2021-07-11T03:40:38.292 INFO:root:teuthology version: 1.1.0-24e3280d 2021-07-11T03:40:38.295 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912 --verbose --description fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} -- /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/orig.config.yaml 2021-07-11T03:40:38.341 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-07-11T03:40:38.406 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912 branch: octopus description: fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests} email: ceph-qa@ceph.io first_in_suite: false job_id: '6260912' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: gibba name: teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba nuke-on-error: true openstack: - volumes: count: 4 size: 30 - machine: disk: 200 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: octopus ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - POOL_APP_NOT_ENABLED - has not responded to cap revoke by MDS for over - MDS_CLIENT_LATE_RELEASE - responding to mclientcaps log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 45c6565d67f116c9b3509842b718398a5b38cf73 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: sha1: 45c6565d67f116c9b3509842b718398a5b38cf73 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: octopus sha1: 54bff90266c0649369e90717e0c4df7b0276c9e1 owner: scheduled_teuthology@teuthology priority: 1000 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 - - client.0 - - client.1 sha1: 45c6565d67f116c9b3509842b718398a5b38cf73 sleep_before_teardown: 0 suite: fs suite_branch: octopus suite_path: /home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 54bff90266c0649369e90717e0c4df7b0276c9e1 targets: gibba004.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD3TbESRYFDs/8GarIwJDMF3tPDJSdI6bDNoa57O3koX+lAtyKu7F09Dt1O5VtJTTnQ3OMXwoajpGcw+pIL92LIcAXvtUbeHzrjrUTY8QjTVC2oqvP8oukXlqjyx4tziMrRNHMM9qSWXSWp289KBb3ObMSpnSirOxnYZIYTDE5/JoLdWHG6FUGv5TA5LJBv8TH5xUUJCSNJu/rLzDXagOAF4ZJej7Mo3ikG7USyY6ZszbCgW3hf6deYoDIwzF22dKCpegtkpseMcczs7m+cV1b6O2cfH9CS1os078OGa9bYzSQwG4s1voAyfo/6m5vhCpMxHpAxkjnXgGPWKsXAqvecf3/0r5UotnQTEUzXqfA3eggQtgHTA6jQ0LL29BU3ODn26H82e7Oam9gFSVnm3SnF0sEtC1alfTQJzSKyaUqCoDzk4lom6m5mfqDQp4JLCR4dKsIc+Nh4G2AIB6/e7mUd9QH1pwnLwrMXa7OSoNUAVG5O6tpNH8QL7vTXVoowFc= gibba005.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaso08m3kPXqiogDT2IoHRT9dVQztV03/TfKLvNRXq0ZUP0gbYBb+7QSfV17jfwNmCnuZzq6Gin13X9IMQNhTnc+PrvWLqwKjKyghkD4xF+4ufDYcbpOeOtth8hW0zF5WCG5KBbNfK7JD018NSanXH3IaLHZV6G0OSCY+V6YJLMm+zxZjXDGhaC1smj7/W2u0M7Cn1HSM2owjqariWxNRwgbz7fEZ7XWXd5TIEEH+LS+N9dni0U7rMkvtQb3pxeMpYjrDL/rNyaUwGiOq2oHiLJdkyxx5XetUlQF23pDTBDpKxoQVfh2bXY4ehnmTKam8yBMsnCabRM2VfABG/AxG/pFt/roOtBHGKF4WZ8wiIQJxZIQd1g0hA3CDa0xZXKagCEwX34o3/3ofN1yqJ9CFHVBthU22voVuEbd7VzRw2LEMEmsJ83f/xZX//1vATfS5AmxxXG3dCadCNI4D5k8K5Ibgc7qmuPnyWaUbmURpTc1KSBTxaLwD2VgueKkxPg28= gibba019.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOM8WP6oMGlmnKvf0fkpXhjYzZceJRPDGXBi8WTY0194xZLnJKMlpsfLJ1e33MIbhpTM1U4EawIbrp6ZwUyTspt0JJuHpVFWQbMmsV5/DCaJ+kI0hJ5oAv61fYpuio2B15cOa3INia01tR/eGCzKZG81DPhiXvY7IlEFGGGN7QcX0IiMwd7MQgn1KvbbG1EGsCOX9TtH7d2hU4y+tQZFycAvFgh+3PnFoM8+5V9psiXGZ5wl5ikdcFRG641nkkohcTFVB9Xyy9oAfKiZ4SXoAs4/fzQfBuotNOTr8whnJfssvmokFyHML0OOUPvHsbry3B9XKzt0yNDS8qOYJF1vpA8E4bOvl5uBqoun2EOY78AW1dKipuLHxVIdl4yWBFEOHl0Au3CN1AbbJbhdZlIGtoWF6axGzpE25NQxpYseMpfR2oOi20r3T5Wn9/AnWg5jBeQ19HFEz5YA+WI+Gdkml7g1yepnl0FH4pVf2ZOahX6nZGko+w1wRiaS3uDMlSMQM= gibba023.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUBbOohbK0ZG/bmINdjKAXvH7VlJ7MPrBb1K1+9bxh2Wse+O+/EuO9Ii/ET6Q2UlRXu4r1SXNw3ibsXHam5JyF/RMHMkXU4va4+0lO0Vg+EG/xd+N0LMRXmXgVqygUJuD1iMr5y85vpoJfpiURQQ6eBq3ohE+pC8NB8F6U6e6mfQePXo5amtUwc/Cd02o/uv6L7WgwDBpz4IEcLNPUgs9RuhBdNxHAOTSOFdiriJdcSHDQ+mYrK8qvoj/tdqqZlblbiGVaFtfLzRFFIRHJPYXaE65/Eu8onBaSBxK01bSxdAWDUkOqW3dbfus1RIP0UAXWZTd6YQLMbtZHCcouUNNLFiXeCjclFc2RnMTBmu/YwwKayxT2MH2E5FbPXq4KG/vfMgf+n87QY6bHuNF4yL8j3XUjVnCzQ25fPGjRvMIiJ6zrT0va7PL9f1WhBE7FellcoeIUs9LRIKAnF6jg9d1oOkJQjnuQqp9lyDSlfK6nDJzJSmk6ybJzqx82u+6hims= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_misc teuthology_branch: master teuthology_sha1: 24e3280d7d1fc156628af95b24b27a29e38b2766 timestamp: 2021-07-09_04:10:02 tube: gibba user: teuthology verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.gibba.8273 2021-07-11T03:40:38.407 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa; will attempt to use it 2021-07-11T03:40:38.408 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks 2021-07-11T03:40:38.408 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-07-11T03:40:38.410 INFO:teuthology.task.internal:Checking packages... 2021-07-11T03:40:38.430 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '45c6565d67f116c9b3509842b718398a5b38cf73' 2021-07-11T03:40:38.430 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-07-11T03:40:38.431 INFO:teuthology.packaging:ref: None 2021-07-11T03:40:38.431 INFO:teuthology.packaging:tag: None 2021-07-11T03:40:38.432 INFO:teuthology.packaging:branch: octopus 2021-07-11T03:40:38.432 INFO:teuthology.packaging:sha1: 45c6565d67f116c9b3509842b718398a5b38cf73 2021-07-11T03:40:38.432 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=octopus 2021-07-11T03:40:38.688 INFO:teuthology.task.internal:Found packages for ceph version 15.2.13-132-g54bff902-1focal 2021-07-11T03:40:38.689 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-07-11T03:40:38.700 INFO:teuthology.task.internal:no buildpackages task found 2021-07-11T03:40:38.700 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-07-11T03:40:38.714 INFO:teuthology.task.internal:Saving configuration 2021-07-11T03:40:38.731 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-07-11T03:40:38.742 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-07-11T03:40:38.759 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-07-11 03:35:25.474805', 'description': '/home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAID7Wti1kbSvbKDHcjukEfwmN6+NMj9xp3e6Vkj8wQooC', 'os_version': '20.04', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba004.front.sepia.ceph.com'} 2021-07-11T03:40:38.777 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-07-11 03:35:25.474789', 'description': '/home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIND/fbbiZElc/qrMi9K8v3tKwxLBq+ldsvxQs7owxuae', 'os_version': '20.04', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba005.front.sepia.ceph.com'} 2021-07-11T03:40:38.795 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-07-11 03:35:25.474773', 'description': '/home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHbXs0i1emGRt7I0J8uaShOp4YBXPh0Y1eq+CDKORU68', 'os_version': '20.04', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba019.front.sepia.ceph.com'} 2021-07-11T03:40:38.815 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-07-11 03:35:25.474751', 'description': '/home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912', 'locked_by': 'scheduled_teuthology@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILe8MAyolDJsXUN2JKSGIjQEEem1vw70IMU/ZhILxaCX', 'os_version': '20.04', 'machine_type': 'gibba', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'gibba023.front.sepia.ceph.com'} 2021-07-11T03:40:38.816 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-07-11T03:40:38.901 INFO:teuthology.task.internal:roles: ubuntu@gibba004.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2021-07-11T03:40:38.902 INFO:teuthology.task.internal:roles: ubuntu@gibba005.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2021-07-11T03:40:38.902 INFO:teuthology.task.internal:roles: ubuntu@gibba019.front.sepia.ceph.com - ['client.0'] 2021-07-11T03:40:38.902 INFO:teuthology.task.internal:roles: ubuntu@gibba023.front.sepia.ceph.com - ['client.1'] 2021-07-11T03:40:38.902 INFO:teuthology.run_tasks:Running task console_log... 2021-07-11T03:40:39.105 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd6cccbab70>, signals=[15]) 2021-07-11T03:40:39.106 INFO:teuthology.run_tasks:Running task internal.connect... 2021-07-11T03:40:39.125 INFO:teuthology.task.internal:Opening connections... 2021-07-11T03:40:39.125 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T03:40:39.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba004.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:40:39.167 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T03:40:39.168 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:40:39.200 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba019.front.sepia.ceph.com 2021-07-11T03:40:39.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:40:39.236 DEBUG:teuthology.task.internal:connecting to ubuntu@gibba023.front.sepia.ceph.com 2021-07-11T03:40:39.237 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:40:39.269 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-07-11T03:40:39.283 DEBUG:teuthology.orchestra.run.gibba004:> uname -m 2021-07-11T03:40:39.289 INFO:teuthology.orchestra.run.gibba004.stdout:x86_64 2021-07-11T03:40:39.289 DEBUG:teuthology.orchestra.run.gibba004:> cat /etc/os-release 2021-07-11T03:40:39.338 INFO:teuthology.orchestra.run.gibba004.stdout:NAME="Ubuntu" 2021-07-11T03:40:39.339 INFO:teuthology.orchestra.run.gibba004.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-07-11T03:40:39.339 INFO:teuthology.orchestra.run.gibba004.stdout:ID=ubuntu 2021-07-11T03:40:39.339 INFO:teuthology.orchestra.run.gibba004.stdout:ID_LIKE=debian 2021-07-11T03:40:39.339 INFO:teuthology.orchestra.run.gibba004.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-07-11T03:40:39.339 INFO:teuthology.orchestra.run.gibba004.stdout:VERSION_ID="20.04" 2021-07-11T03:40:39.340 INFO:teuthology.orchestra.run.gibba004.stdout:HOME_URL="https://www.ubuntu.com/" 2021-07-11T03:40:39.340 INFO:teuthology.orchestra.run.gibba004.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-07-11T03:40:39.340 INFO:teuthology.orchestra.run.gibba004.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-07-11T03:40:39.340 INFO:teuthology.orchestra.run.gibba004.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-07-11T03:40:39.341 INFO:teuthology.orchestra.run.gibba004.stdout:VERSION_CODENAME=focal 2021-07-11T03:40:39.341 INFO:teuthology.orchestra.run.gibba004.stdout:UBUNTU_CODENAME=focal 2021-07-11T03:40:39.342 INFO:teuthology.lock.ops:Updating gibba004.front.sepia.ceph.com on lock server 2021-07-11T03:40:39.361 DEBUG:teuthology.orchestra.run.gibba005:> uname -m 2021-07-11T03:40:39.370 INFO:teuthology.orchestra.run.gibba005.stdout:x86_64 2021-07-11T03:40:39.370 DEBUG:teuthology.orchestra.run.gibba005:> cat /etc/os-release 2021-07-11T03:40:39.418 INFO:teuthology.orchestra.run.gibba005.stdout:NAME="Ubuntu" 2021-07-11T03:40:39.418 INFO:teuthology.orchestra.run.gibba005.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-07-11T03:40:39.419 INFO:teuthology.orchestra.run.gibba005.stdout:ID=ubuntu 2021-07-11T03:40:39.419 INFO:teuthology.orchestra.run.gibba005.stdout:ID_LIKE=debian 2021-07-11T03:40:39.419 INFO:teuthology.orchestra.run.gibba005.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-07-11T03:40:39.419 INFO:teuthology.orchestra.run.gibba005.stdout:VERSION_ID="20.04" 2021-07-11T03:40:39.419 INFO:teuthology.orchestra.run.gibba005.stdout:HOME_URL="https://www.ubuntu.com/" 2021-07-11T03:40:39.420 INFO:teuthology.orchestra.run.gibba005.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-07-11T03:40:39.420 INFO:teuthology.orchestra.run.gibba005.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-07-11T03:40:39.420 INFO:teuthology.orchestra.run.gibba005.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-07-11T03:40:39.420 INFO:teuthology.orchestra.run.gibba005.stdout:VERSION_CODENAME=focal 2021-07-11T03:40:39.421 INFO:teuthology.orchestra.run.gibba005.stdout:UBUNTU_CODENAME=focal 2021-07-11T03:40:39.421 INFO:teuthology.lock.ops:Updating gibba005.front.sepia.ceph.com on lock server 2021-07-11T03:40:39.438 DEBUG:teuthology.orchestra.run.gibba019:> uname -m 2021-07-11T03:40:39.447 INFO:teuthology.orchestra.run.gibba019.stdout:x86_64 2021-07-11T03:40:39.447 DEBUG:teuthology.orchestra.run.gibba019:> cat /etc/os-release 2021-07-11T03:40:39.497 INFO:teuthology.orchestra.run.gibba019.stdout:NAME="Ubuntu" 2021-07-11T03:40:39.497 INFO:teuthology.orchestra.run.gibba019.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-07-11T03:40:39.498 INFO:teuthology.orchestra.run.gibba019.stdout:ID=ubuntu 2021-07-11T03:40:39.498 INFO:teuthology.orchestra.run.gibba019.stdout:ID_LIKE=debian 2021-07-11T03:40:39.498 INFO:teuthology.orchestra.run.gibba019.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-07-11T03:40:39.498 INFO:teuthology.orchestra.run.gibba019.stdout:VERSION_ID="20.04" 2021-07-11T03:40:39.498 INFO:teuthology.orchestra.run.gibba019.stdout:HOME_URL="https://www.ubuntu.com/" 2021-07-11T03:40:39.499 INFO:teuthology.orchestra.run.gibba019.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-07-11T03:40:39.499 INFO:teuthology.orchestra.run.gibba019.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-07-11T03:40:39.499 INFO:teuthology.orchestra.run.gibba019.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-07-11T03:40:39.499 INFO:teuthology.orchestra.run.gibba019.stdout:VERSION_CODENAME=focal 2021-07-11T03:40:39.500 INFO:teuthology.orchestra.run.gibba019.stdout:UBUNTU_CODENAME=focal 2021-07-11T03:40:39.500 INFO:teuthology.lock.ops:Updating gibba019.front.sepia.ceph.com on lock server 2021-07-11T03:40:39.517 DEBUG:teuthology.orchestra.run.gibba023:> uname -m 2021-07-11T03:40:39.526 INFO:teuthology.orchestra.run.gibba023.stdout:x86_64 2021-07-11T03:40:39.527 DEBUG:teuthology.orchestra.run.gibba023:> cat /etc/os-release 2021-07-11T03:40:39.572 INFO:teuthology.orchestra.run.gibba023.stdout:NAME="Ubuntu" 2021-07-11T03:40:39.572 INFO:teuthology.orchestra.run.gibba023.stdout:VERSION="20.04.2 LTS (Focal Fossa)" 2021-07-11T03:40:39.572 INFO:teuthology.orchestra.run.gibba023.stdout:ID=ubuntu 2021-07-11T03:40:39.572 INFO:teuthology.orchestra.run.gibba023.stdout:ID_LIKE=debian 2021-07-11T03:40:39.573 INFO:teuthology.orchestra.run.gibba023.stdout:PRETTY_NAME="Ubuntu 20.04.2 LTS" 2021-07-11T03:40:39.573 INFO:teuthology.orchestra.run.gibba023.stdout:VERSION_ID="20.04" 2021-07-11T03:40:39.573 INFO:teuthology.orchestra.run.gibba023.stdout:HOME_URL="https://www.ubuntu.com/" 2021-07-11T03:40:39.573 INFO:teuthology.orchestra.run.gibba023.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-07-11T03:40:39.573 INFO:teuthology.orchestra.run.gibba023.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-07-11T03:40:39.574 INFO:teuthology.orchestra.run.gibba023.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-07-11T03:40:39.574 INFO:teuthology.orchestra.run.gibba023.stdout:VERSION_CODENAME=focal 2021-07-11T03:40:39.574 INFO:teuthology.orchestra.run.gibba023.stdout:UBUNTU_CODENAME=focal 2021-07-11T03:40:39.574 INFO:teuthology.lock.ops:Updating gibba023.front.sepia.ceph.com on lock server 2021-07-11T03:40:39.590 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-07-11T03:40:39.604 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-07-11T03:40:39.615 INFO:teuthology.task.internal:Checking for old test directory... 2021-07-11T03:40:39.615 DEBUG:teuthology.orchestra.run.gibba004:> test '!' -e /home/ubuntu/cephtest 2021-07-11T03:40:39.619 DEBUG:teuthology.orchestra.run.gibba005:> test '!' -e /home/ubuntu/cephtest 2021-07-11T03:40:39.622 DEBUG:teuthology.orchestra.run.gibba019:> test '!' -e /home/ubuntu/cephtest 2021-07-11T03:40:39.626 DEBUG:teuthology.orchestra.run.gibba023:> test '!' -e /home/ubuntu/cephtest 2021-07-11T03:40:39.633 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-07-11T03:40:39.643 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-07-11T03:40:39.643 DEBUG:teuthology.orchestra.run.gibba004:> test -z $(ls -A /var/lib/ceph) 2021-07-11T03:40:39.646 DEBUG:teuthology.orchestra.run.gibba005:> test -z $(ls -A /var/lib/ceph) 2021-07-11T03:40:39.649 DEBUG:teuthology.orchestra.run.gibba019:> test -z $(ls -A /var/lib/ceph) 2021-07-11T03:40:39.652 DEBUG:teuthology.orchestra.run.gibba023:> test -z $(ls -A /var/lib/ceph) 2021-07-11T03:40:39.674 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-07-11T03:40:39.753 INFO:teuthology.run_tasks:Running task kernel... 2021-07-11T03:40:39.770 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}, 'mds.a': {'kdb': True, 'sha1': 'distro'}, 'mds.c': {'kdb': True, 'sha1': 'distro'}, 'osd.0': {'kdb': True, 'sha1': 'distro'}, 'osd.1': {'kdb': True, 'sha1': 'distro'}, 'osd.2': {'kdb': True, 'sha1': 'distro'}, 'osd.3': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}, 'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mds.b': {'kdb': True, 'sha1': 'distro'}, 'osd.4': {'kdb': True, 'sha1': 'distro'}, 'osd.5': {'kdb': True, 'sha1': 'distro'}, 'osd.6': {'kdb': True, 'sha1': 'distro'}, 'osd.7': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}} 2021-07-11T03:40:39.771 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}, 'client.1': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-07-11T03:40:39.771 DEBUG:teuthology.orchestra.run.gibba004:> uname -r 2021-07-11T03:40:39.780 INFO:teuthology.orchestra.run.gibba004.stdout:5.4.0-66-generic 2021-07-11T03:40:39.781 INFO:teuthology.task.kernel:Running kernel on gibba004: 5.4.0-66-generic 2021-07-11T03:40:39.781 DEBUG:teuthology.orchestra.run.gibba004:> sudo apt-get clean 2021-07-11T03:40:39.849 DEBUG:teuthology.orchestra.run.gibba004:> sudo apt-get update 2021-07-11T03:40:39.984 INFO:teuthology.orchestra.run.gibba004.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2021-07-11T03:40:40.101 INFO:teuthology.orchestra.run.gibba004.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:40:40.184 INFO:teuthology.orchestra.run.gibba004.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2021-07-11T03:40:40.218 INFO:teuthology.orchestra.run.gibba004.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [258 kB] 2021-07-11T03:40:40.271 INFO:teuthology.orchestra.run.gibba004.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [745 kB] 2021-07-11T03:40:40.303 INFO:teuthology.orchestra.run.gibba004.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [148 kB] 2021-07-11T03:40:40.306 INFO:teuthology.orchestra.run.gibba004.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [8,036 B] 2021-07-11T03:40:40.307 INFO:teuthology.orchestra.run.gibba004.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [282 kB] 2021-07-11T03:40:40.310 INFO:teuthology.orchestra.run.gibba004.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [16.4 kB] 2021-07-11T03:40:40.311 INFO:teuthology.orchestra.run.gibba004.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [40.9 kB] 2021-07-11T03:40:40.312 INFO:teuthology.orchestra.run.gibba004.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [456 B] 2021-07-11T03:40:40.312 INFO:teuthology.orchestra.run.gibba004.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [629 kB] 2021-07-11T03:40:40.321 INFO:teuthology.orchestra.run.gibba004.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [503 kB] 2021-07-11T03:40:40.327 INFO:teuthology.orchestra.run.gibba004.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [96.2 kB] 2021-07-11T03:40:40.328 INFO:teuthology.orchestra.run.gibba004.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [11.6 kB] 2021-07-11T03:40:40.337 INFO:teuthology.orchestra.run.gibba004.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [5,384 B] 2021-07-11T03:40:40.338 INFO:teuthology.orchestra.run.gibba004.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [19.9 kB] 2021-07-11T03:40:40.338 INFO:teuthology.orchestra.run.gibba004.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [4,316 B] 2021-07-11T03:40:40.338 INFO:teuthology.orchestra.run.gibba004.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [528 B] 2021-07-11T03:40:40.525 INFO:teuthology.orchestra.run.gibba004.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB] 2021-07-11T03:40:40.623 INFO:teuthology.orchestra.run.gibba004.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,086 kB] 2021-07-11T03:40:40.855 INFO:teuthology.orchestra.run.gibba004.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [505 kB] 2021-07-11T03:40:40.893 INFO:teuthology.orchestra.run.gibba004.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [239 kB] 2021-07-11T03:40:40.905 INFO:teuthology.orchestra.run.gibba004.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [13.8 kB] 2021-07-11T03:40:40.907 INFO:teuthology.orchestra.run.gibba004.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [318 kB] 2021-07-11T03:40:40.928 INFO:teuthology.orchestra.run.gibba004.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [17.7 kB] 2021-07-11T03:40:40.928 INFO:teuthology.orchestra.run.gibba004.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [46.1 kB] 2021-07-11T03:40:40.932 INFO:teuthology.orchestra.run.gibba004.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [456 B] 2021-07-11T03:40:40.932 INFO:teuthology.orchestra.run.gibba004.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [840 kB] 2021-07-11T03:40:40.971 INFO:teuthology.orchestra.run.gibba004.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [625 kB] 2021-07-11T03:40:40.988 INFO:teuthology.orchestra.run.gibba004.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [175 kB] 2021-07-11T03:40:41.015 INFO:teuthology.orchestra.run.gibba004.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [18.3 kB] 2021-07-11T03:40:41.015 INFO:teuthology.orchestra.run.gibba004.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [6,376 B] 2021-07-11T03:40:41.016 INFO:teuthology.orchestra.run.gibba004.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [23.6 kB] 2021-07-11T03:40:41.016 INFO:teuthology.orchestra.run.gibba004.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [6,472 B] 2021-07-11T03:40:41.020 INFO:teuthology.orchestra.run.gibba004.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [648 B] 2021-07-11T03:40:41.021 INFO:teuthology.orchestra.run.gibba004.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [2,568 B] 2021-07-11T03:40:41.021 INFO:teuthology.orchestra.run.gibba004.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [2,568 B] 2021-07-11T03:40:41.098 INFO:teuthology.orchestra.run.gibba004.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main Translation-en [1,120 B] 2021-07-11T03:40:41.098 INFO:teuthology.orchestra.run.gibba004.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [400 B] 2021-07-11T03:40:41.098 INFO:teuthology.orchestra.run.gibba004.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [5,792 B] 2021-07-11T03:40:41.099 INFO:teuthology.orchestra.run.gibba004.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [4,724 B] 2021-07-11T03:40:41.102 INFO:teuthology.orchestra.run.gibba004.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [2,060 B] 2021-07-11T03:40:41.103 INFO:teuthology.orchestra.run.gibba004.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [288 B] 2021-07-11T03:40:43.018 INFO:teuthology.orchestra.run.gibba004.stdout:Fetched 7,037 kB in 1s (5,319 kB/s) 2021-07-11T03:40:44.014 INFO:teuthology.orchestra.run.gibba004.stdout:Reading package lists... 2021-07-11T03:40:44.025 DEBUG:teuthology.orchestra.run.gibba004:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-07-11T03:40:44.061 INFO:teuthology.orchestra.run.gibba004.stdout:Reading package lists... 2021-07-11T03:40:44.164 INFO:teuthology.orchestra.run.gibba004.stdout:Building dependency tree... 2021-07-11T03:40:44.165 INFO:teuthology.orchestra.run.gibba004.stdout:Reading state information... 2021-07-11T03:40:44.209 INFO:teuthology.orchestra.run.gibba004.stderr:E: Unable to locate package linux-image-current-generic 2021-07-11T03:40:44.210 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-07-11T03:40:44.211 DEBUG:teuthology.orchestra.run.gibba004:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-07-11T03:40:44.247 INFO:teuthology.orchestra.run.gibba004.stdout:Reading package lists... 2021-07-11T03:40:44.348 INFO:teuthology.orchestra.run.gibba004.stdout:Building dependency tree... 2021-07-11T03:40:44.349 INFO:teuthology.orchestra.run.gibba004.stdout:Reading state information... 2021-07-11T03:40:44.428 INFO:teuthology.orchestra.run.gibba004.stdout:The following additional packages will be installed: 2021-07-11T03:40:44.429 INFO:teuthology.orchestra.run.gibba004.stdout: linux-generic linux-headers-5.4.0-77 linux-headers-5.4.0-77-generic 2021-07-11T03:40:44.429 INFO:teuthology.orchestra.run.gibba004.stdout: linux-headers-generic linux-image-5.4.0-77-generic 2021-07-11T03:40:44.429 INFO:teuthology.orchestra.run.gibba004.stdout: linux-modules-5.4.0-77-generic linux-modules-extra-5.4.0-77-generic 2021-07-11T03:40:44.429 INFO:teuthology.orchestra.run.gibba004.stdout:Suggested packages: 2021-07-11T03:40:44.430 INFO:teuthology.orchestra.run.gibba004.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2021-07-11T03:40:44.430 INFO:teuthology.orchestra.run.gibba004.stdout:Recommended packages: 2021-07-11T03:40:44.430 INFO:teuthology.orchestra.run.gibba004.stdout: thermald 2021-07-11T03:40:44.442 INFO:teuthology.orchestra.run.gibba004.stdout:The following NEW packages will be installed: 2021-07-11T03:40:44.442 INFO:teuthology.orchestra.run.gibba004.stdout: linux-headers-5.4.0-77 linux-headers-5.4.0-77-generic 2021-07-11T03:40:44.443 INFO:teuthology.orchestra.run.gibba004.stdout: linux-image-5.4.0-77-generic linux-modules-5.4.0-77-generic 2021-07-11T03:40:44.443 INFO:teuthology.orchestra.run.gibba004.stdout: linux-modules-extra-5.4.0-77-generic 2021-07-11T03:40:44.444 INFO:teuthology.orchestra.run.gibba004.stdout:The following packages will be upgraded: 2021-07-11T03:40:44.444 INFO:teuthology.orchestra.run.gibba004.stdout: linux-generic linux-headers-generic linux-image-generic 2021-07-11T03:40:44.615 INFO:teuthology.orchestra.run.gibba004.stdout:3 upgraded, 5 newly installed, 0 to remove and 158 not upgraded. 2021-07-11T03:40:44.615 INFO:teuthology.orchestra.run.gibba004.stdout:Need to get 75.6 MB of archives. 2021-07-11T03:40:44.616 INFO:teuthology.orchestra.run.gibba004.stdout:After this operation, 377 MB of additional disk space will be used. 2021-07-11T03:40:44.616 INFO:teuthology.orchestra.run.gibba004.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-77-generic amd64 5.4.0-77.86 [14.8 MB] 2021-07-11T03:40:45.718 INFO:teuthology.orchestra.run.gibba004.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-77-generic amd64 5.4.0-77.86 [8,981 kB] 2021-07-11T03:40:45.974 INFO:teuthology.orchestra.run.gibba004.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-77-generic amd64 5.4.0-77.86 [39.4 MB] 2021-07-11T03:40:47.298 INFO:teuthology.orchestra.run.gibba004.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.77.80 [1,900 B] 2021-07-11T03:40:47.299 INFO:teuthology.orchestra.run.gibba004.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.77.80 [2,576 B] 2021-07-11T03:40:47.300 INFO:teuthology.orchestra.run.gibba004.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-77 all 5.4.0-77.86 [11.0 MB] 2021-07-11T03:40:47.903 INFO:teuthology.orchestra.run.gibba004.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-77-generic amd64 5.4.0-77.86 [1,410 kB] 2021-07-11T03:40:47.965 INFO:teuthology.orchestra.run.gibba004.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.77.80 [2,444 B] 2021-07-11T03:40:48.211 INFO:teuthology.orchestra.run.gibba004.stdout:Fetched 75.6 MB in 4s (21.5 MB/s) 2021-07-11T03:40:48.340 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package linux-modules-5.4.0-77-generic. 2021-07-11T03:40:50.008 INFO:teuthology.orchestra.run.gibba004.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 101691 files and directories currently installed.) 2021-07-11T03:40:50.015 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../0-linux-modules-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:40:50.086 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking linux-modules-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:40:51.527 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package linux-image-5.4.0-77-generic. 2021-07-11T03:40:51.533 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../1-linux-image-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:40:51.636 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:40:52.480 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-77-generic. 2021-07-11T03:40:52.486 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:40:52.529 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking linux-modules-extra-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:40:55.622 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../3-linux-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:40:55.747 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking linux-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:40:56.146 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:40:56.293 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking linux-image-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:40:56.662 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package linux-headers-5.4.0-77. 2021-07-11T03:40:56.670 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../5-linux-headers-5.4.0-77_5.4.0-77.86_all.deb ... 2021-07-11T03:40:56.712 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking linux-headers-5.4.0-77 (5.4.0-77.86) ... 2021-07-11T03:41:00.136 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package linux-headers-5.4.0-77-generic. 2021-07-11T03:41:00.172 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../6-linux-headers-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:41:00.211 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking linux-headers-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:02.026 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:41:02.167 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking linux-headers-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:41:02.561 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up linux-modules-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:03.371 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:04.091 INFO:teuthology.orchestra.run.gibba004.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:41:04.091 INFO:teuthology.orchestra.run.gibba004.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-77-generic 2021-07-11T03:41:04.231 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up linux-headers-5.4.0-77 (5.4.0-77.86) ... 2021-07-11T03:41:04.348 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up linux-modules-extra-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:05.109 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up linux-image-generic (5.4.0.77.80) ... 2021-07-11T03:41:05.260 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up linux-headers-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:05.386 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up linux-headers-generic (5.4.0.77.80) ... 2021-07-11T03:41:05.528 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up linux-generic (5.4.0.77.80) ... 2021-07-11T03:41:05.680 INFO:teuthology.orchestra.run.gibba004.stdout:Processing triggers for linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:05.796 INFO:teuthology.orchestra.run.gibba004.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-07-11T03:41:05.796 INFO:teuthology.orchestra.run.gibba004.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:41:26.938 INFO:teuthology.orchestra.run.gibba004.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-07-11T03:41:26.939 INFO:teuthology.orchestra.run.gibba004.stdout:Sourcing file `/etc/default/grub' 2021-07-11T03:41:26.964 INFO:teuthology.orchestra.run.gibba004.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-07-11T03:41:26.977 INFO:teuthology.orchestra.run.gibba004.stdout:Generating grub configuration file ... 2021-07-11T03:41:27.317 INFO:teuthology.orchestra.run.gibba004.stdout:Found linux image: /boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:41:27.348 INFO:teuthology.orchestra.run.gibba004.stdout:Found initrd image: /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:41:27.782 INFO:teuthology.orchestra.run.gibba004.stdout:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-07-11T03:41:27.796 INFO:teuthology.orchestra.run.gibba004.stdout:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-07-11T03:41:30.567 INFO:teuthology.orchestra.run.gibba004.stdout:done 2021-07-11T03:41:30.873 DEBUG:teuthology.orchestra.run.gibba004:> dpkg -s linux-image-generic 2021-07-11T03:41:30.890 INFO:teuthology.orchestra.run.gibba004.stdout:Package: linux-image-generic 2021-07-11T03:41:30.890 INFO:teuthology.orchestra.run.gibba004.stdout:Status: install ok installed 2021-07-11T03:41:30.890 INFO:teuthology.orchestra.run.gibba004.stdout:Priority: optional 2021-07-11T03:41:30.891 INFO:teuthology.orchestra.run.gibba004.stdout:Section: kernel 2021-07-11T03:41:30.891 INFO:teuthology.orchestra.run.gibba004.stdout:Installed-Size: 18 2021-07-11T03:41:30.891 INFO:teuthology.orchestra.run.gibba004.stdout:Maintainer: Ubuntu Kernel Team 2021-07-11T03:41:30.891 INFO:teuthology.orchestra.run.gibba004.stdout:Architecture: amd64 2021-07-11T03:41:30.891 INFO:teuthology.orchestra.run.gibba004.stdout:Source: linux-meta 2021-07-11T03:41:30.892 INFO:teuthology.orchestra.run.gibba004.stdout:Version: 5.4.0.77.80 2021-07-11T03:41:30.892 INFO:teuthology.orchestra.run.gibba004.stdout:Provides: virtualbox-guest-modules (= 6.1.16-dfsg-6~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.9) 2021-07-11T03:41:30.892 INFO:teuthology.orchestra.run.gibba004.stdout:Depends: linux-image-5.4.0-77-generic, linux-modules-extra-5.4.0-77-generic, linux-firmware, intel-microcode, amd64-microcode 2021-07-11T03:41:30.892 INFO:teuthology.orchestra.run.gibba004.stdout:Recommends: thermald 2021-07-11T03:41:30.893 INFO:teuthology.orchestra.run.gibba004.stdout:Description: Generic Linux kernel image 2021-07-11T03:41:30.893 INFO:teuthology.orchestra.run.gibba004.stdout: This package will always depend on the latest generic kernel image 2021-07-11T03:41:30.893 INFO:teuthology.orchestra.run.gibba004.stdout: available. 2021-07-11T03:41:30.894 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-66-generic Expected: 5.4.0-77-generic 2021-07-11T03:41:30.894 DEBUG:teuthology.orchestra.run.gibba005:> uname -r 2021-07-11T03:41:30.903 INFO:teuthology.orchestra.run.gibba005.stdout:5.4.0-66-generic 2021-07-11T03:41:30.904 INFO:teuthology.task.kernel:Running kernel on gibba005: 5.4.0-66-generic 2021-07-11T03:41:30.904 DEBUG:teuthology.orchestra.run.gibba005:> sudo apt-get clean 2021-07-11T03:41:30.967 DEBUG:teuthology.orchestra.run.gibba005:> sudo apt-get update 2021-07-11T03:41:31.125 INFO:teuthology.orchestra.run.gibba005.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2021-07-11T03:41:31.201 INFO:teuthology.orchestra.run.gibba005.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:41:31.284 INFO:teuthology.orchestra.run.gibba005.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2021-07-11T03:41:31.329 INFO:teuthology.orchestra.run.gibba005.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [258 kB] 2021-07-11T03:41:31.380 INFO:teuthology.orchestra.run.gibba005.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [745 kB] 2021-07-11T03:41:31.410 INFO:teuthology.orchestra.run.gibba005.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [148 kB] 2021-07-11T03:41:31.414 INFO:teuthology.orchestra.run.gibba005.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [8,036 B] 2021-07-11T03:41:31.415 INFO:teuthology.orchestra.run.gibba005.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [282 kB] 2021-07-11T03:41:31.418 INFO:teuthology.orchestra.run.gibba005.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [16.4 kB] 2021-07-11T03:41:31.419 INFO:teuthology.orchestra.run.gibba005.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [40.9 kB] 2021-07-11T03:41:31.419 INFO:teuthology.orchestra.run.gibba005.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [456 B] 2021-07-11T03:41:31.420 INFO:teuthology.orchestra.run.gibba005.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [629 kB] 2021-07-11T03:41:31.428 INFO:teuthology.orchestra.run.gibba005.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [503 kB] 2021-07-11T03:41:31.434 INFO:teuthology.orchestra.run.gibba005.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [96.2 kB] 2021-07-11T03:41:31.435 INFO:teuthology.orchestra.run.gibba005.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [11.6 kB] 2021-07-11T03:41:31.445 INFO:teuthology.orchestra.run.gibba005.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [5,384 B] 2021-07-11T03:41:31.445 INFO:teuthology.orchestra.run.gibba005.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [19.9 kB] 2021-07-11T03:41:31.445 INFO:teuthology.orchestra.run.gibba005.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [4,316 B] 2021-07-11T03:41:31.446 INFO:teuthology.orchestra.run.gibba005.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [528 B] 2021-07-11T03:41:31.628 INFO:teuthology.orchestra.run.gibba005.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB] 2021-07-11T03:41:31.723 INFO:teuthology.orchestra.run.gibba005.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,086 kB] 2021-07-11T03:41:31.956 INFO:teuthology.orchestra.run.gibba005.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [505 kB] 2021-07-11T03:41:31.993 INFO:teuthology.orchestra.run.gibba005.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [239 kB] 2021-07-11T03:41:32.008 INFO:teuthology.orchestra.run.gibba005.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [13.8 kB] 2021-07-11T03:41:32.008 INFO:teuthology.orchestra.run.gibba005.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [318 kB] 2021-07-11T03:41:32.030 INFO:teuthology.orchestra.run.gibba005.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [17.7 kB] 2021-07-11T03:41:32.030 INFO:teuthology.orchestra.run.gibba005.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [46.1 kB] 2021-07-11T03:41:32.034 INFO:teuthology.orchestra.run.gibba005.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [456 B] 2021-07-11T03:41:32.035 INFO:teuthology.orchestra.run.gibba005.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [625 kB] 2021-07-11T03:41:32.064 INFO:teuthology.orchestra.run.gibba005.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [840 kB] 2021-07-11T03:41:32.087 INFO:teuthology.orchestra.run.gibba005.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [175 kB] 2021-07-11T03:41:32.117 INFO:teuthology.orchestra.run.gibba005.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [18.3 kB] 2021-07-11T03:41:32.117 INFO:teuthology.orchestra.run.gibba005.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [23.6 kB] 2021-07-11T03:41:32.118 INFO:teuthology.orchestra.run.gibba005.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [6,376 B] 2021-07-11T03:41:32.118 INFO:teuthology.orchestra.run.gibba005.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [6,472 B] 2021-07-11T03:41:32.121 INFO:teuthology.orchestra.run.gibba005.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [648 B] 2021-07-11T03:41:32.122 INFO:teuthology.orchestra.run.gibba005.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [2,568 B] 2021-07-11T03:41:32.122 INFO:teuthology.orchestra.run.gibba005.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [2,568 B] 2021-07-11T03:41:32.201 INFO:teuthology.orchestra.run.gibba005.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main Translation-en [1,120 B] 2021-07-11T03:41:32.202 INFO:teuthology.orchestra.run.gibba005.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [400 B] 2021-07-11T03:41:32.202 INFO:teuthology.orchestra.run.gibba005.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [4,724 B] 2021-07-11T03:41:32.202 INFO:teuthology.orchestra.run.gibba005.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [5,792 B] 2021-07-11T03:41:32.203 INFO:teuthology.orchestra.run.gibba005.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [2,060 B] 2021-07-11T03:41:32.204 INFO:teuthology.orchestra.run.gibba005.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [288 B] 2021-07-11T03:41:34.038 INFO:teuthology.orchestra.run.gibba005.stdout:Fetched 7,037 kB in 1s (5,373 kB/s) 2021-07-11T03:41:34.965 INFO:teuthology.orchestra.run.gibba005.stdout:Reading package lists... 2021-07-11T03:41:34.976 DEBUG:teuthology.orchestra.run.gibba005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-07-11T03:41:35.011 INFO:teuthology.orchestra.run.gibba005.stdout:Reading package lists... 2021-07-11T03:41:35.112 INFO:teuthology.orchestra.run.gibba005.stdout:Building dependency tree... 2021-07-11T03:41:35.112 INFO:teuthology.orchestra.run.gibba005.stdout:Reading state information... 2021-07-11T03:41:35.158 INFO:teuthology.orchestra.run.gibba005.stderr:E: Unable to locate package linux-image-current-generic 2021-07-11T03:41:35.159 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-07-11T03:41:35.160 DEBUG:teuthology.orchestra.run.gibba005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-07-11T03:41:35.245 INFO:teuthology.orchestra.run.gibba005.stdout:Reading package lists... 2021-07-11T03:41:35.344 INFO:teuthology.orchestra.run.gibba005.stdout:Building dependency tree... 2021-07-11T03:41:35.345 INFO:teuthology.orchestra.run.gibba005.stdout:Reading state information... 2021-07-11T03:41:35.423 INFO:teuthology.orchestra.run.gibba005.stdout:The following additional packages will be installed: 2021-07-11T03:41:35.423 INFO:teuthology.orchestra.run.gibba005.stdout: linux-generic linux-headers-5.4.0-77 linux-headers-5.4.0-77-generic 2021-07-11T03:41:35.423 INFO:teuthology.orchestra.run.gibba005.stdout: linux-headers-generic linux-image-5.4.0-77-generic 2021-07-11T03:41:35.424 INFO:teuthology.orchestra.run.gibba005.stdout: linux-modules-5.4.0-77-generic linux-modules-extra-5.4.0-77-generic 2021-07-11T03:41:35.424 INFO:teuthology.orchestra.run.gibba005.stdout:Suggested packages: 2021-07-11T03:41:35.424 INFO:teuthology.orchestra.run.gibba005.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2021-07-11T03:41:35.424 INFO:teuthology.orchestra.run.gibba005.stdout:Recommended packages: 2021-07-11T03:41:35.425 INFO:teuthology.orchestra.run.gibba005.stdout: thermald 2021-07-11T03:41:35.436 INFO:teuthology.orchestra.run.gibba005.stdout:The following NEW packages will be installed: 2021-07-11T03:41:35.437 INFO:teuthology.orchestra.run.gibba005.stdout: linux-headers-5.4.0-77 linux-headers-5.4.0-77-generic 2021-07-11T03:41:35.437 INFO:teuthology.orchestra.run.gibba005.stdout: linux-image-5.4.0-77-generic linux-modules-5.4.0-77-generic 2021-07-11T03:41:35.437 INFO:teuthology.orchestra.run.gibba005.stdout: linux-modules-extra-5.4.0-77-generic 2021-07-11T03:41:35.438 INFO:teuthology.orchestra.run.gibba005.stdout:The following packages will be upgraded: 2021-07-11T03:41:35.438 INFO:teuthology.orchestra.run.gibba005.stdout: linux-generic linux-headers-generic linux-image-generic 2021-07-11T03:41:35.609 INFO:teuthology.orchestra.run.gibba005.stdout:3 upgraded, 5 newly installed, 0 to remove and 158 not upgraded. 2021-07-11T03:41:35.609 INFO:teuthology.orchestra.run.gibba005.stdout:Need to get 75.6 MB of archives. 2021-07-11T03:41:35.609 INFO:teuthology.orchestra.run.gibba005.stdout:After this operation, 377 MB of additional disk space will be used. 2021-07-11T03:41:35.610 INFO:teuthology.orchestra.run.gibba005.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-77-generic amd64 5.4.0-77.86 [14.8 MB] 2021-07-11T03:41:36.747 INFO:teuthology.orchestra.run.gibba005.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-77-generic amd64 5.4.0-77.86 [8,981 kB] 2021-07-11T03:41:37.001 INFO:teuthology.orchestra.run.gibba005.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-77-generic amd64 5.4.0-77.86 [39.4 MB] 2021-07-11T03:41:38.285 INFO:teuthology.orchestra.run.gibba005.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.77.80 [1,900 B] 2021-07-11T03:41:38.286 INFO:teuthology.orchestra.run.gibba005.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.77.80 [2,576 B] 2021-07-11T03:41:38.286 INFO:teuthology.orchestra.run.gibba005.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-77 all 5.4.0-77.86 [11.0 MB] 2021-07-11T03:41:39.013 INFO:teuthology.orchestra.run.gibba005.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-77-generic amd64 5.4.0-77.86 [1,410 kB] 2021-07-11T03:41:39.114 INFO:teuthology.orchestra.run.gibba005.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.77.80 [2,444 B] 2021-07-11T03:41:39.346 INFO:teuthology.orchestra.run.gibba005.stdout:Fetched 75.6 MB in 4s (20.6 MB/s) 2021-07-11T03:41:39.503 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package linux-modules-5.4.0-77-generic. 2021-07-11T03:41:41.014 INFO:teuthology.orchestra.run.gibba005.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 101691 files and directories currently installed.) 2021-07-11T03:41:41.021 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../0-linux-modules-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:41:41.091 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking linux-modules-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:42.473 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package linux-image-5.4.0-77-generic. 2021-07-11T03:41:42.480 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../1-linux-image-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:41:42.574 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:43.394 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-77-generic. 2021-07-11T03:41:43.412 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:41:43.452 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking linux-modules-extra-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:46.311 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../3-linux-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:41:46.437 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking linux-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:41:46.777 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:41:46.899 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking linux-image-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:41:47.210 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package linux-headers-5.4.0-77. 2021-07-11T03:41:47.217 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../5-linux-headers-5.4.0-77_5.4.0-77.86_all.deb ... 2021-07-11T03:41:47.276 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking linux-headers-5.4.0-77 (5.4.0-77.86) ... 2021-07-11T03:41:50.669 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package linux-headers-5.4.0-77-generic. 2021-07-11T03:41:50.705 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../6-linux-headers-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:41:50.769 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking linux-headers-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:52.368 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:41:52.491 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking linux-headers-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:41:52.844 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up linux-modules-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:53.629 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:54.383 INFO:teuthology.orchestra.run.gibba005.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:41:54.384 INFO:teuthology.orchestra.run.gibba005.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-77-generic 2021-07-11T03:41:54.498 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up linux-headers-5.4.0-77 (5.4.0-77.86) ... 2021-07-11T03:41:54.625 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up linux-modules-extra-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:55.434 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up linux-image-generic (5.4.0.77.80) ... 2021-07-11T03:41:55.560 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up linux-headers-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:55.686 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up linux-headers-generic (5.4.0.77.80) ... 2021-07-11T03:41:55.845 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up linux-generic (5.4.0.77.80) ... 2021-07-11T03:41:55.972 INFO:teuthology.orchestra.run.gibba005.stdout:Processing triggers for linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:41:56.073 INFO:teuthology.orchestra.run.gibba005.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-07-11T03:41:56.073 INFO:teuthology.orchestra.run.gibba005.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:42:16.536 INFO:teuthology.orchestra.run.gibba005.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-07-11T03:42:16.536 INFO:teuthology.orchestra.run.gibba005.stdout:Sourcing file `/etc/default/grub' 2021-07-11T03:42:16.540 INFO:teuthology.orchestra.run.gibba005.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-07-11T03:42:16.565 INFO:teuthology.orchestra.run.gibba005.stdout:Generating grub configuration file ... 2021-07-11T03:42:16.890 INFO:teuthology.orchestra.run.gibba005.stdout:Found linux image: /boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:42:16.948 INFO:teuthology.orchestra.run.gibba005.stdout:Found initrd image: /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:42:17.365 INFO:teuthology.orchestra.run.gibba005.stdout:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-07-11T03:42:17.379 INFO:teuthology.orchestra.run.gibba005.stdout:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-07-11T03:42:20.307 INFO:teuthology.orchestra.run.gibba005.stdout:done 2021-07-11T03:42:20.583 DEBUG:teuthology.orchestra.run.gibba005:> dpkg -s linux-image-generic 2021-07-11T03:42:20.600 INFO:teuthology.orchestra.run.gibba005.stdout:Package: linux-image-generic 2021-07-11T03:42:20.601 INFO:teuthology.orchestra.run.gibba005.stdout:Status: install ok installed 2021-07-11T03:42:20.601 INFO:teuthology.orchestra.run.gibba005.stdout:Priority: optional 2021-07-11T03:42:20.601 INFO:teuthology.orchestra.run.gibba005.stdout:Section: kernel 2021-07-11T03:42:20.601 INFO:teuthology.orchestra.run.gibba005.stdout:Installed-Size: 18 2021-07-11T03:42:20.602 INFO:teuthology.orchestra.run.gibba005.stdout:Maintainer: Ubuntu Kernel Team 2021-07-11T03:42:20.602 INFO:teuthology.orchestra.run.gibba005.stdout:Architecture: amd64 2021-07-11T03:42:20.602 INFO:teuthology.orchestra.run.gibba005.stdout:Source: linux-meta 2021-07-11T03:42:20.602 INFO:teuthology.orchestra.run.gibba005.stdout:Version: 5.4.0.77.80 2021-07-11T03:42:20.603 INFO:teuthology.orchestra.run.gibba005.stdout:Provides: virtualbox-guest-modules (= 6.1.16-dfsg-6~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.9) 2021-07-11T03:42:20.603 INFO:teuthology.orchestra.run.gibba005.stdout:Depends: linux-image-5.4.0-77-generic, linux-modules-extra-5.4.0-77-generic, linux-firmware, intel-microcode, amd64-microcode 2021-07-11T03:42:20.603 INFO:teuthology.orchestra.run.gibba005.stdout:Recommends: thermald 2021-07-11T03:42:20.603 INFO:teuthology.orchestra.run.gibba005.stdout:Description: Generic Linux kernel image 2021-07-11T03:42:20.603 INFO:teuthology.orchestra.run.gibba005.stdout: This package will always depend on the latest generic kernel image 2021-07-11T03:42:20.604 INFO:teuthology.orchestra.run.gibba005.stdout: available. 2021-07-11T03:42:20.604 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-66-generic Expected: 5.4.0-77-generic 2021-07-11T03:42:20.605 DEBUG:teuthology.orchestra.run.gibba019:> uname -r 2021-07-11T03:42:20.613 INFO:teuthology.orchestra.run.gibba019.stdout:5.4.0-66-generic 2021-07-11T03:42:20.613 INFO:teuthology.task.kernel:Running kernel on gibba019: 5.4.0-66-generic 2021-07-11T03:42:20.614 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-get clean 2021-07-11T03:42:20.695 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-get update 2021-07-11T03:42:20.814 INFO:teuthology.orchestra.run.gibba019.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2021-07-11T03:42:20.925 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:42:21.007 INFO:teuthology.orchestra.run.gibba019.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2021-07-11T03:42:21.033 INFO:teuthology.orchestra.run.gibba019.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [745 kB] 2021-07-11T03:42:21.126 INFO:teuthology.orchestra.run.gibba019.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [258 kB] 2021-07-11T03:42:21.132 INFO:teuthology.orchestra.run.gibba019.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [148 kB] 2021-07-11T03:42:21.135 INFO:teuthology.orchestra.run.gibba019.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [8,036 B] 2021-07-11T03:42:21.135 INFO:teuthology.orchestra.run.gibba019.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [16.4 kB] 2021-07-11T03:42:21.136 INFO:teuthology.orchestra.run.gibba019.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [282 kB] 2021-07-11T03:42:21.140 INFO:teuthology.orchestra.run.gibba019.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [40.9 kB] 2021-07-11T03:42:21.140 INFO:teuthology.orchestra.run.gibba019.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [456 B] 2021-07-11T03:42:21.141 INFO:teuthology.orchestra.run.gibba019.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [629 kB] 2021-07-11T03:42:21.150 INFO:teuthology.orchestra.run.gibba019.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [503 kB] 2021-07-11T03:42:21.156 INFO:teuthology.orchestra.run.gibba019.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [96.2 kB] 2021-07-11T03:42:21.158 INFO:teuthology.orchestra.run.gibba019.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [11.6 kB] 2021-07-11T03:42:21.169 INFO:teuthology.orchestra.run.gibba019.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [19.9 kB] 2021-07-11T03:42:21.169 INFO:teuthology.orchestra.run.gibba019.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [5,384 B] 2021-07-11T03:42:21.170 INFO:teuthology.orchestra.run.gibba019.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [4,316 B] 2021-07-11T03:42:21.170 INFO:teuthology.orchestra.run.gibba019.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [528 B] 2021-07-11T03:42:21.344 INFO:teuthology.orchestra.run.gibba019.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB] 2021-07-11T03:42:21.450 INFO:teuthology.orchestra.run.gibba019.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [505 kB] 2021-07-11T03:42:21.591 INFO:teuthology.orchestra.run.gibba019.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,086 kB] 2021-07-11T03:42:21.721 INFO:teuthology.orchestra.run.gibba019.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [239 kB] 2021-07-11T03:42:21.736 INFO:teuthology.orchestra.run.gibba019.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [13.8 kB] 2021-07-11T03:42:21.737 INFO:teuthology.orchestra.run.gibba019.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [318 kB] 2021-07-11T03:42:21.752 INFO:teuthology.orchestra.run.gibba019.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [17.7 kB] 2021-07-11T03:42:21.752 INFO:teuthology.orchestra.run.gibba019.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [46.1 kB] 2021-07-11T03:42:21.754 INFO:teuthology.orchestra.run.gibba019.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [456 B] 2021-07-11T03:42:21.754 INFO:teuthology.orchestra.run.gibba019.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [625 kB] 2021-07-11T03:42:21.787 INFO:teuthology.orchestra.run.gibba019.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [840 kB] 2021-07-11T03:42:21.814 INFO:teuthology.orchestra.run.gibba019.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [175 kB] 2021-07-11T03:42:21.820 INFO:teuthology.orchestra.run.gibba019.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [18.3 kB] 2021-07-11T03:42:21.837 INFO:teuthology.orchestra.run.gibba019.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [23.6 kB] 2021-07-11T03:42:21.838 INFO:teuthology.orchestra.run.gibba019.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [6,376 B] 2021-07-11T03:42:21.838 INFO:teuthology.orchestra.run.gibba019.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [6,472 B] 2021-07-11T03:42:21.885 INFO:teuthology.orchestra.run.gibba019.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [648 B] 2021-07-11T03:42:21.885 INFO:teuthology.orchestra.run.gibba019.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [2,568 B] 2021-07-11T03:42:21.886 INFO:teuthology.orchestra.run.gibba019.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [2,568 B] 2021-07-11T03:42:21.886 INFO:teuthology.orchestra.run.gibba019.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main Translation-en [1,120 B] 2021-07-11T03:42:21.968 INFO:teuthology.orchestra.run.gibba019.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [400 B] 2021-07-11T03:42:21.968 INFO:teuthology.orchestra.run.gibba019.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [4,724 B] 2021-07-11T03:42:21.969 INFO:teuthology.orchestra.run.gibba019.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [5,792 B] 2021-07-11T03:42:21.969 INFO:teuthology.orchestra.run.gibba019.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [2,060 B] 2021-07-11T03:42:21.969 INFO:teuthology.orchestra.run.gibba019.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [288 B] 2021-07-11T03:42:23.590 INFO:teuthology.orchestra.run.gibba019.stdout:Fetched 7,037 kB in 1s (5,527 kB/s) 2021-07-11T03:42:24.525 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-07-11T03:42:24.536 DEBUG:teuthology.orchestra.run.gibba019:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-07-11T03:42:24.571 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-07-11T03:42:24.668 INFO:teuthology.orchestra.run.gibba019.stdout:Building dependency tree... 2021-07-11T03:42:24.669 INFO:teuthology.orchestra.run.gibba019.stdout:Reading state information... 2021-07-11T03:42:24.714 INFO:teuthology.orchestra.run.gibba019.stderr:E: Unable to locate package linux-image-current-generic 2021-07-11T03:42:24.714 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-07-11T03:42:24.715 DEBUG:teuthology.orchestra.run.gibba019:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-07-11T03:42:24.851 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-07-11T03:42:24.951 INFO:teuthology.orchestra.run.gibba019.stdout:Building dependency tree... 2021-07-11T03:42:24.952 INFO:teuthology.orchestra.run.gibba019.stdout:Reading state information... 2021-07-11T03:42:25.029 INFO:teuthology.orchestra.run.gibba019.stdout:The following additional packages will be installed: 2021-07-11T03:42:25.030 INFO:teuthology.orchestra.run.gibba019.stdout: linux-generic linux-headers-5.4.0-77 linux-headers-5.4.0-77-generic 2021-07-11T03:42:25.030 INFO:teuthology.orchestra.run.gibba019.stdout: linux-headers-generic linux-image-5.4.0-77-generic 2021-07-11T03:42:25.030 INFO:teuthology.orchestra.run.gibba019.stdout: linux-modules-5.4.0-77-generic linux-modules-extra-5.4.0-77-generic 2021-07-11T03:42:25.030 INFO:teuthology.orchestra.run.gibba019.stdout:Suggested packages: 2021-07-11T03:42:25.031 INFO:teuthology.orchestra.run.gibba019.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2021-07-11T03:42:25.031 INFO:teuthology.orchestra.run.gibba019.stdout:Recommended packages: 2021-07-11T03:42:25.031 INFO:teuthology.orchestra.run.gibba019.stdout: thermald 2021-07-11T03:42:25.043 INFO:teuthology.orchestra.run.gibba019.stdout:The following NEW packages will be installed: 2021-07-11T03:42:25.043 INFO:teuthology.orchestra.run.gibba019.stdout: linux-headers-5.4.0-77 linux-headers-5.4.0-77-generic 2021-07-11T03:42:25.043 INFO:teuthology.orchestra.run.gibba019.stdout: linux-image-5.4.0-77-generic linux-modules-5.4.0-77-generic 2021-07-11T03:42:25.043 INFO:teuthology.orchestra.run.gibba019.stdout: linux-modules-extra-5.4.0-77-generic 2021-07-11T03:42:25.044 INFO:teuthology.orchestra.run.gibba019.stdout:The following packages will be upgraded: 2021-07-11T03:42:25.044 INFO:teuthology.orchestra.run.gibba019.stdout: linux-generic linux-headers-generic linux-image-generic 2021-07-11T03:42:25.216 INFO:teuthology.orchestra.run.gibba019.stdout:3 upgraded, 5 newly installed, 0 to remove and 158 not upgraded. 2021-07-11T03:42:25.216 INFO:teuthology.orchestra.run.gibba019.stdout:Need to get 75.6 MB of archives. 2021-07-11T03:42:25.216 INFO:teuthology.orchestra.run.gibba019.stdout:After this operation, 377 MB of additional disk space will be used. 2021-07-11T03:42:25.217 INFO:teuthology.orchestra.run.gibba019.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-77-generic amd64 5.4.0-77.86 [14.8 MB] 2021-07-11T03:42:26.378 INFO:teuthology.orchestra.run.gibba019.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-77-generic amd64 5.4.0-77.86 [8,981 kB] 2021-07-11T03:42:26.625 INFO:teuthology.orchestra.run.gibba019.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-77-generic amd64 5.4.0-77.86 [39.4 MB] 2021-07-11T03:42:27.944 INFO:teuthology.orchestra.run.gibba019.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.77.80 [1,900 B] 2021-07-11T03:42:27.944 INFO:teuthology.orchestra.run.gibba019.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.77.80 [2,576 B] 2021-07-11T03:42:27.944 INFO:teuthology.orchestra.run.gibba019.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-77 all 5.4.0-77.86 [11.0 MB] 2021-07-11T03:42:28.322 INFO:teuthology.orchestra.run.gibba019.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-77-generic amd64 5.4.0-77.86 [1,410 kB] 2021-07-11T03:42:28.359 INFO:teuthology.orchestra.run.gibba019.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.77.80 [2,444 B] 2021-07-11T03:42:28.600 INFO:teuthology.orchestra.run.gibba019.stdout:Fetched 75.6 MB in 3s (22.8 MB/s) 2021-07-11T03:42:28.738 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package linux-modules-5.4.0-77-generic. 2021-07-11T03:42:30.318 INFO:teuthology.orchestra.run.gibba019.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 101691 files and directories currently installed.) 2021-07-11T03:42:30.325 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../0-linux-modules-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:42:30.368 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-modules-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:42:31.700 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package linux-image-5.4.0-77-generic. 2021-07-11T03:42:31.825 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../1-linux-image-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:42:31.825 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:42:32.645 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-77-generic. 2021-07-11T03:42:32.663 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:42:32.703 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-modules-extra-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:42:35.594 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../3-linux-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:42:35.712 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:42:36.002 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:42:36.124 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-image-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:42:36.451 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package linux-headers-5.4.0-77. 2021-07-11T03:42:36.471 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../5-linux-headers-5.4.0-77_5.4.0-77.86_all.deb ... 2021-07-11T03:42:36.509 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-headers-5.4.0-77 (5.4.0-77.86) ... 2021-07-11T03:42:39.751 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package linux-headers-5.4.0-77-generic. 2021-07-11T03:42:39.786 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../6-linux-headers-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:42:39.827 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-headers-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:42:41.377 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:42:41.509 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking linux-headers-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:42:41.828 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-modules-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:42:42.680 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:42:43.431 INFO:teuthology.orchestra.run.gibba019.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:42:43.431 INFO:teuthology.orchestra.run.gibba019.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-77-generic 2021-07-11T03:42:43.556 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-headers-5.4.0-77 (5.4.0-77.86) ... 2021-07-11T03:42:43.700 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-modules-extra-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:42:44.443 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-image-generic (5.4.0.77.80) ... 2021-07-11T03:42:44.569 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-headers-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:42:44.695 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-headers-generic (5.4.0.77.80) ... 2021-07-11T03:42:44.837 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up linux-generic (5.4.0.77.80) ... 2021-07-11T03:42:44.964 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:42:45.061 INFO:teuthology.orchestra.run.gibba019.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-07-11T03:42:45.061 INFO:teuthology.orchestra.run.gibba019.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:43:05.273 INFO:teuthology.orchestra.run.gibba019.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-07-11T03:43:05.273 INFO:teuthology.orchestra.run.gibba019.stdout:Sourcing file `/etc/default/grub' 2021-07-11T03:43:05.275 INFO:teuthology.orchestra.run.gibba019.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-07-11T03:43:05.299 INFO:teuthology.orchestra.run.gibba019.stdout:Generating grub configuration file ... 2021-07-11T03:43:05.594 INFO:teuthology.orchestra.run.gibba019.stdout:Found linux image: /boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:43:05.662 INFO:teuthology.orchestra.run.gibba019.stdout:Found initrd image: /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:43:06.058 INFO:teuthology.orchestra.run.gibba019.stdout:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-07-11T03:43:06.072 INFO:teuthology.orchestra.run.gibba019.stdout:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-07-11T03:43:08.727 INFO:teuthology.orchestra.run.gibba019.stdout:done 2021-07-11T03:43:09.025 DEBUG:teuthology.orchestra.run.gibba019:> dpkg -s linux-image-generic 2021-07-11T03:43:09.039 INFO:teuthology.orchestra.run.gibba019.stdout:Package: linux-image-generic 2021-07-11T03:43:09.039 INFO:teuthology.orchestra.run.gibba019.stdout:Status: install ok installed 2021-07-11T03:43:09.039 INFO:teuthology.orchestra.run.gibba019.stdout:Priority: optional 2021-07-11T03:43:09.039 INFO:teuthology.orchestra.run.gibba019.stdout:Section: kernel 2021-07-11T03:43:09.040 INFO:teuthology.orchestra.run.gibba019.stdout:Installed-Size: 18 2021-07-11T03:43:09.040 INFO:teuthology.orchestra.run.gibba019.stdout:Maintainer: Ubuntu Kernel Team 2021-07-11T03:43:09.040 INFO:teuthology.orchestra.run.gibba019.stdout:Architecture: amd64 2021-07-11T03:43:09.040 INFO:teuthology.orchestra.run.gibba019.stdout:Source: linux-meta 2021-07-11T03:43:09.041 INFO:teuthology.orchestra.run.gibba019.stdout:Version: 5.4.0.77.80 2021-07-11T03:43:09.041 INFO:teuthology.orchestra.run.gibba019.stdout:Provides: virtualbox-guest-modules (= 6.1.16-dfsg-6~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.9) 2021-07-11T03:43:09.041 INFO:teuthology.orchestra.run.gibba019.stdout:Depends: linux-image-5.4.0-77-generic, linux-modules-extra-5.4.0-77-generic, linux-firmware, intel-microcode, amd64-microcode 2021-07-11T03:43:09.041 INFO:teuthology.orchestra.run.gibba019.stdout:Recommends: thermald 2021-07-11T03:43:09.041 INFO:teuthology.orchestra.run.gibba019.stdout:Description: Generic Linux kernel image 2021-07-11T03:43:09.042 INFO:teuthology.orchestra.run.gibba019.stdout: This package will always depend on the latest generic kernel image 2021-07-11T03:43:09.042 INFO:teuthology.orchestra.run.gibba019.stdout: available. 2021-07-11T03:43:09.042 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-66-generic Expected: 5.4.0-77-generic 2021-07-11T03:43:09.043 DEBUG:teuthology.orchestra.run.gibba023:> uname -r 2021-07-11T03:43:09.051 INFO:teuthology.orchestra.run.gibba023.stdout:5.4.0-66-generic 2021-07-11T03:43:09.051 INFO:teuthology.task.kernel:Running kernel on gibba023: 5.4.0-66-generic 2021-07-11T03:43:09.052 DEBUG:teuthology.orchestra.run.gibba023:> sudo apt-get clean 2021-07-11T03:43:09.133 DEBUG:teuthology.orchestra.run.gibba023:> sudo apt-get update 2021-07-11T03:43:09.262 INFO:teuthology.orchestra.run.gibba023.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2021-07-11T03:43:09.396 INFO:teuthology.orchestra.run.gibba023.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:43:09.490 INFO:teuthology.orchestra.run.gibba023.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2021-07-11T03:43:09.507 INFO:teuthology.orchestra.run.gibba023.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [745 kB] 2021-07-11T03:43:09.589 INFO:teuthology.orchestra.run.gibba023.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [258 kB] 2021-07-11T03:43:09.595 INFO:teuthology.orchestra.run.gibba023.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [148 kB] 2021-07-11T03:43:09.598 INFO:teuthology.orchestra.run.gibba023.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [8,036 B] 2021-07-11T03:43:09.599 INFO:teuthology.orchestra.run.gibba023.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [16.4 kB] 2021-07-11T03:43:09.600 INFO:teuthology.orchestra.run.gibba023.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [282 kB] 2021-07-11T03:43:09.603 INFO:teuthology.orchestra.run.gibba023.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [40.9 kB] 2021-07-11T03:43:09.604 INFO:teuthology.orchestra.run.gibba023.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [456 B] 2021-07-11T03:43:09.605 INFO:teuthology.orchestra.run.gibba023.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [629 kB] 2021-07-11T03:43:09.613 INFO:teuthology.orchestra.run.gibba023.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [503 kB] 2021-07-11T03:43:09.619 INFO:teuthology.orchestra.run.gibba023.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [96.2 kB] 2021-07-11T03:43:09.620 INFO:teuthology.orchestra.run.gibba023.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [11.6 kB] 2021-07-11T03:43:09.633 INFO:teuthology.orchestra.run.gibba023.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [5,384 B] 2021-07-11T03:43:09.633 INFO:teuthology.orchestra.run.gibba023.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [19.9 kB] 2021-07-11T03:43:09.633 INFO:teuthology.orchestra.run.gibba023.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [4,316 B] 2021-07-11T03:43:09.634 INFO:teuthology.orchestra.run.gibba023.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [528 B] 2021-07-11T03:43:09.822 INFO:teuthology.orchestra.run.gibba023.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [101 kB] 2021-07-11T03:43:09.939 INFO:teuthology.orchestra.run.gibba023.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [1,086 kB] 2021-07-11T03:43:10.182 INFO:teuthology.orchestra.run.gibba023.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [505 kB] 2021-07-11T03:43:10.219 INFO:teuthology.orchestra.run.gibba023.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [239 kB] 2021-07-11T03:43:10.235 INFO:teuthology.orchestra.run.gibba023.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [13.8 kB] 2021-07-11T03:43:10.236 INFO:teuthology.orchestra.run.gibba023.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [318 kB] 2021-07-11T03:43:10.255 INFO:teuthology.orchestra.run.gibba023.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [17.7 kB] 2021-07-11T03:43:10.256 INFO:teuthology.orchestra.run.gibba023.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [46.1 kB] 2021-07-11T03:43:10.259 INFO:teuthology.orchestra.run.gibba023.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [456 B] 2021-07-11T03:43:10.259 INFO:teuthology.orchestra.run.gibba023.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [625 kB] 2021-07-11T03:43:10.287 INFO:teuthology.orchestra.run.gibba023.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [840 kB] 2021-07-11T03:43:10.314 INFO:teuthology.orchestra.run.gibba023.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [175 kB] 2021-07-11T03:43:10.341 INFO:teuthology.orchestra.run.gibba023.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [18.3 kB] 2021-07-11T03:43:10.342 INFO:teuthology.orchestra.run.gibba023.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [6,376 B] 2021-07-11T03:43:10.342 INFO:teuthology.orchestra.run.gibba023.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [23.6 kB] 2021-07-11T03:43:10.343 INFO:teuthology.orchestra.run.gibba023.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [6,472 B] 2021-07-11T03:43:10.347 INFO:teuthology.orchestra.run.gibba023.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [648 B] 2021-07-11T03:43:10.347 INFO:teuthology.orchestra.run.gibba023.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [2,568 B] 2021-07-11T03:43:10.347 INFO:teuthology.orchestra.run.gibba023.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [2,568 B] 2021-07-11T03:43:10.425 INFO:teuthology.orchestra.run.gibba023.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main Translation-en [1,120 B] 2021-07-11T03:43:10.425 INFO:teuthology.orchestra.run.gibba023.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [400 B] 2021-07-11T03:43:10.426 INFO:teuthology.orchestra.run.gibba023.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [4,724 B] 2021-07-11T03:43:10.426 INFO:teuthology.orchestra.run.gibba023.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [5,792 B] 2021-07-11T03:43:10.428 INFO:teuthology.orchestra.run.gibba023.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-backports/universe Translation-en [2,060 B] 2021-07-11T03:43:10.428 INFO:teuthology.orchestra.run.gibba023.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [288 B] 2021-07-11T03:43:12.344 INFO:teuthology.orchestra.run.gibba023.stdout:Fetched 7,037 kB in 1s (5,107 kB/s) 2021-07-11T03:43:13.405 INFO:teuthology.orchestra.run.gibba023.stdout:Reading package lists... 2021-07-11T03:43:13.417 DEBUG:teuthology.orchestra.run.gibba023:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-07-11T03:43:13.452 INFO:teuthology.orchestra.run.gibba023.stdout:Reading package lists... 2021-07-11T03:43:13.551 INFO:teuthology.orchestra.run.gibba023.stdout:Building dependency tree... 2021-07-11T03:43:13.552 INFO:teuthology.orchestra.run.gibba023.stdout:Reading state information... 2021-07-11T03:43:13.595 INFO:teuthology.orchestra.run.gibba023.stderr:E: Unable to locate package linux-image-current-generic 2021-07-11T03:43:13.596 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-07-11T03:43:13.597 DEBUG:teuthology.orchestra.run.gibba023:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-07-11T03:43:13.633 INFO:teuthology.orchestra.run.gibba023.stdout:Reading package lists... 2021-07-11T03:43:13.729 INFO:teuthology.orchestra.run.gibba023.stdout:Building dependency tree... 2021-07-11T03:43:13.730 INFO:teuthology.orchestra.run.gibba023.stdout:Reading state information... 2021-07-11T03:43:13.805 INFO:teuthology.orchestra.run.gibba023.stdout:The following additional packages will be installed: 2021-07-11T03:43:13.806 INFO:teuthology.orchestra.run.gibba023.stdout: linux-generic linux-headers-5.4.0-77 linux-headers-5.4.0-77-generic 2021-07-11T03:43:13.806 INFO:teuthology.orchestra.run.gibba023.stdout: linux-headers-generic linux-image-5.4.0-77-generic 2021-07-11T03:43:13.806 INFO:teuthology.orchestra.run.gibba023.stdout: linux-modules-5.4.0-77-generic linux-modules-extra-5.4.0-77-generic 2021-07-11T03:43:13.807 INFO:teuthology.orchestra.run.gibba023.stdout:Suggested packages: 2021-07-11T03:43:13.807 INFO:teuthology.orchestra.run.gibba023.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2021-07-11T03:43:13.807 INFO:teuthology.orchestra.run.gibba023.stdout:Recommended packages: 2021-07-11T03:43:13.807 INFO:teuthology.orchestra.run.gibba023.stdout: thermald 2021-07-11T03:43:13.819 INFO:teuthology.orchestra.run.gibba023.stdout:The following NEW packages will be installed: 2021-07-11T03:43:13.820 INFO:teuthology.orchestra.run.gibba023.stdout: linux-headers-5.4.0-77 linux-headers-5.4.0-77-generic 2021-07-11T03:43:13.820 INFO:teuthology.orchestra.run.gibba023.stdout: linux-image-5.4.0-77-generic linux-modules-5.4.0-77-generic 2021-07-11T03:43:13.821 INFO:teuthology.orchestra.run.gibba023.stdout: linux-modules-extra-5.4.0-77-generic 2021-07-11T03:43:13.821 INFO:teuthology.orchestra.run.gibba023.stdout:The following packages will be upgraded: 2021-07-11T03:43:13.822 INFO:teuthology.orchestra.run.gibba023.stdout: linux-generic linux-headers-generic linux-image-generic 2021-07-11T03:43:13.991 INFO:teuthology.orchestra.run.gibba023.stdout:3 upgraded, 5 newly installed, 0 to remove and 158 not upgraded. 2021-07-11T03:43:13.991 INFO:teuthology.orchestra.run.gibba023.stdout:Need to get 75.6 MB of archives. 2021-07-11T03:43:13.992 INFO:teuthology.orchestra.run.gibba023.stdout:After this operation, 377 MB of additional disk space will be used. 2021-07-11T03:43:13.992 INFO:teuthology.orchestra.run.gibba023.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-77-generic amd64 5.4.0-77.86 [14.8 MB] 2021-07-11T03:43:15.097 INFO:teuthology.orchestra.run.gibba023.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-77-generic amd64 5.4.0-77.86 [8,981 kB] 2021-07-11T03:43:15.351 INFO:teuthology.orchestra.run.gibba023.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-77-generic amd64 5.4.0-77.86 [39.4 MB] 2021-07-11T03:43:16.684 INFO:teuthology.orchestra.run.gibba023.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.77.80 [1,900 B] 2021-07-11T03:43:16.685 INFO:teuthology.orchestra.run.gibba023.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.77.80 [2,576 B] 2021-07-11T03:43:16.685 INFO:teuthology.orchestra.run.gibba023.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-77 all 5.4.0-77.86 [11.0 MB] 2021-07-11T03:43:17.031 INFO:teuthology.orchestra.run.gibba023.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-77-generic amd64 5.4.0-77.86 [1,410 kB] 2021-07-11T03:43:17.067 INFO:teuthology.orchestra.run.gibba023.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.77.80 [2,444 B] 2021-07-11T03:43:17.299 INFO:teuthology.orchestra.run.gibba023.stdout:Fetched 75.6 MB in 3s (23.3 MB/s) 2021-07-11T03:43:17.462 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package linux-modules-5.4.0-77-generic. 2021-07-11T03:43:19.031 INFO:teuthology.orchestra.run.gibba023.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 101691 files and directories currently installed.) 2021-07-11T03:43:19.038 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../0-linux-modules-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:43:19.087 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking linux-modules-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:43:20.552 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package linux-image-5.4.0-77-generic. 2021-07-11T03:43:20.572 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../1-linux-image-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:43:20.652 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:43:21.490 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-77-generic. 2021-07-11T03:43:21.497 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:43:21.537 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking linux-modules-extra-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:43:24.735 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../3-linux-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:43:24.886 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking linux-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:43:25.221 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:43:25.373 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking linux-image-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:43:25.725 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package linux-headers-5.4.0-77. 2021-07-11T03:43:25.745 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../5-linux-headers-5.4.0-77_5.4.0-77.86_all.deb ... 2021-07-11T03:43:25.811 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking linux-headers-5.4.0-77 (5.4.0-77.86) ... 2021-07-11T03:43:29.273 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package linux-headers-5.4.0-77-generic. 2021-07-11T03:43:29.309 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../6-linux-headers-5.4.0-77-generic_5.4.0-77.86_amd64.deb ... 2021-07-11T03:43:29.348 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking linux-headers-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:43:30.894 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.77.80_amd64.deb ... 2021-07-11T03:43:31.010 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking linux-headers-generic (5.4.0.77.80) over (5.4.0.66.69) ... 2021-07-11T03:43:31.354 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up linux-modules-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:43:32.190 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:43:32.941 INFO:teuthology.orchestra.run.gibba023.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:43:32.941 INFO:teuthology.orchestra.run.gibba023.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-77-generic 2021-07-11T03:43:33.074 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up linux-headers-5.4.0-77 (5.4.0-77.86) ... 2021-07-11T03:43:33.233 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up linux-modules-extra-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:43:34.010 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up linux-image-generic (5.4.0.77.80) ... 2021-07-11T03:43:34.161 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up linux-headers-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:43:34.262 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up linux-headers-generic (5.4.0.77.80) ... 2021-07-11T03:43:34.362 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up linux-generic (5.4.0.77.80) ... 2021-07-11T03:43:34.497 INFO:teuthology.orchestra.run.gibba023.stdout:Processing triggers for linux-image-5.4.0-77-generic (5.4.0-77.86) ... 2021-07-11T03:43:34.604 INFO:teuthology.orchestra.run.gibba023.stdout:/etc/kernel/postinst.d/initramfs-tools: 2021-07-11T03:43:34.604 INFO:teuthology.orchestra.run.gibba023.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:43:54.995 INFO:teuthology.orchestra.run.gibba023.stdout:/etc/kernel/postinst.d/zz-update-grub: 2021-07-11T03:43:54.996 INFO:teuthology.orchestra.run.gibba023.stdout:Sourcing file `/etc/default/grub' 2021-07-11T03:43:54.998 INFO:teuthology.orchestra.run.gibba023.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-07-11T03:43:55.014 INFO:teuthology.orchestra.run.gibba023.stdout:Generating grub configuration file ... 2021-07-11T03:43:55.400 INFO:teuthology.orchestra.run.gibba023.stdout:Found linux image: /boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:43:55.424 INFO:teuthology.orchestra.run.gibba023.stdout:Found initrd image: /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:43:55.855 INFO:teuthology.orchestra.run.gibba023.stdout:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-07-11T03:43:55.869 INFO:teuthology.orchestra.run.gibba023.stdout:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-07-11T03:43:58.884 INFO:teuthology.orchestra.run.gibba023.stdout:done 2021-07-11T03:43:59.163 DEBUG:teuthology.orchestra.run.gibba023:> dpkg -s linux-image-generic 2021-07-11T03:43:59.185 INFO:teuthology.orchestra.run.gibba023.stdout:Package: linux-image-generic 2021-07-11T03:43:59.185 INFO:teuthology.orchestra.run.gibba023.stdout:Status: install ok installed 2021-07-11T03:43:59.185 INFO:teuthology.orchestra.run.gibba023.stdout:Priority: optional 2021-07-11T03:43:59.185 INFO:teuthology.orchestra.run.gibba023.stdout:Section: kernel 2021-07-11T03:43:59.186 INFO:teuthology.orchestra.run.gibba023.stdout:Installed-Size: 18 2021-07-11T03:43:59.186 INFO:teuthology.orchestra.run.gibba023.stdout:Maintainer: Ubuntu Kernel Team 2021-07-11T03:43:59.186 INFO:teuthology.orchestra.run.gibba023.stdout:Architecture: amd64 2021-07-11T03:43:59.186 INFO:teuthology.orchestra.run.gibba023.stdout:Source: linux-meta 2021-07-11T03:43:59.187 INFO:teuthology.orchestra.run.gibba023.stdout:Version: 5.4.0.77.80 2021-07-11T03:43:59.187 INFO:teuthology.orchestra.run.gibba023.stdout:Provides: virtualbox-guest-modules (= 6.1.16-dfsg-6~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.9) 2021-07-11T03:43:59.187 INFO:teuthology.orchestra.run.gibba023.stdout:Depends: linux-image-5.4.0-77-generic, linux-modules-extra-5.4.0-77-generic, linux-firmware, intel-microcode, amd64-microcode 2021-07-11T03:43:59.187 INFO:teuthology.orchestra.run.gibba023.stdout:Recommends: thermald 2021-07-11T03:43:59.187 INFO:teuthology.orchestra.run.gibba023.stdout:Description: Generic Linux kernel image 2021-07-11T03:43:59.188 INFO:teuthology.orchestra.run.gibba023.stdout: This package will always depend on the latest generic kernel image 2021-07-11T03:43:59.188 INFO:teuthology.orchestra.run.gibba023.stdout: available. 2021-07-11T03:43:59.188 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-66-generic Expected: 5.4.0-77-generic 2021-07-11T03:43:59.189 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-07-11T03:43:59.189 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-07-11T03:43:59.189 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-07-11T03:43:59.189 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-07-11T03:43:59.190 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-07-11T03:43:59.190 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2021-07-11T03:43:59.190 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba004.front.sepia.ceph.com, path=None, version=distro) 2021-07-11T03:43:59.191 DEBUG:teuthology.orchestra.run.gibba004:> sudo apt-get clean 2021-07-11T03:43:59.391 DEBUG:teuthology.orchestra.run.gibba004:> sudo apt-get update 2021-07-11T03:43:59.516 INFO:teuthology.orchestra.run.gibba004.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-07-11T03:43:59.610 INFO:teuthology.orchestra.run.gibba004.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:43:59.692 INFO:teuthology.orchestra.run.gibba004.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-07-11T03:43:59.774 INFO:teuthology.orchestra.run.gibba004.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-07-11T03:44:00.517 INFO:teuthology.orchestra.run.gibba004.stdout:Reading package lists... 2021-07-11T03:44:00.528 DEBUG:teuthology.orchestra.run.gibba004:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-07-11T03:44:00.564 INFO:teuthology.orchestra.run.gibba004.stdout:Reading package lists... 2021-07-11T03:44:00.666 INFO:teuthology.orchestra.run.gibba004.stdout:Building dependency tree... 2021-07-11T03:44:00.666 INFO:teuthology.orchestra.run.gibba004.stdout:Reading state information... 2021-07-11T03:44:00.711 INFO:teuthology.orchestra.run.gibba004.stderr:E: Unable to locate package linux-image-current-generic 2021-07-11T03:44:00.712 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-07-11T03:44:00.713 DEBUG:teuthology.orchestra.run.gibba004:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-07-11T03:44:00.750 INFO:teuthology.orchestra.run.gibba004.stdout:Reading package lists... 2021-07-11T03:44:00.850 INFO:teuthology.orchestra.run.gibba004.stdout:Building dependency tree... 2021-07-11T03:44:00.851 INFO:teuthology.orchestra.run.gibba004.stdout:Reading state information... 2021-07-11T03:44:00.937 INFO:teuthology.orchestra.run.gibba004.stdout:linux-image-generic is already the newest version (5.4.0.77.80). 2021-07-11T03:44:00.937 INFO:teuthology.orchestra.run.gibba004.stdout:0 upgraded, 0 newly installed, 0 to remove and 158 not upgraded. 2021-07-11T03:44:00.939 DEBUG:teuthology.orchestra.run.gibba004:> dpkg -s linux-image-generic 2021-07-11T03:44:00.992 INFO:teuthology.orchestra.run.gibba004.stdout:Package: linux-image-generic 2021-07-11T03:44:00.992 INFO:teuthology.orchestra.run.gibba004.stdout:Status: install ok installed 2021-07-11T03:44:00.992 INFO:teuthology.orchestra.run.gibba004.stdout:Priority: optional 2021-07-11T03:44:00.993 INFO:teuthology.orchestra.run.gibba004.stdout:Section: kernel 2021-07-11T03:44:00.993 INFO:teuthology.orchestra.run.gibba004.stdout:Installed-Size: 18 2021-07-11T03:44:00.993 INFO:teuthology.orchestra.run.gibba004.stdout:Maintainer: Ubuntu Kernel Team 2021-07-11T03:44:00.993 INFO:teuthology.orchestra.run.gibba004.stdout:Architecture: amd64 2021-07-11T03:44:00.994 INFO:teuthology.orchestra.run.gibba004.stdout:Source: linux-meta 2021-07-11T03:44:00.994 INFO:teuthology.orchestra.run.gibba004.stdout:Version: 5.4.0.77.80 2021-07-11T03:44:00.994 INFO:teuthology.orchestra.run.gibba004.stdout:Provides: virtualbox-guest-modules (= 6.1.16-dfsg-6~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.9) 2021-07-11T03:44:00.994 INFO:teuthology.orchestra.run.gibba004.stdout:Depends: linux-image-5.4.0-77-generic, linux-modules-extra-5.4.0-77-generic, linux-firmware, intel-microcode, amd64-microcode 2021-07-11T03:44:00.995 INFO:teuthology.orchestra.run.gibba004.stdout:Recommends: thermald 2021-07-11T03:44:00.995 INFO:teuthology.orchestra.run.gibba004.stdout:Description: Generic Linux kernel image 2021-07-11T03:44:00.995 INFO:teuthology.orchestra.run.gibba004.stdout: This package will always depend on the latest generic kernel image 2021-07-11T03:44:00.995 INFO:teuthology.orchestra.run.gibba004.stdout: available. 2021-07-11T03:44:00.996 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:44:01.001 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.nz7FAk6qlr 2021-07-11T03:44:01.001 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /boot/grub/grub.cfg /tmp/tmp.nz7FAk6qlr 2021-07-11T03:44:01.054 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.nz7FAk6qlr 2021-07-11T03:44:01.088 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.nz7FAk6qlr is 8KB 2021-07-11T03:44:01.140 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.nz7FAk6qlr 2021-07-11T03:44:01.149 DEBUG:teuthology.orchestra.run.gibba004:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-07-11T03:44:01.155 DEBUG:teuthology.orchestra.run.gibba004:> set -ex 2021-07-11T03:44:01.155 DEBUG:teuthology.orchestra.run.gibba004:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-07-11T03:44:01.156 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-07-11T03:44:01.243 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-77-generic 2021-07-11T03:44:01.244 DEBUG:teuthology.orchestra.run.gibba004:> sudo update-grub 2021-07-11T03:44:02.508 INFO:teuthology.orchestra.run.gibba004.stderr:Sourcing file `/etc/default/grub' 2021-07-11T03:44:02.513 INFO:teuthology.orchestra.run.gibba004.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-07-11T03:44:02.575 INFO:teuthology.orchestra.run.gibba004.stderr:Generating grub configuration file ... 2021-07-11T03:44:02.897 INFO:teuthology.orchestra.run.gibba004.stderr:Found linux image: /boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:44:02.912 INFO:teuthology.orchestra.run.gibba004.stderr:Found initrd image: /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:44:03.389 INFO:teuthology.orchestra.run.gibba004.stderr:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-07-11T03:44:03.403 INFO:teuthology.orchestra.run.gibba004.stderr:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-07-11T03:44:05.368 INFO:teuthology.orchestra.run.gibba004.stderr:done 2021-07-11T03:44:05.370 DEBUG:teuthology.orchestra.run.gibba004:> sudo shutdown -r now 2021-07-11T03:44:05.374 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2021-07-11T03:44:05.375 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba005.front.sepia.ceph.com, path=None, version=distro) 2021-07-11T03:44:05.375 DEBUG:teuthology.orchestra.run.gibba005:> sudo apt-get clean 2021-07-11T03:44:05.516 DEBUG:teuthology.orchestra.run.gibba005:> sudo apt-get update 2021-07-11T03:44:05.599 INFO:teuthology.orchestra.run.gibba005.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-07-11T03:44:05.725 INFO:teuthology.orchestra.run.gibba005.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:44:05.807 INFO:teuthology.orchestra.run.gibba005.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-07-11T03:44:05.889 INFO:teuthology.orchestra.run.gibba005.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-07-11T03:44:06.716 INFO:teuthology.orchestra.run.gibba005.stdout:Reading package lists... 2021-07-11T03:44:06.727 DEBUG:teuthology.orchestra.run.gibba005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-07-11T03:44:06.763 INFO:teuthology.orchestra.run.gibba005.stdout:Reading package lists... 2021-07-11T03:44:06.860 INFO:teuthology.orchestra.run.gibba005.stdout:Building dependency tree... 2021-07-11T03:44:06.860 INFO:teuthology.orchestra.run.gibba005.stdout:Reading state information... 2021-07-11T03:44:06.904 INFO:teuthology.orchestra.run.gibba005.stderr:E: Unable to locate package linux-image-current-generic 2021-07-11T03:44:06.905 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-07-11T03:44:06.906 DEBUG:teuthology.orchestra.run.gibba005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-07-11T03:44:06.941 INFO:teuthology.orchestra.run.gibba005.stdout:Reading package lists... 2021-07-11T03:44:07.038 INFO:teuthology.orchestra.run.gibba005.stdout:Building dependency tree... 2021-07-11T03:44:07.039 INFO:teuthology.orchestra.run.gibba005.stdout:Reading state information... 2021-07-11T03:44:07.121 INFO:teuthology.orchestra.run.gibba005.stdout:linux-image-generic is already the newest version (5.4.0.77.80). 2021-07-11T03:44:07.122 INFO:teuthology.orchestra.run.gibba005.stdout:0 upgraded, 0 newly installed, 0 to remove and 158 not upgraded. 2021-07-11T03:44:07.123 DEBUG:teuthology.orchestra.run.gibba005:> dpkg -s linux-image-generic 2021-07-11T03:44:07.177 INFO:teuthology.orchestra.run.gibba005.stdout:Package: linux-image-generic 2021-07-11T03:44:07.178 INFO:teuthology.orchestra.run.gibba005.stdout:Status: install ok installed 2021-07-11T03:44:07.178 INFO:teuthology.orchestra.run.gibba005.stdout:Priority: optional 2021-07-11T03:44:07.178 INFO:teuthology.orchestra.run.gibba005.stdout:Section: kernel 2021-07-11T03:44:07.178 INFO:teuthology.orchestra.run.gibba005.stdout:Installed-Size: 18 2021-07-11T03:44:07.179 INFO:teuthology.orchestra.run.gibba005.stdout:Maintainer: Ubuntu Kernel Team 2021-07-11T03:44:07.179 INFO:teuthology.orchestra.run.gibba005.stdout:Architecture: amd64 2021-07-11T03:44:07.179 INFO:teuthology.orchestra.run.gibba005.stdout:Source: linux-meta 2021-07-11T03:44:07.179 INFO:teuthology.orchestra.run.gibba005.stdout:Version: 5.4.0.77.80 2021-07-11T03:44:07.179 INFO:teuthology.orchestra.run.gibba005.stdout:Provides: virtualbox-guest-modules (= 6.1.16-dfsg-6~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.9) 2021-07-11T03:44:07.180 INFO:teuthology.orchestra.run.gibba005.stdout:Depends: linux-image-5.4.0-77-generic, linux-modules-extra-5.4.0-77-generic, linux-firmware, intel-microcode, amd64-microcode 2021-07-11T03:44:07.180 INFO:teuthology.orchestra.run.gibba005.stdout:Recommends: thermald 2021-07-11T03:44:07.180 INFO:teuthology.orchestra.run.gibba005.stdout:Description: Generic Linux kernel image 2021-07-11T03:44:07.180 INFO:teuthology.orchestra.run.gibba005.stdout: This package will always depend on the latest generic kernel image 2021-07-11T03:44:07.181 INFO:teuthology.orchestra.run.gibba005.stdout: available. 2021-07-11T03:44:07.181 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T03:44:07.185 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.kKzzeDEig8 2021-07-11T03:44:07.185 DEBUG:teuthology.orchestra.run.gibba005:> sudo cp /boot/grub/grub.cfg /tmp/tmp.kKzzeDEig8 2021-07-11T03:44:07.197 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.kKzzeDEig8 2021-07-11T03:44:07.286 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.kKzzeDEig8 is 8KB 2021-07-11T03:44:07.301 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.kKzzeDEig8 2021-07-11T03:44:07.304 DEBUG:teuthology.orchestra.run.gibba005:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-07-11T03:44:07.362 DEBUG:teuthology.orchestra.run.gibba005:> set -ex 2021-07-11T03:44:07.362 DEBUG:teuthology.orchestra.run.gibba005:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-07-11T03:44:07.363 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-07-11T03:44:07.410 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-77-generic 2021-07-11T03:44:07.411 DEBUG:teuthology.orchestra.run.gibba005:> sudo update-grub 2021-07-11T03:44:08.737 INFO:teuthology.orchestra.run.gibba005.stderr:Sourcing file `/etc/default/grub' 2021-07-11T03:44:08.739 INFO:teuthology.orchestra.run.gibba005.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-07-11T03:44:08.789 INFO:teuthology.orchestra.run.gibba005.stderr:Generating grub configuration file ... 2021-07-11T03:44:09.120 INFO:teuthology.orchestra.run.gibba005.stderr:Found linux image: /boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:44:09.136 INFO:teuthology.orchestra.run.gibba005.stderr:Found initrd image: /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:44:09.561 INFO:teuthology.orchestra.run.gibba005.stderr:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-07-11T03:44:09.576 INFO:teuthology.orchestra.run.gibba005.stderr:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-07-11T03:44:11.415 INFO:teuthology.orchestra.run.gibba005.stderr:done 2021-07-11T03:44:11.415 DEBUG:teuthology.orchestra.run.gibba005:> sudo shutdown -r now 2021-07-11T03:44:11.418 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2021-07-11T03:44:11.418 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba019.front.sepia.ceph.com, path=None, version=distro) 2021-07-11T03:44:11.418 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-get clean 2021-07-11T03:44:11.560 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-get update 2021-07-11T03:44:11.661 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-07-11T03:44:11.845 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:44:11.938 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-07-11T03:44:12.037 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-07-11T03:44:13.119 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-07-11T03:44:13.138 DEBUG:teuthology.orchestra.run.gibba019:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-07-11T03:44:13.177 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-07-11T03:44:13.274 INFO:teuthology.orchestra.run.gibba019.stdout:Building dependency tree... 2021-07-11T03:44:13.274 INFO:teuthology.orchestra.run.gibba019.stdout:Reading state information... 2021-07-11T03:44:13.319 INFO:teuthology.orchestra.run.gibba019.stderr:E: Unable to locate package linux-image-current-generic 2021-07-11T03:44:13.320 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-07-11T03:44:13.322 DEBUG:teuthology.orchestra.run.gibba019:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-07-11T03:44:13.359 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-07-11T03:44:13.457 INFO:teuthology.orchestra.run.gibba019.stdout:Building dependency tree... 2021-07-11T03:44:13.458 INFO:teuthology.orchestra.run.gibba019.stdout:Reading state information... 2021-07-11T03:44:13.544 INFO:teuthology.orchestra.run.gibba019.stdout:linux-image-generic is already the newest version (5.4.0.77.80). 2021-07-11T03:44:13.545 INFO:teuthology.orchestra.run.gibba019.stdout:0 upgraded, 0 newly installed, 0 to remove and 158 not upgraded. 2021-07-11T03:44:13.547 DEBUG:teuthology.orchestra.run.gibba019:> dpkg -s linux-image-generic 2021-07-11T03:44:13.603 INFO:teuthology.orchestra.run.gibba019.stdout:Package: linux-image-generic 2021-07-11T03:44:13.603 INFO:teuthology.orchestra.run.gibba019.stdout:Status: install ok installed 2021-07-11T03:44:13.603 INFO:teuthology.orchestra.run.gibba019.stdout:Priority: optional 2021-07-11T03:44:13.604 INFO:teuthology.orchestra.run.gibba019.stdout:Section: kernel 2021-07-11T03:44:13.604 INFO:teuthology.orchestra.run.gibba019.stdout:Installed-Size: 18 2021-07-11T03:44:13.604 INFO:teuthology.orchestra.run.gibba019.stdout:Maintainer: Ubuntu Kernel Team 2021-07-11T03:44:13.604 INFO:teuthology.orchestra.run.gibba019.stdout:Architecture: amd64 2021-07-11T03:44:13.604 INFO:teuthology.orchestra.run.gibba019.stdout:Source: linux-meta 2021-07-11T03:44:13.605 INFO:teuthology.orchestra.run.gibba019.stdout:Version: 5.4.0.77.80 2021-07-11T03:44:13.605 INFO:teuthology.orchestra.run.gibba019.stdout:Provides: virtualbox-guest-modules (= 6.1.16-dfsg-6~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.9) 2021-07-11T03:44:13.605 INFO:teuthology.orchestra.run.gibba019.stdout:Depends: linux-image-5.4.0-77-generic, linux-modules-extra-5.4.0-77-generic, linux-firmware, intel-microcode, amd64-microcode 2021-07-11T03:44:13.605 INFO:teuthology.orchestra.run.gibba019.stdout:Recommends: thermald 2021-07-11T03:44:13.606 INFO:teuthology.orchestra.run.gibba019.stdout:Description: Generic Linux kernel image 2021-07-11T03:44:13.606 INFO:teuthology.orchestra.run.gibba019.stdout: This package will always depend on the latest generic kernel image 2021-07-11T03:44:13.606 INFO:teuthology.orchestra.run.gibba019.stdout: available. 2021-07-11T03:44:13.606 DEBUG:teuthology.orchestra.run.gibba019:> mktemp 2021-07-11T03:44:13.610 INFO:teuthology.orchestra.run.gibba019.stdout:/tmp/tmp.HPpHGIYuhR 2021-07-11T03:44:13.611 DEBUG:teuthology.orchestra.run.gibba019:> sudo cp /boot/grub/grub.cfg /tmp/tmp.HPpHGIYuhR 2021-07-11T03:44:13.618 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod 0666 /tmp/tmp.HPpHGIYuhR 2021-07-11T03:44:13.697 DEBUG:teuthology.orchestra.remote:gibba019:/tmp/tmp.HPpHGIYuhR is 8KB 2021-07-11T03:44:13.750 DEBUG:teuthology.orchestra.run.gibba019:> rm -fr /tmp/tmp.HPpHGIYuhR 2021-07-11T03:44:13.756 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-07-11T03:44:13.819 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-07-11T03:44:13.819 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-07-11T03:44:13.820 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-07-11T03:44:13.862 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-77-generic 2021-07-11T03:44:13.863 DEBUG:teuthology.orchestra.run.gibba019:> sudo update-grub 2021-07-11T03:44:14.988 INFO:teuthology.orchestra.run.gibba019.stderr:Sourcing file `/etc/default/grub' 2021-07-11T03:44:14.996 INFO:teuthology.orchestra.run.gibba019.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-07-11T03:44:15.024 INFO:teuthology.orchestra.run.gibba019.stderr:Generating grub configuration file ... 2021-07-11T03:44:15.294 INFO:teuthology.orchestra.run.gibba019.stderr:Found linux image: /boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:44:15.309 INFO:teuthology.orchestra.run.gibba019.stderr:Found initrd image: /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:44:15.766 INFO:teuthology.orchestra.run.gibba019.stderr:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-07-11T03:44:15.781 INFO:teuthology.orchestra.run.gibba019.stderr:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-07-11T03:44:17.506 INFO:teuthology.orchestra.run.gibba019.stderr:done 2021-07-11T03:44:17.508 DEBUG:teuthology.orchestra.run.gibba019:> sudo shutdown -r now 2021-07-11T03:44:17.511 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2021-07-11T03:44:17.512 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@gibba023.front.sepia.ceph.com, path=None, version=distro) 2021-07-11T03:44:17.512 DEBUG:teuthology.orchestra.run.gibba023:> sudo apt-get clean 2021-07-11T03:44:17.582 DEBUG:teuthology.orchestra.run.gibba023:> sudo apt-get update 2021-07-11T03:44:17.660 INFO:teuthology.orchestra.run.gibba023.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-07-11T03:44:17.787 INFO:teuthology.orchestra.run.gibba023.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:44:17.869 INFO:teuthology.orchestra.run.gibba023.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-07-11T03:44:17.951 INFO:teuthology.orchestra.run.gibba023.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-07-11T03:44:18.777 INFO:teuthology.orchestra.run.gibba023.stdout:Reading package lists... 2021-07-11T03:44:18.788 DEBUG:teuthology.orchestra.run.gibba023:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-07-11T03:44:18.824 INFO:teuthology.orchestra.run.gibba023.stdout:Reading package lists... 2021-07-11T03:44:18.920 INFO:teuthology.orchestra.run.gibba023.stdout:Building dependency tree... 2021-07-11T03:44:18.921 INFO:teuthology.orchestra.run.gibba023.stdout:Reading state information... 2021-07-11T03:44:18.965 INFO:teuthology.orchestra.run.gibba023.stderr:E: Unable to locate package linux-image-current-generic 2021-07-11T03:44:18.965 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-07-11T03:44:18.967 DEBUG:teuthology.orchestra.run.gibba023:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-07-11T03:44:19.003 INFO:teuthology.orchestra.run.gibba023.stdout:Reading package lists... 2021-07-11T03:44:19.098 INFO:teuthology.orchestra.run.gibba023.stdout:Building dependency tree... 2021-07-11T03:44:19.099 INFO:teuthology.orchestra.run.gibba023.stdout:Reading state information... 2021-07-11T03:44:19.182 INFO:teuthology.orchestra.run.gibba023.stdout:linux-image-generic is already the newest version (5.4.0.77.80). 2021-07-11T03:44:19.182 INFO:teuthology.orchestra.run.gibba023.stdout:0 upgraded, 0 newly installed, 0 to remove and 158 not upgraded. 2021-07-11T03:44:19.183 DEBUG:teuthology.orchestra.run.gibba023:> dpkg -s linux-image-generic 2021-07-11T03:44:19.193 INFO:teuthology.orchestra.run.gibba023.stdout:Package: linux-image-generic 2021-07-11T03:44:19.194 INFO:teuthology.orchestra.run.gibba023.stdout:Status: install ok installed 2021-07-11T03:44:19.194 INFO:teuthology.orchestra.run.gibba023.stdout:Priority: optional 2021-07-11T03:44:19.194 INFO:teuthology.orchestra.run.gibba023.stdout:Section: kernel 2021-07-11T03:44:19.194 INFO:teuthology.orchestra.run.gibba023.stdout:Installed-Size: 18 2021-07-11T03:44:19.195 INFO:teuthology.orchestra.run.gibba023.stdout:Maintainer: Ubuntu Kernel Team 2021-07-11T03:44:19.195 INFO:teuthology.orchestra.run.gibba023.stdout:Architecture: amd64 2021-07-11T03:44:19.195 INFO:teuthology.orchestra.run.gibba023.stdout:Source: linux-meta 2021-07-11T03:44:19.195 INFO:teuthology.orchestra.run.gibba023.stdout:Version: 5.4.0.77.80 2021-07-11T03:44:19.195 INFO:teuthology.orchestra.run.gibba023.stdout:Provides: virtualbox-guest-modules (= 6.1.16-dfsg-6~ubuntu1.20.04.2), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.9) 2021-07-11T03:44:19.196 INFO:teuthology.orchestra.run.gibba023.stdout:Depends: linux-image-5.4.0-77-generic, linux-modules-extra-5.4.0-77-generic, linux-firmware, intel-microcode, amd64-microcode 2021-07-11T03:44:19.196 INFO:teuthology.orchestra.run.gibba023.stdout:Recommends: thermald 2021-07-11T03:44:19.196 INFO:teuthology.orchestra.run.gibba023.stdout:Description: Generic Linux kernel image 2021-07-11T03:44:19.196 INFO:teuthology.orchestra.run.gibba023.stdout: This package will always depend on the latest generic kernel image 2021-07-11T03:44:19.197 INFO:teuthology.orchestra.run.gibba023.stdout: available. 2021-07-11T03:44:19.197 DEBUG:teuthology.orchestra.run.gibba023:> mktemp 2021-07-11T03:44:19.238 INFO:teuthology.orchestra.run.gibba023.stdout:/tmp/tmp.a4UUSyIsZh 2021-07-11T03:44:19.239 DEBUG:teuthology.orchestra.run.gibba023:> sudo cp /boot/grub/grub.cfg /tmp/tmp.a4UUSyIsZh 2021-07-11T03:44:19.246 DEBUG:teuthology.orchestra.run.gibba023:> sudo chmod 0666 /tmp/tmp.a4UUSyIsZh 2021-07-11T03:44:19.325 DEBUG:teuthology.orchestra.remote:gibba023:/tmp/tmp.a4UUSyIsZh is 8KB 2021-07-11T03:44:19.340 DEBUG:teuthology.orchestra.run.gibba023:> rm -fr /tmp/tmp.a4UUSyIsZh 2021-07-11T03:44:19.345 DEBUG:teuthology.orchestra.run.gibba023:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2021-07-11T03:44:19.395 DEBUG:teuthology.orchestra.run.gibba023:> set -ex 2021-07-11T03:44:19.396 DEBUG:teuthology.orchestra.run.gibba023:> sudo dd of=/etc/grub.d/01_ceph_kernel 2021-07-11T03:44:19.396 DEBUG:teuthology.orchestra.run.gibba023:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2021-07-11T03:44:19.456 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-77-generic 2021-07-11T03:44:19.464 DEBUG:teuthology.orchestra.run.gibba023:> sudo update-grub 2021-07-11T03:44:20.681 INFO:teuthology.orchestra.run.gibba023.stderr:Sourcing file `/etc/default/grub' 2021-07-11T03:44:20.690 INFO:teuthology.orchestra.run.gibba023.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2021-07-11T03:44:20.715 INFO:teuthology.orchestra.run.gibba023.stderr:Generating grub configuration file ... 2021-07-11T03:44:21.021 INFO:teuthology.orchestra.run.gibba023.stderr:Found linux image: /boot/vmlinuz-5.4.0-77-generic 2021-07-11T03:44:21.036 INFO:teuthology.orchestra.run.gibba023.stderr:Found initrd image: /boot/initrd.img-5.4.0-77-generic 2021-07-11T03:44:21.500 INFO:teuthology.orchestra.run.gibba023.stderr:Found linux image: /boot/vmlinuz-5.4.0-66-generic 2021-07-11T03:44:21.515 INFO:teuthology.orchestra.run.gibba023.stderr:Found initrd image: /boot/initrd.img-5.4.0-66-generic 2021-07-11T03:44:23.412 INFO:teuthology.orchestra.run.gibba023.stderr:done 2021-07-11T03:44:23.414 DEBUG:teuthology.orchestra.run.gibba023:> sudo shutdown -r now 2021-07-11T03:44:53.447 INFO:teuthology.misc:Re-opening connections... 2021-07-11T03:44:53.447 INFO:teuthology.misc:trying to connect to ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T03:44:53.448 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:44:53.449 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba004.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:45:11.883 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.104 2021-07-11T03:45:11.884 INFO:teuthology.misc:trying to connect to ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T03:45:11.884 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:45:11.885 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:45:30.319 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.105 2021-07-11T03:45:30.319 INFO:teuthology.misc:trying to connect to ubuntu@gibba019.front.sepia.ceph.com 2021-07-11T03:45:30.320 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:45:30.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:45:33.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.119 2021-07-11T03:45:33.391 INFO:teuthology.misc:trying to connect to ubuntu@gibba023.front.sepia.ceph.com 2021-07-11T03:45:33.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:45:33.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:45:51.819 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.123 2021-07-11T03:45:51.819 DEBUG:teuthology.misc:waited 58.371888160705566 2021-07-11T03:45:52.821 INFO:teuthology.misc:trying to connect to ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T03:45:52.821 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:45:52.822 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba004.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:45:55.882 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.104 2021-07-11T03:45:55.883 INFO:teuthology.misc:trying to connect to ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T03:45:55.883 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:45:55.884 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:45:58.954 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.105 2021-07-11T03:45:58.955 INFO:teuthology.misc:trying to connect to ubuntu@gibba019.front.sepia.ceph.com 2021-07-11T03:45:58.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:45:58.956 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:46:02.031 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.2.119 2021-07-11T03:46:02.031 INFO:teuthology.misc:trying to connect to ubuntu@gibba023.front.sepia.ceph.com 2021-07-11T03:46:02.031 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:46:02.032 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:46:33.798 DEBUG:teuthology.orchestra.run.gibba023:> true 2021-07-11T03:46:34.314 DEBUG:teuthology.misc:waited 100.86638736724854 2021-07-11T03:46:35.315 INFO:teuthology.misc:trying to connect to ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T03:46:35.316 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:46:35.317 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba004.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:46:35.462 DEBUG:teuthology.orchestra.run.gibba004:> true 2021-07-11T03:46:35.959 INFO:teuthology.misc:trying to connect to ubuntu@gibba019.front.sepia.ceph.com 2021-07-11T03:46:35.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:46:35.960 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:46:36.115 DEBUG:teuthology.orchestra.run.gibba019:> true 2021-07-11T03:46:36.544 DEBUG:teuthology.misc:waited 103.09633374214172 2021-07-11T03:46:37.545 INFO:teuthology.misc:trying to connect to ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T03:46:37.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:46:37.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:46:37.750 DEBUG:teuthology.orchestra.run.gibba005:> true 2021-07-11T03:46:38.133 DEBUG:teuthology.misc:waited 104.68554067611694 2021-07-11T03:46:39.134 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2021-07-11T03:46:39.134 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-77-generic"... 2021-07-11T03:46:39.135 DEBUG:teuthology.orchestra.run.gibba004:> uname -r 2021-07-11T03:46:39.144 INFO:teuthology.orchestra.run.gibba004.stdout:5.4.0-77-generic 2021-07-11T03:46:39.144 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-77-generic vs 5.4.0-77-generic 2021-07-11T03:46:39.144 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-07-11T03:46:39.145 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-07-11T03:46:39.145 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2021-07-11T03:46:39.145 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.4.0-77-generic"... 2021-07-11T03:46:39.146 DEBUG:teuthology.orchestra.run.gibba005:> uname -r 2021-07-11T03:46:39.154 INFO:teuthology.orchestra.run.gibba005.stdout:5.4.0-77-generic 2021-07-11T03:46:39.155 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-77-generic vs 5.4.0-77-generic 2021-07-11T03:46:39.155 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-07-11T03:46:39.155 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-07-11T03:46:39.156 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2021-07-11T03:46:39.156 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.4.0-77-generic"... 2021-07-11T03:46:39.156 DEBUG:teuthology.orchestra.run.gibba019:> uname -r 2021-07-11T03:46:39.166 INFO:teuthology.orchestra.run.gibba019.stdout:5.4.0-77-generic 2021-07-11T03:46:39.167 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-77-generic vs 5.4.0-77-generic 2021-07-11T03:46:39.167 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-07-11T03:46:39.167 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-07-11T03:46:39.167 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2021-07-11T03:46:39.168 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.4.0-77-generic"... 2021-07-11T03:46:39.168 DEBUG:teuthology.orchestra.run.gibba023:> uname -r 2021-07-11T03:46:39.177 INFO:teuthology.orchestra.run.gibba023.stdout:5.4.0-77-generic 2021-07-11T03:46:39.178 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-77-generic vs 5.4.0-77-generic 2021-07-11T03:46:39.178 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-07-11T03:46:39.178 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2021-07-11T03:46:40.179 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2021-07-11T03:46:40.180 DEBUG:teuthology.orchestra.run.gibba004:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-07-11T03:46:40.260 INFO:teuthology.orchestra.run.gibba004.stdout:ttyS1 2021-07-11T03:46:40.284 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2021-07-11T03:46:40.285 DEBUG:teuthology.orchestra.run.gibba005:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-07-11T03:46:40.359 INFO:teuthology.orchestra.run.gibba005.stdout:ttyS1 2021-07-11T03:46:40.396 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2021-07-11T03:46:40.397 DEBUG:teuthology.orchestra.run.gibba019:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-07-11T03:46:40.476 INFO:teuthology.orchestra.run.gibba019.stdout:ttyS1 2021-07-11T03:46:40.501 INFO:teuthology.task.kernel:Enabling kdb on client.1... 2021-07-11T03:46:40.502 DEBUG:teuthology.orchestra.run.gibba023:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-07-11T03:46:40.587 INFO:teuthology.orchestra.run.gibba023.stdout:ttyS1 2021-07-11T03:46:40.625 INFO:teuthology.run_tasks:Running task internal.base... 2021-07-11T03:46:40.637 INFO:teuthology.task.internal:Creating test directory... 2021-07-11T03:46:40.637 DEBUG:teuthology.orchestra.run.gibba004:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-07-11T03:46:40.641 DEBUG:teuthology.orchestra.run.gibba005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-07-11T03:46:40.644 DEBUG:teuthology.orchestra.run.gibba019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-07-11T03:46:40.648 DEBUG:teuthology.orchestra.run.gibba023:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-07-11T03:46:40.657 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-07-11T03:46:40.676 INFO:teuthology.run_tasks:Running task internal.archive... 2021-07-11T03:46:40.689 INFO:teuthology.task.internal:Creating archive directory... 2021-07-11T03:46:40.689 DEBUG:teuthology.orchestra.run.gibba004:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-07-11T03:46:40.692 DEBUG:teuthology.orchestra.run.gibba005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-07-11T03:46:40.695 DEBUG:teuthology.orchestra.run.gibba019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-07-11T03:46:40.698 DEBUG:teuthology.orchestra.run.gibba023:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-07-11T03:46:40.719 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-07-11T03:46:40.732 INFO:teuthology.task.internal:Enabling coredump saving... 2021-07-11T03:46:40.733 DEBUG:teuthology.orchestra.run.gibba004:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-07-11T03:46:40.756 DEBUG:teuthology.orchestra.run.gibba005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-07-11T03:46:40.759 DEBUG:teuthology.orchestra.run.gibba019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-07-11T03:46:40.762 DEBUG:teuthology.orchestra.run.gibba023:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-07-11T03:46:40.782 INFO:teuthology.orchestra.run.gibba004.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-07-11T03:46:40.788 INFO:teuthology.orchestra.run.gibba004.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-07-11T03:46:40.789 INFO:teuthology.orchestra.run.gibba005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-07-11T03:46:40.789 INFO:teuthology.orchestra.run.gibba005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-07-11T03:46:40.796 INFO:teuthology.orchestra.run.gibba019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-07-11T03:46:40.797 INFO:teuthology.orchestra.run.gibba023.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-07-11T03:46:40.799 INFO:teuthology.orchestra.run.gibba019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-07-11T03:46:40.801 INFO:teuthology.orchestra.run.gibba023.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-07-11T03:46:40.801 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-07-11T03:46:40.812 INFO:teuthology.task.internal:Configuring sudo... 2021-07-11T03:46:40.813 DEBUG:teuthology.orchestra.run.gibba004:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-07-11T03:46:40.816 DEBUG:teuthology.orchestra.run.gibba005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-07-11T03:46:40.819 DEBUG:teuthology.orchestra.run.gibba019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-07-11T03:46:40.822 DEBUG:teuthology.orchestra.run.gibba023:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-07-11T03:46:40.830 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-07-11T03:46:40.845 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-07-11T03:46:40.845 DEBUG:teuthology.orchestra.run.gibba004:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-07-11T03:46:40.868 DEBUG:teuthology.orchestra.run.gibba005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-07-11T03:46:40.871 DEBUG:teuthology.orchestra.run.gibba019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-07-11T03:46:40.874 DEBUG:teuthology.orchestra.run.gibba023:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-07-11T03:46:40.881 DEBUG:teuthology.orchestra.run.gibba004:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-07-11T03:46:40.891 DEBUG:teuthology.orchestra.run.gibba004:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-07-11T03:46:40.901 DEBUG:teuthology.orchestra.run.gibba004:> set -ex 2021-07-11T03:46:40.901 DEBUG:teuthology.orchestra.run.gibba004:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-07-11T03:46:40.938 DEBUG:teuthology.orchestra.run.gibba005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-07-11T03:46:40.943 DEBUG:teuthology.orchestra.run.gibba005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-07-11T03:46:40.994 DEBUG:teuthology.orchestra.run.gibba005:> set -ex 2021-07-11T03:46:40.995 DEBUG:teuthology.orchestra.run.gibba005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-07-11T03:46:41.024 DEBUG:teuthology.orchestra.run.gibba019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-07-11T03:46:41.034 DEBUG:teuthology.orchestra.run.gibba019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-07-11T03:46:41.087 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-07-11T03:46:41.087 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-07-11T03:46:41.113 DEBUG:teuthology.orchestra.run.gibba023:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-07-11T03:46:41.123 DEBUG:teuthology.orchestra.run.gibba023:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-07-11T03:46:41.128 DEBUG:teuthology.orchestra.run.gibba023:> set -ex 2021-07-11T03:46:41.128 DEBUG:teuthology.orchestra.run.gibba023:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-07-11T03:46:41.187 DEBUG:teuthology.orchestra.run.gibba004:> sudo service rsyslog restart 2021-07-11T03:46:41.190 DEBUG:teuthology.orchestra.run.gibba005:> sudo service rsyslog restart 2021-07-11T03:46:41.194 DEBUG:teuthology.orchestra.run.gibba019:> sudo service rsyslog restart 2021-07-11T03:46:41.197 DEBUG:teuthology.orchestra.run.gibba023:> sudo service rsyslog restart 2021-07-11T03:46:41.249 INFO:teuthology.run_tasks:Running task internal.timer... 2021-07-11T03:46:41.260 INFO:teuthology.task.internal:Starting timer... 2021-07-11T03:46:41.260 INFO:teuthology.run_tasks:Running task pcp... 2021-07-11T03:46:41.276 INFO:teuthology.run_tasks:Running task selinux... 2021-07-11T03:46:41.288 DEBUG:teuthology.task.selinux:Excluding gibba004: OS 'ubuntu' does not support SELinux 2021-07-11T03:46:41.289 DEBUG:teuthology.task.selinux:Excluding gibba005: OS 'ubuntu' does not support SELinux 2021-07-11T03:46:41.289 DEBUG:teuthology.task.selinux:Excluding gibba019: OS 'ubuntu' does not support SELinux 2021-07-11T03:46:41.289 DEBUG:teuthology.task.selinux:Excluding gibba023: OS 'ubuntu' does not support SELinux 2021-07-11T03:46:41.290 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-07-11T03:46:41.290 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2021-07-11T03:46:41.290 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-07-11T03:46:41.291 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-07-11T03:46:41.302 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated or references a specific commit; assuming it is current 2021-07-11T03:46:41.303 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to origin/master 2021-07-11T03:46:41.323 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2021-07-11T03:46:41.325 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit gibba004.front.sepia.ceph.com,gibba005.front.sepia.ceph.com,gibba019.front.sepia.ceph.com,gibba023.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-07-11T03:46:41.699 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-07-11T03:46:42.546 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-07-11T03:46:45.541 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-07-11T03:46:45.579 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:45 +0000 (0:00:00.054) 0:00:00.054 *********** 2021-07-11T03:46:45.593 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:45 +0000 (0:00:00.013) 0:00:00.067 *********** 2021-07-11T03:46:45.604 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:45 +0000 (0:00:00.011) 0:00:00.078 *********** 2021-07-11T03:46:45.615 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:45 +0000 (0:00:00.011) 0:00:00.090 *********** 2021-07-11T03:46:47.615 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-07-11T03:46:47.617 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 20.04 on host gibba023.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba023.front.sepia.ceph.com] 2021-07-11T03:46:47.672 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:47 +0000 (0:00:02.056) 0:00:02.146 *********** 2021-07-11T03:46:47.749 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 20.04 on host gibba019.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba019.front.sepia.ceph.com] 2021-07-11T03:46:47.944 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:47 +0000 (0:00:00.272) 0:00:02.419 *********** 2021-07-11T03:46:47.957 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 20.04 on host gibba004.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. 2021-07-11T03:46:47.959 INFO:teuthology.task.ansible.out: ok: [gibba004.front.sepia.ceph.com] 2021-07-11T03:46:47.995 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 20.04 on host gibba005.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [gibba005.front.sepia.ceph.com] 2021-07-11T03:46:48.034 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:48 +0000 (0:00:00.089) 0:00:02.509 *********** 2021-07-11T03:46:48.072 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-07-11T03:46:48.074 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-07-11T03:46:48.097 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:48 +0000 (0:00:00.063) 0:00:02.572 *********** 2021-07-11T03:46:48.118 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:48 +0000 (0:00:00.020) 0:00:02.593 *********** 2021-07-11T03:46:48.147 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:48 +0000 (0:00:00.028) 0:00:02.621 *********** 2021-07-11T03:46:48.158 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-07-11T03:46:48.182 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:48 +0000 (0:00:00.035) 0:00:02.657 *********** 2021-07-11T03:46:48.212 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-07-11T03:46:48.237 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:48 +0000 (0:00:00.055) 0:00:02.712 *********** 2021-07-11T03:46:48.629 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-07-11T03:46:48.630 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-07-11T03:46:48.657 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:48 +0000 (0:00:00.419) 0:00:03.132 *********** 2021-07-11T03:46:48.671 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-07-11T03:46:48.697 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:48 +0000 (0:00:00.039) 0:00:03.172 *********** 2021-07-11T03:46:48.710 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-07-11T03:46:48.737 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:48 +0000 (0:00:00.039) 0:00:03.211 *********** 2021-07-11T03:46:48.744 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-07-11T03:46:48.773 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:48 +0000 (0:00:00.034) 0:00:03.246 *********** 2021-07-11T03:46:49.016 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.059408", "end": "2021-07-11 03:46:49.336645", "rc": 0, "start": "2021-07-11 03:46:49.277237", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-07-11T03:46:49.041 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:49 +0000 (0:00:00.269) 0:00:03.516 *********** 2021-07-11T03:46:49.053 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.072542", "end": "2021-07-11 03:46:48.950620", "rc": 0, "start": "2021-07-11 03:46:48.878078", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.089062", "end": "2021-07-11 03:46:49.230160", "rc": 0, "start": "2021-07-11 03:46:49.141098", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.062024", "end": "2021-07-11 03:46:49.476431", "rc": 0, "start": "2021-07-11 03:46:49.414407", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-07-11T03:46:49.078 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:49 +0000 (0:00:00.035) 0:00:03.552 *********** 2021-07-11T03:46:49.100 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:49 +0000 (0:00:00.022) 0:00:03.575 *********** 2021-07-11T03:46:49.127 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:49 +0000 (0:00:00.027) 0:00:03.602 *********** 2021-07-11T03:46:49.291 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-07-11T03:46:49.293 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:46:49.295 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [gibba023.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:46:49.296 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:46:49.322 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:49 +0000 (0:00:00.194) 0:00:03.797 *********** 2021-07-11T03:46:49.354 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:49 +0000 (0:00:00.026) 0:00:03.823 *********** 2021-07-11T03:46:49.376 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:49 +0000 (0:00:00.027) 0:00:03.850 *********** 2021-07-11T03:46:49.406 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:49 +0000 (0:00:00.025) 0:00:03.876 *********** 2021-07-11T03:46:50.109 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2021-07-11T03:46:50.111 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975209.4287033-8781-92849691656954/source", "state": "file", "uid": 0} changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975209.3571274-8775-276909873184531/source", "state": "file", "uid": 0} 2021-07-11T03:46:50.134 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:50 +0000 (0:00:00.733) 0:00:04.609 *********** 2021-07-11T03:46:50.159 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:50 +0000 (0:00:00.024) 0:00:04.634 *********** 2021-07-11T03:46:50.167 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975209.489001-8794-191802729175930/source", "state": "file", "uid": 0} 2021-07-11T03:46:50.168 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975209.433429-8788-209512766377075/source", "state": "file", "uid": 0} 2021-07-11T03:46:50.194 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:50 +0000 (0:00:00.034) 0:00:04.669 *********** 2021-07-11T03:46:50.220 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:50 +0000 (0:00:00.025) 0:00:04.695 *********** 2021-07-11T03:46:50.492 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCt 2021-07-11T03:46:50.494 INFO:teuthology.task.ansible.out:oHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cD 2021-07-11T03:46:50.496 INFO:teuthology.task.ansible.out:nCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2z 2021-07-11T03:46:50.497 INFO:teuthology.task.ansible.out:wUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-07-11T03:46:50.537 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4 2021-07-11T03:46:50.539 INFO:teuthology.task.ansible.out:r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.c 2021-07-11T03:46:50.541 INFO:teuthology.task.ansible.out:om\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ak 2021-07-11T03:46:50.542 INFO:teuthology.task.ansible.out:lap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-07-11T03:46:50.657 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-07-11T03:46:50.660 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-07-11T03:46:50.676 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-07-11T03:46:50.678 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:50 +0000 (0:00:00.456) 0:00:05.151 *********** 2021-07-11T03:46:51.107 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] 2021-07-11T03:46:51.216 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] 2021-07-11T03:46:51.230 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] 2021-07-11T03:46:51.251 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] 2021-07-11T03:46:51.272 INFO:teuthology.task.ansible.out: TASK [set_fact] **************************************************************** Sunday 11 July 2021 03:46:51 +0000 (0:00:00.595) 0:00:05.747 *********** 2021-07-11T03:46:51.308 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"ran_from_cephlab_playbook": true}, "changed": false} 2021-07-11T03:46:51.332 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"ran_from_cephlab_playbook": true}, "changed": false} 2021-07-11T03:46:51.357 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"ran_from_cephlab_playbook": true}, "changed": false} 2021-07-11T03:46:51.368 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"ran_from_cephlab_playbook": true}, "changed": false} 2021-07-11T03:46:51.373 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-07-11T03:46:51.548 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:51 +0000 (0:00:00.275) 0:00:06.022 *********** 2021-07-11T03:46:51.569 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:51 +0000 (0:00:00.021) 0:00:06.044 *********** 2021-07-11T03:46:51.592 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:51 +0000 (0:00:00.022) 0:00:06.067 *********** 2021-07-11T03:46:51.614 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:51 +0000 (0:00:00.022) 0:00:06.089 *********** 2021-07-11T03:46:52.224 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba004.front.sepia.ceph.com] 2021-07-11T03:46:52.235 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] 2021-07-11T03:46:52.291 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] 2021-07-11T03:46:52.325 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2021-07-11T03:46:52.327 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:52 +0000 (0:00:00.711) 0:00:06.800 *********** 2021-07-11T03:46:52.352 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:52 +0000 (0:00:00.026) 0:00:06.827 *********** 2021-07-11T03:46:52.380 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] 2021-07-11T03:46:52.427 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:52 +0000 (0:00:00.074) 0:00:06.901 *********** 2021-07-11T03:46:52.465 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:52 +0000 (0:00:00.038) 0:00:06.940 *********** 2021-07-11T03:46:52.852 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-07-11T03:46:52.854 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.228658", "end": "2021-07-11 03:46:52.823521", "rc": 0, "start": "2021-07-11 03:46:52.594863", "stderr": "", "stderr_lines": [], "stdout": "444af9608e8be25a2d7a2892fb588d0daeff9780", "stdout_lines": ["444af9608e8be25a2d7a2892fb588d0daeff9780"]} 2021-07-11T03:46:52.886 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:52 +0000 (0:00:00.418) 0:00:07.358 *********** 2021-07-11T03:46:52.920 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.236712", "end": "2021-07-11 03:46:52.877682", "rc": 0, "start": "2021-07-11 03:46:52.640970", "stderr": "", "stderr_lines": [], "stdout": "444af9608e8be25a2d7a2892fb588d0daeff9780", "stdout_lines": ["444af9608e8be25a2d7a2892fb588d0daeff9780"]} 2021-07-11T03:46:52.950 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:52 +0000 (0:00:00.066) 0:00:07.424 *********** 2021-07-11T03:46:52.991 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.242694", "end": "2021-07-11 03:46:52.935577", "rc": 0, "start": "2021-07-11 03:46:52.692883", "stderr": "", "stderr_lines": [], "stdout": "444af9608e8be25a2d7a2892fb588d0daeff9780", "stdout_lines": ["444af9608e8be25a2d7a2892fb588d0daeff9780"]} 2021-07-11T03:46:53.030 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.079) 0:00:07.505 *********** 2021-07-11T03:46:53.069 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-07-11T03:46:53.071 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.026104", "end": "2021-07-11 03:46:53.265947", "failed_when_result": false, "rc": 0, "start": "2021-07-11 03:46:53.239843", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-07-11T03:46:53.098 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.067) 0:00:07.572 *********** 2021-07-11T03:46:53.109 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013897", "end": "2021-07-11 03:46:53.517449", "failed_when_result": false, "rc": 0, "start": "2021-07-11 03:46:53.503552", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-07-11T03:46:53.136 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.038) 0:00:07.611 *********** 2021-07-11T03:46:53.151 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-07-11T03:46:53.153 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.217272", "end": "2021-07-11 03:46:53.090936", "rc": 0, "start": "2021-07-11 03:46:52.873664", "stderr": "", "stderr_lines": [], "stdout": "444af9608e8be25a2d7a2892fb588d0daeff9780", "stdout_lines": ["444af9608e8be25a2d7a2892fb588d0daeff9780"]} 2021-07-11T03:46:53.187 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.050) 0:00:07.662 *********** 2021-07-11T03:46:53.194 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-07-11T03:46:53.196 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.014328", "end": "2021-07-11 03:46:53.498271", "failed_when_result": false, "rc": 0, "start": "2021-07-11 03:46:53.483943", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} TASK [Determine if we can skip users and pubkeys updates] ********************** 2021-07-11T03:46:53.196 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.228 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.041) 0:00:07.703 *********** 2021-07-11T03:46:53.260 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.031) 0:00:07.734 *********** 2021-07-11T03:46:53.268 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.302 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.042) 0:00:07.776 *********** 2021-07-11T03:46:53.304 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:53.320 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-07-11T03:46:53.322 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.008949", "end": "2021-07-11 03:46:53.238124", "failed_when_result": false, "rc": 0, "start": "2021-07-11 03:46:53.229175", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-07-11T03:46:53.349 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.047) 0:00:07.824 *********** 2021-07-11T03:46:53.373 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.023) 0:00:07.847 *********** 2021-07-11T03:46:53.382 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:53.383 INFO:teuthology.task.ansible.out:TASK [Determine if we can skip users and pubkeys updates] ********************** 2021-07-11T03:46:53.385 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:53.387 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [Merge extra_admin_users into managed_admin_users] ************************ skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.417 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.044) 0:00:07.892 *********** 2021-07-11T03:46:53.449 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.029) 0:00:07.922 *********** 2021-07-11T03:46:53.460 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** 2021-07-11T03:46:53.462 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.492 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.044) 0:00:07.966 *********** 2021-07-11T03:46:53.512 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** 2021-07-11T03:46:53.518 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.555 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.062) 0:00:08.030 *********** 2021-07-11T03:46:53.563 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-07-11T03:46:53.564 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [Remove managed_admin_users from managed_users] *************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.607 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.051) 0:00:08.082 *********** 2021-07-11T03:46:53.638 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.031) 0:00:08.113 *********** 2021-07-11T03:46:53.650 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.690 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.051) 0:00:08.165 *********** 2021-07-11T03:46:53.704 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:53.713 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-07-11T03:46:53.716 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.717 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-07-11T03:46:53.732 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.768 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.077) 0:00:08.242 *********** 2021-07-11T03:46:53.794 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.026) 0:00:08.269 *********** 2021-07-11T03:46:53.804 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** 2021-07-11T03:46:53.806 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.847 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.052) 0:00:08.322 *********** 2021-07-11T03:46:53.856 INFO:teuthology.task.ansible.out: TASK [Filter the managed_admin_users list] ************************************* 2021-07-11T03:46:53.858 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.892 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.045) 0:00:08.367 *********** 2021-07-11T03:46:53.904 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [Remove managed_admin_users from managed_users] *************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:53.930 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.037) 0:00:08.405 *********** 2021-07-11T03:46:53.969 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:53 +0000 (0:00:00.039) 0:00:08.444 *********** 2021-07-11T03:46:53.977 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-07-11T03:46:53.979 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:54.006 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:54 +0000 (0:00:00.036) 0:00:08.480 *********** 2021-07-11T03:46:54.057 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:54.086 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:54 +0000 (0:00:00.077) 0:00:08.558 *********** 2021-07-11T03:46:54.100 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-07-11T03:46:54.102 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [Filter the managed_admin_users list] ************************************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:54.133 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:54 +0000 (0:00:00.048) 0:00:08.607 *********** 2021-07-11T03:46:54.156 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:54 +0000 (0:00:00.023) 0:00:08.631 *********** 2021-07-11T03:46:54.169 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-07-11T03:46:54.170 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [Filter the managed_admin_users list] ************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:46:54.210 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:54 +0000 (0:00:00.053) 0:00:08.685 *********** 2021-07-11T03:46:54.240 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:54 +0000 (0:00:00.029) 0:00:08.714 *********** 2021-07-11T03:46:54.332 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-07-11T03:46:54.334 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:46:54.366 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:54 +0000 (0:00:00.126) 0:00:08.841 *********** 2021-07-11T03:46:54.466 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:46:54.490 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:46:54 +0000 (0:00:00.123) 0:00:08.964 *********** 2021-07-11T03:46:54.536 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-07-11T03:46:54.659 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-07-11T03:46:54.661 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:54.691 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-07-11T03:46:54.768 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-07-11T03:46:54.784 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-07-11T03:46:54.833 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-07-11T03:46:54.892 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-07-11T03:46:54.905 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-07-11T03:46:54.906 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-07-11T03:46:54.918 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-07-11T03:46:55.005 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-07-11T03:46:55.008 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-07-11T03:46:55.019 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-07-11T03:46:55.034 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-07-11T03:46:55.099 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-07-11T03:46:55.114 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-07-11T03:46:55.123 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-07-11T03:46:55.137 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-07-11T03:46:55.228 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-07-11T03:46:55.243 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-07-11T03:46:55.244 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-07-11T03:46:55.246 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-07-11T03:46:55.333 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-07-11T03:46:55.337 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-07-11T03:46:55.350 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-07-11T03:46:55.362 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-07-11T03:46:55.443 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-07-11T03:46:55.475 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-07-11T03:46:55.543 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-07-11T03:46:55.548 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-07-11T03:46:55.562 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-07-11T03:46:55.584 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-07-11T03:46:55.659 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-07-11T03:46:55.661 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-07-11T03:46:55.673 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-07-11T03:46:55.720 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-07-11T03:46:55.753 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-07-11T03:46:55.780 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-07-11T03:46:55.782 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-07-11T03:46:55.825 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-07-11T03:46:55.858 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-07-11T03:46:55.881 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-07-11T03:46:55.882 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-07-11T03:46:55.926 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-07-11T03:46:55.928 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:55.960 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-07-11T03:46:55.985 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-07-11T03:46:55.996 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-07-11T03:46:56.029 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-07-11T03:46:56.031 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:56.063 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-07-11T03:46:56.096 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-07-11T03:46:56.108 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-07-11T03:46:56.145 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-07-11T03:46:56.168 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-07-11T03:46:56.210 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-07-11T03:46:56.235 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-07-11T03:46:56.237 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-07-11T03:46:56.291 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-07-11T03:46:56.293 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:56.305 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-07-11T03:46:56.341 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-07-11T03:46:56.342 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-07-11T03:46:56.397 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-07-11T03:46:56.442 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-07-11T03:46:56.454 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-07-11T03:46:56.476 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-07-11T03:46:56.500 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-07-11T03:46:56.555 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-07-11T03:46:56.556 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-07-11T03:46:56.579 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-07-11T03:46:56.583 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:56.592 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-07-11T03:46:56.646 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-07-11T03:46:56.679 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-07-11T03:46:56.690 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-07-11T03:46:56.692 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-07-11T03:46:56.759 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-07-11T03:46:56.793 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-07-11T03:46:56.805 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-07-11T03:46:56.828 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-07-11T03:46:56.861 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-07-11T03:46:56.886 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-07-11T03:46:56.909 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-07-11T03:46:56.923 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-07-11T03:46:56.957 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-07-11T03:46:56.959 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:56.980 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-07-11T03:46:57.003 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-07-11T03:46:57.015 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-07-11T03:46:57.060 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-07-11T03:46:57.105 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d 2021-07-11T03:46:57.108 INFO:teuthology.task.ansible.out:7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-07-11T03:46:57.117 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-07-11T03:46:57.162 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-07-11T03:46:57.206 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-07-11T03:46:57.217 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-07-11T03:46:57.219 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-07-11T03:46:57.264 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-07-11T03:46:57.308 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-07-11T03:46:57.320 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-07-11T03:46:57.321 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-07-11T03:46:57.368 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-07-11T03:46:57.415 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-07-11T03:46:57.427 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-07-11T03:46:57.452 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-07-11T03:46:57.475 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-07-11T03:46:57.524 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-07-11T03:46:57.536 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-07-11T03:46:57.548 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-07-11T03:46:57.582 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-07-11T03:46:57.637 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-07-11T03:46:57.652 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-07-11T03:46:57.664 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-07-11T03:46:57.687 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-07-11T03:46:57.731 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-07-11T03:46:57.743 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-07-11T03:46:57.756 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-07-11T03:46:57.790 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-07-11T03:46:57.834 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-07-11T03:46:57.846 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-07-11T03:46:57.847 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-07-11T03:46:57.880 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-07-11T03:46:57.924 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-07-11T03:46:57.938 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-07-11T03:46:57.939 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-07-11T03:46:57.982 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-07-11T03:46:58.026 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-07-11T03:46:58.038 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-07-11T03:46:58.050 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-07-11T03:46:58.073 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-07-11T03:46:58.118 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-07-11T03:46:58.151 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-07-11T03:46:58.155 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-07-11T03:46:58.168 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-07-11T03:46:58.212 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-07-11T03:46:58.266 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-07-11T03:46:58.268 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-07-11T03:46:58.281 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-07-11T03:46:58.326 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-07-11T03:46:58.327 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:58.361 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-07-11T03:46:58.373 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-07-11T03:46:58.385 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-07-11T03:46:58.418 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-07-11T03:46:58.420 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:58.453 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-07-11T03:46:58.476 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-07-11T03:46:58.502 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-07-11T03:46:58.526 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-07-11T03:46:58.550 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-07-11T03:46:58.589 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-07-11T03:46:58.611 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-07-11T03:46:58.635 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-07-11T03:46:58.669 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-07-11T03:46:58.703 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-07-11T03:46:58.715 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-07-11T03:46:58.749 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-07-11T03:46:58.802 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-07-11T03:46:58.838 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-07-11T03:46:58.862 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-07-11T03:46:58.908 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-07-11T03:46:58.931 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-07-11T03:46:58.932 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:58.936 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-07-11T03:46:58.961 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-07-11T03:46:58.993 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-07-11T03:46:59.048 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-07-11T03:46:59.049 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-07-11T03:46:59.071 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-07-11T03:46:59.083 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-07-11T03:46:59.158 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-07-11T03:46:59.159 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-07-11T03:46:59.183 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-07-11T03:46:59.184 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-07-11T03:46:59.259 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-07-11T03:46:59.271 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-07-11T03:46:59.283 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-07-11T03:46:59.284 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-07-11T03:46:59.361 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-07-11T03:46:59.373 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-07-11T03:46:59.388 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-07-11T03:46:59.390 INFO:teuthology.task.ansible.out: 2021-07-11T03:46:59.392 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-07-11T03:46:59.471 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-07-11T03:46:59.472 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-07-11T03:46:59.508 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-07-11T03:46:59.530 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-07-11T03:46:59.564 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-07-11T03:46:59.575 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-07-11T03:46:59.610 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-07-11T03:46:59.648 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-07-11T03:46:59.672 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-07-11T03:46:59.683 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-07-11T03:46:59.727 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-07-11T03:46:59.730 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-07-11T03:46:59.788 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-07-11T03:46:59.801 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-07-11T03:46:59.838 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-07-11T03:46:59.851 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-07-11T03:46:59.905 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-07-11T03:46:59.906 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-07-11T03:46:59.929 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-07-11T03:46:59.940 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-07-11T03:47:00.006 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-07-11T03:47:00.008 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-07-11T03:47:00.029 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-07-11T03:47:00.042 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-07-11T03:47:00.110 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-07-11T03:47:00.111 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-07-11T03:47:00.122 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-07-11T03:47:00.134 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-07-11T03:47:00.211 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-07-11T03:47:00.223 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-07-11T03:47:00.224 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-07-11T03:47:00.238 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-07-11T03:47:00.319 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-07-11T03:47:00.343 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-07-11T03:47:00.365 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-07-11T03:47:00.377 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-07-11T03:47:00.379 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:00.445 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-07-11T03:47:00.446 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-07-11T03:47:00.471 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-07-11T03:47:00.472 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-07-11T03:47:00.472 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:00.553 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-07-11T03:47:00.576 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-07-11T03:47:00.578 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:00.590 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-07-11T03:47:00.591 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:00.593 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-07-11T03:47:00.659 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-07-11T03:47:00.661 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:00.675 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-07-11T03:47:00.699 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-07-11T03:47:00.708 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-07-11T03:47:00.710 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:00.776 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-07-11T03:47:00.811 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-07-11T03:47:00.813 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-07-11T03:47:00.824 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-07-11T03:47:00.870 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-07-11T03:47:00.937 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-07-11T03:47:00.957 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-07-11T03:47:00.961 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-07-11T03:47:00.989 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-07-11T03:47:01.058 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-07-11T03:47:01.070 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-07-11T03:47:01.104 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-07-11T03:47:01.158 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-07-11T03:47:01.172 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-07-11T03:47:01.176 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-07-11T03:47:01.229 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-07-11T03:47:01.265 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-07-11T03:47:01.287 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-07-11T03:47:01.300 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-07-11T03:47:01.333 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-07-11T03:47:01.371 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-07-11T03:47:01.394 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-07-11T03:47:01.412 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-07-11T03:47:01.452 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-07-11T03:47:01.455 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:01.479 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-07-11T03:47:01.524 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-07-11T03:47:01.525 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:01.537 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-07-11T03:47:01.600 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-07-11T03:47:01.601 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-07-11T03:47:01.678 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-07-11T03:47:01.690 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-07-11T03:47:01.702 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-07-11T03:47:01.727 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-07-11T03:47:01.797 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-07-11T03:47:01.810 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-07-11T03:47:01.862 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-07-11T03:47:01.874 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-07-11T03:47:01.886 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-07-11T03:47:01.943 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-07-11T03:47:01.968 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-07-11T03:47:01.994 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-07-11T03:47:02.010 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-07-11T03:47:02.062 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-07-11T03:47:02.094 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-07-11T03:47:02.117 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-07-11T03:47:02.129 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-07-11T03:47:02.188 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-07-11T03:47:02.210 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-07-11T03:47:02.245 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-07-11T03:47:02.285 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-07-11T03:47:02.368 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-07-11T03:47:02.369 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-07-11T03:47:02.382 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-07-11T03:47:02.405 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-07-11T03:47:02.507 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-07-11T03:47:02.510 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-07-11T03:47:02.525 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-07-11T03:47:02.527 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:02.538 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-07-11T03:47:02.540 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:02.683 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-07-11T03:47:02.706 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-07-11T03:47:02.707 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-07-11T03:47:02.720 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:02.744 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-07-11T03:47:02.832 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-07-11T03:47:02.895 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-07-11T03:47:02.899 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-07-11T03:47:02.944 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-07-11T03:47:02.979 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-07-11T03:47:02.991 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-07-11T03:47:03.007 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-07-11T03:47:03.057 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-07-11T03:47:03.102 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-07-11T03:47:03.141 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-07-11T03:47:03.158 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-07-11T03:47:03.192 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-07-11T03:47:03.219 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-07-11T03:47:03.247 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-07-11T03:47:03.809 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, 2021-07-11T03:47:03.811 INFO:teuthology.task.ansible.out: "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state" 2021-07-11T03:47:03.814 INFO:teuthology.task.ansible.out:: "present", "uid": 1177} ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} ok: [gibba004.front.sepia.ceph.com] => (item={'name': 2021-07-11T03:47:03.818 INFO:teuthology.task.ansible.out:'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50 2021-07-11T03:47:03.822 INFO:teuthology.task.ansible.out:ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-07-11T03:47:03.896 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-07-11T03:47:03.911 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-07-11T03:47:03.922 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-07-11T03:47:03.945 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-07-11T03:47:04.048 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-07-11T03:47:04.049 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-07-11T03:47:04.084 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-07-11T03:47:04.109 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-07-11T03:47:04.183 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-07-11T03:47:04.195 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-07-11T03:47:04.269 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-07-11T03:47:04.308 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-07-11T03:47:04.322 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-07-11T03:47:04.364 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-07-11T03:47:04.442 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-07-11T03:47:04.452 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-07-11T03:47:04.491 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-07-11T03:47:04.503 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-07-11T03:47:04.584 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-07-11T03:47:04.618 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-07-11T03:47:04.638 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-07-11T03:47:04.643 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:04.664 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-07-11T03:47:04.711 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-07-11T03:47:04.729 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-07-11T03:47:04.803 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-07-11T03:47:04.822 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-07-11T03:47:04.823 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-07-11T03:47:04.878 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-07-11T03:47:04.905 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-07-11T03:47:04.906 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-07-11T03:47:04.976 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-07-11T03:47:04.988 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-07-11T03:47:05.028 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-07-11T03:47:05.050 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-07-11T03:47:05.108 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-07-11T03:47:05.159 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-07-11T03:47:05.160 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-07-11T03:47:05.204 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-07-11T03:47:05.227 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-07-11T03:47:05.276 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-07-11T03:47:05.351 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-07-11T03:47:05.360 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-07-11T03:47:05.364 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-07-11T03:47:05.421 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-07-11T03:47:05.467 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-07-11T03:47:05.494 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-07-11T03:47:05.506 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-07-11T03:47:05.567 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-07-11T03:47:05.580 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-07-11T03:47:05.582 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:05.593 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-07-11T03:47:05.614 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-07-11T03:47:05.684 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-07-11T03:47:05.914 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-07-11T03:47:05.916 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-07-11T03:47:05.922 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-07-11T03:47:05.923 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-07-11T03:47:05.925 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-07-11T03:47:05.926 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-07-11T03:47:05.927 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:05.928 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-07-11T03:47:06.036 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-07-11T03:47:06.039 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-07-11T03:47:06.074 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-07-11T03:47:06.075 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-07-11T03:47:06.143 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-07-11T03:47:06.155 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-07-11T03:47:06.180 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-07-11T03:47:06.216 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-07-11T03:47:06.282 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-07-11T03:47:06.316 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-07-11T03:47:06.344 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-07-11T03:47:06.404 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-07-11T03:47:06.406 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:06.420 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-07-11T03:47:06.448 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-07-11T03:47:06.492 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-07-11T03:47:06.546 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-07-11T03:47:06.560 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-07-11T03:47:06.572 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-07-11T03:47:06.679 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-07-11T03:47:06.706 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-07-11T03:47:06.707 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-07-11T03:47:06.726 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-07-11T03:47:06.835 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-07-11T03:47:06.852 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-07-11T03:47:06.853 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-07-11T03:47:06.910 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-07-11T03:47:06.954 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-07-11T03:47:06.955 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-07-11T03:47:07.004 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-07-11T03:47:07.052 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-07-11T03:47:07.107 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-07-11T03:47:07.150 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-07-11T03:47:07.163 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-07-11T03:47:07.165 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:07.167 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-07-11T03:47:07.170 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:07.218 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-07-11T03:47:07.264 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-07-11T03:47:07.276 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-07-11T03:47:07.316 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-07-11T03:47:07.318 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-07-11T03:47:07.387 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-07-11T03:47:07.391 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:07.416 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-07-11T03:47:07.419 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:07.432 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-07-11T03:47:07.454 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-07-11T03:47:07.503 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-07-11T03:47:07.554 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-07-11T03:47:07.560 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-07-11T03:47:07.625 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-07-11T03:47:07.641 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-07-11T03:47:07.713 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-07-11T03:47:07.723 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-07-11T03:47:07.749 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-07-11T03:47:07.784 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-07-11T03:47:07.819 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-07-11T03:47:07.870 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-07-11T03:47:07.901 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-07-11T03:47:07.924 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-07-11T03:47:07.954 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-07-11T03:47:07.996 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-07-11T03:47:07.997 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-07-11T03:47:08.048 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-07-11T03:47:08.083 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-07-11T03:47:08.142 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-07-11T03:47:08.150 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-07-11T03:47:08.209 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-07-11T03:47:08.224 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-07-11T03:47:08.254 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-07-11T03:47:08.256 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-07-11T03:47:08.260 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:08.331 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-07-11T03:47:08.383 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-07-11T03:47:08.398 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-07-11T03:47:08.422 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-07-11T03:47:08.457 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-07-11T03:47:08.504 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-07-11T03:47:08.505 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-07-11T03:47:08.559 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-07-11T03:47:08.584 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-07-11T03:47:08.622 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-07-11T03:47:08.667 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-07-11T03:47:08.676 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-07-11T03:47:08.692 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-07-11T03:47:08.776 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-07-11T03:47:08.816 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-07-11T03:47:08.817 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-07-11T03:47:08.819 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-07-11T03:47:08.911 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-07-11T03:47:08.934 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-07-11T03:47:08.968 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-07-11T03:47:08.980 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-07-11T03:47:09.046 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-07-11T03:47:09.095 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-07-11T03:47:09.096 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:09.118 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-07-11T03:47:09.161 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-07-11T03:47:09.199 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-07-11T03:47:09.241 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-07-11T03:47:09.242 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-07-11T03:47:09.279 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-07-11T03:47:09.374 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-07-11T03:47:09.385 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-07-11T03:47:09.401 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-07-11T03:47:09.429 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-07-11T03:47:09.516 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-07-11T03:47:09.517 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-07-11T03:47:09.594 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-07-11T03:47:09.595 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-07-11T03:47:09.658 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-07-11T03:47:09.698 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-07-11T03:47:09.699 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-07-11T03:47:09.765 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-07-11T03:47:09.797 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-07-11T03:47:09.824 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-07-11T03:47:09.825 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-07-11T03:47:09.978 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-07-11T03:47:09.990 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-07-11T03:47:10.023 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-07-11T03:47:10.054 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-07-11T03:47:10.124 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-07-11T03:47:10.186 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-07-11T03:47:10.221 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-07-11T03:47:10.247 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-07-11T03:47:10.280 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-07-11T03:47:10.345 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-07-11T03:47:10.382 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-07-11T03:47:10.383 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-07-11T03:47:10.489 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-07-11T03:47:10.504 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-07-11T03:47:10.544 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-07-11T03:47:10.582 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-07-11T03:47:10.623 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-07-11T03:47:10.632 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-07-11T03:47:10.720 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-07-11T03:47:10.769 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-07-11T03:47:10.771 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-07-11T03:47:10.793 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-07-11T03:47:10.839 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-07-11T03:47:10.902 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-07-11T03:47:10.914 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-07-11T03:47:10.941 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-07-11T03:47:10.984 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-07-11T03:47:11.091 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-07-11T03:47:11.103 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-07-11T03:47:11.104 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-07-11T03:47:11.182 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-07-11T03:47:11.206 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-07-11T03:47:11.225 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-07-11T03:47:11.260 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-07-11T03:47:11.300 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-07-11T03:47:11.336 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-07-11T03:47:11.381 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-07-11T03:47:11.421 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-07-11T03:47:11.424 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-07-11T03:47:11.506 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-07-11T03:47:11.545 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-07-11T03:47:11.567 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-07-11T03:47:11.580 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-07-11T03:47:11.644 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-07-11T03:47:11.681 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-07-11T03:47:11.699 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-07-11T03:47:11.760 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-07-11T03:47:11.826 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-07-11T03:47:11.837 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-07-11T03:47:11.888 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-07-11T03:47:11.939 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-07-11T03:47:11.965 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-07-11T03:47:11.966 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-07-11T03:47:12.011 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-07-11T03:47:12.107 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-07-11T03:47:12.122 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-07-11T03:47:12.166 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-07-11T03:47:12.237 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-07-11T03:47:12.305 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-07-11T03:47:12.345 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-07-11T03:47:12.417 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-07-11T03:47:12.527 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "name": "dachary", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-07-11T03:47:12.675 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "name": "dachary", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-07-11T03:47:12.897 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "name": "dachary", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-07-11T03:47:12.949 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "name": "dachary", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-07-11T03:47:13.070 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/amctagga", "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "name": "amctagga", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-07-11T03:47:13.120 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/amctagga", "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "name": "amctagga", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-07-11T03:47:13.332 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/amctagga", "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "name": "amctagga", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-07-11T03:47:13.381 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/amctagga", "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "name": "amctagga", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-07-11T03:47:13.504 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/kevinz", "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "name": "kevinz", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-07-11T03:47:13.564 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/kevinz", "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "name": "kevinz", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-07-11T03:47:13.811 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/kevinz", "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "name": "kevinz", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-07-11T03:47:13.890 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/kevinz", "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "name": "kevinz", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-07-11T03:47:13.991 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/liu-chunmei", "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "name": "liu-chunmei", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-07-11T03:47:14.047 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/liu-chunmei", "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "name": "liu-chunmei", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-07-11T03:47:14.296 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/liu-chunmei", "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "name": "liu-chunmei", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-07-11T03:47:14.414 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10106, "groups": "sudo", "home": "/home/liu-chunmei", "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "name": "liu-chunmei", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-07-11T03:47:14.522 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/cyx1231st", "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "name": "cyx1231st", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2021-07-11T03:47:14.546 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/cyx1231st", "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "name": "cyx1231st", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2021-07-11T03:47:14.738 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/cyx1231st", "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "name": "cyx1231st", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2021-07-11T03:47:14.909 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10107, "groups": "sudo", "home": "/home/cyx1231st", "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "name": "cyx1231st", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10106} 2021-07-11T03:47:14.996 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/sunnagar", "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "name": "sunnagar", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2021-07-11T03:47:15.068 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/sunnagar", "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "name": "sunnagar", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2021-07-11T03:47:15.199 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/sunnagar", "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "name": "sunnagar", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2021-07-11T03:47:15.388 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10108, "groups": "sudo", "home": "/home/sunnagar", "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "name": "sunnagar", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10107} 2021-07-11T03:47:15.638 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10109, "groups": "sudo", "home": "/home/lflores", "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "name": "lflores", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10108} 2021-07-11T03:47:15.698 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10109, "groups": "sudo", "home": "/home/lflores", "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "name": "lflores", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10108} 2021-07-11T03:47:15.722 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10109, "groups": "sudo", "home": "/home/lflores", "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "name": "lflores", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10108} 2021-07-11T03:47:15.816 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10109, "groups": "sudo", "home": "/home/lflores", "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "name": "lflores", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10108} 2021-07-11T03:47:16.048 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10110, "groups": "sudo", "home": "/home/jsawaya", "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "name": "jsawaya", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10109} 2021-07-11T03:47:16.202 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10110, "groups": "sudo", "home": "/home/jsawaya", "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "name": "jsawaya", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10109} 2021-07-11T03:47:16.206 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10110, "groups": "sudo", "home": "/home/jsawaya", "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "name": "jsawaya", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10109} 2021-07-11T03:47:16.381 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10110, "groups": "sudo", "home": "/home/jsawaya", "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "name": "jsawaya", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10109} 2021-07-11T03:47:16.570 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10111, "groups": "sudo", "home": "/home/markw8973240", "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "name": "markw8973240", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10110} 2021-07-11T03:47:16.671 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10111, "groups": "sudo", "home": "/home/markw8973240", "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "name": "markw8973240", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10110} 2021-07-11T03:47:16.728 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10111, "groups": "sudo", "home": "/home/markw8973240", "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "name": "markw8973240", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10110} 2021-07-11T03:47:16.866 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10111, "groups": "sudo", "home": "/home/markw8973240", "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "name": "markw8973240", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10110} 2021-07-11T03:47:17.054 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10112, "groups": "sudo", "home": "/home/luisrico", "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "name": "luisrico", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10111} 2021-07-11T03:47:17.139 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10112, "groups": "sudo", "home": "/home/luisrico", "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "name": "luisrico", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10111} 2021-07-11T03:47:17.246 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10112, "groups": "sudo", "home": "/home/luisrico", "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "name": "luisrico", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10111} 2021-07-11T03:47:17.301 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10112, "groups": "sudo", "home": "/home/luisrico", "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "name": "luisrico", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10111} 2021-07-11T03:47:17.482 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10113, "groups": "sudo", "home": "/home/matan", "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "name": "matan", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10112} 2021-07-11T03:47:17.590 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10113, "groups": "sudo", "home": "/home/matan", "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "name": "matan", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10112} 2021-07-11T03:47:17.762 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10113, "groups": "sudo", "home": "/home/matan", "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "name": "matan", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10112} 2021-07-11T03:47:17.824 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10113, "groups": "sudo", "home": "/home/matan", "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "name": "matan", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10112} 2021-07-11T03:47:18.048 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10114, "groups": "sudo", "home": "/home/ozeneva", "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "name": "ozeneva", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10113} 2021-07-11T03:47:18.104 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10114, "groups": "sudo", "home": "/home/ozeneva", "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "name": "ozeneva", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10113} 2021-07-11T03:47:18.246 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10114, "groups": "sudo", "home": "/home/ozeneva", "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "name": "ozeneva", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10113} 2021-07-11T03:47:18.334 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10114, "groups": "sudo", "home": "/home/ozeneva", "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "name": "ozeneva", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10113} 2021-07-11T03:47:18.538 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10115, "groups": "sudo", "home": "/home/sudhanaruna", "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "name": "sudhanaruna", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10114} 2021-07-11T03:47:18.563 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10115, "groups": "sudo", "home": "/home/sudhanaruna", "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "name": "sudhanaruna", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10114} 2021-07-11T03:47:18.718 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10115, "groups": "sudo", "home": "/home/sudhanaruna", "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "name": "sudhanaruna", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10114} 2021-07-11T03:47:18.759 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10115, "groups": "sudo", "home": "/home/sudhanaruna", "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "name": "sudhanaruna", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10114} 2021-07-11T03:47:18.997 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10116, "groups": "sudo", "home": "/home/zl31wang", "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "name": "zl31wang", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10115} 2021-07-11T03:47:19.155 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10116, "groups": "sudo", "home": "/home/zl31wang", "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "name": "zl31wang", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10115} 2021-07-11T03:47:19.220 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10116, "groups": "sudo", "home": "/home/zl31wang", "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "name": "zl31wang", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10115} 2021-07-11T03:47:19.285 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10116, "groups": "sudo", "home": "/home/zl31wang", "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "name": "zl31wang", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10115} 2021-07-11T03:47:19.410 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10117, "groups": "sudo", "home": "/home/mayagilad", "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "name": "mayagilad", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10116} 2021-07-11T03:47:19.574 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:19 +0000 (0:00:25.084) 0:00:34.049 *********** 2021-07-11T03:47:19.656 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10117, "groups": "sudo", "home": "/home/mayagilad", "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "name": "mayagilad", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10116} 2021-07-11T03:47:19.663 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10117, "groups": "sudo", "home": "/home/mayagilad", "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "name": "mayagilad", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10116} 2021-07-11T03:47:19.685 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-07-11T03:47:19.734 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:19 +0000 (0:00:00.159) 0:00:34.208 *********** 2021-07-11T03:47:19.827 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-07-11T03:47:20.024 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10117, "groups": "sudo", "home": "/home/mayagilad", "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "name": "mayagilad", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10116} 2021-07-11T03:47:20.027 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-07-11T03:47:20.029 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora r 2021-07-11T03:47:20.031 INFO:teuthology.task.ansible.out:T6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c 2021-07-11T03:47:20.033 INFO:teuthology.task.ansible.out:63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD 2021-07-11T03:47:20.035 INFO:teuthology.task.ansible.out:0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9d 2021-07-11T03:47:20.037 INFO:teuthology.task.ansible.out:f7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b 2021-07-11T03:47:20.055 INFO:teuthology.task.ansible.out:660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "o 2021-07-11T03:47:20.057 INFO:teuthology.task.ansible.out:vpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f 2021-07-11T03:47:20.059 INFO:teuthology.task.ansible.out:/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": " 2021-07-11T03:47:20.061 INFO:teuthology.task.ansible.out:alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon" 2021-07-11T03:47:20.063 INFO:teuthology.task.ansible.out:, "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name 2021-07-11T03:47:20.065 INFO:teuthology.task.ansible.out:": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046 2021-07-11T03:47:20.066 INFO:teuthology.task.ansible.out:655420d4d0a79ce5667bec1b469a0100"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449c 2021-07-11T03:47:20.068 INFO:teuthology.task.ansible.out:e80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn" 2021-07-11T03:47:20.070 INFO:teuthology.task.ansible.out:: "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8yt 2021-07-11T03:47:20.071 INFO:teuthology.task.ansible.out:ILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e1850 2021-07-11T03:47:20.073 INFO:teuthology.task.ansible.out:52887b82ba5c343a42eb271747a"}, {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}]}, "changed": false} 2021-07-11T03:47:20.131 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-07-11T03:47:20.226 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:20 +0000 (0:00:00.492) 0:00:34.701 *********** 2021-07-11T03:47:20.262 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:20 +0000 (0:00:00.035) 0:00:34.737 *********** 2021-07-11T03:47:20.292 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:20 +0000 (0:00:00.027) 0:00:34.765 *********** 2021-07-11T03:47:20.336 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:20 +0000 (0:00:00.045) 0:00:34.810 *********** 2021-07-11T03:47:20.342 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-07-11T03:47:20.432 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:20 +0000 (0:00:00.095) 0:00:34.906 *********** 2021-07-11T03:47:20.482 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:20 +0000 (0:00:00.050) 0:00:34.957 *********** 2021-07-11T03:47:20.527 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:20 +0000 (0:00:00.045) 0:00:35.002 *********** 2021-07-11T03:47:20.816 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-07-11T03:47:20.825 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora r 2021-07-11T03:47:20.827 INFO:teuthology.task.ansible.out:T6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c 2021-07-11T03:47:20.828 INFO:teuthology.task.ansible.out:63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD 2021-07-11T03:47:20.830 INFO:teuthology.task.ansible.out:0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9d 2021-07-11T03:47:20.831 INFO:teuthology.task.ansible.out:f7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b 2021-07-11T03:47:20.833 INFO:teuthology.task.ansible.out:660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "o 2021-07-11T03:47:20.835 INFO:teuthology.task.ansible.out:vpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": 2021-07-11T03:47:20.841 INFO:teuthology.task.ansible.out: "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw 2021-07-11T03:47:20.843 INFO:teuthology.task.ansible.out:b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"n 2021-07-11T03:47:20.845 INFO:teuthology.task.ansible.out:ame": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, {"name": "kyr", "ovpn": "kyr@ 2021-07-11T03:47:20.852 INFO:teuthology.task.ansible.out:suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046 2021-07-11T03:47:20.859 INFO:teuthology.task.ansible.out:655420d4d0a79ce5667bec1b469a0100"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCX 2021-07-11T03:47:20.866 INFO:teuthology.task.ansible.out:LUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf946 2021-07-11T03:47:20.868 INFO:teuthology.task.ansible.out:0980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+G 2021-07-11T03:47:20.869 INFO:teuthology.task.ansible.out:HCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}]}, "changed": false} ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce528 2021-07-11T03:47:20.875 INFO:teuthology.task.ansible.out:77eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"na 2021-07-11T03:47:20.877 INFO:teuthology.task.ansible.out:me": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c36 2021-07-11T03:47:20.879 INFO:teuthology.task.ansible.out:68455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f3 2021-07-11T03:47:20.882 INFO:teuthology.task.ansible.out:5cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, {"name": "dang", "ovpn": "dang@ 2021-07-11T03:47:20.882 INFO:teuthology.task.ansible.out:sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81 2021-07-11T03:47:20.884 INFO:teuthology.task.ansible.out:fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba05 2021-07-11T03:47:20.886 INFO:teuthology.task.ansible.out:7e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name 2021-07-11T03:47:20.887 INFO:teuthology.task.ansible.out:": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name" 2021-07-11T03:47:20.889 INFO:teuthology.task.ansible.out:: "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e4 2021-07-11T03:47:20.895 INFO:teuthology.task.ansible.out:9e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d542 2021-07-11T03:47:20.897 INFO:teuthology.task.ansible.out:0c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d 2021-07-11T03:47:20.899 INFO:teuthology.task.ansible.out:2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae80 2021-07-11T03:47:20.900 INFO:teuthology.task.ansible.out:3d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0 2021-07-11T03:47:20.902 INFO:teuthology.task.ansible.out:e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytI 2021-07-11T03:47:20.902 INFO:teuthology.task.ansible.out:LUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7 2021-07-11T03:47:20.904 INFO:teuthology.task.ansible.out:e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27 2021-07-11T03:47:20.905 INFO:teuthology.task.ansible.out:de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}]}, "changed": false} 2021-07-11T03:47:20.909 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora r 2021-07-11T03:47:20.910 INFO:teuthology.task.ansible.out:T6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c 2021-07-11T03:47:20.912 INFO:teuthology.task.ansible.out:63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD 2021-07-11T03:47:20.914 INFO:teuthology.task.ansible.out:0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9d 2021-07-11T03:47:20.915 INFO:teuthology.task.ansible.out:f7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b 2021-07-11T03:47:20.917 INFO:teuthology.task.ansible.out:660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "o 2021-07-11T03:47:20.919 INFO:teuthology.task.ansible.out:vpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f 2021-07-11T03:47:20.920 INFO:teuthology.task.ansible.out:/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": " 2021-07-11T03:47:20.922 INFO:teuthology.task.ansible.out:alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon" 2021-07-11T03:47:20.925 INFO:teuthology.task.ansible.out:, "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/ 2021-07-11T03:47:20.926 INFO:teuthology.task.ansible.out:x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name 2021-07-11T03:47:20.927 INFO:teuthology.task.ansible.out:": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449c 2021-07-11T03:47:20.929 INFO:teuthology.task.ansible.out:e80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn" 2021-07-11T03:47:20.930 INFO:teuthology.task.ansible.out:: "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e1850 2021-07-11T03:47:20.932 INFO:teuthology.task.ansible.out:52887b82ba5c343a42eb271747a"}, {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, {"name": "luisrico", "ovpn": "lricomor@lricomor-m 2021-07-11T03:47:20.935 INFO:teuthology.task.ansible.out:ac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}]}, "changed": false} 2021-07-11T03:47:20.966 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:20 +0000 (0:00:00.438) 0:00:35.441 *********** 2021-07-11T03:47:20.989 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:21.049 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:21 +0000 (0:00:00.082) 0:00:35.524 *********** 2021-07-11T03:47:21.050 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:21.077 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:21 +0000 (0:00:00.027) 0:00:35.551 *********** 2021-07-11T03:47:22.091 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-07-11T03:47:22.093 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"after": "444af9608e8be25a2d7a2892fb588d0daeff9780", "attempts": 1, "before": "444af9608e8be25a2d7a2892fb588d0daeff9780", "changed": false, "remote_url_changed": false} 2021-07-11T03:47:22.125 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:22 +0000 (0:00:01.048) 0:00:36.599 *********** 2021-07-11T03:47:22.340 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"after": "444af9608e8be25a2d7a2892fb588d0daeff9780", "attempts": 1, "before": "444af9608e8be25a2d7a2892fb588d0daeff9780", "changed": false, "remote_url_changed": false} 2021-07-11T03:47:22.395 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:22 +0000 (0:00:00.268) 0:00:36.868 *********** 2021-07-11T03:47:22.505 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"after": "444af9608e8be25a2d7a2892fb588d0daeff9780", "attempts": 1, "before": "444af9608e8be25a2d7a2892fb588d0daeff9780", "changed": false, "remote_url_changed": false} 2021-07-11T03:47:22.780 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:22 +0000 (0:00:00.386) 0:00:37.255 *********** 2021-07-11T03:47:22.840 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-07-11T03:47:22.842 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} TASK [users : Clone the keys repo] ********************************************* ok: [gibba004.front.sepia.ceph.com] => {"after": "444af9608e8be25a2d7a2892fb588d0daeff9780", "attempts": 1, "before": "444af9608e8be25a 2021-07-11T03:47:22.843 INFO:teuthology.task.ansible.out:2d7a2892fb588d0daeff9780", "changed": false, "remote_url_changed": false} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-07-11T03:47:22.878 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:22 +0000 (0:00:00.097) 0:00:37.353 *********** 2021-07-11T03:47:22.939 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-07-11T03:47:22.989 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-07-11T03:47:23.055 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-07-11T03:47:23.088 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-07-11T03:47:23.184 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-07-11T03:47:23.186 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-07-11T03:47:23.206 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-07-11T03:47:23.230 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-07-11T03:47:23.232 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-07-11T03:47:23.339 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-07-11T03:47:23.340 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-07-11T03:47:23.342 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-07-11T03:47:23.343 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-07-11T03:47:23.389 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-07-11T03:47:23.469 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-07-11T03:47:23.471 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-07-11T03:47:23.506 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-07-11T03:47:23.531 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-07-11T03:47:23.554 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-07-11T03:47:23.555 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-07-11T03:47:23.648 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZt 2021-07-11T03:47:23.650 INFO:teuthology.task.ansible.out:IEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-07-11T03:47:23.687 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-07-11T03:47:23.688 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:23.690 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-07-11T03:47:23.696 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-07-11T03:47:23.811 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-07-11T03:47:23.830 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-07-11T03:47:23.844 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-07-11T03:47:23.881 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-07-11T03:47:23.883 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-07-11T03:47:23.972 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-07-11T03:47:23.997 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-07-11T03:47:24.019 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:24.021 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o 2021-07-11T03:47:24.023 INFO:teuthology.task.ansible.out:8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-07-11T03:47:24.130 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-07-11T03:47:24.133 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-07-11T03:47:24.154 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-07-11T03:47:24.168 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-07-11T03:47:24.169 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:24.184 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-07-11T03:47:24.255 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-07-11T03:47:24.257 INFO:teuthology.task.ansible.out:on", "validate_certs": true} 2021-07-11T03:47:24.266 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:24.280 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:24.315 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-07-11T03:47:24.324 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-07-11T03:47:24.342 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-07-11T03:47:24.418 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-07-11T03:47:24.435 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:24.460 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-07-11T03:47:24.470 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-07-11T03:47:24.472 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:24.510 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-07-11T03:47:24.512 INFO:teuthology.task.ansible.out:on", "validate_certs": true} 2021-07-11T03:47:24.527 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:24.568 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:24.590 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-07-11T03:47:24.592 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf59 2021-07-11T03:47:24.594 INFO:teuthology.task.ansible.out:2607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-07-11T03:47:24.692 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMorb 2021-07-11T03:47:24.694 INFO:teuthology.task.ansible.out:nuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-07-11T03:47:24.719 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-07-11T03:47:24.778 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-07-11T03:47:24.780 INFO:teuthology.task.ansible.out:on", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-07-11T03:47:24.796 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTU 2021-07-11T03:47:24.798 INFO:teuthology.task.ansible.out:yWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:24.896 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemers 2021-07-11T03:47:24.898 INFO:teuthology.task.ansible.out:on", "validate_certs": true} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:24.905 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:24.906 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-07-11T03:47:24.907 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-07-11T03:47:24.934 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": n 2021-07-11T03:47:24.936 INFO:teuthology.task.ansible.out:ull, "state": "present", "user": "davidz", "validate_certs": true} 2021-07-11T03:47:24.994 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-07-11T03:47:25.034 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-07-11T03:47:25.078 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-07-11T03:47:25.087 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-07-11T03:47:25.152 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-07-11T03:47:25.155 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-07-11T03:47:25.209 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtSo 2021-07-11T03:47:25.211 INFO:teuthology.task.ansible.out:AWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13 2021-07-11T03:47:25.214 INFO:teuthology.task.ansible.out:cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2021-07-11T03:47:25.260 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-07-11T03:47:25.314 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-07-11T03:47:25.316 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-07-11T03:47:25.326 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-07-11T03:47:25.412 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-07-11T03:47:25.467 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-07-11T03:47:25.481 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": n 2021-07-11T03:47:25.486 INFO:teuthology.task.ansible.out:ull, "state": "present", "user": "davidz", "validate_certs": true} 2021-07-11T03:47:25.494 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": n 2021-07-11T03:47:25.496 INFO:teuthology.task.ansible.out:ull, "state": "present", "user": "davidz", "validate_certs": true} 2021-07-11T03:47:25.598 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-07-11T03:47:25.600 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-07-11T03:47:25.645 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-07-11T03:47:25.658 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-07-11T03:47:25.672 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-07-11T03:47:25.746 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-07-11T03:47:25.748 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-07-11T03:47:25.788 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-07-11T03:47:25.822 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-07-11T03:47:25.826 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-07-11T03:47:25.832 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-07-11T03:47:25.833 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:25.899 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-07-11T03:47:25.974 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-07-11T03:47:25.981 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "pr 2021-07-11T03:47:25.987 INFO:teuthology.task.ansible.out:esent", "user": "wusui", "validate_certs": true} 2021-07-11T03:47:26.005 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-07-11T03:47:26.007 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:26.044 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-07-11T03:47:26.155 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-07-11T03:47:26.157 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-07-11T03:47:26.168 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-07-11T03:47:26.169 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:26.208 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7g 2021-07-11T03:47:26.209 INFO:teuthology.task.ansible.out:AlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrSX8nDL2WaHe0b/rjE3TZhj5VwbrG6nn/hyWmm2dWcUqotSp973yD2G4tQL9UygeSkDGmjb6whrz2mcAFJ5P4oLtIcCLyZC550dcpwV3BtP/cMAIGE6xeEARKiP8KlVGFNDRg9/l1aAd08Bepe+rJyAYD7IxOOBfgKJMKbeOWnT40RTfwiD5CXNwjzQuhzZDEbm+8zjTChFM4P1EkfR3c+0X3MYpA1AsRQY0ElvFtcn6G0yPKR+obGdIs+ydGupKh1+peGWLemHMA5e62Bs4/Lwtyebyp7AOguC3ZBJ3TZWL4Vn1q6z21cQe6AuowjPrneZWC4ifFB6OUQJqM+R00pHBcajcvsfpYOblZvTJHM726Ek1vahOYfcbEMpqWlVYo+WPRUilzl5cI3ReyAcwphebqKPbhmzCuyJw3lWBKIVDhWjEFzGkxV/iFU5rIXFm/Z0JGNFFxmY1U4GlA4lFXtL7OvEdMsiZo1x0cNmFWLu0k9fQBtiQI19ZBiA7FPKc= yehuda@crestview", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-07-11T03:47:26.235 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9C 2021-07-11T03:47:26.237 INFO:teuthology.task.ansible.out:kxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-07-11T03:47:26.309 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-07-11T03:47:26.311 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-07-11T03:47:26.358 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-07-11T03:47:26.372 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBYB 2021-07-11T03:47:26.373 INFO:teuthology.task.ansible.out:0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-07-11T03:47:26.385 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-07-11T03:47:26.477 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-07-11T03:47:26.478 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7g 2021-07-11T03:47:26.480 INFO:teuthology.task.ansible.out:AlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrSX8nDL2WaHe0b/rjE3TZhj5VwbrG6nn/hyWmm2dWcUqotSp973yD2G4tQL9UygeSkDGmjb6whrz2mcAFJ5P4oLtIcCLyZC550dcpwV3BtP/cMAIGE6xeEARKiP8KlVGFNDRg9/l1aAd08Bepe+rJyAYD7IxOOBfgKJMKbeOWnT40RTfwiD5CXNwjzQuhzZDEbm+8zjTChFM4P1EkfR3c+0X3MYpA1AsRQY0ElvFtcn6G0yPKR+obGdIs+ydGupKh1+peGWLemHMA5e62Bs4/Lwtyebyp7AOguC3ZBJ3TZWL4Vn1q6z21cQe6AuowjPrneZWC4ifFB6OUQJqM+R00pHBcajcvsfpYOblZvTJHM726Ek1vahOYfcbEMpqWlVYo+WPRUilzl5cI3ReyAcwphebqKPbhmzCuyJw3lWBKIVDhWjEFzGkxV/iFU5rIXFm/Z0JGNFFxmY1U4GlA4lFXtL7OvEdMsiZo1x0cNmFWLu0k9fQBtiQI19ZBiA7FPKc= yehuda@crestview", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-07-11T03:47:26.496 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-07-11T03:47:26.509 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-07-11T03:47:26.639 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-07-11T03:47:26.689 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-07-11T03:47:26.715 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-07-11T03:47:26.817 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-07-11T03:47:26.839 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-07-11T03:47:26.853 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7g 2021-07-11T03:47:26.855 INFO:teuthology.task.ansible.out:AlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrSX8nDL2WaHe0b/rjE3TZhj5VwbrG6nn/hyWmm2dWcUqotSp973yD2G4tQL9UygeSkDGmjb6whrz2mcAFJ5P4oLtIcCLyZC550dcpwV3BtP/cMAIGE6xeEARKiP8KlVGFNDRg9/l1aAd08Bepe+rJyAYD7IxOOBfgKJMKbeOWnT40RTfwiD5CXNwjzQuhzZDEbm+8zjTChFM4P1EkfR3c+0X3MYpA1AsRQY0ElvFtcn6G0yPKR+obGdIs+ydGupKh1+peGWLemHMA5e62Bs4/Lwtyebyp7AOguC3ZBJ3TZWL4Vn1q6z21cQe6AuowjPrneZWC4ifFB6OUQJqM+R00pHBcajcvsfpYOblZvTJHM726Ek1vahOYfcbEMpqWlVYo+WPRUilzl5cI3ReyAcwphebqKPbhmzCuyJw3lWBKIVDhWjEFzGkxV/iFU5rIXFm/Z0JGNFFxmY1U4GlA4lFXtL7OvEdMsiZo1x0cNmFWLu0k9fQBtiQI19ZBiA7FPKc= yehuda@crestview", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-07-11T03:47:26.882 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7g 2021-07-11T03:47:26.884 INFO:teuthology.task.ansible.out:AlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrSX8nDL2WaHe0b/rjE3TZhj5VwbrG6nn/hyWmm2dWcUqotSp973yD2G4tQL9UygeSkDGmjb6whrz2mcAFJ5P4oLtIcCLyZC550dcpwV3BtP/cMAIGE6xeEARKiP8KlVGFNDRg9/l1aAd08Bepe+rJyAYD7IxOOBfgKJMKbeOWnT40RTfwiD5CXNwjzQuhzZDEbm+8zjTChFM4P1EkfR3c+0X3MYpA1AsRQY0ElvFtcn6G0yPKR+obGdIs+ydGupKh1+peGWLemHMA5e62Bs4/Lwtyebyp7AOguC3ZBJ3TZWL4Vn1q6z21cQe6AuowjPrneZWC4ifFB6OUQJqM+R00pHBcajcvsfpYOblZvTJHM726Ek1vahOYfcbEMpqWlVYo+WPRUilzl5cI3ReyAcwphebqKPbhmzCuyJw3lWBKIVDhWjEFzGkxV/iFU5rIXFm/Z0JGNFFxmY1U4GlA4lFXtL7OvEdMsiZo1x0cNmFWLu0k9fQBtiQI19ZBiA7FPKc= yehuda@crestview", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-07-11T03:47:26.946 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-07-11T03:47:26.972 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-07-11T03:47:27.009 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-07-11T03:47:27.078 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-07-11T03:47:27.080 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-07-11T03:47:27.116 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-07-11T03:47:27.117 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-07-11T03:47:27.118 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:27.167 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-07-11T03:47:27.170 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:27.197 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-07-11T03:47:27.220 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-07-11T03:47:27.262 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-07-11T03:47:27.288 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-07-11T03:47:27.324 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-07-11T03:47:27.374 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-07-11T03:47:27.384 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-07-11T03:47:27.386 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-07-11T03:47:27.424 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-07-11T03:47:27.457 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-07-11T03:47:27.506 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-07-11T03:47:27.528 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-07-11T03:47:27.549 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-07-11T03:47:27.550 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:27.582 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-07-11T03:47:27.615 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-07-11T03:47:27.630 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-07-11T03:47:27.632 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-07-11T03:47:27.641 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-07-11T03:47:27.688 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQvU 2021-07-11T03:47:27.690 INFO:teuthology.task.ansible.out:xUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-07-11T03:47:27.728 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-07-11T03:47:27.773 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-07-11T03:47:27.785 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-07-11T03:47:27.831 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqd 2021-07-11T03:47:27.833 INFO:teuthology.task.ansible.out:V+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-07-11T03:47:27.888 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-07-11T03:47:27.900 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-07-11T03:47:27.948 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-07-11T03:47:27.974 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-07-11T03:47:28.022 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-07-11T03:47:28.036 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-07-11T03:47:28.096 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-07-11T03:47:28.120 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-07-11T03:47:28.121 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-07-11T03:47:28.123 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-07-11T03:47:28.188 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-07-11T03:47:28.257 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-07-11T03:47:28.274 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-07-11T03:47:28.276 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-07-11T03:47:28.286 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-07-11T03:47:28.328 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-07-11T03:47:28.383 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-07-11T03:47:28.406 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-07-11T03:47:28.450 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-07-11T03:47:28.516 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-07-11T03:47:28.527 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-07-11T03:47:28.561 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-07-11T03:47:28.562 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-07-11T03:47:28.564 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-07-11T03:47:28.605 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-07-11T03:47:28.627 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-07-11T03:47:28.650 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-07-11T03:47:28.661 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-07-11T03:47:28.663 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-07-11T03:47:28.732 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-07-11T03:47:28.743 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVT 2021-07-11T03:47:28.744 INFO:teuthology.task.ansible.out:FNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPg 2021-07-11T03:47:28.750 INFO:teuthology.task.ansible.out:sLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-07-11T03:47:28.779 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-07-11T03:47:28.835 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1O 2021-07-11T03:47:28.837 INFO:teuthology.task.ansible.out:MJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-07-11T03:47:28.863 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-07-11T03:47:28.914 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bf 2021-07-11T03:47:28.917 INFO:teuthology.task.ansible.out:ed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWf 2021-07-11T03:47:28.919 INFO:teuthology.task.ansible.out:tbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-07-11T03:47:28.945 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyKz 2021-07-11T03:47:28.947 INFO:teuthology.task.ansible.out:2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.r 2021-07-11T03:47:28.952 INFO:teuthology.task.ansible.out:emote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-07-11T03:47:29.050 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-07-11T03:47:29.063 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-07-11T03:47:29.141 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-07-11T03:47:29.169 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-07-11T03:47:29.170 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVT 2021-07-11T03:47:29.172 INFO:teuthology.task.ansible.out:FNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-07-11T03:47:29.215 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-07-11T03:47:29.294 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-07-11T03:47:29.305 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-07-11T03:47:29.307 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-07-11T03:47:29.309 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-07-11T03:47:29.342 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-07-11T03:47:29.416 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-07-11T03:47:29.418 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-07-11T03:47:29.428 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-07-11T03:47:29.470 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVT 2021-07-11T03:47:29.474 INFO:teuthology.task.ansible.out:FNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-07-11T03:47:29.536 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVT 2021-07-11T03:47:29.538 INFO:teuthology.task.ansible.out:FNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-07-11T03:47:29.550 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-07-11T03:47:29.617 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEf 2021-07-11T03:47:29.619 INFO:teuthology.task.ansible.out:dVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dT 2021-07-11T03:47:29.620 INFO:teuthology.task.ansible.out:zNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-07-11T03:47:29.660 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-07-11T03:47:29.662 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh9 2021-07-11T03:47:29.664 INFO:teuthology.task.ansible.out:0KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYf 2021-07-11T03:47:29.665 INFO:teuthology.task.ansible.out:Y4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-07-11T03:47:29.720 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-07-11T03:47:29.778 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x 2021-07-11T03:47:29.786 INFO:teuthology.task.ansible.out:0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-07-11T03:47:29.822 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-07-11T03:47:29.892 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-07-11T03:47:29.893 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-07-11T03:47:29.912 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-07-11T03:47:29.946 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-07-11T03:47:30.013 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-07-11T03:47:30.028 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-07-11T03:47:30.030 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-07-11T03:47:30.050 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-07-11T03:47:30.052 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-07-11T03:47:30.079 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-07-11T03:47:30.117 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-07-11T03:47:30.150 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-07-11T03:47:30.167 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-07-11T03:47:30.205 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-07-11T03:47:30.230 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-07-11T03:47:30.268 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-07-11T03:47:30.280 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-07-11T03:47:30.315 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-07-11T03:47:30.316 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-07-11T03:47:30.328 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/ 2021-07-11T03:47:30.330 INFO:teuthology.task.ansible.out:nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-07-11T03:47:30.375 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-07-11T03:47:30.377 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-07-11T03:47:30.399 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-07-11T03:47:30.401 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-07-11T03:47:30.425 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-07-11T03:47:30.436 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-07-11T03:47:30.494 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-07-11T03:47:30.506 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-07-11T03:47:30.518 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-07-11T03:47:30.547 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-07-11T03:47:30.601 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-07-11T03:47:30.612 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-07-11T03:47:30.615 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-07-11T03:47:30.636 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-07-11T03:47:30.637 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:30.675 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "g 2021-07-11T03:47:30.679 INFO:teuthology.task.ansible.out:abrioux", "validate_certs": true} 2021-07-11T03:47:30.710 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-07-11T03:47:30.732 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-07-11T03:47:30.744 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-07-11T03:47:30.746 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-07-11T03:47:30.777 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-07-11T03:47:30.822 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-07-11T03:47:30.844 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-07-11T03:47:30.856 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-07-11T03:47:30.858 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:30.891 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-07-11T03:47:30.935 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-07-11T03:47:30.983 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sj 2021-07-11T03:47:30.985 INFO:teuthology.task.ansible.out:z6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin 2021-07-11T03:47:30.987 INFO:teuthology.task.ansible.out:/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-07-11T03:47:31.009 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkk 2021-07-11T03:47:31.011 INFO:teuthology.task.ansible.out:e6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-07-11T03:47:31.062 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-07-11T03:47:31.101 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-07-11T03:47:31.125 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-07-11T03:47:31.157 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-07-11T03:47:31.169 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-07-11T03:47:31.226 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-07-11T03:47:31.228 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:31.242 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-07-11T03:47:31.253 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-07-11T03:47:31.282 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-07-11T03:47:31.348 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-07-11T03:47:31.360 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-07-11T03:47:31.382 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-07-11T03:47:31.416 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-07-11T03:47:31.461 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-07-11T03:47:31.462 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-07-11T03:47:31.499 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-07-11T03:47:31.560 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-07-11T03:47:31.589 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-07-11T03:47:31.590 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-07-11T03:47:31.658 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-07-11T03:47:31.696 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-07-11T03:47:31.698 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-07-11T03:47:31.735 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-07-11T03:47:31.781 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-07-11T03:47:31.819 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-07-11T03:47:31.821 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-07-11T03:47:31.856 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-07-11T03:47:31.893 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-07-11T03:47:31.943 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-07-11T03:47:31.967 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-07-11T03:47:31.992 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-07-11T03:47:32.037 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-07-11T03:47:32.062 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-07-11T03:47:32.065 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:32.098 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-07-11T03:47:32.124 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-07-11T03:47:32.154 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-07-11T03:47:32.197 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-07-11T03:47:32.233 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-07-11T03:47:32.260 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-07-11T03:47:32.290 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-07-11T03:47:32.349 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-07-11T03:47:32.362 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-07-11T03:47:32.416 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-07-11T03:47:32.479 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-07-11T03:47:32.480 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-07-11T03:47:32.496 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-07-11T03:47:32.530 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-07-11T03:47:32.611 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-07-11T03:47:32.626 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-07-11T03:47:32.629 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-07-11T03:47:32.631 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-07-11T03:47:32.747 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-07-11T03:47:32.750 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-07-11T03:47:32.762 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-07-11T03:47:32.783 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-07-11T03:47:32.883 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-07-11T03:47:32.901 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-07-11T03:47:32.913 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-07-11T03:47:32.916 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-07-11T03:47:33.005 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-07-11T03:47:33.021 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-07-11T03:47:33.023 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-07-11T03:47:33.038 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-07-11T03:47:33.119 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-07-11T03:47:33.131 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-07-11T03:47:33.133 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:33.144 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-07-11T03:47:33.169 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-07-11T03:47:33.235 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-07-11T03:47:33.237 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-07-11T03:47:33.248 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-07-11T03:47:33.249 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:33.292 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-07-11T03:47:33.350 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-07-11T03:47:33.359 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-07-11T03:47:33.361 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-07-11T03:47:33.383 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-07-11T03:47:33.385 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-07-11T03:47:33.471 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-07-11T03:47:33.483 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-07-11T03:47:33.495 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-07-11T03:47:33.496 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-07-11T03:47:33.594 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-07-11T03:47:33.616 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-07-11T03:47:33.618 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAAD 2021-07-11T03:47:33.620 INFO:teuthology.task.ansible.out:AQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-07-11T03:47:33.724 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1l 2021-07-11T03:47:33.726 INFO:teuthology.task.ansible.out:XTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-07-11T03:47:33.736 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-07-11T03:47:33.761 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-07-11T03:47:33.763 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-07-11T03:47:33.765 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-07-11T03:47:33.824 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-07-11T03:47:33.866 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-07-11T03:47:33.878 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-07-11T03:47:33.901 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-07-11T03:47:33.944 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-07-11T03:47:33.978 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-07-11T03:47:33.983 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-07-11T03:47:33.989 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-07-11T03:47:34.039 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-07-11T03:47:34.102 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-07-11T03:47:34.124 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-07-11T03:47:34.125 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-07-11T03:47:34.171 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-07-11T03:47:34.173 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:34.209 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-07-11T03:47:34.222 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-07-11T03:47:34.270 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-07-11T03:47:34.281 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-07-11T03:47:34.347 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-07-11T03:47:34.349 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-07-11T03:47:34.384 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.localdo 2021-07-11T03:47:34.390 INFO:teuthology.task.ansible.out:main", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-07-11T03:47:34.410 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-07-11T03:47:34.433 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-07-11T03:47:34.484 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-07-11T03:47:34.486 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-07-11T03:47:34.540 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq2 2021-07-11T03:47:34.541 INFO:teuthology.task.ansible.out:z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-07-11T03:47:34.558 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-07-11T03:47:34.575 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-07-11T03:47:34.633 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-07-11T03:47:34.635 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:34.680 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-07-11T03:47:34.682 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-07-11T03:47:34.722 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-07-11T03:47:34.735 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-07-11T03:47:34.769 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-07-11T03:47:34.771 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-07-11T03:47:34.835 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-07-11T03:47:34.856 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-07-11T03:47:34.872 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-07-11T03:47:34.895 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-07-11T03:47:34.940 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-07-11T03:47:34.942 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-07-11T03:47:35.001 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-07-11T03:47:35.012 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-07-11T03:47:35.071 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-07-11T03:47:35.085 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-07-11T03:47:35.122 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "k 2021-07-11T03:47:35.124 INFO:teuthology.task.ansible.out:ey": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.local", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-07-11T03:47:35.195 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-07-11T03:47:35.205 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-07-11T03:47:35.229 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-07-11T03:47:35.251 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-07-11T03:47:35.334 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-07-11T03:47:35.373 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-07-11T03:47:35.375 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-07-11T03:47:35.392 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-07-11T03:47:35.489 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-07-11T03:47:35.491 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-07-11T03:47:35.493 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-07-11T03:47:35.540 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-07-11T03:47:35.567 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-07-11T03:47:35.627 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-07-11T03:47:35.656 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-07-11T03:47:35.678 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/y 2021-07-11T03:47:35.679 INFO:teuthology.task.ansible.out:BwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-07-11T03:47:35.723 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loca 2021-07-11T03:47:35.724 INFO:teuthology.task.ansible.out:l", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-07-11T03:47:35.770 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-07-11T03:47:35.798 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-07-11T03:47:35.827 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-07-11T03:47:35.839 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-07-11T03:47:35.869 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-07-11T03:47:35.904 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-07-11T03:47:35.916 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-07-11T03:47:35.976 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/y 2021-07-11T03:47:35.977 INFO:teuthology.task.ansible.out:BwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-07-11T03:47:35.993 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-07-11T03:47:36.018 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-07-11T03:47:36.051 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-07-11T03:47:36.096 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-07-11T03:47:36.097 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-07-11T03:47:36.160 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-07-11T03:47:36.173 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-07-11T03:47:36.235 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBT 2021-07-11T03:47:36.237 INFO:teuthology.task.ansible.out:O4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-07-11T03:47:36.271 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-07-11T03:47:36.282 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-07-11T03:47:36.347 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-07-11T03:47:36.372 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-07-11T03:47:36.399 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/y 2021-07-11T03:47:36.401 INFO:teuthology.task.ansible.out:BwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj 2021-07-11T03:47:36.402 INFO:teuthology.task.ansible.out:", "validate_certs": true} 2021-07-11T03:47:36.458 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-07-11T03:47:36.471 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-07-11T03:47:36.519 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-07-11T03:47:36.543 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-07-11T03:47:36.580 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/y 2021-07-11T03:47:36.581 INFO:teuthology.task.ansible.out:BwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-07-11T03:47:36.602 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-07-11T03:47:36.630 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-07-11T03:47:36.647 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-07-11T03:47:36.681 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-07-11T03:47:36.710 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-07-11T03:47:36.734 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-07-11T03:47:36.766 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-07-11T03:47:36.805 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-07-11T03:47:36.816 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-07-11T03:47:36.840 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-07-11T03:47:36.889 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-07-11T03:47:36.922 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-07-11T03:47:36.939 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-07-11T03:47:36.952 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-07-11T03:47:36.997 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-07-11T03:47:37.037 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-07-11T03:47:37.072 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-07-11T03:47:37.074 INFO:teuthology.task.ansible.out: ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-07-11T03:47:37.121 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-07-11T03:47:37.123 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:37.178 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-07-11T03:47:37.180 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-07-11T03:47:37.193 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-07-11T03:47:37.238 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-07-11T03:47:37.272 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-07-11T03:47:37.291 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-07-11T03:47:37.303 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-07-11T03:47:37.347 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-07-11T03:47:37.380 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-07-11T03:47:37.392 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-07-11T03:47:37.432 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-07-11T03:47:37.433 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:37.486 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-07-11T03:47:37.491 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:37.503 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-07-11T03:47:37.515 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-07-11T03:47:37.561 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-07-11T03:47:37.638 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-07-11T03:47:37.673 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-07-11T03:47:37.686 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyr 2021-07-11T03:47:37.688 INFO:teuthology.task.ansible.out:atsas", "validate_certs": true} 2021-07-11T03:47:37.775 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-07-11T03:47:37.801 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-07-11T03:47:37.827 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-07-11T03:47:37.828 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-07-11T03:47:37.884 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-07-11T03:47:37.958 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", " 2021-07-11T03:47:37.959 INFO:teuthology.task.ansible.out:user": "prasad", "validate_certs": true} 2021-07-11T03:47:37.970 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-07-11T03:47:38.015 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-07-11T03:47:38.017 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-07-11T03:47:38.085 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-07-11T03:47:38.088 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-07-11T03:47:38.090 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-07-11T03:47:38.140 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-07-11T03:47:38.225 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-07-11T03:47:38.235 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-07-11T03:47:38.270 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-07-11T03:47:38.287 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-07-11T03:47:38.347 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-07-11T03:47:38.349 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-07-11T03:47:38.382 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-07-11T03:47:38.384 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-07-11T03:47:38.386 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:38.410 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-07-11T03:47:38.470 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-07-11T03:47:38.492 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-07-11T03:47:38.514 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-07-11T03:47:38.538 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-07-11T03:47:38.582 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-07-11T03:47:38.585 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-07-11T03:47:38.628 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-07-11T03:47:38.648 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-07-11T03:47:38.668 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:38.701 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-07-11T03:47:38.703 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-07-11T03:47:38.761 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-07-11T03:47:38.775 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-07-11T03:47:38.777 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-07-11T03:47:38.810 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-07-11T03:47:38.859 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-07-11T03:47:38.883 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-07-11T03:47:38.918 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-07-11T03:47:38.963 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-07-11T03:47:38.986 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-07-11T03:47:39.044 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjLd 2021-07-11T03:47:39.046 INFO:teuthology.task.ansible.out:+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-07-11T03:47:39.056 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-07-11T03:47:39.067 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-07-11T03:47:39.081 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-07-11T03:47:39.169 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+u 2021-07-11T03:47:39.171 INFO:teuthology.task.ansible.out:zp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-07-11T03:47:39.188 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-07-11T03:47:39.223 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-07-11T03:47:39.278 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-07-11T03:47:39.304 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-07-11T03:47:39.306 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-07-11T03:47:39.340 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-07-11T03:47:39.416 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-07-11T03:47:39.419 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-07-11T03:47:39.440 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-07-11T03:47:39.501 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-07-11T03:47:39.534 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-07-11T03:47:39.549 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-07-11T03:47:39.608 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-07-11T03:47:39.622 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-07-11T03:47:39.653 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-07-11T03:47:39.668 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-07-11T03:47:39.729 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-07-11T03:47:39.772 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-07-11T03:47:39.775 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-07-11T03:47:39.812 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-07-11T03:47:39.864 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-07-11T03:47:39.946 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-07-11T03:47:39.963 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJOefD/VJfS15LRyuVDLluKv93Zgocsl8BhiDMimvBNbrGWc795v2+eevWuwaL/SlL44SnsNZ9Mz8FVn9YuFoQiHLg7cHZNLg9EJh5SMXzZzwLZHnF02MdI2CU6U++9nmh8FtBrdBWKzZ7o+01wOw+52dYOxvP7IJ7C/vjwOLgfigEN8Nd1PrxALVNIQz7fsiet0n3odk9PJfoqTu3IloXm9qbQiPFyfMA6b8a20RdLZCJDGGG5rd6gtsyeFHoAPZEz8FEUlNkow7NM7ZhdxOz/fWXyMZ27KmY8WSAgflbxysMzexyifkGzypiE3NQ10hdPGBgoYsHrAN4Qmte1/Z4aI5nxrNzLy/uNc+L/Bp/0XkLj9zfoiFFOReXvlz/laAV6LpbHyacr+iCwv9rlI7echXmvC6mSJXfBPuyR1uromUXpYRhEkn5uiOsCIQCuYpx2jeil7JV+LJlF62lLSndOv4g12Gq2TEZ4Bhsr2902kftPkLFenyfDTh3A2qGGNM= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-07-11T03:47:39.965 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-07-11T03:47:39.988 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-07-11T03:47:40.071 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-07-11T03:47:40.086 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-07-11T03:47:40.107 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "k 2021-07-11T03:47:40.112 INFO:teuthology.task.ansible.out:ey_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-07-11T03:47:40.192 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-07-11T03:47:40.204 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-07-11T03:47:40.252 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-07-11T03:47:40.254 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-07-11T03:47:40.317 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-07-11T03:47:40.348 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-07-11T03:47:40.361 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-07-11T03:47:40.395 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-07-11T03:47:40.420 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJOefD/VJfS15LRyuVDLluKv93Zgocsl8BhiDMimvBNbrGWc795v2+eevWuwaL/SlL44SnsNZ9Mz8FVn9YuFoQiHLg7cHZNLg9EJh5SMXzZzwLZHnF02MdI2CU6U++9nmh8FtBrdBWKzZ7o+01wOw+52dYOxvP7IJ7C/vjwOLgfigEN8Nd1PrxALVNIQz7fsiet0n3odk9PJfoqTu3IloXm9qbQiPFyfMA6b8a20RdLZCJDGGG5rd6gtsyeFHoAPZEz8FEUlNkow7NM7ZhdxOz/fWXyMZ27KmY8WSAgflbxysMzexyifkGzypiE3NQ10hdPGBgoYsHrAN4Qmte1/Z4aI5nxrNzLy/uNc+L/Bp/0XkLj9zfoiFFOReXvlz/laAV6LpbHyacr+iCwv9rlI7echXmvC6mSJXfBPuyR1uromUXpYRhEkn5uiOsCIQCuYpx2jeil7JV+LJlF62lLSndOv4g12Gq2TEZ4Bhsr2902kftPkLFenyfDTh3A2qGGNM= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-07-11T03:47:40.487 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDwVh9KqjV0b+65aBxQV66IUwNo3D5zTbN8U6Kh6eE4QtaIVe5Mmvi2SBbGMSWofE0zIIw2JmsI5LzdOoCyZl/3gNDPHQd6D9o0xRHQv94giZ2a5K+mW79foAxsuppTjV/Fbg8EZ/19W91+BqMWotlMR66ka0RPT/6U8V5mzXoydSminViO/B6q7ZYzIIgGZqaTsSxCMvz5BAzzeEaGfSlGxw5MOye9sF/tDppuRKG4rMUZULEWGuxgXAtabtXCduocAArqAY3BRJEgj7JetPdPNjipGrULyYQmT7GeknvP/waC0sPm5o8t7taJca7Vu9qKKMKPAqiK4KhlrW/yVBFliT2CjAdVuYvGj87LCcEn6NMeF+4ONwDz++YwjrAWQT2zHUJPvLnYOZPiD1LUm3F34cEHD1y4Uf+oeiYgriUMauskfFn+ShbTmWWooFRFyNobzweg9aMBZSkA+kFFifqOV1FwHTOYg342NumRGB8svK0vvTUID7AU1Ev8YXYqCEGbb0UHth/1pEbAF103enLTtPZ6qKua74rE0qJ89OF9dZXv4JY9+8Juyps7bi2XEyHNm1nHZOY5tqdY0eKFmSGJMoWBDzGgdMjmXlWp4jnY2fC619nZqxMrCQpeT70CqnBhtBemUuu6jHYUbbrPjnXIkH3FZ7/qvZt0jF0IMP9FGw== amctagga@redhat.com", "key_options": null, "keyfile": "/home/amctagga/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amctagga", "validate_certs": true} 2021-07-11T03:47:40.488 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-07-11T03:47:40.510 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-07-11T03:47:40.511 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:40.536 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "k 2021-07-11T03:47:40.539 INFO:teuthology.task.ansible.out:ey_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-07-11T03:47:40.584 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-07-11T03:47:40.606 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtiHTDuQk8EDH5R+JfDMucI4ZpOzKN5SLD+j35QLLySgGXholoDxtqjHnmRjM1rpj5SIaXjHBZdJi7iax+1MRywHYBi5BfTtzYPgA+NH8mt+XkcdkEuFPcYxGh6fMQDQWb2g9Euze/6qi0W9mbYXvvb1/Q3bhVCXg1UgKD42BndWEpbMYFfzJLZWPtOD/lg81pwUmf/4BpPFv52sy87lQVfkkvcUliLObR4S0TI5hPLYxBS+RXmHYKgvPMB2QJ9wum62lJ+YM1MxPygS0OWbIB6UQGXmvxRXA4osXkxJ8ksPPCupOgFNiiYro91cQh3mmi6iHc9kMz5R1aNGGOfDdlnLx9DRSIiNcacjXEVNFaoM0x441f4GeU2ZR8/7uq2dfk7QjPlh1eJGxG9pzRiQ8JCDF1car0eYwXdMTsQoQqpIi5imqcELdcqGGp9FhzFElgUwpf15JYlkouVhS6hq4LDjhLNk7dDT8fmCEnHOPcbxaTgN7J3idnJK6zeIniuuk= root@iZj6ce071s2zz3reioxn93Z", "key_options": null, "keyfile": "/home/kevinz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kevinz", "validate_certs": true} 2021-07-11T03:47:40.631 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-07-11T03:47:40.670 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-07-11T03:47:40.693 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1YAaxAvlRgu72huAggHpoDKSgtF/b00ACfUX4Wk5NyLFbYI3PAQapHNcNL4mIsH8pc6YkdjcJ5N/F12aEupJGQ0Q06OS324xapnaBoUy1hQl9IvhtPqTLEwR0X5xLIAhNBLIEJDNVg13MKwr0u5YVUXdrQRu7Kj9RXAPBahNZFo/Ed+cnR3LICQWliHeHFrk/DU2DUHol3a5wAPxrowAG0AxAecP6/7rASMmitvLQwdQcfIJAhWYgfyXzudg+gOmAEgv3O3U4QOikgmkm/EabUXLJnlTj6yuhtuuctdJr73UT3wLGAGmphLKCkug27kb1nTNoDa+PW/+grP63zhU1 chunmei@otccldstore05", "key_options": null, "keyfile": "/home/liu-chunmei/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liu-chunmei", "validate_certs": true} 2021-07-11T03:47:40.707 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-07-11T03:47:40.757 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-07-11T03:47:40.803 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-07-11T03:47:40.805 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:40.846 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLbPdDb+XHOD2fo8g3RacATj0cG07uELwUvqQP35y+JmhNRSb/ttaysFLA5ss0y77TiXx22saP3ECEm5SnuPg5IERkPQmXiw84ATTNfLnzx2G5H223SLaTQiGfSgniBCJHVPCJfNahJiurlZoOAlwDz6SKXTUiivVrFJ22iTBDCQUO6HFQKjE7aM3RAo5mZXKVlTXzM6WOd8vlzk3HMzV0/91JR6H86Lt9MEzXT4dOzqmtxd8gS6U+vA+50zOIspdK+orTRJ0PO2Fn8c1RSYM8g7uqQtpDcOEHderW8TRsqBbDxpyslK6hRX5urUNBX506XE6JNoViLNvCofgseCGj yingxin@yingxin-server", "key_options": null, "keyfile": "/home/cyx1231st/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cyx1231st", "validate_certs": true} 2021-07-11T03:47:40.856 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-07-11T03:47:40.906 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJOefD/VJfS15LRyuVDLluKv93Zgocsl8BhiDMimvBNbrGWc795v2+eevWuwaL/SlL44SnsNZ9Mz8FVn9YuFoQiHLg7cHZNLg9EJh5SMXzZzwLZHnF02MdI2CU6U++9nmh8FtBrdBWKzZ7o+01wOw+52dYOxvP7IJ7C/vjwOLgfigEN8Nd1PrxALVNIQz7fsiet0n3odk9PJfoqTu3IloXm9qbQiPFyfMA6b8a20RdLZCJDGGG5rd6gtsyeFHoAPZEz8FEUlNkow7NM7ZhdxOz/fWXyMZ27KmY8WSAgflbxysMzexyifkGzypiE3NQ10hdPGBgoYsHrAN4Qmte1/Z4aI5nxrNzLy/uNc+L/Bp/0XkLj9zfoiFFOReXvlz/laAV6LpbHyacr+iCwv9rlI7echXmvC6mSJXfBPuyR1uromUXpYRhEkn5uiOsCIQCuYpx2jeil7JV+LJlF62lLSndOv4g12Gq2TEZ4Bhsr2902kftPkLFenyfDTh3A2qGGNM= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-07-11T03:47:40.943 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDwVh9KqjV0b+65aBxQV66IUwNo3D5zTbN8U6Kh6eE4QtaIVe5Mmvi2SBbGMSWofE0zIIw2JmsI5LzdOoCyZl/3gNDPHQd6D9o0xRHQv94giZ2a5K+mW79foAxsuppTjV/Fbg8EZ/19W91+BqMWotlMR66ka0RPT/6U8V5mzXoydSminViO/B6q7ZYzIIgGZqaTsSxCMvz5BAzzeEaGfSlGxw5MOye9sF/tDppuRKG4rMUZULEWGuxgXAtabtXCduocAArqAY3BRJEgj7JetPdPNjipGrULyYQmT7GeknvP/waC0sPm5o8t7taJca7Vu9qKKMKPAqiK4KhlrW/yVBFliT2CjAdVuYvGj87LCcEn6NMeF+4ONwDz++YwjrAWQT2zHUJPvLnYOZPiD1LUm3F34cEHD1y4Uf+oeiYgriUMauskfFn+ShbTmWWooFRFyNobzweg9aMBZSkA+kFFifqOV1FwHTOYg342NumRGB8svK0vvTUID7AU1Ev8YXYqCEGbb0UHth/1pEbAF103enLTtPZ6qKua74rE0qJ89OF9dZXv4JY9+8Juyps7bi2XEyHNm1nHZOY5tqdY0eKFmSGJMoWBDzGgdMjmXlWp4jnY2fC619nZqxMrCQpeT70CqnBhtBemUuu6jHYUbbrPjnXIkH3FZ7/qvZt0jF0IMP9FGw== amctagga@redhat.com", "key_options": null, "keyfile": "/home/amctagga/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amctagga", "validate_certs": true} 2021-07-11T03:47:40.954 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdqw+8an6CHqsagFqXfuzQi1zCFHCh0e9pDtYz+ylA7L+7vnJLjTR7km0bb4Z6VTUh7h0/nUfF6faPzD2KGmTKvS+mTpeOUaN/H8O8j8qqh90f8z4T8F8L3DwqQbPf/R0izXmTgz0FZObFHmj+G8a+D4ndxF80EPzKkML/G0Hg2GEVdYzazSY4imSY2f56ul6qByBknhFshaTo2DUDcdbDDokF6lO6WgXhY0MJrT24CUSMyZmyi+3hF7vowW2dfbJ+SuDft4XmPDnIxRhDWWvi12jZb8Le7SWYzUlu7fAGbWe63VJqNh992myI61aZshApDD24+1xqIoo+Z/XkQD3l sunnagar@localhost.localdomain", "key_options": null, "keyfile": "/home/sunnagar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunnagar", "validate_certs": true} 2021-07-11T03:47:40.988 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-07-11T03:47:41.014 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "k 2021-07-11T03:47:41.016 INFO:teuthology.task.ansible.out:ey_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-07-11T03:47:41.059 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTlsh9TFwLCeeD2RR+56VhRZ0hp9CHj72U3Z1gbMpxpcg6PYEDzb+0qPVQp1/cIhcwwy0sm13yFl/A5i0iCRpyqB2MbyaXjj0HgkNinxLBFkPSiW2C83lfAyAhG/gLk+FcmWE4+xVrE29aoSM0JtcNtp+q83UGdnFcvhFe0pgJbt5vITo3ZMJND/UZrzeewyK8Vy4Rvpfy8aHYi2Eq+7WQrRHwUwxn0CwsxldnUfmZtu58d+Rtt78LRF0mvcScNlC7q5D+8htmYwkWkK639w5IB3xPv+Z5cV49Psi24Q+9BIr9Zo9/OnnCTauAi5ivDlh+kk/iBksAE8rQpMhvfuSFnNoq2O87aPQvNQHdRCcOa7FVIDcuocu6AB8UP4cGcJreT4+BW5Zbk6vQeJZhmj9yExpRj9MuONy5793nKuAMPn9lOjuXRfK7A5Tb3OOhpEJK1/MjtdZz4mNmFFfFIZ24hgTZO0pOiMwDf/tQ6BG7H8NS9g3QmXPyZajUocw6Szs= lflores@redhat.com", "key_options": null, "keyfile": "/home/lflores/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lflores", "validate_certs": true} 2021-07-11T03:47:41.071 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtiHTDuQk8EDH5R+JfDMucI4ZpOzKN5SLD+j35QLLySgGXholoDxtqjHnmRjM1rpj5SIaXjHBZdJi7iax+1MRywHYBi5BfTtzYPgA+NH8mt+XkcdkEuFPcYxGh6fMQDQWb2g9Euze/6qi0W9mbYXvvb1/Q3bhVCXg1UgKD42BndWEpbMYFfzJLZWPtOD/lg81pwUmf/4BpPFv52sy87lQVfkkvcUliLObR4S0TI5hPLYxBS+RXmHYKgvPMB2QJ9wum62lJ+YM1MxPygS0OWbIB6UQGXmvxRXA4osXkxJ8ksPPCupOgFNiiYro91cQh3mmi6iHc9kMz5R1aNGGOfDdlnLx9DRSIiNcacjXEVNFaoM0x441f4GeU2ZR8/7uq2dfk7QjPlh1eJGxG9pzRiQ8JCDF1car0eYwXdMTsQoQqpIi5imqcELdcqGGp9FhzFElgUwpf15JYlkouVhS6hq4LDjhLNk7dDT8fmCEnHOPcbxaTgN7J3idnJK6zeIniuuk= root@iZj6ce071s2zz3reioxn93Z", "key_options": null, "keyfile": "/home/kevinz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kevinz", "validate_certs": true} 2021-07-11T03:47:41.093 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJOefD/VJfS15LRyuVDLluKv93Zgocsl8BhiDMimvBNbrGWc795v2+eevWuwaL/SlL44SnsNZ9Mz8FVn9YuFoQiHLg7cHZNLg9EJh5SMXzZzwLZHnF02MdI2CU6U++9nmh8FtBrdBWKzZ7o+01wOw+52dYOxvP7IJ7C/vjwOLgfigEN8Nd1PrxALVNIQz7fsiet0n3odk9PJfoqTu3IloXm9qbQiPFyfMA6b8a20RdLZCJDGGG5rd6gtsyeFHoAPZEz8FEUlNkow7NM7ZhdxOz/fWXyMZ27KmY8WSAgflbxysMzexyifkGzypiE3NQ10hdPGBgoYsHrAN4Qmte1/Z4aI5nxrNzLy/uNc+L/Bp/0XkLj9zfoiFFOReXvlz/laAV6LpbHyacr+iCwv9rlI7echXmvC6mSJXfBPuyR1uromUXpYRhEkn5uiOsCIQCuYpx2jeil7JV+LJlF62lLSndOv4g12Gq2TEZ4Bhsr2902kftPkLFenyfDTh3A2qGGNM= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-07-11T03:47:41.126 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-07-11T03:47:41.150 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGq2g8OFN5msjIn5CNB2BRss9eIyJRk3r+7RQcnBOQid jsawaya@redhat.com", "key_options": null, "keyfile": "/home/jsawaya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsawaya", "validate_certs": true} 2021-07-11T03:47:41.165 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1YAaxAvlRgu72huAggHpoDKSgtF/b00ACfUX4Wk5NyLFbYI3PAQapHNcNL4mIsH8pc6YkdjcJ5N/F12aEupJGQ0Q06OS324xapnaBoUy1hQl9IvhtPqTLEwR0X5xLIAhNBLIEJDNVg13MKwr0u5YVUXdrQRu7Kj9RXAPBahNZFo/Ed+cnR3LICQWliHeHFrk/DU2DUHol3a5wAPxrowAG0AxAecP6/7rASMmitvLQwdQcfIJAhWYgfyXzudg+gOmAEgv3O3U4QOikgmkm/EabUXLJnlTj6yuhtuuctdJr73UT3wLGAGmphLKCkug27kb1nTNoDa+PW/+grP63zhU1 chunmei@otccldstore05", "key_options": null, "keyfile": "/home/liu-chunmei/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liu-chunmei", "validate_certs": true} 2021-07-11T03:47:41.199 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "k 2021-07-11T03:47:41.201 INFO:teuthology.task.ansible.out:ey_options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-07-11T03:47:41.249 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-07-11T03:47:41.251 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNIdsy+yA0QjwkvjzTiH4lKB+txlh4oGkgC0r1l9swmFvD+MuzNNkD+oB/YscT51tJXsP5/wZDZMuk60xds4Os2ejoIBvzdGBgZ1G2B33etHPmzgTFE7ZegBkMlvWYmrJ9YS+iaMvXn0MMtqfE4zq0kemVAA4NBfFZMhHPJMKCg7Gj31MR0S0b7dcPx7Jnpg4A45oHr0qnufeqZYQaHGgo+LO2xdoV4V2EswWr7YG+b1wI6DzFZBbNsGyQd+cE1BzQ8NmETPNNpDPk5dCyBS7U+4ZTf/WbWk3+Cn+Ky47iWfxcscupQRkCjmTYI/uCMhnrpGU0zEHkbFwLTS6wqDtDmVHm++xof3xSsGJ2cE/vZySZGHqdxqVXfT8gt17h35rME8eX3TU392rjfK6kjEUD3UgwJJp4NFzQIEXi2HTmDYx/Vi9nuId3V1NrDz98QLPUd1ijiy1DuzI0BJPE/DTDmsb7hNXNIcAV8VRc1243W3eaee/Pdg/wmwfkrgjTU3s= shuaiwang@Shuais-MacBook-Pro.local", "key_options": null, "keyfile": "/home/markw8973240/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "markw8973240", "validate_certs": true} 2021-07-11T03:47:41.285 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLbPdDb+XHOD2fo8g3RacATj0cG07uELwUvqQP35y+JmhNRSb/ttaysFLA5ss0y77TiXx22saP3ECEm5SnuPg5IERkPQmXiw84ATTNfLnzx2G5H223SLaTQiGfSgniBCJHVPCJfNahJiurlZoOAlwDz6SKXTUiivVrFJ22iTBDCQUO6HFQKjE7aM3RAo5mZXKVlTXzM6WOd8vlzk3HMzV0/91JR6H86Lt9MEzXT4dOzqmtxd8gS6U+vA+50zOIspdK+orTRJ0PO2Fn8c1RSYM8g7uqQtpDcOEHderW8TRsqBbDxpyslK6hRX5urUNBX506XE6JNoViLNvCofgseCGj yingxin@yingxin-server", "key_options": null, "keyfile": "/home/cyx1231st/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cyx1231st", "validate_certs": true} 2021-07-11T03:47:41.307 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-07-11T03:47:41.361 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+ZvZuogU+WfyGj4BR2xYllNoQqAoxQ6AM0IYddBV2pL1xgTM/SopuAxDRP0q2stpozMtjVjlvuiIgr/caqyTuOgYtYCvB09ppefqdnCDrJJBCOKpjUEnCpAKcBMMuz9V32Zqt5FgpIi0Mu7Gp/QCvB97imxEkoUunpdX/8Ri/xz0nJO+QqoJX5wOn91jG49XnkMCLzx+MuafhT90WFO9CV1sIVREZS+X2C7ACQuhx9Qwk7R4BXt2f3M4dA0K2nLwaAnVW50M1y6Gs50Wl+ojDR1itySOxv8YpwyRlRTOzQTyVMnG0Qsaxxwp+hy4b/vQW3NIUvHYdTyvqALTQdGHT lricomor@lricomor-mac", "key_options": null, "keyfile": "/home/luisrico/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "luisrico", "validate_certs": true} 2021-07-11T03:47:41.363 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDwVh9KqjV0b+65aBxQV66IUwNo3D5zTbN8U6Kh6eE4QtaIVe5Mmvi2SBbGMSWofE0zIIw2JmsI5LzdOoCyZl/3gNDPHQd6D9o0xRHQv94giZ2a5K+mW79foAxsuppTjV/Fbg8EZ/19W91+BqMWotlMR66ka0RPT/6U8V5mzXoydSminViO/B6q7ZYzIIgGZqaTsSxCMvz5BAzzeEaGfSlGxw5MOye9sF/tDppuRKG4rMUZULEWGuxgXAtabtXCduocAArqAY3BRJEgj7JetPdPNjipGrULyYQmT7GeknvP/waC0sPm5o8t7taJca7Vu9qKKMKPAqiK4KhlrW/yVBFliT2CjAdVuYvGj87LCcEn6NMeF+4ONwDz++YwjrAWQT2zHUJPvLnYOZPiD1LUm3F34cEHD1y4Uf+oeiYgriUMauskfFn+ShbTmWWooFRFyNobzweg9aMBZSkA+kFFifqOV1FwHTOYg342NumRGB8svK0vvTUID7AU1Ev8YXYqCEGbb0UHth/1pEbAF103enLTtPZ6qKua74rE0qJ89OF9dZXv4JY9+8Juyps7bi2XEyHNm1nHZOY5tqdY0eKFmSGJMoWBDzGgdMjmXlWp4jnY2fC619nZqxMrCQpeT70CqnBhtBemUuu6jHYUbbrPjnXIkH3FZ7/qvZt0jF0IMP9FGw== amctagga@redhat.com", "key_options": null, "keyfile": "/home/amctagga/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amctagga", "validate_certs": true} 2021-07-11T03:47:41.407 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdqw+8an6CHqsagFqXfuzQi1zCFHCh0e9pDtYz+ylA7L+7vnJLjTR7km0bb4Z6VTUh7h0/nUfF6faPzD2KGmTKvS+mTpeOUaN/H8O8j8qqh90f8z4T8F8L3DwqQbPf/R0izXmTgz0FZObFHmj+G8a+D4ndxF80EPzKkML/G0Hg2GEVdYzazSY4imSY2f56ul6qByBknhFshaTo2DUDcdbDDokF6lO6WgXhY0MJrT24CUSMyZmyi+3hF7vowW2dfbJ+SuDft4XmPDnIxRhDWWvi12jZb8Le7SWYzUlu7fAGbWe63VJqNh992myI61aZshApDD24+1xqIoo+Z/XkQD3l sunnagar@localhost.localdomain", "key_options": null, "keyfile": "/home/sunnagar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunnagar", "validate_certs": true} 2021-07-11T03:47:41.420 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-07-11T03:47:41.489 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtiHTDuQk8EDH5R+JfDMucI4ZpOzKN5SLD+j35QLLySgGXholoDxtqjHnmRjM1rpj5SIaXjHBZdJi7iax+1MRywHYBi5BfTtzYPgA+NH8mt+XkcdkEuFPcYxGh6fMQDQWb2g9Euze/6qi0W9mbYXvvb1/Q3bhVCXg1UgKD42BndWEpbMYFfzJLZWPtOD/lg81pwUmf/4BpPFv52sy87lQVfkkvcUliLObR4S0TI5hPLYxBS+RXmHYKgvPMB2QJ9wum62lJ+YM1MxPygS0OWbIB6UQGXmvxRXA4osXkxJ8ksPPCupOgFNiiYro91cQh3mmi6iHc9kMz5R1aNGGOfDdlnLx9DRSIiNcacjXEVNFaoM0x441f4GeU2ZR8/7uq2dfk7QjPlh1eJGxG9pzRiQ8JCDF1car0eYwXdMTsQoQqpIi5imqcELdcqGGp9FhzFElgUwpf15JYlkouVhS6hq4LDjhLNk7dDT8fmCEnHOPcbxaTgN7J3idnJK6zeIniuuk= root@iZj6ce071s2zz3reioxn93Z", "key_options": null, "keyfile": "/home/kevinz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kevinz", "validate_certs": true} 2021-07-11T03:47:41.503 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTlsh9TFwLCeeD2RR+56VhRZ0hp9CHj72U3Z1gbMpxpcg6PYEDzb+0qPVQp1/cIhcwwy0sm13yFl/A5i0iCRpyqB2MbyaXjj0HgkNinxLBFkPSiW2C83lfAyAhG/gLk+FcmWE4+xVrE29aoSM0JtcNtp+q83UGdnFcvhFe0pgJbt5vITo3ZMJND/UZrzeewyK8Vy4Rvpfy8aHYi2Eq+7WQrRHwUwxn0CwsxldnUfmZtu58d+Rtt78LRF0mvcScNlC7q5D+8htmYwkWkK639w5IB3xPv+Z5cV49Psi24Q+9BIr9Zo9/OnnCTauAi5ivDlh+kk/iBksAE8rQpMhvfuSFnNoq2O87aPQvNQHdRCcOa7FVIDcuocu6AB8UP4cGcJreT4+BW5Zbk6vQeJZhmj9yExpRj9MuONy5793nKuAMPn9lOjuXRfK7A5Tb3OOhpEJK1/MjtdZz4mNmFFfFIZ24hgTZO0pOiMwDf/tQ6BG7H8NS9g3QmXPyZajUocw6Szs= lflores@redhat.com", "key_options": null, "keyfile": "/home/lflores/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lflores", "validate_certs": true} 2021-07-11T03:47:41.507 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICYMwYvykHtnVhhJwcKaiRh9oMpJkqvEA7OAbHPYOv6W matan.brz@gmail.com", "key_options": null, "keyfile": "/home/matan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "matan", "validate_certs": true} 2021-07-11T03:47:41.529 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDwVh9KqjV0b+65aBxQV66IUwNo3D5zTbN8U6Kh6eE4QtaIVe5Mmvi2SBbGMSWofE0zIIw2JmsI5LzdOoCyZl/3gNDPHQd6D9o0xRHQv94giZ2a5K+mW79foAxsuppTjV/Fbg8EZ/19W91+BqMWotlMR66ka0RPT/6U8V5mzXoydSminViO/B6q7ZYzIIgGZqaTsSxCMvz5BAzzeEaGfSlGxw5MOye9sF/tDppuRKG4rMUZULEWGuxgXAtabtXCduocAArqAY3BRJEgj7JetPdPNjipGrULyYQmT7GeknvP/waC0sPm5o8t7taJca7Vu9qKKMKPAqiK4KhlrW/yVBFliT2CjAdVuYvGj87LCcEn6NMeF+4ONwDz++YwjrAWQT2zHUJPvLnYOZPiD1LUm3F34cEHD1y4Uf+oeiYgriUMauskfFn+ShbTmWWooFRFyNobzweg9aMBZSkA+kFFifqOV1FwHTOYg342NumRGB8svK0vvTUID7AU1Ev8YXYqCEGbb0UHth/1pEbAF103enLTtPZ6qKua74rE0qJ89OF9dZXv4JY9+8Juyps7bi2XEyHNm1nHZOY5tqdY0eKFmSGJMoWBDzGgdMjmXlWp4jnY2fC619nZqxMrCQpeT70CqnBhtBemUuu6jHYUbbrPjnXIkH3FZ7/qvZt0jF0IMP9FGw== amctagga@redhat.com", "key_options": null, "keyfile": "/home/amctagga/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amctagga", "validate_certs": true} 2021-07-11T03:47:41.614 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGq2g8OFN5msjIn5CNB2BRss9eIyJRk3r+7RQcnBOQid jsawaya@redhat.com", "key_options": null, "keyfile": "/home/jsawaya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsawaya", "validate_certs": true} 2021-07-11T03:47:41.626 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1YAaxAvlRgu72huAggHpoDKSgtF/b00ACfUX4Wk5NyLFbYI3PAQapHNcNL4mIsH8pc6YkdjcJ5N/F12aEupJGQ0Q06OS324xapnaBoUy1hQl9IvhtPqTLEwR0X5xLIAhNBLIEJDNVg13MKwr0u5YVUXdrQRu7Kj9RXAPBahNZFo/Ed+cnR3LICQWliHeHFrk/DU2DUHol3a5wAPxrowAG0AxAecP6/7rASMmitvLQwdQcfIJAhWYgfyXzudg+gOmAEgv3O3U4QOikgmkm/EabUXLJnlTj6yuhtuuctdJr73UT3wLGAGmphLKCkug27kb1nTNoDa+PW/+grP63zhU1 chunmei@otccldstore05", "key_options": null, "keyfile": "/home/liu-chunmei/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liu-chunmei", "validate_certs": true} 2021-07-11T03:47:41.655 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/Gyl/Xhn6OJ6mdvGv0a7LUMeEU6xDfecjod/OUf8yBFFLm/h0AMifHH8qQYoH9SkeUthXIggbi0fcx9nrgyqYgLOansWiEIPbBbVCpvBz1vT1UR0Bxg6B7bjcrMgKg3M1YhTiHt41PlxdW/nlhTlxyZ1+nM50Fu/WE1+4nVIUaTtyaKv8PnmLmnpDnNly9X9CPDyilx8xpNiZMTv7oaiilCIfnmrKaZEDuvpU0M6h5wa6wJCatxTGIt/RQo2c4/YL+Nmx0xkouoRwR1H6swhkZPbInFgy/IlmLuu+owFR5mCVdxmst04o18CyvlW//Wiepw/5uJA3Xivcu1hwwcesTO+xJgIQbM/hvJ/UmL/eJvnV9G9g1B7ttrVfITbYgXZ1gbtP4pJCUmq132o6q+qeyOsGl2k5f/rRl5Adch5lzepffSSfsYn2tS77b7uXf3ujGvwhlOwJRITRCDZ4nGFwijFwBg8ptFKTaXZsXjMQRtSP2kuHoRonmOm1XbD4ElM= root@omri", "key_options": null, "keyfile": "/home/ozeneva/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ozeneva", "validate_certs": true} 2021-07-11T03:47:41.712 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtiHTDuQk8EDH5R+JfDMucI4ZpOzKN5SLD+j35QLLySgGXholoDxtqjHnmRjM1rpj5SIaXjHBZdJi7iax+1MRywHYBi5BfTtzYPgA+NH8mt+XkcdkEuFPcYxGh6fMQDQWb2g9Euze/6qi0W9mbYXvvb1/Q3bhVCXg1UgKD42BndWEpbMYFfzJLZWPtOD/lg81pwUmf/4BpPFv52sy87lQVfkkvcUliLObR4S0TI5hPLYxBS+RXmHYKgvPMB2QJ9wum62lJ+YM1MxPygS0OWbIB6UQGXmvxRXA4osXkxJ8ksPPCupOgFNiiYro91cQh3mmi6iHc9kMz5R1aNGGOfDdlnLx9DRSIiNcacjXEVNFaoM0x441f4GeU2ZR8/7uq2dfk7QjPlh1eJGxG9pzRiQ8JCDF1car0eYwXdMTsQoQqpIi5imqcELdcqGGp9FhzFElgUwpf15JYlkouVhS6hq4LDjhLNk7dDT8fmCEnHOPcbxaTgN7J3idnJK6zeIniuuk= root@iZj6ce071s2zz3reioxn93Z", "key_options": null, "keyfile": "/home/kevinz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kevinz", "validate_certs": true} 2021-07-11T03:47:41.724 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLbPdDb+XHOD2fo8g3RacATj0cG07uELwUvqQP35y+JmhNRSb/ttaysFLA5ss0y77TiXx22saP3ECEm5SnuPg5IERkPQmXiw84ATTNfLnzx2G5H223SLaTQiGfSgniBCJHVPCJfNahJiurlZoOAlwDz6SKXTUiivVrFJ22iTBDCQUO6HFQKjE7aM3RAo5mZXKVlTXzM6WOd8vlzk3HMzV0/91JR6H86Lt9MEzXT4dOzqmtxd8gS6U+vA+50zOIspdK+orTRJ0PO2Fn8c1RSYM8g7uqQtpDcOEHderW8TRsqBbDxpyslK6hRX5urUNBX506XE6JNoViLNvCofgseCGj yingxin@yingxin-server", "key_options": null, "keyfile": "/home/cyx1231st/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cyx1231st", "validate_certs": true} 2021-07-11T03:47:41.765 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNIdsy+yA0QjwkvjzTiH4lKB+txlh4oGkgC0r1l9swmFvD+MuzNNkD+oB/YscT51tJXsP5/wZDZMuk60xds4Os2ejoIBvzdGBgZ1G2B33etHPmzgTFE7ZegBkMlvWYmrJ9YS+iaMvXn0MMtqfE4zq0kemVAA4NBfFZMhHPJMKCg7Gj31MR0S0b7dcPx7Jnpg4A45oHr0qnufeqZYQaHGgo+LO2xdoV4V2EswWr7YG+b1wI6DzFZBbNsGyQd+cE1BzQ8NmETPNNpDPk5dCyBS7U+4ZTf/WbWk3+Cn+Ky47iWfxcscupQRkCjmTYI/uCMhnrpGU0zEHkbFwLTS6wqDtDmVHm++xof3xSsGJ2cE/vZySZGHqdxqVXfT8gt17h35rME8eX3TU392rjfK6kjEUD3UgwJJp4NFzQIEXi2HTmDYx/Vi9nuId3V1NrDz98QLPUd1ijiy1DuzI0BJPE/DTDmsb7hNXNIcAV8VRc1243W3eaee/Pdg/wmwfkrgjTU3s= shuaiwang@Shuais-MacBook-Pro.local", "key_options": null, "keyfile": "/home/markw8973240/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "markw8973240", "validate_certs": true} 2021-07-11T03:47:41.788 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDepyzA+MeQT5oRM52SxmUUKVAzkcrH/NqFxoCpBAjUSioTMaB1h0PhHE00nvHnKRvqWdPl8LCBWS3Ncjh30r9lwHaP+oPy4iLQEc7D2Mwn7WyPh5cq9N3UF25C0AlMG744k1tBNb97qy73OJI+oc5pAj4vUpCC1Ut9pMP5E5pQCx/kwgSOpC7mk0dYr/H5FxjPb0ca55eisFTLXraHPSChdrHjlYC5XpOZ9csilTwFdF0fJdu0nMDwIxGvwTOaUcIcfNLkpGB2Sj/pAM4L9pVsIbeE3C8d978MZ8Ff+ksQ0T8eNzew2ZjSkSuV+oR1sBCJYZ7mbDQuYLKAs5DanO1VFJvgjBJm1zP2ONu+g0fn/j04qIosuudziwoG/dOTkFo+6dBvF5hQi7TbZOIUVFr5dBlGS+O4eoK0gi/wpzun227E4dadY9uB1wiZ/PTD/5og+gxWG32RkHqqX3ALlK4m8JKzrq214nZ7Dt9CNewiUkR15uRkDasGIdiu6qb4Ez8= ArunaSudhan@sudhan-linux", "key_options": null, "keyfile": "/home/sudhanaruna/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sudhanaruna", "validate_certs": true} 2021-07-11T03:47:41.811 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1YAaxAvlRgu72huAggHpoDKSgtF/b00ACfUX4Wk5NyLFbYI3PAQapHNcNL4mIsH8pc6YkdjcJ5N/F12aEupJGQ0Q06OS324xapnaBoUy1hQl9IvhtPqTLEwR0X5xLIAhNBLIEJDNVg13MKwr0u5YVUXdrQRu7Kj9RXAPBahNZFo/Ed+cnR3LICQWliHeHFrk/DU2DUHol3a5wAPxrowAG0AxAecP6/7rASMmitvLQwdQcfIJAhWYgfyXzudg+gOmAEgv3O3U4QOikgmkm/EabUXLJnlTj6yuhtuuctdJr73UT3wLGAGmphLKCkug27kb1nTNoDa+PW/+grP63zhU1 chunmei@otccldstore05", "key_options": null, "keyfile": "/home/liu-chunmei/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liu-chunmei", "validate_certs": true} 2021-07-11T03:47:41.859 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdqw+8an6CHqsagFqXfuzQi1zCFHCh0e9pDtYz+ylA7L+7vnJLjTR7km0bb4Z6VTUh7h0/nUfF6faPzD2KGmTKvS+mTpeOUaN/H8O8j8qqh90f8z4T8F8L3DwqQbPf/R0izXmTgz0FZObFHmj+G8a+D4ndxF80EPzKkML/G0Hg2GEVdYzazSY4imSY2f56ul6qByBknhFshaTo2DUDcdbDDokF6lO6WgXhY0MJrT24CUSMyZmyi+3hF7vowW2dfbJ+SuDft4XmPDnIxRhDWWvi12jZb8Le7SWYzUlu7fAGbWe63VJqNh992myI61aZshApDD24+1xqIoo+Z/XkQD3l sunnagar@localhost.localdomain", "key_options": null, "keyfile": "/home/sunnagar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunnagar", "validate_certs": true} 2021-07-11T03:47:41.887 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+ZvZuogU+WfyGj4BR2xYllNoQqAoxQ6AM0IYddBV2pL1xgTM/SopuAxDRP0q2stpozMtjVjlvuiIgr/caqyTuOgYtYCvB09ppefqdnCDrJJBCOKpjUEnCpAKcBMMuz9V32Zqt5FgpIi0Mu7Gp/QCvB97imxEkoUunpdX/8Ri/xz0nJO+QqoJX5wOn91jG49XnkMCLzx+MuafhT90WFO9CV1sIVREZS+X2C7ACQuhx9Qwk7R4BXt2f3M4dA0K2nLwaAnVW50M1y6Gs50Wl+ojDR1itySOxv8YpwyRlRTOzQTyVMnG0Qsaxxwp+hy4b/vQW3NIUvHYdTyvqALTQdGHT lricomor@lricomor-mac", "key_options": null, "keyfile": "/home/luisrico/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "luisrico", "validate_certs": true} 2021-07-11T03:47:41.889 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuHIpWAz6Sk3ATLvTG4KS24JUzgqQBEm4EesPgf1cEG+xK4gISUixaVC9OSNS/KR+8/yoWAhmw992p8cp7mWXxqrRC9kIrthLWvD6Jjz3HTXgarO2oNibeVu27kK/Fm2vvUNACARBkoC5YXd6IuoZOEg1bJu06DLw5Y7pGMqZRVDOrE8WUDwr27roZSgU2/N+18Y1LYn4gwSvWpuC/IotLyGSpW7/dgrZAIreRoYboMdTSlIjjuzeWvYx/AnZmkeEo0IhfyuBRl7ZQwxSFYHn9Z1+iQkt+O9hEZYSZuYRuyFPRqHNMKCEj1iYVRMcwbf/x2/dhjfmbWmjOMJoNNwm+DpixHHbId6Idx82GlAtCgQSTe9aG783oCeZYuVUr1plIRjeEsdjY9UBRYKNknEAChKjMSbxysvO3Dy4yrU2r8qo1iBS1fi4HUg12asCLrJX7vSCvOhOn7ltqDTmt5UiLSn+eCzXpOTBDWz7McB28FoYyR+Rwz5GOH3ITFhPyhXc= wzl@manjaro-wzl", "key_options": null, "keyfile": "/home/zl31wang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zl31wang", "validate_certs": true} 2021-07-11T03:47:41.928 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLbPdDb+XHOD2fo8g3RacATj0cG07uELwUvqQP35y+JmhNRSb/ttaysFLA5ss0y77TiXx22saP3ECEm5SnuPg5IERkPQmXiw84ATTNfLnzx2G5H223SLaTQiGfSgniBCJHVPCJfNahJiurlZoOAlwDz6SKXTUiivVrFJ22iTBDCQUO6HFQKjE7aM3RAo5mZXKVlTXzM6WOd8vlzk3HMzV0/91JR6H86Lt9MEzXT4dOzqmtxd8gS6U+vA+50zOIspdK+orTRJ0PO2Fn8c1RSYM8g7uqQtpDcOEHderW8TRsqBbDxpyslK6hRX5urUNBX506XE6JNoViLNvCofgseCGj yingxin@yingxin-server", "key_options": null, "keyfile": "/home/cyx1231st/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cyx1231st", "validate_certs": true} 2021-07-11T03:47:41.975 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTlsh9TFwLCeeD2RR+56VhRZ0hp9CHj72U3Z1gbMpxpcg6PYEDzb+0qPVQp1/cIhcwwy0sm13yFl/A5i0iCRpyqB2MbyaXjj0HgkNinxLBFkPSiW2C83lfAyAhG/gLk+FcmWE4+xVrE29aoSM0JtcNtp+q83UGdnFcvhFe0pgJbt5vITo3ZMJND/UZrzeewyK8Vy4Rvpfy8aHYi2Eq+7WQrRHwUwxn0CwsxldnUfmZtu58d+Rtt78LRF0mvcScNlC7q5D+8htmYwkWkK639w5IB3xPv+Z5cV49Psi24Q+9BIr9Zo9/OnnCTauAi5ivDlh+kk/iBksAE8rQpMhvfuSFnNoq2O87aPQvNQHdRCcOa7FVIDcuocu6AB8UP4cGcJreT4+BW5Zbk6vQeJZhmj9yExpRj9MuONy5793nKuAMPn9lOjuXRfK7A5Tb3OOhpEJK1/MjtdZz4mNmFFfFIZ24hgTZO0pOiMwDf/tQ6BG7H8NS9g3QmXPyZajUocw6Szs= lflores@redhat.com", "key_options": null, "keyfile": "/home/lflores/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lflores", "validate_certs": true} 2021-07-11T03:47:42.034 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIi8JMOVD91QR7EXBkhxeAsB2UO2gyWqYPiP6HjJO/TzzMKf7QtO928XzdORwQ1/jm2hDF1kaNPaU/XIHD0hHSvfICS4ELQcBPa5uPya6d5Kn5fVlpGqBcXBxKGbHB01PGjn0214eupt2LebWxkm7TP/BGDHKXXaoR748AlbyyG2L3TaNoGkwQiEoZe3CWgH4qkrf2CEpyfOa7BM7U0h+lceJxIGe+Yo6Ll14zyngDBa059E+ndFTpkQ6vECII7FdB1+A4Orhu40XHZgyVnlRudD1l80pIVnZves8N2zHZhQo8hp0sB5oY8HENiB09j7Xa6YQ/c1c2wm9E2xJHSNo7muFiHz48vQEgvNmu0lZdbVwXIy3IgLEBHjs6Kyqjtg618V4UaoP/LBOqtWtLd6TvGJ3qBlNyvhO5gZ2DGeLMN2WWoHxfl3VM+QGCGI2dnzUyHhfG1VBDYj8fzWXOuUVpVCMUw8fEg8GWmYLDwm3SbdMACWU87J0kAmaDPKcwwL6AwtWJZr/p89sSypl6y50JlOoDrbDbxrIbHQgbf1tcEgPOuLUN+O06qpi+OxKMtFdbvz+NQ4V4CpXpvMMQZj5AbuHp0EJ5Vme/Kcmpl/xoYGyFDzOEGR8lw6LMz+CpQ4GtnK5XXe1Xn3TJh6Tn2tVyS20p7artemGQQquCN8iQOw== mayag@gett.com", "key_options": null, "keyfile": "/home/mayagilad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mayagilad", "validate_certs": true} 2021-07-11T03:47:42.036 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICYMwYvykHtnVhhJwcKaiRh9oMpJkqvEA7OAbHPYOv6W matan.brz@gmail.com", "key_options": null, "keyfile": "/home/matan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "matan", "validate_certs": true} changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdqw+8an6CHqsagFqXfuzQi1zCFHCh0e9pDtYz+ylA7L+7vnJLjTR7km0bb4Z6VTUh7h0/nUfF6faPzD2KGmTKvS+mTpeOUaN/H8O8j8qqh90f8z4T8F8L3DwqQbPf/R0izXmTgz0FZObFHmj+G8a+D4ndxF80EPzKkML/G0Hg2GEVdYzazSY4imSY2f56ul6qByBknhFshaTo2DUDcdbDDokF6lO6WgXhY0MJrT24CUSMyZmyi+3hF7vowW2dfbJ+SuDft4XmPDnIxRhDWWvi12jZb8Le7SWYzUlu7fAGbWe63VJqNh992myI61aZshApDD24+1xqIoo+Z/XkQD3l sunnagar@localhost.localdomain", "key_options": null, "keyfile": "/home/sunnagar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunnagar", "validate_certs": true} 2021-07-11T03:47:42.108 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGq2g8OFN5msjIn5CNB2BRss9eIyJRk3r+7RQcnBOQid jsawaya@redhat.com", "key_options": null, "keyfile": "/home/jsawaya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsawaya", "validate_certs": true} 2021-07-11T03:47:42.134 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:42 +0000 (0:00:19.255) 0:00:56.609 *********** 2021-07-11T03:47:42.156 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTlsh9TFwLCeeD2RR+56VhRZ0hp9CHj72U3Z1gbMpxpcg6PYEDzb+0qPVQp1/cIhcwwy0sm13yFl/A5i0iCRpyqB2MbyaXjj0HgkNinxLBFkPSiW2C83lfAyAhG/gLk+FcmWE4+xVrE29aoSM0JtcNtp+q83UGdnFcvhFe0pgJbt5vITo3ZMJND/UZrzeewyK8Vy4Rvpfy8aHYi2Eq+7WQrRHwUwxn0CwsxldnUfmZtu58d+Rtt78LRF0mvcScNlC7q5D+8htmYwkWkK639w5IB3xPv+Z5cV49Psi24Q+9BIr9Zo9/OnnCTauAi5ivDlh+kk/iBksAE8rQpMhvfuSFnNoq2O87aPQvNQHdRCcOa7FVIDcuocu6AB8UP4cGcJreT4+BW5Zbk6vQeJZhmj9yExpRj9MuONy5793nKuAMPn9lOjuXRfK7A5Tb3OOhpEJK1/MjtdZz4mNmFFfFIZ24hgTZO0pOiMwDf/tQ6BG7H8NS9g3QmXPyZajUocw6Szs= lflores@redhat.com", "key_options": null, "keyfile": "/home/lflores/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lflores", "validate_certs": true} 2021-07-11T03:47:42.158 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/Gyl/Xhn6OJ6mdvGv0a7LUMeEU6xDfecjod/OUf8yBFFLm/h0AMifHH8qQYoH9SkeUthXIggbi0fcx9nrgyqYgLOansWiEIPbBbVCpvBz1vT1UR0Bxg6B7bjcrMgKg3M1YhTiHt41PlxdW/nlhTlxyZ1+nM50Fu/WE1+4nVIUaTtyaKv8PnmLmnpDnNly9X9CPDyilx8xpNiZMTv7oaiilCIfnmrKaZEDuvpU0M6h5wa6wJCatxTGIt/RQo2c4/YL+Nmx0xkouoRwR1H6swhkZPbInFgy/IlmLuu+owFR5mCVdxmst04o18CyvlW//Wiepw/5uJA3Xivcu1hwwcesTO+xJgIQbM/hvJ/UmL/eJvnV9G9g1B7ttrVfITbYgXZ1gbtP4pJCUmq132o6q+qeyOsGl2k5f/rRl5Adch5lzepffSSfsYn2tS77b7uXf3ujGvwhlOwJRITRCDZ4nGFwijFwBg8ptFKTaXZsXjMQRtSP2kuHoRonmOm1XbD4ElM= root@omri", "key_options": null, "keyfile": "/home/ozeneva/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ozeneva", "validate_certs": true} 2021-07-11T03:47:42.169 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNIdsy+yA0QjwkvjzTiH4lKB+txlh4oGkgC0r1l9swmFvD+MuzNNkD+oB/YscT51tJXsP5/wZDZMuk60xds4Os2ejoIBvzdGBgZ1G2B33etHPmzgTFE7ZegBkMlvWYmrJ9YS+iaMvXn0MMtqfE4zq0kemVAA4NBfFZMhHPJMKCg7Gj31MR0S0b7dcPx7Jnpg4A45oHr0qnufeqZYQaHGgo+LO2xdoV4V2EswWr7YG+b1wI6DzFZBbNsGyQd+cE1BzQ8NmETPNNpDPk5dCyBS7U+4ZTf/WbWk3+Cn+Ky47iWfxcscupQRkCjmTYI/uCMhnrpGU0zEHkbFwLTS6wqDtDmVHm++xof3xSsGJ2cE/vZySZGHqdxqVXfT8gt17h35rME8eX3TU392rjfK6kjEUD3UgwJJp4NFzQIEXi2HTmDYx/Vi9nuId3V1NrDz98QLPUd1ijiy1DuzI0BJPE/DTDmsb7hNXNIcAV8VRc1243W3eaee/Pdg/wmwfkrgjTU3s= shuaiwang@Shuais-MacBook-Pro.local", "key_options": null, "keyfile": "/home/markw8973240/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "markw8973240", "validate_certs": true} 2021-07-11T03:47:42.226 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.238 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.249 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.250 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGq2g8OFN5msjIn5CNB2BRss9eIyJRk3r+7RQcnBOQid jsawaya@redhat.com", "key_options": null, "keyfile": "/home/jsawaya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsawaya", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-07-11T03:47:42.251 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5L 2021-07-11T03:47:42.252 INFO:teuthology.task.ansible.out:FRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.263 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.276 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-07-11T03:47:42.278 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDepyzA+MeQT5oRM52SxmUUKVAzkcrH/NqFxoCpBAjUSioTMaB1h0PhHE00nvHnKRvqWdPl8LCBWS3Ncjh30r9lwHaP+oPy4iLQEc7D2Mwn7WyPh5cq9N3UF25C0AlMG744k1tBNb97qy73OJI+oc5pAj4vUpCC1Ut9pMP5E5pQCx/kwgSOpC7mk0dYr/H5FxjPb0ca55eisFTLXraHPSChdrHjlYC5XpOZ9csilTwFdF0fJdu0nMDwIxGvwTOaUcIcfNLkpGB2Sj/pAM4L9pVsIbeE3C8d978MZ8Ff+ksQ0T8eNzew2ZjSkSuV+oR1sBCJYZ7mbDQuYLKAs5DanO1VFJvgjBJm1zP2ONu+g0fn/j04qIosuudziwoG/dOTkFo+6dBvF5hQi7TbZOIUVFr5dBlGS+O4eoK0gi/wpzun227E4dadY9uB1wiZ/PTD/5og+gxWG32RkHqqX3ALlK4m8JKzrq214nZ7Dt9CNewiUkR15uRkDasGIdiu6qb4Ez8= ArunaSudhan@sudhan-linux", "key_options": null, "keyfile": "/home/sudhanaruna/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sudhanaruna", "validate_certs": true} 2021-07-11T03:47:42.279 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:42.281 INFO:teuthology.task.ansible.out:TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.292 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.294 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.304 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.306 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+ZvZuogU+WfyGj4BR2xYllNoQqAoxQ6AM0IYddBV2pL1xgTM/SopuAxDRP0q2stpozMtjVjlvuiIgr/caqyTuOgYtYCvB09ppefqdnCDrJJBCOKpjUEnCpAKcBMMuz9V32Zqt5FgpIi0Mu7Gp/QCvB97imxEkoUunpdX/8Ri/xz0nJO+QqoJX5wOn91jG49XnkMCLzx+MuafhT90WFO9CV1sIVREZS+X2C7ACQuhx9Qwk7R4BXt2f3M4dA0K2nLwaAnVW50M1y6Gs50Wl+ojDR1itySOxv8YpwyRlRTOzQTyVMnG0Qsaxxwp+hy4b/vQW3NIUvHYdTyvqALTQdGHT lricomor@lricomor-mac", "key_options": null, "keyfile": "/home/luisrico/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "luisrico", "validate_certs": true} 2021-07-11T03:47:42.351 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNIdsy+yA0QjwkvjzTiH4lKB+txlh4oGkgC0r1l9swmFvD+MuzNNkD+oB/YscT51tJXsP5/wZDZMuk60xds4Os2ejoIBvzdGBgZ1G2B33etHPmzgTFE7ZegBkMlvWYmrJ9YS+iaMvXn0MMtqfE4zq0kemVAA4NBfFZMhHPJMKCg7Gj31MR0S0b7dcPx7Jnpg4A45oHr0qnufeqZYQaHGgo+LO2xdoV4V2EswWr7YG+b1wI6DzFZBbNsGyQd+cE1BzQ8NmETPNNpDPk5dCyBS7U+4ZTf/WbWk3+Cn+Ky47iWfxcscupQRkCjmTYI/uCMhnrpGU0zEHkbFwLTS6wqDtDmVHm++xof3xSsGJ2cE/vZySZGHqdxqVXfT8gt17h35rME8eX3TU392rjfK6kjEUD3UgwJJp4NFzQIEXi2HTmDYx/Vi9nuId3V1NrDz98QLPUd1ijiy1DuzI0BJPE/DTDmsb7hNXNIcAV8VRc1243W3eaee/Pdg/wmwfkrgjTU3s= shuaiwang@Shuais-MacBook-Pro.local", "key_options": null, "keyfile": "/home/markw8973240/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "markw8973240", "validate_certs": true} 2021-07-11T03:47:42.385 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuHIpWAz6Sk3ATLvTG4KS24JUzgqQBEm4EesPgf1cEG+xK4gISUixaVC9OSNS/KR+8/yoWAhmw992p8cp7mWXxqrRC9kIrthLWvD6Jjz3HTXgarO2oNibeVu27kK/Fm2vvUNACARBkoC5YXd6IuoZOEg1bJu06DLw5Y7pGMqZRVDOrE8WUDwr27roZSgU2/N+18Y1LYn4gwSvWpuC/IotLyGSpW7/dgrZAIreRoYboMdTSlIjjuzeWvYx/AnZmkeEo0IhfyuBRl7ZQwxSFYHn9Z1+iQkt+O9hEZYSZuYRuyFPRqHNMKCEj1iYVRMcwbf/x2/dhjfmbWmjOMJoNNwm+DpixHHbId6Idx82GlAtCgQSTe9aG783oCeZYuVUr1plIRjeEsdjY9UBRYKNknEAChKjMSbxysvO3Dy4yrU2r8qo1iBS1fi4HUg12asCLrJX7vSCvOhOn7ltqDTmt5UiLSn+eCzXpOTBDWz7McB28FoYyR+Rwz5GOH3ITFhPyhXc= wzl@manjaro-wzl", "key_options": null, "keyfile": "/home/zl31wang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zl31wang", "validate_certs": true} 2021-07-11T03:47:42.407 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-07-11T03:47:42.409 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-07-11T03:47:42.419 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.421 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.431 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-07-11T03:47:42.435 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICYMwYvykHtnVhhJwcKaiRh9oMpJkqvEA7OAbHPYOv6W matan.brz@gmail.com", "key_options": null, "keyfile": "/home/matan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "matan", "validate_certs": true} 2021-07-11T03:47:42.468 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+ZvZuogU+WfyGj4BR2xYllNoQqAoxQ6AM0IYddBV2pL1xgTM/SopuAxDRP0q2stpozMtjVjlvuiIgr/caqyTuOgYtYCvB09ppefqdnCDrJJBCOKpjUEnCpAKcBMMuz9V32Zqt5FgpIi0Mu7Gp/QCvB97imxEkoUunpdX/8Ri/xz0nJO+QqoJX5wOn91jG49XnkMCLzx+MuafhT90WFO9CV1sIVREZS+X2C7ACQuhx9Qwk7R4BXt2f3M4dA0K2nLwaAnVW50M1y6Gs50Wl+ojDR1itySOxv8YpwyRlRTOzQTyVMnG0Qsaxxwp+hy4b/vQW3NIUvHYdTyvqALTQdGHT lricomor@lricomor-mac", "key_options": null, "keyfile": "/home/luisrico/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "luisrico", "validate_certs": true} 2021-07-11T03:47:42.501 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIi8JMOVD91QR7EXBkhxeAsB2UO2gyWqYPiP6HjJO/TzzMKf7QtO928XzdORwQ1/jm2hDF1kaNPaU/XIHD0hHSvfICS4ELQcBPa5uPya6d5Kn5fVlpGqBcXBxKGbHB01PGjn0214eupt2LebWxkm7TP/BGDHKXXaoR748AlbyyG2L3TaNoGkwQiEoZe3CWgH4qkrf2CEpyfOa7BM7U0h+lceJxIGe+Yo6Ll14zyngDBa059E+ndFTpkQ6vECII7FdB1+A4Orhu40XHZgyVnlRudD1l80pIVnZves8N2zHZhQo8hp0sB5oY8HENiB09j7Xa6YQ/c1c2wm9E2xJHSNo7muFiHz48vQEgvNmu0lZdbVwXIy3IgLEBHjs6Kyqjtg618V4UaoP/LBOqtWtLd6TvGJ3qBlNyvhO5gZ2DGeLMN2WWoHxfl3VM+QGCGI2dnzUyHhfG1VBDYj8fzWXOuUVpVCMUw8fEg8GWmYLDwm3SbdMACWU87J0kAmaDPKcwwL6AwtWJZr/p89sSypl6y50JlOoDrbDbxrIbHQgbf1tcEgPOuLUN+O06qpi+OxKMtFdbvz+NQ4V4CpXpvMMQZj5AbuHp0EJ5Vme/Kcmpl/xoYGyFDzOEGR8lw6LMz+CpQ4GtnK5XXe1Xn3TJh6Tn2tVyS20p7artemGQQquCN8iQOw== mayag@gett.com", "key_options": null, "keyfile": "/home/mayagilad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mayagilad", "validate_certs": true} 2021-07-11T03:47:42.584 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/Gyl/Xhn6OJ6mdvGv0a7LUMeEU6xDfecjod/OUf8yBFFLm/h0AMifHH8qQYoH9SkeUthXIggbi0fcx9nrgyqYgLOansWiEIPbBbVCpvBz1vT1UR0Bxg6B7bjcrMgKg3M1YhTiHt41PlxdW/nlhTlxyZ1+nM50Fu/WE1+4nVIUaTtyaKv8PnmLmnpDnNly9X9CPDyilx8xpNiZMTv7oaiilCIfnmrKaZEDuvpU0M6h5wa6wJCatxTGIt/RQo2c4/YL+Nmx0xkouoRwR1H6swhkZPbInFgy/IlmLuu+owFR5mCVdxmst04o18CyvlW//Wiepw/5uJA3Xivcu1hwwcesTO+xJgIQbM/hvJ/UmL/eJvnV9G9g1B7ttrVfITbYgXZ1gbtP4pJCUmq132o6q+qeyOsGl2k5f/rRl5Adch5lzepffSSfsYn2tS77b7uXf3ujGvwhlOwJRITRCDZ4nGFwijFwBg8ptFKTaXZsXjMQRtSP2kuHoRonmOm1XbD4ElM= root@omri", "key_options": null, "keyfile": "/home/ozeneva/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ozeneva", "validate_certs": true} 2021-07-11T03:47:42.585 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-07-11T03:47:42.611 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:42 +0000 (0:00:00.474) 0:00:57.083 *********** 2021-07-11T03:47:42.617 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-07-11T03:47:42.619 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICYMwYvykHtnVhhJwcKaiRh9oMpJkqvEA7OAbHPYOv6W matan.brz@gmail.com", "key_options": null, "keyfile": "/home/matan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "matan", "validate_certs": true} 2021-07-11T03:47:42.646 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [gibba019.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-07-11T03:47:42.658 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.661 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.672 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDepyzA+MeQT5oRM52SxmUUKVAzkcrH/NqFxoCpBAjUSioTMaB1h0PhHE00nvHnKRvqWdPl8LCBWS3Ncjh30r9lwHaP+oPy4iLQEc7D2Mwn7WyPh5cq9N3UF25C0AlMG744k1tBNb97qy73OJI+oc5pAj4vUpCC1Ut9pMP5E5pQCx/kwgSOpC7mk0dYr/H5FxjPb0ca55eisFTLXraHPSChdrHjlYC5XpOZ9csilTwFdF0fJdu0nMDwIxGvwTOaUcIcfNLkpGB2Sj/pAM4L9pVsIbeE3C8d978MZ8Ff+ksQ0T8eNzew2ZjSkSuV+oR1sBCJYZ7mbDQuYLKAs5DanO1VFJvgjBJm1zP2ONu+g0fn/j04qIosuudziwoG/dOTkFo+6dBvF5hQi7TbZOIUVFr5dBlGS+O4eoK0gi/wpzun227E4dadY9uB1wiZ/PTD/5og+gxWG32RkHqqX3ALlK4m8JKzrq214nZ7Dt9CNewiUkR15uRkDasGIdiu6qb4Ez8= ArunaSudhan@sudhan-linux", "key_options": null, "keyfile": "/home/sudhanaruna/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sudhanaruna", "validate_certs": true} 2021-07-11T03:47:42.674 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.688 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.699 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.702 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.710 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.712 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.723 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** 2021-07-11T03:47:42.725 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/Gyl/Xhn6OJ6mdvGv0a7LUMeEU6xDfecjod/OUf8yBFFLm/h0AMifHH8qQYoH9SkeUthXIggbi0fcx9nrgyqYgLOansWiEIPbBbVCpvBz1vT1UR0Bxg6B7bjcrMgKg3M1YhTiHt41PlxdW/nlhTlxyZ1+nM50Fu/WE1+4nVIUaTtyaKv8PnmLmnpDnNly9X9CPDyilx8xpNiZMTv7oaiilCIfnmrKaZEDuvpU0M6h5wa6wJCatxTGIt/RQo2c4/YL+Nmx0xkouoRwR1H6swhkZPbInFgy/IlmLuu+owFR5mCVdxmst04o18CyvlW//Wiepw/5uJA3Xivcu1hwwcesTO+xJgIQbM/hvJ/UmL/eJvnV9G9g1B7ttrVfITbYgXZ1gbtP4pJCUmq132o6q+qeyOsGl2k5f/rRl5Adch5lzepffSSfsYn2tS77b7uXf3ujGvwhlOwJRITRCDZ4nGFwijFwBg8ptFKTaXZsXjMQRtSP2kuHoRonmOm1XbD4ElM= root@omri", "key_options": null, "keyfile": "/home/ozeneva/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ozeneva", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.736 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.738 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r 2021-07-11T03:47:42.739 INFO:teuthology.task.ansible.out:6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.748 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.750 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.752 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.768 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.788 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuHIpWAz6Sk3ATLvTG4KS24JUzgqQBEm4EesPgf1cEG+xK4gISUixaVC9OSNS/KR+8/yoWAhmw992p8cp7mWXxqrRC9kIrthLWvD6Jjz3HTXgarO2oNibeVu27kK/Fm2vvUNACARBkoC5YXd6IuoZOEg1bJu06DLw5Y7pGMqZRVDOrE8WUDwr27roZSgU2/N+18Y1LYn4gwSvWpuC/IotLyGSpW7/dgrZAIreRoYboMdTSlIjjuzeWvYx/AnZmkeEo0IhfyuBRl7ZQwxSFYHn9Z1+iQkt+O9hEZYSZuYRuyFPRqHNMKCEj1iYVRMcwbf/x2/dhjfmbWmjOMJoNNwm+DpixHHbId6Idx82GlAtCgQSTe9aG783oCeZYuVUr1plIRjeEsdjY9UBRYKNknEAChKjMSbxysvO3Dy4yrU2r8qo1iBS1fi4HUg12asCLrJX7vSCvOhOn7ltqDTmt5UiLSn+eCzXpOTBDWz7McB28FoYyR+Rwz5GOH3ITFhPyhXc= wzl@manjaro-wzl", "key_options": null, "keyfile": "/home/zl31wang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zl31wang", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc96 2021-07-11T03:47:42.789 INFO:teuthology.task.ansible.out:4184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.793 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.826 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.829 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.838 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.840 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.841 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.844 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) = 2021-07-11T03:47:42.845 INFO:teuthology.task.ansible.out:> {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.852 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-07-11T03:47:42.853 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDepyzA+MeQT5oRM52SxmUUKVAzkcrH/NqFxoCpBAjUSioTMaB1h0PhHE00nvHnKRvqWdPl8LCBWS3Ncjh30r9lwHaP+oPy4iLQEc7D2Mwn7WyPh5cq9N3UF25C0AlMG744k1tBNb97qy73OJI+oc5pAj4vUpCC1Ut9pMP5E5pQCx/kwgSOpC7mk0dYr/H5FxjPb0ca55eisFTLXraHPSChdrHjlYC5XpOZ9csilTwFdF0fJdu0nMDwIxGvwTOaUcIcfNLkpGB2Sj/pAM4L9pVsIbeE3C8d978MZ8Ff+ksQ0T8eNzew2ZjSkSuV+oR1sBCJYZ7mbDQuYLKAs5DanO1VFJvgjBJm1zP2ONu+g0fn/j04qIosuudziwoG/dOTkFo+6dBvF5hQi7TbZOIUVFr5dBlGS+O4eoK0gi/wpzun227E4dadY9uB1wiZ/PTD/5og+gxWG32RkHqqX3ALlK4m8JKzrq214nZ7Dt9CNewiUkR15uRkDasGIdiu6qb4Ez8= ArunaSudhan@sudhan-linux", "key_options": null, "keyfile": "/home/sudhanaruna/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sudhanaruna", "validate_certs": true} 2021-07-11T03:47:42.867 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-07-11T03:47:42.868 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.879 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.881 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.893 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.895 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.913 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-07-11T03:47:42.915 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** 2021-07-11T03:47:42.917 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIi8JMOVD91QR7EXBkhxeAsB2UO2gyWqYPiP6HjJO/TzzMKf7QtO928XzdORwQ1/jm2hDF1kaNPaU/XIHD0hHSvfICS4ELQcBPa5uPya6d5Kn5fVlpGqBcXBxKGbHB01PGjn0214eupt2LebWxkm7TP/BGDHKXXaoR748AlbyyG2L3TaNoGkwQiEoZe3CWgH4qkrf2CEpyfOa7BM7U0h+lceJxIGe+Yo6Ll14zyngDBa059E+ndFTpkQ6vECII7FdB1+A4Orhu40XHZgyVnlRudD1l80pIVnZves8N2zHZhQo8hp0sB5oY8HENiB09j7Xa6YQ/c1c2wm9E2xJHSNo7muFiHz48vQEgvNmu0lZdbVwXIy3IgLEBHjs6Kyqjtg618V4UaoP/LBOqtWtLd6TvGJ3qBlNyvhO5gZ2DGeLMN2WWoHxfl3VM+QGCGI2dnzUyHhfG1VBDYj8fzWXOuUVpVCMUw8fEg8GWmYLDwm3SbdMACWU87J0kAmaDPKcwwL6AwtWJZr/p89sSypl6y50JlOoDrbDbxrIbHQgbf1tcEgPOuLUN+O06qpi+OxKMtFdbvz+NQ4V4CpXpvMMQZj5AbuHp0EJ5Vme/Kcmpl/xoYGyFDzOEGR8lw6LMz+CpQ4GtnK5XXe1Xn3TJh6Tn2tVyS20p7artemGQQquCN8iQOw== mayag@gett.com", "key_options": null, "keyfile": "/home/mayagilad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mayagilad", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_ 2021-07-11T03:47:42.918 INFO:teuthology.task.ansible.out:loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.919 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.919 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52c 2021-07-11T03:47:42.921 INFO:teuthology.task.ansible.out:f542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.921 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:42.940 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:42.999 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-07-11T03:47:43.049 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-07-11T03:47:43.051 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.053 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.053 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuHIpWAz6Sk3ATLvTG4KS24JUzgqQBEm4EesPgf1cEG+xK4gISUixaVC9OSNS/KR+8/yoWAhmw992p8cp7mWXxqrRC9kIrthLWvD6Jjz3HTXgarO2oNibeVu27kK/Fm2vvUNACARBkoC5YXd6IuoZOEg1bJu06DLw5Y7pGMqZRVDOrE8WUDwr27roZSgU2/N+18Y1LYn4gwSvWpuC/IotLyGSpW7/dgrZAIreRoYboMdTSlIjjuzeWvYx/AnZmkeEo0IhfyuBRl7ZQwxSFYHn9Z1+iQkt+O9hEZYSZuYRuyFPRqHNMKCEj1iYVRMcwbf/x2/dhjfmbWmjOMJoNNwm+DpixHHbId6Idx82GlAtCgQSTe9aG783oCeZYuVUr1plIRjeEsdjY9UBRYKNknEAChKjMSbxysvO3Dy4yrU2r8qo1iBS1fi4HUg12asCLrJX7vSCvOhOn7ltqDTmt5UiLSn+eCzXpOTBDWz7McB28FoYyR+Rwz5GOH3ITFhPyhXc= wzl@manjaro-wzl", "key_options": null, "keyfile": "/home/zl31wang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zl31wang", "validate_certs": true} 2021-07-11T03:47:43.055 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.055 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.055 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.056 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansi 2021-07-11T03:47:43.057 INFO:teuthology.task.ansible.out:ble_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.058 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.058 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAA 2021-07-11T03:47:43.065 INFO:teuthology.task.ansible.out:AB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name' 2021-07-11T03:47:43.067 INFO:teuthology.task.ansible.out:: 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.105 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:43 +0000 (0:00:00.496) 0:00:57.580 *********** 2021-07-11T03:47:43.114 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.116 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.135 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** changed: [gibba005.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIi8JMOVD91QR7EXBkhxeAsB2UO2gyWqYPiP6HjJO/TzzMKf7QtO928XzdORwQ1/jm2hDF1kaNPaU/XIHD0hHSvfICS4ELQcBPa5uPya6d5Kn5fVlpGqBcXBxKGbHB01PGjn0214eupt2LebWxkm7TP/BGDHKXXaoR748AlbyyG2L3TaNoGkwQiEoZe3CWgH4qkrf2CEpyfOa7BM7U0h+lceJxIGe+Yo6Ll14zyngDBa059E+ndFTpkQ6vECII7FdB1+A4Orhu40XHZgyVnlRudD1l80pIVnZves8N2zHZhQo8hp0sB5oY8HENiB09j7Xa6YQ/c1c2wm9E2xJHSNo7muFiHz48vQEgvNmu0lZdbVwXIy3IgLEBHjs6Kyqjtg618V4UaoP/LBOqtWtLd6TvGJ3qBlNyvhO5gZ2DGeLMN2WWoHxfl3VM+QGCGI2dnzUyHhfG1VBDYj8fzWXOuUVpVCMUw8fEg8GWmYLDwm3SbdMACWU87J0kAmaDPKcwwL6AwtWJZr/p89sSypl6y50JlOoDrbDbxrIbHQgbf1tcEgPOuLUN+O06qpi+OxKMtFdbvz+NQ4V4CpXpvMMQZj5AbuHp0EJ5Vme/Kcmpl/xoYGyFDzOEGR8lw6LMz+CpQ4GtnK5XXe1Xn3TJh6Tn2tVyS20p7artemGQQquCN8iQOw== mayag@gett.com", "key_options": null, "keyfile": "/home/mayagilad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mayagilad", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba019.f 2021-07-11T03:47:43.138 INFO:teuthology.task.ansible.out:ront.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.171 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-07-11T03:47:43.276 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-07-11T03:47:43.278 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} ok: [gibba023.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tse 2021-07-11T03:47:43.279 INFO:teuthology.task.ansible.out:rlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was Fal 2021-07-11T03:47:43.281 INFO:teuthology.task.ansible.out:se"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.283 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.285 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.286 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.286 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.288 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.288 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.289 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9 2021-07-11T03:47:43.291 INFO:teuthology.task.ansible.out:tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N 2021-07-11T03:47:43.311 INFO:teuthology.task.ansible.out:/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi 2021-07-11T03:47:43.313 INFO:teuthology.task.ansible.out:8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072 2021-07-11T03:47:43.319 INFO:teuthology.task.ansible.out:acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c6 2021-07-11T03:47:43.321 INFO:teuthology.task.ansible.out:17f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7e 2021-07-11T03:47:43.323 INFO:teuthology.task.ansible.out:ed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.327 INFO:teuthology.task.ansible.out: skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.328 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.328 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] 2021-07-11T03:47:43.329 INFO:teuthology.task.ansible.out:=> (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.356 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:43 +0000 (0:00:00.250) 0:00:57.831 *********** 2021-07-11T03:47:43.368 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.369 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.426 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.438 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-07-11T03:47:43.440 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.441 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.446 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.451 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.453 INFO:teuthology.task.ansible.out: skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.459 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.461 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsK 2021-07-11T03:47:43.463 INFO:teuthology.task.ansible.out:mPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.463 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.464 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'} 2021-07-11T03:47:43.469 INFO:teuthology.task.ansible.out:) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.se 2021-07-11T03:47:43.471 INFO:teuthology.task.ansible.out:pia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXs 2021-07-11T03:47:43.479 INFO:teuthology.task.ansible.out:WP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db6 2021-07-11T03:47:43.481 INFO:teuthology.task.ansible.out:9f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.484 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.485 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.487 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.489 INFO:teuthology.task.ansible.out: skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.489 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.501 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.512 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.514 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.516 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.527 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.539 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.550 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.554 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.557 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401 2021-07-11T03:47:43.558 INFO:teuthology.task.ansible.out:efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng 2021-07-11T03:47:43.560 INFO:teuthology.task.ansible.out:.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.571 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.572 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.572 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.574 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.585 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.586 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.587 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.587 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag" 2021-07-11T03:47:43.589 INFO:teuthology.task.ansible.out:, "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.600 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-07-11T03:47:43.601 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.603 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.615 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.618 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.619 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.619 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.626 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.628 INFO:teuthology.task.ansible.out: skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.638 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.640 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.651 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.653 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.654 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.665 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.667 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.682 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh6 2021-07-11T03:47:43.684 INFO:teuthology.task.ansible.out:5r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.699 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.701 INFO:teuthology.task.ansible.out: skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.702 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-07-11T03:47:43.710 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.712 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.712 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.712 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.726 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.727 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.738 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.740 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.748 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.761 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: 2021-07-11T03:47:43.763 INFO:teuthology.task.ansible.out:[gibba019.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "cha 2021-07-11T03:47:43.764 INFO:teuthology.task.ansible.out:nged": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad 2021-07-11T03:47:43.766 INFO:teuthology.task.ansible.out: tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.787 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.791 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.804 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.806 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.806 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.807 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.808 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.811 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.813 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.813 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {" 2021-07-11T03:47:43.815 INFO:teuthology.task.ansible.out:name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.825 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.830 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "skip_reason": "Conditional result was False"} ok: [gibba005.front.sepia.ceph 2021-07-11T03:47:43.832 INFO:teuthology.task.ansible.out:.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed 2021-07-11T03:47:43.834 INFO:teuthology.task.ansible.out:": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.844 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.846 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.856 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.867 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.868 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.879 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.891 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.910 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.912 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.923 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.925 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.935 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.946 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.962 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.975 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.977 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.978 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.979 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.984 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:43.984 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.990 INFO:teuthology.task.ansible.out: skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.996 INFO:teuthology.task.ansible.out: skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.997 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:43.998 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:44.006 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.007 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.015 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.026 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.029 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.031 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.049 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "skip_reason": "Conditional result was False"} ok: [gibba005.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i7 2021-07-11T03:47:44.051 INFO:teuthology.task.ansible.out:4JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.076 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.087 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:44.090 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.091 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:44.092 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.094 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:44.096 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.098 INFO:teuthology.task.ansible.out: skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.110 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.123 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.126 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:44.129 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.130 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.130 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.132 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.132 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.133 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q 2021-07-11T03:47:44.134 INFO:teuthology.task.ansible.out: bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.136 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:44.152 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.159 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45 2021-07-11T03:47:44.161 INFO:teuthology.task.ansible.out:\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.173 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.175 INFO:teuthology.task.ansible.out: skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.176 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:44.180 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.189 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.190 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:44.192 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.211 INFO:teuthology.task.ansible.out: skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.276 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.277 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.278 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e2 2021-07-11T03:47:44.279 INFO:teuthology.task.ansible.out:17\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.285 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.286 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.286 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.288 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.288 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.289 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.289 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'scarval 2021-07-11T03:47:44.295 INFO:teuthology.task.ansible.out:hojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983 2021-07-11T03:47:44.297 INFO:teuthology.task.ansible.out:217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "it 2021-07-11T03:47:44.299 INFO:teuthology.task.ansible.out:em": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.300 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.303 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.304 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.314 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.361 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:44 +0000 (0:00:01.004) 0:00:58.836 *********** 2021-07-11T03:47:44.376 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.378 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.380 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.381 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.385 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora Ofp 2021-07-11T03:47:44.386 INFO:teuthology.task.ansible.out:UShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450c 2021-07-11T03:47:44.388 INFO:teuthology.task.ansible.out:e2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.401 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.424 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.426 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.427 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.427 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.431 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.443 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.445 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.460 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.462 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.462 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.480 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.483 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.542 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:44 +0000 (0:00:00.180) 0:00:59.016 *********** 2021-07-11T03:47:44.558 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.559 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.560 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.560 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.562 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.562 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.563 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.564 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.570 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.618 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", 2021-07-11T03:47:44.620 INFO:teuthology.task.ansible.out:"ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc 2021-07-11T03:47:44.622 INFO:teuthology.task.ansible.out:4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.640 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.646 INFO:teuthology.task.ansible.out: skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) 2021-07-11T03:47:44.648 INFO:teuthology.task.ansible.out:=> {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d 2021-07-11T03:47:44.650 INFO:teuthology.task.ansible.out:3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.684 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.686 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.704 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.725 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.727 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.786 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.789 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", 2021-07-11T03:47:44.791 INFO:teuthology.task.ansible.out:"ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.791 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.793 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.793 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.793 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:44.794 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.795 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.796 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.796 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.798 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.809 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.871 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.877 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "chan 2021-07-11T03:47:44.879 INFO:teuthology.task.ansible.out:ged": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'pnawrac 2021-07-11T03:47:44.880 INFO:teuthology.task.ansible.out:ay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.911 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t4 2021-07-11T03:47:44.914 INFO:teuthology.task.ansible.out:80 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.924 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.926 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:44.926 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.927 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.927 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:44.929 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.930 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.931 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-07-11T03:47:44.931 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-07-11T03:47:44.931 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-07-11T03:47:44.932 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.932 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.935 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.936 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.951 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.952 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.964 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.966 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:44.966 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:45.000 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.002 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", 2021-07-11T03:47:45.004 INFO:teuthology.task.ansible.out: "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.006 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.007 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.060 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:45 +0000 (0:00:00.518) 0:00:59.535 *********** 2021-07-11T03:47:45.077 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.079 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.098 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.098 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-07-11T03:47:45.099 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-07-11T03:47:45.100 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.101 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.101 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_ 2021-07-11T03:47:45.103 INFO:teuthology.task.ansible.out:reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.122 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.123 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.135 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.136 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.158 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.171 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.173 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.183 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-07-11T03:47:45.184 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.185 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.196 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.198 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} TASK [Filter the revoked_users list] ******************************************* 2021-07-11T03:47:45.198 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.240 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:45 +0000 (0:00:00.178) 0:00:59.714 *********** 2021-07-11T03:47:45.252 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:45.253 INFO:teuthology.task.ansible.out:TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.256 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.258 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.258 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.260 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.262 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.286 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.292 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:45.293 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.301 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:45.303 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.304 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.305 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.305 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 Vkj 2021-07-11T03:47:45.307 INFO:teuthology.task.ansible.out:YUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} TASK [Remove revoked users] **************************************************** ok: [gibba019.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-07-11T03:47:45.330 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.332 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.342 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.355 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.387 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.390 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.401 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.404 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat 2021-07-11T03:47:45.405 INFO:teuthology.task.ansible.out:@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.406 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.406 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjau 2021-07-11T03:47:45.407 INFO:teuthology.task.ansible.out:mOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.444 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.446 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.465 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba019.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.478 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.482 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.484 INFO:teuthology.task.ansible.out: skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.485 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.486 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.501 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.553 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.555 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} TASK [Remove revoked users] **************************************************** ok: [gibba023.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-07-11T03:47:45.556 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-07-11T03:47:45.562 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.607 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:45 +0000 (0:00:00.365) 0:01:00.079 *********** 2021-07-11T03:47:45.615 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba019.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-07-11T03:47:45.628 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:45.635 INFO:teuthology.task.ansible.out:TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.639 INFO:teuthology.task.ansible.out: skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16 2021-07-11T03:47:45.651 INFO:teuthology.task.ansible.out:d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "i 2021-07-11T03:47:45.658 INFO:teuthology.task.ansible.out:tem", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.669 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba023.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.671 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.675 INFO:teuthology.task.ansible.out: skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.710 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.712 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.716 INFO:teuthology.task.ansible.out:TASK [Remove revoked users] **************************************************** 2021-07-11T03:47:45.717 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-07-11T03:47:45.722 INFO:teuthology.task.ansible.out:TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-07-11T03:47:45.728 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "matan", "ovpn": "matanb@main 5ZJUAUM7LCpjmIkH+Ntl6g 27de66609fee4975af229f411a1f49f589a4c64b958d8913d3f5eff128a37afa"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.744 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* 2021-07-11T03:47:45.745 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.807 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:45 +0000 (0:00:00.194) 0:01:00.274 *********** 2021-07-11T03:47:45.809 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-07-11T03:47:45.809 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.811 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:45.811 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.813 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:45.874 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:45.878 INFO:teuthology.task.ansible.out:TASK [Remove revoked users] **************************************************** 2021-07-11T03:47:45.880 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-07-11T03:47:45.883 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-07-11T03:47:45.933 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:45 +0000 (0:00:00.133) 0:01:00.408 *********** 2021-07-11T03:47:45.959 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-07-11T03:47:46.018 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-07-11T03:47:46.041 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* 2021-07-11T03:47:46.042 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:46.071 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:46 +0000 (0:00:00.137) 0:01:00.545 *********** 2021-07-11T03:47:46.099 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-07-11T03:47:46.105 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-07-11T03:47:46.116 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-07-11T03:47:46.132 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-07-11T03:47:46.197 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-07-11T03:47:46.198 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-07-11T03:47:46.284 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-07-11T03:47:46.328 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-07-11T03:47:46.328 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-07-11T03:47:46.330 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:46.331 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-07-11T03:47:46.342 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:46.392 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-07-11T03:47:46.444 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-07-11T03:47:46.472 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-07-11T03:47:46.487 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-07-11T03:47:46.513 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-07-11T03:47:46.579 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} ok: [gibba023.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-07-11T03:47:46.635 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-07-11T03:47:46.652 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-07-11T03:47:46.698 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-07-11T03:47:46.734 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-07-11T03:47:46.778 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-07-11T03:47:46.802 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-07-11T03:47:46.815 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-07-11T03:47:46.817 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:46.831 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-07-11T03:47:46.867 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-07-11T03:47:46.868 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:46.904 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-07-11T03:47:46.926 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-07-11T03:47:46.950 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-07-11T03:47:46.986 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-07-11T03:47:46.999 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-07-11T03:47:47.057 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-07-11T03:47:47.067 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-07-11T03:47:47.101 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-07-11T03:47:47.103 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-07-11T03:47:47.161 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-07-11T03:47:47.204 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-07-11T03:47:47.217 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-07-11T03:47:47.225 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-07-11T03:47:47.260 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-07-11T03:47:47.343 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-07-11T03:47:47.345 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-07-11T03:47:47.375 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-07-11T03:47:47.382 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:47.404 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-07-11T03:47:47.497 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-07-11T03:47:47.508 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-07-11T03:47:47.509 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:47.523 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-07-11T03:47:47.544 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:47.561 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-07-11T03:47:47.620 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-07-11T03:47:47.691 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-07-11T03:47:47.691 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-07-11T03:47:47.738 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-07-11T03:47:47.760 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-07-11T03:47:47.788 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-07-11T03:47:47.812 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-07-11T03:47:47.857 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-07-11T03:47:47.913 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-07-11T03:47:47.927 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-07-11T03:47:47.935 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-07-11T03:47:48.011 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-07-11T03:47:48.027 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-07-11T03:47:48.050 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-07-11T03:47:48.119 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-07-11T03:47:48.131 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-07-11T03:47:48.153 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-07-11T03:47:48.180 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-07-11T03:47:48.218 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-07-11T03:47:48.239 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-07-11T03:47:48.280 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-07-11T03:47:48.281 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-07-11T03:47:48.315 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-07-11T03:47:48.373 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-07-11T03:47:48.375 INFO:teuthology.task.ansible.out: ok: [gibba005.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-07-11T03:47:48.396 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-07-11T03:47:48.421 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-07-11T03:47:48.466 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-07-11T03:47:48.468 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-07-11T03:47:48.511 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-07-11T03:47:48.513 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-07-11T03:47:48.567 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-07-11T03:47:48.612 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-07-11T03:47:48.614 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-07-11T03:47:48.646 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-07-11T03:47:48.656 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-07-11T03:47:48.713 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-07-11T03:47:48.746 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-07-11T03:47:48.757 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-07-11T03:47:48.780 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-07-11T03:47:48.823 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-07-11T03:47:48.838 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-07-11T03:47:48.857 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-07-11T03:47:48.869 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-07-11T03:47:48.919 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-07-11T03:47:48.931 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-07-11T03:47:48.965 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-07-11T03:47:49.026 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-07-11T03:47:49.050 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-07-11T03:47:49.053 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-07-11T03:47:49.065 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-07-11T03:47:49.113 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-07-11T03:47:49.148 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-07-11T03:47:49.149 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-07-11T03:47:49.181 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-07-11T03:47:49.258 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} ok: [gibba004.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} ok: [gibba019.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-07-11T03:47:49.290 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-07-11T03:47:49.292 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:49.352 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-07-11T03:47:49.355 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-07-11T03:47:49.358 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-07-11T03:47:49.392 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-07-11T03:47:49.393 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:49.446 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-07-11T03:47:49.460 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-07-11T03:47:49.472 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-07-11T03:47:49.485 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-07-11T03:47:49.562 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-07-11T03:47:49.576 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-07-11T03:47:49.600 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-07-11T03:47:49.602 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:49.604 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-07-11T03:47:49.666 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-07-11T03:47:49.682 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-07-11T03:47:49.706 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-07-11T03:47:49.783 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-07-11T03:47:49.785 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:49.787 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-07-11T03:47:49.788 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:49.810 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-07-11T03:47:49.875 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-07-11T03:47:49.876 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-07-11T03:47:49.920 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-07-11T03:47:49.982 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-07-11T03:47:50.010 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jdillaman", "name": "jdillaman", "remove": false, "state": "absent"} 2021-07-11T03:47:50.022 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-07-11T03:47:50.070 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-07-11T03:47:50.093 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-07-11T03:47:50.108 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-07-11T03:47:50.178 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-07-11T03:47:50.201 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-07-11T03:47:50.202 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-07-11T03:47:50.278 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-07-11T03:47:50.299 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-07-11T03:47:50.301 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-07-11T03:47:50.365 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-07-11T03:47:50.388 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-07-11T03:47:50.410 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jdillaman", "name": "jdillaman", "remove": false, "state": "absent"} 2021-07-11T03:47:50.422 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-07-11T03:47:50.484 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-07-11T03:47:50.533 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-07-11T03:47:50.570 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-07-11T03:47:50.605 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-07-11T03:47:50.685 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-07-11T03:47:50.728 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-07-11T03:47:50.782 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-07-11T03:47:50.815 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-07-11T03:47:50.863 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jdillaman", "name": "jdillaman", "remove": false, "state": "absent"} 2021-07-11T03:47:50.885 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-07-11T03:47:50.930 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-07-11T03:47:50.954 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-07-11T03:47:50.987 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-07-11T03:47:51.020 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-07-11T03:47:51.054 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-07-11T03:47:51.092 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-07-11T03:47:51.095 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:51.113 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jdillaman", "name": "jdillaman", "remove": false, "state": "absent"} 2021-07-11T03:47:51.180 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-07-11T03:47:51.217 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-07-11T03:47:51.218 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-07-11T03:47:51.220 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-07-11T03:47:51.246 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:51 +0000 (0:00:05.175) 0:01:05.721 *********** 2021-07-11T03:47:51.317 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-07-11T03:47:51.328 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-07-11T03:47:51.365 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-07-11T03:47:51.408 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-07-11T03:47:51.420 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-07-11T03:47:51.477 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-07-11T03:47:51.533 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-07-11T03:47:51.546 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-07-11T03:47:51.548 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:51.594 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-07-11T03:47:51.622 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:51 +0000 (0:00:00.376) 0:01:06.097 *********** 2021-07-11T03:47:51.644 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-07-11T03:47:51.668 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-07-11T03:47:51.670 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "checksum": "946da8f37b852cb4beed25179833ebd51803d462", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "8cbbb9e68d3df7c0e0513064adc74bb8", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975271.344158-23075-136469354352101/source", "state": "file", "uid": 0} 2021-07-11T03:47:51.703 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:51 +0000 (0:00:00.077) 0:01:06.175 *********** 2021-07-11T03:47:51.710 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-07-11T03:47:51.712 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-07-11T03:47:51.734 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-07-11T03:47:51.778 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [gibba019.front.sepia.ceph.com] => { "msg": "Host gibba019.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } 2021-07-11T03:47:51.808 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:51 +0000 (0:00:00.107) 0:01:06.282 *********** 2021-07-11T03:47:51.815 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [gibba004.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-07-11T03:47:51.842 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-07-11T03:47:51.905 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [gibba019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-07-11T03:47:51.936 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:51 +0000 (0:00:00.127) 0:01:06.410 *********** 2021-07-11T03:47:51.942 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-07-11T03:47:51.944 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-07-11T03:47:51.962 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-07-11T03:47:52.006 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:52.033 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.098) 0:01:06.508 *********** 2021-07-11T03:47:52.041 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-07-11T03:47:52.043 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "checksum": "946da8f37b852cb4beed25179833ebd51803d462", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "8cbbb9e68d3df7c0e0513064adc74bb8", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975271.7175775-23142-229609099200368/source", "state": "file", "uid": 0} 2021-07-11T03:47:52.051 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-07-11T03:47:52.066 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-07-11T03:47:52.091 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.057) 0:01:06.566 *********** 2021-07-11T03:47:52.118 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.025) 0:01:06.592 *********** 2021-07-11T03:47:52.125 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-07-11T03:47:52.126 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:52.153 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.035) 0:01:06.627 *********** 2021-07-11T03:47:52.181 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-07-11T03:47:52.183 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-07-11T03:47:52.211 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [gibba023.front.sepia.ceph.com] => { "msg": "Host gibba023.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } 2021-07-11T03:47:52.235 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.081) 0:01:06.709 *********** 2021-07-11T03:47:52.259 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.024) 0:01:06.733 *********** 2021-07-11T03:47:52.267 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-07-11T03:47:52.270 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:52.296 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.036) 0:01:06.770 *********** 2021-07-11T03:47:52.345 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-07-11T03:47:52.349 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-07-11T03:47:52.375 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.078) 0:01:06.849 *********** 2021-07-11T03:47:52.387 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:52.413 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.037) 0:01:06.887 *********** 2021-07-11T03:47:52.494 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-07-11T03:47:52.496 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:52.520 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.106) 0:01:06.994 *********** 2021-07-11T03:47:52.527 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:52.531 INFO:teuthology.task.ansible.out:TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:52.564 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.044) 0:01:07.039 *********** 2021-07-11T03:47:52.566 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:52.577 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:52.579 INFO:teuthology.task.ansible.out:TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-07-11T03:47:52.579 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "checksum": "946da8f37b852cb4beed25179833ebd51803d462", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "8cbbb9e68d3df7c0e0513064adc74bb8", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975272.1932445-23248-251416683634197/source", "state": "file", "uid": 0} 2021-07-11T03:47:52.621 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.057) 0:01:07.096 *********** 2021-07-11T03:47:52.624 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:52.632 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-07-11T03:47:52.635 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:52.681 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.059) 0:01:07.155 *********** 2021-07-11T03:47:52.691 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-07-11T03:47:52.693 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "checksum": "946da8f37b852cb4beed25179833ebd51803d462", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "8cbbb9e68d3df7c0e0513064adc74bb8", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975272.3338363-23276-225789027630964/source", "state": "file", "uid": 0} 2021-07-11T03:47:52.736 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.054) 0:01:07.210 *********** 2021-07-11T03:47:52.757 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.022) 0:01:07.232 *********** 2021-07-11T03:47:52.766 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-07-11T03:47:52.772 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => { "msg": "Host gibba004.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } 2021-07-11T03:47:52.794 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.036) 0:01:07.269 *********** 2021-07-11T03:47:52.796 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:52.807 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:52.809 INFO:teuthology.task.ansible.out:TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-07-11T03:47:52.810 INFO:teuthology.task.ansible.out: skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:52.814 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:52.840 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.045) 0:01:07.314 *********** 2021-07-11T03:47:52.847 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-07-11T03:47:52.849 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => { "msg": "Host gibba005.front.sepia.ceph.com is running Ubuntu 20.04 (focal)" } 2021-07-11T03:47:52.875 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.035) 0:01:07.349 *********** 2021-07-11T03:47:52.883 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-07-11T03:47:52.885 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:52.929 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.052) 0:01:07.402 *********** 2021-07-11T03:47:52.953 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-07-11T03:47:52.955 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.001 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:52 +0000 (0:00:00.073) 0:01:07.475 *********** 2021-07-11T03:47:53.023 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.022) 0:01:07.498 *********** 2021-07-11T03:47:53.031 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2021-07-11T03:47:53.033 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-07-11T03:47:53.060 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.036) 0:01:07.535 *********** 2021-07-11T03:47:53.076 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-07-11T03:47:53.078 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.104 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.044) 0:01:07.579 *********** 2021-07-11T03:47:53.111 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-07-11T03:47:53.113 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.144 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.037) 0:01:07.616 *********** 2021-07-11T03:47:53.166 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.024) 0:01:07.641 *********** 2021-07-11T03:47:53.173 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-07-11T03:47:53.175 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.209 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.042) 0:01:07.683 *********** 2021-07-11T03:47:53.217 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-07-11T03:47:53.219 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:53.247 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.038) 0:01:07.722 *********** 2021-07-11T03:47:53.256 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-07-11T03:47:53.257 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.287 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.037) 0:01:07.760 *********** 2021-07-11T03:47:53.309 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.023) 0:01:07.784 *********** 2021-07-11T03:47:53.316 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-07-11T03:47:53.318 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.347 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.038) 0:01:07.822 *********** 2021-07-11T03:47:53.356 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-07-11T03:47:53.358 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.383 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.036) 0:01:07.858 *********** 2021-07-11T03:47:53.391 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-07-11T03:47:53.391 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-07-11T03:47:53.392 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.420 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.036) 0:01:07.895 *********** 2021-07-11T03:47:53.444 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.023) 0:01:07.919 *********** 2021-07-11T03:47:53.452 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-07-11T03:47:53.454 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.486 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.036) 0:01:07.956 *********** 2021-07-11T03:47:53.490 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:53.492 INFO:teuthology.task.ansible.out:TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.526 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.045) 0:01:08.001 *********** 2021-07-11T03:47:53.536 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:53.537 INFO:teuthology.task.ansible.out:TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set entitlements_path] ****************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.571 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.044) 0:01:08.045 *********** 2021-07-11T03:47:53.602 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.025) 0:01:08.071 *********** 2021-07-11T03:47:53.608 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.645 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.048) 0:01:08.120 *********** 2021-07-11T03:47:53.671 INFO:teuthology.task.ansible.out: TASK [common : Set the releasever] ********************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-07-11T03:47:53.673 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.698 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.052) 0:01:08.172 *********** 2021-07-11T03:47:53.725 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.027) 0:01:08.200 *********** 2021-07-11T03:47:53.754 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [gibba023.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba023.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.781 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.055) 0:01:08.256 *********** 2021-07-11T03:47:53.817 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.035) 0:01:08.291 *********** 2021-07-11T03:47:53.827 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.857 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.040) 0:01:08.332 *********** 2021-07-11T03:47:53.865 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-07-11T03:47:53.867 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:53.898 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.041) 0:01:08.373 *********** 2021-07-11T03:47:53.906 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-07-11T03:47:53.908 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.935 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.035) 0:01:08.408 *********** 2021-07-11T03:47:53.949 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-07-11T03:47:53.950 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:53.976 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:53 +0000 (0:00:00.042) 0:01:08.451 *********** 2021-07-11T03:47:54.010 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.033) 0:01:08.484 *********** 2021-07-11T03:47:54.030 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-07-11T03:47:54.032 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.066 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.056) 0:01:08.540 *********** 2021-07-11T03:47:54.075 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-07-11T03:47:54.077 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set entitlements_path] ****************************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.105 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.038) 0:01:08.579 *********** 2021-07-11T03:47:54.138 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.033) 0:01:08.613 *********** 2021-07-11T03:47:54.150 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.176 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.038) 0:01:08.651 *********** 2021-07-11T03:47:54.186 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-07-11T03:47:54.187 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:54.216 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.037) 0:01:08.689 *********** 2021-07-11T03:47:54.226 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [gibba004.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba004.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:54.258 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.043) 0:01:08.732 *********** 2021-07-11T03:47:54.265 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-07-11T03:47:54.266 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the releasever] ********************************************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.294 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.036) 0:01:08.769 *********** 2021-07-11T03:47:54.318 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.023) 0:01:08.793 *********** 2021-07-11T03:47:54.326 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.362 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.043) 0:01:08.836 *********** 2021-07-11T03:47:54.369 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-07-11T03:47:54.370 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.425 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.063) 0:01:08.900 *********** 2021-07-11T03:47:54.435 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [gibba005.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [gibba005.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:54.477 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.051) 0:01:08.952 *********** 2021-07-11T03:47:54.486 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-07-11T03:47:54.487 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:54.525 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.047) 0:01:08.999 *********** 2021-07-11T03:47:54.545 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.574 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.049) 0:01:09.049 *********** 2021-07-11T03:47:54.593 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.631 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.056) 0:01:09.105 *********** 2021-07-11T03:47:54.641 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-07-11T03:47:54.643 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.679 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.047) 0:01:09.153 *********** 2021-07-11T03:47:54.688 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-07-11T03:47:54.689 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set the releasever] ********************************************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.722 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.043) 0:01:09.196 *********** 2021-07-11T03:47:54.750 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.027) 0:01:09.224 *********** 2021-07-11T03:47:54.759 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-07-11T03:47:54.761 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.797 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.047) 0:01:09.272 *********** 2021-07-11T03:47:54.806 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-07-11T03:47:54.811 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:54.842 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.044) 0:01:09.317 *********** 2021-07-11T03:47:54.850 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-07-11T03:47:54.852 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.880 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.037) 0:01:09.354 *********** 2021-07-11T03:47:54.903 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.023) 0:01:09.378 *********** 2021-07-11T03:47:54.911 INFO:teuthology.task.ansible.out: TASK [common : Set the releasever] ********************************************* 2021-07-11T03:47:54.912 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.913 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:54.939 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.035) 0:01:09.413 *********** 2021-07-11T03:47:54.962 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.023) 0:01:09.437 *********** 2021-07-11T03:47:54.973 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:55.000 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:54 +0000 (0:00:00.037) 0:01:09.474 *********** 2021-07-11T03:47:55.174 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-07-11T03:47:55.175 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:55.202 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.202) 0:01:09.676 *********** 2021-07-11T03:47:55.213 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-07-11T03:47:55.218 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:55.241 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.039) 0:01:09.716 *********** 2021-07-11T03:47:55.251 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:55.252 INFO:teuthology.task.ansible.out:TASK [common : List CDN releases available to system] ************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:55.286 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.045) 0:01:09.761 *********** 2021-07-11T03:47:55.295 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-07-11T03:47:55.297 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:55.461 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.174) 0:01:09.936 *********** 2021-07-11T03:47:55.482 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-07-11T03:47:55.484 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:55.513 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.051) 0:01:09.987 *********** 2021-07-11T03:47:55.521 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-07-11T03:47:55.523 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:55.550 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.037) 0:01:10.025 *********** 2021-07-11T03:47:55.552 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:55.577 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.026) 0:01:10.052 *********** 2021-07-11T03:47:55.579 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:55.584 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-07-11T03:47:55.586 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:55.611 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.034) 0:01:10.086 *********** 2021-07-11T03:47:55.635 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.023) 0:01:10.110 *********** 2021-07-11T03:47:55.637 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:55.642 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-07-11T03:47:55.644 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:55.675 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.040) 0:01:10.150 *********** 2021-07-11T03:47:55.683 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-07-11T03:47:55.685 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:55.711 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.035) 0:01:10.185 *********** 2021-07-11T03:47:55.718 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-07-11T03:47:55.719 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:55.746 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.035) 0:01:10.220 *********** 2021-07-11T03:47:55.768 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.022) 0:01:10.243 *********** 2021-07-11T03:47:55.780 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:55.802 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.033) 0:01:10.276 *********** 2021-07-11T03:47:55.809 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-07-11T03:47:55.811 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:55.836 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.034) 0:01:10.311 *********** 2021-07-11T03:47:55.844 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-07-11T03:47:55.846 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:55.885 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.049) 0:01:10.360 *********** 2021-07-11T03:47:55.920 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.034) 0:01:10.394 *********** 2021-07-11T03:47:55.934 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:55.962 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.042) 0:01:10.437 *********** 2021-07-11T03:47:55.988 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:55 +0000 (0:00:00.025) 0:01:10.463 *********** 2021-07-11T03:47:55.996 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-07-11T03:47:55.997 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-07-11T03:47:55.998 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.032 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.043) 0:01:10.506 *********** 2021-07-11T03:47:56.057 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.025) 0:01:10.531 *********** 2021-07-11T03:47:56.065 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-07-11T03:47:56.067 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.096 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.039) 0:01:10.571 *********** 2021-07-11T03:47:56.130 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.033) 0:01:10.604 *********** 2021-07-11T03:47:56.136 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-07-11T03:47:56.138 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:56.139 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.166 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.036) 0:01:10.641 *********** 2021-07-11T03:47:56.190 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.024) 0:01:10.665 *********** 2021-07-11T03:47:56.200 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-07-11T03:47:56.201 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.228 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.038) 0:01:10.703 *********** 2021-07-11T03:47:56.237 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-07-11T03:47:56.239 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.288 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.043) 0:01:10.746 *********** TASK [common : Clean yum cache] ************************************************ skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.322 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.050) 0:01:10.797 *********** 2021-07-11T03:47:56.348 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.026) 0:01:10.823 *********** 2021-07-11T03:47:56.359 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-07-11T03:47:56.361 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.387 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.038) 0:01:10.862 *********** 2021-07-11T03:47:56.412 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.024) 0:01:10.887 *********** 2021-07-11T03:47:56.420 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-07-11T03:47:56.422 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Disable all rhsm repos] ***************************************** skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:56.448 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.035) 0:01:10.923 *********** 2021-07-11T03:47:56.473 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.024) 0:01:10.947 *********** 2021-07-11T03:47:56.482 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-07-11T03:47:56.484 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.512 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.039) 0:01:10.987 *********** 2021-07-11T03:47:56.522 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-07-11T03:47:56.523 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.549 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.036) 0:01:11.024 *********** 2021-07-11T03:47:56.572 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.022) 0:01:11.046 *********** 2021-07-11T03:47:56.579 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-07-11T03:47:56.580 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.606 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.034) 0:01:11.081 *********** 2021-07-11T03:47:56.615 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.641 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.035) 0:01:11.116 *********** 2021-07-11T03:47:56.651 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-07-11T03:47:56.653 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.681 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.037) 0:01:11.154 *********** 2021-07-11T03:47:56.712 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.032) 0:01:11.186 *********** 2021-07-11T03:47:56.720 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-07-11T03:47:56.722 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Store list of enabled repos] ************************************ skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.747 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.035) 0:01:11.222 *********** 2021-07-11T03:47:56.771 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.023) 0:01:11.246 *********** 2021-07-11T03:47:56.780 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-07-11T03:47:56.781 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.806 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.035) 0:01:11.281 *********** 2021-07-11T03:47:56.814 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-07-11T03:47:56.816 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:56.845 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.038) 0:01:11.320 *********** 2021-07-11T03:47:56.853 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-07-11T03:47:56.855 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Increase the yum timeout.] ************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.887 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.041) 0:01:11.361 *********** 2021-07-11T03:47:56.917 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.029) 0:01:11.391 *********** 2021-07-11T03:47:56.927 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-07-11T03:47:56.928 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Enable necessary rhsm repos] ************************************ skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:56.959 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.042) 0:01:11.433 *********** 2021-07-11T03:47:56.981 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:56 +0000 (0:00:00.021) 0:01:11.455 *********** 2021-07-11T03:47:56.988 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-07-11T03:47:56.990 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.024 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.042) 0:01:11.498 *********** 2021-07-11T03:47:57.031 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-07-11T03:47:57.033 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.034 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.037 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:57.039 INFO:teuthology.task.ansible.out:TASK [common : Disable all rhsm repos] ***************************************** skipping: [gibba004.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:47:57.064 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.040) 0:01:11.539 *********** 2021-07-11T03:47:57.087 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.022) 0:01:11.561 *********** 2021-07-11T03:47:57.094 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-07-11T03:47:57.096 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.131 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.043) 0:01:11.605 *********** 2021-07-11T03:47:57.162 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.031) 0:01:11.636 *********** 2021-07-11T03:47:57.173 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-07-11T03:47:57.175 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.177 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* skipping: [gibba019.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.178 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.219 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.057) 0:01:11.693 *********** 2021-07-11T03:47:57.255 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.035) 0:01:11.729 *********** 2021-07-11T03:47:57.264 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-07-11T03:47:57.265 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Increase the yum timeout.] ************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.295 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.039) 0:01:11.769 *********** 2021-07-11T03:47:57.320 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.025) 0:01:11.794 *********** 2021-07-11T03:47:57.339 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-07-11T03:47:57.340 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.341 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.368 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.048) 0:01:11.843 *********** 2021-07-11T03:47:57.393 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.024) 0:01:11.867 *********** 2021-07-11T03:47:57.405 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-07-11T03:47:57.407 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.432 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.039) 0:01:11.907 *********** 2021-07-11T03:47:57.440 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-07-11T03:47:57.442 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.470 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.037) 0:01:11.944 *********** 2021-07-11T03:47:57.477 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-07-11T03:47:57.479 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.507 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.037) 0:01:11.982 *********** 2021-07-11T03:47:57.515 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-07-11T03:47:57.518 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.544 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.036) 0:01:12.019 *********** 2021-07-11T03:47:57.578 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-07-11T03:47:57.580 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.610 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.066) 0:01:12.085 *********** 2021-07-11T03:47:57.619 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-07-11T03:47:57.620 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.007567", "end": "2021-07-11 03:47:57.515980", "rc": 0, "start": "2021-07-11 03:47:57.508413", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.622 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.649 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.038) 0:01:12.123 *********** 2021-07-11T03:47:57.680 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.028) 0:01:12.152 *********** 2021-07-11T03:47:57.686 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-07-11T03:47:57.690 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Clean yum cache] ************************************************ skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.718 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.040) 0:01:12.193 *********** 2021-07-11T03:47:57.761 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.042) 0:01:12.235 *********** 2021-07-11T03:47:57.772 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-07-11T03:47:57.773 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.775 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.804 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.042) 0:01:12.278 *********** 2021-07-11T03:47:57.827 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.023) 0:01:12.301 *********** 2021-07-11T03:47:57.835 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-07-11T03:47:57.836 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.864 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.036) 0:01:12.338 *********** 2021-07-11T03:47:57.884 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-07-11T03:47:57.886 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.911 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.047) 0:01:12.386 *********** 2021-07-11T03:47:57.919 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-07-11T03:47:57.920 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.947 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.035) 0:01:12.421 *********** 2021-07-11T03:47:57.954 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-07-11T03:47:57.956 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.007187", "end": "2021-07-11 03:47:58.278861", "rc": 0, "start": "2021-07-11 03:47:58.271674", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-07-11T03:47:57.957 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* skipping: [gibba005.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2021-07-11T03:47:57.983 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:57 +0000 (0:00:00.036) 0:01:12.458 *********** 2021-07-11T03:47:57.991 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-07-11T03:47:58.017 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:58 +0000 (0:00:00.033) 0:01:12.492 *********** 2021-07-11T03:47:58.067 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:58.098 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:58 +0000 (0:00:00.080) 0:01:12.573 *********** 2021-07-11T03:47:58.108 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-07-11T03:47:58.109 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.008925", "end": "2021-07-11 03:47:58.276426", "rc": 0, "start": "2021-07-11 03:47:58.267501", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-07-11T03:47:58.137 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:58 +0000 (0:00:00.038) 0:01:12.611 *********** 2021-07-11T03:47:58.145 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-07-11T03:47:58.147 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:58.177 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:58 +0000 (0:00:00.040) 0:01:12.651 *********** 2021-07-11T03:47:58.184 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-07-11T03:47:58.186 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:58.215 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:58 +0000 (0:00:00.036) 0:01:12.688 *********** 2021-07-11T03:47:58.222 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-07-11T03:47:58.222 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:58.252 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:58 +0000 (0:00:00.037) 0:01:12.726 *********** 2021-07-11T03:47:58.337 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-07-11T03:47:58.339 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.012284", "end": "2021-07-11 03:47:58.771939", "rc": 0, "start": "2021-07-11 03:47:58.759655", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} TASK [common : Inform the OS of the new timezone] ****************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:58.364 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:58 +0000 (0:00:00.112) 0:01:12.839 *********** 2021-07-11T03:47:58.393 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:58 +0000 (0:00:00.028) 0:01:12.868 *********** 2021-07-11T03:47:58.395 INFO:teuthology.task.ansible.out: 2021-07-11T03:47:58.479 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-07-11T03:47:58.481 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:58.505 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:58 +0000 (0:00:00.111) 0:01:12.979 *********** 2021-07-11T03:47:58.588 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:58.612 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:58 +0000 (0:00:00.107) 0:01:13.087 *********** 2021-07-11T03:47:58.620 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-07-11T03:47:58.621 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "name": "sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit sleep.target is masked.\"", "LoadState": "masked", "Names": "sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:58.622 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "name": "sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit sleep.target is masked.\"", "LoadState": "masked", "Names": "sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:58.634 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "name": "sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit sleep.target is masked.\"", "LoadState": "masked", "Names": "sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:58.730 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "name": "suspend.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "suspend.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "suspend.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit suspend.target is masked.\"", "LoadState": "masked", "Names": "suspend.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:58.773 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "name": "suspend.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "suspend.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "suspend.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit suspend.target is masked.\"", "LoadState": "masked", "Names": "suspend.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:58.774 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "name": "suspend.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "suspend.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "suspend.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit suspend.target is masked.\"", "LoadState": "masked", "Names": "suspend.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:58.819 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "name": "sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit sleep.target is masked.\"", "LoadState": "masked", "Names": "sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:58.884 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "name": "hibernate.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hibernate.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hibernate.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hibernate.target is masked.\"", "LoadState": "masked", "Names": "hibernate.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:58.885 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "name": "hibernate.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hibernate.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hibernate.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hibernate.target is masked.\"", "LoadState": "masked", "Names": "hibernate.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:58.918 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "name": "hibernate.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hibernate.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hibernate.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hibernate.target is masked.\"", "LoadState": "masked", "Names": "hibernate.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:58.951 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "name": "suspend.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "suspend.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "suspend.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit suspend.target is masked.\"", "LoadState": "masked", "Names": "suspend.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:59.008 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "name": "hybrid-sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hybrid-sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hybrid-sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hybrid-sleep.target is masked.\"", "LoadState": "masked", "Names": "hybrid-sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:59.039 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.426) 0:01:13.513 *********** 2021-07-11T03:47:59.047 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "name": "hybrid-sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hybrid-sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hybrid-sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hybrid-sleep.target is masked.\"", "LoadState": "masked", "Names": "hybrid-sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:59.051 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "name": "hybrid-sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hybrid-sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hybrid-sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hybrid-sleep.target is masked.\"", "LoadState": "masked", "Names": "hybrid-sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:59.081 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.041) 0:01:13.555 *********** 2021-07-11T03:47:59.103 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.022) 0:01:13.578 *********** 2021-07-11T03:47:59.113 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "name": "hibernate.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hibernate.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hibernate.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hibernate.target is masked.\"", "LoadState": "masked", "Names": "hibernate.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:59.115 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.142 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.038) 0:01:13.617 *********** 2021-07-11T03:47:59.151 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.178 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.035) 0:01:13.652 *********** 2021-07-11T03:47:59.185 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.214 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.036) 0:01:13.688 *********** 2021-07-11T03:47:59.223 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-07-11T03:47:59.223 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "name": "hybrid-sleep.target", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "AllowIsolate": "no", "AssertResult": "no", "AssertTimestampMonotonic": "0", "CanIsolate": "no", "CanReload": "no", "CanStart": "no", "CanStop": "yes", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "DefaultDependencies": "yes", "Description": "hybrid-sleep.target", "FailureAction": "none", "FragmentPath": "/dev/null", "Id": "hybrid-sleep.target", "IgnoreOnIsolate": "no", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "LoadError": "org.freedesktop.systemd1.UnitMasked \"Unit hybrid-sleep.target is masked.\"", "LoadState": "masked", "Names": "hybrid-sleep.target", "NeedDaemonReload": "no", "OnFailureJobMode": "replace", "Perpetual": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StateChangeTimestampMonotonic": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SuccessAction": "none", "Transient": "no", "UnitFileState": "masked"}} 2021-07-11T03:47:59.228 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-07-11T03:47:59.230 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.258 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.043) 0:01:13.732 *********** 2021-07-11T03:47:59.286 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.027) 0:01:13.760 *********** 2021-07-11T03:47:59.295 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.297 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.324 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.037) 0:01:13.798 *********** 2021-07-11T03:47:59.350 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.025) 0:01:13.824 *********** 2021-07-11T03:47:59.359 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-07-11T03:47:59.361 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.389 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.039) 0:01:13.864 *********** 2021-07-11T03:47:59.423 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.033) 0:01:13.897 *********** 2021-07-11T03:47:59.431 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.433 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.460 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.037) 0:01:13.934 *********** 2021-07-11T03:47:59.487 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.027) 0:01:13.962 *********** 2021-07-11T03:47:59.496 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-07-11T03:47:59.498 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.526 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.038) 0:01:14.001 *********** 2021-07-11T03:47:59.554 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.025) 0:01:14.026 *********** 2021-07-11T03:47:59.560 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.562 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.597 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.045) 0:01:14.071 *********** 2021-07-11T03:47:59.909 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.032) 0:01:14.104 *********** TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 11 July 2021 03:47:59 +0000 (0:00:00.044) 0:01:14.148 *********** TASK [common : Refresh repos] ************************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 11 July 2021 03:47:59 +0000 (0:00:00.041) 0:01:14.189 *********** Sunday 11 July 2021 03:47:59 +0000 (0:00:00.023) 0:01:14.213 *********** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 11 July 2021 03:47:59 +0000 (0:00:00.033) 0:01:14.247 *********** TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 11 July 2021 03:47:59 +0000 (0:00:00.044) 0:01:14.292 *********** TASK [common : Refresh repos] ************************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:47:59.928 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:47:59 +0000 (0:00:00.111) 0:01:14.403 *********** 2021-07-11T03:48:00.064 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-07-11T03:48:00.066 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.035847", "end": "2021-07-11 03:48:00.500851", "rc": 0, "start": "2021-07-11 03:48:00.465004", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:00.089 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:00 +0000 (0:00:00.161) 0:01:14.564 *********** 2021-07-11T03:48:00.160 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.289986", "end": "2021-07-11 03:48:00.500934", "rc": 0, "start": "2021-07-11 03:48:00.210948", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:00.188 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:00 +0000 (0:00:00.098) 0:01:14.662 *********** 2021-07-11T03:48:00.195 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-07-11T03:48:00.197 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:00.224 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:00 +0000 (0:00:00.035) 0:01:14.698 *********** 2021-07-11T03:48:00.252 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:00.278 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:00 +0000 (0:00:00.054) 0:01:14.753 *********** 2021-07-11T03:48:00.297 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-07-11T03:48:00.299 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.480436", "end": "2021-07-11 03:48:00.500906", "rc": 0, "start": "2021-07-11 03:48:00.020470", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:00.326 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:00 +0000 (0:00:00.047) 0:01:14.800 *********** 2021-07-11T03:48:00.398 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-07-11T03:48:00.399 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:00.422 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:00 +0000 (0:00:00.097) 0:01:14.897 *********** 2021-07-11T03:48:00.494 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ [WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. 2021-07-11T03:48:00.496 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.107361", "end": "2021-07-11 03:48:00.912418", "rc": 0, "start": "2021-07-11 03:48:00.805057", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:00.522 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:00 +0000 (0:00:00.099) 0:01:14.996 *********** 2021-07-11T03:48:00.566 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-07-11T03:48:00.568 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.719174", "end": "2021-07-11 03:48:00.500929", "rc": 0, "start": "2021-07-11 03:47:59.781755", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:00.596 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:00 +0000 (0:00:00.073) 0:01:15.070 *********** 2021-07-11T03:48:00.597 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:00.680 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-07-11T03:48:00.682 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:00.705 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:00 +0000 (0:00:00.109) 0:01:15.180 *********** 2021-07-11T03:48:00.713 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-07-11T03:48:00.714 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.265662", "end": "2021-07-11 03:48:01.027060", "rc": 0, "start": "2021-07-11 03:48:00.761398", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:00.741 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:00 +0000 (0:00:00.035) 0:01:15.215 *********** 2021-07-11T03:48:00.749 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.121421", "end": "2021-07-11 03:48:00.905901", "rc": 0, "start": "2021-07-11 03:48:00.784480", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:00.779 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:00 +0000 (0:00:00.038) 0:01:15.254 *********** 2021-07-11T03:48:01.010 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.127460", "end": "2021-07-11 03:48:00.929560", "rc": 0, "start": "2021-07-11 03:48:00.802100", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:01.042 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:01 +0000 (0:00:00.263) 0:01:15.517 *********** 2021-07-11T03:48:03.674 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-07-11T03:48:03.676 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:48:03.699 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:03 +0000 (0:00:02.656) 0:01:18.174 *********** 2021-07-11T03:48:03.836 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:48:03.862 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:03 +0000 (0:00:00.162) 0:01:18.336 *********** 2021-07-11T03:48:03.933 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1625974990, "cache_updated": false, "changed": false} 2021-07-11T03:48:03.960 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:03 +0000 (0:00:00.098) 0:01:18.435 *********** 2021-07-11T03:48:04.109 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:48:04.110 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:04.137 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:04 +0000 (0:00:00.177) 0:01:18.612 *********** 2021-07-11T03:48:04.230 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-07-11T03:48:04.232 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:48:04.257 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:04 +0000 (0:00:00.119) 0:01:18.731 *********** 2021-07-11T03:48:04.328 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-07-11T03:48:04.329 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:04.352 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:04 +0000 (0:00:00.094) 0:01:18.826 *********** 2021-07-11T03:48:04.412 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-07-11T03:48:04.414 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:48:04.439 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:04 +0000 (0:00:00.087) 0:01:18.914 *********** 2021-07-11T03:48:04.459 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} 2021-07-11T03:48:04.485 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:04 +0000 (0:00:00.046) 0:01:18.960 *********** 2021-07-11T03:48:04.524 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-07-11T03:48:04.526 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:04.549 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:04 +0000 (0:00:00.064) 0:01:19.024 *********** 2021-07-11T03:48:04.571 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:04.598 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:04 +0000 (0:00:00.048) 0:01:19.073 *********** 2021-07-11T03:48:04.637 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-07-11T03:48:04.639 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:48:04.665 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:04 +0000 (0:00:00.066) 0:01:19.140 *********** 2021-07-11T03:48:04.757 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-07-11T03:48:04.758 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:04.786 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:04 +0000 (0:00:00.121) 0:01:19.261 *********** 2021-07-11T03:48:04.966 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-07-11T03:48:04.968 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-07-11T03:48:04.992 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:04 +0000 (0:00:00.205) 0:01:19.467 *********** 2021-07-11T03:48:05.001 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-07-11T03:48:05.002 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-07-11T03:48:05.029 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:05 +0000 (0:00:00.036) 0:01:19.504 *********** 2021-07-11T03:48:05.059 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:05 +0000 (0:00:00.029) 0:01:19.533 *********** 2021-07-11T03:48:05.088 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-07-11T03:48:05.115 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:05 +0000 (0:00:00.056) 0:01:19.589 *********** 2021-07-11T03:48:05.470 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-07-11T03:48:05.472 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:48:05.506 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:05 +0000 (0:00:00.391) 0:01:19.981 *********** 2021-07-11T03:48:05.518 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:48:05.545 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:05 +0000 (0:00:00.038) 0:01:20.019 *********** 2021-07-11T03:48:05.553 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} 2021-07-11T03:48:05.579 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:05 +0000 (0:00:00.034) 0:01:20.054 *********** 2021-07-11T03:48:05.608 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:48:05.635 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:05 +0000 (0:00:00.055) 0:01:20.110 *********** 2021-07-11T03:48:05.835 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-07-11T03:48:05.836 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-07-11T03:48:05.868 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-07-11T03:48:05.870 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:05.921 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-07-11T03:48:06.008 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-07-11T03:48:06.128 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-07-11T03:48:06.130 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-07-11T03:48:06.183 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-07-11T03:48:06.264 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-07-11T03:48:06.337 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-07-11T03:48:06.366 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:06 +0000 (0:00:00.730) 0:01:20.840 *********** 2021-07-11T03:48:06.374 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-07-11T03:48:06.404 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:06 +0000 (0:00:00.038) 0:01:20.879 *********** 2021-07-11T03:48:06.413 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-07-11T03:48:06.446 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:06 +0000 (0:00:00.042) 0:01:20.921 *********** 2021-07-11T03:48:06.507 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-07-11T03:48:06.508 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:48:06.509 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:48:06.510 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [gibba019.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-07-11T03:48:06.540 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:06 +0000 (0:00:00.093) 0:01:21.014 *********** 2021-07-11T03:48:06.565 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:06 +0000 (0:00:00.025) 0:01:21.040 *********** 2021-07-11T03:48:06.591 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:06 +0000 (0:00:00.025) 0:01:21.065 *********** 2021-07-11T03:48:06.599 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-07-11T03:48:06.601 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:48:06.628 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:06 +0000 (0:00:00.037) 0:01:21.103 *********** 2021-07-11T03:48:06.702 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:48:06.733 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:06 +0000 (0:00:00.104) 0:01:21.208 *********** 2021-07-11T03:48:06.855 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [gibba004.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-07-11T03:48:06.868 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-07-11T03:48:06.870 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:06.957 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-07-11T03:48:07.054 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-07-11T03:48:07.087 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-07-11T03:48:07.101 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-07-11T03:48:07.198 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-07-11T03:48:07.273 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-07-11T03:48:07.372 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": true, "checksum": "2fbc67b36d59f66fc2621f3ca2dcf60dea2167a5", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "md5sum": "fae049f2bb3feda9c19a2dc30bce838a", "mode": "0755", "owner": "root", "size": 3612, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975287.0984535-25385-33179631209076/source", "state": "file", "uid": 0} 2021-07-11T03:48:07.398 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.665) 0:01:21.873 *********** 2021-07-11T03:48:07.435 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": true, "checksum": "2fbc67b36d59f66fc2621f3ca2dcf60dea2167a5", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "md5sum": "fae049f2bb3feda9c19a2dc30bce838a", "mode": "0755", "owner": "root", "size": 3612, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975287.1103954-25378-207081801084343/source", "state": "file", "uid": 0} 2021-07-11T03:48:07.465 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.066) 0:01:21.940 *********** 2021-07-11T03:48:07.484 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": true, "checksum": "2fbc67b36d59f66fc2621f3ca2dcf60dea2167a5", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "md5sum": "fae049f2bb3feda9c19a2dc30bce838a", "mode": "0755", "owner": "root", "size": 3612, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975287.2132545-25395-218654333997708/source", "state": "file", "uid": 0} 2021-07-11T03:48:07.489 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-07-11T03:48:07.491 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-07-11T03:48:07.516 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.050) 0:01:21.990 *********** 2021-07-11T03:48:07.544 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.027) 0:01:22.018 *********** 2021-07-11T03:48:07.553 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-07-11T03:48:07.589 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.045) 0:01:22.064 *********** 2021-07-11T03:48:07.597 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-07-11T03:48:07.599 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": true, "checksum": "2fbc67b36d59f66fc2621f3ca2dcf60dea2167a5", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "md5sum": "fae049f2bb3feda9c19a2dc30bce838a", "mode": "0755", "owner": "root", "size": 3612, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975287.2983668-25416-253694925333740/source", "state": "file", "uid": 0} 2021-07-11T03:48:07.627 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.037) 0:01:22.101 *********** 2021-07-11T03:48:07.635 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:07.636 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-07-11T03:48:07.662 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.034) 0:01:22.136 *********** 2021-07-11T03:48:07.664 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:07.687 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.024) 0:01:22.161 *********** 2021-07-11T03:48:07.694 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-07-11T03:48:07.696 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:07.721 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.034) 0:01:22.196 *********** 2021-07-11T03:48:07.729 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-07-11T03:48:07.731 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-07-11T03:48:07.755 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.034) 0:01:22.230 *********** 2021-07-11T03:48:07.763 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:07.764 INFO:teuthology.task.ansible.out:TASK [common : Check for epel] ************************************************* skipping: [gibba005.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:48:07.765 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:07.792 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.036) 0:01:22.266 *********** 2021-07-11T03:48:07.816 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.023) 0:01:22.290 *********** 2021-07-11T03:48:07.822 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-07-11T03:48:07.823 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:48:07.849 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.033) 0:01:22.324 *********** 2021-07-11T03:48:07.856 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-07-11T03:48:07.858 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:07.883 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.034) 0:01:22.358 *********** 2021-07-11T03:48:07.891 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-07-11T03:48:07.892 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:07.904 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* skipping: [gibba023.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Install nrpe without epel] ************************************** skipping: [gibba005.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:07.906 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:07.934 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.049) 0:01:22.408 *********** 2021-07-11T03:48:07.943 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:07.944 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:07.969 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:07 +0000 (0:00:00.035) 0:01:22.443 *********** 2021-07-11T03:48:07.976 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:07.978 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} TASK [common : Check for epel] ************************************************* skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Install nrpe without epel] ************************************** skipping: [gibba004.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:08.009 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.040) 0:01:22.484 *********** 2021-07-11T03:48:08.043 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.031) 0:01:22.516 *********** 2021-07-11T03:48:08.056 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:08.058 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:08.083 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.041) 0:01:22.557 *********** 2021-07-11T03:48:08.090 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:08.092 INFO:teuthology.task.ansible.out:TASK [common : Install nrpe without epel] ************************************** skipping: [gibba023.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:08.095 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:08.134 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.050) 0:01:22.608 *********** 2021-07-11T03:48:08.142 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-07-11T03:48:08.144 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:08.170 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.036) 0:01:22.644 *********** 2021-07-11T03:48:08.176 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-07-11T03:48:08.178 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:08.189 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:08.191 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:08.202 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:08.204 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:08.230 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.059) 0:01:22.704 *********** 2021-07-11T03:48:08.254 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.024) 0:01:22.728 *********** 2021-07-11T03:48:08.312 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:08.336 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.082) 0:01:22.811 *********** 2021-07-11T03:48:08.598 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-07-11T03:48:08.600 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:48:08.637 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.301) 0:01:23.112 *********** 2021-07-11T03:48:08.660 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:48:08.685 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.047) 0:01:23.160 *********** 2021-07-11T03:48:08.744 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} 2021-07-11T03:48:08.774 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.088) 0:01:23.248 *********** 2021-07-11T03:48:08.836 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:48:08.873 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.098) 0:01:23.347 *********** 2021-07-11T03:48:08.936 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-07-11T03:48:08.938 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-07-11T03:48:08.967 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:08 +0000 (0:00:00.093) 0:01:23.442 *********** 2021-07-11T03:48:08.998 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-07-11T03:48:09.025 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.057) 0:01:23.499 *********** 2021-07-11T03:48:09.077 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-07-11T03:48:09.104 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.079) 0:01:23.579 *********** 2021-07-11T03:48:09.188 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-07-11T03:48:09.216 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.111) 0:01:23.691 *********** 2021-07-11T03:48:09.310 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-07-11T03:48:09.312 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-07-11T03:48:09.340 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.123) 0:01:23.815 *********** 2021-07-11T03:48:09.378 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-07-11T03:48:09.403 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.063) 0:01:23.878 *********** 2021-07-11T03:48:09.442 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-07-11T03:48:09.468 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.064) 0:01:23.942 *********** 2021-07-11T03:48:09.507 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-07-11T03:48:09.509 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-07-11T03:48:09.541 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-07-11T03:48:09.542 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-07-11T03:48:09.565 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.097) 0:01:24.040 *********** 2021-07-11T03:48:09.583 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-07-11T03:48:09.584 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-07-11T03:48:09.605 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-07-11T03:48:09.631 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.066) 0:01:24.106 *********** 2021-07-11T03:48:09.650 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-07-11T03:48:09.672 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-07-11T03:48:09.700 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.068) 0:01:24.174 *********** 2021-07-11T03:48:09.719 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-07-11T03:48:09.721 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:09.746 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.046) 0:01:24.221 *********** 2021-07-11T03:48:09.755 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-07-11T03:48:09.756 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-07-11T03:48:09.791 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.045) 0:01:24.266 *********** 2021-07-11T03:48:09.801 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-07-11T03:48:09.802 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:09.829 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.037) 0:01:24.304 *********** 2021-07-11T03:48:09.840 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-07-11T03:48:09.842 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:48:09.871 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.041) 0:01:24.346 *********** 2021-07-11T03:48:09.881 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ ok: [gibba019.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-07-11T03:48:09.883 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-07-11T03:48:09.885 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:09.914 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.043) 0:01:24.389 *********** 2021-07-11T03:48:09.938 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.023) 0:01:24.412 *********** 2021-07-11T03:48:09.945 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-07-11T03:48:09.947 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:48:09.973 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:09 +0000 (0:00:00.035) 0:01:24.448 *********** 2021-07-11T03:48:09.980 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-07-11T03:48:09.982 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:10.008 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.034) 0:01:24.482 *********** 2021-07-11T03:48:10.016 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-07-11T03:48:10.018 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Check firewalld status] ***************************************** skipping: [gibba023.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:48:10.048 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.039) 0:01:24.522 *********** 2021-07-11T03:48:10.050 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:10.076 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.028) 0:01:24.551 *********** 2021-07-11T03:48:10.085 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-07-11T03:48:10.086 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:10.113 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.037) 0:01:24.588 *********** 2021-07-11T03:48:10.134 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-07-11T03:48:10.135 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-07-11T03:48:10.164 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.050) 0:01:24.638 *********** 2021-07-11T03:48:10.174 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-07-11T03:48:10.175 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:10.202 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.038) 0:01:24.676 *********** 2021-07-11T03:48:10.243 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:10.277 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.075) 0:01:24.752 *********** 2021-07-11T03:48:10.316 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-07-11T03:48:10.318 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-07-11T03:48:10.349 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.072) 0:01:24.824 *********** 2021-07-11T03:48:10.436 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-07-11T03:48:10.465 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.115) 0:01:24.939 *********** 2021-07-11T03:48:10.527 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-07-11T03:48:10.553 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.088) 0:01:25.027 *********** 2021-07-11T03:48:10.571 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-07-11T03:48:10.574 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:46:20 UTC", "ActiveEnterTimestampMonotonic": "13658745", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target systemd-tmpfiles-setup.service local-fs.target nss-lookup.target system.slice network.target -.mount systemd-journald.socket var-run.mount sysinit.target basic.target time-sync.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:46:20 UTC", "AssertTimestampMonotonic": "13651210", "Before": "shutdown.target getty@tty1.service plymouth-quit.service multi-user.target xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:46:20 UTC", "Con 2021-07-11T03:48:10.576 INFO:teuthology.task.ansible.out:ditionTimestampMonotonic": "13651209", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "990", "ExecMainStartTimestamp": "Sun 2021-07-11 03:46:20 UTC", "ExecMainStartTimestampMonotonic": "13658639", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sun 2021-07-11 03:46:20 UTC] ; stop_time=[n/a] ; pid=990 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[Sun 2021-07-11 03:46:20 UTC] ; stop_time=[n/a] ; pid=990 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID 2021-07-11T03:48:10.577 INFO:teuthology.task.ansible.out:": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:46:20 UTC", "InactiveExitTimestampMonotonic": "13658745", "InvocationID": "4c9cfab4ff40460d990d5b981ab6476b", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127825", "LimitNPROCSoft": "127825", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "990", "MemoryAccounting": "yes", "MemoryCurrent": "1376256", "MemoryDe 2021-07-11T03:48:10.579 INFO:teuthology.task.ansible.out:nyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:46:20 UTC", "StateChangeTimestampMonotonic": "13658745", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", 2021-07-11T03:48:10.580 INFO:teuthology.task.ansible.out: "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38347", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} TASK [common : Upload nagios nrpe config.] ************************************* ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-07-11T03:48:10.601 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.048) 0:01:25.076 *********** 2021-07-11T03:48:10.624 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.022) 0:01:25.098 *********** 2021-07-11T03:48:10.674 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-07-11T03:48:10.676 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:10.700 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.076) 0:01:25.175 *********** 2021-07-11T03:48:10.709 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-07-11T03:48:10.711 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:46:12 UTC", "ActiveEnterTimestampMonotonic": "14512590", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount system.slice var-run.mount sysinit.target local-fs.target basic.target network.target systemd-journald.socket remote-fs.target time-sync.target systemd-tmpfiles-setup.service nss-lookup.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:46:12 UTC", "AssertTimestampMonotonic": "14484395", "Before": "getty@tty1.service multi-user.target shutdown.target plymouth-quit.service xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:46:12 UTC", "Con 2021-07-11T03:48:10.712 INFO:teuthology.task.ansible.out:ditionTimestampMonotonic": "14484394", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1012", "ExecMainStartTimestamp": "Sun 2021-07-11 03:46:12 UTC", "ExecMainStartTimestampMonotonic": "14512490", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sun 2021-07-11 03:46:12 UTC] ; stop_time=[n/a] ; pid=1012 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[Sun 2021-07-11 03:46:12 UTC] ; stop_time=[n/a] ; pid=1012 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", " 2021-07-11T03:48:10.714 INFO:teuthology.task.ansible.out:GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:46:12 UTC", "InactiveExitTimestampMonotonic": "14512590", "InvocationID": "36f454dd847b423288689c9d6debc5c6", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127826", "LimitNPROCSoft": "127826", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127826", "LimitSIGPENDINGSoft": "127826", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "1012", "MemoryAccounting": "yes", "MemoryCurrent": "1261568", "Memo 2021-07-11T03:48:10.716 INFO:teuthology.task.ansible.out:ryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:46:12 UTC", "StateChangeTimestampMonotonic": "14512590", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": " 2021-07-11T03:48:10.717 INFO:teuthology.task.ansible.out:no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38347", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:48:10.738 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.037) 0:01:25.213 *********** 2021-07-11T03:48:10.768 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:46:30 UTC", "ActiveEnterTimestampMonotonic": "12308125", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target local-fs.target time-sync.target network.target var-run.mount -.mount system.slice basic.target systemd-journald.socket nss-lookup.target systemd-tmpfiles-setup.service sysinit.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:46:30 UTC", "AssertTimestampMonotonic": "12299102", "Before": "plymouth-quit.service multi-user.target getty@tty1.service xdm.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:46:30 UTC", "Con 2021-07-11T03:48:10.770 INFO:teuthology.task.ansible.out:ditionTimestampMonotonic": "12299100", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "961", "ExecMainStartTimestamp": "Sun 2021-07-11 03:46:30 UTC", "ExecMainStartTimestampMonotonic": "12308031", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sun 2021-07-11 03:46:30 UTC] ; stop_time=[n/a] ; pid=961 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[Sun 2021-07-11 03:46:30 UTC] ; stop_time=[n/a] ; pid=961 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID 2021-07-11T03:48:10.772 INFO:teuthology.task.ansible.out:": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:46:30 UTC", "InactiveExitTimestampMonotonic": "12308125", "InvocationID": "75c60783bc804141a150abe366e97616", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127825", "LimitNPROCSoft": "127825", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "961", "MemoryAccounting": "yes", "MemoryCurrent": "1331200", "MemoryDe 2021-07-11T03:48:10.773 INFO:teuthology.task.ansible.out:nyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:46:30 UTC", "StateChangeTimestampMonotonic": "12308125", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", 2021-07-11T03:48:10.775 INFO:teuthology.task.ansible.out: "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", 2021-07-11T03:48:10.775 INFO:teuthology.task.ansible.out:"TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38347", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:48:10.777 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-07-11T03:48:10.779 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:10.802 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.064) 0:01:25.277 *********** 2021-07-11T03:48:10.826 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.023) 0:01:25.300 *********** 2021-07-11T03:48:10.833 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-07-11T03:48:10.834 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:10.868 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.041) 0:01:25.342 *********** 2021-07-11T03:48:10.877 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-07-11T03:48:10.881 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:46:25 UTC", "ActiveEnterTimestampMonotonic": "14703576", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target nss-lookup.target local-fs.target -.mount basic.target time-sync.target system.slice remote-fs.target systemd-tmpfiles-setup.service var-run.mount sysinit.target systemd-journald.socket", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:46:25 UTC", "AssertTimestampMonotonic": "14695165", "Before": "getty@tty1.service multi-user.target shutdown.target plymouth-quit.service xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:46:25 UTC", "Con 2021-07-11T03:48:10.883 INFO:teuthology.task.ansible.out:ditionTimestampMonotonic": "14695164", "ConfigurationDirectoryMode": "0755", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "963", "ExecMainStartTimestamp": "Sun 2021-07-11 03:46:25 UTC", "ExecMainStartTimestampMonotonic": "14703468", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[Sun 2021-07-11 03:46:25 UTC] ; stop_time=[n/a] ; pid=963 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; flags= ; start_time=[Sun 2021-07-11 03:46:25 UTC] ; stop_time=[n/a] ; pid=963 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPostEx": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nagios/nrpe.pid ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID 2021-07-11T03:48:10.884 INFO:teuthology.task.ansible.out:": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:46:25 UTC", "InactiveExitTimestampMonotonic": "14703576", "InvocationID": "da0553dab06f415abbe01ac5d62a8e43", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127825", "LimitNPROCSoft": "127825", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "963", "MemoryAccounting": "yes", "MemoryCurrent": "1413120", "MemoryDe 2021-07-11T03:48:10.885 INFO:teuthology.task.ansible.out:nyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/run/nagios/nrpe.pid", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:46:25 UTC", "StateChangeTimestampMonotonic": "14703576", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", 2021-07-11T03:48:10.887 INFO:teuthology.task.ansible.out: "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "38347", "TimeoutAbortUSec": "1min", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "110", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:10.914 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.046) 0:01:25.389 *********** 2021-07-11T03:48:10.946 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.027) 0:01:25.416 *********** 2021-07-11T03:48:10.952 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-07-11T03:48:10.953 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:10.982 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:10 +0000 (0:00:00.040) 0:01:25.457 *********** 2021-07-11T03:48:11.010 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.027) 0:01:25.484 *********** 2021-07-11T03:48:11.019 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-07-11T03:48:11.020 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Get SELinux status] ********************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.049 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.039) 0:01:25.524 *********** 2021-07-11T03:48:11.074 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.024) 0:01:25.548 *********** 2021-07-11T03:48:11.082 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-07-11T03:48:11.083 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.083 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.112 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.037) 0:01:25.586 *********** 2021-07-11T03:48:11.137 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.025) 0:01:25.612 *********** 2021-07-11T03:48:11.147 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-07-11T03:48:11.148 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.174 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.036) 0:01:25.649 *********** 2021-07-11T03:48:11.200 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.026) 0:01:25.675 *********** 2021-07-11T03:48:11.208 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-07-11T03:48:11.210 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.234 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.033) 0:01:25.708 *********** 2021-07-11T03:48:11.242 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-07-11T03:48:11.244 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.278 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.043) 0:01:25.752 *********** 2021-07-11T03:48:11.285 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-07-11T03:48:11.287 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.313 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.034) 0:01:25.787 *********** 2021-07-11T03:48:11.336 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.023) 0:01:25.811 *********** 2021-07-11T03:48:11.344 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-07-11T03:48:11.346 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.371 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.034) 0:01:25.845 *********** 2021-07-11T03:48:11.378 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-07-11T03:48:11.379 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.379 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-07-11T03:48:11.380 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.407 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.036) 0:01:25.882 *********** 2021-07-11T03:48:11.428 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.021) 0:01:25.903 *********** 2021-07-11T03:48:11.435 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-07-11T03:48:11.437 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.463 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.034) 0:01:25.938 *********** 2021-07-11T03:48:11.471 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-07-11T03:48:11.473 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.500 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.036) 0:01:25.974 *********** 2021-07-11T03:48:11.506 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-07-11T03:48:11.508 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.532 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.033) 0:01:26.007 *********** 2021-07-11T03:48:11.659 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-07-11T03:48:11.661 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.686 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.153) 0:01:26.160 *********** 2021-07-11T03:48:11.706 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.742 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.056) 0:01:26.217 *********** 2021-07-11T03:48:11.753 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-07-11T03:48:11.755 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.780 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.038) 0:01:26.255 *********** 2021-07-11T03:48:11.790 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-07-11T03:48:11.791 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:11.831 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:11 +0000 (0:00:00.050) 0:01:26.306 *********** 2021-07-11T03:48:11.905 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:12.058 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.226) 0:01:26.532 *********** 2021-07-11T03:48:12.065 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:12.066 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Remove temporary files] ********************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:12.092 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.033) 0:01:26.566 *********** 2021-07-11T03:48:12.115 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.023) 0:01:26.589 *********** 2021-07-11T03:48:12.148 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.033) 0:01:26.623 *********** 2021-07-11T03:48:12.156 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-07-11T03:48:12.158 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:12.183 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.035) 0:01:26.658 *********** 2021-07-11T03:48:12.190 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-07-11T03:48:12.192 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:12.225 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.041) 0:01:26.700 *********** 2021-07-11T03:48:12.234 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-07-11T03:48:12.236 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} TASK [common : nrpe - Compile SELinux module file] ***************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:12.263 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.037) 0:01:26.737 *********** 2021-07-11T03:48:12.287 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.024) 0:01:26.762 *********** 2021-07-11T03:48:12.296 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-07-11T03:48:12.298 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:12.324 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.036) 0:01:26.798 *********** 2021-07-11T03:48:12.331 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-07-11T03:48:12.333 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:12.358 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.034) 0:01:26.833 *********** 2021-07-11T03:48:12.367 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-07-11T03:48:12.368 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-07-11T03:48:12.398 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.039) 0:01:26.872 *********** 2021-07-11T03:48:12.405 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-07-11T03:48:12.407 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:12.434 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.036) 0:01:26.909 *********** 2021-07-11T03:48:12.466 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.031) 0:01:26.941 *********** 2021-07-11T03:48:12.475 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-07-11T03:48:12.477 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-07-11T03:48:12.478 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** ok: [gibba005.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} 2021-07-11T03:48:12.505 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.039) 0:01:26.980 *********** 2021-07-11T03:48:12.529 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.023) 0:01:27.004 *********** 2021-07-11T03:48:12.538 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-07-11T03:48:12.540 INFO:teuthology.task.ansible.out: skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : nrpe - Remove temporary files] ********************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:12.573 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.044) 0:01:27.048 *********** 2021-07-11T03:48:12.606 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.031) 0:01:27.080 *********** 2021-07-11T03:48:12.613 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-07-11T03:48:12.616 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-07-11T03:48:12.643 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.037) 0:01:27.118 *********** 2021-07-11T03:48:12.651 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-07-11T03:48:12.653 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-07-11T03:48:12.653 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-07-11T03:48:12.654 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:12.684 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.040) 0:01:27.158 *********** 2021-07-11T03:48:12.709 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.025) 0:01:27.184 *********** 2021-07-11T03:48:12.716 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-07-11T03:48:12.718 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-07-11T03:48:12.745 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.035) 0:01:27.219 *********** 2021-07-11T03:48:12.752 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-07-11T03:48:12.754 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} 2021-07-11T03:48:12.791 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.046) 0:01:27.266 *********** 2021-07-11T03:48:12.801 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-07-11T03:48:12.802 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} TASK [testnode : Include package type specific vars.] ************************** ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-07-11T03:48:12.833 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.042) 0:01:27.308 *********** 2021-07-11T03:48:12.864 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.030) 0:01:27.339 *********** 2021-07-11T03:48:12.874 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-07-11T03:48:12.875 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-07-11T03:48:12.877 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [gibba004.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-07-11T03:48:12.905 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.039) 0:01:27.379 *********** 2021-07-11T03:48:12.929 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.024) 0:01:27.403 *********** 2021-07-11T03:48:12.954 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-07-11T03:48:12.955 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-07-11T03:48:12.998 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:12 +0000 (0:00:00.069) 0:01:27.473 *********** 2021-07-11T03:48:13.016 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-07-11T03:48:13.018 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} TASK [testnode : Including major version specific variables.] ****************** ok: [gibba023.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} 2021-07-11T03:48:13.045 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.046) 0:01:27.520 *********** 2021-07-11T03:48:13.069 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.023) 0:01:27.544 *********** 2021-07-11T03:48:13.071 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:13.078 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-07-11T03:48:13.079 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-07-11T03:48:13.106 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.036) 0:01:27.580 *********** 2021-07-11T03:48:13.115 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-07-11T03:48:13.116 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "lttng-tools", "libtool-bin", "docker.io", "python3-nose", "python3-virtualenv", "python3-configobj", "python3-gevent", "python3-numpy", "python3-matplotlib", "python3-setuptools"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_20.yml"} 2021-07-11T03:48:13.142 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.036) 0:01:27.617 *********** 2021-07-11T03:48:13.181 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-07-11T03:48:13.183 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} TASK [testnode : Create the teuthology user.] ********************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:13.184 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-07-11T03:48:13.208 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.065) 0:01:27.683 *********** 2021-07-11T03:48:13.232 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.023) 0:01:27.706 *********** 2021-07-11T03:48:13.239 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-07-11T03:48:13.241 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} TASK [testnode : Including version specific variables.] ************************ ok: [gibba019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-07-11T03:48:13.266 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.034) 0:01:27.741 *********** 2021-07-11T03:48:13.289 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.022) 0:01:27.764 *********** 2021-07-11T03:48:13.308 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-07-11T03:48:13.310 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:13.343 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.052) 0:01:27.816 *********** 2021-07-11T03:48:13.350 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-07-11T03:48:13.351 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} TASK [testnode : Create the teuthology user.] ********************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:13.378 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.036) 0:01:27.852 *********** 2021-07-11T03:48:13.413 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.034) 0:01:27.887 *********** 2021-07-11T03:48:13.420 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-07-11T03:48:13.422 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-07-11T03:48:13.452 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.039) 0:01:27.927 *********** 2021-07-11T03:48:13.461 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-07-11T03:48:13.462 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:13.500 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.048) 0:01:27.975 *********** 2021-07-11T03:48:13.522 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-07-11T03:48:13.524 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-07-11T03:48:13.549 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.047) 0:01:28.023 *********** 2021-07-11T03:48:13.602 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-07-11T03:48:13.604 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-07-11T03:48:13.628 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.080) 0:01:28.103 *********** 2021-07-11T03:48:13.656 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.027) 0:01:28.131 *********** 2021-07-11T03:48:13.675 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-07-11T03:48:13.676 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-07-11T03:48:13.703 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.046) 0:01:28.177 *********** 2021-07-11T03:48:13.722 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-07-11T03:48:13.724 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:13.751 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.047) 0:01:28.225 *********** 2021-07-11T03:48:13.758 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:13.785 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.033) 0:01:28.259 *********** 2021-07-11T03:48:13.822 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-07-11T03:48:13.824 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:13.851 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.067) 0:01:28.326 *********** 2021-07-11T03:48:13.871 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:13.873 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:13.909 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.057) 0:01:28.384 *********** 2021-07-11T03:48:13.939 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-07-11T03:48:13.940 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-07-11T03:48:13.966 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:13 +0000 (0:00:00.056) 0:01:28.440 *********** 2021-07-11T03:48:14.071 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-07-11T03:48:14.073 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba005"}, "changed": false} TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [gibba019.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-07-11T03:48:14.097 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:14 +0000 (0:00:00.131) 0:01:28.572 *********** 2021-07-11T03:48:14.122 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:14 +0000 (0:00:00.024) 0:01:28.597 *********** 2021-07-11T03:48:14.130 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-07-11T03:48:14.132 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-07-11T03:48:14.162 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:14 +0000 (0:00:00.039) 0:01:28.637 *********** 2021-07-11T03:48:14.184 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:14 +0000 (0:00:00.022) 0:01:28.659 *********** 2021-07-11T03:48:14.245 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-07-11T03:48:14.247 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba004"}, "changed": false} 2021-07-11T03:48:14.273 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:14 +0000 (0:00:00.088) 0:01:28.747 *********** 2021-07-11T03:48:14.513 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-07-11T03:48:14.514 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-07-11T03:48:14.541 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:14 +0000 (0:00:00.268) 0:01:29.016 *********** 2021-07-11T03:48:14.582 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-07-11T03:48:14.609 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:14 +0000 (0:00:00.067) 0:01:29.083 *********** 2021-07-11T03:48:14.626 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-07-11T03:48:14.627 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba019"}, "changed": false} 2021-07-11T03:48:14.654 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:14 +0000 (0:00:00.043) 0:01:29.127 *********** 2021-07-11T03:48:14.692 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "gibba023"}, "changed": false} 2021-07-11T03:48:14.717 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:14 +0000 (0:00:00.064) 0:01:29.191 *********** 2021-07-11T03:48:14.841 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-07-11T03:48:14.843 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba004.front.sepia.ceph.com", "ansible_hostname": "gibba004", "ansible_nodename": "gibba004"}, "changed": true, "name": "gibba004"} changed: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba005.front.sepia.ceph.com", "ansible_hostname": "gibba005", "ansible_nodename": "gibba005"}, "changed": true, "name": "gibba005"} 2021-07-11T03:48:14.876 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:14 +0000 (0:00:00.159) 0:01:29.351 *********** 2021-07-11T03:48:14.905 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:14 +0000 (0:00:00.028) 0:01:29.379 *********** 2021-07-11T03:48:14.978 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba019.front.sepia.ceph.com", "ansible_hostname": "gibba019", "ansible_nodename": "gibba019"}, "changed": true, "name": "gibba019"} 2021-07-11T03:48:15.004 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:15 +0000 (0:00:00.098) 0:01:29.478 *********** 2021-07-11T03:48:15.034 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "gibba023.front.sepia.ceph.com", "ansible_hostname": "gibba023", "ansible_nodename": "gibba023"}, "changed": true, "name": "gibba023"} 2021-07-11T03:48:15.062 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:15 +0000 (0:00:00.058) 0:01:29.537 *********** 2021-07-11T03:48:15.176 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-07-11T03:48:15.177 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-07-11T03:48:15.203 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:15 +0000 (0:00:00.140) 0:01:29.677 *********** 2021-07-11T03:48:15.204 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:15.232 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-07-11T03:48:15.262 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:15 +0000 (0:00:00.059) 0:01:29.737 *********** 2021-07-11T03:48:15.315 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-07-11T03:48:15.316 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:15.346 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:15 +0000 (0:00:00.083) 0:01:29.820 *********** 2021-07-11T03:48:15.377 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-07-11T03:48:15.379 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:15.406 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:15 +0000 (0:00:00.059) 0:01:29.880 *********** 2021-07-11T03:48:15.497 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-07-11T03:48:15.498 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-07-11T03:48:15.523 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:15 +0000 (0:00:00.117) 0:01:29.998 *********** 2021-07-11T03:48:15.573 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-07-11T03:48:15.598 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:15 +0000 (0:00:00.074) 0:01:30.073 *********** 2021-07-11T03:48:15.691 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-07-11T03:48:15.727 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:15 +0000 (0:00:00.128) 0:01:30.201 *********** 2021-07-11T03:48:15.737 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-07-11T03:48:15.770 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:15 +0000 (0:00:00.043) 0:01:30.245 *********** 2021-07-11T03:48:15.939 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-07-11T03:48:15.940 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-07-11T03:48:15.974 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:15 +0000 (0:00:00.203) 0:01:30.449 *********** 2021-07-11T03:48:16.060 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-07-11T03:48:16.061 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.084 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.110) 0:01:30.559 *********** 2021-07-11T03:48:16.091 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-07-11T03:48:16.093 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-07-11T03:48:16.121 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.037) 0:01:30.596 *********** 2021-07-11T03:48:16.152 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-07-11T03:48:16.153 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.181 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.059) 0:01:30.656 *********** 2021-07-11T03:48:16.199 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-07-11T03:48:16.201 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.226 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.044) 0:01:30.701 *********** 2021-07-11T03:48:16.245 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-07-11T03:48:16.246 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-07-11T03:48:16.272 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.046) 0:01:30.747 *********** 2021-07-11T03:48:16.281 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-07-11T03:48:16.283 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.307 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.035) 0:01:30.782 *********** 2021-07-11T03:48:16.315 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-07-11T03:48:16.317 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.342 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.035) 0:01:30.817 *********** 2021-07-11T03:48:16.364 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.021) 0:01:30.839 *********** 2021-07-11T03:48:16.372 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-07-11T03:48:16.373 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.399 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.034) 0:01:30.874 *********** 2021-07-11T03:48:16.407 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-07-11T03:48:16.408 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.409 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-07-11T03:48:16.409 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.437 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.037) 0:01:30.911 *********** 2021-07-11T03:48:16.462 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.025) 0:01:30.937 *********** 2021-07-11T03:48:16.471 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-07-11T03:48:16.472 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.504 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.039) 0:01:30.976 *********** 2021-07-11T03:48:16.510 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-07-11T03:48:16.511 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.513 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.540 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.037) 0:01:31.014 *********** 2021-07-11T03:48:16.564 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.024) 0:01:31.039 *********** 2021-07-11T03:48:16.573 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-07-11T03:48:16.575 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.600 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.035) 0:01:31.075 *********** 2021-07-11T03:48:16.609 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-07-11T03:48:16.610 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.637 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.036) 0:01:31.111 *********** 2021-07-11T03:48:16.659 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.022) 0:01:31.133 *********** 2021-07-11T03:48:16.666 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-07-11T03:48:16.667 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.696 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.034) 0:01:31.168 *********** 2021-07-11T03:48:16.701 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.729 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.035) 0:01:31.204 *********** 2021-07-11T03:48:16.739 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-07-11T03:48:16.741 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.768 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.039) 0:01:31.243 *********** 2021-07-11T03:48:16.791 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.022) 0:01:31.265 *********** 2021-07-11T03:48:16.797 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-07-11T03:48:16.799 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.824 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.033) 0:01:31.299 *********** 2021-07-11T03:48:16.833 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.859 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.035) 0:01:31.334 *********** 2021-07-11T03:48:16.868 INFO:teuthology.task.ansible.out: TASK [testnode : Clean up Stream distro-provided repos] ************************ 2021-07-11T03:48:16.870 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.902 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.042) 0:01:31.376 *********** 2021-07-11T03:48:16.927 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.024) 0:01:31.401 *********** 2021-07-11T03:48:16.935 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-07-11T03:48:16.937 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.962 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.035) 0:01:31.437 *********** 2021-07-11T03:48:16.971 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-07-11T03:48:16.973 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:16.997 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:16 +0000 (0:00:00.035) 0:01:31.472 *********** 2021-07-11T03:48:17.005 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:48:17.007 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.032 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.034) 0:01:31.507 *********** 2021-07-11T03:48:17.055 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.022) 0:01:31.530 *********** 2021-07-11T03:48:17.062 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-07-11T03:48:17.064 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.090 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.034) 0:01:31.564 *********** 2021-07-11T03:48:17.098 INFO:teuthology.task.ansible.out: TASK [testnode : Clean up Stream distro-provided repos] ************************ 2021-07-11T03:48:17.098 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.099 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-07-11T03:48:17.099 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.126 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.036) 0:01:31.600 *********** 2021-07-11T03:48:17.150 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.024) 0:01:31.625 *********** 2021-07-11T03:48:17.158 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:17.160 INFO:teuthology.task.ansible.out:TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.186 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.036) 0:01:31.661 *********** 2021-07-11T03:48:17.209 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.023) 0:01:31.684 *********** 2021-07-11T03:48:17.216 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-07-11T03:48:17.242 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.032) 0:01:31.717 *********** 2021-07-11T03:48:17.250 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:48:17.252 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.286 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.043) 0:01:31.761 *********** 2021-07-11T03:48:17.295 INFO:teuthology.task.ansible.out: TASK [testnode : Clean up Stream distro-provided repos] ************************ 2021-07-11T03:48:17.296 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.296 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-07-11T03:48:17.297 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.323 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.037) 0:01:31.798 *********** 2021-07-11T03:48:17.347 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.023) 0:01:31.821 *********** 2021-07-11T03:48:17.355 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-07-11T03:48:17.356 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-07-11T03:48:17.356 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.382 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.035) 0:01:31.857 *********** 2021-07-11T03:48:17.405 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.022) 0:01:31.880 *********** 2021-07-11T03:48:17.412 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:48:17.413 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.413 INFO:teuthology.task.ansible.out: TASK [testnode : Clean up Stream distro-provided repos] ************************ 2021-07-11T03:48:17.414 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.440 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.034) 0:01:31.915 *********** 2021-07-11T03:48:17.463 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.023) 0:01:31.938 *********** 2021-07-11T03:48:17.471 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-07-11T03:48:17.509 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.045) 0:01:31.984 *********** 2021-07-11T03:48:17.523 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-07-11T03:48:17.525 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.550 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.040) 0:01:32.024 *********** 2021-07-11T03:48:17.576 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.026) 0:01:32.051 *********** 2021-07-11T03:48:17.583 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:48:17.585 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.610 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.033) 0:01:32.084 *********** 2021-07-11T03:48:17.617 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-07-11T03:48:17.619 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.643 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.033) 0:01:32.118 *********** 2021-07-11T03:48:17.651 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-07-11T03:48:17.652 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-07-11T03:48:17.685 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.041) 0:01:32.160 *********** 2021-07-11T03:48:17.708 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.022) 0:01:32.183 *********** 2021-07-11T03:48:17.716 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-07-11T03:48:17.718 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.744 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.036) 0:01:32.219 *********** 2021-07-11T03:48:17.752 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-07-11T03:48:17.754 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.755 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-07-11T03:48:17.789 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.044) 0:01:32.264 *********** 2021-07-11T03:48:17.817 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.027) 0:01:32.291 *********** 2021-07-11T03:48:17.825 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-07-11T03:48:17.827 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-07-11T03:48:17.862 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.044) 0:01:32.337 *********** 2021-07-11T03:48:17.898 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.036) 0:01:32.373 *********** 2021-07-11T03:48:17.909 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-07-11T03:48:17.910 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install GPG] ************************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:17.943 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.044) 0:01:32.418 *********** 2021-07-11T03:48:17.968 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:17 +0000 (0:00:00.025) 0:01:32.443 *********** 2021-07-11T03:48:17.977 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-07-11T03:48:17.979 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-07-11T03:48:18.005 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.037) 0:01:32.480 *********** 2021-07-11T03:48:18.030 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.024) 0:01:32.505 *********** 2021-07-11T03:48:18.037 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:48:18.038 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.063 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.033) 0:01:32.538 *********** 2021-07-11T03:48:18.071 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-07-11T03:48:18.073 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.104 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.040) 0:01:32.578 *********** 2021-07-11T03:48:18.112 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-07-11T03:48:18.114 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable copr repos] ******************************************** 2021-07-11T03:48:18.143 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.038) 0:01:32.617 *********** 2021-07-11T03:48:18.168 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.025) 0:01:32.643 *********** 2021-07-11T03:48:18.176 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-07-11T03:48:18.177 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Clean yum cache] ********************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.212 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.043) 0:01:32.687 *********** 2021-07-11T03:48:18.238 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.026) 0:01:32.713 *********** 2021-07-11T03:48:18.247 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-07-11T03:48:18.249 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.249 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.276 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.038) 0:01:32.751 *********** 2021-07-11T03:48:18.300 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.022) 0:01:32.774 *********** 2021-07-11T03:48:18.306 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-07-11T03:48:18.308 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.333 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.033) 0:01:32.808 *********** 2021-07-11T03:48:18.341 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:48:18.343 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.368 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.034) 0:01:32.843 *********** 2021-07-11T03:48:18.377 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:18.378 INFO:teuthology.task.ansible.out:TASK [testnode : Clean yum cache] ********************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install GPG keys] ********************************************* 2021-07-11T03:48:18.379 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.380 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.406 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.037) 0:01:32.881 *********** 2021-07-11T03:48:18.429 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.022) 0:01:32.903 *********** 2021-07-11T03:48:18.436 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-07-11T03:48:18.438 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.464 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.035) 0:01:32.938 *********** 2021-07-11T03:48:18.473 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-07-11T03:48:18.475 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.507 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.043) 0:01:32.981 *********** 2021-07-11T03:48:18.516 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:48:18.518 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Clean yum cache] ********************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.545 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.037) 0:01:33.019 *********** 2021-07-11T03:48:18.569 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.024) 0:01:33.044 *********** 2021-07-11T03:48:18.578 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-07-11T03:48:18.580 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Removing saved yum transactions] ****************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.606 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.037) 0:01:33.081 *********** 2021-07-11T03:48:18.630 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.023) 0:01:33.105 *********** 2021-07-11T03:48:18.637 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-07-11T03:48:18.639 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : set_fact] ***************************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.667 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.037) 0:01:33.142 *********** 2021-07-11T03:48:18.692 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.025) 0:01:33.167 *********** 2021-07-11T03:48:18.699 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-07-11T03:48:18.701 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.726 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.033) 0:01:33.201 *********** 2021-07-11T03:48:18.734 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.763 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.036) 0:01:33.237 *********** 2021-07-11T03:48:18.771 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-07-11T03:48:18.773 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.800 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.036) 0:01:33.274 *********** 2021-07-11T03:48:18.822 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.021) 0:01:33.296 *********** 2021-07-11T03:48:18.828 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-07-11T03:48:18.830 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.856 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.034) 0:01:33.331 *********** 2021-07-11T03:48:18.864 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-07-11T03:48:18.866 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.896 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.039) 0:01:33.371 *********** 2021-07-11T03:48:18.904 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-07-11T03:48:18.906 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.933 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.036) 0:01:33.407 *********** 2021-07-11T03:48:18.942 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-07-11T03:48:18.944 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:18.973 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.039) 0:01:33.447 *********** 2021-07-11T03:48:18.997 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:18 +0000 (0:00:00.024) 0:01:33.472 *********** 2021-07-11T03:48:19.006 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.031 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.033) 0:01:33.505 *********** 2021-07-11T03:48:19.038 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-07-11T03:48:19.039 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.065 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.034) 0:01:33.540 *********** 2021-07-11T03:48:19.074 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-07-11T03:48:19.076 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check if ceph-debuginfo is installed] ************************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.104 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.038) 0:01:33.579 *********** 2021-07-11T03:48:19.128 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.023) 0:01:33.603 *********** 2021-07-11T03:48:19.135 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-07-11T03:48:19.137 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.176 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.047) 0:01:33.650 *********** 2021-07-11T03:48:19.186 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.216 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.040) 0:01:33.691 *********** 2021-07-11T03:48:19.242 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.025) 0:01:33.717 *********** 2021-07-11T03:48:19.244 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:19.250 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-07-11T03:48:19.252 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.277 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.034) 0:01:33.752 *********** 2021-07-11T03:48:19.285 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-07-11T03:48:19.286 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.319 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.042) 0:01:33.794 *********** 2021-07-11T03:48:19.328 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-07-11T03:48:19.330 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.356 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.036) 0:01:33.830 *********** 2021-07-11T03:48:19.381 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.024) 0:01:33.855 *********** 2021-07-11T03:48:19.390 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-07-11T03:48:19.392 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.417 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.036) 0:01:33.892 *********** 2021-07-11T03:48:19.424 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-07-11T03:48:19.426 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.451 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.033) 0:01:33.926 *********** 2021-07-11T03:48:19.459 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-07-11T03:48:19.460 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.487 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.035) 0:01:33.961 *********** 2021-07-11T03:48:19.509 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.022) 0:01:33.984 *********** 2021-07-11T03:48:19.520 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-07-11T03:48:19.522 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.548 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.037) 0:01:34.022 *********** 2021-07-11T03:48:19.571 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.023) 0:01:34.046 *********** 2021-07-11T03:48:19.579 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.605 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.034) 0:01:34.080 *********** 2021-07-11T03:48:19.615 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-07-11T03:48:19.617 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.643 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.036) 0:01:34.116 *********** 2021-07-11T03:48:19.649 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-07-11T03:48:19.651 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.676 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.034) 0:01:34.151 *********** 2021-07-11T03:48:19.684 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-07-11T03:48:19.685 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.717 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.040) 0:01:34.192 *********** 2021-07-11T03:48:19.725 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-07-11T03:48:19.727 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.752 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.034) 0:01:34.226 *********** 2021-07-11T03:48:19.775 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.023) 0:01:34.250 *********** 2021-07-11T03:48:19.783 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-07-11T03:48:19.785 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.810 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.034) 0:01:34.284 *********** 2021-07-11T03:48:19.817 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-07-11T03:48:19.819 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.844 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.034) 0:01:34.319 *********** 2021-07-11T03:48:19.851 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-07-11T03:48:19.852 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upgrade packages] ********************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.878 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.033) 0:01:34.352 *********** 2021-07-11T03:48:19.901 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.022) 0:01:34.375 *********** 2021-07-11T03:48:19.908 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-07-11T03:48:19.910 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.934 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.033) 0:01:34.409 *********** 2021-07-11T03:48:19.942 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-07-11T03:48:19.944 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:19.969 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:19 +0000 (0:00:00.034) 0:01:34.444 *********** 2021-07-11T03:48:19.976 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-07-11T03:48:19.977 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable firewalld] *********************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.003 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.033) 0:01:34.477 *********** 2021-07-11T03:48:20.025 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.021) 0:01:34.499 *********** 2021-07-11T03:48:20.032 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-07-11T03:48:20.034 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.061 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.036) 0:01:34.535 *********** 2021-07-11T03:48:20.210 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 11 July 2021 03:48:20 +0000 (0:00:00.040) 0:01:34.576 *********** TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable iptables] ************************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 11 July 2021 03:48:20 +0000 (0:00:00.039) 0:01:34.615 *********** Sunday 11 July 2021 03:48:20 +0000 (0:00:00.026) 0:01:34.642 *********** TASK [testnode : Install abrt] ************************************************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.211 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.042) 0:01:34.685 *********** 2021-07-11T03:48:20.219 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-07-11T03:48:20.220 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.247 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.037) 0:01:34.722 *********** 2021-07-11T03:48:20.272 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.024) 0:01:34.747 *********** 2021-07-11T03:48:20.280 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.307 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.034) 0:01:34.782 *********** 2021-07-11T03:48:20.315 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-07-11T03:48:20.317 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.341 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.033) 0:01:34.816 *********** 2021-07-11T03:48:20.348 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-07-11T03:48:20.350 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.374 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.033) 0:01:34.849 *********** 2021-07-11T03:48:20.382 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-07-11T03:48:20.384 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install abrt] ************************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.411 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.035) 0:01:34.885 *********** 2021-07-11T03:48:20.435 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.024) 0:01:34.909 *********** 2021-07-11T03:48:20.442 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-07-11T03:48:20.444 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.472 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.035) 0:01:34.944 *********** 2021-07-11T03:48:20.490 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-07-11T03:48:20.492 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.524 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.053) 0:01:34.998 *********** 2021-07-11T03:48:20.532 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-07-11T03:48:20.534 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.534 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-07-11T03:48:20.535 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001334", "end": "2021-07-11 03:48:20.732929", "rc": 0, "start": "2021-07-11 03:48:20.731595", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:20.562 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.038) 0:01:35.037 *********** 2021-07-11T03:48:20.587 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.023) 0:01:35.061 *********** 2021-07-11T03:48:20.594 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-07-11T03:48:20.596 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.621 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.034) 0:01:35.096 *********** 2021-07-11T03:48:20.630 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-07-11T03:48:20.632 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.658 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.036) 0:01:35.133 *********** 2021-07-11T03:48:20.665 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-07-11T03:48:20.668 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.693 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.035) 0:01:35.168 *********** 2021-07-11T03:48:20.717 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.023) 0:01:35.191 *********** 2021-07-11T03:48:20.724 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-07-11T03:48:20.725 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.725 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.756 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.039) 0:01:35.230 *********** 2021-07-11T03:48:20.782 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.025) 0:01:35.256 *********** 2021-07-11T03:48:20.800 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-07-11T03:48:20.801 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.829 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.047) 0:01:35.304 *********** 2021-07-11T03:48:20.861 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-07-11T03:48:20.862 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.888 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.058) 0:01:35.362 *********** 2021-07-11T03:48:20.912 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:20.937 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.049) 0:01:35.412 *********** 2021-07-11T03:48:20.945 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-07-11T03:48:20.947 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001289", "end": "2021-07-11 03:48:21.352956", "rc": 0, "start": "2021-07-11 03:48:21.351667", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:20.980 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:20 +0000 (0:00:00.042) 0:01:35.455 *********** 2021-07-11T03:48:20.999 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-07-11T03:48:21.001 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-07-11T03:48:21.026 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.045) 0:01:35.501 *********** 2021-07-11T03:48:21.036 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:21.064 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.037) 0:01:35.539 *********** 2021-07-11T03:48:21.072 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-07-11T03:48:21.073 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001366", "end": "2021-07-11 03:48:21.006686", "rc": 0, "start": "2021-07-11 03:48:21.005320", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:21.102 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.037) 0:01:35.576 *********** 2021-07-11T03:48:21.134 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.032) 0:01:35.608 *********** 2021-07-11T03:48:21.145 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-07-11T03:48:21.146 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Restart abrtd] ************************************************ skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:21.178 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.044) 0:01:35.653 *********** 2021-07-11T03:48:21.206 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.027) 0:01:35.681 *********** 2021-07-11T03:48:21.226 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-07-11T03:48:21.258 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.051) 0:01:35.732 *********** 2021-07-11T03:48:21.378 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-07-11T03:48:21.380 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.001325", "end": "2021-07-11 03:48:21.712997", "rc": 0, "start": "2021-07-11 03:48:21.711672", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:21.405 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.146) 0:01:35.879 *********** 2021-07-11T03:48:21.423 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-07-11T03:48:21.425 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-07-11T03:48:21.455 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.048) 0:01:35.928 *********** 2021-07-11T03:48:21.495 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-07-11T03:48:21.521 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.067) 0:01:35.996 *********** 2021-07-11T03:48:21.538 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-07-11T03:48:21.540 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:21.564 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.042) 0:01:36.038 *********** 2021-07-11T03:48:21.592 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-07-11T03:48:21.592 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-07-11T03:48:21.593 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:21.625 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.061) 0:01:36.100 *********** 2021-07-11T03:48:21.697 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-07-11T03:48:21.699 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:21.723 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.097) 0:01:36.197 *********** 2021-07-11T03:48:21.804 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-07-11T03:48:21.806 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-07-11T03:48:21.833 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.110) 0:01:36.308 *********** 2021-07-11T03:48:21.906 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-07-11T03:48:21.908 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:21.931 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:21 +0000 (0:00:00.098) 0:01:36.406 *********** 2021-07-11T03:48:22.236 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-07-11T03:48:22.237 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-07-11T03:48:22.364 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-07-11T03:48:22.460 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-07-11T03:48:22.566 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-07-11T03:48:22.875 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-07-11T03:48:22.899 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:22 +0000 (0:00:00.967) 0:01:37.374 *********** 2021-07-11T03:48:23.078 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-07-11T03:48:23.111 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:23 +0000 (0:00:00.211) 0:01:37.585 *********** 2021-07-11T03:48:23.178 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-07-11T03:48:23.216 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:23 +0000 (0:00:00.105) 0:01:37.691 *********** 2021-07-11T03:48:23.240 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-07-11T03:48:23.272 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:23 +0000 (0:00:00.055) 0:01:37.746 *********** 2021-07-11T03:48:23.397 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-07-11T03:48:23.398 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:48:23.422 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:23 +0000 (0:00:00.150) 0:01:37.897 *********** 2021-07-11T03:48:23.493 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-07-11T03:48:23.520 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:23 +0000 (0:00:00.097) 0:01:37.995 *********** 2021-07-11T03:48:23.601 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-07-11T03:48:23.602 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:48:23.629 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:23 +0000 (0:00:00.108) 0:01:38.103 *********** 2021-07-11T03:48:23.670 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-07-11T03:48:23.672 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.011345", "end": "2021-07-11 03:48:23.874746", "rc": 0, "start": "2021-07-11 03:48:23.863401", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:23.697 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:23 +0000 (0:00:00.068) 0:01:38.172 *********** 2021-07-11T03:48:23.705 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-07-11T03:48:23.706 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} TASK [testnode : Add local apt repos.] ***************************************** 2021-07-11T03:48:23.732 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:23 +0000 (0:00:00.034) 0:01:38.207 *********** 2021-07-11T03:48:23.757 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:23 +0000 (0:00:00.024) 0:01:38.232 *********** 2021-07-11T03:48:23.765 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-07-11T03:48:23.769 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} 2021-07-11T03:48:23.793 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:23 +0000 (0:00:00.036) 0:01:38.268 *********** 2021-07-11T03:48:23.814 INFO:teuthology.task.ansible.out: 2021-07-11T03:48:23.815 INFO:teuthology.task.ansible.out:TASK [testnode : Add local apt repos.] ***************************************** 2021-07-11T03:48:23.841 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:23 +0000 (0:00:00.047) 0:01:38.316 *********** 2021-07-11T03:48:24.054 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:24 +0000 (0:00:00.212) 0:01:38.528 *********** 2021-07-11T03:48:24.074 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-07-11T03:48:24.075 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.011451", "end": "2021-07-11 03:48:24.006260", "rc": 0, "start": "2021-07-11 03:48:23.994809", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:24.106 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:24 +0000 (0:00:00.052) 0:01:38.581 *********** 2021-07-11T03:48:24.200 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.011326", "end": "2021-07-11 03:48:24.635060", "rc": 0, "start": "2021-07-11 03:48:24.623734", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:24.229 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:24 +0000 (0:00:00.122) 0:01:38.704 *********** 2021-07-11T03:48:24.364 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.011292", "end": "2021-07-11 03:48:24.701611", "rc": 0, "start": "2021-07-11 03:48:24.690319", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:48:24.387 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:24 +0000 (0:00:00.158) 0:01:38.862 *********** 2021-07-11T03:48:25.250 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-07-11T03:48:25.253 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:48:25.275 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:25 +0000 (0:00:00.887) 0:01:39.749 *********** 2021-07-11T03:48:25.876 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:48:25.903 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:25 +0000 (0:00:00.627) 0:01:40.377 *********** 2021-07-11T03:48:25.946 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:48:25.975 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:25 +0000 (0:00:00.072) 0:01:40.450 *********** 2021-07-11T03:48:26.101 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} 2021-07-11T03:48:26.127 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:26 +0000 (0:00:00.150) 0:01:40.601 *********** 2021-07-11T03:48:35.730 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-07-11T03:48:35.732 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0\n libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla\n libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0\n libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf\n qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 152 not upgraded.\nAfter this operation, 62.7 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138110 files and directories currently installed.)\r\ 2021-07-11T03:48:35.735 INFO:teuthology.task.ansible.out:nRemoving qemu-utils (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...\r\nRemoving librbd1 (15.2.8-0ubuntu0.20.04.1) ...\r\nRemoving librados2 (15.2.8-0ubuntu0.20.04.1) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0", " libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla", " libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0", " libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf", " qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 152 not upgraded.", "After this operation, 62.7 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Rea 2021-07-11T03:48:35.738 INFO:teuthology.task.ansible.out:ding database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138110 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-common (1:4.2-3ubuntu6.14) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...", "Removing librbd1 (15.2.8-0ubuntu0.20.04.1) ...", "Removing librados2 (15.2.8-0ubuntu0.20.04.1) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-07-11T03:48:35.757 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:35 +0000 (0:00:09.630) 0:01:50.232 *********** 2021-07-11T03:48:35.871 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0\n libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla\n libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0\n libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf\n qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 152 not upgraded.\nAfter this operation, 62.7 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138110 files and directories currently installed.)\r\ 2021-07-11T03:48:35.873 INFO:teuthology.task.ansible.out:nRemoving qemu-utils (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...\r\nRemoving librbd1 (15.2.8-0ubuntu0.20.04.1) ...\r\nRemoving librados2 (15.2.8-0ubuntu0.20.04.1) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0", " libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla", " libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0", " libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf", " qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 152 not upgraded.", "After this operation, 62.7 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Rea 2021-07-11T03:48:35.874 INFO:teuthology.task.ansible.out:ding database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138110 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-common (1:4.2-3ubuntu6.14) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...", "Removing librbd1 (15.2.8-0ubuntu0.20.04.1) ...", "Removing librados2 (15.2.8-0ubuntu0.20.04.1) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-07-11T03:48:35.900 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:35 +0000 (0:00:00.143) 0:01:50.375 *********** 2021-07-11T03:48:36.188 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0\n libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla\n libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0\n libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf\n qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 152 not upgraded.\nAfter this operation, 62.7 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138110 files and directories currently installed.)\r\ 2021-07-11T03:48:36.190 INFO:teuthology.task.ansible.out:nRemoving qemu-utils (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...\r\nRemoving librbd1 (15.2.8-0ubuntu0.20.04.1) ...\r\nRemoving librados2 (15.2.8-0ubuntu0.20.04.1) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0", " libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla", " libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0", " libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf", " qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 152 not upgraded.", "After this operation, 62.7 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Rea 2021-07-11T03:48:36.192 INFO:teuthology.task.ansible.out:ding database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138110 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-common (1:4.2-3ubuntu6.14) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...", "Removing librbd1 (15.2.8-0ubuntu0.20.04.1) ...", "Removing librados2 (15.2.8-0ubuntu0.20.04.1) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-07-11T03:48:36.219 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:36 +0000 (0:00:00.313) 0:01:50.689 *********** 2021-07-11T03:48:36.614 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0\n libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla\n libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0\n libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf\n qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 152 not upgraded.\nAfter this operation, 62.7 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138110 files and directories currently installed.)\r\ 2021-07-11T03:48:36.616 INFO:teuthology.task.ansible.out:nRemoving qemu-utils (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-x86 (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-system-common (1:4.2-3ubuntu6.14) ...\r\nRemoving qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...\r\nRemoving librbd1 (15.2.8-0ubuntu0.20.04.1) ...\r\nRemoving librados2 (15.2.8-0ubuntu0.20.04.1) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0", " libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla", " libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0", " libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf", " qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 152 not upgraded.", "After this operation, 62.7 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Rea 2021-07-11T03:48:36.617 INFO:teuthology.task.ansible.out:ding database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138110 files and directories currently installed.)", "Removing qemu-utils (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-x86 (1:4.2-3ubuntu6.14) ...", "Removing qemu-system-common (1:4.2-3ubuntu6.14) ...", "Removing qemu-block-extra:amd64 (1:4.2-3ubuntu6.14) ...", "Removing librbd1 (15.2.8-0ubuntu0.20.04.1) ...", "Removing librados2 (15.2.8-0ubuntu0.20.04.1) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-07-11T03:48:36.638 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:36 +0000 (0:00:00.424) 0:01:51.113 *********** 2021-07-11T03:48:56.351 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-07-11T03:48:56.354 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0\n libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla\n libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0\n libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf\n qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apache2-bin apache2-data apache2-utils apt-utils libapt-pkg6.0\nSuggested packages:\n apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser\n apt-doc aptitude | synaptic | wajig\nThe following packages will be upgraded:\n apache2 apache2-bin apache2-data apache2-utils apt apt-utils libapt-pkg6.0\n7 upgraded, 0 newly installed, 0 to remove and 145 not upgraded.\nNeed to get 3865 kB of archives.\nAfter this operation, 42.0 kB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.6 [835 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.6 [1296 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.6 [216 kB]\nG 2021-07-11T03:48:56.355 INFO:teuthology.task.ansible.out:et:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.3 [95.5 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.3 [1179 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.3 [159 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.3 [84.0 kB]\nFetched 3865 kB in 1s (4076 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../libapt-pkg6.0_2.0.6_amd64.deb ...\r\nUnpacking libapt-pkg6.0:amd64 (2.0.6) over (2.0.4) ...\r\nSetting up libapt-pkg6.0:amd64 (2.0.6) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../archives/apt_2.0.6_amd64.deb ...\r\nUnpacking apt (2.0.6) 2021-07-11T03:48:56.357 INFO:teuthology.task.ansible.out: over (2.0.4) ...\r\nSetting up apt (2.0.6) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../apt-utils_2.0.6_amd64.deb ...\r\nUnpacking apt-utils (2.0.6) over (2.0.4) ...\r\nPreparing to unpack .../apache2_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2 (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-bin_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2-bin (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-data_2.4.41-4ubuntu3.3_all.deb ...\r\nUnpacking apache2-data (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-utils_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2-utils (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nSetting up apt-utils (2.0.6) ...\r\nSetting up apache2-bin (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2-data (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2-utils (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2 (2.4.41-4ubuntu3.3) ...\r\nProcessing triggers for ufw (0.36-6) ...\r\nProcessing triggers for systemd (245.4-4ubuntu3.4) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer req 2021-07-11T03:48:56.358 INFO:teuthology.task.ansible.out:uired:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0", " libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla", " libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0", " libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf", " qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apache2-bin apache2-data apache2-utils apt-utils libapt-pkg6.0", "Suggested packages:", " apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser", " apt-doc aptitude | synaptic | wajig", "The following packages will be upgraded:", " apache2 apache2-bin apache2-data apache2-utils apt apt-utils libapt-pkg6.0", "7 upgraded, 0 newly installed, 0 to remove and 145 not upgraded.", "Need to get 3865 kB of archives.", "After this operation, 42.0 kB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.6 [835 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.6 [1296 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.6 [216 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.3 [95.5 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.3 [1179 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.3 [159 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.3 [84.0 kB]" 2021-07-11T03:48:56.360 INFO:teuthology.task.ansible.out:, "Fetched 3865 kB in 1s (4076 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../libapt-pkg6.0_2.0.6_amd64.deb ...", "Unpacking libapt-pkg6.0:amd64 (2.0.6) over (2.0.4) ...", "Setting up libapt-pkg6.0:amd64 (2.0.6) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../archives/apt_2.0.6_amd64.deb ...", "Unpacking apt (2.0.6) over (2.0.4) ...", "Setting up apt (2.0.6) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading data 2021-07-11T03:48:56.362 INFO:teuthology.task.ansible.out:base ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../apt-utils_2.0.6_amd64.deb ...", "Unpacking apt-utils (2.0.6) over (2.0.4) ...", "Preparing to unpack .../apache2_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2 (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-bin_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2-bin (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-data_2.4.41-4ubuntu3.3_all.deb ...", "Unpacking apache2-data (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-utils_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2-utils (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Setting up apt-utils (2.0.6) ...", "Setting up apache2-bin (2.4.41-4ubuntu3.3) ...", "Setting up apache2-data (2.4.41-4ubuntu3.3) ...", "Setting up apache2-utils (2.4.41-4ubuntu3.3) ...", "Setting up apache2 (2.4.41-4ubuntu3.3) ...", "Processing triggers for ufw (0.36-6) ...", "Processing triggers for systemd (245.4-4ubuntu3.4) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-07-11T03:48:56.383 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:56 +0000 (0:00:19.744) 0:02:10.857 *********** 2021-07-11T03:48:56.474 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:56.494 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:56 +0000 (0:00:00.111) 0:02:10.969 *********** 2021-07-11T03:48:57.748 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* changed: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0\n libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla\n libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0\n libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf\n qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apache2-bin apache2-data apache2-utils apt-utils libapt-pkg6.0\nSuggested packages:\n apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser\n apt-doc aptitude | synaptic | wajig\nThe following packages will be upgraded:\n apache2 apache2-bin apache2-data apache2-utils apt apt-utils libapt-pkg6.0\n7 upgraded, 0 newly installed, 0 to remove and 145 not upgraded.\nNeed to get 3865 kB of archives.\nAfter this operation, 42.0 kB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.6 [835 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.6 [1296 kB]\nGet:3 http:// 2021-07-11T03:48:57.750 INFO:teuthology.task.ansible.out:archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.6 [216 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.3 [95.5 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.3 [1179 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.3 [159 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.3 [84.0 kB]\nFetched 3865 kB in 1s (4038 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../libapt-pkg6.0_2.0.6_amd64.deb ...\r\nUnpacking libapt-pkg6.0:amd64 (2.0.6) over (2.0.4) ...\r\nSetting up libapt-pkg6.0:amd64 (2.0.6) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\ 2021-07-11T03:48:57.752 INFO:teuthology.task.ansible.out:r\nPreparing to unpack .../archives/apt_2.0.6_amd64.deb ...\r\nUnpacking apt (2.0.6) over (2.0.4) ...\r\nSetting up apt (2.0.6) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../apt-utils_2.0.6_amd64.deb ...\r\nUnpacking apt-utils (2.0.6) over (2.0.4) ...\r\nPreparing to unpack .../apache2_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2 (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-bin_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2-bin (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-data_2.4.41-4ubuntu3.3_all.deb ...\r\nUnpacking apache2-data (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-utils_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2-utils (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nSetting up apt-utils (2.0.6) ...\r\nSetting up apache2-bin (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2-data (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2-utils (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2 (2.4.41-4ubuntu3.3) ...\r\nProcessing triggers for ufw (0.36-6) ...\r\nProcessing triggers for systemd (245.4-4ubuntu3.4) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state informa 2021-07-11T03:48:57.753 INFO:teuthology.task.ansible.out:tion...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0", " libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla", " libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0", " libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf", " qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apache2-bin apache2-data apache2-utils apt-utils libapt-pkg6.0", "Suggested packages:", " apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser", " apt-doc aptitude | synaptic | wajig", "The following packages will be upgraded:", " apache2 apache2-bin apache2-data apache2-utils apt apt-utils libapt-pkg6.0", "7 upgraded, 0 newly installed, 0 to remove and 145 not upgraded.", "Need to get 3865 kB of archives.", "After this operation, 42.0 kB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.6 [835 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.6 [1296 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.6 [216 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.3 [95.5 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.3 [1179 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.3 [159 kB]", "Get:7 http://archive.ubuntu. 2021-07-11T03:48:57.755 INFO:teuthology.task.ansible.out:com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.3 [84.0 kB]", "Fetched 3865 kB in 1s (4038 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../libapt-pkg6.0_2.0.6_amd64.deb ...", "Unpacking libapt-pkg6.0:amd64 (2.0.6) over (2.0.4) ...", "Setting up libapt-pkg6.0:amd64 (2.0.6) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../archives/apt_2.0.6_amd64.deb ...", "Unpacking apt (2.0.6) over (2.0.4) ...", "Setting up apt (2.0.6) ...", "(Reading database ... ", "(R 2021-07-11T03:48:57.757 INFO:teuthology.task.ansible.out:eading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../apt-utils_2.0.6_amd64.deb ...", "Unpacking apt-utils (2.0.6) over (2.0.4) ...", "Preparing to unpack .../apache2_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2 (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-bin_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2-bin (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-data_2.4.41-4ubuntu3.3_all.deb ...", "Unpacking apache2-data (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-utils_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2-utils (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Setting up apt-utils (2.0.6) ...", "Setting up apache2-bin (2.4.41-4ubuntu3.3) ...", "Setting up apache2-data (2.4.41-4ubuntu3.3) ...", "Setting up apache2-utils (2.4.41-4ubuntu3.3) ...", "Setting up apache2 (2.4.41-4ubuntu3.3) ...", "Processing triggers for ufw (0.36-6) ...", "Processing triggers for systemd (245.4-4ubuntu3.4) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-07-11T03:48:57.784 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:57 +0000 (0:00:01.289) 0:02:12.258 *********** 2021-07-11T03:48:57.797 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0\n libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla\n libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0\n libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf\n qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apache2-bin apache2-data apache2-utils apt-utils libapt-pkg6.0\nSuggested packages:\n apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser\n apt-doc aptitude | synaptic | wajig\nThe following packages will be upgraded:\n apache2 apache2-bin apache2-data apache2-utils apt apt-utils libapt-pkg6.0\n7 upgraded, 0 newly installed, 0 to remove and 145 not upgraded.\nNeed to get 3865 kB of archives.\nAfter this operation, 42.0 kB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.6 [835 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.6 [1296 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.6 [216 kB]\nG 2021-07-11T03:48:57.798 INFO:teuthology.task.ansible.out:et:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.3 [95.5 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.3 [1179 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.3 [159 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.3 [84.0 kB]\nFetched 3865 kB in 1s (4087 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../libapt-pkg6.0_2.0.6_amd64.deb ...\r\nUnpacking libapt-pkg6.0:amd64 (2.0.6) over (2.0.4) ...\r\nSetting up libapt-pkg6.0:amd64 (2.0.6) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../archives/apt_2.0.6_amd64.deb ...\r\nUnpacking apt (2.0.6) 2021-07-11T03:48:57.800 INFO:teuthology.task.ansible.out: over (2.0.4) ...\r\nSetting up apt (2.0.6) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../apt-utils_2.0.6_amd64.deb ...\r\nUnpacking apt-utils (2.0.6) over (2.0.4) ...\r\nPreparing to unpack .../apache2_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2 (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-bin_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2-bin (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-data_2.4.41-4ubuntu3.3_all.deb ...\r\nUnpacking apache2-data (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-utils_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2-utils (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nSetting up apt-utils (2.0.6) ...\r\nSetting up apache2-bin (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2-data (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2-utils (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2 (2.4.41-4ubuntu3.3) ...\r\nProcessing triggers for ufw (0.36-6) ...\r\nProcessing triggers for systemd (245.4-4ubuntu3.4) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer req 2021-07-11T03:48:57.802 INFO:teuthology.task.ansible.out:uired:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0", " libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla", " libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0", " libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf", " qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apache2-bin apache2-data apache2-utils apt-utils libapt-pkg6.0", "Suggested packages:", " apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser", " apt-doc aptitude | synaptic | wajig", "The following packages will be upgraded:", " apache2 apache2-bin apache2-data apache2-utils apt apt-utils libapt-pkg6.0", "7 upgraded, 0 newly installed, 0 to remove and 145 not upgraded.", "Need to get 3865 kB of archives.", "After this operation, 42.0 kB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.6 [835 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.6 [1296 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.6 [216 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.3 [95.5 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.3 [1179 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.3 [159 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.3 [84.0 kB]" 2021-07-11T03:48:57.804 INFO:teuthology.task.ansible.out:, "Fetched 3865 kB in 1s (4087 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../libapt-pkg6.0_2.0.6_amd64.deb ...", "Unpacking libapt-pkg6.0:amd64 (2.0.6) over (2.0.4) ...", "Setting up libapt-pkg6.0:amd64 (2.0.6) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../archives/apt_2.0.6_amd64.deb ...", "Unpacking apt (2.0.6) over (2.0.4) ...", "Setting up apt (2.0.6) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading data 2021-07-11T03:48:57.806 INFO:teuthology.task.ansible.out:base ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../apt-utils_2.0.6_amd64.deb ...", "Unpacking apt-utils (2.0.6) over (2.0.4) ...", "Preparing to unpack .../apache2_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2 (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-bin_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2-bin (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-data_2.4.41-4ubuntu3.3_all.deb ...", "Unpacking apache2-data (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-utils_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2-utils (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Setting up apt-utils (2.0.6) ...", "Setting up apache2-bin (2.4.41-4ubuntu3.3) ...", "Setting up apache2-data (2.4.41-4ubuntu3.3) ...", "Setting up apache2-utils (2.4.41-4ubuntu3.3) ...", "Setting up apache2 (2.4.41-4ubuntu3.3) ...", "Processing triggers for ufw (0.36-6) ...", "Processing triggers for systemd (245.4-4ubuntu3.4) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-07-11T03:48:57.830 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:57 +0000 (0:00:00.045) 0:02:12.304 *********** 2021-07-11T03:48:57.862 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-07-11T03:48:57.864 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:57.890 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:57 +0000 (0:00:00.059) 0:02:12.364 *********** 2021-07-11T03:48:57.908 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:57.934 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:57 +0000 (0:00:00.044) 0:02:12.409 *********** 2021-07-11T03:48:58.942 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-07-11T03:48:58.944 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1\n libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0\n libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0\n libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0\n libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla\n libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0\n libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf\n qemu-system-data qemu-system-gui seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following additional packages will be installed:\n apache2-bin apache2-data apache2-utils apt-utils libapt-pkg6.0\nSuggested packages:\n apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser\n apt-doc aptitude | synaptic | wajig\nThe following packages will be upgraded:\n apache2 apache2-bin apache2-data apache2-utils apt apt-utils libapt-pkg6.0\n7 upgraded, 0 newly installed, 0 to remove and 145 not upgraded.\nNeed to get 3865 kB of archives.\nAfter this operation, 42.0 kB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.6 [835 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.6 [1296 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.6 [216 kB]\nG 2021-07-11T03:48:58.946 INFO:teuthology.task.ansible.out:et:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.3 [95.5 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.3 [1179 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.3 [159 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.3 [84.0 kB]\nFetched 3865 kB in 1s (4127 kB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../libapt-pkg6.0_2.0.6_amd64.deb ...\r\nUnpacking libapt-pkg6.0:amd64 (2.0.6) over (2.0.4) ...\r\nSetting up libapt-pkg6.0:amd64 (2.0.6) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../archives/apt_2.0.6_amd64.deb ...\r\nUnpacking apt (2.0.6) 2021-07-11T03:48:58.947 INFO:teuthology.task.ansible.out: over (2.0.4) ...\r\nSetting up apt (2.0.6) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../apt-utils_2.0.6_amd64.deb ...\r\nUnpacking apt-utils (2.0.6) over (2.0.4) ...\r\nPreparing to unpack .../apache2_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2 (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-bin_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2-bin (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-data_2.4.41-4ubuntu3.3_all.deb ...\r\nUnpacking apache2-data (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nPreparing to unpack .../apache2-utils_2.4.41-4ubuntu3.3_amd64.deb ...\r\nUnpacking apache2-utils (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...\r\nSetting up apt-utils (2.0.6) ...\r\nSetting up apache2-bin (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2-data (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2-utils (2.4.41-4ubuntu3.3) ...\r\nSetting up apache2 (2.4.41-4ubuntu3.3) ...\r\nProcessing triggers for ufw (0.36-6) ...\r\nProcessing triggers for systemd (245.4-4ubuntu3.4) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer req 2021-07-11T03:48:58.950 INFO:teuthology.task.ansible.out:uired:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1", " libboost-iostreams1.71.0 libboost-thread1.71.0 libbrlapi0.7 libcacard0", " libcdparanoia0 libdv4 libgbm1 libgstreamer-plugins-base1.0-0", " libgstreamer-plugins-good1.0-0 libgudev-1.0-0 libiscsi7 liborc-0.4-0", " libpmem1 libshout3 libslirp0 libspice-server1 libtag1v5 libtag1v5-vanilla", " libusbredirparser1 libv4l-0 libv4lconvert0 libvirglrenderer1 libvisual-0.4-0", " libvte-2.91-0 libvte-2.91-common libwayland-server0 msr-tools ovmf", " qemu-system-data qemu-system-gui seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following additional packages will be installed:", " apache2-bin apache2-data apache2-utils apt-utils libapt-pkg6.0", "Suggested packages:", " apache2-doc apache2-suexec-pristine | apache2-suexec-custom www-browser", " apt-doc aptitude | synaptic | wajig", "The following packages will be upgraded:", " apache2 apache2-bin apache2-data apache2-utils apt apt-utils libapt-pkg6.0", "7 upgraded, 0 newly installed, 0 to remove and 145 not upgraded.", "Need to get 3865 kB of archives.", "After this operation, 42.0 kB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libapt-pkg6.0 amd64 2.0.6 [835 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt amd64 2.0.6 [1296 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apt-utils amd64 2.0.6 [216 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2 amd64 2.4.41-4ubuntu3.3 [95.5 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-bin amd64 2.4.41-4ubuntu3.3 [1179 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-data all 2.4.41-4ubuntu3.3 [159 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 apache2-utils amd64 2.4.41-4ubuntu3.3 [84.0 kB]" 2021-07-11T03:48:58.952 INFO:teuthology.task.ansible.out:, "Fetched 3865 kB in 1s (4127 kB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../libapt-pkg6.0_2.0.6_amd64.deb ...", "Unpacking libapt-pkg6.0:amd64 (2.0.6) over (2.0.4) ...", "Setting up libapt-pkg6.0:amd64 (2.0.6) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../archives/apt_2.0.6_amd64.deb ...", "Unpacking apt (2.0.6) over (2.0.4) ...", "Setting up apt (2.0.6) ...", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading data 2021-07-11T03:48:58.956 INFO:teuthology.task.ansible.out:base ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../apt-utils_2.0.6_amd64.deb ...", "Unpacking apt-utils (2.0.6) over (2.0.4) ...", "Preparing to unpack .../apache2_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2 (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-bin_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2-bin (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-data_2.4.41-4ubuntu3.3_all.deb ...", "Unpacking apache2-data (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Preparing to unpack .../apache2-utils_2.4.41-4ubuntu3.3_amd64.deb ...", "Unpacking apache2-utils (2.4.41-4ubuntu3.3) over (2.4.41-4ubuntu3.1) ...", "Setting up apt-utils (2.0.6) ...", "Setting up apache2-bin (2.4.41-4ubuntu3.3) ...", "Setting up apache2-data (2.4.41-4ubuntu3.3) ...", "Setting up apache2-utils (2.4.41-4ubuntu3.3) ...", "Setting up apache2 (2.4.41-4ubuntu3.3) ...", "Processing triggers for ufw (0.36-6) ...", "Processing triggers for systemd (245.4-4ubuntu3.4) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ..."]} 2021-07-11T03:48:58.974 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:58 +0000 (0:00:01.039) 0:02:13.449 *********** 2021-07-11T03:48:59.059 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-07-11T03:48:59.061 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:48:59.085 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:48:59 +0000 (0:00:00.110) 0:02:13.559 *********** 2021-07-11T03:49:07.876 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-07-11T03:49:07.877 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git qemu-system-data qemu-system-gui\n3 upgraded, 6 newly installed, 0 to remove and 142 not upgraded.\nNeed to get 18.8 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.12-0ubuntu0.20.04.1 [3216 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.12-0ubuntu0.20.04.1 [1619 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.16 [53.4 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.16 [1055 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.16 [562 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.16 [39.9 kB]\nGet:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x8 2021-07-11T03:49:07.878 INFO:teuthology.task.ansible.out:6 amd64 1:4.2-3ubuntu6.16 [6722 kB]\nGet:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.16 [973 kB]\nFetched 18.8 MB in 1s (14.3 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...\r\nUnpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...\r\nSelecting previously unselected package librados2.\r\nPreparing to unpack .../1-librados2_15.2.12-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librados2 (15.2.12-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../2-librbd1_15.2.12-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librbd1 (15.2.12-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6.16) ...\r\nPreparing to unpack .../5-qemu-system-data_1%3a4.2-3ubuntu6.16_all.deb ...\r\nUnpacking qemu-system-data (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...\r\nPreparing to unpack .../6-qemu-system-gui_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-gui:amd64 2021-07-11T03:49:07.880 INFO:teuthology.task.ansible.out:(1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../7-qemu-system-x86_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6.16) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../8-qemu-utils_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6.16) ...\r\nSetting up librados2 (15.2.12-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) ...\r\nSetting up git (1:2.25.1-1ubuntu3.1) ...\r\nSetting up librbd1 (15.2.12-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-data (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6.16) ...\r\nProcessing triggers for hicolor-icon-theme (0.17-2) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for mime-support (3.64ubuntu1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data", " qemu-system-gui qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git qemu-system-data qemu-system-gui", "3 upgraded, 6 newly installed, 0 to remove and 142 not upgraded.", "Need to get 18.8 MB of archives.", "After this operation, 62.7 MB of additional disk space will be used.", "Get:1 http://archive.u 2021-07-11T03:49:07.881 INFO:teuthology.task.ansible.out:buntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.12-0ubuntu0.20.04.1 [3216 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.12-0ubuntu0.20.04.1 [1619 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.16 [53.4 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.16 [1055 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.16 [562 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.16 [39.9 kB]", "Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.16 [6722 kB]", "Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.16 [973 kB]", "Fetched 18.8 MB in 1s (14.3 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...", "Unpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...", "Selecting previously unselected package librados2.", "Preparing to unpack .../1-librados2_15.2.12-0ubuntu0.20.04.1_amd64.deb .. 2021-07-11T03:49:07.883 INFO:teuthology.task.ansible.out:.", "Unpacking librados2 (15.2.12-0ubuntu0.20.04.1) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../2-librbd1_15.2.12-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librbd1 (15.2.12-0ubuntu0.20.04.1) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6.16) ...", "Preparing to unpack .../5-qemu-system-data_1%3a4.2-3ubuntu6.16_all.deb ...", "Unpacking qemu-system-data (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...", "Preparing to unpack .../6-qemu-system-gui_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../7-qemu-system-x86_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6.16) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../8-qemu-utils_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6.16) ...", "Setting up librados2 (15.2.12-0ubuntu0.20.04.1) ...", "Setting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) ...", "Setting up git (1:2.25.1-1ubuntu3.1) ...", "Setting up librbd1 (15.2.12-0ubuntu0.20.04.1) ...", "Setting up qemu-system-data (1:4.2-3ubuntu6.16) ...", "Setting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6.16) ...", "Setting up qemu-system-x86 (1:4.2-3ubuntu6.16) ...", "Setting up qemu-utils (1:4.2-3ubuntu6.16) ...", "Processing triggers for hicolor-icon-theme (0.17-2) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing 2021-07-11T03:49:07.885 INFO:teuthology.task.ansible.out:triggers for mime-support (3.64ubuntu1) ..."]} 2021-07-11T03:49:07.903 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:07 +0000 (0:00:08.818) 0:02:22.378 *********** 2021-07-11T03:49:08.375 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-07-11T03:49:08.377 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:49:08.400 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:08 +0000 (0:00:00.496) 0:02:22.874 *********** 2021-07-11T03:49:08.900 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-07-11T03:49:08.902 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:49:08.927 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:08 +0000 (0:00:00.524) 0:02:23.399 *********** 2021-07-11T03:49:09.082 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-07-11T03:49:09.084 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-07-11T03:49:09.107 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:09 +0000 (0:00:00.182) 0:02:23.582 *********** 2021-07-11T03:49:09.179 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-07-11T03:49:09.181 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:09.204 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:09 +0000 (0:00:00.096) 0:02:23.678 *********** 2021-07-11T03:49:09.385 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-07-11T03:49:09.411 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:09 +0000 (0:00:00.205) 0:02:23.884 *********** 2021-07-11T03:49:09.734 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-07-11T03:49:09.735 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-07-11T03:49:09.758 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:09 +0000 (0:00:00.349) 0:02:24.233 *********** 2021-07-11T03:49:09.938 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-07-11T03:49:09.940 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-07-11T03:49:09.969 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:09 +0000 (0:00:00.210) 0:02:24.444 *********** 2021-07-11T03:49:10.020 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-07-11T03:49:10.021 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git qemu-system-data qemu-system-gui\n3 upgraded, 6 newly installed, 0 to remove and 142 not upgraded.\nNeed to get 18.8 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.12-0ubuntu0.20.04.1 [3216 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.12-0ubuntu0.20.04.1 [1619 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.16 [53.4 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.16 [1055 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.16 [562 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.16 [39.9 kB]\nGet:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x8 2021-07-11T03:49:10.023 INFO:teuthology.task.ansible.out:6 amd64 1:4.2-3ubuntu6.16 [6722 kB]\nGet:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.16 [973 kB]\nFetched 18.8 MB in 1s (13.3 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...\r\nUnpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...\r\nSelecting previously unselected package librados2.\r\nPreparing to unpack .../1-librados2_15.2.12-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librados2 (15.2.12-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../2-librbd1_15.2.12-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librbd1 (15.2.12-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6.16) ...\r\nPreparing to unpack .../5-qemu-system-data_1%3a4.2-3ubuntu6.16_all.deb ...\r\nUnpacking qemu-system-data (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...\r\nPreparing to unpack .../6-qemu-system-gui_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-gui:amd64 2021-07-11T03:49:10.024 INFO:teuthology.task.ansible.out:(1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../7-qemu-system-x86_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6.16) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../8-qemu-utils_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6.16) ...\r\nSetting up librados2 (15.2.12-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) ...\r\nSetting up git (1:2.25.1-1ubuntu3.1) ...\r\nSetting up librbd1 (15.2.12-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-data (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6.16) ...\r\nProcessing triggers for hicolor-icon-theme (0.17-2) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for mime-support (3.64ubuntu1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data", " qemu-system-gui qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git qemu-system-data qemu-system-gui", "3 upgraded, 6 newly installed, 0 to remove and 142 not upgraded.", "Need to get 18.8 MB of archives.", "After this operation, 62.7 MB of additional disk space will be used.", "Get:1 http://archive.u 2021-07-11T03:49:10.026 INFO:teuthology.task.ansible.out:buntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.12-0ubuntu0.20.04.1 [3216 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.12-0ubuntu0.20.04.1 [1619 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.16 [53.4 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.16 [1055 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.16 [562 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.16 [39.9 kB]", "Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.16 [6722 kB]", "Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.16 [973 kB]", "Fetched 18.8 MB in 1s (13.3 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...", "Unpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...", "Selecting previously unselected package librados2.", "Preparing to unpack .../1-librados2_15.2.12-0ubuntu0.20.04.1_amd64.deb .. 2021-07-11T03:49:10.027 INFO:teuthology.task.ansible.out:.", "Unpacking librados2 (15.2.12-0ubuntu0.20.04.1) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../2-librbd1_15.2.12-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librbd1 (15.2.12-0ubuntu0.20.04.1) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6.16) ...", "Preparing to unpack .../5-qemu-system-data_1%3a4.2-3ubuntu6.16_all.deb ...", "Unpacking qemu-system-data (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...", "Preparing to unpack .../6-qemu-system-gui_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../7-qemu-system-x86_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6.16) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../8-qemu-utils_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6.16) ...", "Setting up librados2 (15.2.12-0ubuntu0.20.04.1) ...", "Setting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) ...", "Setting up git (1:2.25.1-1ubuntu3.1) ...", "Setting up librbd1 (15.2.12-0ubuntu0.20.04.1) ...", "Setting up qemu-system-data (1:4.2-3ubuntu6.16) ...", "Setting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6.16) ...", "Setting 2021-07-11T03:49:10.027 INFO:teuthology.task.ansible.out:up qemu-system-x86 (1:4.2-3ubuntu6.16) ...", "Setting up qemu-utils (1:4.2-3ubuntu6.16) ...", "Processing triggers for hicolor-icon-theme (0.17-2) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing 2021-07-11T03:49:10.029 INFO:teuthology.task.ansible.out:triggers for mime-support (3.64ubuntu1) ..."]} 2021-07-11T03:49:10.055 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:10 +0000 (0:00:00.085) 0:02:24.529 *********** 2021-07-11T03:49:10.076 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git qemu-system-data qemu-system-gui\n3 upgraded, 6 newly installed, 0 to remove and 142 not upgraded.\nNeed to get 18.8 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.12-0ubuntu0.20.04.1 [3216 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.12-0ubuntu0.20.04.1 [1619 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.16 [53.4 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.16 [1055 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.16 [562 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.16 [39.9 kB]\nGet:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x8 2021-07-11T03:49:10.077 INFO:teuthology.task.ansible.out:6 amd64 1:4.2-3ubuntu6.16 [6722 kB]\nGet:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.16 [973 kB]\nFetched 18.8 MB in 1s (13.2 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...\r\nUnpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...\r\nSelecting previously unselected package librados2.\r\nPreparing to unpack .../1-librados2_15.2.12-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librados2 (15.2.12-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../2-librbd1_15.2.12-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librbd1 (15.2.12-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6.16) ...\r\nPreparing to unpack .../5-qemu-system-data_1%3a4.2-3ubuntu6.16_all.deb ...\r\nUnpacking qemu-system-data (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...\r\nPreparing to unpack .../6-qemu-system-gui_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-gui:amd64 2021-07-11T03:49:10.079 INFO:teuthology.task.ansible.out:(1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../7-qemu-system-x86_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6.16) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../8-qemu-utils_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6.16) ...\r\nSetting up librados2 (15.2.12-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) ...\r\nSetting up git (1:2.25.1-1ubuntu3.1) ...\r\nSetting up librbd1 (15.2.12-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-data (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6.16) ...\r\nProcessing triggers for hicolor-icon-theme (0.17-2) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for mime-support (3.64ubuntu1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data", " qemu-system-gui qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git qemu-system-data qemu-system-gui", "3 upgraded, 6 newly installed, 0 to remove and 142 not upgraded.", "Need to get 18.8 MB of archives.", "After this operation, 62.7 MB of additional disk space will be used.", "Get:1 http://archive.u 2021-07-11T03:49:10.081 INFO:teuthology.task.ansible.out:buntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.12-0ubuntu0.20.04.1 [3216 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.12-0ubuntu0.20.04.1 [1619 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.16 [53.4 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.16 [1055 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.16 [562 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.16 [39.9 kB]", "Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.16 [6722 kB]", "Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.16 [973 kB]", "Fetched 18.8 MB in 1s (13.2 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...", "Unpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...", "Selecting previously unselected package librados2.", "Preparing to unpack .../1-librados2_15.2.12-0ubuntu0.20.04.1_amd64.deb .. 2021-07-11T03:49:10.082 INFO:teuthology.task.ansible.out:.", "Unpacking librados2 (15.2.12-0ubuntu0.20.04.1) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../2-librbd1_15.2.12-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librbd1 (15.2.12-0ubuntu0.20.04.1) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6.16) ...", "Preparing to unpack .../5-qemu-system-data_1%3a4.2-3ubuntu6.16_all.deb ...", "Unpacking qemu-system-data (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...", "Preparing to unpack .../6-qemu-system-gui_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../7-qemu-system-x86_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6.16) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../8-qemu-utils_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6.16) ...", "Setting up librados2 (15.2.12-0ubuntu0.20.04.1) ...", "Setting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) ...", "Setting up git (1:2.25.1-1ubuntu3.1) ...", "Setting up librbd1 (15.2.12-0ubuntu0.20.04.1) ...", "Setting up qemu-system-data (1:4.2-3ubuntu6.16) ...", "Setting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6.16) ...", "Setting up qemu-system-x86 (1:4.2-3ubuntu6.16) ...", "Setting up qemu-utils (1:4.2-3ubuntu6.16) ...", "Processing triggers for hicolor-icon-theme (0.17-2) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing 2021-07-11T03:49:10.084 INFO:teuthology.task.ansible.out:triggers for mime-support (3.64ubuntu1) ..."]} 2021-07-11T03:49:10.106 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:10 +0000 (0:00:00.051) 0:02:24.580 *********** 2021-07-11T03:49:10.132 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-07-11T03:49:10.134 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:49:10.160 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:10 +0000 (0:00:00.053) 0:02:24.634 *********** 2021-07-11T03:49:10.408 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:48:51 UTC", "ActiveEnterTimestampMonotonic": "161162921", "ActiveExitTimestamp": "Sun 2021-07-11 03:48:51 UTC", "ActiveExitTimestampMonotonic": "161120144", "ActiveState": "active", "After": "-.mount systemd-tmpfiles-setup.service sysinit.target remote-fs.target system.slice nss-lookup.target systemd-journald.socket basic.target network.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:48:51 UTC", "AssertTimestampMonotonic": "161151156", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-0 2021-07-11T03:49:10.410 INFO:teuthology.task.ansible.out:7-11 03:48:51 UTC", "ConditionTimestampMonotonic": "161151156", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "6678", "ExecMainStartTimestamp": "Sun 2021-07-11 03:48:51 UTC", "ExecMainStartTimestampMonotonic": "161162911", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IORe 2021-07-11T03:49:10.412 INFO:teuthology.task.ansible.out:adOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-07-11 03:48:51 UTC", "InactiveEnterTimestampMonotonic": "161150731", "InactiveExitTimestamp": "Sun 2021-07-11 03:48:51 UTC", "InactiveExitTimestampMonotonic": "161151837", "InvocationID": "fb9eba1611ba4bb3a3607af8e9f936fc", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127825", "LimitNPROCSoft": "127825", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "6678", "MemoryAccounting": "yes", "MemoryCurrent": "7049216", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit" 2021-07-11T03:49:10.413 INFO:teuthology.task.ansible.out:: "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:48:51 UTC", "StateChangeTimestampMonotonic": "161162921", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix 2021-07-11T03:49:10.415 INFO:teuthology.task.ansible.out:": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38347", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:49:10.435 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:10 +0000 (0:00:00.274) 0:02:24.909 *********** 2021-07-11T03:49:10.508 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:10.535 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:10 +0000 (0:00:00.098) 0:02:25.008 *********** 2021-07-11T03:49:10.544 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} 2021-07-11T03:49:10.579 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:10 +0000 (0:00:00.044) 0:02:25.053 *********** 2021-07-11T03:49:10.599 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:49:10.601 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:10.634 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:10 +0000 (0:00:00.056) 0:02:25.109 *********** 2021-07-11T03:49:10.665 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:10 +0000 (0:00:00.029) 0:02:25.139 *********** 2021-07-11T03:49:10.764 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-07-11T03:49:10.765 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:10.788 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:10 +0000 (0:00:00.124) 0:02:25.263 *********** 2021-07-11T03:49:10.790 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:10.861 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:10.887 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:10 +0000 (0:00:00.099) 0:02:25.362 *********** 2021-07-11T03:49:10.972 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:10.995 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:10 +0000 (0:00:00.108) 0:02:25.470 *********** 2021-07-11T03:49:11.074 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:11.102 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.103) 0:02:25.573 *********** 2021-07-11T03:49:11.128 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.028) 0:02:25.602 *********** 2021-07-11T03:49:11.136 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-07-11T03:49:11.138 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:49:11.171 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.043) 0:02:25.646 *********** 2021-07-11T03:49:11.190 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-07-11T03:49:11.192 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git qemu-system-data qemu-system-gui\n3 upgraded, 6 newly installed, 0 to remove and 142 not upgraded.\nNeed to get 18.8 MB of archives.\nAfter this operation, 62.7 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.12-0ubuntu0.20.04.1 [3216 kB]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.12-0ubuntu0.20.04.1 [1619 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.16 [53.4 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.16 [1055 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.16 [562 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.16 [39.9 kB]\nGet:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x8 2021-07-11T03:49:11.194 INFO:teuthology.task.ansible.out:6 amd64 1:4.2-3ubuntu6.16 [6722 kB]\nGet:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.16 [973 kB]\nFetched 18.8 MB in 1s (13.6 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138038 files and directories currently installed.)\r\nPreparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...\r\nUnpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...\r\nSelecting previously unselected package librados2.\r\nPreparing to unpack .../1-librados2_15.2.12-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librados2 (15.2.12-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../2-librbd1_15.2.12-0ubuntu0.20.04.1_amd64.deb ...\r\nUnpacking librbd1 (15.2.12-0ubuntu0.20.04.1) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-common (1:4.2-3ubuntu6.16) ...\r\nPreparing to unpack .../5-qemu-system-data_1%3a4.2-3ubuntu6.16_all.deb ...\r\nUnpacking qemu-system-data (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...\r\nPreparing to unpack .../6-qemu-system-gui_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-gui:amd64 2021-07-11T03:49:11.195 INFO:teuthology.task.ansible.out:(1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../7-qemu-system-x86_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:4.2-3ubuntu6.16) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../8-qemu-utils_1%3a4.2-3ubuntu6.16_amd64.deb ...\r\nUnpacking qemu-utils (1:4.2-3ubuntu6.16) ...\r\nSetting up librados2 (15.2.12-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) ...\r\nSetting up git (1:2.25.1-1ubuntu3.1) ...\r\nSetting up librbd1 (15.2.12-0ubuntu0.20.04.1) ...\r\nSetting up qemu-system-data (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-system-common (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-system-x86 (1:4.2-3ubuntu6.16) ...\r\nSetting up qemu-utils (1:4.2-3ubuntu6.16) ...\r\nProcessing triggers for hicolor-icon-theme (0.17-2) ...\r\nProcessing triggers for libc-bin (2.31-0ubuntu9.2) ...\r\nProcessing triggers for man-db (2.9.1-1) ...\r\nProcessing triggers for mime-support (3.64ubuntu1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data", " qemu-system-gui qemu-utils", "Suggested packages:", " git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk", " gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "The following packages will be upgraded:", " git qemu-system-data qemu-system-gui", "3 upgraded, 6 newly installed, 0 to remove and 142 not upgraded.", "Need to get 18.8 MB of archives.", "After this operation, 62.7 MB of additional disk space will be used.", "Get:1 http://archive.u 2021-07-11T03:49:11.197 INFO:teuthology.task.ansible.out:buntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.1 [4557 kB]", "Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.12-0ubuntu0.20.04.1 [3216 kB]", "Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.12-0ubuntu0.20.04.1 [1619 kB]", "Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.16 [53.4 kB]", "Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.16 [1055 kB]", "Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.16 [562 kB]", "Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.16 [39.9 kB]", "Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.16 [6722 kB]", "Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.16 [973 kB]", "Fetched 18.8 MB in 1s (13.6 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138038 files and directories currently installed.)", "Preparing to unpack .../0-git_1%3a2.25.1-1ubuntu3.1_amd64.deb ...", "Unpacking git (1:2.25.1-1ubuntu3.1) over (1:2.25.1-1ubuntu3) ...", "Selecting previously unselected package librados2.", "Preparing to unpack .../1-librados2_15.2.12-0ubuntu0.20.04.1_amd64.deb .. 2021-07-11T03:49:11.199 INFO:teuthology.task.ansible.out:.", "Unpacking librados2 (15.2.12-0ubuntu0.20.04.1) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../2-librbd1_15.2.12-0ubuntu0.20.04.1_amd64.deb ...", "Unpacking librbd1 (15.2.12-0ubuntu0.20.04.1) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../3-qemu-block-extra_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../4-qemu-system-common_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-common (1:4.2-3ubuntu6.16) ...", "Preparing to unpack .../5-qemu-system-data_1%3a4.2-3ubuntu6.16_all.deb ...", "Unpacking qemu-system-data (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...", "Preparing to unpack .../6-qemu-system-gui_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) over (1:4.2-3ubuntu6.14) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../7-qemu-system-x86_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-system-x86 (1:4.2-3ubuntu6.16) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../8-qemu-utils_1%3a4.2-3ubuntu6.16_amd64.deb ...", "Unpacking qemu-utils (1:4.2-3ubuntu6.16) ...", "Setting up librados2 (15.2.12-0ubuntu0.20.04.1) ...", "Setting up qemu-system-gui:amd64 (1:4.2-3ubuntu6.16) ...", "Setting up git (1:2.25.1-1ubuntu3.1) ...", "Setting up librbd1 (15.2.12-0ubuntu0.20.04.1) ...", "Setting up qemu-system-data (1:4.2-3ubuntu6.16) ...", "Setting up qemu-block-extra:amd64 (1:4.2-3ubuntu6.16) ...", "Setting up qemu-system-common (1:4.2-3ubuntu6.16) ...", "Setting up qemu-system-x86 (1:4.2-3ubuntu6.16) ...", "Setting up qemu-utils (1:4.2-3ubuntu6.16) ...", "Processing triggers for hicolor-icon-theme (0.17-2) ...", "Processing triggers for libc-bin (2.31-0ubuntu9.2) ...", "Processing triggers for man-db (2.9.1-1) ...", "Processing 2021-07-11T03:49:11.200 INFO:teuthology.task.ansible.out:triggers for mime-support (3.64ubuntu1) ..."]} 2021-07-11T03:49:11.225 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.054) 0:02:25.700 *********** 2021-07-11T03:49:11.234 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:11.235 INFO:teuthology.task.ansible.out:TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:11.287 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.060) 0:02:25.760 *********** 2021-07-11T03:49:11.414 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} Sunday 11 July 2021 03:49:11 +0000 (0:00:00.051) 0:02:25.812 *********** TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Sunday 11 July 2021 03:49:11 +0000 (0:00:00.062) 0:02:25.875 *********** TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-07-11T03:49:11.438 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.037) 0:02:25.912 *********** 2021-07-11T03:49:11.446 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:11.475 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.036) 0:02:25.949 *********** 2021-07-11T03:49:11.484 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:11.486 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:11.515 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.040) 0:02:25.990 *********** 2021-07-11T03:49:11.526 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:11.555 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.039) 0:02:26.030 *********** 2021-07-11T03:49:11.632 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:11.656 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.100) 0:02:26.131 *********** 2021-07-11T03:49:11.677 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-07-11T03:49:11.704 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.047) 0:02:26.178 *********** 2021-07-11T03:49:11.725 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-07-11T03:49:11.726 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:49:11.728 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-07-11T03:49:11.762 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.058) 0:02:26.237 *********** 2021-07-11T03:49:11.797 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.034) 0:02:26.271 *********** 2021-07-11T03:49:11.807 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-07-11T03:49:11.808 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:11.843 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.046) 0:02:26.318 *********** 2021-07-11T03:49:11.915 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-07-11T03:49:11.917 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:11.941 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:11 +0000 (0:00:00.097) 0:02:26.415 *********** 2021-07-11T03:49:12.009 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:12.033 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.092) 0:02:26.507 *********** 2021-07-11T03:49:12.072 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-07-11T03:49:12.074 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-07-11T03:49:12.101 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.068) 0:02:26.576 *********** 2021-07-11T03:49:12.121 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-07-11T03:49:12.123 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:12.149 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.047) 0:02:26.624 *********** 2021-07-11T03:49:12.158 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-07-11T03:49:12.159 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-07-11T03:49:12.189 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.039) 0:02:26.664 *********** 2021-07-11T03:49:12.220 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-07-11T03:49:12.222 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:12.249 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.059) 0:02:26.724 *********** 2021-07-11T03:49:12.257 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-07-11T03:49:12.259 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:49:12.286 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.036) 0:02:26.760 *********** 2021-07-11T03:49:12.294 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-07-11T03:49:12.295 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-07-11T03:49:12.326 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.038) 0:02:26.799 *********** 2021-07-11T03:49:12.331 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-07-11T03:49:12.333 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:12.362 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.037) 0:02:26.837 *********** 2021-07-11T03:49:12.382 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-07-11T03:49:12.384 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-07-11T03:49:12.415 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.053) 0:02:26.890 *********** 2021-07-11T03:49:12.456 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-07-11T03:49:12.457 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-07-11T03:49:12.493 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.077) 0:02:26.968 *********** 2021-07-11T03:49:12.523 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.027) 0:02:26.996 *********** 2021-07-11T03:49:12.533 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** ok: [gibba023.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:49:12.561 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.038) 0:02:27.034 *********** 2021-07-11T03:49:12.584 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:12.613 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.053) 0:02:27.088 *********** 2021-07-11T03:49:12.627 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-07-11T03:49:12.629 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:12.662 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.048) 0:02:27.137 *********** 2021-07-11T03:49:12.687 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.024) 0:02:27.161 *********** 2021-07-11T03:49:12.788 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:12.812 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.125) 0:02:27.287 *********** 2021-07-11T03:49:12.821 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-07-11T03:49:12.823 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-07-11T03:49:12.848 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.035) 0:02:27.323 *********** 2021-07-11T03:49:12.857 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-07-11T03:49:12.859 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ActiveEnterTimestampMonotonic": "154359515", "ActiveExitTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ActiveExitTimestampMonotonic": "154316720", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service systemd-journald.socket -.mount sysinit.target nss-lookup.target basic.target remote-fs.target network.target system.slice", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:48:52 UTC", "AssertTimestampMonotonic": "154347195", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ConditionTimestampMonotonic": "154347195", "ConfigurationDirect 2021-07-11T03:49:12.861 INFO:teuthology.task.ansible.out:oryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "6667", "ExecMainStartTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ExecMainStartTimestampMonotonic": "154359505", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriori 2021-07-11T03:49:12.862 INFO:teuthology.task.ansible.out:ty": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-07-11 03:48:52 UTC", "InactiveEnterTimestampMonotonic": "154346767", "InactiveExitTimestamp": "Sun 2021-07-11 03:48:52 UTC", "InactiveExitTimestampMonotonic": "154347871", "InvocationID": "ebf0a1285c0c44119e372d367db8fec4", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127825", "LimitNPROCSoft": "127825", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "6667", "MemoryAccounting": "yes", "MemoryCurrent": "7143424", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySw 2021-07-11T03:49:12.864 INFO:teuthology.task.ansible.out:apMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:48:52 UTC", "StateChangeTimestampMonotonic": "154359515", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "T 2021-07-11T03:49:12.865 INFO:teuthology.task.ansible.out:TYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38347", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:49:12.887 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.038) 0:02:27.362 *********** 2021-07-11T03:49:12.895 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-07-11T03:49:12.897 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:12.924 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.036) 0:02:27.398 *********** 2021-07-11T03:49:12.932 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-07-11T03:49:12.934 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ActiveEnterTimestampMonotonic": "166380129", "ActiveExitTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ActiveExitTimestampMonotonic": "166337232", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service system.slice remote-fs.target -.mount systemd-journald.socket basic.target sysinit.target nss-lookup.target network.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:48:52 UTC", "AssertTimestampMonotonic": "166368028", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ConditionTimestampMonotonic": "166368028", "ConfigurationDirect 2021-07-11T03:49:12.936 INFO:teuthology.task.ansible.out:oryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "6678", "ExecMainStartTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ExecMainStartTimestampMonotonic": "166380119", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriori 2021-07-11T03:49:12.937 INFO:teuthology.task.ansible.out:ty": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-07-11 03:48:52 UTC", "InactiveEnterTimestampMonotonic": "166367649", "InactiveExitTimestamp": "Sun 2021-07-11 03:48:52 UTC", "InactiveExitTimestampMonotonic": "166368720", "InvocationID": "3f17f3c0ca0e49dc8da3f9c5186a83f7", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127825", "LimitNPROCSoft": "127825", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "6678", "MemoryAccounting": "yes", "MemoryCurrent": "7139328", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySw 2021-07-11T03:49:12.939 INFO:teuthology.task.ansible.out:apMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:48:52 UTC", "StateChangeTimestampMonotonic": "166380129", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "T 2021-07-11T03:49:12.941 INFO:teuthology.task.ansible.out:TYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38347", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:49:12.961 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.037) 0:02:27.436 *********** 2021-07-11T03:49:12.968 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-07-11T03:49:12.970 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:12.996 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:12 +0000 (0:00:00.034) 0:02:27.471 *********** 2021-07-11T03:49:13.004 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-07-11T03:49:13.005 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.043 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.046) 0:02:27.517 *********** 2021-07-11T03:49:13.061 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-07-11T03:49:13.063 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.089 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.046) 0:02:27.564 *********** 2021-07-11T03:49:13.098 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-07-11T03:49:13.100 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.126 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.037) 0:02:27.601 *********** 2021-07-11T03:49:13.133 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-07-11T03:49:13.135 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.163 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.036) 0:02:27.637 *********** 2021-07-11T03:49:13.170 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-07-11T03:49:13.172 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.197 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.034) 0:02:27.672 *********** 2021-07-11T03:49:13.205 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-07-11T03:49:13.207 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-07-11T03:49:13.234 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.036) 0:02:27.708 *********** 2021-07-11T03:49:13.246 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-07-11T03:49:13.247 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.249 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.277 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.042) 0:02:27.751 *********** 2021-07-11T03:49:13.299 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.023) 0:02:27.774 *********** 2021-07-11T03:49:13.307 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-07-11T03:49:13.309 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.337 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.037) 0:02:27.812 *********** 2021-07-11T03:49:13.358 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-07-11T03:49:13.360 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.385 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.047) 0:02:27.860 *********** 2021-07-11T03:49:13.395 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.421 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.036) 0:02:27.896 *********** 2021-07-11T03:49:13.429 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-07-11T03:49:13.431 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.457 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.036) 0:02:27.932 *********** 2021-07-11T03:49:13.490 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.032) 0:02:27.965 *********** 2021-07-11T03:49:13.500 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:13.502 INFO:teuthology.task.ansible.out:TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.532 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.041) 0:02:28.006 *********** 2021-07-11T03:49:13.534 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:13.540 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-07-11T03:49:13.542 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.572 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.039) 0:02:28.046 *********** 2021-07-11T03:49:13.582 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.611 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.038) 0:02:28.085 *********** 2021-07-11T03:49:13.628 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-07-11T03:49:13.630 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:49:13.630 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.655 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.044) 0:02:28.130 *********** 2021-07-11T03:49:13.679 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.023) 0:02:28.153 *********** 2021-07-11T03:49:13.686 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-07-11T03:49:13.687 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.713 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.033) 0:02:28.187 *********** 2021-07-11T03:49:13.736 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.023) 0:02:28.211 *********** 2021-07-11T03:49:13.795 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-07-11T03:49:13.798 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.824 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.087) 0:02:28.298 *********** 2021-07-11T03:49:13.833 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.865 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.040) 0:02:28.339 *********** 2021-07-11T03:49:13.875 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-07-11T03:49:13.905 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.040) 0:02:28.379 *********** 2021-07-11T03:49:13.913 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-07-11T03:49:13.915 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.948 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.042) 0:02:28.422 *********** 2021-07-11T03:49:13.957 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-07-11T03:49:13.959 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ActiveEnterTimestampMonotonic": "174119711", "ActiveExitTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ActiveExitTimestampMonotonic": "174077787", "ActiveState": "active", "After": "basic.target network.target sysinit.target system.slice -.mount remote-fs.target systemd-journald.socket nss-lookup.target systemd-tmpfiles-setup.service", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:48:52 UTC", "AssertTimestampMonotonic": "174108403", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ConditionTimestampMonotonic": "174108402", "ConfigurationDirect 2021-07-11T03:49:13.960 INFO:teuthology.task.ansible.out:oryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", "Documentation": "https://httpd.apache.org/docs/2.4/", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "6717", "ExecMainStartTimestamp": "Sun 2021-07-11 03:48:52 UTC", "ExecMainStartTimestampMonotonic": "174119701", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopEx": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriori 2021-07-11T03:49:13.962 INFO:teuthology.task.ansible.out:ty": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestamp": "Sun 2021-07-11 03:48:52 UTC", "InactiveEnterTimestampMonotonic": "174107996", "InactiveExitTimestamp": "Sun 2021-07-11 03:48:52 UTC", "InactiveExitTimestampMonotonic": "174109094", "InvocationID": "7862db19ad3d4a759acf3d44dd59bbd2", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127826", "LimitNPROCSoft": "127826", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127826", "LimitSIGPENDINGSoft": "127826", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "6717", "MemoryAccounting": "yes", "MemoryCurrent": "7327744", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySw 2021-07-11T03:49:13.964 INFO:teuthology.task.ansible.out:apMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:48:52 UTC", "StateChangeTimestampMonotonic": "174119711", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "T 2021-07-11T03:49:13.965 INFO:teuthology.task.ansible.out:TYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "38347", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:13.988 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:13 +0000 (0:00:00.040) 0:02:28.463 *********** 2021-07-11T03:49:14.015 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.026) 0:02:28.489 *********** 2021-07-11T03:49:14.016 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:14.039 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.076 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.061) 0:02:28.551 *********** 2021-07-11T03:49:14.085 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-07-11T03:49:14.087 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.115 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.038) 0:02:28.589 *********** 2021-07-11T03:49:14.123 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:14.124 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.165 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.050) 0:02:28.640 *********** 2021-07-11T03:49:14.173 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-07-11T03:49:14.175 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.201 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.035) 0:02:28.676 *********** 2021-07-11T03:49:14.209 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-07-11T03:49:14.211 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.236 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.035) 0:02:28.711 *********** 2021-07-11T03:49:14.255 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-07-11T03:49:14.257 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} TASK [testnode : Remove packages] ********************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.284 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.047) 0:02:28.758 *********** 2021-07-11T03:49:14.315 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.030) 0:02:28.789 *********** 2021-07-11T03:49:14.322 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-07-11T03:49:14.324 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.351 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.036) 0:02:28.826 *********** 2021-07-11T03:49:14.376 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.024) 0:02:28.850 *********** 2021-07-11T03:49:14.395 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-07-11T03:49:14.397 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.429 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.053) 0:02:28.903 *********** 2021-07-11T03:49:14.454 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.024) 0:02:28.928 *********** 2021-07-11T03:49:14.463 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-07-11T03:49:14.465 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upgrade packages] ********************************************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.497 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.041) 0:02:28.970 *********** 2021-07-11T03:49:14.520 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.024) 0:02:28.994 *********** 2021-07-11T03:49:14.538 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.539 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:14.568 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.048) 0:02:29.042 *********** 2021-07-11T03:49:14.577 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-07-11T03:49:14.584 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.606 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.038) 0:02:29.081 *********** 2021-07-11T03:49:14.615 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-07-11T03:49:14.617 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.652 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.043) 0:02:29.124 *********** 2021-07-11T03:49:14.660 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.690 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.038) 0:02:29.163 *********** 2021-07-11T03:49:14.697 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.729 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.040) 0:02:29.203 *********** 2021-07-11T03:49:14.737 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.764 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.035) 0:02:29.239 *********** 2021-07-11T03:49:14.774 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.802 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.037) 0:02:29.276 *********** 2021-07-11T03:49:14.827 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.022) 0:02:29.299 *********** 2021-07-11T03:49:14.831 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-07-11T03:49:14.833 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.868 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.043) 0:02:29.342 *********** 2021-07-11T03:49:14.879 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-07-11T03:49:14.881 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.911 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.043) 0:02:29.386 *********** 2021-07-11T03:49:14.930 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.958 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.045) 0:02:29.432 *********** 2021-07-11T03:49:14.967 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:14.995 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:14 +0000 (0:00:00.037) 0:02:29.470 *********** 2021-07-11T03:49:15.004 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-07-11T03:49:15.005 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.033 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.036) 0:02:29.507 *********** 2021-07-11T03:49:15.053 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.081 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.048) 0:02:29.555 *********** 2021-07-11T03:49:15.087 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:15.089 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.118 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.037) 0:02:29.593 *********** 2021-07-11T03:49:15.126 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-07-11T03:49:15.127 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.160 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.039) 0:02:29.633 *********** 2021-07-11T03:49:15.166 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-07-11T03:49:15.168 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.195 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.036) 0:02:29.669 *********** 2021-07-11T03:49:15.218 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.023) 0:02:29.693 *********** 2021-07-11T03:49:15.226 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-07-11T03:49:15.229 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.230 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.261 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.043) 0:02:29.736 *********** 2021-07-11T03:49:15.263 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:15.291 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.029) 0:02:29.765 *********** 2021-07-11T03:49:15.310 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.341 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.050) 0:02:29.816 *********** 2021-07-11T03:49:15.350 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.379 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.038) 0:02:29.854 *********** 2021-07-11T03:49:15.389 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-07-11T03:49:15.392 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.422 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.041) 0:02:29.895 *********** 2021-07-11T03:49:15.430 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.460 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.039) 0:02:29.935 *********** 2021-07-11T03:49:15.468 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-07-11T03:49:15.469 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.502 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.040) 0:02:29.976 *********** 2021-07-11T03:49:15.514 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-07-11T03:49:15.516 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-07-11T03:49:15.544 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.042) 0:02:30.019 *********** 2021-07-11T03:49:15.572 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.027) 0:02:30.046 *********** 2021-07-11T03:49:15.581 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-07-11T03:49:15.583 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.611 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.039) 0:02:30.086 *********** 2021-07-11T03:49:15.638 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.026) 0:02:30.112 *********** 2021-07-11T03:49:15.645 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-07-11T03:49:15.646 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.676 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.038) 0:02:30.150 *********** 2021-07-11T03:49:15.694 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-07-11T03:49:15.696 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.721 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.045) 0:02:30.196 *********** 2021-07-11T03:49:15.730 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-07-11T03:49:15.732 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.758 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.036) 0:02:30.233 *********** 2021-07-11T03:49:15.765 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-07-11T03:49:15.767 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.798 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.040) 0:02:30.273 *********** 2021-07-11T03:49:15.821 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.022) 0:02:30.296 *********** 2021-07-11T03:49:15.830 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-07-11T03:49:15.832 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.833 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.861 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.039) 0:02:30.335 *********** 2021-07-11T03:49:15.892 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.031) 0:02:30.366 *********** 2021-07-11T03:49:15.900 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-07-11T03:49:15.902 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.930 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.038) 0:02:30.405 *********** 2021-07-11T03:49:15.941 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:15.970 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:15 +0000 (0:00:00.039) 0:02:30.444 *********** 2021-07-11T03:49:15.979 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-07-11T03:49:15.981 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.012 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.041) 0:02:30.486 *********** 2021-07-11T03:49:16.043 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.030) 0:02:30.517 *********** 2021-07-11T03:49:16.050 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.052 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.082 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.039) 0:02:30.556 *********** 2021-07-11T03:49:16.109 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.027) 0:02:30.584 *********** 2021-07-11T03:49:16.118 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-07-11T03:49:16.119 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.145 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.035) 0:02:30.620 *********** 2021-07-11T03:49:16.153 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-07-11T03:49:16.154 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.180 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.034) 0:02:30.654 *********** 2021-07-11T03:49:16.188 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-07-11T03:49:16.190 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.191 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.228 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.046) 0:02:30.701 *********** 2021-07-11T03:49:16.250 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.023) 0:02:30.725 *********** 2021-07-11T03:49:16.260 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.262 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.297 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.046) 0:02:30.772 *********** 2021-07-11T03:49:16.336 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.039) 0:02:30.811 *********** 2021-07-11T03:49:16.350 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.379 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.042) 0:02:30.854 *********** 2021-07-11T03:49:16.410 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.027) 0:02:30.881 *********** 2021-07-11T03:49:16.416 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-07-11T03:49:16.418 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.444 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.037) 0:02:30.919 *********** 2021-07-11T03:49:16.456 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-07-11T03:49:16.457 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.493 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.048) 0:02:30.967 *********** 2021-07-11T03:49:16.494 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:16.541 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-07-11T03:49:16.543 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.573 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.080) 0:02:31.048 *********** 2021-07-11T03:49:16.574 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:16.590 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-07-11T03:49:16.592 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.630 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.056) 0:02:31.104 *********** 2021-07-11T03:49:16.640 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-07-11T03:49:16.641 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-07-11T03:49:16.685 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.054) 0:02:31.159 *********** 2021-07-11T03:49:16.710 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-07-11T03:49:16.712 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove /etc/ceph] ********************************************* ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-07-11T03:49:16.748 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.063) 0:02:31.223 *********** 2021-07-11T03:49:16.807 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.058) 0:02:31.281 *********** 2021-07-11T03:49:16.815 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:16.817 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-07-11T03:49:16.850 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.043) 0:02:31.325 *********** 2021-07-11T03:49:16.878 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:16.920 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:16 +0000 (0:00:00.069) 0:02:31.395 *********** 2021-07-11T03:49:16.958 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-07-11T03:49:16.960 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.028 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.031 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.034 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.035 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-07-11T03:49:17.045 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [gibba019.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.046 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.079 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.158) 0:02:31.554 *********** 2021-07-11T03:49:17.089 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.090 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.092 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.136 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.057) 0:02:31.611 *********** 2021-07-11T03:49:17.144 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-07-11T03:49:17.145 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-07-11T03:49:17.185 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.048) 0:02:31.660 *********** 2021-07-11T03:49:17.197 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-07-11T03:49:17.198 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.233 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.047) 0:02:31.707 *********** 2021-07-11T03:49:17.262 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-07-11T03:49:17.288 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.053) 0:02:31.761 *********** 2021-07-11T03:49:17.294 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-07-11T03:49:17.300 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.328 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.041) 0:02:31.803 *********** 2021-07-11T03:49:17.345 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.367 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.038) 0:02:31.841 *********** 2021-07-11T03:49:17.386 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-07-11T03:49:17.388 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.421 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.054) 0:02:31.896 *********** 2021-07-11T03:49:17.454 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.487 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.065) 0:02:31.962 *********** 2021-07-11T03:49:17.497 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-07-11T03:49:17.499 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.005241", "end": "2021-07-11 03:49:17.426228", "rc": 0, "start": "2021-07-11 03:49:17.420987", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/bin/teuthology-lock"]} 2021-07-11T03:49:17.556 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.068) 0:02:32.031 *********** 2021-07-11T03:49:17.668 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.699 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.143) 0:02:32.174 *********** 2021-07-11T03:49:17.713 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:17.714 INFO:teuthology.task.ansible.out:TASK [testnode : Remove /etc/ceph] ********************************************* 2021-07-11T03:49:17.716 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-07-11T03:49:17.718 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:17.757 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.057) 0:02:32.232 *********** 2021-07-11T03:49:17.760 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:17.777 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-07-11T03:49:17.779 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-07-11T03:49:17.812 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.054) 0:02:32.286 *********** 2021-07-11T03:49:17.837 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:17.872 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.060) 0:02:32.347 *********** 2021-07-11T03:49:17.874 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:17.918 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-07-11T03:49:17.921 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-07-11T03:49:17.953 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:17 +0000 (0:00:00.080) 0:02:32.427 *********** 2021-07-11T03:49:17.954 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:18.013 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:18.014 INFO:teuthology.task.ansible.out:TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-07-11T03:49:18.016 INFO:teuthology.task.ansible.out: skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:18.045 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:18 +0000 (0:00:00.091) 0:02:32.519 *********** 2021-07-11T03:49:18.047 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:18.199 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-07-11T03:49:18.201 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-07-11T03:49:18.202 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:18.227 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:18 +0000 (0:00:00.182) 0:02:32.702 *********** 2021-07-11T03:49:18.297 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-07-11T03:49:18.299 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-07-11T03:49:18.331 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:18 +0000 (0:00:00.101) 0:02:32.803 *********** 2021-07-11T03:49:18.339 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-07-11T03:49:18.341 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:18.374 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:18 +0000 (0:00:00.045) 0:02:32.849 *********** 2021-07-11T03:49:18.394 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-07-11T03:49:18.396 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-07-11T03:49:18.428 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:18 +0000 (0:00:00.054) 0:02:32.903 *********** 2021-07-11T03:49:18.604 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:49:18.637 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:18 +0000 (0:00:00.209) 0:02:33.112 *********** 2021-07-11T03:49:18.701 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-07-11T03:49:18.703 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-07-11T03:49:18.755 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:18 +0000 (0:00:00.117) 0:02:33.229 *********** 2021-07-11T03:49:18.764 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-07-11T03:49:18.766 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:18.798 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:18 +0000 (0:00:00.043) 0:02:33.273 *********** 2021-07-11T03:49:18.832 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:18 +0000 (0:00:00.033) 0:02:33.307 *********** 2021-07-11T03:49:18.841 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-07-11T03:49:18.842 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-07-11T03:49:18.885 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:18 +0000 (0:00:00.053) 0:02:33.360 *********** 2021-07-11T03:49:18.887 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:18.942 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-07-11T03:49:18.946 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item={'block_used': 13858793, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236315, 'size_available': 926663946240, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150122, 'block_size': 4096, 'inode_available': 60904854}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236315, "block_size": 4096, "block_total": 240095108, "block_used": 13858793, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904854, "inode_total": 61054976, "inode_used": 150122, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663946240, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}} 2021-07-11T03:49:18.989 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:18 +0000 (0:00:00.102) 0:02:33.462 *********** 2021-07-11T03:49:18.990 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:18.998 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:19.001 INFO:teuthology.task.ansible.out:TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-07-11T03:49:19.003 INFO:teuthology.task.ansible.out: ok: [gibba005.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:49:19.044 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.056) 0:02:33.519 *********** 2021-07-11T03:49:19.046 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:19.055 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:49:19.056 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:19.107 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.062) 0:02:33.581 *********** 2021-07-11T03:49:19.141 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:19.171 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.064) 0:02:33.646 *********** 2021-07-11T03:49:19.188 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-07-11T03:49:19.190 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002474", "end": "2021-07-11 03:49:19.094047", "rc": 0, "start": "2021-07-11 03:49:19.091573", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:19.238 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.066) 0:02:33.713 *********** 2021-07-11T03:49:19.248 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-07-11T03:49:19.250 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:19.280 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.042) 0:02:33.755 *********** 2021-07-11T03:49:19.289 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-07-11T03:49:19.291 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-07-11T03:49:19.319 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.038) 0:02:33.794 *********** 2021-07-11T03:49:19.346 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.024) 0:02:33.819 *********** 2021-07-11T03:49:19.353 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-07-11T03:49:19.355 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'block_used': 13858793, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236315, 'size_available': 926663946240, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150122, 'block_size': 4096, 'inode_available': 60904854}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236315, "block_size": 4096, "block_total": 240095108, "block_used": 13858793, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904854, "inode_total": 61054976, "inode_used": 150122, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663946240, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:19.391 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.046) 0:02:33.865 *********** 2021-07-11T03:49:19.399 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-07-11T03:49:19.400 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:19.430 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.039) 0:02:33.905 *********** 2021-07-11T03:49:19.452 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-07-11T03:49:19.454 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:19.487 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.053) 0:02:33.958 *********** 2021-07-11T03:49:19.494 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-07-11T03:49:19.535 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.051) 0:02:34.010 *********** 2021-07-11T03:49:19.546 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-07-11T03:49:19.548 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:19.577 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.042) 0:02:34.052 *********** 2021-07-11T03:49:19.590 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:19.637 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.058) 0:02:34.110 *********** 2021-07-11T03:49:19.658 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:19.703 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.066) 0:02:34.177 *********** 2021-07-11T03:49:19.741 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:19.743 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:19.779 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.072) 0:02:34.250 *********** 2021-07-11T03:49:19.826 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.050) 0:02:34.300 *********** 2021-07-11T03:49:19.834 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:19.836 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:19.870 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.042) 0:02:34.342 *********** 2021-07-11T03:49:19.909 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-07-11T03:49:19.941 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.072) 0:02:34.415 *********** 2021-07-11T03:49:19.961 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-07-11T03:49:19.987 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:19 +0000 (0:00:00.046) 0:02:34.462 *********** 2021-07-11T03:49:19.999 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-07-11T03:49:20.001 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:49:20.038 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.050) 0:02:34.512 *********** 2021-07-11T03:49:20.064 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-07-11T03:49:20.076 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.091 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.093 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.094 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-07-11T03:49:20.094 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.110 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.112 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.122 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.129 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.135 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.137 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.148 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.172 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.134) 0:02:34.647 *********** 2021-07-11T03:49:20.181 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-07-11T03:49:20.183 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.186 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.188 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} ok: [gibba023.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-07-11T03:49:20.189 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.189 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.190 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.231 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.058) 0:02:34.706 *********** 2021-07-11T03:49:20.241 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.243 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.277 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.045) 0:02:34.751 *********** 2021-07-11T03:49:20.284 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.311 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.034) 0:02:34.785 *********** 2021-07-11T03:49:20.386 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-07-11T03:49:20.388 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.412 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.101) 0:02:34.887 *********** 2021-07-11T03:49:20.496 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.521 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.108) 0:02:34.995 *********** 2021-07-11T03:49:20.528 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-07-11T03:49:20.530 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.005828", "end": "2021-07-11 03:49:20.482720", "rc": 0, "start": "2021-07-11 03:49:20.476892", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/bin/teuthology-lock"]} 2021-07-11T03:49:20.561 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.040) 0:02:35.036 *********** 2021-07-11T03:49:20.570 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003723", "end": "2021-07-11 03:49:20.515939", "rc": 0, "start": "2021-07-11 03:49:20.512216", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/bin/teuthology-lock"]} 2021-07-11T03:49:20.571 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:20.603 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.042) 0:02:35.078 *********** 2021-07-11T03:49:20.612 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-07-11T03:49:20.613 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.655 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.051) 0:02:35.129 *********** 2021-07-11T03:49:20.665 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.709 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.054) 0:02:35.184 *********** 2021-07-11T03:49:20.717 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.752 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.042) 0:02:35.226 *********** 2021-07-11T03:49:20.765 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-07-11T03:49:20.801 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.049) 0:02:35.276 *********** 2021-07-11T03:49:20.819 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-07-11T03:49:20.821 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.851 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.050) 0:02:35.326 *********** 2021-07-11T03:49:20.859 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.901 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.048) 0:02:35.375 *********** 2021-07-11T03:49:20.910 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-07-11T03:49:20.912 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.912 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.913 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.925 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-07-11T03:49:20.926 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [gibba004.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:20.956 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:20 +0000 (0:00:00.055) 0:02:35.431 *********** 2021-07-11T03:49:20.967 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-07-11T03:49:20.980 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-07-11T03:49:20.982 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.014 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:21 +0000 (0:00:00.057) 0:02:35.488 *********** 2021-07-11T03:49:21.117 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-07-11T03:49:21.119 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.130 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.132 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.146 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.156 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.158 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.168 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.198 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:21 +0000 (0:00:00.184) 0:02:35.673 *********** 2021-07-11T03:49:21.216 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-07-11T03:49:21.218 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermPxQaCEQNZ5qnmFMVMwXNK2Jla7IjakCo'], 'uuids': ['660ba162-623e-4554-8641-a7eb8d65491e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.428817", "end": "2021-07-11 03:49:21.133345", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermPxQaCEQNZ5qnmFMVMwXNK2Jla7IjakCo"], "labels": [], "masters": [], "uuids": ["660ba162-623e-4554-8641-a7eb8d65491e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:19.704528", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:21.219 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.s 2021-07-11T03:49:21.220 INFO:teuthology.task.ansible.out:epia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.231 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.462 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-07-11T03:49:21.464 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:49:21.487 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:21 +0000 (0:00:00.288) 0:02:35.961 *********** 2021-07-11T03:49:21.526 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} 2021-07-11T03:49:21.527 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:21.553 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:21 +0000 (0:00:00.066) 0:02:36.028 *********** 2021-07-11T03:49:21.607 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004452", "end": "2021-07-11 03:49:21.583535", "rc": 0, "start": "2021-07-11 03:49:21.579083", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/bin/teuthology-lock"]} 2021-07-11T03:49:21.636 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:21 +0000 (0:00:00.083) 0:02:36.111 *********** 2021-07-11T03:49:21.734 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.768 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:21 +0000 (0:00:00.131) 0:02:36.243 *********** 2021-07-11T03:49:21.820 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.823 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:21.861 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:21 +0000 (0:00:00.092) 0:02:36.335 *********** 2021-07-11T03:49:21.928 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-07-11T03:49:21.930 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:21.965 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:21 +0000 (0:00:00.104) 0:02:36.440 *********** 2021-07-11T03:49:21.974 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-07-11T03:49:21.976 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item={'block_used': 13858783, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236325, 'size_available': 926663987200, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150120, 'block_size': 4096, 'inode_available': 60904856}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236325, "block_size": 4096, "block_total": 240095108, "block_used": 13858783, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904856, "inode_total": 61054976, "inode_used": 150120, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663987200, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}} 2021-07-11T03:49:22.005 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:22 +0000 (0:00:00.039) 0:02:36.479 *********** 2021-07-11T03:49:22.051 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item={'block_used': 13858785, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236323, 'size_available': 926663979008, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150120, 'block_size': 4096, 'inode_available': 60904856}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236323, "block_size": 4096, "block_total": 240095108, "block_used": 13858785, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904856, "inode_total": 61054976, "inode_used": 150120, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663979008, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}} 2021-07-11T03:49:22.056 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:22.087 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:22 +0000 (0:00:00.082) 0:02:36.562 *********** 2021-07-11T03:49:22.112 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:22 +0000 (0:00:00.024) 0:02:36.587 *********** 2021-07-11T03:49:22.183 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-07-11T03:49:22.184 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:22.185 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002492", "end": "2021-07-11 03:49:22.509880", "rc": 0, "start": "2021-07-11 03:49:22.507388", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:22.215 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:22 +0000 (0:00:00.103) 0:02:36.690 *********** 2021-07-11T03:49:22.240 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:22 +0000 (0:00:00.024) 0:02:36.714 *********** 2021-07-11T03:49:22.314 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002386", "end": "2021-07-11 03:49:22.737501", "rc": 0, "start": "2021-07-11 03:49:22.735115", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:22.342 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:22 +0000 (0:00:00.102) 0:02:36.816 *********** 2021-07-11T03:49:22.425 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-07-11T03:49:22.427 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermGzs63hUv9zh24qomJoauXS3n4as28gMw'], 'uuids': ['be4ad59f-44a3-4d48-a0c2-ba82a594a324']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.085768", "end": "2021-07-11 03:49:22.349205", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermGzs63hUv9zh24qomJoauXS3n4as28gMw"], "labels": [], "masters": [], "uuids": ["be4ad59f-44a3-4d48-a0c2-ba82a594a324"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:21.263437", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:22.441 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF', 'wwn-0x50000399ab9813cb'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF-part1', 'wwn-0x50000399ab9813cb-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813cb', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2JXF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF", "wwn-0x50000399ab9813cb"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF-part1", "wwn-0x50000399ab9813cb-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2JXF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813cb"}}, "skip_reason": "Cond 2021-07-11T03:49:22.443 INFO:teuthology.task.ansible.out:itional result was False"} 2021-07-11T03:49:22.525 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [gibba023.front.sepia.ceph.com] => (item={'block_used': 13858783, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236325, 'size_available': 926663987200, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150120, 'block_size': 4096, 'inode_available': 60904856}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236325, "block_size": 4096, "block_total": 240095108, "block_used": 13858783, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904856, "inode_total": 61054976, "inode_used": 150120, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663987200, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:22.559 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:22 +0000 (0:00:00.216) 0:02:37.033 *********** 2021-07-11T03:49:22.605 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'block_used': 13858785, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236323, 'size_available': 926663979008, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150120, 'block_size': 4096, 'inode_available': 60904856}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236323, "block_size": 4096, "block_total": 240095108, "block_used": 13858785, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904856, "inode_total": 61054976, "inode_used": 150120, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663979008, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:22.650 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:22 +0000 (0:00:00.087) 0:02:37.120 *********** 2021-07-11T03:49:22.744 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-07-11T03:49:22.746 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:49:22.783 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:22 +0000 (0:00:00.137) 0:02:37.258 *********** 2021-07-11T03:49:22.834 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-07-11T03:49:22.861 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:22 +0000 (0:00:00.077) 0:02:37.335 *********** 2021-07-11T03:49:22.930 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:22 +0000 (0:00:00.069) 0:02:37.405 *********** 2021-07-11T03:49:23.114 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-07-11T03:49:23.116 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:23.168 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:23 +0000 (0:00:00.237) 0:02:37.642 *********** 2021-07-11T03:49:23.178 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-07-11T03:49:23.180 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:23.225 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:23.455 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-07-11T03:49:23.457 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item={'block_used': 13858784, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236324, 'size_available': 926663983104, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150120, 'block_size': 4096, 'inode_available': 60904856}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236324, "block_size": 4096, "block_total": 240095108, "block_used": 13858784, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904856, "inode_total": 61054976, "inode_used": 150120, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663983104, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}} 2021-07-11T03:49:23.620 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:23 +0000 (0:00:00.451) 0:02:38.094 *********** 2021-07-11T03:49:23.676 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-07-11T03:49:23.677 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-SmKxdL-mU0K-4Xts-CdEF-GiIr-xwv5-zzgpyN', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400BD375A', 'nvme-nvme.8086-50484b4d393133343030424433373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.127709", "end": "2021-07-11 03:49:23.597314", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-SmKxdL-mU0K-4Xts-CdEF-GiIr-xwv5-zzgpyN", "nvme-INTEL_SSDPEL1K375GA_PHKM913400BD375A", "nvme-nvme.8086-50484b4d393133343030424433373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:22.469605", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures destroyed! You m 2021-07-11T03:49:23.679 INFO:teuthology.task.ansible.out:ay now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:23.700 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:23.794 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.002556", "end": "2021-07-11 03:49:23.993229", "rc": 0, "start": "2021-07-11 03:49:23.990673", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:23.826 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:23 +0000 (0:00:00.206) 0:02:38.301 *********** 2021-07-11T03:49:23.916 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [gibba004.front.sepia.ceph.com] => (item={'block_used': 13858784, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236324, 'size_available': 926663983104, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150120, 'block_size': 4096, 'inode_available': 60904856}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236324, "block_size": 4096, "block_total": 240095108, "block_used": 13858784, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904856, "inode_total": 61054976, "inode_used": 150120, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663983104, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:23.933 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:23 +0000 (0:00:00.107) 0:02:38.408 *********** 2021-07-11T03:49:24.007 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-07-11T03:49:24.030 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:24 +0000 (0:00:00.097) 0:02:38.505 *********** 2021-07-11T03:49:24.127 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:24.128 INFO:teuthology.task.ansible.out:TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:24.723 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfHs0onjCiOr8pxxbsLKWXRUXRxKmhdlww'], 'uuids': ['b13eb44e-71ec-4ce4-88c8-e3ab06d62ef9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.483135", "end": "2021-07-11 03:49:25.044853", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfHs0onjCiOr8pxxbsLKWXRUXRxKmhdlww"], "labels": [], "masters": [], "uuids": ["b13eb44e-71ec-4ce4-88c8-e3ab06d62ef9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:23.561718", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:24.739 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:24.743 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:24.914 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermDnDFLTBuDVeMjKFmoo76TfWko9o2RgPf'], 'uuids': ['67896801-eb55-4e3c-8e74-7f993e79508e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.115767", "end": "2021-07-11 03:49:24.842028", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermDnDFLTBuDVeMjKFmoo76TfWko9o2RgPf"], "labels": [], "masters": [], "uuids": ["67896801-eb55-4e3c-8e74-7f993e79508e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:23.726261", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:24.931 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:24.942 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOinXBaYhzQToeXZbIwOq6hFUzG02kBPMMm'], 'uuids': ['5b7fac19-5c84-4e84-9a76-a968c3826efb']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.606385", "end": "2021-07-11 03:49:25.364828", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOinXBaYhzQToeXZbIwOq6hFUzG02kBPMMm"], "labels": [], "masters": [], "uuids": ["5b7fac19-5c84-4e84-9a76-a968c3826efb"]}, "model": null, "partitions": {}, "removable": "0", "rotatio 2021-07-11T03:49:24.944 INFO:teuthology.task.ansible.out:nal": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:23.758443", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:24.953 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:24.967 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:25.750 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8kEdPYlzbyL0oQvhGMqYyT2S1pWXdLag3'], 'uuids': ['48533652-5e1f-43f1-80e6-3b69415c3a8f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.522621", "end": "2021-07-11 03:49:25.942850", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8kEdPYlzbyL0oQvhGMqYyT2S1pWXdLag3"], "labels": [], "masters": [], "uuids": ["48533652-5e1f-43f1-80e6-3b69415c3a8f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:24.420229", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:25.763 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:25.778 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:25.937 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfwFMIR1XNiminAgSphSDSSqPLLmhShsqv'], 'uuids': ['2eab9878-28c7-42d8-89e1-92635b064c8d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.099490", "end": "2021-07-11 03:49:26.268090", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfwFMIR1XNiminAgSphSDSSqPLLmhShsqv"], "labels": [], "masters": [], "uuids": ["2eab9878-28c7-42d8-89e1-92635b064c8d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:25.168600", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:25.953 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JYF6XF', 'wwn-0x50000399ab9813cc'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JYF6XF-part1', 'wwn-0x50000399ab9813cc-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813cc', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2JYF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JYF6XF", "wwn-0x50000399ab9813cc"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JYF6XF-part1", "wwn-0x50000399ab9813cc-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2JYF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813cc"}}, "skip_reason": "Cond 2021-07-11T03:49:25.958 INFO:teuthology.task.ansible.out:itional result was False"} 2021-07-11T03:49:26.158 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermz9vIPOJb6tZkgr2ipitpp1JvafA2pRlZ'], 'uuids': ['682e8392-ac93-4de8-b2f1-cdf6ddf2af5c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.111253", "end": "2021-07-11 03:49:26.084504", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermz9vIPOJb6tZkgr2ipitpp1JvafA2pRlZ"], "labels": [], "masters": [], "uuids": ["682e8392-ac93-4de8-b2f1-cdf6ddf2af5c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:24.973251", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:26.174 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:26.176 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOiBGBvmQldOv91jNB3ExNPdSGFKOC2Htch'], 'uuids': ['064476a9-7b64-4f17-ae5b-d11fde019f6b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.097981", "end": "2021-07-11 03:49:26.591753", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOiBGBvmQldOv91jNB3ExNPdSGFKOC2Htch"], "labels": [], "masters": [], "uuids": ["064476a9-7b64-4f17-ae5b-d11fde019f6b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:25.493772", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:26.177 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38MF6XF', 'wwn-0x50000399aba8166c'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38MF6XF-part1', 'wwn-0x50000399aba8166c-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399aba8166c', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I8K38MF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38MF6XF", "wwn-0x50000399aba8166c"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38MF6XF-part1", "wwn-0x50000399aba8166c-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I8K38MF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399aba8166c"}}, "skip_reason": "Cond 2021-07-11T03:49:26.179 INFO:teuthology.task.ansible.out:itional result was False"} 2021-07-11T03:49:26.994 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8eo5Gsm0vOZ3Vz2Xk1nhOpZoTKV8xNcJi'], 'uuids': ['006ea518-7838-40b2-ab0b-768e62ddba17']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.119269", "end": "2021-07-11 03:49:27.187297", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8eo5Gsm0vOZ3Vz2Xk1nhOpZoTKV8xNcJi"], "labels": [], "masters": [], "uuids": ["006ea518-7838-40b2-ab0b-768e62ddba17"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:26.068028", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:27.006 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2K0F6XF', 'wwn-0x50000399ab9813ce'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2K0F6XF-part1', 'wwn-0x50000399ab9813ce-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813ce', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2K0F6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2K0F6XF", "wwn-0x50000399ab9813ce"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2K0F6XF-part1", "wwn-0x50000399ab9813ce-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2K0F6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813ce"}}, "skip_reason": "Cond 2021-07-11T03:49:27.008 INFO:teuthology.task.ansible.out:itional result was False"} 2021-07-11T03:49:27.166 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-ri1s3s-0pBe-U9it-Sugf-ck4O-g0xU-Jb7dV3', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400CL375A', 'nvme-nvme.8086-50484b4d393133343030434c33373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.126812", "end": "2021-07-11 03:49:27.496490", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-ri1s3s-0pBe-U9it-Sugf-ck4O-g0xU-Jb7dV3", "nvme-INTEL_SSDPEL1K375GA_PHKM913400CL375A", "nvme-nvme.8086-50484b4d393133343030434c33373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:26.369678", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures destroyed! You m 2021-07-11T03:49:27.168 INFO:teuthology.task.ansible.out:ay now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:27.178 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:27.200 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:27.361 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermZqlOAhj5YUibVlkwfdKMI0W5EP24JFE0'], 'uuids': ['05dbad88-7bd1-4bd8-b1f2-a0e4ad3a64a9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.088274", "end": "2021-07-11 03:49:27.276954", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermZqlOAhj5YUibVlkwfdKMI0W5EP24JFE0"], "labels": [], "masters": [], "uuids": ["05dbad88-7bd1-4bd8-b1f2-a0e4ad3a64a9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:26.188680", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:27.406 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:27 +0000 (0:00:03.370) 0:02:41.876 *********** 2021-07-11T03:49:27.413 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-D9kWTv-HDO0-PdnJ-xAQw-ODdj-GFz3-mTxQvF', 'nvme-INTEL_SSDPEL1K375GA_PHKM9200003U375A', 'nvme-nvme.8086-50484b4d393230303030335533373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.114797", "end": "2021-07-11 03:49:27.819339", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-D9kWTv-HDO0-PdnJ-xAQw-ODdj-GFz3-mTxQvF", "nvme-INTEL_SSDPEL1K375GA_PHKM9200003U375A", "nvme-nvme.8086-50484b4d393230303030335533373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:26.704542", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures destroyed! You m 2021-07-11T03:49:27.418 INFO:teuthology.task.ansible.out:ay now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:27.428 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:27.430 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:27.509 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-07-11T03:49:27.511 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:27.685 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermPxQaCEQNZ5qnmFMVMwXNK2Jla7IjakCo'], 'uuids': ['660ba162-623e-4554-8641-a7eb8d65491e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.076997", "end": "2021-07-11 03:49:27.615628", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermPxQaCEQNZ5qnmFMVMwXNK2Jla7IjakCo"], "labels": [], "masters": [], "uuids": ["660ba162-623e-4554-8641-a7eb8d65491e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:27.538631", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0756506 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0756506 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:27.697 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:27.699 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:27.874 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermGzs63hUv9zh24qomJoauXS3n4as28gMw'], 'uuids': ['be4ad59f-44a3-4d48-a0c2-ba82a594a324']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.077677", "end": "2021-07-11 03:49:27.808836", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermGzs63hUv9zh24qomJoauXS3n4as28gMw"], "labels": [], "masters": [], "uuids": ["be4ad59f-44a3-4d48-a0c2-ba82a594a324"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:27.731159", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0763021 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0763021 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:27.891 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF', 'wwn-0x50000399ab9813cb'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF-part1', 'wwn-0x50000399ab9813cb-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813cb', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2JXF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF", "wwn-0x50000399ab9813cb"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JXF6XF-part1", "wwn-0x50000399ab9813cb-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2JXF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813cb"}}, "skip_reason": "Cond 2021-07-11T03:49:27.893 INFO:teuthology.task.ansible.out:itional result was False"} 2021-07-11T03:49:28.016 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-SmKxdL-mU0K-4Xts-CdEF-GiIr-xwv5-zzgpyN', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400BD375A', 'nvme-nvme.8086-50484b4d393133343030424433373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.025002", "end": "2021-07-11 03:49:27.941790", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-SmKxdL-mU0K-4Xts-CdEF-GiIr-xwv5-zzgpyN", "nvme-INTEL_SSDPEL1K375GA_PHKM913400BD375A", "nvme-nvme.8086-50484b4d393133343030424433373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:27.916788", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0237054 s, 4.9 GB/s", "stderr_lines": ["1 2021-07-11T03:49:28.018 INFO:teuthology.task.ansible.out:10+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0237054 s, 4.9 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:28.028 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:28.042 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:28.202 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-07-11T03:49:28.204 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-Ui4Hoq-VUwz-Pn2f-DZPO-Azak-CZBh-B2NY5y', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400JX375A', 'nvme-nvme.8086-50484b4d3931333430304a5833373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:00.901113", "end": "2021-07-11 03:49:28.183101", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-Ui4Hoq-VUwz-Pn2f-DZPO-Azak-CZBh-B2NY5y", "nvme-INTEL_SSDPEL1K375GA_PHKM913400JX375A", "nvme-nvme.8086-50484b4d3931333430304a5833373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:27.281988", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nGPT data structures destroyed! You m 2021-07-11T03:49:28.205 INFO:teuthology.task.ansible.out:ay now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:28.214 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:28.220 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermDnDFLTBuDVeMjKFmoo76TfWko9o2RgPf'], 'uuids': ['67896801-eb55-4e3c-8e74-7f993e79508e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.077532", "end": "2021-07-11 03:49:28.141307", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermDnDFLTBuDVeMjKFmoo76TfWko9o2RgPf"], "labels": [], "masters": [], "uuids": ["67896801-eb55-4e3c-8e74-7f993e79508e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:28.063775", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0762997 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0762997 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle' 2021-07-11T03:49:28.223 INFO:teuthology.task.ansible.out:: None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:28.242 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:28.419 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNf2MseNCDon3b8UMtZySVcim4k8b4PDSfl'], 'uuids': ['6cbdf0d0-c3e3-49df-abbf-5dab30d9f2bb']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.108333", "end": "2021-07-11 03:49:28.743354", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNf2MseNCDon3b8UMtZySVcim4k8b4PDSfl"], "labels": [], "masters": [], "uuids": ["6cbdf0d0-c3e3-49df-abbf-5dab30d9f2bb"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:27.635021", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now par 2021-07-11T03:49:28.421 INFO:teuthology.task.ansible.out:tition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:28.435 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermz9vIPOJb6tZkgr2ipitpp1JvafA2pRlZ'], 'uuids': ['682e8392-ac93-4de8-b2f1-cdf6ddf2af5c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.078307", "end": "2021-07-11 03:49:28.368597", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermz9vIPOJb6tZkgr2ipitpp1JvafA2pRlZ"], "labels": [], "masters": [], "uuids": ["682e8392-ac93-4de8-b2f1-cdf6 2021-07-11T03:49:28.441 INFO:teuthology.task.ansible.out:ddf2af5c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:28.290290", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0770661 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0770661 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} TASK [testnode : Zap all non-root disks] *************************************** skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:28.479 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-07-11T03:49:28.481 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:28.644 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermZqlOAhj5YUibVlkwfdKMI0W5EP24JFE0'], 'uuids': ['05dbad88-7bd1-4bd8-b1f2-a0e4ad3a64a9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.077195", "end": "2021-07-11 03:49:28.575437", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-K74QM5zaBtEuNSUC9dgsBQG9mr3yyermZqlOAhj5YUibVlkwfdKMI0W5EP24JFE0"], "labels": [], "masters": [], "uuids": ["05dbad88-7bd1-4bd8-b1f2-a0e4ad3a64a9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:28.498242", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0759743 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0759743 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:28.680 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:28 +0000 (0:00:01.279) 0:02:43.155 *********** 2021-07-11T03:49:28.690 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-07-11T03:49:28.691 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOi5WTVm3QL4C4d5ybDe5iurhOyToVCB2eK'], 'uuids': ['25e4c92f-8c87-4e42-9e0a-d362aff51737']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.096247", "end": "2021-07-11 03:49:29.075215", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOi5WTVm3QL4C4d5ybDe5iurhOyToVCB2eK"], "labels": [], "masters": [], "uuids": ["25e4c92f-8c87-4e42-9e0a-d362aff51737"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:27.978968", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:28.695 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:28.696 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:29.049 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [gibba019.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.173022", "end": "2021-07-11 03:49:28.905542", "rc": 0, "start": "2021-07-11 03:49:28.732520", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:29.074 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:29 +0000 (0:00:00.392) 0:02:43.547 *********** 2021-07-11T03:49:29.305 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-07-11T03:49:29.307 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.070888", "end": "2021-07-11 03:49:29.232578", "rc": 0, "start": "2021-07-11 03:49:29.161690", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:29.332 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:29 +0000 (0:00:00.257) 0:02:43.805 *********** 2021-07-11T03:49:29.408 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-07-11T03:49:29.493 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:29 +0000 (0:00:00.161) 0:02:43.966 *********** 2021-07-11T03:49:29.501 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-07-11T03:49:29.503 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE88zg8uQlNue99T3Fhp5Tj9tPfayv6r5RB'], 'uuids': ['ee4bae7c-c614-4851-83dc-e8c15cb8d622']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.105659", "end": "2021-07-11 03:49:29.400120", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE88zg8uQlNue99T3Fhp5Tj9tPfayv6r5RB"], "labels": [], "masters": [], "uuids": ["ee4bae7c-c614-4851-83dc-e8c15cb8d622"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:28.294461", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:29.504 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sep 2021-07-11T03:49:29.505 INFO:teuthology.task.ansible.out:ia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:29.516 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:29.547 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:29 +0000 (0:00:00.055) 0:02:44.022 *********** 2021-07-11T03:49:29.577 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-07-11T03:49:29.578 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:29.604 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:29 +0000 (0:00:00.055) 0:02:44.077 *********** 2021-07-11T03:49:29.634 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNf9MxRqxAX76lSLbZN9ibUsTmiYJfsMQyB'], 'uuids': ['264118ac-5cf2-4a55-828f-929068c4caad']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.094589", "end": "2021-07-11 03:49:29.960350", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNf9MxRqxAX76lSLbZN9ibUsTmiYJfsMQyB"], "labels": [], "masters": [], "uuids": ["264118ac-5cf2-4a55-828f-929068c4caad"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:28.865761", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now p 2021-07-11T03:49:29.636 INFO:teuthology.task.ansible.out:artition the disk using fdisk or", "other utilities."]} TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:29.662 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:29 +0000 (0:00:00.059) 0:02:44.137 *********** 2021-07-11T03:49:29.664 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:29.671 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-07-11T03:49:29.673 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:29.698 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-07-11T03:49:29.700 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:29.726 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:29 +0000 (0:00:00.063) 0:02:44.201 *********** 2021-07-11T03:49:29.836 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-07-11T03:49:29.837 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:49:29.863 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:29 +0000 (0:00:00.137) 0:02:44.338 *********** 2021-07-11T03:49:29.886 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-07-11T03:49:29.888 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOisIxHHJ3oVeVAI6zNvyr5KGp81F0CmXrF'], 'uuids': ['a272f6c1-afd1-4547-be3f-8b4086c99f30']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.088141", "end": "2021-07-11 03:49:30.292539", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOisIxHHJ3oVeVAI6zNvyr5KGp81F0CmXrF"], "labels": [], "masters": [], "uuids": ["a272f6c1-afd1-4547-be3f-8b4086c99f30"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:29.204398", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:29.897 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:29.901 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:29.912 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-07-11T03:49:29.914 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'block_used': 13858793, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236315, 'size_available': 926663946240, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150122, 'block_size': 4096, 'inode_available': 60904854}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236315, "block_size": 4096, "block_total": 240095108, "block_used": 13858793, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904854, "inode_total": 61054976, "inode_used": 150122, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663946240, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:29.937 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:29 +0000 (0:00:00.073) 0:02:44.411 *********** 2021-07-11T03:49:29.977 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-07-11T03:49:29.979 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:30.005 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:30 +0000 (0:00:00.068) 0:02:44.480 *********** 2021-07-11T03:49:30.041 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:30.064 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:30 +0000 (0:00:00.058) 0:02:44.539 *********** 2021-07-11T03:49:30.105 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:30.148 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:30 +0000 (0:00:00.083) 0:02:44.623 *********** 2021-07-11T03:49:30.175 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-07-11T03:49:30.181 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:30.202 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:30 +0000 (0:00:00.051) 0:02:44.674 *********** 2021-07-11T03:49:30.242 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-07-11T03:49:30.244 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:30.271 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:30 +0000 (0:00:00.072) 0:02:44.746 *********** 2021-07-11T03:49:30.317 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:30.341 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:30 +0000 (0:00:00.069) 0:02:44.815 *********** 2021-07-11T03:49:30.658 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8Ho98Y6vn1q6f42kY7FSDU3Q7ZbO3H7tA'], 'uuids': ['c19933ee-4647-4ef6-8562-1bad86277352']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.105075", "end": "2021-07-11 03:49:30.634206", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8Ho98Y6vn1q6f42kY7FSDU3Q7ZbO3H7tA"], "labels": [], "masters": [], "uuids": ["c19933ee-4647-4ef6-8562-1bad86277352"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:29.529131", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now p 2021-07-11T03:49:30.660 INFO:teuthology.task.ansible.out:artition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:30.677 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:30.888 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfsVKrklTyaCz8lxgDG2Dclz7xluWcpEFI'], 'uuids': ['d658bedb-8e1a-44a7-bebe-e4b50e14ffb6']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.093515", "end": "2021-07-11 03:49:31.210764", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfsVKrklTyaCz8lxgDG2Dclz7xluWcpEFI"], "labels": [], "masters": [], "uuids": ["d658bedb-8e1a-44a7-bebe-e4b50e14ffb6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:30.117249", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:30.923 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:30 +0000 (0:00:00.582) 0:02:45.398 *********** 2021-07-11T03:49:30.953 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-07-11T03:49:30.954 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-07-11T03:49:30.981 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:30 +0000 (0:00:00.057) 0:02:45.455 *********** 2021-07-11T03:49:31.000 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-07-11T03:49:31.002 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:31.094 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-07-11T03:49:31.096 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOiVNs9BDnKAOLPEGYOnUTTGslh1VfJD69M'], 'uuids': ['b3922aee-d8b4-4b9e-a6c8-3084eaeafa1a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.104803", "end": "2021-07-11 03:49:31.512511", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOiVNs9BDnKAOLPEGYOnUTTGslh1VfJD69M"], "labels": [], "masters": [], "uuids": ["b3922aee-d8b4-4b9e-a6c8-3084eaeafa1a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:30.407708", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-07-11T03:49:31.142 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:31 +0000 (0:00:00.161) 0:02:45.617 *********** 2021-07-11T03:49:31.212 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-07-11T03:49:31.214 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfHs0onjCiOr8pxxbsLKWXRUXRxKmhdlww'], 'uuids': ['b13eb44e-71ec-4ce4-88c8-e3ab06d62ef9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.077235", "end": "2021-07-11 03:49:31.495157", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfHs0onjCiOr8pxxbsLKWXRUXRxKmhdlww"], "labels": [], "masters": [], "uuids": ["b13eb44e-71ec-4ce4-88c8-e3ab06d62ef9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:31.417922", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0759296 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0759296 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 2021-07-11T03:49:31.216 INFO:teuthology.task.ansible.out:'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:31.227 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:31.240 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:31.402 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfwFMIR1XNiminAgSphSDSSqPLLmhShsqv'], 'uuids': ['2eab9878-28c7-42d8-89e1-92635b064c8d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.077884", "end": "2021-07-11 03:49:31.733567", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfwFMIR1XNiminAgSphSDSSqPLLmhShsqv"], "labels": [], "masters": [], "uuids": ["2eab9878-28c7-42d8-89e1-92635b064c8d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:31.655683", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0766141 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0766141 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:31.413 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JYF6XF', 'wwn-0x50000399ab9813cc'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2JYF6XF-part1', 'wwn-0x50000399ab9813cc-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813cc', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2JYF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JYF6XF", "wwn-0x50000399ab9813cc"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2JYF6XF-part1", "wwn-0x50000399ab9813cc-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2JYF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813cc"}}, "skip_reason": "Cond 2021-07-11T03:49:31.415 INFO:teuthology.task.ansible.out:itional result was False"} changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOinXBaYhzQToeXZbIwOq6hFUzG02kBPMMm'], 'uuids': ['5b7fac19-5c84-4e84-9a76-a968c3826efb']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "-1 day, 23:59:59.626030", "end": "2021-07-11 03:49:31.391138", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOinXBaYhzQToeXZbIwOq6hFUzG02kBPMMm"], "labels": [], "masters": [], "uuids": ["5b7fac19-5c84-4e84-9a76-a968c3826efb"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:31.765108", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.076293 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.076293 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:31.420 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:31.440 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:31.542 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-ri1s3s-0pBe-U9it-Sugf-ck4O-g0xU-Jb7dV3', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400CL375A', 'nvme-nvme.8086-50484b4d393133343030434c33373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.024825", "end": "2021-07-11 03:49:31.866237", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-ri1s3s-0pBe-U9it-Sugf-ck4O-g0xU-Jb7dV3", "nvme-INTEL_SSDPEL1K375GA_PHKM913400CL375A", "nvme-nvme.8086-50484b4d393133343030434c33373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:31.841412", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0236264 s, 4.9 GB/s", "stderr_lines": ["1 2021-07-11T03:49:31.544 INFO:teuthology.task.ansible.out:10+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0236264 s, 4.9 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:31.554 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:31.556 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:31.590 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:31.612 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOiBGBvmQldOv91jNB3ExNPdSGFKOC2Htch'], 'uuids': ['064476a9-7b64-4f17-ae5b-d11fde019f6b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.078515", "end": "2021-07-11 03:49:31.584229", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOiBGBvmQldOv91jNB3ExNPdSGFKOC2Htch"], "labels": [], "masters": [], "uuids": ["064476a9-7b64-4f17-ae5b-d11fde019f6b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:31.505714", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0772585 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0772585 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:31.626 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38MF6XF', 'wwn-0x50000399aba8166c'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I8K38MF6XF-part1', 'wwn-0x50000399aba8166c-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399aba8166c', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I8K38MF6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38MF6XF", "wwn-0x50000399aba8166c"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I8K38MF6XF-part1", "wwn-0x50000399aba8166c-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I8K38MF6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399aba8166c"}}, "skip_reason": "Cond 2021-07-11T03:49:31.628 INFO:teuthology.task.ansible.out:itional result was False"} 2021-07-11T03:49:31.740 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-D9kWTv-HDO0-PdnJ-xAQw-ODdj-GFz3-mTxQvF', 'nvme-INTEL_SSDPEL1K375GA_PHKM9200003U375A', 'nvme-nvme.8086-50484b4d393230303030335533373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.025169", "end": "2021-07-11 03:49:31.720699", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-D9kWTv-HDO0-PdnJ-xAQw-ODdj-GFz3-mTxQvF", "nvme-INTEL_SSDPEL1K375GA_PHKM9200003U375A", "nvme-nvme.8086-50484b4d393230303030335533373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:31.695530", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.02397 s, 4.8 GB/s", "stderr_lines": ["110 2021-07-11T03:49:32.005 INFO:teuthology.task.ansible.out:+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.02397 s, 4.8 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNf2MseNCDon3b8UMtZySVcim4k8b4PDSfl'], 'uuids': ['6cbdf0d0-c3e3-49df-abbf-5dab30d9f2bb']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.078423", "end": "2021-07-11 03:49:32.085948", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNf2MseNCDon3b8UMtZySVcim4k8b4PDSfl"], "labels": [], " 2021-07-11T03:49:32.007 INFO:teuthology.task.ansible.out:masters": [], "uuids": ["6cbdf0d0-c3e3-49df-abbf-5dab30d9f2bb"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:32.007525", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0771388 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0771388 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.0 2021-07-11T03:49:32.009 INFO:teuthology.task.ansible.out:0 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} TASK [testnode : Zap all non-root disks] *************************************** changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8iPuhSSCAe9fA7Td6Ow5Qiru8bNwqgI0N'], 'uuids': ['e1cba92d-4570-487d-b10b-7147975183f2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_disc 2021-07-11T03:49:32.011 INFO:teuthology.task.ansible.out:ard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.132732", "end": "2021-07-11 03:49:31.906778", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8iPuhSSCAe9fA7Td6Ow5Qiru8bNwqgI0N"], "labels": [], "masters": [], "uuids": ["e1cba92d-4570-487d-b10b-7147975183f2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:30.774046", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries in memory.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries in memory.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} Sunday 11 July 2021 03:49:31 +0000 (0:00:00.839) 0:02:46.456 *********** TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} TASK [testnode : Blow away lingering OSD data and FSIDs] ******** 2021-07-11T03:49:32.013 INFO:teuthology.task.ansible.out:*************** changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOi5WTVm3QL4C4d5ybDe5iurhOyToVCB2eK'], 'uuids': ['25e4c92f-8c87-4e42-9e0a-d362aff51737']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.078218", "end": "2021-07-11 03:49:31.964939", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOi5WTVm3QL4C4d5ybDe5iurhOyToVCB2eK"], "labels": [], "masters": [], "uuids": ["25e4c92f-8c87-4e42-9e0a-d362aff51737"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:31.886721", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0770399 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0770399 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNf9MxRqxAX76lSLbZN9ibUsTmiYJfsMQyB'], 'uuids': ['264118ac-5cf2-4a55-828f-929068c4caad']}, 'sas_device_handle': None, 'sas_add 2021-07-11T03:49:32.014 INFO:teuthology.task.ansible.out:ress': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.077663", "end": "2021-07-11 03:49:32.321273", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNf9MxRqxAX76lSLbZN9ibUsTmiYJfsMQyB"], "labels": [], "masters": [], "uuids": ["264118ac-5cf2-4a55-828f-929068c4caad"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:32.243610", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0764361 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0764361 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": 2021-07-11T03:49:32.016 INFO:teuthology.task.ansible.out: "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:32.022 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:32.034 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:32.191 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfsVKrklTyaCz8lxgDG2Dclz7xluWcpEFI'], 'uuids': ['d658bedb-8e1a-44a7-bebe-e4b50e14ffb6']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.078305", "end": "2021-07-11 03:49:32.521331", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-jpa1PN2A1WyXVb41j3AmkdGeB04mvUNfsVKrklTyaCz8lxgDG2Dclz7xluWcpEFI"], "labels": [], "masters": [], "uuids": ["d658bedb-8e1a-44a7-bebe-e4b50e14ffb6"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:32.443026", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0771085 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0771085 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:32.235 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:32 +0000 (0:00:00.253) 0:02:46.710 *********** 2021-07-11T03:49:32.248 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOisIxHHJ3oVeVAI6zNvyr5KGp81F0CmXrF'], 'uuids': ['a272f6c1-afd1-4547-be3f-8b4086c99f30']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.077755", "end": "2021-07-11 03:49:32.185777", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOisIxHHJ3oVeVAI6zNvyr5KGp81F0CmXrF"], "labels": [], "masters": [], "uuids": ["a272f6c1-afd1-4547-be3f-8b4086c99f30"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:32.108022", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0765603 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0765603 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:32.250 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8kEdPYlzbyL0oQvhGMqYyT2S1pWXdLag3'], 'uuids': ['48533652-5e1f-43f1-80e6-3b69415c3a8f']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.077154", "end": "2021-07-11 03:49:32.192987", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8kEdPYlzbyL0oQvhGMqYyT2S1pWXdLag3"], "labels": [], "masters": [], "uuids": ["48533652-5e1f-43f1-80e6-3b69415c3a8f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:32.115833", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0759204 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0759204 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:32.252 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:32.253 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:32.271 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:32.312 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:32.401 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOiVNs9BDnKAOLPEGYOnUTTGslh1VfJD69M'], 'uuids': ['b3922aee-d8b4-4b9e-a6c8-3084eaeafa1a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.078034", "end": "2021-07-11 03:49:32.370734", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-mt5JsOa1ANBfl0se4zOr1nnWheru3BOiVNs9BDnKAOLPEGYOnUTTGslh1VfJD69M"], "labels": [], "masters": [], "uuids": ["b3922aee-d8b4-4b9e-a6c8-3084eaeafa1a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:32.292700", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.076829 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.076829 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:32.443 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:32 +0000 (0:00:00.207) 0:02:46.917 *********** 2021-07-11T03:49:32.468 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8eo5Gsm0vOZ3Vz2Xk1nhOpZoTKV8xNcJi'], 'uuids': ['006ea518-7838-40b2-ab0b-768e62ddba17']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.077703", "end": "2021-07-11 03:49:32.445804", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8eo5Gsm0vOZ3Vz2Xk1nhOpZoTKV8xNcJi"], "labels": [], "masters": [], "uuids": ["006ea518-7838-40b2-ab0b-768e62ddba17"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:32.368101", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0764577 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0764577 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:32.486 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2K0F6XF', 'wwn-0x50000399ab9813ce'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'links': {'masters': [], 'labels': [], 'ids': ['ata-TOSHIBA_MG04ACA100N_Y9I6K2K0F6XF-part1', 'wwn-0x50000399ab9813ce-part1'], 'uuids': ['a79497c0-8b9d-46cc-9a4d-f403ab32c811']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x50000399ab9813ce', 'model': 'TOSHIBA MG04ACA1', 'serial': 'Y9I6K2K0F6XF', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)", "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2K0F6XF", "wwn-0x50000399ab9813ce"], "labels": [], "masters": [], "uuids": []}, "model": "TOSHIBA MG04ACA1", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-TOSHIBA_MG04ACA100N_Y9I6K2K0F6XF-part1", "wwn-0x50000399ab9813ce-part1"], "labels": [], "masters": [], "uuids": ["a79497c0-8b9d-46cc-9a4d-f403ab32c811"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Y9I6K2K0F6XF", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x50000399ab9813ce"}}, "skip_reason": "Cond 2021-07-11T03:49:32.490 INFO:teuthology.task.ansible.out:itional result was False"} 2021-07-11T03:49:32.561 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [gibba023.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.181418", "end": "2021-07-11 03:49:32.868724", "rc": 0, "start": "2021-07-11 03:49:32.687306", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:32.589 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:32 +0000 (0:00:00.146) 0:02:47.063 *********** 2021-07-11T03:49:32.629 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-07-11T03:49:32.631 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '732585168', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-Ui4Hoq-VUwz-Pn2f-DZPO-Azak-CZBh-B2NY5y', 'nvme-INTEL_SSDPEL1K375GA_PHKM913400JX375A', 'nvme-nvme.8086-50484b4d3931333430304a5833373541-494e54454c2053534450454c314b3337354741-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEL1K375GA', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '349.32 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.025639", "end": "2021-07-11 03:49:32.610335", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation NVMe Datacenter SSD [Optane]", "links": {"ids": ["lvm-pv-uuid-Ui4Hoq-VUwz-Pn2f-DZPO-Azak-CZBh-B2NY5y", "nvme-INTEL_SSDPEL1K375GA_PHKM913400JX375A", "nvme-nvme.8086-50484b4d3931333430304a5833373541-494e54454c2053534450454c314b3337354741-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEL1K375GA", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "732585168", "sectorsize": "512", "size": "349.32 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:32.584696", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.024338 s, 4.7 GB/s", "stderr_lines": ["11 2021-07-11T03:49:32.635 INFO:teuthology.task.ansible.out:0+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.024338 s, 4.7 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:32.653 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:32.655 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:32.785 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:32.797 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.096453", "end": "2021-07-11 03:49:33.120216", "rc": 0, "start": "2021-07-11 03:49:33.023763", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:32.821 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:32 +0000 (0:00:00.232) 0:02:47.296 *********** 2021-07-11T03:49:32.832 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-07-11T03:49:32.834 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.178721", "end": "2021-07-11 03:49:32.776175", "rc": 0, "start": "2021-07-11 03:49:32.597454", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:32.862 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:32 +0000 (0:00:00.040) 0:02:47.336 *********** 2021-07-11T03:49:32.869 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-07-11T03:49:32.873 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '27262976', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE88zg8uQlNue99T3Fhp5Tj9tPfayv6r5RB'], 'uuids': ['ee4bae7c-c614-4851-83dc-e8c15cb8d622']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '13.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.079226", "end": "2021-07-11 03:49:32.819214", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE88zg8uQlNue99T3Fhp5Tj9tPfayv6r5RB"], "labels": [], "masters": [], "uuids": ["ee4bae7c-c614-4851-83dc-e8c15cb8d622"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "27262976", "sectorsize": "512", "size": "13.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:32.739988", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0779412 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0779412 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 's 2021-07-11T03:49:32.875 INFO:teuthology.task.ansible.out:ize': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:32.875 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-07-11T03:49:32.906 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:32 +0000 (0:00:00.043) 0:02:47.380 *********** 2021-07-11T03:49:32.935 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-07-11T03:49:32.936 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:32.963 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:32 +0000 (0:00:00.057) 0:02:47.438 *********** 2021-07-11T03:49:32.991 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-07-11T03:49:32.993 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.019 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.054) 0:02:47.492 *********** 2021-07-11T03:49:33.047 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-07-11T03:49:33.049 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for physical volumes] *********************************** changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.079739", "end": "2021-07-11 03:49:33.027422", "rc": 0, "start": "2021-07-11 03:49:32.947683", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:33.074 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.056) 0:02:47.548 *********** 2021-07-11T03:49:33.101 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.027) 0:02:47.576 *********** 2021-07-11T03:49:33.109 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-07-11T03:49:33.110 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8Ho98Y6vn1q6f42kY7FSDU3Q7ZbO3H7tA'], 'uuids': ['c19933ee-4647-4ef6-8562-1bad86277352']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.078334", "end": "2021-07-11 03:49:33.056766", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8Ho98Y6vn1q6f42kY7FSDU3Q7ZbO3H7tA"], "labels": [], "masters": [], "uuids": ["c19933ee-4647-4ef6-8562-1bad86277352"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:32.978432", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0770639 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0770639 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:33.111 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.112 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-07-11T03:49:33.142 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.040) 0:02:47.616 *********** 2021-07-11T03:49:33.150 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-07-11T03:49:33.151 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.178 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.035) 0:02:47.652 *********** 2021-07-11T03:49:33.191 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.224 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.047) 0:02:47.699 *********** 2021-07-11T03:49:33.234 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-07-11T03:49:33.235 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:33.265 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.039) 0:02:47.739 *********** 2021-07-11T03:49:33.274 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-07-11T03:49:33.278 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.301 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.036) 0:02:47.775 *********** 2021-07-11T03:49:33.310 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-07-11T03:49:33.311 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '176160768', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8iPuhSSCAe9fA7Td6Ow5Qiru8bNwqgI0N'], 'uuids': ['e1cba92d-4570-487d-b10b-7147975183f2']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '84.00 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.078819", "end": "2021-07-11 03:49:33.255873", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-KodtOCIZbymUKhe4KfFNB5RJwU6G0nE8iPuhSSCAe9fA7Td6Ow5Qiru8bNwqgI0N"], "labels": [], "masters": [], "uuids": ["e1cba92d-4570-487d-b10b-7147975183f2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "176160768", "sectorsize": "512", "size": "84.00 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-07-11 03:49:33.177054", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0774979 s, 1.5 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0774979 s, 1.5 GB/s"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:33.326 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ ok: [gibba023.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:49:33.351 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.049) 0:02:47.825 *********** 2021-07-11T03:49:33.374 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.023) 0:02:47.849 *********** 2021-07-11T03:49:33.380 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-07-11T03:49:33.382 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.412 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.038) 0:02:47.887 *********** 2021-07-11T03:49:33.414 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:33.421 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-07-11T03:49:33.423 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-07-11T03:49:33.455 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.042) 0:02:47.929 *********** 2021-07-11T03:49:33.465 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [gibba023.front.sepia.ceph.com] => (item={'block_used': 13858783, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236325, 'size_available': 926663987200, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150120, 'block_size': 4096, 'inode_available': 60904856}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236325, "block_size": 4096, "block_total": 240095108, "block_used": 13858783, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904856, "inode_total": 61054976, "inode_used": 150120, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663987200, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.467 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-07-11T03:49:33.467 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.493 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.037) 0:02:47.967 *********** 2021-07-11T03:49:33.516 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.022) 0:02:47.990 *********** 2021-07-11T03:49:33.545 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-07-11T03:49:33.546 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.568 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.053) 0:02:48.043 *********** 2021-07-11T03:49:33.598 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-07-11T03:49:33.599 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} TASK [testnode : Write /scratch_devs] ****************************************** changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:33.635 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.066) 0:02:48.109 *********** 2021-07-11T03:49:33.643 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:33.645 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.670 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.034) 0:02:48.144 *********** 2021-07-11T03:49:33.677 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-07-11T03:49:33.679 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.174778", "end": "2021-07-11 03:49:33.611375", "rc": 0, "start": "2021-07-11 03:49:33.436597", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:33.720 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.049) 0:02:48.194 *********** 2021-07-11T03:49:33.733 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-07-11T03:49:33.735 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'block_used': 13858785, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236323, 'size_available': 926663979008, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150120, 'block_size': 4096, 'inode_available': 60904856}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236323, "block_size": 4096, "block_total": 240095108, "block_used": 13858785, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904856, "inode_total": 61054976, "inode_used": 150120, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663979008, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.737 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-07-11T03:49:33.738 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:33.740 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.770 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.050) 0:02:48.244 *********** 2021-07-11T03:49:33.795 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.024) 0:02:48.270 *********** 2021-07-11T03:49:33.805 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-07-11T03:49:33.807 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:33.820 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-07-11T03:49:33.822 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.846 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.051) 0:02:48.321 *********** 2021-07-11T03:49:33.857 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.882 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.034) 0:02:48.355 *********** 2021-07-11T03:49:33.895 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.896 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-07-11T03:49:33.904 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:33.931 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.045) 0:02:48.401 *********** 2021-07-11T03:49:33.938 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.939 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-07-11T03:49:33.940 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:33.967 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.040) 0:02:48.441 *********** 2021-07-11T03:49:33.993 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:33 +0000 (0:00:00.025) 0:02:48.467 *********** 2021-07-11T03:49:33.999 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:34.000 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.002 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.115709", "end": "2021-07-11 03:49:33.935708", "rc": 0, "start": "2021-07-11 03:49:33.819999", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:34.002 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.028 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.036) 0:02:48.503 *********** 2021-07-11T03:49:34.030 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:34.054 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.024) 0:02:48.528 *********** 2021-07-11T03:49:34.097 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.043) 0:02:48.571 *********** 2021-07-11T03:49:34.105 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-07-11T03:49:34.106 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.107 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:34.108 INFO:teuthology.task.ansible.out:TASK [testnode : Remove physical volumes] ************************************** TASK [testnode : Create volume_groups dict] ************************************ skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.141 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.044) 0:02:48.615 *********** 2021-07-11T03:49:34.167 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.025) 0:02:48.641 *********** 2021-07-11T03:49:34.193 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.025) 0:02:48.667 *********** 2021-07-11T03:49:34.202 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-07-11T03:49:34.203 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.233 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.040) 0:02:48.708 *********** 2021-07-11T03:49:34.257 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.023) 0:02:48.732 *********** 2021-07-11T03:49:34.265 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-07-11T03:49:34.267 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.296 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.038) 0:02:48.771 *********** 2021-07-11T03:49:34.303 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-07-11T03:49:34.305 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.337 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.037) 0:02:48.808 *********** 2021-07-11T03:49:34.358 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.384 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.050) 0:02:48.859 *********** 2021-07-11T03:49:34.393 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-07-11T03:49:34.394 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:49:34.424 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.039) 0:02:48.899 *********** 2021-07-11T03:49:34.451 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.026) 0:02:48.925 *********** 2021-07-11T03:49:34.471 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-07-11T03:49:34.472 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.505 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.052) 0:02:48.978 *********** 2021-07-11T03:49:34.518 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-07-11T03:49:34.519 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-07-11T03:49:34.565 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.062) 0:02:49.040 *********** 2021-07-11T03:49:34.632 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-07-11T03:49:34.634 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-07-11T03:49:34.659 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.092) 0:02:49.133 *********** 2021-07-11T03:49:34.712 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-07-11T03:49:34.714 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'block_used': 13858784, 'uuid': 'a79497c0-8b9d-46cc-9a4d-f403ab32c811', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226236324, 'size_available': 926663983104, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 150120, 'block_size': 4096, 'inode_available': 60904856}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226236324, "block_size": 4096, "block_total": 240095108, "block_used": 13858784, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60904856, "inode_total": 61054976, "inode_used": 150120, "mount": "/", "options": "rw,relatime,errors=remount-ro", "size_available": 926663983104, "size_total": 983429562368, "uuid": "a79497c0-8b9d-46cc-9a4d-f403ab32c811"}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.746 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.087) 0:02:49.221 *********** 2021-07-11T03:49:34.805 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.807 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-07-11T03:49:34.840 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.090) 0:02:49.311 *********** 2021-07-11T03:49:34.865 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.028) 0:02:49.340 *********** 2021-07-11T03:49:34.886 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:34.888 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:34.926 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:34 +0000 (0:00:00.060) 0:02:49.400 *********** 2021-07-11T03:49:34.975 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-07-11T03:49:34.981 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} TASK [testnode : set_fact] ***************************************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:35.005 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.076) 0:02:49.477 *********** 2021-07-11T03:49:35.033 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.029) 0:02:49.507 *********** 2021-07-11T03:49:35.052 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-07-11T03:49:35.054 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-07-11T03:49:35.054 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true} 2021-07-11T03:49:35.084 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.051) 0:02:49.559 *********** 2021-07-11T03:49:35.146 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.061) 0:02:49.620 *********** 2021-07-11T03:49:35.154 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:35.154 INFO:teuthology.task.ansible.out:TASK [testnode : Determine desired logical volume percentage size] ************* 2021-07-11T03:49:35.155 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:35.192 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.047) 0:02:49.667 *********** 2021-07-11T03:49:35.234 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-07-11T03:49:35.236 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:35.263 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.070) 0:02:49.737 *********** 2021-07-11T03:49:35.384 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-07-11T03:49:35.389 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:35.483 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:35.653 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-07-11T03:49:35.677 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.414) 0:02:50.151 *********** 2021-07-11T03:49:35.706 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-07-11T03:49:35.708 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:35.736 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.055) 0:02:50.207 *********** 2021-07-11T03:49:35.765 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:35.786 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.053) 0:02:50.261 *********** 2021-07-11T03:49:35.795 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:35.817 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:35.842 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.055) 0:02:50.317 *********** 2021-07-11T03:49:35.860 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-07-11T03:49:35.862 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-07-11T03:49:35.890 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.048) 0:02:50.365 *********** 2021-07-11T03:49:35.901 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-07-11T03:49:35.903 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create logical volume(s)] ************************************* 2021-07-11T03:49:35.903 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:35.932 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:35 +0000 (0:00:00.041) 0:02:50.406 *********** 2021-07-11T03:49:36.060 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-07-11T03:49:36.083 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:36 +0000 (0:00:00.151) 0:02:50.558 *********** 2021-07-11T03:49:36.197 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-07-11T03:49:36.199 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:36.208 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:36.209 INFO:teuthology.task.ansible.out:TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-07-11T03:49:36.212 INFO:teuthology.task.ansible.out: ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:49:36.243 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:36 +0000 (0:00:00.159) 0:02:50.718 *********** 2021-07-11T03:49:36.274 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-07-11T03:49:36.276 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:36.359 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:36.533 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:36.547 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-07-11T03:49:36.585 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:36 +0000 (0:00:00.342) 0:02:51.060 *********** 2021-07-11T03:49:36.605 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:36.665 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:36.905 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-07-11T03:49:36.907 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-07-11T03:49:36.936 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:36 +0000 (0:00:00.350) 0:02:51.410 *********** 2021-07-11T03:49:36.944 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-07-11T03:49:36.946 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:36.983 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:36 +0000 (0:00:00.046) 0:02:51.457 *********** 2021-07-11T03:49:37.001 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:37.036 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.052) 0:02:51.509 *********** 2021-07-11T03:49:37.056 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:37.136 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-07-11T03:49:37.138 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-07-11T03:49:37.167 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.132) 0:02:51.641 *********** 2021-07-11T03:49:37.176 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-07-11T03:49:37.205 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.038) 0:02:51.680 *********** 2021-07-11T03:49:37.338 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-07-11T03:49:37.340 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:37.365 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:37.402 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-07-11T03:49:37.404 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:37.415 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:49:37.443 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.235) 0:02:51.915 *********** 2021-07-11T03:49:37.450 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-07-11T03:49:37.452 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:37.496 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:37.501 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:37.535 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:37.568 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:37.634 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:37.645 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:37.679 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.238) 0:02:52.154 *********** 2021-07-11T03:49:37.689 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:37.690 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:37.723 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.041) 0:02:52.196 *********** 2021-07-11T03:49:37.729 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:37.762 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.041) 0:02:52.237 *********** 2021-07-11T03:49:37.771 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-07-11T03:49:37.773 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.140572", "end": "2021-07-11 03:49:37.663331", "rc": 0, "start": "2021-07-11 03:49:37.522759", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} TASK [testnode : Create filesystems] ******************************************* skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:37.802 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.039) 0:02:52.277 *********** 2021-07-11T03:49:37.833 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.030) 0:02:52.308 *********** 2021-07-11T03:49:37.842 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:37.844 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "msg": ""} 2021-07-11T03:49:37.846 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:37.883 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.049) 0:02:52.357 *********** 2021-07-11T03:49:37.912 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.029) 0:02:52.387 *********** 2021-07-11T03:49:37.944 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:37 +0000 (0:00:00.032) 0:02:52.419 *********** 2021-07-11T03:49:37.952 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-07-11T03:49:37.954 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:38.003 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.058) 0:02:52.478 *********** 2021-07-11T03:49:38.031 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-07-11T03:49:38.065 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.062) 0:02:52.540 *********** 2021-07-11T03:49:38.074 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-07-11T03:49:38.075 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:38.082 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:49:38.111 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.044) 0:02:52.585 *********** 2021-07-11T03:49:38.142 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.031) 0:02:52.617 *********** 2021-07-11T03:49:38.151 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:49:38.184 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.041) 0:02:52.658 *********** 2021-07-11T03:49:38.195 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-07-11T03:49:38.229 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.045) 0:02:52.704 *********** 2021-07-11T03:49:38.241 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-07-11T03:49:38.265 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.036) 0:02:52.740 *********** 2021-07-11T03:49:38.267 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:38.274 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-07-11T03:49:38.282 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:38.363 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:38.438 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-07-11T03:49:38.440 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true} 2021-07-11T03:49:38.464 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.198) 0:02:52.939 *********** 2021-07-11T03:49:38.473 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:38.474 INFO:teuthology.task.ansible.out:TASK [testnode : Write /scratch_devs] ****************************************** 2021-07-11T03:49:38.478 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [gibba023.front.sepia.ceph.com] => {"changed": true} 2021-07-11T03:49:38.531 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.067) 0:02:53.006 *********** 2021-07-11T03:49:38.586 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** changed: [gibba004.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '84G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "84G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-07-11T03:49:38.595 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '13G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "13G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:38.602 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-07-11T03:49:38.603 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:49:38.644 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.112) 0:02:53.119 *********** 2021-07-11T03:49:38.671 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.026) 0:02:53.145 *********** 2021-07-11T03:49:38.689 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} TASK [testnode : Create filesystems] ******************************************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:38.721 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.049) 0:02:53.195 *********** 2021-07-11T03:49:38.759 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.034) 0:02:53.230 *********** 2021-07-11T03:49:38.769 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Mount filesystems] ******************************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-07-11T03:49:38.801 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.045) 0:02:53.276 *********** 2021-07-11T03:49:38.824 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.023) 0:02:53.299 *********** 2021-07-11T03:49:38.853 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.028) 0:02:53.328 *********** 2021-07-11T03:49:38.860 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-07-11T03:49:38.861 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:38.888 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.035) 0:02:53.363 *********** 2021-07-11T03:49:38.896 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:38.925 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.037) 0:02:53.400 *********** 2021-07-11T03:49:38.932 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-07-11T03:49:38.933 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:38.961 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.035) 0:02:53.435 *********** 2021-07-11T03:49:38.969 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-07-11T03:49:38.971 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} TASK [testnode : Install cachefilesd conf file] ******************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:38.997 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:38 +0000 (0:00:00.036) 0:02:53.472 *********** 2021-07-11T03:49:39.020 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.022) 0:02:53.494 *********** 2021-07-11T03:49:39.027 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-07-11T03:49:39.029 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:39.070 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.050) 0:02:53.545 *********** 2021-07-11T03:49:39.081 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-07-11T03:49:39.083 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} TASK [testnode : Restart cachefilesd] ****************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:39.113 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.042) 0:02:53.587 *********** 2021-07-11T03:49:39.136 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.022) 0:02:53.610 *********** 2021-07-11T03:49:39.143 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-07-11T03:49:39.148 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Create the ntp.conf file.] ************************************ ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-07-11T03:49:39.186 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.047) 0:02:53.657 *********** 2021-07-11T03:49:39.219 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.035) 0:02:53.693 *********** 2021-07-11T03:49:39.232 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-07-11T03:49:39.233 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:39.268 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.048) 0:02:53.742 *********** 2021-07-11T03:49:39.275 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-07-11T03:49:39.277 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:39.315 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.047) 0:02:53.789 *********** 2021-07-11T03:49:39.325 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-07-11T03:49:39.327 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true} 2021-07-11T03:49:39.354 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.039) 0:02:53.829 *********** 2021-07-11T03:49:39.355 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:39.366 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-07-11T03:49:39.368 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-07-11T03:49:39.415 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.060) 0:02:53.889 *********** 2021-07-11T03:49:39.445 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-07-11T03:49:39.474 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.059) 0:02:53.949 *********** 2021-07-11T03:49:39.517 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-07-11T03:49:39.519 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:46:25 UTC", "ActiveEnterTimestampMonotonic": "14755060", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount sysinit.target systemd-tmpfiles-setup.service network.target basic.target system.slice systemd-journald.socket", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:46:24 UTC", "AssertTimestampMonotonic": "14566821", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:46:24 UTC", "ConditionTimestampMonotonic": "14566820", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": 2021-07-11T03:49:39.520 INFO:teuthology.task.ansible.out:"/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "946", "ExecMainStartTimestamp": "Sun 2021-07-11 03:46:25 UTC", "ExecMainStartTimestampMonotonic": "14755052", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:46:24 UTC", "InactiveExitTimestampMonotonic": "14567537", "InvocationID": "ec633a608a344fe8be0ab4cb4f1c623f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "Limi 2021-07-11T03:49:39.522 INFO:teuthology.task.ansible.out:tCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127825", "LimitNPROCSoft": "127825", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "946", "MemoryAccounting": "yes", "MemoryCurrent": "3792896", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "Requir 2021-07-11T03:49:39.523 INFO:teuthology.task.ansible.out:esMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:46:25 UTC", "StateChangeTimestampMonotonic": "14755060", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38347", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:49:39.548 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.073) 0:02:54.023 *********** 2021-07-11T03:49:39.571 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:49:39.604 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.055) 0:02:54.079 *********** 2021-07-11T03:49:39.612 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:39.614 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-07-11T03:49:39.648 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.044) 0:02:54.123 *********** 2021-07-11T03:49:39.676 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.027) 0:02:54.151 *********** 2021-07-11T03:49:39.690 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-07-11T03:49:39.692 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:49:39.726 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.048) 0:02:54.200 *********** 2021-07-11T03:49:39.733 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:39.762 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.036) 0:02:54.236 *********** 2021-07-11T03:49:39.773 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:39.775 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-07-11T03:49:39.802 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.040) 0:02:54.277 *********** 2021-07-11T03:49:39.821 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:39.845 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.042) 0:02:54.320 *********** 2021-07-11T03:49:39.858 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:39.886 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.040) 0:02:54.360 *********** 2021-07-11T03:49:39.895 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:39.927 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.041) 0:02:54.402 *********** 2021-07-11T03:49:39.951 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-07-11T03:49:39.952 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-07-11T03:49:39.954 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-07-11T03:49:39.956 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:39.979 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.051) 0:02:54.453 *********** 2021-07-11T03:49:40.005 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:39 +0000 (0:00:00.022) 0:02:54.475 *********** 2021-07-11T03:49:40.025 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-07-11T03:49:40.053 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:40 +0000 (0:00:00.052) 0:02:54.527 *********** 2021-07-11T03:49:40.101 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-07-11T03:49:40.103 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-07-11T03:49:40.148 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:40 +0000 (0:00:00.095) 0:02:54.623 *********** 2021-07-11T03:49:40.279 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-07-11T03:49:40.281 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:49:40.310 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:40 +0000 (0:00:00.158) 0:02:54.782 *********** 2021-07-11T03:49:40.320 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-07-11T03:49:40.356 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:40 +0000 (0:00:00.048) 0:02:54.830 *********** 2021-07-11T03:49:40.357 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:40.388 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-07-11T03:49:40.389 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:40.416 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:40 +0000 (0:00:00.059) 0:02:54.890 *********** 2021-07-11T03:49:40.600 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-07-11T03:49:40.602 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-07-11T03:49:40.630 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:40 +0000 (0:00:00.214) 0:02:55.104 *********** 2021-07-11T03:49:40.790 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-07-11T03:49:40.792 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:49:40.818 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:40 +0000 (0:00:00.187) 0:02:55.292 *********** 2021-07-11T03:49:40.858 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} 2021-07-11T03:49:40.885 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:40 +0000 (0:00:00.067) 0:02:55.359 *********** 2021-07-11T03:49:40.915 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-07-11T03:49:40.917 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} TASK [testnode : Install python-pip on apt based systems.] ********************* ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:49:40.945 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:40 +0000 (0:00:00.060) 0:02:55.419 *********** 2021-07-11T03:49:40.974 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:40 +0000 (0:00:00.028) 0:02:55.448 *********** 2021-07-11T03:49:41.019 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:41.047 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.070) 0:02:55.518 *********** 2021-07-11T03:49:41.073 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-07-11T03:49:41.075 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.106845", "end": "2021-07-11 03:49:41.047313", "rc": 0, "start": "2021-07-11 03:49:40.940468", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} 2021-07-11T03:49:41.113 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.063) 0:02:55.582 *********** 2021-07-11T03:49:41.128 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.100744", "end": "2021-07-11 03:49:41.451572", "rc": 0, "start": "2021-07-11 03:49:41.350828", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} 2021-07-11T03:49:41.162 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.048) 0:02:55.630 *********** 2021-07-11T03:49:41.164 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:41.202 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.046) 0:02:55.676 *********** 2021-07-11T03:49:41.211 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-07-11T03:49:41.213 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-07-11T03:49:41.243 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.041) 0:02:55.718 *********** 2021-07-11T03:49:41.252 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-07-11T03:49:41.254 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:41.282 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.039) 0:02:55.757 *********** 2021-07-11T03:49:41.311 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.029) 0:02:55.786 *********** 2021-07-11T03:49:41.314 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:41.346 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:41.378 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.066) 0:02:55.853 *********** 2021-07-11T03:49:41.441 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:49:41.469 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.090) 0:02:55.943 *********** 2021-07-11T03:49:41.595 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-07-11T03:49:41.597 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-07-11T03:49:41.642 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.169) 0:02:56.112 *********** 2021-07-11T03:49:41.710 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-07-11T03:49:41.712 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:49:41.744 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.105) 0:02:56.218 *********** 2021-07-11T03:49:41.752 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-07-11T03:49:41.753 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.142166", "end": "2021-07-11 03:49:41.705350", "rc": 0, "start": "2021-07-11 03:49:41.563184", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.30.0/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.30.0/Amazon/S3.pm"]} 2021-07-11T03:49:41.779 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.035) 0:02:56.254 *********** 2021-07-11T03:49:41.786 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-07-11T03:49:41.788 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:41.813 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.033) 0:02:56.288 *********** 2021-07-11T03:49:41.821 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-07-11T03:49:41.823 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:41.850 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.037) 0:02:56.325 *********** 2021-07-11T03:49:41.874 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.023) 0:02:56.348 *********** 2021-07-11T03:49:41.896 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:41.936 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.062) 0:02:56.411 *********** 2021-07-11T03:49:41.956 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:41.996 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:41 +0000 (0:00:00.059) 0:02:56.471 *********** 2021-07-11T03:49:42.103 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-07-11T03:49:42.104 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-07-11T03:49:42.133 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.136) 0:02:56.607 *********** 2021-07-11T03:49:42.162 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-07-11T03:49:42.164 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:42.189 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.056) 0:02:56.663 *********** 2021-07-11T03:49:42.289 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-07-11T03:49:42.291 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-07-11T03:49:42.316 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.127) 0:02:56.791 *********** 2021-07-11T03:49:42.344 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-07-11T03:49:42.346 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:42.371 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.054) 0:02:56.845 *********** 2021-07-11T03:49:42.379 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-07-11T03:49:42.381 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:46:18 UTC", "ActiveEnterTimestampMonotonic": "12439331", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target system.slice basic.target -.mount systemd-journald.socket systemd-tmpfiles-setup.service network.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:46:18 UTC", "AssertTimestampMonotonic": "12339024", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:46:18 UTC", "ConditionTimestampMonotonic": "12339023", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": 2021-07-11T03:49:42.382 INFO:teuthology.task.ansible.out:"/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "945", "ExecMainStartTimestamp": "Sun 2021-07-11 03:46:18 UTC", "ExecMainStartTimestampMonotonic": "12439321", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:46:18 UTC", "InactiveExitTimestampMonotonic": "12339664", "InvocationID": "6db3e2674dff406a96b85e0e6c55af88", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "Limi 2021-07-11T03:49:42.384 INFO:teuthology.task.ansible.out:tCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127825", "LimitNPROCSoft": "127825", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "945", "MemoryAccounting": "yes", "MemoryCurrent": "3579904", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "Requir 2021-07-11T03:49:42.386 INFO:teuthology.task.ansible.out:esMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:46:18 UTC", "StateChangeTimestampMonotonic": "12439331", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38347", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:49:42.412 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.040) 0:02:56.886 *********** 2021-07-11T03:49:42.422 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-07-11T03:49:42.423 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:49:42.451 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.039) 0:02:56.926 *********** 2021-07-11T03:49:42.460 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:42.462 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-07-11T03:49:42.493 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.041) 0:02:56.967 *********** 2021-07-11T03:49:42.502 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-07-11T03:49:42.504 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:42.528 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.035) 0:02:57.002 *********** 2021-07-11T03:49:42.545 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:42.547 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-07-11T03:49:42.579 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.051) 0:02:57.054 *********** 2021-07-11T03:49:42.587 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-07-11T03:49:42.589 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:46:30 UTC", "ActiveEnterTimestampMonotonic": "12360706", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "sysinit.target system.slice systemd-tmpfiles-setup.service basic.target systemd-journald.socket network.target -.mount", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:46:30 UTC", "AssertTimestampMonotonic": "12175018", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:46:30 UTC", "ConditionTimestampMonotonic": "12175017", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": 2021-07-11T03:49:42.591 INFO:teuthology.task.ansible.out:"/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "949", "ExecMainStartTimestamp": "Sun 2021-07-11 03:46:30 UTC", "ExecMainStartTimestampMonotonic": "12360698", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:46:30 UTC", "InactiveExitTimestampMonotonic": "12175648", "InvocationID": "0d8afec7579c476a94b69695ba6919b2", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "Limi 2021-07-11T03:49:42.592 INFO:teuthology.task.ansible.out:tCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127825", "LimitNPROCSoft": "127825", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "949", "MemoryAccounting": "yes", "MemoryCurrent": "3743744", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "Requir 2021-07-11T03:49:42.594 INFO:teuthology.task.ansible.out:esMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:46:30 UTC", "StateChangeTimestampMonotonic": "12360706", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38347", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:49:42.618 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.038) 0:02:57.092 *********** 2021-07-11T03:49:42.626 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-07-11T03:49:42.628 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:42.653 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.035) 0:02:57.128 *********** 2021-07-11T03:49:42.661 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:42.663 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-07-11T03:49:42.695 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.041) 0:02:57.170 *********** 2021-07-11T03:49:42.734 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:42.736 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-07-11T03:49:42.758 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.063) 0:02:57.233 *********** 2021-07-11T03:49:42.787 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-07-11T03:49:42.790 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:42.813 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.054) 0:02:57.288 *********** 2021-07-11T03:49:42.839 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.025) 0:02:57.313 *********** 2021-07-11T03:49:42.845 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-07-11T03:49:42.847 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:42.878 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:42 +0000 (0:00:00.039) 0:02:57.353 *********** 2021-07-11T03:49:43.052 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-07-11T03:49:43.054 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:46:10 UTC", "ActiveEnterTimestampMonotonic": "12449041", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "-.mount sysinit.target basic.target network.target system.slice systemd-tmpfiles-setup.service systemd-journald.socket", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:46:10 UTC", "AssertTimestampMonotonic": "12220292", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:46:10 UTC", "ConditionTimestampMonotonic": "12220291", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": 2021-07-11T03:49:43.055 INFO:teuthology.task.ansible.out:"/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documentation": "man:ntpd(8)", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "950", "ExecMainStartTimestamp": "Sun 2021-07-11 03:46:10 UTC", "ExecMainStartTimestampMonotonic": "12449032", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressPackets": "[no data]", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:46:10 UTC", "InactiveExitTimestampMonotonic": "12220967", "InvocationID": "d340d12d8302480d85ade3485a45930e", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "Limi 2021-07-11T03:49:43.057 INFO:teuthology.task.ansible.out:tCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "524288", "LimitNOFILESoft": "1024", "LimitNPROC": "127826", "LimitNPROCSoft": "127826", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127826", "LimitSIGPENDINGSoft": "127826", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "950", "MemoryAccounting": "yes", "MemoryCurrent": "4005888", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMPolicy": "stop", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice -.mount", "Requir 2021-07-11T03:49:43.059 INFO:teuthology.task.ansible.out:esMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartKillSignal": "15", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:46:10 UTC", "StateChangeTimestampMonotonic": "12449041", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "38347", "TimeoutAbortUSec": "1min 30s", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:49:43.079 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:43 +0000 (0:00:00.200) 0:02:57.553 *********** 2021-07-11T03:49:43.107 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-07-11T03:49:43.138 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:43 +0000 (0:00:00.052) 0:02:57.606 *********** 2021-07-11T03:49:43.170 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-07-11T03:49:43.194 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:43 +0000 (0:00:00.062) 0:02:57.669 *********** 2021-07-11T03:49:43.229 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:43.231 INFO:teuthology.task.ansible.out:TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-07-11T03:49:43.233 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:43.263 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:43 +0000 (0:00:00.066) 0:02:57.735 *********** 2021-07-11T03:49:43.477 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} 2021-07-11T03:49:43.504 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:43 +0000 (0:00:00.243) 0:02:57.979 *********** 2021-07-11T03:49:43.675 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:49:43.703 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:43 +0000 (0:00:00.198) 0:02:58.177 *********** 2021-07-11T03:49:43.732 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:43.757 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:43 +0000 (0:00:00.054) 0:02:58.231 *********** 2021-07-11T03:49:43.861 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-07-11T03:49:43.863 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false} 2021-07-11T03:49:43.896 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:43 +0000 (0:00:00.139) 0:02:58.371 *********** 2021-07-11T03:49:43.909 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-07-11T03:49:43.912 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} TASK [testnode : Install python-pip on apt based systems.] ********************* ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} 2021-07-11T03:49:43.938 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:43 +0000 (0:00:00.041) 0:02:58.412 *********** 2021-07-11T03:49:43.964 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:43 +0000 (0:00:00.026) 0:02:58.438 *********** 2021-07-11T03:49:43.972 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-07-11T03:49:43.974 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-07-11T03:49:44.003 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.039) 0:02:58.478 *********** 2021-07-11T03:49:44.013 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-07-11T03:49:44.014 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:44.043 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.039) 0:02:58.518 *********** 2021-07-11T03:49:44.144 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:44.149 INFO:teuthology.task.ansible.out:TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-07-11T03:49:44.149 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1625974818.7200074, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1625974818.7160072, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 158, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1625974818.7160072, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1582242096", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-07-11T03:49:44.182 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.139) 0:02:58.657 *********** 2021-07-11T03:49:44.190 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-07-11T03:49:44.192 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-07-11T03:49:44.195 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:44.223 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.040) 0:02:58.698 *********** 2021-07-11T03:49:44.344 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-07-11T03:49:44.346 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-07-11T03:49:44.372 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.149) 0:02:58.847 *********** 2021-07-11T03:49:44.374 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:44.393 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-07-11T03:49:44.395 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:49:44.431 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.058) 0:02:58.906 *********** 2021-07-11T03:49:44.433 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:44.476 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:44.477 INFO:teuthology.task.ansible.out:TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:44.510 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.077) 0:02:58.983 *********** 2021-07-11T03:49:44.595 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-07-11T03:49:44.597 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-07-11T03:49:44.624 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.115) 0:02:59.099 *********** 2021-07-11T03:49:44.645 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-07-11T03:49:44.647 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-07-11T03:49:44.677 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.052) 0:02:59.151 *********** 2021-07-11T03:49:44.704 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.027) 0:02:59.179 *********** 2021-07-11T03:49:44.746 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-07-11T03:49:44.748 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-07-11T03:49:44.778 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.073) 0:02:59.252 *********** 2021-07-11T03:49:44.808 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:44.837 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:44 +0000 (0:00:00.058) 0:02:59.310 *********** 2021-07-11T03:49:45.018 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-07-11T03:49:45.019 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-07-11T03:49:45.050 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.212) 0:02:59.523 *********** 2021-07-11T03:49:45.057 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:45.059 INFO:teuthology.task.ansible.out:TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-07-11T03:49:45.085 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.036) 0:02:59.560 *********** 2021-07-11T03:49:45.227 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-07-11T03:49:45.228 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-07-11T03:49:45.258 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.172) 0:02:59.732 *********** 2021-07-11T03:49:45.299 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.325 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.066) 0:02:59.799 *********** 2021-07-11T03:49:45.365 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-07-11T03:49:45.366 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.379 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.382 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.406 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.081) 0:02:59.880 *********** 2021-07-11T03:49:45.465 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [gibba019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.479 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba019.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.512 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.106) 0:02:59.987 *********** 2021-07-11T03:49:45.544 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.570 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.057) 0:03:00.044 *********** 2021-07-11T03:49:45.597 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-07-11T03:49:45.599 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.621 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.051) 0:03:00.096 *********** 2021-07-11T03:49:45.649 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-07-11T03:49:45.651 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.680 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.058) 0:03:00.154 *********** 2021-07-11T03:49:45.710 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.735 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.055) 0:03:00.209 *********** 2021-07-11T03:49:45.764 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-07-11T03:49:45.765 INFO:teuthology.task.ansible.out:skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.791 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.055) 0:03:00.265 *********** 2021-07-11T03:49:45.821 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.852 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.055) 0:03:00.321 *********** 2021-07-11T03:49:45.891 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.918 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.067) 0:03:00.388 *********** 2021-07-11T03:49:45.944 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:45.968 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:45 +0000 (0:00:00.054) 0:03:00.443 *********** 2021-07-11T03:49:46.000 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-07-11T03:49:46.001 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} 2021-07-11T03:49:46.029 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:46 +0000 (0:00:00.060) 0:03:00.503 *********** 2021-07-11T03:49:46.132 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:46.133 INFO:teuthology.task.ansible.out:TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-07-11T03:49:46.133 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002494", "end": "2021-07-11 03:49:46.095704", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-07-11 03:49:46.093210", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:46.241 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001731", "end": "2021-07-11 03:49:46.214950", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-07-11 03:49:46.213219", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:46.252 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-07-11T03:49:46.254 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} 2021-07-11T03:49:46.278 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:46 +0000 (0:00:00.248) 0:03:00.752 *********** 2021-07-11T03:49:46.343 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [gibba019.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001651", "end": "2021-07-11 03:49:46.318943", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-07-11 03:49:46.317292", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:46.388 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* ok: [gibba005.front.sepia.ceph.com] => {"changed": false} 2021-07-11T03:49:46.423 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:46 +0000 (0:00:00.145) 0:03:00.898 *********** 2021-07-11T03:49:46.453 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-07-11T03:49:46.490 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:46 +0000 (0:00:00.064) 0:03:00.963 *********** 2021-07-11T03:49:46.506 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-07-11T03:49:46.508 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.028857", "end": "2021-07-11 03:49:46.459161", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-07-11 03:49:46.430304", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:46.678 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:46 +0000 (0:00:00.189) 0:03:01.153 *********** 2021-07-11T03:49:46.686 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-07-11T03:49:46.687 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1625974817.9128273, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1625974817.9088273, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 158, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1625974817.9088273, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1582242096", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-07-11T03:49:46.688 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* ok: [gibba023.front.sepia.ceph.com] => {"changed": false} 2021-07-11T03:49:46.717 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:46 +0000 (0:00:00.039) 0:03:01.192 *********** 2021-07-11T03:49:46.747 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:46 +0000 (0:00:00.030) 0:03:01.222 *********** 2021-07-11T03:49:46.754 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-07-11T03:49:46.789 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:46 +0000 (0:00:00.041) 0:03:01.263 *********** 2021-07-11T03:49:46.805 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-07-11T03:49:46.806 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-07-11T03:49:46.833 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:46 +0000 (0:00:00.044) 0:03:01.308 *********** 2021-07-11T03:49:46.842 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:46.889 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-07-11T03:49:46.891 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)\nRequirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.8/dist-packages (1.2.0)", "Requirement already satisfied: execnet in /usr/local/lib/python3.8/dist-packages (from remoto>=0.0.35) (1.8.0)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python3.8/dist-packages (from execnet->remoto>=0.0.35) (1.5)"], "version": null, "virtualenv": null} TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-07-11T03:49:46.924 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:46 +0000 (0:00:00.090) 0:03:01.399 *********** 2021-07-11T03:49:46.950 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:46 +0000 (0:00:00.026) 0:03:01.425 *********** 2021-07-11T03:49:47.001 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-07-11T03:49:47.003 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1625974819.0245526, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1625974819.0205526, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 158, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1625974819.0205526, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1582242096", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-07-11T03:49:47.043 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.092) 0:03:01.517 *********** 2021-07-11T03:49:47.052 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-07-11T03:49:47.089 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.038) 0:03:01.556 *********** 2021-07-11T03:49:47.139 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-07-11T03:49:47.141 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:47.173 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.091) 0:03:01.648 *********** 2021-07-11T03:49:47.213 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-07-11T03:49:47.215 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-07-11T03:49:47.240 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.066) 0:03:01.715 *********** 2021-07-11T03:49:47.269 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-07-11T03:49:47.271 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-07-11T03:49:47.296 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.055) 0:03:01.770 *********** 2021-07-11T03:49:47.304 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-07-11T03:49:47.305 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false} TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [gibba005.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-07-11T03:49:47.333 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.037) 0:03:01.808 *********** 2021-07-11T03:49:47.357 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.024) 0:03:01.832 *********** 2021-07-11T03:49:47.365 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-07-11T03:49:47.367 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:47.368 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-07-11T03:49:47.370 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp1s0f0"}, "changed": false} 2021-07-11T03:49:47.396 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.038) 0:03:01.871 *********** 2021-07-11T03:49:47.420 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.023) 0:03:01.894 *********** 2021-07-11T03:49:47.422 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:47.504 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-07-11T03:49:47.506 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-07-11T03:49:47.535 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.112) 0:03:02.007 *********** 2021-07-11T03:49:47.541 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-07-11T03:49:47.542 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1625974824.2249348, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1625974824.2209349, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 158, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1625974824.2209349, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "1582242096", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-07-11T03:49:47.569 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.036) 0:03:02.043 *********** 2021-07-11T03:49:47.576 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-07-11T03:49:47.577 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [gibba023.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-07-11T03:49:47.610 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.034) 0:03:02.078 *********** 2021-07-11T03:49:47.627 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.023) 0:03:02.102 *********** 2021-07-11T03:49:47.657 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-07-11T03:49:47.659 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:47.670 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:47.672 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:47.710 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.081) 0:03:02.183 *********** 2021-07-11T03:49:47.721 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-07-11T03:49:47.722 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-07-11T03:49:47.769 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.057) 0:03:02.241 *********** 2021-07-11T03:49:47.792 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [gibba023.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} TASK [testnode : Define net_to_configure var] ********************************** skipping: [gibba005.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:47.793 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:47.794 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:47.826 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.052) 0:03:02.294 *********** 2021-07-11T03:49:47.846 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.026) 0:03:02.320 *********** 2021-07-11T03:49:47.853 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-07-11T03:49:47.855 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} TASK [testnode : set_fact] ***************************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:47.880 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.034) 0:03:02.355 *********** 2021-07-11T03:49:47.906 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.025) 0:03:02.381 *********** 2021-07-11T03:49:47.918 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : ifdown and ifup] ********************************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:47.939 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.032) 0:03:02.414 *********** 2021-07-11T03:49:47.961 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.021) 0:03:02.435 *********** 2021-07-11T03:49:47.968 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-07-11T03:49:47.970 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:47.997 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:47 +0000 (0:00:00.035) 0:03:02.471 *********** 2021-07-11T03:49:48.021 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [gibba023.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.031 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.061 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.063) 0:03:02.535 *********** 2021-07-11T03:49:48.084 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.022) 0:03:02.558 *********** 2021-07-11T03:49:48.090 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-07-11T03:49:48.091 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-07-11T03:49:48.119 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.035) 0:03:02.594 *********** 2021-07-11T03:49:48.129 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-07-11T03:49:48.131 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.161 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.041) 0:03:02.636 *********** 2021-07-11T03:49:48.173 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** skipping: [gibba023.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba023.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.176 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.186 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.188 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.225 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.063) 0:03:02.699 *********** 2021-07-11T03:49:48.248 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.023) 0:03:02.723 *********** 2021-07-11T03:49:48.267 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-07-11T03:49:48.268 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [gibba004.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-07-11T03:49:48.292 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.044) 0:03:02.767 *********** 2021-07-11T03:49:48.318 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.025) 0:03:02.792 *********** 2021-07-11T03:49:48.325 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:48.327 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.356 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.037) 0:03:02.830 *********** 2021-07-11T03:49:48.382 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.025) 0:03:02.856 *********** 2021-07-11T03:49:48.391 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-07-11T03:49:48.392 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.393 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [gibba004.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.422 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.039) 0:03:02.896 *********** 2021-07-11T03:49:48.430 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=enp1s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.431 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=enp1s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.433 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.435 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-07-11T03:49:48.436 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.461 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.039) 0:03:02.936 *********** 2021-07-11T03:49:48.484 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.022) 0:03:02.959 *********** 2021-07-11T03:49:48.493 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-07-11T03:49:48.494 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Define net_to_configure var] ********************************** skipping: [gibba004.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.526 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.041) 0:03:03.000 *********** 2021-07-11T03:49:48.527 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:48.538 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.539 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp1s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp1s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [gibba004.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.571 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.045) 0:03:03.045 *********** 2021-07-11T03:49:48.581 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-07-11T03:49:48.582 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.611 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.039) 0:03:03.085 *********** 2021-07-11T03:49:48.619 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-07-11T03:49:48.620 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.659 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.048) 0:03:03.134 *********** 2021-07-11T03:49:48.670 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-07-11T03:49:48.671 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.709 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.049) 0:03:03.183 *********** 2021-07-11T03:49:48.718 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-07-11T03:49:48.719 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-07-11T03:49:48.720 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002614", "end": "2021-07-11 03:49:48.651092", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-07-11 03:49:48.648478", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:48.758 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.048) 0:03:03.231 *********** 2021-07-11T03:49:48.777 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-07-11T03:49:48.779 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.812 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.055) 0:03:03.287 *********** 2021-07-11T03:49:48.821 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-07-11T03:49:48.823 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [gibba005.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001753", "end": "2021-07-11 03:49:48.787833", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-07-11 03:49:48.786080", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:48.862 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.049) 0:03:03.336 *********** 2021-07-11T03:49:48.871 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:48.872 INFO:teuthology.task.ansible.out:TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.902 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.040) 0:03:03.376 *********** 2021-07-11T03:49:48.906 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:48.916 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-07-11T03:49:48.918 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.945 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.040) 0:03:03.417 *********** 2021-07-11T03:49:48.952 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:48.987 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:48 +0000 (0:00:00.045) 0:03:03.462 *********** 2021-07-11T03:49:48.998 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [gibba005.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001636", "end": "2021-07-11 03:49:48.915778", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-07-11 03:49:48.914142", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:49.038 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:49 +0000 (0:00:00.045) 0:03:03.507 *********** 2021-07-11T03:49:49.041 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-07-11T03:49:49.042 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:49.067 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:49 +0000 (0:00:00.034) 0:03:03.541 *********** 2021-07-11T03:49:49.073 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:49.104 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:49 +0000 (0:00:00.037) 0:03:03.579 *********** 2021-07-11T03:49:49.126 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-07-11T03:49:49.127 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.026111", "end": "2021-07-11 03:49:49.056437", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-07-11 03:49:49.030326", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:49.157 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:49 +0000 (0:00:00.053) 0:03:03.632 *********** 2021-07-11T03:49:49.167 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:49.168 INFO:teuthology.task.ansible.out:TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:49.214 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:49 +0000 (0:00:00.056) 0:03:03.688 *********** 2021-07-11T03:49:49.223 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:49.225 INFO:teuthology.task.ansible.out:TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-07-11T03:49:49.227 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002537", "end": "2021-07-11 03:49:49.179097", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-07-11 03:49:49.176560", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:49.258 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:49 +0000 (0:00:00.044) 0:03:03.732 *********** 2021-07-11T03:49:49.266 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:49.294 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:49 +0000 (0:00:00.036) 0:03:03.768 *********** 2021-07-11T03:49:49.302 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-07-11T03:49:49.303 INFO:teuthology.task.ansible.out:skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:49.334 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [gibba023.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001736", "end": "2021-07-11 03:49:49.290905", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-07-11 03:49:49.289169", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:49.424 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001622", "end": "2021-07-11 03:49:49.400964", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-07-11 03:49:49.399342", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:49.435 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.002500", "end": "2021-07-11 03:49:49.413973", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-07-11 03:49:49.411473", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:49.548 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.001755", "end": "2021-07-11 03:49:49.520673", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-07-11 03:49:49.518918", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:49.572 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.022940", "end": "2021-07-11 03:49:49.540005", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-07-11 03:49:49.517065", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:49.599 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:49 +0000 (0:00:00.305) 0:03:04.074 *********** 2021-07-11T03:49:49.601 INFO:teuthology.task.ansible.out: 2021-07-11T03:49:49.628 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-07-11T03:49:49.652 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:49 +0000 (0:00:00.052) 0:03:04.127 *********** 2021-07-11T03:49:49.670 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-07-11T03:49:49.671 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.001780", "end": "2021-07-11 03:49:49.638018", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-07-11 03:49:49.636238", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:49.682 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:49.799 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-07-11T03:49:49.801 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.031354", "end": "2021-07-11 03:49:49.775515", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-07-11 03:49:49.744161", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:49:49.823 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:49 +0000 (0:00:00.171) 0:03:04.298 *********** 2021-07-11T03:49:49.850 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-07-11T03:49:49.881 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:49 +0000 (0:00:00.057) 0:03:04.355 *********** 2021-07-11T03:49:49.911 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-07-11T03:49:49.913 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:50.002 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-07-11T03:49:50.023 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:50 +0000 (0:00:00.142) 0:03:04.498 *********** 2021-07-11T03:49:50.050 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:50 +0000 (0:00:00.024) 0:03:04.522 *********** 2021-07-11T03:49:50.081 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:50 +0000 (0:00:00.024) 0:03:04.547 *********** 2021-07-11T03:49:50.096 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:50 +0000 (0:00:00.023) 0:03:04.571 *********** 2021-07-11T03:49:50.957 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [gibba019.front.sepia.ceph.com] 2021-07-11T03:49:51.016 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.919) 0:03:05.491 *********** 2021-07-11T03:49:51.040 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] 2021-07-11T03:49:51.053 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] 2021-07-11T03:49:51.068 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] 2021-07-11T03:49:51.078 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-07-11T03:49:51.079 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-07-11T03:49:51.131 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.115) 0:03:05.606 *********** 2021-07-11T03:49:51.159 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:51.185 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.053) 0:03:05.660 *********** 2021-07-11T03:49:51.216 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.031) 0:03:05.691 *********** 2021-07-11T03:49:51.245 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.028) 0:03:05.719 *********** 2021-07-11T03:49:51.271 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.025) 0:03:05.745 *********** 2021-07-11T03:49:51.280 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-07-11T03:49:51.282 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-07-11T03:49:51.284 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** ok: [gibba019.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-07-11T03:49:51.314 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.041) 0:03:05.787 *********** 2021-07-11T03:49:51.337 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.024) 0:03:05.811 *********** 2021-07-11T03:49:51.359 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.022) 0:03:05.833 *********** 2021-07-11T03:49:51.366 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-07-11T03:49:51.368 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} TASK [container-host : set_fact] *********************************************** skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:51.396 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.037) 0:03:05.871 *********** 2021-07-11T03:49:51.432 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.035) 0:03:05.906 *********** 2021-07-11T03:49:51.459 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.025) 0:03:05.932 *********** 2021-07-11T03:49:51.466 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-07-11T03:49:51.468 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-07-11T03:49:51.514 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.055) 0:03:05.988 *********** 2021-07-11T03:49:51.524 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-07-11T03:49:51.527 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-07-11T03:49:51.529 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:49:51.564 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.050) 0:03:06.039 *********** 2021-07-11T03:49:51.600 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.036) 0:03:06.075 *********** 2021-07-11T03:49:51.651 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** ok: [gibba023.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-07-11T03:49:51.676 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:49:51 +0000 (0:00:00.075) 0:03:06.151 *********** 2021-07-11T03:50:03.964 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** changed: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSuggested packages:\n aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 141 not upgraded.\nNeed to get 36.9 MB of archives.\nAfter this operation, 931 kB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.2-0ubuntu1~20.04.2 [36.9 MB]\nPreconfiguring packages ...\nFetched 36.9 MB in 3s (13.0 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138110 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_20.10.2-0ubuntu1~20.04.2_amd64.deb ...\r\nUnpacking docker.io (20.10.2-0ubuntu1~20.04.2) over (19.03.8-0ubuntu1.20.04.2) ...\r\nSetting up docker.io (20.10.2-0ubuntu1~20.04.2) ...\r\nCreated symlink /etc/systemd/system/multi-user.target.wants/docker.service -> /lib/systemd/system/docker.service.\r\nProcessing triggers for man-db (2.9.1-1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Suggested packages:", " aufs 2021-07-11T03:50:03.966 INFO:teuthology.task.ansible.out:-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following packages will be upgraded:", " docker.io", "1 upgraded, 0 newly installed, 0 to remove and 141 not upgraded.", "Need to get 36.9 MB of archives.", "After this operation, 931 kB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.2-0ubuntu1~20.04.2 [36.9 MB]", "Preconfiguring packages ...", "Fetched 36.9 MB in 3s (13.0 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138110 files and directories currently installed.)", "Preparing to unpack .../docker.io_20.10.2-0ubuntu1~20.04.2_amd64.deb ...", "Unpacking docker.io (20.10.2-0ubuntu1~20.04.2) over (19.03.8-0ubuntu1.20.04.2) ...", "Setting up docker.io (20.10.2-0ubuntu1~20.04.2) ...", "Created symlink /etc/systemd/system/multi-user.target.wants/docker.service -> /lib/systemd/system/docker.service.", "Processing triggers for man-db (2.9.1-1) ..."]} 2021-07-11T03:50:03.999 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:03 +0000 (0:00:12.312) 0:03:18.463 *********** 2021-07-11T03:50:04.033 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [gibba019.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:50:04.057 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.068) 0:03:18.532 *********** 2021-07-11T03:50:04.112 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** ok: [gibba019.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-07-11T03:50:04.133 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.075) 0:03:18.607 *********** 2021-07-11T03:50:04.165 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-07-11T03:50:04.171 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSuggested packages:\n aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 141 not upgraded.\nNeed to get 36.9 MB of archives.\nAfter this operation, 931 kB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.2-0ubuntu1~20.04.2 [36.9 MB]\nPreconfiguring packages ...\nFetched 36.9 MB in 3s (12.7 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138110 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_20.10.2-0ubuntu1~20.04.2_amd64.deb ...\r\nUnpacking docker.io (20.10.2-0ubuntu1~20.04.2) over (19.03.8-0ubuntu1.20.04.2) ...\r\nSetting up docker.io (20.10.2-0ubuntu1~20.04.2) ...\r\nCreated symlink /etc/systemd/system/multi-user.target.wants/docker.service -> /lib/systemd/system/docker.service.\r\nProcessing triggers for man-db (2.9.1-1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Suggested packages:", " aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following 2021-07-11T03:50:04.172 INFO:teuthology.task.ansible.out: packages will be upgraded:", " docker.io", "1 upgraded, 0 newly installed, 0 to remove and 141 not upgraded.", "Need to get 36.9 MB of archives.", "After this operation, 931 kB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.2-0ubuntu1~20.04.2 [36.9 MB]", "Preconfiguring packages ...", "Fetched 36.9 MB in 3s (12.7 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138110 files and directories currently installed.)", "Preparing to unpack .../docker.io_20.10.2-0ubuntu1~20.04.2_amd64.deb ...", "Unpacking docker.io (20.10.2-0ubuntu1~20.04.2) over (19.03.8-0ubuntu1.20.04.2) ...", "Setting up docker.io (20.10.2-0ubuntu1~20.04.2) ...", "Created symlink /etc/systemd/system/multi-user.target.wants/docker.service -> /lib/systemd/system/docker.service.", "Processing triggers for man-db (2.9.1-1) ..."]} 2021-07-11T03:50:04.200 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.067) 0:03:18.675 *********** 2021-07-11T03:50:04.236 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-07-11T03:50:04.237 INFO:teuthology.task.ansible.out:skipping: [gibba023.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:50:04.262 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.062) 0:03:18.737 *********** 2021-07-11T03:50:04.333 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** ok: [gibba023.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-07-11T03:50:04.358 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.095) 0:03:18.833 *********** 2021-07-11T03:50:04.378 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-07-11T03:50:04.380 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:50:04.393 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-07-11T03:50:04.394 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSuggested packages:\n aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 141 not upgraded.\nNeed to get 36.9 MB of archives.\nAfter this operation, 931 kB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.2-0ubuntu1~20.04.2 [36.9 MB]\nPreconfiguring packages ...\nFetched 36.9 MB in 3s (12.9 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138110 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_20.10.2-0ubuntu1~20.04.2_amd64.deb ...\r\nUnpacking docker.io (20.10.2-0ubuntu1~20.04.2) over (19.03.8-0ubuntu1.20.04.2) ...\r\nSetting up docker.io (20.10.2-0ubuntu1~20.04.2) ...\r\nCreated symlink /etc/systemd/system/multi-user.target.wants/docker.service -> /lib/systemd/system/docker.service.\r\nProcessing triggers for man-db (2.9.1-1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Suggested packages:", " aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following 2021-07-11T03:50:04.396 INFO:teuthology.task.ansible.out: packages will be upgraded:", " docker.io", "1 upgraded, 0 newly installed, 0 to remove and 141 not upgraded.", "Need to get 36.9 MB of archives.", "After this operation, 931 kB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.2-0ubuntu1~20.04.2 [36.9 MB]", "Preconfiguring packages ...", "Fetched 36.9 MB in 3s (12.9 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138110 files and directories currently installed.)", "Preparing to unpack .../docker.io_20.10.2-0ubuntu1~20.04.2_amd64.deb ...", "Unpacking docker.io (20.10.2-0ubuntu1~20.04.2) over (19.03.8-0ubuntu1.20.04.2) ...", "Setting up docker.io (20.10.2-0ubuntu1~20.04.2) ...", "Created symlink /etc/systemd/system/multi-user.target.wants/docker.service -> /lib/systemd/system/docker.service.", "Processing triggers for man-db (2.9.1-1) ..."]} 2021-07-11T03:50:04.431 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.064) 0:03:18.897 *********** 2021-07-11T03:50:04.466 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [gibba005.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:50:04.510 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.081) 0:03:18.979 *********** 2021-07-11T03:50:04.536 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-07-11T03:50:04.538 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:50:04.577 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.072) 0:03:19.051 *********** 2021-07-11T03:50:04.588 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-07-11T03:50:04.590 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-07-11T03:50:04.590 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:50:04.638 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.056) 0:03:19.108 *********** 2021-07-11T03:50:04.669 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-07-11T03:50:04.671 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": true, "stderr": "", "stderr_lines": [], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nSuggested packages:\n aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils\nThe following packages will be upgraded:\n docker.io\n1 upgraded, 0 newly installed, 0 to remove and 141 not upgraded.\nNeed to get 36.9 MB of archives.\nAfter this operation, 931 kB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.2-0ubuntu1~20.04.2 [36.9 MB]\nPreconfiguring packages ...\nFetched 36.9 MB in 3s (12.8 MB/s)\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 138110 files and directories currently installed.)\r\nPreparing to unpack .../docker.io_20.10.2-0ubuntu1~20.04.2_amd64.deb ...\r\nUnpacking docker.io (20.10.2-0ubuntu1~20.04.2) over (19.03.8-0ubuntu1.20.04.2) ...\r\nSetting up docker.io (20.10.2-0ubuntu1~20.04.2) ...\r\nCreated symlink /etc/systemd/system/multi-user.target.wants/docker.service -> /lib/systemd/system/docker.service.\r\nProcessing triggers for man-db (2.9.1-1) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "Suggested packages:", " aufs-tools btrfs-progs debootstrap docker-doc rinse zfs-fuse | zfsutils", "The following 2021-07-11T03:50:04.675 INFO:teuthology.task.ansible.out: packages will be upgraded:", " docker.io", "1 upgraded, 0 newly installed, 0 to remove and 141 not upgraded.", "Need to get 36.9 MB of archives.", "After this operation, 931 kB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 docker.io amd64 20.10.2-0ubuntu1~20.04.2 [36.9 MB]", "Preconfiguring packages ...", "Fetched 36.9 MB in 3s (12.8 MB/s)", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 138110 files and directories currently installed.)", "Preparing to unpack .../docker.io_20.10.2-0ubuntu1~20.04.2_amd64.deb ...", "Unpacking docker.io (20.10.2-0ubuntu1~20.04.2) over (19.03.8-0ubuntu1.20.04.2) ...", "Setting up docker.io (20.10.2-0ubuntu1~20.04.2) ...", "Created symlink /etc/systemd/system/multi-user.target.wants/docker.service -> /lib/systemd/system/docker.service.", "Processing triggers for man-db (2.9.1-1) ..."]} 2021-07-11T03:50:04.706 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.072) 0:03:19.180 *********** 2021-07-11T03:50:04.743 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-07-11T03:50:04.754 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:50:04.832 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.126) 0:03:19.307 *********** 2021-07-11T03:50:04.860 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-07-11T03:50:04.863 INFO:teuthology.task.ansible.out:skipping: [gibba004.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-07-11T03:50:04.871 INFO:teuthology.task.ansible.out: 2021-07-11T03:50:04.945 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:04 +0000 (0:00:00.111) 0:03:19.418 *********** 2021-07-11T03:50:04.954 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-07-11T03:50:04.956 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:50:05.085 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:50:05.156 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:05 +0000 (0:00:00.212) 0:03:19.630 *********** 2021-07-11T03:50:05.187 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-07-11T03:50:05.189 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-07-11T03:50:05.262 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:05 +0000 (0:00:00.103) 0:03:19.734 *********** 2021-07-11T03:50:05.276 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-07-11T03:50:05.278 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975404.675403-15245-46444019833777/source", "state": "file", "uid": 0} 2021-07-11T03:50:05.528 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975404.9811645-15310-133653961744480/source", "state": "file", "uid": 0} 2021-07-11T03:50:05.593 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* changed: [gibba004.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:50:05.701 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-07-11T03:50:05.703 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975405.282124-15388-252420538083327/source", "state": "file", "uid": 0} 2021-07-11T03:50:05.759 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-07-11T03:50:05.761 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-07-11T03:50:05.762 INFO:teuthology.task.ansible.out: 2021-07-11T03:50:05.812 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:05 +0000 (0:00:00.551) 0:03:20.285 *********** 2021-07-11T03:50:05.844 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-07-11T03:50:05.847 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975405.3154047-15245-170292565861333/source", "state": "file", "uid": 0} 2021-07-11T03:50:05.887 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:05 +0000 (0:00:00.076) 0:03:20.362 *********** 2021-07-11T03:50:06.027 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975405.5554519-15310-198600984292424/source", "state": "file", "uid": 0} 2021-07-11T03:50:06.069 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:06 +0000 (0:00:00.181) 0:03:20.544 *********** 2021-07-11T03:50:06.144 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975405.7432537-15388-158768132678281/source", "state": "file", "uid": 0} 2021-07-11T03:50:06.179 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:06 +0000 (0:00:00.110) 0:03:20.654 *********** 2021-07-11T03:50:06.315 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975405.9148607-15567-180234858671820/source", "state": "file", "uid": 0} 2021-07-11T03:50:06.421 INFO:teuthology.task.ansible.out: TASK [container-host : Ensure git is installed] ******************************** ok: [gibba019.front.sepia.ceph.com] => {"cache_update_time": 1625974942, "cache_updated": false, "changed": false} 2021-07-11T03:50:06.440 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:06 +0000 (0:00:00.260) 0:03:20.914 *********** 2021-07-11T03:50:06.574 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] => {"cache_update_time": 1625974990, "cache_updated": false, "changed": false} 2021-07-11T03:50:06.626 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:06 +0000 (0:00:00.184) 0:03:21.099 *********** 2021-07-11T03:50:06.697 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"cache_update_time": 1625974892, "cache_updated": false, "changed": false} 2021-07-11T03:50:06.723 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:06 +0000 (0:00:00.093) 0:03:21.193 *********** 2021-07-11T03:50:06.774 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-07-11T03:50:06.776 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975406.3382392-15567-32398544350168/source", "state": "file", "uid": 0} 2021-07-11T03:50:06.815 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:06 +0000 (0:00:00.096) 0:03:21.290 *********** 2021-07-11T03:50:07.346 INFO:teuthology.task.ansible.out: TASK [container-host : Ensure git is installed] ******************************** 2021-07-11T03:50:07.348 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"cache_update_time": 1625974840, "cache_updated": false, "changed": false} 2021-07-11T03:50:07.379 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:07 +0000 (0:00:00.564) 0:03:21.854 *********** 2021-07-11T03:50:10.137 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": " Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-sgvja3fp\n", "stderr_lines": [" Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-sgvja3fp"], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-sgvja3fp\nCollecting docopt\n Downloading docopt-0.6.2.tar.gz (25 kB)\nCollecting toml\n Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)\nBuilding wheels for collected packages: registries-conf-ctl, docopt\n Building wheel for registries-conf-ctl (setup.py): started\n Building wheel for registries-conf-ctl (setup.py): finished with status 'done'\n Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=04ea4a67b0b437c3df758faeb90817b9da55643d2dd5eee435220b3df62b1934\n Stored in directory: /tmp/pip-ephem-wheel-cache-whkvg2jf/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42\n Building wheel for docopt (setup.py): started\n Building wheel for docopt (setup.py): finished with status 'done'\n Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=5295f7f3e52f784e1f9b9ef791f868e7c18e7afb5dc41877f378710c5db38358\n Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c\nSuccessfully built registries-conf-ctl docopt\nInstalling collected packages: docopt, toml, registries-conf-ctl\nSuccessfully installed docopt-0.6.2 regis 2021-07-11T03:50:10.139 INFO:teuthology.task.ansible.out:tries-conf-ctl-1.0 toml-0.10.2\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-sgvja3fp", "Collecting docopt", " Downloading docopt-0.6.2.tar.gz (25 kB)", "Collecting toml", " Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)", "Building wheels for collected packages: registries-conf-ctl, docopt", " Building wheel for registries-conf-ctl (setup.py): started", " Building wheel for registries-conf-ctl (setup.py): finished with status 'done'", " Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=04ea4a67b0b437c3df758faeb90817b9da55643d2dd5eee435220b3df62b1934", " Stored in directory: /tmp/pip-ephem-wheel-cache-whkvg2jf/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42", " Building wheel for docopt (setup.py): started", " Building wheel for docopt (setup.py): finished with status 'done'", " Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=5295f7f3e52f784e1f9b9ef791f868e7c18e7afb5dc41877f378710c5db38358", " Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c", "Successfully built registries-conf-ctl docopt", "Installing collected packages: docopt, toml, registries-conf-ctl", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2"], "version": null, "virtualenv": null} 2021-07-11T03:50:10.173 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:10 +0000 (0:00:02.793) 0:03:24.648 *********** 2021-07-11T03:50:10.331 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-07-11T03:50:10.333 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} TASK [container-host : Install registries-conf-ctl] **************************** changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": " Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-fqejgrpy\n", "stderr_lines": [" Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-fqejgrpy"], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-fqejgrpy\nCollecting docopt\n Downloading docopt-0.6.2.tar.gz (25 kB)\nCollecting toml\n Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)\nBuilding wheels for collected packages: registries-conf-ctl, docopt\n Building wheel for registries-conf-ctl (setup.py): started\n Building wheel for registries-conf-ctl (setup.py): finished with status 'done'\n Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=04ea4a67b0b437c3df758faeb90817b9da55643d2dd5eee435220b3df62b1934\n Stored in directory: /tmp/pip-ephem-wheel-cache-_jfa6nge/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42\n Building wheel for docopt (setup.py): started\n Building wheel for docopt (setup.py): finished with status 'done'\n Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=5295f7f3e52f784e1f9b9ef791f868e7c18e7afb5dc41877f378710c5db38358\n Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c\nSuccessfully built registries-conf-ctl docopt\nInstalling collected 2021-07-11T03:50:10.335 INFO:teuthology.task.ansible.out:packages: docopt, toml, registries-conf-ctl\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-fqejgrpy", "Collecting docopt", " Downloading docopt-0.6.2.tar.gz (25 kB)", "Collecting toml", " Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)", "Building wheels for collected packages: registries-conf-ctl, docopt", " Building wheel for registries-conf-ctl (setup.py): started", " Building wheel for registries-conf-ctl (setup.py): finished with status 'done'", " Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=04ea4a67b0b437c3df758faeb90817b9da55643d2dd5eee435220b3df62b1934", " Stored in directory: /tmp/pip-ephem-wheel-cache-_jfa6nge/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42", " Building wheel for docopt (setup.py): started", " Building wheel for docopt (setup.py): finished with status 'done'", " Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=5295f7f3e52f784e1f9b9ef791f868e7c18e7afb5dc41877f378710c5db38358", " Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c", "Successfully built registries-conf-ctl docopt", "Installing collected packages: docopt, toml, registries-conf-ctl", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2"], "version": null, "virtualenv": null} 2021-07-11T03:50:10.378 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:10 +0000 (0:00:00.194) 0:03:24.843 *********** 2021-07-11T03:50:10.408 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:10 +0000 (0:00:00.039) 0:03:24.882 *********** 2021-07-11T03:50:10.618 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": " Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-zkfug__b\n", "stderr_lines": [" Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-zkfug__b"], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-zkfug__b\nCollecting docopt\n Downloading docopt-0.6.2.tar.gz (25 kB)\nCollecting toml\n Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)\nBuilding wheels for collected packages: registries-conf-ctl, docopt\n Building wheel for registries-conf-ctl (setup.py): started\n Building wheel for registries-conf-ctl (setup.py): finished with status 'done'\n Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=04ea4a67b0b437c3df758faeb90817b9da55643d2dd5eee435220b3df62b1934\n Stored in directory: /tmp/pip-ephem-wheel-cache-h36z31tj/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42\n Building wheel for docopt (setup.py): started\n Building wheel for docopt (setup.py): finished with status 'done'\n Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=f95f092f66515728a64ec060b4a0b5128d43fdacf08ef27f6a98460077210f90\n Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c\nSuccessfully built registries-conf-ctl docopt\nInstalling collected packages: docopt, toml, registries-conf-ctl\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2\n", "stdout_lines": ["Collecting git+https://github.c 2021-07-11T03:50:10.620 INFO:teuthology.task.ansible.out:om/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-zkfug__b", "Collecting docopt", " Downloading docopt-0.6.2.tar.gz (25 kB)", "Collecting toml", " Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)", "Building wheels for collected packages: registries-conf-ctl, docopt", " Building wheel for registries-conf-ctl (setup.py): started", " Building wheel for registries-conf-ctl (setup.py): finished with status 'done'", " Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=04ea4a67b0b437c3df758faeb90817b9da55643d2dd5eee435220b3df62b1934", " Stored in directory: /tmp/pip-ephem-wheel-cache-h36z31tj/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42", " Building wheel for docopt (setup.py): started", " Building wheel for docopt (setup.py): finished with status 'done'", " Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=f95f092f66515728a64ec060b4a0b5128d43fdacf08ef27f6a98460077210f90", " Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c", "Successfully built registries-conf-ctl docopt", "Installing collected packages: docopt, toml, registries-conf-ctl", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2"], "version": null, "virtualenv": null} TASK [container-host : Check for docker's daemon.json] ************************* ok: [gibba023.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-07-11T03:50:10.655 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:10 +0000 (0:00:00.246) 0:03:25.129 *********** 2021-07-11T03:50:10.683 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:10 +0000 (0:00:00.028) 0:03:25.158 *********** 2021-07-11T03:50:10.818 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-07-11T03:50:10.870 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:10 +0000 (0:00:00.186) 0:03:25.345 *********** 2021-07-11T03:50:11.006 INFO:teuthology.task.ansible.out: 2021-07-11T03:50:11.016 INFO:teuthology.task.ansible.out:TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975410.4553149-16676-134569326935938/source", "state": "file", "uid": 0} 2021-07-11T03:50:11.071 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:11 +0000 (0:00:00.189) 0:03:25.534 *********** 2021-07-11T03:50:11.280 INFO:teuthology.task.ansible.out: 2021-07-11T03:50:11.286 INFO:teuthology.task.ansible.out:TASK [container-host : Add local docker.io registry mirror] ******************** changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.019142", "end": "2021-07-11 03:50:11.255345", "rc": 0, "start": "2021-07-11 03:50:11.236203", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:50:11.311 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:11 +0000 (0:00:00.249) 0:03:25.783 *********** 2021-07-11T03:50:11.318 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-07-11T03:50:11.320 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975410.8540063-16772-147256149603274/source", "state": "file", "uid": 0} 2021-07-11T03:50:11.363 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:11 +0000 (0:00:00.054) 0:03:25.838 *********** 2021-07-11T03:50:11.389 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975410.98253-16822-29781651181995/source", "state": "file", "uid": 0} 2021-07-11T03:50:11.457 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:11 +0000 (0:00:00.093) 0:03:25.932 *********** 2021-07-11T03:50:11.506 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-07-11T03:50:11.511 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": " Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-q1p9rkwj\n", "stderr_lines": [" Running command git clone -q https://github.com/sebastian-philipp/registries-conf-ctl /tmp/pip-req-build-q1p9rkwj"], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-q1p9rkwj\nCollecting docopt\n Downloading docopt-0.6.2.tar.gz (25 kB)\nCollecting toml\n Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)\nBuilding wheels for collected packages: registries-conf-ctl, docopt\n Building wheel for registries-conf-ctl (setup.py): started\n Building wheel for registries-conf-ctl (setup.py): finished with status 'done'\n Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=fafde2ca333d33a59e9bc0ed55395761006e308afe799a8211a526e77133fc2b\n Stored in directory: /tmp/pip-ephem-wheel-cache-_dhtpqnm/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42\n Building wheel for docopt (setup.py): started\n Building wheel for docopt (setup.py): finished with status 'done'\n Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=43df9f3564819520a8d0da0e62f4181d09af940f690b87613b43baa8ba294f90\n Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c\nSuccessfully built registries-conf-ctl docopt\nInstalling collected packages: docopt, toml, registries-conf-ctl\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2\n", "stdout_lines": ["Collecting git+https://github.c 2021-07-11T03:50:11.513 INFO:teuthology.task.ansible.out:om/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-req-build-q1p9rkwj", "Collecting docopt", " Downloading docopt-0.6.2.tar.gz (25 kB)", "Collecting toml", " Downloading toml-0.10.2-py2.py3-none-any.whl (16 kB)", "Building wheels for collected packages: registries-conf-ctl, docopt", " Building wheel for registries-conf-ctl (setup.py): started", " Building wheel for registries-conf-ctl (setup.py): finished with status 'done'", " Created wheel for registries-conf-ctl: filename=registries_conf_ctl-1.0-py3-none-any.whl size=6035 sha256=fafde2ca333d33a59e9bc0ed55395761006e308afe799a8211a526e77133fc2b", " Stored in directory: /tmp/pip-ephem-wheel-cache-_dhtpqnm/wheels/66/49/be/0ca4a7322f6cd17532451655790e478b4f82776818b44d6d42", " Building wheel for docopt (setup.py): started", " Building wheel for docopt (setup.py): finished with status 'done'", " Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=43df9f3564819520a8d0da0e62f4181d09af940f690b87613b43baa8ba294f90", " Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c", "Successfully built registries-conf-ctl docopt", "Installing collected packages: docopt, toml, registries-conf-ctl", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2"], "version": null, "virtualenv": null} 2021-07-11T03:50:11.545 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:11 +0000 (0:00:00.087) 0:03:26.019 *********** 2021-07-11T03:50:11.546 INFO:teuthology.task.ansible.out: 2021-07-11T03:50:11.596 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-07-11T03:50:11.598 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.018691", "end": "2021-07-11 03:50:11.569061", "rc": 0, "start": "2021-07-11 03:50:11.550370", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:50:11.599 INFO:teuthology.task.ansible.out: 2021-07-11T03:50:11.648 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:11 +0000 (0:00:00.103) 0:03:26.123 *********** 2021-07-11T03:50:11.653 INFO:teuthology.task.ansible.out: 2021-07-11T03:50:11.666 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.018963", "end": "2021-07-11 03:50:11.621398", "rc": 0, "start": "2021-07-11 03:50:11.602435", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:50:11.668 INFO:teuthology.task.ansible.out: 2021-07-11T03:50:11.700 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:11 +0000 (0:00:00.048) 0:03:26.172 *********** 2021-07-11T03:50:11.718 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-07-11T03:50:11.719 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-07-11T03:50:11.775 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:11 +0000 (0:00:00.073) 0:03:26.245 *********** 2021-07-11T03:50:12.204 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1625975411.8584294-17050-122133851917582/source", "state": "file", "uid": 0} 2021-07-11T03:50:12.236 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:12 +0000 (0:00:00.465) 0:03:26.710 *********** 2021-07-11T03:50:12.419 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-07-11T03:50:12.421 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.019016", "end": "2021-07-11 03:50:12.400205", "rc": 0, "start": "2021-07-11 03:50:12.381189", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-07-11T03:50:12.461 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:12 +0000 (0:00:00.225) 0:03:26.936 *********** 2021-07-11T03:50:12.971 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-07-11T03:50:12.973 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:50:01 UTC", "ActiveEnterTimestampMonotonic": "231054947", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "containerd.service sysinit.target docker.socket network-online.target systemd-journald.socket firewalld.service system.slice basic.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:49:59 UTC", "AssertTimestampMonotonic": "229376048", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:49:59 UTC", "ConditionTimestampMonotonic": "229376048", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slic 2021-07-11T03:50:12.975 INFO:teuthology.task.ansible.out:e/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "yes", "DelegateControllers": "cpu cpuacct cpuset io blkio memory devices pids bpf-firewall bpf-devices", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8982", "ExecMainStartTimestamp": "Sun 2021-07-11 03:49:59 UTC", "ExecMainStartTimestampMonotonic": "229376434", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[Sun 2021-07-11 03:49:59 UTC] ; stop_time=[n/a] ; pid=8982 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; flags= ; start_time=[Sun 2021-07-11 03:49:59 UTC] ; stop_time=[n/a] ; pid=8982 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressP 2021-07-11T03:50:12.977 INFO:teuthology.task.ansible.out:ackets": "[no data]", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:49:59 UTC", "InactiveExitTimestampMonotonic": "229376543", "InvocationID": "4af30c8c79d8446eb76620a436d3cbe5", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "8982", "MemoryAccounting": "yes", "MemoryCurrent": "55545856", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMPolicy": "continu 2021-07-11T03:50:12.979 INFO:teuthology.task.ansible.out:e", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket sysinit.target system.slice", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:50:01 UTC", "StateChangeTimestampMonotonic": "231054947", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "21", "TasksMax": "infinity", "TimeoutAbortUSec": "infinity", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "n 2021-07-11T03:50:12.981 INFO:teuthology.task.ansible.out:otify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "Wants": "network-online.target containerd.service", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:50:13.279 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:50:01 UTC", "ActiveEnterTimestampMonotonic": "223874803", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "containerd.service basic.target systemd-journald.socket system.slice sysinit.target docker.socket network-online.target firewalld.service", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:50:00 UTC", "AssertTimestampMonotonic": "222213075", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:50:00 UTC", "ConditionTimestampMonotonic": "222213075", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slic 2021-07-11T03:50:13.281 INFO:teuthology.task.ansible.out:e/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "yes", "DelegateControllers": "cpu cpuacct cpuset io blkio memory devices pids bpf-firewall bpf-devices", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8962", "ExecMainStartTimestamp": "Sun 2021-07-11 03:50:00 UTC", "ExecMainStartTimestampMonotonic": "222213467", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[Sun 2021-07-11 03:50:00 UTC] ; stop_time=[n/a] ; pid=8962 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; flags= ; start_time=[Sun 2021-07-11 03:50:00 UTC] ; stop_time=[n/a] ; pid=8962 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressP 2021-07-11T03:50:13.284 INFO:teuthology.task.ansible.out:ackets": "[no data]", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:50:00 UTC", "InactiveExitTimestampMonotonic": "222213584", "InvocationID": "e78fa15175df40fb9c5b39cca3bd9b47", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "8962", "MemoryAccounting": "yes", "MemoryCurrent": "55832576", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMPolicy": "continu 2021-07-11T03:50:13.286 INFO:teuthology.task.ansible.out:e", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice docker.socket", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:50:01 UTC", "StateChangeTimestampMonotonic": "223874803", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "24", "TasksMax": "infinity", "TimeoutAbortUSec": "infinity", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "n 2021-07-11T03:50:13.288 INFO:teuthology.task.ansible.out:otify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "Wants": "network-online.target containerd.service", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:50:13.356 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:50:02 UTC", "ActiveEnterTimestampMonotonic": "236067241", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "docker.socket network-online.target firewalld.service systemd-journald.socket sysinit.target basic.target containerd.service system.slice", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:50:00 UTC", "AssertTimestampMonotonic": "234190319", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:50:00 UTC", "ConditionTimestampMonotonic": "234190318", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slic 2021-07-11T03:50:13.358 INFO:teuthology.task.ansible.out:e/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "yes", "DelegateControllers": "cpu cpuacct cpuset io blkio memory devices pids bpf-firewall bpf-devices", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "9003", "ExecMainStartTimestamp": "Sun 2021-07-11 03:50:00 UTC", "ExecMainStartTimestampMonotonic": "234190712", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[Sun 2021-07-11 03:50:00 UTC] ; stop_time=[n/a] ; pid=9003 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; flags= ; start_time=[Sun 2021-07-11 03:50:00 UTC] ; stop_time=[n/a] ; pid=9003 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressP 2021-07-11T03:50:13.360 INFO:teuthology.task.ansible.out:ackets": "[no data]", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:50:00 UTC", "InactiveExitTimestampMonotonic": "234190853", "InvocationID": "f6e2797c4bb4456599e0d2b230a5e1bd", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127825", "LimitSIGPENDINGSoft": "127825", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "9003", "MemoryAccounting": "yes", "MemoryCurrent": "56684544", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMPolicy": "continu 2021-07-11T03:50:13.362 INFO:teuthology.task.ansible.out:e", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target system.slice docker.socket", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:50:02 UTC", "StateChangeTimestampMonotonic": "236067241", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "22", "TasksMax": "infinity", "TimeoutAbortUSec": "infinity", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "n 2021-07-11T03:50:13.363 INFO:teuthology.task.ansible.out:otify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "Wants": "containerd.service network- 2021-07-11T03:50:13.366 INFO:teuthology.task.ansible.out:online.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:50:14.181 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestamp": "Sun 2021-07-11 03:50:02 UTC", "ActiveEnterTimestampMonotonic": "243999956", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "firewalld.service systemd-journald.socket docker.socket containerd.service system.slice sysinit.target network-online.target basic.target", "AllowIsolate": "no", "AllowedCPUs": "", "AllowedMemoryNodes": "", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Sun 2021-07-11 03:50:00 UTC", "AssertTimestampMonotonic": "241949312", "Before": "multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUAffinityFromNUMA": "no", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend cap_audit_read", "CleanResult": "success", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Sun 2021-07-11 03:50:00 UTC", "ConditionTimestampMonotonic": "241949312", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slic 2021-07-11T03:50:14.183 INFO:teuthology.task.ansible.out:e/docker.service", "ControlPID": "0", "DefaultDependencies": "yes", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "yes", "DelegateControllers": "cpu cpuacct cpuset io blkio memory devices pids bpf-firewall bpf-devices", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "9013", "ExecMainStartTimestamp": "Sun 2021-07-11 03:50:00 UTC", "ExecMainStartTimestampMonotonic": "241949725", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecReloadEx": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; flags= ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[Sun 2021-07-11 03:50:00 UTC] ; stop_time=[n/a] ; pid=9013 ; code=(null) ; status=0/0 }", "ExecStartEx": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; flags= ; start_time=[Sun 2021-07-11 03:50:00 UTC] ; stop_time=[n/a] ; pid=9013 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FinalKillSignal": "9", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOReadBytes": "18446744073709551615", "IOReadOperations": "18446744073709551615", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IOWriteBytes": "18446744073709551615", "IOWriteOperations": "18446744073709551615", "IPAccounting": "no", "IPEgressBytes": "[no data]", "IPEgressPackets": "[no data]", "IPIngressBytes": "[no data]", "IPIngressP 2021-07-11T03:50:14.185 INFO:teuthology.task.ansible.out:ackets": "[no data]", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Sun 2021-07-11 03:50:00 UTC", "InactiveExitTimestampMonotonic": "241949833", "InvocationID": "12bf7c5da5a247539a6a94da299875bf", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127826", "LimitSIGPENDINGSoft": "127826", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "LogsDirectoryMode": "0755", "MainPID": "9013", "MemoryAccounting": "yes", "MemoryCurrent": "55779328", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMPolicy": "continu 2021-07-11T03:50:14.187 INFO:teuthology.task.ansible.out:e", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "Perpetual": "no", "PrivateDevices": "no", "PrivateMounts": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectHostname": "no", "ProtectKernelLogs": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "ReloadResult": "success", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket system.slice sysinit.target", "Restart": "always", "RestartKillSignal": "15", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Sun 2021-07-11 03:50:02 UTC", "StateChangeTimestampMonotonic": "243999956", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "21", "TasksMax": "infinity", "TimeoutAbortUSec": "infinity", "TimeoutCleanUSec": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000 2021-07-11T03:50:14.187 INFO:teuthology.task.ansible.out:", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "n 2021-07-11T03:50:14.189 INFO:teuthology.task.ansible.out:otify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "Wants": "containerd.service network-online.target", "WatchdogSignal": "6", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-07-11T03:50:14.265 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** 2021-07-11T03:50:14.267 INFO:teuthology.task.ansible.out:skipping: no hosts matched PLAY [paddles] ***************************************************************** skipping: no hosts matched 2021-07-11T03:50:14.268 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2021-07-11T03:50:14.269 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-07-11T03:50:14.287 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-07-11T03:50:14.289 INFO:teuthology.task.ansible.out:Sunday 11 July 2021 03:50:14 +0000 (0:00:01.826) 0:03:28.762 *********** 2021-07-11T03:50:15.236 INFO:teuthology.task.ansible.out:ok: [gibba023.front.sepia.ceph.com] 2021-07-11T03:50:15.271 INFO:teuthology.task.ansible.out:ok: [gibba005.front.sepia.ceph.com] 2021-07-11T03:50:16.081 INFO:teuthology.task.ansible.out:ok: [gibba004.front.sepia.ceph.com] 2021-07-11T03:50:16.114 INFO:teuthology.task.ansible.out:ok: [gibba019.front.sepia.ceph.com] 2021-07-11T03:50:16.142 INFO:teuthology.task.ansible.out: TASK [Touch /ceph-qa-ready] **************************************************** Sunday 11 July 2021 03:50:16 +0000 (0:00:01.852) 0:03:30.615 *********** 2021-07-11T03:50:16.317 INFO:teuthology.task.ansible.out:changed: [gibba004.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-07-11T03:50:16.376 INFO:teuthology.task.ansible.out:changed: [gibba005.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-07-11T03:50:16.441 INFO:teuthology.task.ansible.out:changed: [gibba023.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-07-11T03:50:16.464 INFO:teuthology.task.ansible.out:changed: [gibba019.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-07-11T03:50:16.468 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* 2021-07-11T03:50:16.470 INFO:teuthology.task.ansible.out:gibba004.front.sepia.ceph.com : ok=133 changed=41 unreachable=0 failed=0 skipped=186 rescued=0 ignored=0 gibba005.front.sepia.ceph.com : ok=133 changed=41 unreachable=0 failed=0 skipped=186 rescued=0 ignored=0 gibba019.front.sepia.ceph.com : ok=133 changed=41 unreachable=0 failed=0 skipped=186 rescued=0 ignored=0 gibba023.front.sepia.ceph.com : ok=133 changed=41 unreachable=0 failed=0 skipped=186 rescued=0 ignored=0 Sunday 11 July 2021 03:50:16 +0000 (0:00:00.328) 0:03:30.943 *********** =============================================================================== Create all admin users with sudo access. ------------------------------- 25.08s testnode : Upgrade packages -------------------------------------------- 19.74s users : Update authorized_keys using the keys repo --------------------- 19.26s container-host : Install container packages ---------------------------- 12.31s testnode : Ensure packages are not present. ----------------------------- 9.63s testnode : Install packages --------------------------------------------- 8.82s Remove revoked users ---------------------------------------------------- 5.18s testnode : Zap all non-root disks --------------------------------------- 3.37s container-host : Install registries-conf-ctl ---------------------------- 2.79s common : Update apt cache ----------------------------------------------- 2.66s Gathering Facts --------------------------------------------------------- 2.06s Gathering Facts --------------------------------------------------------- 1.85s container-host : Restart docker service --------------------------------- 1.83s users : Clone the keys repo --------------------------------------------- 1.05s users : Update authorized_keys for each user with literal keys ---------- 1.01s testnode : Install apt keys --------------------------------------------- 0.97s Gathering Facts ---------------------------------- 2021-07-11T03:50:16.471 INFO:teuthology.task.ansible.out:----------------------- 0.92s testnode : Update apt cache. -------------------------------------------- 0.89s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 0.73s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 0.73s 2021-07-11T03:50:16.981 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@gibba004.front.sepia.ceph.com'), Remote(name='ubuntu@gibba005.front.sepia.ceph.com'), Remote(name='ubuntu@gibba019.front.sepia.ceph.com'), Remote(name='ubuntu@gibba023.front.sepia.ceph.com')] 2021-07-11T03:50:16.983 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:50:16.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba004.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:50:17.029 DEBUG:teuthology.orchestra.run.gibba004:> true 2021-07-11T03:50:17.120 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:50:17.121 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:50:17.166 DEBUG:teuthology.orchestra.run.gibba005:> true 2021-07-11T03:50:17.249 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:50:17.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:50:17.295 DEBUG:teuthology.orchestra.run.gibba019:> true 2021-07-11T03:50:17.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-07-11T03:50:17.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'gibba023.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-07-11T03:50:17.423 DEBUG:teuthology.orchestra.run.gibba023:> true 2021-07-11T03:50:17.502 INFO:teuthology.run_tasks:Running task clock... 2021-07-11T03:50:17.517 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-07-11T03:50:17.518 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-07-11T03:50:17.520 DEBUG:teuthology.orchestra.run.gibba004:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-07-11T03:50:17.542 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: ntpd 4.2.8p12@1.3728-o (1): Starting 2021-07-11T03:50:17.543 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: Command line: ntpd -gq 2021-07-11T03:50:17.543 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: proto: precision = 0.034 usec (-25) 2021-07-11T03:50:17.543 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-07-11T03:50:17.544 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: restrict ::: KOD does nothing without LIMITED. 2021-07-11T03:50:17.544 INFO:teuthology.orchestra.run.gibba004.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-07-11T03:50:17.544 INFO:teuthology.orchestra.run.gibba004.stderr:restrict ::: KOD does nothing without LIMITED. 2021-07-11T03:50:17.546 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: Listen and drop on 0 v6wildcard [::]:123 2021-07-11T03:50:17.546 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-07-11T03:50:17.546 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: Listen normally on 2 lo 127.0.0.1:123 2021-07-11T03:50:17.547 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: Listen normally on 3 enp1s0f0 172.21.2.104:123 2021-07-11T03:50:17.547 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: Listen normally on 4 lo [::1]:123 2021-07-11T03:50:17.547 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:8bfe%4]:123 2021-07-11T03:50:17.547 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:17 ntpd[10407]: Listening on routing socket on fd #22 for interface updates 2021-07-11T03:50:18.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:18.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:18.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:18.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:18.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:18.546 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:18.546 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:18.546 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:18.546 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:18.547 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:18.547 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:18.548 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:18 ntpd[10407]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:20.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:20.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:20.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:20.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:20.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:20.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:20.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:20.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:20.546 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:20.546 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:20.546 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:20.546 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:20 ntpd[10407]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:21.543 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:21 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:21.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:21 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:21.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:21 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:21.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:21 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:21.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:21 ntpd[10407]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:21.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:21 ntpd[10407]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:22.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:22.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:22.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:22.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:22.546 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:22.546 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:22.547 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:22.547 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:22.547 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:22.548 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:22.548 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:22.549 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:22.549 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:22.549 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:22.549 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:22.550 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:22.550 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:22.550 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:22 ntpd[10407]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:23.543 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:23 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:23.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:23 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:23.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:23 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:23.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:23 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:23.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:23 ntpd[10407]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:23.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:23 ntpd[10407]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:24.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:24.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:24.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:24.544 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:24.545 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:24.546 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:24.547 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:24.547 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:24.547 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:24.548 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:24.549 INFO:teuthology.orchestra.run.gibba004.stdout:11 Jul 03:50:24 ntpd[10407]: ntpd: time slew +0.000719 s 2021-07-11T03:50:24.550 INFO:teuthology.orchestra.run.gibba004.stdout:ntpd: time slew +0.000719s 2021-07-11T03:50:24.553 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-07-11T03:50:24.553 INFO:teuthology.orchestra.run.gibba004.stderr:11 Jul 03:50:24 ntpd[10407]: can't open /var/log/ntpstats/loopstats.20210711: Permission denied 2021-07-11T03:50:24.631 INFO:teuthology.orchestra.run.gibba004.stdout: remote refid st t when poll reach delay offset jitter 2021-07-11T03:50:24.632 INFO:teuthology.orchestra.run.gibba004.stdout:============================================================================== 2021-07-11T03:50:24.632 INFO:teuthology.orchestra.run.gibba004.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:24.632 INFO:teuthology.orchestra.run.gibba004.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:24.633 INFO:teuthology.orchestra.run.gibba004.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:24.633 INFO:teuthology.orchestra.run.gibba004.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:24.634 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-07-11T03:50:24.635 DEBUG:teuthology.orchestra.run.gibba005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-07-11T03:50:24.657 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: ntpd 4.2.8p12@1.3728-o (1): Starting 2021-07-11T03:50:24.657 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: Command line: ntpd -gq 2021-07-11T03:50:24.657 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: proto: precision = 0.034 usec (-25) 2021-07-11T03:50:24.658 INFO:teuthology.orchestra.run.gibba005.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-07-11T03:50:24.659 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-07-11T03:50:24.659 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: restrict ::: KOD does nothing without LIMITED. 2021-07-11T03:50:24.659 INFO:teuthology.orchestra.run.gibba005.stderr:restrict ::: KOD does nothing without LIMITED. 2021-07-11T03:50:24.670 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: Listen and drop on 0 v6wildcard [::]:123 2021-07-11T03:50:24.671 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-07-11T03:50:24.671 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: Listen normally on 2 lo 127.0.0.1:123 2021-07-11T03:50:24.671 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: Listen normally on 3 enp1s0f0 172.21.2.105:123 2021-07-11T03:50:24.671 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: Listen normally on 4 lo [::1]:123 2021-07-11T03:50:24.672 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:8dba%4]:123 2021-07-11T03:50:24.672 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:24 ntpd[10405]: Listening on routing socket on fd #22 for interface updates 2021-07-11T03:50:25.657 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:25 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:25.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:25 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:25.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:25 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:25.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:25 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:25.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:25 ntpd[10405]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:25.660 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:25 ntpd[10405]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:27.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:27 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:27.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:27 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:27.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:27 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:27.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:27 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:27.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:27 ntpd[10405]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:27.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:27 ntpd[10405]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:28.660 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:28.660 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:28.661 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:28.661 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:28.661 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:28.661 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:28.661 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:28.662 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:28.662 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:28.662 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:28.663 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:28.663 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:28.664 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:28.664 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:28.664 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:28.665 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:28.665 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:28.665 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:28 ntpd[10405]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:29.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:29 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:29.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:29 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:29.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:29 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:29.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:29 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:29.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:29 ntpd[10405]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:29.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:29 ntpd[10405]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:30.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:30.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:30.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:30.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:30.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:30.660 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:30.661 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:30.661 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:30.661 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:30.661 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:30.662 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:30.662 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:30.662 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:30.662 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:30.663 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:30.663 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:30.663 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:30.663 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:30 ntpd[10405]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:31.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:31 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:31.658 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:31 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:31.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:31 ntpd[10405]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:31.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:31 ntpd[10405]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:31.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:31 ntpd[10405]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:31.659 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:31 ntpd[10405]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:31.660 INFO:teuthology.orchestra.run.gibba005.stdout:11 Jul 03:50:31 ntpd[10405]: ntpd: time slew +0.004911 s 2021-07-11T03:50:31.660 INFO:teuthology.orchestra.run.gibba005.stdout:ntpd: time slew +0.004911s 2021-07-11T03:50:31.661 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:31 ntpd[10405]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-07-11T03:50:31.661 INFO:teuthology.orchestra.run.gibba005.stderr:11 Jul 03:50:31 ntpd[10405]: can't open /var/log/ntpstats/loopstats.20210711: Permission denied 2021-07-11T03:50:31.700 INFO:teuthology.orchestra.run.gibba005.stdout: remote refid st t when poll reach delay offset jitter 2021-07-11T03:50:31.701 INFO:teuthology.orchestra.run.gibba005.stdout:============================================================================== 2021-07-11T03:50:31.701 INFO:teuthology.orchestra.run.gibba005.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:31.702 INFO:teuthology.orchestra.run.gibba005.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:31.702 INFO:teuthology.orchestra.run.gibba005.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:31.702 INFO:teuthology.orchestra.run.gibba005.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:31.702 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-07-11T03:50:31.703 DEBUG:teuthology.orchestra.run.gibba019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-07-11T03:50:31.726 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: ntpd 4.2.8p12@1.3728-o (1): Starting 2021-07-11T03:50:31.727 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: Command line: ntpd -gq 2021-07-11T03:50:31.727 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: proto: precision = 0.033 usec (-25) 2021-07-11T03:50:31.727 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-07-11T03:50:31.727 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: restrict ::: KOD does nothing without LIMITED. 2021-07-11T03:50:31.728 INFO:teuthology.orchestra.run.gibba019.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-07-11T03:50:31.728 INFO:teuthology.orchestra.run.gibba019.stderr:restrict ::: KOD does nothing without LIMITED. 2021-07-11T03:50:31.729 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: Listen and drop on 0 v6wildcard [::]:123 2021-07-11T03:50:31.730 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-07-11T03:50:31.730 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: Listen normally on 2 lo 127.0.0.1:123 2021-07-11T03:50:31.730 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: Listen normally on 3 enp1s0f0 172.21.2.119:123 2021-07-11T03:50:31.733 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: Listen normally on 4 lo [::1]:123 2021-07-11T03:50:31.733 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:8c52%4]:123 2021-07-11T03:50:31.733 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:31 ntpd[10387]: Listening on routing socket on fd #22 for interface updates 2021-07-11T03:50:32.729 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:32.729 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:32.730 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:32.730 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:32.730 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:32.730 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:32.731 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:32.731 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:32.731 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:32.731 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:32.731 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:32.733 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:32 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:33.727 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:33 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:33.727 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:33 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:33.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:33 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:33.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:33 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:33.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:33 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:33.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:33 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:34.735 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:34.735 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:34.735 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:34.735 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:34.741 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:34.742 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:34.743 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:34.743 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:34.743 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:34.743 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:34.744 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:34.744 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:34.744 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:34.744 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:34.745 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:34.745 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:34.745 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:34.745 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:34 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:35.727 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:35 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:35.727 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:35 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:35.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:35 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:35.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:35 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:35.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:35 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:35.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:35 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:36.734 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:36.734 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:36.735 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:36.735 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:36.736 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:36.736 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:36.736 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:36.737 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:36.737 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:36.738 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:36.738 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:36.739 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:36.743 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:36.743 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:36.744 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:36.744 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:36.745 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:36.745 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:36 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:37.727 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:37 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:37.727 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:37 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:37.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:37 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:37.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:37 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:37.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:37 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:37.728 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:37 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:38.732 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:38.733 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:38.733 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:38.733 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:38.734 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:38.734 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:38.734 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:38.734 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:38.734 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:38.735 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:38.735 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:38.735 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:38.735 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:38.736 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:38.736 INFO:teuthology.orchestra.run.gibba019.stdout:11 Jul 03:50:38 ntpd[10387]: ntpd: time slew -0.002344 s 2021-07-11T03:50:38.736 INFO:teuthology.orchestra.run.gibba019.stdout:ntpd: time slew -0.002344s 2021-07-11T03:50:38.737 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-07-11T03:50:38.737 INFO:teuthology.orchestra.run.gibba019.stderr:11 Jul 03:50:38 ntpd[10387]: can't open /var/log/ntpstats/loopstats.20210711: Permission denied 2021-07-11T03:50:38.763 INFO:teuthology.orchestra.run.gibba019.stdout: remote refid st t when poll reach delay offset jitter 2021-07-11T03:50:38.763 INFO:teuthology.orchestra.run.gibba019.stdout:============================================================================== 2021-07-11T03:50:38.764 INFO:teuthology.orchestra.run.gibba019.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:38.764 INFO:teuthology.orchestra.run.gibba019.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:38.764 INFO:teuthology.orchestra.run.gibba019.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:38.764 INFO:teuthology.orchestra.run.gibba019.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:38.765 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-07-11T03:50:38.765 DEBUG:teuthology.orchestra.run.gibba023:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-07-11T03:50:38.793 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: ntpd 4.2.8p12@1.3728-o (1): Starting 2021-07-11T03:50:38.793 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: Command line: ntpd -gq 2021-07-11T03:50:38.793 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: proto: precision = 0.033 usec (-25) 2021-07-11T03:50:38.793 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-07-11T03:50:38.798 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: restrict ::: KOD does nothing without LIMITED. 2021-07-11T03:50:38.799 INFO:teuthology.orchestra.run.gibba023.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-07-11T03:50:38.799 INFO:teuthology.orchestra.run.gibba023.stderr:restrict ::: KOD does nothing without LIMITED. 2021-07-11T03:50:38.800 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: Listen and drop on 0 v6wildcard [::]:123 2021-07-11T03:50:38.800 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-07-11T03:50:38.800 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: Listen normally on 2 lo 127.0.0.1:123 2021-07-11T03:50:38.801 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: Listen normally on 3 enp1s0f0 172.21.2.123:123 2021-07-11T03:50:38.801 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: Listen normally on 4 lo [::1]:123 2021-07-11T03:50:38.801 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: Listen normally on 5 enp1s0f0 [fe80::3eec:efff:fe3d:8bf6%4]:123 2021-07-11T03:50:38.801 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:38 ntpd[10371]: Listening on routing socket on fd #22 for interface updates 2021-07-11T03:50:39.787 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:39 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:39.787 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:39 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:39.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:39 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:39.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:39 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:39.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:39 ntpd[10371]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:39.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:39 ntpd[10371]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:40.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:40 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:40.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:40 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:40.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:40 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:40.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:40 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:40.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:40 ntpd[10371]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:40.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:40 ntpd[10371]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:41.787 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:41 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:41.787 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:41 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:41.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:41 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:41.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:41 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:41.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:41 ntpd[10371]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:41.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:41 ntpd[10371]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:42.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:42 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:42.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:42 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:42.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:42 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:42.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:42 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:42.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:42 ntpd[10371]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:42.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:42 ntpd[10371]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:43.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:43.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:43.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:43.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:43.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:43.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:43.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:43.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:43.792 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:43.793 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:43.793 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:43.793 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:43.794 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:43.794 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:43.795 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:43.795 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:43.799 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:43.800 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:43 ntpd[10371]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:44.792 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:44 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:44.792 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:44 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:44.792 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:44 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:44.793 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:44 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:44.793 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:44 ntpd[10371]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:44.793 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:44 ntpd[10371]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:45.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:45.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:45.788 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:45.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:45.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:45.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:45.789 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:45.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:45.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:45.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:45.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-07-11T03:50:45.790 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: can't open /var/log/ntpstats/rawstats.20210711: Permission denied 2021-07-11T03:50:45.791 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-07-11T03:50:45.791 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: can't open /var/log/ntpstats/peerstats.20210711: Permission denied 2021-07-11T03:50:45.792 INFO:teuthology.orchestra.run.gibba023.stdout:11 Jul 03:50:45 ntpd[10371]: ntpd: time slew +0.000645 s 2021-07-11T03:50:45.793 INFO:teuthology.orchestra.run.gibba023.stdout:ntpd: time slew +0.000645s 2021-07-11T03:50:45.793 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-07-11T03:50:45.793 INFO:teuthology.orchestra.run.gibba023.stderr:11 Jul 03:50:45 ntpd[10371]: can't open /var/log/ntpstats/loopstats.20210711: Permission denied 2021-07-11T03:50:45.831 INFO:teuthology.orchestra.run.gibba023.stdout: remote refid st t when poll reach delay offset jitter 2021-07-11T03:50:45.831 INFO:teuthology.orchestra.run.gibba023.stdout:============================================================================== 2021-07-11T03:50:45.832 INFO:teuthology.orchestra.run.gibba023.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:45.832 INFO:teuthology.orchestra.run.gibba023.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:45.832 INFO:teuthology.orchestra.run.gibba023.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:45.832 INFO:teuthology.orchestra.run.gibba023.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-07-11T03:50:45.833 INFO:teuthology.run_tasks:Running task install... 2021-07-11T03:50:45.848 DEBUG:teuthology.task.install:project ceph 2021-07-11T03:50:45.849 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '45c6565d67f116c9b3509842b718398a5b38cf73'}} 2021-07-11T03:50:45.849 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '45c6565d67f116c9b3509842b718398a5b38cf73'} 2021-07-11T03:50:45.850 INFO:teuthology.task.install:Using flavor: basic 2021-07-11T03:50:45.863 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-07-11T03:50:45.863 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2021-07-11T03:50:45.864 DEBUG:teuthology.orchestra.run.gibba004:> sudo lsb_release -is 2021-07-11T03:50:45.892 INFO:teuthology.orchestra.run.gibba004.stdout:Ubuntu 2021-07-11T03:50:45.894 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:50:45.895 DEBUG:teuthology.orchestra.run.gibba005:> sudo lsb_release -is 2021-07-11T03:50:45.896 DEBUG:teuthology.orchestra.run.gibba004:> sudo apt-key list | grep Ceph 2021-07-11T03:50:45.922 INFO:teuthology.orchestra.run.gibba005.stdout:Ubuntu 2021-07-11T03:50:45.924 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:50:45.925 DEBUG:teuthology.orchestra.run.gibba019:> sudo lsb_release -is 2021-07-11T03:50:45.925 DEBUG:teuthology.orchestra.run.gibba005:> sudo apt-key list | grep Ceph 2021-07-11T03:50:45.951 INFO:teuthology.orchestra.run.gibba019.stdout:Ubuntu 2021-07-11T03:50:45.954 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:50:45.954 DEBUG:teuthology.orchestra.run.gibba023:> sudo lsb_release -is 2021-07-11T03:50:45.955 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-key list | grep Ceph 2021-07-11T03:50:45.982 INFO:teuthology.orchestra.run.gibba023.stdout:Ubuntu 2021-07-11T03:50:45.985 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:50:45.986 DEBUG:teuthology.orchestra.run.gibba023:> sudo apt-key list | grep Ceph 2021-07-11T03:50:45.988 INFO:teuthology.orchestra.run.gibba019.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-07-11T03:50:45.990 INFO:teuthology.orchestra.run.gibba004.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-07-11T03:50:46.014 INFO:teuthology.orchestra.run.gibba004.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-07-11T03:50:46.014 INFO:teuthology.orchestra.run.gibba004.stdout:uid [ unknown] Ceph.com (release key) 2021-07-11T03:50:46.015 DEBUG:teuthology.orchestra.run.gibba004:> sudo lsb_release -is 2021-07-11T03:50:46.017 INFO:teuthology.orchestra.run.gibba019.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-07-11T03:50:46.017 INFO:teuthology.orchestra.run.gibba019.stdout:uid [ unknown] Ceph.com (release key) 2021-07-11T03:50:46.017 DEBUG:teuthology.orchestra.run.gibba019:> sudo lsb_release -is 2021-07-11T03:50:46.021 INFO:teuthology.orchestra.run.gibba005.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-07-11T03:50:46.045 INFO:teuthology.orchestra.run.gibba005.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-07-11T03:50:46.045 INFO:teuthology.orchestra.run.gibba005.stdout:uid [ unknown] Ceph.com (release key) 2021-07-11T03:50:46.046 DEBUG:teuthology.orchestra.run.gibba005:> sudo lsb_release -is 2021-07-11T03:50:46.063 INFO:teuthology.orchestra.run.gibba004.stdout:Ubuntu 2021-07-11T03:50:46.067 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:50:46.068 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-07-11T03:50:46.068 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-07-11T03:50:46.069 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=45c6565d67f116c9b3509842b718398a5b38cf73 2021-07-11T03:50:46.076 INFO:teuthology.orchestra.run.gibba023.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-07-11T03:50:46.078 INFO:teuthology.orchestra.run.gibba005.stdout:Ubuntu 2021-07-11T03:50:46.087 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:50:46.088 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-07-11T03:50:46.088 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-07-11T03:50:46.088 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=45c6565d67f116c9b3509842b718398a5b38cf73 2021-07-11T03:50:46.096 INFO:teuthology.orchestra.run.gibba019.stdout:Ubuntu 2021-07-11T03:50:46.111 INFO:teuthology.orchestra.run.gibba023.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-07-11T03:50:46.111 INFO:teuthology.orchestra.run.gibba023.stdout:uid [ unknown] Ceph.com (release key) 2021-07-11T03:50:46.112 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:50:46.112 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-07-11T03:50:46.112 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-07-11T03:50:46.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=45c6565d67f116c9b3509842b718398a5b38cf73 2021-07-11T03:50:46.114 DEBUG:teuthology.orchestra.run.gibba023:> sudo lsb_release -is 2021-07-11T03:50:46.202 INFO:teuthology.orchestra.run.gibba023.stdout:Ubuntu 2021-07-11T03:50:46.205 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:50:46.206 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2021-07-11T03:50:46.206 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2021-07-11T03:50:46.206 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=45c6565d67f116c9b3509842b718398a5b38cf73 2021-07-11T03:50:46.271 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default/ 2021-07-11T03:50:46.272 INFO:teuthology.task.install.deb:Package version is 15.2.13-130-g45c6565d-1focal 2021-07-11T03:50:46.285 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default/ 2021-07-11T03:50:46.285 INFO:teuthology.task.install.deb:Package version is 15.2.13-130-g45c6565d-1focal 2021-07-11T03:50:46.297 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default/ 2021-07-11T03:50:46.298 INFO:teuthology.task.install.deb:Package version is 15.2.13-130-g45c6565d-1focal 2021-07-11T03:50:46.394 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default/ 2021-07-11T03:50:46.402 INFO:teuthology.task.install.deb:Package version is 15.2.13-130-g45c6565d-1focal 2021-07-11T03:50:46.412 DEBUG:teuthology.orchestra.run.gibba004:> set -ex 2021-07-11T03:50:46.412 DEBUG:teuthology.orchestra.run.gibba004:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-07-11T03:50:46.430 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-07-11T03:50:46.431 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-07-11T03:50:46.431 DEBUG:teuthology.orchestra.run.gibba005:> set -ex 2021-07-11T03:50:46.432 DEBUG:teuthology.orchestra.run.gibba005:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-07-11T03:50:46.468 DEBUG:teuthology.orchestra.run.gibba005:> sudo apt-get update 2021-07-11T03:50:46.485 DEBUG:teuthology.orchestra.run.gibba004:> sudo apt-get update 2021-07-11T03:50:46.494 DEBUG:teuthology.orchestra.run.gibba019:> sudo apt-get update 2021-07-11T03:50:46.547 DEBUG:teuthology.orchestra.run.gibba023:> set -ex 2021-07-11T03:50:46.548 DEBUG:teuthology.orchestra.run.gibba023:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-07-11T03:50:46.564 DEBUG:teuthology.orchestra.run.gibba023:> sudo apt-get update 2021-07-11T03:50:46.574 INFO:teuthology.orchestra.run.gibba004.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-07-11T03:50:46.588 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-07-11T03:50:46.593 INFO:teuthology.orchestra.run.gibba005.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-07-11T03:50:46.649 INFO:teuthology.orchestra.run.gibba004.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal InRelease 2021-07-11T03:50:46.653 INFO:teuthology.orchestra.run.gibba019.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal InRelease 2021-07-11T03:50:46.654 INFO:teuthology.orchestra.run.gibba005.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal InRelease 2021-07-11T03:50:46.673 INFO:teuthology.orchestra.run.gibba004.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal Release [7,680 B] 2021-07-11T03:50:46.677 INFO:teuthology.orchestra.run.gibba019.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal Release [7,680 B] 2021-07-11T03:50:46.678 INFO:teuthology.orchestra.run.gibba005.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal Release [7,680 B] 2021-07-11T03:50:46.698 INFO:teuthology.orchestra.run.gibba004.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:50:46.700 INFO:teuthology.orchestra.run.gibba004.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal Release.gpg 2021-07-11T03:50:46.701 INFO:teuthology.orchestra.run.gibba023.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2021-07-11T03:50:46.705 INFO:teuthology.orchestra.run.gibba019.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal Release.gpg 2021-07-11T03:50:46.706 INFO:teuthology.orchestra.run.gibba005.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal Release.gpg 2021-07-11T03:50:46.718 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:50:46.719 INFO:teuthology.orchestra.run.gibba005.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:50:46.724 INFO:teuthology.orchestra.run.gibba004.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main i386 Packages [3,053 B] 2021-07-11T03:50:46.725 INFO:teuthology.orchestra.run.gibba005.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main i386 Packages [3,053 B] 2021-07-11T03:50:46.727 INFO:teuthology.orchestra.run.gibba019.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main i386 Packages [3,053 B] 2021-07-11T03:50:46.752 INFO:teuthology.orchestra.run.gibba005.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2021-07-11T03:50:46.754 INFO:teuthology.orchestra.run.gibba019.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2021-07-11T03:50:46.755 INFO:teuthology.orchestra.run.gibba004.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2021-07-11T03:50:46.772 INFO:teuthology.orchestra.run.gibba023.stdout:Ign:2 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal InRelease 2021-07-11T03:50:46.779 INFO:teuthology.orchestra.run.gibba004.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-07-11T03:50:46.796 INFO:teuthology.orchestra.run.gibba023.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal Release [7,680 B] 2021-07-11T03:50:46.798 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-07-11T03:50:46.801 INFO:teuthology.orchestra.run.gibba005.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-07-11T03:50:46.823 INFO:teuthology.orchestra.run.gibba023.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal Release.gpg 2021-07-11T03:50:46.827 INFO:teuthology.orchestra.run.gibba023.stdout:Hit:5 http://archive.ubuntu.com/ubuntu focal InRelease 2021-07-11T03:50:46.847 INFO:teuthology.orchestra.run.gibba023.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 Packages [15.8 kB] 2021-07-11T03:50:46.861 INFO:teuthology.orchestra.run.gibba004.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-07-11T03:50:46.877 INFO:teuthology.orchestra.run.gibba023.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main i386 Packages [3,053 B] 2021-07-11T03:50:46.881 INFO:teuthology.orchestra.run.gibba019.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-07-11T03:50:46.883 INFO:teuthology.orchestra.run.gibba005.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-07-11T03:50:46.910 INFO:teuthology.orchestra.run.gibba023.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2021-07-11T03:50:46.992 INFO:teuthology.orchestra.run.gibba023.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2021-07-11T03:50:47.199 INFO:teuthology.orchestra.run.gibba019.stdout:Fetched 26.5 kB in 1s (48.0 kB/s) 2021-07-11T03:50:47.216 INFO:teuthology.orchestra.run.gibba005.stdout:Fetched 26.5 kB in 1s (42.6 kB/s) 2021-07-11T03:50:47.218 INFO:teuthology.orchestra.run.gibba023.stdout:Fetched 26.5 kB in 0s (55.8 kB/s) 2021-07-11T03:50:47.288 INFO:teuthology.orchestra.run.gibba004.stdout:Fetched 26.5 kB in 1s (40.3 kB/s) 2021-07-11T03:50:47.693 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-07-11T03:50:47.704 DEBUG:teuthology.orchestra.run.gibba019:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.13-130-g45c6565d-1focal cephadm=15.2.13-130-g45c6565d-1focal ceph-mds=15.2.13-130-g45c6565d-1focal ceph-mgr=15.2.13-130-g45c6565d-1focal ceph-common=15.2.13-130-g45c6565d-1focal ceph-fuse=15.2.13-130-g45c6565d-1focal ceph-test=15.2.13-130-g45c6565d-1focal radosgw=15.2.13-130-g45c6565d-1focal python3-rados=15.2.13-130-g45c6565d-1focal python3-rgw=15.2.13-130-g45c6565d-1focal python3-cephfs=15.2.13-130-g45c6565d-1focal python3-rbd=15.2.13-130-g45c6565d-1focal libcephfs2=15.2.13-130-g45c6565d-1focal libcephfs-dev=15.2.13-130-g45c6565d-1focal librados2=15.2.13-130-g45c6565d-1focal librbd1=15.2.13-130-g45c6565d-1focal rbd-fuse=15.2.13-130-g45c6565d-1focal python3-cephfs=15.2.13-130-g45c6565d-1focal cephfs-shell=15.2.13-130-g45c6565d-1focal 2021-07-11T03:50:47.709 INFO:teuthology.orchestra.run.gibba023.stdout:Reading package lists... 2021-07-11T03:50:47.712 INFO:teuthology.orchestra.run.gibba005.stdout:Reading package lists... 2021-07-11T03:50:47.720 DEBUG:teuthology.orchestra.run.gibba023:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.13-130-g45c6565d-1focal cephadm=15.2.13-130-g45c6565d-1focal ceph-mds=15.2.13-130-g45c6565d-1focal ceph-mgr=15.2.13-130-g45c6565d-1focal ceph-common=15.2.13-130-g45c6565d-1focal ceph-fuse=15.2.13-130-g45c6565d-1focal ceph-test=15.2.13-130-g45c6565d-1focal radosgw=15.2.13-130-g45c6565d-1focal python3-rados=15.2.13-130-g45c6565d-1focal python3-rgw=15.2.13-130-g45c6565d-1focal python3-cephfs=15.2.13-130-g45c6565d-1focal python3-rbd=15.2.13-130-g45c6565d-1focal libcephfs2=15.2.13-130-g45c6565d-1focal libcephfs-dev=15.2.13-130-g45c6565d-1focal librados2=15.2.13-130-g45c6565d-1focal librbd1=15.2.13-130-g45c6565d-1focal rbd-fuse=15.2.13-130-g45c6565d-1focal python3-cephfs=15.2.13-130-g45c6565d-1focal cephfs-shell=15.2.13-130-g45c6565d-1focal 2021-07-11T03:50:47.726 DEBUG:teuthology.orchestra.run.gibba005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.13-130-g45c6565d-1focal cephadm=15.2.13-130-g45c6565d-1focal ceph-mds=15.2.13-130-g45c6565d-1focal ceph-mgr=15.2.13-130-g45c6565d-1focal ceph-common=15.2.13-130-g45c6565d-1focal ceph-fuse=15.2.13-130-g45c6565d-1focal ceph-test=15.2.13-130-g45c6565d-1focal radosgw=15.2.13-130-g45c6565d-1focal python3-rados=15.2.13-130-g45c6565d-1focal python3-rgw=15.2.13-130-g45c6565d-1focal python3-cephfs=15.2.13-130-g45c6565d-1focal python3-rbd=15.2.13-130-g45c6565d-1focal libcephfs2=15.2.13-130-g45c6565d-1focal libcephfs-dev=15.2.13-130-g45c6565d-1focal librados2=15.2.13-130-g45c6565d-1focal librbd1=15.2.13-130-g45c6565d-1focal rbd-fuse=15.2.13-130-g45c6565d-1focal python3-cephfs=15.2.13-130-g45c6565d-1focal cephfs-shell=15.2.13-130-g45c6565d-1focal 2021-07-11T03:50:47.740 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-07-11T03:50:47.757 INFO:teuthology.orchestra.run.gibba023.stdout:Reading package lists... 2021-07-11T03:50:47.764 INFO:teuthology.orchestra.run.gibba005.stdout:Reading package lists... 2021-07-11T03:50:47.791 INFO:teuthology.orchestra.run.gibba004.stdout:Reading package lists... 2021-07-11T03:50:47.796 DEBUG:teuthology.orchestra.run.gibba004:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=15.2.13-130-g45c6565d-1focal cephadm=15.2.13-130-g45c6565d-1focal ceph-mds=15.2.13-130-g45c6565d-1focal ceph-mgr=15.2.13-130-g45c6565d-1focal ceph-common=15.2.13-130-g45c6565d-1focal ceph-fuse=15.2.13-130-g45c6565d-1focal ceph-test=15.2.13-130-g45c6565d-1focal radosgw=15.2.13-130-g45c6565d-1focal python3-rados=15.2.13-130-g45c6565d-1focal python3-rgw=15.2.13-130-g45c6565d-1focal python3-cephfs=15.2.13-130-g45c6565d-1focal python3-rbd=15.2.13-130-g45c6565d-1focal libcephfs2=15.2.13-130-g45c6565d-1focal libcephfs-dev=15.2.13-130-g45c6565d-1focal librados2=15.2.13-130-g45c6565d-1focal librbd1=15.2.13-130-g45c6565d-1focal rbd-fuse=15.2.13-130-g45c6565d-1focal python3-cephfs=15.2.13-130-g45c6565d-1focal cephfs-shell=15.2.13-130-g45c6565d-1focal 2021-07-11T03:50:47.834 INFO:teuthology.orchestra.run.gibba004.stdout:Reading package lists... 2021-07-11T03:50:47.838 INFO:teuthology.orchestra.run.gibba019.stdout:Building dependency tree... 2021-07-11T03:50:47.839 INFO:teuthology.orchestra.run.gibba019.stdout:Reading state information... 2021-07-11T03:50:47.855 INFO:teuthology.orchestra.run.gibba023.stdout:Building dependency tree... 2021-07-11T03:50:47.856 INFO:teuthology.orchestra.run.gibba023.stdout:Reading state information... 2021-07-11T03:50:47.860 INFO:teuthology.orchestra.run.gibba005.stdout:Building dependency tree... 2021-07-11T03:50:47.860 INFO:teuthology.orchestra.run.gibba005.stdout:Reading state information... 2021-07-11T03:50:47.920 INFO:teuthology.orchestra.run.gibba019.stdout:The following packages were automatically installed and are no longer required: 2021-07-11T03:50:47.920 INFO:teuthology.orchestra.run.gibba019.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2021-07-11T03:50:47.920 INFO:teuthology.orchestra.run.gibba019.stdout:Use 'sudo apt autoremove' to remove them. 2021-07-11T03:50:47.920 INFO:teuthology.orchestra.run.gibba019.stdout:The following additional packages will be installed: 2021-07-11T03:50:47.921 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-07-11T03:50:47.921 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-07-11T03:50:47.922 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2021-07-11T03:50:47.922 INFO:teuthology.orchestra.run.gibba019.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2021-07-11T03:50:47.923 INFO:teuthology.orchestra.run.gibba019.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2021-07-11T03:50:47.923 INFO:teuthology.orchestra.run.gibba019.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2021-07-11T03:50:47.923 INFO:teuthology.orchestra.run.gibba019.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2021-07-11T03:50:47.923 INFO:teuthology.orchestra.run.gibba019.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-07-11T03:50:47.924 INFO:teuthology.orchestra.run.gibba019.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2021-07-11T03:50:47.924 INFO:teuthology.orchestra.run.gibba019.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2021-07-11T03:50:47.924 INFO:teuthology.orchestra.run.gibba019.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2021-07-11T03:50:47.924 INFO:teuthology.orchestra.run.gibba019.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2021-07-11T03:50:47.925 INFO:teuthology.orchestra.run.gibba019.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2021-07-11T03:50:47.925 INFO:teuthology.orchestra.run.gibba019.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2021-07-11T03:50:47.925 INFO:teuthology.orchestra.run.gibba019.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2021-07-11T03:50:47.925 INFO:teuthology.orchestra.run.gibba019.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2021-07-11T03:50:47.926 INFO:teuthology.orchestra.run.gibba019.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2021-07-11T03:50:47.926 INFO:teuthology.orchestra.run.gibba019.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2021-07-11T03:50:47.926 INFO:teuthology.orchestra.run.gibba019.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2021-07-11T03:50:47.926 INFO:teuthology.orchestra.run.gibba019.stdout: xmlstarlet 2021-07-11T03:50:47.927 INFO:teuthology.orchestra.run.gibba019.stdout:Suggested packages: 2021-07-11T03:50:47.927 INFO:teuthology.orchestra.run.gibba019.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2021-07-11T03:50:47.927 INFO:teuthology.orchestra.run.gibba019.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2021-07-11T03:50:47.927 INFO:teuthology.orchestra.run.gibba019.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2021-07-11T03:50:47.928 INFO:teuthology.orchestra.run.gibba019.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2021-07-11T03:50:47.928 INFO:teuthology.orchestra.run.gibba019.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2021-07-11T03:50:47.928 INFO:teuthology.orchestra.run.gibba019.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2021-07-11T03:50:47.928 INFO:teuthology.orchestra.run.gibba019.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2021-07-11T03:50:47.929 INFO:teuthology.orchestra.run.gibba019.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2021-07-11T03:50:47.935 INFO:teuthology.orchestra.run.gibba019.stdout:Recommended packages: 2021-07-11T03:50:47.935 INFO:teuthology.orchestra.run.gibba019.stdout: btrfs-tools 2021-07-11T03:50:47.936 INFO:teuthology.orchestra.run.gibba004.stdout:Building dependency tree... 2021-07-11T03:50:47.937 INFO:teuthology.orchestra.run.gibba004.stdout:Reading state information... 2021-07-11T03:50:47.939 INFO:teuthology.orchestra.run.gibba005.stdout:The following packages were automatically installed and are no longer required: 2021-07-11T03:50:47.939 INFO:teuthology.orchestra.run.gibba005.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2021-07-11T03:50:47.939 INFO:teuthology.orchestra.run.gibba005.stdout:Use 'sudo apt autoremove' to remove them. 2021-07-11T03:50:47.940 INFO:teuthology.orchestra.run.gibba005.stdout:The following additional packages will be installed: 2021-07-11T03:50:47.940 INFO:teuthology.orchestra.run.gibba005.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-07-11T03:50:47.940 INFO:teuthology.orchestra.run.gibba005.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-07-11T03:50:47.940 INFO:teuthology.orchestra.run.gibba005.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2021-07-11T03:50:47.940 INFO:teuthology.orchestra.run.gibba005.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2021-07-11T03:50:47.941 INFO:teuthology.orchestra.run.gibba005.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2021-07-11T03:50:47.941 INFO:teuthology.orchestra.run.gibba005.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2021-07-11T03:50:47.941 INFO:teuthology.orchestra.run.gibba005.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2021-07-11T03:50:47.941 INFO:teuthology.orchestra.run.gibba005.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-07-11T03:50:47.941 INFO:teuthology.orchestra.run.gibba005.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2021-07-11T03:50:47.942 INFO:teuthology.orchestra.run.gibba005.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2021-07-11T03:50:47.942 INFO:teuthology.orchestra.run.gibba005.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2021-07-11T03:50:47.942 INFO:teuthology.orchestra.run.gibba005.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2021-07-11T03:50:47.953 INFO:teuthology.orchestra.run.gibba005.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2021-07-11T03:50:47.954 INFO:teuthology.orchestra.run.gibba005.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2021-07-11T03:50:47.954 INFO:teuthology.orchestra.run.gibba005.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2021-07-11T03:50:47.954 INFO:teuthology.orchestra.run.gibba005.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2021-07-11T03:50:47.954 INFO:teuthology.orchestra.run.gibba005.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2021-07-11T03:50:47.955 INFO:teuthology.orchestra.run.gibba005.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2021-07-11T03:50:47.955 INFO:teuthology.orchestra.run.gibba005.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2021-07-11T03:50:47.955 INFO:teuthology.orchestra.run.gibba005.stdout: xmlstarlet 2021-07-11T03:50:47.955 INFO:teuthology.orchestra.run.gibba005.stdout:Suggested packages: 2021-07-11T03:50:47.956 INFO:teuthology.orchestra.run.gibba005.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2021-07-11T03:50:47.956 INFO:teuthology.orchestra.run.gibba005.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2021-07-11T03:50:47.957 INFO:teuthology.orchestra.run.gibba005.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2021-07-11T03:50:47.957 INFO:teuthology.orchestra.run.gibba005.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2021-07-11T03:50:47.957 INFO:teuthology.orchestra.run.gibba005.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2021-07-11T03:50:47.958 INFO:teuthology.orchestra.run.gibba005.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2021-07-11T03:50:47.958 INFO:teuthology.orchestra.run.gibba005.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2021-07-11T03:50:47.958 INFO:teuthology.orchestra.run.gibba005.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2021-07-11T03:50:47.958 INFO:teuthology.orchestra.run.gibba005.stdout:Recommended packages: 2021-07-11T03:50:47.959 INFO:teuthology.orchestra.run.gibba005.stdout: btrfs-tools 2021-07-11T03:50:47.959 INFO:teuthology.orchestra.run.gibba023.stdout:The following packages were automatically installed and are no longer required: 2021-07-11T03:50:47.959 INFO:teuthology.orchestra.run.gibba023.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2021-07-11T03:50:47.959 INFO:teuthology.orchestra.run.gibba023.stdout:Use 'sudo apt autoremove' to remove them. 2021-07-11T03:50:47.960 INFO:teuthology.orchestra.run.gibba023.stdout:The following additional packages will be installed: 2021-07-11T03:50:47.960 INFO:teuthology.orchestra.run.gibba023.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-07-11T03:50:47.960 INFO:teuthology.orchestra.run.gibba023.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-07-11T03:50:47.960 INFO:teuthology.orchestra.run.gibba023.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2021-07-11T03:50:47.960 INFO:teuthology.orchestra.run.gibba023.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2021-07-11T03:50:47.961 INFO:teuthology.orchestra.run.gibba023.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2021-07-11T03:50:47.961 INFO:teuthology.orchestra.run.gibba023.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2021-07-11T03:50:47.961 INFO:teuthology.orchestra.run.gibba023.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2021-07-11T03:50:47.961 INFO:teuthology.orchestra.run.gibba023.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-07-11T03:50:47.961 INFO:teuthology.orchestra.run.gibba023.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2021-07-11T03:50:47.962 INFO:teuthology.orchestra.run.gibba023.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2021-07-11T03:50:47.962 INFO:teuthology.orchestra.run.gibba023.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2021-07-11T03:50:47.962 INFO:teuthology.orchestra.run.gibba023.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2021-07-11T03:50:47.962 INFO:teuthology.orchestra.run.gibba023.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2021-07-11T03:50:47.963 INFO:teuthology.orchestra.run.gibba023.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2021-07-11T03:50:47.963 INFO:teuthology.orchestra.run.gibba023.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2021-07-11T03:50:47.963 INFO:teuthology.orchestra.run.gibba023.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2021-07-11T03:50:47.963 INFO:teuthology.orchestra.run.gibba023.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2021-07-11T03:50:47.964 INFO:teuthology.orchestra.run.gibba023.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2021-07-11T03:50:47.965 INFO:teuthology.orchestra.run.gibba023.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2021-07-11T03:50:47.965 INFO:teuthology.orchestra.run.gibba023.stdout: xmlstarlet 2021-07-11T03:50:47.965 INFO:teuthology.orchestra.run.gibba023.stdout:Suggested packages: 2021-07-11T03:50:47.965 INFO:teuthology.orchestra.run.gibba023.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2021-07-11T03:50:47.966 INFO:teuthology.orchestra.run.gibba023.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2021-07-11T03:50:47.966 INFO:teuthology.orchestra.run.gibba023.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2021-07-11T03:50:47.966 INFO:teuthology.orchestra.run.gibba023.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2021-07-11T03:50:47.966 INFO:teuthology.orchestra.run.gibba023.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2021-07-11T03:50:47.966 INFO:teuthology.orchestra.run.gibba023.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2021-07-11T03:50:47.967 INFO:teuthology.orchestra.run.gibba023.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2021-07-11T03:50:47.967 INFO:teuthology.orchestra.run.gibba023.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2021-07-11T03:50:47.967 INFO:teuthology.orchestra.run.gibba023.stdout:Recommended packages: 2021-07-11T03:50:47.967 INFO:teuthology.orchestra.run.gibba023.stdout: btrfs-tools 2021-07-11T03:50:47.969 INFO:teuthology.orchestra.run.gibba019.stdout:The following NEW packages will be installed: 2021-07-11T03:50:47.969 INFO:teuthology.orchestra.run.gibba019.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-07-11T03:50:47.970 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-07-11T03:50:47.970 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-07-11T03:50:47.970 INFO:teuthology.orchestra.run.gibba019.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2021-07-11T03:50:47.970 INFO:teuthology.orchestra.run.gibba019.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2021-07-11T03:50:47.971 INFO:teuthology.orchestra.run.gibba019.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2021-07-11T03:50:47.971 INFO:teuthology.orchestra.run.gibba019.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2021-07-11T03:50:47.971 INFO:teuthology.orchestra.run.gibba019.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2021-07-11T03:50:47.971 INFO:teuthology.orchestra.run.gibba019.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2021-07-11T03:50:47.972 INFO:teuthology.orchestra.run.gibba019.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2021-07-11T03:50:47.972 INFO:teuthology.orchestra.run.gibba019.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2021-07-11T03:50:47.972 INFO:teuthology.orchestra.run.gibba019.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-07-11T03:50:47.972 INFO:teuthology.orchestra.run.gibba019.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2021-07-11T03:50:47.973 INFO:teuthology.orchestra.run.gibba019.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2021-07-11T03:50:47.973 INFO:teuthology.orchestra.run.gibba019.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2021-07-11T03:50:47.973 INFO:teuthology.orchestra.run.gibba019.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2021-07-11T03:50:47.973 INFO:teuthology.orchestra.run.gibba019.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2021-07-11T03:50:47.973 INFO:teuthology.orchestra.run.gibba019.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2021-07-11T03:50:47.974 INFO:teuthology.orchestra.run.gibba019.stdout: python3-simplejson python3-singledispatch python3-sklearn 2021-07-11T03:50:47.974 INFO:teuthology.orchestra.run.gibba019.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2021-07-11T03:50:47.974 INFO:teuthology.orchestra.run.gibba019.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2021-07-11T03:50:47.974 INFO:teuthology.orchestra.run.gibba019.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-07-11T03:50:47.975 INFO:teuthology.orchestra.run.gibba019.stdout:The following packages will be upgraded: 2021-07-11T03:50:47.975 INFO:teuthology.orchestra.run.gibba019.stdout: librados2 librbd1 2021-07-11T03:50:47.977 INFO:teuthology.orchestra.run.gibba023.stdout:The following NEW packages will be installed: 2021-07-11T03:50:47.977 INFO:teuthology.orchestra.run.gibba023.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-07-11T03:50:47.978 INFO:teuthology.orchestra.run.gibba023.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-07-11T03:50:47.978 INFO:teuthology.orchestra.run.gibba023.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-07-11T03:50:47.978 INFO:teuthology.orchestra.run.gibba023.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2021-07-11T03:50:47.978 INFO:teuthology.orchestra.run.gibba023.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2021-07-11T03:50:47.979 INFO:teuthology.orchestra.run.gibba023.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2021-07-11T03:50:47.979 INFO:teuthology.orchestra.run.gibba023.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2021-07-11T03:50:47.979 INFO:teuthology.orchestra.run.gibba023.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2021-07-11T03:50:47.979 INFO:teuthology.orchestra.run.gibba023.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2021-07-11T03:50:47.979 INFO:teuthology.orchestra.run.gibba023.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2021-07-11T03:50:47.980 INFO:teuthology.orchestra.run.gibba023.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2021-07-11T03:50:47.980 INFO:teuthology.orchestra.run.gibba023.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-07-11T03:50:47.980 INFO:teuthology.orchestra.run.gibba023.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2021-07-11T03:50:47.980 INFO:teuthology.orchestra.run.gibba023.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2021-07-11T03:50:47.981 INFO:teuthology.orchestra.run.gibba023.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2021-07-11T03:50:47.981 INFO:teuthology.orchestra.run.gibba023.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2021-07-11T03:50:47.981 INFO:teuthology.orchestra.run.gibba023.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2021-07-11T03:50:47.981 INFO:teuthology.orchestra.run.gibba023.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2021-07-11T03:50:47.981 INFO:teuthology.orchestra.run.gibba023.stdout: python3-simplejson python3-singledispatch python3-sklearn 2021-07-11T03:50:47.982 INFO:teuthology.orchestra.run.gibba023.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2021-07-11T03:50:47.982 INFO:teuthology.orchestra.run.gibba023.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2021-07-11T03:50:47.982 INFO:teuthology.orchestra.run.gibba023.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-07-11T03:50:47.983 INFO:teuthology.orchestra.run.gibba023.stdout:The following packages will be upgraded: 2021-07-11T03:50:47.983 INFO:teuthology.orchestra.run.gibba023.stdout: librados2 librbd1 2021-07-11T03:50:47.985 INFO:teuthology.orchestra.run.gibba005.stdout:The following NEW packages will be installed: 2021-07-11T03:50:47.985 INFO:teuthology.orchestra.run.gibba005.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-07-11T03:50:47.985 INFO:teuthology.orchestra.run.gibba005.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-07-11T03:50:47.985 INFO:teuthology.orchestra.run.gibba005.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-07-11T03:50:47.986 INFO:teuthology.orchestra.run.gibba005.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2021-07-11T03:50:47.986 INFO:teuthology.orchestra.run.gibba005.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2021-07-11T03:50:47.986 INFO:teuthology.orchestra.run.gibba005.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2021-07-11T03:50:47.987 INFO:teuthology.orchestra.run.gibba005.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2021-07-11T03:50:47.987 INFO:teuthology.orchestra.run.gibba005.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2021-07-11T03:50:47.987 INFO:teuthology.orchestra.run.gibba005.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2021-07-11T03:50:47.988 INFO:teuthology.orchestra.run.gibba005.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2021-07-11T03:50:47.988 INFO:teuthology.orchestra.run.gibba005.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2021-07-11T03:50:47.988 INFO:teuthology.orchestra.run.gibba005.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-07-11T03:50:47.988 INFO:teuthology.orchestra.run.gibba005.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2021-07-11T03:50:47.989 INFO:teuthology.orchestra.run.gibba005.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2021-07-11T03:50:47.989 INFO:teuthology.orchestra.run.gibba005.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2021-07-11T03:50:47.989 INFO:teuthology.orchestra.run.gibba005.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2021-07-11T03:50:47.989 INFO:teuthology.orchestra.run.gibba005.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2021-07-11T03:50:47.990 INFO:teuthology.orchestra.run.gibba005.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2021-07-11T03:50:47.990 INFO:teuthology.orchestra.run.gibba005.stdout: python3-simplejson python3-singledispatch python3-sklearn 2021-07-11T03:50:47.990 INFO:teuthology.orchestra.run.gibba005.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2021-07-11T03:50:47.990 INFO:teuthology.orchestra.run.gibba005.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2021-07-11T03:50:47.990 INFO:teuthology.orchestra.run.gibba005.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-07-11T03:50:47.991 INFO:teuthology.orchestra.run.gibba005.stdout:The following packages will be upgraded: 2021-07-11T03:50:47.991 INFO:teuthology.orchestra.run.gibba005.stdout: librados2 librbd1 2021-07-11T03:50:48.016 INFO:teuthology.orchestra.run.gibba004.stdout:The following packages were automatically installed and are no longer required: 2021-07-11T03:50:48.016 INFO:teuthology.orchestra.run.gibba004.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2021-07-11T03:50:48.016 INFO:teuthology.orchestra.run.gibba004.stdout:Use 'sudo apt autoremove' to remove them. 2021-07-11T03:50:48.017 INFO:teuthology.orchestra.run.gibba004.stdout:The following additional packages will be installed: 2021-07-11T03:50:48.017 INFO:teuthology.orchestra.run.gibba004.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-07-11T03:50:48.017 INFO:teuthology.orchestra.run.gibba004.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-07-11T03:50:48.017 INFO:teuthology.orchestra.run.gibba004.stdout: ceph-mon ceph-osd jq libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 2021-07-11T03:50:48.017 INFO:teuthology.orchestra.run.gibba004.stdout: libleveldb1d liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 2021-07-11T03:50:48.018 INFO:teuthology.orchestra.run.gibba004.stdout: librdkafka1 librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl 2021-07-11T03:50:48.018 INFO:teuthology.orchestra.run.gibba004.stdout: python3-atomicwrites python3-attr python3-bcrypt python3-blinker python3-bs4 2021-07-11T03:50:48.018 INFO:teuthology.orchestra.run.gibba004.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common 2021-07-11T03:50:48.018 INFO:teuthology.orchestra.run.gibba004.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2021-07-11T03:50:48.019 INFO:teuthology.orchestra.run.gibba004.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2021-07-11T03:50:48.019 INFO:teuthology.orchestra.run.gibba004.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2021-07-11T03:50:48.019 INFO:teuthology.orchestra.run.gibba004.stdout: python3-mako python3-markupsafe python3-oauthlib python3-openssl 2021-07-11T03:50:48.019 INFO:teuthology.orchestra.run.gibba004.stdout: python3-packaging python3-paste python3-pastedeploy python3-pastescript 2021-07-11T03:50:48.019 INFO:teuthology.orchestra.run.gibba004.stdout: python3-pecan python3-pluggy python3-prettytable python3-psutil python3-py 2021-07-11T03:50:48.020 INFO:teuthology.orchestra.run.gibba004.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyperclip 2021-07-11T03:50:48.020 INFO:teuthology.orchestra.run.gibba004.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2021-07-11T03:50:48.020 INFO:teuthology.orchestra.run.gibba004.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2021-07-11T03:50:48.020 INFO:teuthology.orchestra.run.gibba004.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2021-07-11T03:50:48.021 INFO:teuthology.orchestra.run.gibba004.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2021-07-11T03:50:48.021 INFO:teuthology.orchestra.run.gibba004.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug socat 2021-07-11T03:50:48.021 INFO:teuthology.orchestra.run.gibba004.stdout: xmlstarlet 2021-07-11T03:50:48.021 INFO:teuthology.orchestra.run.gibba004.stdout:Suggested packages: 2021-07-11T03:50:48.021 INFO:teuthology.orchestra.run.gibba004.stdout: python3-influxdb python-attr-doc python-blinker-doc python-cryptography-doc 2021-07-11T03:50:48.022 INFO:teuthology.orchestra.run.gibba004.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2021-07-11T03:50:48.022 INFO:teuthology.orchestra.run.gibba004.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2021-07-11T03:50:48.022 INFO:teuthology.orchestra.run.gibba004.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2021-07-11T03:50:48.023 INFO:teuthology.orchestra.run.gibba004.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2021-07-11T03:50:48.023 INFO:teuthology.orchestra.run.gibba004.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2021-07-11T03:50:48.023 INFO:teuthology.orchestra.run.gibba004.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2021-07-11T03:50:48.024 INFO:teuthology.orchestra.run.gibba004.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2021-07-11T03:50:48.024 INFO:teuthology.orchestra.run.gibba004.stdout:Recommended packages: 2021-07-11T03:50:48.024 INFO:teuthology.orchestra.run.gibba004.stdout: btrfs-tools 2021-07-11T03:50:48.061 INFO:teuthology.orchestra.run.gibba004.stdout:The following NEW packages will be installed: 2021-07-11T03:50:48.062 INFO:teuthology.orchestra.run.gibba004.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-07-11T03:50:48.062 INFO:teuthology.orchestra.run.gibba004.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-cloud 2021-07-11T03:50:48.062 INFO:teuthology.orchestra.run.gibba004.stdout: ceph-mgr-diskprediction-local ceph-mgr-k8sevents ceph-mgr-modules-core 2021-07-11T03:50:48.062 INFO:teuthology.orchestra.run.gibba004.stdout: ceph-mon ceph-osd ceph-test cephadm cephfs-shell jq libcephfs-dev libcephfs2 2021-07-11T03:50:48.063 INFO:teuthology.orchestra.run.gibba004.stdout: libjq1 libjs-sphinxdoc libjs-underscore liblbfgsb0 libleveldb1d 2021-07-11T03:50:48.063 INFO:teuthology.orchestra.run.gibba004.stdout: liblttng-ust0 liboath0 libonig5 librabbitmq4 libradosstriper1 librdkafka1 2021-07-11T03:50:48.063 INFO:teuthology.orchestra.run.gibba004.stdout: librgw2 libxslt1.1 nvme-cli python-pastedeploy-tpl python3-atomicwrites 2021-07-11T03:50:48.063 INFO:teuthology.orchestra.run.gibba004.stdout: python3-attr python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2021-07-11T03:50:48.063 INFO:teuthology.orchestra.run.gibba004.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cherrypy3 2021-07-11T03:50:48.064 INFO:teuthology.orchestra.run.gibba004.stdout: python3-cmd2 python3-colorama python3-cryptography python3-decorator 2021-07-11T03:50:48.064 INFO:teuthology.orchestra.run.gibba004.stdout: python3-google-auth python3-html5lib python3-jinja2 python3-joblib 2021-07-11T03:50:48.064 INFO:teuthology.orchestra.run.gibba004.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-07-11T03:50:48.064 INFO:teuthology.orchestra.run.gibba004.stdout: python3-markupsafe python3-oauthlib python3-openssl python3-packaging 2021-07-11T03:50:48.065 INFO:teuthology.orchestra.run.gibba004.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2021-07-11T03:50:48.065 INFO:teuthology.orchestra.run.gibba004.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2021-07-11T03:50:48.065 INFO:teuthology.orchestra.run.gibba004.stdout: python3-pyasn1-modules python3-pyinotify python3-pyperclip python3-pytest 2021-07-11T03:50:48.065 INFO:teuthology.orchestra.run.gibba004.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests-oauthlib 2021-07-11T03:50:48.065 INFO:teuthology.orchestra.run.gibba004.stdout: python3-rgw python3-routes python3-rsa python3-scipy python3-simplegeneric 2021-07-11T03:50:48.066 INFO:teuthology.orchestra.run.gibba004.stdout: python3-simplejson python3-singledispatch python3-sklearn 2021-07-11T03:50:48.066 INFO:teuthology.orchestra.run.gibba004.stdout: python3-sklearn-lib python3-soupsieve python3-tempita python3-waitress 2021-07-11T03:50:48.066 INFO:teuthology.orchestra.run.gibba004.stdout: python3-wcwidth python3-webencodings python3-webob python3-websocket 2021-07-11T03:50:48.067 INFO:teuthology.orchestra.run.gibba004.stdout: python3-webtest python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2021-07-11T03:50:48.067 INFO:teuthology.orchestra.run.gibba004.stdout:The following packages will be upgraded: 2021-07-11T03:50:48.067 INFO:teuthology.orchestra.run.gibba004.stdout: librados2 librbd1 2021-07-11T03:50:48.109 INFO:teuthology.orchestra.run.gibba019.stdout:2 upgraded, 101 newly installed, 0 to remove and 141 not upgraded. 2021-07-11T03:50:48.110 INFO:teuthology.orchestra.run.gibba019.stdout:Need to get 153 MB of archives. 2021-07-11T03:50:48.110 INFO:teuthology.orchestra.run.gibba019.stdout:After this operation, 698 MB of additional disk space will be used. 2021-07-11T03:50:48.110 INFO:teuthology.orchestra.run.gibba019.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.13-130-g45c6565d-1focal [1,698 kB] 2021-07-11T03:50:48.111 INFO:teuthology.orchestra.run.gibba023.stdout:2 upgraded, 101 newly installed, 0 to remove and 141 not upgraded. 2021-07-11T03:50:48.111 INFO:teuthology.orchestra.run.gibba023.stdout:Need to get 153 MB of archives. 2021-07-11T03:50:48.111 INFO:teuthology.orchestra.run.gibba023.stdout:After this operation, 698 MB of additional disk space will be used. 2021-07-11T03:50:48.112 INFO:teuthology.orchestra.run.gibba023.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.13-130-g45c6565d-1focal [1,698 kB] 2021-07-11T03:50:48.117 INFO:teuthology.orchestra.run.gibba005.stdout:2 upgraded, 101 newly installed, 0 to remove and 141 not upgraded. 2021-07-11T03:50:48.118 INFO:teuthology.orchestra.run.gibba005.stdout:Need to get 153 MB of archives. 2021-07-11T03:50:48.118 INFO:teuthology.orchestra.run.gibba005.stdout:After this operation, 698 MB of additional disk space will be used. 2021-07-11T03:50:48.118 INFO:teuthology.orchestra.run.gibba005.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.13-130-g45c6565d-1focal [1,698 kB] 2021-07-11T03:50:48.140 INFO:teuthology.orchestra.run.gibba019.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2021-07-11T03:50:48.152 INFO:teuthology.orchestra.run.gibba023.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2021-07-11T03:50:48.158 INFO:teuthology.orchestra.run.gibba005.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2021-07-11T03:50:48.199 INFO:teuthology.orchestra.run.gibba004.stdout:2 upgraded, 101 newly installed, 0 to remove and 141 not upgraded. 2021-07-11T03:50:48.199 INFO:teuthology.orchestra.run.gibba004.stdout:Need to get 153 MB of archives. 2021-07-11T03:50:48.199 INFO:teuthology.orchestra.run.gibba004.stdout:After this operation, 698 MB of additional disk space will be used. 2021-07-11T03:50:48.200 INFO:teuthology.orchestra.run.gibba004.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 15.2.13-130-g45c6565d-1focal [1,698 kB] 2021-07-11T03:50:48.224 INFO:teuthology.orchestra.run.gibba019.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2021-07-11T03:50:48.237 INFO:teuthology.orchestra.run.gibba004.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyperclip all 1.7.0-1 [10.5 kB] 2021-07-11T03:50:48.243 INFO:teuthology.orchestra.run.gibba005.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2021-07-11T03:50:48.255 INFO:teuthology.orchestra.run.gibba019.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2021-07-11T03:50:48.272 INFO:teuthology.orchestra.run.gibba005.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2021-07-11T03:50:48.317 INFO:teuthology.orchestra.run.gibba023.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2021-07-11T03:50:48.321 INFO:teuthology.orchestra.run.gibba004.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2021-07-11T03:50:48.346 INFO:teuthology.orchestra.run.gibba004.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2021-07-11T03:50:48.346 INFO:teuthology.orchestra.run.gibba023.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cmd2 all 0.8.5-2.1 [49.6 kB] 2021-07-11T03:50:48.362 INFO:teuthology.orchestra.run.gibba019.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.13-130-g45c6565d-1focal [3,247 kB] 2021-07-11T03:50:48.370 INFO:teuthology.orchestra.run.gibba019.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2021-07-11T03:50:48.374 INFO:teuthology.orchestra.run.gibba023.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.13-130-g45c6565d-1focal [3,247 kB] 2021-07-11T03:50:48.380 INFO:teuthology.orchestra.run.gibba005.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.13-130-g45c6565d-1focal [3,247 kB] 2021-07-11T03:50:48.390 INFO:teuthology.orchestra.run.gibba005.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2021-07-11T03:50:48.439 INFO:teuthology.orchestra.run.gibba023.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2021-07-11T03:50:48.453 INFO:teuthology.orchestra.run.gibba004.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2021-07-11T03:50:48.495 INFO:teuthology.orchestra.run.gibba004.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 15.2.13-130-g45c6565d-1focal [3,247 kB] 2021-07-11T03:50:48.509 INFO:teuthology.orchestra.run.gibba019.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2021-07-11T03:50:48.514 INFO:teuthology.orchestra.run.gibba019.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2021-07-11T03:50:48.517 INFO:teuthology.orchestra.run.gibba005.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2021-07-11T03:50:48.526 INFO:teuthology.orchestra.run.gibba005.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2021-07-11T03:50:48.529 INFO:teuthology.orchestra.run.gibba019.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2021-07-11T03:50:48.535 INFO:teuthology.orchestra.run.gibba005.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2021-07-11T03:50:48.592 INFO:teuthology.orchestra.run.gibba023.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2021-07-11T03:50:48.599 INFO:teuthology.orchestra.run.gibba023.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2021-07-11T03:50:48.602 INFO:teuthology.orchestra.run.gibba004.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2021-07-11T03:50:48.608 INFO:teuthology.orchestra.run.gibba004.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2021-07-11T03:50:48.614 INFO:teuthology.orchestra.run.gibba023.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2021-07-11T03:50:48.621 INFO:teuthology.orchestra.run.gibba004.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2021-07-11T03:50:48.631 INFO:teuthology.orchestra.run.gibba019.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2021-07-11T03:50:48.642 INFO:teuthology.orchestra.run.gibba005.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2021-07-11T03:50:48.653 INFO:teuthology.orchestra.run.gibba019.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2021-07-11T03:50:48.660 INFO:teuthology.orchestra.run.gibba019.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2021-07-11T03:50:48.666 INFO:teuthology.orchestra.run.gibba005.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2021-07-11T03:50:48.671 INFO:teuthology.orchestra.run.gibba005.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2021-07-11T03:50:48.683 INFO:teuthology.orchestra.run.gibba019.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2021-07-11T03:50:48.690 INFO:teuthology.orchestra.run.gibba019.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2021-07-11T03:50:48.693 INFO:teuthology.orchestra.run.gibba019.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2021-07-11T03:50:48.697 INFO:teuthology.orchestra.run.gibba005.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2021-07-11T03:50:48.705 INFO:teuthology.orchestra.run.gibba005.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2021-07-11T03:50:48.705 INFO:teuthology.orchestra.run.gibba005.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2021-07-11T03:50:48.706 INFO:teuthology.orchestra.run.gibba019.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2021-07-11T03:50:48.709 INFO:teuthology.orchestra.run.gibba019.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2021-07-11T03:50:48.710 INFO:teuthology.orchestra.run.gibba019.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2021-07-11T03:50:48.715 INFO:teuthology.orchestra.run.gibba023.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2021-07-11T03:50:48.721 INFO:teuthology.orchestra.run.gibba005.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2021-07-11T03:50:48.721 INFO:teuthology.orchestra.run.gibba004.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libleveldb1d amd64 1.22-3ubuntu2 [139 kB] 2021-07-11T03:50:48.722 INFO:teuthology.orchestra.run.gibba005.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2021-07-11T03:50:48.725 INFO:teuthology.orchestra.run.gibba005.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2021-07-11T03:50:48.739 INFO:teuthology.orchestra.run.gibba023.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2021-07-11T03:50:48.739 INFO:teuthology.orchestra.run.gibba019.stdout:Get:19 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.13-130-g45c6565d-1focal [463 kB] 2021-07-11T03:50:48.744 INFO:teuthology.orchestra.run.gibba004.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2021-07-11T03:50:48.745 INFO:teuthology.orchestra.run.gibba023.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2021-07-11T03:50:48.751 INFO:teuthology.orchestra.run.gibba004.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2021-07-11T03:50:48.755 INFO:teuthology.orchestra.run.gibba019.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2021-07-11T03:50:48.758 INFO:teuthology.orchestra.run.gibba019.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2021-07-11T03:50:48.758 INFO:teuthology.orchestra.run.gibba019.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2021-07-11T03:50:48.769 INFO:teuthology.orchestra.run.gibba023.stdout:Get:13 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.13-130-g45c6565d-1focal [463 kB] 2021-07-11T03:50:48.770 INFO:teuthology.orchestra.run.gibba005.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2021-07-11T03:50:48.770 INFO:teuthology.orchestra.run.gibba005.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2021-07-11T03:50:48.770 INFO:teuthology.orchestra.run.gibba005.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2021-07-11T03:50:48.780 INFO:teuthology.orchestra.run.gibba005.stdout:Get:22 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.13-130-g45c6565d-1focal [463 kB] 2021-07-11T03:50:48.780 INFO:teuthology.orchestra.run.gibba023.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2021-07-11T03:50:48.781 INFO:teuthology.orchestra.run.gibba023.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2021-07-11T03:50:48.786 INFO:teuthology.orchestra.run.gibba004.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2021-07-11T03:50:48.786 INFO:teuthology.orchestra.run.gibba023.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2021-07-11T03:50:48.793 INFO:teuthology.orchestra.run.gibba019.stdout:Get:23 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.13-130-g45c6565d-1focal [366 kB] 2021-07-11T03:50:48.794 INFO:teuthology.orchestra.run.gibba004.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2021-07-11T03:50:48.796 INFO:teuthology.orchestra.run.gibba004.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2021-07-11T03:50:48.797 INFO:teuthology.orchestra.run.gibba023.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2021-07-11T03:50:48.797 INFO:teuthology.orchestra.run.gibba023.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2021-07-11T03:50:48.808 INFO:teuthology.orchestra.run.gibba023.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2021-07-11T03:50:48.809 INFO:teuthology.orchestra.run.gibba004.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jwt all 1.7.1-2ubuntu2 [17.4 kB] 2021-07-11T03:50:48.809 INFO:teuthology.orchestra.run.gibba004.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2021-07-11T03:50:48.810 INFO:teuthology.orchestra.run.gibba004.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2021-07-11T03:50:48.824 INFO:teuthology.orchestra.run.gibba023.stdout:Get:20 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.13-130-g45c6565d-1focal [366 kB] 2021-07-11T03:50:48.827 INFO:teuthology.orchestra.run.gibba005.stdout:Get:23 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.13-130-g45c6565d-1focal [366 kB] 2021-07-11T03:50:48.837 INFO:teuthology.orchestra.run.gibba019.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2021-07-11T03:50:48.839 INFO:teuthology.orchestra.run.gibba019.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2021-07-11T03:50:48.840 INFO:teuthology.orchestra.run.gibba019.stdout:Get:26 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.13-130-g45c6565d-1focal [20.8 kB] 2021-07-11T03:50:48.841 INFO:teuthology.orchestra.run.gibba019.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2021-07-11T03:50:48.841 INFO:teuthology.orchestra.run.gibba019.stdout:Get:28 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.13-130-g45c6565d-1focal [156 kB] 2021-07-11T03:50:48.841 INFO:teuthology.orchestra.run.gibba023.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2021-07-11T03:50:48.844 INFO:teuthology.orchestra.run.gibba019.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-waitress all 1.4.1-1 [80.6 kB] 2021-07-11T03:50:48.844 INFO:teuthology.orchestra.run.gibba023.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2021-07-11T03:50:48.845 INFO:teuthology.orchestra.run.gibba023.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2021-07-11T03:50:48.847 INFO:teuthology.orchestra.run.gibba005.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2021-07-11T03:50:48.850 INFO:teuthology.orchestra.run.gibba004.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2 [59.1 kB] 2021-07-11T03:50:48.850 INFO:teuthology.orchestra.run.gibba019.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2021-07-11T03:50:48.850 INFO:teuthology.orchestra.run.gibba019.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2021-07-11T03:50:48.851 INFO:teuthology.orchestra.run.gibba005.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2021-07-11T03:50:48.851 INFO:teuthology.orchestra.run.gibba005.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2021-07-11T03:50:48.852 INFO:teuthology.orchestra.run.gibba004.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2021-07-11T03:50:48.853 INFO:teuthology.orchestra.run.gibba004.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2021-07-11T03:50:48.855 INFO:teuthology.orchestra.run.gibba005.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-waitress all 1.4.1-1 [80.6 kB] 2021-07-11T03:50:48.858 INFO:teuthology.orchestra.run.gibba005.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2021-07-11T03:50:48.861 INFO:teuthology.orchestra.run.gibba005.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2021-07-11T03:50:48.869 INFO:teuthology.orchestra.run.gibba019.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2021-07-11T03:50:48.870 INFO:teuthology.orchestra.run.gibba023.stdout:Get:24 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.13-130-g45c6565d-1focal [20.8 kB] 2021-07-11T03:50:48.870 INFO:teuthology.orchestra.run.gibba023.stdout:Get:25 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.13-130-g45c6565d-1focal [156 kB] 2021-07-11T03:50:48.879 INFO:teuthology.orchestra.run.gibba005.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2021-07-11T03:50:48.879 INFO:teuthology.orchestra.run.gibba005.stdout:Get:31 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.13-130-g45c6565d-1focal [20.8 kB] 2021-07-11T03:50:48.880 INFO:teuthology.orchestra.run.gibba005.stdout:Get:32 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.13-130-g45c6565d-1focal [156 kB] 2021-07-11T03:50:48.886 INFO:teuthology.orchestra.run.gibba019.stdout:Get:33 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.13-130-g45c6565d-1focal [29.1 kB] 2021-07-11T03:50:48.887 INFO:teuthology.orchestra.run.gibba019.stdout:Get:34 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.13-130-g45c6565d-1focal [362 kB] 2021-07-11T03:50:48.899 INFO:teuthology.orchestra.run.gibba019.stdout:Get:35 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.13-130-g45c6565d-1focal [3,141 kB] 2021-07-11T03:50:48.911 INFO:teuthology.orchestra.run.gibba023.stdout:Get:26 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.13-130-g45c6565d-1focal [29.1 kB] 2021-07-11T03:50:48.911 INFO:teuthology.orchestra.run.gibba023.stdout:Get:27 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.13-130-g45c6565d-1focal [362 kB] 2021-07-11T03:50:48.919 INFO:teuthology.orchestra.run.gibba019.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2021-07-11T03:50:48.921 INFO:teuthology.orchestra.run.gibba019.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2021-07-11T03:50:48.922 INFO:teuthology.orchestra.run.gibba019.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2021-07-11T03:50:48.923 INFO:teuthology.orchestra.run.gibba023.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2021-07-11T03:50:48.926 INFO:teuthology.orchestra.run.gibba023.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2021-07-11T03:50:48.927 INFO:teuthology.orchestra.run.gibba023.stdout:Get:30 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.13-130-g45c6565d-1focal [3,141 kB] 2021-07-11T03:50:48.927 INFO:teuthology.orchestra.run.gibba005.stdout:Get:33 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.13-130-g45c6565d-1focal [29.1 kB] 2021-07-11T03:50:48.928 INFO:teuthology.orchestra.run.gibba023.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2021-07-11T03:50:48.928 INFO:teuthology.orchestra.run.gibba005.stdout:Get:34 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.13-130-g45c6565d-1focal [362 kB] 2021-07-11T03:50:48.929 INFO:teuthology.orchestra.run.gibba005.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2021-07-11T03:50:48.931 INFO:teuthology.orchestra.run.gibba004.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2021-07-11T03:50:48.931 INFO:teuthology.orchestra.run.gibba005.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2021-07-11T03:50:48.932 INFO:teuthology.orchestra.run.gibba005.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2021-07-11T03:50:48.932 INFO:teuthology.orchestra.run.gibba023.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-waitress all 1.4.1-1 [80.6 kB] 2021-07-11T03:50:48.936 INFO:teuthology.orchestra.run.gibba023.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2021-07-11T03:50:48.936 INFO:teuthology.orchestra.run.gibba004.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2021-07-11T03:50:48.936 INFO:teuthology.orchestra.run.gibba004.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2021-07-11T03:50:48.937 INFO:teuthology.orchestra.run.gibba023.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2021-07-11T03:50:48.939 INFO:teuthology.orchestra.run.gibba004.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-waitress all 1.4.1-1 [80.6 kB] 2021-07-11T03:50:48.941 INFO:teuthology.orchestra.run.gibba005.stdout:Get:38 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.13-130-g45c6565d-1focal [3,141 kB] 2021-07-11T03:50:48.948 INFO:teuthology.orchestra.run.gibba004.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2021-07-11T03:50:48.948 INFO:teuthology.orchestra.run.gibba004.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2021-07-11T03:50:48.977 INFO:teuthology.orchestra.run.gibba023.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2021-07-11T03:50:48.977 INFO:teuthology.orchestra.run.gibba004.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2021-07-11T03:50:48.983 INFO:teuthology.orchestra.run.gibba004.stdout:Get:29 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 15.2.13-130-g45c6565d-1focal [463 kB] 2021-07-11T03:50:49.003 INFO:teuthology.orchestra.run.gibba019.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2021-07-11T03:50:49.004 INFO:teuthology.orchestra.run.gibba023.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2021-07-11T03:50:49.006 INFO:teuthology.orchestra.run.gibba019.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2021-07-11T03:50:49.007 INFO:teuthology.orchestra.run.gibba023.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2021-07-11T03:50:49.014 INFO:teuthology.orchestra.run.gibba005.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2021-07-11T03:50:49.015 INFO:teuthology.orchestra.run.gibba023.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2021-07-11T03:50:49.016 INFO:teuthology.orchestra.run.gibba019.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2021-07-11T03:50:49.016 INFO:teuthology.orchestra.run.gibba019.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2021-07-11T03:50:49.016 INFO:teuthology.orchestra.run.gibba019.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2021-07-11T03:50:49.017 INFO:teuthology.orchestra.run.gibba019.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2021-07-11T03:50:49.017 INFO:teuthology.orchestra.run.gibba019.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2021-07-11T03:50:49.017 INFO:teuthology.orchestra.run.gibba004.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2021-07-11T03:50:49.018 INFO:teuthology.orchestra.run.gibba004.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2021-07-11T03:50:49.030 INFO:teuthology.orchestra.run.gibba004.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2021-07-11T03:50:49.050 INFO:teuthology.orchestra.run.gibba005.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2021-07-11T03:50:49.051 INFO:teuthology.orchestra.run.gibba005.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2021-07-11T03:50:49.051 INFO:teuthology.orchestra.run.gibba005.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2021-07-11T03:50:49.051 INFO:teuthology.orchestra.run.gibba005.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2021-07-11T03:50:49.051 INFO:teuthology.orchestra.run.gibba005.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2021-07-11T03:50:49.052 INFO:teuthology.orchestra.run.gibba005.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2021-07-11T03:50:49.052 INFO:teuthology.orchestra.run.gibba004.stdout:Get:33 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 15.2.13-130-g45c6565d-1focal [366 kB] 2021-07-11T03:50:49.083 INFO:teuthology.orchestra.run.gibba019.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2021-07-11T03:50:49.083 INFO:teuthology.orchestra.run.gibba019.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2021-07-11T03:50:49.091 INFO:teuthology.orchestra.run.gibba023.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2021-07-11T03:50:49.095 INFO:teuthology.orchestra.run.gibba023.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2021-07-11T03:50:49.101 INFO:teuthology.orchestra.run.gibba023.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2021-07-11T03:50:49.102 INFO:teuthology.orchestra.run.gibba023.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2021-07-11T03:50:49.102 INFO:teuthology.orchestra.run.gibba005.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2021-07-11T03:50:49.102 INFO:teuthology.orchestra.run.gibba005.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2021-07-11T03:50:49.103 INFO:teuthology.orchestra.run.gibba004.stdout:Get:34 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 15.2.13-130-g45c6565d-1focal [20.8 kB] 2021-07-11T03:50:49.103 INFO:teuthology.orchestra.run.gibba004.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-werkzeug all 0.16.1+dfsg1-2 [183 kB] 2021-07-11T03:50:49.104 INFO:teuthology.orchestra.run.gibba004.stdout:Get:36 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 15.2.13-130-g45c6565d-1focal [156 kB] 2021-07-11T03:50:49.104 INFO:teuthology.orchestra.run.gibba023.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2021-07-11T03:50:49.107 INFO:teuthology.orchestra.run.gibba023.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2021-07-11T03:50:49.111 INFO:teuthology.orchestra.run.gibba023.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2021-07-11T03:50:49.115 INFO:teuthology.orchestra.run.gibba004.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2021-07-11T03:50:49.121 INFO:teuthology.orchestra.run.gibba004.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2021-07-11T03:50:49.122 INFO:teuthology.orchestra.run.gibba004.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2021-07-11T03:50:49.130 INFO:teuthology.orchestra.run.gibba004.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2021-07-11T03:50:49.138 INFO:teuthology.orchestra.run.gibba004.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2021-07-11T03:50:49.141 INFO:teuthology.orchestra.run.gibba004.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2021-07-11T03:50:49.178 INFO:teuthology.orchestra.run.gibba004.stdout:Get:43 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 15.2.13-130-g45c6565d-1focal [29.1 kB] 2021-07-11T03:50:49.180 INFO:teuthology.orchestra.run.gibba004.stdout:Get:44 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 15.2.13-130-g45c6565d-1focal [362 kB] 2021-07-11T03:50:49.187 INFO:teuthology.orchestra.run.gibba004.stdout:Get:45 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 15.2.13-130-g45c6565d-1focal [3,141 kB] 2021-07-11T03:50:49.188 INFO:teuthology.orchestra.run.gibba023.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2021-07-11T03:50:49.189 INFO:teuthology.orchestra.run.gibba023.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2021-07-11T03:50:49.210 INFO:teuthology.orchestra.run.gibba004.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2021-07-11T03:50:49.210 INFO:teuthology.orchestra.run.gibba004.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2021-07-11T03:50:49.393 INFO:teuthology.orchestra.run.gibba019.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.13-130-g45c6565d-1focal [115 kB] 2021-07-11T03:50:49.393 INFO:teuthology.orchestra.run.gibba019.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.13-130-g45c6565d-1focal [361 kB] 2021-07-11T03:50:49.427 INFO:teuthology.orchestra.run.gibba023.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.13-130-g45c6565d-1focal [115 kB] 2021-07-11T03:50:49.429 INFO:teuthology.orchestra.run.gibba023.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.13-130-g45c6565d-1focal [361 kB] 2021-07-11T03:50:49.435 INFO:teuthology.orchestra.run.gibba005.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.13-130-g45c6565d-1focal [115 kB] 2021-07-11T03:50:49.435 INFO:teuthology.orchestra.run.gibba005.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.13-130-g45c6565d-1focal [361 kB] 2021-07-11T03:50:49.436 INFO:teuthology.orchestra.run.gibba019.stdout:Get:50 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.13-130-g45c6565d-1focal [20.0 MB] 2021-07-11T03:50:49.453 INFO:teuthology.orchestra.run.gibba023.stdout:Get:50 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.13-130-g45c6565d-1focal [20.0 MB] 2021-07-11T03:50:49.462 INFO:teuthology.orchestra.run.gibba005.stdout:Get:50 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.13-130-g45c6565d-1focal [20.0 MB] 2021-07-11T03:50:49.762 INFO:teuthology.orchestra.run.gibba004.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 15.2.13-130-g45c6565d-1focal [115 kB] 2021-07-11T03:50:49.763 INFO:teuthology.orchestra.run.gibba005.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2021-07-11T03:50:49.764 INFO:teuthology.orchestra.run.gibba004.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 15.2.13-130-g45c6565d-1focal [361 kB] 2021-07-11T03:50:49.764 INFO:teuthology.orchestra.run.gibba019.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2021-07-11T03:50:49.796 INFO:teuthology.orchestra.run.gibba004.stdout:Get:50 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 15.2.13-130-g45c6565d-1focal [20.0 MB] 2021-07-11T03:50:49.872 INFO:teuthology.orchestra.run.gibba019.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2021-07-11T03:50:49.876 INFO:teuthology.orchestra.run.gibba019.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2021-07-11T03:50:49.879 INFO:teuthology.orchestra.run.gibba005.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2021-07-11T03:50:49.884 INFO:teuthology.orchestra.run.gibba005.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2021-07-11T03:50:49.918 INFO:teuthology.orchestra.run.gibba004.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2021-07-11T03:50:50.055 INFO:teuthology.orchestra.run.gibba005.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2021-07-11T03:50:50.055 INFO:teuthology.orchestra.run.gibba005.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2021-07-11T03:50:50.056 INFO:teuthology.orchestra.run.gibba005.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2021-07-11T03:50:50.057 INFO:teuthology.orchestra.run.gibba005.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2021-07-11T03:50:50.057 INFO:teuthology.orchestra.run.gibba005.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2021-07-11T03:50:50.057 INFO:teuthology.orchestra.run.gibba005.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2021-07-11T03:50:50.058 INFO:teuthology.orchestra.run.gibba005.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2021-07-11T03:50:50.059 INFO:teuthology.orchestra.run.gibba019.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2021-07-11T03:50:50.059 INFO:teuthology.orchestra.run.gibba019.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2021-07-11T03:50:50.059 INFO:teuthology.orchestra.run.gibba019.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2021-07-11T03:50:50.060 INFO:teuthology.orchestra.run.gibba019.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2021-07-11T03:50:50.060 INFO:teuthology.orchestra.run.gibba019.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2021-07-11T03:50:50.060 INFO:teuthology.orchestra.run.gibba019.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2021-07-11T03:50:50.060 INFO:teuthology.orchestra.run.gibba019.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2021-07-11T03:50:50.092 INFO:teuthology.orchestra.run.gibba019.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2021-07-11T03:50:50.092 INFO:teuthology.orchestra.run.gibba019.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2021-07-11T03:50:50.095 INFO:teuthology.orchestra.run.gibba004.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2021-07-11T03:50:50.096 INFO:teuthology.orchestra.run.gibba004.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2021-07-11T03:50:50.096 INFO:teuthology.orchestra.run.gibba005.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2021-07-11T03:50:50.096 INFO:teuthology.orchestra.run.gibba005.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2021-07-11T03:50:50.156 INFO:teuthology.orchestra.run.gibba023.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2021-07-11T03:50:50.177 INFO:teuthology.orchestra.run.gibba019.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2021-07-11T03:50:50.178 INFO:teuthology.orchestra.run.gibba004.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2021-07-11T03:50:50.178 INFO:teuthology.orchestra.run.gibba004.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2021-07-11T03:50:50.179 INFO:teuthology.orchestra.run.gibba004.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2021-07-11T03:50:50.180 INFO:teuthology.orchestra.run.gibba005.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2021-07-11T03:50:50.182 INFO:teuthology.orchestra.run.gibba004.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2021-07-11T03:50:50.183 INFO:teuthology.orchestra.run.gibba004.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2021-07-11T03:50:50.185 INFO:teuthology.orchestra.run.gibba004.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2021-07-11T03:50:50.185 INFO:teuthology.orchestra.run.gibba004.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2021-07-11T03:50:50.187 INFO:teuthology.orchestra.run.gibba019.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2021-07-11T03:50:50.189 INFO:teuthology.orchestra.run.gibba005.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2021-07-11T03:50:50.192 INFO:teuthology.orchestra.run.gibba019.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2021-07-11T03:50:50.194 INFO:teuthology.orchestra.run.gibba005.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2021-07-11T03:50:50.196 INFO:teuthology.orchestra.run.gibba019.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2021-07-11T03:50:50.199 INFO:teuthology.orchestra.run.gibba019.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2021-07-11T03:50:50.199 INFO:teuthology.orchestra.run.gibba005.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2021-07-11T03:50:50.202 INFO:teuthology.orchestra.run.gibba005.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2021-07-11T03:50:50.204 INFO:teuthology.orchestra.run.gibba019.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2021-07-11T03:50:50.208 INFO:teuthology.orchestra.run.gibba019.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2021-07-11T03:50:50.212 INFO:teuthology.orchestra.run.gibba005.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2021-07-11T03:50:50.216 INFO:teuthology.orchestra.run.gibba005.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2021-07-11T03:50:50.217 INFO:teuthology.orchestra.run.gibba019.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2021-07-11T03:50:50.221 INFO:teuthology.orchestra.run.gibba005.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2021-07-11T03:50:50.225 INFO:teuthology.orchestra.run.gibba023.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2021-07-11T03:50:50.228 INFO:teuthology.orchestra.run.gibba023.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2021-07-11T03:50:50.261 INFO:teuthology.orchestra.run.gibba019.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2021-07-11T03:50:50.262 INFO:teuthology.orchestra.run.gibba005.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2021-07-11T03:50:50.271 INFO:teuthology.orchestra.run.gibba019.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2021-07-11T03:50:50.272 INFO:teuthology.orchestra.run.gibba004.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2021-07-11T03:50:50.273 INFO:teuthology.orchestra.run.gibba004.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2021-07-11T03:50:50.273 INFO:teuthology.orchestra.run.gibba004.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2021-07-11T03:50:50.274 INFO:teuthology.orchestra.run.gibba005.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2021-07-11T03:50:50.275 INFO:teuthology.orchestra.run.gibba004.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2021-07-11T03:50:50.282 INFO:teuthology.orchestra.run.gibba004.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2021-07-11T03:50:50.283 INFO:teuthology.orchestra.run.gibba004.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2021-07-11T03:50:50.284 INFO:teuthology.orchestra.run.gibba004.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2021-07-11T03:50:50.290 INFO:teuthology.orchestra.run.gibba004.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2021-07-11T03:50:50.296 INFO:teuthology.orchestra.run.gibba023.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2021-07-11T03:50:50.296 INFO:teuthology.orchestra.run.gibba023.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2021-07-11T03:50:50.297 INFO:teuthology.orchestra.run.gibba023.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2021-07-11T03:50:50.297 INFO:teuthology.orchestra.run.gibba023.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2021-07-11T03:50:50.298 INFO:teuthology.orchestra.run.gibba023.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2021-07-11T03:50:50.298 INFO:teuthology.orchestra.run.gibba023.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2021-07-11T03:50:50.299 INFO:teuthology.orchestra.run.gibba023.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2021-07-11T03:50:50.323 INFO:teuthology.orchestra.run.gibba004.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2021-07-11T03:50:50.326 INFO:teuthology.orchestra.run.gibba004.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2021-07-11T03:50:50.331 INFO:teuthology.orchestra.run.gibba023.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2021-07-11T03:50:50.331 INFO:teuthology.orchestra.run.gibba023.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2021-07-11T03:50:50.342 INFO:teuthology.orchestra.run.gibba019.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2021-07-11T03:50:50.343 INFO:teuthology.orchestra.run.gibba019.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2021-07-11T03:50:50.344 INFO:teuthology.orchestra.run.gibba019.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2021-07-11T03:50:50.345 INFO:teuthology.orchestra.run.gibba005.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2021-07-11T03:50:50.348 INFO:teuthology.orchestra.run.gibba019.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.3 [1,382 kB] 2021-07-11T03:50:50.348 INFO:teuthology.orchestra.run.gibba005.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2021-07-11T03:50:50.349 INFO:teuthology.orchestra.run.gibba005.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2021-07-11T03:50:50.351 INFO:teuthology.orchestra.run.gibba005.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.3 [1,382 kB] 2021-07-11T03:50:50.380 INFO:teuthology.orchestra.run.gibba019.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2021-07-11T03:50:50.380 INFO:teuthology.orchestra.run.gibba019.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2021-07-11T03:50:50.381 INFO:teuthology.orchestra.run.gibba019.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2021-07-11T03:50:50.382 INFO:teuthology.orchestra.run.gibba019.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2021-07-11T03:50:50.398 INFO:teuthology.orchestra.run.gibba005.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2021-07-11T03:50:50.399 INFO:teuthology.orchestra.run.gibba005.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2021-07-11T03:50:50.399 INFO:teuthology.orchestra.run.gibba005.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2021-07-11T03:50:50.399 INFO:teuthology.orchestra.run.gibba005.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2021-07-11T03:50:50.400 INFO:teuthology.orchestra.run.gibba004.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2021-07-11T03:50:50.410 INFO:teuthology.orchestra.run.gibba004.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2021-07-11T03:50:50.411 INFO:teuthology.orchestra.run.gibba004.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2021-07-11T03:50:50.412 INFO:teuthology.orchestra.run.gibba023.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2021-07-11T03:50:50.413 INFO:teuthology.orchestra.run.gibba004.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2021-07-11T03:50:50.413 INFO:teuthology.orchestra.run.gibba004.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2021-07-11T03:50:50.416 INFO:teuthology.orchestra.run.gibba004.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.3 [1,382 kB] 2021-07-11T03:50:50.429 INFO:teuthology.orchestra.run.gibba023.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2021-07-11T03:50:50.432 INFO:teuthology.orchestra.run.gibba019.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-py all 1.8.1-1 [65.4 kB] 2021-07-11T03:50:50.433 INFO:teuthology.orchestra.run.gibba019.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2021-07-11T03:50:50.436 INFO:teuthology.orchestra.run.gibba005.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-py all 1.8.1-1 [65.4 kB] 2021-07-11T03:50:50.439 INFO:teuthology.orchestra.run.gibba005.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2021-07-11T03:50:50.440 INFO:teuthology.orchestra.run.gibba023.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2021-07-11T03:50:50.440 INFO:teuthology.orchestra.run.gibba004.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2021-07-11T03:50:50.440 INFO:teuthology.orchestra.run.gibba004.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2021-07-11T03:50:50.443 INFO:teuthology.orchestra.run.gibba023.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2021-07-11T03:50:50.445 INFO:teuthology.orchestra.run.gibba023.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2021-07-11T03:50:50.456 INFO:teuthology.orchestra.run.gibba023.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 libxslt1.1 amd64 1.1.34-4 [152 kB] 2021-07-11T03:50:50.460 INFO:teuthology.orchestra.run.gibba023.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2021-07-11T03:50:50.464 INFO:teuthology.orchestra.run.gibba023.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-colorama all 0.4.3-1build1 [23.9 kB] 2021-07-11T03:50:50.482 INFO:teuthology.orchestra.run.gibba004.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2021-07-11T03:50:50.483 INFO:teuthology.orchestra.run.gibba004.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2021-07-11T03:50:50.494 INFO:teuthology.orchestra.run.gibba023.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2021-07-11T03:50:50.503 INFO:teuthology.orchestra.run.gibba023.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2021-07-11T03:50:50.519 INFO:teuthology.orchestra.run.gibba019.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2021-07-11T03:50:50.519 INFO:teuthology.orchestra.run.gibba005.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2021-07-11T03:50:50.520 INFO:teuthology.orchestra.run.gibba019.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2021-07-11T03:50:50.525 INFO:teuthology.orchestra.run.gibba005.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2021-07-11T03:50:50.564 INFO:teuthology.orchestra.run.gibba004.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-py all 1.8.1-1 [65.4 kB] 2021-07-11T03:50:50.566 INFO:teuthology.orchestra.run.gibba004.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2021-07-11T03:50:50.567 INFO:teuthology.orchestra.run.gibba004.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2021-07-11T03:50:50.572 INFO:teuthology.orchestra.run.gibba004.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2021-07-11T03:50:50.577 INFO:teuthology.orchestra.run.gibba023.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2021-07-11T03:50:50.578 INFO:teuthology.orchestra.run.gibba023.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2021-07-11T03:50:50.580 INFO:teuthology.orchestra.run.gibba023.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2021-07-11T03:50:50.583 INFO:teuthology.orchestra.run.gibba023.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.3 [1,382 kB] 2021-07-11T03:50:50.624 INFO:teuthology.orchestra.run.gibba023.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2021-07-11T03:50:50.625 INFO:teuthology.orchestra.run.gibba023.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2021-07-11T03:50:50.626 INFO:teuthology.orchestra.run.gibba023.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2021-07-11T03:50:50.627 INFO:teuthology.orchestra.run.gibba023.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2021-07-11T03:50:50.667 INFO:teuthology.orchestra.run.gibba023.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-py all 1.8.1-1 [65.4 kB] 2021-07-11T03:50:50.667 INFO:teuthology.orchestra.run.gibba023.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2021-07-11T03:50:50.742 INFO:teuthology.orchestra.run.gibba023.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2021-07-11T03:50:50.752 INFO:teuthology.orchestra.run.gibba023.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2021-07-11T03:50:52.202 INFO:teuthology.orchestra.run.gibba019.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.13-130-g45c6565d-1focal [5,303 kB] 2021-07-11T03:50:52.267 INFO:teuthology.orchestra.run.gibba004.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.13-130-g45c6565d-1focal [5,303 kB] 2021-07-11T03:50:52.356 INFO:teuthology.orchestra.run.gibba023.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.13-130-g45c6565d-1focal [5,303 kB] 2021-07-11T03:50:52.439 INFO:teuthology.orchestra.run.gibba005.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 15.2.13-130-g45c6565d-1focal [5,303 kB] 2021-07-11T03:50:52.790 INFO:teuthology.orchestra.run.gibba004.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.13-130-g45c6565d-1focal [162 kB] 2021-07-11T03:50:52.817 INFO:teuthology.orchestra.run.gibba019.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.13-130-g45c6565d-1focal [162 kB] 2021-07-11T03:50:52.825 INFO:teuthology.orchestra.run.gibba004.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.13-130-g45c6565d-1focal [1,401 kB] 2021-07-11T03:50:52.851 INFO:teuthology.orchestra.run.gibba019.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.13-130-g45c6565d-1focal [1,401 kB] 2021-07-11T03:50:52.979 INFO:teuthology.orchestra.run.gibba004.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.13-130-g45c6565d-1focal [6,171 kB] 2021-07-11T03:50:53.030 INFO:teuthology.orchestra.run.gibba023.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.13-130-g45c6565d-1focal [162 kB] 2021-07-11T03:50:53.032 INFO:teuthology.orchestra.run.gibba019.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.13-130-g45c6565d-1focal [6,171 kB] 2021-07-11T03:50:53.040 INFO:teuthology.orchestra.run.gibba023.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.13-130-g45c6565d-1focal [1,401 kB] 2021-07-11T03:50:53.102 INFO:teuthology.orchestra.run.gibba005.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 15.2.13-130-g45c6565d-1focal [162 kB] 2021-07-11T03:50:53.135 INFO:teuthology.orchestra.run.gibba005.stdout:Get:87 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 15.2.13-130-g45c6565d-1focal [1,401 kB] 2021-07-11T03:50:53.241 INFO:teuthology.orchestra.run.gibba023.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.13-130-g45c6565d-1focal [6,171 kB] 2021-07-11T03:50:53.327 INFO:teuthology.orchestra.run.gibba005.stdout:Get:88 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 15.2.13-130-g45c6565d-1focal [6,171 kB] 2021-07-11T03:50:53.706 INFO:teuthology.orchestra.run.gibba004.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.13-130-g45c6565d-1focal [23.8 MB] 2021-07-11T03:50:53.851 INFO:teuthology.orchestra.run.gibba019.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.13-130-g45c6565d-1focal [23.8 MB] 2021-07-11T03:50:54.091 INFO:teuthology.orchestra.run.gibba023.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.13-130-g45c6565d-1focal [23.8 MB] 2021-07-11T03:50:54.188 INFO:teuthology.orchestra.run.gibba005.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 15.2.13-130-g45c6565d-1focal [23.8 MB] 2021-07-11T03:50:56.691 INFO:teuthology.orchestra.run.gibba004.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.13-130-g45c6565d-1focal [3,988 B] 2021-07-11T03:50:56.691 INFO:teuthology.orchestra.run.gibba004.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.13-130-g45c6565d-1focal [538 kB] 2021-07-11T03:50:56.706 INFO:teuthology.orchestra.run.gibba004.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.13-130-g45c6565d-1focal [2,007 kB] 2021-07-11T03:50:56.949 INFO:teuthology.orchestra.run.gibba004.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.13-130-g45c6565d-1focal [53.1 kB] 2021-07-11T03:50:56.951 INFO:teuthology.orchestra.run.gibba004.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.13-130-g45c6565d-1focal [58.1 kB] 2021-07-11T03:50:56.955 INFO:teuthology.orchestra.run.gibba004.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.13-130-g45c6565d-1focal [3,200 kB] 2021-07-11T03:50:57.145 INFO:teuthology.orchestra.run.gibba019.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.13-130-g45c6565d-1focal [3,988 B] 2021-07-11T03:50:57.146 INFO:teuthology.orchestra.run.gibba019.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.13-130-g45c6565d-1focal [538 kB] 2021-07-11T03:50:57.161 INFO:teuthology.orchestra.run.gibba019.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.13-130-g45c6565d-1focal [2,007 kB] 2021-07-11T03:50:57.332 INFO:teuthology.orchestra.run.gibba004.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.13-130-g45c6565d-1focal [34.3 kB] 2021-07-11T03:50:57.333 INFO:teuthology.orchestra.run.gibba004.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.13-130-g45c6565d-1focal [7,880 kB] 2021-07-11T03:50:57.419 INFO:teuthology.orchestra.run.gibba019.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.13-130-g45c6565d-1focal [53.1 kB] 2021-07-11T03:50:57.435 INFO:teuthology.orchestra.run.gibba019.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.13-130-g45c6565d-1focal [58.1 kB] 2021-07-11T03:50:57.437 INFO:teuthology.orchestra.run.gibba019.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.13-130-g45c6565d-1focal [3,200 kB] 2021-07-11T03:50:57.470 INFO:teuthology.orchestra.run.gibba023.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.13-130-g45c6565d-1focal [3,988 B] 2021-07-11T03:50:57.471 INFO:teuthology.orchestra.run.gibba023.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.13-130-g45c6565d-1focal [538 kB] 2021-07-11T03:50:57.489 INFO:teuthology.orchestra.run.gibba023.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.13-130-g45c6565d-1focal [2,007 kB] 2021-07-11T03:50:57.564 INFO:teuthology.orchestra.run.gibba005.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 15.2.13-130-g45c6565d-1focal [3,988 B] 2021-07-11T03:50:57.611 INFO:teuthology.orchestra.run.gibba005.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 15.2.13-130-g45c6565d-1focal [538 kB] 2021-07-11T03:50:57.623 INFO:teuthology.orchestra.run.gibba005.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 15.2.13-130-g45c6565d-1focal [2,007 kB] 2021-07-11T03:50:57.763 INFO:teuthology.orchestra.run.gibba023.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.13-130-g45c6565d-1focal [53.1 kB] 2021-07-11T03:50:57.763 INFO:teuthology.orchestra.run.gibba023.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.13-130-g45c6565d-1focal [58.1 kB] 2021-07-11T03:50:57.770 INFO:teuthology.orchestra.run.gibba023.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.13-130-g45c6565d-1focal [3,200 kB] 2021-07-11T03:50:57.855 INFO:teuthology.orchestra.run.gibba005.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 15.2.13-130-g45c6565d-1focal [53.1 kB] 2021-07-11T03:50:57.858 INFO:teuthology.orchestra.run.gibba019.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.13-130-g45c6565d-1focal [34.3 kB] 2021-07-11T03:50:57.859 INFO:teuthology.orchestra.run.gibba019.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.13-130-g45c6565d-1focal [7,880 kB] 2021-07-11T03:50:57.873 INFO:teuthology.orchestra.run.gibba005.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 15.2.13-130-g45c6565d-1focal [58.1 kB] 2021-07-11T03:50:57.875 INFO:teuthology.orchestra.run.gibba005.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 15.2.13-130-g45c6565d-1focal [3,200 kB] 2021-07-11T03:50:58.183 INFO:teuthology.orchestra.run.gibba023.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.13-130-g45c6565d-1focal [34.3 kB] 2021-07-11T03:50:58.184 INFO:teuthology.orchestra.run.gibba023.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.13-130-g45c6565d-1focal [7,880 kB] 2021-07-11T03:50:58.256 INFO:teuthology.orchestra.run.gibba004.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.13-130-g45c6565d-1focal [4,104 B] 2021-07-11T03:50:58.257 INFO:teuthology.orchestra.run.gibba004.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.13-130-g45c6565d-1focal [39.9 MB] 2021-07-11T03:50:58.318 INFO:teuthology.orchestra.run.gibba005.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-cloud all 15.2.13-130-g45c6565d-1focal [34.3 kB] 2021-07-11T03:50:58.318 INFO:teuthology.orchestra.run.gibba005.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 15.2.13-130-g45c6565d-1focal [7,880 kB] 2021-07-11T03:50:58.890 INFO:teuthology.orchestra.run.gibba019.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.13-130-g45c6565d-1focal [4,104 B] 2021-07-11T03:50:58.891 INFO:teuthology.orchestra.run.gibba019.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.13-130-g45c6565d-1focal [39.9 MB] 2021-07-11T03:50:59.276 INFO:teuthology.orchestra.run.gibba023.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.13-130-g45c6565d-1focal [4,104 B] 2021-07-11T03:50:59.277 INFO:teuthology.orchestra.run.gibba023.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.13-130-g45c6565d-1focal [39.9 MB] 2021-07-11T03:50:59.375 INFO:teuthology.orchestra.run.gibba005.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 15.2.13-130-g45c6565d-1focal [4,104 B] 2021-07-11T03:50:59.376 INFO:teuthology.orchestra.run.gibba005.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 15.2.13-130-g45c6565d-1focal [39.9 MB] 2021-07-11T03:51:03.333 INFO:teuthology.orchestra.run.gibba004.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.13-130-g45c6565d-1focal [15.8 kB] 2021-07-11T03:51:03.334 INFO:teuthology.orchestra.run.gibba004.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.13-130-g45c6565d-1focal [18.1 kB] 2021-07-11T03:51:03.334 INFO:teuthology.orchestra.run.gibba004.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.13-130-g45c6565d-1focal [9,726 kB] 2021-07-11T03:51:04.320 INFO:teuthology.orchestra.run.gibba019.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.13-130-g45c6565d-1focal [15.8 kB] 2021-07-11T03:51:04.320 INFO:teuthology.orchestra.run.gibba019.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.13-130-g45c6565d-1focal [18.1 kB] 2021-07-11T03:51:04.321 INFO:teuthology.orchestra.run.gibba019.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.13-130-g45c6565d-1focal [9,726 kB] 2021-07-11T03:51:04.504 INFO:teuthology.orchestra.run.gibba004.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.13-130-g45c6565d-1focal [60.9 kB] 2021-07-11T03:51:04.705 INFO:teuthology.orchestra.run.gibba023.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.13-130-g45c6565d-1focal [15.8 kB] 2021-07-11T03:51:04.706 INFO:teuthology.orchestra.run.gibba023.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.13-130-g45c6565d-1focal [18.1 kB] 2021-07-11T03:51:04.706 INFO:teuthology.orchestra.run.gibba023.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.13-130-g45c6565d-1focal [9,726 kB] 2021-07-11T03:51:04.796 INFO:teuthology.orchestra.run.gibba005.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 cephfs-shell all 15.2.13-130-g45c6565d-1focal [15.8 kB] 2021-07-11T03:51:04.797 INFO:teuthology.orchestra.run.gibba005.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 15.2.13-130-g45c6565d-1focal [18.1 kB] 2021-07-11T03:51:04.797 INFO:teuthology.orchestra.run.gibba005.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 15.2.13-130-g45c6565d-1focal [9,726 kB] 2021-07-11T03:51:04.823 INFO:teuthology.orchestra.run.gibba004.stdout:Fetched 153 MB in 16s (9,297 kB/s) 2021-07-11T03:51:04.953 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-pyperclip. 2021-07-11T03:51:05.021 INFO:teuthology.orchestra.run.gibba004.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138106 files and directories currently installed.) 2021-07-11T03:51:05.023 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2021-07-11T03:51:05.068 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2021-07-11T03:51:05.202 INFO:teuthology.orchestra.run.gibba019.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.13-130-g45c6565d-1focal [60.9 kB] 2021-07-11T03:51:05.336 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-wcwidth. 2021-07-11T03:51:05.362 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2021-07-11T03:51:05.402 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2021-07-11T03:51:05.417 INFO:teuthology.orchestra.run.gibba023.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.13-130-g45c6565d-1focal [60.9 kB] 2021-07-11T03:51:05.473 INFO:teuthology.orchestra.run.gibba005.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/octopus/45c6565d67f116c9b3509842b718398a5b38cf73/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 15.2.13-130-g45c6565d-1focal [60.9 kB] 2021-07-11T03:51:05.519 INFO:teuthology.orchestra.run.gibba019.stdout:Fetched 153 MB in 17s (8,868 kB/s) 2021-07-11T03:51:05.648 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pyperclip. 2021-07-11T03:51:05.681 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-cmd2. 2021-07-11T03:51:05.706 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2021-07-11T03:51:05.715 INFO:teuthology.orchestra.run.gibba019.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138106 files and directories currently installed.) 2021-07-11T03:51:05.718 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2021-07-11T03:51:05.735 INFO:teuthology.orchestra.run.gibba023.stdout:Fetched 153 MB in 17s (8,764 kB/s) 2021-07-11T03:51:05.747 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2021-07-11T03:51:05.763 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2021-07-11T03:51:05.787 INFO:teuthology.orchestra.run.gibba005.stdout:Fetched 153 MB in 17s (8,742 kB/s) 2021-07-11T03:51:05.864 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-pyperclip. 2021-07-11T03:51:05.895 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-pyperclip. 2021-07-11T03:51:05.931 INFO:teuthology.orchestra.run.gibba023.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138106 files and directories currently installed.) 2021-07-11T03:51:05.933 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2021-07-11T03:51:05.962 INFO:teuthology.orchestra.run.gibba005.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 138106 files and directories currently installed.) 2021-07-11T03:51:05.964 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../000-python3-pyperclip_1.7.0-1_all.deb ... 2021-07-11T03:51:05.970 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2021-07-11T03:51:06.001 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-pyperclip (1.7.0-1) ... 2021-07-11T03:51:06.032 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-wcwidth. 2021-07-11T03:51:06.060 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2021-07-11T03:51:06.098 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2021-07-11T03:51:06.135 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-07-11T03:51:06.142 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2021-07-11T03:51:06.225 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2021-07-11T03:51:06.288 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-wcwidth. 2021-07-11T03:51:06.306 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-wcwidth. 2021-07-11T03:51:06.313 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2021-07-11T03:51:06.331 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../001-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2021-07-11T03:51:06.346 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2021-07-11T03:51:06.372 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2021-07-11T03:51:06.377 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-cmd2. 2021-07-11T03:51:06.385 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2021-07-11T03:51:06.426 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2021-07-11T03:51:06.631 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-cmd2. 2021-07-11T03:51:06.655 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2021-07-11T03:51:06.695 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../004-librbd1_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:06.697 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2021-07-11T03:51:06.699 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-cmd2. 2021-07-11T03:51:06.709 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../002-python3-cmd2_0.8.5-2.1_all.deb ... 2021-07-11T03:51:06.750 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-cmd2 (0.8.5-2.1) ... 2021-07-11T03:51:06.754 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-07-11T03:51:06.763 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2021-07-11T03:51:06.820 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking librbd1 (15.2.13-130-g45c6565d-1focal) over (15.2.12-0ubuntu0.20.04.1) ... 2021-07-11T03:51:06.846 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2021-07-11T03:51:07.066 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-07-11T03:51:07.077 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-07-11T03:51:07.077 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2021-07-11T03:51:07.102 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../003-liblttng-ust0_2.11.0-1_amd64.deb ... 2021-07-11T03:51:07.158 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2021-07-11T03:51:07.168 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2021-07-11T03:51:07.182 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../004-librbd1_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:07.267 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../005-librados2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:07.307 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking librbd1 (15.2.13-130-g45c6565d-1focal) over (15.2.12-0ubuntu0.20.04.1) ... 2021-07-11T03:51:07.399 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking librados2 (15.2.13-130-g45c6565d-1focal) over (15.2.12-0ubuntu0.20.04.1) ... 2021-07-11T03:51:07.463 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../004-librbd1_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:07.529 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../004-librbd1_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:07.589 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking librbd1 (15.2.13-130-g45c6565d-1focal) over (15.2.12-0ubuntu0.20.04.1) ... 2021-07-11T03:51:07.679 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking librbd1 (15.2.13-130-g45c6565d-1focal) over (15.2.12-0ubuntu0.20.04.1) ... 2021-07-11T03:51:07.770 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../005-librados2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:07.878 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking librados2 (15.2.13-130-g45c6565d-1focal) over (15.2.12-0ubuntu0.20.04.1) ... 2021-07-11T03:51:07.994 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libcephfs2. 2021-07-11T03:51:08.003 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../006-libcephfs2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:08.044 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libcephfs2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:08.117 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../005-librados2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:08.183 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../005-librados2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:08.242 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking librados2 (15.2.13-130-g45c6565d-1focal) over (15.2.12-0ubuntu0.20.04.1) ... 2021-07-11T03:51:08.282 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking librados2 (15.2.13-130-g45c6565d-1focal) over (15.2.12-0ubuntu0.20.04.1) ... 2021-07-11T03:51:08.341 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-rados. 2021-07-11T03:51:08.367 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../007-python3-rados_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:08.408 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-rados (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:08.457 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libcephfs2. 2021-07-11T03:51:08.482 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../006-libcephfs2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:08.523 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libcephfs2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:08.744 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-07-11T03:51:08.752 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:08.826 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-ceph-argparse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:08.829 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libcephfs2. 2021-07-11T03:51:08.836 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-rados. 2021-07-11T03:51:08.837 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../006-libcephfs2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:08.845 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../007-python3-rados_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:08.878 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libcephfs2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:08.886 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-rados (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:08.904 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libcephfs2. 2021-07-11T03:51:08.929 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../006-libcephfs2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:08.970 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libcephfs2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:09.137 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-cephfs. 2021-07-11T03:51:09.163 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../009-python3-cephfs_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:09.169 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-rados. 2021-07-11T03:51:09.195 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../007-python3-rados_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:09.204 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-cephfs (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:09.236 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-rados (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:09.297 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-rados. 2021-07-11T03:51:09.305 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../007-python3-rados_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:09.346 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-rados (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:09.356 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-07-11T03:51:09.364 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:09.405 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-ceph-argparse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:09.532 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-ceph-common. 2021-07-11T03:51:09.541 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:09.573 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:09.668 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-07-11T03:51:09.693 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:09.725 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-cephfs. 2021-07-11T03:51:09.735 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-ceph-argparse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:09.750 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../009-python3-cephfs_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:09.780 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-07-11T03:51:09.788 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../008-python3-ceph-argparse_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:09.790 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-cephfs (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:09.829 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-ceph-argparse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:09.859 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-prettytable. 2021-07-11T03:51:09.885 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2021-07-11T03:51:09.917 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2021-07-11T03:51:10.069 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-cephfs. 2021-07-11T03:51:10.078 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../009-python3-cephfs_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:10.110 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-ceph-common. 2021-07-11T03:51:10.119 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:10.119 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-cephfs (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:10.123 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-cephfs. 2021-07-11T03:51:10.149 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../009-python3-cephfs_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:10.153 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-rbd. 2021-07-11T03:51:10.178 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../012-python3-rbd_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:10.191 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-cephfs (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:10.193 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:10.237 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-rbd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:10.455 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-prettytable. 2021-07-11T03:51:10.455 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-ceph-common. 2021-07-11T03:51:10.480 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2021-07-11T03:51:10.481 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:10.518 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2021-07-11T03:51:10.522 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:10.544 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-ceph-common. 2021-07-11T03:51:10.551 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../010-python3-ceph-common_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:10.593 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:10.673 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-07-11T03:51:10.682 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2021-07-11T03:51:10.715 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-rbd. 2021-07-11T03:51:10.722 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2021-07-11T03:51:10.741 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../012-python3-rbd_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:10.781 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-rbd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:10.834 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-prettytable. 2021-07-11T03:51:10.842 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2021-07-11T03:51:10.875 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2021-07-11T03:51:10.887 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-prettytable. 2021-07-11T03:51:10.913 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../011-python3-prettytable_0.7.2-5_all.deb ... 2021-07-11T03:51:10.953 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2021-07-11T03:51:11.001 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-07-11T03:51:11.026 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2021-07-11T03:51:11.059 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-07-11T03:51:11.134 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-07-11T03:51:11.142 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2021-07-11T03:51:11.161 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-rbd. 2021-07-11T03:51:11.183 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2021-07-11T03:51:11.186 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../012-python3-rbd_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:11.228 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-rbd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:11.248 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-rbd. 2021-07-11T03:51:11.273 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../012-python3-rbd_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:11.314 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-rbd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:11.361 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package librgw2. 2021-07-11T03:51:11.387 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../015-librgw2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:11.428 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking librgw2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:11.529 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-07-11T03:51:11.554 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2021-07-11T03:51:11.587 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-07-11T03:51:11.628 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-07-11T03:51:11.635 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2021-07-11T03:51:11.638 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-07-11T03:51:11.647 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../013-librabbitmq4_0.10.0-1_amd64.deb ... 2021-07-11T03:51:11.674 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2021-07-11T03:51:11.696 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2021-07-11T03:51:11.856 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package librgw2. 2021-07-11T03:51:11.881 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../015-librgw2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:11.916 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking librgw2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:11.947 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-rgw. 2021-07-11T03:51:11.956 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../016-python3-rgw_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:11.996 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-rgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:12.028 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-07-11T03:51:12.054 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2021-07-11T03:51:12.066 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-07-11T03:51:12.091 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../014-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2021-07-11T03:51:12.094 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-07-11T03:51:12.133 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-07-11T03:51:12.350 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libleveldb1d:amd64. 2021-07-11T03:51:12.367 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-rgw. 2021-07-11T03:51:12.375 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2021-07-11T03:51:12.392 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../016-python3-rgw_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:12.416 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-07-11T03:51:12.433 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-rgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:12.439 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package librgw2. 2021-07-11T03:51:12.447 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../015-librgw2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:12.488 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking librgw2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:12.511 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package librgw2. 2021-07-11T03:51:12.520 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../015-librgw2_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:12.561 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking librgw2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:12.745 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package liboath0. 2021-07-11T03:51:12.752 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2021-07-11T03:51:12.761 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libleveldb1d:amd64. 2021-07-11T03:51:12.770 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2021-07-11T03:51:12.785 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2021-07-11T03:51:12.810 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-07-11T03:51:13.024 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-rgw. 2021-07-11T03:51:13.038 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-rgw. 2021-07-11T03:51:13.049 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../016-python3-rgw_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:13.063 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../016-python3-rgw_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:13.071 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libradosstriper1. 2021-07-11T03:51:13.090 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-rgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:13.096 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../019-libradosstriper1_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:13.097 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-rgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:13.137 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libradosstriper1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:13.147 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package liboath0. 2021-07-11T03:51:13.172 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2021-07-11T03:51:13.205 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2021-07-11T03:51:13.376 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libleveldb1d:amd64. 2021-07-11T03:51:13.384 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2021-07-11T03:51:13.389 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libleveldb1d:amd64. 2021-07-11T03:51:13.413 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../017-libleveldb1d_1.22-3ubuntu2_amd64.deb ... 2021-07-11T03:51:13.417 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-07-11T03:51:13.457 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-07-11T03:51:13.458 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libradosstriper1. 2021-07-11T03:51:13.483 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../019-libradosstriper1_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:13.524 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libradosstriper1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:13.599 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-common. 2021-07-11T03:51:13.607 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../020-ceph-common_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:13.648 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:13.788 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package liboath0. 2021-07-11T03:51:13.794 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package liboath0. 2021-07-11T03:51:13.801 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2021-07-11T03:51:13.813 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../018-liboath0_2.6.1-1.3_amd64.deb ... 2021-07-11T03:51:13.843 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2021-07-11T03:51:13.854 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2021-07-11T03:51:13.952 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-common. 2021-07-11T03:51:13.960 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../020-ceph-common_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:14.001 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:14.173 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libradosstriper1. 2021-07-11T03:51:14.181 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../019-libradosstriper1_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:14.222 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libradosstriper1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:14.252 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libradosstriper1. 2021-07-11T03:51:14.259 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../019-libradosstriper1_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:14.354 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libradosstriper1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:14.643 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-common. 2021-07-11T03:51:14.651 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../020-ceph-common_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:14.876 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:14.877 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-common. 2021-07-11T03:51:14.877 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../020-ceph-common_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:14.877 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:15.446 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-base. 2021-07-11T03:51:15.453 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../021-ceph-base_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:15.490 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-base (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:15.658 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-base. 2021-07-11T03:51:15.681 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../021-ceph-base_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:15.730 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-base (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:16.371 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-cryptography. 2021-07-11T03:51:16.397 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2021-07-11T03:51:16.444 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-cryptography. 2021-07-11T03:51:16.456 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2021-07-11T03:51:16.459 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2021-07-11T03:51:16.464 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-base. 2021-07-11T03:51:16.486 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2021-07-11T03:51:16.489 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../021-ceph-base_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:16.536 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-base (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:16.622 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-base. 2021-07-11T03:51:16.631 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../021-ceph-base_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:16.675 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-base (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:16.853 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-openssl. 2021-07-11T03:51:16.879 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2021-07-11T03:51:16.883 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-openssl. 2021-07-11T03:51:16.908 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2021-07-11T03:51:16.920 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2021-07-11T03:51:16.949 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2021-07-11T03:51:17.201 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-07-11T03:51:17.214 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-07-11T03:51:17.222 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:17.226 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:17.263 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-mgr-modules-core (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:17.267 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-modules-core (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:17.326 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-cryptography. 2021-07-11T03:51:17.335 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2021-07-11T03:51:17.379 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2021-07-11T03:51:17.476 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-cryptography. 2021-07-11T03:51:17.501 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../022-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2021-07-11T03:51:17.542 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2021-07-11T03:51:17.622 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-bcrypt. 2021-07-11T03:51:17.630 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2021-07-11T03:51:17.671 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2021-07-11T03:51:17.730 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-openssl. 2021-07-11T03:51:17.732 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-bcrypt. 2021-07-11T03:51:17.741 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2021-07-11T03:51:17.755 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2021-07-11T03:51:17.813 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2021-07-11T03:51:17.871 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-openssl. 2021-07-11T03:51:17.896 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../023-python3-openssl_19.0.0-1build1_all.deb ... 2021-07-11T03:51:17.937 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2021-07-11T03:51:17.975 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-cherrypy3. 2021-07-11T03:51:18.000 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2021-07-11T03:51:18.041 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2021-07-11T03:51:18.074 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-07-11T03:51:18.102 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:18.140 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-mgr-modules-core (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:18.232 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-07-11T03:51:18.241 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2021-07-11T03:51:18.241 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../024-ceph-mgr-modules-core_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:18.281 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-mgr-modules-core (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:18.396 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-jwt. 2021-07-11T03:51:18.405 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2021-07-11T03:51:18.485 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-bcrypt. 2021-07-11T03:51:18.511 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2021-07-11T03:51:18.519 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2021-07-11T03:51:18.551 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2021-07-11T03:51:18.627 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-bcrypt. 2021-07-11T03:51:18.657 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../025-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2021-07-11T03:51:18.693 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2021-07-11T03:51:18.821 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-cherrypy3. 2021-07-11T03:51:18.846 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2021-07-11T03:51:18.847 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-logutils. 2021-07-11T03:51:18.872 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2021-07-11T03:51:18.887 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2021-07-11T03:51:18.888 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-cherrypy3. 2021-07-11T03:51:18.895 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2021-07-11T03:51:18.913 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-logutils (0.3.3-7) ... 2021-07-11T03:51:18.937 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2021-07-11T03:51:19.039 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-cherrypy3. 2021-07-11T03:51:19.048 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../026-python3-cherrypy3_8.9.1-6_all.deb ... 2021-07-11T03:51:19.088 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2021-07-11T03:51:19.232 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-markupsafe. 2021-07-11T03:51:19.240 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2021-07-11T03:51:19.250 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-jwt. 2021-07-11T03:51:19.256 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-jwt. 2021-07-11T03:51:19.265 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2021-07-11T03:51:19.274 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2021-07-11T03:51:19.282 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2021-07-11T03:51:19.306 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2021-07-11T03:51:19.315 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2021-07-11T03:51:19.433 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-jwt. 2021-07-11T03:51:19.458 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../027-python3-jwt_1.7.1-2ubuntu2_all.deb ... 2021-07-11T03:51:19.499 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2) ... 2021-07-11T03:51:19.615 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-mako. 2021-07-11T03:51:19.625 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2021-07-11T03:51:19.634 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-logutils. 2021-07-11T03:51:19.652 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-logutils. 2021-07-11T03:51:19.666 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2021-07-11T03:51:19.667 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2021-07-11T03:51:19.700 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-logutils (0.3.3-7) ... 2021-07-11T03:51:19.709 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-logutils (0.3.3-7) ... 2021-07-11T03:51:20.045 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-markupsafe. 2021-07-11T03:51:20.051 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2021-07-11T03:51:20.095 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2021-07-11T03:51:20.121 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-07-11T03:51:20.415 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-mako. 2021-07-11T03:51:20.440 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2021-07-11T03:51:20.445 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-logutils. 2021-07-11T03:51:20.452 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../028-python3-logutils_0.3.3-7_all.deb ... 2021-07-11T03:51:20.494 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-logutils (0.3.3-7) ... 2021-07-11T03:51:20.514 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-07-11T03:51:20.579 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-markupsafe. 2021-07-11T03:51:20.605 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2021-07-11T03:51:20.645 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2021-07-11T03:51:20.697 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-simplegeneric. 2021-07-11T03:51:20.722 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2021-07-11T03:51:20.763 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2021-07-11T03:51:20.830 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-markupsafe. 2021-07-11T03:51:20.834 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-simplegeneric. 2021-07-11T03:51:20.844 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2021-07-11T03:51:20.855 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../029-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2021-07-11T03:51:20.887 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2021-07-11T03:51:20.903 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2021-07-11T03:51:20.957 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-mako. 2021-07-11T03:51:20.966 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2021-07-11T03:51:21.006 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-07-11T03:51:21.066 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-singledispatch. 2021-07-11T03:51:21.074 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2021-07-11T03:51:21.118 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2021-07-11T03:51:21.208 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-mako. 2021-07-11T03:51:21.212 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-singledispatch. 2021-07-11T03:51:21.219 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../030-python3-mako_1.1.0+ds1-1ubuntu2_all.deb ... 2021-07-11T03:51:21.239 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2021-07-11T03:51:21.258 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-07-11T03:51:21.278 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2021-07-11T03:51:21.318 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-simplegeneric. 2021-07-11T03:51:21.326 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2021-07-11T03:51:21.359 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2021-07-11T03:51:21.427 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-webob. 2021-07-11T03:51:21.455 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2021-07-11T03:51:21.493 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2021-07-11T03:51:21.595 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-simplegeneric. 2021-07-11T03:51:21.596 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-singledispatch. 2021-07-11T03:51:21.620 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../031-python3-simplegeneric_0.8.1-3_all.deb ... 2021-07-11T03:51:21.620 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2021-07-11T03:51:21.622 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-webob. 2021-07-11T03:51:21.630 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2021-07-11T03:51:21.661 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2021-07-11T03:51:21.662 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2021-07-11T03:51:21.680 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2021-07-11T03:51:21.829 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-soupsieve. 2021-07-11T03:51:21.838 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2021-07-11T03:51:21.879 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2021-07-11T03:51:21.982 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-webob. 2021-07-11T03:51:21.989 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-singledispatch. 2021-07-11T03:51:21.989 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2021-07-11T03:51:21.997 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../032-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2021-07-11T03:51:22.034 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2021-07-11T03:51:22.038 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2021-07-11T03:51:22.050 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-soupsieve. 2021-07-11T03:51:22.076 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2021-07-11T03:51:22.117 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2021-07-11T03:51:22.173 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-bs4. 2021-07-11T03:51:22.199 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2021-07-11T03:51:22.242 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2021-07-11T03:51:22.368 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-soupsieve. 2021-07-11T03:51:22.380 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-webob. 2021-07-11T03:51:22.392 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2021-07-11T03:51:22.401 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../033-python3-webob_1%3a1.8.5-2_all.deb ... 2021-07-11T03:51:22.434 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2021-07-11T03:51:22.450 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2021-07-11T03:51:22.485 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-bs4. 2021-07-11T03:51:22.486 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2021-07-11T03:51:22.528 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2021-07-11T03:51:22.592 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-waitress. 2021-07-11T03:51:22.601 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1_all.deb ... 2021-07-11T03:51:22.644 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-waitress (1.4.1-1) ... 2021-07-11T03:51:22.770 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-bs4. 2021-07-11T03:51:22.778 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2021-07-11T03:51:22.828 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-soupsieve. 2021-07-11T03:51:22.835 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../034-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2021-07-11T03:51:22.844 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2021-07-11T03:51:22.877 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2021-07-11T03:51:22.923 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-waitress. 2021-07-11T03:51:22.948 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1_all.deb ... 2021-07-11T03:51:22.956 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-tempita. 2021-07-11T03:51:22.979 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2021-07-11T03:51:22.993 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-waitress (1.4.1-1) ... 2021-07-11T03:51:23.049 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-tempita (0.5.2-6) ... 2021-07-11T03:51:23.256 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-waitress. 2021-07-11T03:51:23.264 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-bs4. 2021-07-11T03:51:23.284 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1_all.deb ... 2021-07-11T03:51:23.288 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../035-python3-bs4_4.8.2-1_all.deb ... 2021-07-11T03:51:23.326 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-waitress (1.4.1-1) ... 2021-07-11T03:51:23.334 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2021-07-11T03:51:23.350 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-tempita. 2021-07-11T03:51:23.356 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-paste. 2021-07-11T03:51:23.359 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2021-07-11T03:51:23.365 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2021-07-11T03:51:23.400 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-tempita (0.5.2-6) ... 2021-07-11T03:51:23.406 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2021-07-11T03:51:23.676 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-tempita. 2021-07-11T03:51:23.684 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-07-11T03:51:23.684 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2021-07-11T03:51:23.709 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-waitress. 2021-07-11T03:51:23.710 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2021-07-11T03:51:23.716 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../036-python3-waitress_1.4.1-1_all.deb ... 2021-07-11T03:51:23.725 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-tempita (0.5.2-6) ... 2021-07-11T03:51:23.740 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-paste. 2021-07-11T03:51:23.750 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2021-07-11T03:51:23.751 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-waitress (1.4.1-1) ... 2021-07-11T03:51:23.762 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2021-07-11T03:51:23.811 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2021-07-11T03:51:23.986 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-paste. 2021-07-11T03:51:24.011 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2021-07-11T03:51:24.044 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2021-07-11T03:51:24.063 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-tempita. 2021-07-11T03:51:24.079 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pastedeploy. 2021-07-11T03:51:24.087 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../037-python3-tempita_0.5.2-6_all.deb ... 2021-07-11T03:51:24.087 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2021-07-11T03:51:24.106 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-07-11T03:51:24.130 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-tempita (0.5.2-6) ... 2021-07-11T03:51:24.130 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2021-07-11T03:51:24.132 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2021-07-11T03:51:24.206 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2021-07-11T03:51:24.347 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-07-11T03:51:24.372 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2021-07-11T03:51:24.413 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2021-07-11T03:51:24.439 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-paste. 2021-07-11T03:51:24.448 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../038-python3-paste_3.4.0+dfsg1-1_all.deb ... 2021-07-11T03:51:24.465 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-webtest. 2021-07-11T03:51:24.488 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2021-07-11T03:51:24.489 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2021-07-11T03:51:24.517 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-pastedeploy. 2021-07-11T03:51:24.530 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2021-07-11T03:51:24.531 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-webtest (2.0.32-2) ... 2021-07-11T03:51:24.566 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2021-07-11T03:51:24.750 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-pastedeploy. 2021-07-11T03:51:24.758 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2021-07-11T03:51:24.799 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2021-07-11T03:51:24.800 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pecan. 2021-07-11T03:51:24.815 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2021-07-11T03:51:24.818 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-07-11T03:51:24.841 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2021-07-11T03:51:24.843 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../039-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2021-07-11T03:51:24.884 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2021-07-11T03:51:24.911 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-webtest. 2021-07-11T03:51:24.936 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2021-07-11T03:51:24.977 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-webtest (2.0.32-2) ... 2021-07-11T03:51:25.111 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-webtest. 2021-07-11T03:51:25.136 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2021-07-11T03:51:25.138 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-werkzeug. 2021-07-11T03:51:25.164 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2021-07-11T03:51:25.177 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-webtest (2.0.32-2) ... 2021-07-11T03:51:25.204 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2021-07-11T03:51:25.212 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-pastedeploy. 2021-07-11T03:51:25.219 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../040-python3-pastedeploy_2.1.0-1_all.deb ... 2021-07-11T03:51:25.261 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2021-07-11T03:51:25.322 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-pecan. 2021-07-11T03:51:25.330 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2021-07-11T03:51:25.379 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2021-07-11T03:51:25.513 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-pecan. 2021-07-11T03:51:25.520 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2021-07-11T03:51:25.554 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2021-07-11T03:51:25.555 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr. 2021-07-11T03:51:25.557 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../044-ceph-mgr_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:25.598 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:25.606 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-webtest. 2021-07-11T03:51:25.631 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../041-python3-webtest_2.0.32-2_all.deb ... 2021-07-11T03:51:25.672 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-webtest (2.0.32-2) ... 2021-07-11T03:51:25.743 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-werkzeug. 2021-07-11T03:51:25.771 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2021-07-11T03:51:25.809 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2021-07-11T03:51:25.842 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-werkzeug. 2021-07-11T03:51:25.868 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2021-07-11T03:51:25.909 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2021-07-11T03:51:26.000 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-pecan. 2021-07-11T03:51:26.008 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../042-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2021-07-11T03:51:26.035 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mon. 2021-07-11T03:51:26.049 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2021-07-11T03:51:26.060 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../045-ceph-mon_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:26.101 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mon (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:26.221 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-mgr. 2021-07-11T03:51:26.228 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../044-ceph-mgr_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:26.270 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-mgr (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:26.287 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-mgr. 2021-07-11T03:51:26.296 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../044-ceph-mgr_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:26.336 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-mgr (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:26.430 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-werkzeug. 2021-07-11T03:51:26.455 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../043-python3-werkzeug_0.16.1+dfsg1-2_all.deb ... 2021-07-11T03:51:26.496 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2) ... 2021-07-11T03:51:26.731 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-mon. 2021-07-11T03:51:26.743 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../045-ceph-mon_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:26.747 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-mon. 2021-07-11T03:51:26.773 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../045-ceph-mon_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:26.780 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-mon (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:26.813 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-mon (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:26.883 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-mgr. 2021-07-11T03:51:26.890 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../044-ceph-mgr_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:26.955 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-mgr (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:26.972 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-osd. 2021-07-11T03:51:26.997 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../046-ceph-osd_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:27.044 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-osd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:27.411 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-mon. 2021-07-11T03:51:27.419 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../045-ceph-mon_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:27.460 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-mon (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:27.679 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-osd. 2021-07-11T03:51:27.698 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-osd. 2021-07-11T03:51:27.701 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../046-ceph-osd_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:27.702 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../046-ceph-osd_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:27.742 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-osd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:27.743 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-osd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:28.377 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-osd. 2021-07-11T03:51:28.379 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../046-ceph-osd_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:28.421 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-osd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:28.960 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph. 2021-07-11T03:51:28.989 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../047-ceph_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:29.026 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:29.337 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-fuse. 2021-07-11T03:51:29.362 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../048-ceph-fuse_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:29.403 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:29.689 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph. 2021-07-11T03:51:29.697 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../047-ceph_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:29.739 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:29.748 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph. 2021-07-11T03:51:29.756 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../047-ceph_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:29.790 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mds. 2021-07-11T03:51:29.797 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:29.799 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../049-ceph-mds_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:29.840 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mds (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:30.007 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-fuse. 2021-07-11T03:51:30.015 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../048-ceph-fuse_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:30.048 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:30.151 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-fuse. 2021-07-11T03:51:30.177 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../048-ceph-fuse_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:30.217 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:30.310 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package cephadm. 2021-07-11T03:51:30.335 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../050-cephadm_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:30.343 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph. 2021-07-11T03:51:30.352 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../047-ceph_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:30.369 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-mds. 2021-07-11T03:51:30.377 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../049-ceph-mds_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:30.384 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:30.392 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:30.418 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-mds (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:30.570 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-mds. 2021-07-11T03:51:30.579 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../049-ceph-mds_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:30.619 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-mds (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:30.695 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-jinja2. 2021-07-11T03:51:30.703 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2021-07-11T03:51:30.744 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2021-07-11T03:51:30.746 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-fuse. 2021-07-11T03:51:30.771 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../048-ceph-fuse_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:30.846 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:30.863 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package cephadm. 2021-07-11T03:51:30.888 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../050-cephadm_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:30.929 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:31.065 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-07-11T03:51:31.081 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package cephadm. 2021-07-11T03:51:31.089 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../050-cephadm_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:31.091 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:31.130 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:31.131 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:31.248 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-jinja2. 2021-07-11T03:51:31.256 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2021-07-11T03:51:31.266 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-mds. 2021-07-11T03:51:31.273 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../049-ceph-mds_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:31.297 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2021-07-11T03:51:31.315 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-mds (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:31.459 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-repoze.lru. 2021-07-11T03:51:31.483 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-jinja2. 2021-07-11T03:51:31.485 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2021-07-11T03:51:31.508 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2021-07-11T03:51:31.526 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2021-07-11T03:51:31.549 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2021-07-11T03:51:31.617 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-07-11T03:51:31.643 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:31.683 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-mgr-cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:31.810 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package cephadm. 2021-07-11T03:51:31.812 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libjs-underscore. 2021-07-11T03:51:31.818 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../050-cephadm_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:31.837 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2021-07-11T03:51:31.859 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:31.878 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-07-11T03:51:31.903 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-07-11T03:51:31.911 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:31.952 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-mgr-cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:32.012 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-repoze.lru. 2021-07-11T03:51:32.021 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2021-07-11T03:51:32.061 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2021-07-11T03:51:32.164 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libjs-sphinxdoc. 2021-07-11T03:51:32.180 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-jinja2. 2021-07-11T03:51:32.189 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2021-07-11T03:51:32.205 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../051-python3-jinja2_2.10.1-2_all.deb ... 2021-07-11T03:51:32.230 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-07-11T03:51:32.239 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-repoze.lru. 2021-07-11T03:51:32.246 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2021-07-11T03:51:32.264 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2021-07-11T03:51:32.306 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libjs-underscore. 2021-07-11T03:51:32.321 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2021-07-11T03:51:32.331 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2021-07-11T03:51:32.372 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-07-11T03:51:32.532 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libjs-underscore. 2021-07-11T03:51:32.558 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2021-07-11T03:51:32.567 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-routes. 2021-07-11T03:51:32.590 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-07-11T03:51:32.591 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-07-11T03:51:32.592 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2021-07-11T03:51:32.598 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../052-ceph-mgr-cephadm_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:32.632 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-mgr-cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:32.633 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-routes (2.4.1-2) ... 2021-07-11T03:51:32.649 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libjs-sphinxdoc. 2021-07-11T03:51:32.679 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2021-07-11T03:51:32.716 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-07-11T03:51:32.859 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libjs-sphinxdoc. 2021-07-11T03:51:32.885 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2021-07-11T03:51:32.925 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-07-11T03:51:32.952 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-repoze.lru. 2021-07-11T03:51:32.961 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-07-11T03:51:32.977 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../053-python3-repoze.lru_0.7-2_all.deb ... 2021-07-11T03:51:32.986 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:33.018 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2021-07-11T03:51:33.027 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-dashboard (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:33.093 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-routes. 2021-07-11T03:51:33.101 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2021-07-11T03:51:33.142 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-routes (2.4.1-2) ... 2021-07-11T03:51:33.246 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-routes. 2021-07-11T03:51:33.255 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2021-07-11T03:51:33.296 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-routes (2.4.1-2) ... 2021-07-11T03:51:33.304 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libjs-underscore. 2021-07-11T03:51:33.329 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../054-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2021-07-11T03:51:33.370 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-07-11T03:51:33.465 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-07-11T03:51:33.489 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:33.529 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-mgr-dashboard (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:33.623 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-07-11T03:51:33.649 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:33.690 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libjs-sphinxdoc. 2021-07-11T03:51:33.697 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../055-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2021-07-11T03:51:33.698 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-mgr-dashboard (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:33.740 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-07-11T03:51:33.776 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-07-11T03:51:33.783 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:33.826 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:34.076 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-routes. 2021-07-11T03:51:34.101 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../056-python3-routes_2.4.1-2_all.deb ... 2021-07-11T03:51:34.142 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-routes (2.4.1-2) ... 2021-07-11T03:51:34.145 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-decorator. 2021-07-11T03:51:34.171 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2021-07-11T03:51:34.220 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2021-07-11T03:51:34.260 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-07-11T03:51:34.287 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:34.337 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:34.495 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-07-11T03:51:34.496 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-07-11T03:51:34.504 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:34.506 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:34.544 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-mgr-dashboard (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:34.554 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:34.573 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2021-07-11T03:51:34.580 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2021-07-11T03:51:34.621 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-07-11T03:51:34.670 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-decorator. 2021-07-11T03:51:34.679 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2021-07-11T03:51:34.720 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2021-07-11T03:51:34.907 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-decorator. 2021-07-11T03:51:34.924 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-scipy. 2021-07-11T03:51:34.940 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2021-07-11T03:51:34.950 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2021-07-11T03:51:34.982 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2021-07-11T03:51:34.991 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2021-07-11T03:51:35.024 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2021-07-11T03:51:35.049 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2021-07-11T03:51:35.089 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-07-11T03:51:35.335 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2021-07-11T03:51:35.343 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2021-07-11T03:51:35.384 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-07-11T03:51:35.394 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-mgr-diskprediction-cloud. 2021-07-11T03:51:35.401 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../058-ceph-mgr-diskprediction-cloud_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:35.427 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-scipy. 2021-07-11T03:51:35.433 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2021-07-11T03:51:35.468 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-mgr-diskprediction-cloud (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:35.475 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2021-07-11T03:51:35.737 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-scipy. 2021-07-11T03:51:35.763 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2021-07-11T03:51:35.796 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-decorator. 2021-07-11T03:51:35.803 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2021-07-11T03:51:35.824 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../059-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2021-07-11T03:51:35.892 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-07-11T03:51:35.893 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2021-07-11T03:51:35.918 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2021-07-11T03:51:35.965 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-07-11T03:51:36.182 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2021-07-11T03:51:36.190 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../060-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2021-07-11T03:51:36.256 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-07-11T03:51:36.403 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-joblib. 2021-07-11T03:51:36.412 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2021-07-11T03:51:36.419 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-07-11T03:51:36.426 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2021-07-11T03:51:36.445 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-joblib (0.14.0-3) ... 2021-07-11T03:51:36.466 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-07-11T03:51:36.568 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-scipy. 2021-07-11T03:51:36.594 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../061-python3-scipy_1.3.3-3build1_amd64.deb ... 2021-07-11T03:51:36.660 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2021-07-11T03:51:36.728 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-07-11T03:51:36.755 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2021-07-11T03:51:36.756 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-sklearn. 2021-07-11T03:51:36.783 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2021-07-11T03:51:36.794 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-07-11T03:51:36.822 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-07-11T03:51:36.922 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-joblib. 2021-07-11T03:51:36.946 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2021-07-11T03:51:37.019 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-joblib (0.14.0-3) ... 2021-07-11T03:51:37.282 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-joblib. 2021-07-11T03:51:37.291 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2021-07-11T03:51:37.356 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-sklearn. 2021-07-11T03:51:37.364 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2021-07-11T03:51:37.372 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-07-11T03:51:37.373 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-joblib (0.14.0-3) ... 2021-07-11T03:51:37.380 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:37.396 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-07-11T03:51:37.421 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:37.594 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-07-11T03:51:37.602 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../062-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2021-07-11T03:51:37.643 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-07-11T03:51:37.718 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-sklearn. 2021-07-11T03:51:37.745 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2021-07-11T03:51:37.788 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-07-11T03:51:37.954 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-07-11T03:51:37.964 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:38.012 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:38.171 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-joblib. 2021-07-11T03:51:38.197 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../063-python3-joblib_0.14.0-3_all.deb ... 2021-07-11T03:51:38.238 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-joblib (0.14.0-3) ... 2021-07-11T03:51:38.316 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-cachetools. 2021-07-11T03:51:38.324 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2021-07-11T03:51:38.366 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2021-07-11T03:51:38.367 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-07-11T03:51:38.375 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:38.417 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:38.558 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-sklearn. 2021-07-11T03:51:38.584 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../064-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2021-07-11T03:51:38.624 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-07-11T03:51:38.677 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pyasn1. 2021-07-11T03:51:38.703 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2021-07-11T03:51:38.746 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2021-07-11T03:51:38.958 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-cachetools. 2021-07-11T03:51:38.984 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2021-07-11T03:51:39.024 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2021-07-11T03:51:39.090 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-07-11T03:51:39.098 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2021-07-11T03:51:39.139 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-07-11T03:51:39.224 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-07-11T03:51:39.251 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../065-ceph-mgr-diskprediction-local_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:39.290 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-mgr-diskprediction-local (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:39.318 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-pyasn1. 2021-07-11T03:51:39.345 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2021-07-11T03:51:39.385 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2021-07-11T03:51:39.386 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-cachetools. 2021-07-11T03:51:39.396 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2021-07-11T03:51:39.442 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-rsa. 2021-07-11T03:51:39.444 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2021-07-11T03:51:39.468 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2021-07-11T03:51:39.508 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-rsa (4.0-3) ... 2021-07-11T03:51:39.755 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-07-11T03:51:39.764 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2021-07-11T03:51:39.804 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-07-11T03:51:39.828 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-google-auth. 2021-07-11T03:51:39.836 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2021-07-11T03:51:39.839 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-pyasn1. 2021-07-11T03:51:39.865 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2021-07-11T03:51:39.877 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2021-07-11T03:51:39.905 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2021-07-11T03:51:40.116 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-rsa. 2021-07-11T03:51:40.144 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2021-07-11T03:51:40.182 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-rsa (4.0-3) ... 2021-07-11T03:51:40.198 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-blinker. 2021-07-11T03:51:40.225 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2021-07-11T03:51:40.252 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-07-11T03:51:40.260 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-cachetools. 2021-07-11T03:51:40.260 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2021-07-11T03:51:40.264 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-07-11T03:51:40.267 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../066-python3-cachetools_4.0.0-1_all.deb ... 2021-07-11T03:51:40.301 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-07-11T03:51:40.309 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2021-07-11T03:51:40.486 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-google-auth. 2021-07-11T03:51:40.494 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2021-07-11T03:51:40.527 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2021-07-11T03:51:40.608 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-oauthlib. 2021-07-11T03:51:40.617 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2021-07-11T03:51:40.646 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-rsa. 2021-07-11T03:51:40.647 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-pyasn1. 2021-07-11T03:51:40.672 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2021-07-11T03:51:40.673 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../067-python3-pyasn1_0.4.2-3build1_all.deb ... 2021-07-11T03:51:40.675 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2021-07-11T03:51:40.712 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-rsa (4.0-3) ... 2021-07-11T03:51:40.856 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2021-07-11T03:51:41.102 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-google-auth. 2021-07-11T03:51:41.102 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2021-07-11T03:51:41.103 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-requests-oauthlib. 2021-07-11T03:51:41.103 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2021-07-11T03:51:41.103 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-07-11T03:51:41.106 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2021-07-11T03:51:41.205 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-blinker. 2021-07-11T03:51:41.214 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2021-07-11T03:51:41.283 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-07-11T03:51:41.332 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-websocket. 2021-07-11T03:51:41.358 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2021-07-11T03:51:41.398 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2021-07-11T03:51:41.435 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-blinker. 2021-07-11T03:51:41.459 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-07-11T03:51:41.462 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2021-07-11T03:51:41.484 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../068-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2021-07-11T03:51:41.510 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-07-11T03:51:41.525 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-07-11T03:51:41.591 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-oauthlib. 2021-07-11T03:51:41.617 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2021-07-11T03:51:41.657 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2021-07-11T03:51:41.684 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-kubernetes. 2021-07-11T03:51:41.692 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2021-07-11T03:51:41.733 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2021-07-11T03:51:41.861 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-rsa. 2021-07-11T03:51:41.879 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-oauthlib. 2021-07-11T03:51:41.887 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../069-python3-rsa_4.0-3_all.deb ... 2021-07-11T03:51:41.888 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2021-07-11T03:51:41.928 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-rsa (4.0-3) ... 2021-07-11T03:51:41.929 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2021-07-11T03:51:41.978 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-requests-oauthlib. 2021-07-11T03:51:41.986 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2021-07-11T03:51:42.027 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-07-11T03:51:42.155 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-07-11T03:51:42.182 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:42.221 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-mgr-k8sevents (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:42.273 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-google-auth. 2021-07-11T03:51:42.280 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../070-python3-google-auth_1.5.1-2_all.deb ... 2021-07-11T03:51:42.299 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-requests-oauthlib. 2021-07-11T03:51:42.324 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2021-07-11T03:51:42.325 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2021-07-11T03:51:42.372 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-websocket. 2021-07-11T03:51:42.373 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-07-11T03:51:42.398 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2021-07-11T03:51:42.446 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2021-07-11T03:51:42.560 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libonig5:amd64. 2021-07-11T03:51:42.567 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2021-07-11T03:51:42.607 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2021-07-11T03:51:42.659 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-blinker. 2021-07-11T03:51:42.685 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../071-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2021-07-11T03:51:42.725 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-07-11T03:51:42.739 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-websocket. 2021-07-11T03:51:42.743 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2021-07-11T03:51:42.757 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-kubernetes. 2021-07-11T03:51:42.765 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2021-07-11T03:51:42.784 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2021-07-11T03:51:42.807 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2021-07-11T03:51:42.960 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libjq1:amd64. 2021-07-11T03:51:42.987 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-07-11T03:51:43.043 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:51:43.078 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-oauthlib. 2021-07-11T03:51:43.104 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../072-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2021-07-11T03:51:43.121 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-kubernetes. 2021-07-11T03:51:43.145 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2021-07-11T03:51:43.147 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2021-07-11T03:51:43.187 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2021-07-11T03:51:43.229 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-07-11T03:51:43.255 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:43.295 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-mgr-k8sevents (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:43.296 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package jq. 2021-07-11T03:51:43.322 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-07-11T03:51:43.362 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:51:43.490 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-requests-oauthlib. 2021-07-11T03:51:43.517 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../073-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2021-07-11T03:51:43.565 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-07-11T03:51:43.631 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libonig5:amd64. 2021-07-11T03:51:43.631 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package socat. 2021-07-11T03:51:43.639 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2021-07-11T03:51:43.640 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2021-07-11T03:51:43.641 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-07-11T03:51:43.668 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:43.680 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2021-07-11T03:51:43.681 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking socat (1.7.3.3-2) ... 2021-07-11T03:51:43.708 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-mgr-k8sevents (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:43.885 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-websocket. 2021-07-11T03:51:43.910 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../074-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2021-07-11T03:51:43.992 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2021-07-11T03:51:44.033 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libjq1:amd64. 2021-07-11T03:51:44.060 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-07-11T03:51:44.065 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libonig5:amd64. 2021-07-11T03:51:44.069 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2021-07-11T03:51:44.099 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:51:44.102 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2021-07-11T03:51:44.136 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-07-11T03:51:44.163 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2021-07-11T03:51:44.211 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2021-07-11T03:51:44.312 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-kubernetes. 2021-07-11T03:51:44.338 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../075-python3-kubernetes_7.0.0~a1-2_all.deb ... 2021-07-11T03:51:44.377 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package jq. 2021-07-11T03:51:44.386 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-07-11T03:51:44.395 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2021-07-11T03:51:44.427 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:51:44.455 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libjq1:amd64. 2021-07-11T03:51:44.482 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-07-11T03:51:44.505 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package xmlstarlet. 2021-07-11T03:51:44.514 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2021-07-11T03:51:44.521 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:51:44.555 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2021-07-11T03:51:44.688 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package socat. 2021-07-11T03:51:44.715 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2021-07-11T03:51:44.754 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking socat (1.7.3.3-2) ... 2021-07-11T03:51:44.791 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package jq. 2021-07-11T03:51:44.817 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-07-11T03:51:44.868 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package ceph-test. 2021-07-11T03:51:44.882 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:51:44.895 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../083-ceph-test_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:44.917 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-07-11T03:51:44.943 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking ceph-test (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:44.943 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:44.983 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-mgr-k8sevents (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:45.151 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package socat. 2021-07-11T03:51:45.167 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-07-11T03:51:45.178 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2021-07-11T03:51:45.194 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2021-07-11T03:51:45.217 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking socat (1.7.3.3-2) ... 2021-07-11T03:51:45.233 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2021-07-11T03:51:45.336 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libonig5:amd64. 2021-07-11T03:51:45.344 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../077-libonig5_6.9.4-1_amd64.deb ... 2021-07-11T03:51:45.385 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2021-07-11T03:51:45.536 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package xmlstarlet. 2021-07-11T03:51:45.543 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2021-07-11T03:51:45.586 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2021-07-11T03:51:45.648 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-07-11T03:51:45.675 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2021-07-11T03:51:45.714 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2021-07-11T03:51:45.755 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libjq1:amd64. 2021-07-11T03:51:45.781 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../078-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-07-11T03:51:45.821 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:51:45.873 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package ceph-test. 2021-07-11T03:51:45.900 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../083-ceph-test_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:45.944 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking ceph-test (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:46.017 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package xmlstarlet. 2021-07-11T03:51:46.044 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2021-07-11T03:51:46.083 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2021-07-11T03:51:46.150 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package jq. 2021-07-11T03:51:46.176 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../079-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2021-07-11T03:51:46.216 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:51:46.429 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package ceph-test. 2021-07-11T03:51:46.437 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../083-ceph-test_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:46.479 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking ceph-test (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:46.503 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package socat. 2021-07-11T03:51:46.529 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../080-socat_1.7.3.3-2_amd64.deb ... 2021-07-11T03:51:46.568 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking socat (1.7.3.3-2) ... 2021-07-11T03:51:47.098 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-07-11T03:51:47.104 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../081-libxslt1.1_1.1.34-4_amd64.deb ... 2021-07-11T03:51:47.148 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libxslt1.1:amd64 (1.1.34-4) ... 2021-07-11T03:51:47.451 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package xmlstarlet. 2021-07-11T03:51:47.478 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../082-xmlstarlet_1.6.1-2build1_amd64.deb ... 2021-07-11T03:51:47.518 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2021-07-11T03:51:47.596 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-colorama. 2021-07-11T03:51:47.623 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2021-07-11T03:51:47.671 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2021-07-11T03:51:47.855 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package ceph-test. 2021-07-11T03:51:47.881 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../083-ceph-test_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:47.921 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking ceph-test (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:47.923 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package cephfs-shell. 2021-07-11T03:51:47.950 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../085-cephfs-shell_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:47.998 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking cephfs-shell (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:48.260 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libcephfs-dev. 2021-07-11T03:51:48.268 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:48.309 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libcephfs-dev (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:48.618 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-colorama. 2021-07-11T03:51:48.645 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2021-07-11T03:51:48.685 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2021-07-11T03:51:48.704 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package nvme-cli. 2021-07-11T03:51:48.730 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2021-07-11T03:51:48.801 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2021-07-11T03:51:48.963 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package cephfs-shell. 2021-07-11T03:51:48.989 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../085-cephfs-shell_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:49.029 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking cephfs-shell (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:49.116 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-atomicwrites. 2021-07-11T03:51:49.124 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2021-07-11T03:51:49.166 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2021-07-11T03:51:49.265 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libcephfs-dev. 2021-07-11T03:51:49.291 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:49.322 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libcephfs-dev (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:49.470 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-attr. 2021-07-11T03:51:49.496 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2021-07-11T03:51:49.536 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-attr (19.3.0-2) ... 2021-07-11T03:51:49.541 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-colorama. 2021-07-11T03:51:49.549 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2021-07-11T03:51:49.591 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2021-07-11T03:51:49.634 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package nvme-cli. 2021-07-11T03:51:49.641 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2021-07-11T03:51:49.681 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2021-07-11T03:51:49.852 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package cephfs-shell. 2021-07-11T03:51:49.872 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-webencodings. 2021-07-11T03:51:49.880 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../085-cephfs-shell_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:49.881 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2021-07-11T03:51:49.921 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2021-07-11T03:51:49.927 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking cephfs-shell (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:49.980 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-atomicwrites. 2021-07-11T03:51:49.989 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2021-07-11T03:51:50.021 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2021-07-11T03:51:50.212 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libcephfs-dev. 2021-07-11T03:51:50.221 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:50.259 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-html5lib. 2021-07-11T03:51:50.270 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libcephfs-dev (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:50.274 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-attr. 2021-07-11T03:51:50.284 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2021-07-11T03:51:50.300 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2021-07-11T03:51:50.325 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2021-07-11T03:51:50.340 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-attr (19.3.0-2) ... 2021-07-11T03:51:50.601 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-colorama. 2021-07-11T03:51:50.627 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-webencodings. 2021-07-11T03:51:50.628 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../084-python3-colorama_0.4.3-1build1_all.deb ... 2021-07-11T03:51:50.653 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2021-07-11T03:51:50.667 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-colorama (0.4.3-1build1) ... 2021-07-11T03:51:50.669 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-07-11T03:51:50.677 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.3_amd64.deb ... 2021-07-11T03:51:50.692 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2021-07-11T03:51:50.716 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package nvme-cli. 2021-07-11T03:51:50.718 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-07-11T03:51:50.725 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2021-07-11T03:51:50.772 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2021-07-11T03:51:50.953 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package cephfs-shell. 2021-07-11T03:51:50.979 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../085-cephfs-shell_15.2.13-130-g45c6565d-1focal_all.deb ... 2021-07-11T03:51:50.995 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-html5lib. 2021-07-11T03:51:51.004 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2021-07-11T03:51:51.019 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking cephfs-shell (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:51.044 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2021-07-11T03:51:51.153 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-atomicwrites. 2021-07-11T03:51:51.182 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2021-07-11T03:51:51.219 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2021-07-11T03:51:51.314 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libcephfs-dev. 2021-07-11T03:51:51.340 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../086-libcephfs-dev_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:51.380 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libcephfs-dev (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:51.398 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-07-11T03:51:51.424 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.3_amd64.deb ... 2021-07-11T03:51:51.467 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-07-11T03:51:51.547 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-attr. 2021-07-11T03:51:51.556 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2021-07-11T03:51:51.597 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-packaging. 2021-07-11T03:51:51.598 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-attr (19.3.0-2) ... 2021-07-11T03:51:51.623 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2021-07-11T03:51:51.663 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-packaging (20.3-1) ... 2021-07-11T03:51:51.808 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package nvme-cli. 2021-07-11T03:51:51.816 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../087-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2021-07-11T03:51:51.861 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-packaging. 2021-07-11T03:51:51.864 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2021-07-11T03:51:51.868 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2021-07-11T03:51:51.908 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-packaging (20.3-1) ... 2021-07-11T03:51:51.966 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pastescript. 2021-07-11T03:51:51.975 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2021-07-11T03:51:51.976 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-webencodings. 2021-07-11T03:51:52.002 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2021-07-11T03:51:52.015 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2021-07-11T03:51:52.050 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2021-07-11T03:51:52.197 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-atomicwrites. 2021-07-11T03:51:52.223 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../088-python3-atomicwrites_1.1.5-2build1_all.deb ... 2021-07-11T03:51:52.255 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-pastescript. 2021-07-11T03:51:52.263 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2021-07-11T03:51:52.279 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2021-07-11T03:51:52.319 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2021-07-11T03:51:52.327 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pluggy. 2021-07-11T03:51:52.354 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2021-07-11T03:51:52.403 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2021-07-11T03:51:52.420 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-html5lib. 2021-07-11T03:51:52.428 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2021-07-11T03:51:52.468 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2021-07-11T03:51:52.591 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-attr. 2021-07-11T03:51:52.617 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../089-python3-attr_19.3.0-2_all.deb ... 2021-07-11T03:51:52.647 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-pluggy. 2021-07-11T03:51:52.655 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2021-07-11T03:51:52.657 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-attr (19.3.0-2) ... 2021-07-11T03:51:52.696 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2021-07-11T03:51:52.697 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-psutil. 2021-07-11T03:51:52.706 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2021-07-11T03:51:52.755 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2021-07-11T03:51:52.830 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-07-11T03:51:52.857 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.3_amd64.deb ... 2021-07-11T03:51:52.896 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-07-11T03:51:53.011 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-webencodings. 2021-07-11T03:51:53.012 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-psutil. 2021-07-11T03:51:53.017 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../090-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2021-07-11T03:51:53.035 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2021-07-11T03:51:53.060 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2021-07-11T03:51:53.075 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2021-07-11T03:51:53.124 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-py. 2021-07-11T03:51:53.151 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../097-python3-py_1.8.1-1_all.deb ... 2021-07-11T03:51:53.199 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-py (1.8.1-1) ... 2021-07-11T03:51:53.346 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-html5lib. 2021-07-11T03:51:53.350 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-packaging. 2021-07-11T03:51:53.360 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2021-07-11T03:51:53.372 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../091-python3-html5lib_1.0.1-2_all.deb ... 2021-07-11T03:51:53.399 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-packaging (20.3-1) ... 2021-07-11T03:51:53.411 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-py. 2021-07-11T03:51:53.412 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2021-07-11T03:51:53.420 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../097-python3-py_1.8.1-1_all.deb ... 2021-07-11T03:51:53.460 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-py (1.8.1-1) ... 2021-07-11T03:51:53.527 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pyinotify. 2021-07-11T03:51:53.536 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2021-07-11T03:51:53.585 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-07-11T03:51:53.707 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-07-11T03:51:53.711 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-pastescript. 2021-07-11T03:51:53.733 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../092-python3-lxml_4.5.0-1ubuntu0.3_amd64.deb ... 2021-07-11T03:51:53.738 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2021-07-11T03:51:53.773 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-07-11T03:51:53.794 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2021-07-11T03:51:53.797 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-pyinotify. 2021-07-11T03:51:53.823 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2021-07-11T03:51:53.863 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-07-11T03:51:53.905 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-pytest. 2021-07-11T03:51:53.932 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2021-07-11T03:51:53.972 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-pytest (4.6.9-1) ... 2021-07-11T03:51:54.114 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-pluggy. 2021-07-11T03:51:54.122 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2021-07-11T03:51:54.165 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2021-07-11T03:51:54.168 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-pytest. 2021-07-11T03:51:54.175 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2021-07-11T03:51:54.216 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-pytest (4.6.9-1) ... 2021-07-11T03:51:54.218 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-packaging. 2021-07-11T03:51:54.245 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../093-python3-packaging_20.3-1_all.deb ... 2021-07-11T03:51:54.276 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-packaging (20.3-1) ... 2021-07-11T03:51:54.308 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package python3-simplejson. 2021-07-11T03:51:54.316 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2021-07-11T03:51:54.357 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2021-07-11T03:51:54.501 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-psutil. 2021-07-11T03:51:54.527 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2021-07-11T03:51:54.536 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package python3-simplejson. 2021-07-11T03:51:54.555 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-pastescript. 2021-07-11T03:51:54.563 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2021-07-11T03:51:54.575 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2021-07-11T03:51:54.581 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../094-python3-pastescript_2.0.2-4_all.deb ... 2021-07-11T03:51:54.610 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2021-07-11T03:51:54.621 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2021-07-11T03:51:54.786 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package radosgw. 2021-07-11T03:51:54.795 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../101-radosgw_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:54.835 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking radosgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:54.919 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-py. 2021-07-11T03:51:54.928 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../097-python3-py_1.8.1-1_all.deb ... 2021-07-11T03:51:54.975 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-pluggy. 2021-07-11T03:51:54.977 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-py (1.8.1-1) ... 2021-07-11T03:51:54.997 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package radosgw. 2021-07-11T03:51:55.000 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../095-python3-pluggy_0.13.0-2_all.deb ... 2021-07-11T03:51:55.005 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../101-radosgw_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:55.041 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2021-07-11T03:51:55.046 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking radosgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:55.339 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-pyinotify. 2021-07-11T03:51:55.365 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2021-07-11T03:51:55.394 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-psutil. 2021-07-11T03:51:55.400 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../096-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2021-07-11T03:51:55.413 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-07-11T03:51:55.442 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2021-07-11T03:51:55.723 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package rbd-fuse. 2021-07-11T03:51:55.750 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../102-rbd-fuse_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:55.774 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-pytest. 2021-07-11T03:51:55.782 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2021-07-11T03:51:55.789 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking rbd-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:55.822 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-py. 2021-07-11T03:51:55.823 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-pytest (4.6.9-1) ... 2021-07-11T03:51:55.848 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../097-python3-py_1.8.1-1_all.deb ... 2021-07-11T03:51:55.888 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-py (1.8.1-1) ... 2021-07-11T03:51:55.976 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package rbd-fuse. 2021-07-11T03:51:56.002 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../102-rbd-fuse_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:56.042 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking rbd-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:56.112 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-attr (19.3.0-2) ... 2021-07-11T03:51:56.187 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package python3-simplejson. 2021-07-11T03:51:56.213 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2021-07-11T03:51:56.224 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-pyinotify. 2021-07-11T03:51:56.251 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../098-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2021-07-11T03:51:56.253 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2021-07-11T03:51:56.299 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-07-11T03:51:56.351 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-07-11T03:51:56.372 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-attr (19.3.0-2) ... 2021-07-11T03:51:56.453 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2021-07-11T03:51:56.554 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2021-07-11T03:51:56.635 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-pytest. 2021-07-11T03:51:56.638 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-07-11T03:51:56.662 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../099-python3-pytest_4.6.9-1_all.deb ... 2021-07-11T03:51:56.706 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package radosgw. 2021-07-11T03:51:56.716 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../101-radosgw_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:56.738 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2021-07-11T03:51:56.743 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-pytest (4.6.9-1) ... 2021-07-11T03:51:56.755 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking radosgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:56.756 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:56.864 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2021-07-11T03:51:57.047 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package python3-simplejson. 2021-07-11T03:51:57.075 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../100-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2021-07-11T03:51:57.091 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:57.140 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2021-07-11T03:51:57.235 INFO:teuthology.orchestra.run.gibba019.stdout:Adding system user cephadm....done 2021-07-11T03:51:57.413 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-waitress (1.4.1-1) ... 2021-07-11T03:51:57.600 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package radosgw. 2021-07-11T03:51:57.608 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../101-radosgw_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:57.613 INFO:teuthology.orchestra.run.gibba023.stdout:Adding system user cephadm....done 2021-07-11T03:51:57.626 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package rbd-fuse. 2021-07-11T03:51:57.636 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-repoze.lru (0.7-2) ... 2021-07-11T03:51:57.649 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking radosgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:57.652 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../102-rbd-fuse_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:57.692 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking rbd-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:57.762 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-waitress (1.4.1-1) ... 2021-07-11T03:51:57.863 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up liboath0 (2.6.1-1.3) ... 2021-07-11T03:51:57.988 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-repoze.lru (0.7-2) ... 2021-07-11T03:51:57.989 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-py (1.8.1-1) ... 2021-07-11T03:51:57.990 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-attr (19.3.0-2) ... 2021-07-11T03:51:58.214 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up liboath0 (2.6.1-1.3) ... 2021-07-11T03:51:58.230 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-07-11T03:51:58.257 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-joblib (0.14.0-3) ... 2021-07-11T03:51:58.341 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-py (1.8.1-1) ... 2021-07-11T03:51:58.356 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2021-07-11T03:51:58.523 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2021-07-11T03:51:58.570 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package rbd-fuse. 2021-07-11T03:51:58.596 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../102-rbd-fuse_15.2.13-130-g45c6565d-1focal_amd64.deb ... 2021-07-11T03:51:58.600 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2021-07-11T03:51:58.609 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-joblib (0.14.0-3) ... 2021-07-11T03:51:58.636 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking rbd-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:58.733 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:58.851 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-cachetools (4.0.0-1) ... 2021-07-11T03:51:58.960 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-attr (19.3.0-2) ... 2021-07-11T03:51:58.968 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2021-07-11T03:51:59.103 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2021-07-11T03:51:59.153 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-cachetools (4.0.0-1) ... 2021-07-11T03:51:59.216 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libleveldb1d:amd64 (1.22-3ubuntu2) ... 2021-07-11T03:51:59.305 INFO:teuthology.orchestra.run.gibba004.stdout:Adding system user cephadm....done 2021-07-11T03:51:59.317 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2021-07-11T03:51:59.329 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2021-07-11T03:51:59.379 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2021-07-11T03:51:59.418 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2021-07-11T03:51:59.489 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-waitress (1.4.1-1) ... 2021-07-11T03:51:59.555 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-07-11T03:51:59.613 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2021-07-11T03:51:59.628 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:59.731 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-repoze.lru (0.7-2) ... 2021-07-11T03:51:59.756 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-ceph-argparse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:51:59.839 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-07-11T03:51:59.974 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up liboath0 (2.6.1-1.3) ... 2021-07-11T03:52:00.007 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-07-11T03:52:00.082 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-ceph-argparse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:00.100 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-py (1.8.1-1) ... 2021-07-11T03:52:00.151 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2021-07-11T03:52:00.200 INFO:teuthology.orchestra.run.gibba005.stdout:Adding system user cephadm....done 2021-07-11T03:52:00.324 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-07-11T03:52:00.334 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-joblib (0.14.0-3) ... 2021-07-11T03:52:00.351 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2021-07-11T03:52:00.392 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-waitress (1.4.1-1) ... 2021-07-11T03:52:00.484 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2021-07-11T03:52:00.594 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2021-07-11T03:52:00.643 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-repoze.lru (0.7-2) ... 2021-07-11T03:52:00.653 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2021-07-11T03:52:00.685 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2021-07-11T03:52:00.795 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2021-07-11T03:52:00.869 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up liboath0 (2.6.1-1.3) ... 2021-07-11T03:52:00.871 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-cachetools (4.0.0-1) ... 2021-07-11T03:52:00.903 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2021-07-11T03:52:01.029 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-py (1.8.1-1) ... 2021-07-11T03:52:01.037 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2021-07-11T03:52:01.097 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2021-07-11T03:52:01.120 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2021-07-11T03:52:01.230 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2021-07-11T03:52:01.254 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-joblib (0.14.0-3) ... 2021-07-11T03:52:01.363 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2021-07-11T03:52:01.364 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2021-07-11T03:52:01.532 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2021-07-11T03:52:01.581 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2021-07-11T03:52:01.607 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-07-11T03:52:01.615 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-jwt (1.7.1-2ubuntu2) ... 2021-07-11T03:52:01.800 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2021-07-11T03:52:01.840 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2021-07-11T03:52:01.841 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-cachetools (4.0.0-1) ... 2021-07-11T03:52:01.883 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-ceph-argparse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:02.067 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-colorama (0.4.3-1build1) ... 2021-07-11T03:52:02.076 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-tempita (0.5.2-6) ... 2021-07-11T03:52:02.099 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2021-07-11T03:52:02.159 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-07-11T03:52:02.285 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2021-07-11T03:52:02.327 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-packaging (20.3-1) ... 2021-07-11T03:52:02.335 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2021-07-11T03:52:02.375 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-tempita (0.5.2-6) ... 2021-07-11T03:52:02.527 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2021-07-11T03:52:02.569 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2021-07-11T03:52:02.578 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2021-07-11T03:52:02.626 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-packaging (20.3-1) ... 2021-07-11T03:52:02.679 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2021-07-11T03:52:02.754 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2021-07-11T03:52:02.812 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-ceph-argparse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:02.863 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2021-07-11T03:52:02.891 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2021-07-11T03:52:02.980 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2021-07-11T03:52:02.996 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2021-07-11T03:52:03.080 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2021-07-11T03:52:03.140 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2021-07-11T03:52:03.206 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-pyperclip (1.7.0-1) ... 2021-07-11T03:52:03.222 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2021-07-11T03:52:03.282 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2021-07-11T03:52:03.400 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2021-07-11T03:52:03.481 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2021-07-11T03:52:03.524 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2021-07-11T03:52:03.531 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2021-07-11T03:52:03.684 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2021-07-11T03:52:03.689 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2021-07-11T03:52:03.749 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2021-07-11T03:52:03.817 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2021-07-11T03:52:03.918 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2021-07-11T03:52:03.978 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up socat (1.7.3.3-2) ... 2021-07-11T03:52:04.000 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2021-07-11T03:52:04.068 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2021-07-11T03:52:04.104 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:04.202 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2021-07-11T03:52:04.293 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up socat (1.7.3.3-2) ... 2021-07-11T03:52:04.310 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-tempita (0.5.2-6) ... 2021-07-11T03:52:04.313 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-html5lib (1.0.1-2) ... 2021-07-11T03:52:04.419 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:04.420 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2021-07-11T03:52:04.569 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-packaging (20.3-1) ... 2021-07-11T03:52:04.623 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-07-11T03:52:04.629 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-html5lib (1.0.1-2) ... 2021-07-11T03:52:04.738 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2021-07-11T03:52:04.749 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pluggy (0.13.0-2) ... 2021-07-11T03:52:04.804 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2021-07-11T03:52:04.888 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-07-11T03:52:04.948 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2021-07-11T03:52:04.959 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2021-07-11T03:52:04.972 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2021-07-11T03:52:04.990 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-pluggy (0.13.0-2) ... 2021-07-11T03:52:05.077 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2021-07-11T03:52:05.140 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2021-07-11T03:52:05.215 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2021-07-11T03:52:05.273 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-tempita (0.5.2-6) ... 2021-07-11T03:52:05.333 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2021-07-11T03:52:05.345 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2021-07-11T03:52:05.441 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2021-07-11T03:52:05.524 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-packaging (20.3-1) ... 2021-07-11T03:52:05.592 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2021-07-11T03:52:05.595 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-logutils (0.3.3-7) ... 2021-07-11T03:52:05.692 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2021-07-11T03:52:05.793 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2021-07-11T03:52:05.835 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-logutils (0.3.3-7) ... 2021-07-11T03:52:05.855 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2021-07-11T03:52:05.894 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2021-07-11T03:52:05.987 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2021-07-11T03:52:05.993 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2021-07-11T03:52:06.086 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2021-07-11T03:52:06.119 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2021-07-11T03:52:06.223 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-prettytable (0.7.2-5) ... 2021-07-11T03:52:06.238 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2021-07-11T03:52:06.320 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up socat (1.7.3.3-2) ... 2021-07-11T03:52:06.413 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-prettytable (0.7.2-5) ... 2021-07-11T03:52:06.429 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:06.448 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-cmd2 (0.8.5-2.1) ... 2021-07-11T03:52:06.475 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2021-07-11T03:52:06.597 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2021-07-11T03:52:06.623 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-html5lib (1.0.1-2) ... 2021-07-11T03:52:06.697 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2) ... 2021-07-11T03:52:06.734 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2021-07-11T03:52:06.831 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2021-07-11T03:52:06.907 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-07-11T03:52:07.015 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2021-07-11T03:52:07.033 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-pluggy (0.13.0-2) ... 2021-07-11T03:52:07.167 INFO:teuthology.orchestra.run.gibba019.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-07-11T03:52:07.242 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2021-07-11T03:52:07.286 INFO:teuthology.orchestra.run.gibba023.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-07-11T03:52:07.311 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2021-07-11T03:52:07.316 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up socat (1.7.3.3-2) ... 2021-07-11T03:52:07.369 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2021-07-11T03:52:07.419 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:07.434 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2021-07-11T03:52:07.438 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-07-11T03:52:07.560 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-07-11T03:52:07.653 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-html5lib (1.0.1-2) ... 2021-07-11T03:52:07.670 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2021-07-11T03:52:07.705 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2021-07-11T03:52:07.827 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2021-07-11T03:52:07.871 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2021-07-11T03:52:07.921 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-logutils (0.3.3-7) ... 2021-07-11T03:52:07.998 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-07-11T03:52:08.022 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-pluggy (0.13.0-2) ... 2021-07-11T03:52:08.124 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-07-11T03:52:08.161 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-07-11T03:52:08.163 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2021-07-11T03:52:08.222 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libxslt1.1:amd64 (1.1.34-4) ... 2021-07-11T03:52:08.251 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-webob (1:1.8.5-2) ... 2021-07-11T03:52:08.264 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2021-07-11T03:52:08.272 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-07-11T03:52:08.373 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-webob (1:1.8.5-2) ... 2021-07-11T03:52:08.382 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2021-07-11T03:52:08.484 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-07-11T03:52:08.491 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-prettytable (0.7.2-5) ... 2021-07-11T03:52:08.582 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-07-11T03:52:08.650 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2021-07-11T03:52:08.725 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2021-07-11T03:52:08.728 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up librados2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:08.766 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up librados2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:08.854 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2021-07-11T03:52:08.892 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2021-07-11T03:52:08.911 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-logutils (0.3.3-7) ... 2021-07-11T03:52:08.935 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2021-07-11T03:52:09.103 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2021-07-11T03:52:09.204 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2021-07-11T03:52:09.371 INFO:teuthology.orchestra.run.gibba004.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-07-11T03:52:09.471 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-prettytable (0.7.2-5) ... 2021-07-11T03:52:09.546 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2021-07-11T03:52:09.671 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-07-11T03:52:09.722 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2021-07-11T03:52:09.926 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up librgw2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:09.930 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up librgw2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:09.972 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2021-07-11T03:52:09.981 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2021-07-11T03:52:10.031 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:52:10.076 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:52:10.132 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-pytest (4.6.9-1) ... 2021-07-11T03:52:10.186 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pytest (4.6.9-1) ... 2021-07-11T03:52:10.198 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-07-11T03:52:10.300 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-07-11T03:52:10.400 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2021-07-11T03:52:10.422 INFO:teuthology.orchestra.run.gibba005.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-07-11T03:52:10.455 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2021-07-11T03:52:10.459 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-webob (1:1.8.5-2) ... 2021-07-11T03:52:10.600 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2021-07-11T03:52:10.609 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2021-07-11T03:52:10.685 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-07-11T03:52:10.705 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2021-07-11T03:52:10.710 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2) ... 2021-07-11T03:52:10.855 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-bs4 (4.8.2-1) ... 2021-07-11T03:52:10.911 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2021-07-11T03:52:10.953 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up librados2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:10.956 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-bs4 (4.8.2-1) ... 2021-07-11T03:52:11.079 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2021-07-11T03:52:11.145 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libcephfs2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:11.205 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2021-07-11T03:52:11.249 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libcephfs2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:11.296 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2021-07-11T03:52:11.331 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2021-07-11T03:52:11.376 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2021-07-11T03:52:11.481 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-webob (1:1.8.5-2) ... 2021-07-11T03:52:11.522 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libradosstriper1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:11.624 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-07-11T03:52:11.643 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libradosstriper1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:11.770 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-07-11T03:52:11.874 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-07-11T03:52:12.055 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-07-11T03:52:12.101 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2021-07-11T03:52:12.126 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up librgw2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:12.252 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:52:12.350 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up librados2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:12.411 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-pytest (4.6.9-1) ... 2021-07-11T03:52:12.476 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2021-07-11T03:52:12.629 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up librbd1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:12.670 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2021-07-11T03:52:12.755 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-07-11T03:52:12.794 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up librbd1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:12.879 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2021-07-11T03:52:12.921 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2021-07-11T03:52:12.944 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-07-11T03:52:12.991 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-07-11T03:52:13.070 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2021-07-11T03:52:13.130 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-bs4 (4.8.2-1) ... 2021-07-11T03:52:13.192 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:52:13.196 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-07-11T03:52:13.301 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-rsa (4.0-3) ... 2021-07-11T03:52:13.414 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:52:13.415 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libcephfs2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:13.510 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-webtest (2.0.32-2) ... 2021-07-11T03:52:13.516 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2021-07-11T03:52:13.532 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up librgw2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:13.540 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-rsa (4.0-3) ... 2021-07-11T03:52:13.658 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:52:13.727 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libradosstriper1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:13.749 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-webtest (2.0.32-2) ... 2021-07-11T03:52:13.761 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-mgr-modules-core (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:13.784 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-pytest (4.6.9-1) ... 2021-07-11T03:52:13.853 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-07-11T03:52:13.887 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:14.000 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-modules-core (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:14.045 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-07-11T03:52:14.045 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-07-11T03:52:14.085 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2021-07-11T03:52:14.086 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-07-11T03:52:14.127 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:14.319 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2021-07-11T03:52:14.328 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-07-11T03:52:14.328 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-07-11T03:52:14.338 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libcephfs-dev (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:14.456 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-07-11T03:52:14.545 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-bs4 (4.8.2-1) ... 2021-07-11T03:52:14.595 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libcephfs-dev (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:14.673 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-pastescript (2.0.2-4) ... 2021-07-11T03:52:14.729 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-07-11T03:52:14.830 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libcephfs2 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:14.866 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up librbd1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:14.917 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2021-07-11T03:52:14.930 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pastescript (2.0.2-4) ... 2021-07-11T03:52:14.940 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2021-07-11T03:52:14.967 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-07-11T03:52:15.110 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2021-07-11T03:52:15.185 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-rados (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:15.190 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2021-07-11T03:52:15.208 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libradosstriper1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:15.237 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-07-11T03:52:15.311 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-rbd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:15.334 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2021-07-11T03:52:15.454 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:52:15.470 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up rbd-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:15.484 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-rados (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:15.596 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-routes (2.4.1-2) ... 2021-07-11T03:52:15.605 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-rsa (4.0-3) ... 2021-07-11T03:52:15.610 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-rbd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:15.635 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2021-07-11T03:52:15.752 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up rbd-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:15.856 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-webtest (2.0.32-2) ... 2021-07-11T03:52:15.873 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-google-auth (1.5.1-2) ... 2021-07-11T03:52:15.878 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-routes (2.4.1-2) ... 2021-07-11T03:52:16.099 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-rgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:16.115 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-mgr-modules-core (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:16.155 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-google-auth (1.5.1-2) ... 2021-07-11T03:52:16.226 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-cephfs (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:16.233 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:16.407 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-rgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:16.410 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up cephfs-shell (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:16.416 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-07-11T03:52:16.416 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-07-11T03:52:16.424 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up librbd1 (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:16.519 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2021-07-11T03:52:16.534 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-cephfs (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:16.575 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2021-07-11T03:52:16.701 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2021-07-11T03:52:16.709 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libcephfs-dev (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:16.743 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up cephfs-shell (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:16.810 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-07-11T03:52:16.827 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.3) ... 2021-07-11T03:52:16.869 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2021-07-11T03:52:17.036 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-pastescript (2.0.2-4) ... 2021-07-11T03:52:17.069 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2021-07-11T03:52:17.155 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:17.170 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-rsa (4.0-3) ... 2021-07-11T03:52:17.296 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2021-07-11T03:52:17.404 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-webtest (2.0.32-2) ... 2021-07-11T03:52:17.564 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:17.597 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-mgr-modules-core (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:17.632 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-rados (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:17.677 INFO:teuthology.orchestra.run.gibba023.stdout:Adding group ceph....done 2021-07-11T03:52:17.699 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:17.758 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-rbd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:17.883 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-07-11T03:52:17.883 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-07-11T03:52:17.908 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up rbd-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:18.034 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-routes (2.4.1-2) ... 2021-07-11T03:52:18.053 INFO:teuthology.orchestra.run.gibba023.stdout:Adding system user ceph....done 2021-07-11T03:52:18.082 INFO:teuthology.orchestra.run.gibba019.stdout:Adding group ceph....done 2021-07-11T03:52:18.195 INFO:teuthology.orchestra.run.gibba023.stdout:Setting system user ceph properties....done 2021-07-11T03:52:18.200 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libcephfs-dev (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:18.208 INFO:teuthology.orchestra.run.gibba023.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-07-11T03:52:18.285 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-07-11T03:52:18.326 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2021-07-11T03:52:18.361 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-google-auth (1.5.1-2) ... 2021-07-11T03:52:18.472 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-07-11T03:52:18.486 INFO:teuthology.orchestra.run.gibba019.stdout:Adding system user ceph....done 2021-07-11T03:52:18.544 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-pastescript (2.0.2-4) ... 2021-07-11T03:52:18.597 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-rgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:18.675 INFO:teuthology.orchestra.run.gibba019.stdout:Setting system user ceph properties....done 2021-07-11T03:52:18.697 INFO:teuthology.orchestra.run.gibba019.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-07-11T03:52:18.723 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-cephfs (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:18.771 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-07-11T03:52:18.772 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-test (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:18.811 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2021-07-11T03:52:18.882 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up radosgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:18.954 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-07-11T03:52:18.965 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up cephfs-shell (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:19.085 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-07-11T03:52:19.085 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-07-11T03:52:19.091 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2021-07-11T03:52:19.113 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-rados (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:19.214 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-rbd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:19.256 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-test (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:19.349 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up rbd-fuse (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:19.382 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up radosgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:19.475 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-routes (2.4.1-2) ... 2021-07-11T03:52:19.492 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-base (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:19.585 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-07-11T03:52:19.585 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-07-11T03:52:19.758 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-07-11T03:52:19.769 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:19.811 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-google-auth (1.5.1-2) ... 2021-07-11T03:52:19.976 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-base (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:20.004 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-rgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:20.069 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-mds (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:20.130 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-cephfs (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:20.215 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-07-11T03:52:20.215 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-07-11T03:52:20.246 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-07-11T03:52:20.356 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up cephfs-shell (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:20.498 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2021-07-11T03:52:20.512 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-mgr (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:20.536 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mds (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:20.679 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-07-11T03:52:20.680 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-07-11T03:52:20.686 INFO:teuthology.orchestra.run.gibba004.stdout:Adding group ceph....done 2021-07-11T03:52:20.697 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-07-11T03:52:20.697 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-07-11T03:52:20.962 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:20.963 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:21.058 INFO:teuthology.orchestra.run.gibba004.stdout:Adding system user ceph....done 2021-07-11T03:52:21.072 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-osd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:21.123 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-07-11T03:52:21.123 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-07-11T03:52:21.127 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-common (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:21.254 INFO:teuthology.orchestra.run.gibba004.stdout:Setting system user ceph properties....done 2021-07-11T03:52:21.271 INFO:teuthology.orchestra.run.gibba004.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-07-11T03:52:21.353 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-07-11T03:52:21.397 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:21.474 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-07-11T03:52:21.474 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-07-11T03:52:21.537 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-07-11T03:52:21.556 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-osd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:21.640 INFO:teuthology.orchestra.run.gibba005.stdout:Adding group ceph....done 2021-07-11T03:52:21.765 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-mgr-k8sevents (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:21.828 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-test (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:21.866 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-mgr-diskprediction-local (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:21.937 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up radosgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:21.982 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-07-11T03:52:21.982 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-07-11T03:52:21.984 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-mon (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:22.028 INFO:teuthology.orchestra.run.gibba005.stdout:Adding system user ceph....done 2021-07-11T03:52:22.109 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-07-11T03:52:22.110 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-07-11T03:52:22.167 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-07-11T03:52:22.168 INFO:teuthology.orchestra.run.gibba023.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-07-11T03:52:22.187 INFO:teuthology.orchestra.run.gibba005.stdout:Setting system user ceph properties....done 2021-07-11T03:52:22.197 INFO:teuthology.orchestra.run.gibba005.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2021-07-11T03:52:22.275 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-07-11T03:52:22.383 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-k8sevents (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:22.452 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-mgr-cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:22.463 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-07-11T03:52:22.497 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-base (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:22.501 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-diskprediction-local (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:22.586 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:22.619 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mon (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:22.713 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up ceph-mgr-dashboard (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:22.749 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-07-11T03:52:22.761 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-07-11T03:52:22.761 INFO:teuthology.orchestra.run.gibba019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-07-11T03:52:22.795 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-test (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:22.897 INFO:teuthology.orchestra.run.gibba023.stdout:Processing triggers for systemd (245.4-4ubuntu3.4) ... 2021-07-11T03:52:22.921 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up radosgw (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:23.031 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:23.057 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-mds (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:23.107 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-07-11T03:52:23.108 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-07-11T03:52:23.122 INFO:teuthology.orchestra.run.gibba023.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-07-11T03:52:23.163 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:23.225 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-07-11T03:52:23.225 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-07-11T03:52:23.289 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up ceph-mgr-dashboard (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:23.415 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for systemd (245.4-4ubuntu3.4) ... 2021-07-11T03:52:23.531 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-base (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:23.543 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-mgr (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:23.640 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-07-11T03:52:23.748 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-07-11T03:52:23.748 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-07-11T03:52:23.764 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-07-11T03:52:24.035 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:24.041 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-mds (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:24.169 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-osd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:24.245 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-07-11T03:52:24.245 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-07-11T03:52:24.525 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-mgr (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:24.563 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-07-11T03:52:24.564 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-07-11T03:52:24.671 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-07-11T03:52:24.671 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-07-11T03:52:24.709 INFO:teuthology.orchestra.run.gibba023.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-07-11T03:52:24.838 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-mgr-k8sevents (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:24.953 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-mgr-diskprediction-cloud (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:24.981 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-mgr-diskprediction-local (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:25.013 INFO:teuthology.orchestra.run.gibba023.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-07-11T03:52:25.015 DEBUG:teuthology.orchestra.run.gibba023:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-07-11T03:52:25.096 INFO:teuthology.orchestra.run.gibba023.stdout:Reading package lists... 2021-07-11T03:52:25.097 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-mon (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:25.129 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-osd (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:25.194 INFO:teuthology.orchestra.run.gibba023.stdout:Building dependency tree... 2021-07-11T03:52:25.195 INFO:teuthology.orchestra.run.gibba023.stdout:Reading state information... 2021-07-11T03:52:25.209 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-07-11T03:52:25.271 INFO:teuthology.orchestra.run.gibba023.stdout:libtool-bin is already the newest version (2.4.6-14). 2021-07-11T03:52:25.272 INFO:teuthology.orchestra.run.gibba023.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2021-07-11T03:52:25.272 INFO:teuthology.orchestra.run.gibba023.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.1). 2021-07-11T03:52:25.272 INFO:teuthology.orchestra.run.gibba023.stdout:The following packages were automatically installed and are no longer required: 2021-07-11T03:52:25.272 INFO:teuthology.orchestra.run.gibba023.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2021-07-11T03:52:25.273 INFO:teuthology.orchestra.run.gibba023.stdout:Use 'sudo apt autoremove' to remove them. 2021-07-11T03:52:25.273 INFO:teuthology.orchestra.run.gibba023.stdout:The following additional packages will be installed: 2021-07-11T03:52:25.273 INFO:teuthology.orchestra.run.gibba023.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2021-07-11T03:52:25.273 INFO:teuthology.orchestra.run.gibba023.stdout:Suggested packages: 2021-07-11T03:52:25.273 INFO:teuthology.orchestra.run.gibba023.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-07-11T03:52:25.280 INFO:teuthology.orchestra.run.gibba023.stdout:The following NEW packages will be installed: 2021-07-11T03:52:25.280 INFO:teuthology.orchestra.run.gibba023.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2021-07-11T03:52:25.280 INFO:teuthology.orchestra.run.gibba023.stdout: libssl-dev 2021-07-11T03:52:25.281 INFO:teuthology.orchestra.run.gibba023.stdout:The following packages will be upgraded: 2021-07-11T03:52:25.281 INFO:teuthology.orchestra.run.gibba023.stdout: libssl1.1 2021-07-11T03:52:25.302 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-07-11T03:52:25.302 INFO:teuthology.orchestra.run.gibba004.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-07-11T03:52:25.454 INFO:teuthology.orchestra.run.gibba023.stdout:1 upgraded, 10 newly installed, 0 to remove and 140 not upgraded. 2021-07-11T03:52:25.455 INFO:teuthology.orchestra.run.gibba023.stdout:Need to get 4,231 kB of archives. 2021-07-11T03:52:25.455 INFO:teuthology.orchestra.run.gibba023.stdout:After this operation, 12.4 MB of additional disk space will be used. 2021-07-11T03:52:25.455 INFO:teuthology.orchestra.run.gibba023.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2021-07-11T03:52:25.514 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-07-11T03:52:25.514 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-07-11T03:52:25.533 INFO:teuthology.orchestra.run.gibba019.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-07-11T03:52:25.534 DEBUG:teuthology.orchestra.run.gibba019:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-07-11T03:52:25.570 INFO:teuthology.orchestra.run.gibba019.stdout:Reading package lists... 2021-07-11T03:52:25.575 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-mgr-cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:25.666 INFO:teuthology.orchestra.run.gibba019.stdout:Building dependency tree... 2021-07-11T03:52:25.667 INFO:teuthology.orchestra.run.gibba019.stdout:Reading state information... 2021-07-11T03:52:25.709 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:25.741 INFO:teuthology.orchestra.run.gibba019.stdout:libtool-bin is already the newest version (2.4.6-14). 2021-07-11T03:52:25.741 INFO:teuthology.orchestra.run.gibba019.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2021-07-11T03:52:25.741 INFO:teuthology.orchestra.run.gibba019.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.1). 2021-07-11T03:52:25.741 INFO:teuthology.orchestra.run.gibba019.stdout:The following packages were automatically installed and are no longer required: 2021-07-11T03:52:25.742 INFO:teuthology.orchestra.run.gibba019.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2021-07-11T03:52:25.742 INFO:teuthology.orchestra.run.gibba019.stdout:Use 'sudo apt autoremove' to remove them. 2021-07-11T03:52:25.742 INFO:teuthology.orchestra.run.gibba019.stdout:The following additional packages will be installed: 2021-07-11T03:52:25.742 INFO:teuthology.orchestra.run.gibba019.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2021-07-11T03:52:25.743 INFO:teuthology.orchestra.run.gibba019.stdout:Suggested packages: 2021-07-11T03:52:25.743 INFO:teuthology.orchestra.run.gibba019.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-07-11T03:52:25.750 INFO:teuthology.orchestra.run.gibba019.stdout:The following NEW packages will be installed: 2021-07-11T03:52:25.751 INFO:teuthology.orchestra.run.gibba019.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2021-07-11T03:52:25.751 INFO:teuthology.orchestra.run.gibba019.stdout: libssl-dev 2021-07-11T03:52:25.751 INFO:teuthology.orchestra.run.gibba019.stdout:The following packages will be upgraded: 2021-07-11T03:52:25.752 INFO:teuthology.orchestra.run.gibba019.stdout: libssl1.1 2021-07-11T03:52:25.803 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-mgr-k8sevents (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:25.868 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up ceph-mgr-dashboard (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:25.910 INFO:teuthology.orchestra.run.gibba023.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.4 [1,319 kB] 2021-07-11T03:52:25.915 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-mgr-diskprediction-local (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:25.923 INFO:teuthology.orchestra.run.gibba019.stdout:1 upgraded, 10 newly installed, 0 to remove and 140 not upgraded. 2021-07-11T03:52:25.924 INFO:teuthology.orchestra.run.gibba019.stdout:Need to get 4,231 kB of archives. 2021-07-11T03:52:25.924 INFO:teuthology.orchestra.run.gibba019.stdout:After this operation, 12.4 MB of additional disk space will be used. 2021-07-11T03:52:25.924 INFO:teuthology.orchestra.run.gibba019.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2021-07-11T03:52:26.003 INFO:teuthology.orchestra.run.gibba004.stdout:Processing triggers for systemd (245.4-4ubuntu3.4) ... 2021-07-11T03:52:26.032 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-mon (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:26.115 INFO:teuthology.orchestra.run.gibba023.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2021-07-11T03:52:26.163 INFO:teuthology.orchestra.run.gibba023.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2021-07-11T03:52:26.165 INFO:teuthology.orchestra.run.gibba023.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2021-07-11T03:52:26.171 INFO:teuthology.orchestra.run.gibba023.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2021-07-11T03:52:26.174 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-07-11T03:52:26.174 INFO:teuthology.orchestra.run.gibba005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-07-11T03:52:26.175 INFO:teuthology.orchestra.run.gibba023.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2021-07-11T03:52:26.175 INFO:teuthology.orchestra.run.gibba023.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2021-07-11T03:52:26.176 INFO:teuthology.orchestra.run.gibba023.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2021-07-11T03:52:26.177 INFO:teuthology.orchestra.run.gibba023.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2021-07-11T03:52:26.177 INFO:teuthology.orchestra.run.gibba023.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.4 [1,583 kB] 2021-07-11T03:52:26.255 INFO:teuthology.orchestra.run.gibba004.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-07-11T03:52:26.393 INFO:teuthology.orchestra.run.gibba019.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.4 [1,319 kB] 2021-07-11T03:52:26.400 INFO:teuthology.orchestra.run.gibba023.stdout:Preconfiguring packages ... 2021-07-11T03:52:26.467 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-mgr-cephadm (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:26.525 INFO:teuthology.orchestra.run.gibba023.stdout:Fetched 4,231 kB in 1s (4,444 kB/s) 2021-07-11T03:52:26.602 INFO:teuthology.orchestra.run.gibba019.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2021-07-11T03:52:26.634 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:26.648 INFO:teuthology.orchestra.run.gibba019.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2021-07-11T03:52:26.650 INFO:teuthology.orchestra.run.gibba019.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2021-07-11T03:52:26.656 INFO:teuthology.orchestra.run.gibba019.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2021-07-11T03:52:26.657 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package flex. 2021-07-11T03:52:26.663 INFO:teuthology.orchestra.run.gibba019.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2021-07-11T03:52:26.664 INFO:teuthology.orchestra.run.gibba019.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2021-07-11T03:52:26.665 INFO:teuthology.orchestra.run.gibba019.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2021-07-11T03:52:26.666 INFO:teuthology.orchestra.run.gibba019.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2021-07-11T03:52:26.667 INFO:teuthology.orchestra.run.gibba019.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.4 [1,583 kB] 2021-07-11T03:52:26.725 INFO:teuthology.orchestra.run.gibba023.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145503 files and directories currently installed.) 2021-07-11T03:52:26.727 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:26.760 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up ceph-mgr-dashboard (15.2.13-130-g45c6565d-1focal) ... 2021-07-11T03:52:26.771 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking flex (2.6.4-6.2) ... 2021-07-11T03:52:26.887 INFO:teuthology.orchestra.run.gibba019.stdout:Preconfiguring packages ... 2021-07-11T03:52:26.896 INFO:teuthology.orchestra.run.gibba005.stdout:Processing triggers for systemd (245.4-4ubuntu3.4) ... 2021-07-11T03:52:26.956 INFO:teuthology.orchestra.run.gibba019.stdout:Fetched 4,231 kB in 1s (4,363 kB/s) 2021-07-11T03:52:27.067 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package flex. 2021-07-11T03:52:27.134 INFO:teuthology.orchestra.run.gibba019.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145503 files and directories currently installed.) 2021-07-11T03:52:27.136 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:27.154 INFO:teuthology.orchestra.run.gibba005.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-07-11T03:52:27.192 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking flex (2.6.4-6.2) ... 2021-07-11T03:52:27.343 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.4_amd64.deb ... 2021-07-11T03:52:27.643 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.4) over (1.1.1f-1ubuntu2.2) ... 2021-07-11T03:52:27.771 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.4_amd64.deb ... 2021-07-11T03:52:27.937 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.4) over (1.1.1f-1ubuntu2.2) ... 2021-07-11T03:52:28.081 INFO:teuthology.orchestra.run.gibba004.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-07-11T03:52:28.189 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package bison. 2021-07-11T03:52:28.197 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2021-07-11T03:52:28.240 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2021-07-11T03:52:28.424 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package bison. 2021-07-11T03:52:28.431 INFO:teuthology.orchestra.run.gibba004.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-07-11T03:52:28.432 DEBUG:teuthology.orchestra.run.gibba004:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-07-11T03:52:28.433 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2021-07-11T03:52:28.475 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2021-07-11T03:52:28.516 INFO:teuthology.orchestra.run.gibba004.stdout:Reading package lists... 2021-07-11T03:52:28.609 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package liblzo2-2:amd64. 2021-07-11T03:52:28.616 INFO:teuthology.orchestra.run.gibba004.stdout:Building dependency tree... 2021-07-11T03:52:28.617 INFO:teuthology.orchestra.run.gibba004.stdout:Reading state information... 2021-07-11T03:52:28.636 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2021-07-11T03:52:28.643 INFO:teuthology.orchestra.run.gibba005.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-07-11T03:52:28.692 INFO:teuthology.orchestra.run.gibba004.stdout:libtool-bin is already the newest version (2.4.6-14). 2021-07-11T03:52:28.693 INFO:teuthology.orchestra.run.gibba004.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2021-07-11T03:52:28.693 INFO:teuthology.orchestra.run.gibba004.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.1). 2021-07-11T03:52:28.693 INFO:teuthology.orchestra.run.gibba004.stdout:The following packages were automatically installed and are no longer required: 2021-07-11T03:52:28.694 INFO:teuthology.orchestra.run.gibba004.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2021-07-11T03:52:28.694 INFO:teuthology.orchestra.run.gibba004.stdout:Use 'sudo apt autoremove' to remove them. 2021-07-11T03:52:28.694 INFO:teuthology.orchestra.run.gibba004.stdout:The following additional packages will be installed: 2021-07-11T03:52:28.695 INFO:teuthology.orchestra.run.gibba004.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2021-07-11T03:52:28.695 INFO:teuthology.orchestra.run.gibba004.stdout:Suggested packages: 2021-07-11T03:52:28.695 INFO:teuthology.orchestra.run.gibba004.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-07-11T03:52:28.702 INFO:teuthology.orchestra.run.gibba004.stdout:The following NEW packages will be installed: 2021-07-11T03:52:28.703 INFO:teuthology.orchestra.run.gibba004.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2021-07-11T03:52:28.703 INFO:teuthology.orchestra.run.gibba004.stdout: libssl-dev 2021-07-11T03:52:28.704 INFO:teuthology.orchestra.run.gibba004.stdout:The following packages will be upgraded: 2021-07-11T03:52:28.704 INFO:teuthology.orchestra.run.gibba004.stdout: libssl1.1 2021-07-11T03:52:28.708 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2021-07-11T03:52:28.896 INFO:teuthology.orchestra.run.gibba004.stdout:1 upgraded, 10 newly installed, 0 to remove and 140 not upgraded. 2021-07-11T03:52:28.897 INFO:teuthology.orchestra.run.gibba004.stdout:Need to get 4,231 kB of archives. 2021-07-11T03:52:28.897 INFO:teuthology.orchestra.run.gibba004.stdout:After this operation, 12.4 MB of additional disk space will be used. 2021-07-11T03:52:28.897 INFO:teuthology.orchestra.run.gibba004.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2021-07-11T03:52:28.903 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package liblzo2-2:amd64. 2021-07-11T03:52:28.929 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2021-07-11T03:52:28.969 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2021-07-11T03:52:29.007 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package dump. 2021-07-11T03:52:29.011 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2021-07-11T03:52:29.032 INFO:teuthology.orchestra.run.gibba005.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-07-11T03:52:29.033 DEBUG:teuthology.orchestra.run.gibba005:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2021-07-11T03:52:29.052 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking dump (0.4b46-6) ... 2021-07-11T03:52:29.111 INFO:teuthology.orchestra.run.gibba005.stdout:Reading package lists... 2021-07-11T03:52:29.209 INFO:teuthology.orchestra.run.gibba005.stdout:Building dependency tree... 2021-07-11T03:52:29.210 INFO:teuthology.orchestra.run.gibba005.stdout:Reading state information... 2021-07-11T03:52:29.256 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package dump. 2021-07-11T03:52:29.265 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2021-07-11T03:52:29.284 INFO:teuthology.orchestra.run.gibba005.stdout:libtool-bin is already the newest version (2.4.6-14). 2021-07-11T03:52:29.284 INFO:teuthology.orchestra.run.gibba005.stdout:xfslibs-dev is already the newest version (5.3.0-1ubuntu2). 2021-07-11T03:52:29.284 INFO:teuthology.orchestra.run.gibba005.stdout:uuid-dev is already the newest version (2.34-0.1ubuntu9.1). 2021-07-11T03:52:29.284 INFO:teuthology.orchestra.run.gibba005.stdout:The following packages were automatically installed and are no longer required: 2021-07-11T03:52:29.285 INFO:teuthology.orchestra.run.gibba005.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2021-07-11T03:52:29.285 INFO:teuthology.orchestra.run.gibba005.stdout:Use 'sudo apt autoremove' to remove them. 2021-07-11T03:52:29.286 INFO:teuthology.orchestra.run.gibba005.stdout:The following additional packages will be installed: 2021-07-11T03:52:29.286 INFO:teuthology.orchestra.run.gibba005.stdout: libfl-dev libfl2 liblzo2-2 libssl1.1 2021-07-11T03:52:29.286 INFO:teuthology.orchestra.run.gibba005.stdout:Suggested packages: 2021-07-11T03:52:29.286 INFO:teuthology.orchestra.run.gibba005.stdout: bison-doc flex-doc indent-doc libssl-doc 2021-07-11T03:52:29.294 INFO:teuthology.orchestra.run.gibba005.stdout:The following NEW packages will be installed: 2021-07-11T03:52:29.294 INFO:teuthology.orchestra.run.gibba005.stdout: bison dump flex indent libaio-dev libelf-dev libfl-dev libfl2 liblzo2-2 2021-07-11T03:52:29.294 INFO:teuthology.orchestra.run.gibba005.stdout: libssl-dev 2021-07-11T03:52:29.295 INFO:teuthology.orchestra.run.gibba005.stdout:The following packages will be upgraded: 2021-07-11T03:52:29.295 INFO:teuthology.orchestra.run.gibba005.stdout: libssl1.1 2021-07-11T03:52:29.305 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking dump (0.4b46-6) ... 2021-07-11T03:52:29.323 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package indent. 2021-07-11T03:52:29.341 INFO:teuthology.orchestra.run.gibba004.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.4 [1,319 kB] 2021-07-11T03:52:29.350 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2021-07-11T03:52:29.389 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking indent (2.2.12-1) ... 2021-07-11T03:52:29.468 INFO:teuthology.orchestra.run.gibba005.stdout:1 upgraded, 10 newly installed, 0 to remove and 140 not upgraded. 2021-07-11T03:52:29.469 INFO:teuthology.orchestra.run.gibba005.stdout:Need to get 4,231 kB of archives. 2021-07-11T03:52:29.469 INFO:teuthology.orchestra.run.gibba005.stdout:After this operation, 12.4 MB of additional disk space will be used. 2021-07-11T03:52:29.469 INFO:teuthology.orchestra.run.gibba005.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 flex amd64 2.6.4-6.2 [317 kB] 2021-07-11T03:52:29.551 INFO:teuthology.orchestra.run.gibba004.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2021-07-11T03:52:29.559 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package indent. 2021-07-11T03:52:29.586 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2021-07-11T03:52:29.600 INFO:teuthology.orchestra.run.gibba004.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2021-07-11T03:52:29.601 INFO:teuthology.orchestra.run.gibba004.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2021-07-11T03:52:29.608 INFO:teuthology.orchestra.run.gibba004.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2021-07-11T03:52:29.611 INFO:teuthology.orchestra.run.gibba004.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2021-07-11T03:52:29.612 INFO:teuthology.orchestra.run.gibba004.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2021-07-11T03:52:29.613 INFO:teuthology.orchestra.run.gibba004.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2021-07-11T03:52:29.613 INFO:teuthology.orchestra.run.gibba004.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2021-07-11T03:52:29.614 INFO:teuthology.orchestra.run.gibba004.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.4 [1,583 kB] 2021-07-11T03:52:29.625 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking indent (2.2.12-1) ... 2021-07-11T03:52:29.684 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-07-11T03:52:29.692 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2021-07-11T03:52:29.733 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2021-07-11T03:52:29.840 INFO:teuthology.orchestra.run.gibba004.stdout:Preconfiguring packages ... 2021-07-11T03:52:29.895 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-07-11T03:52:29.921 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2021-07-11T03:52:29.933 INFO:teuthology.orchestra.run.gibba005.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl1.1 amd64 1.1.1f-1ubuntu2.4 [1,319 kB] 2021-07-11T03:52:29.954 INFO:teuthology.orchestra.run.gibba004.stdout:Fetched 4,231 kB in 1s (4,370 kB/s) 2021-07-11T03:52:29.977 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2021-07-11T03:52:30.027 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-07-11T03:52:30.034 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2021-07-11T03:52:30.055 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package flex. 2021-07-11T03:52:30.077 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2021-07-11T03:52:30.120 INFO:teuthology.orchestra.run.gibba004.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145503 files and directories currently installed.) 2021-07-11T03:52:30.123 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:30.143 INFO:teuthology.orchestra.run.gibba005.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 bison amd64 2:3.5.1+dfsg-1 [657 kB] 2021-07-11T03:52:30.160 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking flex (2.6.4-6.2) ... 2021-07-11T03:52:30.190 INFO:teuthology.orchestra.run.gibba005.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblzo2-2 amd64 2.10-2 [50.8 kB] 2021-07-11T03:52:30.192 INFO:teuthology.orchestra.run.gibba005.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/universe amd64 dump amd64 0.4b46-6 [130 kB] 2021-07-11T03:52:30.197 INFO:teuthology.orchestra.run.gibba005.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/main amd64 indent amd64 2.2.12-1 [86.9 kB] 2021-07-11T03:52:30.200 INFO:teuthology.orchestra.run.gibba005.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/main amd64 libaio-dev amd64 0.3.112-5 [13.7 kB] 2021-07-11T03:52:30.201 INFO:teuthology.orchestra.run.gibba005.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal/main amd64 libelf-dev amd64 0.176-1.1build1 [57.0 kB] 2021-07-11T03:52:30.203 INFO:teuthology.orchestra.run.gibba005.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl2 amd64 2.6.4-6.2 [11.5 kB] 2021-07-11T03:52:30.204 INFO:teuthology.orchestra.run.gibba005.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal/main amd64 libfl-dev amd64 2.6.4-6.2 [6,316 B] 2021-07-11T03:52:30.204 INFO:teuthology.orchestra.run.gibba005.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libssl-dev amd64 1.1.1f-1ubuntu2.4 [1,583 kB] 2021-07-11T03:52:30.386 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-07-11T03:52:30.412 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2021-07-11T03:52:30.431 INFO:teuthology.orchestra.run.gibba005.stdout:Preconfiguring packages ... 2021-07-11T03:52:30.458 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2021-07-11T03:52:30.473 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libfl2:amd64. 2021-07-11T03:52:30.482 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:30.513 INFO:teuthology.orchestra.run.gibba005.stdout:Fetched 4,231 kB in 1s (4,371 kB/s) 2021-07-11T03:52:30.522 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:30.639 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package flex. 2021-07-11T03:52:30.683 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.4_amd64.deb ... 2021-07-11T03:52:30.709 INFO:teuthology.orchestra.run.gibba005.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 145503 files and directories currently installed.) 2021-07-11T03:52:30.709 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../00-flex_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:30.753 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking flex (2.6.4-6.2) ... 2021-07-11T03:52:30.783 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-07-11T03:52:30.805 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libfl2:amd64. 2021-07-11T03:52:30.809 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:30.813 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:30.850 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:30.863 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:31.012 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.4) over (1.1.1f-1ubuntu2.2) ... 2021-07-11T03:52:31.115 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-07-11T03:52:31.119 INFO:teuthology.orchestra.run.gibba023.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-07-11T03:52:31.142 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:31.146 INFO:teuthology.orchestra.run.gibba023.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.4_amd64.deb ... 2021-07-11T03:52:31.182 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:31.228 INFO:teuthology.orchestra.run.gibba023.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:31.360 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../01-libssl1.1_1.1.1f-1ubuntu2.4_amd64.deb ... 2021-07-11T03:52:31.463 INFO:teuthology.orchestra.run.gibba019.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-07-11T03:52:31.487 INFO:teuthology.orchestra.run.gibba019.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.4_amd64.deb ... 2021-07-11T03:52:31.518 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libssl1.1:amd64 (1.1.1f-1ubuntu2.4) over (1.1.1f-1ubuntu2.2) ... 2021-07-11T03:52:31.527 INFO:teuthology.orchestra.run.gibba019.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:31.604 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package bison. 2021-07-11T03:52:31.631 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2021-07-11T03:52:31.674 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up flex (2.6.4-6.2) ... 2021-07-11T03:52:31.684 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2021-07-11T03:52:31.782 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:31.947 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up flex (2.6.4-6.2) ... 2021-07-11T03:52:32.013 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package bison. 2021-07-11T03:52:32.022 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../02-bison_2%3a3.5.1+dfsg-1_amd64.deb ... 2021-07-11T03:52:32.033 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2021-07-11T03:52:32.065 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking bison (2:3.5.1+dfsg-1) ... 2021-07-11T03:52:32.073 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:32.124 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package liblzo2-2:amd64. 2021-07-11T03:52:32.133 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2021-07-11T03:52:32.160 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2021-07-11T03:52:32.215 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2021-07-11T03:52:32.285 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:32.391 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2021-07-11T03:52:32.411 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2021-07-11T03:52:32.476 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package liblzo2-2:amd64. 2021-07-11T03:52:32.485 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package dump. 2021-07-11T03:52:32.503 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../03-liblzo2-2_2.10-2_amd64.deb ... 2021-07-11T03:52:32.511 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2021-07-11T03:52:32.537 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:32.542 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking liblzo2-2:amd64 (2.10-2) ... 2021-07-11T03:52:32.551 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking dump (0.4b46-6) ... 2021-07-11T03:52:32.660 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2021-07-11T03:52:32.662 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up dump (0.4b46-6) ... 2021-07-11T03:52:32.786 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:32.834 INFO:teuthology.orchestra.run.gibba023.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-07-11T03:52:32.872 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package indent. 2021-07-11T03:52:32.899 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2021-07-11T03:52:32.936 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2021-07-11T03:52:32.938 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking indent (2.2.12-1) ... 2021-07-11T03:52:32.939 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2021-07-11T03:52:33.015 INFO:teuthology.orchestra.run.gibba023.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-07-11T03:52:33.063 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:33.065 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up indent (2.2.12-1) ... 2021-07-11T03:52:33.166 INFO:teuthology.orchestra.run.gibba023.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:33.192 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up dump (0.4b46-6) ... 2021-07-11T03:52:33.232 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-07-11T03:52:33.238 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package dump. 2021-07-11T03:52:33.241 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2021-07-11T03:52:33.246 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../04-dump_0.4b46-6_amd64.deb ... 2021-07-11T03:52:33.270 INFO:teuthology.orchestra.run.gibba019.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-07-11T03:52:33.277 INFO:teuthology.orchestra.run.gibba023.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-07-11T03:52:33.282 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2021-07-11T03:52:33.287 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking dump (0.4b46-6) ... 2021-07-11T03:52:33.323 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2021-07-11T03:52:33.398 INFO:teuthology.orchestra.run.gibba019.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-07-11T03:52:33.420 INFO:teuthology.orchestra.run.gibba023.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-07-11T03:52:33.459 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up indent (2.2.12-1) ... 2021-07-11T03:52:33.558 INFO:teuthology.orchestra.run.gibba019.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:33.559 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package indent. 2021-07-11T03:52:33.577 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-07-11T03:52:33.585 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../05-indent_2.2.12-1_amd64.deb ... 2021-07-11T03:52:33.603 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2021-07-11T03:52:33.625 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking indent (2.2.12-1) ... 2021-07-11T03:52:33.660 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2021-07-11T03:52:33.686 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-07-11T03:52:33.828 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-07-11T03:52:33.919 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libaio-dev:amd64. 2021-07-11T03:52:33.946 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../06-libaio-dev_0.3.112-5_amd64.deb ... 2021-07-11T03:52:33.985 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libfl2:amd64. 2021-07-11T03:52:33.986 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libaio-dev:amd64 (0.3.112-5) ... 2021-07-11T03:52:33.993 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:34.026 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:34.270 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-07-11T03:52:34.297 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:34.337 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:34.356 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libelf-dev:amd64. 2021-07-11T03:52:34.365 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../07-libelf-dev_0.176-1.1build1_amd64.deb ... 2021-07-11T03:52:34.406 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libelf-dev:amd64 (0.176-1.1build1) ... 2021-07-11T03:52:34.573 INFO:teuthology.orchestra.run.gibba004.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-07-11T03:52:34.599 INFO:teuthology.orchestra.run.gibba004.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.4_amd64.deb ... 2021-07-11T03:52:34.639 INFO:teuthology.orchestra.run.gibba004.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:34.776 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libfl2:amd64. 2021-07-11T03:52:34.802 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../08-libfl2_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:34.822 INFO:teuthology.orchestra.run.gibba023.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2021-07-11T03:52:34.842 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libfl2:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:35.095 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libfl-dev:amd64. 2021-07-11T03:52:35.111 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up flex (2.6.4-6.2) ... 2021-07-11T03:52:35.121 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../09-libfl-dev_2.6.4-6.2_amd64.deb ... 2021-07-11T03:52:35.161 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libfl-dev:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:35.236 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:35.456 INFO:teuthology.orchestra.run.gibba005.stdout:Selecting previously unselected package libssl-dev:amd64. 2021-07-11T03:52:35.465 INFO:teuthology.orchestra.run.gibba005.stdout:Preparing to unpack .../10-libssl-dev_1.1.1f-1ubuntu2.4_amd64.deb ... 2021-07-11T03:52:35.504 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2021-07-11T03:52:35.505 INFO:teuthology.orchestra.run.gibba005.stdout:Unpacking libssl-dev:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:35.543 INFO:teuthology.orchestra.run.gibba023.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-07-11T03:52:35.545 DEBUG:teuthology.parallel:result is None 2021-07-11T03:52:35.630 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2021-07-11T03:52:35.679 INFO:teuthology.orchestra.run.gibba019.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2021-07-11T03:52:35.755 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:35.857 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2021-07-11T03:52:35.969 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up flex (2.6.4-6.2) ... 2021-07-11T03:52:35.982 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:36.083 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up dump (0.4b46-6) ... 2021-07-11T03:52:36.093 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libssl1.1:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:36.164 INFO:teuthology.orchestra.run.gibba004.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-07-11T03:52:36.327 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2021-07-11T03:52:36.361 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up liblzo2-2:amd64 (2.10-2) ... 2021-07-11T03:52:36.368 INFO:teuthology.orchestra.run.gibba019.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-07-11T03:52:36.369 DEBUG:teuthology.parallel:result is None 2021-07-11T03:52:36.453 INFO:teuthology.orchestra.run.gibba004.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-07-11T03:52:36.487 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libaio-dev:amd64 (0.3.112-5) ... 2021-07-11T03:52:36.521 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up indent (2.2.12-1) ... 2021-07-11T03:52:36.646 INFO:teuthology.orchestra.run.gibba004.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:36.655 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libssl-dev:amd64 (1.1.1f-1ubuntu2.4) ... 2021-07-11T03:52:36.756 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libelf-dev:amd64 (0.176-1.1build1) ... 2021-07-11T03:52:36.773 INFO:teuthology.orchestra.run.gibba004.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-07-11T03:52:36.857 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libfl2:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:36.941 INFO:teuthology.orchestra.run.gibba004.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-07-11T03:52:37.000 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up dump (0.4b46-6) ... 2021-07-11T03:52:37.081 INFO:teuthology.orchestra.run.gibba005.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2021-07-11T03:52:37.142 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up bison (2:3.5.1+dfsg-1) ... 2021-07-11T03:52:37.234 INFO:teuthology.orchestra.run.gibba005.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2021-07-11T03:52:37.303 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up indent (2.2.12-1) ... 2021-07-11T03:52:37.462 INFO:teuthology.orchestra.run.gibba005.stdout:Setting up libfl-dev:amd64 (2.6.4-6.2) ... 2021-07-11T03:52:37.589 INFO:teuthology.orchestra.run.gibba005.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.2) ... 2021-07-11T03:52:37.757 INFO:teuthology.orchestra.run.gibba005.stdout:Processing triggers for man-db (2.9.1-1) ... 2021-07-11T03:52:38.152 INFO:teuthology.orchestra.run.gibba004.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2021-07-11T03:52:38.918 INFO:teuthology.orchestra.run.gibba004.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-07-11T03:52:38.920 DEBUG:teuthology.parallel:result is None 2021-07-11T03:52:39.201 INFO:teuthology.orchestra.run.gibba005.stdout:Processing triggers for install-info (6.7.0.dfsg.2-5) ... 2021-07-11T03:52:39.924 INFO:teuthology.orchestra.run.gibba005.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-07-11T03:52:39.927 DEBUG:teuthology.parallel:result is None 2021-07-11T03:52:39.928 DEBUG:teuthology.orchestra.run.gibba004:> sudo lsb_release -is 2021-07-11T03:52:39.956 INFO:teuthology.orchestra.run.gibba004.stdout:Ubuntu 2021-07-11T03:52:39.959 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:52:39.960 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=45c6565d67f116c9b3509842b718398a5b38cf73 2021-07-11T03:52:40.147 DEBUG:teuthology.orchestra.run.gibba004:> dpkg-query -W -f '${Version}' ceph 2021-07-11T03:52:40.192 INFO:teuthology.orchestra.run.gibba004.stdout:15.2.13-130-g45c6565d-1focal 2021-07-11T03:52:40.193 INFO:teuthology.packaging:The installed version of ceph is 15.2.13-130-g45c6565d-1focal 2021-07-11T03:52:40.193 INFO:teuthology.task.install:The correct ceph version 15.2.13-130-g45c6565d-1focal is installed. 2021-07-11T03:52:40.194 DEBUG:teuthology.orchestra.run.gibba005:> sudo lsb_release -is 2021-07-11T03:52:40.221 INFO:teuthology.orchestra.run.gibba005.stdout:Ubuntu 2021-07-11T03:52:40.224 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:52:40.224 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=45c6565d67f116c9b3509842b718398a5b38cf73 2021-07-11T03:52:40.410 DEBUG:teuthology.orchestra.run.gibba005:> dpkg-query -W -f '${Version}' ceph 2021-07-11T03:52:40.426 INFO:teuthology.orchestra.run.gibba005.stdout:15.2.13-130-g45c6565d-1focal 2021-07-11T03:52:40.426 INFO:teuthology.packaging:The installed version of ceph is 15.2.13-130-g45c6565d-1focal 2021-07-11T03:52:40.427 INFO:teuthology.task.install:The correct ceph version 15.2.13-130-g45c6565d-1focal is installed. 2021-07-11T03:52:40.427 DEBUG:teuthology.orchestra.run.gibba019:> sudo lsb_release -is 2021-07-11T03:52:40.455 INFO:teuthology.orchestra.run.gibba019.stdout:Ubuntu 2021-07-11T03:52:40.458 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:52:40.459 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=45c6565d67f116c9b3509842b718398a5b38cf73 2021-07-11T03:52:40.625 DEBUG:teuthology.orchestra.run.gibba019:> dpkg-query -W -f '${Version}' ceph 2021-07-11T03:52:40.670 INFO:teuthology.orchestra.run.gibba019.stdout:15.2.13-130-g45c6565d-1focal 2021-07-11T03:52:40.670 INFO:teuthology.packaging:The installed version of ceph is 15.2.13-130-g45c6565d-1focal 2021-07-11T03:52:40.671 INFO:teuthology.task.install:The correct ceph version 15.2.13-130-g45c6565d-1focal is installed. 2021-07-11T03:52:40.671 DEBUG:teuthology.orchestra.run.gibba023:> sudo lsb_release -is 2021-07-11T03:52:40.699 INFO:teuthology.orchestra.run.gibba023.stdout:Ubuntu 2021-07-11T03:52:40.702 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-07-11T03:52:40.703 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=45c6565d67f116c9b3509842b718398a5b38cf73 2021-07-11T03:52:40.867 DEBUG:teuthology.orchestra.run.gibba023:> dpkg-query -W -f '${Version}' ceph 2021-07-11T03:52:40.911 INFO:teuthology.orchestra.run.gibba023.stdout:15.2.13-130-g45c6565d-1focal 2021-07-11T03:52:40.911 INFO:teuthology.packaging:The installed version of ceph is 15.2.13-130-g45c6565d-1focal 2021-07-11T03:52:40.912 INFO:teuthology.task.install:The correct ceph version 15.2.13-130-g45c6565d-1focal is installed. 2021-07-11T03:52:40.912 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-07-11T03:52:40.913 DEBUG:teuthology.orchestra.run.gibba004:> set -ex 2021-07-11T03:52:40.913 DEBUG:teuthology.orchestra.run.gibba004:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-07-11T03:52:40.925 DEBUG:teuthology.orchestra.run.gibba005:> set -ex 2021-07-11T03:52:40.926 DEBUG:teuthology.orchestra.run.gibba005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-07-11T03:52:40.937 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-07-11T03:52:40.937 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-07-11T03:52:40.949 DEBUG:teuthology.orchestra.run.gibba023:> set -ex 2021-07-11T03:52:40.949 DEBUG:teuthology.orchestra.run.gibba023:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-07-11T03:52:40.970 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-07-11T03:52:40.971 DEBUG:teuthology.orchestra.run.gibba004:> set -ex 2021-07-11T03:52:40.971 DEBUG:teuthology.orchestra.run.gibba004:> sudo dd of=/usr/bin/daemon-helper 2021-07-11T03:52:40.979 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-07-11T03:52:40.989 DEBUG:teuthology.orchestra.run.gibba005:> set -ex 2021-07-11T03:52:40.989 DEBUG:teuthology.orchestra.run.gibba005:> sudo dd of=/usr/bin/daemon-helper 2021-07-11T03:52:40.999 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-07-11T03:52:41.011 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-07-11T03:52:41.012 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/usr/bin/daemon-helper 2021-07-11T03:52:41.023 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-07-11T03:52:41.037 DEBUG:teuthology.orchestra.run.gibba023:> set -ex 2021-07-11T03:52:41.037 DEBUG:teuthology.orchestra.run.gibba023:> sudo dd of=/usr/bin/daemon-helper 2021-07-11T03:52:41.059 DEBUG:teuthology.orchestra.run.gibba023:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-07-11T03:52:41.081 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-07-11T03:52:41.081 DEBUG:teuthology.orchestra.run.gibba004:> set -ex 2021-07-11T03:52:41.082 DEBUG:teuthology.orchestra.run.gibba004:> sudo dd of=/usr/bin/adjust-ulimits 2021-07-11T03:52:41.104 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-07-11T03:52:41.125 DEBUG:teuthology.orchestra.run.gibba005:> set -ex 2021-07-11T03:52:41.125 DEBUG:teuthology.orchestra.run.gibba005:> sudo dd of=/usr/bin/adjust-ulimits 2021-07-11T03:52:41.147 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-07-11T03:52:41.210 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-07-11T03:52:41.211 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/usr/bin/adjust-ulimits 2021-07-11T03:52:41.233 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-07-11T03:52:41.255 DEBUG:teuthology.orchestra.run.gibba023:> set -ex 2021-07-11T03:52:41.255 DEBUG:teuthology.orchestra.run.gibba023:> sudo dd of=/usr/bin/adjust-ulimits 2021-07-11T03:52:41.267 DEBUG:teuthology.orchestra.run.gibba023:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-07-11T03:52:41.273 INFO:teuthology.run_tasks:Running task ceph... 2021-07-11T03:52:41.333 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2021-07-11T03:52:41.334 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 777 /var/log/ceph 2021-07-11T03:52:41.338 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 777 /var/log/ceph 2021-07-11T03:52:41.341 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod 777 /var/log/ceph 2021-07-11T03:52:41.344 DEBUG:teuthology.orchestra.run.gibba023:> sudo chmod 777 /var/log/ceph 2021-07-11T03:52:41.359 INFO:tasks.ceph:Disabling ceph logrotate... 2021-07-11T03:52:41.359 DEBUG:teuthology.orchestra.run.gibba004:> sudo rm -f -- /etc/logrotate.d/ceph 2021-07-11T03:52:41.391 DEBUG:teuthology.orchestra.run.gibba005:> sudo rm -f -- /etc/logrotate.d/ceph 2021-07-11T03:52:41.404 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm -f -- /etc/logrotate.d/ceph 2021-07-11T03:52:41.408 DEBUG:teuthology.orchestra.run.gibba023:> sudo rm -f -- /etc/logrotate.d/ceph 2021-07-11T03:52:41.422 INFO:tasks.ceph:Creating extra log directories... 2021-07-11T03:52:41.423 DEBUG:teuthology.orchestra.run.gibba004:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-07-11T03:52:41.426 DEBUG:teuthology.orchestra.run.gibba005:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-07-11T03:52:41.430 DEBUG:teuthology.orchestra.run.gibba019:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-07-11T03:52:41.457 DEBUG:teuthology.orchestra.run.gibba023:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-07-11T03:52:41.479 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2021-07-11T03:52:41.480 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2021-07-11T03:52:41.480 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2021-07-11T03:52:41.481 DEBUG:teuthology.orchestra.run.gibba004:> set -ex 2021-07-11T03:52:41.481 DEBUG:teuthology.orchestra.run.gibba004:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-07-11T03:52:41.486 DEBUG:teuthology.orchestra.run.gibba004:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-07-11T03:52:41.546 DEBUG:teuthology.orchestra.run.gibba005:> set -ex 2021-07-11T03:52:41.547 DEBUG:teuthology.orchestra.run.gibba005:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-07-11T03:52:41.557 DEBUG:teuthology.orchestra.run.gibba005:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-07-11T03:52:41.577 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-07-11T03:52:41.577 DEBUG:teuthology.orchestra.run.gibba019:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-07-11T03:52:41.586 DEBUG:teuthology.orchestra.run.gibba019:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-07-11T03:52:41.647 DEBUG:teuthology.orchestra.run.gibba023:> set -ex 2021-07-11T03:52:41.648 DEBUG:teuthology.orchestra.run.gibba023:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2021-07-11T03:52:41.657 DEBUG:teuthology.orchestra.run.gibba023:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2021-07-11T03:52:41.677 INFO:tasks.ceph:Creating ceph cluster ceph... 2021-07-11T03:52:41.677 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-07-11T03:52:41.678 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912', 'branch': 'octopus', 'description': 'fs/multiclient/{begin clusters/1-mds-2-client conf/{client mds mon osd} distros/ubuntu_latest mount/fuse objectstore-ec/bluestore-comp overrides/{frag_enable whitelist_health whitelist_wrongly_marked_down} tasks/cephfs_misc_tests}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '6260912', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'gibba', 'name': 'teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 30}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'octopus'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'POOL_APP_NOT_ENABLED', 'has not responded to cap revoke by MDS for over', 'MDS_CLIENT_LATE_RELEASE', 'responding to mclientcaps'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '45c6565d67f116c9b3509842b718398a5b38cf73'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'sha1': '45c6565d67f116c9b3509842b718398a5b38cf73'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'octopus', 'sha1': '54bff90266c0649369e90717e0c4df7b0276c9e1'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 1000, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'], ['client.0'], ['client.1']], 'sha1': '45c6565d67f116c9b3509842b718398a5b38cf73', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'octopus', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '54bff90266c0649369e90717e0c4df7b0276c9e1', 'targets': {'gibba004.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDD3TbESRYFDs/8GarIwJDMF3tPDJSdI6bDNoa57O3koX+lAtyKu7F09Dt1O5VtJTTnQ3OMXwoajpGcw+pIL92LIcAXvtUbeHzrjrUTY8QjTVC2oqvP8oukXlqjyx4tziMrRNHMM9qSWXSWp289KBb3ObMSpnSirOxnYZIYTDE5/JoLdWHG6FUGv5TA5LJBv8TH5xUUJCSNJu/rLzDXagOAF4ZJej7Mo3ikG7USyY6ZszbCgW3hf6deYoDIwzF22dKCpegtkpseMcczs7m+cV1b6O2cfH9CS1os078OGa9bYzSQwG4s1voAyfo/6m5vhCpMxHpAxkjnXgGPWKsXAqvecf3/0r5UotnQTEUzXqfA3eggQtgHTA6jQ0LL29BU3ODn26H82e7Oam9gFSVnm3SnF0sEtC1alfTQJzSKyaUqCoDzk4lom6m5mfqDQp4JLCR4dKsIc+Nh4G2AIB6/e7mUd9QH1pwnLwrMXa7OSoNUAVG5O6tpNH8QL7vTXVoowFc=', 'gibba005.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDaso08m3kPXqiogDT2IoHRT9dVQztV03/TfKLvNRXq0ZUP0gbYBb+7QSfV17jfwNmCnuZzq6Gin13X9IMQNhTnc+PrvWLqwKjKyghkD4xF+4ufDYcbpOeOtth8hW0zF5WCG5KBbNfK7JD018NSanXH3IaLHZV6G0OSCY+V6YJLMm+zxZjXDGhaC1smj7/W2u0M7Cn1HSM2owjqariWxNRwgbz7fEZ7XWXd5TIEEH+LS+N9dni0U7rMkvtQb3pxeMpYjrDL/rNyaUwGiOq2oHiLJdkyxx5XetUlQF23pDTBDpKxoQVfh2bXY4ehnmTKam8yBMsnCabRM2VfABG/AxG/pFt/roOtBHGKF4WZ8wiIQJxZIQd1g0hA3CDa0xZXKagCEwX34o3/3ofN1yqJ9CFHVBthU22voVuEbd7VzRw2LEMEmsJ83f/xZX//1vATfS5AmxxXG3dCadCNI4D5k8K5Ibgc7qmuPnyWaUbmURpTc1KSBTxaLwD2VgueKkxPg28=', 'gibba019.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOM8WP6oMGlmnKvf0fkpXhjYzZceJRPDGXBi8WTY0194xZLnJKMlpsfLJ1e33MIbhpTM1U4EawIbrp6ZwUyTspt0JJuHpVFWQbMmsV5/DCaJ+kI0hJ5oAv61fYpuio2B15cOa3INia01tR/eGCzKZG81DPhiXvY7IlEFGGGN7QcX0IiMwd7MQgn1KvbbG1EGsCOX9TtH7d2hU4y+tQZFycAvFgh+3PnFoM8+5V9psiXGZ5wl5ikdcFRG641nkkohcTFVB9Xyy9oAfKiZ4SXoAs4/fzQfBuotNOTr8whnJfssvmokFyHML0OOUPvHsbry3B9XKzt0yNDS8qOYJF1vpA8E4bOvl5uBqoun2EOY78AW1dKipuLHxVIdl4yWBFEOHl0Au3CN1AbbJbhdZlIGtoWF6axGzpE25NQxpYseMpfR2oOi20r3T5Wn9/AnWg5jBeQ19HFEz5YA+WI+Gdkml7g1yepnl0FH4pVf2ZOahX6nZGko+w1wRiaS3uDMlSMQM=', 'gibba023.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDUBbOohbK0ZG/bmINdjKAXvH7VlJ7MPrBb1K1+9bxh2Wse+O+/EuO9Ii/ET6Q2UlRXu4r1SXNw3ibsXHam5JyF/RMHMkXU4va4+0lO0Vg+EG/xd+N0LMRXmXgVqygUJuD1iMr5y85vpoJfpiURQQ6eBq3ohE+pC8NB8F6U6e6mfQePXo5amtUwc/Cd02o/uv6L7WgwDBpz4IEcLNPUgs9RuhBdNxHAOTSOFdiriJdcSHDQ+mYrK8qvoj/tdqqZlblbiGVaFtfLzRFFIRHJPYXaE65/Eu8onBaSBxK01bSxdAWDUkOqW3dbfus1RIP0UAXWZTd6YQLMbtZHCcouUNNLFiXeCjclFc2RnMTBmu/YwwKayxT2MH2E5FbPXq4KG/vfMgf+n87QY6bHuNF4yL8j3XUjVnCzQ25fPGjRvMIiJ6zrT0va7PL9f1WhBE7FellcoeIUs9LRIKAnF6jg9d1oOkJQjnuQqp9lyDSlfK6nDJzJSmk6ybJzqx82u+6hims='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': '45c6565d67f116c9b3509842b718398a5b38cf73'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_misc']}}], 'teuthology_branch': 'master', 'teuthology_sha1': '24e3280d7d1fc156628af95b24b27a29e38b2766', 'timestamp': '2021-07-09_04:10:02', 'tube': 'gibba', 'user': 'teuthology', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.gibba.8273'} 2021-07-11T03:52:41.678 DEBUG:teuthology.orchestra.run.gibba004:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-07-11T03:52:41.682 DEBUG:teuthology.orchestra.run.gibba005:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-07-11T03:52:41.685 DEBUG:teuthology.orchestra.run.gibba019:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-07-11T03:52:41.687 DEBUG:teuthology.orchestra.run.gibba023:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2021-07-11T03:52:41.692 DEBUG:teuthology.orchestra.run.gibba004:> sudo install -d -m0777 -- /var/run/ceph 2021-07-11T03:52:41.735 DEBUG:teuthology.orchestra.run.gibba005:> sudo install -d -m0777 -- /var/run/ceph 2021-07-11T03:52:41.738 DEBUG:teuthology.orchestra.run.gibba019:> sudo install -d -m0777 -- /var/run/ceph 2021-07-11T03:52:41.742 DEBUG:teuthology.orchestra.run.gibba023:> sudo install -d -m0777 -- /var/run/ceph 2021-07-11T03:52:41.763 DEBUG:teuthology.orchestra.run.gibba004:> set -ex 2021-07-11T03:52:41.764 DEBUG:teuthology.orchestra.run.gibba004:> dd if=/scratch_devs of=/dev/stdout 2021-07-11T03:52:41.773 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-07-11T03:52:41.774 DEBUG:teuthology.orchestra.run.gibba004:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-07-11T03:52:41.826 INFO:teuthology.orchestra.run.gibba004.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-07-11T03:52:41.826 INFO:teuthology.orchestra.run.gibba004.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-07-11T03:52:41.826 INFO:teuthology.orchestra.run.gibba004.stdout:Device: 6h/6d Inode: 346 Links: 1 2021-07-11T03:52:41.827 INFO:teuthology.orchestra.run.gibba004.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-07-11T03:52:41.827 INFO:teuthology.orchestra.run.gibba004.stdout:Access: 2021-07-11 03:49:36.436171771 +0000 2021-07-11T03:52:41.827 INFO:teuthology.orchestra.run.gibba004.stdout:Modify: 2021-07-11 03:49:36.252171776 +0000 2021-07-11T03:52:41.828 INFO:teuthology.orchestra.run.gibba004.stdout:Change: 2021-07-11 03:49:36.252171776 +0000 2021-07-11T03:52:41.828 INFO:teuthology.orchestra.run.gibba004.stdout: Birth: - 2021-07-11T03:52:41.834 INFO:teuthology.orchestra.run.gibba004.stderr:1+0 records in 2021-07-11T03:52:41.834 INFO:teuthology.orchestra.run.gibba004.stderr:1+0 records out 2021-07-11T03:52:41.834 INFO:teuthology.orchestra.run.gibba004.stderr:512 bytes copied, 0.000255039 s, 2.0 MB/s 2021-07-11T03:52:41.836 DEBUG:teuthology.orchestra.run.gibba004:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-07-11T03:52:41.839 INFO:teuthology.orchestra.run.gibba004.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-07-11T03:52:41.840 INFO:teuthology.orchestra.run.gibba004.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-07-11T03:52:41.840 INFO:teuthology.orchestra.run.gibba004.stdout:Device: 6h/6d Inode: 351 Links: 1 2021-07-11T03:52:41.841 INFO:teuthology.orchestra.run.gibba004.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-07-11T03:52:41.841 INFO:teuthology.orchestra.run.gibba004.stdout:Access: 2021-07-11 03:49:36.752171763 +0000 2021-07-11T03:52:41.841 INFO:teuthology.orchestra.run.gibba004.stdout:Modify: 2021-07-11 03:49:36.644171766 +0000 2021-07-11T03:52:41.841 INFO:teuthology.orchestra.run.gibba004.stdout:Change: 2021-07-11 03:49:36.644171766 +0000 2021-07-11T03:52:41.842 INFO:teuthology.orchestra.run.gibba004.stdout: Birth: - 2021-07-11T03:52:41.844 INFO:teuthology.orchestra.run.gibba004.stderr:1+0 records in 2021-07-11T03:52:41.845 INFO:teuthology.orchestra.run.gibba004.stderr:1+0 records out 2021-07-11T03:52:41.845 INFO:teuthology.orchestra.run.gibba004.stderr:512 bytes copied, 0.000219115 s, 2.3 MB/s 2021-07-11T03:52:41.845 DEBUG:teuthology.orchestra.run.gibba004:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-07-11T03:52:41.850 INFO:teuthology.orchestra.run.gibba004.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-07-11T03:52:41.850 INFO:teuthology.orchestra.run.gibba004.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-07-11T03:52:41.851 INFO:teuthology.orchestra.run.gibba004.stdout:Device: 6h/6d Inode: 359 Links: 1 2021-07-11T03:52:41.851 INFO:teuthology.orchestra.run.gibba004.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-07-11T03:52:41.852 INFO:teuthology.orchestra.run.gibba004.stdout:Access: 2021-07-11 03:49:37.128171753 +0000 2021-07-11T03:52:41.852 INFO:teuthology.orchestra.run.gibba004.stdout:Modify: 2021-07-11 03:49:37.000171757 +0000 2021-07-11T03:52:41.852 INFO:teuthology.orchestra.run.gibba004.stdout:Change: 2021-07-11 03:49:37.000171757 +0000 2021-07-11T03:52:41.852 INFO:teuthology.orchestra.run.gibba004.stdout: Birth: - 2021-07-11T03:52:41.853 INFO:teuthology.orchestra.run.gibba004.stderr:1+0 records in 2021-07-11T03:52:41.853 INFO:teuthology.orchestra.run.gibba004.stderr:1+0 records out 2021-07-11T03:52:41.854 INFO:teuthology.orchestra.run.gibba004.stderr:512 bytes copied, 0.000171007 s, 3.0 MB/s 2021-07-11T03:52:41.855 DEBUG:teuthology.orchestra.run.gibba004:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-07-11T03:52:41.903 INFO:teuthology.orchestra.run.gibba004.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-07-11T03:52:41.904 INFO:teuthology.orchestra.run.gibba004.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-07-11T03:52:41.904 INFO:teuthology.orchestra.run.gibba004.stdout:Device: 6h/6d Inode: 364 Links: 1 2021-07-11T03:52:41.904 INFO:teuthology.orchestra.run.gibba004.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-07-11T03:52:41.905 INFO:teuthology.orchestra.run.gibba004.stdout:Access: 2021-07-11 03:49:37.492171744 +0000 2021-07-11T03:52:41.905 INFO:teuthology.orchestra.run.gibba004.stdout:Modify: 2021-07-11 03:49:37.368171747 +0000 2021-07-11T03:52:41.906 INFO:teuthology.orchestra.run.gibba004.stdout:Change: 2021-07-11 03:49:37.368171747 +0000 2021-07-11T03:52:41.906 INFO:teuthology.orchestra.run.gibba004.stdout: Birth: - 2021-07-11T03:52:41.915 INFO:teuthology.orchestra.run.gibba004.stderr:1+0 records in 2021-07-11T03:52:41.916 INFO:teuthology.orchestra.run.gibba004.stderr:1+0 records out 2021-07-11T03:52:41.916 INFO:teuthology.orchestra.run.gibba004.stderr:512 bytes copied, 0.00049594 s, 1.0 MB/s 2021-07-11T03:52:41.922 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-07-11T03:52:41.922 DEBUG:teuthology.orchestra.run.gibba005:> set -ex 2021-07-11T03:52:41.923 DEBUG:teuthology.orchestra.run.gibba005:> dd if=/scratch_devs of=/dev/stdout 2021-07-11T03:52:41.933 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-07-11T03:52:41.934 DEBUG:teuthology.orchestra.run.gibba005:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-07-11T03:52:41.982 INFO:teuthology.orchestra.run.gibba005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-07-11T03:52:41.982 INFO:teuthology.orchestra.run.gibba005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-07-11T03:52:41.983 INFO:teuthology.orchestra.run.gibba005.stdout:Device: 6h/6d Inode: 339 Links: 1 2021-07-11T03:52:41.983 INFO:teuthology.orchestra.run.gibba005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-07-11T03:52:41.984 INFO:teuthology.orchestra.run.gibba005.stdout:Access: 2021-07-11 03:49:35.552632891 +0000 2021-07-11T03:52:41.984 INFO:teuthology.orchestra.run.gibba005.stdout:Modify: 2021-07-11 03:49:35.376632896 +0000 2021-07-11T03:52:41.984 INFO:teuthology.orchestra.run.gibba005.stdout:Change: 2021-07-11 03:49:35.376632896 +0000 2021-07-11T03:52:41.984 INFO:teuthology.orchestra.run.gibba005.stdout: Birth: - 2021-07-11T03:52:41.986 INFO:teuthology.orchestra.run.gibba005.stderr:1+0 records in 2021-07-11T03:52:41.986 INFO:teuthology.orchestra.run.gibba005.stderr:1+0 records out 2021-07-11T03:52:41.986 INFO:teuthology.orchestra.run.gibba005.stderr:512 bytes copied, 0.000216983 s, 2.4 MB/s 2021-07-11T03:52:41.988 DEBUG:teuthology.orchestra.run.gibba005:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-07-11T03:52:41.993 INFO:teuthology.orchestra.run.gibba005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-07-11T03:52:41.993 INFO:teuthology.orchestra.run.gibba005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-07-11T03:52:41.994 INFO:teuthology.orchestra.run.gibba005.stdout:Device: 6h/6d Inode: 350 Links: 1 2021-07-11T03:52:41.994 INFO:teuthology.orchestra.run.gibba005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-07-11T03:52:41.994 INFO:teuthology.orchestra.run.gibba005.stdout:Access: 2021-07-11 03:49:35.944632881 +0000 2021-07-11T03:52:41.994 INFO:teuthology.orchestra.run.gibba005.stdout:Modify: 2021-07-11 03:49:35.888632883 +0000 2021-07-11T03:52:41.995 INFO:teuthology.orchestra.run.gibba005.stdout:Change: 2021-07-11 03:49:35.888632883 +0000 2021-07-11T03:52:41.995 INFO:teuthology.orchestra.run.gibba005.stdout: Birth: - 2021-07-11T03:52:41.998 INFO:teuthology.orchestra.run.gibba005.stderr:1+0 records in 2021-07-11T03:52:41.999 INFO:teuthology.orchestra.run.gibba005.stderr:1+0 records out 2021-07-11T03:52:41.999 INFO:teuthology.orchestra.run.gibba005.stderr:512 bytes copied, 0.000306578 s, 1.7 MB/s 2021-07-11T03:52:42.002 DEBUG:teuthology.orchestra.run.gibba005:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-07-11T03:52:42.009 INFO:teuthology.orchestra.run.gibba005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-07-11T03:52:42.009 INFO:teuthology.orchestra.run.gibba005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-07-11T03:52:42.010 INFO:teuthology.orchestra.run.gibba005.stdout:Device: 6h/6d Inode: 353 Links: 1 2021-07-11T03:52:42.010 INFO:teuthology.orchestra.run.gibba005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-07-11T03:52:42.010 INFO:teuthology.orchestra.run.gibba005.stdout:Access: 2021-07-11 03:49:36.352632871 +0000 2021-07-11T03:52:42.011 INFO:teuthology.orchestra.run.gibba005.stdout:Modify: 2021-07-11 03:49:36.300632872 +0000 2021-07-11T03:52:42.011 INFO:teuthology.orchestra.run.gibba005.stdout:Change: 2021-07-11 03:49:36.300632872 +0000 2021-07-11T03:52:42.011 INFO:teuthology.orchestra.run.gibba005.stdout: Birth: - 2021-07-11T03:52:42.015 INFO:teuthology.orchestra.run.gibba005.stderr:1+0 records in 2021-07-11T03:52:42.016 INFO:teuthology.orchestra.run.gibba005.stderr:1+0 records out 2021-07-11T03:52:42.016 INFO:teuthology.orchestra.run.gibba005.stderr:512 bytes copied, 0.000220338 s, 2.3 MB/s 2021-07-11T03:52:42.018 DEBUG:teuthology.orchestra.run.gibba005:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-07-11T03:52:42.066 INFO:teuthology.orchestra.run.gibba005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-07-11T03:52:42.067 INFO:teuthology.orchestra.run.gibba005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-07-11T03:52:42.067 INFO:teuthology.orchestra.run.gibba005.stdout:Device: 6h/6d Inode: 361 Links: 1 2021-07-11T03:52:42.068 INFO:teuthology.orchestra.run.gibba005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-07-11T03:52:42.068 INFO:teuthology.orchestra.run.gibba005.stdout:Access: 2021-07-11 03:49:36.696632862 +0000 2021-07-11T03:52:42.068 INFO:teuthology.orchestra.run.gibba005.stdout:Modify: 2021-07-11 03:49:36.552632865 +0000 2021-07-11T03:52:42.069 INFO:teuthology.orchestra.run.gibba005.stdout:Change: 2021-07-11 03:49:36.552632865 +0000 2021-07-11T03:52:42.069 INFO:teuthology.orchestra.run.gibba005.stdout: Birth: - 2021-07-11T03:52:42.071 INFO:teuthology.orchestra.run.gibba005.stderr:1+0 records in 2021-07-11T03:52:42.071 INFO:teuthology.orchestra.run.gibba005.stderr:1+0 records out 2021-07-11T03:52:42.071 INFO:teuthology.orchestra.run.gibba005.stderr:512 bytes copied, 0.000219669 s, 2.3 MB/s 2021-07-11T03:52:42.073 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-07-11T03:52:42.074 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@gibba004.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@gibba005.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-07-11T03:52:42.074 INFO:tasks.ceph:Generating config... 2021-07-11T03:52:42.076 INFO:tasks.ceph:[client] client mount timeout = 600 2021-07-11T03:52:42.076 INFO:tasks.ceph:[client] debug client = 20 2021-07-11T03:52:42.076 INFO:tasks.ceph:[client] debug ms = 1 2021-07-11T03:52:42.077 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2021-07-11T03:52:42.077 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2021-07-11T03:52:42.077 INFO:tasks.ceph:[mds] debug mds = 20 2021-07-11T03:52:42.078 INFO:tasks.ceph:[mds] debug ms = 1 2021-07-11T03:52:42.078 INFO:tasks.ceph:[mds] mds bal frag = True 2021-07-11T03:52:42.078 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2021-07-11T03:52:42.079 INFO:tasks.ceph:[mds] mds bal merge size = 5 2021-07-11T03:52:42.079 INFO:tasks.ceph:[mds] mds bal split bits = 3 2021-07-11T03:52:42.079 INFO:tasks.ceph:[mds] mds bal split size = 100 2021-07-11T03:52:42.080 INFO:tasks.ceph:[mds] mds debug frag = True 2021-07-11T03:52:42.080 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2021-07-11T03:52:42.080 INFO:tasks.ceph:[mds] mds op complaint time = 180 2021-07-11T03:52:42.080 INFO:tasks.ceph:[mds] mds verify scatter = True 2021-07-11T03:52:42.081 INFO:tasks.ceph:[mds] osd op complaint time = 180 2021-07-11T03:52:42.081 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2021-07-11T03:52:42.081 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2021-07-11T03:52:42.082 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-07-11T03:52:42.082 INFO:tasks.ceph:[mgr] debug ms = 1 2021-07-11T03:52:42.082 INFO:tasks.ceph:[mon] debug mon = 20 2021-07-11T03:52:42.083 INFO:tasks.ceph:[mon] debug ms = 1 2021-07-11T03:52:42.083 INFO:tasks.ceph:[mon] debug paxos = 20 2021-07-11T03:52:42.083 INFO:tasks.ceph:[mon] mon op complaint time = 120 2021-07-11T03:52:42.084 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-07-11T03:52:42.084 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2021-07-11T03:52:42.084 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-07-11T03:52:42.084 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-07-11T03:52:42.085 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-07-11T03:52:42.085 INFO:tasks.ceph:[osd] debug ms = 1 2021-07-11T03:52:42.085 INFO:tasks.ceph:[osd] debug osd = 20 2021-07-11T03:52:42.086 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-07-11T03:52:42.086 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-07-11T03:52:42.086 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-07-11T03:52:42.086 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-07-11T03:52:42.087 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-07-11T03:52:42.087 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-07-11T03:52:42.087 INFO:tasks.ceph:[osd] osd op complaint time = 180 2021-07-11T03:52:42.088 INFO:tasks.ceph:Setting up mon.a... 2021-07-11T03:52:42.088 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2021-07-11T03:52:42.118 INFO:teuthology.orchestra.run.gibba004.stdout:creating /etc/ceph/ceph.keyring 2021-07-11T03:52:42.121 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2021-07-11T03:52:42.183 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-07-11T03:52:42.189 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.2.104'), ('mon.b', '172.21.2.105'), ('mon.c', '[v2:172.21.2.105:3301,v1:172.21.2.105:6790]')] 2021-07-11T03:52:42.190 DEBUG:teuthology.orchestra.run.gibba004:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.2.104 --add b 172.21.2.105 --addv c '[v2:172.21.2.105:3301,v1:172.21.2.105:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:42.210 INFO:teuthology.orchestra.run.gibba004.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:42.210 INFO:teuthology.orchestra.run.gibba004.stdout:monmaptool: generated fsid 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T03:52:42.210 INFO:teuthology.orchestra.run.gibba004.stdout:epoch 0 2021-07-11T03:52:42.211 INFO:teuthology.orchestra.run.gibba004.stdout:fsid 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T03:52:42.211 INFO:teuthology.orchestra.run.gibba004.stdout:last_changed 2021-07-11T03:52:42.210013+0000 2021-07-11T03:52:42.211 INFO:teuthology.orchestra.run.gibba004.stdout:created 2021-07-11T03:52:42.210013+0000 2021-07-11T03:52:42.211 INFO:teuthology.orchestra.run.gibba004.stdout:min_mon_release 0 (unknown) 2021-07-11T03:52:42.212 INFO:teuthology.orchestra.run.gibba004.stdout:0: [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] mon.a 2021-07-11T03:52:42.212 INFO:teuthology.orchestra.run.gibba004.stdout:1: [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] mon.b 2021-07-11T03:52:42.212 INFO:teuthology.orchestra.run.gibba004.stdout:2: [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] mon.c 2021-07-11T03:52:42.212 INFO:teuthology.orchestra.run.gibba004.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2021-07-11T03:52:42.214 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 57c05e2e-f3b6-453c-878e-145bef41bd6c... 2021-07-11T03:52:42.215 DEBUG:teuthology.orchestra.run.gibba004:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T03:52:42.259 DEBUG:teuthology.orchestra.run.gibba005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T03:52:42.262 DEBUG:teuthology.orchestra.run.gibba019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T03:52:42.266 DEBUG:teuthology.orchestra.run.gibba023:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T03:52:42.274 INFO:teuthology.orchestra.run.gibba004.stdout:[global] 2021-07-11T03:52:42.275 INFO:teuthology.orchestra.run.gibba004.stdout: chdir = "" 2021-07-11T03:52:42.275 INFO:teuthology.orchestra.run.gibba004.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T03:52:42.275 INFO:teuthology.orchestra.run.gibba004.stdout: auth supported = cephx 2021-07-11T03:52:42.276 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.276 INFO:teuthology.orchestra.run.gibba004.stdout: filestore xattr use omap = true 2021-07-11T03:52:42.277 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.277 INFO:teuthology.orchestra.run.gibba004.stdout: mon clock drift allowed = 1.000 2021-07-11T03:52:42.277 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.277 INFO:teuthology.orchestra.run.gibba004.stdout: osd crush chooseleaf type = 0 2021-07-11T03:52:42.278 INFO:teuthology.orchestra.run.gibba004.stdout: auth debug = true 2021-07-11T03:52:42.278 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.278 INFO:teuthology.orchestra.run.gibba004.stdout: ms die on old message = true 2021-07-11T03:52:42.278 INFO:teuthology.orchestra.run.gibba004.stdout: ms die on bug = true 2021-07-11T03:52:42.279 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.279 INFO:teuthology.orchestra.run.gibba004.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T03:52:42.279 INFO:teuthology.orchestra.run.gibba004.stdout: mon pg warn max object skew = 0 2021-07-11T03:52:42.279 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.279 INFO:teuthology.orchestra.run.gibba004.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T03:52:42.280 INFO:teuthology.orchestra.run.gibba004.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T03:52:42.280 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.280 INFO:teuthology.orchestra.run.gibba004.stdout: osd pool default size = 2 2021-07-11T03:52:42.280 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.280 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd allow primary affinity = true 2021-07-11T03:52:42.281 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd allow pg remap = true 2021-07-11T03:52:42.281 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on legacy crush tunables = false 2021-07-11T03:52:42.281 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on crush straw calc version zero = false 2021-07-11T03:52:42.281 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on no sortbitwise = false 2021-07-11T03:52:42.281 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on osd down out interval zero = false 2021-07-11T03:52:42.282 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on too few osds = false 2021-07-11T03:52:42.282 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T03:52:42.282 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T03:52:42.282 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.283 INFO:teuthology.orchestra.run.gibba004.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T03:52:42.283 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.283 INFO:teuthology.orchestra.run.gibba004.stdout: osd default data pool replay window = 5 2021-07-11T03:52:42.283 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.283 INFO:teuthology.orchestra.run.gibba004.stdout: mon allow pool delete = true 2021-07-11T03:52:42.284 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.284 INFO:teuthology.orchestra.run.gibba004.stdout: mon cluster log file level = debug 2021-07-11T03:52:42.284 INFO:teuthology.orchestra.run.gibba004.stdout: debug asserts on shutdown = true 2021-07-11T03:52:42.284 INFO:teuthology.orchestra.run.gibba004.stdout: mon health detail to clog = false 2021-07-11T03:52:42.284 INFO:teuthology.orchestra.run.gibba004.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T03:52:42.285 INFO:teuthology.orchestra.run.gibba004.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T03:52:42.285 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.285 INFO:teuthology.orchestra.run.gibba004.stdout:[osd] 2021-07-11T03:52:42.285 INFO:teuthology.orchestra.run.gibba004.stdout: osd journal size = 100 2021-07-11T03:52:42.286 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.286 INFO:teuthology.orchestra.run.gibba004.stdout: osd scrub load threshold = 5.0 2021-07-11T03:52:42.286 INFO:teuthology.orchestra.run.gibba004.stdout: osd scrub max interval = 600 2021-07-11T03:52:42.286 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.286 INFO:teuthology.orchestra.run.gibba004.stdout: osd recover clone overlap = true 2021-07-11T03:52:42.287 INFO:teuthology.orchestra.run.gibba004.stdout: osd recovery max chunk = 1048576 2021-07-11T03:52:42.287 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.287 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug shutdown = true 2021-07-11T03:52:42.287 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug op order = true 2021-07-11T03:52:42.288 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug verify stray on activate = true 2021-07-11T03:52:42.288 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.288 INFO:teuthology.orchestra.run.gibba004.stdout: osd open classes on start = true 2021-07-11T03:52:42.288 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug pg log writeout = true 2021-07-11T03:52:42.288 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.289 INFO:teuthology.orchestra.run.gibba004.stdout: osd deep scrub update digest min age = 30 2021-07-11T03:52:42.289 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.289 INFO:teuthology.orchestra.run.gibba004.stdout: osd map max advance = 10 2021-07-11T03:52:42.289 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.289 INFO:teuthology.orchestra.run.gibba004.stdout: journal zero on create = true 2021-07-11T03:52:42.290 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.290 INFO:teuthology.orchestra.run.gibba004.stdout: filestore ondisk finisher threads = 3 2021-07-11T03:52:42.290 INFO:teuthology.orchestra.run.gibba004.stdout: filestore apply finisher threads = 3 2021-07-11T03:52:42.290 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.290 INFO:teuthology.orchestra.run.gibba004.stdout: bdev debug aio = true 2021-07-11T03:52:42.291 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug misdirected ops = true 2021-07-11T03:52:42.291 INFO:teuthology.orchestra.run.gibba004.stdout: bluestore block size = 96636764160 2021-07-11T03:52:42.291 INFO:teuthology.orchestra.run.gibba004.stdout: bluestore compression mode = aggressive 2021-07-11T03:52:42.291 INFO:teuthology.orchestra.run.gibba004.stdout: bluestore fsck on mount = True 2021-07-11T03:52:42.292 INFO:teuthology.orchestra.run.gibba004.stdout: debug bluefs = 20 2021-07-11T03:52:42.292 INFO:teuthology.orchestra.run.gibba004.stdout: debug bluestore = 20 2021-07-11T03:52:42.292 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T03:52:42.292 INFO:teuthology.orchestra.run.gibba004.stdout: debug osd = 20 2021-07-11T03:52:42.292 INFO:teuthology.orchestra.run.gibba004.stdout: debug rocksdb = 10 2021-07-11T03:52:42.293 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T03:52:42.293 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd full ratio = 0.9 2021-07-11T03:52:42.293 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd nearfull ratio = 0.8 2021-07-11T03:52:42.293 INFO:teuthology.orchestra.run.gibba004.stdout: osd failsafe full ratio = 0.95 2021-07-11T03:52:42.293 INFO:teuthology.orchestra.run.gibba004.stdout: osd objectstore = bluestore 2021-07-11T03:52:42.294 INFO:teuthology.orchestra.run.gibba004.stdout: osd op complaint time = 180 2021-07-11T03:52:42.294 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.294 INFO:teuthology.orchestra.run.gibba004.stdout:[mgr] 2021-07-11T03:52:42.294 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T03:52:42.295 INFO:teuthology.orchestra.run.gibba004.stdout: debug mgr = 20 2021-07-11T03:52:42.295 INFO:teuthology.orchestra.run.gibba004.stdout: debug mon = 20 2021-07-11T03:52:42.295 INFO:teuthology.orchestra.run.gibba004.stdout: debug auth = 20 2021-07-11T03:52:42.295 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min pgs per osd = 4 2021-07-11T03:52:42.295 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min bytes per osd = 10 2021-07-11T03:52:42.296 INFO:teuthology.orchestra.run.gibba004.stdout: mgr/telemetry/nag = false 2021-07-11T03:52:42.296 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.296 INFO:teuthology.orchestra.run.gibba004.stdout:[mon] 2021-07-11T03:52:42.296 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T03:52:42.296 INFO:teuthology.orchestra.run.gibba004.stdout: debug mon = 20 2021-07-11T03:52:42.297 INFO:teuthology.orchestra.run.gibba004.stdout: debug paxos = 20 2021-07-11T03:52:42.297 INFO:teuthology.orchestra.run.gibba004.stdout: debug auth = 20 2021-07-11T03:52:42.297 INFO:teuthology.orchestra.run.gibba004.stdout: mon data avail warn = 5 2021-07-11T03:52:42.297 INFO:teuthology.orchestra.run.gibba004.stdout: mon mgr mkfs grace = 240 2021-07-11T03:52:42.298 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min pgs per osd = 4 2021-07-11T03:52:42.298 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd reporter subtree level = osd 2021-07-11T03:52:42.298 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd prime pg temp = true 2021-07-11T03:52:42.298 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min bytes per osd = 10 2021-07-11T03:52:42.298 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.299 INFO:teuthology.orchestra.run.gibba004.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T03:52:42.299 INFO:teuthology.orchestra.run.gibba004.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T03:52:42.299 INFO:teuthology.orchestra.run.gibba004.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T03:52:42.299 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.299 INFO:teuthology.orchestra.run.gibba004.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T03:52:42.300 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T03:52:42.300 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T03:52:42.300 INFO:teuthology.orchestra.run.gibba004.stdout: mon op complaint time = 120 2021-07-11T03:52:42.300 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:52:42.300 INFO:teuthology.orchestra.run.gibba004.stdout:[client] 2021-07-11T03:52:42.301 INFO:teuthology.orchestra.run.gibba004.stdout: rgw cache enabled = true 2021-07-11T03:52:42.301 INFO:teuthology.orchestra.run.gibba004.stdout: rgw enable ops log = true 2021-07-11T03:52:42.301 INFO:teuthology.orchestra.run.gibba004.stdout: rgw enable usage log = true 2021-07-11T03:52:42.301 INFO:teuthology.orchestra.run.gibba004.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T03:52:42.301 INFO:teuthology.orchestra.run.gibba004.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T03:52:42.302 INFO:teuthology.orchestra.run.gibba004.stdout: client mount timeout = 600 2021-07-11T03:52:42.302 INFO:teuthology.orchestra.run.gibba004.stdout: debug client = 20 2021-07-11T03:52:42.302 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T03:52:42.302 INFO:teuthology.orchestra.run.gibba004.stdout: rados mon op timeout = 15m 2021-07-11T03:52:42.303 INFO:teuthology.orchestra.run.gibba004.stdout: rados osd op timeout = 15m 2021-07-11T03:52:42.303 INFO:teuthology.orchestra.run.gibba004.stdout:[mon.a] 2021-07-11T03:52:42.303 INFO:teuthology.orchestra.run.gibba004.stdout:[mon.b] 2021-07-11T03:52:42.303 INFO:teuthology.orchestra.run.gibba004.stdout:[mon.c] 2021-07-11T03:52:42.303 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.a] 2021-07-11T03:52:42.304 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.c] 2021-07-11T03:52:42.304 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.b] 2021-07-11T03:52:42.304 INFO:teuthology.orchestra.run.gibba004.stdout:[mds] 2021-07-11T03:52:42.304 INFO:teuthology.orchestra.run.gibba004.stdout: debug mds = 20 2021-07-11T03:52:42.305 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T03:52:42.305 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal frag = True 2021-07-11T03:52:42.305 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal fragment size max = 10000 2021-07-11T03:52:42.305 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal merge size = 5 2021-07-11T03:52:42.305 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal split bits = 3 2021-07-11T03:52:42.306 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal split size = 100 2021-07-11T03:52:42.306 INFO:teuthology.orchestra.run.gibba004.stdout: mds debug frag = True 2021-07-11T03:52:42.306 INFO:teuthology.orchestra.run.gibba004.stdout: mds debug scatterstat = True 2021-07-11T03:52:42.306 INFO:teuthology.orchestra.run.gibba004.stdout: mds op complaint time = 180 2021-07-11T03:52:42.306 INFO:teuthology.orchestra.run.gibba004.stdout: mds verify scatter = True 2021-07-11T03:52:42.307 INFO:teuthology.orchestra.run.gibba004.stdout: osd op complaint time = 180 2021-07-11T03:52:42.307 INFO:teuthology.orchestra.run.gibba004.stdout: rados mon op timeout = 15m 2021-07-11T03:52:42.307 INFO:teuthology.orchestra.run.gibba004.stdout: rados osd op timeout = 15m 2021-07-11T03:52:42.310 INFO:teuthology.orchestra.run.gibba023.stdout:[global] 2021-07-11T03:52:42.311 INFO:teuthology.orchestra.run.gibba023.stdout: chdir = "" 2021-07-11T03:52:42.311 INFO:teuthology.orchestra.run.gibba023.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T03:52:42.311 INFO:teuthology.orchestra.run.gibba023.stdout: auth supported = cephx 2021-07-11T03:52:42.311 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.311 INFO:teuthology.orchestra.run.gibba023.stdout: filestore xattr use omap = true 2021-07-11T03:52:42.312 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.312 INFO:teuthology.orchestra.run.gibba023.stdout: mon clock drift allowed = 1.000 2021-07-11T03:52:42.312 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.312 INFO:teuthology.orchestra.run.gibba023.stdout: osd crush chooseleaf type = 0 2021-07-11T03:52:42.313 INFO:teuthology.orchestra.run.gibba023.stdout: auth debug = true 2021-07-11T03:52:42.313 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.313 INFO:teuthology.orchestra.run.gibba023.stdout: ms die on old message = true 2021-07-11T03:52:42.313 INFO:teuthology.orchestra.run.gibba023.stdout: ms die on bug = true 2021-07-11T03:52:42.313 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.314 INFO:teuthology.orchestra.run.gibba023.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T03:52:42.314 INFO:teuthology.orchestra.run.gibba023.stdout: mon pg warn max object skew = 0 2021-07-11T03:52:42.314 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.314 INFO:teuthology.orchestra.run.gibba023.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T03:52:42.314 INFO:teuthology.orchestra.run.gibba023.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T03:52:42.315 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.315 INFO:teuthology.orchestra.run.gibba023.stdout: osd pool default size = 2 2021-07-11T03:52:42.315 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.315 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd allow primary affinity = true 2021-07-11T03:52:42.316 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd allow pg remap = true 2021-07-11T03:52:42.316 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on legacy crush tunables = false 2021-07-11T03:52:42.316 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on crush straw calc version zero = false 2021-07-11T03:52:42.316 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on no sortbitwise = false 2021-07-11T03:52:42.316 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on osd down out interval zero = false 2021-07-11T03:52:42.317 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on too few osds = false 2021-07-11T03:52:42.317 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T03:52:42.317 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T03:52:42.317 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.317 INFO:teuthology.orchestra.run.gibba023.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T03:52:42.318 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.318 INFO:teuthology.orchestra.run.gibba023.stdout: osd default data pool replay window = 5 2021-07-11T03:52:42.318 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.318 INFO:teuthology.orchestra.run.gibba023.stdout: mon allow pool delete = true 2021-07-11T03:52:42.319 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.319 INFO:teuthology.orchestra.run.gibba023.stdout: mon cluster log file level = debug 2021-07-11T03:52:42.319 INFO:teuthology.orchestra.run.gibba023.stdout: debug asserts on shutdown = true 2021-07-11T03:52:42.319 INFO:teuthology.orchestra.run.gibba023.stdout: mon health detail to clog = false 2021-07-11T03:52:42.319 INFO:teuthology.orchestra.run.gibba023.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T03:52:42.320 INFO:teuthology.orchestra.run.gibba023.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T03:52:42.320 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.320 INFO:teuthology.orchestra.run.gibba023.stdout:[osd] 2021-07-11T03:52:42.320 INFO:teuthology.orchestra.run.gibba023.stdout: osd journal size = 100 2021-07-11T03:52:42.321 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.321 INFO:teuthology.orchestra.run.gibba023.stdout: osd scrub load threshold = 5.0 2021-07-11T03:52:42.321 INFO:teuthology.orchestra.run.gibba023.stdout: osd scrub max interval = 600 2021-07-11T03:52:42.321 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.321 INFO:teuthology.orchestra.run.gibba023.stdout: osd recover clone overlap = true 2021-07-11T03:52:42.322 INFO:teuthology.orchestra.run.gibba023.stdout: osd recovery max chunk = 1048576 2021-07-11T03:52:42.322 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.322 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug shutdown = true 2021-07-11T03:52:42.322 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug op order = true 2021-07-11T03:52:42.322 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug verify stray on activate = true 2021-07-11T03:52:42.323 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.323 INFO:teuthology.orchestra.run.gibba023.stdout: osd open classes on start = true 2021-07-11T03:52:42.323 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug pg log writeout = true 2021-07-11T03:52:42.323 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.324 INFO:teuthology.orchestra.run.gibba023.stdout: osd deep scrub update digest min age = 30 2021-07-11T03:52:42.324 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.324 INFO:teuthology.orchestra.run.gibba023.stdout: osd map max advance = 10 2021-07-11T03:52:42.324 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.324 INFO:teuthology.orchestra.run.gibba023.stdout: journal zero on create = true 2021-07-11T03:52:42.325 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.325 INFO:teuthology.orchestra.run.gibba023.stdout: filestore ondisk finisher threads = 3 2021-07-11T03:52:42.325 INFO:teuthology.orchestra.run.gibba023.stdout: filestore apply finisher threads = 3 2021-07-11T03:52:42.325 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.326 INFO:teuthology.orchestra.run.gibba023.stdout: bdev debug aio = true 2021-07-11T03:52:42.326 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug misdirected ops = true 2021-07-11T03:52:42.326 INFO:teuthology.orchestra.run.gibba023.stdout: bluestore block size = 96636764160 2021-07-11T03:52:42.326 INFO:teuthology.orchestra.run.gibba023.stdout: bluestore compression mode = aggressive 2021-07-11T03:52:42.326 INFO:teuthology.orchestra.run.gibba023.stdout: bluestore fsck on mount = True 2021-07-11T03:52:42.327 INFO:teuthology.orchestra.run.gibba023.stdout: debug bluefs = 20 2021-07-11T03:52:42.327 INFO:teuthology.orchestra.run.gibba023.stdout: debug bluestore = 20 2021-07-11T03:52:42.327 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T03:52:42.327 INFO:teuthology.orchestra.run.gibba023.stdout: debug osd = 20 2021-07-11T03:52:42.327 INFO:teuthology.orchestra.run.gibba023.stdout: debug rocksdb = 10 2021-07-11T03:52:42.328 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T03:52:42.328 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd full ratio = 0.9 2021-07-11T03:52:42.328 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd nearfull ratio = 0.8 2021-07-11T03:52:42.328 INFO:teuthology.orchestra.run.gibba023.stdout: osd failsafe full ratio = 0.95 2021-07-11T03:52:42.329 INFO:teuthology.orchestra.run.gibba023.stdout: osd objectstore = bluestore 2021-07-11T03:52:42.329 INFO:teuthology.orchestra.run.gibba023.stdout: osd op complaint time = 180 2021-07-11T03:52:42.329 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.329 INFO:teuthology.orchestra.run.gibba023.stdout:[mgr] 2021-07-11T03:52:42.329 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T03:52:42.330 INFO:teuthology.orchestra.run.gibba023.stdout: debug mgr = 20 2021-07-11T03:52:42.330 INFO:teuthology.orchestra.run.gibba023.stdout: debug mon = 20 2021-07-11T03:52:42.330 INFO:teuthology.orchestra.run.gibba023.stdout: debug auth = 20 2021-07-11T03:52:42.330 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min pgs per osd = 4 2021-07-11T03:52:42.330 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min bytes per osd = 10 2021-07-11T03:52:42.331 INFO:teuthology.orchestra.run.gibba023.stdout: mgr/telemetry/nag = false 2021-07-11T03:52:42.331 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.331 INFO:teuthology.orchestra.run.gibba023.stdout:[mon] 2021-07-11T03:52:42.331 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T03:52:42.331 INFO:teuthology.orchestra.run.gibba023.stdout: debug mon = 20 2021-07-11T03:52:42.332 INFO:teuthology.orchestra.run.gibba023.stdout: debug paxos = 20 2021-07-11T03:52:42.332 INFO:teuthology.orchestra.run.gibba023.stdout: debug auth = 20 2021-07-11T03:52:42.332 INFO:teuthology.orchestra.run.gibba023.stdout: mon data avail warn = 5 2021-07-11T03:52:42.332 INFO:teuthology.orchestra.run.gibba023.stdout: mon mgr mkfs grace = 240 2021-07-11T03:52:42.333 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min pgs per osd = 4 2021-07-11T03:52:42.333 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd reporter subtree level = osd 2021-07-11T03:52:42.333 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd prime pg temp = true 2021-07-11T03:52:42.333 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min bytes per osd = 10 2021-07-11T03:52:42.333 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.334 INFO:teuthology.orchestra.run.gibba023.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T03:52:42.334 INFO:teuthology.orchestra.run.gibba023.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T03:52:42.334 INFO:teuthology.orchestra.run.gibba023.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T03:52:42.334 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.334 INFO:teuthology.orchestra.run.gibba023.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T03:52:42.335 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T03:52:42.335 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T03:52:42.335 INFO:teuthology.orchestra.run.gibba023.stdout: mon op complaint time = 120 2021-07-11T03:52:42.335 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T03:52:42.336 INFO:teuthology.orchestra.run.gibba023.stdout:[client] 2021-07-11T03:52:42.336 INFO:teuthology.orchestra.run.gibba023.stdout: rgw cache enabled = true 2021-07-11T03:52:42.336 INFO:teuthology.orchestra.run.gibba023.stdout: rgw enable ops log = true 2021-07-11T03:52:42.336 INFO:teuthology.orchestra.run.gibba023.stdout: rgw enable usage log = true 2021-07-11T03:52:42.336 INFO:teuthology.orchestra.run.gibba023.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T03:52:42.337 INFO:teuthology.orchestra.run.gibba023.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T03:52:42.337 INFO:teuthology.orchestra.run.gibba023.stdout: client mount timeout = 600 2021-07-11T03:52:42.337 INFO:teuthology.orchestra.run.gibba023.stdout: debug client = 20 2021-07-11T03:52:42.337 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T03:52:42.337 INFO:teuthology.orchestra.run.gibba023.stdout: rados mon op timeout = 15m 2021-07-11T03:52:42.338 INFO:teuthology.orchestra.run.gibba023.stdout: rados osd op timeout = 15m 2021-07-11T03:52:42.338 INFO:teuthology.orchestra.run.gibba023.stdout:[mon.a] 2021-07-11T03:52:42.338 INFO:teuthology.orchestra.run.gibba023.stdout:[mon.b] 2021-07-11T03:52:42.338 INFO:teuthology.orchestra.run.gibba023.stdout:[mon.c] 2021-07-11T03:52:42.338 INFO:teuthology.orchestra.run.gibba023.stdout:[mds.a] 2021-07-11T03:52:42.339 INFO:teuthology.orchestra.run.gibba023.stdout:[mds.c] 2021-07-11T03:52:42.339 INFO:teuthology.orchestra.run.gibba023.stdout:[mds.b] 2021-07-11T03:52:42.339 INFO:teuthology.orchestra.run.gibba023.stdout:[mds] 2021-07-11T03:52:42.339 INFO:teuthology.orchestra.run.gibba023.stdout: debug mds = 20 2021-07-11T03:52:42.340 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T03:52:42.340 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal frag = True 2021-07-11T03:52:42.340 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal fragment size max = 10000 2021-07-11T03:52:42.340 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal merge size = 5 2021-07-11T03:52:42.340 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal split bits = 3 2021-07-11T03:52:42.341 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal split size = 100 2021-07-11T03:52:42.341 INFO:teuthology.orchestra.run.gibba023.stdout: mds debug frag = True 2021-07-11T03:52:42.341 INFO:teuthology.orchestra.run.gibba023.stdout: mds debug scatterstat = True 2021-07-11T03:52:42.341 INFO:teuthology.orchestra.run.gibba023.stdout: mds op complaint time = 180 2021-07-11T03:52:42.341 INFO:teuthology.orchestra.run.gibba023.stdout: mds verify scatter = True 2021-07-11T03:52:42.342 INFO:teuthology.orchestra.run.gibba023.stdout: osd op complaint time = 180 2021-07-11T03:52:42.342 INFO:teuthology.orchestra.run.gibba023.stdout: rados mon op timeout = 15m 2021-07-11T03:52:42.342 INFO:teuthology.orchestra.run.gibba023.stdout: rados osd op timeout = 15m 2021-07-11T03:52:42.342 INFO:teuthology.orchestra.run.gibba019.stdout:[global] 2021-07-11T03:52:42.343 INFO:teuthology.orchestra.run.gibba019.stdout: chdir = "" 2021-07-11T03:52:42.343 INFO:teuthology.orchestra.run.gibba019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T03:52:42.343 INFO:teuthology.orchestra.run.gibba019.stdout: auth supported = cephx 2021-07-11T03:52:42.343 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.344 INFO:teuthology.orchestra.run.gibba019.stdout: filestore xattr use omap = true 2021-07-11T03:52:42.344 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.344 INFO:teuthology.orchestra.run.gibba019.stdout: mon clock drift allowed = 1.000 2021-07-11T03:52:42.344 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.345 INFO:teuthology.orchestra.run.gibba019.stdout: osd crush chooseleaf type = 0 2021-07-11T03:52:42.345 INFO:teuthology.orchestra.run.gibba019.stdout: auth debug = true 2021-07-11T03:52:42.345 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.345 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on old message = true 2021-07-11T03:52:42.345 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on bug = true 2021-07-11T03:52:42.346 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.346 INFO:teuthology.orchestra.run.gibba019.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T03:52:42.346 INFO:teuthology.orchestra.run.gibba019.stdout: mon pg warn max object skew = 0 2021-07-11T03:52:42.346 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.347 INFO:teuthology.orchestra.run.gibba019.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T03:52:42.347 INFO:teuthology.orchestra.run.gibba019.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T03:52:42.347 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.347 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default size = 2 2021-07-11T03:52:42.347 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.348 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow primary affinity = true 2021-07-11T03:52:42.348 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow pg remap = true 2021-07-11T03:52:42.348 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on legacy crush tunables = false 2021-07-11T03:52:42.348 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on crush straw calc version zero = false 2021-07-11T03:52:42.348 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on no sortbitwise = false 2021-07-11T03:52:42.349 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on osd down out interval zero = false 2021-07-11T03:52:42.349 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on too few osds = false 2021-07-11T03:52:42.349 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T03:52:42.349 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T03:52:42.350 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.350 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T03:52:42.350 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.350 INFO:teuthology.orchestra.run.gibba019.stdout: osd default data pool replay window = 5 2021-07-11T03:52:42.350 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.351 INFO:teuthology.orchestra.run.gibba019.stdout: mon allow pool delete = true 2021-07-11T03:52:42.351 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.351 INFO:teuthology.orchestra.run.gibba019.stdout: mon cluster log file level = debug 2021-07-11T03:52:42.351 INFO:teuthology.orchestra.run.gibba019.stdout: debug asserts on shutdown = true 2021-07-11T03:52:42.351 INFO:teuthology.orchestra.run.gibba019.stdout: mon health detail to clog = false 2021-07-11T03:52:42.352 INFO:teuthology.orchestra.run.gibba019.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T03:52:42.352 INFO:teuthology.orchestra.run.gibba019.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T03:52:42.352 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.352 INFO:teuthology.orchestra.run.gibba019.stdout:[osd] 2021-07-11T03:52:42.352 INFO:teuthology.orchestra.run.gibba019.stdout: osd journal size = 100 2021-07-11T03:52:42.353 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.353 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub load threshold = 5.0 2021-07-11T03:52:42.353 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub max interval = 600 2021-07-11T03:52:42.353 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.354 INFO:teuthology.orchestra.run.gibba019.stdout: osd recover clone overlap = true 2021-07-11T03:52:42.354 INFO:teuthology.orchestra.run.gibba019.stdout: osd recovery max chunk = 1048576 2021-07-11T03:52:42.354 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.354 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug shutdown = true 2021-07-11T03:52:42.354 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug op order = true 2021-07-11T03:52:42.355 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug verify stray on activate = true 2021-07-11T03:52:42.355 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.355 INFO:teuthology.orchestra.run.gibba019.stdout: osd open classes on start = true 2021-07-11T03:52:42.355 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug pg log writeout = true 2021-07-11T03:52:42.355 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.356 INFO:teuthology.orchestra.run.gibba019.stdout: osd deep scrub update digest min age = 30 2021-07-11T03:52:42.356 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.356 INFO:teuthology.orchestra.run.gibba019.stdout: osd map max advance = 10 2021-07-11T03:52:42.356 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.357 INFO:teuthology.orchestra.run.gibba019.stdout: journal zero on create = true 2021-07-11T03:52:42.357 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.357 INFO:teuthology.orchestra.run.gibba019.stdout: filestore ondisk finisher threads = 3 2021-07-11T03:52:42.357 INFO:teuthology.orchestra.run.gibba019.stdout: filestore apply finisher threads = 3 2021-07-11T03:52:42.357 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.358 INFO:teuthology.orchestra.run.gibba019.stdout: bdev debug aio = true 2021-07-11T03:52:42.358 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug misdirected ops = true 2021-07-11T03:52:42.358 INFO:teuthology.orchestra.run.gibba019.stdout: bluestore block size = 96636764160 2021-07-11T03:52:42.358 INFO:teuthology.orchestra.run.gibba019.stdout: bluestore compression mode = aggressive 2021-07-11T03:52:42.358 INFO:teuthology.orchestra.run.gibba019.stdout: bluestore fsck on mount = True 2021-07-11T03:52:42.359 INFO:teuthology.orchestra.run.gibba019.stdout: debug bluefs = 20 2021-07-11T03:52:42.359 INFO:teuthology.orchestra.run.gibba019.stdout: debug bluestore = 20 2021-07-11T03:52:42.359 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T03:52:42.359 INFO:teuthology.orchestra.run.gibba019.stdout: debug osd = 20 2021-07-11T03:52:42.360 INFO:teuthology.orchestra.run.gibba019.stdout: debug rocksdb = 10 2021-07-11T03:52:42.360 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T03:52:42.360 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd full ratio = 0.9 2021-07-11T03:52:42.360 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd nearfull ratio = 0.8 2021-07-11T03:52:42.360 INFO:teuthology.orchestra.run.gibba019.stdout: osd failsafe full ratio = 0.95 2021-07-11T03:52:42.361 INFO:teuthology.orchestra.run.gibba019.stdout: osd objectstore = bluestore 2021-07-11T03:52:42.361 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-07-11T03:52:42.361 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.361 INFO:teuthology.orchestra.run.gibba019.stdout:[mgr] 2021-07-11T03:52:42.361 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T03:52:42.362 INFO:teuthology.orchestra.run.gibba019.stdout: debug mgr = 20 2021-07-11T03:52:42.362 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-07-11T03:52:42.362 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-07-11T03:52:42.362 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-07-11T03:52:42.363 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-07-11T03:52:42.363 INFO:teuthology.orchestra.run.gibba019.stdout: mgr/telemetry/nag = false 2021-07-11T03:52:42.363 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.363 INFO:teuthology.orchestra.run.gibba019.stdout:[mon] 2021-07-11T03:52:42.364 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T03:52:42.364 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-07-11T03:52:42.364 INFO:teuthology.orchestra.run.gibba019.stdout: debug paxos = 20 2021-07-11T03:52:42.364 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-07-11T03:52:42.364 INFO:teuthology.orchestra.run.gibba019.stdout: mon data avail warn = 5 2021-07-11T03:52:42.365 INFO:teuthology.orchestra.run.gibba019.stdout: mon mgr mkfs grace = 240 2021-07-11T03:52:42.365 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-07-11T03:52:42.365 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd reporter subtree level = osd 2021-07-11T03:52:42.365 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd prime pg temp = true 2021-07-11T03:52:42.366 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-07-11T03:52:42.366 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.366 INFO:teuthology.orchestra.run.gibba019.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T03:52:42.366 INFO:teuthology.orchestra.run.gibba019.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T03:52:42.366 INFO:teuthology.orchestra.run.gibba019.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T03:52:42.367 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.367 INFO:teuthology.orchestra.run.gibba019.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T03:52:42.367 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T03:52:42.367 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T03:52:42.368 INFO:teuthology.orchestra.run.gibba019.stdout: mon op complaint time = 120 2021-07-11T03:52:42.368 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T03:52:42.368 INFO:teuthology.orchestra.run.gibba019.stdout:[client] 2021-07-11T03:52:42.368 INFO:teuthology.orchestra.run.gibba019.stdout: rgw cache enabled = true 2021-07-11T03:52:42.368 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable ops log = true 2021-07-11T03:52:42.369 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable usage log = true 2021-07-11T03:52:42.369 INFO:teuthology.orchestra.run.gibba019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T03:52:42.369 INFO:teuthology.orchestra.run.gibba019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T03:52:42.369 INFO:teuthology.orchestra.run.gibba019.stdout: client mount timeout = 600 2021-07-11T03:52:42.370 INFO:teuthology.orchestra.run.gibba019.stdout: debug client = 20 2021-07-11T03:52:42.370 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T03:52:42.370 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-07-11T03:52:42.370 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-07-11T03:52:42.370 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.a] 2021-07-11T03:52:42.371 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.b] 2021-07-11T03:52:42.371 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.c] 2021-07-11T03:52:42.371 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.a] 2021-07-11T03:52:42.371 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.c] 2021-07-11T03:52:42.372 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.b] 2021-07-11T03:52:42.372 INFO:teuthology.orchestra.run.gibba019.stdout:[mds] 2021-07-11T03:52:42.372 INFO:teuthology.orchestra.run.gibba019.stdout: debug mds = 20 2021-07-11T03:52:42.372 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T03:52:42.372 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal frag = True 2021-07-11T03:52:42.373 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal fragment size max = 10000 2021-07-11T03:52:42.373 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal merge size = 5 2021-07-11T03:52:42.373 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split bits = 3 2021-07-11T03:52:42.373 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split size = 100 2021-07-11T03:52:42.374 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug frag = True 2021-07-11T03:52:42.374 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug scatterstat = True 2021-07-11T03:52:42.374 INFO:teuthology.orchestra.run.gibba019.stdout: mds op complaint time = 180 2021-07-11T03:52:42.374 INFO:teuthology.orchestra.run.gibba019.stdout: mds verify scatter = True 2021-07-11T03:52:42.374 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-07-11T03:52:42.375 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-07-11T03:52:42.375 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-07-11T03:52:42.375 INFO:teuthology.orchestra.run.gibba005.stdout:[global] 2021-07-11T03:52:42.375 INFO:teuthology.orchestra.run.gibba005.stdout: chdir = "" 2021-07-11T03:52:42.376 INFO:teuthology.orchestra.run.gibba005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T03:52:42.376 INFO:teuthology.orchestra.run.gibba005.stdout: auth supported = cephx 2021-07-11T03:52:42.376 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.376 INFO:teuthology.orchestra.run.gibba005.stdout: filestore xattr use omap = true 2021-07-11T03:52:42.376 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.377 INFO:teuthology.orchestra.run.gibba005.stdout: mon clock drift allowed = 1.000 2021-07-11T03:52:42.377 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.377 INFO:teuthology.orchestra.run.gibba005.stdout: osd crush chooseleaf type = 0 2021-07-11T03:52:42.377 INFO:teuthology.orchestra.run.gibba005.stdout: auth debug = true 2021-07-11T03:52:42.378 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.378 INFO:teuthology.orchestra.run.gibba005.stdout: ms die on old message = true 2021-07-11T03:52:42.378 INFO:teuthology.orchestra.run.gibba005.stdout: ms die on bug = true 2021-07-11T03:52:42.378 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.378 INFO:teuthology.orchestra.run.gibba005.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T03:52:42.379 INFO:teuthology.orchestra.run.gibba005.stdout: mon pg warn max object skew = 0 2021-07-11T03:52:42.379 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.379 INFO:teuthology.orchestra.run.gibba005.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T03:52:42.379 INFO:teuthology.orchestra.run.gibba005.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T03:52:42.380 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.380 INFO:teuthology.orchestra.run.gibba005.stdout: osd pool default size = 2 2021-07-11T03:52:42.380 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.380 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd allow primary affinity = true 2021-07-11T03:52:42.380 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd allow pg remap = true 2021-07-11T03:52:42.381 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on legacy crush tunables = false 2021-07-11T03:52:42.381 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on crush straw calc version zero = false 2021-07-11T03:52:42.381 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on no sortbitwise = false 2021-07-11T03:52:42.381 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on osd down out interval zero = false 2021-07-11T03:52:42.381 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on too few osds = false 2021-07-11T03:52:42.382 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T03:52:42.382 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T03:52:42.382 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.382 INFO:teuthology.orchestra.run.gibba005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T03:52:42.383 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.383 INFO:teuthology.orchestra.run.gibba005.stdout: osd default data pool replay window = 5 2021-07-11T03:52:42.383 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.383 INFO:teuthology.orchestra.run.gibba005.stdout: mon allow pool delete = true 2021-07-11T03:52:42.383 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.384 INFO:teuthology.orchestra.run.gibba005.stdout: mon cluster log file level = debug 2021-07-11T03:52:42.384 INFO:teuthology.orchestra.run.gibba005.stdout: debug asserts on shutdown = true 2021-07-11T03:52:42.384 INFO:teuthology.orchestra.run.gibba005.stdout: mon health detail to clog = false 2021-07-11T03:52:42.384 INFO:teuthology.orchestra.run.gibba005.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T03:52:42.384 INFO:teuthology.orchestra.run.gibba005.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T03:52:42.385 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.385 INFO:teuthology.orchestra.run.gibba005.stdout:[osd] 2021-07-11T03:52:42.385 INFO:teuthology.orchestra.run.gibba005.stdout: osd journal size = 100 2021-07-11T03:52:42.385 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.386 INFO:teuthology.orchestra.run.gibba005.stdout: osd scrub load threshold = 5.0 2021-07-11T03:52:42.386 INFO:teuthology.orchestra.run.gibba005.stdout: osd scrub max interval = 600 2021-07-11T03:52:42.386 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.386 INFO:teuthology.orchestra.run.gibba005.stdout: osd recover clone overlap = true 2021-07-11T03:52:42.386 INFO:teuthology.orchestra.run.gibba005.stdout: osd recovery max chunk = 1048576 2021-07-11T03:52:42.387 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.387 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug shutdown = true 2021-07-11T03:52:42.387 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug op order = true 2021-07-11T03:52:42.387 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug verify stray on activate = true 2021-07-11T03:52:42.388 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.388 INFO:teuthology.orchestra.run.gibba005.stdout: osd open classes on start = true 2021-07-11T03:52:42.388 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug pg log writeout = true 2021-07-11T03:52:42.388 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.389 INFO:teuthology.orchestra.run.gibba005.stdout: osd deep scrub update digest min age = 30 2021-07-11T03:52:42.389 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.389 INFO:teuthology.orchestra.run.gibba005.stdout: osd map max advance = 10 2021-07-11T03:52:42.389 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.389 INFO:teuthology.orchestra.run.gibba005.stdout: journal zero on create = true 2021-07-11T03:52:42.390 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.390 INFO:teuthology.orchestra.run.gibba005.stdout: filestore ondisk finisher threads = 3 2021-07-11T03:52:42.390 INFO:teuthology.orchestra.run.gibba005.stdout: filestore apply finisher threads = 3 2021-07-11T03:52:42.390 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.391 INFO:teuthology.orchestra.run.gibba005.stdout: bdev debug aio = true 2021-07-11T03:52:42.391 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug misdirected ops = true 2021-07-11T03:52:42.391 INFO:teuthology.orchestra.run.gibba005.stdout: bluestore block size = 96636764160 2021-07-11T03:52:42.391 INFO:teuthology.orchestra.run.gibba005.stdout: bluestore compression mode = aggressive 2021-07-11T03:52:42.392 INFO:teuthology.orchestra.run.gibba005.stdout: bluestore fsck on mount = True 2021-07-11T03:52:42.392 INFO:teuthology.orchestra.run.gibba005.stdout: debug bluefs = 20 2021-07-11T03:52:42.392 INFO:teuthology.orchestra.run.gibba005.stdout: debug bluestore = 20 2021-07-11T03:52:42.392 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T03:52:42.392 INFO:teuthology.orchestra.run.gibba005.stdout: debug osd = 20 2021-07-11T03:52:42.393 INFO:teuthology.orchestra.run.gibba005.stdout: debug rocksdb = 10 2021-07-11T03:52:42.393 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T03:52:42.393 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd full ratio = 0.9 2021-07-11T03:52:42.393 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd nearfull ratio = 0.8 2021-07-11T03:52:42.394 INFO:teuthology.orchestra.run.gibba005.stdout: osd failsafe full ratio = 0.95 2021-07-11T03:52:42.394 INFO:teuthology.orchestra.run.gibba005.stdout: osd objectstore = bluestore 2021-07-11T03:52:42.394 INFO:teuthology.orchestra.run.gibba005.stdout: osd op complaint time = 180 2021-07-11T03:52:42.394 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.395 INFO:teuthology.orchestra.run.gibba005.stdout:[mgr] 2021-07-11T03:52:42.395 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T03:52:42.395 INFO:teuthology.orchestra.run.gibba005.stdout: debug mgr = 20 2021-07-11T03:52:42.395 INFO:teuthology.orchestra.run.gibba005.stdout: debug mon = 20 2021-07-11T03:52:42.395 INFO:teuthology.orchestra.run.gibba005.stdout: debug auth = 20 2021-07-11T03:52:42.396 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min pgs per osd = 4 2021-07-11T03:52:42.396 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min bytes per osd = 10 2021-07-11T03:52:42.396 INFO:teuthology.orchestra.run.gibba005.stdout: mgr/telemetry/nag = false 2021-07-11T03:52:42.396 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.397 INFO:teuthology.orchestra.run.gibba005.stdout:[mon] 2021-07-11T03:52:42.397 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T03:52:42.397 INFO:teuthology.orchestra.run.gibba005.stdout: debug mon = 20 2021-07-11T03:52:42.397 INFO:teuthology.orchestra.run.gibba005.stdout: debug paxos = 20 2021-07-11T03:52:42.398 INFO:teuthology.orchestra.run.gibba005.stdout: debug auth = 20 2021-07-11T03:52:42.398 INFO:teuthology.orchestra.run.gibba005.stdout: mon data avail warn = 5 2021-07-11T03:52:42.398 INFO:teuthology.orchestra.run.gibba005.stdout: mon mgr mkfs grace = 240 2021-07-11T03:52:42.398 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min pgs per osd = 4 2021-07-11T03:52:42.398 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd reporter subtree level = osd 2021-07-11T03:52:42.399 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd prime pg temp = true 2021-07-11T03:52:42.399 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min bytes per osd = 10 2021-07-11T03:52:42.399 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.399 INFO:teuthology.orchestra.run.gibba005.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T03:52:42.400 INFO:teuthology.orchestra.run.gibba005.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T03:52:42.400 INFO:teuthology.orchestra.run.gibba005.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T03:52:42.400 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.400 INFO:teuthology.orchestra.run.gibba005.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T03:52:42.400 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T03:52:42.401 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T03:52:42.401 INFO:teuthology.orchestra.run.gibba005.stdout: mon op complaint time = 120 2021-07-11T03:52:42.401 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T03:52:42.401 INFO:teuthology.orchestra.run.gibba005.stdout:[client] 2021-07-11T03:52:42.402 INFO:teuthology.orchestra.run.gibba005.stdout: rgw cache enabled = true 2021-07-11T03:52:42.402 INFO:teuthology.orchestra.run.gibba005.stdout: rgw enable ops log = true 2021-07-11T03:52:42.402 INFO:teuthology.orchestra.run.gibba005.stdout: rgw enable usage log = true 2021-07-11T03:52:42.402 INFO:teuthology.orchestra.run.gibba005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T03:52:42.402 INFO:teuthology.orchestra.run.gibba005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T03:52:42.403 INFO:teuthology.orchestra.run.gibba005.stdout: client mount timeout = 600 2021-07-11T03:52:42.403 INFO:teuthology.orchestra.run.gibba005.stdout: debug client = 20 2021-07-11T03:52:42.403 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T03:52:42.403 INFO:teuthology.orchestra.run.gibba005.stdout: rados mon op timeout = 15m 2021-07-11T03:52:42.404 INFO:teuthology.orchestra.run.gibba005.stdout: rados osd op timeout = 15m 2021-07-11T03:52:42.404 INFO:teuthology.orchestra.run.gibba005.stdout:[mon.a] 2021-07-11T03:52:42.404 INFO:teuthology.orchestra.run.gibba005.stdout:[mon.b] 2021-07-11T03:52:42.404 INFO:teuthology.orchestra.run.gibba005.stdout:[mon.c] 2021-07-11T03:52:42.404 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.a] 2021-07-11T03:52:42.405 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.c] 2021-07-11T03:52:42.405 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.b] 2021-07-11T03:52:42.405 INFO:teuthology.orchestra.run.gibba005.stdout:[mds] 2021-07-11T03:52:42.405 INFO:teuthology.orchestra.run.gibba005.stdout: debug mds = 20 2021-07-11T03:52:42.406 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T03:52:42.406 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal frag = True 2021-07-11T03:52:42.406 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal fragment size max = 10000 2021-07-11T03:52:42.406 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal merge size = 5 2021-07-11T03:52:42.407 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal split bits = 3 2021-07-11T03:52:42.407 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal split size = 100 2021-07-11T03:52:42.407 INFO:teuthology.orchestra.run.gibba005.stdout: mds debug frag = True 2021-07-11T03:52:42.407 INFO:teuthology.orchestra.run.gibba005.stdout: mds debug scatterstat = True 2021-07-11T03:52:42.407 INFO:teuthology.orchestra.run.gibba005.stdout: mds op complaint time = 180 2021-07-11T03:52:42.408 INFO:teuthology.orchestra.run.gibba005.stdout: mds verify scatter = True 2021-07-11T03:52:42.408 INFO:teuthology.orchestra.run.gibba005.stdout: osd op complaint time = 180 2021-07-11T03:52:42.408 INFO:teuthology.orchestra.run.gibba005.stdout: rados mon op timeout = 15m 2021-07-11T03:52:42.408 INFO:teuthology.orchestra.run.gibba005.stdout: rados osd op timeout = 15m 2021-07-11T03:52:42.409 INFO:tasks.ceph:Creating admin key on mon.a... 2021-07-11T03:52:42.409 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2021-07-11T03:52:42.449 INFO:tasks.ceph:Copying monmap to all nodes... 2021-07-11T03:52:42.499 DEBUG:teuthology.orchestra.remote:gibba004:/etc/ceph/ceph.keyring is 206B 2021-07-11T03:52:42.557 DEBUG:teuthology.orchestra.remote:gibba004:/home/ubuntu/cephtest/ceph.monmap is 379B 2021-07-11T03:52:42.577 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T03:52:42.577 DEBUG:teuthology.orchestra.run.gibba004:> set -ex 2021-07-11T03:52:42.578 DEBUG:teuthology.orchestra.run.gibba004:> sudo dd of=/etc/ceph/ceph.keyring 2021-07-11T03:52:42.578 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-07-11T03:52:42.595 DEBUG:teuthology.orchestra.run.gibba004:> set -ex 2021-07-11T03:52:42.596 DEBUG:teuthology.orchestra.run.gibba004:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:42.642 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T03:52:42.643 DEBUG:teuthology.orchestra.run.gibba005:> set -ex 2021-07-11T03:52:42.643 DEBUG:teuthology.orchestra.run.gibba005:> sudo dd of=/etc/ceph/ceph.keyring 2021-07-11T03:52:42.644 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-07-11T03:52:42.662 DEBUG:teuthology.orchestra.run.gibba005:> set -ex 2021-07-11T03:52:42.662 DEBUG:teuthology.orchestra.run.gibba005:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:42.666 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba019.front.sepia.ceph.com 2021-07-11T03:52:42.667 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-07-11T03:52:42.667 DEBUG:teuthology.orchestra.run.gibba019:> sudo dd of=/etc/ceph/ceph.keyring 2021-07-11T03:52:42.667 DEBUG:teuthology.orchestra.run.gibba019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-07-11T03:52:42.685 DEBUG:teuthology.orchestra.run.gibba019:> set -ex 2021-07-11T03:52:42.685 DEBUG:teuthology.orchestra.run.gibba019:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:42.689 INFO:tasks.ceph:Sending monmap to node ubuntu@gibba023.front.sepia.ceph.com 2021-07-11T03:52:42.689 DEBUG:teuthology.orchestra.run.gibba023:> set -ex 2021-07-11T03:52:42.690 DEBUG:teuthology.orchestra.run.gibba023:> sudo dd of=/etc/ceph/ceph.keyring 2021-07-11T03:52:42.690 DEBUG:teuthology.orchestra.run.gibba023:> sudo chmod 0644 /etc/ceph/ceph.keyring 2021-07-11T03:52:42.707 DEBUG:teuthology.orchestra.run.gibba023:> set -ex 2021-07-11T03:52:42.708 DEBUG:teuthology.orchestra.run.gibba023:> dd of=/home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:42.712 INFO:tasks.ceph:Setting up mon nodes... 2021-07-11T03:52:42.712 INFO:tasks.ceph:Setting up mgr nodes... 2021-07-11T03:52:42.713 DEBUG:teuthology.orchestra.run.gibba004:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2021-07-11T03:52:42.785 INFO:teuthology.orchestra.run.gibba004.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2021-07-11T03:52:42.788 DEBUG:teuthology.orchestra.run.gibba005:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2021-07-11T03:52:42.838 INFO:teuthology.orchestra.run.gibba005.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2021-07-11T03:52:42.839 INFO:tasks.ceph:Setting up mds nodes... 2021-07-11T03:52:42.840 DEBUG:teuthology.orchestra.run.gibba004:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2021-07-11T03:52:42.907 INFO:teuthology.orchestra.run.gibba004.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2021-07-11T03:52:42.909 DEBUG:teuthology.orchestra.run.gibba004:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2021-07-11T03:52:42.919 DEBUG:teuthology.orchestra.run.gibba004:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2021-07-11T03:52:42.942 INFO:teuthology.orchestra.run.gibba004.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2021-07-11T03:52:42.944 DEBUG:teuthology.orchestra.run.gibba004:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2021-07-11T03:52:42.951 DEBUG:teuthology.orchestra.run.gibba005:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2021-07-11T03:52:43.020 INFO:teuthology.orchestra.run.gibba005.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2021-07-11T03:52:43.024 DEBUG:teuthology.orchestra.run.gibba005:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2021-07-11T03:52:43.037 INFO:tasks.ceph_client:Setting up client nodes... 2021-07-11T03:52:43.039 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2021-07-11T03:52:43.082 INFO:teuthology.orchestra.run.gibba019.stdout:creating /etc/ceph/ceph.client.0.keyring 2021-07-11T03:52:43.086 DEBUG:teuthology.orchestra.run.gibba023:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2021-07-11T03:52:43.123 INFO:teuthology.orchestra.run.gibba023.stdout:creating /etc/ceph/ceph.client.1.keyring 2021-07-11T03:52:43.127 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-07-11T03:52:43.128 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@gibba004.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@gibba005.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2021-07-11T03:52:43.128 DEBUG:teuthology.orchestra.run.gibba004:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2021-07-11T03:52:43.140 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-07-11T03:52:43.141 INFO:tasks.ceph:role: osd.0 2021-07-11T03:52:43.141 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T03:52:43.141 DEBUG:teuthology.orchestra.run.gibba004:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-07-11T03:52:43.182 INFO:teuthology.orchestra.run.gibba004.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5505024 blks 2021-07-11T03:52:43.182 INFO:teuthology.orchestra.run.gibba004.stdout: = sectsz=512 attr=2, projid32bit=1 2021-07-11T03:52:43.183 INFO:teuthology.orchestra.run.gibba004.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-07-11T03:52:43.183 INFO:teuthology.orchestra.run.gibba004.stdout: = reflink=1 2021-07-11T03:52:43.183 INFO:teuthology.orchestra.run.gibba004.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-07-11T03:52:43.183 INFO:teuthology.orchestra.run.gibba004.stdout: = sunit=0 swidth=0 blks 2021-07-11T03:52:43.184 INFO:teuthology.orchestra.run.gibba004.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-07-11T03:52:43.184 INFO:teuthology.orchestra.run.gibba004.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-07-11T03:52:43.184 INFO:teuthology.orchestra.run.gibba004.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-07-11T03:52:43.184 INFO:teuthology.orchestra.run.gibba004.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-07-11T03:52:43.185 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@gibba004.front.sepia.ceph.com -o noatime 2021-07-11T03:52:43.185 DEBUG:teuthology.orchestra.run.gibba004:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2021-07-11T03:52:43.239 DEBUG:teuthology.orchestra.run.gibba004:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2021-07-11T03:52:43.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:52:43.249 INFO:teuthology.orchestra.run.gibba004.stderr:sudo: /sbin/restorecon: command not found 2021-07-11T03:52:43.250 DEBUG:teuthology.orchestra.run.gibba004:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2021-07-11T03:52:43.257 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-07-11T03:52:43.257 INFO:tasks.ceph:role: osd.1 2021-07-11T03:52:43.258 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T03:52:43.258 DEBUG:teuthology.orchestra.run.gibba004:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-07-11T03:52:43.346 INFO:teuthology.orchestra.run.gibba004.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5505024 blks 2021-07-11T03:52:43.347 INFO:teuthology.orchestra.run.gibba004.stdout: = sectsz=512 attr=2, projid32bit=1 2021-07-11T03:52:43.347 INFO:teuthology.orchestra.run.gibba004.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-07-11T03:52:43.347 INFO:teuthology.orchestra.run.gibba004.stdout: = reflink=1 2021-07-11T03:52:43.348 INFO:teuthology.orchestra.run.gibba004.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-07-11T03:52:43.348 INFO:teuthology.orchestra.run.gibba004.stdout: = sunit=0 swidth=0 blks 2021-07-11T03:52:43.348 INFO:teuthology.orchestra.run.gibba004.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-07-11T03:52:43.348 INFO:teuthology.orchestra.run.gibba004.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-07-11T03:52:43.349 INFO:teuthology.orchestra.run.gibba004.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-07-11T03:52:43.349 INFO:teuthology.orchestra.run.gibba004.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-07-11T03:52:43.350 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@gibba004.front.sepia.ceph.com -o noatime 2021-07-11T03:52:43.350 DEBUG:teuthology.orchestra.run.gibba004:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2021-07-11T03:52:43.368 DEBUG:teuthology.orchestra.run.gibba004:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2021-07-11T03:52:43.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:52:43.373 INFO:teuthology.orchestra.run.gibba004.stderr:sudo: /sbin/restorecon: command not found 2021-07-11T03:52:43.373 DEBUG:teuthology.orchestra.run.gibba004:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2021-07-11T03:52:43.426 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-07-11T03:52:43.426 INFO:tasks.ceph:role: osd.2 2021-07-11T03:52:43.427 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T03:52:43.427 DEBUG:teuthology.orchestra.run.gibba004:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-07-11T03:52:43.477 INFO:teuthology.orchestra.run.gibba004.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5505024 blks 2021-07-11T03:52:43.477 INFO:teuthology.orchestra.run.gibba004.stdout: = sectsz=512 attr=2, projid32bit=1 2021-07-11T03:52:43.477 INFO:teuthology.orchestra.run.gibba004.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-07-11T03:52:43.478 INFO:teuthology.orchestra.run.gibba004.stdout: = reflink=1 2021-07-11T03:52:43.478 INFO:teuthology.orchestra.run.gibba004.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-07-11T03:52:43.478 INFO:teuthology.orchestra.run.gibba004.stdout: = sunit=0 swidth=0 blks 2021-07-11T03:52:43.478 INFO:teuthology.orchestra.run.gibba004.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-07-11T03:52:43.479 INFO:teuthology.orchestra.run.gibba004.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-07-11T03:52:43.479 INFO:teuthology.orchestra.run.gibba004.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-07-11T03:52:43.479 INFO:teuthology.orchestra.run.gibba004.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-07-11T03:52:43.480 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@gibba004.front.sepia.ceph.com -o noatime 2021-07-11T03:52:43.481 DEBUG:teuthology.orchestra.run.gibba004:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2021-07-11T03:52:43.496 DEBUG:teuthology.orchestra.run.gibba004:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2021-07-11T03:52:43.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:52:43.544 INFO:teuthology.orchestra.run.gibba004.stderr:sudo: /sbin/restorecon: command not found 2021-07-11T03:52:43.545 DEBUG:teuthology.orchestra.run.gibba004:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2021-07-11T03:52:43.558 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2021-07-11T03:52:43.559 INFO:tasks.ceph:role: osd.3 2021-07-11T03:52:43.559 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T03:52:43.560 DEBUG:teuthology.orchestra.run.gibba004:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-07-11T03:52:43.602 INFO:teuthology.orchestra.run.gibba004.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5505024 blks 2021-07-11T03:52:43.602 INFO:teuthology.orchestra.run.gibba004.stdout: = sectsz=512 attr=2, projid32bit=1 2021-07-11T03:52:43.603 INFO:teuthology.orchestra.run.gibba004.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-07-11T03:52:43.603 INFO:teuthology.orchestra.run.gibba004.stdout: = reflink=1 2021-07-11T03:52:43.603 INFO:teuthology.orchestra.run.gibba004.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-07-11T03:52:43.603 INFO:teuthology.orchestra.run.gibba004.stdout: = sunit=0 swidth=0 blks 2021-07-11T03:52:43.604 INFO:teuthology.orchestra.run.gibba004.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-07-11T03:52:43.604 INFO:teuthology.orchestra.run.gibba004.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-07-11T03:52:43.604 INFO:teuthology.orchestra.run.gibba004.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-07-11T03:52:43.604 INFO:teuthology.orchestra.run.gibba004.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-07-11T03:52:43.605 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@gibba004.front.sepia.ceph.com -o noatime 2021-07-11T03:52:43.606 DEBUG:teuthology.orchestra.run.gibba004:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2021-07-11T03:52:43.683 DEBUG:teuthology.orchestra.run.gibba004:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2021-07-11T03:52:43.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:52:43.693 INFO:teuthology.orchestra.run.gibba004.stderr:sudo: /sbin/restorecon: command not found 2021-07-11T03:52:43.694 DEBUG:teuthology.orchestra.run.gibba004:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:43.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:43.760+0000 7effeff8ed80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2021-07-11T03:52:43.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:43.760+0000 7effeff8ed80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2021-07-11T03:52:43.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:43.760+0000 7effeff8ed80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-07-11T03:52:43.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:43.760+0000 7effeff8ed80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-07-11T03:52:43.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:43.760+0000 7effeff8ed80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2021-07-11T03:52:43.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:43.764+0000 7effeff8ed80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2021-07-11T03:52:43.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:43.776+0000 7effeff8ed80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-07-11T03:52:46.233 DEBUG:teuthology.orchestra.run.gibba004:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-07-11T03:52:46.240 DEBUG:teuthology.orchestra.run.gibba004:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:46.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:46.256+0000 7f81eea80d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2021-07-11T03:52:46.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:46.256+0000 7f81eea80d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2021-07-11T03:52:46.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:46.256+0000 7f81eea80d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-07-11T03:52:46.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:46.256+0000 7f81eea80d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-07-11T03:52:46.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:46.256+0000 7f81eea80d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2021-07-11T03:52:46.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:46.256+0000 7f81eea80d80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2021-07-11T03:52:46.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:46.264+0000 7f81eea80d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-07-11T03:52:48.726 DEBUG:teuthology.orchestra.run.gibba004:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-07-11T03:52:48.776 DEBUG:teuthology.orchestra.run.gibba004:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:48.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:48.796+0000 7f3df529dd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2021-07-11T03:52:48.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:48.796+0000 7f3df529dd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2021-07-11T03:52:48.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:48.796+0000 7f3df529dd80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-07-11T03:52:48.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:48.796+0000 7f3df529dd80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-07-11T03:52:48.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:48.796+0000 7f3df529dd80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2021-07-11T03:52:48.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:48.796+0000 7f3df529dd80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2021-07-11T03:52:48.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:48.804+0000 7f3df529dd80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-07-11T03:52:51.282 DEBUG:teuthology.orchestra.run.gibba004:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-07-11T03:52:51.293 DEBUG:teuthology.orchestra.run.gibba004:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:51.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:51.308+0000 7f6cca430d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2021-07-11T03:52:51.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:51.308+0000 7f6cca430d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2021-07-11T03:52:51.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:51.308+0000 7f6cca430d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-07-11T03:52:51.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:51.308+0000 7f6cca430d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-07-11T03:52:51.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:51.308+0000 7f6cca430d80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2021-07-11T03:52:51.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:51.308+0000 7f6cca430d80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2021-07-11T03:52:51.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:52:51.316+0000 7f6cca430d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-07-11T03:52:53.798 DEBUG:teuthology.orchestra.run.gibba004:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-07-11T03:52:53.852 DEBUG:teuthology.orchestra.run.gibba005:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2021-07-11T03:52:53.863 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-07-11T03:52:53.864 INFO:tasks.ceph:role: osd.4 2021-07-11T03:52:53.864 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T03:52:53.865 DEBUG:teuthology.orchestra.run.gibba005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-07-11T03:52:53.902 INFO:teuthology.orchestra.run.gibba005.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5505024 blks 2021-07-11T03:52:53.903 INFO:teuthology.orchestra.run.gibba005.stdout: = sectsz=512 attr=2, projid32bit=1 2021-07-11T03:52:53.903 INFO:teuthology.orchestra.run.gibba005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-07-11T03:52:53.903 INFO:teuthology.orchestra.run.gibba005.stdout: = reflink=1 2021-07-11T03:52:53.903 INFO:teuthology.orchestra.run.gibba005.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-07-11T03:52:53.904 INFO:teuthology.orchestra.run.gibba005.stdout: = sunit=0 swidth=0 blks 2021-07-11T03:52:53.904 INFO:teuthology.orchestra.run.gibba005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-07-11T03:52:53.904 INFO:teuthology.orchestra.run.gibba005.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-07-11T03:52:53.904 INFO:teuthology.orchestra.run.gibba005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-07-11T03:52:53.904 INFO:teuthology.orchestra.run.gibba005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-07-11T03:52:53.906 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@gibba005.front.sepia.ceph.com -o noatime 2021-07-11T03:52:53.906 DEBUG:teuthology.orchestra.run.gibba005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2021-07-11T03:52:53.967 DEBUG:teuthology.orchestra.run.gibba005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2021-07-11T03:52:53.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:52:53.983 INFO:teuthology.orchestra.run.gibba005.stderr:sudo: /sbin/restorecon: command not found 2021-07-11T03:52:53.983 DEBUG:teuthology.orchestra.run.gibba005:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2021-07-11T03:52:54.002 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-07-11T03:52:54.003 INFO:tasks.ceph:role: osd.5 2021-07-11T03:52:54.003 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T03:52:54.003 DEBUG:teuthology.orchestra.run.gibba005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-07-11T03:52:54.091 INFO:teuthology.orchestra.run.gibba005.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5505024 blks 2021-07-11T03:52:54.091 INFO:teuthology.orchestra.run.gibba005.stdout: = sectsz=512 attr=2, projid32bit=1 2021-07-11T03:52:54.092 INFO:teuthology.orchestra.run.gibba005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-07-11T03:52:54.092 INFO:teuthology.orchestra.run.gibba005.stdout: = reflink=1 2021-07-11T03:52:54.092 INFO:teuthology.orchestra.run.gibba005.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-07-11T03:52:54.092 INFO:teuthology.orchestra.run.gibba005.stdout: = sunit=0 swidth=0 blks 2021-07-11T03:52:54.093 INFO:teuthology.orchestra.run.gibba005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-07-11T03:52:54.093 INFO:teuthology.orchestra.run.gibba005.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-07-11T03:52:54.093 INFO:teuthology.orchestra.run.gibba005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-07-11T03:52:54.093 INFO:teuthology.orchestra.run.gibba005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-07-11T03:52:54.094 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@gibba005.front.sepia.ceph.com -o noatime 2021-07-11T03:52:54.095 DEBUG:teuthology.orchestra.run.gibba005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2021-07-11T03:52:54.111 DEBUG:teuthology.orchestra.run.gibba005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2021-07-11T03:52:54.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:52:54.131 INFO:teuthology.orchestra.run.gibba005.stderr:sudo: /sbin/restorecon: command not found 2021-07-11T03:52:54.131 DEBUG:teuthology.orchestra.run.gibba005:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2021-07-11T03:52:54.185 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-07-11T03:52:54.185 INFO:tasks.ceph:role: osd.6 2021-07-11T03:52:54.186 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T03:52:54.186 DEBUG:teuthology.orchestra.run.gibba005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-07-11T03:52:54.228 INFO:teuthology.orchestra.run.gibba005.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5505024 blks 2021-07-11T03:52:54.229 INFO:teuthology.orchestra.run.gibba005.stdout: = sectsz=512 attr=2, projid32bit=1 2021-07-11T03:52:54.229 INFO:teuthology.orchestra.run.gibba005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-07-11T03:52:54.229 INFO:teuthology.orchestra.run.gibba005.stdout: = reflink=1 2021-07-11T03:52:54.229 INFO:teuthology.orchestra.run.gibba005.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-07-11T03:52:54.229 INFO:teuthology.orchestra.run.gibba005.stdout: = sunit=0 swidth=0 blks 2021-07-11T03:52:54.230 INFO:teuthology.orchestra.run.gibba005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-07-11T03:52:54.230 INFO:teuthology.orchestra.run.gibba005.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-07-11T03:52:54.231 INFO:teuthology.orchestra.run.gibba005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-07-11T03:52:54.231 INFO:teuthology.orchestra.run.gibba005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-07-11T03:52:54.233 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@gibba005.front.sepia.ceph.com -o noatime 2021-07-11T03:52:54.233 DEBUG:teuthology.orchestra.run.gibba005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2021-07-11T03:52:54.302 DEBUG:teuthology.orchestra.run.gibba005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2021-07-11T03:52:54.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:52:54.307 INFO:teuthology.orchestra.run.gibba005.stderr:sudo: /sbin/restorecon: command not found 2021-07-11T03:52:54.307 DEBUG:teuthology.orchestra.run.gibba005:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2021-07-11T03:52:54.314 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2021-07-11T03:52:54.314 INFO:tasks.ceph:role: osd.7 2021-07-11T03:52:54.314 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T03:52:54.315 DEBUG:teuthology.orchestra.run.gibba005:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2021-07-11T03:52:54.403 INFO:teuthology.orchestra.run.gibba005.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5505024 blks 2021-07-11T03:52:54.404 INFO:teuthology.orchestra.run.gibba005.stdout: = sectsz=512 attr=2, projid32bit=1 2021-07-11T03:52:54.404 INFO:teuthology.orchestra.run.gibba005.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2021-07-11T03:52:54.404 INFO:teuthology.orchestra.run.gibba005.stdout: = reflink=1 2021-07-11T03:52:54.405 INFO:teuthology.orchestra.run.gibba005.stdout:data = bsize=4096 blocks=22020096, imaxpct=25 2021-07-11T03:52:54.405 INFO:teuthology.orchestra.run.gibba005.stdout: = sunit=0 swidth=0 blks 2021-07-11T03:52:54.405 INFO:teuthology.orchestra.run.gibba005.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2021-07-11T03:52:54.405 INFO:teuthology.orchestra.run.gibba005.stdout:log =internal log bsize=4096 blocks=10752, version=2 2021-07-11T03:52:54.405 INFO:teuthology.orchestra.run.gibba005.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-07-11T03:52:54.406 INFO:teuthology.orchestra.run.gibba005.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-07-11T03:52:54.407 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@gibba005.front.sepia.ceph.com -o noatime 2021-07-11T03:52:54.407 DEBUG:teuthology.orchestra.run.gibba005:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2021-07-11T03:52:54.439 DEBUG:teuthology.orchestra.run.gibba005:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2021-07-11T03:52:54.454 INFO:teuthology.orchestra.run.gibba005.stderr:sudo: /sbin/restorecon: command not found 2021-07-11T03:52:54.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:52:54.456 DEBUG:teuthology.orchestra.run.gibba005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:54.540 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:54.537+0000 7f3c42f15d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2021-07-11T03:52:54.541 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:54.537+0000 7f3c42f15d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2021-07-11T03:52:54.541 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:54.537+0000 7f3c42f15d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-07-11T03:52:54.541 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:54.537+0000 7f3c42f15d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-07-11T03:52:54.541 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:54.537+0000 7f3c42f15d80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2021-07-11T03:52:54.542 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:54.537+0000 7f3c42f15d80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2021-07-11T03:52:54.549 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:54.545+0000 7f3c42f15d80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-07-11T03:52:56.991 DEBUG:teuthology.orchestra.run.gibba005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-07-11T03:52:56.997 DEBUG:teuthology.orchestra.run.gibba005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:57.015 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:57.013+0000 7f6d48e3dd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2021-07-11T03:52:57.016 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:57.013+0000 7f6d48e3dd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2021-07-11T03:52:57.016 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:57.013+0000 7f6d48e3dd80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-07-11T03:52:57.016 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:57.013+0000 7f6d48e3dd80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-07-11T03:52:57.017 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:57.013+0000 7f6d48e3dd80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2021-07-11T03:52:57.017 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:57.013+0000 7f6d48e3dd80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2021-07-11T03:52:57.026 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:57.021+0000 7f6d48e3dd80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-07-11T03:52:59.500 DEBUG:teuthology.orchestra.run.gibba005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-07-11T03:52:59.552 DEBUG:teuthology.orchestra.run.gibba005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:52:59.569 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:59.565+0000 7f939ee1cd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2021-07-11T03:52:59.570 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:59.565+0000 7f939ee1cd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2021-07-11T03:52:59.570 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:59.565+0000 7f939ee1cd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-07-11T03:52:59.570 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:59.565+0000 7f939ee1cd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-07-11T03:52:59.570 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:59.565+0000 7f939ee1cd80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2021-07-11T03:52:59.571 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:59.569+0000 7f939ee1cd80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2021-07-11T03:52:59.577 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:52:59.573+0000 7f939ee1cd80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-07-11T03:53:02.047 DEBUG:teuthology.orchestra.run.gibba005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2021-07-11T03:53:02.060 DEBUG:teuthology.orchestra.run.gibba005:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:53:02.076 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:02.073+0000 7fee617dad80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2021-07-11T03:53:02.076 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:02.073+0000 7fee617dad80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2021-07-11T03:53:02.077 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:02.073+0000 7fee617dad80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-07-11T03:53:02.077 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:02.073+0000 7fee617dad80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-07-11T03:53:02.077 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:02.073+0000 7fee617dad80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2021-07-11T03:53:02.078 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:02.073+0000 7fee617dad80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2021-07-11T03:53:02.084 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:02.081+0000 7fee617dad80 -1 freelist read_size_meta_from_db missing size meta in DB 2021-07-11T03:53:04.571 DEBUG:teuthology.orchestra.run.gibba005:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2021-07-11T03:53:04.629 INFO:tasks.ceph:Reading keys from all nodes... 2021-07-11T03:53:04.630 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:53:04.638 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.QEpMa1VaIN 2021-07-11T03:53:04.639 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.QEpMa1VaIN 2021-07-11T03:53:04.692 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.QEpMa1VaIN 2021-07-11T03:53:04.706 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.QEpMa1VaIN is 56B 2021-07-11T03:53:04.726 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.QEpMa1VaIN 2021-07-11T03:53:04.735 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:53:04.786 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.PbCSjl3GNx 2021-07-11T03:53:04.786 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /var/lib/ceph/mds/ceph-a/keyring /tmp/tmp.PbCSjl3GNx 2021-07-11T03:53:04.808 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.PbCSjl3GNx 2021-07-11T03:53:04.889 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.PbCSjl3GNx is 56B 2021-07-11T03:53:04.903 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.PbCSjl3GNx 2021-07-11T03:53:04.907 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:53:04.953 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.WNLvcqVEeU 2021-07-11T03:53:04.953 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /var/lib/ceph/mds/ceph-c/keyring /tmp/tmp.WNLvcqVEeU 2021-07-11T03:53:04.965 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.WNLvcqVEeU 2021-07-11T03:53:05.036 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.WNLvcqVEeU is 56B 2021-07-11T03:53:05.057 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.WNLvcqVEeU 2021-07-11T03:53:05.066 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:53:05.117 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.zm2dNyYn2P 2021-07-11T03:53:05.117 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.zm2dNyYn2P 2021-07-11T03:53:05.140 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.zm2dNyYn2P 2021-07-11T03:53:05.217 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.zm2dNyYn2P is 56B 2021-07-11T03:53:05.232 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.zm2dNyYn2P 2021-07-11T03:53:05.237 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:53:05.285 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.QXJwtLDcxV 2021-07-11T03:53:05.286 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.QXJwtLDcxV 2021-07-11T03:53:05.308 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.QXJwtLDcxV 2021-07-11T03:53:05.385 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.QXJwtLDcxV is 56B 2021-07-11T03:53:05.402 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.QXJwtLDcxV 2021-07-11T03:53:05.406 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:53:05.454 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.WnAGipfx82 2021-07-11T03:53:05.455 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.WnAGipfx82 2021-07-11T03:53:05.469 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.WnAGipfx82 2021-07-11T03:53:05.541 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.WnAGipfx82 is 56B 2021-07-11T03:53:05.551 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.WnAGipfx82 2021-07-11T03:53:05.559 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:53:05.567 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.zd5ui9WlW1 2021-07-11T03:53:05.568 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.zd5ui9WlW1 2021-07-11T03:53:05.620 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.zd5ui9WlW1 2021-07-11T03:53:05.634 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.zd5ui9WlW1 is 56B 2021-07-11T03:53:05.690 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.zd5ui9WlW1 2021-07-11T03:53:05.699 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T03:53:05.708 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.e0Op8hcSrO 2021-07-11T03:53:05.709 DEBUG:teuthology.orchestra.run.gibba005:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.e0Op8hcSrO 2021-07-11T03:53:05.716 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.e0Op8hcSrO 2021-07-11T03:53:05.775 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.e0Op8hcSrO is 56B 2021-07-11T03:53:05.829 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.e0Op8hcSrO 2021-07-11T03:53:05.838 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T03:53:05.842 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.f1oFMGuBgv 2021-07-11T03:53:05.842 DEBUG:teuthology.orchestra.run.gibba005:> sudo cp /var/lib/ceph/mds/ceph-b/keyring /tmp/tmp.f1oFMGuBgv 2021-07-11T03:53:05.895 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.f1oFMGuBgv 2021-07-11T03:53:05.916 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.f1oFMGuBgv is 56B 2021-07-11T03:53:05.969 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.f1oFMGuBgv 2021-07-11T03:53:05.974 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T03:53:06.022 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.X2GG6QuP3n 2021-07-11T03:53:06.022 DEBUG:teuthology.orchestra.run.gibba005:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.X2GG6QuP3n 2021-07-11T03:53:06.044 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.X2GG6QuP3n 2021-07-11T03:53:06.122 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.X2GG6QuP3n is 56B 2021-07-11T03:53:06.141 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.X2GG6QuP3n 2021-07-11T03:53:06.150 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T03:53:06.202 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.niR3IHUbDG 2021-07-11T03:53:06.203 DEBUG:teuthology.orchestra.run.gibba005:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.niR3IHUbDG 2021-07-11T03:53:06.227 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.niR3IHUbDG 2021-07-11T03:53:06.306 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.niR3IHUbDG is 56B 2021-07-11T03:53:06.326 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.niR3IHUbDG 2021-07-11T03:53:06.335 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T03:53:06.386 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.v3HULo7SAA 2021-07-11T03:53:06.387 DEBUG:teuthology.orchestra.run.gibba005:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.v3HULo7SAA 2021-07-11T03:53:06.409 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.v3HULo7SAA 2021-07-11T03:53:06.486 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.v3HULo7SAA is 56B 2021-07-11T03:53:06.501 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.v3HULo7SAA 2021-07-11T03:53:06.506 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T03:53:06.551 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.vSSo1B5t7d 2021-07-11T03:53:06.552 DEBUG:teuthology.orchestra.run.gibba005:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.vSSo1B5t7d 2021-07-11T03:53:06.564 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.vSSo1B5t7d 2021-07-11T03:53:06.635 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.vSSo1B5t7d is 56B 2021-07-11T03:53:06.652 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.vSSo1B5t7d 2021-07-11T03:53:06.668 DEBUG:teuthology.orchestra.remote:gibba019:/etc/ceph/ceph.client.0.keyring is 59B 2021-07-11T03:53:06.731 DEBUG:teuthology.orchestra.remote:gibba023:/etc/ceph/ceph.client.1.keyring is 59B 2021-07-11T03:53:06.784 INFO:tasks.ceph:Adding keys to all mons... 2021-07-11T03:53:06.784 DEBUG:teuthology.orchestra.run.gibba004:> sudo tee -a /etc/ceph/ceph.keyring 2021-07-11T03:53:06.788 DEBUG:teuthology.orchestra.run.gibba005:> sudo tee -a /etc/ceph/ceph.keyring 2021-07-11T03:53:06.795 INFO:teuthology.orchestra.run.gibba004.stdout:[mgr.y] 2021-07-11T03:53:06.796 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ== 2021-07-11T03:53:06.796 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.a] 2021-07-11T03:53:06.797 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q== 2021-07-11T03:53:06.797 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.c] 2021-07-11T03:53:06.797 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA== 2021-07-11T03:53:06.797 INFO:teuthology.orchestra.run.gibba004.stdout:[osd.0] 2021-07-11T03:53:06.798 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A== 2021-07-11T03:53:06.798 INFO:teuthology.orchestra.run.gibba004.stdout:[osd.1] 2021-07-11T03:53:06.798 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw== 2021-07-11T03:53:06.798 INFO:teuthology.orchestra.run.gibba004.stdout:[osd.2] 2021-07-11T03:53:06.799 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ== 2021-07-11T03:53:06.799 INFO:teuthology.orchestra.run.gibba004.stdout:[osd.3] 2021-07-11T03:53:06.799 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw== 2021-07-11T03:53:06.800 INFO:teuthology.orchestra.run.gibba004.stdout:[mgr.x] 2021-07-11T03:53:06.800 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ== 2021-07-11T03:53:06.800 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.b] 2021-07-11T03:53:06.800 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w== 2021-07-11T03:53:06.801 INFO:teuthology.orchestra.run.gibba004.stdout:[osd.4] 2021-07-11T03:53:06.801 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ== 2021-07-11T03:53:06.801 INFO:teuthology.orchestra.run.gibba004.stdout:[osd.5] 2021-07-11T03:53:06.801 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg== 2021-07-11T03:53:06.802 INFO:teuthology.orchestra.run.gibba004.stdout:[osd.6] 2021-07-11T03:53:06.802 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ== 2021-07-11T03:53:06.802 INFO:teuthology.orchestra.run.gibba004.stdout:[osd.7] 2021-07-11T03:53:06.802 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA== 2021-07-11T03:53:06.802 INFO:teuthology.orchestra.run.gibba004.stdout:[client.0] 2021-07-11T03:53:06.803 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw== 2021-07-11T03:53:06.803 INFO:teuthology.orchestra.run.gibba004.stdout:[client.1] 2021-07-11T03:53:06.803 INFO:teuthology.orchestra.run.gibba004.stdout: key = AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg== 2021-07-11T03:53:06.805 INFO:teuthology.orchestra.run.gibba005.stdout:[mgr.y] 2021-07-11T03:53:06.805 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ== 2021-07-11T03:53:06.805 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.a] 2021-07-11T03:53:06.806 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q== 2021-07-11T03:53:06.806 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.c] 2021-07-11T03:53:06.806 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA== 2021-07-11T03:53:06.806 INFO:teuthology.orchestra.run.gibba005.stdout:[osd.0] 2021-07-11T03:53:06.806 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A== 2021-07-11T03:53:06.807 INFO:teuthology.orchestra.run.gibba005.stdout:[osd.1] 2021-07-11T03:53:06.807 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw== 2021-07-11T03:53:06.807 INFO:teuthology.orchestra.run.gibba005.stdout:[osd.2] 2021-07-11T03:53:06.807 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ== 2021-07-11T03:53:06.807 INFO:teuthology.orchestra.run.gibba005.stdout:[osd.3] 2021-07-11T03:53:06.808 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw== 2021-07-11T03:53:06.808 INFO:teuthology.orchestra.run.gibba005.stdout:[mgr.x] 2021-07-11T03:53:06.808 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ== 2021-07-11T03:53:06.808 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.b] 2021-07-11T03:53:06.809 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w== 2021-07-11T03:53:06.809 INFO:teuthology.orchestra.run.gibba005.stdout:[osd.4] 2021-07-11T03:53:06.809 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ== 2021-07-11T03:53:06.809 INFO:teuthology.orchestra.run.gibba005.stdout:[osd.5] 2021-07-11T03:53:06.809 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg== 2021-07-11T03:53:06.810 INFO:teuthology.orchestra.run.gibba005.stdout:[osd.6] 2021-07-11T03:53:06.810 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ== 2021-07-11T03:53:06.810 INFO:teuthology.orchestra.run.gibba005.stdout:[osd.7] 2021-07-11T03:53:06.810 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA== 2021-07-11T03:53:06.810 INFO:teuthology.orchestra.run.gibba005.stdout:[client.0] 2021-07-11T03:53:06.811 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw== 2021-07-11T03:53:06.811 INFO:teuthology.orchestra.run.gibba005.stdout:[client.1] 2021-07-11T03:53:06.811 INFO:teuthology.orchestra.run.gibba005.stdout: key = AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg== 2021-07-11T03:53:06.811 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-07-11T03:53:06.851 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-07-11T03:53:06.899 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-07-11T03:53:06.902 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-07-11T03:53:06.945 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-07-11T03:53:06.967 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-07-11T03:53:07.011 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.014 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.035 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.079 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.107 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.110 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.139 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.141 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.236 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-07-11T03:53:07.239 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-07-11T03:53:07.293 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-07-11T03:53:07.296 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2021-07-11T03:53:07.326 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.363 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.428 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.431 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.493 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.496 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.533 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.536 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-07-11T03:53:07.566 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-07-11T03:53:07.599 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-07-11T03:53:07.672 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-07-11T03:53:07.675 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-07-11T03:53:07.698 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-07-11T03:53:07.699 DEBUG:teuthology.orchestra.run.gibba004:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2021-07-11T03:53:07.705 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-07-11T03:53:07.795 DEBUG:teuthology.orchestra.run.gibba004:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2021-07-11T03:53:07.807 DEBUG:teuthology.orchestra.run.gibba005:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2021-07-11T03:53:07.819 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-07-11T03:53:07.908 DEBUG:teuthology.orchestra.run.gibba005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2021-07-11T03:53:07.929 DEBUG:teuthology.orchestra.run.gibba005:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2021-07-11T03:53:07.951 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2021-07-11T03:53:08.001 DEBUG:teuthology.orchestra.run.gibba005:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2021-07-11T03:53:08.011 DEBUG:teuthology.orchestra.run.gibba004:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:53:08.015 DEBUG:teuthology.orchestra.run.gibba005:> rm -- /home/ubuntu/cephtest/ceph.monmap 2021-07-11T03:53:08.020 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2021-07-11T03:53:08.021 INFO:tasks.ceph.mon.a:Restarting daemon 2021-07-11T03:53:08.021 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2021-07-11T03:53:08.064 INFO:tasks.ceph.mon.a:Started 2021-07-11T03:53:08.064 INFO:tasks.ceph.mon.b:Restarting daemon 2021-07-11T03:53:08.065 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2021-07-11T03:53:08.067 INFO:tasks.ceph.mon.b:Started 2021-07-11T03:53:08.068 INFO:tasks.ceph.mon.c:Restarting daemon 2021-07-11T03:53:08.068 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2021-07-11T03:53:08.070 INFO:tasks.ceph.mon.c:Started 2021-07-11T03:53:08.070 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2021-07-11T03:53:08.071 INFO:tasks.ceph.mgr.y:Restarting daemon 2021-07-11T03:53:08.071 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2021-07-11T03:53:08.074 INFO:tasks.ceph.mgr.y:Started 2021-07-11T03:53:08.074 INFO:tasks.ceph.mgr.x:Restarting daemon 2021-07-11T03:53:08.074 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2021-07-11T03:53:08.077 INFO:tasks.ceph.mgr.x:Started 2021-07-11T03:53:08.078 INFO:tasks.ceph:Setting crush tunables to default 2021-07-11T03:53:08.078 DEBUG:teuthology.orchestra.run.gibba004:> sudo ceph --cluster ceph osd crush tunables default 2021-07-11T03:53:08.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61ff59e700 1 Processor -- start 2021-07-11T03:53:08.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61ff59e700 1 -- start start 2021-07-11T03:53:08.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61ff59e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 0x7f6200061700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:08.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61ff59e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f62000ad5c0 0x7f62000b9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:08.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61ff59e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62000aecf0 con 0x7f6200140440 2021-07-11T03:53:08.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61ff59e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62000b6f20 con 0x7f62000ad5c0 2021-07-11T03:53:08.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61ff59e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62000aef20 con 0x7f6200061320 2021-07-11T03:53:08.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fdd9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f62000ad5c0 0x7f62000b9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:08.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fdd9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f62000ad5c0 0x7f62000b9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41166/0 (socket says 172.21.2.104:41166) 2021-07-11T03:53:08.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fdd9b700 1 -- 172.21.2.104:0/3498397245 learned_addr learned my addr 172.21.2.104:0/3498397245 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:08.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fdd9b700 1 -- 172.21.2.104:0/3498397245 >> v2:172.21.2.104:3300/0 conn(0x7f62000ad5c0 msgr2=0x7f62000b9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2021-07-11T03:53:08.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fdd9b700 1 -- 172.21.2.104:0/3498397245 >> v2:172.21.2.104:3300/0 conn(0x7f62000ad5c0 msgr2=0x7f62000b9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-07-11T03:53:08.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fdd9b700 1 --2- 172.21.2.104:0/3498397245 >> v2:172.21.2.104:3300/0 conn(0x7f62000ad5c0 0x7f62000b9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-07-11T03:53:08.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fe59c700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 0x7f6200061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:08.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fdd9b700 1 --2- 172.21.2.104:0/3498397245 >> v2:172.21.2.104:3300/0 conn(0x7f62000ad5c0 0x7f62000b9e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-07-11T03:53:08.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fe59c700 1 -- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 msgr2=0x7f6200061700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2021-07-11T03:53:08.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fe59c700 1 -- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 msgr2=0x7f6200061700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2021-07-11T03:53:08.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fe59c700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 0x7f6200061700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-07-11T03:53:08.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.136+0000 7f61fe59c700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 0x7f6200061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-07-11T03:53:08.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359833208 0 0) 0x7f62000aecf0 con 0x7f6200140440 2021-07-11T03:53:08.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61e4003260 con 0x7f6200140440 2021-07-11T03:53:08.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3626133294 0 0) 0x7f61e4003260 con 0x7f6200140440 2021-07-11T03:53:08.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 msgr2=0x7f6200061700 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-07-11T03:53:08.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 0x7f6200061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:08.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 >> v2:172.21.2.104:3300/0 conn(0x7f62000ad5c0 msgr2=0x7f62000b9e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2021-07-11T03:53:08.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 --2- 172.21.2.104:0/3498397245 >> v2:172.21.2.104:3300/0 conn(0x7f62000ad5c0 0x7f62000b9e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:08.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62000af1c0 con 0x7f6200140440 2021-07-11T03:53:08.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f61f0001660 con 0x7f6200140440 2021-07-11T03:53:08.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 >> v1:172.21.2.104:6789/0 conn(0x7f6200140440 legacy=0x7f6200146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:08.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 0x7f61e40037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:08.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e4003e00 0x7f61e4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:08.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61e40049f0 0x7f61e4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:08.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62000b6f20 con 0x7f61e4003e00 2021-07-11T03:53:08.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62000aef20 con 0x7f61e40049f0 2021-07-11T03:53:08.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e400b8e0 con 0x7f6200061320 2021-07-11T03:53:08.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fdd9b700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e4003e00 0x7f61e4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:08.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fed9d700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61e40049f0 0x7f61e4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:08.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.204+0000 7f61fe59c700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 0x7f61e40037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:08.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.208+0000 7f61fed9d700 1 -- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 msgr2=0x7f61e40037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:08.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.208+0000 7f61fed9d700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6200061320 0x7f61e40037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:08.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.208+0000 7f61fed9d700 1 -- 172.21.2.104:0/3498397245 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e4003e00 msgr2=0x7f61e4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:08.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.208+0000 7f61fed9d700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e4003e00 0x7f61e4004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:08.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.208+0000 7f61fed9d700 1 -- 172.21.2.104:0/3498397245 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e4003260 con 0x7f61e40049f0 2021-07-11T03:53:08.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.208+0000 7f61fdd9b700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e4003e00 0x7f61e4004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T03:53:08.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.208+0000 7f61fed9d700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61e40049f0 0x7f61e4004e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f61f0001370 tx=0x7f62000af350).ready entity=mon.1 client_cookie=d3cad96105e2e2c6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:08.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.212+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f0007030 con 0x7f61e40049f0 2021-07-11T03:53:08.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.212+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f0007190 con 0x7f61e40049f0 2021-07-11T03:53:08.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.212+0000 7f61fd59a700 1 -- 172.21.2.104:0/3498397245 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f000c8f0 con 0x7f61e40049f0 2021-07-11T03:53:08.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.212+0000 7f61ff59e700 1 -- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61e40049f0 msgr2=0x7f61e4004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:08.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.212+0000 7f61ff59e700 1 --2- 172.21.2.104:0/3498397245 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61e40049f0 0x7f61e4004e70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f61f0001370 tx=0x7f62000af350).stop 2021-07-11T03:53:08.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.212+0000 7f61fed9d700 1 -- 172.21.2.104:0/3498397245 reap_dead start 2021-07-11T03:53:08.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.212+0000 7f61ff59e700 1 -- 172.21.2.104:0/3498397245 shutdown_connections 2021-07-11T03:53:08.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.212+0000 7f61ff59e700 1 -- 172.21.2.104:0/3498397245 >> 172.21.2.104:0/3498397245 conn(0x7f6200066de0 msgr2=0x7f62000671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:08.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.212+0000 7f61ff59e700 1 -- 172.21.2.104:0/3498397245 shutdown_connections 2021-07-11T03:53:08.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.212+0000 7f61ff59e700 1 -- 172.21.2.104:0/3498397245 wait complete. 2021-07-11T03:53:08.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61ff59e700 1 Processor -- start 2021-07-11T03:53:08.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61ff59e700 1 -- start start 2021-07-11T03:53:08.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61ff59e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f62000aad20 0x7f62000b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:08.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61ff59e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f62000b3690 0x7f62000b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:08.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61ff59e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62000b2630 0x7f62000ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:08.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61ff59e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62000b1a60 con 0x7f62000b2630 2021-07-11T03:53:08.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61ff59e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6200152dc0 con 0x7f62000aad20 2021-07-11T03:53:08.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61ff59e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62000ab100 con 0x7f62000b3690 2021-07-11T03:53:08.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61fed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62000b2630 0x7f62000ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:08.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61fe59c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f62000aad20 0x7f62000b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:08.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61fdd9b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f62000b3690 0x7f62000b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:08.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61fed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62000b2630 0x7f62000ac6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41194/0 (socket says 172.21.2.104:41194) 2021-07-11T03:53:08.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.220+0000 7f61fed9d700 1 -- 172.21.2.104:0/1924074672 learned_addr learned my addr 172.21.2.104:0/1924074672 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:08.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.224+0000 7f61fed9d700 1 -- 172.21.2.104:0/1924074672 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f62000b3690 msgr2=0x7f62000b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:08.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.224+0000 7f61fed9d700 1 --2- 172.21.2.104:0/1924074672 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f62000b3690 0x7f62000b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:08.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.224+0000 7f61fed9d700 1 -- 172.21.2.104:0/1924074672 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f62000aad20 msgr2=0x7f62000b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:08.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.224+0000 7f61fed9d700 1 --2- 172.21.2.104:0/1924074672 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f62000aad20 0x7f62000b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:08.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.224+0000 7f61fed9d700 1 -- 172.21.2.104:0/1924074672 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6200152ff0 con 0x7f62000b2630 2021-07-11T03:53:08.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.224+0000 7f61fed9d700 1 --2- 172.21.2.104:0/1924074672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62000b2630 0x7f62000ac6f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f62000b9510 tx=0x7f61f0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:08.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.228+0000 7f61ef7fe700 1 -- 172.21.2.104:0/1924074672 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f0002c80 con 0x7f62000b2630 2021-07-11T03:53:08.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.228+0000 7f61ef7fe700 1 -- 172.21.2.104:0/1924074672 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61f000ccf0 con 0x7f62000b2630 2021-07-11T03:53:08.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.228+0000 7f61ef7fe700 1 -- 172.21.2.104:0/1924074672 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61f0006700 con 0x7f62000b2630 2021-07-11T03:53:08.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.228+0000 7f61ff59e700 1 -- 172.21.2.104:0/1924074672 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62000b15c0 con 0x7f62000b2630 2021-07-11T03:53:08.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.228+0000 7f61ff59e700 1 -- 172.21.2.104:0/1924074672 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62000b2ab0 con 0x7f62000b2630 2021-07-11T03:53:08.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.228+0000 7f61ef7fe700 1 -- 172.21.2.104:0/1924074672 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f61f0023030 con 0x7f62000b2630 2021-07-11T03:53:08.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.228+0000 7f61ef7fe700 1 -- 172.21.2.104:0/1924074672 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f61f0006c20 con 0x7f62000b2630 2021-07-11T03:53:08.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.232+0000 7f61ff59e700 1 -- 172.21.2.104:0/1924074672 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62000b1750 con 0x7f62000b2630 2021-07-11T03:53:08.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.240+0000 7f61ef7fe700 1 -- 172.21.2.104:0/1924074672 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f61f002b060 con 0x7f62000b2630 2021-07-11T03:53:08.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.324+0000 7f61ff59e700 1 -- 172.21.2.104:0/1924074672 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6200057350 con 0x7f62000b2630 2021-07-11T03:53:08.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.328+0000 7f61ef7fe700 1 -- 172.21.2.104:0/1924074672 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f61f0021070 con 0x7f62000b2630 2021-07-11T03:53:08.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.328+0000 7f61ff59e700 1 -- 172.21.2.104:0/1924074672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62000b2630 msgr2=0x7f62000ac6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:08.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.328+0000 7f61ff59e700 1 --2- 172.21.2.104:0/1924074672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62000b2630 0x7f62000ac6f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f62000b9510 tx=0x7f61f0014000).stop 2021-07-11T03:53:08.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.328+0000 7f61ff59e700 1 -- 172.21.2.104:0/1924074672 shutdown_connections 2021-07-11T03:53:08.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.328+0000 7f61ff59e700 1 --2- 172.21.2.104:0/1924074672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62000b2630 0x7f62000ac6f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:08.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.328+0000 7f61ff59e700 1 --2- 172.21.2.104:0/1924074672 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f62000b3690 0x7f62000b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:08.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.328+0000 7f61ff59e700 1 --2- 172.21.2.104:0/1924074672 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f62000aad20 0x7f62000b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:08.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.328+0000 7f61ff59e700 1 -- 172.21.2.104:0/1924074672 >> 172.21.2.104:0/1924074672 conn(0x7f6200066de0 msgr2=0x7f6200140e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:08.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.328+0000 7f61ff59e700 1 -- 172.21.2.104:0/1924074672 shutdown_connections 2021-07-11T03:53:08.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:08.328+0000 7f61ff59e700 1 -- 172.21.2.104:0/1924074672 wait complete. 2021-07-11T03:53:08.333 INFO:teuthology.orchestra.run.gibba004.stderr:adjusted tunables profile to default 2021-07-11T03:53:08.337 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2021-07-11T03:53:08.337 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:53:08.341 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.ncMwkSecTp 2021-07-11T03:53:08.341 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.ncMwkSecTp 2021-07-11T03:53:08.347 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.ncMwkSecTp 2021-07-11T03:53:08.403 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.ncMwkSecTp is 37B 2021-07-11T03:53:08.412 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.ncMwkSecTp 2021-07-11T03:53:08.415 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:53:08.464 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.exvUywAeeL 2021-07-11T03:53:08.464 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.exvUywAeeL 2021-07-11T03:53:08.470 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.exvUywAeeL 2021-07-11T03:53:08.482 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.exvUywAeeL is 37B 2021-07-11T03:53:08.534 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.exvUywAeeL 2021-07-11T03:53:08.538 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:53:08.541 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.jHJKk3gf68 2021-07-11T03:53:08.542 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.jHJKk3gf68 2021-07-11T03:53:08.547 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.jHJKk3gf68 2021-07-11T03:53:08.607 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.jHJKk3gf68 is 37B 2021-07-11T03:53:08.621 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.jHJKk3gf68 2021-07-11T03:53:08.630 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T03:53:08.680 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.fk9nKgcKgC 2021-07-11T03:53:08.680 DEBUG:teuthology.orchestra.run.gibba004:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.fk9nKgcKgC 2021-07-11T03:53:08.687 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.fk9nKgcKgC 2021-07-11T03:53:08.743 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.fk9nKgcKgC is 37B 2021-07-11T03:53:08.752 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.fk9nKgcKgC 2021-07-11T03:53:08.756 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T03:53:08.759 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.SgFD7ILsDu 2021-07-11T03:53:08.760 DEBUG:teuthology.orchestra.run.gibba005:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.SgFD7ILsDu 2021-07-11T03:53:08.808 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.SgFD7ILsDu 2021-07-11T03:53:08.821 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.SgFD7ILsDu is 37B 2021-07-11T03:53:08.874 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.SgFD7ILsDu 2021-07-11T03:53:08.883 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T03:53:08.887 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.m3aqyb1XTz 2021-07-11T03:53:08.887 DEBUG:teuthology.orchestra.run.gibba005:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.m3aqyb1XTz 2021-07-11T03:53:08.892 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.m3aqyb1XTz 2021-07-11T03:53:08.947 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.m3aqyb1XTz is 37B 2021-07-11T03:53:08.958 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.m3aqyb1XTz 2021-07-11T03:53:08.961 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T03:53:09.009 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.ovGoJEsAaY 2021-07-11T03:53:09.010 DEBUG:teuthology.orchestra.run.gibba005:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.ovGoJEsAaY 2021-07-11T03:53:09.017 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.ovGoJEsAaY 2021-07-11T03:53:09.073 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.ovGoJEsAaY is 37B 2021-07-11T03:53:09.082 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.ovGoJEsAaY 2021-07-11T03:53:09.088 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T03:53:09.134 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.Zd1iBkBoWs 2021-07-11T03:53:09.134 DEBUG:teuthology.orchestra.run.gibba005:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.Zd1iBkBoWs 2021-07-11T03:53:09.141 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.Zd1iBkBoWs 2021-07-11T03:53:09.196 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.Zd1iBkBoWs is 37B 2021-07-11T03:53:09.248 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.Zd1iBkBoWs 2021-07-11T03:53:09.252 DEBUG:teuthology.orchestra.run.gibba005:> sudo ceph --cluster ceph osd new a9f3dcb9-1813-4b31-8816-f0e71b6104b6 0 2021-07-11T03:53:09.345 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 Processor -- start 2021-07-11T03:53:09.345 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 -- start start 2021-07-11T03:53:09.346 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7faee405fc00 0x7faee40b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.346 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faee40b5990 0x7faee40b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.346 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee4143f60 con 0x7faee40b6560 2021-07-11T03:53:09.346 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee40ba6b0 con 0x7faee40b5990 2021-07-11T03:53:09.347 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee405ffe0 con 0x7faee405fc00 2021-07-11T03:53:09.347 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee259c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7faee405fc00 0x7faee40b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.347 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee1d9b700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7faee40b6560 0x7faee40b9fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.105:57130/0 (socket says 172.21.2.105:57130) 2021-07-11T03:53:09.347 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee1d9b700 1 -- 172.21.2.105:0/1161862927 learned_addr learned my addr 172.21.2.105:0/1161862927 (peer_addr_for_me v1:172.21.2.105:0/0) 2021-07-11T03:53:09.348 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee2d9d700 1 --2- 172.21.2.105:0/1161862927 >> v2:172.21.2.104:3300/0 conn(0x7faee40b5990 0x7faee40b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.348 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee259c700 1 -- 172.21.2.105:0/1161862927 >> v2:172.21.2.104:3300/0 conn(0x7faee40b5990 msgr2=0x7faee40b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:09.348 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee259c700 1 --2- 172.21.2.105:0/1161862927 >> v2:172.21.2.104:3300/0 conn(0x7faee40b5990 0x7faee40b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.348 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee259c700 1 -- 172.21.2.105:0/1161862927 >> v1:172.21.2.105:6789/0 conn(0x7faee40b6560 legacy=0x7faee40b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.349 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee259c700 1 -- 172.21.2.105:0/1161862927 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee4143a50 con 0x7faee405fc00 2021-07-11T03:53:09.349 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee259c700 1 --2- 172.21.2.105:0/1161862927 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faee405fc00 0x7faee40b41b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7faed4002f70 tx=0x7faed4006df0).ready entity=mon.1 client_cookie=9fa1d35173ec3958 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.350 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee159a700 1 -- 172.21.2.105:0/1161862927 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faed400d5c0 con 0x7faee405fc00 2021-07-11T03:53:09.350 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee159a700 1 -- 172.21.2.105:0/1161862927 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed4000e90 con 0x7faee405fc00 2021-07-11T03:53:09.350 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee159a700 1 -- 172.21.2.105:0/1161862927 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faed400d910 con 0x7faee405fc00 2021-07-11T03:53:09.350 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 -- 172.21.2.105:0/1161862927 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faee405fc00 msgr2=0x7faee40b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.351 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 --2- 172.21.2.105:0/1161862927 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faee405fc00 0x7faee40b41b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7faed4002f70 tx=0x7faed4006df0).stop 2021-07-11T03:53:09.351 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 -- 172.21.2.105:0/1161862927 shutdown_connections 2021-07-11T03:53:09.351 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 --2- 172.21.2.105:0/1161862927 >> v2:172.21.2.104:3300/0 conn(0x7faee40b5990 0x7faee40b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.351 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 --2- 172.21.2.105:0/1161862927 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faee405fc00 0x7faee40b41b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.351 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 -- 172.21.2.105:0/1161862927 >> 172.21.2.105:0/1161862927 conn(0x7faee40623a0 msgr2=0x7faee4066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.352 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 -- 172.21.2.105:0/1161862927 shutdown_connections 2021-07-11T03:53:09.352 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 -- 172.21.2.105:0/1161862927 wait complete. 2021-07-11T03:53:09.355 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 Processor -- start 2021-07-11T03:53:09.355 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.341+0000 7faee882a700 1 -- start start 2021-07-11T03:53:09.355 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee882a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faee405fc00 0x7faee412c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.356 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee882a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faee40b5990 0x7faee41521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.356 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee882a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faee40b6560 0x7faee41526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.356 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee882a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee413fdb0 con 0x7faee40b6560 2021-07-11T03:53:09.356 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee882a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee41564d0 con 0x7faee405fc00 2021-07-11T03:53:09.356 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee882a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee4144720 con 0x7faee40b5990 2021-07-11T03:53:09.357 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee259c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faee405fc00 0x7faee412c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.357 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee259c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faee405fc00 0x7faee412c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.105:40334/0 (socket says 172.21.2.105:40334) 2021-07-11T03:53:09.357 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee259c700 1 -- 172.21.2.105:0/3321500185 learned_addr learned my addr 172.21.2.105:0/3321500185 (peer_addr_for_me v2:172.21.2.105:0/0) 2021-07-11T03:53:09.357 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee2d9d700 1 --2- 172.21.2.105:0/3321500185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faee40b6560 0x7faee41526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.358 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee1d9b700 1 --2- 172.21.2.105:0/3321500185 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faee40b5990 0x7faee41521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.358 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee259c700 1 -- 172.21.2.105:0/3321500185 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faee40b5990 msgr2=0x7faee41521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.358 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee259c700 1 --2- 172.21.2.105:0/3321500185 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faee40b5990 0x7faee41521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.359 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee259c700 1 -- 172.21.2.105:0/3321500185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faee40b6560 msgr2=0x7faee41526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.359 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee259c700 1 --2- 172.21.2.105:0/3321500185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faee40b6560 0x7faee41526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.359 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee259c700 1 -- 172.21.2.105:0/3321500185 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed400b180 con 0x7faee405fc00 2021-07-11T03:53:09.360 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee259c700 1 --2- 172.21.2.105:0/3321500185 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faee405fc00 0x7faee412c1d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7faee4144d40 tx=0x7faed4014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.360 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faedb7fe700 1 -- 172.21.2.105:0/3321500185 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faed4012d50 con 0x7faee405fc00 2021-07-11T03:53:09.360 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faedb7fe700 1 -- 172.21.2.105:0/3321500185 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed401f440 con 0x7faee405fc00 2021-07-11T03:53:09.360 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee882a700 1 -- 172.21.2.105:0/3321500185 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee41449c0 con 0x7faee405fc00 2021-07-11T03:53:09.361 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faedb7fe700 1 -- 172.21.2.105:0/3321500185 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faed401f7a0 con 0x7faee405fc00 2021-07-11T03:53:09.361 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee882a700 1 -- 172.21.2.105:0/3321500185 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee412c6c0 con 0x7faee405fc00 2021-07-11T03:53:09.361 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faedb7fe700 1 -- 172.21.2.105:0/3321500185 <== mon.1 v2:172.21.2.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7faed4025020 con 0x7faee405fc00 2021-07-11T03:53:09.361 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faedb7fe700 1 -- 172.21.2.105:0/3321500185 <== mon.1 v2:172.21.2.105:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7faed401f900 con 0x7faee405fc00 2021-07-11T03:53:09.362 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faee882a700 1 -- 172.21.2.105:0/3321500185 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee40afba0 con 0x7faee405fc00 2021-07-11T03:53:09.362 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.345+0000 7faedb7fe700 1 -- 172.21.2.105:0/3321500185 <== mon.1 v2:172.21.2.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7faed4028440 con 0x7faee405fc00 2021-07-11T03:53:09.393 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.389+0000 7faee882a700 1 -- 172.21.2.105:0/3321500185 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", "id": 0} v 0) v1 -- 0x7faee41566d0 con 0x7faee405fc00 2021-07-11T03:53:09.397 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.393+0000 7faedb7fe700 1 -- 172.21.2.105:0/3321500185 <== mon.1 v2:172.21.2.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7faee41566d0 con 0x7faee405fc00 2021-07-11T03:53:09.398 INFO:teuthology.orchestra.run.gibba005.stdout:0 2021-07-11T03:53:09.399 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.393+0000 7faee882a700 1 -- 172.21.2.105:0/3321500185 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faee405fc00 msgr2=0x7faee412c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.399 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.393+0000 7faee882a700 1 --2- 172.21.2.105:0/3321500185 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faee405fc00 0x7faee412c1d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7faee4144d40 tx=0x7faed4014000).stop 2021-07-11T03:53:09.400 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.393+0000 7faee882a700 1 -- 172.21.2.105:0/3321500185 shutdown_connections 2021-07-11T03:53:09.400 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.393+0000 7faee882a700 1 --2- 172.21.2.105:0/3321500185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faee40b6560 0x7faee41526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.401 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.393+0000 7faee882a700 1 --2- 172.21.2.105:0/3321500185 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faee40b5990 0x7faee41521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.401 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.393+0000 7faee882a700 1 --2- 172.21.2.105:0/3321500185 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faee405fc00 0x7faee412c1d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.401 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.393+0000 7faee882a700 1 -- 172.21.2.105:0/3321500185 >> 172.21.2.105:0/3321500185 conn(0x7faee40623a0 msgr2=0x7faee4149110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.402 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.393+0000 7faee882a700 1 -- 172.21.2.105:0/3321500185 shutdown_connections 2021-07-11T03:53:09.402 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.393+0000 7faee882a700 1 -- 172.21.2.105:0/3321500185 wait complete. 2021-07-11T03:53:09.404 DEBUG:teuthology.orchestra.run.gibba005:> sudo ceph --cluster ceph osd new 61e6a59d-2129-494a-9723-2c792e38c8ed 1 2021-07-11T03:53:09.458 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 Processor -- start 2021-07-11T03:53:09.458 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- start start 2021-07-11T03:53:09.459 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1cbc061b80 0x7f1cbc0b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.459 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1cbc14b760 0x7f1cbc0b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.459 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cbc14ad80 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.459 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cbc14ab90 con 0x7f1cbc061b80 2021-07-11T03:53:09.459 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cbc14b020 con 0x7f1cbc14b760 2021-07-11T03:53:09.460 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc1e20700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1cbc14b760 0x7f1cbc0b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.460 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc1e20700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1cbc14b760 0x7f1cbc0b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.105:40340/0 (socket says 172.21.2.105:40340) 2021-07-11T03:53:09.460 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc1e20700 1 -- 172.21.2.105:0/49953159 learned_addr learned my addr 172.21.2.105:0/49953159 (peer_addr_for_me v2:172.21.2.105:0/0) 2021-07-11T03:53:09.460 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc161f700 1 --2- 172.21.2.105:0/49953159 >> v2:172.21.2.104:3300/0 conn(0x7f1cbc061b80 0x7f1cbc0b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.461 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc1e20700 1 -- 172.21.2.105:0/49953159 >> v2:172.21.2.104:3300/0 conn(0x7f1cbc061b80 msgr2=0x7f1cbc0b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:09.461 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc1e20700 1 --2- 172.21.2.105:0/49953159 >> v2:172.21.2.104:3300/0 conn(0x7f1cbc061b80 0x7f1cbc0b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.461 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc1e20700 1 -- 172.21.2.105:0/49953159 >> v1:172.21.2.104:6789/0 conn(0x7f1cbc0b39d0 legacy=0x7f1cbc144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.461 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc1e20700 1 -- 172.21.2.105:0/49953159 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc14b2c0 con 0x7f1cbc14b760 2021-07-11T03:53:09.461 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc1e20700 1 --2- 172.21.2.105:0/49953159 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cbc14b760 0x7f1cbc0b15f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f1cb40032f0 tx=0x7f1cb4009290).ready entity=mon.1 client_cookie=5c6ad5efe25d0cc5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.462 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1caffff700 1 -- 172.21.2.105:0/49953159 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cb400b870 con 0x7f1cbc14b760 2021-07-11T03:53:09.462 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1caffff700 1 -- 172.21.2.105:0/49953159 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb400b9d0 con 0x7f1cbc14b760 2021-07-11T03:53:09.462 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1caffff700 1 -- 172.21.2.105:0/49953159 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cb400bb40 con 0x7f1cbc14b760 2021-07-11T03:53:09.463 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- 172.21.2.105:0/49953159 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cbc14b760 msgr2=0x7f1cbc0b15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.463 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 --2- 172.21.2.105:0/49953159 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cbc14b760 0x7f1cbc0b15f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f1cb40032f0 tx=0x7f1cb4009290).stop 2021-07-11T03:53:09.463 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- 172.21.2.105:0/49953159 shutdown_connections 2021-07-11T03:53:09.464 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 --2- 172.21.2.105:0/49953159 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cbc14b760 0x7f1cbc0b15f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.464 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 --2- 172.21.2.105:0/49953159 >> v2:172.21.2.104:3300/0 conn(0x7f1cbc061b80 0x7f1cbc0b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.465 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- 172.21.2.105:0/49953159 >> 172.21.2.105:0/49953159 conn(0x7f1cbc0639b0 msgr2=0x7f1cbc063d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.465 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- 172.21.2.105:0/49953159 shutdown_connections 2021-07-11T03:53:09.466 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- 172.21.2.105:0/49953159 wait complete. 2021-07-11T03:53:09.466 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 Processor -- start 2021-07-11T03:53:09.467 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- start start 2021-07-11T03:53:09.467 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cbc061b80 0x7f1cbc143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.467 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cbc0b39d0 0x7f1cbc0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.468 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cbc14b760 0x7f1cbc0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.469 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc066b30 con 0x7f1cbc14b760 2021-07-11T03:53:09.469 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc066d00 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.470 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc3883700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0af1b0 con 0x7f1cbc061b80 2021-07-11T03:53:09.475 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc161f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cbc061b80 0x7f1cbc143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.475 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc161f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cbc061b80 0x7f1cbc143660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.105:50076/0 (socket says 172.21.2.105:50076) 2021-07-11T03:53:09.476 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc0e1e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cbc0b39d0 0x7f1cbc0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.477 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc161f700 1 -- 172.21.2.105:0/3124387624 learned_addr learned my addr 172.21.2.105:0/3124387624 (peer_addr_for_me v2:172.21.2.105:0/0) 2021-07-11T03:53:09.477 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.453+0000 7f1cc1e20700 1 --2- 172.21.2.105:0/3124387624 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cbc14b760 0x7f1cbc0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.477 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cc0e1e700 1 -- 172.21.2.105:0/3124387624 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cbc061b80 msgr2=0x7f1cbc143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.477 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cc0e1e700 1 --2- 172.21.2.105:0/3124387624 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cbc061b80 0x7f1cbc143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.478 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cc0e1e700 1 -- 172.21.2.105:0/3124387624 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cbc14b760 msgr2=0x7f1cbc0afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.478 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cc0e1e700 1 --2- 172.21.2.105:0/3124387624 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cbc14b760 0x7f1cbc0afda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.478 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cc0e1e700 1 -- 172.21.2.105:0/3124387624 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4009530 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.478 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cc1e20700 1 --2- 172.21.2.105:0/3124387624 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cbc14b760 0x7f1cbc0afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:09.479 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cc0e1e700 1 --2- 172.21.2.105:0/3124387624 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cbc0b39d0 0x7f1cbc0af8b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f1cbc0add90 tx=0x7f1cb0007b90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.479 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cae7fc700 1 -- 172.21.2.105:0/3124387624 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cb0012400 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.479 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cae7fc700 1 -- 172.21.2.105:0/3124387624 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb0012560 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.479 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cc3883700 1 -- 172.21.2.105:0/3124387624 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc066890 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.480 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cae7fc700 1 -- 172.21.2.105:0/3124387624 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cb00128c0 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.480 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cc3883700 1 -- 172.21.2.105:0/3124387624 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc066490 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.480 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cae7fc700 1 -- 172.21.2.105:0/3124387624 <== mon.1 v2:172.21.2.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f1cb0019030 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.480 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cae7fc700 1 -- 172.21.2.105:0/3124387624 <== mon.1 v2:172.21.2.105:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f1cb001c440 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.480 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cc3883700 1 -- 172.21.2.105:0/3124387624 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc057350 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.481 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.457+0000 7f1cae7fc700 1 -- 172.21.2.105:0/3124387624 <== mon.1 v2:172.21.2.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f1cb0017070 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.506 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.501+0000 7f1cc3883700 1 -- 172.21.2.105:0/3124387624 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", "id": 1} v 0) v1 -- 0x7f1cbc1444f0 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.510 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.505+0000 7f1cae7fc700 1 -- 172.21.2.105:0/3124387624 <== mon.1 v2:172.21.2.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1cb001c600 con 0x7f1cbc0b39d0 2021-07-11T03:53:09.511 INFO:teuthology.orchestra.run.gibba005.stdout:1 2021-07-11T03:53:09.512 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.509+0000 7f1cc3883700 1 -- 172.21.2.105:0/3124387624 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cbc0b39d0 msgr2=0x7f1cbc0af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.513 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.509+0000 7f1cc3883700 1 --2- 172.21.2.105:0/3124387624 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cbc0b39d0 0x7f1cbc0af8b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f1cbc0add90 tx=0x7f1cb0007b90).stop 2021-07-11T03:53:09.513 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.509+0000 7f1cc3883700 1 -- 172.21.2.105:0/3124387624 shutdown_connections 2021-07-11T03:53:09.513 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.509+0000 7f1cc3883700 1 --2- 172.21.2.105:0/3124387624 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cbc14b760 0x7f1cbc0afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.514 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.509+0000 7f1cc3883700 1 --2- 172.21.2.105:0/3124387624 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cbc0b39d0 0x7f1cbc0af8b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.515 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.509+0000 7f1cc3883700 1 --2- 172.21.2.105:0/3124387624 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cbc061b80 0x7f1cbc143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.515 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.509+0000 7f1cc3883700 1 -- 172.21.2.105:0/3124387624 >> 172.21.2.105:0/3124387624 conn(0x7f1cbc0639b0 msgr2=0x7f1cbc063d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.516 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.509+0000 7f1cc3883700 1 -- 172.21.2.105:0/3124387624 shutdown_connections 2021-07-11T03:53:09.516 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.509+0000 7f1cc3883700 1 -- 172.21.2.105:0/3124387624 wait complete. 2021-07-11T03:53:09.518 DEBUG:teuthology.orchestra.run.gibba005:> sudo ceph --cluster ceph osd new 9b13c5df-5184-4f5e-a553-cc41d8143a5f 2 2021-07-11T03:53:09.570 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.565+0000 7f981c2bb700 1 Processor -- start 2021-07-11T03:53:09.570 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.565+0000 7f981c2bb700 1 -- start start 2021-07-11T03:53:09.571 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.565+0000 7f981c2bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98140b6560 0x7f98140b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.571 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.565+0000 7f981c2bb700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9814143f60 con 0x7f98140b5990 2021-07-11T03:53:09.572 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.565+0000 7f981c2bb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f981405ffe0 con 0x7f981405fc00 2021-07-11T03:53:09.572 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.565+0000 7f981c2bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98140ba6b0 con 0x7f98140b6560 2021-07-11T03:53:09.578 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.565+0000 7f9819856700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98140b6560 0x7f98140b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.578 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.565+0000 7f981a057700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f981405fc00 0x7f98140b9fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.105:57150/0 (socket says 172.21.2.105:57150) 2021-07-11T03:53:09.578 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.565+0000 7f981a057700 1 -- 172.21.2.105:0/3991618939 learned_addr learned my addr 172.21.2.105:0/3991618939 (peer_addr_for_me v1:172.21.2.105:0/0) 2021-07-11T03:53:09.579 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.565+0000 7f9819055700 1 -- 172.21.2.105:0/3991618939 <== mon.1 v1:172.21.2.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982889357 0 0) 0x7f981405ffe0 con 0x7f981405fc00 2021-07-11T03:53:09.579 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819055700 1 -- 172.21.2.105:0/3991618939 --> v1:172.21.2.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97fc003260 con 0x7f981405fc00 2021-07-11T03:53:09.579 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 -- 172.21.2.105:0/3991618939 >> v1:172.21.2.105:6789/0 conn(0x7f981405fc00 legacy=0x7f98140b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.579 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 -- 172.21.2.105:0/3991618939 >> v1:172.21.2.104:6789/0 conn(0x7f98140b5990 legacy=0x7f98140b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:09.580 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 -- 172.21.2.105:0/3991618939 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9814143a50 con 0x7f98140b6560 2021-07-11T03:53:09.580 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 --2- 172.21.2.105:0/3991618939 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98140b6560 0x7f98140b41b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f98140616a0 tx=0x7f9810006df0).ready entity=mon.2 client_cookie=c5cb73e849229470 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.580 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819055700 1 -- 172.21.2.105:0/3991618939 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f981000d570 con 0x7f98140b6560 2021-07-11T03:53:09.580 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819055700 1 -- 172.21.2.105:0/3991618939 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9810000e90 con 0x7f98140b6560 2021-07-11T03:53:09.580 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819055700 1 -- 172.21.2.105:0/3991618939 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f981000d8c0 con 0x7f98140b6560 2021-07-11T03:53:09.581 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- 172.21.2.105:0/3991618939 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98140b6560 msgr2=0x7f98140b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.581 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 --2- 172.21.2.105:0/3991618939 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98140b6560 0x7f98140b41b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f98140616a0 tx=0x7f9810006df0).stop 2021-07-11T03:53:09.581 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- 172.21.2.105:0/3991618939 shutdown_connections 2021-07-11T03:53:09.581 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 --2- 172.21.2.105:0/3991618939 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98140b6560 0x7f98140b41b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.582 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- 172.21.2.105:0/3991618939 >> 172.21.2.105:0/3991618939 conn(0x7f98140623a0 msgr2=0x7f9814066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.582 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- 172.21.2.105:0/3991618939 shutdown_connections 2021-07-11T03:53:09.582 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- 172.21.2.105:0/3991618939 wait complete. 2021-07-11T03:53:09.582 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 Processor -- start 2021-07-11T03:53:09.583 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- start start 2021-07-11T03:53:09.583 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f981405fc00 0x7f9814156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.583 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98140b5990 0x7f98141447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.583 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98140b6560 0x7f9814140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.584 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9814144cd0 con 0x7f981405fc00 2021-07-11T03:53:09.584 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9814155970 con 0x7f98140b5990 2021-07-11T03:53:09.584 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9814155c10 con 0x7f98140b6560 2021-07-11T03:53:09.584 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98140b5990 0x7f98141447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.584 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98140b5990 0x7f98141447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.105:40358/0 (socket says 172.21.2.105:40358) 2021-07-11T03:53:09.585 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 -- 172.21.2.105:0/844404599 learned_addr learned my addr 172.21.2.105:0/844404599 (peer_addr_for_me v2:172.21.2.105:0/0) 2021-07-11T03:53:09.585 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981a858700 1 --2- 172.21.2.105:0/844404599 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98140b6560 0x7f9814140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.585 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981a057700 1 --2- 172.21.2.105:0/844404599 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f981405fc00 0x7f9814156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.585 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 -- 172.21.2.105:0/844404599 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98140b6560 msgr2=0x7f9814140020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.585 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 --2- 172.21.2.105:0/844404599 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98140b6560 0x7f9814140020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.586 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 -- 172.21.2.105:0/844404599 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f981405fc00 msgr2=0x7f9814156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.586 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 --2- 172.21.2.105:0/844404599 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f981405fc00 0x7f9814156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.586 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 -- 172.21.2.105:0/844404599 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981000b180 con 0x7f98140b5990 2021-07-11T03:53:09.586 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981a057700 1 --2- 172.21.2.105:0/844404599 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f981405fc00 0x7f9814156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:53:09.587 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f9819856700 1 --2- 172.21.2.105:0/844404599 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98140b5990 0x7f98141447e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f98140616a0 tx=0x7f9810014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.587 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f98077fe700 1 -- 172.21.2.105:0/844404599 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9810006b20 con 0x7f98140b5990 2021-07-11T03:53:09.587 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f98077fe700 1 -- 172.21.2.105:0/844404599 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9810002f70 con 0x7f98140b5990 2021-07-11T03:53:09.587 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f98077fe700 1 -- 172.21.2.105:0/844404599 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f981001f630 con 0x7f98140b5990 2021-07-11T03:53:09.588 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- 172.21.2.105:0/844404599 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9814155eb0 con 0x7f98140b5990 2021-07-11T03:53:09.588 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- 172.21.2.105:0/844404599 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981413faf0 con 0x7f98140b5990 2021-07-11T03:53:09.588 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f98077fe700 1 -- 172.21.2.105:0/844404599 <== mon.1 v2:172.21.2.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f9810025020 con 0x7f98140b5990 2021-07-11T03:53:09.588 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f98077fe700 1 -- 172.21.2.105:0/844404599 <== mon.1 v2:172.21.2.105:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f981001f960 con 0x7f98140b5990 2021-07-11T03:53:09.588 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.569+0000 7f981c2bb700 1 -- 172.21.2.105:0/844404599 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9814156040 con 0x7f98140b5990 2021-07-11T03:53:09.589 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.573+0000 7f98077fe700 1 -- 172.21.2.105:0/844404599 <== mon.1 v2:172.21.2.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f981003e870 con 0x7f98140b5990 2021-07-11T03:53:09.620 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.617+0000 7f981c2bb700 1 -- 172.21.2.105:0/844404599 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", "id": 2} v 0) v1 -- 0x7f9814057350 con 0x7f98140b5990 2021-07-11T03:53:09.624 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.621+0000 7f98077fe700 1 -- 172.21.2.105:0/844404599 <== mon.1 v2:172.21.2.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f981002e070 con 0x7f98140b5990 2021-07-11T03:53:09.624 INFO:teuthology.orchestra.run.gibba005.stdout:2 2021-07-11T03:53:09.626 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.621+0000 7f981c2bb700 1 -- 172.21.2.105:0/844404599 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98140b5990 msgr2=0x7f98141447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.626 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.621+0000 7f981c2bb700 1 --2- 172.21.2.105:0/844404599 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98140b5990 0x7f98141447e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f98140616a0 tx=0x7f9810014000).stop 2021-07-11T03:53:09.626 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.621+0000 7f981c2bb700 1 -- 172.21.2.105:0/844404599 shutdown_connections 2021-07-11T03:53:09.626 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.621+0000 7f981c2bb700 1 --2- 172.21.2.105:0/844404599 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98140b6560 0x7f9814140020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.627 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.621+0000 7f981c2bb700 1 --2- 172.21.2.105:0/844404599 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98140b5990 0x7f98141447e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.627 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.621+0000 7f981c2bb700 1 --2- 172.21.2.105:0/844404599 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f981405fc00 0x7f9814156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.627 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.621+0000 7f981c2bb700 1 -- 172.21.2.105:0/844404599 >> 172.21.2.105:0/844404599 conn(0x7f98140623a0 msgr2=0x7f98140b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.627 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.621+0000 7f981c2bb700 1 -- 172.21.2.105:0/844404599 shutdown_connections 2021-07-11T03:53:09.627 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.621+0000 7f981c2bb700 1 -- 172.21.2.105:0/844404599 wait complete. 2021-07-11T03:53:09.632 DEBUG:teuthology.orchestra.run.gibba005:> sudo ceph --cluster ceph osd new b75b5b84-a78e-4b67-8044-8ab08825f5fb 3 2021-07-11T03:53:09.683 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 Processor -- start 2021-07-11T03:53:09.684 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 -- start start 2021-07-11T03:53:09.685 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fba000b6560 0x7fba000b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.686 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba00143f60 con 0x7fba0005fc00 2021-07-11T03:53:09.686 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0005ffe0 con 0x7fba000b5990 2021-07-11T03:53:09.686 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba000ba6b0 con 0x7fba000b6560 2021-07-11T03:53:09.687 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fb9ff7fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fba000b6560 0x7fba000b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.687 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba04e8b700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fba000b5990 0x7fba000b4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.105:57166/0 (socket says 172.21.2.105:57166) 2021-07-11T03:53:09.687 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba04e8b700 1 -- 172.21.2.105:0/942702756 learned_addr learned my addr 172.21.2.105:0/942702756 (peer_addr_for_me v1:172.21.2.105:0/0) 2021-07-11T03:53:09.687 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fb9ff7fe700 1 -- 172.21.2.105:0/942702756 >> v1:172.21.2.105:6789/0 conn(0x7fba000b5990 legacy=0x7fba000b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.688 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fb9ff7fe700 1 -- 172.21.2.105:0/942702756 >> v1:172.21.2.104:6789/0 conn(0x7fba0005fc00 legacy=0x7fba000b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.688 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fb9ff7fe700 1 -- 172.21.2.105:0/942702756 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba00143a50 con 0x7fba000b6560 2021-07-11T03:53:09.688 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fb9ff7fe700 1 --2- 172.21.2.105:0/942702756 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba000b6560 0x7fba000b41b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fb9f0002f70 tx=0x7fb9f0006df0).ready entity=mon.1 client_cookie=cc499fc9ba0faec8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.689 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fb9feffd700 1 -- 172.21.2.105:0/942702756 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9f000d610 con 0x7fba000b6560 2021-07-11T03:53:09.689 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fb9feffd700 1 -- 172.21.2.105:0/942702756 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f0000e90 con 0x7fba000b6560 2021-07-11T03:53:09.689 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fb9feffd700 1 -- 172.21.2.105:0/942702756 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9f000d960 con 0x7fba000b6560 2021-07-11T03:53:09.689 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 -- 172.21.2.105:0/942702756 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba000b6560 msgr2=0x7fba000b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.690 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 --2- 172.21.2.105:0/942702756 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba000b6560 0x7fba000b41b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fb9f0002f70 tx=0x7fb9f0006df0).stop 2021-07-11T03:53:09.690 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 -- 172.21.2.105:0/942702756 shutdown_connections 2021-07-11T03:53:09.690 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 --2- 172.21.2.105:0/942702756 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba000b6560 0x7fba000b41b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.690 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 -- 172.21.2.105:0/942702756 >> 172.21.2.105:0/942702756 conn(0x7fba000623a0 msgr2=0x7fba00066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.691 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 -- 172.21.2.105:0/942702756 shutdown_connections 2021-07-11T03:53:09.691 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 -- 172.21.2.105:0/942702756 wait complete. 2021-07-11T03:53:09.691 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 Processor -- start 2021-07-11T03:53:09.691 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 -- start start 2021-07-11T03:53:09.694 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba0005fc00 0x7fba0012d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.694 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba000b5990 0x7fba0013fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.694 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.681+0000 7fba068ee700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba000b6560 0x7fba0013ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.694 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba068ee700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba000b0050 con 0x7fba0005fc00 2021-07-11T03:53:09.695 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba068ee700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba000af000 con 0x7fba000b6560 2021-07-11T03:53:09.695 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba068ee700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba00144cf0 con 0x7fba000b5990 2021-07-11T03:53:09.695 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba04e8b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba000b6560 0x7fba0013ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.695 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba04e8b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba000b6560 0x7fba0013ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.105:40372/0 (socket says 172.21.2.105:40372) 2021-07-11T03:53:09.696 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba04e8b700 1 -- 172.21.2.105:0/4135882724 learned_addr learned my addr 172.21.2.105:0/4135882724 (peer_addr_for_me v2:172.21.2.105:0/0) 2021-07-11T03:53:09.696 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fb9ff7fe700 1 --2- 172.21.2.105:0/4135882724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba000b5990 0x7fba0013fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.696 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fb9fffff700 1 --2- 172.21.2.105:0/4135882724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba0005fc00 0x7fba0012d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.696 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba04e8b700 1 -- 172.21.2.105:0/4135882724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba000b5990 msgr2=0x7fba0013fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.697 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba04e8b700 1 --2- 172.21.2.105:0/4135882724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba000b5990 0x7fba0013fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.697 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba04e8b700 1 -- 172.21.2.105:0/4135882724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba0005fc00 msgr2=0x7fba0012d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.697 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba04e8b700 1 --2- 172.21.2.105:0/4135882724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba0005fc00 0x7fba0012d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.697 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba04e8b700 1 -- 172.21.2.105:0/4135882724 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f000b180 con 0x7fba000b6560 2021-07-11T03:53:09.697 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba04e8b700 1 --2- 172.21.2.105:0/4135882724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba000b6560 0x7fba0013ff00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fba000afcd0 tx=0x7fb9f8007f00).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.698 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fb9fd7fa700 1 -- 172.21.2.105:0/4135882724 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9f8005ba0 con 0x7fba000b6560 2021-07-11T03:53:09.698 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fb9fd7fa700 1 -- 172.21.2.105:0/4135882724 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f8005d00 con 0x7fba000b6560 2021-07-11T03:53:09.698 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fb9fd7fa700 1 -- 172.21.2.105:0/4135882724 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9f80072c0 con 0x7fba000b6560 2021-07-11T03:53:09.698 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba068ee700 1 -- 172.21.2.105:0/4135882724 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba001563a0 con 0x7fba000b6560 2021-07-11T03:53:09.699 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba068ee700 1 -- 172.21.2.105:0/4135882724 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba000afdb0 con 0x7fba000b6560 2021-07-11T03:53:09.699 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fb9fd7fa700 1 -- 172.21.2.105:0/4135882724 <== mon.1 v2:172.21.2.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fb9f8010070 con 0x7fba000b6560 2021-07-11T03:53:09.700 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fb9fd7fa700 1 -- 172.21.2.105:0/4135882724 <== mon.1 v2:172.21.2.105:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7fb9f8013910 con 0x7fba000b6560 2021-07-11T03:53:09.700 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fba068ee700 1 -- 172.21.2.105:0/4135882724 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba00156530 con 0x7fba000b6560 2021-07-11T03:53:09.700 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.685+0000 7fb9fd7fa700 1 -- 172.21.2.105:0/4135882724 <== mon.1 v2:172.21.2.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fb9f8013bb0 con 0x7fba000b6560 2021-07-11T03:53:09.734 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.729+0000 7fba068ee700 1 -- 172.21.2.105:0/4135882724 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", "id": 3} v 0) v1 -- 0x7fba001561e0 con 0x7fba000b6560 2021-07-11T03:53:09.741 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.737+0000 7fb9fd7fa700 1 -- 172.21.2.105:0/4135882724 <== mon.1 v2:172.21.2.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb9f8013dd0 con 0x7fba000b6560 2021-07-11T03:53:09.742 INFO:teuthology.orchestra.run.gibba005.stdout:3 2021-07-11T03:53:09.743 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.737+0000 7fba068ee700 1 -- 172.21.2.105:0/4135882724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba000b6560 msgr2=0x7fba0013ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.743 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.737+0000 7fba068ee700 1 --2- 172.21.2.105:0/4135882724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba000b6560 0x7fba0013ff00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fba000afcd0 tx=0x7fb9f8007f00).stop 2021-07-11T03:53:09.744 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.737+0000 7fba068ee700 1 -- 172.21.2.105:0/4135882724 shutdown_connections 2021-07-11T03:53:09.744 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.737+0000 7fba068ee700 1 --2- 172.21.2.105:0/4135882724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba000b6560 0x7fba0013ff00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.744 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.737+0000 7fba068ee700 1 --2- 172.21.2.105:0/4135882724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba000b5990 0x7fba0013fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.744 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.737+0000 7fba068ee700 1 --2- 172.21.2.105:0/4135882724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba0005fc00 0x7fba0012d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.745 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.737+0000 7fba068ee700 1 -- 172.21.2.105:0/4135882724 >> 172.21.2.105:0/4135882724 conn(0x7fba000623a0 msgr2=0x7fba0012bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.745 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.741+0000 7fba068ee700 1 -- 172.21.2.105:0/4135882724 shutdown_connections 2021-07-11T03:53:09.745 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.741+0000 7fba068ee700 1 -- 172.21.2.105:0/4135882724 wait complete. 2021-07-11T03:53:09.748 DEBUG:teuthology.orchestra.run.gibba005:> sudo ceph --cluster ceph osd new 70205843-19d1-4c4f-a9e4-6aa9ebfa4c28 4 2021-07-11T03:53:09.801 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3f254700 1 Processor -- start 2021-07-11T03:53:09.801 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3f254700 1 -- start start 2021-07-11T03:53:09.801 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3f254700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcd380b6560 0x7fcd380b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.802 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3f254700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd380b5990 0x7fcd380b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.802 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3f254700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd38143f60 con 0x7fcd3805fc00 2021-07-11T03:53:09.802 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3f254700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd380ba6b0 con 0x7fcd380b6560 2021-07-11T03:53:09.802 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3f254700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3805ffe0 con 0x7fcd380b5990 2021-07-11T03:53:09.806 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3d7f1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd380b5990 0x7fcd380b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.807 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3d7f1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd380b5990 0x7fcd380b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.105:50106/0 (socket says 172.21.2.105:50106) 2021-07-11T03:53:09.807 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd37fff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcd380b6560 0x7fcd380b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.807 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3d7f1700 1 -- 172.21.2.105:0/3912068508 learned_addr learned my addr 172.21.2.105:0/3912068508 (peer_addr_for_me v2:172.21.2.105:0/0) 2021-07-11T03:53:09.807 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd37fff700 1 -- 172.21.2.105:0/3912068508 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd380b5990 msgr2=0x7fcd380b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:09.808 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd37fff700 1 --2- 172.21.2.105:0/3912068508 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd380b5990 0x7fcd380b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.808 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd37fff700 1 -- 172.21.2.105:0/3912068508 >> v1:172.21.2.105:6789/0 conn(0x7fcd3805fc00 legacy=0x7fcd380b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.808 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd37fff700 1 -- 172.21.2.105:0/3912068508 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd38143a50 con 0x7fcd380b6560 2021-07-11T03:53:09.808 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd37fff700 1 --2- 172.21.2.105:0/3912068508 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd380b6560 0x7fcd380b41b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fcd20001110 tx=0x7fcd28006df0).ready entity=mon.1 client_cookie=1764b597cc099f96 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.809 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd377fe700 1 -- 172.21.2.105:0/3912068508 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2800d4a0 con 0x7fcd380b6560 2021-07-11T03:53:09.809 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd377fe700 1 -- 172.21.2.105:0/3912068508 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd28000e90 con 0x7fcd380b6560 2021-07-11T03:53:09.809 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd377fe700 1 -- 172.21.2.105:0/3912068508 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2800d7f0 con 0x7fcd380b6560 2021-07-11T03:53:09.809 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3f254700 1 -- 172.21.2.105:0/3912068508 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd380b6560 msgr2=0x7fcd380b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.809 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.797+0000 7fcd3f254700 1 --2- 172.21.2.105:0/3912068508 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd380b6560 0x7fcd380b41b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7fcd20001110 tx=0x7fcd28006df0).stop 2021-07-11T03:53:09.810 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 -- 172.21.2.105:0/3912068508 shutdown_connections 2021-07-11T03:53:09.810 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 --2- 172.21.2.105:0/3912068508 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd380b5990 0x7fcd380b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.810 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 --2- 172.21.2.105:0/3912068508 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd380b6560 0x7fcd380b41b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.811 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 -- 172.21.2.105:0/3912068508 >> 172.21.2.105:0/3912068508 conn(0x7fcd380623a0 msgr2=0x7fcd38066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.811 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 -- 172.21.2.105:0/3912068508 shutdown_connections 2021-07-11T03:53:09.811 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 -- 172.21.2.105:0/3912068508 wait complete. 2021-07-11T03:53:09.811 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 Processor -- start 2021-07-11T03:53:09.812 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 -- start start 2021-07-11T03:53:09.812 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd3805fc00 0x7fcd38128870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.812 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd380b5990 0x7fcd380afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.812 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd380b6560 0x7fcd380affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.813 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3812c2f0 con 0x7fcd380b5990 2021-07-11T03:53:09.813 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380ae370 con 0x7fcd380b6560 2021-07-11T03:53:09.813 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380ae610 con 0x7fcd3805fc00 2021-07-11T03:53:09.813 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3cff0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd3805fc00 0x7fcd38128870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.814 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3d7f1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd380b6560 0x7fcd380affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.814 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3cff0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd3805fc00 0x7fcd38128870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.105:50112/0 (socket says 172.21.2.105:50112) 2021-07-11T03:53:09.814 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3d7f1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd380b6560 0x7fcd380affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.105:40386/0 (socket says 172.21.2.105:40386) 2021-07-11T03:53:09.814 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3cff0700 1 -- 172.21.2.105:0/2892011443 learned_addr learned my addr 172.21.2.105:0/2892011443 (peer_addr_for_me v2:172.21.2.105:0/0) 2021-07-11T03:53:09.814 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd37fff700 1 --2- 172.21.2.105:0/2892011443 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd380b5990 0x7fcd380afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.815 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3cff0700 1 -- 172.21.2.105:0/2892011443 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd380b6560 msgr2=0x7fcd380affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.815 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3cff0700 1 --2- 172.21.2.105:0/2892011443 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd380b6560 0x7fcd380affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.815 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3cff0700 1 -- 172.21.2.105:0/2892011443 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd380b5990 msgr2=0x7fcd380afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.815 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3cff0700 1 --2- 172.21.2.105:0/2892011443 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd380b5990 0x7fcd380afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.816 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3cff0700 1 -- 172.21.2.105:0/2892011443 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2800b180 con 0x7fcd3805fc00 2021-07-11T03:53:09.816 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3d7f1700 1 --2- 172.21.2.105:0/2892011443 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd380b6560 0x7fcd380affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T03:53:09.816 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3cff0700 1 --2- 172.21.2.105:0/2892011443 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd3805fc00 0x7fcd38128870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fcd200011d0 tx=0x7fcd20007c00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.817 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd35ffb700 1 -- 172.21.2.105:0/2892011443 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd20007240 con 0x7fcd3805fc00 2021-07-11T03:53:09.817 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd35ffb700 1 -- 172.21.2.105:0/2892011443 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd200073a0 con 0x7fcd3805fc00 2021-07-11T03:53:09.817 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd35ffb700 1 -- 172.21.2.105:0/2892011443 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd20006700 con 0x7fcd3805fc00 2021-07-11T03:53:09.817 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 -- 172.21.2.105:0/2892011443 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3812c590 con 0x7fcd3805fc00 2021-07-11T03:53:09.818 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 -- 172.21.2.105:0/2892011443 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd38128ea0 con 0x7fcd3805fc00 2021-07-11T03:53:09.818 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd35ffb700 1 -- 172.21.2.105:0/2892011443 <== mon.2 v2:172.21.2.105:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fcd20010030 con 0x7fcd3805fc00 2021-07-11T03:53:09.818 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd35ffb700 1 -- 172.21.2.105:0/2892011443 <== mon.2 v2:172.21.2.105:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7fcd20013940 con 0x7fcd3805fc00 2021-07-11T03:53:09.818 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.801+0000 7fcd3f254700 1 -- 172.21.2.105:0/2892011443 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd380ae7d0 con 0x7fcd3805fc00 2021-07-11T03:53:09.819 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.805+0000 7fcd35ffb700 1 -- 172.21.2.105:0/2892011443 <== mon.2 v2:172.21.2.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fcd20013c20 con 0x7fcd3805fc00 2021-07-11T03:53:09.851 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.849+0000 7fcd3f254700 1 -- 172.21.2.105:0/2892011443 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", "id": 4} v 0) v1 -- 0x7fcd38057350 con 0x7fcd3805fc00 2021-07-11T03:53:09.855 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.853+0000 7fcd35ffb700 1 -- 172.21.2.105:0/2892011443 <== mon.2 v2:172.21.2.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcd20013dc0 con 0x7fcd3805fc00 2021-07-11T03:53:09.855 INFO:teuthology.orchestra.run.gibba005.stdout:4 2021-07-11T03:53:09.856 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.853+0000 7fcd3f254700 1 -- 172.21.2.105:0/2892011443 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd3805fc00 msgr2=0x7fcd38128870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.857 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.853+0000 7fcd3f254700 1 --2- 172.21.2.105:0/2892011443 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd3805fc00 0x7fcd38128870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fcd200011d0 tx=0x7fcd20007c00).stop 2021-07-11T03:53:09.857 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.853+0000 7fcd3f254700 1 -- 172.21.2.105:0/2892011443 shutdown_connections 2021-07-11T03:53:09.857 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.853+0000 7fcd3f254700 1 --2- 172.21.2.105:0/2892011443 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd380b6560 0x7fcd380affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.857 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.853+0000 7fcd3f254700 1 --2- 172.21.2.105:0/2892011443 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd380b5990 0x7fcd380afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.858 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.853+0000 7fcd3f254700 1 --2- 172.21.2.105:0/2892011443 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd3805fc00 0x7fcd38128870 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.858 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.853+0000 7fcd3f254700 1 -- 172.21.2.105:0/2892011443 >> 172.21.2.105:0/2892011443 conn(0x7fcd380623a0 msgr2=0x7fcd380b7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.858 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.853+0000 7fcd3f254700 1 -- 172.21.2.105:0/2892011443 shutdown_connections 2021-07-11T03:53:09.858 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.853+0000 7fcd3f254700 1 -- 172.21.2.105:0/2892011443 wait complete. 2021-07-11T03:53:09.862 DEBUG:teuthology.orchestra.run.gibba005:> sudo ceph --cluster ceph osd new e19543c5-2378-49c8-a626-c71dbbb2a718 5 2021-07-11T03:53:09.913 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13e1f5b700 1 Processor -- start 2021-07-11T03:53:09.914 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13e1f5b700 1 -- start start 2021-07-11T03:53:09.914 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13e1f5b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 0x7f13dc0b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.914 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13e1f5b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13dc143f60 con 0x7f13dc05fc00 2021-07-11T03:53:09.914 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13e1f5b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13dc05ffe0 con 0x7f13dc0b6560 2021-07-11T03:53:09.915 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13e1f5b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc0ba6b0 con 0x7f13dc0b5990 2021-07-11T03:53:09.915 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13daffd700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f13dc0b6560 0x7f13dc0b4a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.105:57186/0 (socket says 172.21.2.105:57186) 2021-07-11T03:53:09.915 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13daffd700 1 -- 172.21.2.105:0/1452939982 learned_addr learned my addr 172.21.2.105:0/1452939982 (peer_addr_for_me v1:172.21.2.105:0/0) 2021-07-11T03:53:09.916 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13dbfff700 1 --2- 172.21.2.105:0/1452939982 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 0x7f13dc0b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.918 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13dbfff700 1 -- 172.21.2.105:0/1452939982 >> v1:172.21.2.105:6789/0 conn(0x7f13dc0b6560 legacy=0x7f13dc0b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.918 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13dbfff700 1 -- 172.21.2.105:0/1452939982 >> v1:172.21.2.104:6789/0 conn(0x7f13dc05fc00 legacy=0x7f13dc0b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.919 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13dbfff700 1 -- 172.21.2.105:0/1452939982 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc143a50 con 0x7f13dc0b5990 2021-07-11T03:53:09.919 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13dbfff700 1 --2- 172.21.2.105:0/1452939982 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 0x7f13dc0b41b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f13cc001110 tx=0x7f13d40091d0).ready entity=mon.2 client_cookie=22159a9d685daea4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.919 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13da7fc700 1 -- 172.21.2.105:0/1452939982 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13d400b630 con 0x7f13dc0b5990 2021-07-11T03:53:09.919 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13da7fc700 1 -- 172.21.2.105:0/1452939982 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d400b790 con 0x7f13dc0b5990 2021-07-11T03:53:09.920 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13da7fc700 1 -- 172.21.2.105:0/1452939982 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13d400b900 con 0x7f13dc0b5990 2021-07-11T03:53:09.920 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13e1f5b700 1 -- 172.21.2.105:0/1452939982 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 msgr2=0x7f13dc0b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.920 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.909+0000 7f13e1f5b700 1 --2- 172.21.2.105:0/1452939982 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 0x7f13dc0b41b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f13cc001110 tx=0x7f13d40091d0).stop 2021-07-11T03:53:09.920 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 -- 172.21.2.105:0/1452939982 shutdown_connections 2021-07-11T03:53:09.921 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 --2- 172.21.2.105:0/1452939982 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 0x7f13dc0b41b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.921 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 -- 172.21.2.105:0/1452939982 >> 172.21.2.105:0/1452939982 conn(0x7f13dc0623a0 msgr2=0x7f13dc066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.921 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 -- 172.21.2.105:0/1452939982 shutdown_connections 2021-07-11T03:53:09.921 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 -- 172.21.2.105:0/1452939982 wait complete. 2021-07-11T03:53:09.922 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 Processor -- start 2021-07-11T03:53:09.922 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 -- start start 2021-07-11T03:53:09.922 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13dc05fc00 0x7f13dc14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.923 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 0x7f13dc156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.923 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13dc0b6560 0x7f13dc157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:09.923 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc157600 con 0x7f13dc05fc00 2021-07-11T03:53:09.923 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc0afc40 con 0x7f13dc0b6560 2021-07-11T03:53:09.923 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc0afee0 con 0x7f13dc0b5990 2021-07-11T03:53:09.924 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13daffd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 0x7f13dc156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.924 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13daffd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 0x7f13dc156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.105:50124/0 (socket says 172.21.2.105:50124) 2021-07-11T03:53:09.924 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13daffd700 1 -- 172.21.2.105:0/872493820 learned_addr learned my addr 172.21.2.105:0/872493820 (peer_addr_for_me v2:172.21.2.105:0/0) 2021-07-11T03:53:09.924 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13dbfff700 1 --2- 172.21.2.105:0/872493820 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13dc0b6560 0x7f13dc157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.925 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13db7fe700 1 --2- 172.21.2.105:0/872493820 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13dc05fc00 0x7f13dc14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:09.925 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13dbfff700 1 -- 172.21.2.105:0/872493820 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 msgr2=0x7f13dc156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.925 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13dbfff700 1 --2- 172.21.2.105:0/872493820 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 0x7f13dc156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.925 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13dbfff700 1 -- 172.21.2.105:0/872493820 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13dc05fc00 msgr2=0x7f13dc14d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.926 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13dbfff700 1 --2- 172.21.2.105:0/872493820 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13dc05fc00 0x7f13dc14d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.926 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13dbfff700 1 -- 172.21.2.105:0/872493820 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4009470 con 0x7f13dc0b6560 2021-07-11T03:53:09.926 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13daffd700 1 --2- 172.21.2.105:0/872493820 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 0x7f13dc156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T03:53:09.926 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13dbfff700 1 --2- 172.21.2.105:0/872493820 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13dc0b6560 0x7f13dc157110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f13dc1566a0 tx=0x7f13d4016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:09.927 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13d8ff9700 1 -- 172.21.2.105:0/872493820 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13d4006ea0 con 0x7f13dc0b6560 2021-07-11T03:53:09.927 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13d8ff9700 1 -- 172.21.2.105:0/872493820 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d4011de0 con 0x7f13dc0b6560 2021-07-11T03:53:09.927 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 -- 172.21.2.105:0/872493820 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13dc0b0180 con 0x7f13dc0b6560 2021-07-11T03:53:09.927 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13d8ff9700 1 -- 172.21.2.105:0/872493820 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13d4021630 con 0x7f13dc0b6560 2021-07-11T03:53:09.928 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 -- 172.21.2.105:0/872493820 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13dc0ae230 con 0x7f13dc0b6560 2021-07-11T03:53:09.928 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13d8ff9700 1 -- 172.21.2.105:0/872493820 <== mon.1 v2:172.21.2.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f13d4027020 con 0x7f13dc0b6560 2021-07-11T03:53:09.928 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13d8ff9700 1 -- 172.21.2.105:0/872493820 <== mon.1 v2:172.21.2.105:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f13d400be80 con 0x7f13dc0b6560 2021-07-11T03:53:09.928 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13e1f5b700 1 -- 172.21.2.105:0/872493820 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13dc057350 con 0x7f13dc0b6560 2021-07-11T03:53:09.929 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.913+0000 7f13d8ff9700 1 -- 172.21.2.105:0/872493820 <== mon.1 v2:172.21.2.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f13d4015020 con 0x7f13dc0b6560 2021-07-11T03:53:09.963 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.961+0000 7f13e1f5b700 1 -- 172.21.2.105:0/872493820 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", "id": 5} v 0) v1 -- 0x7f13dc147440 con 0x7f13dc0b6560 2021-07-11T03:53:09.967 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.965+0000 7f13d8ff9700 1 -- 172.21.2.105:0/872493820 <== mon.1 v2:172.21.2.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f13d40032d0 con 0x7f13dc0b6560 2021-07-11T03:53:09.967 INFO:teuthology.orchestra.run.gibba005.stdout:5 2021-07-11T03:53:09.968 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.965+0000 7f13e1f5b700 1 -- 172.21.2.105:0/872493820 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13dc0b6560 msgr2=0x7f13dc157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:09.969 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.965+0000 7f13e1f5b700 1 --2- 172.21.2.105:0/872493820 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13dc0b6560 0x7f13dc157110 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f13dc1566a0 tx=0x7f13d4016000).stop 2021-07-11T03:53:09.969 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.965+0000 7f13e1f5b700 1 -- 172.21.2.105:0/872493820 shutdown_connections 2021-07-11T03:53:09.969 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.965+0000 7f13e1f5b700 1 --2- 172.21.2.105:0/872493820 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13dc0b6560 0x7f13dc157110 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.969 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.965+0000 7f13e1f5b700 1 --2- 172.21.2.105:0/872493820 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13dc0b5990 0x7f13dc156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.970 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.965+0000 7f13e1f5b700 1 --2- 172.21.2.105:0/872493820 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13dc05fc00 0x7f13dc14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:09.970 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.965+0000 7f13e1f5b700 1 -- 172.21.2.105:0/872493820 >> 172.21.2.105:0/872493820 conn(0x7f13dc0623a0 msgr2=0x7f13dc05bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:09.970 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.965+0000 7f13e1f5b700 1 -- 172.21.2.105:0/872493820 shutdown_connections 2021-07-11T03:53:09.970 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:09.965+0000 7f13e1f5b700 1 -- 172.21.2.105:0/872493820 wait complete. 2021-07-11T03:53:09.974 DEBUG:teuthology.orchestra.run.gibba005:> sudo ceph --cluster ceph osd new fc1e39b4-4c50-4f6a-a8ff-19687bf442b2 6 2021-07-11T03:53:10.025 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0ca0c700 1 Processor -- start 2021-07-11T03:53:10.025 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0ca0c700 1 -- start start 2021-07-11T03:53:10.027 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0ca0c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 0x7fcd080b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:10.027 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0ca0c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd08143f60 con 0x7fcd080b5990 2021-07-11T03:53:10.027 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0ca0c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0805ffe0 con 0x7fcd0805fc00 2021-07-11T03:53:10.028 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0ca0c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080ba6b0 con 0x7fcd080b6560 2021-07-11T03:53:10.028 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0659c700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fcd0805fc00 0x7fcd080b9fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.105:57198/0 (socket says 172.21.2.105:57198) 2021-07-11T03:53:10.028 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd05d9b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 0x7fcd080b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:10.028 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0659c700 1 -- 172.21.2.105:0/2937288470 learned_addr learned my addr 172.21.2.105:0/2937288470 (peer_addr_for_me v1:172.21.2.105:0/0) 2021-07-11T03:53:10.029 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd05d9b700 1 -- 172.21.2.105:0/2937288470 >> v1:172.21.2.105:6789/0 conn(0x7fcd0805fc00 legacy=0x7fcd080b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:10.029 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd05d9b700 1 -- 172.21.2.105:0/2937288470 >> v1:172.21.2.104:6789/0 conn(0x7fcd080b5990 legacy=0x7fcd080b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:10.029 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd05d9b700 1 -- 172.21.2.105:0/2937288470 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd08143a50 con 0x7fcd080b6560 2021-07-11T03:53:10.029 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd05d9b700 1 --2- 172.21.2.105:0/2937288470 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 0x7fcd080b41b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fccf8001110 tx=0x7fccf0006df0).ready entity=mon.2 client_cookie=bcabf51688222ab9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:10.029 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0559a700 1 -- 172.21.2.105:0/2937288470 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccf000d570 con 0x7fcd080b6560 2021-07-11T03:53:10.030 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0559a700 1 -- 172.21.2.105:0/2937288470 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf0000e90 con 0x7fcd080b6560 2021-07-11T03:53:10.030 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0559a700 1 -- 172.21.2.105:0/2937288470 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccf000d8c0 con 0x7fcd080b6560 2021-07-11T03:53:10.030 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/2937288470 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 msgr2=0x7fcd080b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:10.030 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.021+0000 7fcd0ca0c700 1 --2- 172.21.2.105:0/2937288470 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 0x7fcd080b41b0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fccf8001110 tx=0x7fccf0006df0).stop 2021-07-11T03:53:10.031 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/2937288470 shutdown_connections 2021-07-11T03:53:10.031 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 --2- 172.21.2.105:0/2937288470 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 0x7fcd080b41b0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.031 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/2937288470 >> 172.21.2.105:0/2937288470 conn(0x7fcd080623a0 msgr2=0x7fcd08066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:10.031 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/2937288470 shutdown_connections 2021-07-11T03:53:10.031 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/2937288470 wait complete. 2021-07-11T03:53:10.032 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 Processor -- start 2021-07-11T03:53:10.032 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 -- start start 2021-07-11T03:53:10.034 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd0805fc00 0x7fcd080b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:10.034 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd080b5990 0x7fcd08144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:10.035 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 0x7fcd08145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:10.035 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd080aecd0 con 0x7fcd080b5990 2021-07-11T03:53:10.035 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd08146d00 con 0x7fcd0805fc00 2021-07-11T03:53:10.035 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd08145770 con 0x7fcd080b6560 2021-07-11T03:53:10.036 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd06d9d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 0x7fcd08145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:10.036 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0659c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd0805fc00 0x7fcd080b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:10.036 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd06d9d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 0x7fcd08145280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.105:50140/0 (socket says 172.21.2.105:50140) 2021-07-11T03:53:10.036 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd06d9d700 1 -- 172.21.2.105:0/901999742 learned_addr learned my addr 172.21.2.105:0/901999742 (peer_addr_for_me v2:172.21.2.105:0/0) 2021-07-11T03:53:10.036 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd05d9b700 1 --2- 172.21.2.105:0/901999742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd080b5990 0x7fcd08144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:10.037 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd06d9d700 1 -- 172.21.2.105:0/901999742 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd0805fc00 msgr2=0x7fcd080b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:10.037 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd06d9d700 1 --2- 172.21.2.105:0/901999742 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd0805fc00 0x7fcd080b8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.037 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd06d9d700 1 -- 172.21.2.105:0/901999742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd080b5990 msgr2=0x7fcd08144d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:10.037 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd06d9d700 1 --2- 172.21.2.105:0/901999742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd080b5990 0x7fcd08144d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.037 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd06d9d700 1 -- 172.21.2.105:0/901999742 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf000b180 con 0x7fcd080b6560 2021-07-11T03:53:10.038 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd06d9d700 1 --2- 172.21.2.105:0/901999742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 0x7fcd08145280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fcd080616a0 tx=0x7fcd00007f40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:10.038 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fccff7fe700 1 -- 172.21.2.105:0/901999742 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd0000f470 con 0x7fcd080b6560 2021-07-11T03:53:10.038 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fccff7fe700 1 -- 172.21.2.105:0/901999742 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0000f5d0 con 0x7fcd080b6560 2021-07-11T03:53:10.038 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/901999742 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd080af1f0 con 0x7fcd080b6560 2021-07-11T03:53:10.039 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fccff7fe700 1 -- 172.21.2.105:0/901999742 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd0000f930 con 0x7fcd080b6560 2021-07-11T03:53:10.039 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/901999742 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd080af730 con 0x7fcd080b6560 2021-07-11T03:53:10.039 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fccff7fe700 1 -- 172.21.2.105:0/901999742 <== mon.2 v2:172.21.2.105:3301/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7fcd00019030 con 0x7fcd080b6560 2021-07-11T03:53:10.039 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fccff7fe700 1 -- 172.21.2.105:0/901999742 <== mon.2 v2:172.21.2.105:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7fcd0001b9d0 con 0x7fcd080b6560 2021-07-11T03:53:10.039 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.025+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/901999742 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd08147090 con 0x7fcd080b6560 2021-07-11T03:53:10.040 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.029+0000 7fccff7fe700 1 -- 172.21.2.105:0/901999742 <== mon.2 v2:172.21.2.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7fcd00017070 con 0x7fcd080b6560 2021-07-11T03:53:10.075 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.073+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/901999742 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", "id": 6} v 0) v1 -- 0x7fcd08057350 con 0x7fcd080b6560 2021-07-11T03:53:10.079 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.077+0000 7fccff7fe700 1 -- 172.21.2.105:0/901999742 <== mon.2 v2:172.21.2.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcd0001bcb0 con 0x7fcd080b6560 2021-07-11T03:53:10.080 INFO:teuthology.orchestra.run.gibba005.stdout:6 2021-07-11T03:53:10.080 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.077+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/901999742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 msgr2=0x7fcd08145280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:10.081 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.077+0000 7fcd0ca0c700 1 --2- 172.21.2.105:0/901999742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 0x7fcd08145280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fcd080616a0 tx=0x7fcd00007f40).stop 2021-07-11T03:53:10.081 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.077+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/901999742 shutdown_connections 2021-07-11T03:53:10.081 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.077+0000 7fcd0ca0c700 1 --2- 172.21.2.105:0/901999742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd080b6560 0x7fcd08145280 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.081 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.077+0000 7fcd0ca0c700 1 --2- 172.21.2.105:0/901999742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd080b5990 0x7fcd08144d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.082 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.077+0000 7fcd0ca0c700 1 --2- 172.21.2.105:0/901999742 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd0805fc00 0x7fcd080b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.082 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.077+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/901999742 >> 172.21.2.105:0/901999742 conn(0x7fcd080623a0 msgr2=0x7fcd080b7410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:10.082 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.077+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/901999742 shutdown_connections 2021-07-11T03:53:10.082 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.077+0000 7fcd0ca0c700 1 -- 172.21.2.105:0/901999742 wait complete. 2021-07-11T03:53:10.086 DEBUG:teuthology.orchestra.run.gibba005:> sudo ceph --cluster ceph osd new 2df0639d-90fe-4158-80ae-24cf0f10dc86 7 2021-07-11T03:53:10.139 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd7f2c700 1 Processor -- start 2021-07-11T03:53:10.140 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd7f2c700 1 -- start start 2021-07-11T03:53:10.140 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd7f2c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 0x7f9dd0064870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:10.140 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd7f2c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd00653a0 con 0x7f9dd00b38a0 2021-07-11T03:53:10.141 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd7f2c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd00b30d0 con 0x7f9dd01443a0 2021-07-11T03:53:10.141 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd7f2c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd0065fd0 con 0x7f9dd013e460 2021-07-11T03:53:10.141 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd5cc8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 0x7f9dd0064870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:10.141 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd64c9700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f9dd01443a0 0x7f9dd0066da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.105:57212/0 (socket says 172.21.2.105:57212) 2021-07-11T03:53:10.142 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd64c9700 1 -- 172.21.2.105:0/1620368087 learned_addr learned my addr 172.21.2.105:0/1620368087 (peer_addr_for_me v1:172.21.2.105:0/0) 2021-07-11T03:53:10.142 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd4cc6700 1 -- 172.21.2.105:0/1620368087 <== mon.1 v1:172.21.2.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928984230 0 0) 0x7f9dd00b30d0 con 0x7f9dd01443a0 2021-07-11T03:53:10.142 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd4cc6700 1 -- 172.21.2.105:0/1620368087 --> v1:172.21.2.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9db8003260 con 0x7f9dd01443a0 2021-07-11T03:53:10.142 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd5cc8700 1 -- 172.21.2.105:0/1620368087 >> v1:172.21.2.105:6789/0 conn(0x7f9dd01443a0 legacy=0x7f9dd0066da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:10.143 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd5cc8700 1 -- 172.21.2.105:0/1620368087 >> v1:172.21.2.104:6789/0 conn(0x7f9dd00b38a0 legacy=0x7f9dd00b7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:10.143 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd5cc8700 1 -- 172.21.2.105:0/1620368087 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd00b7c40 con 0x7f9dd013e460 2021-07-11T03:53:10.143 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd5cc8700 1 --2- 172.21.2.105:0/1620368087 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 0x7f9dd0064870 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f9dc4002f70 tx=0x7f9dc4006df0).ready entity=mon.2 client_cookie=48e7907311dbeb9f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:10.144 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd4cc6700 1 -- 172.21.2.105:0/1620368087 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dc400d570 con 0x7f9dd013e460 2021-07-11T03:53:10.144 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd4cc6700 1 -- 172.21.2.105:0/1620368087 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc4000e90 con 0x7f9dd013e460 2021-07-11T03:53:10.144 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd4cc6700 1 -- 172.21.2.105:0/1620368087 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dc400d8c0 con 0x7f9dd013e460 2021-07-11T03:53:10.144 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/1620368087 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 msgr2=0x7f9dd0064870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:10.145 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.133+0000 7f9dd7f2c700 1 --2- 172.21.2.105:0/1620368087 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 0x7f9dd0064870 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f9dc4002f70 tx=0x7f9dc4006df0).stop 2021-07-11T03:53:10.145 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/1620368087 shutdown_connections 2021-07-11T03:53:10.145 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 --2- 172.21.2.105:0/1620368087 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 0x7f9dd0064870 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.145 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/1620368087 >> 172.21.2.105:0/1620368087 conn(0x7f9dd00629d0 msgr2=0x7f9dd0062de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:10.145 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/1620368087 shutdown_connections 2021-07-11T03:53:10.146 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/1620368087 wait complete. 2021-07-11T03:53:10.146 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 Processor -- start 2021-07-11T03:53:10.146 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 -- start start 2021-07-11T03:53:10.149 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9dd00b38a0 0x7f9dd0146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:10.149 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 0x7f9dd00afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:10.149 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9dd01443a0 0x7f9dd00b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:10.149 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd00af750 con 0x7f9dd01443a0 2021-07-11T03:53:10.150 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd00b7180 con 0x7f9dd00b38a0 2021-07-11T03:53:10.150 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd00b7420 con 0x7f9dd013e460 2021-07-11T03:53:10.150 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd5cc8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9dd00b38a0 0x7f9dd0146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:10.150 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd5cc8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9dd00b38a0 0x7f9dd0146d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.105:40418/0 (socket says 172.21.2.105:40418) 2021-07-11T03:53:10.151 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd5cc8700 1 -- 172.21.2.105:0/4120060549 learned_addr learned my addr 172.21.2.105:0/4120060549 (peer_addr_for_me v2:172.21.2.105:0/0) 2021-07-11T03:53:10.151 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd54c7700 1 --2- 172.21.2.105:0/4120060549 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 0x7f9dd00afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:10.151 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd64c9700 1 --2- 172.21.2.105:0/4120060549 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9dd01443a0 0x7f9dd00b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:10.151 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd5cc8700 1 -- 172.21.2.105:0/4120060549 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 msgr2=0x7f9dd00afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:10.152 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd5cc8700 1 --2- 172.21.2.105:0/4120060549 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 0x7f9dd00afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.152 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd5cc8700 1 -- 172.21.2.105:0/4120060549 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9dd01443a0 msgr2=0x7f9dd00b0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:10.152 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd5cc8700 1 --2- 172.21.2.105:0/4120060549 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9dd01443a0 0x7f9dd00b0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.152 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd5cc8700 1 -- 172.21.2.105:0/4120060549 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc400b180 con 0x7f9dd00b38a0 2021-07-11T03:53:10.153 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd54c7700 1 --2- 172.21.2.105:0/4120060549 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 0x7f9dd00afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T03:53:10.153 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd5cc8700 1 --2- 172.21.2.105:0/4120060549 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9dd00b38a0 0x7f9dd0146d20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9dc4002f70 tx=0x7f9dc4014000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:10.153 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dc2ffd700 1 -- 172.21.2.105:0/4120060549 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dc4012e80 con 0x7f9dd00b38a0 2021-07-11T03:53:10.153 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dc2ffd700 1 -- 172.21.2.105:0/4120060549 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc4000e90 con 0x7f9dd00b38a0 2021-07-11T03:53:10.153 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/4120060549 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd00b76c0 con 0x7f9dd00b38a0 2021-07-11T03:53:10.154 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dc2ffd700 1 -- 172.21.2.105:0/4120060549 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dc401e5e0 con 0x7f9dd00b38a0 2021-07-11T03:53:10.154 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/4120060549 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd00aebf0 con 0x7f9dd00b38a0 2021-07-11T03:53:10.154 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dc2ffd700 1 -- 172.21.2.105:0/4120060549 <== mon.1 v2:172.21.2.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 345+0+0 (secure 0 0 0) 0x7f9dc4024020 con 0x7f9dd00b38a0 2021-07-11T03:53:10.154 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dc2ffd700 1 -- 172.21.2.105:0/4120060549 <== mon.1 v2:172.21.2.105:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f9dc4027ad0 con 0x7f9dd00b38a0 2021-07-11T03:53:10.155 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.137+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/4120060549 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd0057350 con 0x7f9dd00b38a0 2021-07-11T03:53:10.155 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.141+0000 7f9dc2ffd700 1 -- 172.21.2.105:0/4120060549 <== mon.1 v2:172.21.2.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f9dc4027db0 con 0x7f9dd00b38a0 2021-07-11T03:53:10.188 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.185+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/4120060549 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", "id": 7} v 0) v1 -- 0x7f9dd00b7a20 con 0x7f9dd00b38a0 2021-07-11T03:53:10.191 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.189+0000 7f9dc2ffd700 1 -- 172.21.2.105:0/4120060549 <== mon.1 v2:172.21.2.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9dc4027440 con 0x7f9dd00b38a0 2021-07-11T03:53:10.192 INFO:teuthology.orchestra.run.gibba005.stdout:7 2021-07-11T03:53:10.193 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.189+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/4120060549 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9dd00b38a0 msgr2=0x7f9dd0146d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:10.193 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.189+0000 7f9dd7f2c700 1 --2- 172.21.2.105:0/4120060549 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9dd00b38a0 0x7f9dd0146d20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f9dc4002f70 tx=0x7f9dc4014000).stop 2021-07-11T03:53:10.193 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.189+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/4120060549 shutdown_connections 2021-07-11T03:53:10.194 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.189+0000 7f9dd7f2c700 1 --2- 172.21.2.105:0/4120060549 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9dd01443a0 0x7f9dd00b0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.194 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.189+0000 7f9dd7f2c700 1 --2- 172.21.2.105:0/4120060549 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9dd013e460 0x7f9dd00afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.195 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.189+0000 7f9dd7f2c700 1 --2- 172.21.2.105:0/4120060549 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9dd00b38a0 0x7f9dd0146d20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:10.195 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.189+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/4120060549 >> 172.21.2.105:0/4120060549 conn(0x7f9dd00629d0 msgr2=0x7f9dd0060d70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:10.195 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.189+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/4120060549 shutdown_connections 2021-07-11T03:53:10.196 INFO:teuthology.orchestra.run.gibba005.stderr:2021-07-11T03:53:10.189+0000 7f9dd7f2c700 1 -- 172.21.2.105:0/4120060549 wait complete. 2021-07-11T03:53:10.199 INFO:tasks.ceph.osd.0:Restarting daemon 2021-07-11T03:53:10.200 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2021-07-11T03:53:10.202 INFO:tasks.ceph.osd.0:Started 2021-07-11T03:53:10.202 INFO:tasks.ceph.osd.1:Restarting daemon 2021-07-11T03:53:10.203 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2021-07-11T03:53:10.205 INFO:tasks.ceph.osd.1:Started 2021-07-11T03:53:10.206 INFO:tasks.ceph.osd.2:Restarting daemon 2021-07-11T03:53:10.206 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2021-07-11T03:53:10.208 INFO:tasks.ceph.osd.2:Started 2021-07-11T03:53:10.208 INFO:tasks.ceph.osd.3:Restarting daemon 2021-07-11T03:53:10.209 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2021-07-11T03:53:10.210 INFO:tasks.ceph.osd.3:Started 2021-07-11T03:53:10.211 INFO:tasks.ceph.osd.4:Restarting daemon 2021-07-11T03:53:10.211 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2021-07-11T03:53:10.213 INFO:tasks.ceph.osd.4:Started 2021-07-11T03:53:10.213 INFO:tasks.ceph.osd.5:Restarting daemon 2021-07-11T03:53:10.214 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2021-07-11T03:53:10.215 INFO:tasks.ceph.osd.5:Started 2021-07-11T03:53:10.216 INFO:tasks.ceph.osd.6:Restarting daemon 2021-07-11T03:53:10.217 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2021-07-11T03:53:10.218 INFO:tasks.ceph.osd.6:Started 2021-07-11T03:53:10.219 INFO:tasks.ceph.osd.7:Restarting daemon 2021-07-11T03:53:10.219 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2021-07-11T03:53:10.221 INFO:tasks.ceph.osd.7:Started 2021-07-11T03:53:10.222 INFO:tasks.ceph:Waiting for OSDs to come up 2021-07-11T03:53:10.533 INFO:tasks.ceph.osd.4.gibba005.stderr:2021-07-11T03:53:10.529+0000 7f1231aefd80 -1 Falling back to public interface 2021-07-11T03:53:10.792 INFO:tasks.ceph.osd.0.gibba004.stderr:2021-07-11T03:53:10.788+0000 7ff080b53d80 -1 Falling back to public interface 2021-07-11T03:53:10.796 INFO:tasks.ceph.osd.1.gibba004.stderr:2021-07-11T03:53:10.792+0000 7f22970f5d80 -1 Falling back to public interface 2021-07-11T03:53:10.822 INFO:tasks.ceph.osd.5.gibba005.stderr:2021-07-11T03:53:10.817+0000 7fcebca85d80 -1 Falling back to public interface 2021-07-11T03:53:10.822 INFO:tasks.ceph.osd.6.gibba005.stderr:2021-07-11T03:53:10.817+0000 7f8da0121d80 -1 Falling back to public interface 2021-07-11T03:53:10.823 INFO:tasks.ceph.osd.7.gibba005.stderr:2021-07-11T03:53:10.817+0000 7f3ce2db2d80 -1 Falling back to public interface 2021-07-11T03:53:10.834 INFO:tasks.ceph.osd.2.gibba004.stderr:2021-07-11T03:53:10.832+0000 7f09db92bd80 -1 Falling back to public interface 2021-07-11T03:53:10.838 INFO:tasks.ceph.osd.3.gibba004.stderr:2021-07-11T03:53:10.836+0000 7fb303957d80 -1 Falling back to public interface 2021-07-11T03:53:11.026 DEBUG:teuthology.orchestra.run.gibba004:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:11.080 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.076+0000 7f9a6ae8f700 1 Processor -- start 2021-07-11T03:53:11.081 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.076+0000 7f9a6ae8f700 1 -- start start 2021-07-11T03:53:11.081 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.076+0000 7f9a6ae8f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9a640aebe0 0x7f9a640b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:11.081 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.076+0000 7f9a6ae8f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9a6414b250 0x7f9a64152090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:11.082 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.076+0000 7f9a6ae8f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a640b6250 con 0x7f9a640b1200 2021-07-11T03:53:11.082 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.076+0000 7f9a6ae8f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a640aefc0 con 0x7f9a6414b250 2021-07-11T03:53:11.082 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.076+0000 7f9a6ae8f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640b00e0 con 0x7f9a640aebe0 2021-07-11T03:53:11.083 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6942c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9a6414b250 0x7f9a64152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:11.084 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a68c2b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9a640aebe0 0x7f9a640b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:11.084 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6942c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9a6414b250 0x7f9a64152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:48784/0 (socket says 172.21.2.104:48784) 2021-07-11T03:53:11.084 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6942c700 1 -- 172.21.2.104:0/4050178618 learned_addr learned my addr 172.21.2.104:0/4050178618 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:11.084 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6942c700 1 -- 172.21.2.104:0/4050178618 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9a640aebe0 msgr2=0x7f9a640b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:11.084 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6942c700 1 --2- 172.21.2.104:0/4050178618 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9a640aebe0 0x7f9a640b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:11.085 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6942c700 1 -- 172.21.2.104:0/4050178618 >> v1:172.21.2.105:6789/0 conn(0x7f9a640b1200 legacy=0x7f9a6414ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:11.085 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6942c700 1 -- 172.21.2.104:0/4050178618 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a640a8c80 con 0x7f9a6414b250 2021-07-11T03:53:11.085 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6942c700 1 --2- 172.21.2.104:0/4050178618 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9a6414b250 0x7f9a64152090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f9a5c0032f0 tx=0x7f9a5c009290).ready entity=mon.1 client_cookie=8886013a895fa1fe server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:11.088 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a637fe700 1 -- 172.21.2.104:0/4050178618 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a5c00b8a0 con 0x7f9a6414b250 2021-07-11T03:53:11.089 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a637fe700 1 -- 172.21.2.104:0/4050178618 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c00ba00 con 0x7f9a6414b250 2021-07-11T03:53:11.089 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a637fe700 1 -- 172.21.2.104:0/4050178618 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a5c00bb70 con 0x7f9a6414b250 2021-07-11T03:53:11.089 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4050178618 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9a6414b250 msgr2=0x7f9a64152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:11.089 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6ae8f700 1 --2- 172.21.2.104:0/4050178618 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9a6414b250 0x7f9a64152090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f9a5c0032f0 tx=0x7f9a5c009290).stop 2021-07-11T03:53:11.090 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4050178618 shutdown_connections 2021-07-11T03:53:11.090 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6ae8f700 1 --2- 172.21.2.104:0/4050178618 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9a6414b250 0x7f9a64152090 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:11.090 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6ae8f700 1 --2- 172.21.2.104:0/4050178618 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9a640aebe0 0x7f9a640b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:11.090 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4050178618 >> 172.21.2.104:0/4050178618 conn(0x7f9a640b5420 msgr2=0x7f9a640b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:11.091 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4050178618 shutdown_connections 2021-07-11T03:53:11.091 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.080+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4050178618 wait complete. 2021-07-11T03:53:11.091 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6ae8f700 1 Processor -- start 2021-07-11T03:53:11.091 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6ae8f700 1 -- start start 2021-07-11T03:53:11.091 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6ae8f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9a640aebe0 0x7f9a641564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:11.092 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6ae8f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9a640b1200 0x7f9a640aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:11.092 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6ae8f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9a6414b250 0x7f9a6414e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:11.092 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6ae8f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a640a93d0 con 0x7f9a640b1200 2021-07-11T03:53:11.092 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6ae8f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6414d450 con 0x7f9a6414b250 2021-07-11T03:53:11.092 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6ae8f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6414d6f0 con 0x7f9a640aebe0 2021-07-11T03:53:11.093 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a63fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9a640b1200 0x7f9a640aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:11.093 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a63fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9a640b1200 0x7f9a640aa1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41230/0 (socket says 172.21.2.104:41230) 2021-07-11T03:53:11.093 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a63fff700 1 -- 172.21.2.104:0/4175761740 learned_addr learned my addr 172.21.2.104:0/4175761740 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:11.093 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a68c2b700 1 --2- 172.21.2.104:0/4175761740 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9a640aebe0 0x7f9a641564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:11.094 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6942c700 1 --2- 172.21.2.104:0/4175761740 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9a6414b250 0x7f9a6414e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:11.094 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a63fff700 1 -- 172.21.2.104:0/4175761740 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9a640aebe0 msgr2=0x7f9a641564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:11.094 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a63fff700 1 --2- 172.21.2.104:0/4175761740 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9a640aebe0 0x7f9a641564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:11.094 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a63fff700 1 -- 172.21.2.104:0/4175761740 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9a6414b250 msgr2=0x7f9a6414e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:11.094 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a63fff700 1 --2- 172.21.2.104:0/4175761740 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9a6414b250 0x7f9a6414e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:11.095 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a63fff700 1 -- 172.21.2.104:0/4175761740 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c009530 con 0x7f9a640b1200 2021-07-11T03:53:11.095 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a63fff700 1 --2- 172.21.2.104:0/4175761740 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9a640b1200 0x7f9a640aa1b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f9a6406a250 tx=0x7f9a4c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:11.095 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a61ffb700 1 -- 172.21.2.104:0/4175761740 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a4c00ce30 con 0x7f9a640b1200 2021-07-11T03:53:11.095 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a61ffb700 1 -- 172.21.2.104:0/4175761740 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c00f470 con 0x7f9a640b1200 2021-07-11T03:53:11.095 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4175761740 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6414d990 con 0x7f9a640b1200 2021-07-11T03:53:11.096 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a61ffb700 1 -- 172.21.2.104:0/4175761740 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9a4c00f7d0 con 0x7f9a640b1200 2021-07-11T03:53:11.096 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4175761740 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6414e5c0 con 0x7f9a640b1200 2021-07-11T03:53:11.096 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a61ffb700 1 -- 172.21.2.104:0/4175761740 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 2) v1 ==== 32652+0+0 (secure 0 0 0) 0x7f9a4c019030 con 0x7f9a640b1200 2021-07-11T03:53:11.096 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4175761740 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6414d210 con 0x7f9a640b1200 2021-07-11T03:53:11.097 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.084+0000 7f9a61ffb700 1 -- 172.21.2.104:0/4175761740 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f9a4c039530 con 0x7f9a640b1200 2021-07-11T03:53:11.097 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.088+0000 7f9a61ffb700 1 -- 172.21.2.104:0/4175761740 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+67674 (secure 0 0 0) 0x7f9a6414d210 con 0x7f9a640b1200 2021-07-11T03:53:11.129 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.124+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4175761740 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a6414d210 con 0x7f9a640b1200 2021-07-11T03:53:11.130 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.124+0000 7f9a61ffb700 1 -- 172.21.2.104:0/4175761740 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+6849 (secure 0 0 0) 0x7f9a6414d210 con 0x7f9a640b1200 2021-07-11T03:53:11.130 INFO:teuthology.misc.health.gibba004.stdout: 2021-07-11T03:53:11.131 INFO:teuthology.misc.health.gibba004.stdout:{"epoch":10,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:10.188268+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:11.132 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.124+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4175761740 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9a640b1200 msgr2=0x7f9a640aa1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:11.132 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.124+0000 7f9a6ae8f700 1 --2- 172.21.2.104:0/4175761740 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9a640b1200 0x7f9a640aa1b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f9a6406a250 tx=0x7f9a4c007c50).stop 2021-07-11T03:53:11.132 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.124+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4175761740 shutdown_connections 2021-07-11T03:53:11.132 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.124+0000 7f9a6ae8f700 1 --2- 172.21.2.104:0/4175761740 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9a6414b250 0x7f9a6414e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:11.132 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.124+0000 7f9a6ae8f700 1 --2- 172.21.2.104:0/4175761740 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9a640b1200 0x7f9a640aa1b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:11.133 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.124+0000 7f9a6ae8f700 1 --2- 172.21.2.104:0/4175761740 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9a640aebe0 0x7f9a641564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:11.133 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.124+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4175761740 >> 172.21.2.104:0/4175761740 conn(0x7f9a640b5420 msgr2=0x7f9a640b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:11.133 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.124+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4175761740 shutdown_connections 2021-07-11T03:53:11.133 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:11.124+0000 7f9a6ae8f700 1 -- 172.21.2.104:0/4175761740 wait complete. 2021-07-11T03:53:11.135 DEBUG:teuthology.misc:0 of 8 OSDs are up 2021-07-11T03:53:11.680 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:53:11.683 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:53:11.686 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:53:11.689 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:53:11.699 INFO:tasks.ceph.osd.4.gibba005.stderr:2021-07-11T03:53:11.697+0000 7f1231aefd80 -1 osd.4 0 log_to_monitors {default=true} 2021-07-11T03:53:11.969 INFO:tasks.ceph.osd.0.gibba004.stderr:2021-07-11T03:53:11.964+0000 7ff080b53d80 -1 osd.0 0 log_to_monitors {default=true} 2021-07-11T03:53:11.970 INFO:tasks.ceph.osd.1.gibba004.stderr:2021-07-11T03:53:11.968+0000 7f22970f5d80 -1 osd.1 0 log_to_monitors {default=true} 2021-07-11T03:53:12.041 INFO:tasks.ceph.osd.6.gibba005.stderr:2021-07-11T03:53:12.037+0000 7f8da0121d80 -1 osd.6 0 log_to_monitors {default=true} 2021-07-11T03:53:12.043 INFO:tasks.ceph.osd.5.gibba005.stderr:2021-07-11T03:53:12.041+0000 7fcebca85d80 -1 osd.5 0 log_to_monitors {default=true} 2021-07-11T03:53:12.079 INFO:tasks.ceph.osd.7.gibba005.stderr:2021-07-11T03:53:12.077+0000 7f3ce2db2d80 -1 osd.7 0 log_to_monitors {default=true} 2021-07-11T03:53:12.083 INFO:tasks.ceph.osd.2.gibba004.stderr:2021-07-11T03:53:12.080+0000 7f09db92bd80 -1 osd.2 0 log_to_monitors {default=true} 2021-07-11T03:53:12.169 INFO:tasks.ceph.osd.3.gibba004.stderr:2021-07-11T03:53:12.168+0000 7fb303957d80 -1 osd.3 0 log_to_monitors {default=true} 2021-07-11T03:53:13.397 INFO:tasks.ceph.osd.1.gibba004.stderr:2021-07-11T03:53:13.392+0000 7f22891b2700 -1 osd.1 0 waiting for initial osdmap 2021-07-11T03:53:13.397 INFO:tasks.ceph.osd.3.gibba004.stderr:2021-07-11T03:53:13.392+0000 7fb2f5a14700 -1 osd.3 0 waiting for initial osdmap 2021-07-11T03:53:13.398 INFO:tasks.ceph.osd.0.gibba004.stderr:2021-07-11T03:53:13.392+0000 7ff072c10700 -1 osd.0 0 waiting for initial osdmap 2021-07-11T03:53:13.400 INFO:tasks.ceph.osd.7.gibba005.stderr:2021-07-11T03:53:13.397+0000 7f3cd4e6f700 -1 osd.7 0 waiting for initial osdmap 2021-07-11T03:53:13.400 INFO:tasks.ceph.osd.4.gibba005.stderr:2021-07-11T03:53:13.397+0000 7f1223bac700 -1 osd.4 0 waiting for initial osdmap 2021-07-11T03:53:13.401 INFO:tasks.ceph.osd.5.gibba005.stderr:2021-07-11T03:53:13.397+0000 7fceaeb42700 -1 osd.5 0 waiting for initial osdmap 2021-07-11T03:53:13.401 INFO:tasks.ceph.osd.6.gibba005.stderr:2021-07-11T03:53:13.397+0000 7f8d921de700 -1 osd.6 0 waiting for initial osdmap 2021-07-11T03:53:13.402 INFO:tasks.ceph.osd.2.gibba004.stderr:2021-07-11T03:53:13.396+0000 7f09cd9e8700 -1 osd.2 0 waiting for initial osdmap 2021-07-11T03:53:13.403 INFO:tasks.ceph.osd.3.gibba004.stderr:2021-07-11T03:53:13.400+0000 7fb2fce7a700 -1 osd.3 13 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-07-11T03:53:13.404 INFO:tasks.ceph.osd.5.gibba005.stderr:2021-07-11T03:53:13.401+0000 7fceb5fa8700 -1 osd.5 13 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-07-11T03:53:13.404 INFO:tasks.ceph.osd.2.gibba004.stderr:2021-07-11T03:53:13.400+0000 7f09d4e4e700 -1 osd.2 13 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-07-11T03:53:13.405 INFO:tasks.ceph.osd.6.gibba005.stderr:2021-07-11T03:53:13.401+0000 7f8d99644700 -1 osd.6 13 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-07-11T03:53:13.405 INFO:tasks.ceph.osd.0.gibba004.stderr:2021-07-11T03:53:13.400+0000 7ff07a076700 -1 osd.0 13 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-07-11T03:53:13.406 INFO:tasks.ceph.osd.4.gibba005.stderr:2021-07-11T03:53:13.401+0000 7f122b012700 -1 osd.4 13 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-07-11T03:53:13.406 INFO:tasks.ceph.osd.1.gibba004.stderr:2021-07-11T03:53:13.400+0000 7f2290618700 -1 osd.1 13 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-07-11T03:53:13.406 INFO:tasks.ceph.osd.7.gibba005.stderr:2021-07-11T03:53:13.401+0000 7f3cdc2d5700 -1 osd.7 13 set_numa_affinity unable to identify public interface 'enp1s0f0' numa node: (0) Success 2021-07-11T03:53:17.944 DEBUG:teuthology.orchestra.run.gibba004:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:17.998 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.992+0000 7fa3fe637700 1 Processor -- start 2021-07-11T03:53:17.998 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- start start 2021-07-11T03:53:17.999 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa3f80b7580 0x7fa3f80b7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:17.999 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3f8145110 0x7fa3f8060000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:17.999 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f813eaf0 con 0x7fa3f80b1fb0 2021-07-11T03:53:17.999 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f80604f0 con 0x7fa3f80b7580 2021-07-11T03:53:17.999 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f8144490 con 0x7fa3f8145110 2021-07-11T03:53:18.000 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f7fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa3f80b7580 0x7fa3f80b7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:18.000 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f7fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa3f80b7580 0x7fa3f80b7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41460/0 (socket says 172.21.2.104:41460) 2021-07-11T03:53:18.000 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f7fff700 1 -- 172.21.2.104:0/641341924 learned_addr learned my addr 172.21.2.104:0/641341924 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:18.003 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f7fff700 1 -- 172.21.2.104:0/641341924 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3f8145110 msgr2=0x7fa3f8060000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:18.003 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f7fff700 1 --2- 172.21.2.104:0/641341924 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3f8145110 0x7fa3f8060000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:18.004 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f7fff700 1 -- 172.21.2.104:0/641341924 >> v1:172.21.2.105:6789/0 conn(0x7fa3f80b1fb0 legacy=0x7fa3f81422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:18.004 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f7fff700 1 -- 172.21.2.104:0/641341924 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f8144730 con 0x7fa3f80b7580 2021-07-11T03:53:18.004 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f7fff700 1 --2- 172.21.2.104:0/641341924 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3f80b7580 0x7fa3f80b7960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fa3f8066860 tx=0x7fa3e0006eb0).ready entity=mon.0 client_cookie=35ccba8450a07249 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:18.004 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f6ffd700 1 -- 172.21.2.104:0/641341924 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3e000d570 con 0x7fa3f80b7580 2021-07-11T03:53:18.005 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f6ffd700 1 -- 172.21.2.104:0/641341924 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e0000e90 con 0x7fa3f80b7580 2021-07-11T03:53:18.005 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f6ffd700 1 -- 172.21.2.104:0/641341924 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3e000d8c0 con 0x7fa3f80b7580 2021-07-11T03:53:18.005 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- 172.21.2.104:0/641341924 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3f80b7580 msgr2=0x7fa3f80b7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:18.005 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 --2- 172.21.2.104:0/641341924 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3f80b7580 0x7fa3f80b7960 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fa3f8066860 tx=0x7fa3e0006eb0).stop 2021-07-11T03:53:18.006 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- 172.21.2.104:0/641341924 shutdown_connections 2021-07-11T03:53:18.006 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 --2- 172.21.2.104:0/641341924 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3f8145110 0x7fa3f8060000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:18.006 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 --2- 172.21.2.104:0/641341924 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3f80b7580 0x7fa3f80b7960 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:18.006 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- 172.21.2.104:0/641341924 >> 172.21.2.104:0/641341924 conn(0x7fa3f8064890 msgr2=0x7fa3f8064c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:18.007 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- 172.21.2.104:0/641341924 shutdown_connections 2021-07-11T03:53:18.008 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- 172.21.2.104:0/641341924 wait complete. 2021-07-11T03:53:18.008 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 Processor -- start 2021-07-11T03:53:18.008 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- start start 2021-07-11T03:53:18.008 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3f80b1fb0 0x7fa3f813f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:18.009 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3f80b7580 0x7fa3f8153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:18.009 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3f8145110 0x7fa3f8154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:18.009 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80aada0 con 0x7fa3f8145110 2021-07-11T03:53:18.009 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81538c0 con 0x7fa3f80b1fb0 2021-07-11T03:53:18.009 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f80aea90 con 0x7fa3f80b7580 2021-07-11T03:53:18.010 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fcbd4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3f8145110 0x7fa3f8154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:18.010 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fcbd4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3f8145110 0x7fa3f8154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41470/0 (socket says 172.21.2.104:41470) 2021-07-11T03:53:18.010 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fcbd4700 1 -- 172.21.2.104:0/2691738861 learned_addr learned my addr 172.21.2.104:0/2691738861 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:18.010 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fcbd4700 1 -- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3f80b7580 msgr2=0x7fa3f8153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:18.011 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fcbd4700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3f80b7580 0x7fa3f8153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:18.011 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fcbd4700 1 -- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3f80b1fb0 msgr2=0x7fa3f813f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:18.011 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fcbd4700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3f80b1fb0 0x7fa3f813f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:18.011 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fcbd4700 1 -- 172.21.2.104:0/2691738861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e000b1e0 con 0x7fa3f8145110 2021-07-11T03:53:18.011 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fcbd4700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3f8145110 0x7fa3f8154250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa3f0003310 tx=0x7fa3f0009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:18.012 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f57fa700 1 -- 172.21.2.104:0/2691738861 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3f0001500 con 0x7fa3f8145110 2021-07-11T03:53:18.012 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f57fa700 1 -- 172.21.2.104:0/2691738861 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f0001660 con 0x7fa3f8145110 2021-07-11T03:53:18.012 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3fe637700 1 -- 172.21.2.104:0/2691738861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f8141380 con 0x7fa3f8145110 2021-07-11T03:53:18.012 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:17.996+0000 7fa3f57fa700 1 -- 172.21.2.104:0/2691738861 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3f00019c0 con 0x7fa3f8145110 2021-07-11T03:53:18.013 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.000+0000 7fa3fe637700 1 -- 172.21.2.104:0/2691738861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f8141ec0 con 0x7fa3f8145110 2021-07-11T03:53:18.013 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.000+0000 7fa3fe637700 1 -- 172.21.2.104:0/2691738861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f8153a80 con 0x7fa3f8145110 2021-07-11T03:53:18.013 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.000+0000 7fa3f57fa700 1 -- 172.21.2.104:0/2691738861 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa3f0018030 con 0x7fa3f8145110 2021-07-11T03:53:18.013 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.000+0000 7fa3f57fa700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3dc0531f0 0x7fa3dc0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:18.013 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.000+0000 7fa3f57fa700 1 -- 172.21.2.104:0/2691738861 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4896+0+0 (secure 0 0 0) 0x7fa3f006aab0 con 0x7fa3f8145110 2021-07-11T03:53:18.014 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.000+0000 7fa3f7fff700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3dc0531f0 0x7fa3dc0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:18.014 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.000+0000 7fa3f7fff700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3dc0531f0 0x7fa3dc0556b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa3f8066860 tx=0x7fa3e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:18.015 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.000+0000 7fa3f57fa700 1 -- 172.21.2.104:0/2691738861 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa3f0042230 con 0x7fa3f8145110 2021-07-11T03:53:18.088 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.084+0000 7fa3fe637700 1 -- 172.21.2.104:0/2691738861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3f8057350 con 0x7fa3f8145110 2021-07-11T03:53:18.089 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.084+0000 7fa3f57fa700 1 -- 172.21.2.104:0/2691738861 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11805 (secure 0 0 0) 0x7fa3f0022e30 con 0x7fa3f8145110 2021-07-11T03:53:18.089 INFO:teuthology.misc.health.gibba004.stdout: 2021-07-11T03:53:18.090 INFO:teuthology.misc.health.gibba004.stdout:{"epoch":16,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:16.418947+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:18.091 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 -- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3dc0531f0 msgr2=0x7fa3dc0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:18.091 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3dc0531f0 0x7fa3dc0556b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fa3f8066860 tx=0x7fa3e0015000).stop 2021-07-11T03:53:18.091 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 -- 172.21.2.104:0/2691738861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3f8145110 msgr2=0x7fa3f8154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:18.092 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3f8145110 0x7fa3f8154250 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa3f0003310 tx=0x7fa3f0009120).stop 2021-07-11T03:53:18.092 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 -- 172.21.2.104:0/2691738861 shutdown_connections 2021-07-11T03:53:18.092 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3dc0531f0 0x7fa3dc0556b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:18.092 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3f8145110 0x7fa3f8154250 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:18.092 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3f80b7580 0x7fa3f8153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:18.093 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 --2- 172.21.2.104:0/2691738861 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3f80b1fb0 0x7fa3f813f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:18.093 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 -- 172.21.2.104:0/2691738861 >> 172.21.2.104:0/2691738861 conn(0x7fa3f8064890 msgr2=0x7fa3f813d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:18.093 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 -- 172.21.2.104:0/2691738861 shutdown_connections 2021-07-11T03:53:18.093 INFO:teuthology.misc.health.gibba004.stderr:2021-07-11T03:53:18.088+0000 7fa3fe637700 1 -- 172.21.2.104:0/2691738861 wait complete. 2021-07-11T03:53:18.097 DEBUG:teuthology.misc:8 of 8 OSDs are up 2021-07-11T03:53:18.097 INFO:tasks.ceph:Creating RBD pool 2021-07-11T03:53:18.098 DEBUG:teuthology.orchestra.run.gibba004:> sudo ceph --cluster ceph osd pool create rbd 8 2021-07-11T03:53:18.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 Processor -- start 2021-07-11T03:53:18.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 -- start start 2021-07-11T03:53:18.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd300b1ec0 0x7fcd300b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:18.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcd300b22a0 0x7fcd30141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:18.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd300a77d0 con 0x7fcd300affb0 2021-07-11T03:53:18.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd300b03f0 con 0x7fcd300b22a0 2021-07-11T03:53:18.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300b0650 con 0x7fcd300b1ec0 2021-07-11T03:53:18.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2f7fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd300b1ec0 0x7fcd300b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:18.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2effd700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fcd300affb0 0x7fcd300a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:37616/0 (socket says 172.21.2.104:37616) 2021-07-11T03:53:18.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2effd700 1 -- 172.21.2.104:0/1759211542 learned_addr learned my addr 172.21.2.104:0/1759211542 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:18.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2ffff700 1 --2- 172.21.2.104:0/1759211542 >> v2:172.21.2.105:3300/0 conn(0x7fcd300b22a0 0x7fcd30141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:18.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2ffff700 1 -- 172.21.2.104:0/1759211542 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd300b1ec0 msgr2=0x7fcd300b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:18.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2ffff700 1 --2- 172.21.2.104:0/1759211542 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd300b1ec0 0x7fcd300b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:18.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2ffff700 1 -- 172.21.2.104:0/1759211542 >> v1:172.21.2.105:6789/0 conn(0x7fcd300affb0 legacy=0x7fcd300a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:18.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2ffff700 1 -- 172.21.2.104:0/1759211542 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300b08f0 con 0x7fcd300b22a0 2021-07-11T03:53:18.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2ffff700 1 --2- 172.21.2.104:0/1759211542 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd300b22a0 0x7fcd30141250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fcd280032f0 tx=0x7fcd280091d0).ready entity=mon.1 client_cookie=3314b2c05b74e4d7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:18.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2e7fc700 1 -- 172.21.2.104:0/1759211542 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2800b6c0 con 0x7fcd300b22a0 2021-07-11T03:53:18.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2e7fc700 1 -- 172.21.2.104:0/1759211542 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2800b820 con 0x7fcd300b22a0 2021-07-11T03:53:18.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd2e7fc700 1 -- 172.21.2.104:0/1759211542 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2800b990 con 0x7fcd300b22a0 2021-07-11T03:53:18.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 -- 172.21.2.104:0/1759211542 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd300b22a0 msgr2=0x7fcd30141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:18.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 --2- 172.21.2.104:0/1759211542 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd300b22a0 0x7fcd30141250 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fcd280032f0 tx=0x7fcd280091d0).stop 2021-07-11T03:53:18.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 -- 172.21.2.104:0/1759211542 shutdown_connections 2021-07-11T03:53:18.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 --2- 172.21.2.104:0/1759211542 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd300b22a0 0x7fcd30141250 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:18.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 --2- 172.21.2.104:0/1759211542 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd300b1ec0 0x7fcd300b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:18.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 -- 172.21.2.104:0/1759211542 >> 172.21.2.104:0/1759211542 conn(0x7fcd3005f6b0 msgr2=0x7fcd3005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:18.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 -- 172.21.2.104:0/1759211542 shutdown_connections 2021-07-11T03:53:18.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.148+0000 7fcd3617a700 1 -- 172.21.2.104:0/1759211542 wait complete. 2021-07-11T03:53:18.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd3617a700 1 Processor -- start 2021-07-11T03:53:18.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd3617a700 1 -- start start 2021-07-11T03:53:18.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd3617a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd300affb0 0x7fcd3013dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:18.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd3617a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd300b1ec0 0x7fcd3014a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:18.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd3617a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd300b22a0 0x7fcd300b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:18.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd3617a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300b3e80 con 0x7fcd300affb0 2021-07-11T03:53:18.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd3617a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3014a570 con 0x7fcd300b22a0 2021-07-11T03:53:18.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd3617a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3014b8c0 con 0x7fcd300b1ec0 2021-07-11T03:53:18.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd300affb0 0x7fcd3013dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:18.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd300affb0 0x7fcd3013dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41482/0 (socket says 172.21.2.104:41482) 2021-07-11T03:53:18.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2f7fe700 1 -- 172.21.2.104:0/2554233870 learned_addr learned my addr 172.21.2.104:0/2554233870 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:18.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2ffff700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd300b22a0 0x7fcd300b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:18.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2effd700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd300b1ec0 0x7fcd3014a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:18.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2f7fe700 1 -- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd300b1ec0 msgr2=0x7fcd3014a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:18.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2f7fe700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd300b1ec0 0x7fcd3014a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:18.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2f7fe700 1 -- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd300b22a0 msgr2=0x7fcd300b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:18.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2f7fe700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd300b22a0 0x7fcd300b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:18.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2f7fe700 1 -- 172.21.2.104:0/2554233870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd28009470 con 0x7fcd300affb0 2021-07-11T03:53:18.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2f7fe700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd300affb0 0x7fcd3013dd00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fcd3014a700 tx=0x7fcd20006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:18.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2cff9700 1 -- 172.21.2.104:0/2554233870 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2000c830 con 0x7fcd300affb0 2021-07-11T03:53:18.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2cff9700 1 -- 172.21.2.104:0/2554233870 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2000c990 con 0x7fcd300affb0 2021-07-11T03:53:18.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd3617a700 1 -- 172.21.2.104:0/2554233870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd300abc90 con 0x7fcd300affb0 2021-07-11T03:53:18.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2cff9700 1 -- 172.21.2.104:0/2554233870 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd2000ccf0 con 0x7fcd300affb0 2021-07-11T03:53:18.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd3617a700 1 -- 172.21.2.104:0/2554233870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd300b5d10 con 0x7fcd300affb0 2021-07-11T03:53:18.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd3617a700 1 -- 172.21.2.104:0/2554233870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd30057350 con 0x7fcd300affb0 2021-07-11T03:53:18.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2cff9700 1 -- 172.21.2.104:0/2554233870 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcd2001c460 con 0x7fcd300affb0 2021-07-11T03:53:18.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2cff9700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd14053210 0x7fcd140556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:18.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2cff9700 1 -- 172.21.2.104:0/2554233870 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4896+0+0 (secure 0 0 0) 0x7fcd2006b560 con 0x7fcd300affb0 2021-07-11T03:53:18.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2effd700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd14053210 0x7fcd140556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:18.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.152+0000 7fcd2effd700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd14053210 0x7fcd140556d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fcd1800b000 tx=0x7fcd18009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:18.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.156+0000 7fcd2cff9700 1 -- 172.21.2.104:0/2554233870 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcd20042ce0 con 0x7fcd300affb0 2021-07-11T03:53:18.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:18.240+0000 7fcd3617a700 1 -- 172.21.2.104:0/2554233870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fcd3014af80 con 0x7fcd300affb0 2021-07-11T03:53:19.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd2cff9700 1 -- 172.21.2.104:0/2554233870 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fcd20046300 con 0x7fcd300affb0 2021-07-11T03:53:19.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 -- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd14053210 msgr2=0x7fcd140556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:19.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd14053210 0x7fcd140556d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fcd1800b000 tx=0x7fcd18009040).stop 2021-07-11T03:53:19.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 -- 172.21.2.104:0/2554233870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd300affb0 msgr2=0x7fcd3013dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:19.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd300affb0 0x7fcd3013dd00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fcd3014a700 tx=0x7fcd20006df0).stop 2021-07-11T03:53:19.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 -- 172.21.2.104:0/2554233870 shutdown_connections 2021-07-11T03:53:19.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd300b22a0 0x7fcd300b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:19.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd300b1ec0 0x7fcd3014a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:19.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd14053210 0x7fcd140556d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:19.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 --2- 172.21.2.104:0/2554233870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd300affb0 0x7fcd3013dd00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:19.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 -- 172.21.2.104:0/2554233870 >> 172.21.2.104:0/2554233870 conn(0x7fcd3005f6b0 msgr2=0x7fcd300a8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:19.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 -- 172.21.2.104:0/2554233870 shutdown_connections 2021-07-11T03:53:19.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.136+0000 7fcd3617a700 1 -- 172.21.2.104:0/2554233870 wait complete. 2021-07-11T03:53:19.144 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'rbd' created 2021-07-11T03:53:19.148 DEBUG:teuthology.orchestra.run.gibba004:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2021-07-11T03:53:19.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 Processor -- start 2021-07-11T03:53:19.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 -- start start 2021-07-11T03:53:19.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f85880aa120 0x7f85880ae440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:19.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85880b0880 0x7f85881405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:19.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85880b2af0 con 0x7f85880b04a0 2021-07-11T03:53:19.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85880b2770 con 0x7f85880aa120 2021-07-11T03:53:19.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85880b2d20 con 0x7f85880b0880 2021-07-11T03:53:19.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f85867fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f85880aa120 0x7f85880ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:19.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f85867fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f85880aa120 0x7f85880ae440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41492/0 (socket says 172.21.2.104:41492) 2021-07-11T03:53:19.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f85867fc700 1 -- 172.21.2.104:0/16875290 learned_addr learned my addr 172.21.2.104:0/16875290 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:19.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f85867fc700 1 -- 172.21.2.104:0/16875290 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85880b0880 msgr2=0x7f85881405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:19.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f85867fc700 1 --2- 172.21.2.104:0/16875290 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85880b0880 0x7f85881405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:19.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f85867fc700 1 -- 172.21.2.104:0/16875290 >> v1:172.21.2.105:6789/0 conn(0x7f85880b04a0 legacy=0x7f85880add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:19.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f85867fc700 1 -- 172.21.2.104:0/16875290 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85880a9b70 con 0x7f85880aa120 2021-07-11T03:53:19.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f85867fc700 1 --2- 172.21.2.104:0/16875290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85880aa120 0x7f85880ae440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f85880b7770 tx=0x7f857c006f40).ready entity=mon.0 client_cookie=b789166db41d8f40 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:19.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f8585ffb700 1 -- 172.21.2.104:0/16875290 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857c00dc30 con 0x7f85880aa120 2021-07-11T03:53:19.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f8585ffb700 1 -- 172.21.2.104:0/16875290 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c000e90 con 0x7f85880aa120 2021-07-11T03:53:19.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f8585ffb700 1 -- 172.21.2.104:0/16875290 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857c00be90 con 0x7f85880aa120 2021-07-11T03:53:19.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 -- 172.21.2.104:0/16875290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85880aa120 msgr2=0x7f85880ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:19.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 --2- 172.21.2.104:0/16875290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85880aa120 0x7f85880ae440 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f85880b7770 tx=0x7f857c006f40).stop 2021-07-11T03:53:19.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 -- 172.21.2.104:0/16875290 shutdown_connections 2021-07-11T03:53:19.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 --2- 172.21.2.104:0/16875290 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85880b0880 0x7f85881405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:19.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 --2- 172.21.2.104:0/16875290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85880aa120 0x7f85880ae440 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:19.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 -- 172.21.2.104:0/16875290 >> 172.21.2.104:0/16875290 conn(0x7f85880607c0 msgr2=0x7f8588060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:19.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 -- 172.21.2.104:0/16875290 shutdown_connections 2021-07-11T03:53:19.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 -- 172.21.2.104:0/16875290 wait complete. 2021-07-11T03:53:19.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 Processor -- start 2021-07-11T03:53:19.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.200+0000 7f858d7cd700 1 -- start start 2021-07-11T03:53:19.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f858d7cd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85880aa120 0x7f85880b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:19.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f858d7cd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85880b04a0 0x7f858815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:19.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f858d7cd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85880b0880 0x7f858815a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:19.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f858d7cd700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8588145510 con 0x7f85880b0880 2021-07-11T03:53:19.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f858d7cd700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85881452a0 con 0x7f85880aa120 2021-07-11T03:53:19.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f858d7cd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8588143d10 con 0x7f85880b04a0 2021-07-11T03:53:19.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f85877fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85880b0880 0x7f858815a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:19.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f85877fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85880b0880 0x7f858815a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41498/0 (socket says 172.21.2.104:41498) 2021-07-11T03:53:19.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f85877fe700 1 -- 172.21.2.104:0/2966816097 learned_addr learned my addr 172.21.2.104:0/2966816097 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:19.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f8586ffd700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85880aa120 0x7f85880b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:19.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f85877fe700 1 -- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85880b04a0 msgr2=0x7f858815a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:19.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f85877fe700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85880b04a0 0x7f858815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:19.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f85877fe700 1 -- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85880aa120 msgr2=0x7f85880b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:19.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f85877fe700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85880aa120 0x7f85880b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:19.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f85877fe700 1 -- 172.21.2.104:0/2966816097 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c00b460 con 0x7f85880b0880 2021-07-11T03:53:19.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f85877fe700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85880b0880 0x7f858815a8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f85800032f0 tx=0x7f85800091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:19.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f8577fff700 1 -- 172.21.2.104:0/2966816097 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f858000f510 con 0x7f85880b0880 2021-07-11T03:53:19.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f8577fff700 1 -- 172.21.2.104:0/2966816097 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858000f670 con 0x7f85880b0880 2021-07-11T03:53:19.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f858d7cd700 1 -- 172.21.2.104:0/2966816097 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8588143fb0 con 0x7f85880b0880 2021-07-11T03:53:19.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f858d7cd700 1 -- 172.21.2.104:0/2966816097 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858815ada0 con 0x7f85880b0880 2021-07-11T03:53:19.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f8577fff700 1 -- 172.21.2.104:0/2966816097 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f858000f7e0 con 0x7f85880b0880 2021-07-11T03:53:19.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f858d7cd700 1 -- 172.21.2.104:0/2966816097 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8568003410 con 0x7f85880b0880 2021-07-11T03:53:19.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f8577fff700 1 -- 172.21.2.104:0/2966816097 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f858000f070 con 0x7f85880b0880 2021-07-11T03:53:19.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f8577fff700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85640532d0 0x7f8564055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:19.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f8577fff700 1 -- 172.21.2.104:0/2966816097 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5238+0+0 (secure 0 0 0) 0x7f858006b9e0 con 0x7f85880b0880 2021-07-11T03:53:19.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f8586ffd700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85640532d0 0x7f8564055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:19.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.204+0000 7f8586ffd700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85640532d0 0x7f8564055790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f8578008000 tx=0x7f8578009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:19.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.208+0000 7f8577fff700 1 -- 172.21.2.104:0/2966816097 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8580042ef0 con 0x7f85880b0880 2021-07-11T03:53:19.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:19.288+0000 7f858d7cd700 1 -- 172.21.2.104:0/2966816097 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8568003990 con 0x7f85880b0880 2021-07-11T03:53:20.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f8577fff700 1 -- 172.21.2.104:0/2966816097 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f85800280c0 con 0x7f85880b0880 2021-07-11T03:53:20.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 -- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85640532d0 msgr2=0x7f8564055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85640532d0 0x7f8564055790 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f8578008000 tx=0x7f8578009040).stop 2021-07-11T03:53:20.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 -- 172.21.2.104:0/2966816097 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85880b0880 msgr2=0x7f858815a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85880b0880 0x7f858815a8b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f85800032f0 tx=0x7f85800091a0).stop 2021-07-11T03:53:20.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 -- 172.21.2.104:0/2966816097 shutdown_connections 2021-07-11T03:53:20.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85640532d0 0x7f8564055790 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85880b0880 0x7f858815a8b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85880b04a0 0x7f858815a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 --2- 172.21.2.104:0/2966816097 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85880aa120 0x7f85880b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 -- 172.21.2.104:0/2966816097 >> 172.21.2.104:0/2966816097 conn(0x7f85880607c0 msgr2=0x7f85880b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:20.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 -- 172.21.2.104:0/2966816097 shutdown_connections 2021-07-11T03:53:20.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.148+0000 7f858d7cd700 1 -- 172.21.2.104:0/2966816097 wait complete. 2021-07-11T03:53:20.155 INFO:teuthology.orchestra.run.gibba004.stderr:enabled application 'rbd' on pool 'rbd' 2021-07-11T03:53:20.159 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2021-07-11T03:53:20.159 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T03:53:20.160 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T03:53:20.161 INFO:tasks.ceph.mds.a:Started 2021-07-11T03:53:20.162 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T03:53:20.162 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T03:53:20.164 INFO:tasks.ceph.mds.c:Started 2021-07-11T03:53:20.165 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T03:53:20.165 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T03:53:20.167 INFO:tasks.ceph.mds.b:Started 2021-07-11T03:53:20.168 INFO:tasks.ceph:Setting up CephFS filesystem... 2021-07-11T03:53:20.169 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:20.196 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T03:53:20.197 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T03:53:20.203 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T03:53:20.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 Processor -- start 2021-07-11T03:53:20.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 -- start start 2021-07-11T03:53:20.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb2080ad150 0x7fb2080b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2080ac370 con 0x7fb2080aa860 2021-07-11T03:53:20.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2080aac40 con 0x7fb2080a9da0 2021-07-11T03:53:20.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2080abb50 con 0x7fb2080ad150 2021-07-11T03:53:20.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb2077fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb2080ad150 0x7fb2080b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb207fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb2080aa860 0x7fb2080aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38294/0 (socket says 172.21.2.104:38294) 2021-07-11T03:53:20.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb2077fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb2080ad150 0x7fb2080b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41528/0 (socket says 172.21.2.104:41528) 2021-07-11T03:53:20.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb207fff700 1 -- 172.21.2.104:0/4041135863 learned_addr learned my addr 172.21.2.104:0/4041135863 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:20.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb2077fe700 1 -- 172.21.2.104:0/4041135863 >> v1:172.21.2.105:6789/0 conn(0x7fb2080a9da0 legacy=0x7fb208140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb2077fe700 1 -- 172.21.2.104:0/4041135863 >> v1:172.21.2.104:6789/0 conn(0x7fb2080aa860 legacy=0x7fb2080aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb2077fe700 1 -- 172.21.2.104:0/4041135863 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2080b16b0 con 0x7fb2080ad150 2021-07-11T03:53:20.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb2077fe700 1 --2- 172.21.2.104:0/4041135863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2080ad150 0x7fb2080b11c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fb1fc002f70 tx=0x7fb1fc006df0).ready entity=mon.0 client_cookie=f6c41260aa18c24c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb206ffd700 1 -- 172.21.2.104:0/4041135863 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1fc00d610 con 0x7fb2080ad150 2021-07-11T03:53:20.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb206ffd700 1 -- 172.21.2.104:0/4041135863 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1fc000e90 con 0x7fb2080ad150 2021-07-11T03:53:20.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb206ffd700 1 -- 172.21.2.104:0/4041135863 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1fc00d960 con 0x7fb2080ad150 2021-07-11T03:53:20.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 -- 172.21.2.104:0/4041135863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2080ad150 msgr2=0x7fb2080b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 --2- 172.21.2.104:0/4041135863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2080ad150 0x7fb2080b11c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fb1fc002f70 tx=0x7fb1fc006df0).stop 2021-07-11T03:53:20.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 -- 172.21.2.104:0/4041135863 shutdown_connections 2021-07-11T03:53:20.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 --2- 172.21.2.104:0/4041135863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2080ad150 0x7fb2080b11c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 -- 172.21.2.104:0/4041135863 >> 172.21.2.104:0/4041135863 conn(0x7fb2080b64d0 msgr2=0x7fb2080b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:20.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 -- 172.21.2.104:0/4041135863 shutdown_connections 2021-07-11T03:53:20.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 -- 172.21.2.104:0/4041135863 wait complete. 2021-07-11T03:53:20.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 Processor -- start 2021-07-11T03:53:20.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.220+0000 7fb20e7fb700 1 -- start start 2021-07-11T03:53:20.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20e7fb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb2080a9da0 0x7fb208152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20e7fb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb2080aa860 0x7fb208066050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20e7fb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2080ad150 0x7fb208143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20e7fb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2081533b0 con 0x7fb2080ad150 2021-07-11T03:53:20.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20e7fb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb208066e50 con 0x7fb2080a9da0 2021-07-11T03:53:20.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20e7fb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080a19e0 con 0x7fb2080aa860 2021-07-11T03:53:20.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20cd98700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2080ad150 0x7fb208143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20cd98700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2080ad150 0x7fb208143370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41536/0 (socket says 172.21.2.104:41536) 2021-07-11T03:53:20.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20cd98700 1 -- 172.21.2.104:0/3187639070 learned_addr learned my addr 172.21.2.104:0/3187639070 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:20.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb207fff700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb2080a9da0 0x7fb208152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb2077fe700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb2080aa860 0x7fb208066050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20cd98700 1 -- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb2080aa860 msgr2=0x7fb208066050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20cd98700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb2080aa860 0x7fb208066050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20cd98700 1 -- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb2080a9da0 msgr2=0x7fb208152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20cd98700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb2080a9da0 0x7fb208152ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20cd98700 1 -- 172.21.2.104:0/3187639070 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc00b180 con 0x7fb2080ad150 2021-07-11T03:53:20.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20cd98700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2080ad150 0x7fb208143370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb2080b63e0 tx=0x7fb1f8007b80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb2057fa700 1 -- 172.21.2.104:0/3187639070 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1f8012400 con 0x7fb2080ad150 2021-07-11T03:53:20.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb2057fa700 1 -- 172.21.2.104:0/3187639070 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f8012560 con 0x7fb2080ad150 2021-07-11T03:53:20.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20e7fb700 1 -- 172.21.2.104:0/3187639070 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2080a1c80 con 0x7fb2080ad150 2021-07-11T03:53:20.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb2057fa700 1 -- 172.21.2.104:0/3187639070 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1f80128c0 con 0x7fb2080ad150 2021-07-11T03:53:20.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20e7fb700 1 -- 172.21.2.104:0/3187639070 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2080a15f0 con 0x7fb2080ad150 2021-07-11T03:53:20.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb20e7fb700 1 -- 172.21.2.104:0/3187639070 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb208057350 con 0x7fb2080ad150 2021-07-11T03:53:20.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb2057fa700 1 -- 172.21.2.104:0/3187639070 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb1f8019030 con 0x7fb2080ad150 2021-07-11T03:53:20.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb2057fa700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1e8053380 0x7fb1e8055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb2057fa700 1 -- 172.21.2.104:0/3187639070 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb1f806a460 con 0x7fb2080ad150 2021-07-11T03:53:20.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb207fff700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1e8053380 0x7fb1e8055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.224+0000 7fb207fff700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1e8053380 0x7fb1e8055840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb20000d000 tx=0x7fb200009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.228+0000 7fb2057fa700 1 -- 172.21.2.104:0/3187639070 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb1f8041ad0 con 0x7fb2080ad150 2021-07-11T03:53:20.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 -- 172.21.2.104:0/3187639070 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2080636e0 con 0x7fb2080ad150 2021-07-11T03:53:20.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb2057fa700 1 -- 172.21.2.104:0/3187639070 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13191 (secure 0 0 0) 0x7fb1f8017020 con 0x7fb2080ad150 2021-07-11T03:53:20.316 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:20.316 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":18,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:20.137353+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:20.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 -- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1e8053380 msgr2=0x7fb1e8055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1e8053380 0x7fb1e8055840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fb20000d000 tx=0x7fb200009040).stop 2021-07-11T03:53:20.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 -- 172.21.2.104:0/3187639070 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2080ad150 msgr2=0x7fb208143370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2080ad150 0x7fb208143370 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb2080b63e0 tx=0x7fb1f8007b80).stop 2021-07-11T03:53:20.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 -- 172.21.2.104:0/3187639070 shutdown_connections 2021-07-11T03:53:20.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1e8053380 0x7fb1e8055840 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2080ad150 0x7fb208143370 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb2080aa860 0x7fb208066050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 --2- 172.21.2.104:0/3187639070 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb2080a9da0 0x7fb208152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 -- 172.21.2.104:0/3187639070 >> 172.21.2.104:0/3187639070 conn(0x7fb2080b64d0 msgr2=0x7fb2080b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:20.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 -- 172.21.2.104:0/3187639070 shutdown_connections 2021-07-11T03:53:20.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.312+0000 7fb20e7fb700 1 -- 172.21.2.104:0/3187639070 wait complete. 2021-07-11T03:53:20.323 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T03:53:20.324 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:53:20.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 Processor -- start 2021-07-11T03:53:20.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 -- start start 2021-07-11T03:53:20.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fbf00060100 0x7fbf00064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbf00144440 0x7fbf00065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf000604e0 con 0x7fbf000b38d0 2021-07-11T03:53:20.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf000656f0 con 0x7fbf00144440 2021-07-11T03:53:20.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf00065c80 con 0x7fbf00060100 2021-07-11T03:53:20.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf05166700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fbf000b38d0 0x7fbf00144970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38312/0 (socket says 172.21.2.104:38312) 2021-07-11T03:53:20.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf05166700 1 -- 172.21.2.104:0/4239651586 learned_addr learned my addr 172.21.2.104:0/4239651586 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:20.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbefffff700 1 --2- 172.21.2.104:0/4239651586 >> v2:172.21.2.104:3300/0 conn(0x7fbf00144440 0x7fbf00065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf04965700 1 --2- 172.21.2.104:0/4239651586 >> v2:172.21.2.105:3300/0 conn(0x7fbf00060100 0x7fbf00064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbefffff700 1 -- 172.21.2.104:0/4239651586 >> v2:172.21.2.105:3300/0 conn(0x7fbf00060100 msgr2=0x7fbf00064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:20.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbefffff700 1 --2- 172.21.2.104:0/4239651586 >> v2:172.21.2.105:3300/0 conn(0x7fbf00060100 0x7fbf00064d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbefffff700 1 -- 172.21.2.104:0/4239651586 >> v1:172.21.2.104:6789/0 conn(0x7fbf000b38d0 legacy=0x7fbf00144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbefffff700 1 -- 172.21.2.104:0/4239651586 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf000b23b0 con 0x7fbf00144440 2021-07-11T03:53:20.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbefffff700 1 --2- 172.21.2.104:0/4239651586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbf00144440 0x7fbf00065200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fbef0001110 tx=0x7fbef4006df0).ready entity=mon.0 client_cookie=676d8c30f4694b80 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbeff7fe700 1 -- 172.21.2.104:0/4239651586 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef400d570 con 0x7fbf00144440 2021-07-11T03:53:20.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbeff7fe700 1 -- 172.21.2.104:0/4239651586 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef4000e90 con 0x7fbf00144440 2021-07-11T03:53:20.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbeff7fe700 1 -- 172.21.2.104:0/4239651586 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef400d8c0 con 0x7fbf00144440 2021-07-11T03:53:20.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 -- 172.21.2.104:0/4239651586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbf00144440 msgr2=0x7fbf00065200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 --2- 172.21.2.104:0/4239651586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbf00144440 0x7fbf00065200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fbef0001110 tx=0x7fbef4006df0).stop 2021-07-11T03:53:20.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 -- 172.21.2.104:0/4239651586 shutdown_connections 2021-07-11T03:53:20.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 --2- 172.21.2.104:0/4239651586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbf00144440 0x7fbf00065200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 --2- 172.21.2.104:0/4239651586 >> v2:172.21.2.105:3300/0 conn(0x7fbf00060100 0x7fbf00064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 -- 172.21.2.104:0/4239651586 >> 172.21.2.104:0/4239651586 conn(0x7fbf0005cf60 msgr2=0x7fbf0005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:20.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 -- 172.21.2.104:0/4239651586 shutdown_connections 2021-07-11T03:53:20.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.416+0000 7fbf06bc9700 1 -- 172.21.2.104:0/4239651586 wait complete. 2021-07-11T03:53:20.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf06bc9700 1 Processor -- start 2021-07-11T03:53:20.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf06bc9700 1 -- start start 2021-07-11T03:53:20.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf06bc9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbf00060100 0x7fbf00142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf06bc9700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbf000b38d0 0x7fbf000b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf06bc9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbf000b6cd0 0x7fbf00138f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf06bc9700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000aff80 con 0x7fbf000b6cd0 2021-07-11T03:53:20.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf06bc9700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf00142530 con 0x7fbf000b38d0 2021-07-11T03:53:20.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf06bc9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000b0250 con 0x7fbf00060100 2021-07-11T03:53:20.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf05166700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbf000b6cd0 0x7fbf00138f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf05166700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbf000b6cd0 0x7fbf00138f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41550/0 (socket says 172.21.2.104:41550) 2021-07-11T03:53:20.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf05166700 1 -- 172.21.2.104:0/1265664683 learned_addr learned my addr 172.21.2.104:0/1265664683 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:20.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf05166700 1 -- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbf00060100 msgr2=0x7fbf00142040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:20.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf05166700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbf00060100 0x7fbf00142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf05166700 1 -- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbf000b38d0 msgr2=0x7fbf000b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf05166700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbf000b38d0 0x7fbf000b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf05166700 1 -- 172.21.2.104:0/1265664683 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef400b180 con 0x7fbf000b6cd0 2021-07-11T03:53:20.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf05166700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbf000b6cd0 0x7fbf00138f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fbef00011d0 tx=0x7fbef0007b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbefdffb700 1 -- 172.21.2.104:0/1265664683 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef00072e0 con 0x7fbf000b6cd0 2021-07-11T03:53:20.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbefdffb700 1 -- 172.21.2.104:0/1265664683 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef0007440 con 0x7fbf000b6cd0 2021-07-11T03:53:20.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbefdffb700 1 -- 172.21.2.104:0/1265664683 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbef0005800 con 0x7fbf000b6cd0 2021-07-11T03:53:20.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf06bc9700 1 -- 172.21.2.104:0/1265664683 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf000b04f0 con 0x7fbf000b6cd0 2021-07-11T03:53:20.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf06bc9700 1 -- 172.21.2.104:0/1265664683 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf000b1120 con 0x7fbf000b6cd0 2021-07-11T03:53:20.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf06bc9700 1 -- 172.21.2.104:0/1265664683 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf00057350 con 0x7fbf000b6cd0 2021-07-11T03:53:20.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbefdffb700 1 -- 172.21.2.104:0/1265664683 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbef0010070 con 0x7fbf000b6cd0 2021-07-11T03:53:20.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbefdffb700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbee00531f0 0x7fbee00556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbefdffb700 1 -- 172.21.2.104:0/1265664683 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbef006a9e0 con 0x7fbf000b6cd0 2021-07-11T03:53:20.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.420+0000 7fbf04965700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbee00531f0 0x7fbee00556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.424+0000 7fbf04965700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbee00531f0 0x7fbee00556b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fbef8007000 tx=0x7fbef8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.424+0000 7fbefdffb700 1 -- 172.21.2.104:0/1265664683 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbef0042050 con 0x7fbf000b6cd0 2021-07-11T03:53:20.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 -- 172.21.2.104:0/1265664683 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbf000698e0 con 0x7fbf000b6cd0 2021-07-11T03:53:20.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbefdffb700 1 -- 172.21.2.104:0/1265664683 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbef0045670 con 0x7fbf000b6cd0 2021-07-11T03:53:20.512 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:53:20.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 -- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbee00531f0 msgr2=0x7fbee00556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbee00531f0 0x7fbee00556b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fbef8007000 tx=0x7fbef8008040).stop 2021-07-11T03:53:20.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 -- 172.21.2.104:0/1265664683 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbf000b6cd0 msgr2=0x7fbf00138f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbf000b6cd0 0x7fbf00138f00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fbef00011d0 tx=0x7fbef0007b50).stop 2021-07-11T03:53:20.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 -- 172.21.2.104:0/1265664683 shutdown_connections 2021-07-11T03:53:20.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbee00531f0 0x7fbee00556b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbf000b6cd0 0x7fbf00138f00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbf000b38d0 0x7fbf000b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 --2- 172.21.2.104:0/1265664683 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbf00060100 0x7fbf00142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 -- 172.21.2.104:0/1265664683 >> 172.21.2.104:0/1265664683 conn(0x7fbf0005cf60 msgr2=0x7fbf0013f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:20.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 -- 172.21.2.104:0/1265664683 shutdown_connections 2021-07-11T03:53:20.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.508+0000 7fbf06bc9700 1 -- 172.21.2.104:0/1265664683 wait complete. 2021-07-11T03:53:20.519 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:53:20.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 Processor -- start 2021-07-11T03:53:20.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 -- start start 2021-07-11T03:53:20.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9bbc0b4180 0x7f9bbc140940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9bbc0b4690 0x7f9bbc0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bbc143b60 con 0x7f9bbc146950 2021-07-11T03:53:20.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc147510 con 0x7f9bbc0b4180 2021-07-11T03:53:20.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bbc143e00 con 0x7f9bbc0b4690 2021-07-11T03:53:20.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc15ec700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f9bbc146950 0x7f9bbc140230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38322/0 (socket says 172.21.2.104:38322) 2021-07-11T03:53:20.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc15ec700 1 -- 172.21.2.104:0/1436954738 learned_addr learned my addr 172.21.2.104:0/1436954738 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:20.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc0deb700 1 --2- 172.21.2.104:0/1436954738 >> v2:172.21.2.104:3300/0 conn(0x7f9bbc0b4180 0x7f9bbc140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc0deb700 1 -- 172.21.2.104:0/1436954738 >> v2:172.21.2.105:3300/0 conn(0x7f9bbc0b4690 msgr2=0x7f9bbc0b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:20.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc0deb700 1 --2- 172.21.2.104:0/1436954738 >> v2:172.21.2.105:3300/0 conn(0x7f9bbc0b4690 0x7f9bbc0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc0deb700 1 -- 172.21.2.104:0/1436954738 >> v1:172.21.2.104:6789/0 conn(0x7f9bbc146950 legacy=0x7f9bbc140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc0deb700 1 -- 172.21.2.104:0/1436954738 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc0b0c90 con 0x7f9bbc0b4180 2021-07-11T03:53:20.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc0deb700 1 --2- 172.21.2.104:0/1436954738 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9bbc0b4180 0x7f9bbc140940 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9bb8002f70 tx=0x7f9bb8006df0).ready entity=mon.0 client_cookie=590dac163d04a3f3 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9baffff700 1 -- 172.21.2.104:0/1436954738 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bb800d5c0 con 0x7f9bbc0b4180 2021-07-11T03:53:20.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9baffff700 1 -- 172.21.2.104:0/1436954738 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb8000e90 con 0x7f9bbc0b4180 2021-07-11T03:53:20.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9baffff700 1 -- 172.21.2.104:0/1436954738 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bb800d910 con 0x7f9bbc0b4180 2021-07-11T03:53:20.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 -- 172.21.2.104:0/1436954738 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9bbc0b4180 msgr2=0x7f9bbc140940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 --2- 172.21.2.104:0/1436954738 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9bbc0b4180 0x7f9bbc140940 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9bb8002f70 tx=0x7f9bb8006df0).stop 2021-07-11T03:53:20.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 -- 172.21.2.104:0/1436954738 shutdown_connections 2021-07-11T03:53:20.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 --2- 172.21.2.104:0/1436954738 >> v2:172.21.2.105:3300/0 conn(0x7f9bbc0b4690 0x7f9bbc0b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 --2- 172.21.2.104:0/1436954738 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9bbc0b4180 0x7f9bbc140940 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.568+0000 7f9bc3850700 1 -- 172.21.2.104:0/1436954738 >> 172.21.2.104:0/1436954738 conn(0x7f9bbc05c490 msgr2=0x7f9bbc05c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:20.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 -- 172.21.2.104:0/1436954738 shutdown_connections 2021-07-11T03:53:20.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 -- 172.21.2.104:0/1436954738 wait complete. 2021-07-11T03:53:20.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 Processor -- start 2021-07-11T03:53:20.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 -- start start 2021-07-11T03:53:20.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9bbc0b4180 0x7f9bbc0b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9bbc0b4690 0x7f9bbc13a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9bbc146950 0x7f9bbc13acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0afc50 con 0x7f9bbc0b4690 2021-07-11T03:53:20.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0afe80 con 0x7f9bbc146950 2021-07-11T03:53:20.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bbc0b0120 con 0x7f9bbc0b4180 2021-07-11T03:53:20.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc0deb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9bbc0b4690 0x7f9bbc13a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc0deb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9bbc0b4690 0x7f9bbc13a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41562/0 (socket says 172.21.2.104:41562) 2021-07-11T03:53:20.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc0deb700 1 -- 172.21.2.104:0/542320429 learned_addr learned my addr 172.21.2.104:0/542320429 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:20.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc0deb700 1 -- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9bbc0b4180 msgr2=0x7f9bbc0b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc0deb700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9bbc0b4180 0x7f9bbc0b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc0deb700 1 -- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9bbc146950 msgr2=0x7f9bbc13acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc0deb700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9bbc146950 0x7f9bbc13acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc0deb700 1 -- 172.21.2.104:0/542320429 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb800b180 con 0x7f9bbc0b4690 2021-07-11T03:53:20.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc0deb700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9bbc0b4690 0x7f9bbc13a7c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9bb8002f70 tx=0x7f9bb8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bae7fc700 1 -- 172.21.2.104:0/542320429 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bb8006b20 con 0x7f9bbc0b4690 2021-07-11T03:53:20.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bae7fc700 1 -- 172.21.2.104:0/542320429 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb801f440 con 0x7f9bbc0b4690 2021-07-11T03:53:20.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 -- 172.21.2.104:0/542320429 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bbc0b03c0 con 0x7f9bbc0b4690 2021-07-11T03:53:20.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bae7fc700 1 -- 172.21.2.104:0/542320429 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9bb801f7a0 con 0x7f9bbc0b4690 2021-07-11T03:53:20.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 -- 172.21.2.104:0/542320429 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bbc1542f0 con 0x7f9bbc0b4690 2021-07-11T03:53:20.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc3850700 1 -- 172.21.2.104:0/542320429 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bbc0b6d90 con 0x7f9bbc0b4690 2021-07-11T03:53:20.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bae7fc700 1 -- 172.21.2.104:0/542320429 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9bb8025020 con 0x7f9bbc0b4690 2021-07-11T03:53:20.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bae7fc700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b9c053300 0x7f9b9c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bae7fc700 1 -- 172.21.2.104:0/542320429 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9bb806c360 con 0x7f9bbc0b4690 2021-07-11T03:53:20.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.572+0000 7f9bc15ec700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b9c053300 0x7f9b9c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.576+0000 7f9bc15ec700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b9c053300 0x7f9b9c0557c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9bbc060c50 tx=0x7f9bb0007860).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.576+0000 7f9bae7fc700 1 -- 172.21.2.104:0/542320429 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9bb8043a80 con 0x7f9bbc0b4690 2021-07-11T03:53:20.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 -- 172.21.2.104:0/542320429 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9bbc057350 con 0x7f9bbc0b4690 2021-07-11T03:53:20.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bae7fc700 1 -- 172.21.2.104:0/542320429 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9bb80470a0 con 0x7f9bbc0b4690 2021-07-11T03:53:20.663 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:20.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 -- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b9c053300 msgr2=0x7f9b9c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b9c053300 0x7f9b9c0557c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9bbc060c50 tx=0x7f9bb0007860).stop 2021-07-11T03:53:20.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 -- 172.21.2.104:0/542320429 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9bbc0b4690 msgr2=0x7f9bbc13a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9bbc0b4690 0x7f9bbc13a7c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f9bb8002f70 tx=0x7f9bb8014000).stop 2021-07-11T03:53:20.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 -- 172.21.2.104:0/542320429 shutdown_connections 2021-07-11T03:53:20.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9bbc146950 0x7f9bbc13acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b9c053300 0x7f9b9c0557c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9bbc0b4690 0x7f9bbc13a7c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 --2- 172.21.2.104:0/542320429 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9bbc0b4180 0x7f9bbc0b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 -- 172.21.2.104:0/542320429 >> 172.21.2.104:0/542320429 conn(0x7f9bbc05c490 msgr2=0x7f9bbc13dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:20.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 -- 172.21.2.104:0/542320429 shutdown_connections 2021-07-11T03:53:20.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.660+0000 7f9bc3850700 1 -- 172.21.2.104:0/542320429 wait complete. 2021-07-11T03:53:20.670 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T03:53:20.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f7883354700 1 Processor -- start 2021-07-11T03:53:20.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f7883354700 1 -- start start 2021-07-11T03:53:20.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f7883354700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f787c060100 0x7f787c0b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f7883354700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f787c067300 0x7f787c1438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f7883354700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c0aabb0 con 0x7f787c0b4c40 2021-07-11T03:53:20.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f7883354700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787c0604e0 con 0x7f787c060100 2021-07-11T03:53:20.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f7883354700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c066ea0 con 0x7f787c067300 2021-07-11T03:53:20.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f78818f1700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f787c0b4c40 0x7f787c144510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38336/0 (socket says 172.21.2.104:38336) 2021-07-11T03:53:20.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f78818f1700 1 -- 172.21.2.104:0/2535915119 learned_addr learned my addr 172.21.2.104:0/2535915119 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:20.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f78810f0700 1 --2- 172.21.2.104:0/2535915119 >> v2:172.21.2.104:3300/0 conn(0x7f787c060100 0x7f787c0b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f78810f0700 1 -- 172.21.2.104:0/2535915119 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f787c067300 msgr2=0x7f787c1438f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:20.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f78810f0700 1 --2- 172.21.2.104:0/2535915119 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f787c067300 0x7f787c1438f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f78810f0700 1 -- 172.21.2.104:0/2535915119 >> v1:172.21.2.104:6789/0 conn(0x7f787c0b4c40 legacy=0x7f787c144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f786ffff700 1 -- 172.21.2.104:0/2535915119 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789151214 0 0) 0x7f787c0aabb0 con 0x7f787c0b4c40 2021-07-11T03:53:20.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.716+0000 7f78810f0700 1 -- 172.21.2.104:0/2535915119 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0b33b0 con 0x7f787c060100 2021-07-11T03:53:20.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f78810f0700 1 --2- 172.21.2.104:0/2535915119 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f787c060100 0x7f787c0b4750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f7878002f70 tx=0x7f7878006df0).ready entity=mon.0 client_cookie=cea8287bc9568c50 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f786ffff700 1 -- 172.21.2.104:0/2535915119 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787800d570 con 0x7f787c060100 2021-07-11T03:53:20.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f786ffff700 1 -- 172.21.2.104:0/2535915119 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7878000e90 con 0x7f787c060100 2021-07-11T03:53:20.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f786ffff700 1 -- 172.21.2.104:0/2535915119 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f787800d8c0 con 0x7f787c060100 2021-07-11T03:53:20.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 -- 172.21.2.104:0/2535915119 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f787c060100 msgr2=0x7f787c0b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 --2- 172.21.2.104:0/2535915119 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f787c060100 0x7f787c0b4750 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f7878002f70 tx=0x7f7878006df0).stop 2021-07-11T03:53:20.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 -- 172.21.2.104:0/2535915119 shutdown_connections 2021-07-11T03:53:20.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 --2- 172.21.2.104:0/2535915119 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f787c067300 0x7f787c1438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 --2- 172.21.2.104:0/2535915119 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f787c060100 0x7f787c0b4750 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 -- 172.21.2.104:0/2535915119 >> 172.21.2.104:0/2535915119 conn(0x7f787c05cf30 msgr2=0x7f787c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:20.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 -- 172.21.2.104:0/2535915119 shutdown_connections 2021-07-11T03:53:20.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 -- 172.21.2.104:0/2535915119 wait complete. 2021-07-11T03:53:20.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 Processor -- start 2021-07-11T03:53:20.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 -- start start 2021-07-11T03:53:20.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f787c060100 0x7f787c0622b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f787c067300 0x7f787c0af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f787c0b4c40 0x7f787c0af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0618d0 con 0x7f787c060100 2021-07-11T03:53:20.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0a9fe0 con 0x7f787c067300 2021-07-11T03:53:20.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0aa280 con 0x7f787c0b4c40 2021-07-11T03:53:20.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f78810f0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f787c060100 0x7f787c0622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f78810f0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f787c060100 0x7f787c0622b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41572/0 (socket says 172.21.2.104:41572) 2021-07-11T03:53:20.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f78810f0700 1 -- 172.21.2.104:0/3289970075 learned_addr learned my addr 172.21.2.104:0/3289970075 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:20.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f78810f0700 1 -- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f787c0b4c40 msgr2=0x7f787c0af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f78810f0700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f787c0b4c40 0x7f787c0af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f78810f0700 1 -- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f787c067300 msgr2=0x7f787c0af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f78810f0700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f787c067300 0x7f787c0af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f78810f0700 1 -- 172.21.2.104:0/3289970075 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787800b180 con 0x7f787c060100 2021-07-11T03:53:20.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f78810f0700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f787c060100 0x7f787c0622b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7878002f70 tx=0x7f7878015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f786e7fc700 1 -- 172.21.2.104:0/3289970075 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78780119d0 con 0x7f787c060100 2021-07-11T03:53:20.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f786e7fc700 1 -- 172.21.2.104:0/3289970075 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7878011b30 con 0x7f787c060100 2021-07-11T03:53:20.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f786e7fc700 1 -- 172.21.2.104:0/3289970075 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7878011e90 con 0x7f787c060100 2021-07-11T03:53:20.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 -- 172.21.2.104:0/3289970075 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c0abe60 con 0x7f787c060100 2021-07-11T03:53:20.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.720+0000 7f7883354700 1 -- 172.21.2.104:0/3289970075 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c0aea20 con 0x7f787c060100 2021-07-11T03:53:20.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.724+0000 7f7883354700 1 -- 172.21.2.104:0/3289970075 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c057350 con 0x7f787c060100 2021-07-11T03:53:20.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.724+0000 7f786e7fc700 1 -- 172.21.2.104:0/3289970075 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f787800f070 con 0x7f787c060100 2021-07-11T03:53:20.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.724+0000 7f786e7fc700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f785c053270 0x7f785c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.724+0000 7f786e7fc700 1 -- 172.21.2.104:0/3289970075 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f787806cb10 con 0x7f787c060100 2021-07-11T03:53:20.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.724+0000 7f78808ef700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f785c053270 0x7f785c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.724+0000 7f78808ef700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f785c053270 0x7f785c055730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7870002fb0 tx=0x7f78700069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.724+0000 7f786e7fc700 1 -- 172.21.2.104:0/3289970075 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7878044180 con 0x7f787c060100 2021-07-11T03:53:20.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 -- 172.21.2.104:0/3289970075 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f787c0698e0 con 0x7f787c060100 2021-07-11T03:53:20.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f786e7fc700 1 -- 172.21.2.104:0/3289970075 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+143 (secure 0 0 0) 0x7f78780477a0 con 0x7f787c060100 2021-07-11T03:53:20.812 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:20.813 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T03:53:20.813 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:20.814 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T03:53:20.814 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T03:53:20.814 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:20.815 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:20.815 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T03:53:20.816 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T03:53:20.816 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:20.816 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T03:53:20.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 -- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f785c053270 msgr2=0x7f785c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f785c053270 0x7f785c055730 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7870002fb0 tx=0x7f78700069d0).stop 2021-07-11T03:53:20.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 -- 172.21.2.104:0/3289970075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f787c060100 msgr2=0x7f787c0622b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f787c060100 0x7f787c0622b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f7878002f70 tx=0x7f7878015000).stop 2021-07-11T03:53:20.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 -- 172.21.2.104:0/3289970075 shutdown_connections 2021-07-11T03:53:20.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f787c0b4c40 0x7f787c0af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f787c067300 0x7f787c0af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f785c053270 0x7f785c055730 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 --2- 172.21.2.104:0/3289970075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f787c060100 0x7f787c0622b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 -- 172.21.2.104:0/3289970075 >> 172.21.2.104:0/3289970075 conn(0x7f787c05cf30 msgr2=0x7f787c0b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:20.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 -- 172.21.2.104:0/3289970075 shutdown_connections 2021-07-11T03:53:20.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.808+0000 7f7883354700 1 -- 172.21.2.104:0/3289970075 wait complete. 2021-07-11T03:53:20.822 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T03:53:20.823 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T03:53:20.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 Processor -- start 2021-07-11T03:53:20.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 -- start start 2021-07-11T03:53:20.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb0580b05b0 0x7fb0580b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb058143920 con 0x7fb0580b3710 2021-07-11T03:53:20.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb058143bc0 con 0x7fb0580a9c10 2021-07-11T03:53:20.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb058143e60 con 0x7fb0580b05b0 2021-07-11T03:53:20.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb0567fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb0580b3710 0x7fb05813d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38352/0 (socket says 172.21.2.104:38352) 2021-07-11T03:53:20.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb0567fc700 1 -- 172.21.2.104:0/443413093 learned_addr learned my addr 172.21.2.104:0/443413093 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:20.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb056ffd700 1 --2- 172.21.2.104:0/443413093 >> v2:172.21.2.104:3300/0 conn(0x7fb0580b05b0 0x7fb0580b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb056ffd700 1 -- 172.21.2.104:0/443413093 >> v1:172.21.2.105:6789/0 conn(0x7fb0580a9c10 legacy=0x7fb0580b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:20.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb056ffd700 1 -- 172.21.2.104:0/443413093 >> v1:172.21.2.104:6789/0 conn(0x7fb0580b3710 legacy=0x7fb05813d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb056ffd700 1 -- 172.21.2.104:0/443413093 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058144100 con 0x7fb0580b05b0 2021-07-11T03:53:20.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb056ffd700 1 --2- 172.21.2.104:0/443413093 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0580b05b0 0x7fb0580b0990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb05806aa90 tx=0x7fb048006df0).ready entity=mon.0 client_cookie=6fcd135c4231a92a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb055ffb700 1 -- 172.21.2.104:0/443413093 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb04800d5c0 con 0x7fb0580b05b0 2021-07-11T03:53:20.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb055ffb700 1 -- 172.21.2.104:0/443413093 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb048000e90 con 0x7fb0580b05b0 2021-07-11T03:53:20.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb055ffb700 1 -- 172.21.2.104:0/443413093 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb04800d910 con 0x7fb0580b05b0 2021-07-11T03:53:20.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 -- 172.21.2.104:0/443413093 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0580b05b0 msgr2=0x7fb0580b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 --2- 172.21.2.104:0/443413093 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0580b05b0 0x7fb0580b0990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb05806aa90 tx=0x7fb048006df0).stop 2021-07-11T03:53:20.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 -- 172.21.2.104:0/443413093 shutdown_connections 2021-07-11T03:53:20.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 --2- 172.21.2.104:0/443413093 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0580b05b0 0x7fb0580b0990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:20.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 -- 172.21.2.104:0/443413093 >> 172.21.2.104:0/443413093 conn(0x7fb0580607c0 msgr2=0x7fb058060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:20.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 -- 172.21.2.104:0/443413093 shutdown_connections 2021-07-11T03:53:20.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.880+0000 7fb05d460700 1 -- 172.21.2.104:0/443413093 wait complete. 2021-07-11T03:53:20.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb05d460700 1 Processor -- start 2021-07-11T03:53:20.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb05d460700 1 -- start start 2021-07-11T03:53:20.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb05d460700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0580a9c10 0x7fb058147e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb05d460700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0580b05b0 0x7fb058063ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb05d460700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0580b3710 0x7fb058065ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb05d460700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb05813f120 con 0x7fb0580b05b0 2021-07-11T03:53:20.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb05d460700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb058064090 con 0x7fb0580a9c10 2021-07-11T03:53:20.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb05d460700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb05813df80 con 0x7fb0580b3710 2021-07-11T03:53:20.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb0567fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0580b05b0 0x7fb058063ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb0567fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0580b05b0 0x7fb058063ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41588/0 (socket says 172.21.2.104:41588) 2021-07-11T03:53:20.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb0567fc700 1 -- 172.21.2.104:0/528615802 learned_addr learned my addr 172.21.2.104:0/528615802 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:20.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb0567fc700 1 -- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0580b3710 msgr2=0x7fb058065ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb0567fc700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0580b3710 0x7fb058065ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb0567fc700 1 -- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0580a9c10 msgr2=0x7fb058147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:20.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb0567fc700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0580a9c10 0x7fb058147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:20.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb0567fc700 1 -- 172.21.2.104:0/528615802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04800b180 con 0x7fb0580b05b0 2021-07-11T03:53:20.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb0567fc700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0580b05b0 0x7fb058063ba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb058064220 tx=0x7fb040007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb04ffff700 1 -- 172.21.2.104:0/528615802 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb04000ce20 con 0x7fb0580b05b0 2021-07-11T03:53:20.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb04ffff700 1 -- 172.21.2.104:0/528615802 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0400072e0 con 0x7fb0580b05b0 2021-07-11T03:53:20.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb05d460700 1 -- 172.21.2.104:0/528615802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb05813ebe0 con 0x7fb0580b05b0 2021-07-11T03:53:20.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb05d460700 1 -- 172.21.2.104:0/528615802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0580afd70 con 0x7fb0580b05b0 2021-07-11T03:53:20.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb04ffff700 1 -- 172.21.2.104:0/528615802 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb040005800 con 0x7fb0580b05b0 2021-07-11T03:53:20.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb05d460700 1 -- 172.21.2.104:0/528615802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb030003410 con 0x7fb0580b05b0 2021-07-11T03:53:20.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb04ffff700 1 -- 172.21.2.104:0/528615802 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb040010070 con 0x7fb0580b05b0 2021-07-11T03:53:20.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb04ffff700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb03c0531a0 0x7fb03c055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:20.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb04ffff700 1 -- 172.21.2.104:0/528615802 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb04006a780 con 0x7fb0580b05b0 2021-07-11T03:53:20.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.884+0000 7fb056ffd700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb03c0531a0 0x7fb03c055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:20.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.888+0000 7fb056ffd700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb03c0531a0 0x7fb03c055660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb05806aa90 tx=0x7fb048012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:20.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.888+0000 7fb04ffff700 1 -- 172.21.2.104:0/528615802 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb040041df0 con 0x7fb0580b05b0 2021-07-11T03:53:20.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:20.968+0000 7fb05d460700 1 -- 172.21.2.104:0/528615802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb030003c40 con 0x7fb0580b05b0 2021-07-11T03:53:21.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb04ffff700 1 -- 172.21.2.104:0/528615802 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v19) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb040045410 con 0x7fb0580b05b0 2021-07-11T03:53:21.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 -- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb03c0531a0 msgr2=0x7fb03c055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:21.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb03c0531a0 0x7fb03c055660 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb05806aa90 tx=0x7fb048012000).stop 2021-07-11T03:53:21.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 -- 172.21.2.104:0/528615802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0580b05b0 msgr2=0x7fb058063ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:21.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0580b05b0 0x7fb058063ba0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fb058064220 tx=0x7fb040007c70).stop 2021-07-11T03:53:21.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 -- 172.21.2.104:0/528615802 shutdown_connections 2021-07-11T03:53:21.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0580b3710 0x7fb058065ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:21.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb03c0531a0 0x7fb03c055660 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:21.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0580b05b0 0x7fb058063ba0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:21.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 --2- 172.21.2.104:0/528615802 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0580a9c10 0x7fb058147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:21.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 -- 172.21.2.104:0/528615802 >> 172.21.2.104:0/528615802 conn(0x7fb0580607c0 msgr2=0x7fb0580ae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:21.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 -- 172.21.2.104:0/528615802 shutdown_connections 2021-07-11T03:53:21.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.164+0000 7fb05d460700 1 -- 172.21.2.104:0/528615802 wait complete. 2021-07-11T03:53:21.171 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T03:53:21.175 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T03:53:21.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cbb497700 1 Processor -- start 2021-07-11T03:53:21.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cbb497700 1 -- start start 2021-07-11T03:53:21.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cbb497700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f3cb40b3710 0x7f3cb40b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:21.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cbb497700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb4143920 con 0x7f3cb40a9c10 2021-07-11T03:53:21.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cbb497700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb4143bc0 con 0x7f3cb40b05b0 2021-07-11T03:53:21.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cbb497700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb4143e60 con 0x7f3cb40b3710 2021-07-11T03:53:21.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cb9a34700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f3cb40a9c10 0x7f3cb40b10b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38362/0 (socket says 172.21.2.104:38362) 2021-07-11T03:53:21.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cb9a34700 1 -- 172.21.2.104:0/3392661492 learned_addr learned my addr 172.21.2.104:0/3392661492 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:21.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482204313 0 0) 0x7f3cb4143920 con 0x7f3cb40a9c10 2021-07-11T03:53:21.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c9c003260 con 0x7f3cb40a9c10 2021-07-11T03:53:21.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cb8a32700 1 --2- 172.21.2.104:0/3392661492 >> v2:172.21.2.105:3300/0 conn(0x7f3cb40b3710 0x7f3cb40b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:21.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1898244237 0 0) 0x7f3c9c003260 con 0x7f3cb40a9c10 2021-07-11T03:53:21.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 >> v2:172.21.2.105:3300/0 conn(0x7f3cb40b3710 msgr2=0x7f3cb40b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:21.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 --2- 172.21.2.104:0/3392661492 >> v2:172.21.2.105:3300/0 conn(0x7f3cb40b3710 0x7f3cb40b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:21.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 >> v1:172.21.2.105:6789/0 conn(0x7f3cb40b05b0 legacy=0x7f3cb413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:21.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb4144100 con 0x7f3cb40a9c10 2021-07-11T03:53:21.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f3cac0019e0 con 0x7f3cb40a9c10 2021-07-11T03:53:21.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 >> v1:172.21.2.104:6789/0 conn(0x7f3cb40a9c10 legacy=0x7f3cb40b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:21.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 --2- 172.21.2.104:0/3392661492 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c9c003820 0x7f3c9c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:21.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 --2- 172.21.2.104:0/3392661492 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c9c004500 0x7f3c9c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:21.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 --2- 172.21.2.104:0/3392661492 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3c9c004f70 0x7f3c9c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:21.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4143e60 con 0x7f3c9c004500 2021-07-11T03:53:21.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c00bef0 con 0x7f3c9c003820 2021-07-11T03:53:21.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c00c0b0 con 0x7f3c9c004f70 2021-07-11T03:53:21.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cb8a32700 1 --2- 172.21.2.104:0/3392661492 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c9c004500 0x7f3c9c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:21.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cb8a32700 1 -- 172.21.2.104:0/3392661492 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3c9c004f70 msgr2=0x7f3c9c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:21.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cb8a32700 1 --2- 172.21.2.104:0/3392661492 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3c9c004f70 0x7f3c9c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:21.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cb8a32700 1 -- 172.21.2.104:0/3392661492 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c9c003820 msgr2=0x7f3c9c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:21.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cb8a32700 1 --2- 172.21.2.104:0/3392661492 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c9c003820 0x7f3c9c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:21.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cb8a32700 1 -- 172.21.2.104:0/3392661492 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c003260 con 0x7f3c9c004500 2021-07-11T03:53:21.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cb9a34700 1 -- 172.21.2.104:0/3392661492 reap_dead start 2021-07-11T03:53:21.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cb8a32700 1 --2- 172.21.2.104:0/3392661492 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c9c004500 0x7f3c9c004980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3ca400a000 tx=0x7f3ca4008040).ready entity=mon.0 client_cookie=9a7446f625f2249 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:21.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ca4013670 con 0x7f3c9c004500 2021-07-11T03:53:21.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca40137d0 con 0x7f3c9c004500 2021-07-11T03:53:21.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3ca3fff700 1 -- 172.21.2.104:0/3392661492 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ca4013b30 con 0x7f3c9c004500 2021-07-11T03:53:21.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cbb497700 1 -- 172.21.2.104:0/3392661492 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c9c004500 msgr2=0x7f3c9c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:21.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.236+0000 7f3cbb497700 1 --2- 172.21.2.104:0/3392661492 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c9c004500 0x7f3c9c004980 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3ca400a000 tx=0x7f3ca4008040).stop 2021-07-11T03:53:21.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 -- 172.21.2.104:0/3392661492 shutdown_connections 2021-07-11T03:53:21.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 --2- 172.21.2.104:0/3392661492 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c9c004500 0x7f3c9c004980 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:21.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 -- 172.21.2.104:0/3392661492 >> 172.21.2.104:0/3392661492 conn(0x7f3cb40607c0 msgr2=0x7f3cb4060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:21.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 -- 172.21.2.104:0/3392661492 shutdown_connections 2021-07-11T03:53:21.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 -- 172.21.2.104:0/3392661492 wait complete. 2021-07-11T03:53:21.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 Processor -- start 2021-07-11T03:53:21.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 -- start start 2021-07-11T03:53:21.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c9c004500 0x7f3cb40b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:21.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cb4063de0 0x7f3cb4067200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:21.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cb40b52a0 0x7f3cb40b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:21.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40ae4b0 con 0x7f3cb4063de0 2021-07-11T03:53:21.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb4065df0 con 0x7f3c9c004500 2021-07-11T03:53:21.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40638d0 con 0x7f3cb40b52a0 2021-07-11T03:53:21.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cb8a32700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cb4063de0 0x7f3cb4067200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:21.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cb8a32700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cb4063de0 0x7f3cb4067200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41610/0 (socket says 172.21.2.104:41610) 2021-07-11T03:53:21.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cb8a32700 1 -- 172.21.2.104:0/3161321181 learned_addr learned my addr 172.21.2.104:0/3161321181 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:21.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cb8a32700 1 -- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cb40b52a0 msgr2=0x7f3cb40b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:21.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cb8a32700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cb40b52a0 0x7f3cb40b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:21.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cb8a32700 1 -- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c9c004500 msgr2=0x7f3cb40b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:21.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cb8a32700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c9c004500 0x7f3cb40b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:21.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cb8a32700 1 -- 172.21.2.104:0/3161321181 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb4063b00 con 0x7f3cb4063de0 2021-07-11T03:53:21.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cb8a32700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cb4063de0 0x7f3cb4067200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f3cb40b6f50 tx=0x7f3ca4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:21.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3ca27fc700 1 -- 172.21.2.104:0/3161321181 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ca4021070 con 0x7f3cb4063de0 2021-07-11T03:53:21.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3ca27fc700 1 -- 172.21.2.104:0/3161321181 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca4012930 con 0x7f3cb4063de0 2021-07-11T03:53:21.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 -- 172.21.2.104:0/3161321181 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb40aa5c0 con 0x7f3cb4063de0 2021-07-11T03:53:21.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 -- 172.21.2.104:0/3161321181 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb40aa860 con 0x7f3cb4063de0 2021-07-11T03:53:21.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3ca27fc700 1 -- 172.21.2.104:0/3161321181 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ca4012c90 con 0x7f3cb4063de0 2021-07-11T03:53:21.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cbb497700 1 -- 172.21.2.104:0/3161321181 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb4057350 con 0x7f3cb4063de0 2021-07-11T03:53:21.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3ca27fc700 1 -- 172.21.2.104:0/3161321181 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3ca401d440 con 0x7f3cb4063de0 2021-07-11T03:53:21.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3ca27fc700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c940532d0 0x7f3c94055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:21.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3ca27fc700 1 -- 172.21.2.104:0/3161321181 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f3ca406c300 con 0x7f3cb4063de0 2021-07-11T03:53:21.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.240+0000 7f3cb9233700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c940532d0 0x7f3c94055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:21.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.244+0000 7f3cb9233700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c940532d0 0x7f3c94055790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3cb0001fa0 tx=0x7f3cb00077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:21.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.244+0000 7f3ca27fc700 1 -- 172.21.2.104:0/3161321181 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3ca40436a0 con 0x7f3cb4063de0 2021-07-11T03:53:21.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:21.328+0000 7f3cbb497700 1 -- 172.21.2.104:0/3161321181 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f3cb413e770 con 0x7f3cb4063de0 2021-07-11T03:53:22.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.168+0000 7f3ca27fc700 1 -- 172.21.2.104:0/3161321181 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v20) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3ca4046cc0 con 0x7f3cb4063de0 2021-07-11T03:53:22.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.168+0000 7f3cbb497700 1 -- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c940532d0 msgr2=0x7f3c94055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.168+0000 7f3cbb497700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c940532d0 0x7f3c94055790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3cb0001fa0 tx=0x7f3cb00077f0).stop 2021-07-11T03:53:22.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.168+0000 7f3cbb497700 1 -- 172.21.2.104:0/3161321181 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cb4063de0 msgr2=0x7f3cb4067200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.168+0000 7f3cbb497700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cb4063de0 0x7f3cb4067200 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f3cb40b6f50 tx=0x7f3ca4008040).stop 2021-07-11T03:53:22.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.168+0000 7f3cbb497700 1 -- 172.21.2.104:0/3161321181 shutdown_connections 2021-07-11T03:53:22.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.168+0000 7f3cbb497700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cb40b52a0 0x7f3cb40b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.168+0000 7f3cbb497700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c940532d0 0x7f3c94055790 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.168+0000 7f3cbb497700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cb4063de0 0x7f3cb4067200 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0x7f3cb40b6f50 tx=0x7f3ca4008040).stop 2021-07-11T03:53:22.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.168+0000 7f3cbb497700 1 --2- 172.21.2.104:0/3161321181 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c9c004500 0x7f3cb40b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.168+0000 7f3cbb497700 1 -- 172.21.2.104:0/3161321181 >> 172.21.2.104:0/3161321181 conn(0x7f3cb40607c0 msgr2=0x7f3cb40ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:22.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.172+0000 7f3cbb497700 1 -- 172.21.2.104:0/3161321181 shutdown_connections 2021-07-11T03:53:22.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.172+0000 7f3cbb497700 1 -- 172.21.2.104:0/3161321181 wait complete. 2021-07-11T03:53:22.176 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T03:53:22.181 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T03:53:22.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 Processor -- start 2021-07-11T03:53:22.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 -- start start 2021-07-11T03:53:22.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd094141b40 0x7fd09414cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0940b31d0 con 0x7fd0940b1170 2021-07-11T03:53:22.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0940b7bf0 con 0x7fd0940b7810 2021-07-11T03:53:22.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0940675c0 con 0x7fd094141b40 2021-07-11T03:53:22.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd098fe2700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd0940b1170 0x7fd094141430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38382/0 (socket says 172.21.2.104:38382) 2021-07-11T03:53:22.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd098fe2700 1 -- 172.21.2.104:0/4077144722 learned_addr learned my addr 172.21.2.104:0/4077144722 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:22.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd099fe4700 1 --2- 172.21.2.104:0/4077144722 >> v2:172.21.2.104:3300/0 conn(0x7fd094141b40 0x7fd09414cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd099fe4700 1 -- 172.21.2.104:0/4077144722 >> v1:172.21.2.105:6789/0 conn(0x7fd0940b7810 legacy=0x7fd0941445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:22.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd099fe4700 1 -- 172.21.2.104:0/4077144722 >> v1:172.21.2.104:6789/0 conn(0x7fd0940b1170 legacy=0x7fd094141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd099fe4700 1 -- 172.21.2.104:0/4077144722 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0940aae70 con 0x7fd094141b40 2021-07-11T03:53:22.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd099fe4700 1 --2- 172.21.2.104:0/4077144722 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd094141b40 0x7fd09414cb80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fd094060740 tx=0x7fd08c0091d0).ready entity=mon.0 client_cookie=2eedc6fc3d8f2c47 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd087fff700 1 -- 172.21.2.104:0/4077144722 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd08c009ea0 con 0x7fd094141b40 2021-07-11T03:53:22.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd087fff700 1 -- 172.21.2.104:0/4077144722 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c00b860 con 0x7fd094141b40 2021-07-11T03:53:22.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd087fff700 1 -- 172.21.2.104:0/4077144722 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd08c00b9d0 con 0x7fd094141b40 2021-07-11T03:53:22.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 -- 172.21.2.104:0/4077144722 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd094141b40 msgr2=0x7fd09414cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 --2- 172.21.2.104:0/4077144722 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd094141b40 0x7fd09414cb80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fd094060740 tx=0x7fd08c0091d0).stop 2021-07-11T03:53:22.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 -- 172.21.2.104:0/4077144722 shutdown_connections 2021-07-11T03:53:22.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 --2- 172.21.2.104:0/4077144722 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd094141b40 0x7fd09414cb80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 -- 172.21.2.104:0/4077144722 >> 172.21.2.104:0/4077144722 conn(0x7fd09405f6b0 msgr2=0x7fd09405fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:22.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 -- 172.21.2.104:0/4077144722 shutdown_connections 2021-07-11T03:53:22.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 -- 172.21.2.104:0/4077144722 wait complete. 2021-07-11T03:53:22.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 Processor -- start 2021-07-11T03:53:22.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.232+0000 7fd09ba47700 1 -- start start 2021-07-11T03:53:22.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd09ba47700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0940b1170 0x7fd09405cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd09ba47700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0940b7810 0x7fd0940612d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd09ba47700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd094141b40 0x7fd094156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd09ba47700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd094154450 con 0x7fd094141b40 2021-07-11T03:53:22.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd09ba47700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940a8c00 con 0x7fd0940b1170 2021-07-11T03:53:22.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd09ba47700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd094061940 con 0x7fd0940b7810 2021-07-11T03:53:22.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd099fe4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd094141b40 0x7fd094156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd099fe4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd094141b40 0x7fd094156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41620/0 (socket says 172.21.2.104:41620) 2021-07-11T03:53:22.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd099fe4700 1 -- 172.21.2.104:0/2224306038 learned_addr learned my addr 172.21.2.104:0/2224306038 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:22.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd099fe4700 1 -- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0940b7810 msgr2=0x7fd0940612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd099fe4700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0940b7810 0x7fd0940612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd099fe4700 1 -- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0940b1170 msgr2=0x7fd09405cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd099fe4700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0940b1170 0x7fd09405cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd099fe4700 1 -- 172.21.2.104:0/2224306038 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c009470 con 0x7fd094141b40 2021-07-11T03:53:22.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd099fe4700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd094141b40 0x7fd094156720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd094060740 tx=0x7fd08c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd0867fc700 1 -- 172.21.2.104:0/2224306038 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd08c021440 con 0x7fd094141b40 2021-07-11T03:53:22.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd0867fc700 1 -- 172.21.2.104:0/2224306038 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c0215a0 con 0x7fd094141b40 2021-07-11T03:53:22.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd09ba47700 1 -- 172.21.2.104:0/2224306038 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09415aa80 con 0x7fd094141b40 2021-07-11T03:53:22.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd09ba47700 1 -- 172.21.2.104:0/2224306038 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd094061e10 con 0x7fd094141b40 2021-07-11T03:53:22.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd0867fc700 1 -- 172.21.2.104:0/2224306038 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd08c021900 con 0x7fd094141b40 2021-07-11T03:53:22.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd09ba47700 1 -- 172.21.2.104:0/2224306038 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd094061030 con 0x7fd094141b40 2021-07-11T03:53:22.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd0867fc700 1 -- 172.21.2.104:0/2224306038 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd08c00f070 con 0x7fd094141b40 2021-07-11T03:53:22.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd0867fc700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd0740532d0 0x7fd074055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd0867fc700 1 -- 172.21.2.104:0/2224306038 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fd08c06c8a0 con 0x7fd094141b40 2021-07-11T03:53:22.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd0997e3700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd0740532d0 0x7fd074055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.236+0000 7fd0997e3700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd0740532d0 0x7fd074055790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fd090002f90 tx=0x7fd0900069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.240+0000 7fd0867fc700 1 -- 172.21.2.104:0/2224306038 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd094061030 con 0x7fd094141b40 2021-07-11T03:53:22.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.348+0000 7fd09ba47700 1 -- 172.21.2.104:0/2224306038 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fd094061030 con 0x7fd094141b40 2021-07-11T03:53:22.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.352+0000 7fd0867fc700 1 -- 172.21.2.104:0/2224306038 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 187+0+0 (secure 0 0 0) 0x7fd094061030 con 0x7fd094141b40 2021-07-11T03:53:22.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 -- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd0740532d0 msgr2=0x7fd074055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd0740532d0 0x7fd074055790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fd090002f90 tx=0x7fd0900069b0).stop 2021-07-11T03:53:22.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 -- 172.21.2.104:0/2224306038 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd094141b40 msgr2=0x7fd094156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd094141b40 0x7fd094156720 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd094060740 tx=0x7fd08c016000).stop 2021-07-11T03:53:22.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 -- 172.21.2.104:0/2224306038 shutdown_connections 2021-07-11T03:53:22.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd0740532d0 0x7fd074055790 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd094141b40 0x7fd094156720 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0940b7810 0x7fd0940612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 --2- 172.21.2.104:0/2224306038 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0940b1170 0x7fd09405cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 -- 172.21.2.104:0/2224306038 >> 172.21.2.104:0/2224306038 conn(0x7fd09405f6b0 msgr2=0x7fd0940b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:22.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 -- 172.21.2.104:0/2224306038 shutdown_connections 2021-07-11T03:53:22.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.356+0000 7fd09ba47700 1 -- 172.21.2.104:0/2224306038 wait complete. 2021-07-11T03:53:22.363 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 3 and data pool 4 2021-07-11T03:53:22.367 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:22.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 Processor -- start 2021-07-11T03:53:22.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- start start 2021-07-11T03:53:22.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f186c0b7810 0x7f186c1445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f186c144a90 0x7f186c064f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186c0b31d0 con 0x7f186c0b1170 2021-07-11T03:53:22.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c144f20 con 0x7f186c0b7810 2021-07-11T03:53:22.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c0b7bf0 con 0x7f186c144a90 2021-07-11T03:53:22.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f186c0b7810 0x7f186c1445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f186c0b7810 0x7f186c1445a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41638/0 (socket says 172.21.2.104:41638) 2021-07-11T03:53:22.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 -- 172.21.2.104:0/2904422033 learned_addr learned my addr 172.21.2.104:0/2904422033 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:22.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 -- 172.21.2.104:0/2904422033 >> v2:172.21.2.105:3300/0 conn(0x7f186c144a90 msgr2=0x7f186c064f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:22.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 --2- 172.21.2.104:0/2904422033 >> v2:172.21.2.105:3300/0 conn(0x7f186c144a90 0x7f186c064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 -- 172.21.2.104:0/2904422033 >> v1:172.21.2.105:6789/0 conn(0x7f186c0b1170 legacy=0x7f186c064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:22.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 -- 172.21.2.104:0/2904422033 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c0675c0 con 0x7f186c0b7810 2021-07-11T03:53:22.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 --2- 172.21.2.104:0/2904422033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0b7810 0x7f186c1445a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f186c067ca0 tx=0x7f185c006eb0).ready entity=mon.0 client_cookie=c9969137d1d8bfcf server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186a7fc700 1 -- 172.21.2.104:0/2904422033 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f185c00d5c0 con 0x7f186c0b7810 2021-07-11T03:53:22.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186a7fc700 1 -- 172.21.2.104:0/2904422033 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c000e90 con 0x7f186c0b7810 2021-07-11T03:53:22.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186a7fc700 1 -- 172.21.2.104:0/2904422033 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f185c00d910 con 0x7f186c0b7810 2021-07-11T03:53:22.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- 172.21.2.104:0/2904422033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0b7810 msgr2=0x7f186c1445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 --2- 172.21.2.104:0/2904422033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0b7810 0x7f186c1445a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f186c067ca0 tx=0x7f185c006eb0).stop 2021-07-11T03:53:22.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- 172.21.2.104:0/2904422033 shutdown_connections 2021-07-11T03:53:22.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 --2- 172.21.2.104:0/2904422033 >> v2:172.21.2.105:3300/0 conn(0x7f186c144a90 0x7f186c064f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 --2- 172.21.2.104:0/2904422033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0b7810 0x7f186c1445a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- 172.21.2.104:0/2904422033 >> 172.21.2.104:0/2904422033 conn(0x7f186c05f6b0 msgr2=0x7f186c05fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:22.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- 172.21.2.104:0/2904422033 shutdown_connections 2021-07-11T03:53:22.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- 172.21.2.104:0/2904422033 wait complete. 2021-07-11T03:53:22.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 Processor -- start 2021-07-11T03:53:22.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- start start 2021-07-11T03:53:22.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0b1170 0x7f186c09ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f186c0b7810 0x7f186c152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c144a90 0x7f186c0607c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1415f0 con 0x7f186c0b1170 2021-07-11T03:53:22.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1493d0 con 0x7f186c0b7810 2021-07-11T03:53:22.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f1871eee700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c152800 con 0x7f186c144a90 2021-07-11T03:53:22.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0b1170 0x7f186c09ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0b1170 0x7f186c09ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41644/0 (socket says 172.21.2.104:41644) 2021-07-11T03:53:22.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 -- 172.21.2.104:0/1554047778 learned_addr learned my addr 172.21.2.104:0/1554047778 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:22.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 -- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c144a90 msgr2=0x7f186c0607c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c144a90 0x7f186c0607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 -- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f186c0b7810 msgr2=0x7f186c152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f186c0b7810 0x7f186c152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 -- 172.21.2.104:0/1554047778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c00b1e0 con 0x7f186c0b1170 2021-07-11T03:53:22.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.416+0000 7f186b7fe700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0b1170 0x7f186c09ebf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f186c067ca0 tx=0x7f185c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f1868ff9700 1 -- 172.21.2.104:0/1554047778 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f185c021070 con 0x7f186c0b1170 2021-07-11T03:53:22.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f1868ff9700 1 -- 172.21.2.104:0/1554047778 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c000e90 con 0x7f186c0b1170 2021-07-11T03:53:22.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f1871eee700 1 -- 172.21.2.104:0/1554047778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186c152aa0 con 0x7f186c0b1170 2021-07-11T03:53:22.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f1868ff9700 1 -- 172.21.2.104:0/1554047778 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f185c006be0 con 0x7f186c0b1170 2021-07-11T03:53:22.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f1871eee700 1 -- 172.21.2.104:0/1554047778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c1476b0 con 0x7f186c0b1170 2021-07-11T03:53:22.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f1871eee700 1 -- 172.21.2.104:0/1554047778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186c060e50 con 0x7f186c0b1170 2021-07-11T03:53:22.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f1868ff9700 1 -- 172.21.2.104:0/1554047778 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f185c00f070 con 0x7f186c0b1170 2021-07-11T03:53:22.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f1868ff9700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1850053210 0x7f18500556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f1868ff9700 1 -- 172.21.2.104:0/1554047778 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f185c06c850 con 0x7f186c0b1170 2021-07-11T03:53:22.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f186affd700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1850053210 0x7f18500556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f186affd700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1850053210 0x7f18500556d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1854002f90 tx=0x7f18540069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.420+0000 7f1868ff9700 1 -- 172.21.2.104:0/1554047778 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f185c043c40 con 0x7f186c0b1170 2021-07-11T03:53:22.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.504+0000 7f1871eee700 1 -- 172.21.2.104:0/1554047778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f186c0b3b20 con 0x7f186c0b1170 2021-07-11T03:53:22.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.504+0000 7f1868ff9700 1 -- 172.21.2.104:0/1554047778 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16051 (secure 0 0 0) 0x7f185c002f50 con 0x7f186c0b1170 2021-07-11T03:53:22.510 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:22.510 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":21,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:22.350809+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:22.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 -- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1850053210 msgr2=0x7f18500556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1850053210 0x7f18500556d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1854002f90 tx=0x7f18540069b0).stop 2021-07-11T03:53:22.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 -- 172.21.2.104:0/1554047778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0b1170 msgr2=0x7f186c09ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0b1170 0x7f186c09ebf0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f186c067ca0 tx=0x7f185c015000).stop 2021-07-11T03:53:22.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 -- 172.21.2.104:0/1554047778 shutdown_connections 2021-07-11T03:53:22.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c144a90 0x7f186c0607c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f186c0b7810 0x7f186c152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1850053210 0x7f18500556d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 --2- 172.21.2.104:0/1554047778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0b1170 0x7f186c09ebf0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 -- 172.21.2.104:0/1554047778 >> 172.21.2.104:0/1554047778 conn(0x7f186c05f6b0 msgr2=0x7f186c05c490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:22.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 -- 172.21.2.104:0/1554047778 shutdown_connections 2021-07-11T03:53:22.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.508+0000 7f1871eee700 1 -- 172.21.2.104:0/1554047778 wait complete. 2021-07-11T03:53:22.517 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:22.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.564+0000 7fcc0f458700 1 Processor -- start 2021-07-11T03:53:22.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.564+0000 7fcc0f458700 1 -- start start 2021-07-11T03:53:22.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.564+0000 7fcc0f458700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcc080a76b0 0x7fcc08066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.564+0000 7fcc0f458700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcc080b0060 0x7fcc0805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.564+0000 7fcc0f458700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc080a7a90 con 0x7fcc080b1970 2021-07-11T03:53:22.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.564+0000 7fcc0f458700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc08147620 con 0x7fcc080a76b0 2021-07-11T03:53:22.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.564+0000 7fcc0f458700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc081478c0 con 0x7fcc080b0060 2021-07-11T03:53:22.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.564+0000 7fcc0d1f4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcc080a76b0 0x7fcc08066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.564+0000 7fcc0d1f4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcc080a76b0 0x7fcc08066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41654/0 (socket says 172.21.2.104:41654) 2021-07-11T03:53:22.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.564+0000 7fcc0d1f4700 1 -- 172.21.2.104:0/926481181 learned_addr learned my addr 172.21.2.104:0/926481181 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:22.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 -- 172.21.2.104:0/926481181 >> v2:172.21.2.105:3300/0 conn(0x7fcc080b0060 msgr2=0x7fcc0805fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:22.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 --2- 172.21.2.104:0/926481181 >> v2:172.21.2.105:3300/0 conn(0x7fcc080b0060 0x7fcc0805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 -- 172.21.2.104:0/926481181 >> v1:172.21.2.105:6789/0 conn(0x7fcc080b1970 legacy=0x7fcc0805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:22.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 -- 172.21.2.104:0/926481181 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc08147b60 con 0x7fcc080a76b0 2021-07-11T03:53:22.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 --2- 172.21.2.104:0/926481181 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc080a76b0 0x7fcc08066a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fcc0806a0e0 tx=0x7fcc04006eb0).ready entity=mon.0 client_cookie=5c113269e9c635ad server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcbf7fff700 1 -- 172.21.2.104:0/926481181 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc0400d5c0 con 0x7fcc080a76b0 2021-07-11T03:53:22.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcbf7fff700 1 -- 172.21.2.104:0/926481181 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc04000e90 con 0x7fcc080a76b0 2021-07-11T03:53:22.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcbf7fff700 1 -- 172.21.2.104:0/926481181 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc0400d910 con 0x7fcc080a76b0 2021-07-11T03:53:22.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- 172.21.2.104:0/926481181 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc080a76b0 msgr2=0x7fcc08066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 --2- 172.21.2.104:0/926481181 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc080a76b0 0x7fcc08066a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fcc0806a0e0 tx=0x7fcc04006eb0).stop 2021-07-11T03:53:22.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- 172.21.2.104:0/926481181 shutdown_connections 2021-07-11T03:53:22.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 --2- 172.21.2.104:0/926481181 >> v2:172.21.2.105:3300/0 conn(0x7fcc080b0060 0x7fcc0805fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 --2- 172.21.2.104:0/926481181 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc080a76b0 0x7fcc08066a70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- 172.21.2.104:0/926481181 >> 172.21.2.104:0/926481181 conn(0x7fcc080b53e0 msgr2=0x7fcc080b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:22.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- 172.21.2.104:0/926481181 shutdown_connections 2021-07-11T03:53:22.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- 172.21.2.104:0/926481181 wait complete. 2021-07-11T03:53:22.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 Processor -- start 2021-07-11T03:53:22.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- start start 2021-07-11T03:53:22.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc080a76b0 0x7fcc080ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc080b0060 0x7fcc08159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc080b1970 0x7fcc0815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0815a5f0 con 0x7fcc080a76b0 2021-07-11T03:53:22.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081567e0 con 0x7fcc080b1970 2021-07-11T03:53:22.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08156a80 con 0x7fcc080b0060 2021-07-11T03:53:22.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc080a76b0 0x7fcc080ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc080a76b0 0x7fcc080ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41656/0 (socket says 172.21.2.104:41656) 2021-07-11T03:53:22.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 -- 172.21.2.104:0/2121846022 learned_addr learned my addr 172.21.2.104:0/2121846022 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:22.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 -- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc080b0060 msgr2=0x7fcc08159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc080b0060 0x7fcc08159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 -- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc080b1970 msgr2=0x7fcc0815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc080b1970 0x7fcc0815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 -- 172.21.2.104:0/2121846022 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0400b1e0 con 0x7fcc080a76b0 2021-07-11T03:53:22.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0d1f4700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc080a76b0 0x7fcc080ad060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fcc0806a0e0 tx=0x7fcc04015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcbf67fc700 1 -- 172.21.2.104:0/2121846022 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc04006be0 con 0x7fcc080a76b0 2021-07-11T03:53:22.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcbf67fc700 1 -- 172.21.2.104:0/2121846022 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc04011ce0 con 0x7fcc080a76b0 2021-07-11T03:53:22.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- 172.21.2.104:0/2121846022 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc08156d20 con 0x7fcc080a76b0 2021-07-11T03:53:22.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcbf67fc700 1 -- 172.21.2.104:0/2121846022 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc0401d640 con 0x7fcc080a76b0 2021-07-11T03:53:22.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- 172.21.2.104:0/2121846022 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc08157460 con 0x7fcc080a76b0 2021-07-11T03:53:22.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.568+0000 7fcc0f458700 1 -- 172.21.2.104:0/2121846022 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc08057350 con 0x7fcc080a76b0 2021-07-11T03:53:22.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.572+0000 7fcbf67fc700 1 -- 172.21.2.104:0/2121846022 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcc0400f070 con 0x7fcc080a76b0 2021-07-11T03:53:22.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.572+0000 7fcbf67fc700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcbe8053350 0x7fcbe8055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.572+0000 7fcbf67fc700 1 -- 172.21.2.104:0/2121846022 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcc0406ced0 con 0x7fcc080a76b0 2021-07-11T03:53:22.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.572+0000 7fcc0c9f3700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcbe8053350 0x7fcbe8055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.572+0000 7fcc0c9f3700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcbe8053350 0x7fcbe8055810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fcbf8002f10 tx=0x7fcbf8006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.572+0000 7fcbf67fc700 1 -- 172.21.2.104:0/2121846022 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcc040441b0 con 0x7fcc080a76b0 2021-07-11T03:53:22.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 -- 172.21.2.104:0/2121846022 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc080698e0 con 0x7fcc080a76b0 2021-07-11T03:53:22.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcbf67fc700 1 -- 172.21.2.104:0/2121846022 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16051 (secure 0 0 0) 0x7fcc040477d0 con 0x7fcc080a76b0 2021-07-11T03:53:22.660 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:22.660 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":21,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:22.350809+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:22.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 -- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcbe8053350 msgr2=0x7fcbe8055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcbe8053350 0x7fcbe8055810 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fcbf8002f10 tx=0x7fcbf8006930).stop 2021-07-11T03:53:22.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 -- 172.21.2.104:0/2121846022 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc080a76b0 msgr2=0x7fcc080ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc080a76b0 0x7fcc080ad060 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fcc0806a0e0 tx=0x7fcc04015000).stop 2021-07-11T03:53:22.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 -- 172.21.2.104:0/2121846022 shutdown_connections 2021-07-11T03:53:22.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc080b1970 0x7fcc0815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc080b0060 0x7fcc08159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcbe8053350 0x7fcbe8055810 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 --2- 172.21.2.104:0/2121846022 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc080a76b0 0x7fcc080ad060 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 -- 172.21.2.104:0/2121846022 >> 172.21.2.104:0/2121846022 conn(0x7fcc080b53e0 msgr2=0x7fcc08066820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:22.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 -- 172.21.2.104:0/2121846022 shutdown_connections 2021-07-11T03:53:22.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.656+0000 7fcc0f458700 1 -- 172.21.2.104:0/2121846022 wait complete. 2021-07-11T03:53:22.668 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T03:53:22.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0ee4990700 1 Processor -- start 2021-07-11T03:53:22.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0ee4990700 1 -- start start 2021-07-11T03:53:22.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0ee4990700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ee00a9ec0 0x7f0ee00aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0ee4990700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0ee00603d0 0x7f0ee013ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0ee4990700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee0061990 con 0x7f0ee013d1c0 2021-07-11T03:53:22.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0ee4990700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee0061c30 con 0x7f0ee00603d0 2021-07-11T03:53:22.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0ee4990700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee0060b90 con 0x7f0ee00a9ec0 2021-07-11T03:53:22.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0eded9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f0ee013d1c0 0x7f0ee013f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38434/0 (socket says 172.21.2.104:38434) 2021-07-11T03:53:22.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0eded9d700 1 -- 172.21.2.104:0/1100150375 learned_addr learned my addr 172.21.2.104:0/1100150375 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:22.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687801793 0 0) 0x7f0ee0061990 con 0x7f0ee013d1c0 2021-07-11T03:53:22.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ec0003260 con 0x7f0ee013d1c0 2021-07-11T03:53:22.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0eddd9b700 1 --2- 172.21.2.104:0/1100150375 >> v2:172.21.2.105:3300/0 conn(0x7f0ee00603d0 0x7f0ee013ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0ede59c700 1 --2- 172.21.2.104:0/1100150375 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ee00a9ec0 0x7f0ee00aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3531991647 0 0) 0x7f0ec0003260 con 0x7f0ee013d1c0 2021-07-11T03:53:22.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ee00a9ec0 msgr2=0x7f0ee00aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:22.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 --2- 172.21.2.104:0/1100150375 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ee00a9ec0 0x7f0ee00aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 >> v2:172.21.2.105:3300/0 conn(0x7f0ee00603d0 msgr2=0x7f0ee013ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:22.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 --2- 172.21.2.104:0/1100150375 >> v2:172.21.2.105:3300/0 conn(0x7f0ee00603d0 0x7f0ee013ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:22.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0060e30 con 0x7f0ee013d1c0 2021-07-11T03:53:22.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f0ed80019e0 con 0x7f0ee013d1c0 2021-07-11T03:53:22.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 >> v1:172.21.2.104:6789/0 conn(0x7f0ee013d1c0 legacy=0x7f0ee013f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 --2- 172.21.2.104:0/1100150375 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ec00037d0 0x7f0ec0003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 --2- 172.21.2.104:0/1100150375 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ee00a9ec0 0x7f0ec0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 --2- 172.21.2.104:0/1100150375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ec00049f0 0x7f0ec0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee0061c30 con 0x7f0ec00049f0 2021-07-11T03:53:22.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee0060b90 con 0x7f0ec00037d0 2021-07-11T03:53:22.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec000fd60 con 0x7f0ee00a9ec0 2021-07-11T03:53:22.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.716+0000 7f0eded9d700 1 --2- 172.21.2.104:0/1100150375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ec00049f0 0x7f0ec0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 -- 172.21.2.104:0/1100150375 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ee00a9ec0 msgr2=0x7f0ec0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:22.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 --2- 172.21.2.104:0/1100150375 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ee00a9ec0 0x7f0ec0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 -- 172.21.2.104:0/1100150375 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ec00037d0 msgr2=0x7f0ec0003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:22.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 --2- 172.21.2.104:0/1100150375 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ec00037d0 0x7f0ec0003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 -- 172.21.2.104:0/1100150375 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec0003260 con 0x7f0ec00049f0 2021-07-11T03:53:22.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 --2- 172.21.2.104:0/1100150375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ec00049f0 0x7f0ec0004e70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0ee005f870 tx=0x7f0ee0060fc0).ready entity=mon.0 client_cookie=f5af5b47c06b514 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ed800cb70 con 0x7f0ec00049f0 2021-07-11T03:53:22.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed800ccd0 con 0x7f0ec00049f0 2021-07-11T03:53:22.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0edd59a700 1 -- 172.21.2.104:0/1100150375 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ed8005b80 con 0x7f0ec00049f0 2021-07-11T03:53:22.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- 172.21.2.104:0/1100150375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ec00049f0 msgr2=0x7f0ec0004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 --2- 172.21.2.104:0/1100150375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ec00049f0 0x7f0ec0004e70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0ee005f870 tx=0x7f0ee0060fc0).stop 2021-07-11T03:53:22.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 -- 172.21.2.104:0/1100150375 reap_dead start 2021-07-11T03:53:22.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- 172.21.2.104:0/1100150375 shutdown_connections 2021-07-11T03:53:22.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- 172.21.2.104:0/1100150375 >> 172.21.2.104:0/1100150375 conn(0x7f0ee00b2970 msgr2=0x7f0ee00b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:22.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- 172.21.2.104:0/1100150375 shutdown_connections 2021-07-11T03:53:22.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- 172.21.2.104:0/1100150375 wait complete. 2021-07-11T03:53:22.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 Processor -- start 2021-07-11T03:53:22.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- start start 2021-07-11T03:53:22.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ee01377b0 0x7f0ee013fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ee00a9990 0x7f0ee00a7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ee00a8e80 0x7f0ee0142120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee013ea60 con 0x7f0ee00a8e80 2021-07-11T03:53:22.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee013a910 con 0x7f0ee01377b0 2021-07-11T03:53:22.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee013abb0 con 0x7f0ee00a9990 2021-07-11T03:53:22.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ee00a8e80 0x7f0ee0142120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ee00a8e80 0x7f0ee0142120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41680/0 (socket says 172.21.2.104:41680) 2021-07-11T03:53:22.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 -- 172.21.2.104:0/2214548913 learned_addr learned my addr 172.21.2.104:0/2214548913 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:22.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 -- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ee00a9990 msgr2=0x7f0ee00a7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ee00a9990 0x7f0ee00a7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 -- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ee01377b0 msgr2=0x7f0ee013fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:22.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ee01377b0 0x7f0ee013fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:22.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 -- 172.21.2.104:0/2214548913 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee013f340 con 0x7f0ee00a8e80 2021-07-11T03:53:22.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0eded9d700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ee00a8e80 0x7f0ee0142120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f0ee00abbd0 tx=0x7f0ed8013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ed77fe700 1 -- 172.21.2.104:0/2214548913 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ed8004d10 con 0x7f0ee00a8e80 2021-07-11T03:53:22.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ed77fe700 1 -- 172.21.2.104:0/2214548913 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed8018070 con 0x7f0ee00a8e80 2021-07-11T03:53:22.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ed77fe700 1 -- 172.21.2.104:0/2214548913 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ed80072c0 con 0x7f0ee00a8e80 2021-07-11T03:53:22.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- 172.21.2.104:0/2214548913 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee013f5e0 con 0x7f0ee00a8e80 2021-07-11T03:53:22.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- 172.21.2.104:0/2214548913 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee013a4c0 con 0x7f0ee00a8e80 2021-07-11T03:53:22.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.720+0000 7f0ee4990700 1 -- 172.21.2.104:0/2214548913 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee0057350 con 0x7f0ee00a8e80 2021-07-11T03:53:22.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.724+0000 7f0ed77fe700 1 -- 172.21.2.104:0/2214548913 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0ed8026070 con 0x7f0ee00a8e80 2021-07-11T03:53:22.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.724+0000 7f0ed77fe700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ec4053300 0x7f0ec40557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:22.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.724+0000 7f0ed77fe700 1 -- 172.21.2.104:0/2214548913 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0ed806ba70 con 0x7f0ee00a8e80 2021-07-11T03:53:22.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.724+0000 7f0ede59c700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ec4053300 0x7f0ec40557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:22.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.724+0000 7f0ede59c700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ec4053300 0x7f0ec40557c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0ed000b000 tx=0x7f0ed0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:22.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.724+0000 7f0ed77fe700 1 -- 172.21.2.104:0/2214548913 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0ed8042db0 con 0x7f0ee00a8e80 2021-07-11T03:53:22.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:22.832+0000 7f0ee4990700 1 -- 172.21.2.104:0/2214548913 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0ee00a8c60 con 0x7f0ee00a8e80 2021-07-11T03:53:23.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.372+0000 7f0ed77fe700 1 -- 172.21.2.104:0/2214548913 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0ed806be60 con 0x7f0ee00a8e80 2021-07-11T03:53:23.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 -- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ec4053300 msgr2=0x7f0ec40557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ec4053300 0x7f0ec40557c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0ed000b000 tx=0x7f0ed0008040).stop 2021-07-11T03:53:23.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 -- 172.21.2.104:0/2214548913 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ee00a8e80 msgr2=0x7f0ee0142120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ee00a8e80 0x7f0ee0142120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f0ee00abbd0 tx=0x7f0ed8013000).stop 2021-07-11T03:53:23.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 -- 172.21.2.104:0/2214548913 shutdown_connections 2021-07-11T03:53:23.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ec4053300 0x7f0ec40557c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:23.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ee00a8e80 0x7f0ee0142120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:23.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ee00a9990 0x7f0ee00a7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 --2- 172.21.2.104:0/2214548913 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ee01377b0 0x7f0ee013fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 -- 172.21.2.104:0/2214548913 >> 172.21.2.104:0/2214548913 conn(0x7f0ee00b2970 msgr2=0x7f0ee013b450 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:23.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 -- 172.21.2.104:0/2214548913 shutdown_connections 2021-07-11T03:53:23.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.376+0000 7f0ee4990700 1 -- 172.21.2.104:0/2214548913 wait complete. 2021-07-11T03:53:23.407 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:53:23.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.456+0000 7f83d2c80700 1 Processor -- start 2021-07-11T03:53:23.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.456+0000 7f83d2c80700 1 -- start start 2021-07-11T03:53:23.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.456+0000 7f83d2c80700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83cc0a76b0 0x7f83cc066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.456+0000 7f83d2c80700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f83cc0b1970 0x7f83cc066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.456+0000 7f83d2c80700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83cc067450 con 0x7f83cc0b0060 2021-07-11T03:53:23.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.456+0000 7f83d2c80700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83cc0a7a90 con 0x7f83cc0b1970 2021-07-11T03:53:23.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.456+0000 7f83d2c80700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc147620 con 0x7f83cc0a76b0 2021-07-11T03:53:23.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83cbfff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f83cc0b1970 0x7f83cc066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83cbfff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f83cc0b1970 0x7f83cc066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41688/0 (socket says 172.21.2.104:41688) 2021-07-11T03:53:23.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83cbfff700 1 -- 172.21.2.104:0/3489434950 learned_addr learned my addr 172.21.2.104:0/3489434950 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:23.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83cbfff700 1 -- 172.21.2.104:0/3489434950 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83cc0a76b0 msgr2=0x7f83cc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83cbfff700 1 --2- 172.21.2.104:0/3489434950 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83cc0a76b0 0x7f83cc066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83cbfff700 1 -- 172.21.2.104:0/3489434950 >> v1:172.21.2.105:6789/0 conn(0x7f83cc0b0060 legacy=0x7f83cc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83cbfff700 1 -- 172.21.2.104:0/3489434950 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc1478c0 con 0x7f83cc0b1970 2021-07-11T03:53:23.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83cbfff700 1 --2- 172.21.2.104:0/3489434950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83cc0b1970 0x7f83cc066f60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f83cc0b6460 tx=0x7f83b4006eb0).ready entity=mon.0 client_cookie=c936837d0f0e5804 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83cb7fe700 1 -- 172.21.2.104:0/3489434950 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b400d570 con 0x7f83cc0b1970 2021-07-11T03:53:23.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83cb7fe700 1 -- 172.21.2.104:0/3489434950 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b4000e90 con 0x7f83cc0b1970 2021-07-11T03:53:23.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83cb7fe700 1 -- 172.21.2.104:0/3489434950 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83b400d8c0 con 0x7f83cc0b1970 2021-07-11T03:53:23.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 -- 172.21.2.104:0/3489434950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83cc0b1970 msgr2=0x7f83cc066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 --2- 172.21.2.104:0/3489434950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83cc0b1970 0x7f83cc066f60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f83cc0b6460 tx=0x7f83b4006eb0).stop 2021-07-11T03:53:23.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 -- 172.21.2.104:0/3489434950 shutdown_connections 2021-07-11T03:53:23.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 --2- 172.21.2.104:0/3489434950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83cc0b1970 0x7f83cc066f60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:23.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 --2- 172.21.2.104:0/3489434950 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83cc0a76b0 0x7f83cc066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 -- 172.21.2.104:0/3489434950 >> 172.21.2.104:0/3489434950 conn(0x7f83cc0b53e0 msgr2=0x7f83cc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:23.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 -- 172.21.2.104:0/3489434950 shutdown_connections 2021-07-11T03:53:23.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 -- 172.21.2.104:0/3489434950 wait complete. 2021-07-11T03:53:23.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 Processor -- start 2021-07-11T03:53:23.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 -- start start 2021-07-11T03:53:23.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83cc0a76b0 0x7f83cc0abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83cc0b0060 0x7f83cc0b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83cc0b1970 0x7f83cc0ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0ab070 con 0x7f83cc0b1970 2021-07-11T03:53:23.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0aa8e0 con 0x7f83cc0b0060 2021-07-11T03:53:23.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0ab2a0 con 0x7f83cc0a76b0 2021-07-11T03:53:23.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d121d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83cc0b1970 0x7f83cc0ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d121d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83cc0b1970 0x7f83cc0ab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41694/0 (socket says 172.21.2.104:41694) 2021-07-11T03:53:23.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d121d700 1 -- 172.21.2.104:0/3545363164 learned_addr learned my addr 172.21.2.104:0/3545363164 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:23.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d121d700 1 -- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83cc0a76b0 msgr2=0x7f83cc0abf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d121d700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83cc0a76b0 0x7f83cc0abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d121d700 1 -- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83cc0b0060 msgr2=0x7f83cc0b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d121d700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83cc0b0060 0x7f83cc0b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d121d700 1 -- 172.21.2.104:0/3545363164 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b400b1e0 con 0x7f83cc0b1970 2021-07-11T03:53:23.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d121d700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83cc0b1970 0x7f83cc0ab740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f83c40012f0 tx=0x7f83c40013c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83c9ffb700 1 -- 172.21.2.104:0/3545363164 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83c400e4a0 con 0x7f83cc0b1970 2021-07-11T03:53:23.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83c9ffb700 1 -- 172.21.2.104:0/3545363164 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c400e600 con 0x7f83cc0b1970 2021-07-11T03:53:23.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 -- 172.21.2.104:0/3545363164 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83cc0abc30 con 0x7f83cc0b1970 2021-07-11T03:53:23.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83c9ffb700 1 -- 172.21.2.104:0/3545363164 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83c400e960 con 0x7f83cc0b1970 2021-07-11T03:53:23.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.460+0000 7f83d2c80700 1 -- 172.21.2.104:0/3545363164 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83cc0aa560 con 0x7f83cc0b1970 2021-07-11T03:53:23.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.464+0000 7f83d2c80700 1 -- 172.21.2.104:0/3545363164 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83cc0abdc0 con 0x7f83cc0b1970 2021-07-11T03:53:23.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.464+0000 7f83c9ffb700 1 -- 172.21.2.104:0/3545363164 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f83c4010030 con 0x7f83cc0b1970 2021-07-11T03:53:23.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.464+0000 7f83c9ffb700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83b0053160 0x7f83b0055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.464+0000 7f83c9ffb700 1 -- 172.21.2.104:0/3545363164 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f83c406b780 con 0x7f83cc0b1970 2021-07-11T03:53:23.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.464+0000 7f83d0a1c700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83b0053160 0x7f83b0055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.464+0000 7f83d0a1c700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83b0053160 0x7f83b0055620 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f83bc002f70 tx=0x7f83bc006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.464+0000 7f83c9ffb700 1 -- 172.21.2.104:0/3545363164 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f83c4042ac0 con 0x7f83cc0b1970 2021-07-11T03:53:23.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 -- 172.21.2.104:0/3545363164 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83cc0ab460 con 0x7f83cc0b1970 2021-07-11T03:53:23.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83c9ffb700 1 -- 172.21.2.104:0/3545363164 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+2134 (secure 0 0 0) 0x7f83c40460e0 con 0x7f83cc0b1970 2021-07-11T03:53:23.576 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:23.576 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":5,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3943971217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3943971217},{"type":"v1","addr":"172.21.2.105:6835","nonce":3943971217}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:22.350923+0000","modified":"2021-07-11T03:53:23.355891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4224":{"gid":4224,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/1236541958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":1236541958},{"type":"v1","addr":"172.21.2.104:6835","nonce":1236541958}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-07-11T03:53:23.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 -- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83b0053160 msgr2=0x7f83b0055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83b0053160 0x7f83b0055620 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f83bc002f70 tx=0x7f83bc006a80).stop 2021-07-11T03:53:23.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 -- 172.21.2.104:0/3545363164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83cc0b1970 msgr2=0x7f83cc0ab740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83cc0b1970 0x7f83cc0ab740 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f83c40012f0 tx=0x7f83c40013c0).stop 2021-07-11T03:53:23.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 -- 172.21.2.104:0/3545363164 shutdown_connections 2021-07-11T03:53:23.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83b0053160 0x7f83b0055620 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:23.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83cc0b1970 0x7f83cc0ab740 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:23.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83cc0b0060 0x7f83cc0b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 --2- 172.21.2.104:0/3545363164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83cc0a76b0 0x7f83cc0abf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 -- 172.21.2.104:0/3545363164 >> 172.21.2.104:0/3545363164 conn(0x7f83cc0b53e0 msgr2=0x7f83cc05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:23.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 -- 172.21.2.104:0/3545363164 shutdown_connections 2021-07-11T03:53:23.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.572+0000 7f83d2c80700 1 -- 172.21.2.104:0/3545363164 wait complete. 2021-07-11T03:53:23.580 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 5 2021-07-11T03:53:23.584 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:23.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.628+0000 7fd75d996700 1 Processor -- start 2021-07-11T03:53:23.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.628+0000 7fd75d996700 1 -- start start 2021-07-11T03:53:23.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7580a9de0 0x7fd7580abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7580b0960 con 0x7fd7580b0450 2021-07-11T03:53:23.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7580ab300 con 0x7fd758060100 2021-07-11T03:53:23.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd758067c40 con 0x7fd7580a9de0 2021-07-11T03:53:23.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd7577fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd7580b0450 0x7fd758146900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38468/0 (socket says 172.21.2.104:38468) 2021-07-11T03:53:23.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd7577fe700 1 -- 172.21.2.104:0/1626999119 learned_addr learned my addr 172.21.2.104:0/1626999119 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:23.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201058036 0 0) 0x7fd7580b0960 con 0x7fd7580b0450 2021-07-11T03:53:23.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd738003260 con 0x7fd7580b0450 2021-07-11T03:53:23.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2080863028 0 0) 0x7fd738003260 con 0x7fd7580b0450 2021-07-11T03:53:23.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7580a9de0 msgr2=0x7fd7580abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 --2- 172.21.2.104:0/1626999119 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7580a9de0 0x7fd7580abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 >> v1:172.21.2.105:6789/0 conn(0x7fd758060100 legacy=0x7fd7580ab730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7580604e0 con 0x7fd7580b0450 2021-07-11T03:53:23.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fd7500019e0 con 0x7fd7580b0450 2021-07-11T03:53:23.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 >> v1:172.21.2.104:6789/0 conn(0x7fd7580b0450 legacy=0x7fd758146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 --2- 172.21.2.104:0/1626999119 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd738003800 0x7fd738003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 --2- 172.21.2.104:0/1626999119 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7580a9de0 0x7fd738004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 --2- 172.21.2.104:0/1626999119 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd738004bf0 0x7fd73800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758067c40 con 0x7fd738003800 2021-07-11T03:53:23.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380052d0 con 0x7fd738004bf0 2021-07-11T03:53:23.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd73800bc40 con 0x7fd7580a9de0 2021-07-11T03:53:23.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd756ffd700 1 --2- 172.21.2.104:0/1626999119 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd738003800 0x7fd738003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd7577fe700 1 --2- 172.21.2.104:0/1626999119 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd738004bf0 0x7fd73800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd756ffd700 1 -- 172.21.2.104:0/1626999119 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7580a9de0 msgr2=0x7fd738004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd756ffd700 1 --2- 172.21.2.104:0/1626999119 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7580a9de0 0x7fd738004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd756ffd700 1 -- 172.21.2.104:0/1626999119 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd738004bf0 msgr2=0x7fd73800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd756ffd700 1 --2- 172.21.2.104:0/1626999119 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd738004bf0 0x7fd73800b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:23.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd756ffd700 1 -- 172.21.2.104:0/1626999119 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd738003260 con 0x7fd738003800 2021-07-11T03:53:23.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd756ffd700 1 --2- 172.21.2.104:0/1626999119 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd738003800 0x7fd738003c00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fd740000f70 tx=0x7fd7400079e0).ready entity=mon.0 client_cookie=70b80f70c51d0588 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd740007cf0 con 0x7fd738003800 2021-07-11T03:53:23.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd740014410 con 0x7fd738003800 2021-07-11T03:53:23.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd755ffb700 1 -- 172.21.2.104:0/1626999119 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd740014770 con 0x7fd738003800 2021-07-11T03:53:23.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 -- 172.21.2.104:0/1626999119 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd738003800 msgr2=0x7fd738003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 --2- 172.21.2.104:0/1626999119 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd738003800 0x7fd738003c00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fd740000f70 tx=0x7fd7400079e0).stop 2021-07-11T03:53:23.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd7577fe700 1 -- 172.21.2.104:0/1626999119 reap_dead start 2021-07-11T03:53:23.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 -- 172.21.2.104:0/1626999119 shutdown_connections 2021-07-11T03:53:23.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 -- 172.21.2.104:0/1626999119 >> 172.21.2.104:0/1626999119 conn(0x7fd758061070 msgr2=0x7fd75805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:23.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 -- 172.21.2.104:0/1626999119 shutdown_connections 2021-07-11T03:53:23.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 -- 172.21.2.104:0/1626999119 wait complete. 2021-07-11T03:53:23.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 Processor -- start 2021-07-11T03:53:23.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.632+0000 7fd75d996700 1 -- start start 2021-07-11T03:53:23.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd75d996700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7580af010 0x7fd758149b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd75d996700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd758143ba0 0x7fd7581522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd75d996700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7580a8430 0x7fd7580a7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd75d996700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758143a00 con 0x7fd7580af010 2021-07-11T03:53:23.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd75d996700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75814a440 con 0x7fd758143ba0 2021-07-11T03:53:23.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd75d996700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75814a060 con 0x7fd7580a8430 2021-07-11T03:53:23.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd756ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7580af010 0x7fd758149b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd756ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7580af010 0x7fd758149b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41710/0 (socket says 172.21.2.104:41710) 2021-07-11T03:53:23.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd756ffd700 1 -- 172.21.2.104:0/3747366693 learned_addr learned my addr 172.21.2.104:0/3747366693 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:23.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd756ffd700 1 -- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7580a8430 msgr2=0x7fd7580a7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd756ffd700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7580a8430 0x7fd7580a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd756ffd700 1 -- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd758143ba0 msgr2=0x7fd7581522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd756ffd700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd758143ba0 0x7fd7581522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd756ffd700 1 -- 172.21.2.104:0/3747366693 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7580af3f0 con 0x7fd7580af010 2021-07-11T03:53:23.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd756ffd700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7580af010 0x7fd758149b70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd758143b40 tx=0x7fd740015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd74ffff700 1 -- 172.21.2.104:0/3747366693 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd740007cf0 con 0x7fd7580af010 2021-07-11T03:53:23.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd74ffff700 1 -- 172.21.2.104:0/3747366693 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7400137c0 con 0x7fd7580af010 2021-07-11T03:53:23.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd74ffff700 1 -- 172.21.2.104:0/3747366693 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd740013b20 con 0x7fd7580af010 2021-07-11T03:53:23.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd75d996700 1 -- 172.21.2.104:0/3747366693 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd758156610 con 0x7fd7580af010 2021-07-11T03:53:23.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd75d996700 1 -- 172.21.2.104:0/3747366693 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75814a910 con 0x7fd7580af010 2021-07-11T03:53:23.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd75d996700 1 -- 172.21.2.104:0/3747366693 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd758152980 con 0x7fd7580af010 2021-07-11T03:53:23.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd74ffff700 1 -- 172.21.2.104:0/3747366693 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd740013c80 con 0x7fd7580af010 2021-07-11T03:53:23.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd74ffff700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd73c0531a0 0x7fd73c055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd74ffff700 1 -- 172.21.2.104:0/3747366693 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd74001c040 con 0x7fd7580af010 2021-07-11T03:53:23.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.636+0000 7fd7567fc700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd73c0531a0 0x7fd73c055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.640+0000 7fd7567fc700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd73c0531a0 0x7fd73c055660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd748002f70 tx=0x7fd748006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.640+0000 7fd74ffff700 1 -- 172.21.2.104:0/3747366693 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd740043ad0 con 0x7fd7580af010 2021-07-11T03:53:23.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 -- 172.21.2.104:0/3747366693 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd758057350 con 0x7fd7580af010 2021-07-11T03:53:23.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd74ffff700 1 -- 172.21.2.104:0/3747366693 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16051 (secure 0 0 0) 0x7fd7400470f0 con 0x7fd7580af010 2021-07-11T03:53:23.724 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:23.724 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":22,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:23.356023+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:23.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 -- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd73c0531a0 msgr2=0x7fd73c055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd73c0531a0 0x7fd73c055660 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd748002f70 tx=0x7fd748006a10).stop 2021-07-11T03:53:23.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 -- 172.21.2.104:0/3747366693 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7580af010 msgr2=0x7fd758149b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7580af010 0x7fd758149b70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd758143b40 tx=0x7fd740015000).stop 2021-07-11T03:53:23.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 -- 172.21.2.104:0/3747366693 shutdown_connections 2021-07-11T03:53:23.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7580a8430 0x7fd7580a7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd758143ba0 0x7fd7581522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd73c0531a0 0x7fd73c055660 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:23.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 --2- 172.21.2.104:0/3747366693 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7580af010 0x7fd758149b70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:23.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 -- 172.21.2.104:0/3747366693 >> 172.21.2.104:0/3747366693 conn(0x7fd758061070 msgr2=0x7fd7580aca80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:23.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 -- 172.21.2.104:0/3747366693 shutdown_connections 2021-07-11T03:53:23.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.720+0000 7fd75d996700 1 -- 172.21.2.104:0/3747366693 wait complete. 2021-07-11T03:53:23.732 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:23.733 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-07-11T03:53:23.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 Processor -- start 2021-07-11T03:53:23.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 -- start start 2021-07-11T03:53:23.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f96ec0b1170 0x7f96ec141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96ec141920 0x7f96ec141d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec0b31d0 con 0x7f96ec0b7810 2021-07-11T03:53:23.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec0b7bf0 con 0x7f96ec0b1170 2021-07-11T03:53:23.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0675c0 con 0x7f96ec141920 2021-07-11T03:53:23.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96eaffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f96ec0b1170 0x7f96ec141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96eaffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f96ec0b1170 0x7f96ec141430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41722/0 (socket says 172.21.2.104:41722) 2021-07-11T03:53:23.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96eaffd700 1 -- 172.21.2.104:0/1734592865 learned_addr learned my addr 172.21.2.104:0/1734592865 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:23.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96eaffd700 1 -- 172.21.2.104:0/1734592865 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96ec141920 msgr2=0x7f96ec141d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96eaffd700 1 --2- 172.21.2.104:0/1734592865 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96ec141920 0x7f96ec141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96eaffd700 1 -- 172.21.2.104:0/1734592865 >> v1:172.21.2.105:6789/0 conn(0x7f96ec0b7810 legacy=0x7f96ec1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96eaffd700 1 -- 172.21.2.104:0/1734592865 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec142270 con 0x7f96ec0b1170 2021-07-11T03:53:23.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96eaffd700 1 --2- 172.21.2.104:0/1734592865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96ec0b1170 0x7f96ec141430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f96ec067ca0 tx=0x7f96d4006eb0).ready entity=mon.0 client_cookie=4b33c58fbb10ddb7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96ea7fc700 1 -- 172.21.2.104:0/1734592865 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d400d570 con 0x7f96ec0b1170 2021-07-11T03:53:23.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96ea7fc700 1 -- 172.21.2.104:0/1734592865 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d4000e90 con 0x7f96ec0b1170 2021-07-11T03:53:23.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96ea7fc700 1 -- 172.21.2.104:0/1734592865 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d400d8c0 con 0x7f96ec0b1170 2021-07-11T03:53:23.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 -- 172.21.2.104:0/1734592865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96ec0b1170 msgr2=0x7f96ec141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 --2- 172.21.2.104:0/1734592865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96ec0b1170 0x7f96ec141430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f96ec067ca0 tx=0x7f96d4006eb0).stop 2021-07-11T03:53:23.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 -- 172.21.2.104:0/1734592865 shutdown_connections 2021-07-11T03:53:23.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 --2- 172.21.2.104:0/1734592865 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96ec141920 0x7f96ec141d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 --2- 172.21.2.104:0/1734592865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96ec0b1170 0x7f96ec141430 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:23.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 -- 172.21.2.104:0/1734592865 >> 172.21.2.104:0/1734592865 conn(0x7f96ec05f6b0 msgr2=0x7f96ec05fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:23.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 -- 172.21.2.104:0/1734592865 shutdown_connections 2021-07-11T03:53:23.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.780+0000 7f96f2139700 1 -- 172.21.2.104:0/1734592865 wait complete. 2021-07-11T03:53:23.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96f2139700 1 Processor -- start 2021-07-11T03:53:23.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96f2139700 1 -- start start 2021-07-11T03:53:23.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96f2139700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96ec0b1170 0x7f96ec0adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96f2139700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f96ec0b7810 0x7f96ec0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96f2139700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96ec141920 0x7f96ec150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96f2139700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec143710 con 0x7f96ec0b1170 2021-07-11T03:53:23.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96f2139700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec1439b0 con 0x7f96ec0b7810 2021-07-11T03:53:23.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96f2139700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec143c50 con 0x7f96ec141920 2021-07-11T03:53:23.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96eb7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96ec0b1170 0x7f96ec0adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96eb7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96ec0b1170 0x7f96ec0adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41726/0 (socket says 172.21.2.104:41726) 2021-07-11T03:53:23.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96eb7fe700 1 -- 172.21.2.104:0/218826096 learned_addr learned my addr 172.21.2.104:0/218826096 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:23.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96eb7fe700 1 -- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96ec141920 msgr2=0x7f96ec150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96eb7fe700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96ec141920 0x7f96ec150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96eb7fe700 1 -- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f96ec0b7810 msgr2=0x7f96ec0b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96eb7fe700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f96ec0b7810 0x7f96ec0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96eb7fe700 1 -- 172.21.2.104:0/218826096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d400b1e0 con 0x7f96ec0b1170 2021-07-11T03:53:23.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96eb7fe700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96ec0b1170 0x7f96ec0adb70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f96dc000f70 tx=0x7f96dc007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96e8ff9700 1 -- 172.21.2.104:0/218826096 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96dc00ce80 con 0x7f96ec0b1170 2021-07-11T03:53:23.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96e8ff9700 1 -- 172.21.2.104:0/218826096 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc013450 con 0x7f96ec0b1170 2021-07-11T03:53:23.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96f2139700 1 -- 172.21.2.104:0/218826096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec0aff30 con 0x7f96ec0b1170 2021-07-11T03:53:23.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96e8ff9700 1 -- 172.21.2.104:0/218826096 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96dc0137b0 con 0x7f96ec0b1170 2021-07-11T03:53:23.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96f2139700 1 -- 172.21.2.104:0/218826096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec150500 con 0x7f96ec0b1170 2021-07-11T03:53:23.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96f2139700 1 -- 172.21.2.104:0/218826096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec057350 con 0x7f96ec0b1170 2021-07-11T03:53:23.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96e8ff9700 1 -- 172.21.2.104:0/218826096 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f96dc01a060 con 0x7f96ec0b1170 2021-07-11T03:53:23.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96e8ff9700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96d0053240 0x7f96d0055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96e8ff9700 1 -- 172.21.2.104:0/218826096 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f96dc06afc0 con 0x7f96ec0b1170 2021-07-11T03:53:23.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.784+0000 7f96eaffd700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96d0053240 0x7f96d0055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.788+0000 7f96eaffd700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96d0053240 0x7f96d0055700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f96ec067ca0 tx=0x7f96d4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.788+0000 7f96e8ff9700 1 -- 172.21.2.104:0/218826096 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f96dc0421f0 con 0x7f96ec0b1170 2021-07-11T03:53:23.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.868+0000 7f96f2139700 1 -- 172.21.2.104:0/218826096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96ec0698e0 con 0x7f96ec0b1170 2021-07-11T03:53:23.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.868+0000 7f96e8ff9700 1 -- 172.21.2.104:0/218826096 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16051 (secure 0 0 0) 0x7f96dc018070 con 0x7f96ec0b1170 2021-07-11T03:53:23.873 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:23.873 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":22,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:23.356023+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:23.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.868+0000 7f96f2139700 1 -- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96d0053240 msgr2=0x7f96d0055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.868+0000 7f96f2139700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96d0053240 0x7f96d0055700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f96ec067ca0 tx=0x7f96d4015000).stop 2021-07-11T03:53:23.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.868+0000 7f96f2139700 1 -- 172.21.2.104:0/218826096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96ec0b1170 msgr2=0x7f96ec0adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.868+0000 7f96f2139700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96ec0b1170 0x7f96ec0adb70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f96dc000f70 tx=0x7f96dc007c30).stop 2021-07-11T03:53:23.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.872+0000 7f96f2139700 1 -- 172.21.2.104:0/218826096 shutdown_connections 2021-07-11T03:53:23.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.872+0000 7f96f2139700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96ec141920 0x7f96ec150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.872+0000 7f96f2139700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f96ec0b7810 0x7f96ec0b03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.872+0000 7f96f2139700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96d0053240 0x7f96d0055700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:23.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.872+0000 7f96f2139700 1 --2- 172.21.2.104:0/218826096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96ec0b1170 0x7f96ec0adb70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:23.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.872+0000 7f96f2139700 1 -- 172.21.2.104:0/218826096 >> 172.21.2.104:0/218826096 conn(0x7f96ec05f6b0 msgr2=0x7f96ec140e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:23.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.872+0000 7f96f2139700 1 -- 172.21.2.104:0/218826096 shutdown_connections 2021-07-11T03:53:23.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.872+0000 7f96f2139700 1 -- 172.21.2.104:0/218826096 wait complete. 2021-07-11T03:53:23.880 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:53:20.980780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:53:21.341375+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:53:23.881 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:53:23.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.928+0000 7fd9611dc700 1 Processor -- start 2021-07-11T03:53:23.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.928+0000 7fd9611dc700 1 -- start start 2021-07-11T03:53:23.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd95c0b12d0 0x7fd95c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95c0a77d0 con 0x7fd95c0b22a0 2021-07-11T03:53:23.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95c0b0650 con 0x7fd95c0b1ec0 2021-07-11T03:53:23.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0b1740 con 0x7fd95c0b12d0 2021-07-11T03:53:23.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95b59e700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd95c0b22a0 0x7fd95c141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38504/0 (socket says 172.21.2.104:38504) 2021-07-11T03:53:23.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95b59e700 1 -- 172.21.2.104:0/2875975850 learned_addr learned my addr 172.21.2.104:0/2875975850 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:23.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3398898610 0 0) 0x7fd95c0a77d0 con 0x7fd95c0b22a0 2021-07-11T03:53:23.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd93c003260 con 0x7fd95c0b22a0 2021-07-11T03:53:23.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1764937667 0 0) 0x7fd93c003260 con 0x7fd95c0b22a0 2021-07-11T03:53:23.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd95c0b12d0 msgr2=0x7fd95c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 --2- 172.21.2.104:0/2875975850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd95c0b12d0 0x7fd95c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 >> v1:172.21.2.105:6789/0 conn(0x7fd95c0b1ec0 legacy=0x7fd95c0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c0b08f0 con 0x7fd95c0b22a0 2021-07-11T03:53:23.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fd9540019e0 con 0x7fd95c0b22a0 2021-07-11T03:53:23.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 >> v1:172.21.2.104:6789/0 conn(0x7fd95c0b22a0 legacy=0x7fd95c141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 --2- 172.21.2.104:0/2875975850 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd93c003800 0x7fd93c003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 --2- 172.21.2.104:0/2875975850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd93c004500 0x7fd93c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 --2- 172.21.2.104:0/2875975850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd95c0b12d0 0x7fd93c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0b1740 con 0x7fd93c004500 2021-07-11T03:53:23.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c0052c0 con 0x7fd93c003800 2021-07-11T03:53:23.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93c00bc40 con 0x7fd95c0b12d0 2021-07-11T03:53:23.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95a59c700 1 --2- 172.21.2.104:0/2875975850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd93c004500 0x7fd93c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95a59c700 1 -- 172.21.2.104:0/2875975850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd95c0b12d0 msgr2=0x7fd93c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95a59c700 1 --2- 172.21.2.104:0/2875975850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd95c0b12d0 0x7fd93c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95a59c700 1 -- 172.21.2.104:0/2875975850 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd93c003800 msgr2=0x7fd93c003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:23.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95a59c700 1 --2- 172.21.2.104:0/2875975850 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd93c003800 0x7fd93c003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95a59c700 1 -- 172.21.2.104:0/2875975850 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c003260 con 0x7fd93c004500 2021-07-11T03:53:23.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95a59c700 1 --2- 172.21.2.104:0/2875975850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd93c004500 0x7fd93c004980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd93c0054d0 tx=0x7fd94c006ca0).ready entity=mon.0 client_cookie=261859a278e8bad9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd94c012410 con 0x7fd93c004500 2021-07-11T03:53:23.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd94c012570 con 0x7fd93c004500 2021-07-11T03:53:23.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd959d9b700 1 -- 172.21.2.104:0/2875975850 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd94c0128d0 con 0x7fd93c004500 2021-07-11T03:53:23.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- 172.21.2.104:0/2875975850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd93c004500 msgr2=0x7fd93c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:23.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 --2- 172.21.2.104:0/2875975850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd93c004500 0x7fd93c004980 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fd93c0054d0 tx=0x7fd94c006ca0).stop 2021-07-11T03:53:23.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95b59e700 1 -- 172.21.2.104:0/2875975850 reap_dead start 2021-07-11T03:53:23.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- 172.21.2.104:0/2875975850 shutdown_connections 2021-07-11T03:53:23.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- 172.21.2.104:0/2875975850 >> 172.21.2.104:0/2875975850 conn(0x7fd95c05f6b0 msgr2=0x7fd95c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:23.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- 172.21.2.104:0/2875975850 shutdown_connections 2021-07-11T03:53:23.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- 172.21.2.104:0/2875975850 wait complete. 2021-07-11T03:53:23.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 Processor -- start 2021-07-11T03:53:23.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- start start 2021-07-11T03:53:23.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd95c0ae5a0 0x7fd95c14ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd95c156980 0x7fd95c156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd95c13a790 0x7fd95c13ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c14b730 con 0x7fd95c0ae5a0 2021-07-11T03:53:23.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0ae980 con 0x7fd95c13a790 2021-07-11T03:53:23.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd9611dc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95c0abde0 con 0x7fd95c156980 2021-07-11T03:53:23.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd95c0ae5a0 0x7fd95c14ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd95c0ae5a0 0x7fd95c14ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41746/0 (socket says 172.21.2.104:41746) 2021-07-11T03:53:23.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.932+0000 7fd95ad9d700 1 -- 172.21.2.104:0/1068520410 learned_addr learned my addr 172.21.2.104:0/1068520410 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:23.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd95ad9d700 1 -- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd95c156980 msgr2=0x7fd95c156e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:23.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd95ad9d700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd95c156980 0x7fd95c156e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd95ad9d700 1 -- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd95c13a790 msgr2=0x7fd95c13ac40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:23.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd95ad9d700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd95c13a790 0x7fd95c13ac40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:23.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd95ad9d700 1 -- 172.21.2.104:0/1068520410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c0ab400 con 0x7fd95c0ae5a0 2021-07-11T03:53:23.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd95ad9d700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd95c0ae5a0 0x7fd95c14ae70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd944001fa0 tx=0x7fd944007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd953fff700 1 -- 172.21.2.104:0/1068520410 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd94400ce80 con 0x7fd95c0ae5a0 2021-07-11T03:53:23.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd953fff700 1 -- 172.21.2.104:0/1068520410 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd944000e90 con 0x7fd95c0ae5a0 2021-07-11T03:53:23.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd9611dc700 1 -- 172.21.2.104:0/1068520410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95c0ab6a0 con 0x7fd95c0ae5a0 2021-07-11T03:53:23.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd953fff700 1 -- 172.21.2.104:0/1068520410 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd944007430 con 0x7fd95c0ae5a0 2021-07-11T03:53:23.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd9611dc700 1 -- 172.21.2.104:0/1068520410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd95c14b9c0 con 0x7fd95c0ae5a0 2021-07-11T03:53:23.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd9611dc700 1 -- 172.21.2.104:0/1068520410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd95c14b500 con 0x7fd95c0ae5a0 2021-07-11T03:53:23.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd953fff700 1 -- 172.21.2.104:0/1068520410 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd94401a060 con 0x7fd95c0ae5a0 2021-07-11T03:53:23.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd953fff700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9400532d0 0x7fd940055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:23.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd953fff700 1 -- 172.21.2.104:0/1068520410 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd94406ad30 con 0x7fd95c0ae5a0 2021-07-11T03:53:23.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd95a59c700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9400532d0 0x7fd940055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:23.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.936+0000 7fd95a59c700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9400532d0 0x7fd940055790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd93c0054d0 tx=0x7fd94c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:23.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:23.940+0000 7fd953fff700 1 -- 172.21.2.104:0/1068520410 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd944042070 con 0x7fd95c0ae5a0 2021-07-11T03:53:24.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.020+0000 7fd9611dc700 1 -- 172.21.2.104:0/1068520410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd95c157490 con 0x7fd95c0ae5a0 2021-07-11T03:53:24.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.020+0000 7fd953fff700 1 -- 172.21.2.104:0/1068520410 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd95c157490 con 0x7fd95c0ae5a0 2021-07-11T03:53:24.025 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:53:24.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.020+0000 7fd9611dc700 1 -- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9400532d0 msgr2=0x7fd940055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.020+0000 7fd9611dc700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9400532d0 0x7fd940055790 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fd93c0054d0 tx=0x7fd94c015000).stop 2021-07-11T03:53:24.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.020+0000 7fd9611dc700 1 -- 172.21.2.104:0/1068520410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd95c0ae5a0 msgr2=0x7fd95c14ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.020+0000 7fd9611dc700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd95c0ae5a0 0x7fd95c14ae70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fd944001fa0 tx=0x7fd944007c30).stop 2021-07-11T03:53:24.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.024+0000 7fd9611dc700 1 -- 172.21.2.104:0/1068520410 shutdown_connections 2021-07-11T03:53:24.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.024+0000 7fd9611dc700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd95c13a790 0x7fd95c13ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.024+0000 7fd9611dc700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd95c156980 0x7fd95c156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.024+0000 7fd9611dc700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9400532d0 0x7fd940055790 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.024+0000 7fd9611dc700 1 --2- 172.21.2.104:0/1068520410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd95c0ae5a0 0x7fd95c14ae70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.024+0000 7fd9611dc700 1 -- 172.21.2.104:0/1068520410 >> 172.21.2.104:0/1068520410 conn(0x7fd95c05f6b0 msgr2=0x7fd95c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.024+0000 7fd9611dc700 1 -- 172.21.2.104:0/1068520410 shutdown_connections 2021-07-11T03:53:24.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.024+0000 7fd9611dc700 1 -- 172.21.2.104:0/1068520410 wait complete. 2021-07-11T03:53:24.032 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:53:24.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.100+0000 7f301394e700 1 Processor -- start 2021-07-11T03:53:24.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.100+0000 7f301394e700 1 -- start start 2021-07-11T03:53:24.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.100+0000 7f301394e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f300c0b1ec0 0x7f300c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.100+0000 7f301394e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f300c0affb0 0x7f300c0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.100+0000 7f301394e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f300c0b2790 con 0x7f300c0a70c0 2021-07-11T03:53:24.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.100+0000 7f301394e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300c0b03f0 con 0x7f300c0affb0 2021-07-11T03:53:24.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.100+0000 7f301394e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0b0650 con 0x7f300c0b1ec0 2021-07-11T03:53:24.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f3011eeb700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f300c0a70c0 0x7f300c141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:37890/0 (socket says 172.21.2.104:37890) 2021-07-11T03:53:24.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f3011eeb700 1 -- 172.21.2.104:0/1278573387 learned_addr learned my addr 172.21.2.104:0/1278573387 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:24.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f3010ee9700 1 --2- 172.21.2.104:0/1278573387 >> v2:172.21.2.105:3300/0 conn(0x7f300c0affb0 0x7f300c0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f30116ea700 1 --2- 172.21.2.104:0/1278573387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f300c0b1ec0 0x7f300c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f3010ee9700 1 -- 172.21.2.104:0/1278573387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f300c0b1ec0 msgr2=0x7f300c0b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:24.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f3010ee9700 1 --2- 172.21.2.104:0/1278573387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f300c0b1ec0 0x7f300c0b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f3010ee9700 1 -- 172.21.2.104:0/1278573387 >> v1:172.21.2.105:6789/0 conn(0x7f300c0a70c0 legacy=0x7f300c141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f3010ee9700 1 -- 172.21.2.104:0/1278573387 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c0b08f0 con 0x7f300c0affb0 2021-07-11T03:53:24.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f3010ee9700 1 --2- 172.21.2.104:0/1278573387 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f300c0affb0 0x7f300c0b22a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f2ffc002f70 tx=0x7f2ffc006df0).ready entity=mon.1 client_cookie=fe6c655883356c31 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f2ffbfff700 1 -- 172.21.2.104:0/1278573387 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ffc00d570 con 0x7f300c0affb0 2021-07-11T03:53:24.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f2ffbfff700 1 -- 172.21.2.104:0/1278573387 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc000e90 con 0x7f300c0affb0 2021-07-11T03:53:24.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f2ffbfff700 1 -- 172.21.2.104:0/1278573387 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2ffc00d8c0 con 0x7f300c0affb0 2021-07-11T03:53:24.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f301394e700 1 -- 172.21.2.104:0/1278573387 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f300c0affb0 msgr2=0x7f300c0b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f301394e700 1 --2- 172.21.2.104:0/1278573387 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f300c0affb0 0x7f300c0b22a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f2ffc002f70 tx=0x7f2ffc006df0).stop 2021-07-11T03:53:24.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f301394e700 1 -- 172.21.2.104:0/1278573387 shutdown_connections 2021-07-11T03:53:24.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f301394e700 1 --2- 172.21.2.104:0/1278573387 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f300c0affb0 0x7f300c0b22a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f301394e700 1 --2- 172.21.2.104:0/1278573387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f300c0b1ec0 0x7f300c0b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f301394e700 1 -- 172.21.2.104:0/1278573387 >> 172.21.2.104:0/1278573387 conn(0x7f300c05f6b0 msgr2=0x7f300c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f301394e700 1 -- 172.21.2.104:0/1278573387 shutdown_connections 2021-07-11T03:53:24.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f301394e700 1 -- 172.21.2.104:0/1278573387 wait complete. 2021-07-11T03:53:24.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.104+0000 7f301394e700 1 Processor -- start 2021-07-11T03:53:24.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f301394e700 1 -- start start 2021-07-11T03:53:24.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f301394e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f300c0a70c0 0x7f300c15a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f301394e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f300c0affb0 0x7f300c0abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f301394e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f300c0b1ec0 0x7f300c13c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f301394e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0b5b50 con 0x7f300c0a70c0 2021-07-11T03:53:24.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f301394e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0b5d80 con 0x7f300c0b1ec0 2021-07-11T03:53:24.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f301394e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0b6020 con 0x7f300c0affb0 2021-07-11T03:53:24.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f30116ea700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f300c0a70c0 0x7f300c15a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f30116ea700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f300c0a70c0 0x7f300c15a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41758/0 (socket says 172.21.2.104:41758) 2021-07-11T03:53:24.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f30116ea700 1 -- 172.21.2.104:0/3222294866 learned_addr learned my addr 172.21.2.104:0/3222294866 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:24.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f3011eeb700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f300c0b1ec0 0x7f300c13c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f30116ea700 1 -- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f300c0affb0 msgr2=0x7f300c0abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f3010ee9700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f300c0affb0 0x7f300c0abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f30116ea700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f300c0affb0 0x7f300c0abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f30116ea700 1 -- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f300c0b1ec0 msgr2=0x7f300c13c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f30116ea700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f300c0b1ec0 0x7f300c13c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f30116ea700 1 -- 172.21.2.104:0/3222294866 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc00b180 con 0x7f300c0a70c0 2021-07-11T03:53:24.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f3011eeb700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f300c0b1ec0 0x7f300c13c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:24.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f30116ea700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f300c0a70c0 0x7f300c15a9e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f300800c000 tx=0x7f3008006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f2ffa7fc700 1 -- 172.21.2.104:0/3222294866 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3008010780 con 0x7f300c0a70c0 2021-07-11T03:53:24.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f2ffa7fc700 1 -- 172.21.2.104:0/3222294866 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30080108e0 con 0x7f300c0a70c0 2021-07-11T03:53:24.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f301394e700 1 -- 172.21.2.104:0/3222294866 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c0b62c0 con 0x7f300c0a70c0 2021-07-11T03:53:24.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f2ffa7fc700 1 -- 172.21.2.104:0/3222294866 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3008010c40 con 0x7f300c0a70c0 2021-07-11T03:53:24.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f301394e700 1 -- 172.21.2.104:0/3222294866 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c1464e0 con 0x7f300c0a70c0 2021-07-11T03:53:24.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f301394e700 1 -- 172.21.2.104:0/3222294866 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300c057350 con 0x7f300c0a70c0 2021-07-11T03:53:24.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f2ffa7fc700 1 -- 172.21.2.104:0/3222294866 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f300801c460 con 0x7f300c0a70c0 2021-07-11T03:53:24.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f2ffa7fc700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2fec0532b0 0x7f2fec055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f2ffa7fc700 1 -- 172.21.2.104:0/3222294866 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f300806ba60 con 0x7f300c0a70c0 2021-07-11T03:53:24.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.108+0000 7f3010ee9700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2fec0532b0 0x7f2fec055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.112+0000 7f3010ee9700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2fec0532b0 0x7f2fec055770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2ffc013000 tx=0x7f2ffc014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.112+0000 7f2ffa7fc700 1 -- 172.21.2.104:0/3222294866 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3008042c30 con 0x7f300c0a70c0 2021-07-11T03:53:24.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 -- 172.21.2.104:0/3222294866 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f300c0ac760 con 0x7f300c0a70c0 2021-07-11T03:53:24.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f2ffa7fc700 1 -- 172.21.2.104:0/3222294866 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3008046250 con 0x7f300c0a70c0 2021-07-11T03:53:24.196 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:24.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 -- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2fec0532b0 msgr2=0x7f2fec055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2fec0532b0 0x7f2fec055770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2ffc013000 tx=0x7f2ffc014000).stop 2021-07-11T03:53:24.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 -- 172.21.2.104:0/3222294866 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f300c0a70c0 msgr2=0x7f300c15a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f300c0a70c0 0x7f300c15a9e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f300800c000 tx=0x7f3008006df0).stop 2021-07-11T03:53:24.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 -- 172.21.2.104:0/3222294866 shutdown_connections 2021-07-11T03:53:24.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f300c0b1ec0 0x7f300c13c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f300c0affb0 0x7f300c0abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2fec0532b0 0x7f2fec055770 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 --2- 172.21.2.104:0/3222294866 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f300c0a70c0 0x7f300c15a9e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 -- 172.21.2.104:0/3222294866 >> 172.21.2.104:0/3222294866 conn(0x7f300c05f6b0 msgr2=0x7f300c0a8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 -- 172.21.2.104:0/3222294866 shutdown_connections 2021-07-11T03:53:24.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.192+0000 7f301394e700 1 -- 172.21.2.104:0/3222294866 wait complete. 2021-07-11T03:53:24.204 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:53:24.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 Processor -- start 2021-07-11T03:53:24.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- start start 2021-07-11T03:53:24.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe7940b05b0 0x7fe7940b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe794143920 con 0x7fe7940b3710 2021-07-11T03:53:24.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe794143bc0 con 0x7fe7940a9c10 2021-07-11T03:53:24.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe794143e60 con 0x7fe7940b05b0 2021-07-11T03:53:24.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe792ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe7940b05b0 0x7fe7940b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7927fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe7940b3710 0x7fe79413d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38534/0 (socket says 172.21.2.104:38534) 2021-07-11T03:53:24.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7927fc700 1 -- 172.21.2.104:0/2690306720 learned_addr learned my addr 172.21.2.104:0/2690306720 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:24.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe792ffd700 1 -- 172.21.2.104:0/2690306720 >> v1:172.21.2.105:6789/0 conn(0x7fe7940a9c10 legacy=0x7fe7940b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:24.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe792ffd700 1 -- 172.21.2.104:0/2690306720 >> v1:172.21.2.104:6789/0 conn(0x7fe7940b3710 legacy=0x7fe79413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe792ffd700 1 -- 172.21.2.104:0/2690306720 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe794144100 con 0x7fe7940b05b0 2021-07-11T03:53:24.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe792ffd700 1 --2- 172.21.2.104:0/2690306720 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7940b05b0 0x7fe7940b0990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fe79406aa90 tx=0x7fe784006df0).ready entity=mon.0 client_cookie=c5421a3c588ba42b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe791ffb700 1 -- 172.21.2.104:0/2690306720 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe78400d5c0 con 0x7fe7940b05b0 2021-07-11T03:53:24.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe791ffb700 1 -- 172.21.2.104:0/2690306720 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe784000e90 con 0x7fe7940b05b0 2021-07-11T03:53:24.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe791ffb700 1 -- 172.21.2.104:0/2690306720 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe78400d910 con 0x7fe7940b05b0 2021-07-11T03:53:24.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- 172.21.2.104:0/2690306720 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7940b05b0 msgr2=0x7fe7940b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 --2- 172.21.2.104:0/2690306720 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7940b05b0 0x7fe7940b0990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fe79406aa90 tx=0x7fe784006df0).stop 2021-07-11T03:53:24.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- 172.21.2.104:0/2690306720 shutdown_connections 2021-07-11T03:53:24.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 --2- 172.21.2.104:0/2690306720 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7940b05b0 0x7fe7940b0990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- 172.21.2.104:0/2690306720 >> 172.21.2.104:0/2690306720 conn(0x7fe7940607c0 msgr2=0x7fe794060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- 172.21.2.104:0/2690306720 shutdown_connections 2021-07-11T03:53:24.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- 172.21.2.104:0/2690306720 wait complete. 2021-07-11T03:53:24.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 Processor -- start 2021-07-11T03:53:24.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- start start 2021-07-11T03:53:24.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe7940a9c10 0x7fe79414ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe7940b05b0 0x7fe7940aced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7940b3710 0x7fe7940ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940662c0 con 0x7fe7940b3710 2021-07-11T03:53:24.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7940ab890 con 0x7fe7940a9c10 2021-07-11T03:53:24.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7998eb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe794066d10 con 0x7fe7940b05b0 2021-07-11T03:53:24.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7937fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7940b3710 0x7fe7940ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7937fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7940b3710 0x7fe7940ad3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41776/0 (socket says 172.21.2.104:41776) 2021-07-11T03:53:24.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7937fe700 1 -- 172.21.2.104:0/1744488324 learned_addr learned my addr 172.21.2.104:0/1744488324 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:24.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7937fe700 1 -- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe7940b05b0 msgr2=0x7fe7940aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7937fe700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe7940b05b0 0x7fe7940aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7937fe700 1 -- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe7940a9c10 msgr2=0x7fe79414ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7937fe700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe7940a9c10 0x7fe79414ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.268+0000 7fe7937fe700 1 -- 172.21.2.104:0/1744488324 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78400b180 con 0x7fe7940b3710 2021-07-11T03:53:24.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe7937fe700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7940b3710 0x7fe7940ad3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe78c003310 tx=0x7fe78c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe783fff700 1 -- 172.21.2.104:0/1744488324 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe78c001500 con 0x7fe7940b3710 2021-07-11T03:53:24.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe783fff700 1 -- 172.21.2.104:0/1744488324 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c001660 con 0x7fe7940b3710 2021-07-11T03:53:24.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe7998eb700 1 -- 172.21.2.104:0/1744488324 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7940ae540 con 0x7fe7940b3710 2021-07-11T03:53:24.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe783fff700 1 -- 172.21.2.104:0/1744488324 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe78c0019c0 con 0x7fe7940b3710 2021-07-11T03:53:24.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe7998eb700 1 -- 172.21.2.104:0/1744488324 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe794065c60 con 0x7fe7940b3710 2021-07-11T03:53:24.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe7998eb700 1 -- 172.21.2.104:0/1744488324 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe794057350 con 0x7fe7940b3710 2021-07-11T03:53:24.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe783fff700 1 -- 172.21.2.104:0/1744488324 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe78c018030 con 0x7fe7940b3710 2021-07-11T03:53:24.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe783fff700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe770053350 0x7fe770055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe783fff700 1 -- 172.21.2.104:0/1744488324 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe78c06afc0 con 0x7fe7940b3710 2021-07-11T03:53:24.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe792ffd700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe770053350 0x7fe770055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.272+0000 7fe792ffd700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe770053350 0x7fe770055810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe79406aa90 tx=0x7fe784012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.276+0000 7fe783fff700 1 -- 172.21.2.104:0/1744488324 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe78c0422a0 con 0x7fe7940b3710 2021-07-11T03:53:24.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 -- 172.21.2.104:0/1744488324 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe794066ed0 con 0x7fe7940b3710 2021-07-11T03:53:24.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe783fff700 1 -- 172.21.2.104:0/1744488324 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe78c0458c0 con 0x7fe7940b3710 2021-07-11T03:53:24.361 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:24.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 -- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe770053350 msgr2=0x7fe770055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe770053350 0x7fe770055810 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fe79406aa90 tx=0x7fe784012000).stop 2021-07-11T03:53:24.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 -- 172.21.2.104:0/1744488324 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7940b3710 msgr2=0x7fe7940ad3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7940b3710 0x7fe7940ad3c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe78c003310 tx=0x7fe78c0091a0).stop 2021-07-11T03:53:24.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 -- 172.21.2.104:0/1744488324 shutdown_connections 2021-07-11T03:53:24.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe770053350 0x7fe770055810 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7940b3710 0x7fe7940ad3c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe7940b05b0 0x7fe7940aced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 --2- 172.21.2.104:0/1744488324 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe7940a9c10 0x7fe79414ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 -- 172.21.2.104:0/1744488324 >> 172.21.2.104:0/1744488324 conn(0x7fe7940607c0 msgr2=0x7fe7940b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 -- 172.21.2.104:0/1744488324 shutdown_connections 2021-07-11T03:53:24.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.356+0000 7fe7998eb700 1 -- 172.21.2.104:0/1744488324 wait complete. 2021-07-11T03:53:24.370 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:53:24.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.424+0000 7f5d6909d700 1 Processor -- start 2021-07-11T03:53:24.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.424+0000 7f5d6909d700 1 -- start start 2021-07-11T03:53:24.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.424+0000 7f5d6909d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5d64146950 0x7f5d64140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.424+0000 7f5d6909d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5d640b4180 0x7f5d64140720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.424+0000 7f5d6909d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d64143b60 con 0x7f5d640b4690 2021-07-11T03:53:24.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.424+0000 7f5d6909d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d64147510 con 0x7f5d640b4180 2021-07-11T03:53:24.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.424+0000 7f5d6909d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d64143e00 con 0x7f5d64146950 2021-07-11T03:53:24.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5d640b4180 0x7f5d64140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6359e700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f5d640b4690 0x7f5d640b4ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38550/0 (socket says 172.21.2.104:38550) 2021-07-11T03:53:24.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6359e700 1 -- 172.21.2.104:0/3970380984 learned_addr learned my addr 172.21.2.104:0/3970380984 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:24.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 -- 172.21.2.104:0/3970380984 >> v2:172.21.2.105:3300/0 conn(0x7f5d64146950 msgr2=0x7f5d64140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:24.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 --2- 172.21.2.104:0/3970380984 >> v2:172.21.2.105:3300/0 conn(0x7f5d64146950 0x7f5d64140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 -- 172.21.2.104:0/3970380984 >> v1:172.21.2.104:6789/0 conn(0x7f5d640b4690 legacy=0x7f5d640b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 -- 172.21.2.104:0/3970380984 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d640b0c90 con 0x7f5d640b4180 2021-07-11T03:53:24.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 --2- 172.21.2.104:0/3970380984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d640b4180 0x7f5d64140720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5d4c002f70 tx=0x7f5d4c006df0).ready entity=mon.0 client_cookie=c5e1f6ee54b08643 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d61d9b700 1 -- 172.21.2.104:0/3970380984 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d4c00d5c0 con 0x7f5d640b4180 2021-07-11T03:53:24.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d61d9b700 1 -- 172.21.2.104:0/3970380984 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c000e90 con 0x7f5d640b4180 2021-07-11T03:53:24.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d61d9b700 1 -- 172.21.2.104:0/3970380984 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d4c00d910 con 0x7f5d640b4180 2021-07-11T03:53:24.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 -- 172.21.2.104:0/3970380984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d640b4180 msgr2=0x7f5d64140720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 --2- 172.21.2.104:0/3970380984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d640b4180 0x7f5d64140720 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5d4c002f70 tx=0x7f5d4c006df0).stop 2021-07-11T03:53:24.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 -- 172.21.2.104:0/3970380984 shutdown_connections 2021-07-11T03:53:24.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 --2- 172.21.2.104:0/3970380984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d640b4180 0x7f5d64140720 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 --2- 172.21.2.104:0/3970380984 >> v2:172.21.2.105:3300/0 conn(0x7f5d64146950 0x7f5d64140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 -- 172.21.2.104:0/3970380984 >> 172.21.2.104:0/3970380984 conn(0x7f5d6405c490 msgr2=0x7f5d6405c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 -- 172.21.2.104:0/3970380984 shutdown_connections 2021-07-11T03:53:24.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 -- 172.21.2.104:0/3970380984 wait complete. 2021-07-11T03:53:24.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 Processor -- start 2021-07-11T03:53:24.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 -- start start 2021-07-11T03:53:24.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d640b4180 0x7f5d64147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d640b4690 0x7f5d640b5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d64146950 0x7f5d640b5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d640b9130 con 0x7f5d640b4690 2021-07-11T03:53:24.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d640b6170 con 0x7f5d64146950 2021-07-11T03:53:24.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d640b6a50 con 0x7f5d640b4180 2021-07-11T03:53:24.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d640b4690 0x7f5d640b5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d640b4690 0x7f5d640b5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41790/0 (socket says 172.21.2.104:41790) 2021-07-11T03:53:24.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 -- 172.21.2.104:0/2879119560 learned_addr learned my addr 172.21.2.104:0/2879119560 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:24.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 -- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d640b4180 msgr2=0x7f5d64147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d640b4180 0x7f5d64147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 -- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d64146950 msgr2=0x7f5d640b5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d64146950 0x7f5d640b5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 -- 172.21.2.104:0/2879119560 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c00b180 con 0x7f5d640b4690 2021-07-11T03:53:24.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6259c700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d640b4690 0x7f5d640b5790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f5d64149170 tx=0x7f5d4c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d5bfff700 1 -- 172.21.2.104:0/2879119560 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d4c006b20 con 0x7f5d640b4690 2021-07-11T03:53:24.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d5bfff700 1 -- 172.21.2.104:0/2879119560 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c01e430 con 0x7f5d640b4690 2021-07-11T03:53:24.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d5bfff700 1 -- 172.21.2.104:0/2879119560 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d4c01e790 con 0x7f5d640b4690 2021-07-11T03:53:24.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 -- 172.21.2.104:0/2879119560 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d640b5050 con 0x7f5d640b4690 2021-07-11T03:53:24.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.428+0000 7f5d6909d700 1 -- 172.21.2.104:0/2879119560 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d64148db0 con 0x7f5d640b4690 2021-07-11T03:53:24.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.432+0000 7f5d6909d700 1 -- 172.21.2.104:0/2879119560 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d640b6c50 con 0x7f5d640b4690 2021-07-11T03:53:24.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.432+0000 7f5d5bfff700 1 -- 172.21.2.104:0/2879119560 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5d4c00f030 con 0x7f5d640b4690 2021-07-11T03:53:24.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.432+0000 7f5d5bfff700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d48053330 0x7f5d480557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.432+0000 7f5d5bfff700 1 -- 172.21.2.104:0/2879119560 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5d4c06c5c0 con 0x7f5d640b4690 2021-07-11T03:53:24.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.432+0000 7f5d62d9d700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d48053330 0x7f5d480557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.432+0000 7f5d62d9d700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d48053330 0x7f5d480557f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5d54002f70 tx=0x7f5d540069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.432+0000 7f5d5bfff700 1 -- 172.21.2.104:0/2879119560 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5d4c0438a0 con 0x7f5d640b4690 2021-07-11T03:53:24.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 -- 172.21.2.104:0/2879119560 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5d64057350 con 0x7f5d640b4690 2021-07-11T03:53:24.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d5bfff700 1 -- 172.21.2.104:0/2879119560 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5d4c046ec0 con 0x7f5d640b4690 2021-07-11T03:53:24.520 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:24.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 -- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d48053330 msgr2=0x7f5d480557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d48053330 0x7f5d480557f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5d54002f70 tx=0x7f5d540069e0).stop 2021-07-11T03:53:24.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 -- 172.21.2.104:0/2879119560 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d640b4690 msgr2=0x7f5d640b5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d640b4690 0x7f5d640b5790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f5d64149170 tx=0x7f5d4c014000).stop 2021-07-11T03:53:24.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 -- 172.21.2.104:0/2879119560 shutdown_connections 2021-07-11T03:53:24.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d64146950 0x7f5d640b5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d48053330 0x7f5d480557f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d640b4690 0x7f5d640b5790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 --2- 172.21.2.104:0/2879119560 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d640b4180 0x7f5d64147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 -- 172.21.2.104:0/2879119560 >> 172.21.2.104:0/2879119560 conn(0x7f5d6405c490 msgr2=0x7f5d640b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 -- 172.21.2.104:0/2879119560 shutdown_connections 2021-07-11T03:53:24.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.516+0000 7f5d6909d700 1 -- 172.21.2.104:0/2879119560 wait complete. 2021-07-11T03:53:24.529 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2021-07-11T03:53:24.530 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2021-07-11T03:53:24.530 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2021-07-11T03:53:24.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 Processor -- start 2021-07-11T03:53:24.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- start start 2021-07-11T03:53:24.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d8067120 0x7fd7d80b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd7d813fff0 0x7fd7d8140480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d80b9bd0 con 0x7fd7d81439b0 2021-07-11T03:53:24.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d8143080 con 0x7fd7d813fff0 2021-07-11T03:53:24.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d8067500 con 0x7fd7d8067120 2021-07-11T03:53:24.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d77fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd7d813fff0 0x7fd7d8140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d77fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd7d813fff0 0x7fd7d8140480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41794/0 (socket says 172.21.2.104:41794) 2021-07-11T03:53:24.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d77fe700 1 -- 172.21.2.104:0/2070891791 learned_addr learned my addr 172.21.2.104:0/2070891791 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:24.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d77fe700 1 -- 172.21.2.104:0/2070891791 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d8067120 msgr2=0x7fd7d80b96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:24.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d77fe700 1 --2- 172.21.2.104:0/2070891791 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d8067120 0x7fd7d80b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d77fe700 1 -- 172.21.2.104:0/2070891791 >> v1:172.21.2.105:6789/0 conn(0x7fd7d81439b0 legacy=0x7fd7d80aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:24.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d77fe700 1 -- 172.21.2.104:0/2070891791 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d80af4d0 con 0x7fd7d813fff0 2021-07-11T03:53:24.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d77fe700 1 --2- 172.21.2.104:0/2070891791 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d813fff0 0x7fd7d8140480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd7d80b0b40 tx=0x7fd7d0009290).ready entity=mon.0 client_cookie=119614d3e20ad4f9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d5ffb700 1 -- 172.21.2.104:0/2070891791 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d000b860 con 0x7fd7d813fff0 2021-07-11T03:53:24.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d5ffb700 1 -- 172.21.2.104:0/2070891791 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d000b9c0 con 0x7fd7d813fff0 2021-07-11T03:53:24.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d5ffb700 1 -- 172.21.2.104:0/2070891791 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7d000bb70 con 0x7fd7d813fff0 2021-07-11T03:53:24.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- 172.21.2.104:0/2070891791 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d813fff0 msgr2=0x7fd7d8140480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 --2- 172.21.2.104:0/2070891791 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d813fff0 0x7fd7d8140480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd7d80b0b40 tx=0x7fd7d0009290).stop 2021-07-11T03:53:24.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- 172.21.2.104:0/2070891791 shutdown_connections 2021-07-11T03:53:24.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 --2- 172.21.2.104:0/2070891791 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d813fff0 0x7fd7d8140480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 --2- 172.21.2.104:0/2070891791 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d8067120 0x7fd7d80b96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- 172.21.2.104:0/2070891791 >> 172.21.2.104:0/2070891791 conn(0x7fd7d805f9f0 msgr2=0x7fd7d80616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- 172.21.2.104:0/2070891791 shutdown_connections 2021-07-11T03:53:24.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- 172.21.2.104:0/2070891791 wait complete. 2021-07-11T03:53:24.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 Processor -- start 2021-07-11T03:53:24.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- start start 2021-07-11T03:53:24.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7d8067120 0x7fd7d80b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d813fff0 0x7fd7d80b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d81439b0 0x7fd7d80b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d8146320 con 0x7fd7d813fff0 2021-07-11T03:53:24.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d80b2210 con 0x7fd7d8067120 2021-07-11T03:53:24.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7dd835700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d80b93a0 con 0x7fd7d81439b0 2021-07-11T03:53:24.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d813fff0 0x7fd7d80b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d813fff0 0x7fd7d80b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41802/0 (socket says 172.21.2.104:41802) 2021-07-11T03:53:24.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.584+0000 7fd7d67fc700 1 -- 172.21.2.104:0/25486338 learned_addr learned my addr 172.21.2.104:0/25486338 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:24.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7d67fc700 1 -- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d81439b0 msgr2=0x7fd7d80b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7d67fc700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d81439b0 0x7fd7d80b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7d67fc700 1 -- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7d8067120 msgr2=0x7fd7d80b73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7d67fc700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7d8067120 0x7fd7d80b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7d67fc700 1 -- 172.21.2.104:0/25486338 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d0009530 con 0x7fd7d813fff0 2021-07-11T03:53:24.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7d67fc700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d813fff0 0x7fd7d80b3b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fd7c0002f10 tx=0x7fd7c0006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7cffff700 1 -- 172.21.2.104:0/25486338 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c000f510 con 0x7fd7d813fff0 2021-07-11T03:53:24.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7cffff700 1 -- 172.21.2.104:0/25486338 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c000f670 con 0x7fd7d813fff0 2021-07-11T03:53:24.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7dd835700 1 -- 172.21.2.104:0/25486338 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d80b0b80 con 0x7fd7d813fff0 2021-07-11T03:53:24.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7cffff700 1 -- 172.21.2.104:0/25486338 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c000f7e0 con 0x7fd7d813fff0 2021-07-11T03:53:24.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7dd835700 1 -- 172.21.2.104:0/25486338 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d80b8ae0 con 0x7fd7d813fff0 2021-07-11T03:53:24.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7dd835700 1 -- 172.21.2.104:0/25486338 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d80b23d0 con 0x7fd7d813fff0 2021-07-11T03:53:24.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7cffff700 1 -- 172.21.2.104:0/25486338 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd7c0017020 con 0x7fd7d813fff0 2021-07-11T03:53:24.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7cffff700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7bc053160 0x7fd7bc055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7cffff700 1 -- 172.21.2.104:0/25486338 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd7c006ac40 con 0x7fd7d813fff0 2021-07-11T03:53:24.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7d6ffd700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7bc053160 0x7fd7bc055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.588+0000 7fd7d6ffd700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7bc053160 0x7fd7bc055620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd7c8002f70 tx=0x7fd7c8006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.592+0000 7fd7cffff700 1 -- 172.21.2.104:0/25486338 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd7c0041f80 con 0x7fd7d813fff0 2021-07-11T03:53:24.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 -- 172.21.2.104:0/25486338 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fd7d8057350 con 0x7fd7d813fff0 2021-07-11T03:53:24.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7cffff700 1 -- 172.21.2.104:0/25486338 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+130855 (secure 0 0 0) 0x7fd7c0015070 con 0x7fd7d813fff0 2021-07-11T03:53:24.698 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:24.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 -- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7bc053160 msgr2=0x7fd7bc055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7bc053160 0x7fd7bc055620 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fd7c8002f70 tx=0x7fd7c8006a80).stop 2021-07-11T03:53:24.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 -- 172.21.2.104:0/25486338 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d813fff0 msgr2=0x7fd7d80b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d813fff0 0x7fd7d80b3b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fd7c0002f10 tx=0x7fd7c0006d70).stop 2021-07-11T03:53:24.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 -- 172.21.2.104:0/25486338 shutdown_connections 2021-07-11T03:53:24.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d81439b0 0x7fd7d80b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7bc053160 0x7fd7bc055620 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d813fff0 0x7fd7d80b3b60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 --2- 172.21.2.104:0/25486338 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7d8067120 0x7fd7d80b73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 -- 172.21.2.104:0/25486338 >> 172.21.2.104:0/25486338 conn(0x7fd7d805f9f0 msgr2=0x7fd7d80616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 -- 172.21.2.104:0/25486338 shutdown_connections 2021-07-11T03:53:24.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.692+0000 7fd7dd835700 1 -- 172.21.2.104:0/25486338 wait complete. 2021-07-11T03:53:24.704 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6800","nonce":15271},{"type":"v1","addr":"172.21.2.105:6801","nonce":15271}]},"active_addr":"172.21.2.105:6801/15271","active_change":"2021-07-11T03:53:10.356014+0000","active_mgr_features":4540138292840890367,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138292840890367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.2.105:0","nonce":2168655493}]},{"addrvec":[{"type":"v2","addr":"172.21.2.105:0","nonce":2933309231}]},{"addrvec":[{"type":"v2","addr":"172.21.2.105:0","nonce":865452680}]}]}} 2021-07-11T03:53:24.710 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2021-07-11T03:53:24.711 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2021-07-11T03:53:24.711 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:24.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 Processor -- start 2021-07-11T03:53:24.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 -- start start 2021-07-11T03:53:24.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0f940a9da0 0x7f0f940b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f940ac370 con 0x7f0f940aa860 2021-07-11T03:53:24.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f940aac40 con 0x7f0f940ad150 2021-07-11T03:53:24.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f940abb50 con 0x7f0f940a9da0 2021-07-11T03:53:24.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f998fd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f0f940aa860 0x7f0f940aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38576/0 (socket says 172.21.2.104:38576) 2021-07-11T03:53:24.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f998fd700 1 -- 172.21.2.104:0/3670261364 learned_addr learned my addr 172.21.2.104:0/3670261364 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:24.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194587505 0 0) 0x7f0f940ac370 con 0x7f0f940aa860 2021-07-11T03:53:24.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f7c003260 con 0x7f0f940aa860 2021-07-11T03:53:24.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9a0fe700 1 --2- 172.21.2.104:0/3670261364 >> v2:172.21.2.105:3300/0 conn(0x7f0f940a9da0 0x7f0f940b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2206293870 0 0) 0x7f0f7c003260 con 0x7f0f940aa860 2021-07-11T03:53:24.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 >> v2:172.21.2.105:3300/0 conn(0x7f0f940a9da0 msgr2=0x7f0f940b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:24.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 --2- 172.21.2.104:0/3670261364 >> v2:172.21.2.105:3300/0 conn(0x7f0f940a9da0 0x7f0f940b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 >> v1:172.21.2.105:6789/0 conn(0x7f0f940ad150 legacy=0x7f0f94140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:24.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f940b16b0 con 0x7f0f940aa860 2021-07-11T03:53:24.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f0f90001660 con 0x7f0f940aa860 2021-07-11T03:53:24.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9a0fe700 1 --2- 172.21.2.104:0/3670261364 >> v2:172.21.2.105:3300/0 conn(0x7f0f940a9da0 0x7f0f940b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:24.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 >> v1:172.21.2.104:6789/0 conn(0x7f0f940aa860 legacy=0x7f0f940aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f7c003750 0x7f0f7c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 0x7f0f7c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f7c004f90 0x7f0f7c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c00be50 con 0x7f0f7c004500 2021-07-11T03:53:24.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c00bfc0 con 0x7f0f7c003750 2021-07-11T03:53:24.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c00c180 con 0x7f0f7c004f90 2021-07-11T03:53:24.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f990fc700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 0x7f0f7c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f998fd700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f7c003750 0x7f0f7c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f990fc700 1 -- 172.21.2.104:0/3670261364 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f7c004f90 msgr2=0x7f0f7c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:24.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9a0fe700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f7c004f90 0x7f0f7c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f990fc700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f7c004f90 0x7f0f7c00b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f990fc700 1 -- 172.21.2.104:0/3670261364 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f7c003750 msgr2=0x7f0f7c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:24.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f990fc700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f7c003750 0x7f0f7c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f990fc700 1 -- 172.21.2.104:0/3670261364 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c003260 con 0x7f0f7c004500 2021-07-11T03:53:24.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9a0fe700 1 -- 172.21.2.104:0/3670261364 reap_dead start 2021-07-11T03:53:24.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f998fd700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f7c003750 0x7f0f7c003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:24.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f990fc700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 0x7f0f7c004980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0f88001fa0 tx=0x7f0f88007a00).ready entity=mon.0 client_cookie=206a044c1ed8c684 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f88007030 con 0x7f0f7c004500 2021-07-11T03:53:24.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f88002e90 con 0x7f0f7c004500 2021-07-11T03:53:24.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 -- 172.21.2.104:0/3670261364 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 msgr2=0x7f0f7c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 0x7f0f7c004980 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0f88001fa0 tx=0x7f0f88007a00).stop 2021-07-11T03:53:24.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f988fb700 1 -- 172.21.2.104:0/3670261364 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f880073a0 con 0x7f0f7c004500 2021-07-11T03:53:24.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 -- 172.21.2.104:0/3670261364 shutdown_connections 2021-07-11T03:53:24.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 --2- 172.21.2.104:0/3670261364 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 0x7f0f7c004980 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 -- 172.21.2.104:0/3670261364 >> 172.21.2.104:0/3670261364 conn(0x7f0f940b64d0 msgr2=0x7f0f940b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 -- 172.21.2.104:0/3670261364 shutdown_connections 2021-07-11T03:53:24.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.760+0000 7f0f9bb61700 1 -- 172.21.2.104:0/3670261364 wait complete. 2021-07-11T03:53:24.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f9bb61700 1 Processor -- start 2021-07-11T03:53:24.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f9bb61700 1 -- start start 2021-07-11T03:53:24.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f9bb61700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 0x7f0f940ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f9bb61700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f94156a20 0x7f0f94156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f9bb61700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f94159d50 0x7f0f9415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f9bb61700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f94156470 con 0x7f0f7c004500 2021-07-11T03:53:24.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f9bb61700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9409ffb0 con 0x7f0f94159d50 2021-07-11T03:53:24.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f9bb61700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f940642e0 con 0x7f0f94156a20 2021-07-11T03:53:24.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f998fd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 0x7f0f940ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f998fd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 0x7f0f940ae8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41820/0 (socket says 172.21.2.104:41820) 2021-07-11T03:53:24.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f998fd700 1 -- 172.21.2.104:0/465493153 learned_addr learned my addr 172.21.2.104:0/465493153 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:24.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f998fd700 1 -- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f94156a20 msgr2=0x7f0f94156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f998fd700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f94156a20 0x7f0f94156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f998fd700 1 -- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f94159d50 msgr2=0x7f0f9415a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f998fd700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f94159d50 0x7f0f9415a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f998fd700 1 -- 172.21.2.104:0/465493153 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f94064cf0 con 0x7f0f7c004500 2021-07-11T03:53:24.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f998fd700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 0x7f0f940ae8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f0f94154000 tx=0x7f0f90009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f86ffd700 1 -- 172.21.2.104:0/465493153 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f90007170 con 0x7f0f7c004500 2021-07-11T03:53:24.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f86ffd700 1 -- 172.21.2.104:0/465493153 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f900072d0 con 0x7f0f7c004500 2021-07-11T03:53:24.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f9bb61700 1 -- 172.21.2.104:0/465493153 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f94157390 con 0x7f0f7c004500 2021-07-11T03:53:24.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f86ffd700 1 -- 172.21.2.104:0/465493153 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f90004920 con 0x7f0f7c004500 2021-07-11T03:53:24.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f9bb61700 1 -- 172.21.2.104:0/465493153 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f940647b0 con 0x7f0f7c004500 2021-07-11T03:53:24.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f9bb61700 1 -- 172.21.2.104:0/465493153 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f94159a40 con 0x7f0f7c004500 2021-07-11T03:53:24.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f86ffd700 1 -- 172.21.2.104:0/465493153 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0f9001b060 con 0x7f0f7c004500 2021-07-11T03:53:24.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f86ffd700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f74053300 0x7f0f740557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f86ffd700 1 -- 172.21.2.104:0/465493153 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0f9006b1c0 con 0x7f0f7c004500 2021-07-11T03:53:24.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.764+0000 7f0f990fc700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f74053300 0x7f0f740557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.768+0000 7f0f990fc700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f74053300 0x7f0f740557c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f0f88001fa0 tx=0x7f0f88015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.768+0000 7f0f86ffd700 1 -- 172.21.2.104:0/465493153 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0f900424a0 con 0x7f0f7c004500 2021-07-11T03:53:24.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 -- 172.21.2.104:0/465493153 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f94057350 con 0x7f0f7c004500 2021-07-11T03:53:24.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f86ffd700 1 -- 172.21.2.104:0/465493153 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f0f90019070 con 0x7f0f7c004500 2021-07-11T03:53:24.855 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:24.855 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":23,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:24.367517+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:24.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 -- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f74053300 msgr2=0x7f0f740557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f74053300 0x7f0f740557c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f0f88001fa0 tx=0x7f0f88015000).stop 2021-07-11T03:53:24.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 -- 172.21.2.104:0/465493153 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 msgr2=0x7f0f940ae8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 0x7f0f940ae8e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f0f94154000 tx=0x7f0f90009040).stop 2021-07-11T03:53:24.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 -- 172.21.2.104:0/465493153 shutdown_connections 2021-07-11T03:53:24.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f94159d50 0x7f0f9415a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f94156a20 0x7f0f94156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f74053300 0x7f0f740557c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 --2- 172.21.2.104:0/465493153 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f7c004500 0x7f0f940ae8e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 -- 172.21.2.104:0/465493153 >> 172.21.2.104:0/465493153 conn(0x7f0f940b64d0 msgr2=0x7f0f94061520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 -- 172.21.2.104:0/465493153 shutdown_connections 2021-07-11T03:53:24.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.852+0000 7f0f9bb61700 1 -- 172.21.2.104:0/465493153 wait complete. 2021-07-11T03:53:24.862 INFO:tasks.ceph.ceph_manager.ceph:all up! 2021-07-11T03:53:24.863 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:24.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 Processor -- start 2021-07-11T03:53:24.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 -- start start 2021-07-11T03:53:24.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f06180b1170 0x7f0618141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0618141920 0x7f0618141db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06180b31d0 con 0x7f06180b7810 2021-07-11T03:53:24.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06181422a0 con 0x7f0618141920 2021-07-11T03:53:24.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06180b7bf0 con 0x7f06180b1170 2021-07-11T03:53:24.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f061ed7b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0618141920 0x7f0618141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f061ed7b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0618141920 0x7f0618141db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41830/0 (socket says 172.21.2.104:41830) 2021-07-11T03:53:24.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f061ed7b700 1 -- 172.21.2.104:0/3120989599 learned_addr learned my addr 172.21.2.104:0/3120989599 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:24.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f061ed7b700 1 -- 172.21.2.104:0/3120989599 >> v2:172.21.2.105:3300/0 conn(0x7f06180b1170 msgr2=0x7f0618141430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:24.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f061ed7b700 1 --2- 172.21.2.104:0/3120989599 >> v2:172.21.2.105:3300/0 conn(0x7f06180b1170 0x7f0618141430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f061ed7b700 1 -- 172.21.2.104:0/3120989599 >> v1:172.21.2.105:6789/0 conn(0x7f06180b7810 legacy=0x7f06181445a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:24.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f061ed7b700 1 -- 172.21.2.104:0/3120989599 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06180675c0 con 0x7f0618141920 2021-07-11T03:53:24.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f061ed7b700 1 --2- 172.21.2.104:0/3120989599 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0618141920 0x7f0618141db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f0618067ca0 tx=0x7f0610009290).ready entity=mon.0 client_cookie=faa181b5981373d5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f061d578700 1 -- 172.21.2.104:0/3120989599 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f061000b830 con 0x7f0618141920 2021-07-11T03:53:24.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f061d578700 1 -- 172.21.2.104:0/3120989599 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061000b990 con 0x7f0618141920 2021-07-11T03:53:24.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f061d578700 1 -- 172.21.2.104:0/3120989599 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f061000bb40 con 0x7f0618141920 2021-07-11T03:53:24.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 -- 172.21.2.104:0/3120989599 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0618141920 msgr2=0x7f0618141db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 --2- 172.21.2.104:0/3120989599 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0618141920 0x7f0618141db0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f0618067ca0 tx=0x7f0610009290).stop 2021-07-11T03:53:24.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 -- 172.21.2.104:0/3120989599 shutdown_connections 2021-07-11T03:53:24.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 --2- 172.21.2.104:0/3120989599 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0618141920 0x7f0618141db0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:24.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 --2- 172.21.2.104:0/3120989599 >> v2:172.21.2.105:3300/0 conn(0x7f06180b1170 0x7f0618141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 -- 172.21.2.104:0/3120989599 >> 172.21.2.104:0/3120989599 conn(0x7f061805f6b0 msgr2=0x7f061805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:24.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 -- 172.21.2.104:0/3120989599 shutdown_connections 2021-07-11T03:53:24.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 -- 172.21.2.104:0/3120989599 wait complete. 2021-07-11T03:53:24.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 Processor -- start 2021-07-11T03:53:24.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.912+0000 7f06207de700 1 -- start start 2021-07-11T03:53:24.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06207de700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f06180b1170 0x7f0618061a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06207de700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06180b7810 0x7f0618151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06207de700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0618141920 0x7f0618151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06207de700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061814ef90 con 0x7f06180b7810 2021-07-11T03:53:24.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06207de700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061809f220 con 0x7f06180b1170 2021-07-11T03:53:24.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06207de700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061809f4c0 con 0x7f0618141920 2021-07-11T03:53:24.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f061dd79700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06180b7810 0x7f0618151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f061dd79700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06180b7810 0x7f0618151aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41834/0 (socket says 172.21.2.104:41834) 2021-07-11T03:53:24.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f061dd79700 1 -- 172.21.2.104:0/446857004 learned_addr learned my addr 172.21.2.104:0/446857004 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:24.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f061dd79700 1 -- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0618141920 msgr2=0x7f0618151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f061dd79700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0618141920 0x7f0618151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f061dd79700 1 -- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f06180b1170 msgr2=0x7f0618061a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:24.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f061dd79700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f06180b1170 0x7f0618061a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:24.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f061dd79700 1 -- 172.21.2.104:0/446857004 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0610009530 con 0x7f06180b7810 2021-07-11T03:53:24.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f061dd79700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06180b7810 0x7f0618151aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0608002fb0 tx=0x7f0608006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06077fe700 1 -- 172.21.2.104:0/446857004 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0608013460 con 0x7f06180b7810 2021-07-11T03:53:24.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06077fe700 1 -- 172.21.2.104:0/446857004 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06080135c0 con 0x7f06180b7810 2021-07-11T03:53:24.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06207de700 1 -- 172.21.2.104:0/446857004 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061809f760 con 0x7f06180b7810 2021-07-11T03:53:24.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06207de700 1 -- 172.21.2.104:0/446857004 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06180adae0 con 0x7f06180b7810 2021-07-11T03:53:24.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06077fe700 1 -- 172.21.2.104:0/446857004 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0608013920 con 0x7f06180b7810 2021-07-11T03:53:24.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06207de700 1 -- 172.21.2.104:0/446857004 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0618057350 con 0x7f06180b7810 2021-07-11T03:53:24.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06077fe700 1 -- 172.21.2.104:0/446857004 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f060800f070 con 0x7f06180b7810 2021-07-11T03:53:24.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06077fe700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05f8053260 0x7f05f8055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:24.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f06077fe700 1 -- 172.21.2.104:0/446857004 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f060806af10 con 0x7f06180b7810 2021-07-11T03:53:24.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.916+0000 7f061e57a700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05f8053260 0x7f05f8055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:24.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.920+0000 7f061e57a700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05f8053260 0x7f05f8055720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f0614002fb0 tx=0x7f06140069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:24.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:24.920+0000 7f06077fe700 1 -- 172.21.2.104:0/446857004 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0608042250 con 0x7f06180b7810 2021-07-11T03:53:25.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.000+0000 7f06207de700 1 -- 172.21.2.104:0/446857004 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06180698e0 con 0x7f06180b7810 2021-07-11T03:53:25.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.000+0000 7f06077fe700 1 -- 172.21.2.104:0/446857004 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f0608045870 con 0x7f06180b7810 2021-07-11T03:53:25.005 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:25.006 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":23,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:24.367517+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:25.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 -- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05f8053260 msgr2=0x7f05f8055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05f8053260 0x7f05f8055720 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f0614002fb0 tx=0x7f06140069d0).stop 2021-07-11T03:53:25.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 -- 172.21.2.104:0/446857004 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06180b7810 msgr2=0x7f0618151aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06180b7810 0x7f0618151aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f0608002fb0 tx=0x7f0608006df0).stop 2021-07-11T03:53:25.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 -- 172.21.2.104:0/446857004 shutdown_connections 2021-07-11T03:53:25.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0618141920 0x7f0618151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05f8053260 0x7f05f8055720 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06180b7810 0x7f0618151aa0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 --2- 172.21.2.104:0/446857004 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f06180b1170 0x7f0618061a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 -- 172.21.2.104:0/446857004 >> 172.21.2.104:0/446857004 conn(0x7f061805f6b0 msgr2=0x7f06180b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 -- 172.21.2.104:0/446857004 shutdown_connections 2021-07-11T03:53:25.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.004+0000 7f06207de700 1 -- 172.21.2.104:0/446857004 wait complete. 2021-07-11T03:53:25.013 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2021-07-11T03:53:25.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.060+0000 7fad7fccf700 1 Processor -- start 2021-07-11T03:53:25.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.060+0000 7fad7fccf700 1 -- start start 2021-07-11T03:53:25.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.060+0000 7fad7fccf700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fad78060910 0x7fad780acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.060+0000 7fad7fccf700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad780aa5d0 0x7fad7813c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.060+0000 7fad7fccf700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad780b6ce0 con 0x7fad780b6900 2021-07-11T03:53:25.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.060+0000 7fad7fccf700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad780611e0 con 0x7fad78060910 2021-07-11T03:53:25.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.060+0000 7fad7fccf700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad78061480 con 0x7fad780aa5d0 2021-07-11T03:53:25.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.060+0000 7fad7d26a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fad78060910 0x7fad780acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.060+0000 7fad7d26a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fad78060910 0x7fad780acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41846/0 (socket says 172.21.2.104:41846) 2021-07-11T03:53:25.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.060+0000 7fad7d26a700 1 -- 172.21.2.104:0/2753270257 learned_addr learned my addr 172.21.2.104:0/2753270257 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:25.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7d26a700 1 -- 172.21.2.104:0/2753270257 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad780aa5d0 msgr2=0x7fad7813c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7d26a700 1 --2- 172.21.2.104:0/2753270257 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad780aa5d0 0x7fad7813c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7d26a700 1 -- 172.21.2.104:0/2753270257 >> v1:172.21.2.105:6789/0 conn(0x7fad780b6900 legacy=0x7fad780a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7d26a700 1 -- 172.21.2.104:0/2753270257 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad78061720 con 0x7fad78060910 2021-07-11T03:53:25.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7d26a700 1 --2- 172.21.2.104:0/2753270257 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad78060910 0x7fad780acd50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fad78060300 tx=0x7fad74006f40).ready entity=mon.0 client_cookie=6e7f66f32283d4d4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7ca69700 1 -- 172.21.2.104:0/2753270257 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad7400f940 con 0x7fad78060910 2021-07-11T03:53:25.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7ca69700 1 -- 172.21.2.104:0/2753270257 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad74000e90 con 0x7fad78060910 2021-07-11T03:53:25.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7ca69700 1 -- 172.21.2.104:0/2753270257 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad7400fc90 con 0x7fad78060910 2021-07-11T03:53:25.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- 172.21.2.104:0/2753270257 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad78060910 msgr2=0x7fad780acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 --2- 172.21.2.104:0/2753270257 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad78060910 0x7fad780acd50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fad78060300 tx=0x7fad74006f40).stop 2021-07-11T03:53:25.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- 172.21.2.104:0/2753270257 shutdown_connections 2021-07-11T03:53:25.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 --2- 172.21.2.104:0/2753270257 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad780aa5d0 0x7fad7813c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 --2- 172.21.2.104:0/2753270257 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad78060910 0x7fad780acd50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- 172.21.2.104:0/2753270257 >> 172.21.2.104:0/2753270257 conn(0x7fad780b3480 msgr2=0x7fad780b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- 172.21.2.104:0/2753270257 shutdown_connections 2021-07-11T03:53:25.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- 172.21.2.104:0/2753270257 wait complete. 2021-07-11T03:53:25.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 Processor -- start 2021-07-11T03:53:25.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- start start 2021-07-11T03:53:25.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad78060910 0x7fad780af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad780aa5d0 0x7fad78151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad780b6900 0x7fad780ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78133d90 con 0x7fad78060910 2021-07-11T03:53:25.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78143b50 con 0x7fad780b6900 2021-07-11T03:53:25.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad781523e0 con 0x7fad780aa5d0 2021-07-11T03:53:25.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7da6b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad78060910 0x7fad780af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7da6b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad78060910 0x7fad780af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41852/0 (socket says 172.21.2.104:41852) 2021-07-11T03:53:25.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7da6b700 1 -- 172.21.2.104:0/3447674824 learned_addr learned my addr 172.21.2.104:0/3447674824 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:25.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7da6b700 1 -- 172.21.2.104:0/3447674824 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad780aa5d0 msgr2=0x7fad78151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7da6b700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad780aa5d0 0x7fad78151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7da6b700 1 -- 172.21.2.104:0/3447674824 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad780b6900 msgr2=0x7fad780ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7da6b700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad780b6900 0x7fad780ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7da6b700 1 -- 172.21.2.104:0/3447674824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7400a360 con 0x7fad78060910 2021-07-11T03:53:25.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7da6b700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad78060910 0x7fad780af7d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fad78152520 tx=0x7fad6c006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad6affd700 1 -- 172.21.2.104:0/3447674824 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad6c010960 con 0x7fad78060910 2021-07-11T03:53:25.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad6affd700 1 -- 172.21.2.104:0/3447674824 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c010ac0 con 0x7fad78060910 2021-07-11T03:53:25.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- 172.21.2.104:0/3447674824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad78151d10 con 0x7fad78060910 2021-07-11T03:53:25.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- 172.21.2.104:0/3447674824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad78151f40 con 0x7fad78060910 2021-07-11T03:53:25.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad6affd700 1 -- 172.21.2.104:0/3447674824 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad6c010e20 con 0x7fad78060910 2021-07-11T03:53:25.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad7fccf700 1 -- 172.21.2.104:0/3447674824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad54003660 con 0x7fad78060910 2021-07-11T03:53:25.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad6affd700 1 -- 172.21.2.104:0/3447674824 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fad6c018420 con 0x7fad78060910 2021-07-11T03:53:25.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad6affd700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad580532d0 0x7fad58055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.064+0000 7fad6affd700 1 -- 172.21.2.104:0/3447674824 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fad6c06faf0 con 0x7fad78060910 2021-07-11T03:53:25.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.068+0000 7fad6affd700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x7fad5805a7c0 0x7fad5805cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.068+0000 7fad6affd700 1 -- 172.21.2.104:0/3447674824 --> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad5805d1a0 con 0x7fad5805a7c0 2021-07-11T03:53:25.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.068+0000 7fad6affd700 1 -- 172.21.2.104:0/3447674824 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad6c06fe00 con 0x7fad78060910 2021-07-11T03:53:25.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.068+0000 7fad7e26c700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x7fad5805a7c0 0x7fad5805cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.068+0000 7fad7e26c700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x7fad5805a7c0 0x7fad5805cbe0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.068+0000 7fad7d26a700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad580532d0 0x7fad58055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.068+0000 7fad6affd700 1 -- 172.21.2.104:0/3447674824 <== osd.0 v2:172.21.2.104:6800/15411 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fad5805d1a0 con 0x7fad5805a7c0 2021-07-11T03:53:25.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.068+0000 7fad7d26a700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad580532d0 0x7fad58055790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fad78060300 tx=0x7fad74018000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7fccf700 1 -- 172.21.2.104:0/3447674824 --> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad54002810 con 0x7fad5805a7c0 2021-07-11T03:53:25.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad6affd700 1 -- 172.21.2.104:0/3447674824 <== osd.0 v2:172.21.2.104:6800/15411 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad54002810 con 0x7fad5805a7c0 2021-07-11T03:53:25.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7fccf700 1 -- 172.21.2.104:0/3447674824 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x7fad5805a7c0 msgr2=0x7fad5805cbe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7fccf700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x7fad5805a7c0 0x7fad5805cbe0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7fccf700 1 -- 172.21.2.104:0/3447674824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad580532d0 msgr2=0x7fad58055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7fccf700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad580532d0 0x7fad58055790 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fad78060300 tx=0x7fad74018000).stop 2021-07-11T03:53:25.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7fccf700 1 -- 172.21.2.104:0/3447674824 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad78060910 msgr2=0x7fad780af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7fccf700 1 --2- 172.21.2.104:0/3447674824 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad78060910 0x7fad780af7d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fad78152520 tx=0x7fad6c006d70).stop 2021-07-11T03:53:25.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7e26c700 1 -- 172.21.2.104:0/3447674824 reap_dead start 2021-07-11T03:53:25.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7fccf700 1 -- 172.21.2.104:0/3447674824 shutdown_connections 2021-07-11T03:53:25.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7fccf700 1 -- 172.21.2.104:0/3447674824 >> 172.21.2.104:0/3447674824 conn(0x7fad780b3480 msgr2=0x7fad78136460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7fccf700 1 -- 172.21.2.104:0/3447674824 shutdown_connections 2021-07-11T03:53:25.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.076+0000 7fad7fccf700 1 -- 172.21.2.104:0/3447674824 wait complete. 2021-07-11T03:53:25.088 INFO:teuthology.orchestra.run.gibba004.stdout:60129542147 2021-07-11T03:53:25.088 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2021-07-11T03:53:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f29a57c8700 1 Processor -- start 2021-07-11T03:53:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f29a57c8700 1 -- start start 2021-07-11T03:53:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f29a57c8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f29a0060100 0x7f29a0061650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f29a57c8700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a00604e0 con 0x7f29a0065e20 2021-07-11T03:53:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f29a57c8700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a0060c70 con 0x7f29a0065970 2021-07-11T03:53:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f29a57c8700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29a0060f10 con 0x7f29a0060100 2021-07-11T03:53:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299f7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f29a0065e20 0x7f29a014d9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38630/0 (socket says 172.21.2.104:38630) 2021-07-11T03:53:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299f7fe700 1 -- 172.21.2.104:0/2980147217 learned_addr learned my addr 172.21.2.104:0/2980147217 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:25.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651482431 0 0) 0x7f29a00604e0 con 0x7f29a0065e20 2021-07-11T03:53:25.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2980003260 con 0x7f29a0065e20 2021-07-11T03:53:25.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299effd700 1 --2- 172.21.2.104:0/2980147217 >> v2:172.21.2.105:3300/0 conn(0x7f29a0060100 0x7f29a0061650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3805494717 0 0) 0x7f2980003260 con 0x7f29a0065e20 2021-07-11T03:53:25.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 >> v2:172.21.2.105:3300/0 conn(0x7f29a0060100 msgr2=0x7f29a0061650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 --2- 172.21.2.104:0/2980147217 >> v2:172.21.2.105:3300/0 conn(0x7f29a0060100 0x7f29a0061650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 >> v1:172.21.2.105:6789/0 conn(0x7f29a0065970 legacy=0x7f29a00665d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a00611b0 con 0x7f29a0065e20 2021-07-11T03:53:25.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f29980019e0 con 0x7f29a0065e20 2021-07-11T03:53:25.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 >> v1:172.21.2.104:6789/0 conn(0x7f29a0065e20 legacy=0x7f29a014d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 --2- 172.21.2.104:0/2980147217 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2980003820 0x7f2980003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 --2- 172.21.2.104:0/2980147217 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 0x7f2980004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 --2- 172.21.2.104:0/2980147217 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2980005050 0x7f298000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0060f10 con 0x7f2980004500 2021-07-11T03:53:25.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298000bf40 con 0x7f2980005050 2021-07-11T03:53:25.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298000c100 con 0x7f2980003820 2021-07-11T03:53:25.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.136+0000 7f299e7fc700 1 --2- 172.21.2.104:0/2980147217 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 0x7f2980004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299e7fc700 1 -- 172.21.2.104:0/2980147217 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2980003820 msgr2=0x7f2980003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299e7fc700 1 --2- 172.21.2.104:0/2980147217 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2980003820 0x7f2980003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299e7fc700 1 -- 172.21.2.104:0/2980147217 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2980005050 msgr2=0x7f298000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299e7fc700 1 --2- 172.21.2.104:0/2980147217 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2980005050 0x7f298000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299e7fc700 1 -- 172.21.2.104:0/2980147217 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2980003260 con 0x7f2980004500 2021-07-11T03:53:25.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299f7fe700 1 -- 172.21.2.104:0/2980147217 reap_dead start 2021-07-11T03:53:25.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299e7fc700 1 --2- 172.21.2.104:0/2980147217 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 0x7f2980004980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f2990000f70 tx=0x7f29900079e0).ready entity=mon.0 client_cookie=d656c4dcdbd56b3b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2990007cf0 con 0x7f2980004500 2021-07-11T03:53:25.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2990014410 con 0x7f2980004500 2021-07-11T03:53:25.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299dffb700 1 -- 172.21.2.104:0/2980147217 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2990014770 con 0x7f2980004500 2021-07-11T03:53:25.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- 172.21.2.104:0/2980147217 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 msgr2=0x7f2980004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 --2- 172.21.2.104:0/2980147217 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 0x7f2980004980 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f2990000f70 tx=0x7f29900079e0).stop 2021-07-11T03:53:25.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- 172.21.2.104:0/2980147217 shutdown_connections 2021-07-11T03:53:25.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 --2- 172.21.2.104:0/2980147217 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 0x7f2980004980 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- 172.21.2.104:0/2980147217 >> 172.21.2.104:0/2980147217 conn(0x7f29a005cf30 msgr2=0x7f29a005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- 172.21.2.104:0/2980147217 shutdown_connections 2021-07-11T03:53:25.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- 172.21.2.104:0/2980147217 wait complete. 2021-07-11T03:53:25.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 Processor -- start 2021-07-11T03:53:25.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- start start 2021-07-11T03:53:25.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 0x7f29a01527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f29a0152ce0 0x7f29a00a9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f29a00a8ac0 0x7f29a00a9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a00a8200 con 0x7f2980004500 2021-07-11T03:53:25.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a00a84a0 con 0x7f29a0152ce0 2021-07-11T03:53:25.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a0153160 con 0x7f29a00a8ac0 2021-07-11T03:53:25.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 0x7f29a01527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 0x7f29a01527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41872/0 (socket says 172.21.2.104:41872) 2021-07-11T03:53:25.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299effd700 1 -- 172.21.2.104:0/2147980680 learned_addr learned my addr 172.21.2.104:0/2147980680 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:25.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299f7fe700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f29a00a8ac0 0x7f29a00a9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299effd700 1 -- 172.21.2.104:0/2147980680 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f29a00a8ac0 msgr2=0x7f29a00a9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299effd700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f29a00a8ac0 0x7f29a00a9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299effd700 1 -- 172.21.2.104:0/2147980680 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f29a0152ce0 msgr2=0x7f29a00a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299e7fc700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f29a0152ce0 0x7f29a00a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299effd700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f29a0152ce0 0x7f29a00a9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299effd700 1 -- 172.21.2.104:0/2147980680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a0153400 con 0x7f2980004500 2021-07-11T03:53:25.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f299effd700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 0x7f29a01527f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f2988007000 tx=0x7f2988008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f2997fff700 1 -- 172.21.2.104:0/2147980680 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f298800f4e0 con 0x7f2980004500 2021-07-11T03:53:25.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f2997fff700 1 -- 172.21.2.104:0/2147980680 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2988000e90 con 0x7f2980004500 2021-07-11T03:53:25.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- 172.21.2.104:0/2147980680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a01497c0 con 0x7f2980004500 2021-07-11T03:53:25.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- 172.21.2.104:0/2147980680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a00abba0 con 0x7f2980004500 2021-07-11T03:53:25.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f2997fff700 1 -- 172.21.2.104:0/2147980680 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f298800f830 con 0x7f2980004500 2021-07-11T03:53:25.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f29a57c8700 1 -- 172.21.2.104:0/2147980680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f29a01535e0 con 0x7f2980004500 2021-07-11T03:53:25.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.140+0000 7f2997fff700 1 -- 172.21.2.104:0/2147980680 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f298800f990 con 0x7f2980004500 2021-07-11T03:53:25.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.144+0000 7f2997fff700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2984053300 0x7f29840557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.144+0000 7f2997fff700 1 -- 172.21.2.104:0/2147980680 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f298806b060 con 0x7f2980004500 2021-07-11T03:53:25.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.144+0000 7f2997fff700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] conn(0x7f298405a6f0 0x7f298405cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.144+0000 7f2997fff700 1 -- 172.21.2.104:0/2147980680 --> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f298405d0d0 con 0x7f298405a6f0 2021-07-11T03:53:25.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.144+0000 7f2997fff700 1 -- 172.21.2.104:0/2147980680 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f298806d6f0 con 0x7f2980004500 2021-07-11T03:53:25.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.144+0000 7f299f7fe700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] conn(0x7f298405a6f0 0x7f298405cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.144+0000 7f299f7fe700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] conn(0x7f298405a6f0 0x7f298405cb10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.144+0000 7f299e7fc700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2984053300 0x7f29840557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.144+0000 7f2997fff700 1 -- 172.21.2.104:0/2147980680 <== osd.1 v2:172.21.2.104:6808/15412 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f298405d0d0 con 0x7f298405a6f0 2021-07-11T03:53:25.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.144+0000 7f299e7fc700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2984053300 0x7f29840557c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f2990017000 tx=0x7f2990015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f29a57c8700 1 -- 172.21.2.104:0/2147980680 --> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f29a0057350 con 0x7f298405a6f0 2021-07-11T03:53:25.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f2997fff700 1 -- 172.21.2.104:0/2147980680 <== osd.1 v2:172.21.2.104:6808/15412 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f29a0057350 con 0x7f298405a6f0 2021-07-11T03:53:25.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f29a57c8700 1 -- 172.21.2.104:0/2147980680 >> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] conn(0x7f298405a6f0 msgr2=0x7f298405cb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f29a57c8700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] conn(0x7f298405a6f0 0x7f298405cb10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f29a57c8700 1 -- 172.21.2.104:0/2147980680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2984053300 msgr2=0x7f29840557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f29a57c8700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2984053300 0x7f29840557c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f2990017000 tx=0x7f2990015000).stop 2021-07-11T03:53:25.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f29a57c8700 1 -- 172.21.2.104:0/2147980680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 msgr2=0x7f29a01527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f29a57c8700 1 --2- 172.21.2.104:0/2147980680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2980004500 0x7f29a01527f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f2988007000 tx=0x7f2988008040).stop 2021-07-11T03:53:25.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f299f7fe700 1 -- 172.21.2.104:0/2147980680 reap_dead start 2021-07-11T03:53:25.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f29a57c8700 1 -- 172.21.2.104:0/2147980680 shutdown_connections 2021-07-11T03:53:25.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f29a57c8700 1 -- 172.21.2.104:0/2147980680 >> 172.21.2.104:0/2147980680 conn(0x7f29a005cf30 msgr2=0x7f29a01479a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f29a57c8700 1 -- 172.21.2.104:0/2147980680 shutdown_connections 2021-07-11T03:53:25.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.152+0000 7f29a57c8700 1 -- 172.21.2.104:0/2147980680 wait complete. 2021-07-11T03:53:25.171 INFO:teuthology.orchestra.run.gibba004.stdout:60129542149 2021-07-11T03:53:25.171 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2021-07-11T03:53:25.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.224+0000 7f081c93d700 1 Processor -- start 2021-07-11T03:53:25.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.224+0000 7f081c93d700 1 -- start start 2021-07-11T03:53:25.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0818067300 0x7f08181438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08180aabb0 con 0x7f0818143de0 2021-07-11T03:53:25.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08180604e0 con 0x7f0818060100 2021-07-11T03:53:25.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0818066ea0 con 0x7f0818067300 2021-07-11T03:53:25.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0816d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f0818143de0 0x7f081814b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38650/0 (socket says 172.21.2.104:38650) 2021-07-11T03:53:25.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0816d9d700 1 -- 172.21.2.104:0/2957554969 learned_addr learned my addr 172.21.2.104:0/2957554969 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:25.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036016041 0 0) 0x7f08180aabb0 con 0x7f0818143de0 2021-07-11T03:53:25.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07f8003260 con 0x7f0818143de0 2021-07-11T03:53:25.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0815d9b700 1 --2- 172.21.2.104:0/2957554969 >> v2:172.21.2.105:3300/0 conn(0x7f0818067300 0x7f08181438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3920416749 0 0) 0x7f07f8003260 con 0x7f0818143de0 2021-07-11T03:53:25.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 >> v2:172.21.2.105:3300/0 conn(0x7f0818067300 msgr2=0x7f08181438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 --2- 172.21.2.104:0/2957554969 >> v2:172.21.2.105:3300/0 conn(0x7f0818067300 0x7f08181438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 >> v1:172.21.2.105:6789/0 conn(0x7f0818060100 legacy=0x7f08180b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08180b33b0 con 0x7f0818143de0 2021-07-11T03:53:25.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f08100019e0 con 0x7f0818143de0 2021-07-11T03:53:25.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 >> v1:172.21.2.104:6789/0 conn(0x7f0818143de0 legacy=0x7f081814b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 --2- 172.21.2.104:0/2957554969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f07f8003750 0x7f07f8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 --2- 172.21.2.104:0/2957554969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f07f8004500 0x7f07f8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 --2- 172.21.2.104:0/2957554969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 0x7f07f800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0818066ea0 con 0x7f07f8004ec0 2021-07-11T03:53:25.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f800bda0 con 0x7f07f8003750 2021-07-11T03:53:25.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f800bf60 con 0x7f07f8004500 2021-07-11T03:53:25.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0815d9b700 1 --2- 172.21.2.104:0/2957554969 >> v2:172.21.2.105:3300/0 conn(0x7f0818067300 0x7f08181438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:53:25.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0816d9d700 1 --2- 172.21.2.104:0/2957554969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 0x7f07f800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081659c700 1 --2- 172.21.2.104:0/2957554969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f07f8003750 0x7f07f8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0816d9d700 1 -- 172.21.2.104:0/2957554969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f07f8004500 msgr2=0x7f07f8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0816d9d700 1 --2- 172.21.2.104:0/2957554969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f07f8004500 0x7f07f8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0816d9d700 1 -- 172.21.2.104:0/2957554969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f07f8003750 msgr2=0x7f07f8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0816d9d700 1 --2- 172.21.2.104:0/2957554969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f07f8003750 0x7f07f8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0816d9d700 1 -- 172.21.2.104:0/2957554969 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f8003260 con 0x7f07f8004ec0 2021-07-11T03:53:25.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0816d9d700 1 -- 172.21.2.104:0/2957554969 reap_dead start 2021-07-11T03:53:25.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f0816d9d700 1 --2- 172.21.2.104:0/2957554969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 0x7f07f800b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f08100016f0 tx=0x7f08180b1990).ready entity=mon.0 client_cookie=ac601a03811a0d84 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f081000cca0 con 0x7f07f8004ec0 2021-07-11T03:53:25.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081000ce00 con 0x7f07f8004ec0 2021-07-11T03:53:25.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081559a700 1 -- 172.21.2.104:0/2957554969 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0810005b80 con 0x7f07f8004ec0 2021-07-11T03:53:25.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- 172.21.2.104:0/2957554969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 msgr2=0x7f07f800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 --2- 172.21.2.104:0/2957554969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 0x7f07f800b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f08100016f0 tx=0x7f08180b1990).stop 2021-07-11T03:53:25.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- 172.21.2.104:0/2957554969 shutdown_connections 2021-07-11T03:53:25.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 --2- 172.21.2.104:0/2957554969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 0x7f07f800b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- 172.21.2.104:0/2957554969 >> 172.21.2.104:0/2957554969 conn(0x7f081805cf30 msgr2=0x7f081805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- 172.21.2.104:0/2957554969 shutdown_connections 2021-07-11T03:53:25.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- 172.21.2.104:0/2957554969 wait complete. 2021-07-11T03:53:25.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 Processor -- start 2021-07-11T03:53:25.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- start start 2021-07-11T03:53:25.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 0x7f08181411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f08180ae900 0x7f08180aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f08180af250 0x7f08181539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081813fd00 con 0x7f07f8004ec0 2021-07-11T03:53:25.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08180afd90 con 0x7f08180af250 2021-07-11T03:53:25.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08180ab410 con 0x7f08180ae900 2021-07-11T03:53:25.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 0x7f08181411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 0x7f08181411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41890/0 (socket says 172.21.2.104:41890) 2021-07-11T03:53:25.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081659c700 1 -- 172.21.2.104:0/3612694569 learned_addr learned my addr 172.21.2.104:0/3612694569 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:25.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081659c700 1 -- 172.21.2.104:0/3612694569 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f08180ae900 msgr2=0x7f08180aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081659c700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f08180ae900 0x7f08180aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081659c700 1 -- 172.21.2.104:0/3612694569 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f08180af250 msgr2=0x7f08181539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081659c700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f08180af250 0x7f08181539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081659c700 1 -- 172.21.2.104:0/3612694569 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08180acf40 con 0x7f07f8004ec0 2021-07-11T03:53:25.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081659c700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 0x7f08181411e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f08180afb80 tx=0x7f0808009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f080f7fe700 1 -- 172.21.2.104:0/3612694569 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f08080155f0 con 0x7f07f8004ec0 2021-07-11T03:53:25.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f080f7fe700 1 -- 172.21.2.104:0/3612694569 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0808010040 con 0x7f07f8004ec0 2021-07-11T03:53:25.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- 172.21.2.104:0/3612694569 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08180ad170 con 0x7f07f8004ec0 2021-07-11T03:53:25.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f080f7fe700 1 -- 172.21.2.104:0/3612694569 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0808015940 con 0x7f07f8004ec0 2021-07-11T03:53:25.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.228+0000 7f081c93d700 1 -- 172.21.2.104:0/3612694569 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08180adc40 con 0x7f07f8004ec0 2021-07-11T03:53:25.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f081c93d700 1 -- 172.21.2.104:0/3612694569 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08180b2c80 con 0x7f07f8004ec0 2021-07-11T03:53:25.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f080f7fe700 1 -- 172.21.2.104:0/3612694569 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0808015aa0 con 0x7f07f8004ec0 2021-07-11T03:53:25.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f080f7fe700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f07fc053270 0x7f07fc055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f080f7fe700 1 -- 172.21.2.104:0/3612694569 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f080806b6c0 con 0x7f07f8004ec0 2021-07-11T03:53:25.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f080f7fe700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] conn(0x7f07fc05a6e0 0x7f07fc05cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f080f7fe700 1 -- 172.21.2.104:0/3612694569 --> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07fc05d0c0 con 0x7f07fc05a6e0 2021-07-11T03:53:25.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f080f7fe700 1 -- 172.21.2.104:0/3612694569 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f080806dc20 con 0x7f07f8004ec0 2021-07-11T03:53:25.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f0815d9b700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f07fc053270 0x7f07fc055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f0816d9d700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] conn(0x7f07fc05a6e0 0x7f07fc05cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f0815d9b700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f07fc053270 0x7f07fc055730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f080000a000 tx=0x7f0800008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f0816d9d700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] conn(0x7f07fc05a6e0 0x7f07fc05cb00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.232+0000 7f080f7fe700 1 -- 172.21.2.104:0/3612694569 <== osd.2 v2:172.21.2.104:6816/15413 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f07fc05d0c0 con 0x7f07fc05a6e0 2021-07-11T03:53:25.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f081c93d700 1 -- 172.21.2.104:0/3612694569 --> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08180b21b0 con 0x7f07fc05a6e0 2021-07-11T03:53:25.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f080f7fe700 1 -- 172.21.2.104:0/3612694569 <== osd.2 v2:172.21.2.104:6816/15413 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f08180b21b0 con 0x7f07fc05a6e0 2021-07-11T03:53:25.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f081c93d700 1 -- 172.21.2.104:0/3612694569 >> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] conn(0x7f07fc05a6e0 msgr2=0x7f07fc05cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f081c93d700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] conn(0x7f07fc05a6e0 0x7f07fc05cb00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f081c93d700 1 -- 172.21.2.104:0/3612694569 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f07fc053270 msgr2=0x7f07fc055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f081c93d700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f07fc053270 0x7f07fc055730 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f080000a000 tx=0x7f0800008040).stop 2021-07-11T03:53:25.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f081c93d700 1 -- 172.21.2.104:0/3612694569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 msgr2=0x7f08181411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f081c93d700 1 --2- 172.21.2.104:0/3612694569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07f8004ec0 0x7f08181411e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f08180afb80 tx=0x7f0808009040).stop 2021-07-11T03:53:25.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f0816d9d700 1 -- 172.21.2.104:0/3612694569 reap_dead start 2021-07-11T03:53:25.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f081c93d700 1 -- 172.21.2.104:0/3612694569 shutdown_connections 2021-07-11T03:53:25.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f081c93d700 1 -- 172.21.2.104:0/3612694569 >> 172.21.2.104:0/3612694569 conn(0x7f081805cf30 msgr2=0x7f0818142220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f081c93d700 1 -- 172.21.2.104:0/3612694569 shutdown_connections 2021-07-11T03:53:25.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.240+0000 7f081c93d700 1 -- 172.21.2.104:0/3612694569 wait complete. 2021-07-11T03:53:25.258 INFO:teuthology.orchestra.run.gibba004.stdout:60129542149 2021-07-11T03:53:25.259 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2021-07-11T03:53:25.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 Processor -- start 2021-07-11T03:53:25.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 -- start start 2021-07-11T03:53:25.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4774060100 0x7f47740b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47740b4c40 0x7f477414b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47740aabb0 con 0x7f4774067300 2021-07-11T03:53:25.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47740604e0 con 0x7f4774060100 2021-07-11T03:53:25.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4774066ea0 con 0x7f47740b4c40 2021-07-11T03:53:25.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4772ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4774060100 0x7f47740b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4772ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4774060100 0x7f47740b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41902/0 (socket says 172.21.2.104:41902) 2021-07-11T03:53:25.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4772ffd700 1 -- 172.21.2.104:0/1380538416 learned_addr learned my addr 172.21.2.104:0/1380538416 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:25.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4772ffd700 1 -- 172.21.2.104:0/1380538416 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47740b4c40 msgr2=0x7f477414b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4772ffd700 1 --2- 172.21.2.104:0/1380538416 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47740b4c40 0x7f477414b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4772ffd700 1 -- 172.21.2.104:0/1380538416 >> v1:172.21.2.105:6789/0 conn(0x7f4774067300 legacy=0x7f47741438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4772ffd700 1 -- 172.21.2.104:0/1380538416 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47740b33b0 con 0x7f4774060100 2021-07-11T03:53:25.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4772ffd700 1 --2- 172.21.2.104:0/1380538416 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4774060100 0x7f47740b4750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f477405f1e0 tx=0x7f475c006eb0).ready entity=mon.0 client_cookie=e4e7365cfbab0ad1 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4771ffb700 1 -- 172.21.2.104:0/1380538416 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f475c00d570 con 0x7f4774060100 2021-07-11T03:53:25.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4771ffb700 1 -- 172.21.2.104:0/1380538416 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c000e90 con 0x7f4774060100 2021-07-11T03:53:25.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4771ffb700 1 -- 172.21.2.104:0/1380538416 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f475c00d8c0 con 0x7f4774060100 2021-07-11T03:53:25.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 -- 172.21.2.104:0/1380538416 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4774060100 msgr2=0x7f47740b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 --2- 172.21.2.104:0/1380538416 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4774060100 0x7f47740b4750 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f477405f1e0 tx=0x7f475c006eb0).stop 2021-07-11T03:53:25.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 -- 172.21.2.104:0/1380538416 shutdown_connections 2021-07-11T03:53:25.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 --2- 172.21.2.104:0/1380538416 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47740b4c40 0x7f477414b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 --2- 172.21.2.104:0/1380538416 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4774060100 0x7f47740b4750 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 -- 172.21.2.104:0/1380538416 >> 172.21.2.104:0/1380538416 conn(0x7f477405cf30 msgr2=0x7f477405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 -- 172.21.2.104:0/1380538416 shutdown_connections 2021-07-11T03:53:25.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.308+0000 7f4779693700 1 -- 172.21.2.104:0/1380538416 wait complete. 2021-07-11T03:53:25.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4779693700 1 Processor -- start 2021-07-11T03:53:25.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4779693700 1 -- start start 2021-07-11T03:53:25.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4779693700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4774060100 0x7f4774064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4779693700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4774067300 0x7f4774151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4779693700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47740b4c40 0x7f4774151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4779693700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4774151650 con 0x7f47740b4c40 2021-07-11T03:53:25.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4779693700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477413e630 con 0x7f4774067300 2021-07-11T03:53:25.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4779693700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477414edb0 con 0x7f4774060100 2021-07-11T03:53:25.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f47737fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47740b4c40 0x7f4774151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f47737fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47740b4c40 0x7f4774151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41906/0 (socket says 172.21.2.104:41906) 2021-07-11T03:53:25.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f47737fe700 1 -- 172.21.2.104:0/1234697256 learned_addr learned my addr 172.21.2.104:0/1234697256 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:25.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f47737fe700 1 -- 172.21.2.104:0/1234697256 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4774060100 msgr2=0x7f4774064bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f47737fe700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4774060100 0x7f4774064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f47737fe700 1 -- 172.21.2.104:0/1234697256 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4774067300 msgr2=0x7f4774151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f47737fe700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4774067300 0x7f4774151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f47737fe700 1 -- 172.21.2.104:0/1234697256 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c00b1e0 con 0x7f47740b4c40 2021-07-11T03:53:25.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f47737fe700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47740b4c40 0x7f4774151f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f476c0032f0 tx=0x7f476c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f476bfff700 1 -- 172.21.2.104:0/1234697256 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f476c00f510 con 0x7f47740b4c40 2021-07-11T03:53:25.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f476bfff700 1 -- 172.21.2.104:0/1234697256 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c00f670 con 0x7f47740b4c40 2021-07-11T03:53:25.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4779693700 1 -- 172.21.2.104:0/1234697256 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47741408f0 con 0x7f47740b4c40 2021-07-11T03:53:25.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f476bfff700 1 -- 172.21.2.104:0/1234697256 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f476c00f7e0 con 0x7f47740b4c40 2021-07-11T03:53:25.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4779693700 1 -- 172.21.2.104:0/1234697256 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477414ea50 con 0x7f47740b4c40 2021-07-11T03:53:25.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4779693700 1 -- 172.21.2.104:0/1234697256 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47740657e0 con 0x7f47740b4c40 2021-07-11T03:53:25.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f476bfff700 1 -- 172.21.2.104:0/1234697256 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f476c00f070 con 0x7f47740b4c40 2021-07-11T03:53:25.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f476bfff700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4758053160 0x7f4758055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f476bfff700 1 -- 172.21.2.104:0/1234697256 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f476c06b140 con 0x7f47740b4c40 2021-07-11T03:53:25.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f476bfff700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] conn(0x7f475805a640 0x7f475805ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f476bfff700 1 -- 172.21.2.104:0/1234697256 --> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f475805d0c0 con 0x7f475805a640 2021-07-11T03:53:25.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f476bfff700 1 -- 172.21.2.104:0/1234697256 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f476c06d7d0 con 0x7f47740b4c40 2021-07-11T03:53:25.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f47727fc700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] conn(0x7f475805a640 0x7f475805ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f4772ffd700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4758053160 0x7f4758055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f47727fc700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] conn(0x7f475805a640 0x7f475805ca60 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.312+0000 7f476bfff700 1 -- 172.21.2.104:0/1234697256 <== osd.3 v2:172.21.2.104:6824/15456 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f475805d0c0 con 0x7f475805a640 2021-07-11T03:53:25.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.316+0000 7f4772ffd700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4758053160 0x7f4758055620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f477405f1e0 tx=0x7f475c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f4779693700 1 -- 172.21.2.104:0/1234697256 --> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f47741439e0 con 0x7f475805a640 2021-07-11T03:53:25.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f476bfff700 1 -- 172.21.2.104:0/1234697256 <== osd.3 v2:172.21.2.104:6824/15456 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f47741439e0 con 0x7f475805a640 2021-07-11T03:53:25.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f4779693700 1 -- 172.21.2.104:0/1234697256 >> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] conn(0x7f475805a640 msgr2=0x7f475805ca60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f4779693700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] conn(0x7f475805a640 0x7f475805ca60 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f4779693700 1 -- 172.21.2.104:0/1234697256 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4758053160 msgr2=0x7f4758055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f4779693700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4758053160 0x7f4758055620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f477405f1e0 tx=0x7f475c015000).stop 2021-07-11T03:53:25.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f4779693700 1 -- 172.21.2.104:0/1234697256 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47740b4c40 msgr2=0x7f4774151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f4779693700 1 --2- 172.21.2.104:0/1234697256 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47740b4c40 0x7f4774151f00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f476c0032f0 tx=0x7f476c0091a0).stop 2021-07-11T03:53:25.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f47737fe700 1 -- 172.21.2.104:0/1234697256 reap_dead start 2021-07-11T03:53:25.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f4779693700 1 -- 172.21.2.104:0/1234697256 shutdown_connections 2021-07-11T03:53:25.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f4779693700 1 -- 172.21.2.104:0/1234697256 >> 172.21.2.104:0/1234697256 conn(0x7f477405cf30 msgr2=0x7f477405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f4779693700 1 -- 172.21.2.104:0/1234697256 shutdown_connections 2021-07-11T03:53:25.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.324+0000 7f4779693700 1 -- 172.21.2.104:0/1234697256 wait complete. 2021-07-11T03:53:25.335 INFO:teuthology.orchestra.run.gibba004.stdout:60129542148 2021-07-11T03:53:25.335 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2021-07-11T03:53:25.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.388+0000 7f0197764700 1 Processor -- start 2021-07-11T03:53:25.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.388+0000 7f0197764700 1 -- start start 2021-07-11T03:53:25.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.388+0000 7f0197764700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01900b04a0 0x7f01900add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.388+0000 7f0197764700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01900b2af0 con 0x7f01900aa120 2021-07-11T03:53:25.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.388+0000 7f0197764700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01900b2770 con 0x7f01900b0880 2021-07-11T03:53:25.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.388+0000 7f0197764700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01900b2d20 con 0x7f01900b04a0 2021-07-11T03:53:25.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.388+0000 7f0194cff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f01900aa120 0x7f01900ae220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38688/0 (socket says 172.21.2.104:38688) 2021-07-11T03:53:25.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.388+0000 7f0194cff700 1 -- 172.21.2.104:0/3386181233 learned_addr learned my addr 172.21.2.104:0/3386181233 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:25.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.388+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742839332 0 0) 0x7f01900b2af0 con 0x7f01900aa120 2021-07-11T03:53:25.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.388+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0178003260 con 0x7f01900aa120 2021-07-11T03:53:25.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0195500700 1 --2- 172.21.2.104:0/3386181233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01900b04a0 0x7f01900add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1621822108 0 0) 0x7f0178003260 con 0x7f01900aa120 2021-07-11T03:53:25.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01900b04a0 msgr2=0x7f01900add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 --2- 172.21.2.104:0/3386181233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01900b04a0 0x7f01900add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 >> v1:172.21.2.105:6789/0 conn(0x7f01900b0880 legacy=0x7f01901405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01900a9b70 con 0x7f01900aa120 2021-07-11T03:53:25.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f018c001660 con 0x7f01900aa120 2021-07-11T03:53:25.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 >> v1:172.21.2.104:6789/0 conn(0x7f01900aa120 legacy=0x7f01900ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 --2- 172.21.2.104:0/3386181233 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0178003800 0x7f0178003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 --2- 172.21.2.104:0/3386181233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0178004500 0x7f0178004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 --2- 172.21.2.104:0/3386181233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0178005070 0x7f017800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01900b2d20 con 0x7f0178005070 2021-07-11T03:53:25.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017800bed0 con 0x7f0178003800 2021-07-11T03:53:25.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f017800c090 con 0x7f0178004500 2021-07-11T03:53:25.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0195500700 1 --2- 172.21.2.104:0/3386181233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01900b04a0 0x7f01900add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:25.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0195d01700 1 --2- 172.21.2.104:0/3386181233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0178005070 0x7f017800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0195d01700 1 -- 172.21.2.104:0/3386181233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0178004500 msgr2=0x7f0178004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0195d01700 1 --2- 172.21.2.104:0/3386181233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0178004500 0x7f0178004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0195d01700 1 -- 172.21.2.104:0/3386181233 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0178003800 msgr2=0x7f0178003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0195d01700 1 --2- 172.21.2.104:0/3386181233 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0178003800 0x7f0178003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0195d01700 1 -- 172.21.2.104:0/3386181233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0178003260 con 0x7f0178005070 2021-07-11T03:53:25.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0195d01700 1 --2- 172.21.2.104:0/3386181233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0178005070 0x7f017800b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f01900b0dd0 tx=0x7f0188007d80).ready entity=mon.0 client_cookie=500aee5cbb2394f0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0188001620 con 0x7f0178005070 2021-07-11T03:53:25.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0188006890 con 0x7f0178005070 2021-07-11T03:53:25.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0183fff700 1 -- 172.21.2.104:0/3386181233 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0188006bf0 con 0x7f0178005070 2021-07-11T03:53:25.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- 172.21.2.104:0/3386181233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0178005070 msgr2=0x7f017800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 --2- 172.21.2.104:0/3386181233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0178005070 0x7f017800b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f01900b0dd0 tx=0x7f0188007d80).stop 2021-07-11T03:53:25.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0195d01700 1 -- 172.21.2.104:0/3386181233 reap_dead start 2021-07-11T03:53:25.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- 172.21.2.104:0/3386181233 shutdown_connections 2021-07-11T03:53:25.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- 172.21.2.104:0/3386181233 >> 172.21.2.104:0/3386181233 conn(0x7f01900607c0 msgr2=0x7f0190060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- 172.21.2.104:0/3386181233 shutdown_connections 2021-07-11T03:53:25.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- 172.21.2.104:0/3386181233 wait complete. 2021-07-11T03:53:25.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 Processor -- start 2021-07-11T03:53:25.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- start start 2021-07-11T03:53:25.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0190140330 0x7f0190066f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0190064630 0x7f01900673f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0190142450 0x7f01901428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0190144680 con 0x7f0190064630 2021-07-11T03:53:25.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0190140710 con 0x7f0190140330 2021-07-11T03:53:25.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0190143fd0 con 0x7f0190142450 2021-07-11T03:53:25.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0194cff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0190064630 0x7f01900673f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0194cff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0190064630 0x7f01900673f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41930/0 (socket says 172.21.2.104:41930) 2021-07-11T03:53:25.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0194cff700 1 -- 172.21.2.104:0/1754082739 learned_addr learned my addr 172.21.2.104:0/1754082739 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:25.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0195500700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0190140330 0x7f0190066f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0194cff700 1 -- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0190142450 msgr2=0x7f01901428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0194cff700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0190142450 0x7f01901428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0194cff700 1 -- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0190140330 msgr2=0x7f0190066f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0194cff700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0190140330 0x7f0190066f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0194cff700 1 -- 172.21.2.104:0/1754082739 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01901448b0 con 0x7f0190064630 2021-07-11T03:53:25.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0194cff700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0190064630 0x7f01900673f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f018c001370 tx=0x7f018c001a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f01827fc700 1 -- 172.21.2.104:0/1754082739 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f018c007170 con 0x7f0190064630 2021-07-11T03:53:25.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f01827fc700 1 -- 172.21.2.104:0/1754082739 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018c0072d0 con 0x7f0190064630 2021-07-11T03:53:25.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- 172.21.2.104:0/1754082739 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01900643a0 con 0x7f0190064630 2021-07-11T03:53:25.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f01827fc700 1 -- 172.21.2.104:0/1754082739 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f018c0049e0 con 0x7f0190064630 2021-07-11T03:53:25.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- 172.21.2.104:0/1754082739 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0190144270 con 0x7f0190064630 2021-07-11T03:53:25.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.392+0000 7f0197764700 1 -- 172.21.2.104:0/1754082739 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0190067a80 con 0x7f0190064630 2021-07-11T03:53:25.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.396+0000 7f01827fc700 1 -- 172.21.2.104:0/1754082739 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f018c01b060 con 0x7f0190064630 2021-07-11T03:53:25.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.396+0000 7f01827fc700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0170053270 0x7f0170055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.396+0000 7f01827fc700 1 -- 172.21.2.104:0/1754082739 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f018c06b220 con 0x7f0190064630 2021-07-11T03:53:25.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.396+0000 7f01827fc700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] conn(0x7f017005a660 0x7f017005ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.396+0000 7f01827fc700 1 -- 172.21.2.104:0/1754082739 --> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f017005d040 con 0x7f017005a660 2021-07-11T03:53:25.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.396+0000 7f01827fc700 1 -- 172.21.2.104:0/1754082739 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f018c06d890 con 0x7f0190064630 2021-07-11T03:53:25.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.396+0000 7f0195500700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0170053270 0x7f0170055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.396+0000 7f0195d01700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] conn(0x7f017005a660 0x7f017005ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.396+0000 7f0195d01700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] conn(0x7f017005a660 0x7f017005ca80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.396+0000 7f0195500700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0170053270 0x7f0170055730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f0184008000 tx=0x7f0184008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.396+0000 7f01827fc700 1 -- 172.21.2.104:0/1754082739 <== osd.4 v2:172.21.2.105:6802/15645 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f017005d040 con 0x7f017005a660 2021-07-11T03:53:25.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0197764700 1 -- 172.21.2.104:0/1754082739 --> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f01901430c0 con 0x7f017005a660 2021-07-11T03:53:25.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f01827fc700 1 -- 172.21.2.104:0/1754082739 <== osd.4 v2:172.21.2.105:6802/15645 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f01901430c0 con 0x7f017005a660 2021-07-11T03:53:25.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0197764700 1 -- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] conn(0x7f017005a660 msgr2=0x7f017005ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0197764700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] conn(0x7f017005a660 0x7f017005ca80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0197764700 1 -- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0170053270 msgr2=0x7f0170055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0197764700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0170053270 0x7f0170055730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f0184008000 tx=0x7f0184008040).stop 2021-07-11T03:53:25.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0197764700 1 -- 172.21.2.104:0/1754082739 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0190064630 msgr2=0x7f01900673f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0197764700 1 --2- 172.21.2.104:0/1754082739 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0190064630 0x7f01900673f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f018c001370 tx=0x7f018c001a60).stop 2021-07-11T03:53:25.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0195d01700 1 -- 172.21.2.104:0/1754082739 reap_dead start 2021-07-11T03:53:25.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0197764700 1 -- 172.21.2.104:0/1754082739 shutdown_connections 2021-07-11T03:53:25.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0197764700 1 -- 172.21.2.104:0/1754082739 >> 172.21.2.104:0/1754082739 conn(0x7f01900607c0 msgr2=0x7f01900a8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0197764700 1 -- 172.21.2.104:0/1754082739 shutdown_connections 2021-07-11T03:53:25.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.408+0000 7f0197764700 1 -- 172.21.2.104:0/1754082739 wait complete. 2021-07-11T03:53:25.422 INFO:teuthology.orchestra.run.gibba004.stdout:60129542150 2021-07-11T03:53:25.423 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2021-07-11T03:53:25.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d84dcb700 1 Processor -- start 2021-07-11T03:53:25.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d84dcb700 1 -- start start 2021-07-11T03:53:25.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d84dcb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6d800b5bf0 0x7f6d800b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d84dcb700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d801440b0 con 0x7f6d800b6580 2021-07-11T03:53:25.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d84dcb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d800b6e50 con 0x7f6d800b43c0 2021-07-11T03:53:25.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d84dcb700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d80143b70 con 0x7f6d800b5bf0 2021-07-11T03:53:25.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d7e59c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6d800b6580 0x7f6d80061d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38706/0 (socket says 172.21.2.104:38706) 2021-07-11T03:53:25.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d7e59c700 1 -- 172.21.2.104:0/4272680348 learned_addr learned my addr 172.21.2.104:0/4272680348 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:25.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d7ed9d700 1 --2- 172.21.2.104:0/4272680348 >> v2:172.21.2.104:3300/0 conn(0x7f6d800b5bf0 0x7f6d800b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d7ed9d700 1 -- 172.21.2.104:0/4272680348 >> v1:172.21.2.105:6789/0 conn(0x7f6d800b43c0 legacy=0x7f6d8012ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d7ed9d700 1 -- 172.21.2.104:0/4272680348 >> v1:172.21.2.104:6789/0 conn(0x7f6d800b6580 legacy=0x7f6d80061d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d7ed9d700 1 -- 172.21.2.104:0/4272680348 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d80143e10 con 0x7f6d800b5bf0 2021-07-11T03:53:25.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.476+0000 7f6d7ed9d700 1 --2- 172.21.2.104:0/4272680348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d800b5bf0 0x7f6d800b6960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f6d68001110 tx=0x7f6d780091d0).ready entity=mon.0 client_cookie=237a591cdcf7ff6d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7d59a700 1 -- 172.21.2.104:0/4272680348 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d78009ea0 con 0x7f6d800b5bf0 2021-07-11T03:53:25.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7d59a700 1 -- 172.21.2.104:0/4272680348 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7800b760 con 0x7f6d800b5bf0 2021-07-11T03:53:25.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7d59a700 1 -- 172.21.2.104:0/4272680348 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7800b8d0 con 0x7f6d800b5bf0 2021-07-11T03:53:25.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4272680348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d800b5bf0 msgr2=0x7f6d800b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 --2- 172.21.2.104:0/4272680348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d800b5bf0 0x7f6d800b6960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f6d68001110 tx=0x7f6d780091d0).stop 2021-07-11T03:53:25.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4272680348 shutdown_connections 2021-07-11T03:53:25.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 --2- 172.21.2.104:0/4272680348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d800b5bf0 0x7f6d800b6960 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4272680348 >> 172.21.2.104:0/4272680348 conn(0x7f6d8005c490 msgr2=0x7f6d8005c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4272680348 shutdown_connections 2021-07-11T03:53:25.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4272680348 wait complete. 2021-07-11T03:53:25.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 Processor -- start 2021-07-11T03:53:25.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- start start 2021-07-11T03:53:25.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d800b43c0 0x7f6d80141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d800b5bf0 0x7f6d800af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d800b6580 0x7f6d800afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800aeca0 con 0x7f6d800b5bf0 2021-07-11T03:53:25.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800ad7a0 con 0x7f6d800b6580 2021-07-11T03:53:25.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800ada40 con 0x7f6d800b43c0 2021-07-11T03:53:25.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d800b5bf0 0x7f6d800af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7dd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d800b5bf0 0x7f6d800af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41944/0 (socket says 172.21.2.104:41944) 2021-07-11T03:53:25.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7dd9b700 1 -- 172.21.2.104:0/4074295185 learned_addr learned my addr 172.21.2.104:0/4074295185 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:25.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7dd9b700 1 -- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d800b43c0 msgr2=0x7f6d80141420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7dd9b700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d800b43c0 0x7f6d80141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7dd9b700 1 -- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d800b6580 msgr2=0x7f6d800afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7dd9b700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d800b6580 0x7f6d800afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7dd9b700 1 -- 172.21.2.104:0/4074295185 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78009470 con 0x7f6d800b5bf0 2021-07-11T03:53:25.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7dd9b700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d800b5bf0 0x7f6d800af680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f6d70002f90 tx=0x7f6d70006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d777fe700 1 -- 172.21.2.104:0/4074295185 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d70013460 con 0x7f6d800b5bf0 2021-07-11T03:53:25.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d777fe700 1 -- 172.21.2.104:0/4074295185 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d700135c0 con 0x7f6d800b5bf0 2021-07-11T03:53:25.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d777fe700 1 -- 172.21.2.104:0/4074295185 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d70013920 con 0x7f6d800b5bf0 2021-07-11T03:53:25.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4074295185 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d800adce0 con 0x7f6d800b5bf0 2021-07-11T03:53:25.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4074295185 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d800b0560 con 0x7f6d800b5bf0 2021-07-11T03:53:25.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4074295185 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d80057350 con 0x7f6d800b5bf0 2021-07-11T03:53:25.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d777fe700 1 -- 172.21.2.104:0/4074295185 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6d7000f070 con 0x7f6d800b5bf0 2021-07-11T03:53:25.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d777fe700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d64053270 0x7f6d64055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d777fe700 1 -- 172.21.2.104:0/4074295185 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6d7006ac30 con 0x7f6d800b5bf0 2021-07-11T03:53:25.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d777fe700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] conn(0x7f6d6405a660 0x7f6d6405ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d777fe700 1 -- 172.21.2.104:0/4074295185 --> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d6405d040 con 0x7f6d6405a660 2021-07-11T03:53:25.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d777fe700 1 -- 172.21.2.104:0/4074295185 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d7006d2c0 con 0x7f6d800b5bf0 2021-07-11T03:53:25.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7e59c700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d64053270 0x7f6d64055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.480+0000 7f6d7ed9d700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] conn(0x7f6d6405a660 0x7f6d6405ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.484+0000 7f6d7e59c700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d64053270 0x7f6d64055730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6d68001000 tx=0x7f6d68007820).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.484+0000 7f6d7ed9d700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] conn(0x7f6d6405a660 0x7f6d6405ca80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.484+0000 7f6d777fe700 1 -- 172.21.2.104:0/4074295185 <== osd.5 v2:172.21.2.105:6812/15646 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f6d6405d040 con 0x7f6d6405a660 2021-07-11T03:53:25.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4074295185 --> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6d80142030 con 0x7f6d6405a660 2021-07-11T03:53:25.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d777fe700 1 -- 172.21.2.104:0/4074295185 <== osd.5 v2:172.21.2.105:6812/15646 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6d80142030 con 0x7f6d6405a660 2021-07-11T03:53:25.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] conn(0x7f6d6405a660 msgr2=0x7f6d6405ca80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d84dcb700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] conn(0x7f6d6405a660 0x7f6d6405ca80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d64053270 msgr2=0x7f6d64055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d84dcb700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d64053270 0x7f6d64055730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f6d68001000 tx=0x7f6d68007820).stop 2021-07-11T03:53:25.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4074295185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d800b5bf0 msgr2=0x7f6d800af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d84dcb700 1 --2- 172.21.2.104:0/4074295185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d800b5bf0 0x7f6d800af680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f6d70002f90 tx=0x7f6d70006de0).stop 2021-07-11T03:53:25.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d7ed9d700 1 -- 172.21.2.104:0/4074295185 reap_dead start 2021-07-11T03:53:25.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4074295185 shutdown_connections 2021-07-11T03:53:25.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4074295185 >> 172.21.2.104:0/4074295185 conn(0x7f6d8005c490 msgr2=0x7f6d80128930 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4074295185 shutdown_connections 2021-07-11T03:53:25.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.496+0000 7f6d84dcb700 1 -- 172.21.2.104:0/4074295185 wait complete. 2021-07-11T03:53:25.506 INFO:teuthology.orchestra.run.gibba004.stdout:60129542149 2021-07-11T03:53:25.506 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2021-07-11T03:53:25.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 Processor -- start 2021-07-11T03:53:25.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- start start 2021-07-11T03:53:25.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff8100b04a0 0x7ff8100add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff8100b0880 0x7ff8101405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8100b2af0 con 0x7ff8100aa120 2021-07-11T03:53:25.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8100b2770 con 0x7ff8100b0880 2021-07-11T03:53:25.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8100b2d20 con 0x7ff8100b04a0 2021-07-11T03:53:25.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff814942700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff8100aa120 0x7ff8100ae220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38724/0 (socket says 172.21.2.104:38724) 2021-07-11T03:53:25.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff814942700 1 -- 172.21.2.104:0/1833201821 learned_addr learned my addr 172.21.2.104:0/1833201821 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:25.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff815944700 1 --2- 172.21.2.104:0/1833201821 >> v2:172.21.2.104:3300/0 conn(0x7ff8100b0880 0x7ff8101405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff815944700 1 -- 172.21.2.104:0/1833201821 >> v2:172.21.2.105:3300/0 conn(0x7ff8100b04a0 msgr2=0x7ff8100add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff815944700 1 --2- 172.21.2.104:0/1833201821 >> v2:172.21.2.105:3300/0 conn(0x7ff8100b04a0 0x7ff8100add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff815944700 1 -- 172.21.2.104:0/1833201821 >> v1:172.21.2.104:6789/0 conn(0x7ff8100aa120 legacy=0x7ff8100ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff815944700 1 -- 172.21.2.104:0/1833201821 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100a9b70 con 0x7ff8100b0880 2021-07-11T03:53:25.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff815944700 1 --2- 172.21.2.104:0/1833201821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b0880 0x7ff8101405a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff8080032f0 tx=0x7ff8080091d0).ready entity=mon.0 client_cookie=b28427a1ba7e1f22 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff7fffff700 1 -- 172.21.2.104:0/1833201821 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff808009ea0 con 0x7ff8100b0880 2021-07-11T03:53:25.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff7fffff700 1 -- 172.21.2.104:0/1833201821 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80800b870 con 0x7ff8100b0880 2021-07-11T03:53:25.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff7fffff700 1 -- 172.21.2.104:0/1833201821 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff80800b9e0 con 0x7ff8100b0880 2021-07-11T03:53:25.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- 172.21.2.104:0/1833201821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b0880 msgr2=0x7ff8101405a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 --2- 172.21.2.104:0/1833201821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b0880 0x7ff8101405a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7ff8080032f0 tx=0x7ff8080091d0).stop 2021-07-11T03:53:25.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- 172.21.2.104:0/1833201821 shutdown_connections 2021-07-11T03:53:25.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 --2- 172.21.2.104:0/1833201821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b0880 0x7ff8101405a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 --2- 172.21.2.104:0/1833201821 >> v2:172.21.2.105:3300/0 conn(0x7ff8100b04a0 0x7ff8100add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- 172.21.2.104:0/1833201821 >> 172.21.2.104:0/1833201821 conn(0x7ff8100607c0 msgr2=0x7ff810060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- 172.21.2.104:0/1833201821 shutdown_connections 2021-07-11T03:53:25.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- 172.21.2.104:0/1833201821 wait complete. 2021-07-11T03:53:25.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 Processor -- start 2021-07-11T03:53:25.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- start start 2021-07-11T03:53:25.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100aa120 0x7ff8100abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff8100b04a0 0x7ff810067150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff8100b0880 0x7ff8100638d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff810143590 con 0x7ff8100aa120 2021-07-11T03:53:25.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8100657b0 con 0x7ff8100b04a0 2021-07-11T03:53:25.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff8173a7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff810066a70 con 0x7ff8100b0880 2021-07-11T03:53:25.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff815143700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100aa120 0x7ff8100abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff815143700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100aa120 0x7ff8100abd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41960/0 (socket says 172.21.2.104:41960) 2021-07-11T03:53:25.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff815143700 1 -- 172.21.2.104:0/2204933386 learned_addr learned my addr 172.21.2.104:0/2204933386 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:25.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff815143700 1 -- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff8100b0880 msgr2=0x7ff8100638d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.556+0000 7ff815143700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff8100b0880 0x7ff8100638d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff815143700 1 -- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff8100b04a0 msgr2=0x7ff810067150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff815143700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff8100b04a0 0x7ff810067150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff815143700 1 -- 172.21.2.104:0/2204933386 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff808009470 con 0x7ff8100aa120 2021-07-11T03:53:25.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff815143700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100aa120 0x7ff8100abd00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7ff800002f70 tx=0x7ff800006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff7fe7fc700 1 -- 172.21.2.104:0/2204933386 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff800013460 con 0x7ff8100aa120 2021-07-11T03:53:25.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff7fe7fc700 1 -- 172.21.2.104:0/2204933386 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8000135c0 con 0x7ff8100aa120 2021-07-11T03:53:25.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff7fe7fc700 1 -- 172.21.2.104:0/2204933386 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff800013920 con 0x7ff8100aa120 2021-07-11T03:53:25.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff8173a7700 1 -- 172.21.2.104:0/2204933386 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff810066d10 con 0x7ff8100aa120 2021-07-11T03:53:25.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff8173a7700 1 -- 172.21.2.104:0/2204933386 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8100654a0 con 0x7ff8100aa120 2021-07-11T03:53:25.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff8173a7700 1 -- 172.21.2.104:0/2204933386 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff810065240 con 0x7ff8100aa120 2021-07-11T03:53:25.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff7fe7fc700 1 -- 172.21.2.104:0/2204933386 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff80000f070 con 0x7ff8100aa120 2021-07-11T03:53:25.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff7fe7fc700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7f00531f0 0x7ff7f00556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff7fe7fc700 1 -- 172.21.2.104:0/2204933386 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff80006ae70 con 0x7ff8100aa120 2021-07-11T03:53:25.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff7fe7fc700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] conn(0x7ff7f005a5e0 0x7ff7f005ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff7fe7fc700 1 -- 172.21.2.104:0/2204933386 --> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7f005d060 con 0x7ff7f005a5e0 2021-07-11T03:53:25.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff7fe7fc700 1 -- 172.21.2.104:0/2204933386 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff80006d500 con 0x7ff8100aa120 2021-07-11T03:53:25.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff814942700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7f00531f0 0x7ff7f00556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff815944700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] conn(0x7ff7f005a5e0 0x7ff7f005ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff815944700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] conn(0x7ff7f005a5e0 0x7ff7f005ca00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.560+0000 7ff814942700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7f00531f0 0x7ff7f00556b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ff8100b00b0 tx=0x7ff80c007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.564+0000 7ff7fe7fc700 1 -- 172.21.2.104:0/2204933386 <== osd.6 v2:172.21.2.105:6810/15647 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7ff7f005d060 con 0x7ff7f005a5e0 2021-07-11T03:53:25.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.576+0000 7ff8173a7700 1 -- 172.21.2.104:0/2204933386 --> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff810065dc0 con 0x7ff7f005a5e0 2021-07-11T03:53:25.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff7fe7fc700 1 -- 172.21.2.104:0/2204933386 <== osd.6 v2:172.21.2.105:6810/15647 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff80800b870 con 0x7ff7f005a5e0 2021-07-11T03:53:25.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff8173a7700 1 -- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] conn(0x7ff7f005a5e0 msgr2=0x7ff7f005ca00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff8173a7700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] conn(0x7ff7f005a5e0 0x7ff7f005ca00 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff8173a7700 1 -- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7f00531f0 msgr2=0x7ff7f00556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff8173a7700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7f00531f0 0x7ff7f00556b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7ff8100b00b0 tx=0x7ff80c007800).stop 2021-07-11T03:53:25.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff8173a7700 1 -- 172.21.2.104:0/2204933386 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100aa120 msgr2=0x7ff8100abd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff8173a7700 1 --2- 172.21.2.104:0/2204933386 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100aa120 0x7ff8100abd00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7ff800002f70 tx=0x7ff800006ec0).stop 2021-07-11T03:53:25.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff815944700 1 -- 172.21.2.104:0/2204933386 reap_dead start 2021-07-11T03:53:25.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff8173a7700 1 -- 172.21.2.104:0/2204933386 shutdown_connections 2021-07-11T03:53:25.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff8173a7700 1 -- 172.21.2.104:0/2204933386 >> 172.21.2.104:0/2204933386 conn(0x7ff8100607c0 msgr2=0x7ff8100b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff8173a7700 1 -- 172.21.2.104:0/2204933386 shutdown_connections 2021-07-11T03:53:25.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.580+0000 7ff8173a7700 1 -- 172.21.2.104:0/2204933386 wait complete. 2021-07-11T03:53:25.589 INFO:teuthology.orchestra.run.gibba004.stdout:60129542149 2021-07-11T03:53:25.589 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2021-07-11T03:53:25.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 Processor -- start 2021-07-11T03:53:25.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 -- start start 2021-07-11T03:53:25.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f84840b04a0 0x7f84840add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f84840b0880 0x7f84841405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84840b2af0 con 0x7f84840aa120 2021-07-11T03:53:25.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84840b2770 con 0x7f84840b0880 2021-07-11T03:53:25.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84840b2d20 con 0x7f84840b04a0 2021-07-11T03:53:25.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f84837fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f84840aa120 0x7f84840ae220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38742/0 (socket says 172.21.2.104:38742) 2021-07-11T03:53:25.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f84837fe700 1 -- 172.21.2.104:0/2694171274 learned_addr learned my addr 172.21.2.104:0/2694171274 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:25.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456649738 0 0) 0x7f84840b2af0 con 0x7f84840aa120 2021-07-11T03:53:25.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8464003260 con 0x7f84840aa120 2021-07-11T03:53:25.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3104633974 0 0) 0x7f8464003260 con 0x7f84840aa120 2021-07-11T03:53:25.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f84840b04a0 msgr2=0x7f84840add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 --2- 172.21.2.104:0/2694171274 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f84840b04a0 0x7f84840add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 >> v2:172.21.2.105:3300/0 conn(0x7f84840b0880 msgr2=0x7f84841405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 --2- 172.21.2.104:0/2694171274 >> v2:172.21.2.105:3300/0 conn(0x7f84840b0880 0x7f84841405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84840a9b70 con 0x7f84840aa120 2021-07-11T03:53:25.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f846c001660 con 0x7f84840aa120 2021-07-11T03:53:25.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 >> v1:172.21.2.104:6789/0 conn(0x7f84840aa120 legacy=0x7f84840ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 --2- 172.21.2.104:0/2694171274 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8464003750 0x7f8464003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 --2- 172.21.2.104:0/2694171274 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8464004500 0x7f8464004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 --2- 172.21.2.104:0/2694171274 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8464004f90 0x7f846400d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84840b2770 con 0x7f8464004500 2021-07-11T03:53:25.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84840b2d20 con 0x7f8464004f90 2021-07-11T03:53:25.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846400e060 con 0x7f8464003750 2021-07-11T03:53:25.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f84837fe700 1 --2- 172.21.2.104:0/2694171274 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8464004500 0x7f8464004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f84837fe700 1 -- 172.21.2.104:0/2694171274 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8464003750 msgr2=0x7f8464003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f84837fe700 1 --2- 172.21.2.104:0/2694171274 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8464003750 0x7f8464003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f84837fe700 1 -- 172.21.2.104:0/2694171274 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8464004f90 msgr2=0x7f846400d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f84837fe700 1 --2- 172.21.2.104:0/2694171274 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8464004f90 0x7f846400d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f84837fe700 1 -- 172.21.2.104:0/2694171274 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8464003260 con 0x7f8464004500 2021-07-11T03:53:25.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f84837fe700 1 --2- 172.21.2.104:0/2694171274 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8464004500 0x7f8464004980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f846c001370 tx=0x7f846c001e50).ready entity=mon.0 client_cookie=18edd4c1e89f214d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846c0072c0 con 0x7f8464004500 2021-07-11T03:53:25.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c007420 con 0x7f8464004500 2021-07-11T03:53:25.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8482ffd700 1 -- 172.21.2.104:0/2694171274 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846c00cbb0 con 0x7f8464004500 2021-07-11T03:53:25.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 -- 172.21.2.104:0/2694171274 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8464004500 msgr2=0x7f8464004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 --2- 172.21.2.104:0/2694171274 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8464004500 0x7f8464004980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f846c001370 tx=0x7f846c001e50).stop 2021-07-11T03:53:25.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f8488d4e700 1 -- 172.21.2.104:0/2694171274 reap_dead start 2021-07-11T03:53:25.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 -- 172.21.2.104:0/2694171274 shutdown_connections 2021-07-11T03:53:25.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 -- 172.21.2.104:0/2694171274 >> 172.21.2.104:0/2694171274 conn(0x7f84840607c0 msgr2=0x7f8484060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 -- 172.21.2.104:0/2694171274 shutdown_connections 2021-07-11T03:53:25.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.640+0000 7f848a7b1700 1 -- 172.21.2.104:0/2694171274 wait complete. 2021-07-11T03:53:25.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f848a7b1700 1 Processor -- start 2021-07-11T03:53:25.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f848a7b1700 1 -- start start 2021-07-11T03:53:25.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f848a7b1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f848413e220 0x7f8484143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f848a7b1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8484144260 0x7f848413b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f848a7b1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f848413bea0 0x7f8484151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f848a7b1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8484142a60 con 0x7f848413bea0 2021-07-11T03:53:25.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f848a7b1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84840ab2c0 con 0x7f8484144260 2021-07-11T03:53:25.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f848a7b1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84840ab950 con 0x7f848413e220 2021-07-11T03:53:25.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f8488d4e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f848413bea0 0x7f8484151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f8488d4e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f848413bea0 0x7f8484151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:41982/0 (socket says 172.21.2.104:41982) 2021-07-11T03:53:25.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f8488d4e700 1 -- 172.21.2.104:0/2732795747 learned_addr learned my addr 172.21.2.104:0/2732795747 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:25.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f8488d4e700 1 -- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f848413e220 msgr2=0x7f8484143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f8488d4e700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f848413e220 0x7f8484143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f8488d4e700 1 -- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8484144260 msgr2=0x7f848413b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f8488d4e700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8484144260 0x7f848413b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:25.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f8488d4e700 1 -- 172.21.2.104:0/2732795747 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84841454b0 con 0x7f848413bea0 2021-07-11T03:53:25.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f8488d4e700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f848413bea0 0x7f8484151500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f846c0018c0 tx=0x7f847c00a060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f84817fa700 1 -- 172.21.2.104:0/2732795747 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f847c00ce40 con 0x7f848413bea0 2021-07-11T03:53:25.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f84817fa700 1 -- 172.21.2.104:0/2732795747 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c005c30 con 0x7f848413bea0 2021-07-11T03:53:25.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f848a7b1700 1 -- 172.21.2.104:0/2732795747 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848413e600 con 0x7f848413bea0 2021-07-11T03:53:25.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f84817fa700 1 -- 172.21.2.104:0/2732795747 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f847c007070 con 0x7f848413bea0 2021-07-11T03:53:25.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f848a7b1700 1 -- 172.21.2.104:0/2732795747 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84840ab4b0 con 0x7f848413bea0 2021-07-11T03:53:25.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f848a7b1700 1 -- 172.21.2.104:0/2732795747 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8484142640 con 0x7f848413bea0 2021-07-11T03:53:25.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f84817fa700 1 -- 172.21.2.104:0/2732795747 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f847c01b060 con 0x7f848413bea0 2021-07-11T03:53:25.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f84817fa700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8468053210 0x7f84680556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f84817fa700 1 -- 172.21.2.104:0/2732795747 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f847c06abc0 con 0x7f848413bea0 2021-07-11T03:53:25.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f84817fa700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] conn(0x7f846805a600 0x7f846805ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f84817fa700 1 -- 172.21.2.104:0/2732795747 --> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f846805d080 con 0x7f846805a600 2021-07-11T03:53:25.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f84817fa700 1 -- 172.21.2.104:0/2732795747 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f847c070220 con 0x7f848413bea0 2021-07-11T03:53:25.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f8483fff700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8468053210 0x7f84680556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f84837fe700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] conn(0x7f846805a600 0x7f846805ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.644+0000 7f8483fff700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8468053210 0x7f84680556d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f8474002f70 tx=0x7f8474006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.648+0000 7f84837fe700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] conn(0x7f846805a600 0x7f846805ca20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.648+0000 7f84817fa700 1 -- 172.21.2.104:0/2732795747 <== osd.7 v2:172.21.2.105:6811/15656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f846805d080 con 0x7f846805a600 2021-07-11T03:53:25.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.656+0000 7f848a7b1700 1 -- 172.21.2.104:0/2732795747 --> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f848413c4f0 con 0x7f846805a600 2021-07-11T03:53:25.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f84817fa700 1 -- 172.21.2.104:0/2732795747 <== osd.7 v2:172.21.2.105:6811/15656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f848413c4f0 con 0x7f846805a600 2021-07-11T03:53:25.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f848a7b1700 1 -- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] conn(0x7f846805a600 msgr2=0x7f846805ca20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f848a7b1700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] conn(0x7f846805a600 0x7f846805ca20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f848a7b1700 1 -- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8468053210 msgr2=0x7f84680556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f848a7b1700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8468053210 0x7f84680556d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f8474002f70 tx=0x7f8474006a00).stop 2021-07-11T03:53:25.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f848a7b1700 1 -- 172.21.2.104:0/2732795747 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f848413bea0 msgr2=0x7f8484151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f848a7b1700 1 --2- 172.21.2.104:0/2732795747 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f848413bea0 0x7f8484151500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f846c0018c0 tx=0x7f847c00a060).stop 2021-07-11T03:53:25.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f8488d4e700 1 -- 172.21.2.104:0/2732795747 reap_dead start 2021-07-11T03:53:25.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f848a7b1700 1 -- 172.21.2.104:0/2732795747 shutdown_connections 2021-07-11T03:53:25.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f848a7b1700 1 -- 172.21.2.104:0/2732795747 >> 172.21.2.104:0/2732795747 conn(0x7f84840607c0 msgr2=0x7f84840b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f848a7b1700 1 -- 172.21.2.104:0/2732795747 shutdown_connections 2021-07-11T03:53:25.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.660+0000 7f848a7b1700 1 -- 172.21.2.104:0/2732795747 wait complete. 2021-07-11T03:53:25.671 INFO:teuthology.orchestra.run.gibba004.stdout:60129542149 2021-07-11T03:53:25.671 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2021-07-11T03:53:25.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7b5f83700 1 Processor -- start 2021-07-11T03:53:25.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7b5f83700 1 -- start start 2021-07-11T03:53:25.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7b5f83700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb7b00b7810 0x7fb7b01445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7b5f83700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b0144a90 0x7fb7b0064f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7b5f83700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7b00b31d0 con 0x7fb7b00b1170 2021-07-11T03:53:25.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7b5f83700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7b0144ef0 con 0x7fb7b00b7810 2021-07-11T03:53:25.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7b5f83700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b00b7bf0 con 0x7fb7b0144a90 2021-07-11T03:53:25.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7af7fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb7b00b7810 0x7fb7b01445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7affff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b0144a90 0x7fb7b0064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7aeffd700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fb7b00b1170 0x7fb7b0064870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:38132/0 (socket says 172.21.2.104:38132) 2021-07-11T03:53:25.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7aeffd700 1 -- 172.21.2.104:0/1541644041 learned_addr learned my addr 172.21.2.104:0/1541644041 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:25.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7affff700 1 -- 172.21.2.104:0/1541644041 >> v2:172.21.2.105:3300/0 conn(0x7fb7b00b7810 msgr2=0x7fb7b01445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:25.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7affff700 1 --2- 172.21.2.104:0/1541644041 >> v2:172.21.2.105:3300/0 conn(0x7fb7b00b7810 0x7fb7b01445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7affff700 1 -- 172.21.2.104:0/1541644041 >> v1:172.21.2.105:6789/0 conn(0x7fb7b00b1170 legacy=0x7fb7b0064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7affff700 1 -- 172.21.2.104:0/1541644041 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00675c0 con 0x7fb7b0144a90 2021-07-11T03:53:25.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7affff700 1 --2- 172.21.2.104:0/1541644041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b0144a90 0x7fb7b0064f80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fb7a80032f0 tx=0x7fb7a80091d0).ready entity=mon.2 client_cookie=9e6c83b852c7eae1 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.720+0000 7fb7ae7fc700 1 -- 172.21.2.104:0/1541644041 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7a800b6c0 con 0x7fb7b0144a90 2021-07-11T03:53:25.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7ae7fc700 1 -- 172.21.2.104:0/1541644041 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a800b820 con 0x7fb7b0144a90 2021-07-11T03:53:25.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7ae7fc700 1 -- 172.21.2.104:0/1541644041 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7a800b990 con 0x7fb7b0144a90 2021-07-11T03:53:25.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1541644041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b0144a90 msgr2=0x7fb7b0064f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 --2- 172.21.2.104:0/1541644041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b0144a90 0x7fb7b0064f80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fb7a80032f0 tx=0x7fb7a80091d0).stop 2021-07-11T03:53:25.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1541644041 shutdown_connections 2021-07-11T03:53:25.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 --2- 172.21.2.104:0/1541644041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b0144a90 0x7fb7b0064f80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 --2- 172.21.2.104:0/1541644041 >> v2:172.21.2.105:3300/0 conn(0x7fb7b00b7810 0x7fb7b01445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1541644041 >> 172.21.2.104:0/1541644041 conn(0x7fb7b005f6b0 msgr2=0x7fb7b005fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1541644041 shutdown_connections 2021-07-11T03:53:25.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1541644041 wait complete. 2021-07-11T03:53:25.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 Processor -- start 2021-07-11T03:53:25.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- start start 2021-07-11T03:53:25.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b00b1170 0x7fb7b00b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b00b7810 0x7fb7b00ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7b0144a90 0x7fb7b00adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b00ae090 con 0x7fb7b00b7810 2021-07-11T03:53:25.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b00ae5b0 con 0x7fb7b0144a90 2021-07-11T03:53:25.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b00ad2c0 con 0x7fb7b00b1170 2021-07-11T03:53:25.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7aeffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b00b7810 0x7fb7b00ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7aeffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b00b7810 0x7fb7b00ad6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42002/0 (socket says 172.21.2.104:42002) 2021-07-11T03:53:25.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7aeffd700 1 -- 172.21.2.104:0/1618783472 learned_addr learned my addr 172.21.2.104:0/1618783472 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:25.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7af7fe700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b00b1170 0x7fb7b00b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7affff700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7b0144a90 0x7fb7b00adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7aeffd700 1 -- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b00b1170 msgr2=0x7fb7b00b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7aeffd700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b00b1170 0x7fb7b00b5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7aeffd700 1 -- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7b0144a90 msgr2=0x7fb7b00adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7aeffd700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7b0144a90 0x7fb7b00adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7aeffd700 1 -- 172.21.2.104:0/1618783472 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a8009470 con 0x7fb7b00b7810 2021-07-11T03:53:25.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7aeffd700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b00b7810 0x7fb7b00ad6b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb7b0067ca0 tx=0x7fb798007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7acff9700 1 -- 172.21.2.104:0/1618783472 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb79800ce30 con 0x7fb7b00b7810 2021-07-11T03:53:25.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7acff9700 1 -- 172.21.2.104:0/1618783472 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7980072e0 con 0x7fb7b00b7810 2021-07-11T03:53:25.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1618783472 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b0066aa0 con 0x7fb7b00b7810 2021-07-11T03:53:25.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7acff9700 1 -- 172.21.2.104:0/1618783472 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb798005800 con 0x7fb7b00b7810 2021-07-11T03:53:25.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1618783472 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b0143d00 con 0x7fb7b00b7810 2021-07-11T03:53:25.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1618783472 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b0057350 con 0x7fb7b00b7810 2021-07-11T03:53:25.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7acff9700 1 -- 172.21.2.104:0/1618783472 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb798010070 con 0x7fb7b00b7810 2021-07-11T03:53:25.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7acff9700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb794053180 0x7fb794055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:25.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.724+0000 7fb7acff9700 1 -- 172.21.2.104:0/1618783472 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb79806ab10 con 0x7fb7b00b7810 2021-07-11T03:53:25.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.728+0000 7fb7af7fe700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb794053180 0x7fb794055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:25.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.728+0000 7fb7af7fe700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb794053180 0x7fb794055640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fb7a000b000 tx=0x7fb7a0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:25.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.728+0000 7fb7acff9700 1 -- 172.21.2.104:0/1618783472 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb798041e50 con 0x7fb7b00b7810 2021-07-11T03:53:25.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1618783472 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb7b00627b0 con 0x7fb7b00b7810 2021-07-11T03:53:25.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7acff9700 1 -- 172.21.2.104:0/1618783472 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb798045470 con 0x7fb7b00b7810 2021-07-11T03:53:25.815 INFO:teuthology.orchestra.run.gibba004.stdout:60129542146 2021-07-11T03:53:25.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb794053180 msgr2=0x7fb794055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb794053180 0x7fb794055640 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fb7a000b000 tx=0x7fb7a0008040).stop 2021-07-11T03:53:25.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1618783472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b00b7810 msgr2=0x7fb7b00ad6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:25.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b00b7810 0x7fb7b00ad6b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb7b0067ca0 tx=0x7fb798007c40).stop 2021-07-11T03:53:25.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1618783472 shutdown_connections 2021-07-11T03:53:25.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7b0144a90 0x7fb7b00adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb794053180 0x7fb794055640 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b00b7810 0x7fb7b00ad6b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 --2- 172.21.2.104:0/1618783472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b00b1170 0x7fb7b00b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:25.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1618783472 >> 172.21.2.104:0/1618783472 conn(0x7fb7b005f6b0 msgr2=0x7fb7b005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:25.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1618783472 shutdown_connections 2021-07-11T03:53:25.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:25.812+0000 7fb7b5f83700 1 -- 172.21.2.104:0/1618783472 wait complete. 2021-07-11T03:53:25.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.0 2021-07-11T03:53:26.824 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2021-07-11T03:53:26.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe25d7a6700 1 Processor -- start 2021-07-11T03:53:26.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe25d7a6700 1 -- start start 2021-07-11T03:53:26.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe25d7a6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2580aa860 0x7fe2580b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:26.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe25d7a6700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe2580a9da0 0x7fe2580aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:26.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe25d7a6700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe2580acf30 0x7fe2581434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:26.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe25d7a6700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2580abb50 con 0x7fe2580a9da0 2021-07-11T03:53:26.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe25d7a6700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2580ac370 con 0x7fe2580acf30 2021-07-11T03:53:26.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe25d7a6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2580aac40 con 0x7fe2580aa860 2021-07-11T03:53:26.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe2567fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe2580a9da0 0x7fe2580aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:26.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe2567fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe2580a9da0 0x7fe2580aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42006/0 (socket says 172.21.2.104:42006) 2021-07-11T03:53:26.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe2567fc700 1 -- 172.21.2.104:0/83010671 learned_addr learned my addr 172.21.2.104:0/83010671 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:26.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe2567fc700 1 -- 172.21.2.104:0/83010671 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2580aa860 msgr2=0x7fe2580b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:26.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe2567fc700 1 --2- 172.21.2.104:0/83010671 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2580aa860 0x7fe2580b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:26.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe2567fc700 1 -- 172.21.2.104:0/83010671 >> v2:172.21.2.105:3300/0 conn(0x7fe2580acf30 msgr2=0x7fe2581434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:26.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe2567fc700 1 --2- 172.21.2.104:0/83010671 >> v2:172.21.2.105:3300/0 conn(0x7fe2580acf30 0x7fe2581434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:26.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe2567fc700 1 -- 172.21.2.104:0/83010671 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2580b16b0 con 0x7fe2580a9da0 2021-07-11T03:53:26.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe2567fc700 1 --2- 172.21.2.104:0/83010671 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2580a9da0 0x7fe2580aca40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fe258067210 tx=0x7fe248006f40).ready entity=mon.0 client_cookie=c18f61fe529aa474 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:26.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe255ffb700 1 -- 172.21.2.104:0/83010671 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24800dc30 con 0x7fe2580a9da0 2021-07-11T03:53:26.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe255ffb700 1 -- 172.21.2.104:0/83010671 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe248000e90 con 0x7fe2580a9da0 2021-07-11T03:53:26.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe255ffb700 1 -- 172.21.2.104:0/83010671 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24800be90 con 0x7fe2580a9da0 2021-07-11T03:53:26.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe25d7a6700 1 -- 172.21.2.104:0/83010671 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2580a9da0 msgr2=0x7fe2580aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:26.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.876+0000 7fe25d7a6700 1 --2- 172.21.2.104:0/83010671 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2580a9da0 0x7fe2580aca40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fe258067210 tx=0x7fe248006f40).stop 2021-07-11T03:53:26.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 -- 172.21.2.104:0/83010671 shutdown_connections 2021-07-11T03:53:26.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 --2- 172.21.2.104:0/83010671 >> v2:172.21.2.105:3300/0 conn(0x7fe2580acf30 0x7fe2581434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:26.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 --2- 172.21.2.104:0/83010671 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2580a9da0 0x7fe2580aca40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:26.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 --2- 172.21.2.104:0/83010671 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2580aa860 0x7fe2580b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:26.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 -- 172.21.2.104:0/83010671 >> 172.21.2.104:0/83010671 conn(0x7fe2580b64d0 msgr2=0x7fe2580b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:26.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 -- 172.21.2.104:0/83010671 shutdown_connections 2021-07-11T03:53:26.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 -- 172.21.2.104:0/83010671 wait complete. 2021-07-11T03:53:26.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 Processor -- start 2021-07-11T03:53:26.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 -- start start 2021-07-11T03:53:26.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2580a9da0 0x7fe258142410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:26.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe2580aa860 0x7fe25813aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:26.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2580acf30 0x7fe25813af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:26.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25813d6a0 con 0x7fe2580a9da0 2021-07-11T03:53:26.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe258141a50 con 0x7fe2580aa860 2021-07-11T03:53:26.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe258141f20 con 0x7fe2580acf30 2021-07-11T03:53:26.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe256ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2580a9da0 0x7fe258142410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:26.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe256ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2580a9da0 0x7fe258142410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42010/0 (socket says 172.21.2.104:42010) 2021-07-11T03:53:26.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe256ffd700 1 -- 172.21.2.104:0/2208803524 learned_addr learned my addr 172.21.2.104:0/2208803524 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:26.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe256ffd700 1 -- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2580acf30 msgr2=0x7fe25813af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:26.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe256ffd700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2580acf30 0x7fe25813af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:26.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe256ffd700 1 -- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe2580aa860 msgr2=0x7fe25813aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:26.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe256ffd700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe2580aa860 0x7fe25813aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:26.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe256ffd700 1 -- 172.21.2.104:0/2208803524 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24800b460 con 0x7fe2580a9da0 2021-07-11T03:53:26.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe256ffd700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2580a9da0 0x7fe258142410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fe24c002f70 tx=0x7fe24c006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:26.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe247fff700 1 -- 172.21.2.104:0/2208803524 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24c013460 con 0x7fe2580a9da0 2021-07-11T03:53:26.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe247fff700 1 -- 172.21.2.104:0/2208803524 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c0135c0 con 0x7fe2580a9da0 2021-07-11T03:53:26.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe247fff700 1 -- 172.21.2.104:0/2208803524 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe24c013920 con 0x7fe2580a9da0 2021-07-11T03:53:26.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 -- 172.21.2.104:0/2208803524 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe258141c80 con 0x7fe2580a9da0 2021-07-11T03:53:26.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 -- 172.21.2.104:0/2208803524 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe258143ff0 con 0x7fe2580a9da0 2021-07-11T03:53:26.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe25d7a6700 1 -- 172.21.2.104:0/2208803524 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe258142f10 con 0x7fe2580a9da0 2021-07-11T03:53:26.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe247fff700 1 -- 172.21.2.104:0/2208803524 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe24c01a060 con 0x7fe2580a9da0 2021-07-11T03:53:26.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe247fff700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2340531a0 0x7fe234055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:26.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.880+0000 7fe247fff700 1 -- 172.21.2.104:0/2208803524 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe24c06b030 con 0x7fe2580a9da0 2021-07-11T03:53:26.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.884+0000 7fe2567fc700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2340531a0 0x7fe234055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:26.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.884+0000 7fe2567fc700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2340531a0 0x7fe234055660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fe258067210 tx=0x7fe2580ac290).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:26.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.884+0000 7fe247fff700 1 -- 172.21.2.104:0/2208803524 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe258142f10 con 0x7fe2580a9da0 2021-07-11T03:53:26.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 -- 172.21.2.104:0/2208803524 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe258142f10 con 0x7fe2580a9da0 2021-07-11T03:53:26.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe247fff700 1 -- 172.21.2.104:0/2208803524 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe258142f10 con 0x7fe2580a9da0 2021-07-11T03:53:26.969 INFO:teuthology.orchestra.run.gibba004.stdout:60129542146 2021-07-11T03:53:26.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 -- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2340531a0 msgr2=0x7fe234055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:26.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2340531a0 0x7fe234055660 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fe258067210 tx=0x7fe2580ac290).stop 2021-07-11T03:53:26.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 -- 172.21.2.104:0/2208803524 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2580a9da0 msgr2=0x7fe258142410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:26.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2580a9da0 0x7fe258142410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fe24c002f70 tx=0x7fe24c006ec0).stop 2021-07-11T03:53:26.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 -- 172.21.2.104:0/2208803524 shutdown_connections 2021-07-11T03:53:26.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2580acf30 0x7fe25813af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:26.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe2580aa860 0x7fe25813aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:26.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2340531a0 0x7fe234055660 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:26.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 --2- 172.21.2.104:0/2208803524 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2580a9da0 0x7fe258142410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:26.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 -- 172.21.2.104:0/2208803524 >> 172.21.2.104:0/2208803524 conn(0x7fe2580b64d0 msgr2=0x7fe258060a10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:26.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 -- 172.21.2.104:0/2208803524 shutdown_connections 2021-07-11T03:53:26.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:26.964+0000 7fe25d7a6700 1 -- 172.21.2.104:0/2208803524 wait complete. 2021-07-11T03:53:26.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.0 2021-07-11T03:53:27.980 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2021-07-11T03:53:28.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.032+0000 7fa6fa7cc700 1 Processor -- start 2021-07-11T03:53:28.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.032+0000 7fa6fa7cc700 1 -- start start 2021-07-11T03:53:28.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.032+0000 7fa6fa7cc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6f40aa860 0x7fa6f40b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.032+0000 7fa6fa7cc700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa6f40ad150 0x7fa6f4140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.032+0000 7fa6fa7cc700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f40ac370 con 0x7fa6f40a9da0 2021-07-11T03:53:28.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.032+0000 7fa6fa7cc700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f40abb50 con 0x7fa6f40ad150 2021-07-11T03:53:28.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.032+0000 7fa6fa7cc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40aac40 con 0x7fa6f40aa860 2021-07-11T03:53:28.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f3fff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6f40aa860 0x7fa6f40b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f8d69700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa6f40ad150 0x7fa6f4140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f8d69700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa6f40ad150 0x7fa6f4140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:49582/0 (socket says 172.21.2.104:49582) 2021-07-11T03:53:28.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f3fff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6f40aa860 0x7fa6f40b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:45182/0 (socket says 172.21.2.104:45182) 2021-07-11T03:53:28.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f8d69700 1 -- 172.21.2.104:0/2128615162 learned_addr learned my addr 172.21.2.104:0/2128615162 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:28.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f37fe700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fa6f40a9da0 0x7fa6f40aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:38158/0 (socket says 172.21.2.104:38158) 2021-07-11T03:53:28.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f8d69700 1 -- 172.21.2.104:0/2128615162 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6f40aa860 msgr2=0x7fa6f40b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f8d69700 1 --2- 172.21.2.104:0/2128615162 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6f40aa860 0x7fa6f40b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f8d69700 1 -- 172.21.2.104:0/2128615162 >> v1:172.21.2.105:6789/0 conn(0x7fa6f40a9da0 legacy=0x7fa6f40aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f8d69700 1 -- 172.21.2.104:0/2128615162 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f40b16b0 con 0x7fa6f40ad150 2021-07-11T03:53:28.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f8d69700 1 --2- 172.21.2.104:0/2128615162 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6f40ad150 0x7fa6f4140340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa6ec0032f0 tx=0x7fa6ec009290).ready entity=mon.1 client_cookie=56fecceb88851e68 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f2ffd700 1 -- 172.21.2.104:0/2128615162 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6ec00b8a0 con 0x7fa6f40ad150 2021-07-11T03:53:28.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f2ffd700 1 -- 172.21.2.104:0/2128615162 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec00ba00 con 0x7fa6f40ad150 2021-07-11T03:53:28.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/2128615162 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6f40ad150 msgr2=0x7fa6f4140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 --2- 172.21.2.104:0/2128615162 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6f40ad150 0x7fa6f4140340 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa6ec0032f0 tx=0x7fa6ec009290).stop 2021-07-11T03:53:28.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/2128615162 shutdown_connections 2021-07-11T03:53:28.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 --2- 172.21.2.104:0/2128615162 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6f40ad150 0x7fa6f4140340 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 --2- 172.21.2.104:0/2128615162 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6f40aa860 0x7fa6f40b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/2128615162 >> 172.21.2.104:0/2128615162 conn(0x7fa6f40b64d0 msgr2=0x7fa6f40b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/2128615162 shutdown_connections 2021-07-11T03:53:28.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/2128615162 wait complete. 2021-07-11T03:53:28.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 Processor -- start 2021-07-11T03:53:28.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 -- start start 2021-07-11T03:53:28.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6f40a9da0 0x7fa6f40b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6f40aa860 0x7fa6f40a8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6f40ad150 0x7fa6f4065b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4064fa0 con 0x7fa6f40aa860 2021-07-11T03:53:28.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4066fa0 con 0x7fa6f40a9da0 2021-07-11T03:53:28.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6fa7cc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f40667c0 con 0x7fa6f40ad150 2021-07-11T03:53:28.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f37fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6f40aa860 0x7fa6f40a8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f37fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6f40aa860 0x7fa6f40a8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42028/0 (socket says 172.21.2.104:42028) 2021-07-11T03:53:28.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.036+0000 7fa6f37fe700 1 -- 172.21.2.104:0/3867687221 learned_addr learned my addr 172.21.2.104:0/3867687221 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:28.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f37fe700 1 -- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6f40ad150 msgr2=0x7fa6f4065b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f37fe700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6f40ad150 0x7fa6f4065b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f3fff700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6f40a9da0 0x7fa6f40b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f37fe700 1 -- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6f40a9da0 msgr2=0x7fa6f40b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f37fe700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6f40a9da0 0x7fa6f40b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f37fe700 1 -- 172.21.2.104:0/3867687221 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ec009530 con 0x7fa6f40aa860 2021-07-11T03:53:28.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f37fe700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6f40aa860 0x7fa6f40a8cf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa6f4065770 tx=0x7fa6e8007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f17fa700 1 -- 172.21.2.104:0/3867687221 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6e800ce20 con 0x7fa6f40aa860 2021-07-11T03:53:28.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f17fa700 1 -- 172.21.2.104:0/3867687221 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e80072e0 con 0x7fa6f40aa860 2021-07-11T03:53:28.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/3867687221 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f4066a60 con 0x7fa6f40aa860 2021-07-11T03:53:28.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f17fa700 1 -- 172.21.2.104:0/3867687221 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa6e8007450 con 0x7fa6f40aa860 2021-07-11T03:53:28.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/3867687221 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f40636e0 con 0x7fa6f40aa860 2021-07-11T03:53:28.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/3867687221 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f4065820 con 0x7fa6f40aa860 2021-07-11T03:53:28.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f17fa700 1 -- 172.21.2.104:0/3867687221 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa6e8010070 con 0x7fa6f40aa860 2021-07-11T03:53:28.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f17fa700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa6d40531f0 0x7fa6d40556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f17fa700 1 -- 172.21.2.104:0/3867687221 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa6e806ad10 con 0x7fa6f40aa860 2021-07-11T03:53:28.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f3fff700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa6d40531f0 0x7fa6d40556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.040+0000 7fa6f3fff700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa6d40531f0 0x7fa6d40556b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa6e400b000 tx=0x7fa6e4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.044+0000 7fa6f17fa700 1 -- 172.21.2.104:0/3867687221 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa6e8070020 con 0x7fa6f40aa860 2021-07-11T03:53:28.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/3867687221 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa6f4057350 con 0x7fa6f40aa860 2021-07-11T03:53:28.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6f17fa700 1 -- 172.21.2.104:0/3867687221 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa6e8041ee0 con 0x7fa6f40aa860 2021-07-11T03:53:28.128 INFO:teuthology.orchestra.run.gibba004.stdout:60129542148 2021-07-11T03:53:28.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa6d40531f0 msgr2=0x7fa6d40556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa6d40531f0 0x7fa6d40556b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fa6e400b000 tx=0x7fa6e4008040).stop 2021-07-11T03:53:28.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/3867687221 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6f40aa860 msgr2=0x7fa6f40a8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6f40aa860 0x7fa6f40a8cf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa6f4065770 tx=0x7fa6e8007c40).stop 2021-07-11T03:53:28.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/3867687221 shutdown_connections 2021-07-11T03:53:28.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6f40ad150 0x7fa6f4065b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa6d40531f0 0x7fa6d40556b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6f40aa860 0x7fa6f40a8cf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 --2- 172.21.2.104:0/3867687221 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6f40a9da0 0x7fa6f40b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/3867687221 >> 172.21.2.104:0/3867687221 conn(0x7fa6f40b64d0 msgr2=0x7fa6f40a70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/3867687221 shutdown_connections 2021-07-11T03:53:28.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.124+0000 7fa6fa7cc700 1 -- 172.21.2.104:0/3867687221 wait complete. 2021-07-11T03:53:28.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542148 for osd.0 2021-07-11T03:53:28.135 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2021-07-11T03:53:28.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 Processor -- start 2021-07-11T03:53:28.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 -- start start 2021-07-11T03:53:28.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f90b00aa860 0x7f90b00b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f90b00a9da0 0x7f90b00aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b00ac370 con 0x7f90b00acf30 2021-07-11T03:53:28.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b00abb50 con 0x7f90b00a9da0 2021-07-11T03:53:28.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b00aac40 con 0x7f90b00aa860 2021-07-11T03:53:28.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90ae7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f90b00a9da0 0x7f90b00aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90ae7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f90b00a9da0 0x7f90b00aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42034/0 (socket says 172.21.2.104:42034) 2021-07-11T03:53:28.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90ae7fc700 1 -- 172.21.2.104:0/1693395187 learned_addr learned my addr 172.21.2.104:0/1693395187 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:28.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90ae7fc700 1 -- 172.21.2.104:0/1693395187 >> v2:172.21.2.105:3300/0 conn(0x7f90b00aa860 msgr2=0x7f90b00b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90ae7fc700 1 --2- 172.21.2.104:0/1693395187 >> v2:172.21.2.105:3300/0 conn(0x7f90b00aa860 0x7f90b00b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90ae7fc700 1 -- 172.21.2.104:0/1693395187 >> v1:172.21.2.105:6789/0 conn(0x7f90b00acf30 legacy=0x7f90b01434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90ae7fc700 1 -- 172.21.2.104:0/1693395187 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b00b16b0 con 0x7f90b00a9da0 2021-07-11T03:53:28.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90ae7fc700 1 --2- 172.21.2.104:0/1693395187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90b00a9da0 0x7f90b00aca40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f90b00b63e0 tx=0x7f90a0006eb0).ready entity=mon.0 client_cookie=3a5f4e05247a1d44 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90adffb700 1 -- 172.21.2.104:0/1693395187 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90a000d5c0 con 0x7f90b00a9da0 2021-07-11T03:53:28.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90adffb700 1 -- 172.21.2.104:0/1693395187 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a0000e90 con 0x7f90b00a9da0 2021-07-11T03:53:28.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 -- 172.21.2.104:0/1693395187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90b00a9da0 msgr2=0x7f90b00aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90adffb700 1 -- 172.21.2.104:0/1693395187 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90a000d910 con 0x7f90b00a9da0 2021-07-11T03:53:28.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 --2- 172.21.2.104:0/1693395187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90b00a9da0 0x7f90b00aca40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f90b00b63e0 tx=0x7f90a0006eb0).stop 2021-07-11T03:53:28.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 -- 172.21.2.104:0/1693395187 shutdown_connections 2021-07-11T03:53:28.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 --2- 172.21.2.104:0/1693395187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90b00a9da0 0x7f90b00aca40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 --2- 172.21.2.104:0/1693395187 >> v2:172.21.2.105:3300/0 conn(0x7f90b00aa860 0x7f90b00b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 -- 172.21.2.104:0/1693395187 >> 172.21.2.104:0/1693395187 conn(0x7f90b00b64d0 msgr2=0x7f90b00b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 -- 172.21.2.104:0/1693395187 shutdown_connections 2021-07-11T03:53:28.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.184+0000 7f90b578e700 1 -- 172.21.2.104:0/1693395187 wait complete. 2021-07-11T03:53:28.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90b578e700 1 Processor -- start 2021-07-11T03:53:28.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90b578e700 1 -- start start 2021-07-11T03:53:28.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90b578e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90b00a9da0 0x7f90b0066200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90b578e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90b00aa860 0x7f90b0063c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90b578e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90b00acf30 0x7f90b0064180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90b578e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0146610 con 0x7f90b00a9da0 2021-07-11T03:53:28.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90b578e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b0066bd0 con 0x7f90b00acf30 2021-07-11T03:53:28.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90b578e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b01459b0 con 0x7f90b00aa860 2021-07-11T03:53:28.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90aeffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90b00a9da0 0x7f90b0066200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90aeffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90b00a9da0 0x7f90b0066200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42038/0 (socket says 172.21.2.104:42038) 2021-07-11T03:53:28.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90aeffd700 1 -- 172.21.2.104:0/1309616235 learned_addr learned my addr 172.21.2.104:0/1309616235 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:28.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90af7fe700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90b00acf30 0x7f90b0064180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90ae7fc700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90b00aa860 0x7f90b0063c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90aeffd700 1 -- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90b00aa860 msgr2=0x7f90b0063c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90aeffd700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90b00aa860 0x7f90b0063c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90aeffd700 1 -- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90b00acf30 msgr2=0x7f90b0064180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90aeffd700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90b00acf30 0x7f90b0064180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90aeffd700 1 -- 172.21.2.104:0/1309616235 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a000b1e0 con 0x7f90b00a9da0 2021-07-11T03:53:28.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90aeffd700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90b00a9da0 0x7f90b0066200 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f9098002f70 tx=0x7f9098006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90a7fff700 1 -- 172.21.2.104:0/1309616235 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f909800f510 con 0x7f90b00a9da0 2021-07-11T03:53:28.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90a7fff700 1 -- 172.21.2.104:0/1309616235 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909800f670 con 0x7f90b00a9da0 2021-07-11T03:53:28.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90b578e700 1 -- 172.21.2.104:0/1309616235 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b0066e00 con 0x7f90b00a9da0 2021-07-11T03:53:28.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90a7fff700 1 -- 172.21.2.104:0/1309616235 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f909800f7e0 con 0x7f90b00a9da0 2021-07-11T03:53:28.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90b578e700 1 -- 172.21.2.104:0/1309616235 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b00637f0 con 0x7f90b00a9da0 2021-07-11T03:53:28.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90b578e700 1 -- 172.21.2.104:0/1309616235 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b0057350 con 0x7f90b00a9da0 2021-07-11T03:53:28.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90a7fff700 1 -- 172.21.2.104:0/1309616235 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f909800f070 con 0x7f90b00a9da0 2021-07-11T03:53:28.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90a7fff700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90940531f0 0x7f90940556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90a7fff700 1 -- 172.21.2.104:0/1309616235 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f909806aed0 con 0x7f90b00a9da0 2021-07-11T03:53:28.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90ae7fc700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90940531f0 0x7f90940556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.188+0000 7f90ae7fc700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90940531f0 0x7f90940556b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f90a0019000 tx=0x7f90a0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.192+0000 7f90a7fff700 1 -- 172.21.2.104:0/1309616235 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9098042210 con 0x7f90b00a9da0 2021-07-11T03:53:28.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90b578e700 1 -- 172.21.2.104:0/1309616235 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f90b0145b20 con 0x7f90b00a9da0 2021-07-11T03:53:28.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90a7fff700 1 -- 172.21.2.104:0/1309616235 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9098045830 con 0x7f90b00a9da0 2021-07-11T03:53:28.277 INFO:teuthology.orchestra.run.gibba004.stdout:60129542150 2021-07-11T03:53:28.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90b578e700 1 -- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90940531f0 msgr2=0x7f90940556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90b578e700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90940531f0 0x7f90940556b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f90a0019000 tx=0x7f90a0015000).stop 2021-07-11T03:53:28.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90b578e700 1 -- 172.21.2.104:0/1309616235 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90b00a9da0 msgr2=0x7f90b0066200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90b578e700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90b00a9da0 0x7f90b0066200 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f9098002f70 tx=0x7f9098006ef0).stop 2021-07-11T03:53:28.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90b578e700 1 -- 172.21.2.104:0/1309616235 shutdown_connections 2021-07-11T03:53:28.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90b578e700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90b00acf30 0x7f90b0064180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90b578e700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90b00aa860 0x7f90b0063c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90b578e700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90940531f0 0x7f90940556b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90b578e700 1 --2- 172.21.2.104:0/1309616235 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90b00a9da0 0x7f90b0066200 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.272+0000 7f90b578e700 1 -- 172.21.2.104:0/1309616235 >> 172.21.2.104:0/1309616235 conn(0x7f90b00b64d0 msgr2=0x7f90b00b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.276+0000 7f90b578e700 1 -- 172.21.2.104:0/1309616235 shutdown_connections 2021-07-11T03:53:28.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.276+0000 7f90b578e700 1 -- 172.21.2.104:0/1309616235 wait complete. 2021-07-11T03:53:28.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.1 2021-07-11T03:53:28.284 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2021-07-11T03:53:28.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f1520738700 1 Processor -- start 2021-07-11T03:53:28.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f1520738700 1 -- start start 2021-07-11T03:53:28.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f1520738700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f15180b1170 0x7f1518141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f1520738700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1518141920 0x7f1518141d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f1520738700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15180b31d0 con 0x7f15180b7810 2021-07-11T03:53:28.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f1520738700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15180b7bf0 con 0x7f15180b1170 2021-07-11T03:53:28.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f1520738700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15180675c0 con 0x7f1518141920 2021-07-11T03:53:28.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f151e4d4700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f15180b7810 0x7f15181445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38814/0 (socket says 172.21.2.104:38814) 2021-07-11T03:53:28.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f151e4d4700 1 -- 172.21.2.104:0/3883057505 learned_addr learned my addr 172.21.2.104:0/3883057505 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:28.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f151dcd3700 1 --2- 172.21.2.104:0/3883057505 >> v2:172.21.2.104:3300/0 conn(0x7f15180b1170 0x7f1518141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f151dcd3700 1 -- 172.21.2.104:0/3883057505 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1518141920 msgr2=0x7f1518141d80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:28.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f151dcd3700 1 --2- 172.21.2.104:0/3883057505 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1518141920 0x7f1518141d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f151dcd3700 1 -- 172.21.2.104:0/3883057505 >> v1:172.21.2.104:6789/0 conn(0x7f15180b7810 legacy=0x7f15181445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f151dcd3700 1 -- 172.21.2.104:0/3883057505 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1518142270 con 0x7f15180b1170 2021-07-11T03:53:28.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.332+0000 7f151dcd3700 1 --2- 172.21.2.104:0/3883057505 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15180b1170 0x7f1518141430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f1508002f70 tx=0x7f1508006df0).ready entity=mon.0 client_cookie=39601fe2c61b1ec5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151d4d2700 1 -- 172.21.2.104:0/3883057505 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f150800d570 con 0x7f15180b1170 2021-07-11T03:53:28.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151d4d2700 1 -- 172.21.2.104:0/3883057505 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1508000e90 con 0x7f15180b1170 2021-07-11T03:53:28.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151d4d2700 1 -- 172.21.2.104:0/3883057505 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f150800d900 con 0x7f15180b1170 2021-07-11T03:53:28.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- 172.21.2.104:0/3883057505 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15180b1170 msgr2=0x7f1518141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 --2- 172.21.2.104:0/3883057505 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15180b1170 0x7f1518141430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f1508002f70 tx=0x7f1508006df0).stop 2021-07-11T03:53:28.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- 172.21.2.104:0/3883057505 shutdown_connections 2021-07-11T03:53:28.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 --2- 172.21.2.104:0/3883057505 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15180b1170 0x7f1518141430 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 --2- 172.21.2.104:0/3883057505 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1518141920 0x7f1518141d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- 172.21.2.104:0/3883057505 >> 172.21.2.104:0/3883057505 conn(0x7f151805f6b0 msgr2=0x7f151805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- 172.21.2.104:0/3883057505 shutdown_connections 2021-07-11T03:53:28.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- 172.21.2.104:0/3883057505 wait complete. 2021-07-11T03:53:28.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 Processor -- start 2021-07-11T03:53:28.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- start start 2021-07-11T03:53:28.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f15180b1170 0x7f1518140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f15180b7810 0x7f15181433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1518141920 0x7f15181438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15180abdb0 con 0x7f1518141920 2021-07-11T03:53:28.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1518144770 con 0x7f15180b1170 2021-07-11T03:53:28.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15180ac290 con 0x7f15180b7810 2021-07-11T03:53:28.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151ecd5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1518141920 0x7f15181438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151ecd5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1518141920 0x7f15181438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42056/0 (socket says 172.21.2.104:42056) 2021-07-11T03:53:28.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151ecd5700 1 -- 172.21.2.104:0/3936953898 learned_addr learned my addr 172.21.2.104:0/3936953898 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:28.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151ecd5700 1 -- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f15180b7810 msgr2=0x7f15181433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151ecd5700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f15180b7810 0x7f15181433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151ecd5700 1 -- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f15180b1170 msgr2=0x7f1518140970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151ecd5700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f15180b1170 0x7f1518140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151ecd5700 1 -- 172.21.2.104:0/3936953898 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150800b180 con 0x7f1518141920 2021-07-11T03:53:28.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f151ecd5700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1518141920 0x7f15181438c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f1510003270 tx=0x7f15100090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f15077fe700 1 -- 172.21.2.104:0/3936953898 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f151000d4a0 con 0x7f1518141920 2021-07-11T03:53:28.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f15077fe700 1 -- 172.21.2.104:0/3936953898 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151000d600 con 0x7f1518141920 2021-07-11T03:53:28.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- 172.21.2.104:0/3936953898 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15180ac530 con 0x7f1518141920 2021-07-11T03:53:28.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f15077fe700 1 -- 172.21.2.104:0/3936953898 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f151000d960 con 0x7f1518141920 2021-07-11T03:53:28.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- 172.21.2.104:0/3936953898 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15180ab910 con 0x7f1518141920 2021-07-11T03:53:28.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f1520738700 1 -- 172.21.2.104:0/3936953898 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1518143f50 con 0x7f1518141920 2021-07-11T03:53:28.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f15077fe700 1 -- 172.21.2.104:0/3936953898 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1510018030 con 0x7f1518141920 2021-07-11T03:53:28.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f15077fe700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14f8053270 0x7f14f8055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.336+0000 7f15077fe700 1 -- 172.21.2.104:0/3936953898 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f151006acc0 con 0x7f1518141920 2021-07-11T03:53:28.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.340+0000 7f151e4d4700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14f8053270 0x7f14f8055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.340+0000 7f151e4d4700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14f8053270 0x7f14f8055730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f1518067ca0 tx=0x7f1514007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.340+0000 7f15077fe700 1 -- 172.21.2.104:0/3936953898 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1518143f50 con 0x7f1518141920 2021-07-11T03:53:28.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 -- 172.21.2.104:0/3936953898 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1518143f50 con 0x7f1518141920 2021-07-11T03:53:28.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f15077fe700 1 -- 172.21.2.104:0/3936953898 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1518143f50 con 0x7f1518141920 2021-07-11T03:53:28.428 INFO:teuthology.orchestra.run.gibba004.stdout:60129542150 2021-07-11T03:53:28.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 -- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14f8053270 msgr2=0x7f14f8055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14f8053270 0x7f14f8055730 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f1518067ca0 tx=0x7f1514007800).stop 2021-07-11T03:53:28.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 -- 172.21.2.104:0/3936953898 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1518141920 msgr2=0x7f15181438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1518141920 0x7f15181438c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f1510003270 tx=0x7f15100090f0).stop 2021-07-11T03:53:28.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 -- 172.21.2.104:0/3936953898 shutdown_connections 2021-07-11T03:53:28.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14f8053270 0x7f14f8055730 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1518141920 0x7f15181438c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f15180b7810 0x7f15181433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 --2- 172.21.2.104:0/3936953898 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f15180b1170 0x7f1518140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 -- 172.21.2.104:0/3936953898 >> 172.21.2.104:0/3936953898 conn(0x7f151805f6b0 msgr2=0x7f151805cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 -- 172.21.2.104:0/3936953898 shutdown_connections 2021-07-11T03:53:28.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.424+0000 7f1520738700 1 -- 172.21.2.104:0/3936953898 wait complete. 2021-07-11T03:53:28.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.2 2021-07-11T03:53:28.435 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2021-07-11T03:53:28.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 Processor -- start 2021-07-11T03:53:28.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 -- start start 2021-07-11T03:53:28.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fde1c0b96e0 0x7fde1c0aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fde1c13fff0 0x7fde1c140480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde1c067500 con 0x7fde1c067120 2021-07-11T03:53:28.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1c143080 con 0x7fde1c13fff0 2021-07-11T03:53:28.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1c0af4d0 con 0x7fde1c0b96e0 2021-07-11T03:53:28.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde1b7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fde1c13fff0 0x7fde1c140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde1b7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fde1c13fff0 0x7fde1c140480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42062/0 (socket says 172.21.2.104:42062) 2021-07-11T03:53:28.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde1b7fe700 1 -- 172.21.2.104:0/2512620882 learned_addr learned my addr 172.21.2.104:0/2512620882 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:28.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde1b7fe700 1 -- 172.21.2.104:0/2512620882 >> v2:172.21.2.105:3300/0 conn(0x7fde1c0b96e0 msgr2=0x7fde1c0aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde1b7fe700 1 --2- 172.21.2.104:0/2512620882 >> v2:172.21.2.105:3300/0 conn(0x7fde1c0b96e0 0x7fde1c0aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde1b7fe700 1 -- 172.21.2.104:0/2512620882 >> v1:172.21.2.105:6789/0 conn(0x7fde1c067120 legacy=0x7fde1c1439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde1b7fe700 1 -- 172.21.2.104:0/2512620882 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c143270 con 0x7fde1c13fff0 2021-07-11T03:53:28.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde1b7fe700 1 --2- 172.21.2.104:0/2512620882 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde1c13fff0 0x7fde1c140480 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fde1c060280 tx=0x7fde14009290).ready entity=mon.0 client_cookie=2e7677308d3cf87a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde19ffb700 1 -- 172.21.2.104:0/2512620882 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde1400b830 con 0x7fde1c13fff0 2021-07-11T03:53:28.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde19ffb700 1 -- 172.21.2.104:0/2512620882 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1400b990 con 0x7fde1c13fff0 2021-07-11T03:53:28.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde19ffb700 1 -- 172.21.2.104:0/2512620882 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde1400bb00 con 0x7fde1c13fff0 2021-07-11T03:53:28.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 -- 172.21.2.104:0/2512620882 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde1c13fff0 msgr2=0x7fde1c140480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 --2- 172.21.2.104:0/2512620882 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde1c13fff0 0x7fde1c140480 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fde1c060280 tx=0x7fde14009290).stop 2021-07-11T03:53:28.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 -- 172.21.2.104:0/2512620882 shutdown_connections 2021-07-11T03:53:28.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 --2- 172.21.2.104:0/2512620882 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde1c13fff0 0x7fde1c140480 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 --2- 172.21.2.104:0/2512620882 >> v2:172.21.2.105:3300/0 conn(0x7fde1c0b96e0 0x7fde1c0aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 -- 172.21.2.104:0/2512620882 >> 172.21.2.104:0/2512620882 conn(0x7fde1c05f9f0 msgr2=0x7fde1c0616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 -- 172.21.2.104:0/2512620882 shutdown_connections 2021-07-11T03:53:28.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.484+0000 7fde214a2700 1 -- 172.21.2.104:0/2512620882 wait complete. 2021-07-11T03:53:28.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde214a2700 1 Processor -- start 2021-07-11T03:53:28.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde214a2700 1 -- start start 2021-07-11T03:53:28.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde214a2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde1c067120 0x7fde1c145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde214a2700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde1c0b96e0 0x7fde1c1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde214a2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde1c13fff0 0x7fde1c1527b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde214a2700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0b90e0 con 0x7fde1c13fff0 2021-07-11T03:53:28.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde214a2700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0b2aa0 con 0x7fde1c0b96e0 2021-07-11T03:53:28.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde214a2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1c0b8e40 con 0x7fde1c067120 2021-07-11T03:53:28.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde1b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde1c13fff0 0x7fde1c1527b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde1b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde1c13fff0 0x7fde1c1527b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42070/0 (socket says 172.21.2.104:42070) 2021-07-11T03:53:28.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde1b7fe700 1 -- 172.21.2.104:0/3358549824 learned_addr learned my addr 172.21.2.104:0/3358549824 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:28.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde1b7fe700 1 -- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde1c067120 msgr2=0x7fde1c145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde1b7fe700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde1c067120 0x7fde1c145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde1b7fe700 1 -- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde1c0b96e0 msgr2=0x7fde1c1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde1b7fe700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde1c0b96e0 0x7fde1c1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde1b7fe700 1 -- 172.21.2.104:0/3358549824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde14009530 con 0x7fde1c13fff0 2021-07-11T03:53:28.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde1b7fe700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde1c13fff0 0x7fde1c1527b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fde1c060280 tx=0x7fde14016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde13fff700 1 -- 172.21.2.104:0/3358549824 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde14011c80 con 0x7fde1c13fff0 2021-07-11T03:53:28.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde13fff700 1 -- 172.21.2.104:0/3358549824 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde14011de0 con 0x7fde1c13fff0 2021-07-11T03:53:28.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde13fff700 1 -- 172.21.2.104:0/3358549824 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde1401f610 con 0x7fde1c13fff0 2021-07-11T03:53:28.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde214a2700 1 -- 172.21.2.104:0/3358549824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1c0b2d00 con 0x7fde1c13fff0 2021-07-11T03:53:28.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde214a2700 1 -- 172.21.2.104:0/3358549824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1c0af900 con 0x7fde1c13fff0 2021-07-11T03:53:28.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde214a2700 1 -- 172.21.2.104:0/3358549824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1c0b2e90 con 0x7fde1c13fff0 2021-07-11T03:53:28.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde13fff700 1 -- 172.21.2.104:0/3358549824 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fde14026030 con 0x7fde1c13fff0 2021-07-11T03:53:28.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde13fff700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde00053240 0x7fde00055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde13fff700 1 -- 172.21.2.104:0/3358549824 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fde1406bff0 con 0x7fde1c13fff0 2021-07-11T03:53:28.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.488+0000 7fde1affd700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde00053240 0x7fde00055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.492+0000 7fde1affd700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde00053240 0x7fde00055700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fde0c002f90 tx=0x7fde0c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.492+0000 7fde13fff700 1 -- 172.21.2.104:0/3358549824 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fde14043240 con 0x7fde1c13fff0 2021-07-11T03:53:28.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.572+0000 7fde214a2700 1 -- 172.21.2.104:0/3358549824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fde1c057350 con 0x7fde1c13fff0 2021-07-11T03:53:28.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.572+0000 7fde13fff700 1 -- 172.21.2.104:0/3358549824 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fde14015020 con 0x7fde1c13fff0 2021-07-11T03:53:28.578 INFO:teuthology.orchestra.run.gibba004.stdout:60129542149 2021-07-11T03:53:28.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 -- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde00053240 msgr2=0x7fde00055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde00053240 0x7fde00055700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fde0c002f90 tx=0x7fde0c0069b0).stop 2021-07-11T03:53:28.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 -- 172.21.2.104:0/3358549824 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde1c13fff0 msgr2=0x7fde1c1527b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde1c13fff0 0x7fde1c1527b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fde1c060280 tx=0x7fde14016000).stop 2021-07-11T03:53:28.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 -- 172.21.2.104:0/3358549824 shutdown_connections 2021-07-11T03:53:28.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde00053240 0x7fde00055700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde1c13fff0 0x7fde1c1527b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde1c0b96e0 0x7fde1c1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 --2- 172.21.2.104:0/3358549824 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde1c067120 0x7fde1c145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 -- 172.21.2.104:0/3358549824 >> 172.21.2.104:0/3358549824 conn(0x7fde1c05f9f0 msgr2=0x7fde1c142850 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 -- 172.21.2.104:0/3358549824 shutdown_connections 2021-07-11T03:53:28.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.576+0000 7fde214a2700 1 -- 172.21.2.104:0/3358549824 wait complete. 2021-07-11T03:53:28.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.3 2021-07-11T03:53:28.585 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2021-07-11T03:53:28.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f746aa27700 1 Processor -- start 2021-07-11T03:53:28.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f746aa27700 1 -- start start 2021-07-11T03:53:28.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f746aa27700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f74640b7810 0x7f74641445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f746aa27700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7464144a90 0x7f7464064f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f746aa27700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74640b31d0 con 0x7f74640b1170 2021-07-11T03:53:28.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f746aa27700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7464144f20 con 0x7f7464144a90 2021-07-11T03:53:28.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f746aa27700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74640b7bf0 con 0x7f74640b7810 2021-07-11T03:53:28.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f74637fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f74640b1170 0x7f7464064870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38844/0 (socket says 172.21.2.104:38844) 2021-07-11T03:53:28.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f7468fc4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7464144a90 0x7f7464064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f74637fe700 1 -- 172.21.2.104:0/2579572309 learned_addr learned my addr 172.21.2.104:0/2579572309 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:28.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f7468fc4700 1 -- 172.21.2.104:0/2579572309 >> v2:172.21.2.105:3300/0 conn(0x7f74640b7810 msgr2=0x7f74641445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f7468fc4700 1 --2- 172.21.2.104:0/2579572309 >> v2:172.21.2.105:3300/0 conn(0x7f74640b7810 0x7f74641445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f7468fc4700 1 -- 172.21.2.104:0/2579572309 >> v1:172.21.2.104:6789/0 conn(0x7f74640b1170 legacy=0x7f7464064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f7468fc4700 1 -- 172.21.2.104:0/2579572309 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74640675c0 con 0x7f7464144a90 2021-07-11T03:53:28.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f7468fc4700 1 --2- 172.21.2.104:0/2579572309 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7464144a90 0x7f7464064f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f745c0032f0 tx=0x7f745c0091d0).ready entity=mon.0 client_cookie=24fad9818dd0cd2a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f7462ffd700 1 -- 172.21.2.104:0/2579572309 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745c009ea0 con 0x7f7464144a90 2021-07-11T03:53:28.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f7462ffd700 1 -- 172.21.2.104:0/2579572309 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c00b870 con 0x7f7464144a90 2021-07-11T03:53:28.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f7462ffd700 1 -- 172.21.2.104:0/2579572309 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745c00b9e0 con 0x7f7464144a90 2021-07-11T03:53:28.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f746aa27700 1 -- 172.21.2.104:0/2579572309 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7464144a90 msgr2=0x7f7464064f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.632+0000 7f746aa27700 1 --2- 172.21.2.104:0/2579572309 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7464144a90 0x7f7464064f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f745c0032f0 tx=0x7f745c0091d0).stop 2021-07-11T03:53:28.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 -- 172.21.2.104:0/2579572309 shutdown_connections 2021-07-11T03:53:28.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 --2- 172.21.2.104:0/2579572309 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7464144a90 0x7f7464064f80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 --2- 172.21.2.104:0/2579572309 >> v2:172.21.2.105:3300/0 conn(0x7f74640b7810 0x7f74641445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 -- 172.21.2.104:0/2579572309 >> 172.21.2.104:0/2579572309 conn(0x7f746405f6b0 msgr2=0x7f746405fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 -- 172.21.2.104:0/2579572309 shutdown_connections 2021-07-11T03:53:28.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 -- 172.21.2.104:0/2579572309 wait complete. 2021-07-11T03:53:28.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 Processor -- start 2021-07-11T03:53:28.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 -- start start 2021-07-11T03:53:28.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74640b1170 0x7f74640adda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74640b7810 0x7f74640abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7464144a90 0x7f7464156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7464156570 con 0x7f7464144a90 2021-07-11T03:53:28.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74640b0b20 con 0x7f74640b1170 2021-07-11T03:53:28.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74641567a0 con 0x7f74640b7810 2021-07-11T03:53:28.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f7468fc4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7464144a90 0x7f7464156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f7468fc4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7464144a90 0x7f7464156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42082/0 (socket says 172.21.2.104:42082) 2021-07-11T03:53:28.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f7468fc4700 1 -- 172.21.2.104:0/811949727 learned_addr learned my addr 172.21.2.104:0/811949727 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:28.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f7468fc4700 1 -- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74640b7810 msgr2=0x7f74640abd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f7468fc4700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74640b7810 0x7f74640abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f7468fc4700 1 -- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74640b1170 msgr2=0x7f74640adda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f7468fc4700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74640b1170 0x7f74640adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f7468fc4700 1 -- 172.21.2.104:0/811949727 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f745c009470 con 0x7f7464144a90 2021-07-11T03:53:28.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f7468fc4700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7464144a90 0x7f7464156c40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f745c0032f0 tx=0x7f745c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f74617fa700 1 -- 172.21.2.104:0/811949727 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745c021440 con 0x7f7464144a90 2021-07-11T03:53:28.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f74617fa700 1 -- 172.21.2.104:0/811949727 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c0215a0 con 0x7f7464144a90 2021-07-11T03:53:28.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f74617fa700 1 -- 172.21.2.104:0/811949727 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f745c021900 con 0x7f7464144a90 2021-07-11T03:53:28.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 -- 172.21.2.104:0/811949727 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74640b0dc0 con 0x7f7464144a90 2021-07-11T03:53:28.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 -- 172.21.2.104:0/811949727 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74640ac330 con 0x7f7464144a90 2021-07-11T03:53:28.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f746aa27700 1 -- 172.21.2.104:0/811949727 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74640ae5d0 con 0x7f7464144a90 2021-07-11T03:53:28.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f74617fa700 1 -- 172.21.2.104:0/811949727 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f745c027020 con 0x7f7464144a90 2021-07-11T03:53:28.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f74617fa700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7448053210 0x7f74480556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f74617fa700 1 -- 172.21.2.104:0/811949727 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f745c06ca30 con 0x7f7464144a90 2021-07-11T03:53:28.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.636+0000 7f7463fff700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7448053210 0x7f74480556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.640+0000 7f7463fff700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7448053210 0x7f74480556d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7454002f70 tx=0x7f7454006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.640+0000 7f74617fa700 1 -- 172.21.2.104:0/811949727 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f745c043c60 con 0x7f7464144a90 2021-07-11T03:53:28.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.720+0000 7f746aa27700 1 -- 172.21.2.104:0/811949727 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f74641564d0 con 0x7f7464144a90 2021-07-11T03:53:28.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.720+0000 7f74617fa700 1 -- 172.21.2.104:0/811949727 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f745c015090 con 0x7f7464144a90 2021-07-11T03:53:28.725 INFO:teuthology.orchestra.run.gibba004.stdout:60129542150 2021-07-11T03:53:28.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 -- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7448053210 msgr2=0x7f74480556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7448053210 0x7f74480556d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7454002f70 tx=0x7f7454006a80).stop 2021-07-11T03:53:28.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 -- 172.21.2.104:0/811949727 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7464144a90 msgr2=0x7f7464156c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7464144a90 0x7f7464156c40 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f745c0032f0 tx=0x7f745c016000).stop 2021-07-11T03:53:28.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 -- 172.21.2.104:0/811949727 shutdown_connections 2021-07-11T03:53:28.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7448053210 0x7f74480556d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7464144a90 0x7f7464156c40 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74640b7810 0x7f74640abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 --2- 172.21.2.104:0/811949727 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74640b1170 0x7f74640adda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 -- 172.21.2.104:0/811949727 >> 172.21.2.104:0/811949727 conn(0x7f746405f6b0 msgr2=0x7f74640607c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 -- 172.21.2.104:0/811949727 shutdown_connections 2021-07-11T03:53:28.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.724+0000 7f746aa27700 1 -- 172.21.2.104:0/811949727 wait complete. 2021-07-11T03:53:28.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2021-07-11T03:53:28.733 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2021-07-11T03:53:28.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f8734aeb700 1 Processor -- start 2021-07-11T03:53:28.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f8734aeb700 1 -- start start 2021-07-11T03:53:28.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f8734aeb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87300b1970 0x7f873005fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f8734aeb700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87300a7a90 con 0x7f8730067180 2021-07-11T03:53:28.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f8734aeb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8730147620 con 0x7f87300a76b0 2021-07-11T03:53:28.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f8734aeb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87301478c0 con 0x7f87300b1970 2021-07-11T03:53:28.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872dd9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8730067180 0x7f873005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38858/0 (socket says 172.21.2.104:38858) 2021-07-11T03:53:28.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872dd9b700 1 -- 172.21.2.104:0/904599095 learned_addr learned my addr 172.21.2.104:0/904599095 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:28.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020223887 0 0) 0x7f87300a7a90 con 0x7f8730067180 2021-07-11T03:53:28.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8710003260 con 0x7f8730067180 2021-07-11T03:53:28.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872ed9d700 1 --2- 172.21.2.104:0/904599095 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87300b1970 0x7f873005fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3884357223 0 0) 0x7f8710003260 con 0x7f8730067180 2021-07-11T03:53:28.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87300b1970 msgr2=0x7f873005fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 --2- 172.21.2.104:0/904599095 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87300b1970 0x7f873005fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 >> v1:172.21.2.105:6789/0 conn(0x7f87300a76b0 legacy=0x7f8730066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8730147b60 con 0x7f8730067180 2021-07-11T03:53:28.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f8718001660 con 0x7f8730067180 2021-07-11T03:53:28.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 >> v1:172.21.2.104:6789/0 conn(0x7f8730067180 legacy=0x7f873005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 --2- 172.21.2.104:0/904599095 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8710003800 0x7f8710003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 --2- 172.21.2.104:0/904599095 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87300b1970 0x7f8710004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 --2- 172.21.2.104:0/904599095 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8710004bf0 0x7f871000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87301478c0 con 0x7f8710003800 2021-07-11T03:53:28.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87100052a0 con 0x7f8710004bf0 2021-07-11T03:53:28.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871000bc40 con 0x7f87300b1970 2021-07-11T03:53:28.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872e59c700 1 --2- 172.21.2.104:0/904599095 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8710003800 0x7f8710003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872e59c700 1 -- 172.21.2.104:0/904599095 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87300b1970 msgr2=0x7f8710004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872e59c700 1 --2- 172.21.2.104:0/904599095 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87300b1970 0x7f8710004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872e59c700 1 -- 172.21.2.104:0/904599095 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8710004bf0 msgr2=0x7f871000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872e59c700 1 --2- 172.21.2.104:0/904599095 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8710004bf0 0x7f871000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.780+0000 7f872e59c700 1 -- 172.21.2.104:0/904599095 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8710003260 con 0x7f8710003800 2021-07-11T03:53:28.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872e59c700 1 --2- 172.21.2.104:0/904599095 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8710003800 0x7f8710003c00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f8720001fa0 tx=0x7f8720007a00).ready entity=mon.0 client_cookie=57e06c293e39e13c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8720007070 con 0x7f8710003800 2021-07-11T03:53:28.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87200071d0 con 0x7f8710003800 2021-07-11T03:53:28.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872d59a700 1 -- 172.21.2.104:0/904599095 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8720006510 con 0x7f8710003800 2021-07-11T03:53:28.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- 172.21.2.104:0/904599095 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8710003800 msgr2=0x7f8710003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 --2- 172.21.2.104:0/904599095 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8710003800 0x7f8710003c00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f8720001fa0 tx=0x7f8720007a00).stop 2021-07-11T03:53:28.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872ed9d700 1 -- 172.21.2.104:0/904599095 reap_dead start 2021-07-11T03:53:28.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- 172.21.2.104:0/904599095 shutdown_connections 2021-07-11T03:53:28.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- 172.21.2.104:0/904599095 >> 172.21.2.104:0/904599095 conn(0x7f87300b53e0 msgr2=0x7f87300b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- 172.21.2.104:0/904599095 shutdown_connections 2021-07-11T03:53:28.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- 172.21.2.104:0/904599095 wait complete. 2021-07-11T03:53:28.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 Processor -- start 2021-07-11T03:53:28.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- start start 2021-07-11T03:53:28.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8730145a80 0x7f87300aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87300ae390 0x7f8730144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8730144af0 0x7f8730159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8730145e60 con 0x7f8730144af0 2021-07-11T03:53:28.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87300af040 con 0x7f8730145a80 2021-07-11T03:53:28.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87301473b0 con 0x7f87300ae390 2021-07-11T03:53:28.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8730144af0 0x7f8730159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8730144af0 0x7f8730159ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42104/0 (socket says 172.21.2.104:42104) 2021-07-11T03:53:28.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872ed9d700 1 -- 172.21.2.104:0/790262190 learned_addr learned my addr 172.21.2.104:0/790262190 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:28.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872ed9d700 1 -- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87300ae390 msgr2=0x7f8730144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872ed9d700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87300ae390 0x7f8730144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872ed9d700 1 -- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8730145a80 msgr2=0x7f87300aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872ed9d700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8730145a80 0x7f87300aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872ed9d700 1 -- 172.21.2.104:0/790262190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8730146710 con 0x7f8730144af0 2021-07-11T03:53:28.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f872ed9d700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8730144af0 0x7f8730159ac0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f8718001840 tx=0x7f8728008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f87277fe700 1 -- 172.21.2.104:0/790262190 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8728010580 con 0x7f8730144af0 2021-07-11T03:53:28.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f87277fe700 1 -- 172.21.2.104:0/790262190 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87280106e0 con 0x7f8730144af0 2021-07-11T03:53:28.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f87277fe700 1 -- 172.21.2.104:0/790262190 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8728010a40 con 0x7f8730144af0 2021-07-11T03:53:28.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- 172.21.2.104:0/790262190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87300aa7f0 con 0x7f8730144af0 2021-07-11T03:53:28.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- 172.21.2.104:0/790262190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87300aad30 con 0x7f8730144af0 2021-07-11T03:53:28.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f8734aeb700 1 -- 172.21.2.104:0/790262190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87300ae090 con 0x7f8730144af0 2021-07-11T03:53:28.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f87277fe700 1 -- 172.21.2.104:0/790262190 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8728010ba0 con 0x7f8730144af0 2021-07-11T03:53:28.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f87277fe700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8714053270 0x7f8714055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.784+0000 7f87277fe700 1 -- 172.21.2.104:0/790262190 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f872806b5a0 con 0x7f8730144af0 2021-07-11T03:53:28.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.788+0000 7f872e59c700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8714053270 0x7f8714055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.788+0000 7f872e59c700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8714053270 0x7f8714055730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f8720001fa0 tx=0x7f8720015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.788+0000 7f87277fe700 1 -- 172.21.2.104:0/790262190 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8728042880 con 0x7f8730144af0 2021-07-11T03:53:28.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.868+0000 7f8734aeb700 1 -- 172.21.2.104:0/790262190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8730057350 con 0x7f8730144af0 2021-07-11T03:53:28.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f87277fe700 1 -- 172.21.2.104:0/790262190 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8728045ea0 con 0x7f8730144af0 2021-07-11T03:53:28.875 INFO:teuthology.orchestra.run.gibba004.stdout:60129542149 2021-07-11T03:53:28.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 -- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8714053270 msgr2=0x7f8714055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8714053270 0x7f8714055730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f8720001fa0 tx=0x7f8720015000).stop 2021-07-11T03:53:28.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 -- 172.21.2.104:0/790262190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8730144af0 msgr2=0x7f8730159ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8730144af0 0x7f8730159ac0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f8718001840 tx=0x7f8728008040).stop 2021-07-11T03:53:28.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 -- 172.21.2.104:0/790262190 shutdown_connections 2021-07-11T03:53:28.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8714053270 0x7f8714055730 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8730144af0 0x7f8730159ac0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87300ae390 0x7f8730144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 --2- 172.21.2.104:0/790262190 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8730145a80 0x7f87300aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 -- 172.21.2.104:0/790262190 >> 172.21.2.104:0/790262190 conn(0x7f87300b53e0 msgr2=0x7f8730149da0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 -- 172.21.2.104:0/790262190 shutdown_connections 2021-07-11T03:53:28.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.872+0000 7f8734aeb700 1 -- 172.21.2.104:0/790262190 wait complete. 2021-07-11T03:53:28.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.5 2021-07-11T03:53:28.882 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2021-07-11T03:53:28.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 Processor -- start 2021-07-11T03:53:28.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 -- start start 2021-07-11T03:53:28.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c141b40 0x7fab9c14cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c0b31d0 con 0x7fab9c0b7810 2021-07-11T03:53:28.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c0b7bf0 con 0x7fab9c0b1170 2021-07-11T03:53:28.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0675c0 con 0x7fab9c141b40 2021-07-11T03:53:28.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2289700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fab9c0b7810 0x7fab9c1445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38876/0 (socket says 172.21.2.104:38876) 2021-07-11T03:53:28.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2289700 1 -- 172.21.2.104:0/3379286449 learned_addr learned my addr 172.21.2.104:0/3379286449 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:28.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160811023 0 0) 0x7fab9c0b31d0 con 0x7fab9c0b7810 2021-07-11T03:53:28.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab84003260 con 0x7fab9c0b7810 2021-07-11T03:53:28.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2a8a700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c141b40 0x7fab9c14cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3241054620 0 0) 0x7fab84003260 con 0x7fab9c0b7810 2021-07-11T03:53:28.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c141b40 msgr2=0x7fab9c14cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c141b40 0x7fab9c14cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 >> v1:172.21.2.105:6789/0 conn(0x7fab9c0b1170 legacy=0x7fab9c141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c0aae70 con 0x7fab9c0b7810 2021-07-11T03:53:28.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fab8c001660 con 0x7fab9c0b7810 2021-07-11T03:53:28.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 >> v1:172.21.2.104:6789/0 conn(0x7fab9c0b7810 legacy=0x7fab9c1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c141b40 0x7fab84003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab84003ef0 0x7fab84004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab84004bf0 0x7fab8400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0675c0 con 0x7fab84004bf0 2021-07-11T03:53:28.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab840052f0 con 0x7fab84003ef0 2021-07-11T03:53:28.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab8400bc40 con 0x7fab9c141b40 2021-07-11T03:53:28.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2a8a700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab84004bf0 0x7fab8400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1a88700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab84003ef0 0x7fab84004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2289700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c141b40 0x7fab84003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2a8a700 1 -- 172.21.2.104:0/3379286449 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c141b40 msgr2=0x7fab84003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2a8a700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c141b40 0x7fab84003800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2a8a700 1 -- 172.21.2.104:0/3379286449 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab84003ef0 msgr2=0x7fab84004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:28.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2a8a700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab84003ef0 0x7fab84004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2a8a700 1 -- 172.21.2.104:0/3379286449 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab84003260 con 0x7fab84004bf0 2021-07-11T03:53:28.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2a8a700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab84004bf0 0x7fab8400b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fab94007000 tx=0x7fab94008040).ready entity=mon.0 client_cookie=3406a1d86ebd4abf server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab9400c5e0 con 0x7fab84004bf0 2021-07-11T03:53:28.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9400c740 con 0x7fab84004bf0 2021-07-11T03:53:28.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba1287700 1 -- 172.21.2.104:0/3379286449 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab9400caa0 con 0x7fab84004bf0 2021-07-11T03:53:28.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 -- 172.21.2.104:0/3379286449 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab84004bf0 msgr2=0x7fab8400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 --2- 172.21.2.104:0/3379286449 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab84004bf0 0x7fab8400b750 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fab94007000 tx=0x7fab94008040).stop 2021-07-11T03:53:28.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba2a8a700 1 -- 172.21.2.104:0/3379286449 reap_dead start 2021-07-11T03:53:28.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 -- 172.21.2.104:0/3379286449 shutdown_connections 2021-07-11T03:53:28.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 -- 172.21.2.104:0/3379286449 >> 172.21.2.104:0/3379286449 conn(0x7fab9c05f6b0 msgr2=0x7fab9c05fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:28.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 -- 172.21.2.104:0/3379286449 shutdown_connections 2021-07-11T03:53:28.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 -- 172.21.2.104:0/3379286449 wait complete. 2021-07-11T03:53:28.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 Processor -- start 2021-07-11T03:53:28.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.932+0000 7faba44ed700 1 -- start start 2021-07-11T03:53:28.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba44ed700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab9c0ad850 0x7fab9c14fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba44ed700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab9c0adc30 0x7fab9c150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba44ed700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c0a9cc0 0x7fab9c0aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba44ed700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0ae3f0 con 0x7fab9c0adc30 2021-07-11T03:53:28.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba44ed700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0ad330 con 0x7fab9c0ad850 2021-07-11T03:53:28.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba44ed700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0ad090 con 0x7fab9c0a9cc0 2021-07-11T03:53:28.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba1a88700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab9c0adc30 0x7fab9c150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba1a88700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab9c0adc30 0x7fab9c150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42124/0 (socket says 172.21.2.104:42124) 2021-07-11T03:53:28.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba1a88700 1 -- 172.21.2.104:0/2032617256 learned_addr learned my addr 172.21.2.104:0/2032617256 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:28.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba2289700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab9c0ad850 0x7fab9c14fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba1a88700 1 -- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c0a9cc0 msgr2=0x7fab9c0aa170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba1a88700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c0a9cc0 0x7fab9c0aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:28.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba1a88700 1 -- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab9c0ad850 msgr2=0x7fab9c14fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:28.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba1a88700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab9c0ad850 0x7fab9c14fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:28.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba1a88700 1 -- 172.21.2.104:0/2032617256 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c0abc30 con 0x7fab9c0adc30 2021-07-11T03:53:28.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba1a88700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab9c0adc30 0x7fab9c150370 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fab9800b000 tx=0x7fab98009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7fab8b7fe700 1 -- 172.21.2.104:0/2032617256 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab98015630 con 0x7fab9c0adc30 2021-07-11T03:53:28.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7fab8b7fe700 1 -- 172.21.2.104:0/2032617256 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab98000e90 con 0x7fab9c0adc30 2021-07-11T03:53:28.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba44ed700 1 -- 172.21.2.104:0/2032617256 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c150860 con 0x7fab9c0adc30 2021-07-11T03:53:28.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7fab8b7fe700 1 -- 172.21.2.104:0/2032617256 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab98015980 con 0x7fab9c0adc30 2021-07-11T03:53:28.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba44ed700 1 -- 172.21.2.104:0/2032617256 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c0b01d0 con 0x7fab9c0adc30 2021-07-11T03:53:28.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba44ed700 1 -- 172.21.2.104:0/2032617256 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c0abdc0 con 0x7fab9c0adc30 2021-07-11T03:53:28.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7fab8b7fe700 1 -- 172.21.2.104:0/2032617256 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fab98015ae0 con 0x7fab9c0adc30 2021-07-11T03:53:28.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7fab8b7fe700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab7c053270 0x7fab7c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:28.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7fab8b7fe700 1 -- 172.21.2.104:0/2032617256 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fab9806b260 con 0x7fab9c0adc30 2021-07-11T03:53:28.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba2289700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab7c053270 0x7fab7c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:28.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.936+0000 7faba2289700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab7c053270 0x7fab7c055730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fab8c009000 tx=0x7fab8c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:28.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:28.940+0000 7fab8b7fe700 1 -- 172.21.2.104:0/2032617256 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fab980425a0 con 0x7fab9c0adc30 2021-07-11T03:53:29.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.020+0000 7faba44ed700 1 -- 172.21.2.104:0/2032617256 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fab9c057350 con 0x7fab9c0adc30 2021-07-11T03:53:29.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7fab8b7fe700 1 -- 172.21.2.104:0/2032617256 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fab98045bc0 con 0x7fab9c0adc30 2021-07-11T03:53:29.026 INFO:teuthology.orchestra.run.gibba004.stdout:60129542149 2021-07-11T03:53:29.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 -- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab7c053270 msgr2=0x7fab7c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab7c053270 0x7fab7c055730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7fab8c009000 tx=0x7fab8c009040).stop 2021-07-11T03:53:29.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 -- 172.21.2.104:0/2032617256 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab9c0adc30 msgr2=0x7fab9c150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab9c0adc30 0x7fab9c150370 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fab9800b000 tx=0x7fab98009040).stop 2021-07-11T03:53:29.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 -- 172.21.2.104:0/2032617256 shutdown_connections 2021-07-11T03:53:29.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab9c0a9cc0 0x7fab9c0aa170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab7c053270 0x7fab7c055730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab9c0adc30 0x7fab9c150370 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 --2- 172.21.2.104:0/2032617256 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab9c0ad850 0x7fab9c14fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 -- 172.21.2.104:0/2032617256 >> 172.21.2.104:0/2032617256 conn(0x7fab9c05f6b0 msgr2=0x7fab9c05bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 -- 172.21.2.104:0/2032617256 shutdown_connections 2021-07-11T03:53:29.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.024+0000 7faba44ed700 1 -- 172.21.2.104:0/2032617256 wait complete. 2021-07-11T03:53:29.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2021-07-11T03:53:29.034 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2021-07-11T03:53:29.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.080+0000 7f1d2f793700 1 Processor -- start 2021-07-11T03:53:29.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.080+0000 7f1d2f793700 1 -- start start 2021-07-11T03:53:29.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.080+0000 7f1d2f793700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d280b96e0 0x7f1d280aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.080+0000 7f1d2f793700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1d2813fff0 0x7f1d28140480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.080+0000 7f1d2f793700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d28067500 con 0x7f1d28067120 2021-07-11T03:53:29.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.080+0000 7f1d2f793700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d28143080 con 0x7f1d2813fff0 2021-07-11T03:53:29.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d280af4d0 con 0x7f1d280b96e0 2021-07-11T03:53:29.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2d52f700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1d28067120 0x7f1d281439b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38896/0 (socket says 172.21.2.104:38896) 2021-07-11T03:53:29.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2d52f700 1 -- 172.21.2.104:0/529934617 learned_addr learned my addr 172.21.2.104:0/529934617 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:29.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2dd30700 1 --2- 172.21.2.104:0/529934617 >> v2:172.21.2.104:3300/0 conn(0x7f1d2813fff0 0x7f1d28140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d1bfff700 1 -- 172.21.2.104:0/529934617 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729565711 0 0) 0x7f1d28067500 con 0x7f1d28067120 2021-07-11T03:53:29.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d1bfff700 1 -- 172.21.2.104:0/529934617 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d10003260 con 0x7f1d28067120 2021-07-11T03:53:29.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2dd30700 1 -- 172.21.2.104:0/529934617 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d280b96e0 msgr2=0x7f1d280aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:29.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2dd30700 1 --2- 172.21.2.104:0/529934617 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d280b96e0 0x7f1d280aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2dd30700 1 -- 172.21.2.104:0/529934617 >> v1:172.21.2.104:6789/0 conn(0x7f1d28067120 legacy=0x7f1d281439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2dd30700 1 -- 172.21.2.104:0/529934617 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d28143270 con 0x7f1d2813fff0 2021-07-11T03:53:29.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2dd30700 1 --2- 172.21.2.104:0/529934617 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d2813fff0 0x7f1d28140480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f1d200032f0 tx=0x7f1d200091d0).ready entity=mon.0 client_cookie=fbc50fbe9e0aea3 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d1bfff700 1 -- 172.21.2.104:0/529934617 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d2000b6c0 con 0x7f1d2813fff0 2021-07-11T03:53:29.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d1bfff700 1 -- 172.21.2.104:0/529934617 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2000b820 con 0x7f1d2813fff0 2021-07-11T03:53:29.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d1bfff700 1 -- 172.21.2.104:0/529934617 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d2000b990 con 0x7f1d2813fff0 2021-07-11T03:53:29.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- 172.21.2.104:0/529934617 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d2813fff0 msgr2=0x7f1d28140480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 --2- 172.21.2.104:0/529934617 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d2813fff0 0x7f1d28140480 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f1d200032f0 tx=0x7f1d200091d0).stop 2021-07-11T03:53:29.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- 172.21.2.104:0/529934617 shutdown_connections 2021-07-11T03:53:29.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 --2- 172.21.2.104:0/529934617 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d2813fff0 0x7f1d28140480 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 --2- 172.21.2.104:0/529934617 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d280b96e0 0x7f1d280aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- 172.21.2.104:0/529934617 >> 172.21.2.104:0/529934617 conn(0x7f1d2805f9f0 msgr2=0x7f1d280616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- 172.21.2.104:0/529934617 shutdown_connections 2021-07-11T03:53:29.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- 172.21.2.104:0/529934617 wait complete. 2021-07-11T03:53:29.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 Processor -- start 2021-07-11T03:53:29.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- start start 2021-07-11T03:53:29.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d28067120 0x7f1d28156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d280b96e0 0x7f1d28159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d2813fff0 0x7f1d2805bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2813fb40 con 0x7f1d28067120 2021-07-11T03:53:29.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d281572c0 con 0x7f1d280b96e0 2021-07-11T03:53:29.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d28064a80 con 0x7f1d2813fff0 2021-07-11T03:53:29.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2d52f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d28067120 0x7f1d28156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2d52f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d28067120 0x7f1d28156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42134/0 (socket says 172.21.2.104:42134) 2021-07-11T03:53:29.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2d52f700 1 -- 172.21.2.104:0/1809537357 learned_addr learned my addr 172.21.2.104:0/1809537357 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:29.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2d52f700 1 -- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d2813fff0 msgr2=0x7f1d2805bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2d52f700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d2813fff0 0x7f1d2805bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2d52f700 1 -- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d280b96e0 msgr2=0x7f1d28159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2cd2e700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d280b96e0 0x7f1d28159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2d52f700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d280b96e0 0x7f1d28159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2d52f700 1 -- 172.21.2.104:0/1809537357 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d20009470 con 0x7f1d28067120 2021-07-11T03:53:29.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2cd2e700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d280b96e0 0x7f1d28159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:29.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2d52f700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d28067120 0x7f1d28156dd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f1d1c001110 tx=0x7f1d1c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d1a7fc700 1 -- 172.21.2.104:0/1809537357 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c0072e0 con 0x7f1d28067120 2021-07-11T03:53:29.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d1a7fc700 1 -- 172.21.2.104:0/1809537357 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c007440 con 0x7f1d28067120 2021-07-11T03:53:29.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- 172.21.2.104:0/1809537357 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d28064d20 con 0x7f1d28067120 2021-07-11T03:53:29.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d1a7fc700 1 -- 172.21.2.104:0/1809537357 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1c005800 con 0x7f1d28067120 2021-07-11T03:53:29.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.084+0000 7f1d2f793700 1 -- 172.21.2.104:0/1809537357 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d28156690 con 0x7f1d28067120 2021-07-11T03:53:29.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.088+0000 7f1d2f793700 1 -- 172.21.2.104:0/1809537357 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d28057350 con 0x7f1d28067120 2021-07-11T03:53:29.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.088+0000 7f1d1a7fc700 1 -- 172.21.2.104:0/1809537357 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1d1c010070 con 0x7f1d28067120 2021-07-11T03:53:29.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.088+0000 7f1d1a7fc700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d08053270 0x7f1d08055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.088+0000 7f1d1a7fc700 1 -- 172.21.2.104:0/1809537357 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1d1c06abe0 con 0x7f1d28067120 2021-07-11T03:53:29.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.088+0000 7f1d2cd2e700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d08053270 0x7f1d08055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.088+0000 7f1d2cd2e700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d08053270 0x7f1d08055730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1d24007000 tx=0x7f1d24008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.088+0000 7f1d1a7fc700 1 -- 172.21.2.104:0/1809537357 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1d1c041f20 con 0x7f1d28067120 2021-07-11T03:53:29.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.168+0000 7f1d2f793700 1 -- 172.21.2.104:0/1809537357 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f1d28156ad0 con 0x7f1d28067120 2021-07-11T03:53:29.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d1a7fc700 1 -- 172.21.2.104:0/1809537357 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1d1c045540 con 0x7f1d28067120 2021-07-11T03:53:29.174 INFO:teuthology.orchestra.run.gibba004.stdout:60129542149 2021-07-11T03:53:29.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 -- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d08053270 msgr2=0x7f1d08055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d08053270 0x7f1d08055730 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f1d24007000 tx=0x7f1d24008040).stop 2021-07-11T03:53:29.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 -- 172.21.2.104:0/1809537357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d28067120 msgr2=0x7f1d28156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d28067120 0x7f1d28156dd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f1d1c001110 tx=0x7f1d1c007ad0).stop 2021-07-11T03:53:29.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 -- 172.21.2.104:0/1809537357 shutdown_connections 2021-07-11T03:53:29.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d2813fff0 0x7f1d2805bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d280b96e0 0x7f1d28159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d08053270 0x7f1d08055730 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 --2- 172.21.2.104:0/1809537357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d28067120 0x7f1d28156dd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 -- 172.21.2.104:0/1809537357 >> 172.21.2.104:0/1809537357 conn(0x7f1d2805f9f0 msgr2=0x7f1d281427b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 -- 172.21.2.104:0/1809537357 shutdown_connections 2021-07-11T03:53:29.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.172+0000 7f1d2f793700 1 -- 172.21.2.104:0/1809537357 wait complete. 2021-07-11T03:53:29.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2021-07-11T03:53:29.182 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2021-07-11T03:53:29.182 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-07-11T03:53:29.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 Processor -- start 2021-07-11T03:53:29.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- start start 2021-07-11T03:53:29.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4e40b9a20 0x7fe4e4147570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe4e40b0760 0x7fe4e40ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e40b9e00 con 0x7fe4e40b47b0 2021-07-11T03:53:29.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4e40b54e0 con 0x7fe4e40b0760 2021-07-11T03:53:29.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40b4cc0 con 0x7fe4e40b9a20 2021-07-11T03:53:29.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e1d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe4e40b0760 0x7fe4e40ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e2d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe4e40b47b0 0x7fe4e4144690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38912/0 (socket says 172.21.2.104:38912) 2021-07-11T03:53:29.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e2d9d700 1 -- 172.21.2.104:0/2355052485 learned_addr learned my addr 172.21.2.104:0/2355052485 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:29.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e259c700 1 --2- 172.21.2.104:0/2355052485 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4e40b9a20 0x7fe4e4147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e1d9b700 1 -- 172.21.2.104:0/2355052485 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4e40b9a20 msgr2=0x7fe4e4147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:29.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e1d9b700 1 --2- 172.21.2.104:0/2355052485 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4e40b9a20 0x7fe4e4147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e1d9b700 1 -- 172.21.2.104:0/2355052485 >> v1:172.21.2.104:6789/0 conn(0x7fe4e40b47b0 legacy=0x7fe4e4144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e1d9b700 1 -- 172.21.2.104:0/2355052485 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e40ad8f0 con 0x7fe4e40b0760 2021-07-11T03:53:29.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e1d9b700 1 --2- 172.21.2.104:0/2355052485 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4e40b0760 0x7fe4e40ae030 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fe4e40af9b0 tx=0x7fe4d4006df0).ready entity=mon.0 client_cookie=590a6b2191d67e1d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e159a700 1 -- 172.21.2.104:0/2355052485 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4d400d570 con 0x7fe4e40b0760 2021-07-11T03:53:29.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e159a700 1 -- 172.21.2.104:0/2355052485 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d4000e90 con 0x7fe4e40b0760 2021-07-11T03:53:29.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e159a700 1 -- 172.21.2.104:0/2355052485 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4d400d8c0 con 0x7fe4e40b0760 2021-07-11T03:53:29.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- 172.21.2.104:0/2355052485 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4e40b0760 msgr2=0x7fe4e40ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 --2- 172.21.2.104:0/2355052485 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4e40b0760 0x7fe4e40ae030 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fe4e40af9b0 tx=0x7fe4d4006df0).stop 2021-07-11T03:53:29.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- 172.21.2.104:0/2355052485 shutdown_connections 2021-07-11T03:53:29.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 --2- 172.21.2.104:0/2355052485 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4e40b0760 0x7fe4e40ae030 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 --2- 172.21.2.104:0/2355052485 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4e40b9a20 0x7fe4e4147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- 172.21.2.104:0/2355052485 >> 172.21.2.104:0/2355052485 conn(0x7fe4e405f9f0 msgr2=0x7fe4e40616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- 172.21.2.104:0/2355052485 shutdown_connections 2021-07-11T03:53:29.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- 172.21.2.104:0/2355052485 wait complete. 2021-07-11T03:53:29.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 Processor -- start 2021-07-11T03:53:29.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- start start 2021-07-11T03:53:29.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4e40b0760 0x7fe4e40b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe4e40b47b0 0x7fe4e415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4e40b9a20 0x7fe4e4146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40b3680 con 0x7fe4e40b0760 2021-07-11T03:53:29.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40b0d60 con 0x7fe4e40b47b0 2021-07-11T03:53:29.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e8b81700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e40b1880 con 0x7fe4e40b9a20 2021-07-11T03:53:29.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4e40b0760 0x7fe4e40b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4e40b0760 0x7fe4e40b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42148/0 (socket says 172.21.2.104:42148) 2021-07-11T03:53:29.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e259c700 1 -- 172.21.2.104:0/913542561 learned_addr learned my addr 172.21.2.104:0/913542561 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:29.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e1d9b700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe4e40b47b0 0x7fe4e415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e2d9d700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4e40b9a20 0x7fe4e4146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e259c700 1 -- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4e40b9a20 msgr2=0x7fe4e4146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e259c700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4e40b9a20 0x7fe4e4146330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e259c700 1 -- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe4e40b47b0 msgr2=0x7fe4e415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e259c700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe4e40b47b0 0x7fe4e415a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e259c700 1 -- 172.21.2.104:0/913542561 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d400b180 con 0x7fe4e40b0760 2021-07-11T03:53:29.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e2d9d700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4e40b9a20 0x7fe4e4146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:29.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4e259c700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4e40b0760 0x7fe4e40b2f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe4cc00c000 tx=0x7fe4cc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.232+0000 7fe4db7fe700 1 -- 172.21.2.104:0/913542561 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4cc010780 con 0x7fe4e40b0760 2021-07-11T03:53:29.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.236+0000 7fe4db7fe700 1 -- 172.21.2.104:0/913542561 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc0108e0 con 0x7fe4e40b0760 2021-07-11T03:53:29.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.236+0000 7fe4e8b81700 1 -- 172.21.2.104:0/913542561 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e40b1ab0 con 0x7fe4e40b0760 2021-07-11T03:53:29.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.236+0000 7fe4db7fe700 1 -- 172.21.2.104:0/913542561 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe4cc010c40 con 0x7fe4e40b0760 2021-07-11T03:53:29.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.236+0000 7fe4e8b81700 1 -- 172.21.2.104:0/913542561 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e40b1150 con 0x7fe4e40b0760 2021-07-11T03:53:29.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.236+0000 7fe4e8b81700 1 -- 172.21.2.104:0/913542561 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e4057350 con 0x7fe4e40b0760 2021-07-11T03:53:29.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.236+0000 7fe4db7fe700 1 -- 172.21.2.104:0/913542561 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe4cc01c460 con 0x7fe4e40b0760 2021-07-11T03:53:29.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.236+0000 7fe4db7fe700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4c80531f0 0x7fe4c80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.236+0000 7fe4db7fe700 1 -- 172.21.2.104:0/913542561 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe4cc06b960 con 0x7fe4e40b0760 2021-07-11T03:53:29.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.236+0000 7fe4e1d9b700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4c80531f0 0x7fe4c80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.236+0000 7fe4e1d9b700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4c80531f0 0x7fe4c80556b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fe4d4017000 tx=0x7fe4d4014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.236+0000 7fe4db7fe700 1 -- 172.21.2.104:0/913542561 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe4cc042c40 con 0x7fe4e40b0760 2021-07-11T03:53:29.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.320+0000 7fe4e8b81700 1 -- 172.21.2.104:0/913542561 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4e40b0eb0 con 0x7fe4c80531f0 2021-07-11T03:53:29.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4db7fe700 1 -- 172.21.2.104:0/913542561 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82337 (secure 0 0 0) 0x7fe4d400d600 con 0x7fe4c80531f0 2021-07-11T03:53:29.332 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:29.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 -- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4c80531f0 msgr2=0x7fe4c80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4c80531f0 0x7fe4c80556b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fe4d4017000 tx=0x7fe4d4014000).stop 2021-07-11T03:53:29.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 -- 172.21.2.104:0/913542561 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4e40b0760 msgr2=0x7fe4e40b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4e40b0760 0x7fe4e40b2f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fe4cc00c000 tx=0x7fe4cc006df0).stop 2021-07-11T03:53:29.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 -- 172.21.2.104:0/913542561 shutdown_connections 2021-07-11T03:53:29.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4e40b9a20 0x7fe4e4146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe4e40b47b0 0x7fe4e415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4c80531f0 0x7fe4c80556b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 --2- 172.21.2.104:0/913542561 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4e40b0760 0x7fe4e40b2f90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 -- 172.21.2.104:0/913542561 >> 172.21.2.104:0/913542561 conn(0x7fe4e405f9f0 msgr2=0x7fe4e4149aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 -- 172.21.2.104:0/913542561 shutdown_connections 2021-07-11T03:53:29.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.328+0000 7fe4e8b81700 1 -- 172.21.2.104:0/913542561 wait complete. 2021-07-11T03:53:29.336 INFO:teuthology.orchestra.run.gibba004.stderr:dumped all 2021-07-11T03:53:29.339 INFO:teuthology.orchestra.run.gibba004.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2021-07-11T03:53:28.366276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390552,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584168,"statfs":{"total":773094113280,"available":764502188032,"internally_reserved":8589934592,"allocated":1925120,"data_stored":701229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.008262"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.377538+0000","last_change":"2021-07-11T03:53:23.376788+0000","last_active":"2021-07-11T03:53:23.377538+0000","last_peered":"2021-07-11T03:53:23.377538+0000","last_clean":"2021-07-11T03:53:23.377538+0000","last_became_active":"2021-07-11T03:53:23.376580+0000","last_became_peered":"2021-07-11T03:53:23.376580+0000","last_unstale":"2021-07-11T03:53:23.377538+0000","last_undegraded":"2021-07-11T03:53:23.377538+0000","last_fullsized":"2021-07-11T03:53:23.377538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.370366+0000","last_change":"2021-07-11T03:53:15.964238+0000","last_active":"2021-07-11T03:53:23.370366+0000","last_peered":"2021-07-11T03:53:23.370366+0000","last_clean":"2021-07-11T03:53:23.370366+0000","last_became_active":"2021-07-11T03:53:15.962801+0000","last_became_peered":"2021-07-11T03:53:15.962801+0000","last_unstale":"2021-07-11T03:53:23.370366+0000","last_undegraded":"2021-07-11T03:53:23.370366+0000","last_fullsized":"2021-07-11T03:53:23.370366+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:11.361771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:11.361771+0000","last_clean_scrub_stamp":"2021-07-11T03:53:11.361771+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.510092+0000","last_change":"2021-07-11T03:53:20.153399+0000","last_active":"2021-07-11T03:53:24.510092+0000","last_peered":"2021-07-11T03:53:24.510092+0000","last_clean":"2021-07-11T03:53:24.510092+0000","last_became_active":"2021-07-11T03:53:20.153135+0000","last_became_peered":"2021-07-11T03:53:20.153135+0000","last_unstale":"2021-07-11T03:53:24.510092+0000","last_undegraded":"2021-07-11T03:53:24.510092+0000","last_fullsized":"2021-07-11T03:53:24.510092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"21'1","reported_seq":"11","reported_epoch":"21","state":"active+clean","last_fresh":"2021-07-11T03:53:22.360825+0000","last_change":"2021-07-11T03:53:22.175189+0000","last_active":"2021-07-11T03:53:22.360825+0000","last_peered":"2021-07-11T03:53:22.360825+0000","last_clean":"2021-07-11T03:53:22.360825+0000","last_became_active":"2021-07-11T03:53:22.174938+0000","last_became_peered":"2021-07-11T03:53:22.174938+0000","last_unstale":"2021-07-11T03:53:22.360825+0000","last_undegraded":"2021-07-11T03:53:22.360825+0000","last_fullsized":"2021-07-11T03:53:22.360825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.506050+0000","last_change":"2021-07-11T03:53:22.359991+0000","last_active":"2021-07-11T03:53:24.506050+0000","last_peered":"2021-07-11T03:53:24.506050+0000","last_clean":"2021-07-11T03:53:24.506050+0000","last_became_active":"2021-07-11T03:53:22.359630+0000","last_became_peered":"2021-07-11T03:53:22.359630+0000","last_unstale":"2021-07-11T03:53:24.506050+0000","last_undegraded":"2021-07-11T03:53:24.506050+0000","last_fullsized":"2021-07-11T03:53:24.506050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.370292+0000","last_change":"2021-07-11T03:53:20.152489+0000","last_active":"2021-07-11T03:53:23.370292+0000","last_peered":"2021-07-11T03:53:23.370292+0000","last_clean":"2021-07-11T03:53:23.370292+0000","last_became_active":"2021-07-11T03:53:20.152282+0000","last_became_peered":"2021-07-11T03:53:20.152282+0000","last_unstale":"2021-07-11T03:53:23.370292+0000","last_undegraded":"2021-07-11T03:53:23.370292+0000","last_fullsized":"2021-07-11T03:53:23.370292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"11","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.506027+0000","last_change":"2021-07-11T03:53:22.175499+0000","last_active":"2021-07-11T03:53:24.506027+0000","last_peered":"2021-07-11T03:53:24.506027+0000","last_clean":"2021-07-11T03:53:24.506027+0000","last_became_active":"2021-07-11T03:53:22.175299+0000","last_became_peered":"2021-07-11T03:53:22.175299+0000","last_unstale":"2021-07-11T03:53:24.506027+0000","last_undegraded":"2021-07-11T03:53:24.506027+0000","last_fullsized":"2021-07-11T03:53:24.506027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.375038+0000","last_change":"2021-07-11T03:53:22.364550+0000","last_active":"2021-07-11T03:53:23.375038+0000","last_peered":"2021-07-11T03:53:23.375038+0000","last_clean":"2021-07-11T03:53:23.375038+0000","last_became_active":"2021-07-11T03:53:22.364318+0000","last_became_peered":"2021-07-11T03:53:22.364318+0000","last_unstale":"2021-07-11T03:53:23.375038+0000","last_undegraded":"2021-07-11T03:53:23.375038+0000","last_fullsized":"2021-07-11T03:53:23.375038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2021-07-11T03:53:22.359516+0000","last_change":"2021-07-11T03:53:20.153985+0000","last_active":"2021-07-11T03:53:22.359516+0000","last_peered":"2021-07-11T03:53:22.359516+0000","last_clean":"2021-07-11T03:53:22.359516+0000","last_became_active":"2021-07-11T03:53:20.153695+0000","last_became_peered":"2021-07-11T03:53:20.153695+0000","last_unstale":"2021-07-11T03:53:22.359516+0000","last_undegraded":"2021-07-11T03:53:22.359516+0000","last_fullsized":"2021-07-11T03:53:22.359516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"21'8","reported_seq":"19","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.374393+0000","last_change":"2021-07-11T03:53:22.174070+0000","last_active":"2021-07-11T03:53:23.374393+0000","last_peered":"2021-07-11T03:53:23.374393+0000","last_clean":"2021-07-11T03:53:23.374393+0000","last_became_active":"2021-07-11T03:53:22.173837+0000","last_became_peered":"2021-07-11T03:53:22.173837+0000","last_unstale":"2021-07-11T03:53:23.374393+0000","last_undegraded":"2021-07-11T03:53:23.374393+0000","last_fullsized":"2021-07-11T03:53:23.374393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.374687+0000","last_change":"2021-07-11T03:53:22.360989+0000","last_active":"2021-07-11T03:53:23.374687+0000","last_peered":"2021-07-11T03:53:23.374687+0000","last_clean":"2021-07-11T03:53:23.374687+0000","last_became_active":"2021-07-11T03:53:22.360703+0000","last_became_peered":"2021-07-11T03:53:22.360703+0000","last_unstale":"2021-07-11T03:53:23.374687+0000","last_undegraded":"2021-07-11T03:53:23.374687+0000","last_fullsized":"2021-07-11T03:53:23.374687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.509504+0000","last_change":"2021-07-11T03:53:20.153402+0000","last_active":"2021-07-11T03:53:24.509504+0000","last_peered":"2021-07-11T03:53:24.509504+0000","last_clean":"2021-07-11T03:53:24.509504+0000","last_became_active":"2021-07-11T03:53:20.153235+0000","last_became_peered":"2021-07-11T03:53:20.153235+0000","last_unstale":"2021-07-11T03:53:24.509504+0000","last_undegraded":"2021-07-11T03:53:24.509504+0000","last_fullsized":"2021-07-11T03:53:24.509504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'5","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.370127+0000","last_change":"2021-07-11T03:53:22.175125+0000","last_active":"2021-07-11T03:53:23.370127+0000","last_peered":"2021-07-11T03:53:23.370127+0000","last_clean":"2021-07-11T03:53:23.370127+0000","last_became_active":"2021-07-11T03:53:22.174940+0000","last_became_peered":"2021-07-11T03:53:22.174940+0000","last_unstale":"2021-07-11T03:53:23.370127+0000","last_undegraded":"2021-07-11T03:53:23.370127+0000","last_fullsized":"2021-07-11T03:53:23.370127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.374636+0000","last_change":"2021-07-11T03:53:22.364551+0000","last_active":"2021-07-11T03:53:23.374636+0000","last_peered":"2021-07-11T03:53:23.374636+0000","last_clean":"2021-07-11T03:53:23.374636+0000","last_became_active":"2021-07-11T03:53:22.364326+0000","last_became_peered":"2021-07-11T03:53:22.364326+0000","last_unstale":"2021-07-11T03:53:23.374636+0000","last_undegraded":"2021-07-11T03:53:23.374636+0000","last_fullsized":"2021-07-11T03:53:23.374636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"21'6","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.509680+0000","last_change":"2021-07-11T03:53:22.175328+0000","last_active":"2021-07-11T03:53:24.509680+0000","last_peered":"2021-07-11T03:53:24.509680+0000","last_clean":"2021-07-11T03:53:24.509680+0000","last_became_active":"2021-07-11T03:53:22.175161+0000","last_became_peered":"2021-07-11T03:53:22.175161+0000","last_unstale":"2021-07-11T03:53:24.509680+0000","last_undegraded":"2021-07-11T03:53:24.509680+0000","last_fullsized":"2021-07-11T03:53:24.509680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.374991+0000","last_change":"2021-07-11T03:53:20.286895+0000","last_active":"2021-07-11T03:53:23.374991+0000","last_peered":"2021-07-11T03:53:23.374991+0000","last_clean":"2021-07-11T03:53:23.374991+0000","last_became_active":"2021-07-11T03:53:20.286695+0000","last_became_peered":"2021-07-11T03:53:20.286695+0000","last_unstale":"2021-07-11T03:53:23.374991+0000","last_undegraded":"2021-07-11T03:53:23.374991+0000","last_fullsized":"2021-07-11T03:53:23.374991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.506057+0000","last_change":"2021-07-11T03:53:22.364636+0000","last_active":"2021-07-11T03:53:24.506057+0000","last_peered":"2021-07-11T03:53:24.506057+0000","last_clean":"2021-07-11T03:53:24.506057+0000","last_became_active":"2021-07-11T03:53:22.364374+0000","last_became_peered":"2021-07-11T03:53:22.364374+0000","last_unstale":"2021-07-11T03:53:24.506057+0000","last_undegraded":"2021-07-11T03:53:24.506057+0000","last_fullsized":"2021-07-11T03:53:24.506057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"21'8","reported_seq":"19","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.374893+0000","last_change":"2021-07-11T03:53:22.174091+0000","last_active":"2021-07-11T03:53:23.374893+0000","last_peered":"2021-07-11T03:53:23.374893+0000","last_clean":"2021-07-11T03:53:23.374893+0000","last_became_active":"2021-07-11T03:53:22.173847+0000","last_became_peered":"2021-07-11T03:53:22.173847+0000","last_unstale":"2021-07-11T03:53:23.374893+0000","last_undegraded":"2021-07-11T03:53:23.374893+0000","last_fullsized":"2021-07-11T03:53:23.374893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.370354+0000","last_change":"2021-07-11T03:53:20.288329+0000","last_active":"2021-07-11T03:53:23.370354+0000","last_peered":"2021-07-11T03:53:23.370354+0000","last_clean":"2021-07-11T03:53:23.370354+0000","last_became_active":"2021-07-11T03:53:20.287190+0000","last_became_peered":"2021-07-11T03:53:20.287190+0000","last_unstale":"2021-07-11T03:53:23.370354+0000","last_undegraded":"2021-07-11T03:53:23.370354+0000","last_fullsized":"2021-07-11T03:53:23.370354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2021-07-11T03:53:22.365174+0000","last_change":"2021-07-11T03:53:22.364440+0000","last_active":"2021-07-11T03:53:22.365174+0000","last_peered":"2021-07-11T03:53:22.365174+0000","last_clean":"2021-07-11T03:53:22.365174+0000","last_became_active":"2021-07-11T03:53:22.364223+0000","last_became_peered":"2021-07-11T03:53:22.364223+0000","last_unstale":"2021-07-11T03:53:22.365174+0000","last_undegraded":"2021-07-11T03:53:22.365174+0000","last_fullsized":"2021-07-11T03:53:22.365174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"21'7","reported_seq":"17","reported_epoch":"21","state":"active+clean","last_fresh":"2021-07-11T03:53:22.364171+0000","last_change":"2021-07-11T03:53:22.174256+0000","last_active":"2021-07-11T03:53:22.364171+0000","last_peered":"2021-07-11T03:53:22.364171+0000","last_clean":"2021-07-11T03:53:22.364171+0000","last_became_active":"2021-07-11T03:53:22.173993+0000","last_became_peered":"2021-07-11T03:53:22.173993+0000","last_unstale":"2021-07-11T03:53:22.364171+0000","last_undegraded":"2021-07-11T03:53:22.364171+0000","last_fullsized":"2021-07-11T03:53:22.364171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.375017+0000","last_change":"2021-07-11T03:53:20.153427+0000","last_active":"2021-07-11T03:53:23.375017+0000","last_peered":"2021-07-11T03:53:23.375017+0000","last_clean":"2021-07-11T03:53:23.375017+0000","last_became_active":"2021-07-11T03:53:20.153186+0000","last_became_peered":"2021-07-11T03:53:20.153186+0000","last_unstale":"2021-07-11T03:53:23.375017+0000","last_undegraded":"2021-07-11T03:53:23.375017+0000","last_fullsized":"2021-07-11T03:53:23.375017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.370096+0000","last_change":"2021-07-11T03:53:22.365406+0000","last_active":"2021-07-11T03:53:23.370096+0000","last_peered":"2021-07-11T03:53:23.370096+0000","last_clean":"2021-07-11T03:53:23.370096+0000","last_became_active":"2021-07-11T03:53:22.365249+0000","last_became_peered":"2021-07-11T03:53:22.365249+0000","last_unstale":"2021-07-11T03:53:23.370096+0000","last_undegraded":"2021-07-11T03:53:23.370096+0000","last_fullsized":"2021-07-11T03:53:23.370096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"23'4","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-07-11T03:53:24.873588+0000","last_change":"2021-07-11T03:53:22.174014+0000","last_active":"2021-07-11T03:53:24.873588+0000","last_peered":"2021-07-11T03:53:24.873588+0000","last_clean":"2021-07-11T03:53:24.873588+0000","last_became_active":"2021-07-11T03:53:22.173761+0000","last_became_peered":"2021-07-11T03:53:22.173761+0000","last_unstale":"2021-07-11T03:53:24.873588+0000","last_undegraded":"2021-07-11T03:53:24.873588+0000","last_fullsized":"2021-07-11T03:53:24.873588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.375276+0000","last_change":"2021-07-11T03:53:20.153252+0000","last_active":"2021-07-11T03:53:23.375276+0000","last_peered":"2021-07-11T03:53:23.375276+0000","last_clean":"2021-07-11T03:53:23.375276+0000","last_became_active":"2021-07-11T03:53:20.153046+0000","last_became_peered":"2021-07-11T03:53:20.153046+0000","last_unstale":"2021-07-11T03:53:23.375276+0000","last_undegraded":"2021-07-11T03:53:23.375276+0000","last_fullsized":"2021-07-11T03:53:23.375276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048824,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323016,"statfs":{"total":96636764160,"available":95562768384,"internally_reserved":1073741824,"allocated":245760,"data_stored":89606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6599999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048800,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323040,"statfs":{"total":96636764160,"available":95562792960,"internally_reserved":1073741824,"allocated":221184,"data_stored":81808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048824,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323016,"statfs":{"total":96636764160,"available":95562768384,"internally_reserved":1073741824,"allocated":245760,"data_stored":89191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":88462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5840000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":89766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.17799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.217}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048828,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323012,"statfs":{"total":96636764160,"available":95562764288,"internally_reserved":1073741824,"allocated":249856,"data_stored":90566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":83946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.224}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323028,"statfs":{"total":96636764160,"available":95562780672,"internally_reserved":1073741824,"allocated":233472,"data_stored":87884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]}]}} 2021-07-11T03:53:29.341 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-07-11T03:53:29.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aab2ad700 1 Processor -- start 2021-07-11T03:53:29.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aab2ad700 1 -- start start 2021-07-11T03:53:29.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aab2ad700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3aa40a76b0 0x7f3aa4066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aab2ad700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3aa40b0060 0x7f3aa405fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aab2ad700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa40a7a90 con 0x7f3aa40b1970 2021-07-11T03:53:29.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aab2ad700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa4147620 con 0x7f3aa40b0060 2021-07-11T03:53:29.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aab2ad700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa41478c0 con 0x7f3aa40a76b0 2021-07-11T03:53:29.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aa984a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3aa40b0060 0x7f3aa405fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aa984a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3aa40b0060 0x7f3aa405fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42160/0 (socket says 172.21.2.104:42160) 2021-07-11T03:53:29.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aa984a700 1 -- 172.21.2.104:0/147805497 learned_addr learned my addr 172.21.2.104:0/147805497 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:29.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aa984a700 1 -- 172.21.2.104:0/147805497 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3aa40a76b0 msgr2=0x7f3aa4066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:29.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aa984a700 1 --2- 172.21.2.104:0/147805497 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3aa40a76b0 0x7f3aa4066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aa984a700 1 -- 172.21.2.104:0/147805497 >> v1:172.21.2.105:6789/0 conn(0x7f3aa40b1970 legacy=0x7f3aa405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:29.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aa984a700 1 -- 172.21.2.104:0/147805497 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4147b60 con 0x7f3aa40b0060 2021-07-11T03:53:29.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.388+0000 7f3aa984a700 1 --2- 172.21.2.104:0/147805497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3aa40b0060 0x7f3aa405fdc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f3aa40b6460 tx=0x7f3a9c009290).ready entity=mon.0 client_cookie=f1748c906fa25d14 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3a97fff700 1 -- 172.21.2.104:0/147805497 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a9c00b860 con 0x7f3aa40b0060 2021-07-11T03:53:29.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3a97fff700 1 -- 172.21.2.104:0/147805497 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c00b9c0 con 0x7f3aa40b0060 2021-07-11T03:53:29.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3a97fff700 1 -- 172.21.2.104:0/147805497 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a9c00bb70 con 0x7f3aa40b0060 2021-07-11T03:53:29.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- 172.21.2.104:0/147805497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3aa40b0060 msgr2=0x7f3aa405fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 --2- 172.21.2.104:0/147805497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3aa40b0060 0x7f3aa405fdc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f3aa40b6460 tx=0x7f3a9c009290).stop 2021-07-11T03:53:29.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- 172.21.2.104:0/147805497 shutdown_connections 2021-07-11T03:53:29.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 --2- 172.21.2.104:0/147805497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3aa40b0060 0x7f3aa405fdc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 --2- 172.21.2.104:0/147805497 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3aa40a76b0 0x7f3aa4066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- 172.21.2.104:0/147805497 >> 172.21.2.104:0/147805497 conn(0x7f3aa40b53e0 msgr2=0x7f3aa40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- 172.21.2.104:0/147805497 shutdown_connections 2021-07-11T03:53:29.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- 172.21.2.104:0/147805497 wait complete. 2021-07-11T03:53:29.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 Processor -- start 2021-07-11T03:53:29.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- start start 2021-07-11T03:53:29.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3aa40a76b0 0x7f3aa414ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3aa40b0060 0x7f3aa4144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3aa40b1970 0x7f3aa4144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40ac670 con 0x7f3aa40b0060 2021-07-11T03:53:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40ab240 con 0x7f3aa40a76b0 2021-07-11T03:53:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40ab4e0 con 0x7f3aa40b1970 2021-07-11T03:53:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aa8848700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3aa40b0060 0x7f3aa4144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aa8848700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3aa40b0060 0x7f3aa4144600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42162/0 (socket says 172.21.2.104:42162) 2021-07-11T03:53:29.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aa8848700 1 -- 172.21.2.104:0/3445438886 learned_addr learned my addr 172.21.2.104:0/3445438886 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:29.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aa8848700 1 -- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3aa40b1970 msgr2=0x7f3aa4144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aa8848700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3aa40b1970 0x7f3aa4144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aa8848700 1 -- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3aa40a76b0 msgr2=0x7f3aa414ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aa8848700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3aa40a76b0 0x7f3aa414ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aa8848700 1 -- 172.21.2.104:0/3445438886 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c009530 con 0x7f3aa40b0060 2021-07-11T03:53:29.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aa8848700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3aa40b0060 0x7f3aa4144600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f3aa0002f10 tx=0x7f3aa0006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3a967fc700 1 -- 172.21.2.104:0/3445438886 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3aa000f510 con 0x7f3aa40b0060 2021-07-11T03:53:29.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3a967fc700 1 -- 172.21.2.104:0/3445438886 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa000f670 con 0x7f3aa40b0060 2021-07-11T03:53:29.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- 172.21.2.104:0/3445438886 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa40ab780 con 0x7f3aa40b0060 2021-07-11T03:53:29.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3a967fc700 1 -- 172.21.2.104:0/3445438886 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3aa000f7e0 con 0x7f3aa40b0060 2021-07-11T03:53:29.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- 172.21.2.104:0/3445438886 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa40ae3f0 con 0x7f3aa40b0060 2021-07-11T03:53:29.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3aab2ad700 1 -- 172.21.2.104:0/3445438886 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa4057350 con 0x7f3aa40b0060 2021-07-11T03:53:29.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3a967fc700 1 -- 172.21.2.104:0/3445438886 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3aa0017020 con 0x7f3aa40b0060 2021-07-11T03:53:29.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.392+0000 7f3a967fc700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a840531f0 0x7f3a840556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.396+0000 7f3a967fc700 1 -- 172.21.2.104:0/3445438886 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3aa006ac40 con 0x7f3aa40b0060 2021-07-11T03:53:29.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.396+0000 7f3aa9049700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a840531f0 0x7f3a840556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.396+0000 7f3aa9049700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a840531f0 0x7f3a840556b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3a98002f70 tx=0x7f3a98006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.396+0000 7f3a967fc700 1 -- 172.21.2.104:0/3445438886 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3aa0041f80 con 0x7f3aa40b0060 2021-07-11T03:53:29.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.476+0000 7f3aab2ad700 1 -- 172.21.2.104:0/3445438886 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3aa40abc20 con 0x7f3a840531f0 2021-07-11T03:53:29.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3a967fc700 1 -- 172.21.2.104:0/3445438886 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82337 (secure 0 0 0) 0x7f3aa40abc20 con 0x7f3a840531f0 2021-07-11T03:53:29.492 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:29.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 -- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a840531f0 msgr2=0x7f3a840556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a840531f0 0x7f3a840556b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3a98002f70 tx=0x7f3a98006a80).stop 2021-07-11T03:53:29.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 -- 172.21.2.104:0/3445438886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3aa40b0060 msgr2=0x7f3aa4144600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3aa40b0060 0x7f3aa4144600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f3aa0002f10 tx=0x7f3aa0006d70).stop 2021-07-11T03:53:29.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 -- 172.21.2.104:0/3445438886 shutdown_connections 2021-07-11T03:53:29.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3aa40b1970 0x7f3aa4144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a840531f0 0x7f3a840556b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3aa40b0060 0x7f3aa4144600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 --2- 172.21.2.104:0/3445438886 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3aa40a76b0 0x7f3aa414ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 -- 172.21.2.104:0/3445438886 >> 172.21.2.104:0/3445438886 conn(0x7f3aa40b53e0 msgr2=0x7f3aa4149cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 -- 172.21.2.104:0/3445438886 shutdown_connections 2021-07-11T03:53:29.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.488+0000 7f3aab2ad700 1 -- 172.21.2.104:0/3445438886 wait complete. 2021-07-11T03:53:29.496 INFO:teuthology.orchestra.run.gibba004.stderr:dumped all 2021-07-11T03:53:29.498 INFO:teuthology.orchestra.run.gibba004.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2021-07-11T03:53:28.366276+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":44,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390552,"kb_used_data":1880,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584168,"statfs":{"total":773094113280,"available":764502188032,"internally_reserved":8589934592,"allocated":1925120,"data_stored":701229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.008262"},"pg_stats":[{"pgid":"4.5","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.377538+0000","last_change":"2021-07-11T03:53:23.376788+0000","last_active":"2021-07-11T03:53:23.377538+0000","last_peered":"2021-07-11T03:53:23.377538+0000","last_clean":"2021-07-11T03:53:23.377538+0000","last_became_active":"2021-07-11T03:53:23.376580+0000","last_became_peered":"2021-07-11T03:53:23.376580+0000","last_unstale":"2021-07-11T03:53:23.377538+0000","last_undegraded":"2021-07-11T03:53:23.377538+0000","last_fullsized":"2021-07-11T03:53:23.377538+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.370366+0000","last_change":"2021-07-11T03:53:15.964238+0000","last_active":"2021-07-11T03:53:23.370366+0000","last_peered":"2021-07-11T03:53:23.370366+0000","last_clean":"2021-07-11T03:53:23.370366+0000","last_became_active":"2021-07-11T03:53:15.962801+0000","last_became_peered":"2021-07-11T03:53:15.962801+0000","last_unstale":"2021-07-11T03:53:23.370366+0000","last_undegraded":"2021-07-11T03:53:23.370366+0000","last_fullsized":"2021-07-11T03:53:23.370366+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:11.361771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:11.361771+0000","last_clean_scrub_stamp":"2021-07-11T03:53:11.361771+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.510092+0000","last_change":"2021-07-11T03:53:20.153399+0000","last_active":"2021-07-11T03:53:24.510092+0000","last_peered":"2021-07-11T03:53:24.510092+0000","last_clean":"2021-07-11T03:53:24.510092+0000","last_became_active":"2021-07-11T03:53:20.153135+0000","last_became_peered":"2021-07-11T03:53:20.153135+0000","last_unstale":"2021-07-11T03:53:24.510092+0000","last_undegraded":"2021-07-11T03:53:24.510092+0000","last_fullsized":"2021-07-11T03:53:24.510092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"21'1","reported_seq":"11","reported_epoch":"21","state":"active+clean","last_fresh":"2021-07-11T03:53:22.360825+0000","last_change":"2021-07-11T03:53:22.175189+0000","last_active":"2021-07-11T03:53:22.360825+0000","last_peered":"2021-07-11T03:53:22.360825+0000","last_clean":"2021-07-11T03:53:22.360825+0000","last_became_active":"2021-07-11T03:53:22.174938+0000","last_became_peered":"2021-07-11T03:53:22.174938+0000","last_unstale":"2021-07-11T03:53:22.360825+0000","last_undegraded":"2021-07-11T03:53:22.360825+0000","last_fullsized":"2021-07-11T03:53:22.360825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.506050+0000","last_change":"2021-07-11T03:53:22.359991+0000","last_active":"2021-07-11T03:53:24.506050+0000","last_peered":"2021-07-11T03:53:24.506050+0000","last_clean":"2021-07-11T03:53:24.506050+0000","last_became_active":"2021-07-11T03:53:22.359630+0000","last_became_peered":"2021-07-11T03:53:22.359630+0000","last_unstale":"2021-07-11T03:53:24.506050+0000","last_undegraded":"2021-07-11T03:53:24.506050+0000","last_fullsized":"2021-07-11T03:53:24.506050+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.370292+0000","last_change":"2021-07-11T03:53:20.152489+0000","last_active":"2021-07-11T03:53:23.370292+0000","last_peered":"2021-07-11T03:53:23.370292+0000","last_clean":"2021-07-11T03:53:23.370292+0000","last_became_active":"2021-07-11T03:53:20.152282+0000","last_became_peered":"2021-07-11T03:53:20.152282+0000","last_unstale":"2021-07-11T03:53:23.370292+0000","last_undegraded":"2021-07-11T03:53:23.370292+0000","last_fullsized":"2021-07-11T03:53:23.370292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"11","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.506027+0000","last_change":"2021-07-11T03:53:22.175499+0000","last_active":"2021-07-11T03:53:24.506027+0000","last_peered":"2021-07-11T03:53:24.506027+0000","last_clean":"2021-07-11T03:53:24.506027+0000","last_became_active":"2021-07-11T03:53:22.175299+0000","last_became_peered":"2021-07-11T03:53:22.175299+0000","last_unstale":"2021-07-11T03:53:24.506027+0000","last_undegraded":"2021-07-11T03:53:24.506027+0000","last_fullsized":"2021-07-11T03:53:24.506027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.375038+0000","last_change":"2021-07-11T03:53:22.364550+0000","last_active":"2021-07-11T03:53:23.375038+0000","last_peered":"2021-07-11T03:53:23.375038+0000","last_clean":"2021-07-11T03:53:23.375038+0000","last_became_active":"2021-07-11T03:53:22.364318+0000","last_became_peered":"2021-07-11T03:53:22.364318+0000","last_unstale":"2021-07-11T03:53:23.375038+0000","last_undegraded":"2021-07-11T03:53:23.375038+0000","last_fullsized":"2021-07-11T03:53:23.375038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"12","reported_epoch":"21","state":"active+clean","last_fresh":"2021-07-11T03:53:22.359516+0000","last_change":"2021-07-11T03:53:20.153985+0000","last_active":"2021-07-11T03:53:22.359516+0000","last_peered":"2021-07-11T03:53:22.359516+0000","last_clean":"2021-07-11T03:53:22.359516+0000","last_became_active":"2021-07-11T03:53:20.153695+0000","last_became_peered":"2021-07-11T03:53:20.153695+0000","last_unstale":"2021-07-11T03:53:22.359516+0000","last_undegraded":"2021-07-11T03:53:22.359516+0000","last_fullsized":"2021-07-11T03:53:22.359516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"21'8","reported_seq":"19","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.374393+0000","last_change":"2021-07-11T03:53:22.174070+0000","last_active":"2021-07-11T03:53:23.374393+0000","last_peered":"2021-07-11T03:53:23.374393+0000","last_clean":"2021-07-11T03:53:23.374393+0000","last_became_active":"2021-07-11T03:53:22.173837+0000","last_became_peered":"2021-07-11T03:53:22.173837+0000","last_unstale":"2021-07-11T03:53:23.374393+0000","last_undegraded":"2021-07-11T03:53:23.374393+0000","last_fullsized":"2021-07-11T03:53:23.374393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.374687+0000","last_change":"2021-07-11T03:53:22.360989+0000","last_active":"2021-07-11T03:53:23.374687+0000","last_peered":"2021-07-11T03:53:23.374687+0000","last_clean":"2021-07-11T03:53:23.374687+0000","last_became_active":"2021-07-11T03:53:22.360703+0000","last_became_peered":"2021-07-11T03:53:22.360703+0000","last_unstale":"2021-07-11T03:53:23.374687+0000","last_undegraded":"2021-07-11T03:53:23.374687+0000","last_fullsized":"2021-07-11T03:53:23.374687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.509504+0000","last_change":"2021-07-11T03:53:20.153402+0000","last_active":"2021-07-11T03:53:24.509504+0000","last_peered":"2021-07-11T03:53:24.509504+0000","last_clean":"2021-07-11T03:53:24.509504+0000","last_became_active":"2021-07-11T03:53:20.153235+0000","last_became_peered":"2021-07-11T03:53:20.153235+0000","last_unstale":"2021-07-11T03:53:24.509504+0000","last_undegraded":"2021-07-11T03:53:24.509504+0000","last_fullsized":"2021-07-11T03:53:24.509504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"21'5","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.370127+0000","last_change":"2021-07-11T03:53:22.175125+0000","last_active":"2021-07-11T03:53:23.370127+0000","last_peered":"2021-07-11T03:53:23.370127+0000","last_clean":"2021-07-11T03:53:23.370127+0000","last_became_active":"2021-07-11T03:53:22.174940+0000","last_became_peered":"2021-07-11T03:53:22.174940+0000","last_unstale":"2021-07-11T03:53:23.370127+0000","last_undegraded":"2021-07-11T03:53:23.370127+0000","last_fullsized":"2021-07-11T03:53:23.370127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.374636+0000","last_change":"2021-07-11T03:53:22.364551+0000","last_active":"2021-07-11T03:53:23.374636+0000","last_peered":"2021-07-11T03:53:23.374636+0000","last_clean":"2021-07-11T03:53:23.374636+0000","last_became_active":"2021-07-11T03:53:22.364326+0000","last_became_peered":"2021-07-11T03:53:22.364326+0000","last_unstale":"2021-07-11T03:53:23.374636+0000","last_undegraded":"2021-07-11T03:53:23.374636+0000","last_fullsized":"2021-07-11T03:53:23.374636+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"21'6","reported_seq":"17","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.509680+0000","last_change":"2021-07-11T03:53:22.175328+0000","last_active":"2021-07-11T03:53:24.509680+0000","last_peered":"2021-07-11T03:53:24.509680+0000","last_clean":"2021-07-11T03:53:24.509680+0000","last_became_active":"2021-07-11T03:53:22.175161+0000","last_became_peered":"2021-07-11T03:53:22.175161+0000","last_unstale":"2021-07-11T03:53:24.509680+0000","last_undegraded":"2021-07-11T03:53:24.509680+0000","last_fullsized":"2021-07-11T03:53:24.509680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.374991+0000","last_change":"2021-07-11T03:53:20.286895+0000","last_active":"2021-07-11T03:53:23.374991+0000","last_peered":"2021-07-11T03:53:23.374991+0000","last_clean":"2021-07-11T03:53:23.374991+0000","last_became_active":"2021-07-11T03:53:20.286695+0000","last_became_peered":"2021-07-11T03:53:20.286695+0000","last_unstale":"2021-07-11T03:53:23.374991+0000","last_undegraded":"2021-07-11T03:53:23.374991+0000","last_fullsized":"2021-07-11T03:53:23.374991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:24.506057+0000","last_change":"2021-07-11T03:53:22.364636+0000","last_active":"2021-07-11T03:53:24.506057+0000","last_peered":"2021-07-11T03:53:24.506057+0000","last_clean":"2021-07-11T03:53:24.506057+0000","last_became_active":"2021-07-11T03:53:22.364374+0000","last_became_peered":"2021-07-11T03:53:22.364374+0000","last_unstale":"2021-07-11T03:53:24.506057+0000","last_undegraded":"2021-07-11T03:53:24.506057+0000","last_fullsized":"2021-07-11T03:53:24.506057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"21'8","reported_seq":"19","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.374893+0000","last_change":"2021-07-11T03:53:22.174091+0000","last_active":"2021-07-11T03:53:23.374893+0000","last_peered":"2021-07-11T03:53:23.374893+0000","last_clean":"2021-07-11T03:53:23.374893+0000","last_became_active":"2021-07-11T03:53:22.173847+0000","last_became_peered":"2021-07-11T03:53:22.173847+0000","last_unstale":"2021-07-11T03:53:23.374893+0000","last_undegraded":"2021-07-11T03:53:23.374893+0000","last_fullsized":"2021-07-11T03:53:23.374893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.370354+0000","last_change":"2021-07-11T03:53:20.288329+0000","last_active":"2021-07-11T03:53:23.370354+0000","last_peered":"2021-07-11T03:53:23.370354+0000","last_clean":"2021-07-11T03:53:23.370354+0000","last_became_active":"2021-07-11T03:53:20.287190+0000","last_became_peered":"2021-07-11T03:53:20.287190+0000","last_unstale":"2021-07-11T03:53:23.370354+0000","last_undegraded":"2021-07-11T03:53:23.370354+0000","last_fullsized":"2021-07-11T03:53:23.370354+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"21","state":"active+clean","last_fresh":"2021-07-11T03:53:22.365174+0000","last_change":"2021-07-11T03:53:22.364440+0000","last_active":"2021-07-11T03:53:22.365174+0000","last_peered":"2021-07-11T03:53:22.365174+0000","last_clean":"2021-07-11T03:53:22.365174+0000","last_became_active":"2021-07-11T03:53:22.364223+0000","last_became_peered":"2021-07-11T03:53:22.364223+0000","last_unstale":"2021-07-11T03:53:22.365174+0000","last_undegraded":"2021-07-11T03:53:22.365174+0000","last_fullsized":"2021-07-11T03:53:22.365174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"21'7","reported_seq":"17","reported_epoch":"21","state":"active+clean","last_fresh":"2021-07-11T03:53:22.364171+0000","last_change":"2021-07-11T03:53:22.174256+0000","last_active":"2021-07-11T03:53:22.364171+0000","last_peered":"2021-07-11T03:53:22.364171+0000","last_clean":"2021-07-11T03:53:22.364171+0000","last_became_active":"2021-07-11T03:53:22.173993+0000","last_became_peered":"2021-07-11T03:53:22.173993+0000","last_unstale":"2021-07-11T03:53:22.364171+0000","last_undegraded":"2021-07-11T03:53:22.364171+0000","last_fullsized":"2021-07-11T03:53:22.364171+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":7,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.375017+0000","last_change":"2021-07-11T03:53:20.153427+0000","last_active":"2021-07-11T03:53:23.375017+0000","last_peered":"2021-07-11T03:53:23.375017+0000","last_clean":"2021-07-11T03:53:23.375017+0000","last_became_active":"2021-07-11T03:53:20.153186+0000","last_became_peered":"2021-07-11T03:53:20.153186+0000","last_unstale":"2021-07-11T03:53:23.375017+0000","last_undegraded":"2021-07-11T03:53:23.375017+0000","last_fullsized":"2021-07-11T03:53:23.375017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.370096+0000","last_change":"2021-07-11T03:53:22.365406+0000","last_active":"2021-07-11T03:53:23.370096+0000","last_peered":"2021-07-11T03:53:23.370096+0000","last_clean":"2021-07-11T03:53:23.370096+0000","last_became_active":"2021-07-11T03:53:22.365249+0000","last_became_peered":"2021-07-11T03:53:22.365249+0000","last_unstale":"2021-07-11T03:53:23.370096+0000","last_undegraded":"2021-07-11T03:53:23.370096+0000","last_fullsized":"2021-07-11T03:53:23.370096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:22.158178+0000","last_clean_scrub_stamp":"2021-07-11T03:53:22.158178+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"23'4","reported_seq":"15","reported_epoch":"23","state":"active+clean","last_fresh":"2021-07-11T03:53:24.873588+0000","last_change":"2021-07-11T03:53:22.174014+0000","last_active":"2021-07-11T03:53:24.873588+0000","last_peered":"2021-07-11T03:53:24.873588+0000","last_clean":"2021-07-11T03:53:24.873588+0000","last_became_active":"2021-07-11T03:53:22.173761+0000","last_became_peered":"2021-07-11T03:53:22.173761+0000","last_unstale":"2021-07-11T03:53:24.873588+0000","last_undegraded":"2021-07-11T03:53:24.873588+0000","last_fullsized":"2021-07-11T03:53:24.873588+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:21.152205+0000","last_clean_scrub_stamp":"2021-07-11T03:53:21.152205+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"12","reported_epoch":"22","state":"active+clean","last_fresh":"2021-07-11T03:53:23.375276+0000","last_change":"2021-07-11T03:53:20.153252+0000","last_active":"2021-07-11T03:53:23.375276+0000","last_peered":"2021-07-11T03:53:23.375276+0000","last_clean":"2021-07-11T03:53:23.375276+0000","last_became_active":"2021-07-11T03:53:20.153046+0000","last_became_peered":"2021-07-11T03:53:20.153046+0000","last_unstale":"2021-07-11T03:53:23.375276+0000","last_undegraded":"2021-07-11T03:53:23.375276+0000","last_fullsized":"2021-07-11T03:53:23.375276+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":16,"acting":16,"num_store_stats":7},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048824,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323016,"statfs":{"total":96636764160,"available":95562768384,"internally_reserved":1073741824,"allocated":245760,"data_stored":89606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6599999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048800,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323040,"statfs":{"total":96636764160,"available":95562792960,"internally_reserved":1073741824,"allocated":221184,"data_stored":81808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.577}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.214}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048824,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323016,"statfs":{"total":96636764160,"available":95562768384,"internally_reserved":1073741824,"allocated":245760,"data_stored":89191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0580000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":88462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5840000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048832,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323008,"statfs":{"total":96636764160,"available":95562760192,"internally_reserved":1073741824,"allocated":253952,"data_stored":89766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.17799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.217}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94899999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048828,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323012,"statfs":{"total":96636764160,"available":95562764288,"internally_reserved":1073741824,"allocated":249856,"data_stored":90566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048816,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323024,"statfs":{"total":96636764160,"available":95562776576,"internally_reserved":1073741824,"allocated":237568,"data_stored":83946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.224}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.252}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048812,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323028,"statfs":{"total":96636764160,"available":95562780672,"internally_reserved":1073741824,"allocated":233472,"data_stored":87884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.21099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94299999999999995}]}]}]}} 2021-07-11T03:53:29.500 INFO:tasks.ceph.ceph_manager.ceph:clean! 2021-07-11T03:53:29.501 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2021-07-11T03:53:29.501 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2021-07-11T03:53:29.502 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2021-07-11T03:53:29.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa309834700 1 Processor -- start 2021-07-11T03:53:29.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa309834700 1 -- start start 2021-07-11T03:53:29.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa309834700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa3040b7810 0x7fa3041445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa309834700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3040b1170 0x7fa304144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa309834700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3040b31d0 con 0x7fa304064870 2021-07-11T03:53:29.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa309834700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3040b7bf0 con 0x7fa3040b7810 2021-07-11T03:53:29.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa309834700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040675c0 con 0x7fa3040b1170 2021-07-11T03:53:29.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa302ffd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa3040b7810 0x7fa3041445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa3027fc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3040b1170 0x7fa304144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa3027fc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3040b1170 0x7fa304144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:45336/0 (socket says 172.21.2.104:45336) 2021-07-11T03:53:29.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa3027fc700 1 -- 172.21.2.104:0/3418083050 learned_addr learned my addr 172.21.2.104:0/3418083050 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:29.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.548+0000 7fa3037fe700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fa304064870 0x7fa304064cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:38312/0 (socket says 172.21.2.104:38312) 2021-07-11T03:53:29.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 -- 172.21.2.104:0/3418083050 >> v2:172.21.2.105:3300/0 conn(0x7fa3040b7810 msgr2=0x7fa3041445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:29.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 --2- 172.21.2.104:0/3418083050 >> v2:172.21.2.105:3300/0 conn(0x7fa3040b7810 0x7fa3041445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 -- 172.21.2.104:0/3418083050 >> v1:172.21.2.105:6789/0 conn(0x7fa304064870 legacy=0x7fa304064cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 -- 172.21.2.104:0/3418083050 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3040aae70 con 0x7fa3040b1170 2021-07-11T03:53:29.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 --2- 172.21.2.104:0/3418083050 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3040b1170 0x7fa304144a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fa2ec002f70 tx=0x7fa2ec006eb0).ready entity=mon.2 client_cookie=8e2fa85afcef98b2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa301ffb700 1 -- 172.21.2.104:0/3418083050 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2ec00d570 con 0x7fa3040b1170 2021-07-11T03:53:29.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa301ffb700 1 -- 172.21.2.104:0/3418083050 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec000e90 con 0x7fa3040b1170 2021-07-11T03:53:29.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa301ffb700 1 -- 172.21.2.104:0/3418083050 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2ec00d900 con 0x7fa3040b1170 2021-07-11T03:53:29.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 -- 172.21.2.104:0/3418083050 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3040b1170 msgr2=0x7fa304144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 --2- 172.21.2.104:0/3418083050 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3040b1170 0x7fa304144a90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fa2ec002f70 tx=0x7fa2ec006eb0).stop 2021-07-11T03:53:29.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 -- 172.21.2.104:0/3418083050 shutdown_connections 2021-07-11T03:53:29.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 --2- 172.21.2.104:0/3418083050 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3040b1170 0x7fa304144a90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 --2- 172.21.2.104:0/3418083050 >> v2:172.21.2.105:3300/0 conn(0x7fa3040b7810 0x7fa3041445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 -- 172.21.2.104:0/3418083050 >> 172.21.2.104:0/3418083050 conn(0x7fa30405f6b0 msgr2=0x7fa30405fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 -- 172.21.2.104:0/3418083050 shutdown_connections 2021-07-11T03:53:29.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 -- 172.21.2.104:0/3418083050 wait complete. 2021-07-11T03:53:29.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 Processor -- start 2021-07-11T03:53:29.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 -- start start 2021-07-11T03:53:29.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa304064870 0x7fa3040add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3040b1170 0x7fa30413f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3040b7810 0x7fa30413f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3041579f0 con 0x7fa3040b1170 2021-07-11T03:53:29.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040a95a0 con 0x7fa3040b7810 2021-07-11T03:53:29.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa309834700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3040a9840 con 0x7fa304064870 2021-07-11T03:53:29.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3040b1170 0x7fa30413f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3040b1170 0x7fa30413f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42180/0 (socket says 172.21.2.104:42180) 2021-07-11T03:53:29.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 -- 172.21.2.104:0/2041294965 learned_addr learned my addr 172.21.2.104:0/2041294965 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:29.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3037fe700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3040b7810 0x7fa30413f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa302ffd700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa304064870 0x7fa3040add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 -- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa304064870 msgr2=0x7fa3040add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa304064870 0x7fa3040add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 -- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3040b7810 msgr2=0x7fa30413f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3040b7810 0x7fa30413f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa3027fc700 1 -- 172.21.2.104:0/2041294965 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec00b1e0 con 0x7fa3040b1170 2021-07-11T03:53:29.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.552+0000 7fa302ffd700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa304064870 0x7fa3040add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:29.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa3027fc700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3040b1170 0x7fa30413f410 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fa2ec002f70 tx=0x7fa2ec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa2fbfff700 1 -- 172.21.2.104:0/2041294965 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2ec011860 con 0x7fa3040b1170 2021-07-11T03:53:29.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa2fbfff700 1 -- 172.21.2.104:0/2041294965 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec0119c0 con 0x7fa3040b1170 2021-07-11T03:53:29.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa309834700 1 -- 172.21.2.104:0/2041294965 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa304157670 con 0x7fa3040b1170 2021-07-11T03:53:29.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa309834700 1 -- 172.21.2.104:0/2041294965 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30413ff90 con 0x7fa3040b1170 2021-07-11T03:53:29.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa2fbfff700 1 -- 172.21.2.104:0/2041294965 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa2ec011d20 con 0x7fa3040b1170 2021-07-11T03:53:29.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa309834700 1 -- 172.21.2.104:0/2041294965 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa304057350 con 0x7fa3040b1170 2021-07-11T03:53:29.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa2fbfff700 1 -- 172.21.2.104:0/2041294965 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa2ec00f070 con 0x7fa3040b1170 2021-07-11T03:53:29.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa2fbfff700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2e8053220 0x7fa2e80556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa2fbfff700 1 -- 172.21.2.104:0/2041294965 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa2ec06cc80 con 0x7fa3040b1170 2021-07-11T03:53:29.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa302ffd700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2e8053220 0x7fa2e80556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.556+0000 7fa302ffd700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2e8053220 0x7fa2e80556e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa2f400b000 tx=0x7fa2f4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.560+0000 7fa2fbfff700 1 -- 172.21.2.104:0/2041294965 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa2ec043fc0 con 0x7fa3040b1170 2021-07-11T03:53:29.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 -- 172.21.2.104:0/2041294965 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa3040b0e50 con 0x7fa3040b1170 2021-07-11T03:53:29.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa2fbfff700 1 -- 172.21.2.104:0/2041294965 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7fa2ec0475e0 con 0x7fa3040b1170 2021-07-11T03:53:29.672 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:29.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 -- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2e8053220 msgr2=0x7fa2e80556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2e8053220 0x7fa2e80556e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa2f400b000 tx=0x7fa2f4008040).stop 2021-07-11T03:53:29.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 -- 172.21.2.104:0/2041294965 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3040b1170 msgr2=0x7fa30413f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3040b1170 0x7fa30413f410 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fa2ec002f70 tx=0x7fa2ec015000).stop 2021-07-11T03:53:29.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 -- 172.21.2.104:0/2041294965 shutdown_connections 2021-07-11T03:53:29.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3040b7810 0x7fa30413f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2e8053220 0x7fa2e80556e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3040b1170 0x7fa30413f410 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 --2- 172.21.2.104:0/2041294965 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa304064870 0x7fa3040add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 -- 172.21.2.104:0/2041294965 >> 172.21.2.104:0/2041294965 conn(0x7fa30405f6b0 msgr2=0x7fa304144cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 -- 172.21.2.104:0/2041294965 shutdown_connections 2021-07-11T03:53:29.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.668+0000 7fa309834700 1 -- 172.21.2.104:0/2041294965 wait complete. 2021-07-11T03:53:29.679 INFO:teuthology.orchestra.run.gibba004.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-07-11T03:53:29.680 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2021-07-11T03:53:29.680 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:29.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.724+0000 7fce5f5ce700 1 Processor -- start 2021-07-11T03:53:29.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- start start 2021-07-11T03:53:29.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fce58067180 0x7fce5805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fce580b1970 0x7fce5805fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce580a7a90 con 0x7fce580a76b0 2021-07-11T03:53:29.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce58147620 con 0x7fce58067180 2021-07-11T03:53:29.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce581478c0 con 0x7fce580b1970 2021-07-11T03:53:29.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5d36a700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fce580a76b0 0x7fce58066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38952/0 (socket says 172.21.2.104:38952) 2021-07-11T03:53:29.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5d36a700 1 -- 172.21.2.104:0/3654039472 learned_addr learned my addr 172.21.2.104:0/3654039472 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:29.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5cb69700 1 --2- 172.21.2.104:0/3654039472 >> v2:172.21.2.104:3300/0 conn(0x7fce58067180 0x7fce5805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5cb69700 1 -- 172.21.2.104:0/3654039472 >> v2:172.21.2.105:3300/0 conn(0x7fce580b1970 msgr2=0x7fce5805fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:29.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5cb69700 1 --2- 172.21.2.104:0/3654039472 >> v2:172.21.2.105:3300/0 conn(0x7fce580b1970 0x7fce5805fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5cb69700 1 -- 172.21.2.104:0/3654039472 >> v1:172.21.2.104:6789/0 conn(0x7fce580a76b0 legacy=0x7fce58066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5cb69700 1 -- 172.21.2.104:0/3654039472 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58147b60 con 0x7fce58067180 2021-07-11T03:53:29.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5cb69700 1 --2- 172.21.2.104:0/3654039472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce58067180 0x7fce5805f6b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fce54002f70 tx=0x7fce54006df0).ready entity=mon.0 client_cookie=704503db44ee2b5b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce4bfff700 1 -- 172.21.2.104:0/3654039472 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce5400d5c0 con 0x7fce58067180 2021-07-11T03:53:29.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce4bfff700 1 -- 172.21.2.104:0/3654039472 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce54000e90 con 0x7fce58067180 2021-07-11T03:53:29.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce4bfff700 1 -- 172.21.2.104:0/3654039472 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce5400d950 con 0x7fce58067180 2021-07-11T03:53:29.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- 172.21.2.104:0/3654039472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce58067180 msgr2=0x7fce5805f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 --2- 172.21.2.104:0/3654039472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce58067180 0x7fce5805f6b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fce54002f70 tx=0x7fce54006df0).stop 2021-07-11T03:53:29.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- 172.21.2.104:0/3654039472 shutdown_connections 2021-07-11T03:53:29.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 --2- 172.21.2.104:0/3654039472 >> v2:172.21.2.105:3300/0 conn(0x7fce580b1970 0x7fce5805fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 --2- 172.21.2.104:0/3654039472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce58067180 0x7fce5805f6b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- 172.21.2.104:0/3654039472 >> 172.21.2.104:0/3654039472 conn(0x7fce580b53e0 msgr2=0x7fce580b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- 172.21.2.104:0/3654039472 shutdown_connections 2021-07-11T03:53:29.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- 172.21.2.104:0/3654039472 wait complete. 2021-07-11T03:53:29.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 Processor -- start 2021-07-11T03:53:29.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- start start 2021-07-11T03:53:29.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce58067180 0x7fce580ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce580a76b0 0x7fce580ad8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce580b1970 0x7fce580b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce580ac780 con 0x7fce580b1970 2021-07-11T03:53:29.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce580addc0 con 0x7fce58067180 2021-07-11T03:53:29.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5f5ce700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce580af2f0 con 0x7fce580a76b0 2021-07-11T03:53:29.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5db6b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce580b1970 0x7fce580b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5db6b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce580b1970 0x7fce580b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42194/0 (socket says 172.21.2.104:42194) 2021-07-11T03:53:29.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.728+0000 7fce5db6b700 1 -- 172.21.2.104:0/2619606355 learned_addr learned my addr 172.21.2.104:0/2619606355 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:29.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce5db6b700 1 -- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce580a76b0 msgr2=0x7fce580ad8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce5db6b700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce580a76b0 0x7fce580ad8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce5db6b700 1 -- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce58067180 msgr2=0x7fce580ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce5db6b700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce58067180 0x7fce580ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce5db6b700 1 -- 172.21.2.104:0/2619606355 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5400b180 con 0x7fce580b1970 2021-07-11T03:53:29.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce5db6b700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce580b1970 0x7fce580b42d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fce580ac220 tx=0x7fce500091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce4a7fc700 1 -- 172.21.2.104:0/2619606355 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce5000d4a0 con 0x7fce580b1970 2021-07-11T03:53:29.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce4a7fc700 1 -- 172.21.2.104:0/2619606355 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5000d600 con 0x7fce580b1970 2021-07-11T03:53:29.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce4a7fc700 1 -- 172.21.2.104:0/2619606355 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce5000d960 con 0x7fce580b1970 2021-07-11T03:53:29.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce5f5ce700 1 -- 172.21.2.104:0/2619606355 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce580a99b0 con 0x7fce580b1970 2021-07-11T03:53:29.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce5f5ce700 1 -- 172.21.2.104:0/2619606355 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce580a9d40 con 0x7fce580b1970 2021-07-11T03:53:29.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce5f5ce700 1 -- 172.21.2.104:0/2619606355 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce58057350 con 0x7fce580b1970 2021-07-11T03:53:29.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce4a7fc700 1 -- 172.21.2.104:0/2619606355 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fce50018030 con 0x7fce580b1970 2021-07-11T03:53:29.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce4a7fc700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce380532d0 0x7fce38055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce4a7fc700 1 -- 172.21.2.104:0/2619606355 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fce5006afa0 con 0x7fce580b1970 2021-07-11T03:53:29.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce5d36a700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce380532d0 0x7fce38055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.732+0000 7fce5d36a700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce380532d0 0x7fce38055790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fce5806a0e0 tx=0x7fce4c007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.736+0000 7fce4a7fc700 1 -- 172.21.2.104:0/2619606355 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fce50042280 con 0x7fce580b1970 2021-07-11T03:53:29.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 -- 172.21.2.104:0/2619606355 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce580698e0 con 0x7fce580b1970 2021-07-11T03:53:29.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce4a7fc700 1 -- 172.21.2.104:0/2619606355 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16038 (secure 0 0 0) 0x7fce50023ec0 con 0x7fce580b1970 2021-07-11T03:53:29.819 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:29.820 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":23,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:24.367517+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:29.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 -- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce380532d0 msgr2=0x7fce38055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce380532d0 0x7fce38055790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7fce5806a0e0 tx=0x7fce4c007800).stop 2021-07-11T03:53:29.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 -- 172.21.2.104:0/2619606355 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce580b1970 msgr2=0x7fce580b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce580b1970 0x7fce580b42d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fce580ac220 tx=0x7fce500091a0).stop 2021-07-11T03:53:29.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 -- 172.21.2.104:0/2619606355 shutdown_connections 2021-07-11T03:53:29.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce380532d0 0x7fce38055790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce580b1970 0x7fce580b42d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce580a76b0 0x7fce580ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 --2- 172.21.2.104:0/2619606355 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce58067180 0x7fce580ad3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 -- 172.21.2.104:0/2619606355 >> 172.21.2.104:0/2619606355 conn(0x7fce580b53e0 msgr2=0x7fce58060210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 -- 172.21.2.104:0/2619606355 shutdown_connections 2021-07-11T03:53:29.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.816+0000 7fce5f5ce700 1 -- 172.21.2.104:0/2619606355 wait complete. 2021-07-11T03:53:29.827 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:53:20.980780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:53:21.341375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:53:29.828 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:53:29.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.876+0000 7f98a4e30700 1 Processor -- start 2021-07-11T03:53:29.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.876+0000 7f98a4e30700 1 -- start start 2021-07-11T03:53:29.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.876+0000 7f98a4e30700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a00b12d0 0x7f98a00affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.876+0000 7f98a4e30700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f98a00b22a0 0x7f98a0141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.876+0000 7f98a4e30700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a00a77d0 con 0x7f98a00b1ec0 2021-07-11T03:53:29.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.876+0000 7f98a4e30700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a00b1740 con 0x7f98a00b22a0 2021-07-11T03:53:29.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f98a4e30700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00b0650 con 0x7f98a00b12d0 2021-07-11T03:53:29.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989e59c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f98a00b1ec0 0x7f98a00a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38970/0 (socket says 172.21.2.104:38970) 2021-07-11T03:53:29.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989e59c700 1 -- 172.21.2.104:0/1178807120 learned_addr learned my addr 172.21.2.104:0/1178807120 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:29.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989ed9d700 1 --2- 172.21.2.104:0/1178807120 >> v2:172.21.2.105:3300/0 conn(0x7f98a00b22a0 0x7f98a0141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989dd9b700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a00b12d0 0x7f98a00affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650037053 0 0) 0x7f98a00a77d0 con 0x7f98a00b1ec0 2021-07-11T03:53:29.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9880003260 con 0x7f98a00b1ec0 2021-07-11T03:53:29.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2188350624 0 0) 0x7f9880003260 con 0x7f98a00b1ec0 2021-07-11T03:53:29.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a00b12d0 msgr2=0x7f98a00affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:29.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a00b12d0 0x7f98a00affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 >> v2:172.21.2.105:3300/0 conn(0x7f98a00b22a0 msgr2=0x7f98a0141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:29.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 --2- 172.21.2.104:0/1178807120 >> v2:172.21.2.105:3300/0 conn(0x7f98a00b22a0 0x7f98a0141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00b08f0 con 0x7f98a00b1ec0 2021-07-11T03:53:29.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f9890001660 con 0x7f98a00b1ec0 2021-07-11T03:53:29.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 >> v1:172.21.2.104:6789/0 conn(0x7f98a00b1ec0 legacy=0x7f98a00a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a00b12d0 0x7f9880003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9880003c80 0x7f9880004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98800049f0 0x7f9880004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00b1740 con 0x7f9880003c80 2021-07-11T03:53:29.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00b0650 con 0x7f98800049f0 2021-07-11T03:53:29.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988000b750 con 0x7f98a00b12d0 2021-07-11T03:53:29.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989dd9b700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9880003c80 0x7f9880004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989e59c700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a00b12d0 0x7f9880003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989ed9d700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98800049f0 0x7f9880004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989dd9b700 1 -- 172.21.2.104:0/1178807120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a00b12d0 msgr2=0x7f9880003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:29.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989dd9b700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a00b12d0 0x7f9880003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989dd9b700 1 -- 172.21.2.104:0/1178807120 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98800049f0 msgr2=0x7f9880004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:29.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989dd9b700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98800049f0 0x7f9880004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989dd9b700 1 -- 172.21.2.104:0/1178807120 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a00b08f0 con 0x7f9880003c80 2021-07-11T03:53:29.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989dd9b700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9880003c80 0x7f9880004500 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f988800c000 tx=0x7f9888006df0).ready entity=mon.0 client_cookie=6fa98f428b542b81 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f988800e830 con 0x7f9880003c80 2021-07-11T03:53:29.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988800e990 con 0x7f9880003c80 2021-07-11T03:53:29.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989d59a700 1 -- 172.21.2.104:0/1178807120 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f988800ecf0 con 0x7f9880003c80 2021-07-11T03:53:29.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f98a4e30700 1 -- 172.21.2.104:0/1178807120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9880003c80 msgr2=0x7f9880004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f98a4e30700 1 --2- 172.21.2.104:0/1178807120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9880003c80 0x7f9880004500 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f988800c000 tx=0x7f9888006df0).stop 2021-07-11T03:53:29.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f989ed9d700 1 -- 172.21.2.104:0/1178807120 reap_dead start 2021-07-11T03:53:29.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f98a4e30700 1 -- 172.21.2.104:0/1178807120 shutdown_connections 2021-07-11T03:53:29.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f98a4e30700 1 -- 172.21.2.104:0/1178807120 >> 172.21.2.104:0/1178807120 conn(0x7f98a005f6b0 msgr2=0x7f98a005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f98a4e30700 1 -- 172.21.2.104:0/1178807120 shutdown_connections 2021-07-11T03:53:29.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f98a4e30700 1 -- 172.21.2.104:0/1178807120 wait complete. 2021-07-11T03:53:29.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f98a4e30700 1 Processor -- start 2021-07-11T03:53:29.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.880+0000 7f98a4e30700 1 -- start start 2021-07-11T03:53:29.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98a4e30700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98a00ab870 0x7f98a00b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98a4e30700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98a00b4b00 0x7f98a015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98a4e30700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a015ab40 0x7f98a015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98a4e30700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00b56c0 con 0x7f98a00ab870 2021-07-11T03:53:29.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98a4e30700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00a9d30 con 0x7f98a00b4b00 2021-07-11T03:53:29.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98a4e30700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00abc50 con 0x7f98a015ab40 2021-07-11T03:53:29.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98a00ab870 0x7f98a00b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98a00ab870 0x7f98a00b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42210/0 (socket says 172.21.2.104:42210) 2021-07-11T03:53:29.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989e59c700 1 -- 172.21.2.104:0/3084909829 learned_addr learned my addr 172.21.2.104:0/3084909829 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:29.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989dd9b700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98a00b4b00 0x7f98a015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989e59c700 1 -- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a015ab40 msgr2=0x7f98a015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989e59c700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a015ab40 0x7f98a015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989e59c700 1 -- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98a00b4b00 msgr2=0x7f98a015a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989e59c700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98a00b4b00 0x7f98a015a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989e59c700 1 -- 172.21.2.104:0/3084909829 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a006aa50 con 0x7f98a00ab870 2021-07-11T03:53:29.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989e59c700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98a00ab870 0x7f98a00b4610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f98a00b3bf0 tx=0x7f98900014c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98977fe700 1 -- 172.21.2.104:0/3084909829 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f989000c770 con 0x7f98a00ab870 2021-07-11T03:53:29.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98977fe700 1 -- 172.21.2.104:0/3084909829 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f989000c8d0 con 0x7f98a00ab870 2021-07-11T03:53:29.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98a4e30700 1 -- 172.21.2.104:0/3084909829 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a00ac4f0 con 0x7f98a00ab870 2021-07-11T03:53:29.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98977fe700 1 -- 172.21.2.104:0/3084909829 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f989000cc30 con 0x7f98a00ab870 2021-07-11T03:53:29.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98a4e30700 1 -- 172.21.2.104:0/3084909829 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a015b4a0 con 0x7f98a00ab870 2021-07-11T03:53:29.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98a4e30700 1 -- 172.21.2.104:0/3084909829 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a014adf0 con 0x7f98a00ab870 2021-07-11T03:53:29.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98977fe700 1 -- 172.21.2.104:0/3084909829 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9890014430 con 0x7f98a00ab870 2021-07-11T03:53:29.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98977fe700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f98840532d0 0x7f9884055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:29.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f98977fe700 1 -- 172.21.2.104:0/3084909829 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f989006b550 con 0x7f98a00ab870 2021-07-11T03:53:29.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989dd9b700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f98840532d0 0x7f9884055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:29.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.884+0000 7f989dd9b700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f98840532d0 0x7f9884055790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f988801a000 tx=0x7f9888019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:29.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.888+0000 7f98977fe700 1 -- 172.21.2.104:0/3084909829 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f98a014adf0 con 0x7f98a00ab870 2021-07-11T03:53:29.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 -- 172.21.2.104:0/3084909829 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f98a014adf0 con 0x7f98a00ab870 2021-07-11T03:53:29.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98977fe700 1 -- 172.21.2.104:0/3084909829 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f98a014adf0 con 0x7f98a00ab870 2021-07-11T03:53:29.971 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:53:29.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 -- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f98840532d0 msgr2=0x7f9884055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f98840532d0 0x7f9884055790 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f988801a000 tx=0x7f9888019000).stop 2021-07-11T03:53:29.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 -- 172.21.2.104:0/3084909829 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98a00ab870 msgr2=0x7f98a00b4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:29.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98a00ab870 0x7f98a00b4610 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f98a00b3bf0 tx=0x7f98900014c0).stop 2021-07-11T03:53:29.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 -- 172.21.2.104:0/3084909829 shutdown_connections 2021-07-11T03:53:29.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98a015ab40 0x7f98a015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:29.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98a00b4b00 0x7f98a015a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f98840532d0 0x7f9884055790 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 --2- 172.21.2.104:0/3084909829 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98a00ab870 0x7f98a00b4610 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:29.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 -- 172.21.2.104:0/3084909829 >> 172.21.2.104:0/3084909829 conn(0x7f98a005f6b0 msgr2=0x7f98a0141020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:29.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 -- 172.21.2.104:0/3084909829 shutdown_connections 2021-07-11T03:53:29.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:29.968+0000 7f98a4e30700 1 -- 172.21.2.104:0/3084909829 wait complete. 2021-07-11T03:53:29.978 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:53:30.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61dd9a3700 1 Processor -- start 2021-07-11T03:53:30.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61dd9a3700 1 -- start start 2021-07-11T03:53:30.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61dd9a3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d80b1ec0 0x7f61d80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61dd9a3700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d80a77d0 con 0x7f61d80b22a0 2021-07-11T03:53:30.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61dd9a3700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d80b03f0 con 0x7f61d80affb0 2021-07-11T03:53:30.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61dd9a3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80b0650 con 0x7f61d80b1ec0 2021-07-11T03:53:30.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d77fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f61d80b22a0 0x7f61d8141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:38986/0 (socket says 172.21.2.104:38986) 2021-07-11T03:53:30.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d77fe700 1 -- 172.21.2.104:0/2750497110 learned_addr learned my addr 172.21.2.104:0/2750497110 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:30.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188121730 0 0) 0x7f61d80a77d0 con 0x7f61d80b22a0 2021-07-11T03:53:30.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b8003260 con 0x7f61d80b22a0 2021-07-11T03:53:30.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d6ffd700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d80b1ec0 0x7f61d80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3994691418 0 0) 0x7f61b8003260 con 0x7f61d80b22a0 2021-07-11T03:53:30.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d80b1ec0 msgr2=0x7f61d80b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:30.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d80b1ec0 0x7f61d80b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 >> v1:172.21.2.105:6789/0 conn(0x7f61d80affb0 legacy=0x7f61d80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d80b08f0 con 0x7f61d80b22a0 2021-07-11T03:53:30.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f61d00019e0 con 0x7f61d80b22a0 2021-07-11T03:53:30.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 >> v1:172.21.2.104:6789/0 conn(0x7f61d80b22a0 legacy=0x7f61d8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61b8003780 0x7f61b8003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61b8004500 0x7f61b8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d80b1ec0 0x7f61b8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80b0650 con 0x7f61b8003780 2021-07-11T03:53:30.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b800b780 con 0x7f61b8004500 2021-07-11T03:53:30.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b800b940 con 0x7f61d80b1ec0 2021-07-11T03:53:30.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d6ffd700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61b8003780 0x7f61b8003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d67fc700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61b8004500 0x7f61b8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d6ffd700 1 -- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d80b1ec0 msgr2=0x7f61b8004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:30.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d77fe700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d80b1ec0 0x7f61b8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d6ffd700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d80b1ec0 0x7f61b8004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d6ffd700 1 -- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61b8004500 msgr2=0x7f61b8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:30.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d6ffd700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61b8004500 0x7f61b8004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d6ffd700 1 -- 172.21.2.104:0/2750497110 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d80b08f0 con 0x7f61b8003780 2021-07-11T03:53:30.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d77fe700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d80b1ec0 0x7f61b8004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:30.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.028+0000 7f61d6ffd700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61b8003780 0x7f61b8003ba0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f61c800c000 tx=0x7f61c8006df0).ready entity=mon.0 client_cookie=6a2ba29584d134dd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c800e830 con 0x7f61b8003780 2021-07-11T03:53:30.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c800e990 con 0x7f61b8003780 2021-07-11T03:53:30.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d5ffb700 1 -- 172.21.2.104:0/2750497110 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61c800ecf0 con 0x7f61b8003780 2021-07-11T03:53:30.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- 172.21.2.104:0/2750497110 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61b8003780 msgr2=0x7f61b8003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 --2- 172.21.2.104:0/2750497110 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61b8003780 0x7f61b8003ba0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f61c800c000 tx=0x7f61c8006df0).stop 2021-07-11T03:53:30.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d77fe700 1 -- 172.21.2.104:0/2750497110 reap_dead start 2021-07-11T03:53:30.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- 172.21.2.104:0/2750497110 shutdown_connections 2021-07-11T03:53:30.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- 172.21.2.104:0/2750497110 >> 172.21.2.104:0/2750497110 conn(0x7f61d805f6b0 msgr2=0x7f61d805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:30.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- 172.21.2.104:0/2750497110 shutdown_connections 2021-07-11T03:53:30.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- 172.21.2.104:0/2750497110 wait complete. 2021-07-11T03:53:30.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 Processor -- start 2021-07-11T03:53:30.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- start start 2021-07-11T03:53:30.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d813cf00 0x7f61d80aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61d80ab920 0x7f61d8138770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61d8138c60 0x7f61d815a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d813d2e0 con 0x7f61d8138c60 2021-07-11T03:53:30.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80a9420 con 0x7f61d80ab920 2021-07-11T03:53:30.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d80aa170 con 0x7f61d813cf00 2021-07-11T03:53:30.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d6ffd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d813cf00 0x7f61d80aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d67fc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61d80ab920 0x7f61d8138770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61d8138c60 0x7f61d815a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d67fc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61d80ab920 0x7f61d8138770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:49792/0 (socket says 172.21.2.104:49792) 2021-07-11T03:53:30.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d67fc700 1 -- 172.21.2.104:0/1522180425 learned_addr learned my addr 172.21.2.104:0/1522180425 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:30.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d77fe700 1 -- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d813cf00 msgr2=0x7f61d80aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d77fe700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d813cf00 0x7f61d80aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d77fe700 1 -- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61d80ab920 msgr2=0x7f61d8138770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d77fe700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61d80ab920 0x7f61d8138770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d77fe700 1 -- 172.21.2.104:0/1522180425 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d80aa410 con 0x7f61d8138c60 2021-07-11T03:53:30.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d67fc700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61d80ab920 0x7f61d8138770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:53:30.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d77fe700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61d8138c60 0x7f61d815a6c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f61d0001bc0 tx=0x7f61d0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61cffff700 1 -- 172.21.2.104:0/1522180425 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61d0004c40 con 0x7f61d8138c60 2021-07-11T03:53:30.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61cffff700 1 -- 172.21.2.104:0/1522180425 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d0004da0 con 0x7f61d8138c60 2021-07-11T03:53:30.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61cffff700 1 -- 172.21.2.104:0/1522180425 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61d00072c0 con 0x7f61d8138c60 2021-07-11T03:53:30.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- 172.21.2.104:0/1522180425 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d80a9680 con 0x7f61d8138c60 2021-07-11T03:53:30.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- 172.21.2.104:0/1522180425 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d81390e0 con 0x7f61d8138c60 2021-07-11T03:53:30.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61dd9a3700 1 -- 172.21.2.104:0/1522180425 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d8057350 con 0x7f61d8138c60 2021-07-11T03:53:30.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61cffff700 1 -- 172.21.2.104:0/1522180425 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f61d001c070 con 0x7f61d8138c60 2021-07-11T03:53:30.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61cffff700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61bc0531f0 0x7f61bc0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61cffff700 1 -- 172.21.2.104:0/1522180425 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f61d006aa10 con 0x7f61d8138c60 2021-07-11T03:53:30.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.032+0000 7f61d6ffd700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61bc0531f0 0x7f61bc0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.036+0000 7f61d6ffd700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61bc0531f0 0x7f61bc0556b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f61c801d000 tx=0x7f61c8019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.036+0000 7f61cffff700 1 -- 172.21.2.104:0/1522180425 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f61d0041d50 con 0x7f61d8138c60 2021-07-11T03:53:30.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 -- 172.21.2.104:0/1522180425 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f61d814bb60 con 0x7f61d8138c60 2021-07-11T03:53:30.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61cffff700 1 -- 172.21.2.104:0/1522180425 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f61d0045370 con 0x7f61d8138c60 2021-07-11T03:53:30.124 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:30.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 -- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61bc0531f0 msgr2=0x7f61bc0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61bc0531f0 0x7f61bc0556b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f61c801d000 tx=0x7f61c8019000).stop 2021-07-11T03:53:30.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 -- 172.21.2.104:0/1522180425 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61d8138c60 msgr2=0x7f61d815a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61d8138c60 0x7f61d815a6c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f61d0001bc0 tx=0x7f61d0009040).stop 2021-07-11T03:53:30.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 -- 172.21.2.104:0/1522180425 shutdown_connections 2021-07-11T03:53:30.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61bc0531f0 0x7f61bc0556b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61d8138c60 0x7f61d815a6c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61d80ab920 0x7f61d8138770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 --2- 172.21.2.104:0/1522180425 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61d813cf00 0x7f61d80aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 -- 172.21.2.104:0/1522180425 >> 172.21.2.104:0/1522180425 conn(0x7f61d805f6b0 msgr2=0x7f61d80af630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:30.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 -- 172.21.2.104:0/1522180425 shutdown_connections 2021-07-11T03:53:30.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.120+0000 7f61dd9a3700 1 -- 172.21.2.104:0/1522180425 wait complete. 2021-07-11T03:53:30.132 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:53:30.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.180+0000 7fba59ee4700 1 Processor -- start 2021-07-11T03:53:30.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.180+0000 7fba59ee4700 1 -- start start 2021-07-11T03:53:30.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fba540b31b0 0x7fba5413e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fba54145090 0x7fba5413ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba540ad8f0 con 0x7fba540b3590 2021-07-11T03:53:30.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba540aff00 con 0x7fba540b31b0 2021-07-11T03:53:30.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba54145560 con 0x7fba54145090 2021-07-11T03:53:30.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba537fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fba540b31b0 0x7fba5413e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba537fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fba540b31b0 0x7fba5413e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42238/0 (socket says 172.21.2.104:42238) 2021-07-11T03:53:30.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba537fe700 1 -- 172.21.2.104:0/556140164 learned_addr learned my addr 172.21.2.104:0/556140164 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:30.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba537fe700 1 -- 172.21.2.104:0/556140164 >> v2:172.21.2.105:3300/0 conn(0x7fba54145090 msgr2=0x7fba5413ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:30.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba537fe700 1 --2- 172.21.2.104:0/556140164 >> v2:172.21.2.105:3300/0 conn(0x7fba54145090 0x7fba5413ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba537fe700 1 -- 172.21.2.104:0/556140164 >> v1:172.21.2.105:6789/0 conn(0x7fba540b3590 legacy=0x7fba54141400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:30.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba537fe700 1 -- 172.21.2.104:0/556140164 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540b0160 con 0x7fba540b31b0 2021-07-11T03:53:30.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba537fe700 1 --2- 172.21.2.104:0/556140164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba540b31b0 0x7fba5413e560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fba54141190 tx=0x7fba44006eb0).ready entity=mon.0 client_cookie=6d788826d28508cf server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba527fc700 1 -- 172.21.2.104:0/556140164 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba4400d5c0 con 0x7fba540b31b0 2021-07-11T03:53:30.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba527fc700 1 -- 172.21.2.104:0/556140164 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba44000e90 con 0x7fba540b31b0 2021-07-11T03:53:30.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba527fc700 1 -- 172.21.2.104:0/556140164 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba4400d950 con 0x7fba540b31b0 2021-07-11T03:53:30.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- 172.21.2.104:0/556140164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba540b31b0 msgr2=0x7fba5413e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 --2- 172.21.2.104:0/556140164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba540b31b0 0x7fba5413e560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fba54141190 tx=0x7fba44006eb0).stop 2021-07-11T03:53:30.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- 172.21.2.104:0/556140164 shutdown_connections 2021-07-11T03:53:30.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 --2- 172.21.2.104:0/556140164 >> v2:172.21.2.105:3300/0 conn(0x7fba54145090 0x7fba5413ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 --2- 172.21.2.104:0/556140164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba540b31b0 0x7fba5413e560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- 172.21.2.104:0/556140164 >> 172.21.2.104:0/556140164 conn(0x7fba540607c0 msgr2=0x7fba54060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:30.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- 172.21.2.104:0/556140164 shutdown_connections 2021-07-11T03:53:30.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- 172.21.2.104:0/556140164 wait complete. 2021-07-11T03:53:30.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 Processor -- start 2021-07-11T03:53:30.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- start start 2021-07-11T03:53:30.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba540b31b0 0x7fba541400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba540b3590 0x7fba54159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba54145090 0x7fba5415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba540677a0 con 0x7fba54145090 2021-07-11T03:53:30.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba54144990 con 0x7fba540b3590 2021-07-11T03:53:30.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba54066b20 con 0x7fba540b31b0 2021-07-11T03:53:30.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba53fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba54145090 0x7fba5415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba53fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba54145090 0x7fba5415a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42246/0 (socket says 172.21.2.104:42246) 2021-07-11T03:53:30.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba53fff700 1 -- 172.21.2.104:0/9386049 learned_addr learned my addr 172.21.2.104:0/9386049 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:30.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba53fff700 1 -- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba540b31b0 msgr2=0x7fba541400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba53fff700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba540b31b0 0x7fba541400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba53fff700 1 -- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba540b3590 msgr2=0x7fba54159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba53fff700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba540b3590 0x7fba54159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba53fff700 1 -- 172.21.2.104:0/9386049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4400b1e0 con 0x7fba54145090 2021-07-11T03:53:30.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba53fff700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba54145090 0x7fba5415a480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fba4c0032f0 tx=0x7fba4c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba50ff9700 1 -- 172.21.2.104:0/9386049 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba4c00f510 con 0x7fba54145090 2021-07-11T03:53:30.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba50ff9700 1 -- 172.21.2.104:0/9386049 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c00f670 con 0x7fba54145090 2021-07-11T03:53:30.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- 172.21.2.104:0/9386049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba54066dc0 con 0x7fba54145090 2021-07-11T03:53:30.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba50ff9700 1 -- 172.21.2.104:0/9386049 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba4c00f7e0 con 0x7fba54145090 2021-07-11T03:53:30.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.184+0000 7fba59ee4700 1 -- 172.21.2.104:0/9386049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba54067300 con 0x7fba54145090 2021-07-11T03:53:30.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.188+0000 7fba59ee4700 1 -- 172.21.2.104:0/9386049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba540674c0 con 0x7fba54145090 2021-07-11T03:53:30.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.188+0000 7fba50ff9700 1 -- 172.21.2.104:0/9386049 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fba4c00f070 con 0x7fba54145090 2021-07-11T03:53:30.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.188+0000 7fba50ff9700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba38053210 0x7fba380556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.188+0000 7fba50ff9700 1 -- 172.21.2.104:0/9386049 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fba4c06b0a0 con 0x7fba54145090 2021-07-11T03:53:30.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.188+0000 7fba537fe700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba38053210 0x7fba380556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.188+0000 7fba537fe700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba38053210 0x7fba380556d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fba54141190 tx=0x7fba44015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.188+0000 7fba50ff9700 1 -- 172.21.2.104:0/9386049 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fba4c0423e0 con 0x7fba54145090 2021-07-11T03:53:30.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 -- 172.21.2.104:0/9386049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba54057350 con 0x7fba54145090 2021-07-11T03:53:30.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba50ff9700 1 -- 172.21.2.104:0/9386049 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7fba4c045a00 con 0x7fba54145090 2021-07-11T03:53:30.277 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:30.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 -- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba38053210 msgr2=0x7fba380556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba38053210 0x7fba380556d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fba54141190 tx=0x7fba44015000).stop 2021-07-11T03:53:30.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 -- 172.21.2.104:0/9386049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba54145090 msgr2=0x7fba5415a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba54145090 0x7fba5415a480 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fba4c0032f0 tx=0x7fba4c0091a0).stop 2021-07-11T03:53:30.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 -- 172.21.2.104:0/9386049 shutdown_connections 2021-07-11T03:53:30.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba38053210 0x7fba380556d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba54145090 0x7fba5415a480 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba540b3590 0x7fba54159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 --2- 172.21.2.104:0/9386049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba540b31b0 0x7fba541400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 -- 172.21.2.104:0/9386049 >> 172.21.2.104:0/9386049 conn(0x7fba540607c0 msgr2=0x7fba540b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:30.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.272+0000 7fba59ee4700 1 -- 172.21.2.104:0/9386049 shutdown_connections 2021-07-11T03:53:30.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.276+0000 7fba59ee4700 1 -- 172.21.2.104:0/9386049 wait complete. 2021-07-11T03:53:30.284 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:53:30.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 Processor -- start 2021-07-11T03:53:30.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 -- start start 2021-07-11T03:53:30.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fbba4060100 0x7fbba4064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbba4144440 0x7fbba4065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbba40b38d0 0x7fbba4144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba40656f0 con 0x7fbba4144440 2021-07-11T03:53:30.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba4065c80 con 0x7fbba4060100 2021-07-11T03:53:30.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40604e0 con 0x7fbba40b38d0 2021-07-11T03:53:30.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba1d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbba4144440 0x7fbba4065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba1d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbba4144440 0x7fbba4065200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42256/0 (socket says 172.21.2.104:42256) 2021-07-11T03:53:30.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba1d9b700 1 -- 172.21.2.104:0/1805577353 learned_addr learned my addr 172.21.2.104:0/1805577353 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:30.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba1d9b700 1 -- 172.21.2.104:0/1805577353 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbba40b38d0 msgr2=0x7fbba4144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:30.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba1d9b700 1 --2- 172.21.2.104:0/1805577353 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbba40b38d0 0x7fbba4144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba1d9b700 1 -- 172.21.2.104:0/1805577353 >> v2:172.21.2.105:3300/0 conn(0x7fbba4060100 msgr2=0x7fbba4064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:30.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba1d9b700 1 --2- 172.21.2.104:0/1805577353 >> v2:172.21.2.105:3300/0 conn(0x7fbba4060100 0x7fbba4064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba1d9b700 1 -- 172.21.2.104:0/1805577353 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40b23b0 con 0x7fbba4144440 2021-07-11T03:53:30.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba1d9b700 1 --2- 172.21.2.104:0/1805577353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbba4144440 0x7fbba4065200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fbba40b20d0 tx=0x7fbb8c006eb0).ready entity=mon.0 client_cookie=c5694d80731c83a1 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba159a700 1 -- 172.21.2.104:0/1805577353 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8c00d570 con 0x7fbba4144440 2021-07-11T03:53:30.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba159a700 1 -- 172.21.2.104:0/1805577353 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c000e90 con 0x7fbba4144440 2021-07-11T03:53:30.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba159a700 1 -- 172.21.2.104:0/1805577353 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb8c00d8c0 con 0x7fbba4144440 2021-07-11T03:53:30.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 -- 172.21.2.104:0/1805577353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbba4144440 msgr2=0x7fbba4065200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 --2- 172.21.2.104:0/1805577353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbba4144440 0x7fbba4065200 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fbba40b20d0 tx=0x7fbb8c006eb0).stop 2021-07-11T03:53:30.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 -- 172.21.2.104:0/1805577353 shutdown_connections 2021-07-11T03:53:30.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 --2- 172.21.2.104:0/1805577353 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbba40b38d0 0x7fbba4144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 --2- 172.21.2.104:0/1805577353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbba4144440 0x7fbba4065200 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 --2- 172.21.2.104:0/1805577353 >> v2:172.21.2.105:3300/0 conn(0x7fbba4060100 0x7fbba4064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 -- 172.21.2.104:0/1805577353 >> 172.21.2.104:0/1805577353 conn(0x7fbba405cf60 msgr2=0x7fbba405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:30.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 -- 172.21.2.104:0/1805577353 shutdown_connections 2021-07-11T03:53:30.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.360+0000 7fbba89c7700 1 -- 172.21.2.104:0/1805577353 wait complete. 2021-07-11T03:53:30.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba89c7700 1 Processor -- start 2021-07-11T03:53:30.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba89c7700 1 -- start start 2021-07-11T03:53:30.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba89c7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbba4060100 0x7fbba4061af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba89c7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbba40b38d0 0x7fbba40b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba89c7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbba4145f50 0x7fbba40aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba89c7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40b0440 con 0x7fbba4060100 2021-07-11T03:53:30.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba89c7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba4062180 con 0x7fbba40b38d0 2021-07-11T03:53:30.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba89c7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba4060d10 con 0x7fbba4145f50 2021-07-11T03:53:30.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbba4060100 0x7fbba4061af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbba4060100 0x7fbba4061af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42262/0 (socket says 172.21.2.104:42262) 2021-07-11T03:53:30.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba259c700 1 -- 172.21.2.104:0/2684272322 learned_addr learned my addr 172.21.2.104:0/2684272322 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:30.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba259c700 1 -- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbba4145f50 msgr2=0x7fbba40aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba259c700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbba4145f50 0x7fbba40aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba259c700 1 -- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbba40b38d0 msgr2=0x7fbba40b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba259c700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbba40b38d0 0x7fbba40b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba259c700 1 -- 172.21.2.104:0/2684272322 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c00b1e0 con 0x7fbba4060100 2021-07-11T03:53:30.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba259c700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbba4060100 0x7fbba4061af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fbb94002f70 tx=0x7fbb94006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbb9b7fe700 1 -- 172.21.2.104:0/2684272322 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb94013460 con 0x7fbba4060100 2021-07-11T03:53:30.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbb9b7fe700 1 -- 172.21.2.104:0/2684272322 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb940135c0 con 0x7fbba4060100 2021-07-11T03:53:30.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbb9b7fe700 1 -- 172.21.2.104:0/2684272322 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbb94013920 con 0x7fbba4060100 2021-07-11T03:53:30.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba89c7700 1 -- 172.21.2.104:0/2684272322 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba40623b0 con 0x7fbba4060100 2021-07-11T03:53:30.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba89c7700 1 -- 172.21.2.104:0/2684272322 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba4061650 con 0x7fbba4060100 2021-07-11T03:53:30.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba89c7700 1 -- 172.21.2.104:0/2684272322 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba4061810 con 0x7fbba4060100 2021-07-11T03:53:30.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbb9b7fe700 1 -- 172.21.2.104:0/2684272322 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbb9401a060 con 0x7fbba4060100 2021-07-11T03:53:30.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbb9b7fe700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbb880531a0 0x7fbb88055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbb9b7fe700 1 -- 172.21.2.104:0/2684272322 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbb9406b060 con 0x7fbba4060100 2021-07-11T03:53:30.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.364+0000 7fbba1d9b700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbb880531a0 0x7fbb88055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.368+0000 7fbba1d9b700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbb880531a0 0x7fbb88055660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fbba40b20d0 tx=0x7fbb8c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.368+0000 7fbb9b7fe700 1 -- 172.21.2.104:0/2684272322 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbb94042290 con 0x7fbba4060100 2021-07-11T03:53:30.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 -- 172.21.2.104:0/2684272322 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbba4062540 con 0x7fbba4060100 2021-07-11T03:53:30.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbb9b7fe700 1 -- 172.21.2.104:0/2684272322 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbb94018030 con 0x7fbba4060100 2021-07-11T03:53:30.455 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:30.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 -- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbb880531a0 msgr2=0x7fbb88055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbb880531a0 0x7fbb88055660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fbba40b20d0 tx=0x7fbb8c015000).stop 2021-07-11T03:53:30.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 -- 172.21.2.104:0/2684272322 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbba4060100 msgr2=0x7fbba4061af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbba4060100 0x7fbba4061af0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fbb94002f70 tx=0x7fbb94006ec0).stop 2021-07-11T03:53:30.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 -- 172.21.2.104:0/2684272322 shutdown_connections 2021-07-11T03:53:30.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbba4145f50 0x7fbba40aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbba40b38d0 0x7fbba40b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbb880531a0 0x7fbb88055660 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 --2- 172.21.2.104:0/2684272322 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbba4060100 0x7fbba4061af0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 -- 172.21.2.104:0/2684272322 >> 172.21.2.104:0/2684272322 conn(0x7fbba405cf60 msgr2=0x7fbba40aa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:30.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 -- 172.21.2.104:0/2684272322 shutdown_connections 2021-07-11T03:53:30.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.452+0000 7fbba89c7700 1 -- 172.21.2.104:0/2684272322 wait complete. 2021-07-11T03:53:30.463 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:53:30.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.508+0000 7f69134d5700 1 Processor -- start 2021-07-11T03:53:30.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.508+0000 7f69134d5700 1 -- start start 2021-07-11T03:53:30.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.508+0000 7f69134d5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f690c0b1170 0x7f690c141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.508+0000 7f69134d5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f690c141920 0x7f690c141d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.508+0000 7f69134d5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c0b31d0 con 0x7f690c0b7810 2021-07-11T03:53:30.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.508+0000 7f69134d5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c0b7bf0 con 0x7f690c141920 2021-07-11T03:53:30.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.508+0000 7f69134d5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0675c0 con 0x7f690c0b1170 2021-07-11T03:53:30.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911a72700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f690c141920 0x7f690c141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911a72700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f690c141920 0x7f690c141d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42270/0 (socket says 172.21.2.104:42270) 2021-07-11T03:53:30.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911a72700 1 -- 172.21.2.104:0/570276754 learned_addr learned my addr 172.21.2.104:0/570276754 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:30.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911a72700 1 -- 172.21.2.104:0/570276754 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f690c0b1170 msgr2=0x7f690c141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:30.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911a72700 1 --2- 172.21.2.104:0/570276754 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f690c0b1170 0x7f690c141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911a72700 1 -- 172.21.2.104:0/570276754 >> v1:172.21.2.105:6789/0 conn(0x7f690c0b7810 legacy=0x7f690c1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:30.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911a72700 1 -- 172.21.2.104:0/570276754 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c142270 con 0x7f690c141920 2021-07-11T03:53:30.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911a72700 1 --2- 172.21.2.104:0/570276754 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f690c141920 0x7f690c141d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f690c060740 tx=0x7f6904009290).ready entity=mon.0 client_cookie=f92b7e73484b799c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f68fffff700 1 -- 172.21.2.104:0/570276754 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f690400b860 con 0x7f690c141920 2021-07-11T03:53:30.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f68fffff700 1 -- 172.21.2.104:0/570276754 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690400b9c0 con 0x7f690c141920 2021-07-11T03:53:30.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f68fffff700 1 -- 172.21.2.104:0/570276754 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f690400bb70 con 0x7f690c141920 2021-07-11T03:53:30.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 -- 172.21.2.104:0/570276754 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f690c141920 msgr2=0x7f690c141d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 --2- 172.21.2.104:0/570276754 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f690c141920 0x7f690c141d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f690c060740 tx=0x7f6904009290).stop 2021-07-11T03:53:30.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 -- 172.21.2.104:0/570276754 shutdown_connections 2021-07-11T03:53:30.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 --2- 172.21.2.104:0/570276754 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f690c141920 0x7f690c141d80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 --2- 172.21.2.104:0/570276754 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f690c0b1170 0x7f690c141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:30.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 -- 172.21.2.104:0/570276754 >> 172.21.2.104:0/570276754 conn(0x7f690c05f6b0 msgr2=0x7f690c05fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:30.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 -- 172.21.2.104:0/570276754 shutdown_connections 2021-07-11T03:53:30.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 -- 172.21.2.104:0/570276754 wait complete. 2021-07-11T03:53:30.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 Processor -- start 2021-07-11T03:53:30.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 -- start start 2021-07-11T03:53:30.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f690c0b1170 0x7f690c060ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f690c0b7810 0x7f690c061560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f690c141920 0x7f690c149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0623d0 con 0x7f690c0b1170 2021-07-11T03:53:30.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c060780 con 0x7f690c141920 2021-07-11T03:53:30.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c060970 con 0x7f690c0b7810 2021-07-11T03:53:30.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911271700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f690c0b1170 0x7f690c060ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911271700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f690c0b1170 0x7f690c060ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42276/0 (socket says 172.21.2.104:42276) 2021-07-11T03:53:30.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911271700 1 -- 172.21.2.104:0/241437024 learned_addr learned my addr 172.21.2.104:0/241437024 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:30.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911a72700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f690c141920 0x7f690c149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6910a70700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f690c0b7810 0x7f690c061560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911271700 1 -- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f690c0b7810 msgr2=0x7f690c061560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911271700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f690c0b7810 0x7f690c061560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911271700 1 -- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f690c141920 msgr2=0x7f690c149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911271700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f690c141920 0x7f690c149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911271700 1 -- 172.21.2.104:0/241437024 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6904009530 con 0x7f690c0b1170 2021-07-11T03:53:30.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f6911271700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f690c0b1170 0x7f690c060ee0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f6900000f70 tx=0x7f6900007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f68fe7fc700 1 -- 172.21.2.104:0/241437024 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f690000ce80 con 0x7f690c0b1170 2021-07-11T03:53:30.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f68fe7fc700 1 -- 172.21.2.104:0/241437024 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900013450 con 0x7f690c0b1170 2021-07-11T03:53:30.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 -- 172.21.2.104:0/241437024 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c1499a0 con 0x7f690c0b1170 2021-07-11T03:53:30.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f68fe7fc700 1 -- 172.21.2.104:0/241437024 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69000137b0 con 0x7f690c0b1170 2021-07-11T03:53:30.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.512+0000 7f69134d5700 1 -- 172.21.2.104:0/241437024 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c061a50 con 0x7f690c0b1170 2021-07-11T03:53:30.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.516+0000 7f69134d5700 1 -- 172.21.2.104:0/241437024 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c0b3b40 con 0x7f690c0b1170 2021-07-11T03:53:30.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.516+0000 7f68fe7fc700 1 -- 172.21.2.104:0/241437024 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f690001a060 con 0x7f690c0b1170 2021-07-11T03:53:30.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.516+0000 7f68fe7fc700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f68ec0531f0 0x7f68ec0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:30.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.516+0000 7f68fe7fc700 1 -- 172.21.2.104:0/241437024 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f690006ae10 con 0x7f690c0b1170 2021-07-11T03:53:30.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.516+0000 7f6910a70700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f68ec0531f0 0x7f68ec0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:30.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.516+0000 7f6910a70700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f68ec0531f0 0x7f68ec0556b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f690800a000 tx=0x7f6908008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:30.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.516+0000 7f68fe7fc700 1 -- 172.21.2.104:0/241437024 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f690c0b3b40 con 0x7f690c0b1170 2021-07-11T03:53:30.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 -- 172.21.2.104:0/241437024 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f690c0b3b40 con 0x7f690c0b1170 2021-07-11T03:53:30.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f68fe7fc700 1 -- 172.21.2.104:0/241437024 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2483 (secure 0 0 0) 0x7f690c0b3b40 con 0x7f690c0b1170 2021-07-11T03:53:30.629 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:30.629 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3943971217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3943971217},{"type":"v1","addr":"172.21.2.105:6835","nonce":3943971217}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/668148871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":668148871},{"type":"v1","addr":"172.21.2.104:6833","nonce":668148871}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":6}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:22.350923+0000","modified":"2021-07-11T03:53:23.355891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4224":{"gid":4224,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/1236541958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":1236541958},{"type":"v1","addr":"172.21.2.104:6835","nonce":1236541958}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-07-11T03:53:30.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 -- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f68ec0531f0 msgr2=0x7f68ec0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f68ec0531f0 0x7f68ec0556b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f690800a000 tx=0x7f6908008040).stop 2021-07-11T03:53:30.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 -- 172.21.2.104:0/241437024 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f690c0b1170 msgr2=0x7f690c060ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:30.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f690c0b1170 0x7f690c060ee0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f6900000f70 tx=0x7f6900007c30).stop 2021-07-11T03:53:30.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 -- 172.21.2.104:0/241437024 shutdown_connections 2021-07-11T03:53:30.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f690c141920 0x7f690c149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f690c0b7810 0x7f690c061560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f68ec0531f0 0x7f68ec0556b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 --2- 172.21.2.104:0/241437024 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f690c0b1170 0x7f690c060ee0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:30.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 -- 172.21.2.104:0/241437024 >> 172.21.2.104:0/241437024 conn(0x7f690c05f6b0 msgr2=0x7f690c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:30.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 -- 172.21.2.104:0/241437024 shutdown_connections 2021-07-11T03:53:30.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:30.624+0000 7f69134d5700 1 -- 172.21.2.104:0/241437024 wait complete. 2021-07-11T03:53:30.633 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 6 2021-07-11T03:53:30.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:53:22.350923+0000', 'modified': '2021-07-11T03:53:23.355891+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4224}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4224': {'gid': 4224, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.104:6835/1236541958', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 1236541958}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 1236541958}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:53:30.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:53:30.939 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:53:31.004 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":4224,"want_state":"up:active","state":"up:active","rank_uptime":8.6442573540000005,"mdsmap_epoch":5,"osdmap_epoch":23,"osdmap_epoch_barrier":21,"uptime":10.801202324} 2021-07-11T03:53:31.005 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 4224, "mdsmap_epoch": 5, "osdmap_epoch": 23, "osdmap_epoch_barrier": 21, "rank_uptime": 8.644257354, "state": "up:active", "uptime": 10.801202324, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:53:31.005 INFO:teuthology.run_tasks:Running task ceph-fuse... 2021-07-11T03:53:31.033 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2021-07-11T03:53:31.033 INFO:tasks.ceph_fuse:config is None 2021-07-11T03:53:31.034 INFO:tasks.ceph_fuse:new config is {'client.0': None, 'client.1': None} 2021-07-11T03:53:31.034 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2021-07-11T03:53:31.034 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:31.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.080+0000 7feee8f93700 1 Processor -- start 2021-07-11T03:53:31.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- start start 2021-07-11T03:53:31.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7feee40b2660 0x7feee405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7feee4096cc0 0x7feee4097100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7feee413f560 0x7feee413f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee40b2a40 con 0x7feee413f560 2021-07-11T03:53:31.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee40975f0 con 0x7feee40b2660 2021-07-11T03:53:31.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee4097b80 con 0x7feee4096cc0 2021-07-11T03:53:31.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee2d9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7feee413f560 0x7feee413f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee2d9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7feee413f560 0x7feee413f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42282/0 (socket says 172.21.2.104:42282) 2021-07-11T03:53:31.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee2d9d700 1 -- 172.21.2.104:0/1299324736 learned_addr learned my addr 172.21.2.104:0/1299324736 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:31.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee2d9d700 1 -- 172.21.2.104:0/1299324736 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7feee4096cc0 msgr2=0x7feee4097100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:31.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee2d9d700 1 --2- 172.21.2.104:0/1299324736 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7feee4096cc0 0x7feee4097100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee2d9d700 1 -- 172.21.2.104:0/1299324736 >> v2:172.21.2.105:3300/0 conn(0x7feee40b2660 msgr2=0x7feee405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:31.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee2d9d700 1 --2- 172.21.2.104:0/1299324736 >> v2:172.21.2.105:3300/0 conn(0x7feee40b2660 0x7feee405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee2d9d700 1 -- 172.21.2.104:0/1299324736 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee4097e20 con 0x7feee413f560 2021-07-11T03:53:31.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee2d9d700 1 --2- 172.21.2.104:0/1299324736 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7feee413f560 0x7feee413f9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7feedc0032f0 tx=0x7feedc00b530).ready entity=mon.0 client_cookie=c40bf6ddcacc0e1c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee159a700 1 -- 172.21.2.104:0/1299324736 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feedc00dd00 con 0x7feee413f560 2021-07-11T03:53:31.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee159a700 1 -- 172.21.2.104:0/1299324736 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feedc00de60 con 0x7feee413f560 2021-07-11T03:53:31.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee159a700 1 -- 172.21.2.104:0/1299324736 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feedc012640 con 0x7feee413f560 2021-07-11T03:53:31.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- 172.21.2.104:0/1299324736 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7feee413f560 msgr2=0x7feee413f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 --2- 172.21.2.104:0/1299324736 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7feee413f560 0x7feee413f9f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7feedc0032f0 tx=0x7feedc00b530).stop 2021-07-11T03:53:31.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- 172.21.2.104:0/1299324736 shutdown_connections 2021-07-11T03:53:31.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 --2- 172.21.2.104:0/1299324736 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7feee413f560 0x7feee413f9f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 --2- 172.21.2.104:0/1299324736 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7feee4096cc0 0x7feee4097100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 --2- 172.21.2.104:0/1299324736 >> v2:172.21.2.105:3300/0 conn(0x7feee40b2660 0x7feee405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- 172.21.2.104:0/1299324736 >> 172.21.2.104:0/1299324736 conn(0x7feee4062f10 msgr2=0x7feee40632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- 172.21.2.104:0/1299324736 shutdown_connections 2021-07-11T03:53:31.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- 172.21.2.104:0/1299324736 wait complete. 2021-07-11T03:53:31.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 Processor -- start 2021-07-11T03:53:31.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- start start 2021-07-11T03:53:31.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7feee4096cc0 0x7feee40af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7feee40b2660 0x7feee40a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7feee413f560 0x7feee40a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee4063c90 con 0x7feee4096cc0 2021-07-11T03:53:31.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee40aaf70 con 0x7feee40b2660 2021-07-11T03:53:31.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee8f93700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee4063ec0 con 0x7feee413f560 2021-07-11T03:53:31.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7feee4096cc0 0x7feee40af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7feee4096cc0 0x7feee40af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42286/0 (socket says 172.21.2.104:42286) 2021-07-11T03:53:31.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.084+0000 7feee259c700 1 -- 172.21.2.104:0/2202569118 learned_addr learned my addr 172.21.2.104:0/2202569118 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:31.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feee259c700 1 -- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7feee413f560 msgr2=0x7feee40a9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feee259c700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7feee413f560 0x7feee40a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feee259c700 1 -- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7feee40b2660 msgr2=0x7feee40a9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feee259c700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7feee40b2660 0x7feee40a9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feee259c700 1 -- 172.21.2.104:0/2202569118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc00b7d0 con 0x7feee4096cc0 2021-07-11T03:53:31.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feee259c700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7feee4096cc0 0x7feee40af3e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7feed4002f70 tx=0x7feed4006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feedb7fe700 1 -- 172.21.2.104:0/2202569118 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feed4013460 con 0x7feee4096cc0 2021-07-11T03:53:31.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feedb7fe700 1 -- 172.21.2.104:0/2202569118 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed40135c0 con 0x7feee4096cc0 2021-07-11T03:53:31.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feee8f93700 1 -- 172.21.2.104:0/2202569118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee40ab210 con 0x7feee4096cc0 2021-07-11T03:53:31.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feee8f93700 1 -- 172.21.2.104:0/2202569118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee40ab500 con 0x7feee4096cc0 2021-07-11T03:53:31.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feedb7fe700 1 -- 172.21.2.104:0/2202569118 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feed4013920 con 0x7feee4096cc0 2021-07-11T03:53:31.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feee8f93700 1 -- 172.21.2.104:0/2202569118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc003410 con 0x7feee4096cc0 2021-07-11T03:53:31.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feedb7fe700 1 -- 172.21.2.104:0/2202569118 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7feed401a060 con 0x7feee4096cc0 2021-07-11T03:53:31.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feedb7fe700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7feec8053240 0x7feec8055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feedb7fe700 1 -- 172.21.2.104:0/2202569118 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7feed406afd0 con 0x7feee4096cc0 2021-07-11T03:53:31.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.088+0000 7feee1d9b700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7feec8053240 0x7feec8055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.092+0000 7feee1d9b700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7feec8053240 0x7feec8055700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7feecc002f70 tx=0x7feecc006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.092+0000 7feedb7fe700 1 -- 172.21.2.104:0/2202569118 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7feed4042200 con 0x7feee4096cc0 2021-07-11T03:53:31.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feee8f93700 1 -- 172.21.2.104:0/2202569118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feebc0036c0 con 0x7feee4096cc0 2021-07-11T03:53:31.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feedb7fe700 1 -- 172.21.2.104:0/2202569118 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16038 (secure 0 0 0) 0x7feed4045820 con 0x7feee4096cc0 2021-07-11T03:53:31.177 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:31.177 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":23,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:24.367517+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:31.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feee8f93700 1 -- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7feec8053240 msgr2=0x7feec8055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feee8f93700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7feec8053240 0x7feec8055700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7feecc002f70 tx=0x7feecc006a80).stop 2021-07-11T03:53:31.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feee8f93700 1 -- 172.21.2.104:0/2202569118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7feee4096cc0 msgr2=0x7feee40af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feee8f93700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7feee4096cc0 0x7feee40af3e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7feed4002f70 tx=0x7feed4006e40).stop 2021-07-11T03:53:31.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feee8f93700 1 -- 172.21.2.104:0/2202569118 shutdown_connections 2021-07-11T03:53:31.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feee8f93700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7feee413f560 0x7feee40a9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feee8f93700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7feee40b2660 0x7feee40a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feee8f93700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7feec8053240 0x7feec8055700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feee8f93700 1 --2- 172.21.2.104:0/2202569118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7feee4096cc0 0x7feee40af3e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.172+0000 7feee8f93700 1 -- 172.21.2.104:0/2202569118 >> 172.21.2.104:0/2202569118 conn(0x7feee4062f10 msgr2=0x7feee40b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.176+0000 7feee8f93700 1 -- 172.21.2.104:0/2202569118 shutdown_connections 2021-07-11T03:53:31.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.176+0000 7feee8f93700 1 -- 172.21.2.104:0/2202569118 wait complete. 2021-07-11T03:53:31.184 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:53:20.980780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:53:21.341375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:53:31.185 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:53:31.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 Processor -- start 2021-07-11T03:53:31.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 -- start start 2021-07-11T03:53:31.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a99b0 0x7f8b640b2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b64139bc0 con 0x7f8b640aca50 2021-07-11T03:53:31.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b64140500 con 0x7f8b640a95d0 2021-07-11T03:53:31.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b640b0130 con 0x7f8b640a99b0 2021-07-11T03:53:31.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b69d6c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8b640aca50 0x7f8b640b0870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39062/0 (socket says 172.21.2.104:39062) 2021-07-11T03:53:31.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b69d6c700 1 -- 172.21.2.104:0/3647160684 learned_addr learned my addr 172.21.2.104:0/3647160684 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:31.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6ad6e700 1 --2- 172.21.2.104:0/3647160684 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a99b0 0x7f8b640b2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6ad6e700 1 -- 172.21.2.104:0/3647160684 >> v1:172.21.2.105:6789/0 conn(0x7f8b640a95d0 legacy=0x7f8b640a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6ad6e700 1 -- 172.21.2.104:0/3647160684 >> v1:172.21.2.104:6789/0 conn(0x7f8b640aca50 legacy=0x7f8b640b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6ad6e700 1 -- 172.21.2.104:0/3647160684 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b640b03d0 con 0x7f8b640a99b0 2021-07-11T03:53:31.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6ad6e700 1 --2- 172.21.2.104:0/3647160684 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a99b0 0x7f8b640b2c00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8b5c0032f0 tx=0x7f8b5c0091d0).ready entity=mon.2 client_cookie=eea93e6e17e21cbd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6956b700 1 -- 172.21.2.104:0/3647160684 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5c00b8a0 con 0x7f8b640a99b0 2021-07-11T03:53:31.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6956b700 1 -- 172.21.2.104:0/3647160684 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c00ba00 con 0x7f8b640a99b0 2021-07-11T03:53:31.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6956b700 1 -- 172.21.2.104:0/3647160684 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b5c00bb70 con 0x7f8b640a99b0 2021-07-11T03:53:31.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/3647160684 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a99b0 msgr2=0x7f8b640b2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 --2- 172.21.2.104:0/3647160684 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a99b0 0x7f8b640b2c00 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8b5c0032f0 tx=0x7f8b5c0091d0).stop 2021-07-11T03:53:31.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/3647160684 shutdown_connections 2021-07-11T03:53:31.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 --2- 172.21.2.104:0/3647160684 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a99b0 0x7f8b640b2c00 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/3647160684 >> 172.21.2.104:0/3647160684 conn(0x7f8b6405f6b0 msgr2=0x7f8b6405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/3647160684 shutdown_connections 2021-07-11T03:53:31.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.236+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/3647160684 wait complete. 2021-07-11T03:53:31.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6c7d1700 1 Processor -- start 2021-07-11T03:53:31.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6c7d1700 1 -- start start 2021-07-11T03:53:31.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6c7d1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a95d0 0x7f8b6413cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6c7d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b640a99b0 0x7f8b64064200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6c7d1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b640aca50 0x7f8b640646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6c7d1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6413c540 con 0x7f8b640a99b0 2021-07-11T03:53:31.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6c7d1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6413af20 con 0x7f8b640aca50 2021-07-11T03:53:31.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6c7d1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6413b1c0 con 0x7f8b640a95d0 2021-07-11T03:53:31.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b69d6c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b640a99b0 0x7f8b64064200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b69d6c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b640a99b0 0x7f8b64064200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42304/0 (socket says 172.21.2.104:42304) 2021-07-11T03:53:31.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b69d6c700 1 -- 172.21.2.104:0/2333900951 learned_addr learned my addr 172.21.2.104:0/2333900951 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:31.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6ad6e700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b640aca50 0x7f8b640646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b69d6c700 1 -- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a95d0 msgr2=0x7f8b6413cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6a56d700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a95d0 0x7f8b6413cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b69d6c700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a95d0 0x7f8b6413cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b69d6c700 1 -- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b640aca50 msgr2=0x7f8b640646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b69d6c700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b640aca50 0x7f8b640646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b69d6c700 1 -- 172.21.2.104:0/2333900951 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c009470 con 0x7f8b640a99b0 2021-07-11T03:53:31.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6ad6e700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b640aca50 0x7f8b640646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:53:31.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6a56d700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a95d0 0x7f8b6413cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:31.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b69d6c700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b640a99b0 0x7f8b64064200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f8b6406a0f0 tx=0x7f8b54007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b537fe700 1 -- 172.21.2.104:0/2333900951 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b54001d70 con 0x7f8b640a99b0 2021-07-11T03:53:31.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b537fe700 1 -- 172.21.2.104:0/2333900951 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5400ce30 con 0x7f8b640a99b0 2021-07-11T03:53:31.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b537fe700 1 -- 172.21.2.104:0/2333900951 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b54005030 con 0x7f8b640a99b0 2021-07-11T03:53:31.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/2333900951 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6413b460 con 0x7f8b640a99b0 2021-07-11T03:53:31.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/2333900951 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6413b720 con 0x7f8b640a99b0 2021-07-11T03:53:31.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/2333900951 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b64063fa0 con 0x7f8b640a99b0 2021-07-11T03:53:31.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b537fe700 1 -- 172.21.2.104:0/2333900951 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8b5401b070 con 0x7f8b640a99b0 2021-07-11T03:53:31.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b537fe700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b440532b0 0x7f8b44055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b537fe700 1 -- 172.21.2.104:0/2333900951 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8b5406aab0 con 0x7f8b640a99b0 2021-07-11T03:53:31.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6a56d700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b440532b0 0x7f8b44055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.240+0000 7f8b6a56d700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b440532b0 0x7f8b44055770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8b60008000 tx=0x7f8b60009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.244+0000 7f8b537fe700 1 -- 172.21.2.104:0/2333900951 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8b54041ce0 con 0x7f8b640a99b0 2021-07-11T03:53:31.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.324+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/2333900951 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8b64057350 con 0x7f8b640a99b0 2021-07-11T03:53:31.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.324+0000 7f8b537fe700 1 -- 172.21.2.104:0/2333900951 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8b54045300 con 0x7f8b640a99b0 2021-07-11T03:53:31.330 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:53:31.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b440532b0 msgr2=0x7f8b44055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b440532b0 0x7f8b44055770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f8b60008000 tx=0x7f8b60009040).stop 2021-07-11T03:53:31.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/2333900951 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b640a99b0 msgr2=0x7f8b64064200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b640a99b0 0x7f8b64064200 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f8b6406a0f0 tx=0x7f8b54007c30).stop 2021-07-11T03:53:31.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/2333900951 shutdown_connections 2021-07-11T03:53:31.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b640aca50 0x7f8b640646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b440532b0 0x7f8b44055770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b640a99b0 0x7f8b64064200 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 --2- 172.21.2.104:0/2333900951 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b640a95d0 0x7f8b6413cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/2333900951 >> 172.21.2.104:0/2333900951 conn(0x7f8b6405f6b0 msgr2=0x7f8b6405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/2333900951 shutdown_connections 2021-07-11T03:53:31.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.328+0000 7f8b6c7d1700 1 -- 172.21.2.104:0/2333900951 wait complete. 2021-07-11T03:53:31.337 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:53:31.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86cbe2700 1 Processor -- start 2021-07-11T03:53:31.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86cbe2700 1 -- start start 2021-07-11T03:53:31.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86cbe2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680b1ec0 0x7fe8680b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86cbe2700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe8680affb0 0x7fe8680b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86cbe2700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8680b2790 con 0x7fe8680a70c0 2021-07-11T03:53:31.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86cbe2700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8680b03f0 con 0x7fe8680affb0 2021-07-11T03:53:31.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86cbe2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680b0650 con 0x7fe8680b1ec0 2021-07-11T03:53:31.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86659c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680b1ec0 0x7fe8680b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe865d9b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe8680affb0 0x7fe8680b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86659c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680b1ec0 0x7fe8680b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:45472/0 (socket says 172.21.2.104:45472) 2021-07-11T03:53:31.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86659c700 1 -- 172.21.2.104:0/349756075 learned_addr learned my addr 172.21.2.104:0/349756075 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:31.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86659c700 1 -- 172.21.2.104:0/349756075 >> v2:172.21.2.105:3300/0 conn(0x7fe8680affb0 msgr2=0x7fe8680b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:31.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86659c700 1 --2- 172.21.2.104:0/349756075 >> v2:172.21.2.105:3300/0 conn(0x7fe8680affb0 0x7fe8680b22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86659c700 1 -- 172.21.2.104:0/349756075 >> v1:172.21.2.105:6789/0 conn(0x7fe8680a70c0 legacy=0x7fe868141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.388+0000 7fe86659c700 1 -- 172.21.2.104:0/349756075 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8680b08f0 con 0x7fe8680b1ec0 2021-07-11T03:53:31.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86659c700 1 --2- 172.21.2.104:0/349756075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680b1ec0 0x7fe8680b12d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fe850002f70 tx=0x7fe850006eb0).ready entity=mon.2 client_cookie=4ee1ac84c88544f5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86559a700 1 -- 172.21.2.104:0/349756075 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe85000d570 con 0x7fe8680b1ec0 2021-07-11T03:53:31.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86559a700 1 -- 172.21.2.104:0/349756075 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe850000e90 con 0x7fe8680b1ec0 2021-07-11T03:53:31.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86559a700 1 -- 172.21.2.104:0/349756075 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe85000d900 con 0x7fe8680b1ec0 2021-07-11T03:53:31.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 -- 172.21.2.104:0/349756075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680b1ec0 msgr2=0x7fe8680b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 --2- 172.21.2.104:0/349756075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680b1ec0 0x7fe8680b12d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fe850002f70 tx=0x7fe850006eb0).stop 2021-07-11T03:53:31.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 -- 172.21.2.104:0/349756075 shutdown_connections 2021-07-11T03:53:31.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 --2- 172.21.2.104:0/349756075 >> v2:172.21.2.105:3300/0 conn(0x7fe8680affb0 0x7fe8680b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 --2- 172.21.2.104:0/349756075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680b1ec0 0x7fe8680b12d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 -- 172.21.2.104:0/349756075 >> 172.21.2.104:0/349756075 conn(0x7fe86805f6b0 msgr2=0x7fe86805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 -- 172.21.2.104:0/349756075 shutdown_connections 2021-07-11T03:53:31.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 -- 172.21.2.104:0/349756075 wait complete. 2021-07-11T03:53:31.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 Processor -- start 2021-07-11T03:53:31.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 -- start start 2021-07-11T03:53:31.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680a70c0 0x7fe8680abd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8680affb0 0x7fe8680b5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8680b1ec0 0x7fe86813c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86815af90 con 0x7fe8680b1ec0 2021-07-11T03:53:31.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86814b3f0 con 0x7fe8680affb0 2021-07-11T03:53:31.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86814b650 con 0x7fe8680a70c0 2021-07-11T03:53:31.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe866d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8680b1ec0 0x7fe86813c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe866d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8680b1ec0 0x7fe86813c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42318/0 (socket says 172.21.2.104:42318) 2021-07-11T03:53:31.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe866d9d700 1 -- 172.21.2.104:0/831359950 learned_addr learned my addr 172.21.2.104:0/831359950 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:31.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86659c700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680a70c0 0x7fe8680abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe865d9b700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8680affb0 0x7fe8680b5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe866d9d700 1 -- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680a70c0 msgr2=0x7fe8680abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe866d9d700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680a70c0 0x7fe8680abd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe866d9d700 1 -- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8680affb0 msgr2=0x7fe8680b5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe866d9d700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8680affb0 0x7fe8680b5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe866d9d700 1 -- 172.21.2.104:0/831359950 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85000b1e0 con 0x7fe8680b1ec0 2021-07-11T03:53:31.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe866d9d700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8680b1ec0 0x7fe86813c7b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fe86806a0f0 tx=0x7fe860007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe85f7fe700 1 -- 172.21.2.104:0/831359950 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe86000e4a0 con 0x7fe8680b1ec0 2021-07-11T03:53:31.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe85f7fe700 1 -- 172.21.2.104:0/831359950 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86000e600 con 0x7fe8680b1ec0 2021-07-11T03:53:31.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe85f7fe700 1 -- 172.21.2.104:0/831359950 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe86000e960 con 0x7fe8680b1ec0 2021-07-11T03:53:31.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 -- 172.21.2.104:0/831359950 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86815acf0 con 0x7fe8680b1ec0 2021-07-11T03:53:31.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.392+0000 7fe86cbe2700 1 -- 172.21.2.104:0/831359950 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe868149f20 con 0x7fe8680b1ec0 2021-07-11T03:53:31.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.396+0000 7fe86cbe2700 1 -- 172.21.2.104:0/831359950 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe868057350 con 0x7fe8680b1ec0 2021-07-11T03:53:31.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.396+0000 7fe85f7fe700 1 -- 172.21.2.104:0/831359950 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe860019030 con 0x7fe8680b1ec0 2021-07-11T03:53:31.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.396+0000 7fe85f7fe700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe84c053160 0x7fe84c055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.396+0000 7fe85f7fe700 1 -- 172.21.2.104:0/831359950 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe86006abb0 con 0x7fe8680b1ec0 2021-07-11T03:53:31.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.396+0000 7fe86659c700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe84c053160 0x7fe84c055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.396+0000 7fe86659c700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe84c053160 0x7fe84c055620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe850019000 tx=0x7fe850015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.396+0000 7fe85f7fe700 1 -- 172.21.2.104:0/831359950 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe860071080 con 0x7fe8680b1ec0 2021-07-11T03:53:31.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 -- 172.21.2.104:0/831359950 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe8680a8fc0 con 0x7fe8680b1ec0 2021-07-11T03:53:31.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe85f7fe700 1 -- 172.21.2.104:0/831359950 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe860041e90 con 0x7fe8680b1ec0 2021-07-11T03:53:31.484 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:31.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 -- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe84c053160 msgr2=0x7fe84c055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe84c053160 0x7fe84c055620 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fe850019000 tx=0x7fe850015000).stop 2021-07-11T03:53:31.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 -- 172.21.2.104:0/831359950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8680b1ec0 msgr2=0x7fe86813c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8680b1ec0 0x7fe86813c7b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fe86806a0f0 tx=0x7fe860007f40).stop 2021-07-11T03:53:31.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 -- 172.21.2.104:0/831359950 shutdown_connections 2021-07-11T03:53:31.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe84c053160 0x7fe84c055620 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8680b1ec0 0x7fe86813c7b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8680affb0 0x7fe8680b5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 --2- 172.21.2.104:0/831359950 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8680a70c0 0x7fe8680abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 -- 172.21.2.104:0/831359950 >> 172.21.2.104:0/831359950 conn(0x7fe86805f6b0 msgr2=0x7fe86813edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 -- 172.21.2.104:0/831359950 shutdown_connections 2021-07-11T03:53:31.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.480+0000 7fe86cbe2700 1 -- 172.21.2.104:0/831359950 wait complete. 2021-07-11T03:53:31.491 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:53:31.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 Processor -- start 2021-07-11T03:53:31.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 -- start start 2021-07-11T03:53:31.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f71740b05b0 0x7f71740b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f71740b3710 0x7f717413d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f717413da40 0x7f71740b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7174143920 con 0x7f71740b3710 2021-07-11T03:53:31.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7174143bc0 con 0x7f717413da40 2021-07-11T03:53:31.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7174143e60 con 0x7f71740b05b0 2021-07-11T03:53:31.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f717259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f71740b3710 0x7f717413d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f717259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f71740b3710 0x7f717413d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42322/0 (socket says 172.21.2.104:42322) 2021-07-11T03:53:31.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f717259c700 1 -- 172.21.2.104:0/1167461957 learned_addr learned my addr 172.21.2.104:0/1167461957 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:31.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f717259c700 1 -- 172.21.2.104:0/1167461957 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f71740b05b0 msgr2=0x7f71740b0990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:31.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f717259c700 1 --2- 172.21.2.104:0/1167461957 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f71740b05b0 0x7f71740b0990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f717259c700 1 -- 172.21.2.104:0/1167461957 >> v2:172.21.2.105:3300/0 conn(0x7f717413da40 msgr2=0x7f71740b10b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:31.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f717259c700 1 --2- 172.21.2.104:0/1167461957 >> v2:172.21.2.105:3300/0 conn(0x7f717413da40 0x7f71740b10b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f717259c700 1 -- 172.21.2.104:0/1167461957 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7174144100 con 0x7f71740b3710 2021-07-11T03:53:31.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f717259c700 1 --2- 172.21.2.104:0/1167461957 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f71740b3710 0x7f717413d550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f715c002f70 tx=0x7f715c006eb0).ready entity=mon.0 client_cookie=d4be683b6e63fe69 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f7171d9b700 1 -- 172.21.2.104:0/1167461957 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f715c00d570 con 0x7f71740b3710 2021-07-11T03:53:31.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f7171d9b700 1 -- 172.21.2.104:0/1167461957 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c000e90 con 0x7f71740b3710 2021-07-11T03:53:31.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f7171d9b700 1 -- 172.21.2.104:0/1167461957 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f715c00d900 con 0x7f71740b3710 2021-07-11T03:53:31.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 -- 172.21.2.104:0/1167461957 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f71740b3710 msgr2=0x7f717413d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 --2- 172.21.2.104:0/1167461957 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f71740b3710 0x7f717413d550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f715c002f70 tx=0x7f715c006eb0).stop 2021-07-11T03:53:31.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 -- 172.21.2.104:0/1167461957 shutdown_connections 2021-07-11T03:53:31.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 --2- 172.21.2.104:0/1167461957 >> v2:172.21.2.105:3300/0 conn(0x7f717413da40 0x7f71740b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 --2- 172.21.2.104:0/1167461957 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f71740b3710 0x7f717413d550 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 --2- 172.21.2.104:0/1167461957 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f71740b05b0 0x7f71740b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 -- 172.21.2.104:0/1167461957 >> 172.21.2.104:0/1167461957 conn(0x7f71740607c0 msgr2=0x7f7174060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 -- 172.21.2.104:0/1167461957 shutdown_connections 2021-07-11T03:53:31.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.540+0000 7f71791ab700 1 -- 172.21.2.104:0/1167461957 wait complete. 2021-07-11T03:53:31.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f71791ab700 1 Processor -- start 2021-07-11T03:53:31.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f71791ab700 1 -- start start 2021-07-11T03:53:31.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f71791ab700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f71740b05b0 0x7f7174066ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f71791ab700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f71740b3710 0x7f7174136150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f71791ab700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f717413da40 0x7f7174136640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f71791ab700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7174138120 con 0x7f71740b3710 2021-07-11T03:53:31.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f71791ab700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7174067880 con 0x7f717413da40 2021-07-11T03:53:31.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f71791ab700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7174137aa0 con 0x7f71740b05b0 2021-07-11T03:53:31.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f717259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f71740b3710 0x7f7174136150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f717259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f71740b3710 0x7f7174136150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42326/0 (socket says 172.21.2.104:42326) 2021-07-11T03:53:31.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f717259c700 1 -- 172.21.2.104:0/1365363865 learned_addr learned my addr 172.21.2.104:0/1365363865 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:31.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f717259c700 1 -- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f71740b05b0 msgr2=0x7f7174066ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f717259c700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f71740b05b0 0x7f7174066ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f717259c700 1 -- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f717413da40 msgr2=0x7f7174136640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f717259c700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f717413da40 0x7f7174136640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f717259c700 1 -- 172.21.2.104:0/1365363865 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715c00b1e0 con 0x7f71740b3710 2021-07-11T03:53:31.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f717259c700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f71740b3710 0x7f7174136150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f715c002f70 tx=0x7f715c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f716bfff700 1 -- 172.21.2.104:0/1365363865 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f715c00de10 con 0x7f71740b3710 2021-07-11T03:53:31.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f716bfff700 1 -- 172.21.2.104:0/1365363865 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c011ce0 con 0x7f71740b3710 2021-07-11T03:53:31.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f71791ab700 1 -- 172.21.2.104:0/1365363865 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7174135770 con 0x7f71740b3710 2021-07-11T03:53:31.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f716bfff700 1 -- 172.21.2.104:0/1365363865 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f715c01d640 con 0x7f71740b3710 2021-07-11T03:53:31.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f71791ab700 1 -- 172.21.2.104:0/1365363865 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7174135cb0 con 0x7f71740b3710 2021-07-11T03:53:31.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f71791ab700 1 -- 172.21.2.104:0/1365363865 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7174135e70 con 0x7f71740b3710 2021-07-11T03:53:31.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f716bfff700 1 -- 172.21.2.104:0/1365363865 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f715c00f070 con 0x7f71740b3710 2021-07-11T03:53:31.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f716bfff700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7158053160 0x7f7158055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f716bfff700 1 -- 172.21.2.104:0/1365363865 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f715c06cd30 con 0x7f71740b3710 2021-07-11T03:53:31.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.544+0000 7f7172d9d700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7158053160 0x7f7158055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.548+0000 7f7172d9d700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7158053160 0x7f7158055620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f7164002fb0 tx=0x7f71640069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.548+0000 7f716bfff700 1 -- 172.21.2.104:0/1365363865 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f715c044070 con 0x7f71740b3710 2021-07-11T03:53:31.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.628+0000 7f71791ab700 1 -- 172.21.2.104:0/1365363865 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7174057350 con 0x7f71740b3710 2021-07-11T03:53:31.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.628+0000 7f716bfff700 1 -- 172.21.2.104:0/1365363865 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f715c047690 con 0x7f71740b3710 2021-07-11T03:53:31.633 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:31.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 -- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7158053160 msgr2=0x7f7158055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7158053160 0x7f7158055620 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f7164002fb0 tx=0x7f71640069b0).stop 2021-07-11T03:53:31.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 -- 172.21.2.104:0/1365363865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f71740b3710 msgr2=0x7f7174136150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f71740b3710 0x7f7174136150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f715c002f70 tx=0x7f715c015000).stop 2021-07-11T03:53:31.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 -- 172.21.2.104:0/1365363865 shutdown_connections 2021-07-11T03:53:31.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f717413da40 0x7f7174136640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7158053160 0x7f7158055620 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f71740b3710 0x7f7174136150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 --2- 172.21.2.104:0/1365363865 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f71740b05b0 0x7f7174066ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 -- 172.21.2.104:0/1365363865 >> 172.21.2.104:0/1365363865 conn(0x7f71740607c0 msgr2=0x7f71741413d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 -- 172.21.2.104:0/1365363865 shutdown_connections 2021-07-11T03:53:31.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.632+0000 7f71791ab700 1 -- 172.21.2.104:0/1365363865 wait complete. 2021-07-11T03:53:31.640 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:53:31.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f4140417700 1 Processor -- start 2021-07-11T03:53:31.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f4140417700 1 -- start start 2021-07-11T03:53:31.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f4140417700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4138144440 0x7f4138144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f4140417700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4138065420 0x7f4138144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f4140417700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4138145350 con 0x7f4138060100 2021-07-11T03:53:31.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f4140417700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4138065c80 con 0x7f4138065420 2021-07-11T03:53:31.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f4140417700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41380604e0 con 0x7f4138144440 2021-07-11T03:53:31.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f413e9b4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4138065420 0x7f4138144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f413e1b3700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4138060100 0x7f4138064d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39102/0 (socket says 172.21.2.104:39102) 2021-07-11T03:53:31.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f413e1b3700 1 -- 172.21.2.104:0/1446039867 learned_addr learned my addr 172.21.2.104:0/1446039867 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:31.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f413e9b4700 1 -- 172.21.2.104:0/1446039867 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4138144440 msgr2=0x7f4138144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:31.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f413e9b4700 1 --2- 172.21.2.104:0/1446039867 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4138144440 0x7f4138144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f413e9b4700 1 -- 172.21.2.104:0/1446039867 >> v1:172.21.2.104:6789/0 conn(0x7f4138060100 legacy=0x7f4138064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.688+0000 7f413d1b1700 1 -- 172.21.2.104:0/1446039867 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3164685658 0 0) 0x7f4138145350 con 0x7f4138060100 2021-07-11T03:53:31.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413e9b4700 1 -- 172.21.2.104:0/1446039867 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41380b23b0 con 0x7f4138065420 2021-07-11T03:53:31.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413e9b4700 1 --2- 172.21.2.104:0/1446039867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4138065420 0x7f4138144e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f41300032f0 tx=0x7f41300091d0).ready entity=mon.0 client_cookie=3eac1ee53191226d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d1b1700 1 -- 172.21.2.104:0/1446039867 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4130009ea0 con 0x7f4138065420 2021-07-11T03:53:31.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d1b1700 1 -- 172.21.2.104:0/1446039867 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413000b8a0 con 0x7f4138065420 2021-07-11T03:53:31.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d1b1700 1 -- 172.21.2.104:0/1446039867 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f413000ba10 con 0x7f4138065420 2021-07-11T03:53:31.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 -- 172.21.2.104:0/1446039867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4138065420 msgr2=0x7f4138144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 --2- 172.21.2.104:0/1446039867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4138065420 0x7f4138144e60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f41300032f0 tx=0x7f41300091d0).stop 2021-07-11T03:53:31.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 -- 172.21.2.104:0/1446039867 shutdown_connections 2021-07-11T03:53:31.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 --2- 172.21.2.104:0/1446039867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4138065420 0x7f4138144e60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 --2- 172.21.2.104:0/1446039867 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4138144440 0x7f4138144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 -- 172.21.2.104:0/1446039867 >> 172.21.2.104:0/1446039867 conn(0x7f413805cf60 msgr2=0x7f413805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 -- 172.21.2.104:0/1446039867 shutdown_connections 2021-07-11T03:53:31.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 -- 172.21.2.104:0/1446039867 wait complete. 2021-07-11T03:53:31.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 Processor -- start 2021-07-11T03:53:31.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 -- start start 2021-07-11T03:53:31.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4138060100 0x7f41380ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4138065420 0x7f4138141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f41380b02f0 0x7f4138134380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41380b0f40 con 0x7f4138065420 2021-07-11T03:53:31.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413813bb70 con 0x7f4138060100 2021-07-11T03:53:31.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413813b8d0 con 0x7f41380b02f0 2021-07-11T03:53:31.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d9b2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4138065420 0x7f4138141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d9b2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4138065420 0x7f4138141160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42340/0 (socket says 172.21.2.104:42340) 2021-07-11T03:53:31.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d9b2700 1 -- 172.21.2.104:0/70702647 learned_addr learned my addr 172.21.2.104:0/70702647 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:31.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d9b2700 1 -- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f41380b02f0 msgr2=0x7f4138134380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d9b2700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f41380b02f0 0x7f4138134380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d9b2700 1 -- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4138060100 msgr2=0x7f41380ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d9b2700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4138060100 0x7f41380ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d9b2700 1 -- 172.21.2.104:0/70702647 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4130009470 con 0x7f4138065420 2021-07-11T03:53:31.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f413d9b2700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4138065420 0x7f4138141160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f4134002f70 tx=0x7f4134006e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f41277fe700 1 -- 172.21.2.104:0/70702647 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f413400e470 con 0x7f4138065420 2021-07-11T03:53:31.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f41277fe700 1 -- 172.21.2.104:0/70702647 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f413400e5d0 con 0x7f4138065420 2021-07-11T03:53:31.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 -- 172.21.2.104:0/70702647 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41380b11a0 con 0x7f4138065420 2021-07-11T03:53:31.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f41277fe700 1 -- 172.21.2.104:0/70702647 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f413400e930 con 0x7f4138065420 2021-07-11T03:53:31.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.692+0000 7f4140417700 1 -- 172.21.2.104:0/70702647 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f413813e7e0 con 0x7f4138065420 2021-07-11T03:53:31.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.696+0000 7f4140417700 1 -- 172.21.2.104:0/70702647 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4138057350 con 0x7f4138065420 2021-07-11T03:53:31.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.696+0000 7f41277fe700 1 -- 172.21.2.104:0/70702647 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4134018030 con 0x7f4138065420 2021-07-11T03:53:31.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.696+0000 7f41277fe700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4118053270 0x7f4118055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.696+0000 7f41277fe700 1 -- 172.21.2.104:0/70702647 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f413406a970 con 0x7f4138065420 2021-07-11T03:53:31.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.696+0000 7f413e1b3700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4118053270 0x7f4118055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.696+0000 7f413e1b3700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4118053270 0x7f4118055730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4128000ed0 tx=0x7f41280076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.696+0000 7f41277fe700 1 -- 172.21.2.104:0/70702647 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4134041d60 con 0x7f4138065420 2021-07-11T03:53:31.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.776+0000 7f4140417700 1 -- 172.21.2.104:0/70702647 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f41380698e0 con 0x7f4138065420 2021-07-11T03:53:31.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.776+0000 7f41277fe700 1 -- 172.21.2.104:0/70702647 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4134045380 con 0x7f4138065420 2021-07-11T03:53:31.782 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:31.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 -- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4118053270 msgr2=0x7f4118055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4118053270 0x7f4118055730 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f4128000ed0 tx=0x7f41280076c0).stop 2021-07-11T03:53:31.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 -- 172.21.2.104:0/70702647 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4138065420 msgr2=0x7f4138141160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4138065420 0x7f4138141160 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f4134002f70 tx=0x7f4134006e30).stop 2021-07-11T03:53:31.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 -- 172.21.2.104:0/70702647 shutdown_connections 2021-07-11T03:53:31.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f41380b02f0 0x7f4138134380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4118053270 0x7f4118055730 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4138065420 0x7f4138141160 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 --2- 172.21.2.104:0/70702647 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4138060100 0x7f41380ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 -- 172.21.2.104:0/70702647 >> 172.21.2.104:0/70702647 conn(0x7f413805cf60 msgr2=0x7f4138144900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 -- 172.21.2.104:0/70702647 shutdown_connections 2021-07-11T03:53:31.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.780+0000 7f4140417700 1 -- 172.21.2.104:0/70702647 wait complete. 2021-07-11T03:53:31.789 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T03:53:31.790 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:53:31.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 Processor -- start 2021-07-11T03:53:31.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 -- start start 2021-07-11T03:53:31.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4f500b7810 0x7f4f501445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4f50144a90 0x7f4f50064f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f500b31d0 con 0x7f4f500b1170 2021-07-11T03:53:31.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f50144ef0 con 0x7f4f50144a90 2021-07-11T03:53:31.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f500b7bf0 con 0x7f4f500b7810 2021-07-11T03:53:31.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f55943700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4f50144a90 0x7f4f50064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f54941700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4f500b1170 0x7f4f50064870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39116/0 (socket says 172.21.2.104:39116) 2021-07-11T03:53:31.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f54941700 1 -- 172.21.2.104:0/4108480495 learned_addr learned my addr 172.21.2.104:0/4108480495 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:31.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f55943700 1 -- 172.21.2.104:0/4108480495 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4f500b7810 msgr2=0x7f4f501445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:31.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f55943700 1 --2- 172.21.2.104:0/4108480495 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4f500b7810 0x7f4f501445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f55943700 1 -- 172.21.2.104:0/4108480495 >> v1:172.21.2.104:6789/0 conn(0x7f4f500b1170 legacy=0x7f4f50064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f55943700 1 -- 172.21.2.104:0/4108480495 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f500675c0 con 0x7f4f50144a90 2021-07-11T03:53:31.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f55943700 1 --2- 172.21.2.104:0/4108480495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4f50144a90 0x7f4f50064f80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f4f40001490 tx=0x7f4f48006df0).ready entity=mon.0 client_cookie=9ff1351aa9e89a69 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f4ffff700 1 -- 172.21.2.104:0/4108480495 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f4800d570 con 0x7f4f50144a90 2021-07-11T03:53:31.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f4ffff700 1 -- 172.21.2.104:0/4108480495 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f48000e90 con 0x7f4f50144a90 2021-07-11T03:53:31.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f4ffff700 1 -- 172.21.2.104:0/4108480495 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f4800d8c0 con 0x7f4f50144a90 2021-07-11T03:53:31.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 -- 172.21.2.104:0/4108480495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4f50144a90 msgr2=0x7f4f50064f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 --2- 172.21.2.104:0/4108480495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4f50144a90 0x7f4f50064f80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f4f40001490 tx=0x7f4f48006df0).stop 2021-07-11T03:53:31.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 -- 172.21.2.104:0/4108480495 shutdown_connections 2021-07-11T03:53:31.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 --2- 172.21.2.104:0/4108480495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4f50144a90 0x7f4f50064f80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 --2- 172.21.2.104:0/4108480495 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4f500b7810 0x7f4f501445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 -- 172.21.2.104:0/4108480495 >> 172.21.2.104:0/4108480495 conn(0x7f4f5005f6b0 msgr2=0x7f4f5005fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 -- 172.21.2.104:0/4108480495 shutdown_connections 2021-07-11T03:53:31.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.836+0000 7f4f573a6700 1 -- 172.21.2.104:0/4108480495 wait complete. 2021-07-11T03:53:31.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f573a6700 1 Processor -- start 2021-07-11T03:53:31.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f573a6700 1 -- start start 2021-07-11T03:53:31.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f573a6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4f500b1170 0x7f4f50138f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f573a6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4f500b7810 0x7f4f500af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f573a6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4f50144a90 0x7f4f500afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f573a6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f50061d60 con 0x7f4f500b1170 2021-07-11T03:53:31.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f573a6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f500aef90 con 0x7f4f500b7810 2021-07-11T03:53:31.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f573a6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f500aed70 con 0x7f4f50144a90 2021-07-11T03:53:31.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f55142700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4f500b1170 0x7f4f50138f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f55142700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4f500b1170 0x7f4f50138f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42356/0 (socket says 172.21.2.104:42356) 2021-07-11T03:53:31.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f55142700 1 -- 172.21.2.104:0/1766100802 learned_addr learned my addr 172.21.2.104:0/1766100802 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:31.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f55142700 1 -- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4f50144a90 msgr2=0x7f4f500afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f55142700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4f50144a90 0x7f4f500afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f55142700 1 -- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4f500b7810 msgr2=0x7f4f500af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f55142700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4f500b7810 0x7f4f500af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f55142700 1 -- 172.21.2.104:0/1766100802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4800b180 con 0x7f4f500b1170 2021-07-11T03:53:31.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f55142700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4f500b1170 0x7f4f50138f00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f4f44002f70 tx=0x7f4f44006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f4e7fc700 1 -- 172.21.2.104:0/1766100802 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f44013460 con 0x7f4f500b1170 2021-07-11T03:53:31.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f4e7fc700 1 -- 172.21.2.104:0/1766100802 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f440135c0 con 0x7f4f500b1170 2021-07-11T03:53:31.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f4e7fc700 1 -- 172.21.2.104:0/1766100802 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4f44013920 con 0x7f4f500b1170 2021-07-11T03:53:31.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f573a6700 1 -- 172.21.2.104:0/1766100802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f50061f90 con 0x7f4f500b1170 2021-07-11T03:53:31.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f573a6700 1 -- 172.21.2.104:0/1766100802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f501393f0 con 0x7f4f500b1170 2021-07-11T03:53:31.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f573a6700 1 -- 172.21.2.104:0/1766100802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f50057350 con 0x7f4f500b1170 2021-07-11T03:53:31.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f4e7fc700 1 -- 172.21.2.104:0/1766100802 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4f4401a060 con 0x7f4f500b1170 2021-07-11T03:53:31.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f4e7fc700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4f30053240 0x7f4f30055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:31.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f4e7fc700 1 -- 172.21.2.104:0/1766100802 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4f4406b0b0 con 0x7f4f500b1170 2021-07-11T03:53:31.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.840+0000 7f4f54941700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4f30053240 0x7f4f30055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:31.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.844+0000 7f4f54941700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4f30053240 0x7f4f30055700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4f40001550 tx=0x7f4f40007b50).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:31.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.844+0000 7f4f4e7fc700 1 -- 172.21.2.104:0/1766100802 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4f440422e0 con 0x7f4f500b1170 2021-07-11T03:53:31.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.948+0000 7f4f573a6700 1 -- 172.21.2.104:0/1766100802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f500663a0 con 0x7f4f500b1170 2021-07-11T03:53:31.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.948+0000 7f4f4e7fc700 1 -- 172.21.2.104:0/1766100802 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2483 (secure 0 0 0) 0x7f4f44045900 con 0x7f4f500b1170 2021-07-11T03:53:31.954 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:31.954 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3943971217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3943971217},{"type":"v1","addr":"172.21.2.105:6835","nonce":3943971217}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/668148871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":668148871},{"type":"v1","addr":"172.21.2.104:6833","nonce":668148871}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":6}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:22.350923+0000","modified":"2021-07-11T03:53:23.355891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4224":{"gid":4224,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/1236541958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":1236541958},{"type":"v1","addr":"172.21.2.104:6835","nonce":1236541958}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-07-11T03:53:31.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 -- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4f30053240 msgr2=0x7f4f30055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4f30053240 0x7f4f30055700 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4f40001550 tx=0x7f4f40007b50).stop 2021-07-11T03:53:31.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 -- 172.21.2.104:0/1766100802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4f500b1170 msgr2=0x7f4f50138f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:31.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4f500b1170 0x7f4f50138f00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f4f44002f70 tx=0x7f4f44006ec0).stop 2021-07-11T03:53:31.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 -- 172.21.2.104:0/1766100802 shutdown_connections 2021-07-11T03:53:31.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4f50144a90 0x7f4f500afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4f500b7810 0x7f4f500af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:31.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4f30053240 0x7f4f30055700 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 --2- 172.21.2.104:0/1766100802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4f500b1170 0x7f4f50138f00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:31.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 -- 172.21.2.104:0/1766100802 >> 172.21.2.104:0/1766100802 conn(0x7f4f5005f6b0 msgr2=0x7f4f5005bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:31.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 -- 172.21.2.104:0/1766100802 shutdown_connections 2021-07-11T03:53:31.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:31.952+0000 7f4f573a6700 1 -- 172.21.2.104:0/1766100802 wait complete. 2021-07-11T03:53:31.958 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 6 2021-07-11T03:53:32.264 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:53:22.350923+0000', 'modified': '2021-07-11T03:53:23.355891+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4224}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4224': {'gid': 4224, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.104:6835/1236541958', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 1236541958}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 1236541958}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:53:32.265 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:53:32.265 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:53:32.330 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":4224,"want_state":"up:active","state":"up:active","rank_uptime":9.9702398940000005,"mdsmap_epoch":5,"osdmap_epoch":23,"osdmap_epoch_barrier":21,"uptime":12.127185407000001} 2021-07-11T03:53:32.330 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 4224, "mdsmap_epoch": 5, "osdmap_epoch": 23, "osdmap_epoch_barrier": 21, "rank_uptime": 9.970239894, "state": "up:active", "uptime": 12.127185407, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:53:32.331 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T03:53:32.331 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T03:53:32.331 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T03:53:32.332 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:32.332 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:32.343 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:53:32.355 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:53:32.355 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:53:32.405 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:53:32.406 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:53:32.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:32.407 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:53:32.413 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T03:53:32.413 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:32.416 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:53:32.425 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:53:32.426 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:53:32.431 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:53:32.431 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:53:32.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:32.432 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:53:32.464 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T03:53:32.457+0000 7f1e26375080 -1 init, newargv = 0x5608e33adab0 newargc=9 2021-07-11T03:53:32.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14513]: starting ceph client 2021-07-11T03:53:32.479 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14513]: starting fuse 2021-07-11T03:53:33.537 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:53:33.549 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:53:33.550 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:53:33.598 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:53:33.599 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:53:33.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:33.600 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:53:33.607 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T03:53:33.607 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T03:53:33.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:33.608 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T03:53:33.609 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T03:53:33.609 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T03:53:33.609 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:53:33.610 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T03:53:33.610 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:53:33.610 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T03:53:33.611 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T03:53:33.611 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T03:53:33.611 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:53:33.611 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T03:53:33.612 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T03:53:33.612 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T03:53:33.612 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T03:53:33.613 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:53:33.613 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T03:53:33.613 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T03:53:33.614 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T03:53:33.614 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T03:53:33.614 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T03:53:33.615 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:53:33.615 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T03:53:33.615 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:53:33.644 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14513.asok 2021-07-11T03:53:33.647 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14513.asok 2021-07-11T03:53:33.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:33.648 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14513.asok status 2021-07-11T03:53:33.768 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T03:53:33.769 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T03:53:33.769 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T03:53:33.769 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T03:53:33.770 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T03:53:33.770 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T03:53:33.770 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T03:53:33.770 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14513", 2021-07-11T03:53:33.771 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T03:53:33.771 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:53:33.771 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T03:53:33.771 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T03:53:33.771 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4271, 2021-07-11T03:53:33.772 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T03:53:33.772 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T03:53:33.772 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T03:53:33.772 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4271 2021-07-11T03:53:33.773 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:53:33.773 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:53:33.773 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:53:33.773 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:53:33.773 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3128207794 2021-07-11T03:53:33.774 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T03:53:33.774 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:53:33.774 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:53:33.774 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:53:33.775 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:53:33.775 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3128207794 2021-07-11T03:53:33.775 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:53:33.775 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4271 172.21.2.119:0/3128207794", 2021-07-11T03:53:33.776 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/3128207794", 2021-07-11T03:53:33.776 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T03:53:33.777 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 5, 2021-07-11T03:53:33.777 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 23, 2021-07-11T03:53:33.777 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T03:53:33.778 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T03:53:33.778 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T03:53:33.778 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T03:53:33.780 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:33.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 Processor -- start 2021-07-11T03:53:33.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 -- start start 2021-07-11T03:53:33.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8180a76b0 0x7fe818066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:33.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe8180b1970 0x7fe818066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:33.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe8180b0060 0x7fe81805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:33.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe818067450 con 0x7fe8180b1970 2021-07-11T03:53:33.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8180a7a90 con 0x7fe8180b0060 2021-07-11T03:53:33.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe818147620 con 0x7fe8180a76b0 2021-07-11T03:53:33.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe8177fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe8180b1970 0x7fe818066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:33.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe8177fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe8180b1970 0x7fe818066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42362/0 (socket says 172.21.2.104:42362) 2021-07-11T03:53:33.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe8177fe700 1 -- 172.21.2.104:0/4034244194 learned_addr learned my addr 172.21.2.104:0/4034244194 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:33.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81d000700 1 --2- 172.21.2.104:0/4034244194 >> v2:172.21.2.105:3300/0 conn(0x7fe8180b0060 0x7fe81805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:33.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe817fff700 1 --2- 172.21.2.104:0/4034244194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8180a76b0 0x7fe818066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:33.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe8177fe700 1 -- 172.21.2.104:0/4034244194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8180a76b0 msgr2=0x7fe818066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:33.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe8177fe700 1 --2- 172.21.2.104:0/4034244194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8180a76b0 0x7fe818066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:33.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe8177fe700 1 -- 172.21.2.104:0/4034244194 >> v2:172.21.2.105:3300/0 conn(0x7fe8180b0060 msgr2=0x7fe81805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:33.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe8177fe700 1 --2- 172.21.2.104:0/4034244194 >> v2:172.21.2.105:3300/0 conn(0x7fe8180b0060 0x7fe81805f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:33.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe8177fe700 1 -- 172.21.2.104:0/4034244194 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8181478c0 con 0x7fe8180b1970 2021-07-11T03:53:33.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe817fff700 1 --2- 172.21.2.104:0/4034244194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8180a76b0 0x7fe818066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:33.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe8177fe700 1 --2- 172.21.2.104:0/4034244194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8180b1970 0x7fe818066f60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe800002f70 tx=0x7fe800006eb0).ready entity=mon.0 client_cookie=9314334fda1b2789 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:33.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe816ffd700 1 -- 172.21.2.104:0/4034244194 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe80000d570 con 0x7fe8180b1970 2021-07-11T03:53:33.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe816ffd700 1 -- 172.21.2.104:0/4034244194 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe800000e90 con 0x7fe8180b1970 2021-07-11T03:53:33.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe816ffd700 1 -- 172.21.2.104:0/4034244194 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe80000d900 con 0x7fe8180b1970 2021-07-11T03:53:33.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 -- 172.21.2.104:0/4034244194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8180b1970 msgr2=0x7fe818066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:33.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 --2- 172.21.2.104:0/4034244194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8180b1970 0x7fe818066f60 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe800002f70 tx=0x7fe800006eb0).stop 2021-07-11T03:53:33.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 -- 172.21.2.104:0/4034244194 shutdown_connections 2021-07-11T03:53:33.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 --2- 172.21.2.104:0/4034244194 >> v2:172.21.2.105:3300/0 conn(0x7fe8180b0060 0x7fe81805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:33.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 --2- 172.21.2.104:0/4034244194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8180b1970 0x7fe818066f60 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:33.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 --2- 172.21.2.104:0/4034244194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8180a76b0 0x7fe818066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:33.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 -- 172.21.2.104:0/4034244194 >> 172.21.2.104:0/4034244194 conn(0x7fe8180b53e0 msgr2=0x7fe8180b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:33.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 -- 172.21.2.104:0/4034244194 shutdown_connections 2021-07-11T03:53:33.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.832+0000 7fe81ea63700 1 -- 172.21.2.104:0/4034244194 wait complete. 2021-07-11T03:53:33.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe81ea63700 1 Processor -- start 2021-07-11T03:53:33.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe81ea63700 1 -- start start 2021-07-11T03:53:33.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe81ea63700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8180a76b0 0x7fe8180b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:33.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe81ea63700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8180b0060 0x7fe8180aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:33.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe81ea63700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8180b1970 0x7fe8180a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:33.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe81ea63700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8180ad9f0 con 0x7fe8180b0060 2021-07-11T03:53:33.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe81ea63700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8180ab330 con 0x7fe8180a76b0 2021-07-11T03:53:33.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe81ea63700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8180abbc0 con 0x7fe8180b1970 2021-07-11T03:53:33.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8177fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8180b0060 0x7fe8180aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:33.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8177fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8180b0060 0x7fe8180aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42366/0 (socket says 172.21.2.104:42366) 2021-07-11T03:53:33.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8177fe700 1 -- 172.21.2.104:0/2095878666 learned_addr learned my addr 172.21.2.104:0/2095878666 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:33.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8177fe700 1 -- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8180b1970 msgr2=0x7fe8180a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:33.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe817fff700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8180a76b0 0x7fe8180b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:33.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8177fe700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8180b1970 0x7fe8180a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:33.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8177fe700 1 -- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8180a76b0 msgr2=0x7fe8180b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:33.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8177fe700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8180a76b0 0x7fe8180b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:33.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8177fe700 1 -- 172.21.2.104:0/2095878666 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80000b1e0 con 0x7fe8180b0060 2021-07-11T03:53:33.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe817fff700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8180a76b0 0x7fe8180b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:33.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8177fe700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8180b0060 0x7fe8180aeea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fe800002f70 tx=0x7fe800015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:33.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8157fa700 1 -- 172.21.2.104:0/2095878666 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe800021070 con 0x7fe8180b0060 2021-07-11T03:53:33.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8157fa700 1 -- 172.21.2.104:0/2095878666 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe800000e90 con 0x7fe8180b0060 2021-07-11T03:53:33.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8157fa700 1 -- 172.21.2.104:0/2095878666 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe800011b80 con 0x7fe8180b0060 2021-07-11T03:53:33.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe81ea63700 1 -- 172.21.2.104:0/2095878666 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8181598d0 con 0x7fe8180b0060 2021-07-11T03:53:33.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe81ea63700 1 -- 172.21.2.104:0/2095878666 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8180aca50 con 0x7fe8180b0060 2021-07-11T03:53:33.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe81ea63700 1 -- 172.21.2.104:0/2095878666 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8180ab4f0 con 0x7fe8180b0060 2021-07-11T03:53:33.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8157fa700 1 -- 172.21.2.104:0/2095878666 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe80000f070 con 0x7fe8180b0060 2021-07-11T03:53:33.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8157fa700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe7fc0532d0 0x7fe7fc055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:33.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.836+0000 7fe8157fa700 1 -- 172.21.2.104:0/2095878666 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe80006ca40 con 0x7fe8180b0060 2021-07-11T03:53:33.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.840+0000 7fe817fff700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe7fc0532d0 0x7fe7fc055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:33.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.840+0000 7fe817fff700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe7fc0532d0 0x7fe7fc055790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fe808008000 tx=0x7fe808008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:33.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.840+0000 7fe8157fa700 1 -- 172.21.2.104:0/2095878666 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe800043c10 con 0x7fe8180b0060 2021-07-11T03:53:33.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.920+0000 7fe81ea63700 1 -- 172.21.2.104:0/2095878666 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe818057350 con 0x7fe8180b0060 2021-07-11T03:53:33.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.920+0000 7fe8157fa700 1 -- 172.21.2.104:0/2095878666 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16038 (secure 0 0 0) 0x7fe800047230 con 0x7fe8180b0060 2021-07-11T03:53:33.926 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:33.926 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":23,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:24.367517+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:33.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 -- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe7fc0532d0 msgr2=0x7fe7fc055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:33.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe7fc0532d0 0x7fe7fc055790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fe808008000 tx=0x7fe808008040).stop 2021-07-11T03:53:33.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 -- 172.21.2.104:0/2095878666 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8180b0060 msgr2=0x7fe8180aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:33.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8180b0060 0x7fe8180aeea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fe800002f70 tx=0x7fe800015000).stop 2021-07-11T03:53:33.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 -- 172.21.2.104:0/2095878666 shutdown_connections 2021-07-11T03:53:33.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8180b1970 0x7fe8180a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:33.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe7fc0532d0 0x7fe7fc055790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:33.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8180b0060 0x7fe8180aeea0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:33.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 --2- 172.21.2.104:0/2095878666 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8180a76b0 0x7fe8180b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:33.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 -- 172.21.2.104:0/2095878666 >> 172.21.2.104:0/2095878666 conn(0x7fe8180b53e0 msgr2=0x7fe81805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:33.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 -- 172.21.2.104:0/2095878666 shutdown_connections 2021-07-11T03:53:33.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.924+0000 7fe81ea63700 1 -- 172.21.2.104:0/2095878666 wait complete. 2021-07-11T03:53:33.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:53:20.980780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:53:21.341375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:53:33.934 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:53:33.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff171e66700 1 Processor -- start 2021-07-11T03:53:33.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff171e66700 1 -- start start 2021-07-11T03:53:33.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff171e66700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff16c0b1ec0 0x7ff16c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:33.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff171e66700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff16c0a77d0 con 0x7ff16c0b22a0 2021-07-11T03:53:33.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff171e66700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff16c0b03f0 con 0x7ff16c0affb0 2021-07-11T03:53:33.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff171e66700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0b0650 con 0x7ff16c0b1ec0 2021-07-11T03:53:33.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16bfff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff16c0b22a0 0x7ff16c141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39142/0 (socket says 172.21.2.104:39142) 2021-07-11T03:53:33.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16bfff700 1 -- 172.21.2.104:0/722257180 learned_addr learned my addr 172.21.2.104:0/722257180 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:33.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16b7fe700 1 --2- 172.21.2.104:0/722257180 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff16c0b1ec0 0x7ff16c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:33.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276340012 0 0) 0x7ff16c0a77d0 con 0x7ff16c0b22a0 2021-07-11T03:53:33.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff14c003260 con 0x7ff16c0b22a0 2021-07-11T03:53:33.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1520971752 0 0) 0x7ff14c003260 con 0x7ff16c0b22a0 2021-07-11T03:53:33.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff16c0b1ec0 msgr2=0x7ff16c0b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:33.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 --2- 172.21.2.104:0/722257180 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff16c0b1ec0 0x7ff16c0b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:33.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 >> v1:172.21.2.105:6789/0 conn(0x7ff16c0affb0 legacy=0x7ff16c0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:33.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c0b08f0 con 0x7ff16c0b22a0 2021-07-11T03:53:33.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7ff1640019e0 con 0x7ff16c0b22a0 2021-07-11T03:53:33.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 >> v1:172.21.2.104:6789/0 conn(0x7ff16c0b22a0 legacy=0x7ff16c141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:33.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 --2- 172.21.2.104:0/722257180 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff14c003780 0x7ff14c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:33.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 --2- 172.21.2.104:0/722257180 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff14c004500 0x7ff14c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:33.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 --2- 172.21.2.104:0/722257180 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff14c004e70 0x7ff14c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:33.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0b0650 con 0x7ff14c004e70 2021-07-11T03:53:33.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c00bcf0 con 0x7ff14c003780 2021-07-11T03:53:33.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c00be80 con 0x7ff14c004500 2021-07-11T03:53:33.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16b7fe700 1 --2- 172.21.2.104:0/722257180 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff16c0b1ec0 0x7ff16c0b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:53:33.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16bfff700 1 --2- 172.21.2.104:0/722257180 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff14c004e70 0x7ff14c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:33.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16bfff700 1 -- 172.21.2.104:0/722257180 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff14c004500 msgr2=0x7ff14c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:33.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16bfff700 1 --2- 172.21.2.104:0/722257180 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff14c004500 0x7ff14c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:33.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16bfff700 1 -- 172.21.2.104:0/722257180 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff14c003780 msgr2=0x7ff14c003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:33.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16bfff700 1 --2- 172.21.2.104:0/722257180 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff14c003780 0x7ff14c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:33.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16bfff700 1 -- 172.21.2.104:0/722257180 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c0b08f0 con 0x7ff14c004e70 2021-07-11T03:53:33.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16bfff700 1 --2- 172.21.2.104:0/722257180 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff14c004e70 0x7ff14c00b750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7ff14c0054d0 tx=0x7ff1640079b0).ready entity=mon.0 client_cookie=b13dd141ddf60613 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:33.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff16400cd90 con 0x7ff14c004e70 2021-07-11T03:53:33.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff164005990 con 0x7ff14c004e70 2021-07-11T03:53:33.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff16a7fc700 1 -- 172.21.2.104:0/722257180 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff164005cf0 con 0x7ff14c004e70 2021-07-11T03:53:33.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff171e66700 1 -- 172.21.2.104:0/722257180 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff14c004e70 msgr2=0x7ff14c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.984+0000 7ff171e66700 1 --2- 172.21.2.104:0/722257180 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff14c004e70 0x7ff14c00b750 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7ff14c0054d0 tx=0x7ff1640079b0).stop 2021-07-11T03:53:34.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff16bfff700 1 -- 172.21.2.104:0/722257180 reap_dead start 2021-07-11T03:53:34.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 -- 172.21.2.104:0/722257180 shutdown_connections 2021-07-11T03:53:34.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 -- 172.21.2.104:0/722257180 >> 172.21.2.104:0/722257180 conn(0x7ff16c05f6b0 msgr2=0x7ff16c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:34.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 -- 172.21.2.104:0/722257180 shutdown_connections 2021-07-11T03:53:34.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 -- 172.21.2.104:0/722257180 wait complete. 2021-07-11T03:53:34.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 Processor -- start 2021-07-11T03:53:34.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 -- start start 2021-07-11T03:53:34.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff16c140120 0x7ff16c148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff16c157390 0x7ff16c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff16c1577f0 0x7ff16c15a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0aae80 con 0x7ff16c140120 2021-07-11T03:53:34.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c156a60 con 0x7ff16c1577f0 2021-07-11T03:53:34.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c140500 con 0x7ff16c157390 2021-07-11T03:53:34.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff16b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff16c140120 0x7ff16c148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff16b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff16c140120 0x7ff16c148b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42386/0 (socket says 172.21.2.104:42386) 2021-07-11T03:53:34.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff16b7fe700 1 -- 172.21.2.104:0/957714982 learned_addr learned my addr 172.21.2.104:0/957714982 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:34.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff16b7fe700 1 -- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff16c157390 msgr2=0x7ff16c159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff16b7fe700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff16c157390 0x7ff16c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff16b7fe700 1 -- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff16c1577f0 msgr2=0x7ff16c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff16b7fe700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff16c1577f0 0x7ff16c15a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff16b7fe700 1 -- 172.21.2.104:0/957714982 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c149160 con 0x7ff16c140120 2021-07-11T03:53:34.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff16b7fe700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff16c140120 0x7ff16c148b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7ff15400a000 tx=0x7ff154008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff168ff9700 1 -- 172.21.2.104:0/957714982 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1540106e0 con 0x7ff16c140120 2021-07-11T03:53:34.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff168ff9700 1 -- 172.21.2.104:0/957714982 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff154000e90 con 0x7ff16c140120 2021-07-11T03:53:34.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 -- 172.21.2.104:0/957714982 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c156c50 con 0x7ff16c140120 2021-07-11T03:53:34.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff168ff9700 1 -- 172.21.2.104:0/957714982 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff154010a30 con 0x7ff16c140120 2021-07-11T03:53:34.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 -- 172.21.2.104:0/957714982 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c1474d0 con 0x7ff16c140120 2021-07-11T03:53:34.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff171e66700 1 -- 172.21.2.104:0/957714982 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16c1492f0 con 0x7ff16c140120 2021-07-11T03:53:34.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff168ff9700 1 -- 172.21.2.104:0/957714982 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff154010b90 con 0x7ff16c140120 2021-07-11T03:53:34.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff168ff9700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1500531f0 0x7ff1500556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.988+0000 7ff168ff9700 1 -- 172.21.2.104:0/957714982 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff15406b390 con 0x7ff16c140120 2021-07-11T03:53:34.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.992+0000 7ff16affd700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1500531f0 0x7ff1500556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.992+0000 7ff16affd700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1500531f0 0x7ff1500556b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7ff15c001fa0 tx=0x7ff15c0077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:33.992+0000 7ff168ff9700 1 -- 172.21.2.104:0/957714982 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff1540426d0 con 0x7ff16c140120 2021-07-11T03:53:34.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.072+0000 7ff171e66700 1 -- 172.21.2.104:0/957714982 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff16c057350 con 0x7ff16c140120 2021-07-11T03:53:34.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.072+0000 7ff168ff9700 1 -- 172.21.2.104:0/957714982 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff154045cf0 con 0x7ff16c140120 2021-07-11T03:53:34.077 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:53:34.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 -- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1500531f0 msgr2=0x7ff1500556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1500531f0 0x7ff1500556b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7ff15c001fa0 tx=0x7ff15c0077f0).stop 2021-07-11T03:53:34.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 -- 172.21.2.104:0/957714982 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff16c140120 msgr2=0x7ff16c148b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff16c140120 0x7ff16c148b30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7ff15400a000 tx=0x7ff154008040).stop 2021-07-11T03:53:34.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 -- 172.21.2.104:0/957714982 shutdown_connections 2021-07-11T03:53:34.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff16c1577f0 0x7ff16c15a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff16c157390 0x7ff16c159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1500531f0 0x7ff1500556b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 --2- 172.21.2.104:0/957714982 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff16c140120 0x7ff16c148b30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 -- 172.21.2.104:0/957714982 >> 172.21.2.104:0/957714982 conn(0x7ff16c05f6b0 msgr2=0x7ff16c0a70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:34.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 -- 172.21.2.104:0/957714982 shutdown_connections 2021-07-11T03:53:34.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.076+0000 7ff171e66700 1 -- 172.21.2.104:0/957714982 wait complete. 2021-07-11T03:53:34.086 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:53:34.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 Processor -- start 2021-07-11T03:53:34.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 -- start start 2021-07-11T03:53:34.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff6040b3710 0x7ff6040b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff6040a9c10 0x7ff6040b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff604143920 con 0x7ff6040b05b0 2021-07-11T03:53:34.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff604143bc0 con 0x7ff6040b3710 2021-07-11T03:53:34.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff604143e60 con 0x7ff6040a9c10 2021-07-11T03:53:34.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60a4da700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff6040b05b0 0x7ff60413d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39164/0 (socket says 172.21.2.104:39164) 2021-07-11T03:53:34.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60a4da700 1 -- 172.21.2.104:0/521310831 learned_addr learned my addr 172.21.2.104:0/521310831 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:34.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff609cd9700 1 --2- 172.21.2.104:0/521310831 >> v2:172.21.2.104:3300/0 conn(0x7ff6040b3710 0x7ff6040b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60acdb700 1 --2- 172.21.2.104:0/521310831 >> v2:172.21.2.105:3300/0 conn(0x7ff6040a9c10 0x7ff6040b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff609cd9700 1 -- 172.21.2.104:0/521310831 >> v2:172.21.2.105:3300/0 conn(0x7ff6040a9c10 msgr2=0x7ff6040b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:34.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff609cd9700 1 --2- 172.21.2.104:0/521310831 >> v2:172.21.2.105:3300/0 conn(0x7ff6040a9c10 0x7ff6040b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff609cd9700 1 -- 172.21.2.104:0/521310831 >> v1:172.21.2.104:6789/0 conn(0x7ff6040b05b0 legacy=0x7ff60413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff609cd9700 1 -- 172.21.2.104:0/521310831 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff604144100 con 0x7ff6040b3710 2021-07-11T03:53:34.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff609cd9700 1 --2- 172.21.2.104:0/521310831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6040b3710 0x7ff6040b0990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7ff5fc0032f0 tx=0x7ff5fc00b470).ready entity=mon.0 client_cookie=f890b46b662afaad server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff6094d8700 1 -- 172.21.2.104:0/521310831 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5fc00ddf0 con 0x7ff6040b3710 2021-07-11T03:53:34.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff6094d8700 1 -- 172.21.2.104:0/521310831 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc010470 con 0x7ff6040b3710 2021-07-11T03:53:34.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff6094d8700 1 -- 172.21.2.104:0/521310831 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff5fc0107d0 con 0x7ff6040b3710 2021-07-11T03:53:34.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 -- 172.21.2.104:0/521310831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6040b3710 msgr2=0x7ff6040b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 --2- 172.21.2.104:0/521310831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6040b3710 0x7ff6040b0990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7ff5fc0032f0 tx=0x7ff5fc00b470).stop 2021-07-11T03:53:34.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 -- 172.21.2.104:0/521310831 shutdown_connections 2021-07-11T03:53:34.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 --2- 172.21.2.104:0/521310831 >> v2:172.21.2.105:3300/0 conn(0x7ff6040a9c10 0x7ff6040b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 --2- 172.21.2.104:0/521310831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6040b3710 0x7ff6040b0990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 -- 172.21.2.104:0/521310831 >> 172.21.2.104:0/521310831 conn(0x7ff6040607c0 msgr2=0x7ff604060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:34.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 -- 172.21.2.104:0/521310831 shutdown_connections 2021-07-11T03:53:34.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.136+0000 7ff60c73e700 1 -- 172.21.2.104:0/521310831 wait complete. 2021-07-11T03:53:34.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60c73e700 1 Processor -- start 2021-07-11T03:53:34.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60c73e700 1 -- start start 2021-07-11T03:53:34.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60c73e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6040a9c10 0x7ff6041403c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60c73e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6040b05b0 0x7ff604063c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60c73e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6040b3710 0x7ff604065620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60c73e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60413fa10 con 0x7ff6040b3710 2021-07-11T03:53:34.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60c73e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60413fcb0 con 0x7ff6040a9c10 2021-07-11T03:53:34.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60c73e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60413ff50 con 0x7ff6040b05b0 2021-07-11T03:53:34.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60acdb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6040b3710 0x7ff604065620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60acdb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6040b3710 0x7ff604065620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42402/0 (socket says 172.21.2.104:42402) 2021-07-11T03:53:34.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60acdb700 1 -- 172.21.2.104:0/2335250352 learned_addr learned my addr 172.21.2.104:0/2335250352 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:34.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff609cd9700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6040b05b0 0x7ff604063c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60acdb700 1 -- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6040b05b0 msgr2=0x7ff604063c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60acdb700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6040b05b0 0x7ff604063c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60acdb700 1 -- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6040a9c10 msgr2=0x7ff6041403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60a4da700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6040a9c10 0x7ff6041403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60acdb700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6040a9c10 0x7ff6041403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60acdb700 1 -- 172.21.2.104:0/2335250352 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5fc00b710 con 0x7ff6040b3710 2021-07-11T03:53:34.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff609cd9700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6040b05b0 0x7ff604063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:34.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60a4da700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6040a9c10 0x7ff6041403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:34.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60acdb700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6040b3710 0x7ff604065620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ff600007000 tx=0x7ff600008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff5f37fe700 1 -- 172.21.2.104:0/2335250352 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff600010580 con 0x7ff6040b3710 2021-07-11T03:53:34.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff5f37fe700 1 -- 172.21.2.104:0/2335250352 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6000106e0 con 0x7ff6040b3710 2021-07-11T03:53:34.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60c73e700 1 -- 172.21.2.104:0/2335250352 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60413f100 con 0x7ff6040b3710 2021-07-11T03:53:34.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60c73e700 1 -- 172.21.2.104:0/2335250352 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60413f5a0 con 0x7ff6040b3710 2021-07-11T03:53:34.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff5f37fe700 1 -- 172.21.2.104:0/2335250352 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff600010a40 con 0x7ff6040b3710 2021-07-11T03:53:34.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60c73e700 1 -- 172.21.2.104:0/2335250352 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff604140d40 con 0x7ff6040b3710 2021-07-11T03:53:34.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff5f37fe700 1 -- 172.21.2.104:0/2335250352 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff600010ba0 con 0x7ff6040b3710 2021-07-11T03:53:34.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff5f37fe700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff5e40532d0 0x7ff5e4055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff5f37fe700 1 -- 172.21.2.104:0/2335250352 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff60006b3a0 con 0x7ff6040b3710 2021-07-11T03:53:34.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60a4da700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff5e40532d0 0x7ff5e4055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.140+0000 7ff60a4da700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff5e40532d0 0x7ff5e4055790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ff5f4008000 tx=0x7ff5f4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.144+0000 7ff5f37fe700 1 -- 172.21.2.104:0/2335250352 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff604140d40 con 0x7ff6040b3710 2021-07-11T03:53:34.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.224+0000 7ff60c73e700 1 -- 172.21.2.104:0/2335250352 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff604140d40 con 0x7ff6040b3710 2021-07-11T03:53:34.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff5f37fe700 1 -- 172.21.2.104:0/2335250352 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff604140d40 con 0x7ff6040b3710 2021-07-11T03:53:34.230 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:34.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 -- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff5e40532d0 msgr2=0x7ff5e4055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff5e40532d0 0x7ff5e4055790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ff5f4008000 tx=0x7ff5f4009040).stop 2021-07-11T03:53:34.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 -- 172.21.2.104:0/2335250352 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6040b3710 msgr2=0x7ff604065620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6040b3710 0x7ff604065620 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7ff600007000 tx=0x7ff600008040).stop 2021-07-11T03:53:34.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 -- 172.21.2.104:0/2335250352 shutdown_connections 2021-07-11T03:53:34.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff5e40532d0 0x7ff5e4055790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6040b3710 0x7ff604065620 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6040b05b0 0x7ff604063c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 --2- 172.21.2.104:0/2335250352 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6040a9c10 0x7ff6041403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 -- 172.21.2.104:0/2335250352 >> 172.21.2.104:0/2335250352 conn(0x7ff6040607c0 msgr2=0x7ff6040b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:34.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 -- 172.21.2.104:0/2335250352 shutdown_connections 2021-07-11T03:53:34.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.228+0000 7ff60c73e700 1 -- 172.21.2.104:0/2335250352 wait complete. 2021-07-11T03:53:34.238 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:53:34.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b644d8700 1 Processor -- start 2021-07-11T03:53:34.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b644d8700 1 -- start start 2021-07-11T03:53:34.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b644d8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b5c0b1ec0 0x7f5b5c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b644d8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5b5c0affb0 0x7f5b5c0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b644d8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5b5c0a70c0 0x7f5b5c141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b644d8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5c0b2790 con 0x7f5b5c0a70c0 2021-07-11T03:53:34.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b644d8700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b5c0b03f0 con 0x7f5b5c0affb0 2021-07-11T03:53:34.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b644d8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0b0650 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b62a75700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5b5c0a70c0 0x7f5b5c141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b62a75700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5b5c0a70c0 0x7f5b5c141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42410/0 (socket says 172.21.2.104:42410) 2021-07-11T03:53:34.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b62a75700 1 -- 172.21.2.104:0/3023757780 learned_addr learned my addr 172.21.2.104:0/3023757780 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:34.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b62a75700 1 -- 172.21.2.104:0/3023757780 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b5c0b1ec0 msgr2=0x7f5b5c0b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:34.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b62a75700 1 --2- 172.21.2.104:0/3023757780 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b5c0b1ec0 0x7f5b5c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b62a75700 1 -- 172.21.2.104:0/3023757780 >> v2:172.21.2.105:3300/0 conn(0x7f5b5c0affb0 msgr2=0x7f5b5c0b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:34.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b62a75700 1 --2- 172.21.2.104:0/3023757780 >> v2:172.21.2.105:3300/0 conn(0x7f5b5c0affb0 0x7f5b5c0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b62a75700 1 -- 172.21.2.104:0/3023757780 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c0b08f0 con 0x7f5b5c0a70c0 2021-07-11T03:53:34.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b62a75700 1 --2- 172.21.2.104:0/3023757780 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b5c0a70c0 0x7f5b5c141250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5b540032f0 tx=0x7f5b54009290).ready entity=mon.0 client_cookie=1716729b29230cc9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b61272700 1 -- 172.21.2.104:0/3023757780 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b5400b870 con 0x7f5b5c0a70c0 2021-07-11T03:53:34.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b61272700 1 -- 172.21.2.104:0/3023757780 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b5400b9d0 con 0x7f5b5c0a70c0 2021-07-11T03:53:34.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b61272700 1 -- 172.21.2.104:0/3023757780 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b5400bb40 con 0x7f5b5c0a70c0 2021-07-11T03:53:34.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b644d8700 1 -- 172.21.2.104:0/3023757780 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b5c0a70c0 msgr2=0x7f5b5c141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.288+0000 7f5b644d8700 1 --2- 172.21.2.104:0/3023757780 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b5c0a70c0 0x7f5b5c141250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f5b540032f0 tx=0x7f5b54009290).stop 2021-07-11T03:53:34.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 -- 172.21.2.104:0/3023757780 shutdown_connections 2021-07-11T03:53:34.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 --2- 172.21.2.104:0/3023757780 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b5c0a70c0 0x7f5b5c141250 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 --2- 172.21.2.104:0/3023757780 >> v2:172.21.2.105:3300/0 conn(0x7f5b5c0affb0 0x7f5b5c0b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 --2- 172.21.2.104:0/3023757780 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b5c0b1ec0 0x7f5b5c0b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 -- 172.21.2.104:0/3023757780 >> 172.21.2.104:0/3023757780 conn(0x7f5b5c05f6b0 msgr2=0x7f5b5c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:34.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 -- 172.21.2.104:0/3023757780 shutdown_connections 2021-07-11T03:53:34.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 -- 172.21.2.104:0/3023757780 wait complete. 2021-07-11T03:53:34.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 Processor -- start 2021-07-11T03:53:34.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 -- start start 2021-07-11T03:53:34.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b5c0a70c0 0x7f5b5c0adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b5c0affb0 0x7f5b5c0aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b5c0b1ec0 0x7f5b5c0aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0b4ca0 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0acc10 con 0x7f5b5c0a70c0 2021-07-11T03:53:34.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c0ab730 con 0x7f5b5c0affb0 2021-07-11T03:53:34.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b62a75700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b5c0b1ec0 0x7f5b5c0aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b62a75700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b5c0b1ec0 0x7f5b5c0aacd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42416/0 (socket says 172.21.2.104:42416) 2021-07-11T03:53:34.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b62a75700 1 -- 172.21.2.104:0/2657731279 learned_addr learned my addr 172.21.2.104:0/2657731279 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:34.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b61a73700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b5c0affb0 0x7f5b5c0aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b62a75700 1 -- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b5c0affb0 msgr2=0x7f5b5c0aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b62a75700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b5c0affb0 0x7f5b5c0aee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b62a75700 1 -- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b5c0a70c0 msgr2=0x7f5b5c0adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b62a75700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b5c0a70c0 0x7f5b5c0adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b62a75700 1 -- 172.21.2.104:0/2657731279 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b54009530 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b62a75700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b5c0b1ec0 0x7f5b5c0aacd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f5b5c0a9bc0 tx=0x7f5b54016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b4f7fe700 1 -- 172.21.2.104:0/2657731279 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b54011c70 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b4f7fe700 1 -- 172.21.2.104:0/2657731279 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b54011dd0 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 -- 172.21.2.104:0/2657731279 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c0ad5b0 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b4f7fe700 1 -- 172.21.2.104:0/2657731279 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b54020620 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 -- 172.21.2.104:0/2657731279 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c0b5090 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b644d8700 1 -- 172.21.2.104:0/2657731279 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c057350 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b4f7fe700 1 -- 172.21.2.104:0/2657731279 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5b54028070 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b4f7fe700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b3c0532b0 0x7f5b3c055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.292+0000 7f5b4f7fe700 1 -- 172.21.2.104:0/2657731279 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5b5406bda0 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.296+0000 7f5b62274700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b3c0532b0 0x7f5b3c055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.296+0000 7f5b62274700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b3c0532b0 0x7f5b3c055770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f5b58002f70 tx=0x7f5b58006990).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.296+0000 7f5b4f7fe700 1 -- 172.21.2.104:0/2657731279 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5b540430e0 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.376+0000 7f5b644d8700 1 -- 172.21.2.104:0/2657731279 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b5c0ac760 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.376+0000 7f5b4f7fe700 1 -- 172.21.2.104:0/2657731279 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f5b540150a0 con 0x7f5b5c0b1ec0 2021-07-11T03:53:34.382 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:34.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 -- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b3c0532b0 msgr2=0x7f5b3c055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b3c0532b0 0x7f5b3c055770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f5b58002f70 tx=0x7f5b58006990).stop 2021-07-11T03:53:34.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 -- 172.21.2.104:0/2657731279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b5c0b1ec0 msgr2=0x7f5b5c0aacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b5c0b1ec0 0x7f5b5c0aacd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f5b5c0a9bc0 tx=0x7f5b54016000).stop 2021-07-11T03:53:34.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 -- 172.21.2.104:0/2657731279 shutdown_connections 2021-07-11T03:53:34.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b3c0532b0 0x7f5b3c055770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b5c0b1ec0 0x7f5b5c0aacd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b5c0affb0 0x7f5b5c0aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 --2- 172.21.2.104:0/2657731279 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b5c0a70c0 0x7f5b5c0adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 -- 172.21.2.104:0/2657731279 >> 172.21.2.104:0/2657731279 conn(0x7f5b5c05f6b0 msgr2=0x7f5b5c13e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:34.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 -- 172.21.2.104:0/2657731279 shutdown_connections 2021-07-11T03:53:34.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.380+0000 7f5b644d8700 1 -- 172.21.2.104:0/2657731279 wait complete. 2021-07-11T03:53:34.390 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:53:34.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 Processor -- start 2021-07-11T03:53:34.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- start start 2021-07-11T03:53:34.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd734060100 0x7fd734064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd734144440 0x7fd734065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7340604e0 con 0x7fd7340b38d0 2021-07-11T03:53:34.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7340656f0 con 0x7fd734060100 2021-07-11T03:53:34.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd734065c80 con 0x7fd734144440 2021-07-11T03:53:34.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd733fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd7340b38d0 0x7fd734144970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39192/0 (socket says 172.21.2.104:39192) 2021-07-11T03:53:34.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd733fff700 1 -- 172.21.2.104:0/2571013432 learned_addr learned my addr 172.21.2.104:0/2571013432 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:34.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd7337fe700 1 --2- 172.21.2.104:0/2571013432 >> v2:172.21.2.104:3300/0 conn(0x7fd734060100 0x7fd734064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd7337fe700 1 -- 172.21.2.104:0/2571013432 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd734144440 msgr2=0x7fd734065200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:34.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd7337fe700 1 --2- 172.21.2.104:0/2571013432 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd734144440 0x7fd734065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd7337fe700 1 -- 172.21.2.104:0/2571013432 >> v1:172.21.2.104:6789/0 conn(0x7fd7340b38d0 legacy=0x7fd734144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd7337fe700 1 -- 172.21.2.104:0/2571013432 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7340b23b0 con 0x7fd734060100 2021-07-11T03:53:34.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd7337fe700 1 --2- 172.21.2.104:0/2571013432 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd734060100 0x7fd734064d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fd71c002f70 tx=0x7fd71c006df0).ready entity=mon.0 client_cookie=5c9a6a3448fcf643 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd7327fc700 1 -- 172.21.2.104:0/2571013432 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd71c00d570 con 0x7fd734060100 2021-07-11T03:53:34.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd7327fc700 1 -- 172.21.2.104:0/2571013432 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c000e90 con 0x7fd734060100 2021-07-11T03:53:34.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- 172.21.2.104:0/2571013432 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd734060100 msgr2=0x7fd734064d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 --2- 172.21.2.104:0/2571013432 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd734060100 0x7fd734064d10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fd71c002f70 tx=0x7fd71c006df0).stop 2021-07-11T03:53:34.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd7327fc700 1 -- 172.21.2.104:0/2571013432 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd71c00d8c0 con 0x7fd734060100 2021-07-11T03:53:34.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- 172.21.2.104:0/2571013432 shutdown_connections 2021-07-11T03:53:34.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 --2- 172.21.2.104:0/2571013432 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd734144440 0x7fd734065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 --2- 172.21.2.104:0/2571013432 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd734060100 0x7fd734064d10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- 172.21.2.104:0/2571013432 >> 172.21.2.104:0/2571013432 conn(0x7fd73405cf60 msgr2=0x7fd73405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:34.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- 172.21.2.104:0/2571013432 shutdown_connections 2021-07-11T03:53:34.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- 172.21.2.104:0/2571013432 wait complete. 2021-07-11T03:53:34.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 Processor -- start 2021-07-11T03:53:34.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- start start 2021-07-11T03:53:34.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd734060100 0x7fd734130450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7340b38d0 0x7fd73413a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd73413aba0 0x7fd7340b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7340616f0 con 0x7fd7340b38d0 2021-07-11T03:53:34.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd734061990 con 0x7fd734060100 2021-07-11T03:53:34.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd739ad5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd734061c30 con 0x7fd73413aba0 2021-07-11T03:53:34.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd732ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7340b38d0 0x7fd73413a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd732ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7340b38d0 0x7fd73413a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42428/0 (socket says 172.21.2.104:42428) 2021-07-11T03:53:34.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd732ffd700 1 -- 172.21.2.104:0/2882910068 learned_addr learned my addr 172.21.2.104:0/2882910068 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:34.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd732ffd700 1 -- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd73413aba0 msgr2=0x7fd7340b54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd732ffd700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd73413aba0 0x7fd7340b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd732ffd700 1 -- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd734060100 msgr2=0x7fd734130450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd732ffd700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd734060100 0x7fd734130450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.440+0000 7fd732ffd700 1 -- 172.21.2.104:0/2882910068 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c00b180 con 0x7fd7340b38d0 2021-07-11T03:53:34.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd732ffd700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7340b38d0 0x7fd73413a6b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fd724002f70 tx=0x7fd724006e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd730ff9700 1 -- 172.21.2.104:0/2882910068 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72400e470 con 0x7fd7340b38d0 2021-07-11T03:53:34.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd730ff9700 1 -- 172.21.2.104:0/2882910068 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72400e5d0 con 0x7fd7340b38d0 2021-07-11T03:53:34.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd739ad5700 1 -- 172.21.2.104:0/2882910068 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73405f9f0 con 0x7fd7340b38d0 2021-07-11T03:53:34.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd730ff9700 1 -- 172.21.2.104:0/2882910068 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd72400e930 con 0x7fd7340b38d0 2021-07-11T03:53:34.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd739ad5700 1 -- 172.21.2.104:0/2882910068 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73413b050 con 0x7fd7340b38d0 2021-07-11T03:53:34.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd739ad5700 1 -- 172.21.2.104:0/2882910068 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd734061df0 con 0x7fd7340b38d0 2021-07-11T03:53:34.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd730ff9700 1 -- 172.21.2.104:0/2882910068 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd724018030 con 0x7fd7340b38d0 2021-07-11T03:53:34.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd730ff9700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd718053330 0x7fd7180557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd730ff9700 1 -- 172.21.2.104:0/2882910068 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd72406a960 con 0x7fd7340b38d0 2021-07-11T03:53:34.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd7337fe700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd718053330 0x7fd7180557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd7337fe700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd718053330 0x7fd7180557f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fd71c002f70 tx=0x7fd71c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.444+0000 7fd730ff9700 1 -- 172.21.2.104:0/2882910068 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd724041d50 con 0x7fd7340b38d0 2021-07-11T03:53:34.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.528+0000 7fd739ad5700 1 -- 172.21.2.104:0/2882910068 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd734057350 con 0x7fd7340b38d0 2021-07-11T03:53:34.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.528+0000 7fd730ff9700 1 -- 172.21.2.104:0/2882910068 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd724045370 con 0x7fd7340b38d0 2021-07-11T03:53:34.534 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:34.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 -- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd718053330 msgr2=0x7fd7180557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd718053330 0x7fd7180557f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fd71c002f70 tx=0x7fd71c015000).stop 2021-07-11T03:53:34.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 -- 172.21.2.104:0/2882910068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7340b38d0 msgr2=0x7fd73413a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7340b38d0 0x7fd73413a6b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fd724002f70 tx=0x7fd724006e50).stop 2021-07-11T03:53:34.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 -- 172.21.2.104:0/2882910068 shutdown_connections 2021-07-11T03:53:34.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd73413aba0 0x7fd7340b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd718053330 0x7fd7180557f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7340b38d0 0x7fd73413a6b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 --2- 172.21.2.104:0/2882910068 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd734060100 0x7fd734130450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 -- 172.21.2.104:0/2882910068 >> 172.21.2.104:0/2882910068 conn(0x7fd73405cf60 msgr2=0x7fd7340b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:34.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 -- 172.21.2.104:0/2882910068 shutdown_connections 2021-07-11T03:53:34.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.532+0000 7fd739ad5700 1 -- 172.21.2.104:0/2882910068 wait complete. 2021-07-11T03:53:34.541 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T03:53:34.542 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:53:34.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd929d700 1 Processor -- start 2021-07-11T03:53:34.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd929d700 1 -- start start 2021-07-11T03:53:34.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd929d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f4cd40b1970 0x7f4cd405fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd929d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd40a7a90 con 0x7f4cd4067180 2021-07-11T03:53:34.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd929d700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd4147620 con 0x7f4cd40a76b0 2021-07-11T03:53:34.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd929d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd41478c0 con 0x7f4cd40b1970 2021-07-11T03:53:34.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd27fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4cd4067180 0x7f4cd405f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39206/0 (socket says 172.21.2.104:39206) 2021-07-11T03:53:34.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd27fc700 1 -- 172.21.2.104:0/648654030 learned_addr learned my addr 172.21.2.104:0/648654030 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:34.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806853254 0 0) 0x7f4cd40a7a90 con 0x7f4cd4067180 2021-07-11T03:53:34.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cb4003260 con 0x7f4cd4067180 2021-07-11T03:53:34.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd37fe700 1 --2- 172.21.2.104:0/648654030 >> v2:172.21.2.105:3300/0 conn(0x7f4cd40b1970 0x7f4cd405fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2013899656 0 0) 0x7f4cb4003260 con 0x7f4cd4067180 2021-07-11T03:53:34.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 >> v2:172.21.2.105:3300/0 conn(0x7f4cd40b1970 msgr2=0x7f4cd405fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:34.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 --2- 172.21.2.104:0/648654030 >> v2:172.21.2.105:3300/0 conn(0x7f4cd40b1970 0x7f4cd405fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 >> v1:172.21.2.105:6789/0 conn(0x7f4cd40a76b0 legacy=0x7f4cd4066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd4147b60 con 0x7f4cd4067180 2021-07-11T03:53:34.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f4cbc001660 con 0x7f4cd4067180 2021-07-11T03:53:34.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 >> v1:172.21.2.104:6789/0 conn(0x7f4cd4067180 legacy=0x7f4cd405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 --2- 172.21.2.104:0/648654030 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4cb40037a0 0x7f4cb4003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 --2- 172.21.2.104:0/648654030 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4cb4004500 0x7f4cb4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 --2- 172.21.2.104:0/648654030 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 0x7f4cb400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd41478c0 con 0x7f4cb4004f90 2021-07-11T03:53:34.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb400be00 con 0x7f4cb4004500 2021-07-11T03:53:34.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb400bfc0 con 0x7f4cb40037a0 2021-07-11T03:53:34.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd37fe700 1 --2- 172.21.2.104:0/648654030 >> v2:172.21.2.105:3300/0 conn(0x7f4cd40b1970 0x7f4cd405fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:53:34.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd37fe700 1 --2- 172.21.2.104:0/648654030 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 0x7f4cb400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd2ffd700 1 --2- 172.21.2.104:0/648654030 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4cb40037a0 0x7f4cb4003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd27fc700 1 --2- 172.21.2.104:0/648654030 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4cb4004500 0x7f4cb4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd37fe700 1 -- 172.21.2.104:0/648654030 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4cb40037a0 msgr2=0x7f4cb4003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:34.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd37fe700 1 --2- 172.21.2.104:0/648654030 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4cb40037a0 0x7f4cb4003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd37fe700 1 -- 172.21.2.104:0/648654030 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4cb4004500 msgr2=0x7f4cb4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:34.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd37fe700 1 --2- 172.21.2.104:0/648654030 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4cb4004500 0x7f4cb4004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd37fe700 1 -- 172.21.2.104:0/648654030 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb4003260 con 0x7f4cb4004f90 2021-07-11T03:53:34.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd37fe700 1 -- 172.21.2.104:0/648654030 reap_dead start 2021-07-11T03:53:34.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd37fe700 1 --2- 172.21.2.104:0/648654030 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 0x7f4cb400b750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f4ccc00a000 tx=0x7f4ccc008040).ready entity=mon.0 client_cookie=11abe32190ea1ff6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ccc00c070 con 0x7f4cb4004f90 2021-07-11T03:53:34.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ccc00e620 con 0x7f4cb4004f90 2021-07-11T03:53:34.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd1ffb700 1 -- 172.21.2.104:0/648654030 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4ccc00e980 con 0x7f4cb4004f90 2021-07-11T03:53:34.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd929d700 1 -- 172.21.2.104:0/648654030 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 msgr2=0x7f4cb400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.592+0000 7f4cd929d700 1 --2- 172.21.2.104:0/648654030 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 0x7f4cb400b750 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f4ccc00a000 tx=0x7f4ccc008040).stop 2021-07-11T03:53:34.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 -- 172.21.2.104:0/648654030 shutdown_connections 2021-07-11T03:53:34.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 --2- 172.21.2.104:0/648654030 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 0x7f4cb400b750 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 -- 172.21.2.104:0/648654030 >> 172.21.2.104:0/648654030 conn(0x7f4cd40b53e0 msgr2=0x7f4cd40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:34.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 -- 172.21.2.104:0/648654030 shutdown_connections 2021-07-11T03:53:34.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 -- 172.21.2.104:0/648654030 wait complete. 2021-07-11T03:53:34.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 Processor -- start 2021-07-11T03:53:34.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 -- start start 2021-07-11T03:53:34.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 0x7f4cd40b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4cd40a9230 0x7f4cd40aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4cd40abb30 0x7f4cd415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd40ae270 con 0x7f4cb4004f90 2021-07-11T03:53:34.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd40ae510 con 0x7f4cd40a9230 2021-07-11T03:53:34.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd40ad6f0 con 0x7f4cd40abb30 2021-07-11T03:53:34.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 0x7f4cd40b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 0x7f4cd40b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42448/0 (socket says 172.21.2.104:42448) 2021-07-11T03:53:34.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd2ffd700 1 -- 172.21.2.104:0/1091753975 learned_addr learned my addr 172.21.2.104:0/1091753975 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:34.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd2ffd700 1 -- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4cd40abb30 msgr2=0x7f4cd415a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd2ffd700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4cd40abb30 0x7f4cd415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd2ffd700 1 -- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4cd40a9230 msgr2=0x7f4cd40aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd2ffd700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4cd40a9230 0x7f4cd40aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd2ffd700 1 -- 172.21.2.104:0/1091753975 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd40ad990 con 0x7f4cb4004f90 2021-07-11T03:53:34.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd2ffd700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 0x7f4cd40b4590 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f4cc400b000 tx=0x7f4cc4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4ccbfff700 1 -- 172.21.2.104:0/1091753975 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cc40155f0 con 0x7f4cb4004f90 2021-07-11T03:53:34.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4ccbfff700 1 -- 172.21.2.104:0/1091753975 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc4010040 con 0x7f4cb4004f90 2021-07-11T03:53:34.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 -- 172.21.2.104:0/1091753975 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd40adf60 con 0x7f4cb4004f90 2021-07-11T03:53:34.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4ccbfff700 1 -- 172.21.2.104:0/1091753975 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4cc4015940 con 0x7f4cb4004f90 2021-07-11T03:53:34.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 -- 172.21.2.104:0/1091753975 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd40ac240 con 0x7f4cb4004f90 2021-07-11T03:53:34.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd929d700 1 -- 172.21.2.104:0/1091753975 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd4057350 con 0x7f4cb4004f90 2021-07-11T03:53:34.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4ccbfff700 1 -- 172.21.2.104:0/1091753975 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4cc4015aa0 con 0x7f4cb4004f90 2021-07-11T03:53:34.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4ccbfff700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4cb8053270 0x7f4cb8055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:34.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4ccbfff700 1 -- 172.21.2.104:0/1091753975 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4cc406b670 con 0x7f4cb4004f90 2021-07-11T03:53:34.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.596+0000 7f4cd27fc700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4cb8053270 0x7f4cb8055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:34.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.600+0000 7f4cd27fc700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4cb8053270 0x7f4cb8055730 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f4cbc00b000 tx=0x7f4cd40a8c80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:34.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.600+0000 7f4ccbfff700 1 -- 172.21.2.104:0/1091753975 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4cc40429b0 con 0x7f4cb4004f90 2021-07-11T03:53:34.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 -- 172.21.2.104:0/1091753975 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cd40adb20 con 0x7f4cb4004f90 2021-07-11T03:53:34.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4ccbfff700 1 -- 172.21.2.104:0/1091753975 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2483 (secure 0 0 0) 0x7f4cc4045fd0 con 0x7f4cb4004f90 2021-07-11T03:53:34.715 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:34.715 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3943971217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3943971217},{"type":"v1","addr":"172.21.2.105:6835","nonce":3943971217}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/668148871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":668148871},{"type":"v1","addr":"172.21.2.104:6833","nonce":668148871}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":6}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:22.350923+0000","modified":"2021-07-11T03:53:23.355891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4224":{"gid":4224,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/1236541958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":1236541958},{"type":"v1","addr":"172.21.2.104:6835","nonce":1236541958}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-07-11T03:53:34.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 -- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4cb8053270 msgr2=0x7f4cb8055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4cb8053270 0x7f4cb8055730 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f4cbc00b000 tx=0x7f4cd40a8c80).stop 2021-07-11T03:53:34.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 -- 172.21.2.104:0/1091753975 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 msgr2=0x7f4cd40b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:34.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 0x7f4cd40b4590 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f4cc400b000 tx=0x7f4cc4009040).stop 2021-07-11T03:53:34.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 -- 172.21.2.104:0/1091753975 shutdown_connections 2021-07-11T03:53:34.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4cd40abb30 0x7f4cd415a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4cd40a9230 0x7f4cd40aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:34.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4cb8053270 0x7f4cb8055730 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 --2- 172.21.2.104:0/1091753975 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4cb4004f90 0x7f4cd40b4590 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:34.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 -- 172.21.2.104:0/1091753975 >> 172.21.2.104:0/1091753975 conn(0x7f4cd40b53e0 msgr2=0x7f4cd40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:34.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 -- 172.21.2.104:0/1091753975 shutdown_connections 2021-07-11T03:53:34.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:34.708+0000 7f4cd929d700 1 -- 172.21.2.104:0/1091753975 wait complete. 2021-07-11T03:53:34.719 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 6 2021-07-11T03:53:35.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:53:22.350923+0000', 'modified': '2021-07-11T03:53:23.355891+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4224}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4224': {'gid': 4224, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.104:6835/1236541958', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 1236541958}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 1236541958}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:53:35.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:53:35.024 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:53:35.089 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":4224,"want_state":"up:active","state":"up:active","rank_uptime":12.728997238,"mdsmap_epoch":5,"osdmap_epoch":23,"osdmap_epoch_barrier":21,"uptime":14.885943259999999} 2021-07-11T03:53:35.090 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 4224, "mdsmap_epoch": 5, "osdmap_epoch": 23, "osdmap_epoch_barrier": 21, "rank_uptime": 12.728997238, "state": "up:active", "uptime": 14.88594326, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:53:35.090 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T03:53:35.090 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-07-11T03:53:35.091 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba023.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-07-11T03:53:35.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:35.091 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:53:35.103 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T03:53:35.114 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:53:35.115 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:53:35.121 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:53:35.122 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:53:35.123 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:35.123 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T03:53:35.170 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T03:53:35.171 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:53:35.173 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T03:53:35.188 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:53:35.188 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:53:35.198 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:53:35.199 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:53:35.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:35.201 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T03:53:35.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14465]: starting ceph client 2021-07-11T03:53:35.220 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:2021-07-11T03:53:35.212+0000 7fa693a9e080 -1 init, newargv = 0x5640081aee60 newargc=9 2021-07-11T03:53:35.233 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14465]: starting fuse 2021-07-11T03:53:36.326 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T03:53:36.338 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:53:36.339 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:53:36.389 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:53:36.390 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:53:36.391 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:36.391 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T03:53:36.396 INFO:teuthology.orchestra.run.gibba023.stdout:52 2021-07-11T03:53:36.396 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T03:53:36.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:36.397 DEBUG:teuthology.orchestra.run.gibba023:> sudo python3 -c ' 2021-07-11T03:53:36.397 DEBUG:teuthology.orchestra.run.gibba023:> import glob 2021-07-11T03:53:36.398 DEBUG:teuthology.orchestra.run.gibba023:> import re 2021-07-11T03:53:36.398 DEBUG:teuthology.orchestra.run.gibba023:> import os 2021-07-11T03:53:36.399 DEBUG:teuthology.orchestra.run.gibba023:> import subprocess 2021-07-11T03:53:36.399 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T03:53:36.399 DEBUG:teuthology.orchestra.run.gibba023:> def find_socket(client_name): 2021-07-11T03:53:36.400 DEBUG:teuthology.orchestra.run.gibba023:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-07-11T03:53:36.400 DEBUG:teuthology.orchestra.run.gibba023:> files = glob.glob(asok_path) 2021-07-11T03:53:36.400 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T03:53:36.400 DEBUG:teuthology.orchestra.run.gibba023:> # Given a non-glob path, it better be there 2021-07-11T03:53:36.401 DEBUG:teuthology.orchestra.run.gibba023:> if "*" not in asok_path: 2021-07-11T03:53:36.401 DEBUG:teuthology.orchestra.run.gibba023:> assert(len(files) == 1) 2021-07-11T03:53:36.401 DEBUG:teuthology.orchestra.run.gibba023:> return files[0] 2021-07-11T03:53:36.402 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T03:53:36.402 DEBUG:teuthology.orchestra.run.gibba023:> for f in files: 2021-07-11T03:53:36.402 DEBUG:teuthology.orchestra.run.gibba023:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T03:53:36.403 DEBUG:teuthology.orchestra.run.gibba023:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T03:53:36.403 DEBUG:teuthology.orchestra.run.gibba023:> return f 2021-07-11T03:53:36.403 DEBUG:teuthology.orchestra.run.gibba023:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T03:53:36.404 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T03:53:36.404 DEBUG:teuthology.orchestra.run.gibba023:> print(find_socket("client.1")) 2021-07-11T03:53:36.404 DEBUG:teuthology.orchestra.run.gibba023:> ' 2021-07-11T03:53:36.473 INFO:teuthology.orchestra.run.gibba023.stdout:/var/run/ceph/ceph-client.1.14465.asok 2021-07-11T03:53:36.475 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14465.asok 2021-07-11T03:53:36.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:36.476 DEBUG:teuthology.orchestra.run.gibba023:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14465.asok status 2021-07-11T03:53:36.544 INFO:teuthology.orchestra.run.gibba023.stdout:{ 2021-07-11T03:53:36.544 INFO:teuthology.orchestra.run.gibba023.stdout: "metadata": { 2021-07-11T03:53:36.545 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T03:53:36.545 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T03:53:36.545 INFO:teuthology.orchestra.run.gibba023.stdout: "entity_id": "1", 2021-07-11T03:53:36.545 INFO:teuthology.orchestra.run.gibba023.stdout: "hostname": "gibba023", 2021-07-11T03:53:36.546 INFO:teuthology.orchestra.run.gibba023.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-07-11T03:53:36.546 INFO:teuthology.orchestra.run.gibba023.stdout: "pid": "14465", 2021-07-11T03:53:36.546 INFO:teuthology.orchestra.run.gibba023.stdout: "root": "/" 2021-07-11T03:53:36.546 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T03:53:36.547 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_count": 0, 2021-07-11T03:53:36.547 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_pinned_count": 0, 2021-07-11T03:53:36.547 INFO:teuthology.orchestra.run.gibba023.stdout: "id": 4280, 2021-07-11T03:53:36.547 INFO:teuthology.orchestra.run.gibba023.stdout: "inst": { 2021-07-11T03:53:36.547 INFO:teuthology.orchestra.run.gibba023.stdout: "name": { 2021-07-11T03:53:36.548 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "client", 2021-07-11T03:53:36.548 INFO:teuthology.orchestra.run.gibba023.stdout: "num": 4280 2021-07-11T03:53:36.548 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T03:53:36.548 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T03:53:36.549 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T03:53:36.549 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T03:53:36.549 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 1800095036 2021-07-11T03:53:36.549 INFO:teuthology.orchestra.run.gibba023.stdout: } 2021-07-11T03:53:36.549 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T03:53:36.550 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T03:53:36.550 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T03:53:36.550 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T03:53:36.550 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 1800095036 2021-07-11T03:53:36.551 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T03:53:36.551 INFO:teuthology.orchestra.run.gibba023.stdout: "inst_str": "client.4280 172.21.2.123:0/1800095036", 2021-07-11T03:53:36.551 INFO:teuthology.orchestra.run.gibba023.stdout: "addr_str": "172.21.2.123:0/1800095036", 2021-07-11T03:53:36.551 INFO:teuthology.orchestra.run.gibba023.stdout: "inode_count": 1, 2021-07-11T03:53:36.551 INFO:teuthology.orchestra.run.gibba023.stdout: "mds_epoch": 5, 2021-07-11T03:53:36.552 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch": 23, 2021-07-11T03:53:36.552 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch_barrier": 0, 2021-07-11T03:53:36.552 INFO:teuthology.orchestra.run.gibba023.stdout: "blacklisted": false, 2021-07-11T03:53:36.552 INFO:teuthology.orchestra.run.gibba023.stdout: "fs_name": "cephfs" 2021-07-11T03:53:36.553 INFO:teuthology.orchestra.run.gibba023.stdout:} 2021-07-11T03:53:36.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:36.555 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:36.555 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:36.566 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:53:36.568 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:53:36.568 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:36.569 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:36.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:36.587 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:36.588 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:53:36.594 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T03:53:36.595 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T03:53:36.596 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:36.596 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:53:36.664 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2021-07-11T03:53:36.688 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:36.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fde6bb700 1 Processor -- start 2021-07-11T03:53:36.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fde6bb700 1 -- start start 2021-07-11T03:53:36.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fde6bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5fd80973d0 0x7f5fd805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:36.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fde6bb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5fd805c490 0x7f5fd813f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:36.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fde6bb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd80b2a40 con 0x7f5fd80b2660 2021-07-11T03:53:36.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fde6bb700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd8097b80 con 0x7f5fd805c490 2021-07-11T03:53:36.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fde6bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd8097e20 con 0x7f5fd80973d0 2021-07-11T03:53:36.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fdcc58700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5fd805c490 0x7f5fd813f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:36.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fdcc58700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5fd805c490 0x7f5fd813f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42456/0 (socket says 172.21.2.104:42456) 2021-07-11T03:53:36.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fdcc58700 1 -- 172.21.2.104:0/179064158 learned_addr learned my addr 172.21.2.104:0/179064158 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:36.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fdcc58700 1 -- 172.21.2.104:0/179064158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5fd80973d0 msgr2=0x7f5fd805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:36.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fdcc58700 1 --2- 172.21.2.104:0/179064158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5fd80973d0 0x7f5fd805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:36.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fdcc58700 1 -- 172.21.2.104:0/179064158 >> v1:172.21.2.105:6789/0 conn(0x7f5fd80b2660 legacy=0x7f5fd8096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:36.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fdcc58700 1 -- 172.21.2.104:0/179064158 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd80980c0 con 0x7f5fd805c490 2021-07-11T03:53:36.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fdcc58700 1 --2- 172.21.2.104:0/179064158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5fd805c490 0x7f5fd813f560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f5fd80b20e0 tx=0x7f5fd0009290).ready entity=mon.0 client_cookie=e37df857b8824b47 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:36.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fd6ffd700 1 -- 172.21.2.104:0/179064158 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fd000b860 con 0x7f5fd805c490 2021-07-11T03:53:36.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fd6ffd700 1 -- 172.21.2.104:0/179064158 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd000b9c0 con 0x7f5fd805c490 2021-07-11T03:53:36.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fd6ffd700 1 -- 172.21.2.104:0/179064158 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fd000bb70 con 0x7f5fd805c490 2021-07-11T03:53:36.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fde6bb700 1 -- 172.21.2.104:0/179064158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5fd805c490 msgr2=0x7f5fd813f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:36.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.740+0000 7f5fde6bb700 1 --2- 172.21.2.104:0/179064158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5fd805c490 0x7f5fd813f560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f5fd80b20e0 tx=0x7f5fd0009290).stop 2021-07-11T03:53:36.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 -- 172.21.2.104:0/179064158 shutdown_connections 2021-07-11T03:53:36.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 --2- 172.21.2.104:0/179064158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5fd805c490 0x7f5fd813f560 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:36.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 --2- 172.21.2.104:0/179064158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5fd80973d0 0x7f5fd805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:36.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 -- 172.21.2.104:0/179064158 >> 172.21.2.104:0/179064158 conn(0x7f5fd8062f10 msgr2=0x7f5fd80632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:36.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 -- 172.21.2.104:0/179064158 shutdown_connections 2021-07-11T03:53:36.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 -- 172.21.2.104:0/179064158 wait complete. 2021-07-11T03:53:36.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 Processor -- start 2021-07-11T03:53:36.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 -- start start 2021-07-11T03:53:36.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5fd805c490 0x7f5fd8136050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:36.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5fd80973d0 0x7f5fd80ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:36.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5fd80b2660 0x7f5fd80aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:36.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd813f840 con 0x7f5fd805c490 2021-07-11T03:53:36.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd814a7b0 con 0x7f5fd80973d0 2021-07-11T03:53:36.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80952c0 con 0x7f5fd80b2660 2021-07-11T03:53:36.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5fd805c490 0x7f5fd8136050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:36.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5fd805c490 0x7f5fd8136050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42460/0 (socket says 172.21.2.104:42460) 2021-07-11T03:53:36.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd7fff700 1 -- 172.21.2.104:0/522094336 learned_addr learned my addr 172.21.2.104:0/522094336 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:36.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd7fff700 1 -- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5fd80b2660 msgr2=0x7f5fd80aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:36.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd7fff700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5fd80b2660 0x7f5fd80aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:36.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd7fff700 1 -- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5fd80973d0 msgr2=0x7f5fd80ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:36.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd77fe700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5fd80973d0 0x7f5fd80ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:36.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd7fff700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5fd80973d0 0x7f5fd80ae660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:36.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd7fff700 1 -- 172.21.2.104:0/522094336 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd0009530 con 0x7f5fd805c490 2021-07-11T03:53:36.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd7fff700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5fd805c490 0x7f5fd8136050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f5fd809a090 tx=0x7f5fc0006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:36.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd57fa700 1 -- 172.21.2.104:0/522094336 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc000f510 con 0x7f5fd805c490 2021-07-11T03:53:36.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd57fa700 1 -- 172.21.2.104:0/522094336 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc000f670 con 0x7f5fd805c490 2021-07-11T03:53:36.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 -- 172.21.2.104:0/522094336 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd814a9e0 con 0x7f5fd805c490 2021-07-11T03:53:36.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 -- 172.21.2.104:0/522094336 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd809b550 con 0x7f5fd805c490 2021-07-11T03:53:36.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fd57fa700 1 -- 172.21.2.104:0/522094336 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5fc000f7e0 con 0x7f5fd805c490 2021-07-11T03:53:36.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.744+0000 7f5fde6bb700 1 -- 172.21.2.104:0/522094336 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb0003410 con 0x7f5fd805c490 2021-07-11T03:53:36.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.748+0000 7f5fd57fa700 1 -- 172.21.2.104:0/522094336 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5fc0017030 con 0x7f5fd805c490 2021-07-11T03:53:36.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.748+0000 7f5fd57fa700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5fbc053180 0x7f5fbc055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:36.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.748+0000 7f5fd57fa700 1 -- 172.21.2.104:0/522094336 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5fc006a930 con 0x7f5fd805c490 2021-07-11T03:53:36.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.748+0000 7f5fd77fe700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5fbc053180 0x7f5fbc055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:36.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.748+0000 7f5fd77fe700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5fbc053180 0x7f5fbc055640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f5fc8007000 tx=0x7f5fc8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:36.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.748+0000 7f5fd57fa700 1 -- 172.21.2.104:0/522094336 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5fc0041b60 con 0x7f5fd805c490 2021-07-11T03:53:36.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.828+0000 7f5fde6bb700 1 -- 172.21.2.104:0/522094336 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fb00036c0 con 0x7f5fd805c490 2021-07-11T03:53:36.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fd57fa700 1 -- 172.21.2.104:0/522094336 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f5fc0015070 con 0x7f5fd805c490 2021-07-11T03:53:36.834 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:36.835 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":23,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:24.367517+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:36.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 -- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5fbc053180 msgr2=0x7f5fbc055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:36.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5fbc053180 0x7f5fbc055640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f5fc8007000 tx=0x7f5fc8008040).stop 2021-07-11T03:53:36.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 -- 172.21.2.104:0/522094336 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5fd805c490 msgr2=0x7f5fd8136050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:36.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5fd805c490 0x7f5fd8136050 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f5fd809a090 tx=0x7f5fc0006d70).stop 2021-07-11T03:53:36.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 -- 172.21.2.104:0/522094336 shutdown_connections 2021-07-11T03:53:36.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5fd80b2660 0x7f5fd80aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:36.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5fd80973d0 0x7f5fd80ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:36.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5fbc053180 0x7f5fbc055640 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:36.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 --2- 172.21.2.104:0/522094336 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5fd805c490 0x7f5fd8136050 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:36.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 -- 172.21.2.104:0/522094336 >> 172.21.2.104:0/522094336 conn(0x7f5fd8062f10 msgr2=0x7f5fd8060210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:36.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 -- 172.21.2.104:0/522094336 shutdown_connections 2021-07-11T03:53:36.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.832+0000 7f5fde6bb700 1 -- 172.21.2.104:0/522094336 wait complete. 2021-07-11T03:53:36.842 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:53:20.980780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:53:21.341375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:53:36.843 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:53:36.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 Processor -- start 2021-07-11T03:53:36.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 -- start start 2021-07-11T03:53:36.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb4b80b1ec0 0x7fb4b80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:36.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b80b0420 con 0x7fb4b80affb0 2021-07-11T03:53:36.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b80a77d0 con 0x7fb4b80b22a0 2021-07-11T03:53:36.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b80b0650 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bd164700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb4b80affb0 0x7fb4b80a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39236/0 (socket says 172.21.2.104:39236) 2021-07-11T03:53:36.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bd164700 1 -- 172.21.2.104:0/2683989152 learned_addr learned my addr 172.21.2.104:0/2683989152 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:36.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bd965700 1 --2- 172.21.2.104:0/2683989152 >> v2:172.21.2.104:3300/0 conn(0x7fb4b80b1ec0 0x7fb4b80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:36.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bc963700 1 -- 172.21.2.104:0/2683989152 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2157263704 0 0) 0x7fb4b80b0420 con 0x7fb4b80affb0 2021-07-11T03:53:36.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bc963700 1 -- 172.21.2.104:0/2683989152 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a0003260 con 0x7fb4b80affb0 2021-07-11T03:53:36.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bd965700 1 -- 172.21.2.104:0/2683989152 >> v1:172.21.2.105:6789/0 conn(0x7fb4b80b22a0 legacy=0x7fb4b8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:36.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bd965700 1 -- 172.21.2.104:0/2683989152 >> v1:172.21.2.104:6789/0 conn(0x7fb4b80affb0 legacy=0x7fb4b80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:36.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bd965700 1 -- 172.21.2.104:0/2683989152 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b80b08f0 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bd965700 1 --2- 172.21.2.104:0/2683989152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb4b80b1ec0 0x7fb4b80b12d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fb4b80b2f50 tx=0x7fb4b4006df0).ready entity=mon.0 client_cookie=7d10d8b345c446eb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:36.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bc963700 1 -- 172.21.2.104:0/2683989152 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4b400d5c0 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bc963700 1 -- 172.21.2.104:0/2683989152 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b4000e90 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bc963700 1 -- 172.21.2.104:0/2683989152 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4b400d910 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/2683989152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb4b80b1ec0 msgr2=0x7fb4b80b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:36.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 --2- 172.21.2.104:0/2683989152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb4b80b1ec0 0x7fb4b80b12d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fb4b80b2f50 tx=0x7fb4b4006df0).stop 2021-07-11T03:53:36.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/2683989152 shutdown_connections 2021-07-11T03:53:36.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 --2- 172.21.2.104:0/2683989152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb4b80b1ec0 0x7fb4b80b12d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:36.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/2683989152 >> 172.21.2.104:0/2683989152 conn(0x7fb4b805f6b0 msgr2=0x7fb4b805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:36.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/2683989152 shutdown_connections 2021-07-11T03:53:36.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.892+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/2683989152 wait complete. 2021-07-11T03:53:36.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bfbc9700 1 Processor -- start 2021-07-11T03:53:36.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bfbc9700 1 -- start start 2021-07-11T03:53:36.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bfbc9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4b80affb0 0x7fb4b80ab480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:36.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bfbc9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb4b80b1ec0 0x7fb4b80aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:36.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bfbc9700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4b80b22a0 0x7fb4b8149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:36.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bfbc9700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80a9b40 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bfbc9700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b80a9d70 con 0x7fb4b80b22a0 2021-07-11T03:53:36.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bfbc9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b8145330 con 0x7fb4b80affb0 2021-07-11T03:53:36.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bd164700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb4b80b1ec0 0x7fb4b80aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:36.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bd164700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb4b80b1ec0 0x7fb4b80aabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42474/0 (socket says 172.21.2.104:42474) 2021-07-11T03:53:36.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bd164700 1 -- 172.21.2.104:0/3121847170 learned_addr learned my addr 172.21.2.104:0/3121847170 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:36.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bd164700 1 -- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4b80affb0 msgr2=0x7fb4b80ab480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:36.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bd164700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4b80affb0 0x7fb4b80ab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:36.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bd164700 1 -- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4b80b22a0 msgr2=0x7fb4b8149730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:36.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bd164700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4b80b22a0 0x7fb4b8149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:36.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bd164700 1 -- 172.21.2.104:0/3121847170 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b400b180 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bd164700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb4b80b1ec0 0x7fb4b80aabc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fb4a8001110 tx=0x7fb4a8007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:36.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4a6ffd700 1 -- 172.21.2.104:0/3121847170 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4a80072e0 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4a6ffd700 1 -- 172.21.2.104:0/3121847170 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a8007440 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/3121847170 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b80ab0b0 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/3121847170 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b80aa7a0 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4a6ffd700 1 -- 172.21.2.104:0/3121847170 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb4a8005800 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/3121847170 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b8149f60 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4a6ffd700 1 -- 172.21.2.104:0/3121847170 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb4a8010070 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4a6ffd700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb498053350 0x7fb498055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:36.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4a6ffd700 1 -- 172.21.2.104:0/3121847170 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb4a806ade0 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.896+0000 7fb4bd965700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb498053350 0x7fb498055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:36.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.900+0000 7fb4bd965700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb498053350 0x7fb498055810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fb4b80b2f50 tx=0x7fb4b4014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:36.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.900+0000 7fb4a6ffd700 1 -- 172.21.2.104:0/3121847170 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb4a8041fb0 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.980+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/3121847170 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb4b8057350 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.980+0000 7fb4a6ffd700 1 -- 172.21.2.104:0/3121847170 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb4a80455d0 con 0x7fb4b80b1ec0 2021-07-11T03:53:36.985 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:53:36.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.980+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb498053350 msgr2=0x7fb498055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:36.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.980+0000 7fb4bfbc9700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb498053350 0x7fb498055810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fb4b80b2f50 tx=0x7fb4b4014000).stop 2021-07-11T03:53:36.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.980+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/3121847170 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb4b80b1ec0 msgr2=0x7fb4b80aabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:36.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.980+0000 7fb4bfbc9700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb4b80b1ec0 0x7fb4b80aabc0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fb4a8001110 tx=0x7fb4a8007ad0).stop 2021-07-11T03:53:36.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.984+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/3121847170 shutdown_connections 2021-07-11T03:53:36.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.984+0000 7fb4bfbc9700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4b80b22a0 0x7fb4b8149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:36.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.984+0000 7fb4bfbc9700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb498053350 0x7fb498055810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:36.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.984+0000 7fb4bfbc9700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb4b80b1ec0 0x7fb4b80aabc0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:36.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.984+0000 7fb4bfbc9700 1 --2- 172.21.2.104:0/3121847170 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4b80affb0 0x7fb4b80ab480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:36.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.984+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/3121847170 >> 172.21.2.104:0/3121847170 conn(0x7fb4b805f6b0 msgr2=0x7fb4b80a70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:36.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.984+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/3121847170 shutdown_connections 2021-07-11T03:53:36.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:36.984+0000 7fb4bfbc9700 1 -- 172.21.2.104:0/3121847170 wait complete. 2021-07-11T03:53:36.993 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:53:37.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.040+0000 7f094c09a700 1 Processor -- start 2021-07-11T03:53:37.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.040+0000 7f094c09a700 1 -- start start 2021-07-11T03:53:37.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09440aa2a0 0x7f094414b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0944061990 con 0x7f09440603d0 2021-07-11T03:53:37.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0944061c30 con 0x7f09440a9ec0 2021-07-11T03:53:37.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0944060b90 con 0x7f09440aa2a0 2021-07-11T03:53:37.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0949635700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f09440603d0 0x7f094413f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39250/0 (socket says 172.21.2.104:39250) 2021-07-11T03:53:37.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0949635700 1 -- 172.21.2.104:0/4130757984 learned_addr learned my addr 172.21.2.104:0/4130757984 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:37.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250880950 0 0) 0x7f0944061990 con 0x7f09440603d0 2021-07-11T03:53:37.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f092c003260 con 0x7f09440603d0 2021-07-11T03:53:37.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094a637700 1 --2- 172.21.2.104:0/4130757984 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09440aa2a0 0x7f094414b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3778440114 0 0) 0x7f092c003260 con 0x7f09440603d0 2021-07-11T03:53:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09440aa2a0 msgr2=0x7f094414b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 --2- 172.21.2.104:0/4130757984 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09440aa2a0 0x7f094414b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 >> v1:172.21.2.105:6789/0 conn(0x7f09440a9ec0 legacy=0x7f094413ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944060e30 con 0x7f09440603d0 2021-07-11T03:53:37.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f0938001660 con 0x7f09440603d0 2021-07-11T03:53:37.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 >> v1:172.21.2.104:6789/0 conn(0x7f09440603d0 legacy=0x7f094413f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 --2- 172.21.2.104:0/4130757984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f092c003800 0x7f092c003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 --2- 172.21.2.104:0/4130757984 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09440aa2a0 0x7f092c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 --2- 172.21.2.104:0/4130757984 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f092c004b10 0x7f092c004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0944060b90 con 0x7f092c003800 2021-07-11T03:53:37.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c00b910 con 0x7f092c004b10 2021-07-11T03:53:37.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c00bad0 con 0x7f09440aa2a0 2021-07-11T03:53:37.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0949e36700 1 --2- 172.21.2.104:0/4130757984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f092c003800 0x7f092c003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0949e36700 1 -- 172.21.2.104:0/4130757984 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09440aa2a0 msgr2=0x7f092c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0949e36700 1 --2- 172.21.2.104:0/4130757984 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09440aa2a0 0x7f092c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0949e36700 1 -- 172.21.2.104:0/4130757984 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f092c004b10 msgr2=0x7f092c004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0949e36700 1 --2- 172.21.2.104:0/4130757984 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f092c004b10 0x7f092c004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0949e36700 1 -- 172.21.2.104:0/4130757984 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c003260 con 0x7f092c003800 2021-07-11T03:53:37.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0949e36700 1 --2- 172.21.2.104:0/4130757984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f092c003800 0x7f092c003c00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f0940001fa0 tx=0x7f0940007a00).ready entity=mon.0 client_cookie=fe9ad244525dac99 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0940007070 con 0x7f092c003800 2021-07-11T03:53:37.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09400071d0 con 0x7f092c003800 2021-07-11T03:53:37.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0948e34700 1 -- 172.21.2.104:0/4130757984 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0940006510 con 0x7f092c003800 2021-07-11T03:53:37.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- 172.21.2.104:0/4130757984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f092c003800 msgr2=0x7f092c003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 --2- 172.21.2.104:0/4130757984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f092c003800 0x7f092c003c00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f0940001fa0 tx=0x7f0940007a00).stop 2021-07-11T03:53:37.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094a637700 1 -- 172.21.2.104:0/4130757984 reap_dead start 2021-07-11T03:53:37.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- 172.21.2.104:0/4130757984 shutdown_connections 2021-07-11T03:53:37.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- 172.21.2.104:0/4130757984 >> 172.21.2.104:0/4130757984 conn(0x7f09440b2970 msgr2=0x7f09440b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- 172.21.2.104:0/4130757984 shutdown_connections 2021-07-11T03:53:37.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- 172.21.2.104:0/4130757984 wait complete. 2021-07-11T03:53:37.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 Processor -- start 2021-07-11T03:53:37.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- start start 2021-07-11T03:53:37.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0944063370 0x7f09440ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0944063750 0x7f09440ad630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0944143260 0x7f094412c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09441312b0 con 0x7f0944143260 2021-07-11T03:53:37.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0944130630 con 0x7f0944063370 2021-07-11T03:53:37.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094c09a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0944130390 con 0x7f0944063750 2021-07-11T03:53:37.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094a637700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0944143260 0x7f094412c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094a637700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0944143260 0x7f094412c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42496/0 (socket says 172.21.2.104:42496) 2021-07-11T03:53:37.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094a637700 1 -- 172.21.2.104:0/3055595559 learned_addr learned my addr 172.21.2.104:0/3055595559 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:37.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f0949e36700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0944063370 0x7f09440ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094a637700 1 -- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0944063750 msgr2=0x7f09440ad630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094a637700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0944063750 0x7f09440ad630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094a637700 1 -- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0944063370 msgr2=0x7f09440ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094a637700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0944063370 0x7f09440ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.044+0000 7f094a637700 1 -- 172.21.2.104:0/3055595559 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944131510 con 0x7f0944143260 2021-07-11T03:53:37.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f094a637700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0944143260 0x7f094412c020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f093c000b60 tx=0x7f093c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f0936ffd700 1 -- 172.21.2.104:0/3055595559 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f093c00c070 con 0x7f0944143260 2021-07-11T03:53:37.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f0936ffd700 1 -- 172.21.2.104:0/3055595559 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093c0133f0 con 0x7f0944143260 2021-07-11T03:53:37.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f0936ffd700 1 -- 172.21.2.104:0/3055595559 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f093c013750 con 0x7f0944143260 2021-07-11T03:53:37.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f094c09a700 1 -- 172.21.2.104:0/3055595559 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09440b6540 con 0x7f0944143260 2021-07-11T03:53:37.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f094c09a700 1 -- 172.21.2.104:0/3055595559 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09441319b0 con 0x7f0944143260 2021-07-11T03:53:37.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f094c09a700 1 -- 172.21.2.104:0/3055595559 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09441316a0 con 0x7f0944143260 2021-07-11T03:53:37.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f0936ffd700 1 -- 172.21.2.104:0/3055595559 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f093c0138b0 con 0x7f0944143260 2021-07-11T03:53:37.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f0936ffd700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0924053300 0x7f09240557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f0936ffd700 1 -- 172.21.2.104:0/3055595559 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f093c06afb0 con 0x7f0944143260 2021-07-11T03:53:37.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f0949e36700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0924053300 0x7f09240557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f0949e36700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0924053300 0x7f09240557c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f0940016000 tx=0x7f0940015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.048+0000 7f0936ffd700 1 -- 172.21.2.104:0/3055595559 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f093c0422f0 con 0x7f0944143260 2021-07-11T03:53:37.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.132+0000 7f094c09a700 1 -- 172.21.2.104:0/3055595559 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0944057350 con 0x7f0944143260 2021-07-11T03:53:37.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.132+0000 7f0936ffd700 1 -- 172.21.2.104:0/3055595559 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f093c045910 con 0x7f0944143260 2021-07-11T03:53:37.138 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:37.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 -- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0924053300 msgr2=0x7f09240557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0924053300 0x7f09240557c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f0940016000 tx=0x7f0940015000).stop 2021-07-11T03:53:37.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 -- 172.21.2.104:0/3055595559 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0944143260 msgr2=0x7f094412c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0944143260 0x7f094412c020 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f093c000b60 tx=0x7f093c008040).stop 2021-07-11T03:53:37.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 -- 172.21.2.104:0/3055595559 shutdown_connections 2021-07-11T03:53:37.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0924053300 0x7f09240557c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0944143260 0x7f094412c020 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0944063750 0x7f09440ad630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 --2- 172.21.2.104:0/3055595559 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0944063370 0x7f09440ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 -- 172.21.2.104:0/3055595559 >> 172.21.2.104:0/3055595559 conn(0x7f09440b2970 msgr2=0x7f094412f170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 -- 172.21.2.104:0/3055595559 shutdown_connections 2021-07-11T03:53:37.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.136+0000 7f094c09a700 1 -- 172.21.2.104:0/3055595559 wait complete. 2021-07-11T03:53:37.144 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:53:37.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.192+0000 7f6fd17ff700 1 Processor -- start 2021-07-11T03:53:37.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.192+0000 7f6fd17ff700 1 -- start start 2021-07-11T03:53:37.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6fcc0b05b0 0x7f6fcc0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6fcc0a9c10 0x7f6fcc0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fcc143920 con 0x7f6fcc0b3710 2021-07-11T03:53:37.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc143bc0 con 0x7f6fcc0b05b0 2021-07-11T03:53:37.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fcc143e60 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fca7fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6fcc0b3710 0x7f6fcc13d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39272/0 (socket says 172.21.2.104:39272) 2021-07-11T03:53:37.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fca7fc700 1 -- 172.21.2.104:0/2507786440 learned_addr learned my addr 172.21.2.104:0/2507786440 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:37.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 --2- 172.21.2.104:0/2507786440 >> v2:172.21.2.104:3300/0 conn(0x7f6fcc0b05b0 0x7f6fcc0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 -- 172.21.2.104:0/2507786440 >> v2:172.21.2.105:3300/0 conn(0x7f6fcc0a9c10 msgr2=0x7f6fcc0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 --2- 172.21.2.104:0/2507786440 >> v2:172.21.2.105:3300/0 conn(0x7f6fcc0a9c10 0x7f6fcc0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 -- 172.21.2.104:0/2507786440 >> v1:172.21.2.104:6789/0 conn(0x7f6fcc0b3710 legacy=0x7f6fcc13d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 -- 172.21.2.104:0/2507786440 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc144100 con 0x7f6fcc0b05b0 2021-07-11T03:53:37.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 --2- 172.21.2.104:0/2507786440 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fcc0b05b0 0x7f6fcc0b0990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f6fbc002f70 tx=0x7f6fbc006df0).ready entity=mon.0 client_cookie=ca17de0621869f85 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fc9ffb700 1 -- 172.21.2.104:0/2507786440 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fbc00d5c0 con 0x7f6fcc0b05b0 2021-07-11T03:53:37.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fc9ffb700 1 -- 172.21.2.104:0/2507786440 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc000e90 con 0x7f6fcc0b05b0 2021-07-11T03:53:37.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fc9ffb700 1 -- 172.21.2.104:0/2507786440 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fbc00d910 con 0x7f6fcc0b05b0 2021-07-11T03:53:37.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- 172.21.2.104:0/2507786440 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fcc0b05b0 msgr2=0x7f6fcc0b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 --2- 172.21.2.104:0/2507786440 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fcc0b05b0 0x7f6fcc0b0990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f6fbc002f70 tx=0x7f6fbc006df0).stop 2021-07-11T03:53:37.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- 172.21.2.104:0/2507786440 shutdown_connections 2021-07-11T03:53:37.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 --2- 172.21.2.104:0/2507786440 >> v2:172.21.2.105:3300/0 conn(0x7f6fcc0a9c10 0x7f6fcc0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 --2- 172.21.2.104:0/2507786440 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fcc0b05b0 0x7f6fcc0b0990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- 172.21.2.104:0/2507786440 >> 172.21.2.104:0/2507786440 conn(0x7f6fcc0607c0 msgr2=0x7f6fcc060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- 172.21.2.104:0/2507786440 shutdown_connections 2021-07-11T03:53:37.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- 172.21.2.104:0/2507786440 wait complete. 2021-07-11T03:53:37.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 Processor -- start 2021-07-11T03:53:37.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- start start 2021-07-11T03:53:37.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fcc0a9c10 0x7f6fcc147680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fcc0b05b0 0x7f6fcc147f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fcc0b3710 0x7f6fcc0b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc1409e0 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc1401e0 con 0x7f6fcc0b05b0 2021-07-11T03:53:37.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fd17ff700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc0b5e70 con 0x7f6fcc0b3710 2021-07-11T03:53:37.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fcc0a9c10 0x7f6fcc147680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fcc0a9c10 0x7f6fcc147680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42508/0 (socket says 172.21.2.104:42508) 2021-07-11T03:53:37.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 -- 172.21.2.104:0/3139396128 learned_addr learned my addr 172.21.2.104:0/3139396128 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:37.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fca7fc700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fcc0b05b0 0x7f6fcc147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcb7fe700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fcc0b3710 0x7f6fcc0b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 -- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fcc0b3710 msgr2=0x7f6fcc0b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fcc0b3710 0x7f6fcc0b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 -- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fcc0b05b0 msgr2=0x7f6fcc147f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fcc0b05b0 0x7f6fcc147f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 -- 172.21.2.104:0/3139396128 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc00b180 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcb7fe700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fcc0b3710 0x7f6fcc0b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:53:37.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.196+0000 7f6fcaffd700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fcc0a9c10 0x7f6fcc147680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f6fcc145ab0 tx=0x7f6fbc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fc3fff700 1 -- 172.21.2.104:0/3139396128 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fbc006b20 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fc3fff700 1 -- 172.21.2.104:0/3139396128 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc006780 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fd17ff700 1 -- 172.21.2.104:0/3139396128 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fcc13ff40 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fc3fff700 1 -- 172.21.2.104:0/3139396128 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fbc01f630 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fd17ff700 1 -- 172.21.2.104:0/3139396128 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fcc13efe0 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fd17ff700 1 -- 172.21.2.104:0/3139396128 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fcc057350 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fc3fff700 1 -- 172.21.2.104:0/3139396128 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6fbc025020 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fc3fff700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb00531a0 0x7f6fb0055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fc3fff700 1 -- 172.21.2.104:0/3139396128 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6fbc06c610 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fca7fc700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb00531a0 0x7f6fb0055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fca7fc700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb00531a0 0x7f6fb0055660 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6fb400b000 tx=0x7f6fb4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.200+0000 7f6fc3fff700 1 -- 172.21.2.104:0/3139396128 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6fbc043880 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 -- 172.21.2.104:0/3139396128 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6fcc044650 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fc3fff700 1 -- 172.21.2.104:0/3139396128 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6fbc046e60 con 0x7f6fcc0a9c10 2021-07-11T03:53:37.287 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:37.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 -- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb00531a0 msgr2=0x7f6fb0055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb00531a0 0x7f6fb0055660 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f6fb400b000 tx=0x7f6fb4009040).stop 2021-07-11T03:53:37.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 -- 172.21.2.104:0/3139396128 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fcc0a9c10 msgr2=0x7f6fcc147680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fcc0a9c10 0x7f6fcc147680 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f6fcc145ab0 tx=0x7f6fbc014000).stop 2021-07-11T03:53:37.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 -- 172.21.2.104:0/3139396128 shutdown_connections 2021-07-11T03:53:37.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fcc0b3710 0x7f6fcc0b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fcc0b05b0 0x7f6fcc147f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb00531a0 0x7f6fb0055660 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 --2- 172.21.2.104:0/3139396128 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fcc0a9c10 0x7f6fcc147680 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 -- 172.21.2.104:0/3139396128 >> 172.21.2.104:0/3139396128 conn(0x7f6fcc0607c0 msgr2=0x7f6fcc142eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 -- 172.21.2.104:0/3139396128 shutdown_connections 2021-07-11T03:53:37.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.284+0000 7f6fd17ff700 1 -- 172.21.2.104:0/3139396128 wait complete. 2021-07-11T03:53:37.295 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:53:37.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 Processor -- start 2021-07-11T03:53:37.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 -- start start 2021-07-11T03:53:37.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f68100b1ec0 0x7f68100b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f68100affb0 0x7f68100b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68100b0420 con 0x7f68100a70c0 2021-07-11T03:53:37.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100b2790 con 0x7f68100affb0 2021-07-11T03:53:37.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100b0650 con 0x7f68100b1ec0 2021-07-11T03:53:37.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f68100affb0 0x7f68100b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f68100affb0 0x7f68100b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42518/0 (socket says 172.21.2.104:42518) 2021-07-11T03:53:37.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680f7fe700 1 -- 172.21.2.104:0/2773418285 learned_addr learned my addr 172.21.2.104:0/2773418285 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:37.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680ffff700 1 --2- 172.21.2.104:0/2773418285 >> v2:172.21.2.105:3300/0 conn(0x7f68100b1ec0 0x7f68100b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680f7fe700 1 -- 172.21.2.104:0/2773418285 >> v2:172.21.2.105:3300/0 conn(0x7f68100b1ec0 msgr2=0x7f68100b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680f7fe700 1 --2- 172.21.2.104:0/2773418285 >> v2:172.21.2.105:3300/0 conn(0x7f68100b1ec0 0x7f68100b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680f7fe700 1 -- 172.21.2.104:0/2773418285 >> v1:172.21.2.105:6789/0 conn(0x7f68100a70c0 legacy=0x7f6810141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680f7fe700 1 -- 172.21.2.104:0/2773418285 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100b08f0 con 0x7f68100affb0 2021-07-11T03:53:37.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680f7fe700 1 --2- 172.21.2.104:0/2773418285 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f68100affb0 0x7f68100b22a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f68100ad2d0 tx=0x7f6800006eb0).ready entity=mon.0 client_cookie=aeba6ba466fb0b78 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680effd700 1 -- 172.21.2.104:0/2773418285 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f680000d5c0 con 0x7f68100affb0 2021-07-11T03:53:37.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680effd700 1 -- 172.21.2.104:0/2773418285 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6800000e90 con 0x7f68100affb0 2021-07-11T03:53:37.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f680effd700 1 -- 172.21.2.104:0/2773418285 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f680000d910 con 0x7f68100affb0 2021-07-11T03:53:37.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 -- 172.21.2.104:0/2773418285 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f68100affb0 msgr2=0x7f68100b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 --2- 172.21.2.104:0/2773418285 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f68100affb0 0x7f68100b22a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f68100ad2d0 tx=0x7f6800006eb0).stop 2021-07-11T03:53:37.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 -- 172.21.2.104:0/2773418285 shutdown_connections 2021-07-11T03:53:37.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 --2- 172.21.2.104:0/2773418285 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f68100affb0 0x7f68100b22a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 --2- 172.21.2.104:0/2773418285 >> v2:172.21.2.105:3300/0 conn(0x7f68100b1ec0 0x7f68100b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 -- 172.21.2.104:0/2773418285 >> 172.21.2.104:0/2773418285 conn(0x7f681005f6b0 msgr2=0x7f681005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 -- 172.21.2.104:0/2773418285 shutdown_connections 2021-07-11T03:53:37.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 -- 172.21.2.104:0/2773418285 wait complete. 2021-07-11T03:53:37.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.344+0000 7f68164ea700 1 Processor -- start 2021-07-11T03:53:37.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f68164ea700 1 -- start start 2021-07-11T03:53:37.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f68164ea700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f68100a70c0 0x7f6810159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f68164ea700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f68100affb0 0x7f681015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f68164ea700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f68100b1ec0 0x7f6810148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f68164ea700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68100b3800 con 0x7f68100affb0 2021-07-11T03:53:37.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f68164ea700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68100b3390 con 0x7f68100b1ec0 2021-07-11T03:53:37.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f68164ea700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681015ab70 con 0x7f68100a70c0 2021-07-11T03:53:37.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f68100affb0 0x7f681015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f68100affb0 0x7f681015a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42526/0 (socket says 172.21.2.104:42526) 2021-07-11T03:53:37.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680f7fe700 1 -- 172.21.2.104:0/612869404 learned_addr learned my addr 172.21.2.104:0/612869404 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:37.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680f7fe700 1 -- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f68100a70c0 msgr2=0x7f6810159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680f7fe700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f68100a70c0 0x7f6810159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680f7fe700 1 -- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f68100b1ec0 msgr2=0x7f6810148ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680f7fe700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f68100b1ec0 0x7f6810148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680f7fe700 1 -- 172.21.2.104:0/612869404 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680000b1e0 con 0x7f68100affb0 2021-07-11T03:53:37.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680f7fe700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f68100affb0 0x7f681015a4d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f68100ad2d0 tx=0x7f6800015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680d7fa700 1 -- 172.21.2.104:0/612869404 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6800021070 con 0x7f68100affb0 2021-07-11T03:53:37.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680d7fa700 1 -- 172.21.2.104:0/612869404 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6800000e90 con 0x7f68100affb0 2021-07-11T03:53:37.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680d7fa700 1 -- 172.21.2.104:0/612869404 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6800011ad0 con 0x7f68100affb0 2021-07-11T03:53:37.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f68164ea700 1 -- 172.21.2.104:0/612869404 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68100b5490 con 0x7f68100affb0 2021-07-11T03:53:37.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f68164ea700 1 -- 172.21.2.104:0/612869404 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68100b50a0 con 0x7f68100affb0 2021-07-11T03:53:37.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f68164ea700 1 -- 172.21.2.104:0/612869404 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681015a9c0 con 0x7f68100affb0 2021-07-11T03:53:37.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680d7fa700 1 -- 172.21.2.104:0/612869404 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f680000f070 con 0x7f68100affb0 2021-07-11T03:53:37.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680d7fa700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67f0053300 0x7f67f00557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680d7fa700 1 -- 172.21.2.104:0/612869404 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f680006ca70 con 0x7f68100affb0 2021-07-11T03:53:37.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.348+0000 7f680ffff700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67f0053300 0x7f67f00557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.352+0000 7f680ffff700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67f0053300 0x7f67f00557c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f6804007000 tx=0x7f6804008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.352+0000 7f680d7fa700 1 -- 172.21.2.104:0/612869404 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6800043ca0 con 0x7f68100affb0 2021-07-11T03:53:37.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.432+0000 7f68164ea700 1 -- 172.21.2.104:0/612869404 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6810057350 con 0x7f68100affb0 2021-07-11T03:53:37.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f680d7fa700 1 -- 172.21.2.104:0/612869404 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f68000472c0 con 0x7f68100affb0 2021-07-11T03:53:37.439 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:37.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 -- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67f0053300 msgr2=0x7f67f00557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67f0053300 0x7f67f00557c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f6804007000 tx=0x7f6804008040).stop 2021-07-11T03:53:37.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 -- 172.21.2.104:0/612869404 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f68100affb0 msgr2=0x7f681015a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f68100affb0 0x7f681015a4d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f68100ad2d0 tx=0x7f6800015000).stop 2021-07-11T03:53:37.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 -- 172.21.2.104:0/612869404 shutdown_connections 2021-07-11T03:53:37.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f68100b1ec0 0x7f6810148ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67f0053300 0x7f67f00557c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f68100affb0 0x7f681015a4d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 --2- 172.21.2.104:0/612869404 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f68100a70c0 0x7f6810159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 -- 172.21.2.104:0/612869404 >> 172.21.2.104:0/612869404 conn(0x7f681005f6b0 msgr2=0x7f681014d810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 -- 172.21.2.104:0/612869404 shutdown_connections 2021-07-11T03:53:37.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.436+0000 7f68164ea700 1 -- 172.21.2.104:0/612869404 wait complete. 2021-07-11T03:53:37.447 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:37.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.492+0000 7f6d8e02e700 1 Processor -- start 2021-07-11T03:53:37.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.492+0000 7f6d8e02e700 1 -- start start 2021-07-11T03:53:37.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6d88143540 0x7f6d880a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6d8805f6b0 0x7f6d88143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d880b14c0 con 0x7f6d880a8b60 2021-07-11T03:53:37.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880ab3e0 con 0x7f6d88143540 2021-07-11T03:53:37.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d880b1b90 con 0x7f6d8805f6b0 2021-07-11T03:53:37.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d87fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6d880a8b60 0x7f6d880b0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39302/0 (socket says 172.21.2.104:39302) 2021-07-11T03:53:37.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d87fff700 1 -- 172.21.2.104:0/1357302314 learned_addr learned my addr 172.21.2.104:0/1357302314 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:37.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d877fe700 1 --2- 172.21.2.104:0/1357302314 >> v2:172.21.2.104:3300/0 conn(0x7f6d88143540 0x7f6d880a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d877fe700 1 -- 172.21.2.104:0/1357302314 >> v2:172.21.2.105:3300/0 conn(0x7f6d8805f6b0 msgr2=0x7f6d88143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d877fe700 1 --2- 172.21.2.104:0/1357302314 >> v2:172.21.2.105:3300/0 conn(0x7f6d8805f6b0 0x7f6d88143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d877fe700 1 -- 172.21.2.104:0/1357302314 >> v1:172.21.2.104:6789/0 conn(0x7f6d880a8b60 legacy=0x7f6d880b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d877fe700 1 -- 172.21.2.104:0/1357302314 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d88143fa0 con 0x7f6d88143540 2021-07-11T03:53:37.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d877fe700 1 --2- 172.21.2.104:0/1357302314 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d88143540 0x7f6d880a9300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f6d78002f70 tx=0x7f6d78006e80).ready entity=mon.0 client_cookie=338ce63280871a46 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d867fc700 1 -- 172.21.2.104:0/1357302314 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7800ba40 con 0x7f6d88143540 2021-07-11T03:53:37.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d867fc700 1 -- 172.21.2.104:0/1357302314 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d78000e90 con 0x7f6d88143540 2021-07-11T03:53:37.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d867fc700 1 -- 172.21.2.104:0/1357302314 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d7800bd90 con 0x7f6d88143540 2021-07-11T03:53:37.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1357302314 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d88143540 msgr2=0x7f6d880a9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 --2- 172.21.2.104:0/1357302314 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d88143540 0x7f6d880a9300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f6d78002f70 tx=0x7f6d78006e80).stop 2021-07-11T03:53:37.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1357302314 shutdown_connections 2021-07-11T03:53:37.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 --2- 172.21.2.104:0/1357302314 >> v2:172.21.2.105:3300/0 conn(0x7f6d8805f6b0 0x7f6d88143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 --2- 172.21.2.104:0/1357302314 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d88143540 0x7f6d880a9300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1357302314 >> 172.21.2.104:0/1357302314 conn(0x7f6d880b53e0 msgr2=0x7f6d880b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1357302314 shutdown_connections 2021-07-11T03:53:37.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1357302314 wait complete. 2021-07-11T03:53:37.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 Processor -- start 2021-07-11T03:53:37.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- start start 2021-07-11T03:53:37.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d8805f6b0 0x7f6d880a7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d880a8b60 0x7f6d881400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d88143540 0x7f6d881405a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d880a7c00 con 0x7f6d880a8b60 2021-07-11T03:53:37.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8813f460 con 0x7f6d88143540 2021-07-11T03:53:37.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d8e02e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8813e940 con 0x7f6d8805f6b0 2021-07-11T03:53:37.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d86ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d880a8b60 0x7f6d881400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d86ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d880a8b60 0x7f6d881400b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42536/0 (socket says 172.21.2.104:42536) 2021-07-11T03:53:37.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.496+0000 7f6d86ffd700 1 -- 172.21.2.104:0/1663920828 learned_addr learned my addr 172.21.2.104:0/1663920828 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:37.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d86ffd700 1 -- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d8805f6b0 msgr2=0x7f6d880a7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d86ffd700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d8805f6b0 0x7f6d880a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d86ffd700 1 -- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d88143540 msgr2=0x7f6d881405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d86ffd700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d88143540 0x7f6d881405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d86ffd700 1 -- 172.21.2.104:0/1663920828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7800d410 con 0x7f6d880a8b60 2021-07-11T03:53:37.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d86ffd700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d880a8b60 0x7f6d881400b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f6d880a7d40 tx=0x7f6d70006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d84ff9700 1 -- 172.21.2.104:0/1663920828 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d70013460 con 0x7f6d880a8b60 2021-07-11T03:53:37.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d84ff9700 1 -- 172.21.2.104:0/1663920828 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d700135c0 con 0x7f6d880a8b60 2021-07-11T03:53:37.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1663920828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8813f690 con 0x7f6d880a8b60 2021-07-11T03:53:37.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d84ff9700 1 -- 172.21.2.104:0/1663920828 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d70013920 con 0x7f6d880a8b60 2021-07-11T03:53:37.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1663920828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d88154650 con 0x7f6d880a8b60 2021-07-11T03:53:37.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1663920828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d88057350 con 0x7f6d880a8b60 2021-07-11T03:53:37.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d84ff9700 1 -- 172.21.2.104:0/1663920828 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6d7001a060 con 0x7f6d880a8b60 2021-07-11T03:53:37.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d84ff9700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d6c0531a0 0x7f6d6c055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d84ff9700 1 -- 172.21.2.104:0/1663920828 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6d7006afe0 con 0x7f6d880a8b60 2021-07-11T03:53:37.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d877fe700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d6c0531a0 0x7f6d6c055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.500+0000 7f6d877fe700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d6c0531a0 0x7f6d6c055660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f6d78002f70 tx=0x7f6d880ab6e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.504+0000 7f6d84ff9700 1 -- 172.21.2.104:0/1663920828 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6d70042270 con 0x7f6d880a8b60 2021-07-11T03:53:37.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1663920828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d880aaf50 con 0x7f6d880a8b60 2021-07-11T03:53:37.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d84ff9700 1 -- 172.21.2.104:0/1663920828 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f6d70018070 con 0x7f6d880a8b60 2021-07-11T03:53:37.588 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:37.588 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":23,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:24.367517+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:37.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d6c0531a0 msgr2=0x7f6d6c055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d6c0531a0 0x7f6d6c055660 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f6d78002f70 tx=0x7f6d880ab6e0).stop 2021-07-11T03:53:37.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1663920828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d880a8b60 msgr2=0x7f6d881400b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d880a8b60 0x7f6d881400b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f6d880a7d40 tx=0x7f6d70006ec0).stop 2021-07-11T03:53:37.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1663920828 shutdown_connections 2021-07-11T03:53:37.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d88143540 0x7f6d881405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d6c0531a0 0x7f6d6c055660 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d880a8b60 0x7f6d881400b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 --2- 172.21.2.104:0/1663920828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d8805f6b0 0x7f6d880a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1663920828 >> 172.21.2.104:0/1663920828 conn(0x7f6d880b53e0 msgr2=0x7f6d88141300 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1663920828 shutdown_connections 2021-07-11T03:53:37.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.584+0000 7f6d8e02e700 1 -- 172.21.2.104:0/1663920828 wait complete. 2021-07-11T03:53:37.595 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:53:20.980780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:53:21.341375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:53:37.596 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:53:37.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.644+0000 7f94c6456700 1 Processor -- start 2021-07-11T03:53:37.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.644+0000 7f94c6456700 1 -- start start 2021-07-11T03:53:37.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.644+0000 7f94c6456700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94c0144440 0x7f94c0144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.644+0000 7f94c6456700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f94c0065420 0x7f94c0144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.644+0000 7f94c6456700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c0145350 con 0x7f94c0060100 2021-07-11T03:53:37.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.644+0000 7f94c6456700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c0065c80 con 0x7f94c0065420 2021-07-11T03:53:37.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.644+0000 7f94c6456700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00604e0 con 0x7f94c0144440 2021-07-11T03:53:37.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.644+0000 7f94c49f3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f94c0065420 0x7f94c0144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.644+0000 7f94c49f3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f94c0065420 0x7f94c0144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42546/0 (socket says 172.21.2.104:42546) 2021-07-11T03:53:37.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.644+0000 7f94c49f3700 1 -- 172.21.2.104:0/2182279662 learned_addr learned my addr 172.21.2.104:0/2182279662 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:37.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 -- 172.21.2.104:0/2182279662 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94c0144440 msgr2=0x7f94c0144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 --2- 172.21.2.104:0/2182279662 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94c0144440 0x7f94c0144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 -- 172.21.2.104:0/2182279662 >> v1:172.21.2.105:6789/0 conn(0x7f94c0060100 legacy=0x7f94c0064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 -- 172.21.2.104:0/2182279662 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c00b23b0 con 0x7f94c0065420 2021-07-11T03:53:37.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 --2- 172.21.2.104:0/2182279662 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94c0065420 0x7f94c0144e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f94c0069be0 tx=0x7f94b8009290).ready entity=mon.0 client_cookie=900ba098168c933c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94beffd700 1 -- 172.21.2.104:0/2182279662 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94b800b860 con 0x7f94c0065420 2021-07-11T03:53:37.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94beffd700 1 -- 172.21.2.104:0/2182279662 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b800b9c0 con 0x7f94c0065420 2021-07-11T03:53:37.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94beffd700 1 -- 172.21.2.104:0/2182279662 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94b800bb70 con 0x7f94c0065420 2021-07-11T03:53:37.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 -- 172.21.2.104:0/2182279662 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94c0065420 msgr2=0x7f94c0144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 --2- 172.21.2.104:0/2182279662 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94c0065420 0x7f94c0144e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f94c0069be0 tx=0x7f94b8009290).stop 2021-07-11T03:53:37.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 -- 172.21.2.104:0/2182279662 shutdown_connections 2021-07-11T03:53:37.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 --2- 172.21.2.104:0/2182279662 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94c0065420 0x7f94c0144e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 --2- 172.21.2.104:0/2182279662 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94c0144440 0x7f94c0144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 -- 172.21.2.104:0/2182279662 >> 172.21.2.104:0/2182279662 conn(0x7f94c005cf60 msgr2=0x7f94c005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 -- 172.21.2.104:0/2182279662 shutdown_connections 2021-07-11T03:53:37.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 -- 172.21.2.104:0/2182279662 wait complete. 2021-07-11T03:53:37.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 Processor -- start 2021-07-11T03:53:37.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 -- start start 2021-07-11T03:53:37.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94c0060100 0x7f94c014e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f94c0065420 0x7f94c0145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94c013ed10 0x7f94c01406b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00af6f0 con 0x7f94c013ed10 2021-07-11T03:53:37.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00b0bd0 con 0x7f94c0065420 2021-07-11T03:53:37.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c00b06d0 con 0x7f94c0060100 2021-07-11T03:53:37.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94c013ed10 0x7f94c01406b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94c013ed10 0x7f94c01406b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42554/0 (socket says 172.21.2.104:42554) 2021-07-11T03:53:37.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 -- 172.21.2.104:0/1747020187 learned_addr learned my addr 172.21.2.104:0/1747020187 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:37.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 -- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94c0060100 msgr2=0x7f94c014e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94c0060100 0x7f94c014e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 -- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f94c0065420 msgr2=0x7f94c0145ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f94c0065420 0x7f94c0145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 -- 172.21.2.104:0/1747020187 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b8009530 con 0x7f94c013ed10 2021-07-11T03:53:37.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c49f3700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94c013ed10 0x7f94c01406b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f94c013f1a0 tx=0x7f94b8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94bd7fa700 1 -- 172.21.2.104:0/1747020187 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94b8011c80 con 0x7f94c013ed10 2021-07-11T03:53:37.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94bd7fa700 1 -- 172.21.2.104:0/1747020187 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b8011de0 con 0x7f94c013ed10 2021-07-11T03:53:37.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 -- 172.21.2.104:0/1747020187 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c00af920 con 0x7f94c013ed10 2021-07-11T03:53:37.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94c6456700 1 -- 172.21.2.104:0/1747020187 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c01463b0 con 0x7f94c013ed10 2021-07-11T03:53:37.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.648+0000 7f94bd7fa700 1 -- 172.21.2.104:0/1747020187 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f94b8020620 con 0x7f94c013ed10 2021-07-11T03:53:37.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.652+0000 7f94c6456700 1 -- 172.21.2.104:0/1747020187 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c01403a0 con 0x7f94c013ed10 2021-07-11T03:53:37.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.652+0000 7f94bd7fa700 1 -- 172.21.2.104:0/1747020187 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f94b800f070 con 0x7f94c013ed10 2021-07-11T03:53:37.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.652+0000 7f94bd7fa700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94a4053330 0x7f94a40557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.652+0000 7f94bd7fa700 1 -- 172.21.2.104:0/1747020187 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f94b806c540 con 0x7f94c013ed10 2021-07-11T03:53:37.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.652+0000 7f94bffff700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94a4053330 0x7f94a40557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.652+0000 7f94bffff700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94a4053330 0x7f94a40557f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f94a8002f90 tx=0x7f94a80069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.652+0000 7f94bd7fa700 1 -- 172.21.2.104:0/1747020187 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f94b8043930 con 0x7f94c013ed10 2021-07-11T03:53:37.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 -- 172.21.2.104:0/1747020187 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f94c0057350 con 0x7f94c013ed10 2021-07-11T03:53:37.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94bd7fa700 1 -- 172.21.2.104:0/1747020187 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f94b801b0b0 con 0x7f94c013ed10 2021-07-11T03:53:37.740 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:53:37.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 -- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94a4053330 msgr2=0x7f94a40557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94a4053330 0x7f94a40557f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f94a8002f90 tx=0x7f94a80069b0).stop 2021-07-11T03:53:37.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 -- 172.21.2.104:0/1747020187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94c013ed10 msgr2=0x7f94c01406b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94c013ed10 0x7f94c01406b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f94c013f1a0 tx=0x7f94b8016000).stop 2021-07-11T03:53:37.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 -- 172.21.2.104:0/1747020187 shutdown_connections 2021-07-11T03:53:37.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94a4053330 0x7f94a40557f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94c013ed10 0x7f94c01406b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f94c0065420 0x7f94c0145ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 --2- 172.21.2.104:0/1747020187 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94c0060100 0x7f94c014e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 -- 172.21.2.104:0/1747020187 >> 172.21.2.104:0/1747020187 conn(0x7f94c005cf60 msgr2=0x7f94c00b6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 -- 172.21.2.104:0/1747020187 shutdown_connections 2021-07-11T03:53:37.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.736+0000 7f94c6456700 1 -- 172.21.2.104:0/1747020187 wait complete. 2021-07-11T03:53:37.747 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:53:37.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbfd49b700 1 Processor -- start 2021-07-11T03:53:37.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbfd49b700 1 -- start start 2021-07-11T03:53:37.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbfd49b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fbbf80b1ec0 0x7fbbf80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbfd49b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbbf80b22a0 0x7fbbf8141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbfd49b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf80b0420 con 0x7fbbf80affb0 2021-07-11T03:53:37.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbfd49b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf80a77d0 con 0x7fbbf80b1ec0 2021-07-11T03:53:37.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbfd49b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80b0650 con 0x7fbbf80b22a0 2021-07-11T03:53:37.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf67fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fbbf80affb0 0x7fbbf80a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39328/0 (socket says 172.21.2.104:39328) 2021-07-11T03:53:37.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf67fc700 1 -- 172.21.2.104:0/347355525 learned_addr learned my addr 172.21.2.104:0/347355525 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:37.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752634847 0 0) 0x7fbbf80b0420 con 0x7fbbf80affb0 2021-07-11T03:53:37.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbd8003260 con 0x7fbbf80affb0 2021-07-11T03:53:37.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2005489767 0 0) 0x7fbbd8003260 con 0x7fbbf80affb0 2021-07-11T03:53:37.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbbf80b22a0 msgr2=0x7fbbf8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 --2- 172.21.2.104:0/347355525 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbbf80b22a0 0x7fbbf8141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 >> v2:172.21.2.105:3300/0 conn(0x7fbbf80b1ec0 msgr2=0x7fbbf80b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 --2- 172.21.2.104:0/347355525 >> v2:172.21.2.105:3300/0 conn(0x7fbbf80b1ec0 0x7fbbf80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf80b08f0 con 0x7fbbf80affb0 2021-07-11T03:53:37.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fbbe0001660 con 0x7fbbf80affb0 2021-07-11T03:53:37.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 >> v1:172.21.2.104:6789/0 conn(0x7fbbf80affb0 legacy=0x7fbbf80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 --2- 172.21.2.104:0/347355525 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbbd80037d0 0x7fbbd8003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 --2- 172.21.2.104:0/347355525 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbd8004500 0x7fbbd8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 --2- 172.21.2.104:0/347355525 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbbf80b22a0 0x7fbbd800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80a77d0 con 0x7fbbd8004500 2021-07-11T03:53:37.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf80b0650 con 0x7fbbd80037d0 2021-07-11T03:53:37.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.796+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd8005300 con 0x7fbbf80b22a0 2021-07-11T03:53:37.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 --2- 172.21.2.104:0/347355525 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbd8004500 0x7fbbd8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 -- 172.21.2.104:0/347355525 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbbf80b22a0 msgr2=0x7fbbd800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 --2- 172.21.2.104:0/347355525 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbbf80b22a0 0x7fbbd800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 -- 172.21.2.104:0/347355525 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbbd80037d0 msgr2=0x7fbbd8003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 --2- 172.21.2.104:0/347355525 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbbd80037d0 0x7fbbd8003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 -- 172.21.2.104:0/347355525 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd8003260 con 0x7fbbd8004500 2021-07-11T03:53:37.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 --2- 172.21.2.104:0/347355525 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbd8004500 0x7fbbd8004980 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fbbe0001370 tx=0x7fbbf80a7910).ready entity=mon.0 client_cookie=76d56f0111815b6d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbe00070d0 con 0x7fbbd8004500 2021-07-11T03:53:37.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe0007230 con 0x7fbbd8004500 2021-07-11T03:53:37.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf5ffb700 1 -- 172.21.2.104:0/347355525 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbe000c8f0 con 0x7fbbd8004500 2021-07-11T03:53:37.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- 172.21.2.104:0/347355525 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbd8004500 msgr2=0x7fbbd8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 --2- 172.21.2.104:0/347355525 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbd8004500 0x7fbbd8004980 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fbbe0001370 tx=0x7fbbf80a7910).stop 2021-07-11T03:53:37.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf77fe700 1 -- 172.21.2.104:0/347355525 reap_dead start 2021-07-11T03:53:37.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- 172.21.2.104:0/347355525 shutdown_connections 2021-07-11T03:53:37.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- 172.21.2.104:0/347355525 >> 172.21.2.104:0/347355525 conn(0x7fbbf805f6b0 msgr2=0x7fbbf805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- 172.21.2.104:0/347355525 shutdown_connections 2021-07-11T03:53:37.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- 172.21.2.104:0/347355525 wait complete. 2021-07-11T03:53:37.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 Processor -- start 2021-07-11T03:53:37.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- start start 2021-07-11T03:53:37.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbbf8147aa0 0x7fbbf8138800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbf813e050 0x7fbbf8138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbbf814b4e0 0x7fbbf813c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf8147e80 con 0x7fbbf813e050 2021-07-11T03:53:37.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf814ad20 con 0x7fbbf8147aa0 2021-07-11T03:53:37.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf813e560 con 0x7fbbf814b4e0 2021-07-11T03:53:37.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbf813e050 0x7fbbf8138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbf813e050 0x7fbbf8138cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42572/0 (socket says 172.21.2.104:42572) 2021-07-11T03:53:37.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 -- 172.21.2.104:0/1777679884 learned_addr learned my addr 172.21.2.104:0/1777679884 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:37.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf6ffd700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbbf8147aa0 0x7fbbf8138800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf77fe700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbbf814b4e0 0x7fbbf813c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 -- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbbf814b4e0 msgr2=0x7fbbf813c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbbf814b4e0 0x7fbbf813c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 -- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbbf8147aa0 msgr2=0x7fbbf8138800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbbf8147aa0 0x7fbbf8138800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 -- 172.21.2.104:0/1777679884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf813b920 con 0x7fbbf813e050 2021-07-11T03:53:37.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbf67fc700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbf813e050 0x7fbbf8138cf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fbbe0001370 tx=0x7fbbe0013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbeffff700 1 -- 172.21.2.104:0/1777679884 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbe001a070 con 0x7fbbf813e050 2021-07-11T03:53:37.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbeffff700 1 -- 172.21.2.104:0/1777679884 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe0001bc0 con 0x7fbbf813e050 2021-07-11T03:53:37.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- 172.21.2.104:0/1777679884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf813bbc0 con 0x7fbbf813e050 2021-07-11T03:53:37.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbeffff700 1 -- 172.21.2.104:0/1777679884 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbbe00049e0 con 0x7fbbf813e050 2021-07-11T03:53:37.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- 172.21.2.104:0/1777679884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf813c550 con 0x7fbbf813e050 2021-07-11T03:53:37.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.800+0000 7fbbfd49b700 1 -- 172.21.2.104:0/1777679884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf8057350 con 0x7fbbf813e050 2021-07-11T03:53:37.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.804+0000 7fbbeffff700 1 -- 172.21.2.104:0/1777679884 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbbe0010070 con 0x7fbbf813e050 2021-07-11T03:53:37.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.804+0000 7fbbeffff700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbbdc0532b0 0x7fbbdc055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.804+0000 7fbbeffff700 1 -- 172.21.2.104:0/1777679884 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbbe006b450 con 0x7fbbf813e050 2021-07-11T03:53:37.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.804+0000 7fbbf6ffd700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbbdc0532b0 0x7fbbdc055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.804+0000 7fbbf6ffd700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbbdc0532b0 0x7fbbdc055770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fbbe8007000 tx=0x7fbbe8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.804+0000 7fbbeffff700 1 -- 172.21.2.104:0/1777679884 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbbe0042680 con 0x7fbbf813e050 2021-07-11T03:53:37.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.884+0000 7fbbfd49b700 1 -- 172.21.2.104:0/1777679884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbbf814bb60 con 0x7fbbf813e050 2021-07-11T03:53:37.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.884+0000 7fbbeffff700 1 -- 172.21.2.104:0/1777679884 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbbe0045ca0 con 0x7fbbf813e050 2021-07-11T03:53:37.890 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:37.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 -- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbbdc0532b0 msgr2=0x7fbbdc055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbbdc0532b0 0x7fbbdc055770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7fbbe8007000 tx=0x7fbbe8008040).stop 2021-07-11T03:53:37.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 -- 172.21.2.104:0/1777679884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbf813e050 msgr2=0x7fbbf8138cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbf813e050 0x7fbbf8138cf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fbbe0001370 tx=0x7fbbe0013000).stop 2021-07-11T03:53:37.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 -- 172.21.2.104:0/1777679884 shutdown_connections 2021-07-11T03:53:37.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbbf814b4e0 0x7fbbf813c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbbdc0532b0 0x7fbbdc055770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbbf813e050 0x7fbbf8138cf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 --2- 172.21.2.104:0/1777679884 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbbf8147aa0 0x7fbbf8138800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 -- 172.21.2.104:0/1777679884 >> 172.21.2.104:0/1777679884 conn(0x7fbbf805f6b0 msgr2=0x7fbbf8140020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 -- 172.21.2.104:0/1777679884 shutdown_connections 2021-07-11T03:53:37.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.888+0000 7fbbfd49b700 1 -- 172.21.2.104:0/1777679884 wait complete. 2021-07-11T03:53:37.897 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:53:37.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.944+0000 7f51bbe98700 1 Processor -- start 2021-07-11T03:53:37.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.944+0000 7f51bbe98700 1 -- start start 2021-07-11T03:53:37.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f51b4060100 0x7f51b4064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f51b40b38d0 0x7f51b4065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b40604e0 con 0x7f51b4144440 2021-07-11T03:53:37.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b40656f0 con 0x7f51b4060100 2021-07-11T03:53:37.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b4065c80 con 0x7f51b40b38d0 2021-07-11T03:53:37.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9433700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f51b4144440 0x7f51b4144970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39348/0 (socket says 172.21.2.104:39348) 2021-07-11T03:53:37.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9433700 1 -- 172.21.2.104:0/3294624393 learned_addr learned my addr 172.21.2.104:0/3294624393 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:37.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 --2- 172.21.2.104:0/3294624393 >> v2:172.21.2.104:3300/0 conn(0x7f51b4060100 0x7f51b4064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 -- 172.21.2.104:0/3294624393 >> v2:172.21.2.105:3300/0 conn(0x7f51b40b38d0 msgr2=0x7f51b4065200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:37.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b8c32700 1 -- 172.21.2.104:0/3294624393 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655144706 0 0) 0x7f51b40604e0 con 0x7f51b4144440 2021-07-11T03:53:37.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 --2- 172.21.2.104:0/3294624393 >> v2:172.21.2.105:3300/0 conn(0x7f51b40b38d0 0x7f51b4065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 -- 172.21.2.104:0/3294624393 >> v1:172.21.2.104:6789/0 conn(0x7f51b4144440 legacy=0x7f51b4144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 -- 172.21.2.104:0/3294624393 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b40b23b0 con 0x7f51b4060100 2021-07-11T03:53:37.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 --2- 172.21.2.104:0/3294624393 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51b4060100 0x7f51b4064d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f51a8002f70 tx=0x7f51a8006df0).ready entity=mon.0 client_cookie=2465cd9b1da46870 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b8c32700 1 -- 172.21.2.104:0/3294624393 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a800d5c0 con 0x7f51b4060100 2021-07-11T03:53:37.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b8c32700 1 -- 172.21.2.104:0/3294624393 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a8000e90 con 0x7f51b4060100 2021-07-11T03:53:37.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b8c32700 1 -- 172.21.2.104:0/3294624393 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a800d910 con 0x7f51b4060100 2021-07-11T03:53:37.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- 172.21.2.104:0/3294624393 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51b4060100 msgr2=0x7f51b4064d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 --2- 172.21.2.104:0/3294624393 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51b4060100 0x7f51b4064d10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f51a8002f70 tx=0x7f51a8006df0).stop 2021-07-11T03:53:37.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- 172.21.2.104:0/3294624393 shutdown_connections 2021-07-11T03:53:37.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 --2- 172.21.2.104:0/3294624393 >> v2:172.21.2.105:3300/0 conn(0x7f51b40b38d0 0x7f51b4065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 --2- 172.21.2.104:0/3294624393 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51b4060100 0x7f51b4064d10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:37.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- 172.21.2.104:0/3294624393 >> 172.21.2.104:0/3294624393 conn(0x7f51b405cf60 msgr2=0x7f51b405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:37.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- 172.21.2.104:0/3294624393 shutdown_connections 2021-07-11T03:53:37.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- 172.21.2.104:0/3294624393 wait complete. 2021-07-11T03:53:37.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 Processor -- start 2021-07-11T03:53:37.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- start start 2021-07-11T03:53:37.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51b4060100 0x7f51b41396e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f51b40b38d0 0x7f51b40b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f51b4143a40 0x7f51b40b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b40b5bc0 con 0x7f51b4060100 2021-07-11T03:53:37.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b4136650 con 0x7f51b40b38d0 2021-07-11T03:53:37.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b41368f0 con 0x7f51b4143a40 2021-07-11T03:53:37.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51b4060100 0x7f51b41396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51b4060100 0x7f51b41396e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42584/0 (socket says 172.21.2.104:42584) 2021-07-11T03:53:37.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 -- 172.21.2.104:0/3543994299 learned_addr learned my addr 172.21.2.104:0/3543994299 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:37.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 -- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f51b4143a40 msgr2=0x7f51b40b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f51b4143a40 0x7f51b40b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 -- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f51b40b38d0 msgr2=0x7f51b40b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:37.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f51b40b38d0 0x7f51b40b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:37.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 -- 172.21.2.104:0/3543994299 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a800b180 con 0x7f51b4060100 2021-07-11T03:53:37.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51b9c34700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51b4060100 0x7f51b41396e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f51a8002f70 tx=0x7f51a8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51a6ffd700 1 -- 172.21.2.104:0/3543994299 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a8006b20 con 0x7f51b4060100 2021-07-11T03:53:37.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51a6ffd700 1 -- 172.21.2.104:0/3543994299 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a801f440 con 0x7f51b4060100 2021-07-11T03:53:37.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- 172.21.2.104:0/3543994299 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b41432d0 con 0x7f51b4060100 2021-07-11T03:53:37.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51a6ffd700 1 -- 172.21.2.104:0/3543994299 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51a801f7a0 con 0x7f51b4060100 2021-07-11T03:53:37.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.948+0000 7f51bbe98700 1 -- 172.21.2.104:0/3543994299 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b4143770 con 0x7f51b4060100 2021-07-11T03:53:37.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.952+0000 7f51bbe98700 1 -- 172.21.2.104:0/3543994299 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b4057350 con 0x7f51b4060100 2021-07-11T03:53:37.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.952+0000 7f51a6ffd700 1 -- 172.21.2.104:0/3543994299 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f51a8025020 con 0x7f51b4060100 2021-07-11T03:53:37.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.952+0000 7f51a6ffd700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5194053160 0x7f5194055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:37.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.952+0000 7f51a6ffd700 1 -- 172.21.2.104:0/3543994299 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f51a806c730 con 0x7f51b4060100 2021-07-11T03:53:37.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.952+0000 7f51b9433700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5194053160 0x7f5194055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:37.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.952+0000 7f51b9433700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5194053160 0x7f5194055620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f51b0000ed0 tx=0x7f51b00078a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:37.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:37.952+0000 7f51a6ffd700 1 -- 172.21.2.104:0/3543994299 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f51a8043a70 con 0x7f51b4060100 2021-07-11T03:53:38.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 -- 172.21.2.104:0/3543994299 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51b4136ab0 con 0x7f51b4060100 2021-07-11T03:53:38.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51a6ffd700 1 -- 172.21.2.104:0/3543994299 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f51a8047090 con 0x7f51b4060100 2021-07-11T03:53:38.041 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:38.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 -- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5194053160 msgr2=0x7f5194055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5194053160 0x7f5194055620 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f51b0000ed0 tx=0x7f51b00078a0).stop 2021-07-11T03:53:38.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 -- 172.21.2.104:0/3543994299 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51b4060100 msgr2=0x7f51b41396e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51b4060100 0x7f51b41396e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f51a8002f70 tx=0x7f51a8014000).stop 2021-07-11T03:53:38.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 -- 172.21.2.104:0/3543994299 shutdown_connections 2021-07-11T03:53:38.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f51b4143a40 0x7f51b40b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f51b40b38d0 0x7f51b40b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5194053160 0x7f5194055620 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 --2- 172.21.2.104:0/3543994299 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51b4060100 0x7f51b41396e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 -- 172.21.2.104:0/3543994299 >> 172.21.2.104:0/3543994299 conn(0x7f51b405cf60 msgr2=0x7f51b413a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 -- 172.21.2.104:0/3543994299 shutdown_connections 2021-07-11T03:53:38.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.036+0000 7f51bbe98700 1 -- 172.21.2.104:0/3543994299 wait complete. 2021-07-11T03:53:38.048 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:53:38.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.096+0000 7f8041575700 1 Processor -- start 2021-07-11T03:53:38.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.096+0000 7f8041575700 1 -- start start 2021-07-11T03:53:38.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f803c0a9ec0 0x7f803c0aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f803c0603d0 0x7f803c05c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f803c13ccd0 0x7f803c13f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803c061990 con 0x7f803c0a9ec0 2021-07-11T03:53:38.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803c061c30 con 0x7f803c13ccd0 2021-07-11T03:53:38.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c060b90 con 0x7f803c0603d0 2021-07-11T03:53:38.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803affd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f803c0a9ec0 0x7f803c0aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803affd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f803c0a9ec0 0x7f803c0aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42592/0 (socket says 172.21.2.104:42592) 2021-07-11T03:53:38.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803affd700 1 -- 172.21.2.104:0/3453850860 learned_addr learned my addr 172.21.2.104:0/3453850860 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:38.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803affd700 1 -- 172.21.2.104:0/3453850860 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f803c0603d0 msgr2=0x7f803c05c490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:38.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803affd700 1 --2- 172.21.2.104:0/3453850860 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f803c0603d0 0x7f803c05c490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803affd700 1 -- 172.21.2.104:0/3453850860 >> v2:172.21.2.105:3300/0 conn(0x7f803c13ccd0 msgr2=0x7f803c13f790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:38.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803affd700 1 --2- 172.21.2.104:0/3453850860 >> v2:172.21.2.105:3300/0 conn(0x7f803c13ccd0 0x7f803c13f790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803affd700 1 -- 172.21.2.104:0/3453850860 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c060e30 con 0x7f803c0a9ec0 2021-07-11T03:53:38.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803affd700 1 --2- 172.21.2.104:0/3453850860 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f803c0a9ec0 0x7f803c0aa2a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f802c002f70 tx=0x7f802c006eb0).ready entity=mon.0 client_cookie=b7a6e7917ec12e21 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8039ffb700 1 -- 172.21.2.104:0/3453850860 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f802c00d570 con 0x7f803c0a9ec0 2021-07-11T03:53:38.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8039ffb700 1 -- 172.21.2.104:0/3453850860 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c000e90 con 0x7f803c0a9ec0 2021-07-11T03:53:38.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8039ffb700 1 -- 172.21.2.104:0/3453850860 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f802c00d900 con 0x7f803c0a9ec0 2021-07-11T03:53:38.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- 172.21.2.104:0/3453850860 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f803c0a9ec0 msgr2=0x7f803c0aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 --2- 172.21.2.104:0/3453850860 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f803c0a9ec0 0x7f803c0aa2a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f802c002f70 tx=0x7f802c006eb0).stop 2021-07-11T03:53:38.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- 172.21.2.104:0/3453850860 shutdown_connections 2021-07-11T03:53:38.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 --2- 172.21.2.104:0/3453850860 >> v2:172.21.2.105:3300/0 conn(0x7f803c13ccd0 0x7f803c13f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 --2- 172.21.2.104:0/3453850860 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f803c0603d0 0x7f803c05c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 --2- 172.21.2.104:0/3453850860 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f803c0a9ec0 0x7f803c0aa2a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- 172.21.2.104:0/3453850860 >> 172.21.2.104:0/3453850860 conn(0x7f803c0b2970 msgr2=0x7f803c0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- 172.21.2.104:0/3453850860 shutdown_connections 2021-07-11T03:53:38.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- 172.21.2.104:0/3453850860 wait complete. 2021-07-11T03:53:38.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 Processor -- start 2021-07-11T03:53:38.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- start start 2021-07-11T03:53:38.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f803c0603d0 0x7f803c142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f803c0a9ec0 0x7f803c0a81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f803c13ccd0 0x7f803c0a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c15ae60 con 0x7f803c13ccd0 2021-07-11T03:53:38.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c13d6d0 con 0x7f803c0603d0 2021-07-11T03:53:38.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f8041575700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c13d970 con 0x7f803c0a9ec0 2021-07-11T03:53:38.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f803c13ccd0 0x7f803c0a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f803c13ccd0 0x7f803c0a8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42602/0 (socket says 172.21.2.104:42602) 2021-07-11T03:53:38.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803b7fe700 1 -- 172.21.2.104:0/3853031153 learned_addr learned my addr 172.21.2.104:0/3853031153 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:38.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803b7fe700 1 -- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f803c0a9ec0 msgr2=0x7f803c0a81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803b7fe700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f803c0a9ec0 0x7f803c0a81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803b7fe700 1 -- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f803c0603d0 msgr2=0x7f803c142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803b7fe700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f803c0603d0 0x7f803c142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803b7fe700 1 -- 172.21.2.104:0/3853031153 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c00b1e0 con 0x7f803c13ccd0 2021-07-11T03:53:38.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.100+0000 7f803b7fe700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f803c13ccd0 0x7f803c0a8ca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f8034001250 tx=0x7f80340090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f8033fff700 1 -- 172.21.2.104:0/3853031153 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80340014e0 con 0x7f803c13ccd0 2021-07-11T03:53:38.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f8033fff700 1 -- 172.21.2.104:0/3853031153 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034001640 con 0x7f803c13ccd0 2021-07-11T03:53:38.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f8033fff700 1 -- 172.21.2.104:0/3853031153 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80340019a0 con 0x7f803c13ccd0 2021-07-11T03:53:38.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f8041575700 1 -- 172.21.2.104:0/3853031153 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c13dc10 con 0x7f803c13ccd0 2021-07-11T03:53:38.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f8041575700 1 -- 172.21.2.104:0/3853031153 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c0a86e0 con 0x7f803c13ccd0 2021-07-11T03:53:38.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f8041575700 1 -- 172.21.2.104:0/3853031153 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803c13e2d0 con 0x7f803c13ccd0 2021-07-11T03:53:38.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f8033fff700 1 -- 172.21.2.104:0/3853031153 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8034018030 con 0x7f803c13ccd0 2021-07-11T03:53:38.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f8033fff700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80200532b0 0x7f8020055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f8033fff700 1 -- 172.21.2.104:0/3853031153 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f803406add0 con 0x7f803c13ccd0 2021-07-11T03:53:38.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f803affd700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80200532b0 0x7f8020055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f803affd700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80200532b0 0x7f8020055770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f802c002f70 tx=0x7f802c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.104+0000 7f8033fff700 1 -- 172.21.2.104:0/3853031153 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f803c13e2d0 con 0x7f803c13ccd0 2021-07-11T03:53:38.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.188+0000 7f8041575700 1 -- 172.21.2.104:0/3853031153 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f803c13e2d0 con 0x7f803c13ccd0 2021-07-11T03:53:38.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.188+0000 7f8033fff700 1 -- 172.21.2.104:0/3853031153 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f803c13e2d0 con 0x7f803c13ccd0 2021-07-11T03:53:38.193 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:38.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.188+0000 7f8041575700 1 -- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80200532b0 msgr2=0x7f8020055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.188+0000 7f8041575700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80200532b0 0x7f8020055770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f802c002f70 tx=0x7f802c015000).stop 2021-07-11T03:53:38.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.188+0000 7f8041575700 1 -- 172.21.2.104:0/3853031153 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f803c13ccd0 msgr2=0x7f803c0a8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.188+0000 7f8041575700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f803c13ccd0 0x7f803c0a8ca0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f8034001250 tx=0x7f80340090f0).stop 2021-07-11T03:53:38.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.192+0000 7f8041575700 1 -- 172.21.2.104:0/3853031153 shutdown_connections 2021-07-11T03:53:38.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.192+0000 7f8041575700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80200532b0 0x7f8020055770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.192+0000 7f8041575700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f803c13ccd0 0x7f803c0a8ca0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.192+0000 7f8041575700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f803c0a9ec0 0x7f803c0a81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.192+0000 7f8041575700 1 --2- 172.21.2.104:0/3853031153 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f803c0603d0 0x7f803c142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.192+0000 7f8041575700 1 -- 172.21.2.104:0/3853031153 >> 172.21.2.104:0/3853031153 conn(0x7f803c0b2970 msgr2=0x7f803c0b4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.192+0000 7f8041575700 1 -- 172.21.2.104:0/3853031153 shutdown_connections 2021-07-11T03:53:38.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.192+0000 7f8041575700 1 -- 172.21.2.104:0/3853031153 wait complete. 2021-07-11T03:53:38.201 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:38.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 Processor -- start 2021-07-11T03:53:38.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- start start 2021-07-11T03:53:38.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa0e80b3240 0x7fa0e805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa0e8065790 0x7fa0e8060ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa0e80613e0 0x7fa0e814aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e80b3620 con 0x7fa0e80613e0 2021-07-11T03:53:38.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e8060270 con 0x7fa0e8065790 2021-07-11T03:53:38.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e8060510 con 0x7fa0e80b3240 2021-07-11T03:53:38.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e6d9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa0e80613e0 0x7fa0e814aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e6d9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa0e80613e0 0x7fa0e814aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42606/0 (socket says 172.21.2.104:42606) 2021-07-11T03:53:38.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e6d9d700 1 -- 172.21.2.104:0/2518824973 learned_addr learned my addr 172.21.2.104:0/2518824973 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:38.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e6d9d700 1 -- 172.21.2.104:0/2518824973 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa0e80b3240 msgr2=0x7fa0e805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:38.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e6d9d700 1 --2- 172.21.2.104:0/2518824973 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa0e80b3240 0x7fa0e805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e6d9d700 1 -- 172.21.2.104:0/2518824973 >> v2:172.21.2.105:3300/0 conn(0x7fa0e8065790 msgr2=0x7fa0e8060ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:38.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e6d9d700 1 --2- 172.21.2.104:0/2518824973 >> v2:172.21.2.105:3300/0 conn(0x7fa0e8065790 0x7fa0e8060ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e6d9d700 1 -- 172.21.2.104:0/2518824973 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e80607b0 con 0x7fa0e80613e0 2021-07-11T03:53:38.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e6d9d700 1 --2- 172.21.2.104:0/2518824973 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa0e80613e0 0x7fa0e814aed0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fa0e00032f0 tx=0x7fa0e0009290).ready entity=mon.0 client_cookie=91396fbf5ecbac5d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e559a700 1 -- 172.21.2.104:0/2518824973 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0e000b870 con 0x7fa0e80613e0 2021-07-11T03:53:38.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e559a700 1 -- 172.21.2.104:0/2518824973 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e000b9d0 con 0x7fa0e80613e0 2021-07-11T03:53:38.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0e559a700 1 -- 172.21.2.104:0/2518824973 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0e000bb40 con 0x7fa0e80613e0 2021-07-11T03:53:38.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- 172.21.2.104:0/2518824973 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa0e80613e0 msgr2=0x7fa0e814aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 --2- 172.21.2.104:0/2518824973 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa0e80613e0 0x7fa0e814aed0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fa0e00032f0 tx=0x7fa0e0009290).stop 2021-07-11T03:53:38.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- 172.21.2.104:0/2518824973 shutdown_connections 2021-07-11T03:53:38.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 --2- 172.21.2.104:0/2518824973 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa0e80613e0 0x7fa0e814aed0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 --2- 172.21.2.104:0/2518824973 >> v2:172.21.2.105:3300/0 conn(0x7fa0e8065790 0x7fa0e8060ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 --2- 172.21.2.104:0/2518824973 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa0e80b3240 0x7fa0e805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- 172.21.2.104:0/2518824973 >> 172.21.2.104:0/2518824973 conn(0x7fa0e805c490 msgr2=0x7fa0e805c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- 172.21.2.104:0/2518824973 shutdown_connections 2021-07-11T03:53:38.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- 172.21.2.104:0/2518824973 wait complete. 2021-07-11T03:53:38.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 Processor -- start 2021-07-11T03:53:38.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- start start 2021-07-11T03:53:38.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa0e80613e0 0x7fa0e80af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa0e8065790 0x7fa0e814cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa0e80b3240 0x7fa0e814d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e814b760 con 0x7fa0e8065790 2021-07-11T03:53:38.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80acdb0 con 0x7fa0e80613e0 2021-07-11T03:53:38.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.260+0000 7fa0eceff700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80ad050 con 0x7fa0e80b3240 2021-07-11T03:53:38.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0e5d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa0e8065790 0x7fa0e814cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0e5d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa0e8065790 0x7fa0e814cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42614/0 (socket says 172.21.2.104:42614) 2021-07-11T03:53:38.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0e5d9b700 1 -- 172.21.2.104:0/1643577107 learned_addr learned my addr 172.21.2.104:0/1643577107 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:38.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0e5d9b700 1 -- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa0e80b3240 msgr2=0x7fa0e814d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0e5d9b700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa0e80b3240 0x7fa0e814d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0e5d9b700 1 -- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa0e80613e0 msgr2=0x7fa0e80af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0e5d9b700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa0e80613e0 0x7fa0e80af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0e5d9b700 1 -- 172.21.2.104:0/1643577107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e0009530 con 0x7fa0e8065790 2021-07-11T03:53:38.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0e5d9b700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa0e8065790 0x7fa0e814cca0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa0d8002f70 tx=0x7fa0d8006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0d77fe700 1 -- 172.21.2.104:0/1643577107 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d8013460 con 0x7fa0e8065790 2021-07-11T03:53:38.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0d77fe700 1 -- 172.21.2.104:0/1643577107 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d80135c0 con 0x7fa0e8065790 2021-07-11T03:53:38.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0d77fe700 1 -- 172.21.2.104:0/1643577107 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa0d8013920 con 0x7fa0e8065790 2021-07-11T03:53:38.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0eceff700 1 -- 172.21.2.104:0/1643577107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e80ad2f0 con 0x7fa0e8065790 2021-07-11T03:53:38.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0eceff700 1 -- 172.21.2.104:0/1643577107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e80afb10 con 0x7fa0e8065790 2021-07-11T03:53:38.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0eceff700 1 -- 172.21.2.104:0/1643577107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e8057350 con 0x7fa0e8065790 2021-07-11T03:53:38.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0d77fe700 1 -- 172.21.2.104:0/1643577107 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa0d801a060 con 0x7fa0e8065790 2021-07-11T03:53:38.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0d77fe700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa0c40532d0 0x7fa0c4055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0d77fe700 1 -- 172.21.2.104:0/1643577107 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa0d806afd0 con 0x7fa0e8065790 2021-07-11T03:53:38.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0e659c700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa0c40532d0 0x7fa0c4055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.264+0000 7fa0e659c700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa0c40532d0 0x7fa0c4055790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa0dc002f70 tx=0x7fa0dc006990).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.268+0000 7fa0d77fe700 1 -- 172.21.2.104:0/1643577107 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa0d8042240 con 0x7fa0e8065790 2021-07-11T03:53:38.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.348+0000 7fa0eceff700 1 -- 172.21.2.104:0/1643577107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0e805d760 con 0x7fa0e8065790 2021-07-11T03:53:38.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.348+0000 7fa0d77fe700 1 -- 172.21.2.104:0/1643577107 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16038 (secure 0 0 0) 0x7fa0d8018070 con 0x7fa0e8065790 2021-07-11T03:53:38.353 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:38.354 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":23,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:24.367517+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:38.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 -- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa0c40532d0 msgr2=0x7fa0c4055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa0c40532d0 0x7fa0c4055790 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa0dc002f70 tx=0x7fa0dc006990).stop 2021-07-11T03:53:38.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 -- 172.21.2.104:0/1643577107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa0e8065790 msgr2=0x7fa0e814cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa0e8065790 0x7fa0e814cca0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa0d8002f70 tx=0x7fa0d8006ec0).stop 2021-07-11T03:53:38.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 -- 172.21.2.104:0/1643577107 shutdown_connections 2021-07-11T03:53:38.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa0e80b3240 0x7fa0e814d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa0c40532d0 0x7fa0c4055790 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa0e8065790 0x7fa0e814cca0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 --2- 172.21.2.104:0/1643577107 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa0e80613e0 0x7fa0e80af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 -- 172.21.2.104:0/1643577107 >> 172.21.2.104:0/1643577107 conn(0x7fa0e805c490 msgr2=0x7fa0e80b43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 -- 172.21.2.104:0/1643577107 shutdown_connections 2021-07-11T03:53:38.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.352+0000 7fa0eceff700 1 -- 172.21.2.104:0/1643577107 wait complete. 2021-07-11T03:53:38.361 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:53:20.980780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:53:21.341375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:53:38.362 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:53:38.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 Processor -- start 2021-07-11T03:53:38.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 -- start start 2021-07-11T03:53:38.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8a880af2c0 0x7f8a880af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8a880ad850 0x7f8a880a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a880afda0 con 0x7f8a880ad320 2021-07-11T03:53:38.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8805c610 con 0x7f8a880af2c0 2021-07-11T03:53:38.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a88140800 con 0x7f8a880ad850 2021-07-11T03:53:38.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a87fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8a880ad320 0x7f8a88143150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39386/0 (socket says 172.21.2.104:39386) 2021-07-11T03:53:38.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8cb4b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8a880af2c0 0x7f8a880af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a87fff700 1 -- 172.21.2.104:0/4219900836 learned_addr learned my addr 172.21.2.104:0/4219900836 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:38.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a877fe700 1 -- 172.21.2.104:0/4219900836 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055216667 0 0) 0x7f8a880afda0 con 0x7f8a880ad320 2021-07-11T03:53:38.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a877fe700 1 -- 172.21.2.104:0/4219900836 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a74003260 con 0x7f8a880ad320 2021-07-11T03:53:38.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8cb4b700 1 -- 172.21.2.104:0/4219900836 >> v2:172.21.2.105:3300/0 conn(0x7f8a880ad850 msgr2=0x7f8a880a8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:38.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8cb4b700 1 --2- 172.21.2.104:0/4219900836 >> v2:172.21.2.105:3300/0 conn(0x7f8a880ad850 0x7f8a880a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8cb4b700 1 -- 172.21.2.104:0/4219900836 >> v1:172.21.2.104:6789/0 conn(0x7f8a880ad320 legacy=0x7f8a88143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8cb4b700 1 -- 172.21.2.104:0/4219900836 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a88143860 con 0x7f8a880af2c0 2021-07-11T03:53:38.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8cb4b700 1 --2- 172.21.2.104:0/4219900836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a880af2c0 0x7f8a880af6a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f8a880ad770 tx=0x7f8a7c006df0).ready entity=mon.0 client_cookie=3d85c095ba372fc4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a877fe700 1 -- 172.21.2.104:0/4219900836 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a7c00d5c0 con 0x7f8a880af2c0 2021-07-11T03:53:38.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a877fe700 1 -- 172.21.2.104:0/4219900836 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c000e90 con 0x7f8a880af2c0 2021-07-11T03:53:38.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a877fe700 1 -- 172.21.2.104:0/4219900836 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a7c00d910 con 0x7f8a880af2c0 2021-07-11T03:53:38.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 -- 172.21.2.104:0/4219900836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a880af2c0 msgr2=0x7f8a880af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 --2- 172.21.2.104:0/4219900836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a880af2c0 0x7f8a880af6a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f8a880ad770 tx=0x7f8a7c006df0).stop 2021-07-11T03:53:38.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 -- 172.21.2.104:0/4219900836 shutdown_connections 2021-07-11T03:53:38.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 --2- 172.21.2.104:0/4219900836 >> v2:172.21.2.105:3300/0 conn(0x7f8a880ad850 0x7f8a880a8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 --2- 172.21.2.104:0/4219900836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a880af2c0 0x7f8a880af6a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 -- 172.21.2.104:0/4219900836 >> 172.21.2.104:0/4219900836 conn(0x7f8a88061140 msgr2=0x7f8a88062e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 -- 172.21.2.104:0/4219900836 shutdown_connections 2021-07-11T03:53:38.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.412+0000 7f8a8db4d700 1 -- 172.21.2.104:0/4219900836 wait complete. 2021-07-11T03:53:38.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8db4d700 1 Processor -- start 2021-07-11T03:53:38.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8db4d700 1 -- start start 2021-07-11T03:53:38.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8db4d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a880ad320 0x7f8a88064620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8db4d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a880ad850 0x7f8a88152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8db4d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a880af2c0 0x7f8a88152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8db4d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880a9450 con 0x7f8a880ad850 2021-07-11T03:53:38.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8db4d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a880a9680 con 0x7f8a880ad320 2021-07-11T03:53:38.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8db4d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8814f9f0 con 0x7f8a880af2c0 2021-07-11T03:53:38.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a87fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a880ad850 0x7f8a88152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a87fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a880ad850 0x7f8a88152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42624/0 (socket says 172.21.2.104:42624) 2021-07-11T03:53:38.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a87fff700 1 -- 172.21.2.104:0/3344737598 learned_addr learned my addr 172.21.2.104:0/3344737598 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:38.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a87fff700 1 -- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a880af2c0 msgr2=0x7f8a88152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a87fff700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a880af2c0 0x7f8a88152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a87fff700 1 -- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a880ad320 msgr2=0x7f8a88064620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a87fff700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a880ad320 0x7f8a88064620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a87fff700 1 -- 172.21.2.104:0/3344737598 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a7c00b180 con 0x7f8a880ad850 2021-07-11T03:53:38.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a87fff700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a880ad850 0x7f8a88152280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f8a80001490 tx=0x7f8a80007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a85ffb700 1 -- 172.21.2.104:0/3344737598 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a80005ba0 con 0x7f8a880ad850 2021-07-11T03:53:38.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a85ffb700 1 -- 172.21.2.104:0/3344737598 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a80005d00 con 0x7f8a880ad850 2021-07-11T03:53:38.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8db4d700 1 -- 172.21.2.104:0/3344737598 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a880aa770 con 0x7f8a880ad850 2021-07-11T03:53:38.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a85ffb700 1 -- 172.21.2.104:0/3344737598 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a800072c0 con 0x7f8a880ad850 2021-07-11T03:53:38.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8db4d700 1 -- 172.21.2.104:0/3344737598 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8814f7b0 con 0x7f8a880ad850 2021-07-11T03:53:38.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8db4d700 1 -- 172.21.2.104:0/3344737598 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a88057350 con 0x7f8a880ad850 2021-07-11T03:53:38.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a85ffb700 1 -- 172.21.2.104:0/3344737598 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8a8001a070 con 0x7f8a880ad850 2021-07-11T03:53:38.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a85ffb700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a68053300 0x7f8a680557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a85ffb700 1 -- 172.21.2.104:0/3344737598 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8a8006aa60 con 0x7f8a880ad850 2021-07-11T03:53:38.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8cb4b700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a68053300 0x7f8a680557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.416+0000 7f8a8cb4b700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a68053300 0x7f8a680557c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f8a880ad770 tx=0x7f8a7c014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.420+0000 7f8a85ffb700 1 -- 172.21.2.104:0/3344737598 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8a80041d00 con 0x7f8a880ad850 2021-07-11T03:53:38.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.500+0000 7f8a8db4d700 1 -- 172.21.2.104:0/3344737598 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8a88061990 con 0x7f8a880ad850 2021-07-11T03:53:38.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.500+0000 7f8a85ffb700 1 -- 172.21.2.104:0/3344737598 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8a80045320 con 0x7f8a880ad850 2021-07-11T03:53:38.506 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:53:38.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 -- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a68053300 msgr2=0x7f8a680557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a68053300 0x7f8a680557c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f8a880ad770 tx=0x7f8a7c014000).stop 2021-07-11T03:53:38.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 -- 172.21.2.104:0/3344737598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a880ad850 msgr2=0x7f8a88152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a880ad850 0x7f8a88152280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f8a80001490 tx=0x7f8a80007e20).stop 2021-07-11T03:53:38.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 -- 172.21.2.104:0/3344737598 shutdown_connections 2021-07-11T03:53:38.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a880af2c0 0x7f8a88152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a68053300 0x7f8a680557c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a880ad850 0x7f8a88152280 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 --2- 172.21.2.104:0/3344737598 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a880ad320 0x7f8a88064620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 -- 172.21.2.104:0/3344737598 >> 172.21.2.104:0/3344737598 conn(0x7f8a88061140 msgr2=0x7f8a880ae060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 -- 172.21.2.104:0/3344737598 shutdown_connections 2021-07-11T03:53:38.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.504+0000 7f8a8db4d700 1 -- 172.21.2.104:0/3344737598 wait complete. 2021-07-11T03:53:38.513 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:53:38.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.560+0000 7f1da31c7700 1 Processor -- start 2021-07-11T03:53:38.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.560+0000 7f1da31c7700 1 -- start start 2021-07-11T03:53:38.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c060100 0x7f1d9c064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1d9c144440 0x7f1d9c065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d9c0604e0 con 0x7f1d9c0b38d0 2021-07-11T03:53:38.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9c0656f0 con 0x7f1d9c144440 2021-07-11T03:53:38.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c065c80 con 0x7f1d9c060100 2021-07-11T03:53:38.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da1764700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1d9c0b38d0 0x7f1d9c144970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39398/0 (socket says 172.21.2.104:39398) 2021-07-11T03:53:38.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da1764700 1 -- 172.21.2.104:0/1738683869 learned_addr learned my addr 172.21.2.104:0/1738683869 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:38.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1074291067 0 0) 0x7f1d9c0604e0 con 0x7f1d9c0b38d0 2021-07-11T03:53:38.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d7c003260 con 0x7f1d9c0b38d0 2021-07-11T03:53:38.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9bfff700 1 --2- 172.21.2.104:0/1738683869 >> v2:172.21.2.105:3300/0 conn(0x7f1d9c144440 0x7f1d9c065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3092181411 0 0) 0x7f1d7c003260 con 0x7f1d9c0b38d0 2021-07-11T03:53:38.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c060100 msgr2=0x7f1d9c064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:38.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 --2- 172.21.2.104:0/1738683869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c060100 0x7f1d9c064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 >> v2:172.21.2.105:3300/0 conn(0x7f1d9c144440 msgr2=0x7f1d9c065200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:38.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 --2- 172.21.2.104:0/1738683869 >> v2:172.21.2.105:3300/0 conn(0x7f1d9c144440 0x7f1d9c065200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c0b23b0 con 0x7f1d9c0b38d0 2021-07-11T03:53:38.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f1d940019e0 con 0x7f1d9c0b38d0 2021-07-11T03:53:38.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 >> v1:172.21.2.104:6789/0 conn(0x7f1d9c0b38d0 legacy=0x7f1d9c144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 --2- 172.21.2.104:0/1738683869 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d7c0037d0 0x7f1d7c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 --2- 172.21.2.104:0/1738683869 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d7c004500 0x7f1d7c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 --2- 172.21.2.104:0/1738683869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c060100 0x7f1d7c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0656f0 con 0x7f1d7c004500 2021-07-11T03:53:38.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c065c80 con 0x7f1d7c0037d0 2021-07-11T03:53:38.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c005300 con 0x7f1d9c060100 2021-07-11T03:53:38.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9bfff700 1 --2- 172.21.2.104:0/1738683869 >> v2:172.21.2.105:3300/0 conn(0x7f1d9c144440 0x7f1d9c065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:38.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9bfff700 1 --2- 172.21.2.104:0/1738683869 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d7c004500 0x7f1d7c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9bfff700 1 -- 172.21.2.104:0/1738683869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c060100 msgr2=0x7f1d7c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:38.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9bfff700 1 --2- 172.21.2.104:0/1738683869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c060100 0x7f1d7c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9bfff700 1 -- 172.21.2.104:0/1738683869 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d7c0037d0 msgr2=0x7f1d7c003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:38.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9bfff700 1 --2- 172.21.2.104:0/1738683869 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d7c0037d0 0x7f1d7c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9bfff700 1 -- 172.21.2.104:0/1738683869 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c003260 con 0x7f1d7c004500 2021-07-11T03:53:38.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9bfff700 1 --2- 172.21.2.104:0/1738683869 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d7c004500 0x7f1d7c004980 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f1d8c007000 tx=0x7f1d8c008040).ready entity=mon.0 client_cookie=30b0001022e8e9dd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d8c015070 con 0x7f1d7c004500 2021-07-11T03:53:38.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c011410 con 0x7f1d7c004500 2021-07-11T03:53:38.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9b7fe700 1 -- 172.21.2.104:0/1738683869 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d8c011770 con 0x7f1d7c004500 2021-07-11T03:53:38.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- 172.21.2.104:0/1738683869 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d7c004500 msgr2=0x7f1d7c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 --2- 172.21.2.104:0/1738683869 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d7c004500 0x7f1d7c004980 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f1d8c007000 tx=0x7f1d8c008040).stop 2021-07-11T03:53:38.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da1764700 1 -- 172.21.2.104:0/1738683869 reap_dead start 2021-07-11T03:53:38.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- 172.21.2.104:0/1738683869 shutdown_connections 2021-07-11T03:53:38.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- 172.21.2.104:0/1738683869 >> 172.21.2.104:0/1738683869 conn(0x7f1d9c05cf60 msgr2=0x7f1d9c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- 172.21.2.104:0/1738683869 shutdown_connections 2021-07-11T03:53:38.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- 172.21.2.104:0/1738683869 wait complete. 2021-07-11T03:53:38.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 Processor -- start 2021-07-11T03:53:38.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- start start 2021-07-11T03:53:38.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d9c0b0560 0x7f1d9c0b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0607c0 0x7f1d9c15a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c15a970 0x7f1d9c1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0b1590 con 0x7f1d9c0607c0 2021-07-11T03:53:38.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0ac3b0 con 0x7f1d9c0b0560 2021-07-11T03:53:38.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1da31c7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0abeb0 con 0x7f1d9c15a970 2021-07-11T03:53:38.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.564+0000 7f1d9bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0607c0 0x7f1d9c15a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d9bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0607c0 0x7f1d9c15a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42642/0 (socket says 172.21.2.104:42642) 2021-07-11T03:53:38.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d9bfff700 1 -- 172.21.2.104:0/1810227798 learned_addr learned my addr 172.21.2.104:0/1810227798 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:38.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d9bfff700 1 -- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c15a970 msgr2=0x7f1d9c1568f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:38.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d9bfff700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c15a970 0x7f1d9c1568f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d9bfff700 1 -- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d9c0b0560 msgr2=0x7f1d9c0b0940 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:38.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d9bfff700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d9c0b0560 0x7f1d9c0b0940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d9bfff700 1 -- 172.21.2.104:0/1810227798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c0ac110 con 0x7f1d9c0607c0 2021-07-11T03:53:38.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d9bfff700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0607c0 0x7f1d9c15a480 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f1d9c065420 tx=0x7f1d8c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d99ffb700 1 -- 172.21.2.104:0/1810227798 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d8c01a900 con 0x7f1d9c0607c0 2021-07-11T03:53:38.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d99ffb700 1 -- 172.21.2.104:0/1810227798 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c000e90 con 0x7f1d9c0607c0 2021-07-11T03:53:38.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1da31c7700 1 -- 172.21.2.104:0/1810227798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c159fe0 con 0x7f1d9c0607c0 2021-07-11T03:53:38.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1da31c7700 1 -- 172.21.2.104:0/1810227798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c062680 con 0x7f1d9c0607c0 2021-07-11T03:53:38.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d99ffb700 1 -- 172.21.2.104:0/1810227798 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d8c01ac50 con 0x7f1d9c0607c0 2021-07-11T03:53:38.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1da31c7700 1 -- 172.21.2.104:0/1810227798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c159df0 con 0x7f1d9c0607c0 2021-07-11T03:53:38.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d99ffb700 1 -- 172.21.2.104:0/1810227798 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1d8c02a070 con 0x7f1d9c0607c0 2021-07-11T03:53:38.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d99ffb700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d800532d0 0x7f1d80055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1d99ffb700 1 -- 172.21.2.104:0/1810227798 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1d8c06c6a0 con 0x7f1d9c0607c0 2021-07-11T03:53:38.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1da0f63700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d800532d0 0x7f1d80055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.568+0000 7f1da0f63700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d800532d0 0x7f1d80055790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f1d84002f70 tx=0x7f1d84006980).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.572+0000 7f1d99ffb700 1 -- 172.21.2.104:0/1810227798 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1d8c0438d0 con 0x7f1d9c0607c0 2021-07-11T03:53:38.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 -- 172.21.2.104:0/1810227798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1d9c15a170 con 0x7f1d9c0607c0 2021-07-11T03:53:38.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1d99ffb700 1 -- 172.21.2.104:0/1810227798 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1d8c046ef0 con 0x7f1d9c0607c0 2021-07-11T03:53:38.656 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:38.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 -- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d800532d0 msgr2=0x7f1d80055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d800532d0 0x7f1d80055790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f1d84002f70 tx=0x7f1d84006980).stop 2021-07-11T03:53:38.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 -- 172.21.2.104:0/1810227798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0607c0 msgr2=0x7f1d9c15a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0607c0 0x7f1d9c15a480 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f1d9c065420 tx=0x7f1d8c008040).stop 2021-07-11T03:53:38.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 -- 172.21.2.104:0/1810227798 shutdown_connections 2021-07-11T03:53:38.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c15a970 0x7f1d9c1568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d800532d0 0x7f1d80055790 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0607c0 0x7f1d9c15a480 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 --2- 172.21.2.104:0/1810227798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d9c0b0560 0x7f1d9c0b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 -- 172.21.2.104:0/1810227798 >> 172.21.2.104:0/1810227798 conn(0x7f1d9c05cf60 msgr2=0x7f1d9c0adce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 -- 172.21.2.104:0/1810227798 shutdown_connections 2021-07-11T03:53:38.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.652+0000 7f1da31c7700 1 -- 172.21.2.104:0/1810227798 wait complete. 2021-07-11T03:53:38.666 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:53:38.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 Processor -- start 2021-07-11T03:53:38.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- start start 2021-07-11T03:53:38.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d100b1ec0 0x7f1d100b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1d100b22a0 0x7f1d10141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d100a77d0 con 0x7f1d100affb0 2021-07-11T03:53:38.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d100b03f0 con 0x7f1d100b22a0 2021-07-11T03:53:38.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100b0650 con 0x7f1d100b1ec0 2021-07-11T03:53:38.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0effd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1d100affb0 0x7f1d100a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39422/0 (socket says 172.21.2.104:39422) 2021-07-11T03:53:38.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0effd700 1 -- 172.21.2.104:0/2927458805 learned_addr learned my addr 172.21.2.104:0/2927458805 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:38.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0ffff700 1 --2- 172.21.2.104:0/2927458805 >> v2:172.21.2.104:3300/0 conn(0x7f1d100b22a0 0x7f1d10141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0ffff700 1 -- 172.21.2.104:0/2927458805 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d100b1ec0 msgr2=0x7f1d100b12d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:38.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0ffff700 1 --2- 172.21.2.104:0/2927458805 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d100b1ec0 0x7f1d100b12d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0ffff700 1 -- 172.21.2.104:0/2927458805 >> v1:172.21.2.104:6789/0 conn(0x7f1d100affb0 legacy=0x7f1d100a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0ffff700 1 -- 172.21.2.104:0/2927458805 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d100b08f0 con 0x7f1d100b22a0 2021-07-11T03:53:38.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0ffff700 1 --2- 172.21.2.104:0/2927458805 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d100b22a0 0x7f1d10141250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f1d080032f0 tx=0x7f1d080091d0).ready entity=mon.0 client_cookie=940184e88f75c811 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0e7fc700 1 -- 172.21.2.104:0/2927458805 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d08009ea0 con 0x7f1d100b22a0 2021-07-11T03:53:38.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0e7fc700 1 -- 172.21.2.104:0/2927458805 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0800b8a0 con 0x7f1d100b22a0 2021-07-11T03:53:38.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0e7fc700 1 -- 172.21.2.104:0/2927458805 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d0800ba50 con 0x7f1d100b22a0 2021-07-11T03:53:38.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- 172.21.2.104:0/2927458805 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d100b22a0 msgr2=0x7f1d10141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 --2- 172.21.2.104:0/2927458805 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d100b22a0 0x7f1d10141250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f1d080032f0 tx=0x7f1d080091d0).stop 2021-07-11T03:53:38.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- 172.21.2.104:0/2927458805 shutdown_connections 2021-07-11T03:53:38.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 --2- 172.21.2.104:0/2927458805 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d100b22a0 0x7f1d10141250 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 --2- 172.21.2.104:0/2927458805 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d100b1ec0 0x7f1d100b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- 172.21.2.104:0/2927458805 >> 172.21.2.104:0/2927458805 conn(0x7f1d1005f6b0 msgr2=0x7f1d1005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- 172.21.2.104:0/2927458805 shutdown_connections 2021-07-11T03:53:38.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- 172.21.2.104:0/2927458805 wait complete. 2021-07-11T03:53:38.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 Processor -- start 2021-07-11T03:53:38.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- start start 2021-07-11T03:53:38.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d100affb0 0x7f1d1015a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d100b1ec0 0x7f1d100ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d100b22a0 0x7f1d100ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100b5990 con 0x7f1d100affb0 2021-07-11T03:53:38.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100b5c30 con 0x7f1d100b1ec0 2021-07-11T03:53:38.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d1624a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d100b5ed0 con 0x7f1d100b22a0 2021-07-11T03:53:38.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.720+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d100affb0 0x7f1d1015a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d100affb0 0x7f1d1015a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42658/0 (socket says 172.21.2.104:42658) 2021-07-11T03:53:38.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0f7fe700 1 -- 172.21.2.104:0/1365006902 learned_addr learned my addr 172.21.2.104:0/1365006902 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:38.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0f7fe700 1 -- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d100b22a0 msgr2=0x7f1d100ab850 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:38.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0f7fe700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d100b22a0 0x7f1d100ab850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0f7fe700 1 -- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d100b1ec0 msgr2=0x7f1d100ae280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:38.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0f7fe700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d100b1ec0 0x7f1d100ae280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0f7fe700 1 -- 172.21.2.104:0/1365006902 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d08009470 con 0x7f1d100affb0 2021-07-11T03:53:38.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0f7fe700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d100affb0 0x7f1d1015a9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f1d00002fb0 tx=0x7f1d00006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0cff9700 1 -- 172.21.2.104:0/1365006902 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d00013460 con 0x7f1d100affb0 2021-07-11T03:53:38.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0cff9700 1 -- 172.21.2.104:0/1365006902 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d000135c0 con 0x7f1d100affb0 2021-07-11T03:53:38.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d1624a700 1 -- 172.21.2.104:0/1365006902 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d100ad470 con 0x7f1d100affb0 2021-07-11T03:53:38.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0cff9700 1 -- 172.21.2.104:0/1365006902 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d00013920 con 0x7f1d100affb0 2021-07-11T03:53:38.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d1624a700 1 -- 172.21.2.104:0/1365006902 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d100b5220 con 0x7f1d100affb0 2021-07-11T03:53:38.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d1624a700 1 -- 172.21.2.104:0/1365006902 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d10057350 con 0x7f1d100affb0 2021-07-11T03:53:38.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0cff9700 1 -- 172.21.2.104:0/1365006902 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1d0000f070 con 0x7f1d100affb0 2021-07-11T03:53:38.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0cff9700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf00531f0 0x7f1cf00556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0cff9700 1 -- 172.21.2.104:0/1365006902 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1d0006af20 con 0x7f1d100affb0 2021-07-11T03:53:38.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0effd700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf00531f0 0x7f1cf00556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.724+0000 7f1d0effd700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf00531f0 0x7f1cf00556b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f1d1006a0f0 tx=0x7f1d04007790).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.728+0000 7f1d0cff9700 1 -- 172.21.2.104:0/1365006902 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1d00042260 con 0x7f1d100affb0 2021-07-11T03:53:38.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 -- 172.21.2.104:0/1365006902 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d100ac760 con 0x7f1d100affb0 2021-07-11T03:53:38.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d0cff9700 1 -- 172.21.2.104:0/1365006902 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1d00045880 con 0x7f1d100affb0 2021-07-11T03:53:38.812 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:38.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 -- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf00531f0 msgr2=0x7f1cf00556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf00531f0 0x7f1cf00556b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f1d1006a0f0 tx=0x7f1d04007790).stop 2021-07-11T03:53:38.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 -- 172.21.2.104:0/1365006902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d100affb0 msgr2=0x7f1d1015a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d100affb0 0x7f1d1015a9f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f1d00002fb0 tx=0x7f1d00006df0).stop 2021-07-11T03:53:38.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 -- 172.21.2.104:0/1365006902 shutdown_connections 2021-07-11T03:53:38.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d100b22a0 0x7f1d100ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d100b1ec0 0x7f1d100ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf00531f0 0x7f1cf00556b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 --2- 172.21.2.104:0/1365006902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d100affb0 0x7f1d1015a9f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 -- 172.21.2.104:0/1365006902 >> 172.21.2.104:0/1365006902 conn(0x7f1d1005f6b0 msgr2=0x7f1d10147f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 -- 172.21.2.104:0/1365006902 shutdown_connections 2021-07-11T03:53:38.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.808+0000 7f1d1624a700 1 -- 172.21.2.104:0/1365006902 wait complete. 2021-07-11T03:53:38.819 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:53:38.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 Processor -- start 2021-07-11T03:53:38.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 -- start start 2021-07-11T03:53:38.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7cf805ff60 0x7f7cf8140440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf8064910 con 0x7f7cf80b2710 2021-07-11T03:53:38.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf8064bb0 con 0x7f7cf80611d0 2021-07-11T03:53:38.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf8064e50 con 0x7f7cf805ff60 2021-07-11T03:53:38.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf67fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f7cf80b2710 0x7f7cf8143110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39436/0 (socket says 172.21.2.104:39436) 2021-07-11T03:53:38.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf67fc700 1 -- 172.21.2.104:0/1895249332 learned_addr learned my addr 172.21.2.104:0/1895249332 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:38.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf6ffd700 1 --2- 172.21.2.104:0/1895249332 >> v2:172.21.2.104:3300/0 conn(0x7f7cf805ff60 0x7f7cf8140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf5ffb700 1 -- 172.21.2.104:0/1895249332 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622610625 0 0) 0x7f7cf8064910 con 0x7f7cf80b2710 2021-07-11T03:53:38.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf5ffb700 1 -- 172.21.2.104:0/1895249332 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cd8003260 con 0x7f7cf80b2710 2021-07-11T03:53:38.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf6ffd700 1 -- 172.21.2.104:0/1895249332 >> v1:172.21.2.105:6789/0 conn(0x7f7cf80611d0 legacy=0x7f7cf8143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:38.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf6ffd700 1 -- 172.21.2.104:0/1895249332 >> v1:172.21.2.104:6789/0 conn(0x7f7cf80b2710 legacy=0x7f7cf8143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf6ffd700 1 -- 172.21.2.104:0/1895249332 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf80650f0 con 0x7f7cf805ff60 2021-07-11T03:53:38.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf6ffd700 1 --2- 172.21.2.104:0/1895249332 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7cf805ff60 0x7f7cf8140440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f7ce0001110 tx=0x7f7ce8006df0).ready entity=mon.0 client_cookie=3bbfb3f5de81a7b2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf5ffb700 1 -- 172.21.2.104:0/1895249332 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce800d570 con 0x7f7cf805ff60 2021-07-11T03:53:38.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf5ffb700 1 -- 172.21.2.104:0/1895249332 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce8000e90 con 0x7f7cf805ff60 2021-07-11T03:53:38.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cf5ffb700 1 -- 172.21.2.104:0/1895249332 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce800d8c0 con 0x7f7cf805ff60 2021-07-11T03:53:38.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 -- 172.21.2.104:0/1895249332 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7cf805ff60 msgr2=0x7f7cf8140440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 --2- 172.21.2.104:0/1895249332 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7cf805ff60 0x7f7cf8140440 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f7ce0001110 tx=0x7f7ce8006df0).stop 2021-07-11T03:53:38.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 -- 172.21.2.104:0/1895249332 shutdown_connections 2021-07-11T03:53:38.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 --2- 172.21.2.104:0/1895249332 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7cf805ff60 0x7f7cf8140440 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 -- 172.21.2.104:0/1895249332 >> 172.21.2.104:0/1895249332 conn(0x7f7cf805cf60 msgr2=0x7f7cf805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 -- 172.21.2.104:0/1895249332 shutdown_connections 2021-07-11T03:53:38.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.868+0000 7f7cfd924700 1 -- 172.21.2.104:0/1895249332 wait complete. 2021-07-11T03:53:38.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cfd924700 1 Processor -- start 2021-07-11T03:53:38.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cfd924700 1 -- start start 2021-07-11T03:53:38.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cfd924700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7cf805ff60 0x7f7cf8146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cfd924700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7cf80611d0 0x7f7cf80ab710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cfd924700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7cf80b2710 0x7f7cf81459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cfd924700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80ada20 con 0x7f7cf80611d0 2021-07-11T03:53:38.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cfd924700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80ad240 con 0x7f7cf80b2710 2021-07-11T03:53:38.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cfd924700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80ad4e0 con 0x7f7cf805ff60 2021-07-11T03:53:38.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cf67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7cf80611d0 0x7f7cf80ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cf67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7cf80611d0 0x7f7cf80ab710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42670/0 (socket says 172.21.2.104:42670) 2021-07-11T03:53:38.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cf67fc700 1 -- 172.21.2.104:0/1266431834 learned_addr learned my addr 172.21.2.104:0/1266431834 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:38.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cf67fc700 1 -- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7cf805ff60 msgr2=0x7f7cf8146e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cf67fc700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7cf805ff60 0x7f7cf8146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cf67fc700 1 -- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7cf80b2710 msgr2=0x7f7cf81459c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cf67fc700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7cf80b2710 0x7f7cf81459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cf67fc700 1 -- 172.21.2.104:0/1266431834 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce800b180 con 0x7f7cf80611d0 2021-07-11T03:53:38.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cf67fc700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7cf80611d0 0x7f7cf80ab710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f7ce0000ed0 tx=0x7f7ce0007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7ceffff700 1 -- 172.21.2.104:0/1266431834 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce0007e10 con 0x7f7cf80611d0 2021-07-11T03:53:38.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7ceffff700 1 -- 172.21.2.104:0/1266431834 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce00072e0 con 0x7f7cf80611d0 2021-07-11T03:53:38.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7ceffff700 1 -- 172.21.2.104:0/1266431834 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7ce0005800 con 0x7f7cf80611d0 2021-07-11T03:53:38.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cfd924700 1 -- 172.21.2.104:0/1266431834 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf80ad780 con 0x7f7cf80611d0 2021-07-11T03:53:38.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cfd924700 1 -- 172.21.2.104:0/1266431834 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf80acf30 con 0x7f7cf80611d0 2021-07-11T03:53:38.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cfd924700 1 -- 172.21.2.104:0/1266431834 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf8057350 con 0x7f7cf80611d0 2021-07-11T03:53:38.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7ceffff700 1 -- 172.21.2.104:0/1266431834 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f7ce0010070 con 0x7f7cf80611d0 2021-07-11T03:53:38.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7ceffff700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7cdc053300 0x7f7cdc0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:38.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7ceffff700 1 -- 172.21.2.104:0/1266431834 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7ce006ac50 con 0x7f7cf80611d0 2021-07-11T03:53:38.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.872+0000 7f7cf6ffd700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7cdc053300 0x7f7cdc0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:38.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.876+0000 7f7cf6ffd700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7cdc053300 0x7f7cdc0557c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f7ce0001110 tx=0x7f7ce8014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:38.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.876+0000 7f7ceffff700 1 -- 172.21.2.104:0/1266431834 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7ce0041f90 con 0x7f7cf80611d0 2021-07-11T03:53:38.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.956+0000 7f7cfd924700 1 -- 172.21.2.104:0/1266431834 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7cf80698e0 con 0x7f7cf80611d0 2021-07-11T03:53:38.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.956+0000 7f7ceffff700 1 -- 172.21.2.104:0/1266431834 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7ce00455b0 con 0x7f7cf80611d0 2021-07-11T03:53:38.961 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:38.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.956+0000 7f7cfd924700 1 -- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7cdc053300 msgr2=0x7f7cdc0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.956+0000 7f7cfd924700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7cdc053300 0x7f7cdc0557c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f7ce0001110 tx=0x7f7ce8014000).stop 2021-07-11T03:53:38.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.956+0000 7f7cfd924700 1 -- 172.21.2.104:0/1266431834 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7cf80611d0 msgr2=0x7f7cf80ab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:38.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.956+0000 7f7cfd924700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7cf80611d0 0x7f7cf80ab710 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f7ce0000ed0 tx=0x7f7ce0007ba0).stop 2021-07-11T03:53:38.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.956+0000 7f7cfd924700 1 -- 172.21.2.104:0/1266431834 shutdown_connections 2021-07-11T03:53:38.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.956+0000 7f7cfd924700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7cf80b2710 0x7f7cf81459c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.956+0000 7f7cfd924700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7cdc053300 0x7f7cdc0557c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.956+0000 7f7cfd924700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7cf80611d0 0x7f7cf80ab710 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:38.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.956+0000 7f7cfd924700 1 --2- 172.21.2.104:0/1266431834 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7cf805ff60 0x7f7cf8146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:38.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.960+0000 7f7cfd924700 1 -- 172.21.2.104:0/1266431834 >> 172.21.2.104:0/1266431834 conn(0x7f7cf805cf60 msgr2=0x7f7cf80b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:38.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.960+0000 7f7cfd924700 1 -- 172.21.2.104:0/1266431834 shutdown_connections 2021-07-11T03:53:38.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:38.960+0000 7f7cfd924700 1 -- 172.21.2.104:0/1266431834 wait complete. 2021-07-11T03:53:38.972 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:39.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 Processor -- start 2021-07-11T03:53:39.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 -- start start 2021-07-11T03:53:39.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34a40a76b0 0x7f34a4066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f34a40b1970 0x7f34a4066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f34a40b0060 0x7f34a405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a4067450 con 0x7f34a40b0060 2021-07-11T03:53:39.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34a40a7a90 con 0x7f34a40b1970 2021-07-11T03:53:39.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a4147620 con 0x7f34a40a76b0 2021-07-11T03:53:39.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a8fda700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f34a40b0060 0x7f34a405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a8fda700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f34a40b0060 0x7f34a405f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42680/0 (socket says 172.21.2.104:42680) 2021-07-11T03:53:39.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a8fda700 1 -- 172.21.2.104:0/3867931443 learned_addr learned my addr 172.21.2.104:0/3867931443 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:39.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a8fda700 1 -- 172.21.2.104:0/3867931443 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34a40a76b0 msgr2=0x7f34a4066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:39.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a8fda700 1 --2- 172.21.2.104:0/3867931443 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34a40a76b0 0x7f34a4066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a8fda700 1 -- 172.21.2.104:0/3867931443 >> v2:172.21.2.105:3300/0 conn(0x7f34a40b1970 msgr2=0x7f34a4066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:39.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a8fda700 1 --2- 172.21.2.104:0/3867931443 >> v2:172.21.2.105:3300/0 conn(0x7f34a40b1970 0x7f34a4066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a8fda700 1 -- 172.21.2.104:0/3867931443 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a41478c0 con 0x7f34a40b0060 2021-07-11T03:53:39.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a8fda700 1 --2- 172.21.2.104:0/3867931443 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34a40b0060 0x7f34a405f6b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f349c0032f0 tx=0x7f349c009290).ready entity=mon.0 client_cookie=bcc9a5c4c22088c9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a2ffd700 1 -- 172.21.2.104:0/3867931443 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f349c00b870 con 0x7f34a40b0060 2021-07-11T03:53:39.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a2ffd700 1 -- 172.21.2.104:0/3867931443 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349c00b9d0 con 0x7f34a40b0060 2021-07-11T03:53:39.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34a2ffd700 1 -- 172.21.2.104:0/3867931443 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f349c00bb40 con 0x7f34a40b0060 2021-07-11T03:53:39.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3867931443 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34a40b0060 msgr2=0x7f34a405f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 --2- 172.21.2.104:0/3867931443 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34a40b0060 0x7f34a405f6b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f349c0032f0 tx=0x7f349c009290).stop 2021-07-11T03:53:39.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3867931443 shutdown_connections 2021-07-11T03:53:39.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 --2- 172.21.2.104:0/3867931443 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34a40b0060 0x7f34a405f6b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 --2- 172.21.2.104:0/3867931443 >> v2:172.21.2.105:3300/0 conn(0x7f34a40b1970 0x7f34a4066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 --2- 172.21.2.104:0/3867931443 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34a40a76b0 0x7f34a4066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3867931443 >> 172.21.2.104:0/3867931443 conn(0x7f34a40b53e0 msgr2=0x7f34a40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:39.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3867931443 shutdown_connections 2021-07-11T03:53:39.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.020+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3867931443 wait complete. 2021-07-11T03:53:39.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34aaa3d700 1 Processor -- start 2021-07-11T03:53:39.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34aaa3d700 1 -- start start 2021-07-11T03:53:39.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34aaa3d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34a40a76b0 0x7f34a40abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34aaa3d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f34a40b0060 0x7f34a40b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34aaa3d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34a40b1970 0x7f34a4144a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34aaa3d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40aa800 con 0x7f34a40a76b0 2021-07-11T03:53:39.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34aaa3d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40af560 con 0x7f34a40b0060 2021-07-11T03:53:39.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34aaa3d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a40aaa30 con 0x7f34a40b1970 2021-07-11T03:53:39.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34a40a76b0 0x7f34a40abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34a40a76b0 0x7f34a40abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42684/0 (socket says 172.21.2.104:42684) 2021-07-11T03:53:39.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a3fff700 1 -- 172.21.2.104:0/3613533945 learned_addr learned my addr 172.21.2.104:0/3613533945 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:39.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a3fff700 1 -- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34a40b1970 msgr2=0x7f34a4144a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a3fff700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34a40b1970 0x7f34a4144a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a3fff700 1 -- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f34a40b0060 msgr2=0x7f34a40b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a3fff700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f34a40b0060 0x7f34a40b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a3fff700 1 -- 172.21.2.104:0/3613533945 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349c009530 con 0x7f34a40a76b0 2021-07-11T03:53:39.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a3fff700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34a40a76b0 0x7f34a40abe40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f348c002f70 tx=0x7f348c006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a17fa700 1 -- 172.21.2.104:0/3613533945 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348c013460 con 0x7f34a40a76b0 2021-07-11T03:53:39.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a17fa700 1 -- 172.21.2.104:0/3613533945 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c0135c0 con 0x7f34a40a76b0 2021-07-11T03:53:39.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a17fa700 1 -- 172.21.2.104:0/3613533945 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348c013920 con 0x7f34a40a76b0 2021-07-11T03:53:39.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3613533945 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a40aacd0 con 0x7f34a40a76b0 2021-07-11T03:53:39.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3613533945 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a414b720 con 0x7f34a40a76b0 2021-07-11T03:53:39.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3613533945 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34a40aae60 con 0x7f34a40a76b0 2021-07-11T03:53:39.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a17fa700 1 -- 172.21.2.104:0/3613533945 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f348c00f070 con 0x7f34a40a76b0 2021-07-11T03:53:39.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a17fa700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34880531f0 0x7f34880556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a17fa700 1 -- 172.21.2.104:0/3613533945 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f348c06ade0 con 0x7f34a40a76b0 2021-07-11T03:53:39.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.024+0000 7f34a37fe700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34880531f0 0x7f34880556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.028+0000 7f34a37fe700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34880531f0 0x7f34880556b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f3494002f70 tx=0x7f3494006990).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.028+0000 7f34a17fa700 1 -- 172.21.2.104:0/3613533945 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f348c0421d0 con 0x7f34a40a76b0 2021-07-11T03:53:39.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.108+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3613533945 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34a4057350 con 0x7f34a40a76b0 2021-07-11T03:53:39.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.108+0000 7f34a17fa700 1 -- 172.21.2.104:0/3613533945 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f348c0457f0 con 0x7f34a40a76b0 2021-07-11T03:53:39.113 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:39.113 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":23,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:24.367517+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:20.980780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:21.341375+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:39.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34880531f0 msgr2=0x7f34880556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34880531f0 0x7f34880556b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f3494002f70 tx=0x7f3494006990).stop 2021-07-11T03:53:39.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3613533945 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34a40a76b0 msgr2=0x7f34a40abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34a40a76b0 0x7f34a40abe40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f348c002f70 tx=0x7f348c006ec0).stop 2021-07-11T03:53:39.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3613533945 shutdown_connections 2021-07-11T03:53:39.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34a40b1970 0x7f34a4144a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f34a40b0060 0x7f34a40b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34880531f0 0x7f34880556b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 --2- 172.21.2.104:0/3613533945 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34a40a76b0 0x7f34a40abe40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3613533945 >> 172.21.2.104:0/3613533945 conn(0x7f34a40b53e0 msgr2=0x7f34a40b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:39.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3613533945 shutdown_connections 2021-07-11T03:53:39.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.112+0000 7f34aaa3d700 1 -- 172.21.2.104:0/3613533945 wait complete. 2021-07-11T03:53:39.122 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:53:20.980780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:53:21.341375+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:53:39.122 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:53:39.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 Processor -- start 2021-07-11T03:53:39.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 -- start start 2021-07-11T03:53:39.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe5880b12d0 0x7fe5880affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5880b22a0 0x7fe588141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5880a77d0 con 0x7fe5880b1ec0 2021-07-11T03:53:39.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5880b1740 con 0x7fe5880b12d0 2021-07-11T03:53:39.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880b0650 con 0x7fe5880b22a0 2021-07-11T03:53:39.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe5877fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe5880b12d0 0x7fe5880affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe587fff700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fe5880b1ec0 0x7fe5880a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:38828/0 (socket says 172.21.2.104:38828) 2021-07-11T03:53:39.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe587fff700 1 -- 172.21.2.104:0/3806848022 learned_addr learned my addr 172.21.2.104:0/3806848022 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:39.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58cb1b700 1 --2- 172.21.2.104:0/3806848022 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5880b22a0 0x7fe588141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe5877fe700 1 -- 172.21.2.104:0/3806848022 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5880b22a0 msgr2=0x7fe588141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:39.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe5877fe700 1 --2- 172.21.2.104:0/3806848022 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5880b22a0 0x7fe588141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe5877fe700 1 -- 172.21.2.104:0/3806848022 >> v1:172.21.2.105:6789/0 conn(0x7fe5880b1ec0 legacy=0x7fe5880a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe586ffd700 1 -- 172.21.2.104:0/3806848022 <== mon.1 v1:172.21.2.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150328562 0 0) 0x7fe5880a77d0 con 0x7fe5880b1ec0 2021-07-11T03:53:39.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe5877fe700 1 -- 172.21.2.104:0/3806848022 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5880b08f0 con 0x7fe5880b12d0 2021-07-11T03:53:39.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58cb1b700 1 --2- 172.21.2.104:0/3806848022 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5880b22a0 0x7fe588141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T03:53:39.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe5877fe700 1 --2- 172.21.2.104:0/3806848022 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5880b12d0 0x7fe5880affb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fe57c002f70 tx=0x7fe57c006df0).ready entity=mon.1 client_cookie=843d93f33c84706e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe586ffd700 1 -- 172.21.2.104:0/3806848022 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe57c00d570 con 0x7fe5880b12d0 2021-07-11T03:53:39.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe586ffd700 1 -- 172.21.2.104:0/3806848022 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c000e90 con 0x7fe5880b12d0 2021-07-11T03:53:39.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe586ffd700 1 -- 172.21.2.104:0/3806848022 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe57c00d8c0 con 0x7fe5880b12d0 2021-07-11T03:53:39.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 -- 172.21.2.104:0/3806848022 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5880b12d0 msgr2=0x7fe5880affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 --2- 172.21.2.104:0/3806848022 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5880b12d0 0x7fe5880affb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fe57c002f70 tx=0x7fe57c006df0).stop 2021-07-11T03:53:39.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 -- 172.21.2.104:0/3806848022 shutdown_connections 2021-07-11T03:53:39.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 --2- 172.21.2.104:0/3806848022 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5880b22a0 0x7fe588141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 --2- 172.21.2.104:0/3806848022 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5880b12d0 0x7fe5880affb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 -- 172.21.2.104:0/3806848022 >> 172.21.2.104:0/3806848022 conn(0x7fe58805f6b0 msgr2=0x7fe58805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:39.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 -- 172.21.2.104:0/3806848022 shutdown_connections 2021-07-11T03:53:39.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.184+0000 7fe58e57e700 1 -- 172.21.2.104:0/3806848022 wait complete. 2021-07-11T03:53:39.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58e57e700 1 Processor -- start 2021-07-11T03:53:39.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58e57e700 1 -- start start 2021-07-11T03:53:39.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58e57e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5880b12d0 0x7fe58813a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58e57e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5880b1ec0 0x7fe58814b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58e57e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5880b22a0 0x7fe588139010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58e57e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58814a930 con 0x7fe5880b1ec0 2021-07-11T03:53:39.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58e57e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58813ac30 con 0x7fe5880b12d0 2021-07-11T03:53:39.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58e57e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880aa080 con 0x7fe5880b22a0 2021-07-11T03:53:39.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe587fff700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5880b12d0 0x7fe58813a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe5877fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5880b1ec0 0x7fe58814b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe587fff700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5880b12d0 0x7fe58813a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:50256/0 (socket says 172.21.2.104:50256) 2021-07-11T03:53:39.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe587fff700 1 -- 172.21.2.104:0/4174748969 learned_addr learned my addr 172.21.2.104:0/4174748969 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:39.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58cb1b700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5880b22a0 0x7fe588139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe587fff700 1 -- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5880b22a0 msgr2=0x7fe588139010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe587fff700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5880b22a0 0x7fe588139010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe587fff700 1 -- 172.21.2.104:0/4174748969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5880b1ec0 msgr2=0x7fe58814b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe587fff700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5880b1ec0 0x7fe58814b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe587fff700 1 -- 172.21.2.104:0/4174748969 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe580009470 con 0x7fe5880b12d0 2021-07-11T03:53:39.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe587fff700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5880b12d0 0x7fe58813a430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fe578000bc0 tx=0x7fe578007a60).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe5857fa700 1 -- 172.21.2.104:0/4174748969 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5780070d0 con 0x7fe5880b12d0 2021-07-11T03:53:39.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe5857fa700 1 -- 172.21.2.104:0/4174748969 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578007230 con 0x7fe5880b12d0 2021-07-11T03:53:39.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe5857fa700 1 -- 172.21.2.104:0/4174748969 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe578005610 con 0x7fe5880b12d0 2021-07-11T03:53:39.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58e57e700 1 -- 172.21.2.104:0/4174748969 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c00b180 con 0x7fe5880b12d0 2021-07-11T03:53:39.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58e57e700 1 -- 172.21.2.104:0/4174748969 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe588138720 con 0x7fe5880b12d0 2021-07-11T03:53:39.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe58e57e700 1 -- 172.21.2.104:0/4174748969 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5880a95c0 con 0x7fe5880b12d0 2021-07-11T03:53:39.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe5857fa700 1 -- 172.21.2.104:0/4174748969 <== mon.1 v2:172.21.2.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe578010070 con 0x7fe5880b12d0 2021-07-11T03:53:39.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe5857fa700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe568053300 0x7fe5680557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe5857fa700 1 -- 172.21.2.104:0/4174748969 <== mon.1 v2:172.21.2.105:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe57806acd0 con 0x7fe5880b12d0 2021-07-11T03:53:39.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe5877fe700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe568053300 0x7fe5680557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.188+0000 7fe5877fe700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe568053300 0x7fe5680557c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fe57c019000 tx=0x7fe57c014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.192+0000 7fe5857fa700 1 -- 172.21.2.104:0/4174748969 <== mon.1 v2:172.21.2.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe5880a95c0 con 0x7fe5880b12d0 2021-07-11T03:53:39.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 -- 172.21.2.104:0/4174748969 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe5880a95c0 con 0x7fe5880b12d0 2021-07-11T03:53:39.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe5857fa700 1 -- 172.21.2.104:0/4174748969 <== mon.1 v2:172.21.2.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe5880a95c0 con 0x7fe5880b12d0 2021-07-11T03:53:39.280 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:53:39.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 -- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe568053300 msgr2=0x7fe5680557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe568053300 0x7fe5680557c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fe57c019000 tx=0x7fe57c014000).stop 2021-07-11T03:53:39.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 -- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5880b12d0 msgr2=0x7fe58813a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5880b12d0 0x7fe58813a430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fe578000bc0 tx=0x7fe578007a60).stop 2021-07-11T03:53:39.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 -- 172.21.2.104:0/4174748969 shutdown_connections 2021-07-11T03:53:39.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5880b22a0 0x7fe588139010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe568053300 0x7fe5680557c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5880b1ec0 0x7fe58814b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 --2- 172.21.2.104:0/4174748969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5880b12d0 0x7fe58813a430 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 -- 172.21.2.104:0/4174748969 >> 172.21.2.104:0/4174748969 conn(0x7fe58805f6b0 msgr2=0x7fe58813d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:39.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 -- 172.21.2.104:0/4174748969 shutdown_connections 2021-07-11T03:53:39.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.276+0000 7fe58e57e700 1 -- 172.21.2.104:0/4174748969 wait complete. 2021-07-11T03:53:39.289 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:53:39.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 Processor -- start 2021-07-11T03:53:39.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- start start 2021-07-11T03:53:39.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff1cc146950 0x7ff1cc140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1cc143b60 con 0x7ff1cc0b4690 2021-07-11T03:53:39.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1cc143e00 con 0x7ff1cc0b4180 2021-07-11T03:53:39.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1cc147510 con 0x7ff1cc146950 2021-07-11T03:53:39.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d0a14700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff1cc0b4690 0x7ff1cc0b4ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39476/0 (socket says 172.21.2.104:39476) 2021-07-11T03:53:39.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d0a14700 1 -- 172.21.2.104:0/247616297 learned_addr learned my addr 172.21.2.104:0/247616297 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:39.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 --2- 172.21.2.104:0/247616297 >> v2:172.21.2.104:3300/0 conn(0x7ff1cc146950 0x7ff1cc140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 -- 172.21.2.104:0/247616297 >> v1:172.21.2.105:6789/0 conn(0x7ff1cc0b4180 legacy=0x7ff1cc140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:39.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 -- 172.21.2.104:0/247616297 >> v1:172.21.2.104:6789/0 conn(0x7ff1cc0b4690 legacy=0x7ff1cc0b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 -- 172.21.2.104:0/247616297 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1cc0b0c90 con 0x7ff1cc146950 2021-07-11T03:53:39.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 --2- 172.21.2.104:0/247616297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1cc146950 0x7ff1cc140230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7ff1c4001490 tx=0x7ff1bc006df0).ready entity=mon.0 client_cookie=6dafdaf631e2e5dd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1caffd700 1 -- 172.21.2.104:0/247616297 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1bc00d4a0 con 0x7ff1cc146950 2021-07-11T03:53:39.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1caffd700 1 -- 172.21.2.104:0/247616297 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc000e90 con 0x7ff1cc146950 2021-07-11T03:53:39.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1caffd700 1 -- 172.21.2.104:0/247616297 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1bc00d7f0 con 0x7ff1cc146950 2021-07-11T03:53:39.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- 172.21.2.104:0/247616297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1cc146950 msgr2=0x7ff1cc140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 --2- 172.21.2.104:0/247616297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1cc146950 0x7ff1cc140230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7ff1c4001490 tx=0x7ff1bc006df0).stop 2021-07-11T03:53:39.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- 172.21.2.104:0/247616297 shutdown_connections 2021-07-11T03:53:39.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 --2- 172.21.2.104:0/247616297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1cc146950 0x7ff1cc140230 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- 172.21.2.104:0/247616297 >> 172.21.2.104:0/247616297 conn(0x7ff1cc05c490 msgr2=0x7ff1cc05c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:39.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- 172.21.2.104:0/247616297 shutdown_connections 2021-07-11T03:53:39.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- 172.21.2.104:0/247616297 wait complete. 2021-07-11T03:53:39.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 Processor -- start 2021-07-11T03:53:39.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- start start 2021-07-11T03:53:39.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1cc0b4180 0x7ff1cc141200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1cc0b4690 0x7ff1cc13a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1cc146950 0x7ff1cc13ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1544b0 con 0x7ff1cc0b4180 2021-07-11T03:53:39.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0b0720 con 0x7ff1cc146950 2021-07-11T03:53:39.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1d2477700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc0afd40 con 0x7ff1cc0b4690 2021-07-11T03:53:39.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1cc0b4180 0x7ff1cc141200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1cc0b4180 0x7ff1cc141200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42710/0 (socket says 172.21.2.104:42710) 2021-07-11T03:53:39.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 -- 172.21.2.104:0/2308497987 learned_addr learned my addr 172.21.2.104:0/2308497987 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:39.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 -- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1cc0b4690 msgr2=0x7ff1cc13a730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:39.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1cc0b4690 0x7ff1cc13a730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 -- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1cc146950 msgr2=0x7ff1cc13ac20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:39.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1cc146950 0x7ff1cc13ac20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 -- 172.21.2.104:0/2308497987 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc00b180 con 0x7ff1cc0b4180 2021-07-11T03:53:39.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.340+0000 7ff1cbfff700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1cc0b4180 0x7ff1cc141200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7ff1cc1545f0 tx=0x7ff1bc014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1c97fa700 1 -- 172.21.2.104:0/2308497987 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1bc012a20 con 0x7ff1cc0b4180 2021-07-11T03:53:39.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1c97fa700 1 -- 172.21.2.104:0/2308497987 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1bc012b80 con 0x7ff1cc0b4180 2021-07-11T03:53:39.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1d2477700 1 -- 172.21.2.104:0/2308497987 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc0affe0 con 0x7ff1cc0b4180 2021-07-11T03:53:39.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1c97fa700 1 -- 172.21.2.104:0/2308497987 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1bc01e430 con 0x7ff1cc0b4180 2021-07-11T03:53:39.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1d2477700 1 -- 172.21.2.104:0/2308497987 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc13b110 con 0x7ff1cc0b4180 2021-07-11T03:53:39.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1d2477700 1 -- 172.21.2.104:0/2308497987 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc13b110 con 0x7ff1cc0b4180 2021-07-11T03:53:39.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1c97fa700 1 -- 172.21.2.104:0/2308497987 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff1bc00f030 con 0x7ff1cc0b4180 2021-07-11T03:53:39.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1c97fa700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1b0053300 0x7ff1b00557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1c97fa700 1 -- 172.21.2.104:0/2308497987 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff1bc06c4e0 con 0x7ff1cc0b4180 2021-07-11T03:53:39.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1cb7fe700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1b0053300 0x7ff1b00557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1cb7fe700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1b0053300 0x7ff1b00557c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7ff1b4002f90 tx=0x7ff1b4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.344+0000 7ff1c97fa700 1 -- 172.21.2.104:0/2308497987 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff1bc043860 con 0x7ff1cc0b4180 2021-07-11T03:53:39.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 -- 172.21.2.104:0/2308497987 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff1cc13b110 con 0x7ff1cc0b4180 2021-07-11T03:53:39.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1c97fa700 1 -- 172.21.2.104:0/2308497987 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff1bc046e40 con 0x7ff1cc0b4180 2021-07-11T03:53:39.432 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:39.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 -- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1b0053300 msgr2=0x7ff1b00557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1b0053300 0x7ff1b00557c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7ff1b4002f90 tx=0x7ff1b4006930).stop 2021-07-11T03:53:39.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 -- 172.21.2.104:0/2308497987 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1cc0b4180 msgr2=0x7ff1cc141200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1cc0b4180 0x7ff1cc141200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7ff1cc1545f0 tx=0x7ff1bc014000).stop 2021-07-11T03:53:39.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 -- 172.21.2.104:0/2308497987 shutdown_connections 2021-07-11T03:53:39.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1cc146950 0x7ff1cc13ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1cc0b4690 0x7ff1cc13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff1b0053300 0x7ff1b00557c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 --2- 172.21.2.104:0/2308497987 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1cc0b4180 0x7ff1cc141200 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 -- 172.21.2.104:0/2308497987 >> 172.21.2.104:0/2308497987 conn(0x7ff1cc05c490 msgr2=0x7ff1cc13edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:39.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 -- 172.21.2.104:0/2308497987 shutdown_connections 2021-07-11T03:53:39.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.428+0000 7ff1d2477700 1 -- 172.21.2.104:0/2308497987 wait complete. 2021-07-11T03:53:39.439 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:53:39.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.492+0000 7f8ed53ff700 1 Processor -- start 2021-07-11T03:53:39.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.492+0000 7f8ed53ff700 1 -- start start 2021-07-11T03:53:39.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8ed00a9960 0x7f8ed00b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8ed00b06c0 0x7f8ed00aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed00b1c60 con 0x7f8ed00ac260 2021-07-11T03:53:39.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed00a9d40 con 0x7f8ed00a9960 2021-07-11T03:53:39.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed00b2940 con 0x7f8ed00b06c0 2021-07-11T03:53:39.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ece7fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8ed00ac260 0x7f8ed00aec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39490/0 (socket says 172.21.2.104:39490) 2021-07-11T03:53:39.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ece7fc700 1 -- 172.21.2.104:0/2502841908 learned_addr learned my addr 172.21.2.104:0/2502841908 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:39.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8eceffd700 1 --2- 172.21.2.104:0/2502841908 >> v2:172.21.2.104:3300/0 conn(0x7f8ed00a9960 0x7f8ed00b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8eceffd700 1 -- 172.21.2.104:0/2502841908 >> v2:172.21.2.105:3300/0 conn(0x7f8ed00b06c0 msgr2=0x7f8ed00aafa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:39.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8eceffd700 1 --2- 172.21.2.104:0/2502841908 >> v2:172.21.2.105:3300/0 conn(0x7f8ed00b06c0 0x7f8ed00aafa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8eceffd700 1 -- 172.21.2.104:0/2502841908 >> v1:172.21.2.104:6789/0 conn(0x7f8ed00ac260 legacy=0x7f8ed00aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ecdffb700 1 -- 172.21.2.104:0/2502841908 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481373623 0 0) 0x7f8ec0001110 con 0x7f8ed00ac260 2021-07-11T03:53:39.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8eceffd700 1 -- 172.21.2.104:0/2502841908 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00b00c0 con 0x7f8ed00a9960 2021-07-11T03:53:39.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8eceffd700 1 --2- 172.21.2.104:0/2502841908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ed00a9960 0x7f8ed00b1770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f8eb8002f70 tx=0x7f8eb8006df0).ready entity=mon.0 client_cookie=54d10ff87f9c24ee server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ecdffb700 1 -- 172.21.2.104:0/2502841908 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb800d5c0 con 0x7f8ed00a9960 2021-07-11T03:53:39.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ecdffb700 1 -- 172.21.2.104:0/2502841908 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb8000e90 con 0x7f8ed00a9960 2021-07-11T03:53:39.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ecdffb700 1 -- 172.21.2.104:0/2502841908 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb800d950 con 0x7f8ed00a9960 2021-07-11T03:53:39.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2502841908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ed00a9960 msgr2=0x7f8ed00b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 --2- 172.21.2.104:0/2502841908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ed00a9960 0x7f8ed00b1770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f8eb8002f70 tx=0x7f8eb8006df0).stop 2021-07-11T03:53:39.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2502841908 shutdown_connections 2021-07-11T03:53:39.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 --2- 172.21.2.104:0/2502841908 >> v2:172.21.2.105:3300/0 conn(0x7f8ed00b06c0 0x7f8ed00aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 --2- 172.21.2.104:0/2502841908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ed00a9960 0x7f8ed00b1770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2502841908 >> 172.21.2.104:0/2502841908 conn(0x7f8ed005f6b0 msgr2=0x7f8ed005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:39.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2502841908 shutdown_connections 2021-07-11T03:53:39.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2502841908 wait complete. 2021-07-11T03:53:39.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 Processor -- start 2021-07-11T03:53:39.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- start start 2021-07-11T03:53:39.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ed00a9960 0x7f8ed00b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ed00ac260 0x7f8ed0063e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ed00b06c0 0x7f8ed013c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed009f2f0 con 0x7f8ed00ac260 2021-07-11T03:53:39.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed00650a0 con 0x7f8ed00b06c0 2021-07-11T03:53:39.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0065340 con 0x7f8ed00a9960 2021-07-11T03:53:39.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ece7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ed00ac260 0x7f8ed0063e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ece7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ed00ac260 0x7f8ed0063e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42726/0 (socket says 172.21.2.104:42726) 2021-07-11T03:53:39.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ece7fc700 1 -- 172.21.2.104:0/2213531107 learned_addr learned my addr 172.21.2.104:0/2213531107 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:39.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ece7fc700 1 -- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ed00a9960 msgr2=0x7f8ed00b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ece7fc700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ed00a9960 0x7f8ed00b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ece7fc700 1 -- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ed00b06c0 msgr2=0x7f8ed013c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ece7fc700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ed00b06c0 0x7f8ed013c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ece7fc700 1 -- 172.21.2.104:0/2213531107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb800b180 con 0x7f8ed00ac260 2021-07-11T03:53:39.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ece7fc700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ed00ac260 0x7f8ed0063e80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f8ec0000bc0 tx=0x7f8ec0007900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ec7fff700 1 -- 172.21.2.104:0/2213531107 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ec0001d50 con 0x7f8ed00ac260 2021-07-11T03:53:39.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ec7fff700 1 -- 172.21.2.104:0/2213531107 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec00072e0 con 0x7f8ed00ac260 2021-07-11T03:53:39.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ec7fff700 1 -- 172.21.2.104:0/2213531107 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ec0005610 con 0x7f8ed00ac260 2021-07-11T03:53:39.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2213531107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed00655e0 con 0x7f8ed00ac260 2021-07-11T03:53:39.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.496+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2213531107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed00b3b80 con 0x7f8ed00ac260 2021-07-11T03:53:39.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.500+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2213531107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed0057350 con 0x7f8ed00ac260 2021-07-11T03:53:39.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.500+0000 7f8ec7fff700 1 -- 172.21.2.104:0/2213531107 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8ec0010070 con 0x7f8ed00ac260 2021-07-11T03:53:39.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.500+0000 7f8ec7fff700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8eb40531f0 0x7f8eb40556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.500+0000 7f8ec7fff700 1 -- 172.21.2.104:0/2213531107 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8ec006aaa0 con 0x7f8ed00ac260 2021-07-11T03:53:39.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.500+0000 7f8eceffd700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8eb40531f0 0x7f8eb40556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.500+0000 7f8eceffd700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8eb40531f0 0x7f8eb40556b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f8eb8002f70 tx=0x7f8eb8014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.500+0000 7f8ec7fff700 1 -- 172.21.2.104:0/2213531107 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8ec0041cd0 con 0x7f8ed00ac260 2021-07-11T03:53:39.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2213531107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ed00b1c60 con 0x7f8ed00ac260 2021-07-11T03:53:39.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ec7fff700 1 -- 172.21.2.104:0/2213531107 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8ed00b1c60 con 0x7f8ed00ac260 2021-07-11T03:53:39.588 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:39.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8eb40531f0 msgr2=0x7f8eb40556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8eb40531f0 0x7f8eb40556b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f8eb8002f70 tx=0x7f8eb8014000).stop 2021-07-11T03:53:39.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2213531107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ed00ac260 msgr2=0x7f8ed0063e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ed00ac260 0x7f8ed0063e80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f8ec0000bc0 tx=0x7f8ec0007900).stop 2021-07-11T03:53:39.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2213531107 shutdown_connections 2021-07-11T03:53:39.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ed00b06c0 0x7f8ed013c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8eb40531f0 0x7f8eb40556b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ed00ac260 0x7f8ed0063e80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 --2- 172.21.2.104:0/2213531107 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ed00a9960 0x7f8ed00b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2213531107 >> 172.21.2.104:0/2213531107 conn(0x7f8ed005f6b0 msgr2=0x7f8ed0141930 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:39.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2213531107 shutdown_connections 2021-07-11T03:53:39.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.584+0000 7f8ed53ff700 1 -- 172.21.2.104:0/2213531107 wait complete. 2021-07-11T03:53:39.595 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:53:39.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.644+0000 7fc0226b6700 1 Processor -- start 2021-07-11T03:53:39.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.644+0000 7fc0226b6700 1 -- start start 2021-07-11T03:53:39.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.644+0000 7fc0226b6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc01c0aca50 0x7fc01c0a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.644+0000 7fc0226b6700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc01c0b2c00 0x7fc01c0b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.644+0000 7fc0226b6700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01c139bc0 con 0x7fc01c0a95d0 2021-07-11T03:53:39.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.644+0000 7fc0226b6700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01c140500 con 0x7fc01c0b2c00 2021-07-11T03:53:39.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.644+0000 7fc0226b6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0b0130 con 0x7fc01c0aca50 2021-07-11T03:53:39.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc01b7fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc01c0aca50 0x7fc01c0a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc01b7fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc01c0aca50 0x7fc01c0a99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:45900/0 (socket says 172.21.2.104:45900) 2021-07-11T03:53:39.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc01b7fe700 1 -- 172.21.2.104:0/853660158 learned_addr learned my addr 172.21.2.104:0/853660158 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:39.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc020c53700 1 --2- 172.21.2.104:0/853660158 >> v2:172.21.2.105:3300/0 conn(0x7fc01c0b2c00 0x7fc01c0b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc01b7fe700 1 -- 172.21.2.104:0/853660158 >> v2:172.21.2.105:3300/0 conn(0x7fc01c0b2c00 msgr2=0x7fc01c0b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:39.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc01b7fe700 1 --2- 172.21.2.104:0/853660158 >> v2:172.21.2.105:3300/0 conn(0x7fc01c0b2c00 0x7fc01c0b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc01b7fe700 1 -- 172.21.2.104:0/853660158 >> v1:172.21.2.105:6789/0 conn(0x7fc01c0a95d0 legacy=0x7fc01c0a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc01b7fe700 1 -- 172.21.2.104:0/853660158 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c0b03d0 con 0x7fc01c0aca50 2021-07-11T03:53:39.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc01b7fe700 1 --2- 172.21.2.104:0/853660158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc01c0aca50 0x7fc01c0a99b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fc00c002f70 tx=0x7fc00c006eb0).ready entity=mon.2 client_cookie=a60a56b4495de389 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc01affd700 1 -- 172.21.2.104:0/853660158 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc00c00d570 con 0x7fc01c0aca50 2021-07-11T03:53:39.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc01affd700 1 -- 172.21.2.104:0/853660158 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c000e90 con 0x7fc01c0aca50 2021-07-11T03:53:39.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc01affd700 1 -- 172.21.2.104:0/853660158 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc00c00d900 con 0x7fc01c0aca50 2021-07-11T03:53:39.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc0226b6700 1 -- 172.21.2.104:0/853660158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc01c0aca50 msgr2=0x7fc01c0a99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc0226b6700 1 --2- 172.21.2.104:0/853660158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc01c0aca50 0x7fc01c0a99b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fc00c002f70 tx=0x7fc00c006eb0).stop 2021-07-11T03:53:39.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc0226b6700 1 -- 172.21.2.104:0/853660158 shutdown_connections 2021-07-11T03:53:39.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc0226b6700 1 --2- 172.21.2.104:0/853660158 >> v2:172.21.2.105:3300/0 conn(0x7fc01c0b2c00 0x7fc01c0b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc0226b6700 1 --2- 172.21.2.104:0/853660158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc01c0aca50 0x7fc01c0a99b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc0226b6700 1 -- 172.21.2.104:0/853660158 >> 172.21.2.104:0/853660158 conn(0x7fc01c05f6b0 msgr2=0x7fc01c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:39.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc0226b6700 1 -- 172.21.2.104:0/853660158 shutdown_connections 2021-07-11T03:53:39.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc0226b6700 1 -- 172.21.2.104:0/853660158 wait complete. 2021-07-11T03:53:39.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc0226b6700 1 Processor -- start 2021-07-11T03:53:39.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.648+0000 7fc0226b6700 1 -- start start 2021-07-11T03:53:39.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0226b6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc01c0a95d0 0x7fc01c0aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0226b6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc01c0aca50 0x7fc01c159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0226b6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc01c0b2c00 0x7fc01c15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0226b6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0b6480 con 0x7fc01c0aca50 2021-07-11T03:53:39.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0226b6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0b3320 con 0x7fc01c0a95d0 2021-07-11T03:53:39.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0226b6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0b35c0 con 0x7fc01c0b2c00 2021-07-11T03:53:39.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc01b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc01c0aca50 0x7fc01c159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc01b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc01c0aca50 0x7fc01c159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42742/0 (socket says 172.21.2.104:42742) 2021-07-11T03:53:39.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc01b7fe700 1 -- 172.21.2.104:0/490570932 learned_addr learned my addr 172.21.2.104:0/490570932 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:39.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc01bfff700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc01c0a95d0 0x7fc01c0aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc01b7fe700 1 -- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc01c0b2c00 msgr2=0x7fc01c15a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc01b7fe700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc01c0b2c00 0x7fc01c15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc01b7fe700 1 -- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc01c0a95d0 msgr2=0x7fc01c0aae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc01b7fe700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc01c0a95d0 0x7fc01c0aae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc01b7fe700 1 -- 172.21.2.104:0/490570932 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c00b1e0 con 0x7fc01c0aca50 2021-07-11T03:53:39.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc01b7fe700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc01c0aca50 0x7fc01c159db0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fc00c002f70 tx=0x7fc00c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0197fa700 1 -- 172.21.2.104:0/490570932 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc00c006be0 con 0x7fc01c0aca50 2021-07-11T03:53:39.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0197fa700 1 -- 172.21.2.104:0/490570932 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c00de40 con 0x7fc01c0aca50 2021-07-11T03:53:39.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0226b6700 1 -- 172.21.2.104:0/490570932 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01c0b3860 con 0x7fc01c0aca50 2021-07-11T03:53:39.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0197fa700 1 -- 172.21.2.104:0/490570932 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc00c01d450 con 0x7fc01c0aca50 2021-07-11T03:53:39.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0226b6700 1 -- 172.21.2.104:0/490570932 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c13cf90 con 0x7fc01c0aca50 2021-07-11T03:53:39.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0226b6700 1 -- 172.21.2.104:0/490570932 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c057350 con 0x7fc01c0aca50 2021-07-11T03:53:39.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0197fa700 1 -- 172.21.2.104:0/490570932 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc00c00f070 con 0x7fc01c0aca50 2021-07-11T03:53:39.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0197fa700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc000053270 0x7fc000055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc0197fa700 1 -- 172.21.2.104:0/490570932 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc00c06ccc0 con 0x7fc01c0aca50 2021-07-11T03:53:39.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.652+0000 7fc01bfff700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc000053270 0x7fc000055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.656+0000 7fc01bfff700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc000053270 0x7fc000055730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc004008000 tx=0x7fc004009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.656+0000 7fc0197fa700 1 -- 172.21.2.104:0/490570932 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc00c043f60 con 0x7fc01c0aca50 2021-07-11T03:53:39.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 -- 172.21.2.104:0/490570932 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc01c044650 con 0x7fc01c0aca50 2021-07-11T03:53:39.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0197fa700 1 -- 172.21.2.104:0/490570932 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc00c047580 con 0x7fc01c0aca50 2021-07-11T03:53:39.744 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:39.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 -- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc000053270 msgr2=0x7fc000055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc000053270 0x7fc000055730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc004008000 tx=0x7fc004009040).stop 2021-07-11T03:53:39.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 -- 172.21.2.104:0/490570932 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc01c0aca50 msgr2=0x7fc01c159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc01c0aca50 0x7fc01c159db0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fc00c002f70 tx=0x7fc00c015000).stop 2021-07-11T03:53:39.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 -- 172.21.2.104:0/490570932 shutdown_connections 2021-07-11T03:53:39.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc01c0b2c00 0x7fc01c15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc000053270 0x7fc000055730 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc01c0aca50 0x7fc01c159db0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 --2- 172.21.2.104:0/490570932 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc01c0a95d0 0x7fc01c0aae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 -- 172.21.2.104:0/490570932 >> 172.21.2.104:0/490570932 conn(0x7fc01c05f6b0 msgr2=0x7fc01c13da60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:39.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 -- 172.21.2.104:0/490570932 shutdown_connections 2021-07-11T03:53:39.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.740+0000 7fc0226b6700 1 -- 172.21.2.104:0/490570932 wait complete. 2021-07-11T03:53:39.764 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) 2021-07-11T03:53:39.765 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2021-07-11T03:53:39.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 Processor -- start 2021-07-11T03:53:39.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 -- start start 2021-07-11T03:53:39.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff75c0b96e0 0x7ff75c0aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c067500 con 0x7ff75c13fff0 2021-07-11T03:53:39.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0af4d0 con 0x7ff75c067120 2021-07-11T03:53:39.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c143080 con 0x7ff75c0b96e0 2021-07-11T03:53:39.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff761af7700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff75c13fff0 0x7ff75c140480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39514/0 (socket says 172.21.2.104:39514) 2021-07-11T03:53:39.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff761af7700 1 -- 172.21.2.104:0/657710343 learned_addr learned my addr 172.21.2.104:0/657710343 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:39.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff760af5700 1 --2- 172.21.2.104:0/657710343 >> v2:172.21.2.104:3300/0 conn(0x7ff75c0b96e0 0x7ff75c0aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff74bfff700 1 -- 172.21.2.104:0/657710343 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546159642 0 0) 0x7ff75c067500 con 0x7ff75c13fff0 2021-07-11T03:53:39.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff74bfff700 1 -- 172.21.2.104:0/657710343 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff744003260 con 0x7ff75c13fff0 2021-07-11T03:53:39.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff760af5700 1 -- 172.21.2.104:0/657710343 >> v1:172.21.2.105:6789/0 conn(0x7ff75c067120 legacy=0x7ff75c1439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:39.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff760af5700 1 -- 172.21.2.104:0/657710343 >> v1:172.21.2.104:6789/0 conn(0x7ff75c13fff0 legacy=0x7ff75c140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff760af5700 1 -- 172.21.2.104:0/657710343 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c143270 con 0x7ff75c0b96e0 2021-07-11T03:53:39.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff760af5700 1 --2- 172.21.2.104:0/657710343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0b96e0 0x7ff75c0aec00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7ff75c060320 tx=0x7ff74c006df0).ready entity=mon.0 client_cookie=a9b582a27053a5be server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff74bfff700 1 -- 172.21.2.104:0/657710343 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff74c00d5c0 con 0x7ff75c0b96e0 2021-07-11T03:53:39.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff74bfff700 1 -- 172.21.2.104:0/657710343 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c000e90 con 0x7ff75c0b96e0 2021-07-11T03:53:39.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff74bfff700 1 -- 172.21.2.104:0/657710343 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff74c00d910 con 0x7ff75c0b96e0 2021-07-11T03:53:39.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 -- 172.21.2.104:0/657710343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0b96e0 msgr2=0x7ff75c0aec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 --2- 172.21.2.104:0/657710343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0b96e0 0x7ff75c0aec00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7ff75c060320 tx=0x7ff74c006df0).stop 2021-07-11T03:53:39.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 -- 172.21.2.104:0/657710343 shutdown_connections 2021-07-11T03:53:39.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 --2- 172.21.2.104:0/657710343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0b96e0 0x7ff75c0aec00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:39.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 -- 172.21.2.104:0/657710343 >> 172.21.2.104:0/657710343 conn(0x7ff75c05f9f0 msgr2=0x7ff75c0616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:39.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 -- 172.21.2.104:0/657710343 shutdown_connections 2021-07-11T03:53:39.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.816+0000 7ff76355a700 1 -- 172.21.2.104:0/657710343 wait complete. 2021-07-11T03:53:39.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff76355a700 1 Processor -- start 2021-07-11T03:53:39.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff76355a700 1 -- start start 2021-07-11T03:53:39.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff76355a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff75c067120 0x7ff75c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff76355a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0b96e0 0x7ff75c159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff76355a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff75c13fff0 0x7ff75c15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff76355a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c064470 con 0x7ff75c0b96e0 2021-07-11T03:53:39.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff76355a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c159b80 con 0x7ff75c13fff0 2021-07-11T03:53:39.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff76355a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c064c70 con 0x7ff75c067120 2021-07-11T03:53:39.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff760af5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0b96e0 0x7ff75c159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff760af5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0b96e0 0x7ff75c159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42752/0 (socket says 172.21.2.104:42752) 2021-07-11T03:53:39.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff760af5700 1 -- 172.21.2.104:0/2707798278 learned_addr learned my addr 172.21.2.104:0/2707798278 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:39.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff760af5700 1 -- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff75c067120 msgr2=0x7ff75c0b7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff760af5700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff75c067120 0x7ff75c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff760af5700 1 -- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff75c13fff0 msgr2=0x7ff75c15a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:39.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff760af5700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff75c13fff0 0x7ff75c15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:39.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff760af5700 1 -- 172.21.2.104:0/2707798278 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c00b180 con 0x7ff75c0b96e0 2021-07-11T03:53:39.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff760af5700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0b96e0 0x7ff75c159f70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7ff75c0664a0 tx=0x7ff74c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff74a7fc700 1 -- 172.21.2.104:0/2707798278 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff74c006b20 con 0x7ff75c0b96e0 2021-07-11T03:53:39.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff74a7fc700 1 -- 172.21.2.104:0/2707798278 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c002f70 con 0x7ff75c0b96e0 2021-07-11T03:53:39.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff76355a700 1 -- 172.21.2.104:0/2707798278 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c064660 con 0x7ff75c0b96e0 2021-07-11T03:53:39.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff74a7fc700 1 -- 172.21.2.104:0/2707798278 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff74c01e620 con 0x7ff75c0b96e0 2021-07-11T03:53:39.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff76355a700 1 -- 172.21.2.104:0/2707798278 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c15a950 con 0x7ff75c0b96e0 2021-07-11T03:53:39.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff76355a700 1 -- 172.21.2.104:0/2707798278 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c0b07a0 con 0x7ff75c0b96e0 2021-07-11T03:53:39.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff74a7fc700 1 -- 172.21.2.104:0/2707798278 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff74c00f030 con 0x7ff75c0b96e0 2021-07-11T03:53:39.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff74a7fc700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff73c0532d0 0x7ff73c055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:39.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff74a7fc700 1 -- 172.21.2.104:0/2707798278 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff74c06c3f0 con 0x7ff75c0b96e0 2021-07-11T03:53:39.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.820+0000 7ff7612f6700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff73c0532d0 0x7ff73c055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:39.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.824+0000 7ff7612f6700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff73c0532d0 0x7ff73c055790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff758002f90 tx=0x7ff7580069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:39.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.824+0000 7ff74a7fc700 1 -- 172.21.2.104:0/2707798278 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff74c043730 con 0x7ff75c0b96e0 2021-07-11T03:53:39.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:39.932+0000 7ff76355a700 1 -- 172.21.2.104:0/2707798278 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7ff75c0b07a0 con 0x7ff75c0b96e0 2021-07-11T03:53:40.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff74a7fc700 1 -- 172.21.2.104:0/2707798278 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v32) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff74c046d50 con 0x7ff75c0b96e0 2021-07-11T03:53:40.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 -- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff73c0532d0 msgr2=0x7ff73c055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff73c0532d0 0x7ff73c055790 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7ff758002f90 tx=0x7ff7580069b0).stop 2021-07-11T03:53:40.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 -- 172.21.2.104:0/2707798278 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0b96e0 msgr2=0x7ff75c159f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0b96e0 0x7ff75c159f70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7ff75c0664a0 tx=0x7ff74c014000).stop 2021-07-11T03:53:40.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 -- 172.21.2.104:0/2707798278 shutdown_connections 2021-07-11T03:53:40.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff75c13fff0 0x7ff75c15a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:40.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff73c0532d0 0x7ff73c055790 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:40.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0b96e0 0x7ff75c159f70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:40.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 --2- 172.21.2.104:0/2707798278 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff75c067120 0x7ff75c0b7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:40.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 -- 172.21.2.104:0/2707798278 >> 172.21.2.104:0/2707798278 conn(0x7ff75c05f9f0 msgr2=0x7ff75c141ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:40.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 -- 172.21.2.104:0/2707798278 shutdown_connections 2021-07-11T03:53:40.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.180+0000 7ff76355a700 1 -- 172.21.2.104:0/2707798278 wait complete. 2021-07-11T03:53:40.192 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T03:53:40.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.244+0000 7f5396afa700 1 Processor -- start 2021-07-11T03:53:40.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.244+0000 7f5396afa700 1 -- start start 2021-07-11T03:53:40.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f53900b12d0 0x7f53900affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53900b22a0 0x7f5390141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53900a77d0 con 0x7f53900b1ec0 2021-07-11T03:53:40.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53900b1740 con 0x7f53900b12d0 2021-07-11T03:53:40.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53900b0650 con 0x7f53900b22a0 2021-07-11T03:53:40.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538ffff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f53900b12d0 0x7f53900affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:40.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538ffff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f53900b12d0 0x7f53900affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42762/0 (socket says 172.21.2.104:42762) 2021-07-11T03:53:40.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538ffff700 1 -- 172.21.2.104:0/1681176057 learned_addr learned my addr 172.21.2.104:0/1681176057 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:40.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538ffff700 1 -- 172.21.2.104:0/1681176057 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53900b22a0 msgr2=0x7f5390141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:40.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538ffff700 1 --2- 172.21.2.104:0/1681176057 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53900b22a0 0x7f5390141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:40.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538ffff700 1 -- 172.21.2.104:0/1681176057 >> v1:172.21.2.104:6789/0 conn(0x7f53900b1ec0 legacy=0x7f53900a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538ffff700 1 -- 172.21.2.104:0/1681176057 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53900b08f0 con 0x7f53900b12d0 2021-07-11T03:53:40.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538ffff700 1 --2- 172.21.2.104:0/1681176057 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53900b12d0 0x7f53900affb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f5378002f70 tx=0x7f5378006eb0).ready entity=mon.0 client_cookie=5a137f9383ecbe21 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:40.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538f7fe700 1 -- 172.21.2.104:0/1681176057 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f537800d5c0 con 0x7f53900b12d0 2021-07-11T03:53:40.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538f7fe700 1 -- 172.21.2.104:0/1681176057 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5378000e90 con 0x7f53900b12d0 2021-07-11T03:53:40.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538f7fe700 1 -- 172.21.2.104:0/1681176057 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f537800d950 con 0x7f53900b12d0 2021-07-11T03:53:40.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- 172.21.2.104:0/1681176057 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53900b12d0 msgr2=0x7f53900affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 --2- 172.21.2.104:0/1681176057 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53900b12d0 0x7f53900affb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f5378002f70 tx=0x7f5378006eb0).stop 2021-07-11T03:53:40.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- 172.21.2.104:0/1681176057 shutdown_connections 2021-07-11T03:53:40.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 --2- 172.21.2.104:0/1681176057 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53900b12d0 0x7f53900affb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:40.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 --2- 172.21.2.104:0/1681176057 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53900b22a0 0x7f5390141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:40.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- 172.21.2.104:0/1681176057 >> 172.21.2.104:0/1681176057 conn(0x7f539005f6b0 msgr2=0x7f539005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:40.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- 172.21.2.104:0/1681176057 shutdown_connections 2021-07-11T03:53:40.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- 172.21.2.104:0/1681176057 wait complete. 2021-07-11T03:53:40.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 Processor -- start 2021-07-11T03:53:40.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- start start 2021-07-11T03:53:40.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53900b12d0 0x7f53900af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53900b1ec0 0x7f5390131800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53900b22a0 0x7f5390131cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900ab820 con 0x7f53900b22a0 2021-07-11T03:53:40.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53900a9c10 con 0x7f53900b12d0 2021-07-11T03:53:40.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53901321e0 con 0x7f53900b1ec0 2021-07-11T03:53:40.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5395097700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53900b22a0 0x7f5390131cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:40.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5395097700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53900b22a0 0x7f5390131cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42768/0 (socket says 172.21.2.104:42768) 2021-07-11T03:53:40.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5395097700 1 -- 172.21.2.104:0/1465504288 learned_addr learned my addr 172.21.2.104:0/1465504288 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:40.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5394896700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53900b12d0 0x7f53900af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:40.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5395097700 1 -- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53900b1ec0 msgr2=0x7f5390131800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538ffff700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53900b1ec0 0x7f5390131800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:40.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5395097700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53900b1ec0 0x7f5390131800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:40.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5395097700 1 -- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53900b12d0 msgr2=0x7f53900af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5395097700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53900b12d0 0x7f53900af540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:40.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5395097700 1 -- 172.21.2.104:0/1465504288 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537800b1e0 con 0x7f53900b22a0 2021-07-11T03:53:40.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5394896700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53900b12d0 0x7f53900af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:40.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538ffff700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53900b1ec0 0x7f5390131800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:40.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5395097700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53900b22a0 0x7f5390131cf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f53880032f0 tx=0x7f53880091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:40.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538dffb700 1 -- 172.21.2.104:0/1465504288 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f538800d4a0 con 0x7f53900b22a0 2021-07-11T03:53:40.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538dffb700 1 -- 172.21.2.104:0/1465504288 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538800d600 con 0x7f53900b22a0 2021-07-11T03:53:40.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- 172.21.2.104:0/1465504288 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539013ab40 con 0x7f53900b22a0 2021-07-11T03:53:40.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f538dffb700 1 -- 172.21.2.104:0/1465504288 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f538800d960 con 0x7f53900b22a0 2021-07-11T03:53:40.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.248+0000 7f5396afa700 1 -- 172.21.2.104:0/1465504288 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53900a9420 con 0x7f53900b22a0 2021-07-11T03:53:40.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.252+0000 7f5396afa700 1 -- 172.21.2.104:0/1465504288 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5390057350 con 0x7f53900b22a0 2021-07-11T03:53:40.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.252+0000 7f538dffb700 1 -- 172.21.2.104:0/1465504288 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5388018030 con 0x7f53900b22a0 2021-07-11T03:53:40.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.252+0000 7f538dffb700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5374053160 0x7f5374055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.252+0000 7f538dffb700 1 -- 172.21.2.104:0/1465504288 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f538806aed0 con 0x7f53900b22a0 2021-07-11T03:53:40.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.252+0000 7f5394896700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5374053160 0x7f5374055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:40.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.252+0000 7f5394896700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5374053160 0x7f5374055620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f5380008000 tx=0x7f5380009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:40.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.252+0000 7f538dffb700 1 -- 172.21.2.104:0/1465504288 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f53880421b0 con 0x7f53900b22a0 2021-07-11T03:53:40.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.336+0000 7f5396afa700 1 -- 172.21.2.104:0/1465504288 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5390144f90 con 0x7f53900b22a0 2021-07-11T03:53:40.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.340+0000 7f538dffb700 1 -- 172.21.2.104:0/1465504288 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f53880457d0 con 0x7f53900b22a0 2021-07-11T03:53:40.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 -- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5374053160 msgr2=0x7f5374055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5374053160 0x7f5374055620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f5380008000 tx=0x7f5380009040).stop 2021-07-11T03:53:40.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 -- 172.21.2.104:0/1465504288 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53900b22a0 msgr2=0x7f5390131cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53900b22a0 0x7f5390131cf0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f53880032f0 tx=0x7f53880091a0).stop 2021-07-11T03:53:40.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 -- 172.21.2.104:0/1465504288 shutdown_connections 2021-07-11T03:53:40.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5374053160 0x7f5374055620 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:40.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53900b22a0 0x7f5390131cf0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:40.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53900b1ec0 0x7f5390131800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:40.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 --2- 172.21.2.104:0/1465504288 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53900b12d0 0x7f53900af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:40.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 -- 172.21.2.104:0/1465504288 >> 172.21.2.104:0/1465504288 conn(0x7f539005f6b0 msgr2=0x7f539013b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:40.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 -- 172.21.2.104:0/1465504288 shutdown_connections 2021-07-11T03:53:40.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.344+0000 7f5396afa700 1 -- 172.21.2.104:0/1465504288 wait complete. 2021-07-11T03:53:40.353 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T03:53:40.354 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:53:40.362 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:53:40.375 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:53:40.388 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:53:40.401 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T03:53:40.402 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:40.403 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:40.403 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:40.407 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:53:40.408 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:53:40.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:40.409 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:40.409 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:40.415 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:53:40.416 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:53:40.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:40.417 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:40.417 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:40.471 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-07-11T03:53:40.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:53:40.473 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-07-11T03:53:40.473 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T03:53:40.474 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T03:53:40.474 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:40.490 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:40.491 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:40.491 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:40.496 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T03:53:40.497 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T03:53:40.497 INFO:teuthology.orchestra.run:waiting for 900 2021-07-11T03:53:40.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:53:40.498 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:40.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:40.526 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:40.526 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:53:40.538 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T03:53:40.539 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T03:53:40.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:40.540 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:40.541 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:53:40.593 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T03:53:40.594 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T03:53:40.595 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:40.595 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:40.596 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:53:40.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:53:40.605 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': Transport endpoint is not connected 2021-07-11T03:53:40.606 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.1 2021-07-11T03:53:40.606 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba023.front.sepia.ceph.com... 2021-07-11T03:53:40.607 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T03:53:40.607 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:53:40.622 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:40.623 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:40.623 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:53:40.670 INFO:teuthology.orchestra.run.gibba023.stdout:ext2/ext3 2021-07-11T03:53:40.671 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T03:53:40.672 INFO:teuthology.orchestra.run:waiting for 900 2021-07-11T03:53:40.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:53:40.673 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:53:40.688 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T03:53:40.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 Processor -- start 2021-07-11T03:53:40.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 -- start start 2021-07-11T03:53:40.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab18060100 0x7fab180b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fab18067300 0x7fab181438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab180aabb0 con 0x7fab180b4c40 2021-07-11T03:53:40.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab180604e0 con 0x7fab18067300 2021-07-11T03:53:40.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab18066ea0 con 0x7fab18060100 2021-07-11T03:53:40.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab16d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fab180b4c40 0x7fab18144510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39546/0 (socket says 172.21.2.104:39546) 2021-07-11T03:53:40.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab16d9d700 1 -- 172.21.2.104:0/2860952983 learned_addr learned my addr 172.21.2.104:0/2860952983 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:40.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab15d9b700 1 --2- 172.21.2.104:0/2860952983 >> v2:172.21.2.104:3300/0 conn(0x7fab18067300 0x7fab181438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:40.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1559a700 1 -- 172.21.2.104:0/2860952983 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3779773982 0 0) 0x7fab180aabb0 con 0x7fab180b4c40 2021-07-11T03:53:40.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1559a700 1 -- 172.21.2.104:0/2860952983 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faafc003260 con 0x7fab180b4c40 2021-07-11T03:53:40.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab15d9b700 1 -- 172.21.2.104:0/2860952983 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab18060100 msgr2=0x7fab180b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:40.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab15d9b700 1 --2- 172.21.2.104:0/2860952983 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab18060100 0x7fab180b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:40.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab15d9b700 1 -- 172.21.2.104:0/2860952983 >> v1:172.21.2.104:6789/0 conn(0x7fab180b4c40 legacy=0x7fab18144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab15d9b700 1 -- 172.21.2.104:0/2860952983 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180b33b0 con 0x7fab18067300 2021-07-11T03:53:40.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab15d9b700 1 --2- 172.21.2.104:0/2860952983 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab18067300 0x7fab181438f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fab0c002f70 tx=0x7fab0c006df0).ready entity=mon.0 client_cookie=87085d9fd05604a8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:40.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1559a700 1 -- 172.21.2.104:0/2860952983 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c00d570 con 0x7fab18067300 2021-07-11T03:53:40.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1559a700 1 -- 172.21.2.104:0/2860952983 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c000e90 con 0x7fab18067300 2021-07-11T03:53:40.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 -- 172.21.2.104:0/2860952983 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab18067300 msgr2=0x7fab181438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 --2- 172.21.2.104:0/2860952983 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab18067300 0x7fab181438f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fab0c002f70 tx=0x7fab0c006df0).stop 2021-07-11T03:53:40.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1559a700 1 -- 172.21.2.104:0/2860952983 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c00d8c0 con 0x7fab18067300 2021-07-11T03:53:40.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 -- 172.21.2.104:0/2860952983 shutdown_connections 2021-07-11T03:53:40.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 --2- 172.21.2.104:0/2860952983 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab18067300 0x7fab181438f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:40.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 --2- 172.21.2.104:0/2860952983 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab18060100 0x7fab180b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:40.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 -- 172.21.2.104:0/2860952983 >> 172.21.2.104:0/2860952983 conn(0x7fab1805cf30 msgr2=0x7fab1805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:40.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 -- 172.21.2.104:0/2860952983 shutdown_connections 2021-07-11T03:53:40.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.740+0000 7fab1c8f0700 1 -- 172.21.2.104:0/2860952983 wait complete. 2021-07-11T03:53:40.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1c8f0700 1 Processor -- start 2021-07-11T03:53:40.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1c8f0700 1 -- start start 2021-07-11T03:53:40.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1c8f0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab18060100 0x7fab180ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1c8f0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab18067300 0x7fab181395d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1c8f0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab180b4c40 0x7fab18139ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1c8f0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab18137460 con 0x7fab18067300 2021-07-11T03:53:40.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1c8f0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab180aeeb0 con 0x7fab18060100 2021-07-11T03:53:40.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1c8f0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab180af150 con 0x7fab180b4c40 2021-07-11T03:53:40.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab15d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab18067300 0x7fab181395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:40.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab15d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab18067300 0x7fab181395d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42784/0 (socket says 172.21.2.104:42784) 2021-07-11T03:53:40.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab15d9b700 1 -- 172.21.2.104:0/836754279 learned_addr learned my addr 172.21.2.104:0/836754279 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:40.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab15d9b700 1 -- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab180b4c40 msgr2=0x7fab18139ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab15d9b700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab180b4c40 0x7fab18139ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:40.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab15d9b700 1 -- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab18060100 msgr2=0x7fab180ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:40.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab15d9b700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab18060100 0x7fab180ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:40.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab15d9b700 1 -- 172.21.2.104:0/836754279 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c00b180 con 0x7fab18067300 2021-07-11T03:53:40.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab15d9b700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab18067300 0x7fab181395d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fab0c002f70 tx=0x7fab0c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:40.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab077fe700 1 -- 172.21.2.104:0/836754279 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c00de40 con 0x7fab18067300 2021-07-11T03:53:40.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab077fe700 1 -- 172.21.2.104:0/836754279 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c01e430 con 0x7fab18067300 2021-07-11T03:53:40.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab077fe700 1 -- 172.21.2.104:0/836754279 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c01e790 con 0x7fab18067300 2021-07-11T03:53:40.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1c8f0700 1 -- 172.21.2.104:0/836754279 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab180af3f0 con 0x7fab18067300 2021-07-11T03:53:40.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1c8f0700 1 -- 172.21.2.104:0/836754279 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab180aa730 con 0x7fab18067300 2021-07-11T03:53:40.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1c8f0700 1 -- 172.21.2.104:0/836754279 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab18057350 con 0x7fab18067300 2021-07-11T03:53:40.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab077fe700 1 -- 172.21.2.104:0/836754279 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fab0c00f030 con 0x7fab18067300 2021-07-11T03:53:40.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab077fe700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaf40531a0 0x7faaf4055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:40.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab077fe700 1 -- 172.21.2.104:0/836754279 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fab0c06c4d0 con 0x7fab18067300 2021-07-11T03:53:40.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.744+0000 7fab1659c700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaf40531a0 0x7faaf4055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:40.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.748+0000 7fab1659c700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaf40531a0 0x7faaf4055660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fab08002f70 tx=0x7fab08006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:40.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.748+0000 7fab077fe700 1 -- 172.21.2.104:0/836754279 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fab0c043700 con 0x7fab18067300 2021-07-11T03:53:40.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab1c8f0700 1 -- 172.21.2.104:0/836754279 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fab180698e0 con 0x7fab18067300 2021-07-11T03:53:40.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab077fe700 1 -- 172.21.2.104:0/836754279 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v23) v1 ==== 81+0+33645 (secure 0 0 0) 0x7fab0c046d20 con 0x7fab18067300 2021-07-11T03:53:40.833 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:40.833 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:53:40.834 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 23, 2021-07-11T03:53:40.834 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T03:53:40.834 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T03:53:40.834 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T03:53:24.367517+0000", 2021-07-11T03:53:40.835 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:53:40.835 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:53:40.835 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T03:53:40.835 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T03:53:40.836 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T03:53:40.836 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T03:53:40.836 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T03:53:40.836 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T03:53:40.836 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T03:53:40.837 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:53:40.837 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T03:53:40.837 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T03:53:40.837 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T03:53:40.837 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T03:53:40.838 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T03:53:40.838 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 4, 2021-07-11T03:53:40.838 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T03:53:40.838 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T03:53:40.839 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T03:53:40.839 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T03:53:40.839 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T03:53:40.839 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.839 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T03:53:40.840 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T03:53:40.840 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T03:53:40.843 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:53:40.843 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:53:40.844 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:53:40.844 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:53:40.844 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:53:40.844 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:53:40.845 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:53:40.845 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:53:40.845 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T03:53:40.845 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T03:53:40.846 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T03:53:40.846 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T03:53:40.846 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T03:53:40.846 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:53:40.847 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:53:40.847 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:53:40.847 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:53:40.847 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:53:40.847 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:53:40.848 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:53:40.848 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.848 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T03:53:40.848 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:53:40.849 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:53:40.849 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:53:40.849 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:53:40.850 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:53:40.850 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:53:40.850 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:53:40.851 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:53:40.851 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:53:40.851 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:53:40.851 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:53:40.851 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:53:40.852 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:53:40.852 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:53:40.852 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:53:40.853 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:53:40.853 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:53:40.853 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:53:40.853 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:53:40.853 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:53:40.854 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:53:40.854 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:53:40.854 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:53:40.854 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:53:40.855 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:53:40.855 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:53:40.855 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.855 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:53:40.856 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:53:40.856 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:53:40.856 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:53:40.856 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:53:40.856 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:53:40.857 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:53:40.857 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:53:40.857 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:53:40.857 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:53:40.858 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:53:40.858 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:53:40.858 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T03:53:40.858 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.858 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:53:40.859 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T03:53:40.859 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.859 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.859 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.860 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T03:53:40.860 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T03:53:40.861 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T03:53:40.861 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:53:40.861 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:53:40.861 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:53:40.861 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:53:40.862 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:53:40.862 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:53:40.862 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:53:40.863 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:53:40.863 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:53:40.863 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:53:40.863 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:53:40.864 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:53:40.864 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:53:40.864 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:53:40.864 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:53:40.865 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:53:40.865 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:53:40.865 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:53:40.865 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:53:40.866 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:53:40.866 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.866 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T03:53:40.866 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:53:40.867 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:53:40.867 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:53:40.867 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:53:40.867 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:53:40.868 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:53:40.868 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:53:40.868 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:53:40.868 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:53:40.868 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:53:40.869 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:53:40.869 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:53:40.869 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:53:40.869 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:53:40.870 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:53:40.870 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:53:40.870 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:53:40.871 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:53:40.871 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:53:40.871 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:53:40.871 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:53:40.872 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:53:40.872 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:53:40.872 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:53:40.872 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:53:40.872 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:53:40.873 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.873 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:53:40.873 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:53:40.873 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:53:40.874 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:53:40.874 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:53:40.874 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:53:40.874 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:53:40.875 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:53:40.875 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:53:40.875 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:53:40.875 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:53:40.875 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:53:40.876 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:53:40.876 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T03:53:40.876 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.876 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.877 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.877 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 3, 2021-07-11T03:53:40.877 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T03:53:40.877 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:20.980780+0000", 2021-07-11T03:53:40.878 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:53:40.878 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:53:40.878 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:53:40.878 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:53:40.879 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:53:40.879 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:53:40.879 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:53:40.880 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:53:40.880 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:53:40.880 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:53:40.880 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:53:40.881 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:53:40.882 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:53:40.882 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:53:40.882 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:53:40.882 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:53:40.883 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:53:40.883 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:53:40.883 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:53:40.883 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:53:40.883 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.884 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "21", 2021-07-11T03:53:40.884 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:53:40.884 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:53:40.884 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:53:40.885 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:53:40.885 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:53:40.885 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:53:40.885 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:53:40.886 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:53:40.886 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:53:40.886 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:53:40.886 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:53:40.887 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:53:40.887 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:53:40.887 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:53:40.887 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:53:40.887 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:53:40.888 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:53:40.888 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:53:40.888 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:53:40.888 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:53:40.889 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:53:40.889 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:53:40.889 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:53:40.889 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:53:40.890 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:53:40.891 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:53:40.891 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.892 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:53:40.892 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:53:40.892 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:53:40.892 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:53:40.893 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:53:40.893 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:53:40.893 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:53:40.893 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:53:40.894 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:53:40.894 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:53:40.894 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:53:40.894 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:53:40.894 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T03:53:40.895 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T03:53:40.895 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T03:53:40.895 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.895 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:53:40.895 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T03:53:40.896 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T03:53:40.896 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.896 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.896 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.897 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.897 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 4, 2021-07-11T03:53:40.897 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T03:53:40.897 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:21.341375+0000", 2021-07-11T03:53:40.898 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:53:40.898 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:53:40.898 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:53:40.898 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:53:40.898 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:53:40.899 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:53:40.899 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:53:40.899 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:53:40.899 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:53:40.900 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:53:40.900 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:53:40.900 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:53:40.900 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:53:40.900 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:53:40.901 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:53:40.901 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:53:40.901 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:53:40.901 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:53:40.902 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:53:40.903 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:53:40.903 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.903 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "23", 2021-07-11T03:53:40.903 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:53:40.903 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:53:40.904 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:53:40.904 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:53:40.904 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:53:40.904 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:53:40.905 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:53:40.905 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:53:40.905 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:53:40.905 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:53:40.906 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:53:40.906 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:53:40.906 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:53:40.906 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:53:40.906 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:53:40.907 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:53:40.907 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:53:40.907 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:53:40.907 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:53:40.907 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:53:40.908 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:53:40.908 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:53:40.908 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:53:40.908 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:53:40.909 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:53:40.909 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:53:40.909 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.909 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:53:40.909 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:53:40.910 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:53:40.910 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:53:40.910 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:53:40.911 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:53:40.911 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:53:40.911 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:53:40.911 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:53:40.911 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:53:40.912 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:53:40.912 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:53:40.913 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:53:40.913 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T03:53:40.913 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T03:53:40.913 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.914 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.914 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.914 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:53:40.914 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T03:53:40.915 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.915 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:53:40.915 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T03:53:40.915 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:53:40.916 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:53:40.916 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:53:40.916 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:53:40.916 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:53:40.917 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:53:40.917 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:53:40.917 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 20, 2021-07-11T03:53:40.917 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:53:40.918 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:53:40.918 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:53:40.918 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.918 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.919 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.919 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T03:53:40.919 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:53:40.919 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.919 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.920 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.920 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T03:53:40.920 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:53:40.920 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.921 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.921 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.921 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:53:40.921 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.921 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.922 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.922 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T03:53:40.922 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:53:40.922 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.922 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.924 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.924 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T03:53:40.925 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:53:40.925 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.925 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.925 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.926 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:53:40.926 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.926 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.926 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.927 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T03:53:40.927 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:53:40.927 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.927 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.927 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.928 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T03:53:40.928 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:53:40.928 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.928 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.929 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.929 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:53:40.929 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.929 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.929 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.930 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T03:53:40.930 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:53:40.930 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.930 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.931 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.931 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T03:53:40.931 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:53:40.931 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.931 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.932 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.932 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T03:53:40.932 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T03:53:40.932 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T03:53:40.932 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T03:53:40.933 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:53:40.933 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:53:40.934 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:53:40.934 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.934 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.934 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.935 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:53:40.935 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T03:53:40.935 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:53:40.935 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:53:40.935 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:53:40.936 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:53:40.936 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:53:40.936 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:53:40.936 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:53:40.937 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 20, 2021-07-11T03:53:40.937 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:53:40.937 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:53:40.937 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:53:40.937 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.938 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.938 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.938 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T03:53:40.938 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:53:40.938 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.939 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.939 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.939 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T03:53:40.939 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:53:40.940 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.940 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.940 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.940 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:53:40.940 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.941 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.941 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.941 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T03:53:40.941 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:53:40.941 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.942 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.942 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.942 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T03:53:40.942 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:53:40.943 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.943 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.943 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.943 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:53:40.943 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.944 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.944 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.945 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T03:53:40.945 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:53:40.945 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.945 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.945 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.946 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T03:53:40.946 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:53:40.946 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.946 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.947 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.947 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:53:40.947 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.947 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.948 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.948 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T03:53:40.948 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:53:40.948 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.949 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.949 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.949 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T03:53:40.949 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:53:40.950 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.950 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.950 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.950 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T03:53:40.951 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T03:53:40.951 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T03:53:40.951 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T03:53:40.951 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:53:40.952 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:53:40.952 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:53:40.952 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.952 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.953 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.953 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:53:40.953 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T03:53:40.953 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:53:40.953 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:53:40.954 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:53:40.954 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:53:40.955 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:53:40.955 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:53:40.955 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:53:40.955 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 17, 2021-07-11T03:53:40.956 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:53:40.956 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:53:40.956 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:53:40.956 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.957 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.957 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.957 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T03:53:40.957 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:53:40.957 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.958 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.958 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.958 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T03:53:40.958 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:53:40.959 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.959 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.959 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.959 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:53:40.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.960 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.960 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T03:53:40.960 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:53:40.961 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.961 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.961 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.961 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T03:53:40.961 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:53:40.962 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.962 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.962 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.962 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:53:40.963 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.963 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.963 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.963 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T03:53:40.964 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:53:40.964 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.964 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.964 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.965 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T03:53:40.965 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:53:40.965 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.966 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.966 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.966 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:53:40.966 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.967 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.967 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.967 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T03:53:40.967 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:53:40.967 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.968 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.968 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.968 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T03:53:40.968 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:53:40.968 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.969 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.969 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.969 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T03:53:40.969 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T03:53:40.970 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T03:53:40.970 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T03:53:40.970 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:53:40.970 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:53:40.970 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:53:40.971 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.971 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.971 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.971 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:53:40.972 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T03:53:40.972 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:53:40.972 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:53:40.972 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:53:40.972 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:53:40.973 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:53:40.973 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:53:40.973 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:53:40.973 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 20, 2021-07-11T03:53:40.973 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:53:40.974 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:53:40.974 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:53:40.974 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.974 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.975 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.975 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T03:53:40.976 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:53:40.976 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.976 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.976 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.976 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T03:53:40.977 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:53:40.977 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.977 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.977 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.978 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:53:40.978 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.978 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.978 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.978 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T03:53:40.979 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:53:40.979 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.979 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.979 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.980 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T03:53:40.980 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:53:40.980 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.980 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.980 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.981 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:53:40.981 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.981 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.981 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.982 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T03:53:40.982 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:53:40.982 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.982 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.982 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.983 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T03:53:40.983 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:53:40.983 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.983 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.984 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.984 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:53:40.984 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.984 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.984 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.985 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T03:53:40.985 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:53:40.985 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.986 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.986 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.986 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T03:53:40.987 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:53:40.987 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.987 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.987 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.987 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T03:53:40.988 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T03:53:40.988 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T03:53:40.988 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T03:53:40.988 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:53:40.988 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:53:40.989 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:53:40.989 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.989 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.989 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.990 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:53:40.990 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T03:53:40.990 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:53:40.990 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:53:40.990 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:53:40.991 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:53:40.991 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:53:40.991 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:53:40.991 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:53:40.992 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 20, 2021-07-11T03:53:40.992 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:53:40.992 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:53:40.992 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:53:40.992 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.993 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.993 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.993 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T03:53:40.993 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:53:40.993 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.994 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.994 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.994 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T03:53:40.994 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:53:40.995 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.995 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.995 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.995 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:53:40.995 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:40.996 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.996 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:40.997 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T03:53:40.997 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:53:40.997 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.997 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:40.998 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:40.998 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T03:53:40.998 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:53:40.998 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:40.999 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:40.999 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:40.999 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:53:40.999 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.000 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.000 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.000 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T03:53:41.000 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:53:41.000 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.001 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.001 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.001 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T03:53:41.001 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:53:41.002 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.002 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.002 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.002 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:53:41.003 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.003 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.003 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.003 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T03:53:41.004 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:53:41.004 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.004 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.004 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.005 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T03:53:41.005 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:53:41.005 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.005 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.006 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.006 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T03:53:41.006 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T03:53:41.007 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T03:53:41.007 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T03:53:41.007 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:53:41.008 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:53:41.008 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:53:41.008 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.008 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.008 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.009 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:53:41.009 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T03:53:41.009 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:53:41.009 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:53:41.010 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:53:41.010 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:53:41.010 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:53:41.010 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:53:41.010 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:53:41.011 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 19, 2021-07-11T03:53:41.011 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:53:41.011 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:53:41.011 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:53:41.012 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.012 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.012 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.012 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T03:53:41.012 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:53:41.013 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.013 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.013 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.013 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T03:53:41.014 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:53:41.014 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.014 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.014 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.015 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:53:41.015 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.015 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.015 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.015 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T03:53:41.016 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:53:41.016 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.016 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.016 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.017 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T03:53:41.017 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:53:41.018 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.018 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.018 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.018 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:53:41.019 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.019 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.019 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.020 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T03:53:41.020 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:53:41.020 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.020 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.021 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.021 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T03:53:41.021 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:53:41.021 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.021 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.022 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.022 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:53:41.022 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.022 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.023 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.023 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T03:53:41.023 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:53:41.023 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.024 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.024 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.024 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T03:53:41.024 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:53:41.025 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.025 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.025 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.025 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T03:53:41.026 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T03:53:41.026 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T03:53:41.026 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T03:53:41.026 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:53:41.027 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:53:41.027 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:53:41.027 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.028 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.028 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.028 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:53:41.028 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T03:53:41.029 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:53:41.029 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:53:41.029 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:53:41.029 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:53:41.030 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:53:41.030 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:53:41.030 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:53:41.030 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 21, 2021-07-11T03:53:41.031 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:53:41.031 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:53:41.031 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:53:41.031 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.031 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.032 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.032 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T03:53:41.032 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:53:41.032 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.033 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.033 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.033 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T03:53:41.033 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:53:41.033 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.034 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.034 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.034 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:53:41.034 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.035 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.035 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.035 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T03:53:41.035 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:53:41.035 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.036 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.036 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.036 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T03:53:41.036 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:53:41.037 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.037 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.037 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.037 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:53:41.037 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.038 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.038 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.039 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T03:53:41.039 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:53:41.039 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.039 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.040 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.040 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T03:53:41.040 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:53:41.040 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.041 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.041 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.041 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:53:41.041 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.041 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.042 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.042 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T03:53:41.042 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:53:41.043 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.043 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.043 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.043 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T03:53:41.043 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:53:41.044 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.044 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.044 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.044 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T03:53:41.045 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T03:53:41.045 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T03:53:41.045 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T03:53:41.045 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:53:41.046 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:53:41.046 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:53:41.046 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.046 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.046 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.047 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:53:41.047 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T03:53:41.047 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:53:41.047 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:53:41.048 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:53:41.048 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:53:41.048 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:53:41.049 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:53:41.049 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:53:41.049 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 17, 2021-07-11T03:53:41.049 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:53:41.050 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:53:41.050 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:53:41.050 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.050 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.051 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.051 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T03:53:41.051 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:53:41.051 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.052 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.052 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.052 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T03:53:41.052 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:53:41.053 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.053 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.053 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.053 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:53:41.054 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.054 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.054 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.054 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T03:53:41.054 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:53:41.055 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.055 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.055 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.055 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T03:53:41.056 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:53:41.056 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.056 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.056 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.056 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:53:41.057 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.057 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.057 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.057 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T03:53:41.057 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:53:41.058 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.058 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.058 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.058 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T03:53:41.059 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:53:41.059 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.059 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.059 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.059 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:53:41.060 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:53:41.060 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.060 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:53:41.060 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T03:53:41.061 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:53:41.061 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.061 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.061 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:53:41.061 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T03:53:41.062 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:53:41.062 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.062 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.062 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.063 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T03:53:41.063 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T03:53:41.063 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T03:53:41.063 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T03:53:41.064 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:53:41.064 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:53:41.064 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:53:41.064 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:41.065 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.065 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:53:41.065 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T03:53:41.065 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.065 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:53:41.066 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:53:41.066 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:53:41.066 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:53:41.066 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:53:41.067 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:53:41.067 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T03:53:41.067 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:53:41.067 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.068 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.068 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:53:41.068 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:53:41.068 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:53:41.069 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:53:41.069 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:53:41.069 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:53:41.069 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T03:53:41.069 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:53:41.070 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.070 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.070 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:53:41.070 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:53:41.071 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:53:41.071 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:53:41.071 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:53:41.071 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:53:41.071 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T03:53:41.072 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:53:41.072 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.072 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.072 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:53:41.073 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:53:41.073 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:53:41.073 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:53:41.073 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:53:41.073 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:53:41.074 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T03:53:41.074 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:53:41.074 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.074 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.075 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:53:41.075 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:53:41.075 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:53:41.075 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:53:41.075 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:53:41.076 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:53:41.076 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T03:53:41.076 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:53:41.076 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.076 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.077 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:53:41.077 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:53:41.077 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:53:41.077 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:53:41.078 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:53:41.078 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:53:41.078 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T03:53:41.078 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:53:41.079 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.079 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.079 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:53:41.079 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:53:41.079 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:53:41.080 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:53:41.080 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:53:41.080 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:53:41.080 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T03:53:41.081 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:53:41.081 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.081 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:41.081 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:53:41.081 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:53:41.082 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:53:41.082 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:53:41.082 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:53:41.082 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:53:41.083 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T03:53:41.083 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:53:41.083 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.083 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:53:41.084 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T03:53:41.084 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T03:53:41.084 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T03:53:41.084 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T03:53:41.084 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": {}, 2021-07-11T03:53:41.085 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T03:53:41.085 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T03:53:41.085 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T03:53:41.085 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T03:53:41.085 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T03:53:41.086 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T03:53:41.086 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T03:53:41.086 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T03:53:41.086 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:41.087 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:41.087 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T03:53:41.087 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T03:53:41.087 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T03:53:41.087 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T03:53:41.088 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T03:53:41.088 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:53:41.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab1c8f0700 1 -- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaf40531a0 msgr2=0x7faaf4055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:41.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab1c8f0700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaf40531a0 0x7faaf4055660 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fab08002f70 tx=0x7fab08006a80).stop 2021-07-11T03:53:41.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab1c8f0700 1 -- 172.21.2.104:0/836754279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab18067300 msgr2=0x7fab181395d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:41.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab1c8f0700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab18067300 0x7fab181395d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fab0c002f70 tx=0x7fab0c014000).stop 2021-07-11T03:53:41.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab1c8f0700 1 -- 172.21.2.104:0/836754279 shutdown_connections 2021-07-11T03:53:41.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab1c8f0700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab180b4c40 0x7fab18139ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:41.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab1c8f0700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaf40531a0 0x7faaf4055660 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:41.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab1c8f0700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab18067300 0x7fab181395d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:41.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab1c8f0700 1 --2- 172.21.2.104:0/836754279 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab18060100 0x7fab180ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:41.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.828+0000 7fab1c8f0700 1 -- 172.21.2.104:0/836754279 >> 172.21.2.104:0/836754279 conn(0x7fab1805cf30 msgr2=0x7fab180b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:41.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.832+0000 7fab1c8f0700 1 -- 172.21.2.104:0/836754279 shutdown_connections 2021-07-11T03:53:41.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:40.832+0000 7fab1c8f0700 1 -- 172.21.2.104:0/836754279 wait complete. 2021-07-11T03:53:41.093 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:53:41.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.144+0000 7f00df124700 1 Processor -- start 2021-07-11T03:53:41.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.144+0000 7f00df124700 1 -- start start 2021-07-11T03:53:41.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.144+0000 7f00df124700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f00d8067180 0x7f00d805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.144+0000 7f00df124700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00d80b1970 0x7f00d805fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.144+0000 7f00df124700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d80a7a90 con 0x7f00d80a76b0 2021-07-11T03:53:41.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.144+0000 7f00df124700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d8147620 con 0x7f00d8067180 2021-07-11T03:53:41.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.144+0000 7f00df124700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d81478c0 con 0x7f00d80b1970 2021-07-11T03:53:41.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.144+0000 7f00dd6c1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00d80b1970 0x7f00d805fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:41.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00d7fff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f00d8067180 0x7f00d805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:41.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00dd6c1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00d80b1970 0x7f00d805fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:45954/0 (socket says 172.21.2.104:45954) 2021-07-11T03:53:41.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00dd6c1700 1 -- 172.21.2.104:0/3259781963 learned_addr learned my addr 172.21.2.104:0/3259781963 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:41.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00dd6c1700 1 -- 172.21.2.104:0/3259781963 >> v2:172.21.2.105:3300/0 conn(0x7f00d8067180 msgr2=0x7f00d805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:41.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00dd6c1700 1 --2- 172.21.2.104:0/3259781963 >> v2:172.21.2.105:3300/0 conn(0x7f00d8067180 0x7f00d805f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:41.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00dd6c1700 1 -- 172.21.2.104:0/3259781963 >> v1:172.21.2.105:6789/0 conn(0x7f00d80a76b0 legacy=0x7f00d8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:41.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00dd6c1700 1 -- 172.21.2.104:0/3259781963 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d8147b60 con 0x7f00d80b1970 2021-07-11T03:53:41.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00dd6c1700 1 --2- 172.21.2.104:0/3259781963 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00d80b1970 0x7f00d805fba0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f00d00032f0 tx=0x7f00d0009290).ready entity=mon.2 client_cookie=a4a6ca80f1d9425f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:41.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00d77fe700 1 -- 172.21.2.104:0/3259781963 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00d000b8a0 con 0x7f00d80b1970 2021-07-11T03:53:41.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00d77fe700 1 -- 172.21.2.104:0/3259781963 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d000ba00 con 0x7f00d80b1970 2021-07-11T03:53:41.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00d77fe700 1 -- 172.21.2.104:0/3259781963 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00d000bb70 con 0x7f00d80b1970 2021-07-11T03:53:41.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00df124700 1 -- 172.21.2.104:0/3259781963 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00d80b1970 msgr2=0x7f00d805fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:41.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00df124700 1 --2- 172.21.2.104:0/3259781963 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00d80b1970 0x7f00d805fba0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f00d00032f0 tx=0x7f00d0009290).stop 2021-07-11T03:53:41.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00df124700 1 -- 172.21.2.104:0/3259781963 shutdown_connections 2021-07-11T03:53:41.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00df124700 1 --2- 172.21.2.104:0/3259781963 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00d80b1970 0x7f00d805fba0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:41.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00df124700 1 --2- 172.21.2.104:0/3259781963 >> v2:172.21.2.105:3300/0 conn(0x7f00d8067180 0x7f00d805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:41.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00df124700 1 -- 172.21.2.104:0/3259781963 >> 172.21.2.104:0/3259781963 conn(0x7f00d80b53e0 msgr2=0x7f00d80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:41.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00df124700 1 -- 172.21.2.104:0/3259781963 shutdown_connections 2021-07-11T03:53:41.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00df124700 1 -- 172.21.2.104:0/3259781963 wait complete. 2021-07-11T03:53:41.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00df124700 1 Processor -- start 2021-07-11T03:53:41.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.148+0000 7f00df124700 1 -- start start 2021-07-11T03:53:41.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00df124700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f00d8067180 0x7f00d8145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00df124700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00d80a76b0 0x7f00d80aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00df124700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00d80b1970 0x7f00d80af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00df124700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80ac5c0 con 0x7f00d80b1970 2021-07-11T03:53:41.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00df124700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80ac860 con 0x7f00d8067180 2021-07-11T03:53:41.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00df124700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d80acb00 con 0x7f00d80a76b0 2021-07-11T03:53:41.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00dd6c1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00d80b1970 0x7f00d80af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:41.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00dd6c1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00d80b1970 0x7f00d80af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42794/0 (socket says 172.21.2.104:42794) 2021-07-11T03:53:41.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00dd6c1700 1 -- 172.21.2.104:0/319836832 learned_addr learned my addr 172.21.2.104:0/319836832 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:41.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00dd6c1700 1 -- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00d80a76b0 msgr2=0x7f00d80aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:41.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00dd6c1700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00d80a76b0 0x7f00d80aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:41.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00dd6c1700 1 -- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f00d8067180 msgr2=0x7f00d8145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:41.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00dd6c1700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f00d8067180 0x7f00d8145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:41.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00dd6c1700 1 -- 172.21.2.104:0/319836832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0009530 con 0x7f00d80b1970 2021-07-11T03:53:41.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00dd6c1700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00d80b1970 0x7f00d80af400 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f00d00032f0 tx=0x7f00d0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:41.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00d5ffb700 1 -- 172.21.2.104:0/319836832 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00d0011cc0 con 0x7f00d80b1970 2021-07-11T03:53:41.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00df124700 1 -- 172.21.2.104:0/319836832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d80aac70 con 0x7f00d80b1970 2021-07-11T03:53:41.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00d5ffb700 1 -- 172.21.2.104:0/319836832 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d0011e20 con 0x7f00d80b1970 2021-07-11T03:53:41.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00df124700 1 -- 172.21.2.104:0/319836832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d80aa890 con 0x7f00d80b1970 2021-07-11T03:53:41.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00d5ffb700 1 -- 172.21.2.104:0/319836832 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00d001f610 con 0x7f00d80b1970 2021-07-11T03:53:41.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00df124700 1 -- 172.21.2.104:0/319836832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d80ace80 con 0x7f00d80b1970 2021-07-11T03:53:41.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00d5ffb700 1 -- 172.21.2.104:0/319836832 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f00d000f070 con 0x7f00d80b1970 2021-07-11T03:53:41.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00d5ffb700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00bc053270 0x7f00bc055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00d5ffb700 1 -- 172.21.2.104:0/319836832 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f00d006bdc0 con 0x7f00d80b1970 2021-07-11T03:53:41.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00dcec0700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00bc053270 0x7f00bc055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:41.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.152+0000 7f00dcec0700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00bc053270 0x7f00bc055730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f00d806a0e0 tx=0x7f00c0007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:41.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.156+0000 7f00d5ffb700 1 -- 172.21.2.104:0/319836832 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f00d0043100 con 0x7f00d80b1970 2021-07-11T03:53:41.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.260+0000 7f00df124700 1 -- 172.21.2.104:0/319836832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00d8057350 con 0x7f00d80b1970 2021-07-11T03:53:41.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.260+0000 7f00d5ffb700 1 -- 172.21.2.104:0/319836832 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2483 (secure 0 0 0) 0x7f00d0015030 con 0x7f00d80b1970 2021-07-11T03:53:41.266 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:41.266 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3943971217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3943971217},{"type":"v1","addr":"172.21.2.105:6835","nonce":3943971217}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/668148871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":668148871},{"type":"v1","addr":"172.21.2.104:6833","nonce":668148871}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":6}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:22.350923+0000","modified":"2021-07-11T03:53:23.355891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4224},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4224":{"gid":4224,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/1236541958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":1236541958},{"type":"v1","addr":"172.21.2.104:6835","nonce":1236541958}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-07-11T03:53:41.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 -- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00bc053270 msgr2=0x7f00bc055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:41.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00bc053270 0x7f00bc055730 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f00d806a0e0 tx=0x7f00c0007810).stop 2021-07-11T03:53:41.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 -- 172.21.2.104:0/319836832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00d80b1970 msgr2=0x7f00d80af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:41.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00d80b1970 0x7f00d80af400 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f00d00032f0 tx=0x7f00d0016000).stop 2021-07-11T03:53:41.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 -- 172.21.2.104:0/319836832 shutdown_connections 2021-07-11T03:53:41.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00bc053270 0x7f00bc055730 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:41.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00d80b1970 0x7f00d80af400 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:41.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00d80a76b0 0x7f00d80aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:41.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 --2- 172.21.2.104:0/319836832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f00d8067180 0x7f00d8145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:41.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 -- 172.21.2.104:0/319836832 >> 172.21.2.104:0/319836832 conn(0x7f00d80b53e0 msgr2=0x7f00d8148a00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:41.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 -- 172.21.2.104:0/319836832 shutdown_connections 2021-07-11T03:53:41.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.264+0000 7f00df124700 1 -- 172.21.2.104:0/319836832 wait complete. 2021-07-11T03:53:41.270 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 6 2021-07-11T03:53:41.273 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T03:53:41.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd650294700 1 Processor -- start 2021-07-11T03:53:41.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd650294700 1 -- start start 2021-07-11T03:53:41.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd650294700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd648067180 0x7fd64805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd650294700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd6480b1970 0x7fd64805fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd650294700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6480a7a90 con 0x7fd6480a76b0 2021-07-11T03:53:41.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd650294700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd648147620 con 0x7fd6480b1970 2021-07-11T03:53:41.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd650294700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6481478c0 con 0x7fd648067180 2021-07-11T03:53:41.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd64e831700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd6480b1970 0x7fd64805fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:41.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd64e831700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd6480b1970 0x7fd64805fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42804/0 (socket says 172.21.2.104:42804) 2021-07-11T03:53:41.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd64e831700 1 -- 172.21.2.104:0/2271251703 learned_addr learned my addr 172.21.2.104:0/2271251703 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:41.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd64e831700 1 -- 172.21.2.104:0/2271251703 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd648067180 msgr2=0x7fd64805f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:41.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd64e831700 1 --2- 172.21.2.104:0/2271251703 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd648067180 0x7fd64805f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:41.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd64e831700 1 -- 172.21.2.104:0/2271251703 >> v1:172.21.2.105:6789/0 conn(0x7fd6480a76b0 legacy=0x7fd648066a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:41.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.320+0000 7fd64e831700 1 -- 172.21.2.104:0/2271251703 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd648147b60 con 0x7fd6480b1970 2021-07-11T03:53:41.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64e831700 1 --2- 172.21.2.104:0/2271251703 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6480b1970 0x7fd64805fba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fd648148420 tx=0x7fd644006eb0).ready entity=mon.0 client_cookie=b33ceee1fddd785f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:41.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d02e700 1 -- 172.21.2.104:0/2271251703 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd64400d570 con 0x7fd6480b1970 2021-07-11T03:53:41.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d02e700 1 -- 172.21.2.104:0/2271251703 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd644000e90 con 0x7fd6480b1970 2021-07-11T03:53:41.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d02e700 1 -- 172.21.2.104:0/2271251703 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd64400d8c0 con 0x7fd6480b1970 2021-07-11T03:53:41.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- 172.21.2.104:0/2271251703 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6480b1970 msgr2=0x7fd64805fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:41.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 --2- 172.21.2.104:0/2271251703 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6480b1970 0x7fd64805fba0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fd648148420 tx=0x7fd644006eb0).stop 2021-07-11T03:53:41.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- 172.21.2.104:0/2271251703 shutdown_connections 2021-07-11T03:53:41.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 --2- 172.21.2.104:0/2271251703 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6480b1970 0x7fd64805fba0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:41.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 --2- 172.21.2.104:0/2271251703 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd648067180 0x7fd64805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:41.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- 172.21.2.104:0/2271251703 >> 172.21.2.104:0/2271251703 conn(0x7fd6480b53e0 msgr2=0x7fd6480b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:41.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- 172.21.2.104:0/2271251703 shutdown_connections 2021-07-11T03:53:41.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- 172.21.2.104:0/2271251703 wait complete. 2021-07-11T03:53:41.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 Processor -- start 2021-07-11T03:53:41.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- start start 2021-07-11T03:53:41.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd648067180 0x7fd6481574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6480a76b0 0x7fd6480aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6480b1970 0x7fd6480acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480ad4a0 con 0x7fd6480a76b0 2021-07-11T03:53:41.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480ad940 con 0x7fd648067180 2021-07-11T03:53:41.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6480adbe0 con 0x7fd6480b1970 2021-07-11T03:53:41.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d82f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6480a76b0 0x7fd6480aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:41.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d82f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6480a76b0 0x7fd6480aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42806/0 (socket says 172.21.2.104:42806) 2021-07-11T03:53:41.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d82f700 1 -- 172.21.2.104:0/2034359118 learned_addr learned my addr 172.21.2.104:0/2034359118 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:41.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d82f700 1 -- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6480b1970 msgr2=0x7fd6480acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:41.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d82f700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6480b1970 0x7fd6480acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:41.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d82f700 1 -- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd648067180 msgr2=0x7fd6481574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:41.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d82f700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd648067180 0x7fd6481574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:41.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d82f700 1 -- 172.21.2.104:0/2034359118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64400b1e0 con 0x7fd6480a76b0 2021-07-11T03:53:41.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd64d82f700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6480a76b0 0x7fd6480aee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd640003330 tx=0x7fd6400091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:41.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd6377fe700 1 -- 172.21.2.104:0/2034359118 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd640001710 con 0x7fd6480a76b0 2021-07-11T03:53:41.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd6377fe700 1 -- 172.21.2.104:0/2034359118 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd640001870 con 0x7fd6480a76b0 2021-07-11T03:53:41.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- 172.21.2.104:0/2034359118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6480ade80 con 0x7fd6480a76b0 2021-07-11T03:53:41.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd6377fe700 1 -- 172.21.2.104:0/2034359118 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd640001bd0 con 0x7fd6480a76b0 2021-07-11T03:53:41.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- 172.21.2.104:0/2034359118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6480b3460 con 0x7fd6480a76b0 2021-07-11T03:53:41.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd650294700 1 -- 172.21.2.104:0/2034359118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6480ae010 con 0x7fd6480a76b0 2021-07-11T03:53:41.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd6377fe700 1 -- 172.21.2.104:0/2034359118 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd640018030 con 0x7fd6480a76b0 2021-07-11T03:53:41.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd6377fe700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 0x7fd6280557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:41.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.324+0000 7fd6377fe700 1 -- 172.21.2.104:0/2034359118 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd64006b2d0 con 0x7fd6480a76b0 2021-07-11T03:53:41.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.328+0000 7fd64e030700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 0x7fd6280557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:41.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.328+0000 7fd64e030700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 0x7fd6280557c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fd638002fb0 tx=0x7fd6380069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:41.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.328+0000 7fd6377fe700 1 -- 172.21.2.104:0/2034359118 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd640041cf0 con 0x7fd6480a76b0 2021-07-11T03:53:41.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:41.436+0000 7fd650294700 1 -- 172.21.2.104:0/2034359118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd648057350 con 0x7fd6480a76b0 2021-07-11T03:53:41.749 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:53:41.753 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:53:41.756 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:53:41.759 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:53:42.347 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T03:53:42.345+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T03:53:42.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.353+0000 7fd6377fe700 1 -- 172.21.2.104:0/2034359118 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd6400460f0 con 0x7fd6480a76b0 2021-07-11T03:53:42.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 -- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 msgr2=0x7fd6280557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 0x7fd6280557c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fd638002fb0 tx=0x7fd6380069d0).stop 2021-07-11T03:53:42.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 -- 172.21.2.104:0/2034359118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6480a76b0 msgr2=0x7fd6480aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6480a76b0 0x7fd6480aee40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7fd640003330 tx=0x7fd6400091d0).stop 2021-07-11T03:53:42.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 -- 172.21.2.104:0/2034359118 shutdown_connections 2021-07-11T03:53:42.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6480b1970 0x7fd6480acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:42.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 0x7fd6280557c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:42.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6480a76b0 0x7fd6480aee40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:42.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 --2- 172.21.2.104:0/2034359118 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd648067180 0x7fd6481574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:42.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 -- 172.21.2.104:0/2034359118 >> 172.21.2.104:0/2034359118 conn(0x7fd6480b53e0 msgr2=0x7fd648144810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:42.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 -- 172.21.2.104:0/2034359118 shutdown_connections 2021-07-11T03:53:42.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.357+0000 7fd650294700 1 -- 172.21.2.104:0/2034359118 wait complete. 2021-07-11T03:53:42.362 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T03:53:42.367 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:53:42.370 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T03:53:42.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 Processor -- start 2021-07-11T03:53:42.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- start start 2021-07-11T03:53:42.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0ca413fff0 0x7f0ca4140480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:42.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca4067500 con 0x7f0ca40b96e0 2021-07-11T03:53:42.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ca40af4d0 con 0x7f0ca4067120 2021-07-11T03:53:42.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ca4143080 con 0x7f0ca413fff0 2021-07-11T03:53:42.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca1d9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f0ca40b96e0 0x7f0ca40aec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39594/0 (socket says 172.21.2.104:39594) 2021-07-11T03:53:42.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca1d9b700 1 -- 172.21.2.104:0/21451652 learned_addr learned my addr 172.21.2.104:0/21451652 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:42.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca2d9d700 1 --2- 172.21.2.104:0/21451652 >> v2:172.21.2.104:3300/0 conn(0x7f0ca413fff0 0x7f0ca4140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:42.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca2d9d700 1 -- 172.21.2.104:0/21451652 >> v1:172.21.2.105:6789/0 conn(0x7f0ca4067120 legacy=0x7f0ca41439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:42.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca2d9d700 1 -- 172.21.2.104:0/21451652 >> v1:172.21.2.104:6789/0 conn(0x7f0ca40b96e0 legacy=0x7f0ca40aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca2d9d700 1 -- 172.21.2.104:0/21451652 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca4143270 con 0x7f0ca413fff0 2021-07-11T03:53:42.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca2d9d700 1 --2- 172.21.2.104:0/21451652 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ca413fff0 0x7f0ca4140480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f0ca4060320 tx=0x7f0c9c0091d0).ready entity=mon.0 client_cookie=bf0bfcf45beb0d55 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:42.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca159a700 1 -- 172.21.2.104:0/21451652 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c9c009ea0 con 0x7f0ca413fff0 2021-07-11T03:53:42.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca159a700 1 -- 172.21.2.104:0/21451652 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c9c00b860 con 0x7f0ca413fff0 2021-07-11T03:53:42.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca159a700 1 -- 172.21.2.104:0/21451652 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c9c00b9d0 con 0x7f0ca413fff0 2021-07-11T03:53:42.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- 172.21.2.104:0/21451652 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ca413fff0 msgr2=0x7f0ca4140480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 --2- 172.21.2.104:0/21451652 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ca413fff0 0x7f0ca4140480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f0ca4060320 tx=0x7f0c9c0091d0).stop 2021-07-11T03:53:42.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- 172.21.2.104:0/21451652 shutdown_connections 2021-07-11T03:53:42.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 --2- 172.21.2.104:0/21451652 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ca413fff0 0x7f0ca4140480 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:42.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- 172.21.2.104:0/21451652 >> 172.21.2.104:0/21451652 conn(0x7f0ca405f9f0 msgr2=0x7f0ca40616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:42.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- 172.21.2.104:0/21451652 shutdown_connections 2021-07-11T03:53:42.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- 172.21.2.104:0/21451652 wait complete. 2021-07-11T03:53:42.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 Processor -- start 2021-07-11T03:53:42.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- start start 2021-07-11T03:53:42.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ca4067120 0x7f0ca40b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:42.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ca40b96e0 0x7f0ca4148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:42.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ca413fff0 0x7f0ca4144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:42.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40b8620 con 0x7f0ca4067120 2021-07-11T03:53:42.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40b8850 con 0x7f0ca40b96e0 2021-07-11T03:53:42.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca40b8af0 con 0x7f0ca413fff0 2021-07-11T03:53:42.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ca4067120 0x7f0ca40b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:42.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ca4067120 0x7f0ca40b73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42830/0 (socket says 172.21.2.104:42830) 2021-07-11T03:53:42.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca259c700 1 -- 172.21.2.104:0/1809472348 learned_addr learned my addr 172.21.2.104:0/1809472348 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:42.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca259c700 1 -- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ca413fff0 msgr2=0x7f0ca4144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca259c700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ca413fff0 0x7f0ca4144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:42.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca259c700 1 -- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ca40b96e0 msgr2=0x7f0ca4148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca259c700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ca40b96e0 0x7f0ca4148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:42.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca259c700 1 -- 172.21.2.104:0/1809472348 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9c009470 con 0x7f0ca4067120 2021-07-11T03:53:42.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca259c700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ca4067120 0x7f0ca40b73d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f0c8c002f90 tx=0x7f0c8c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:42.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0c9b7fe700 1 -- 172.21.2.104:0/1809472348 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c8c013460 con 0x7f0ca4067120 2021-07-11T03:53:42.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0c9b7fe700 1 -- 172.21.2.104:0/1809472348 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8c0135c0 con 0x7f0ca4067120 2021-07-11T03:53:42.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- 172.21.2.104:0/1809472348 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca4063c80 con 0x7f0ca4067120 2021-07-11T03:53:42.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0c9b7fe700 1 -- 172.21.2.104:0/1809472348 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c8c013920 con 0x7f0ca4067120 2021-07-11T03:53:42.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.417+0000 7f0ca8b95700 1 -- 172.21.2.104:0/1809472348 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca40b9360 con 0x7f0ca4067120 2021-07-11T03:53:42.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.421+0000 7f0ca8b95700 1 -- 172.21.2.104:0/1809472348 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca40b8cf0 con 0x7f0ca4067120 2021-07-11T03:53:42.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.421+0000 7f0c9b7fe700 1 -- 172.21.2.104:0/1809472348 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0c8c01a060 con 0x7f0ca4067120 2021-07-11T03:53:42.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.421+0000 7f0c9b7fe700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c88053210 0x7f0c880556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:42.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.421+0000 7f0c9b7fe700 1 -- 172.21.2.104:0/1809472348 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f0c8c06ae80 con 0x7f0ca4067120 2021-07-11T03:53:42.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.421+0000 7f0ca1d9b700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c88053210 0x7f0c880556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:42.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.421+0000 7f0ca1d9b700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c88053210 0x7f0c880556d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f0ca4060280 tx=0x7f0c94007820).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:42.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.421+0000 7f0c9b7fe700 1 -- 172.21.2.104:0/1809472348 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0c8c042060 con 0x7f0ca4067120 2021-07-11T03:53:42.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.529+0000 7f0ca8b95700 1 -- 172.21.2.104:0/1809472348 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ca4057350 con 0x7f0ca4067120 2021-07-11T03:53:42.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0c9b7fe700 1 -- 172.21.2.104:0/1809472348 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+2123 (secure 0 0 0) 0x7f0c8c018070 con 0x7f0ca4067120 2021-07-11T03:53:42.534 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:42.535 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4217,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3943971217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3943971217},{"type":"v1","addr":"172.21.2.105:6835","nonce":3943971217}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":3},{"gid":4221,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/668148871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":668148871},{"type":"v1","addr":"172.21.2.104:6833","nonce":668148871}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":6}],"filesystems":[{"mdsmap":{"epoch":7,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:22.350923+0000","modified":"2021-07-11T03:53:42.346425+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":24,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2021-07-11T03:53:42.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 -- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c88053210 msgr2=0x7f0c880556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c88053210 0x7f0c880556d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f0ca4060280 tx=0x7f0c94007820).stop 2021-07-11T03:53:42.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 -- 172.21.2.104:0/1809472348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ca4067120 msgr2=0x7f0ca40b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ca4067120 0x7f0ca40b73d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f0c8c002f90 tx=0x7f0c8c006df0).stop 2021-07-11T03:53:42.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 -- 172.21.2.104:0/1809472348 shutdown_connections 2021-07-11T03:53:42.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ca413fff0 0x7f0ca4144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:42.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ca40b96e0 0x7f0ca4148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:42.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c88053210 0x7f0c880556d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:42.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 --2- 172.21.2.104:0/1809472348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ca4067120 0x7f0ca40b73d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:42.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 -- 172.21.2.104:0/1809472348 >> 172.21.2.104:0/1809472348 conn(0x7f0ca405f9f0 msgr2=0x7f0ca40b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:42.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 -- 172.21.2.104:0/1809472348 shutdown_connections 2021-07-11T03:53:42.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.533+0000 7f0ca8b95700 1 -- 172.21.2.104:0/1809472348 wait complete. 2021-07-11T03:53:42.538 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 7 2021-07-11T03:53:42.542 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T03:53:42.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 Processor -- start 2021-07-11T03:53:42.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- start start 2021-07-11T03:53:42.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff8100b59f0 0x7ff8100b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:42.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff81013ff70 0x7ff8100b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:42.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8100aeb30 con 0x7ff8100b29a0 2021-07-11T03:53:42.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8100ae890 con 0x7ff8100b59f0 2021-07-11T03:53:42.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81013f830 con 0x7ff81013ff70 2021-07-11T03:53:42.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80f7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff8100b29a0 0x7ff8100ae030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39606/0 (socket says 172.21.2.104:39606) 2021-07-11T03:53:42.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80f7fe700 1 -- 172.21.2.104:0/3447258368 learned_addr learned my addr 172.21.2.104:0/3447258368 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:42.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80effd700 1 --2- 172.21.2.104:0/3447258368 >> v2:172.21.2.104:3300/0 conn(0x7ff8100b59f0 0x7ff8100b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:42.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80effd700 1 -- 172.21.2.104:0/3447258368 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff81013ff70 msgr2=0x7ff8100b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:42.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80effd700 1 --2- 172.21.2.104:0/3447258368 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff81013ff70 0x7ff8100b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:42.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80effd700 1 -- 172.21.2.104:0/3447258368 >> v1:172.21.2.104:6789/0 conn(0x7ff8100b29a0 legacy=0x7ff8100ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80effd700 1 -- 172.21.2.104:0/3447258368 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81013fad0 con 0x7ff8100b59f0 2021-07-11T03:53:42.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80effd700 1 --2- 172.21.2.104:0/3447258368 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b59f0 0x7ff8100b3cf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7ff800002f70 tx=0x7ff800007f00).ready entity=mon.0 client_cookie=5e773f01d3b7a413 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:42.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80e7fc700 1 -- 172.21.2.104:0/3447258368 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff80000a500 con 0x7ff8100b59f0 2021-07-11T03:53:42.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80e7fc700 1 -- 172.21.2.104:0/3447258368 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff800000e90 con 0x7ff8100b59f0 2021-07-11T03:53:42.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80e7fc700 1 -- 172.21.2.104:0/3447258368 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff80000a850 con 0x7ff8100b59f0 2021-07-11T03:53:42.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- 172.21.2.104:0/3447258368 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b59f0 msgr2=0x7ff8100b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 --2- 172.21.2.104:0/3447258368 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b59f0 0x7ff8100b3cf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7ff800002f70 tx=0x7ff800007f00).stop 2021-07-11T03:53:42.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- 172.21.2.104:0/3447258368 shutdown_connections 2021-07-11T03:53:42.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 --2- 172.21.2.104:0/3447258368 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b59f0 0x7ff8100b3cf0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:42.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 --2- 172.21.2.104:0/3447258368 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff81013ff70 0x7ff8100b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:42.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- 172.21.2.104:0/3447258368 >> 172.21.2.104:0/3447258368 conn(0x7ff81006a110 msgr2=0x7ff81005cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:42.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- 172.21.2.104:0/3447258368 shutdown_connections 2021-07-11T03:53:42.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- 172.21.2.104:0/3447258368 wait complete. 2021-07-11T03:53:42.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 Processor -- start 2021-07-11T03:53:42.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- start start 2021-07-11T03:53:42.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b29a0 0x7ff8101435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:42.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff8100b59f0 0x7ff81015a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:42.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff81013ff70 0x7ff81015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:42.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff810060240 con 0x7ff8100b29a0 2021-07-11T03:53:42.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8100604e0 con 0x7ff81013ff70 2021-07-11T03:53:42.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff810060780 con 0x7ff8100b59f0 2021-07-11T03:53:42.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b29a0 0x7ff8101435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:42.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b29a0 0x7ff8101435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42844/0 (socket says 172.21.2.104:42844) 2021-07-11T03:53:42.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80f7fe700 1 -- 172.21.2.104:0/1235551399 learned_addr learned my addr 172.21.2.104:0/1235551399 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:42.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80f7fe700 1 -- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff8100b59f0 msgr2=0x7ff81015a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80f7fe700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff8100b59f0 0x7ff81015a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:42.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80f7fe700 1 -- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff81013ff70 msgr2=0x7ff81015a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:42.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80f7fe700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff81013ff70 0x7ff81015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:42.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80f7fe700 1 -- 172.21.2.104:0/1235551399 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80000c200 con 0x7ff8100b29a0 2021-07-11T03:53:42.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80f7fe700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b29a0 0x7ff8101435c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7ff81005d550 tx=0x7ff7f8007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:42.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80cff9700 1 -- 172.21.2.104:0/1235551399 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7f800ce30 con 0x7ff8100b29a0 2021-07-11T03:53:42.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80cff9700 1 -- 172.21.2.104:0/1235551399 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f80072e0 con 0x7ff8100b29a0 2021-07-11T03:53:42.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- 172.21.2.104:0/1235551399 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff810060ef0 con 0x7ff8100b29a0 2021-07-11T03:53:42.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80cff9700 1 -- 172.21.2.104:0/1235551399 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7f8005800 con 0x7ff8100b29a0 2021-07-11T03:53:42.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- 172.21.2.104:0/1235551399 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff810139c90 con 0x7ff8100b29a0 2021-07-11T03:53:42.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff8160f8700 1 -- 172.21.2.104:0/1235551399 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff810060940 con 0x7ff8100b29a0 2021-07-11T03:53:42.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80cff9700 1 -- 172.21.2.104:0/1235551399 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff7f8010070 con 0x7ff8100b29a0 2021-07-11T03:53:42.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80cff9700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7f4053210 0x7ff7f40556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:42.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80cff9700 1 -- 172.21.2.104:0/1235551399 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6155+0+0 (secure 0 0 0) 0x7ff7f806abc0 con 0x7ff8100b29a0 2021-07-11T03:53:42.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.637+0000 7ff80effd700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7f4053210 0x7ff7f40556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:42.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.641+0000 7ff80effd700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7f4053210 0x7ff7f40556d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7ff800002f70 tx=0x7ff800016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:42.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.641+0000 7ff80cff9700 1 -- 172.21.2.104:0/1235551399 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff7f8041eb0 con 0x7ff8100b29a0 2021-07-11T03:53:42.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:42.749+0000 7ff8160f8700 1 -- 172.21.2.104:0/1235551399 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff810057350 con 0x7ff8100b29a0 2021-07-11T03:53:43.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff80cff9700 1 -- 172.21.2.104:0/1235551399 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff7f80454d0 con 0x7ff8100b29a0 2021-07-11T03:53:43.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 -- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7f4053210 msgr2=0x7ff7f40556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:43.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7f4053210 0x7ff7f40556d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7ff800002f70 tx=0x7ff800016000).stop 2021-07-11T03:53:43.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 -- 172.21.2.104:0/1235551399 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b29a0 msgr2=0x7ff8101435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:43.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b29a0 0x7ff8101435c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7ff81005d550 tx=0x7ff7f8007c40).stop 2021-07-11T03:53:43.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 -- 172.21.2.104:0/1235551399 shutdown_connections 2021-07-11T03:53:43.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff81013ff70 0x7ff81015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:43.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff8100b59f0 0x7ff81015a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:43.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7f4053210 0x7ff7f40556d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:43.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 --2- 172.21.2.104:0/1235551399 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff8100b29a0 0x7ff8101435c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:43.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 -- 172.21.2.104:0/1235551399 >> 172.21.2.104:0/1235551399 conn(0x7ff81006a110 msgr2=0x7ff8100b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:43.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 -- 172.21.2.104:0/1235551399 shutdown_connections 2021-07-11T03:53:43.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.361+0000 7ff8160f8700 1 -- 172.21.2.104:0/1235551399 wait complete. 2021-07-11T03:53:43.380 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T03:53:43.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3133379700 1 Processor -- start 2021-07-11T03:53:43.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3133379700 1 -- start start 2021-07-11T03:53:43.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3133379700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f312c1483a0 0x7f312c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:43.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3133379700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312c0b0350 con 0x7f312c0b1db0 2021-07-11T03:53:43.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3133379700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312c0b05f0 con 0x7f312c0a70c0 2021-07-11T03:53:43.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3133379700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312c0a90c0 con 0x7f312c1483a0 2021-07-11T03:53:43.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3131115700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f312c0b1db0 0x7f312c0b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39620/0 (socket says 172.21.2.104:39620) 2021-07-11T03:53:43.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3131115700 1 -- 172.21.2.104:0/3941913007 learned_addr learned my addr 172.21.2.104:0/3941913007 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:43.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3130914700 1 --2- 172.21.2.104:0/3941913007 >> v2:172.21.2.105:3300/0 conn(0x7f312c1483a0 0x7f312c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:43.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3100109400 0 0) 0x7f312c0b0350 con 0x7f312c0b1db0 2021-07-11T03:53:43.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3114003260 con 0x7f312c0b1db0 2021-07-11T03:53:43.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2414444112 0 0) 0x7f3114003260 con 0x7f312c0b1db0 2021-07-11T03:53:43.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 >> v2:172.21.2.105:3300/0 conn(0x7f312c1483a0 msgr2=0x7f312c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:43.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 --2- 172.21.2.104:0/3941913007 >> v2:172.21.2.105:3300/0 conn(0x7f312c1483a0 0x7f312c0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:43.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 >> v1:172.21.2.105:6789/0 conn(0x7f312c0a70c0 legacy=0x7f312c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:43.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c0b0890 con 0x7f312c0b1db0 2021-07-11T03:53:43.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f3128001660 con 0x7f312c0b1db0 2021-07-11T03:53:43.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 >> v1:172.21.2.104:6789/0 conn(0x7f312c0b1db0 legacy=0x7f312c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:43.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 --2- 172.21.2.104:0/3941913007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 0x7f3114003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:43.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 --2- 172.21.2.104:0/3941913007 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3114004500 0x7f3114004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:43.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 --2- 172.21.2.104:0/3941913007 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3114004f90 0x7f311400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:43.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311400bdd0 con 0x7f3114003780 2021-07-11T03:53:43.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311400bf10 con 0x7f3114004f90 2021-07-11T03:53:43.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311400c0d0 con 0x7f3114004500 2021-07-11T03:53:43.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3130914700 1 --2- 172.21.2.104:0/3941913007 >> v2:172.21.2.105:3300/0 conn(0x7f312c1483a0 0x7f312c0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:53:43.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3131115700 1 --2- 172.21.2.104:0/3941913007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 0x7f3114003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:43.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3131115700 1 -- 172.21.2.104:0/3941913007 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3114004500 msgr2=0x7f3114004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:43.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3131115700 1 --2- 172.21.2.104:0/3941913007 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3114004500 0x7f3114004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:43.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3131115700 1 -- 172.21.2.104:0/3941913007 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3114004f90 msgr2=0x7f311400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:43.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3131115700 1 --2- 172.21.2.104:0/3941913007 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3114004f90 0x7f311400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:43.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3131115700 1 -- 172.21.2.104:0/3941913007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c0b0890 con 0x7f3114003780 2021-07-11T03:53:43.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3131916700 1 -- 172.21.2.104:0/3941913007 reap_dead start 2021-07-11T03:53:43.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3131115700 1 --2- 172.21.2.104:0/3941913007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 0x7f3114003ba0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f3128001040 tx=0x7f3128001e00).ready entity=mon.0 client_cookie=99349d9435daf31f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:43.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31280071a0 con 0x7f3114003780 2021-07-11T03:53:43.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3128007300 con 0x7f3114003780 2021-07-11T03:53:43.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f311bfff700 1 -- 172.21.2.104:0/3941913007 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f312800c980 con 0x7f3114003780 2021-07-11T03:53:43.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3133379700 1 -- 172.21.2.104:0/3941913007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 msgr2=0x7f3114003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:43.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.429+0000 7f3133379700 1 --2- 172.21.2.104:0/3941913007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 0x7f3114003ba0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f3128001040 tx=0x7f3128001e00).stop 2021-07-11T03:53:43.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 -- 172.21.2.104:0/3941913007 shutdown_connections 2021-07-11T03:53:43.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 --2- 172.21.2.104:0/3941913007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 0x7f3114003ba0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:43.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 -- 172.21.2.104:0/3941913007 >> 172.21.2.104:0/3941913007 conn(0x7f312c05f6b0 msgr2=0x7f312c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:43.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 -- 172.21.2.104:0/3941913007 shutdown_connections 2021-07-11T03:53:43.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 -- 172.21.2.104:0/3941913007 wait complete. 2021-07-11T03:53:43.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 Processor -- start 2021-07-11T03:53:43.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 -- start start 2021-07-11T03:53:43.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 0x7f312c0a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:43.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f312c0a9350 0x7f312c159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:43.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f312c0ab230 0x7f312c0ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:43.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c148cd0 con 0x7f3114003780 2021-07-11T03:53:43.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c15a4b0 con 0x7f312c0ab230 2021-07-11T03:53:43.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c15ab80 con 0x7f312c0a9350 2021-07-11T03:53:43.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3131115700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 0x7f312c0a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:43.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3131115700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 0x7f312c0a9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42864/0 (socket says 172.21.2.104:42864) 2021-07-11T03:53:43.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3131115700 1 -- 172.21.2.104:0/3070450680 learned_addr learned my addr 172.21.2.104:0/3070450680 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:43.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3131115700 1 -- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f312c0a9350 msgr2=0x7f312c159fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:43.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3131115700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f312c0a9350 0x7f312c159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:43.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3131115700 1 -- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f312c0ab230 msgr2=0x7f312c0ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:43.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3131115700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f312c0ab230 0x7f312c0ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:43.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3131115700 1 -- 172.21.2.104:0/3070450680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c14b760 con 0x7f3114003780 2021-07-11T03:53:43.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3131115700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 0x7f312c0a9a60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f3128001040 tx=0x7f3128015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:43.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f311a7fc700 1 -- 172.21.2.104:0/3070450680 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f312800cd60 con 0x7f3114003780 2021-07-11T03:53:43.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f311a7fc700 1 -- 172.21.2.104:0/3070450680 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3128018070 con 0x7f3114003780 2021-07-11T03:53:43.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 -- 172.21.2.104:0/3070450680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312c14ba00 con 0x7f3114003780 2021-07-11T03:53:43.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f311a7fc700 1 -- 172.21.2.104:0/3070450680 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3128004710 con 0x7f3114003780 2021-07-11T03:53:43.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 -- 172.21.2.104:0/3070450680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c0a97b0 con 0x7f3114003780 2021-07-11T03:53:43.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f3133379700 1 -- 172.21.2.104:0/3070450680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312c057350 con 0x7f3114003780 2021-07-11T03:53:43.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.433+0000 7f311a7fc700 1 -- 172.21.2.104:0/3070450680 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3128010070 con 0x7f3114003780 2021-07-11T03:53:43.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.437+0000 7f311a7fc700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f310c0531f0 0x7f310c0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:43.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.437+0000 7f311a7fc700 1 -- 172.21.2.104:0/3070450680 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f312806bd50 con 0x7f3114003780 2021-07-11T03:53:43.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.437+0000 7f3130914700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f310c0531f0 0x7f310c0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:43.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.437+0000 7f3130914700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f310c0531f0 0x7f310c0556b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f311c00a000 tx=0x7f311c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:43.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.437+0000 7f311a7fc700 1 -- 172.21.2.104:0/3070450680 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3128042f40 con 0x7f3114003780 2021-07-11T03:53:43.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:43.521+0000 7f3133379700 1 -- 172.21.2.104:0/3070450680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f312c044650 con 0x7f3114003780 2021-07-11T03:53:44.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f311a7fc700 1 -- 172.21.2.104:0/3070450680 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 185+0+0 (secure 0 0 0) 0x7f3128046560 con 0x7f3114003780 2021-07-11T03:53:44.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 -- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f310c0531f0 msgr2=0x7f310c0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:44.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f310c0531f0 0x7f310c0556b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f311c00a000 tx=0x7f311c008040).stop 2021-07-11T03:53:44.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 -- 172.21.2.104:0/3070450680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 msgr2=0x7f312c0a9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:44.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 0x7f312c0a9a60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f3128001040 tx=0x7f3128015000).stop 2021-07-11T03:53:44.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 -- 172.21.2.104:0/3070450680 shutdown_connections 2021-07-11T03:53:44.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f312c0ab230 0x7f312c0ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:44.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f312c0a9350 0x7f312c159fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:44.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f310c0531f0 0x7f310c0556b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:44.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 --2- 172.21.2.104:0/3070450680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3114003780 0x7f312c0a9a60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:44.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 -- 172.21.2.104:0/3070450680 >> 172.21.2.104:0/3070450680 conn(0x7f312c05f6b0 msgr2=0x7f312c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:44.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 -- 172.21.2.104:0/3070450680 shutdown_connections 2021-07-11T03:53:44.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.369+0000 7f3133379700 1 -- 172.21.2.104:0/3070450680 wait complete. 2021-07-11T03:53:44.380 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T03:53:44.388 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T03:53:44.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 Processor -- start 2021-07-11T03:53:44.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 -- start start 2021-07-11T03:53:44.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f362c0aa120 0x7f362c0ae440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:44.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0b2af0 con 0x7f362c0b04a0 2021-07-11T03:53:44.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0b2770 con 0x7f362c0b0880 2021-07-11T03:53:44.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c0b2d20 con 0x7f362c0aa120 2021-07-11T03:53:44.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3631b20700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f362c0aa120 0x7f362c0ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:44.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3632321700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f362c0b04a0 0x7f362c0add30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39640/0 (socket says 172.21.2.104:39640) 2021-07-11T03:53:44.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3632321700 1 -- 172.21.2.104:0/2747242435 learned_addr learned my addr 172.21.2.104:0/2747242435 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:44.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3631b20700 1 -- 172.21.2.104:0/2747242435 >> v1:172.21.2.105:6789/0 conn(0x7f362c0b0880 legacy=0x7f362c1405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:44.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3631b20700 1 -- 172.21.2.104:0/2747242435 >> v1:172.21.2.104:6789/0 conn(0x7f362c0b04a0 legacy=0x7f362c0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:44.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f363131f700 1 -- 172.21.2.104:0/2747242435 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964915256 0 0) 0x7f362c0b2af0 con 0x7f362c0b04a0 2021-07-11T03:53:44.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3631b20700 1 -- 172.21.2.104:0/2747242435 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0a9b70 con 0x7f362c0aa120 2021-07-11T03:53:44.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3631b20700 1 --2- 172.21.2.104:0/2747242435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0aa120 0x7f362c0ae440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f362c0b7770 tx=0x7f3628006e80).ready entity=mon.0 client_cookie=9f3989e678353eff server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:44.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f363131f700 1 -- 172.21.2.104:0/2747242435 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f362800ba30 con 0x7f362c0aa120 2021-07-11T03:53:44.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f363131f700 1 -- 172.21.2.104:0/2747242435 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3628000e90 con 0x7f362c0aa120 2021-07-11T03:53:44.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f363131f700 1 -- 172.21.2.104:0/2747242435 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f362800bd80 con 0x7f362c0aa120 2021-07-11T03:53:44.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 -- 172.21.2.104:0/2747242435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0aa120 msgr2=0x7f362c0ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:44.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 --2- 172.21.2.104:0/2747242435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0aa120 0x7f362c0ae440 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f362c0b7770 tx=0x7f3628006e80).stop 2021-07-11T03:53:44.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 -- 172.21.2.104:0/2747242435 shutdown_connections 2021-07-11T03:53:44.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 --2- 172.21.2.104:0/2747242435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0aa120 0x7f362c0ae440 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:44.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 -- 172.21.2.104:0/2747242435 >> 172.21.2.104:0/2747242435 conn(0x7f362c0607c0 msgr2=0x7f362c060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:44.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 -- 172.21.2.104:0/2747242435 shutdown_connections 2021-07-11T03:53:44.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 -- 172.21.2.104:0/2747242435 wait complete. 2021-07-11T03:53:44.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 Processor -- start 2021-07-11T03:53:44.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.441+0000 7f3634585700 1 -- start start 2021-07-11T03:53:44.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3634585700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f362c0aa120 0x7f362c1400b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:44.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3634585700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f362c0b04a0 0x7f362c142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:44.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3634585700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0b0880 0x7f362c1415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:44.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3634585700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c143c90 con 0x7f362c0b0880 2021-07-11T03:53:44.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3634585700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c143f30 con 0x7f362c0aa120 2021-07-11T03:53:44.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3634585700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1441d0 con 0x7f362c0b04a0 2021-07-11T03:53:44.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3632b22700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0b0880 0x7f362c1415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:44.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3632b22700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0b0880 0x7f362c1415b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42882/0 (socket says 172.21.2.104:42882) 2021-07-11T03:53:44.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3632b22700 1 -- 172.21.2.104:0/1299060777 learned_addr learned my addr 172.21.2.104:0/1299060777 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:44.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3632b22700 1 -- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f362c0b04a0 msgr2=0x7f362c142fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:44.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3632b22700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f362c0b04a0 0x7f362c142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:44.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3632b22700 1 -- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f362c0aa120 msgr2=0x7f362c1400b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:44.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3632b22700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f362c0aa120 0x7f362c1400b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:44.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3632b22700 1 -- 172.21.2.104:0/1299060777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362800d410 con 0x7f362c0b0880 2021-07-11T03:53:44.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3632b22700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0b0880 0x7f362c1415b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f3624003290 tx=0x7f36240090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:44.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f361f7fe700 1 -- 172.21.2.104:0/1299060777 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f362400f510 con 0x7f362c0b0880 2021-07-11T03:53:44.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f361f7fe700 1 -- 172.21.2.104:0/1299060777 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362400f670 con 0x7f362c0b0880 2021-07-11T03:53:44.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f361f7fe700 1 -- 172.21.2.104:0/1299060777 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f362400f7e0 con 0x7f362c0b0880 2021-07-11T03:53:44.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3634585700 1 -- 172.21.2.104:0/1299060777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c1447a0 con 0x7f362c0b0880 2021-07-11T03:53:44.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3634585700 1 -- 172.21.2.104:0/1299060777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c065930 con 0x7f362c0b0880 2021-07-11T03:53:44.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3634585700 1 -- 172.21.2.104:0/1299060777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c144de0 con 0x7f362c0b0880 2021-07-11T03:53:44.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f361f7fe700 1 -- 172.21.2.104:0/1299060777 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f362400f070 con 0x7f362c0b0880 2021-07-11T03:53:44.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f361f7fe700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f360c053160 0x7f360c055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:44.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f361f7fe700 1 -- 172.21.2.104:0/1299060777 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f362406af20 con 0x7f362c0b0880 2021-07-11T03:53:44.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3632321700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f360c053160 0x7f360c055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:44.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.445+0000 7f3632321700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f360c053160 0x7f360c055620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f3620000bc0 tx=0x7f36200076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:44.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.449+0000 7f361f7fe700 1 -- 172.21.2.104:0/1299060777 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f36240423c0 con 0x7f362c0b0880 2021-07-11T03:53:44.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:44.529+0000 7f3634585700 1 -- 172.21.2.104:0/1299060777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f362c144930 con 0x7f362c0b0880 2021-07-11T03:53:45.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f361f7fe700 1 -- 172.21.2.104:0/1299060777 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 173+0+0 (secure 0 0 0) 0x7f36240459e0 con 0x7f362c0b0880 2021-07-11T03:53:45.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 -- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f360c053160 msgr2=0x7f360c055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:45.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f360c053160 0x7f360c055620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f3620000bc0 tx=0x7f36200076c0).stop 2021-07-11T03:53:45.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 -- 172.21.2.104:0/1299060777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0b0880 msgr2=0x7f362c1415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:45.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0b0880 0x7f362c1415b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f3624003290 tx=0x7f36240090f0).stop 2021-07-11T03:53:45.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 -- 172.21.2.104:0/1299060777 shutdown_connections 2021-07-11T03:53:45.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f360c053160 0x7f360c055620 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:45.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0b0880 0x7f362c1415b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:45.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f362c0b04a0 0x7f362c142fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:45.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 --2- 172.21.2.104:0/1299060777 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f362c0aa120 0x7f362c1400b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:45.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 -- 172.21.2.104:0/1299060777 >> 172.21.2.104:0/1299060777 conn(0x7f362c0607c0 msgr2=0x7f362c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:45.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 -- 172.21.2.104:0/1299060777 shutdown_connections 2021-07-11T03:53:45.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.377+0000 7f3634585700 1 -- 172.21.2.104:0/1299060777 wait complete. 2021-07-11T03:53:45.387 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T03:53:45.390 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T03:53:45.391 INFO:tasks.ceph.mds.a:Stopping old one... 2021-07-11T03:53:45.391 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T03:53:45.391 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:53:45.392 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T03:53:45.392 INFO:tasks.ceph.mds.c:Stopping old one... 2021-07-11T03:53:45.392 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T03:53:45.393 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:53:45.393 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T03:53:45.393 INFO:tasks.ceph.mds.b:Stopping old one... 2021-07-11T03:53:45.394 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T03:53:45.394 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:53:45.397 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T03:53:45.398 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T03:53:45.399 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T03:53:45.400 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T03:53:45.401 INFO:tasks.ceph.mds.a:Started 2021-07-11T03:53:45.402 DEBUG:teuthology.parallel:result is None 2021-07-11T03:53:45.402 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T03:53:45.403 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T03:53:45.404 INFO:tasks.ceph.mds.c:Started 2021-07-11T03:53:45.404 DEBUG:teuthology.parallel:result is None 2021-07-11T03:53:45.407 INFO:tasks.ceph.mds.b:Started 2021-07-11T03:53:45.407 DEBUG:teuthology.parallel:result is None 2021-07-11T03:53:45.408 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T03:53:45.437 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T03:53:45.437 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T03:53:45.442 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T03:53:45.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 Processor -- start 2021-07-11T03:53:45.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- start start 2021-07-11T03:53:45.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f506c0b7810 0x7f506c1445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:45.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f506c144a90 0x7f506c064f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:45.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506c0b31d0 con 0x7f506c0b1170 2021-07-11T03:53:45.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506c144ef0 con 0x7f506c144a90 2021-07-11T03:53:45.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c0b7bf0 con 0x7f506c0b7810 2021-07-11T03:53:45.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5071685700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f506c144a90 0x7f506c064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:45.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5071685700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f506c144a90 0x7f506c064f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42912/0 (socket says 172.21.2.104:42912) 2021-07-11T03:53:45.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5071685700 1 -- 172.21.2.104:0/2267404925 learned_addr learned my addr 172.21.2.104:0/2267404925 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:45.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5070e84700 1 --2- 172.21.2.104:0/2267404925 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f506c0b7810 0x7f506c1445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:45.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5071685700 1 -- 172.21.2.104:0/2267404925 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f506c0b7810 msgr2=0x7f506c1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:45.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5071685700 1 --2- 172.21.2.104:0/2267404925 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f506c0b7810 0x7f506c1445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:45.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5071685700 1 -- 172.21.2.104:0/2267404925 >> v1:172.21.2.105:6789/0 conn(0x7f506c0b1170 legacy=0x7f506c064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:45.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5071685700 1 -- 172.21.2.104:0/2267404925 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c0675c0 con 0x7f506c144a90 2021-07-11T03:53:45.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5071685700 1 --2- 172.21.2.104:0/2267404925 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f506c144a90 0x7f506c064f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f50640032f0 tx=0x7f5064009290).ready entity=mon.0 client_cookie=9bed0aaa5efcc045 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:45.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f506b7fe700 1 -- 172.21.2.104:0/2267404925 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f506400b8a0 con 0x7f506c144a90 2021-07-11T03:53:45.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f506b7fe700 1 -- 172.21.2.104:0/2267404925 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506400ba00 con 0x7f506c144a90 2021-07-11T03:53:45.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f506b7fe700 1 -- 172.21.2.104:0/2267404925 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f506400bb70 con 0x7f506c144a90 2021-07-11T03:53:45.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- 172.21.2.104:0/2267404925 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f506c144a90 msgr2=0x7f506c064f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:45.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 --2- 172.21.2.104:0/2267404925 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f506c144a90 0x7f506c064f80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f50640032f0 tx=0x7f5064009290).stop 2021-07-11T03:53:45.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- 172.21.2.104:0/2267404925 shutdown_connections 2021-07-11T03:53:45.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 --2- 172.21.2.104:0/2267404925 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f506c144a90 0x7f506c064f80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:45.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 --2- 172.21.2.104:0/2267404925 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f506c0b7810 0x7f506c1445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:45.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- 172.21.2.104:0/2267404925 >> 172.21.2.104:0/2267404925 conn(0x7f506c05f6b0 msgr2=0x7f506c05fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:45.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- 172.21.2.104:0/2267404925 shutdown_connections 2021-07-11T03:53:45.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- 172.21.2.104:0/2267404925 wait complete. 2021-07-11T03:53:45.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 Processor -- start 2021-07-11T03:53:45.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- start start 2021-07-11T03:53:45.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f506c0b1170 0x7f506c143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:45.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f506c0b7810 0x7f506c061770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:45.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f506c144a90 0x7f506c061c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:45.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c061070 con 0x7f506c0b1170 2021-07-11T03:53:45.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c140ea0 con 0x7f506c144a90 2021-07-11T03:53:45.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f50730e8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c140c00 con 0x7f506c0b7810 2021-07-11T03:53:45.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5070e84700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f506c0b1170 0x7f506c143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:45.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5070e84700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f506c0b1170 0x7f506c143a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42916/0 (socket says 172.21.2.104:42916) 2021-07-11T03:53:45.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5070e84700 1 -- 172.21.2.104:0/3509877742 learned_addr learned my addr 172.21.2.104:0/3509877742 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:45.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5070e84700 1 -- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f506c0b7810 msgr2=0x7f506c061770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:45.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.457+0000 7f5070e84700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f506c0b7810 0x7f506c061770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:45.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f5070e84700 1 -- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f506c144a90 msgr2=0x7f506c061c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:45.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f5070e84700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f506c144a90 0x7f506c061c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:45.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f5070e84700 1 -- 172.21.2.104:0/3509877742 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5064009530 con 0x7f506c0b1170 2021-07-11T03:53:45.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f5070e84700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f506c0b1170 0x7f506c143a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f50540030b0 tx=0x7f5054008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:45.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f5069ffb700 1 -- 172.21.2.104:0/3509877742 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5054011780 con 0x7f506c0b1170 2021-07-11T03:53:45.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f5069ffb700 1 -- 172.21.2.104:0/3509877742 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50540118e0 con 0x7f506c0b1170 2021-07-11T03:53:45.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f50730e8700 1 -- 172.21.2.104:0/3509877742 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506c0612d0 con 0x7f506c0b1170 2021-07-11T03:53:45.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f5069ffb700 1 -- 172.21.2.104:0/3509877742 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5054011c40 con 0x7f506c0b1170 2021-07-11T03:53:45.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f50730e8700 1 -- 172.21.2.104:0/3509877742 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c144180 con 0x7f506c0b1170 2021-07-11T03:53:45.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f50730e8700 1 -- 172.21.2.104:0/3509877742 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506c061460 con 0x7f506c0b1170 2021-07-11T03:53:45.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f5069ffb700 1 -- 172.21.2.104:0/3509877742 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f505401c460 con 0x7f506c0b1170 2021-07-11T03:53:45.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f5069ffb700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50500531f0 0x7f50500556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:45.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f5069ffb700 1 -- 172.21.2.104:0/3509877742 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f505406b400 con 0x7f506c0b1170 2021-07-11T03:53:45.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f506bfff700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50500531f0 0x7f50500556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:45.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f506bfff700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50500531f0 0x7f50500556b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f506c060740 tx=0x7f505c007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:45.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.461+0000 7f5069ffb700 1 -- 172.21.2.104:0/3509877742 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f50540429c0 con 0x7f506c0b1170 2021-07-11T03:53:45.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:45.557+0000 7f50730e8700 1 -- 172.21.2.104:0/3509877742 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f506c057350 con 0x7f506c0b1170 2021-07-11T03:53:46.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.453+0000 7f5069ffb700 1 -- 172.21.2.104:0/3509877742 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v29) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5054045fe0 con 0x7f506c0b1170 2021-07-11T03:53:46.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 -- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50500531f0 msgr2=0x7f50500556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50500531f0 0x7f50500556b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f506c060740 tx=0x7f505c007810).stop 2021-07-11T03:53:46.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 -- 172.21.2.104:0/3509877742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f506c0b1170 msgr2=0x7f506c143a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f506c0b1170 0x7f506c143a30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f50540030b0 tx=0x7f5054008040).stop 2021-07-11T03:53:46.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 -- 172.21.2.104:0/3509877742 shutdown_connections 2021-07-11T03:53:46.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f506c144a90 0x7f506c061c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f506c0b7810 0x7f506c061770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50500531f0 0x7f50500556b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 --2- 172.21.2.104:0/3509877742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f506c0b1170 0x7f506c143a30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 -- 172.21.2.104:0/3509877742 >> 172.21.2.104:0/3509877742 conn(0x7f506c05f6b0 msgr2=0x7f506c0b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:46.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 -- 172.21.2.104:0/3509877742 shutdown_connections 2021-07-11T03:53:46.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.461+0000 7f50730e8700 1 -- 172.21.2.104:0/3509877742 wait complete. 2021-07-11T03:53:46.470 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T03:53:46.474 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T03:53:46.475 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T03:53:46.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 Processor -- start 2021-07-11T03:53:46.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 -- start start 2021-07-11T03:53:46.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f222c0b04a0 0x7f222c0add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f222c0b0880 0x7f222c1405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f222c0b2af0 con 0x7f222c0aa120 2021-07-11T03:53:46.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f222c0b2770 con 0x7f222c0b04a0 2021-07-11T03:53:46.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c0b2d20 con 0x7f222c0b0880 2021-07-11T03:53:46.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2230dfc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f222c0aa120 0x7f222c0ae220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39698/0 (socket says 172.21.2.104:39698) 2021-07-11T03:53:46.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2230dfc700 1 -- 172.21.2.104:0/3117137148 learned_addr learned my addr 172.21.2.104:0/3117137148 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:46.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543514428 0 0) 0x7f222c0b2af0 con 0x7f222c0aa120 2021-07-11T03:53:46.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2214003260 con 0x7f222c0aa120 2021-07-11T03:53:46.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2231dfe700 1 --2- 172.21.2.104:0/3117137148 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f222c0b0880 0x7f222c1405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f22315fd700 1 --2- 172.21.2.104:0/3117137148 >> v2:172.21.2.105:3300/0 conn(0x7f222c0b04a0 0x7f222c0add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1970515912 0 0) 0x7f2214003260 con 0x7f222c0aa120 2021-07-11T03:53:46.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f222c0b0880 msgr2=0x7f222c1405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:46.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 --2- 172.21.2.104:0/3117137148 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f222c0b0880 0x7f222c1405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 >> v2:172.21.2.105:3300/0 conn(0x7f222c0b04a0 msgr2=0x7f222c0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:46.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 --2- 172.21.2.104:0/3117137148 >> v2:172.21.2.105:3300/0 conn(0x7f222c0b04a0 0x7f222c0add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c0a9b70 con 0x7f222c0aa120 2021-07-11T03:53:46.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f221c001660 con 0x7f222c0aa120 2021-07-11T03:53:46.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 >> v1:172.21.2.104:6789/0 conn(0x7f222c0aa120 legacy=0x7f222c0ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 --2- 172.21.2.104:0/3117137148 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f222c0b0880 0x7f22140037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 --2- 172.21.2.104:0/3117137148 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2214003ee0 0x7f2214004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 --2- 172.21.2.104:0/3117137148 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2214004bf0 0x7f221400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c0b2770 con 0x7f2214004bf0 2021-07-11T03:53:46.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c0b2d20 con 0x7f2214003ee0 2021-07-11T03:53:46.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22140053a0 con 0x7f222c0b0880 2021-07-11T03:53:46.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2231dfe700 1 --2- 172.21.2.104:0/3117137148 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2214004bf0 0x7f221400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2231dfe700 1 -- 172.21.2.104:0/3117137148 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f222c0b0880 msgr2=0x7f22140037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:46.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2231dfe700 1 --2- 172.21.2.104:0/3117137148 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f222c0b0880 0x7f22140037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2231dfe700 1 -- 172.21.2.104:0/3117137148 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2214003ee0 msgr2=0x7f2214004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:46.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2231dfe700 1 --2- 172.21.2.104:0/3117137148 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2214003ee0 0x7f2214004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2231dfe700 1 -- 172.21.2.104:0/3117137148 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2214003260 con 0x7f2214004bf0 2021-07-11T03:53:46.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2231dfe700 1 --2- 172.21.2.104:0/3117137148 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2214004bf0 0x7f221400b750 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f2224007000 tx=0x7f2224008040).ready entity=mon.0 client_cookie=9614a441720764ec server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:46.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f222400ecf0 con 0x7f2214004bf0 2021-07-11T03:53:46.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222400ee50 con 0x7f2214004bf0 2021-07-11T03:53:46.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f221bfff700 1 -- 172.21.2.104:0/3117137148 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2224010630 con 0x7f2214004bf0 2021-07-11T03:53:46.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 -- 172.21.2.104:0/3117137148 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2214004bf0 msgr2=0x7f221400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 --2- 172.21.2.104:0/3117137148 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2214004bf0 0x7f221400b750 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f2224007000 tx=0x7f2224008040).stop 2021-07-11T03:53:46.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2231dfe700 1 -- 172.21.2.104:0/3117137148 reap_dead start 2021-07-11T03:53:46.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 -- 172.21.2.104:0/3117137148 shutdown_connections 2021-07-11T03:53:46.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 -- 172.21.2.104:0/3117137148 >> 172.21.2.104:0/3117137148 conn(0x7f222c0607c0 msgr2=0x7f222c060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:46.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 -- 172.21.2.104:0/3117137148 shutdown_connections 2021-07-11T03:53:46.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.565+0000 7f2233861700 1 -- 172.21.2.104:0/3117137148 wait complete. 2021-07-11T03:53:46.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2233861700 1 Processor -- start 2021-07-11T03:53:46.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2233861700 1 -- start start 2021-07-11T03:53:46.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2233861700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f222c063c10 0x7f222c156e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2233861700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f222c144c60 0x7f222c066a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2233861700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f222c1414d0 0x7f222c066f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2233861700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c063ff0 con 0x7f222c063c10 2021-07-11T03:53:46.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2233861700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f222c15a7a0 con 0x7f222c1414d0 2021-07-11T03:53:46.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2233861700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f222c15aa40 con 0x7f222c144c60 2021-07-11T03:53:46.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f22315fd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f222c063c10 0x7f222c156e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f22315fd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f222c063c10 0x7f222c156e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42944/0 (socket says 172.21.2.104:42944) 2021-07-11T03:53:46.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f22315fd700 1 -- 172.21.2.104:0/3809273091 learned_addr learned my addr 172.21.2.104:0/3809273091 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:46.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f22315fd700 1 -- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f222c144c60 msgr2=0x7f222c066a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f22315fd700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f222c144c60 0x7f222c066a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f22315fd700 1 -- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f222c1414d0 msgr2=0x7f222c066f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f22315fd700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f222c1414d0 0x7f222c066f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f22315fd700 1 -- 172.21.2.104:0/3809273091 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c156720 con 0x7f222c063c10 2021-07-11T03:53:46.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f22315fd700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f222c063c10 0x7f222c156e60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f2228007000 tx=0x7f2228008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:46.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f221a7fc700 1 -- 172.21.2.104:0/3809273091 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f222800f4e0 con 0x7f222c063c10 2021-07-11T03:53:46.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f221a7fc700 1 -- 172.21.2.104:0/3809273091 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2228000e90 con 0x7f222c063c10 2021-07-11T03:53:46.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f221a7fc700 1 -- 172.21.2.104:0/3809273091 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f222800f830 con 0x7f222c063c10 2021-07-11T03:53:46.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2233861700 1 -- 172.21.2.104:0/3809273091 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f222c1569c0 con 0x7f222c063c10 2021-07-11T03:53:46.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2233861700 1 -- 172.21.2.104:0/3809273091 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c067400 con 0x7f222c063c10 2021-07-11T03:53:46.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2233861700 1 -- 172.21.2.104:0/3809273091 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f222c157690 con 0x7f222c063c10 2021-07-11T03:53:46.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f221a7fc700 1 -- 172.21.2.104:0/3809273091 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f222800f990 con 0x7f222c063c10 2021-07-11T03:53:46.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f221a7fc700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f220c053300 0x7f220c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f221a7fc700 1 -- 172.21.2.104:0/3809273091 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f222806ad40 con 0x7f222c063c10 2021-07-11T03:53:46.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.569+0000 7f2230dfc700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f220c053300 0x7f220c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.573+0000 7f2230dfc700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f220c053300 0x7f220c0557c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f221c001370 tx=0x7f221c001570).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:46.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.573+0000 7f221a7fc700 1 -- 172.21.2.104:0/3809273091 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f222c157690 con 0x7f222c063c10 2021-07-11T03:53:46.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 -- 172.21.2.104:0/3809273091 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f222c157690 con 0x7f222c063c10 2021-07-11T03:53:46.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f221a7fc700 1 -- 172.21.2.104:0/3809273091 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+5621 (secure 0 0 0) 0x7f222c157690 con 0x7f222c063c10 2021-07-11T03:53:46.690 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:46.690 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:53:46.691 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T03:53:46.691 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.691 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T03:53:46.691 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T03:53:46.691 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.692 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:53:46.692 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.692 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.692 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.693 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.693 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.693 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.693 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T03:53:46.693 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T03:53:46.694 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.694 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:53:46.694 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.694 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.695 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.695 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.695 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.695 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.695 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T03:53:46.696 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T03:53:46.696 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.696 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:53:46.696 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.697 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.697 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.697 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.697 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.697 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.698 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T03:53:46.698 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T03:53:46.698 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.698 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.699 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.699 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.699 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.699 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.700 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.700 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T03:53:46.700 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T03:53:46.700 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.701 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.701 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.701 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.701 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.702 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.702 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.702 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T03:53:46.702 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T03:53:46.703 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.703 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.703 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.703 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.704 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.704 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.704 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.704 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T03:53:46.705 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T03:53:46.705 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.705 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.705 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.705 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.706 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.706 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.706 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.706 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T03:53:46.707 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T03:53:46.707 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.707 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.707 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.707 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.708 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.709 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.709 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.709 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T03:53:46.709 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T03:53:46.710 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.710 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.710 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.710 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.711 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.711 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.711 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.711 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T03:53:46.712 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T03:53:46.712 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.712 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.712 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.713 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.713 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.713 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.713 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.714 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T03:53:46.714 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T03:53:46.714 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.714 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.714 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.715 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.715 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.715 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.715 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.716 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T03:53:46.716 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T03:53:46.716 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.716 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:53:46.717 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow r", 2021-07-11T03:53:46.717 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow rw", 2021-07-11T03:53:46.717 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rwx" 2021-07-11T03:53:46.717 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.718 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.718 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.718 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T03:53:46.718 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T03:53:46.719 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.719 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:53:46.719 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow r", 2021-07-11T03:53:46.719 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow rw", 2021-07-11T03:53:46.719 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rwx" 2021-07-11T03:53:46.720 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.720 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.720 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.720 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T03:53:46.721 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T03:53:46.721 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.721 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:53:46.721 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:53:46.721 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:53:46.722 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.722 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.722 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.722 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.723 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.bootstrap-mds", 2021-07-11T03:53:46.723 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAka+pg8NtPChAAAHlCtml1449X0vdjc9vWRg==", 2021-07-11T03:53:46.723 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.723 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile bootstrap-mds" 2021-07-11T03:53:46.723 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.724 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.724 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.724 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.bootstrap-mgr", 2021-07-11T03:53:46.724 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAka+pgTuRPChAAa/pF1gWltEBIlyhrL9Qn7Q==", 2021-07-11T03:53:46.725 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.725 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile bootstrap-mgr" 2021-07-11T03:53:46.725 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.725 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.725 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.726 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.bootstrap-osd", 2021-07-11T03:53:46.726 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAka+pg/OpPChAA9r6/OuuFSk6fjGoGA3fsBw==", 2021-07-11T03:53:46.726 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.726 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile bootstrap-osd" 2021-07-11T03:53:46.727 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.727 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.727 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.727 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.bootstrap-rbd", 2021-07-11T03:53:46.727 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAka+pgxvBPChAAmkBdPkl8uPNiiGognHp+3A==", 2021-07-11T03:53:46.728 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.728 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile bootstrap-rbd" 2021-07-11T03:53:46.728 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.728 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.729 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.729 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.bootstrap-rbd-mirror", 2021-07-11T03:53:46.729 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAka+pgTfdPChAAmgha6ZvRuDCdQVNvG1PsWw==", 2021-07-11T03:53:46.729 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.729 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2021-07-11T03:53:46.730 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.730 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.730 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.730 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.bootstrap-rgw", 2021-07-11T03:53:46.731 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAka+pg2P5PChAAOQRZO4vyLmofej87uBFNhQ==", 2021-07-11T03:53:46.731 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.731 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile bootstrap-rgw" 2021-07-11T03:53:46.732 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.732 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.732 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.732 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T03:53:46.733 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T03:53:46.733 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.733 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:53:46.733 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T03:53:46.734 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.734 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.734 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:46.734 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:46.734 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T03:53:46.735 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T03:53:46.735 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:53:46.735 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:53:46.735 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T03:53:46.736 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:53:46.736 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.736 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:46.736 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:53:46.737 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:53:46.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 -- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f220c053300 msgr2=0x7f220c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f220c053300 0x7f220c0557c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f221c001370 tx=0x7f221c001570).stop 2021-07-11T03:53:46.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 -- 172.21.2.104:0/3809273091 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f222c063c10 msgr2=0x7f222c156e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f222c063c10 0x7f222c156e60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f2228007000 tx=0x7f2228008040).stop 2021-07-11T03:53:46.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 -- 172.21.2.104:0/3809273091 shutdown_connections 2021-07-11T03:53:46.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f222c1414d0 0x7f222c066f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f222c144c60 0x7f222c066a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f220c053300 0x7f220c0557c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 --2- 172.21.2.104:0/3809273091 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f222c063c10 0x7f222c156e60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 -- 172.21.2.104:0/3809273091 >> 172.21.2.104:0/3809273091 conn(0x7f222c0607c0 msgr2=0x7f222c05fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:46.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 -- 172.21.2.104:0/3809273091 shutdown_connections 2021-07-11T03:53:46.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.685+0000 7f2233861700 1 -- 172.21.2.104:0/3809273091 wait complete. 2021-07-11T03:53:46.741 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2021-07-11T03:53:46.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.793+0000 7f64e46a7700 1 Processor -- start 2021-07-11T03:53:46.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.793+0000 7f64e46a7700 1 -- start start 2021-07-11T03:53:46.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.793+0000 7f64e46a7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64dc0b1db0 0x7f64dc0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.793+0000 7f64e46a7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f64dc1483a0 0x7f64dc0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.793+0000 7f64e46a7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f64dc0a70c0 0x7f64dc144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.793+0000 7f64e46a7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64dc0a90c0 con 0x7f64dc0a70c0 2021-07-11T03:53:46.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.793+0000 7f64e46a7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64dc0b0350 con 0x7f64dc1483a0 2021-07-11T03:53:46.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.793+0000 7f64e46a7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc0b05f0 con 0x7f64dc0b1db0 2021-07-11T03:53:46.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.793+0000 7f64e2c44700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f64dc0a70c0 0x7f64dc144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.793+0000 7f64e2c44700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f64dc0a70c0 0x7f64dc144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42950/0 (socket says 172.21.2.104:42950) 2021-07-11T03:53:46.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.793+0000 7f64e2c44700 1 -- 172.21.2.104:0/2392038001 learned_addr learned my addr 172.21.2.104:0/2392038001 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:46.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e2c44700 1 -- 172.21.2.104:0/2392038001 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64dc0b1db0 msgr2=0x7f64dc0b0fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:46.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e2c44700 1 --2- 172.21.2.104:0/2392038001 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64dc0b1db0 0x7f64dc0b0fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e2c44700 1 -- 172.21.2.104:0/2392038001 >> v2:172.21.2.105:3300/0 conn(0x7f64dc1483a0 msgr2=0x7f64dc0b2190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:46.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e2c44700 1 --2- 172.21.2.104:0/2392038001 >> v2:172.21.2.105:3300/0 conn(0x7f64dc1483a0 0x7f64dc0b2190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e2c44700 1 -- 172.21.2.104:0/2392038001 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64dc0b0890 con 0x7f64dc0a70c0 2021-07-11T03:53:46.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e2c44700 1 --2- 172.21.2.104:0/2392038001 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64dc0a70c0 0x7f64dc144720 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f64d40032f0 tx=0x7f64d4009290).ready entity=mon.0 client_cookie=16c32c29f4049b24 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:46.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1441700 1 -- 172.21.2.104:0/2392038001 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64d400b870 con 0x7f64dc0a70c0 2021-07-11T03:53:46.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1441700 1 -- 172.21.2.104:0/2392038001 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d400b9d0 con 0x7f64dc0a70c0 2021-07-11T03:53:46.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1441700 1 -- 172.21.2.104:0/2392038001 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64d400bb80 con 0x7f64dc0a70c0 2021-07-11T03:53:46.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 -- 172.21.2.104:0/2392038001 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64dc0a70c0 msgr2=0x7f64dc144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 --2- 172.21.2.104:0/2392038001 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64dc0a70c0 0x7f64dc144720 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f64d40032f0 tx=0x7f64d4009290).stop 2021-07-11T03:53:46.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 -- 172.21.2.104:0/2392038001 shutdown_connections 2021-07-11T03:53:46.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 --2- 172.21.2.104:0/2392038001 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64dc0a70c0 0x7f64dc144720 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 --2- 172.21.2.104:0/2392038001 >> v2:172.21.2.105:3300/0 conn(0x7f64dc1483a0 0x7f64dc0b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 --2- 172.21.2.104:0/2392038001 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64dc0b1db0 0x7f64dc0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 -- 172.21.2.104:0/2392038001 >> 172.21.2.104:0/2392038001 conn(0x7f64dc05f6b0 msgr2=0x7f64dc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:46.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 -- 172.21.2.104:0/2392038001 shutdown_connections 2021-07-11T03:53:46.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 -- 172.21.2.104:0/2392038001 wait complete. 2021-07-11T03:53:46.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 Processor -- start 2021-07-11T03:53:46.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 -- start start 2021-07-11T03:53:46.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64dc0a70c0 0x7f64dc14e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64dc0b1db0 0x7f64dc14d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64dc1483a0 0x7f64dc0a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc0a9f30 con 0x7f64dc0b1db0 2021-07-11T03:53:46.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc14e810 con 0x7f64dc1483a0 2021-07-11T03:53:46.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc14eab0 con 0x7f64dc0a70c0 2021-07-11T03:53:46.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1c42700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64dc0b1db0 0x7f64dc14d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1c42700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64dc0b1db0 0x7f64dc14d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42958/0 (socket says 172.21.2.104:42958) 2021-07-11T03:53:46.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1c42700 1 -- 172.21.2.104:0/3406315196 learned_addr learned my addr 172.21.2.104:0/3406315196 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:46.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1c42700 1 -- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64dc0a70c0 msgr2=0x7f64dc14e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1c42700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64dc0a70c0 0x7f64dc14e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1c42700 1 -- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64dc1483a0 msgr2=0x7f64dc0a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1c42700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64dc1483a0 0x7f64dc0a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1c42700 1 -- 172.21.2.104:0/3406315196 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d4009530 con 0x7f64dc0b1db0 2021-07-11T03:53:46.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e1c42700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64dc0b1db0 0x7f64dc14d8e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f64d0000ed0 tx=0x7f64d0006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:46.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64cf7fe700 1 -- 172.21.2.104:0/3406315196 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64d0009e00 con 0x7f64dc0b1db0 2021-07-11T03:53:46.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64cf7fe700 1 -- 172.21.2.104:0/3406315196 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d0012410 con 0x7f64dc0b1db0 2021-07-11T03:53:46.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 -- 172.21.2.104:0/3406315196 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64dc14d440 con 0x7f64dc0b1db0 2021-07-11T03:53:46.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64cf7fe700 1 -- 172.21.2.104:0/3406315196 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f64d0012770 con 0x7f64dc0b1db0 2021-07-11T03:53:46.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.797+0000 7f64e46a7700 1 -- 172.21.2.104:0/3406315196 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64dc0a9ad0 con 0x7f64dc0b1db0 2021-07-11T03:53:46.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.801+0000 7f64e46a7700 1 -- 172.21.2.104:0/3406315196 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64dc14d5d0 con 0x7f64dc0b1db0 2021-07-11T03:53:46.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.801+0000 7f64cf7fe700 1 -- 172.21.2.104:0/3406315196 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f64d000f030 con 0x7f64dc0b1db0 2021-07-11T03:53:46.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.801+0000 7f64cf7fe700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64bc0531f0 0x7f64bc0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.801+0000 7f64cf7fe700 1 -- 172.21.2.104:0/3406315196 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f64d006ad60 con 0x7f64dc0b1db0 2021-07-11T03:53:46.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.801+0000 7f64e2443700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64bc0531f0 0x7f64bc0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.801+0000 7f64e2443700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64bc0531f0 0x7f64bc0556b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f64d8002fb0 tx=0x7f64d80069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:46.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.801+0000 7f64cf7fe700 1 -- 172.21.2.104:0/3406315196 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f64d00423d0 con 0x7f64dc0b1db0 2021-07-11T03:53:46.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.909+0000 7f64e46a7700 1 -- 172.21.2.104:0/3406315196 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f64dc057350 con 0x7f64dc0b1db0 2021-07-11T03:53:46.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64cf7fe700 1 -- 172.21.2.104:0/3406315196 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7f64d00459f0 con 0x7f64dc0b1db0 2021-07-11T03:53:46.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 -- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64bc0531f0 msgr2=0x7f64bc0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64bc0531f0 0x7f64bc0556b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f64d8002fb0 tx=0x7f64d80069d0).stop 2021-07-11T03:53:46.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 -- 172.21.2.104:0/3406315196 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64dc0b1db0 msgr2=0x7f64dc14d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64dc0b1db0 0x7f64dc14d8e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f64d0000ed0 tx=0x7f64d0006d60).stop 2021-07-11T03:53:46.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 -- 172.21.2.104:0/3406315196 shutdown_connections 2021-07-11T03:53:46.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64dc1483a0 0x7f64dc0a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64bc0531f0 0x7f64bc0556b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64dc0b1db0 0x7f64dc14d8e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 --2- 172.21.2.104:0/3406315196 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64dc0a70c0 0x7f64dc14e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:46.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 -- 172.21.2.104:0/3406315196 >> 172.21.2.104:0/3406315196 conn(0x7f64dc05f6b0 msgr2=0x7f64dc05ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:46.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 -- 172.21.2.104:0/3406315196 shutdown_connections 2021-07-11T03:53:46.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.917+0000 7f64e46a7700 1 -- 172.21.2.104:0/3406315196 wait complete. 2021-07-11T03:53:46.926 INFO:teuthology.orchestra.run.gibba004.stderr:updated 2021-07-11T03:53:46.929 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2021-07-11T03:53:46.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.973+0000 7fb47e0c5700 1 Processor -- start 2021-07-11T03:53:46.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.973+0000 7fb47e0c5700 1 -- start start 2021-07-11T03:53:46.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.973+0000 7fb47e0c5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4780b1db0 0x7fb4780b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.973+0000 7fb47e0c5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4780b0350 con 0x7fb4780a70c0 2021-07-11T03:53:46.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.973+0000 7fb47e0c5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4780b05f0 con 0x7fb4781483a0 2021-07-11T03:53:46.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.973+0000 7fb47e0c5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780a90c0 con 0x7fb4780b1db0 2021-07-11T03:53:46.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb477fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb4780a70c0 0x7fb478144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39734/0 (socket says 172.21.2.104:39734) 2021-07-11T03:53:46.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb477fff700 1 -- 172.21.2.104:0/2555740841 learned_addr learned my addr 172.21.2.104:0/2555740841 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:46.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019467157 0 0) 0x7fb4780b0350 con 0x7fb4780a70c0 2021-07-11T03:53:46.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb458003260 con 0x7fb4780a70c0 2021-07-11T03:53:46.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4777fe700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4780b1db0 0x7fb4780b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 60304754 0 0) 0x7fb458003260 con 0x7fb4780a70c0 2021-07-11T03:53:46.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4780b1db0 msgr2=0x7fb4780b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:46.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4780b1db0 0x7fb4780b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 >> v1:172.21.2.105:6789/0 conn(0x7fb4781483a0 legacy=0x7fb4780b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780b0890 con 0x7fb4780a70c0 2021-07-11T03:53:46.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fb4700019e0 con 0x7fb4780a70c0 2021-07-11T03:53:46.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 >> v1:172.21.2.104:6789/0 conn(0x7fb4780a70c0 legacy=0x7fb478144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb458003800 0x7fb458003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4780b1db0 0x7fb458004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4580049f0 0x7fb458004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45800b800 con 0x7fb458003800 2021-07-11T03:53:46.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45800b940 con 0x7fb4580049f0 2021-07-11T03:53:46.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45800bb00 con 0x7fb4780b1db0 2021-07-11T03:53:46.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4777fe700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb458003800 0x7fb458003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb476ffd700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4780b1db0 0x7fb458004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb477fff700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4580049f0 0x7fb458004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4777fe700 1 -- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4780b1db0 msgr2=0x7fb458004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:46.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4777fe700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb4780b1db0 0x7fb458004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4777fe700 1 -- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4580049f0 msgr2=0x7fb458004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:46.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4777fe700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4580049f0 0x7fb458004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:46.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4777fe700 1 -- 172.21.2.104:0/2555740841 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780b0890 con 0x7fb458003800 2021-07-11T03:53:46.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb477fff700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4580049f0 0x7fb458004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:46.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4777fe700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb458003800 0x7fb458003c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fb46800c000 tx=0x7fb468006df0).ready entity=mon.0 client_cookie=350ddcae6defcfa9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:46.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46800e830 con 0x7fb458003800 2021-07-11T03:53:46.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46800e990 con 0x7fb458003800 2021-07-11T03:53:46.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4767fc700 1 -- 172.21.2.104:0/2555740841 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46800ecf0 con 0x7fb458003800 2021-07-11T03:53:46.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 -- 172.21.2.104:0/2555740841 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb458003800 msgr2=0x7fb458003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:46.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 --2- 172.21.2.104:0/2555740841 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb458003800 0x7fb458003c20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fb46800c000 tx=0x7fb468006df0).stop 2021-07-11T03:53:46.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb477fff700 1 -- 172.21.2.104:0/2555740841 reap_dead start 2021-07-11T03:53:46.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 -- 172.21.2.104:0/2555740841 shutdown_connections 2021-07-11T03:53:46.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 -- 172.21.2.104:0/2555740841 >> 172.21.2.104:0/2555740841 conn(0x7fb47805f6b0 msgr2=0x7fb47805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:46.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 -- 172.21.2.104:0/2555740841 shutdown_connections 2021-07-11T03:53:46.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 -- 172.21.2.104:0/2555740841 wait complete. 2021-07-11T03:53:46.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 Processor -- start 2021-07-11T03:53:46.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 -- start start 2021-07-11T03:53:46.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb478142360 0x7fb478156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4780af6e0 0x7fb478157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb478145940 0x7fb47813e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:46.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478142740 con 0x7fb478142360 2021-07-11T03:53:46.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478157600 con 0x7fb4780af6e0 2021-07-11T03:53:46.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb47e0c5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4780ad020 con 0x7fb478145940 2021-07-11T03:53:46.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4777fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb478142360 0x7fb478156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:46.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4777fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb478142360 0x7fb478156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42976/0 (socket says 172.21.2.104:42976) 2021-07-11T03:53:46.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.977+0000 7fb4777fe700 1 -- 172.21.2.104:0/69440343 learned_addr learned my addr 172.21.2.104:0/69440343 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:46.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb477fff700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb478145940 0x7fb47813e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb476ffd700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4780af6e0 0x7fb478157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb4777fe700 1 -- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb478145940 msgr2=0x7fb47813e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb4777fe700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb478145940 0x7fb47813e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb4777fe700 1 -- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4780af6e0 msgr2=0x7fb478157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb4777fe700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4780af6e0 0x7fb478157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb4777fe700 1 -- 172.21.2.104:0/69440343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780ad2c0 con 0x7fb478142360 2021-07-11T03:53:47.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb476ffd700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4780af6e0 0x7fb478157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:47.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb4777fe700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb478142360 0x7fb478156c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fb46800c000 tx=0x7fb468019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb474ff9700 1 -- 172.21.2.104:0/69440343 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46801b8c0 con 0x7fb478142360 2021-07-11T03:53:47.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb474ff9700 1 -- 172.21.2.104:0/69440343 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb468000e90 con 0x7fb478142360 2021-07-11T03:53:47.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb47e0c5700 1 -- 172.21.2.104:0/69440343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4780ad560 con 0x7fb478142360 2021-07-11T03:53:47.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb474ff9700 1 -- 172.21.2.104:0/69440343 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb46801bc10 con 0x7fb478142360 2021-07-11T03:53:47.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb47e0c5700 1 -- 172.21.2.104:0/69440343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4780af3d0 con 0x7fb478142360 2021-07-11T03:53:47.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb47e0c5700 1 -- 172.21.2.104:0/69440343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb478057350 con 0x7fb478142360 2021-07-11T03:53:47.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb474ff9700 1 -- 172.21.2.104:0/69440343 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb46802a020 con 0x7fb478142360 2021-07-11T03:53:47.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb474ff9700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb45c053160 0x7fb45c055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb474ff9700 1 -- 172.21.2.104:0/69440343 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb46806c3f0 con 0x7fb478142360 2021-07-11T03:53:47.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb476ffd700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb45c053160 0x7fb45c055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.981+0000 7fb476ffd700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb45c053160 0x7fb45c055620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fb460003000 tx=0x7fb460009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:46.985+0000 7fb474ff9700 1 -- 172.21.2.104:0/69440343 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb468043a60 con 0x7fb478142360 2021-07-11T03:53:47.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.093+0000 7fb47e0c5700 1 -- 172.21.2.104:0/69440343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fb478156980 con 0x7fb478142360 2021-07-11T03:53:47.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb474ff9700 1 -- 172.21.2.104:0/69440343 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb468047080 con 0x7fb478142360 2021-07-11T03:53:47.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 -- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb45c053160 msgr2=0x7fb45c055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb45c053160 0x7fb45c055620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fb460003000 tx=0x7fb460009040).stop 2021-07-11T03:53:47.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 -- 172.21.2.104:0/69440343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb478142360 msgr2=0x7fb478156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb478142360 0x7fb478156c20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7fb46800c000 tx=0x7fb468019000).stop 2021-07-11T03:53:47.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 -- 172.21.2.104:0/69440343 shutdown_connections 2021-07-11T03:53:47.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb478145940 0x7fb47813e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb4780af6e0 0x7fb478157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb45c053160 0x7fb45c055620 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 --2- 172.21.2.104:0/69440343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb478142360 0x7fb478156c20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 -- 172.21.2.104:0/69440343 >> 172.21.2.104:0/69440343 conn(0x7fb47805f6b0 msgr2=0x7fb4780b3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:47.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 -- 172.21.2.104:0/69440343 shutdown_connections 2021-07-11T03:53:47.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.097+0000 7fb47e0c5700 1 -- 172.21.2.104:0/69440343 wait complete. 2021-07-11T03:53:47.107 INFO:teuthology.orchestra.run.gibba004.stderr:updated 2021-07-11T03:53:47.109 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2021-07-11T03:53:47.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 Processor -- start 2021-07-11T03:53:47.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 -- start start 2021-07-11T03:53:47.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f3ccc1483a0 0x7f3ccc0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3ccc0a70c0 0x7f3ccc144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ccc0b0350 con 0x7f3ccc0b1db0 2021-07-11T03:53:47.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc0a90c0 con 0x7f3ccc0a70c0 2021-07-11T03:53:47.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ccc0b05f0 con 0x7f3ccc1483a0 2021-07-11T03:53:47.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd0c7a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3ccc0a70c0 0x7f3ccc144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd0c7a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3ccc0a70c0 0x7f3ccc144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42982/0 (socket says 172.21.2.104:42982) 2021-07-11T03:53:47.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd0c7a700 1 -- 172.21.2.104:0/553115078 learned_addr learned my addr 172.21.2.104:0/553115078 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:47.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd0c7a700 1 -- 172.21.2.104:0/553115078 >> v2:172.21.2.105:3300/0 conn(0x7f3ccc1483a0 msgr2=0x7f3ccc0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd0c7a700 1 --2- 172.21.2.104:0/553115078 >> v2:172.21.2.105:3300/0 conn(0x7f3ccc1483a0 0x7f3ccc0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:47.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd0c7a700 1 -- 172.21.2.104:0/553115078 >> v1:172.21.2.105:6789/0 conn(0x7f3ccc0b1db0 legacy=0x7f3ccc0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd0c7a700 1 -- 172.21.2.104:0/553115078 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc0b0890 con 0x7f3ccc0a70c0 2021-07-11T03:53:47.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd0c7a700 1 --2- 172.21.2.104:0/553115078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ccc0a70c0 0x7f3ccc144720 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f3ccc0ace60 tx=0x7f3cc4009290).ready entity=mon.0 client_cookie=48282b8269e1abd6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3ccaffd700 1 -- 172.21.2.104:0/553115078 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc400b820 con 0x7f3ccc0a70c0 2021-07-11T03:53:47.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3ccaffd700 1 -- 172.21.2.104:0/553115078 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc400b980 con 0x7f3ccc0a70c0 2021-07-11T03:53:47.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3ccaffd700 1 -- 172.21.2.104:0/553115078 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc400bb30 con 0x7f3ccc0a70c0 2021-07-11T03:53:47.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 -- 172.21.2.104:0/553115078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ccc0a70c0 msgr2=0x7f3ccc144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 --2- 172.21.2.104:0/553115078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ccc0a70c0 0x7f3ccc144720 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f3ccc0ace60 tx=0x7f3cc4009290).stop 2021-07-11T03:53:47.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 -- 172.21.2.104:0/553115078 shutdown_connections 2021-07-11T03:53:47.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 --2- 172.21.2.104:0/553115078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ccc0a70c0 0x7f3ccc144720 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 --2- 172.21.2.104:0/553115078 >> v2:172.21.2.105:3300/0 conn(0x7f3ccc1483a0 0x7f3ccc0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:47.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 -- 172.21.2.104:0/553115078 >> 172.21.2.104:0/553115078 conn(0x7f3ccc05f6b0 msgr2=0x7f3ccc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:47.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 -- 172.21.2.104:0/553115078 shutdown_connections 2021-07-11T03:53:47.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 -- 172.21.2.104:0/553115078 wait complete. 2021-07-11T03:53:47.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 Processor -- start 2021-07-11T03:53:47.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.157+0000 7f3cd26dd700 1 -- start start 2021-07-11T03:53:47.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd26dd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3ccc0a70c0 0x7f3ccc0ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd26dd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3ccc0b1db0 0x7f3ccc13f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd26dd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ccc1483a0 0x7f3ccc13fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd26dd700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc147520 con 0x7f3ccc1483a0 2021-07-11T03:53:47.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd26dd700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc1468a0 con 0x7f3ccc0b1db0 2021-07-11T03:53:47.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd26dd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ccc146b40 con 0x7f3ccc0a70c0 2021-07-11T03:53:47.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd0c7a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ccc1483a0 0x7f3ccc13fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd0c7a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ccc1483a0 0x7f3ccc13fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:42992/0 (socket says 172.21.2.104:42992) 2021-07-11T03:53:47.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd0c7a700 1 -- 172.21.2.104:0/406924546 learned_addr learned my addr 172.21.2.104:0/406924546 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:47.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3ccb7fe700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3ccc0b1db0 0x7f3ccc13f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3ccbfff700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3ccc0a70c0 0x7f3ccc0ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd0c7a700 1 -- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3ccc0a70c0 msgr2=0x7f3ccc0ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd0c7a700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3ccc0a70c0 0x7f3ccc0ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd0c7a700 1 -- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3ccc0b1db0 msgr2=0x7f3ccc13f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd0c7a700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3ccc0b1db0 0x7f3ccc13f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd0c7a700 1 -- 172.21.2.104:0/406924546 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4009530 con 0x7f3ccc1483a0 2021-07-11T03:53:47.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3ccbfff700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3ccc0a70c0 0x7f3ccc0ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:47.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd0c7a700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ccc1483a0 0x7f3ccc13fdd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f3ccc0ace60 tx=0x7f3cc4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cc97fa700 1 -- 172.21.2.104:0/406924546 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc4011cc0 con 0x7f3ccc1483a0 2021-07-11T03:53:47.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cc97fa700 1 -- 172.21.2.104:0/406924546 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4011e20 con 0x7f3ccc1483a0 2021-07-11T03:53:47.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd26dd700 1 -- 172.21.2.104:0/406924546 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc146de0 con 0x7f3ccc1483a0 2021-07-11T03:53:47.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cc97fa700 1 -- 172.21.2.104:0/406924546 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cc4020660 con 0x7f3ccc1483a0 2021-07-11T03:53:47.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd26dd700 1 -- 172.21.2.104:0/406924546 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ccc0ad530 con 0x7f3ccc1483a0 2021-07-11T03:53:47.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cd26dd700 1 -- 172.21.2.104:0/406924546 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ccc057350 con 0x7f3ccc1483a0 2021-07-11T03:53:47.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cc97fa700 1 -- 172.21.2.104:0/406924546 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3cc400f070 con 0x7f3ccc1483a0 2021-07-11T03:53:47.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cc97fa700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb00531f0 0x7f3cb00556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3cc97fa700 1 -- 172.21.2.104:0/406924546 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3cc406bd50 con 0x7f3ccc1483a0 2021-07-11T03:53:47.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3ccbfff700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb00531f0 0x7f3cb00556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.161+0000 7f3ccbfff700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb00531f0 0x7f3cb00556b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f3cbc008000 tx=0x7f3cbc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.165+0000 7f3cc97fa700 1 -- 172.21.2.104:0/406924546 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3cc4043470 con 0x7f3ccc1483a0 2021-07-11T03:53:47.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.273+0000 7f3cd26dd700 1 -- 172.21.2.104:0/406924546 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f3ccc0afc80 con 0x7f3ccc1483a0 2021-07-11T03:53:47.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.277+0000 7f3cc97fa700 1 -- 172.21.2.104:0/406924546 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3cc4015030 con 0x7f3ccc1483a0 2021-07-11T03:53:47.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 -- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb00531f0 msgr2=0x7f3cb00556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb00531f0 0x7f3cb00556b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f3cbc008000 tx=0x7f3cbc009040).stop 2021-07-11T03:53:47.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 -- 172.21.2.104:0/406924546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ccc1483a0 msgr2=0x7f3ccc13fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ccc1483a0 0x7f3ccc13fdd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f3ccc0ace60 tx=0x7f3cc4016000).stop 2021-07-11T03:53:47.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 -- 172.21.2.104:0/406924546 shutdown_connections 2021-07-11T03:53:47.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb00531f0 0x7f3cb00556b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ccc1483a0 0x7f3ccc13fdd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3ccc0b1db0 0x7f3ccc13f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 --2- 172.21.2.104:0/406924546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3ccc0a70c0 0x7f3ccc0ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 -- 172.21.2.104:0/406924546 >> 172.21.2.104:0/406924546 conn(0x7f3ccc05f6b0 msgr2=0x7f3ccc147730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:47.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 -- 172.21.2.104:0/406924546 shutdown_connections 2021-07-11T03:53:47.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.281+0000 7f3cd26dd700 1 -- 172.21.2.104:0/406924546 wait complete. 2021-07-11T03:53:47.289 INFO:teuthology.orchestra.run.gibba004.stderr:updated 2021-07-11T03:53:47.293 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2021-07-11T03:53:47.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.337+0000 7f9fed92b700 1 Processor -- start 2021-07-11T03:53:47.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.337+0000 7f9fed92b700 1 -- start start 2021-07-11T03:53:47.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.337+0000 7f9fed92b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe80b0b30 0x7f9fe8149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.337+0000 7f9fed92b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe80b0430 con 0x7f9fe80ad0d0 2021-07-11T03:53:47.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.337+0000 7f9fed92b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe80ad4b0 con 0x7f9fe80afc20 2021-07-11T03:53:47.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.337+0000 7f9fed92b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80b5750 con 0x7f9fe80b0b30 2021-07-11T03:53:47.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.337+0000 7f9fe6ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f9fe80ad0d0 0x7f9fe813f0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39764/0 (socket says 172.21.2.104:39764) 2021-07-11T03:53:47.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.337+0000 7f9fe6ffd700 1 -- 172.21.2.104:0/364764659 learned_addr learned my addr 172.21.2.104:0/364764659 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:47.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1604743716 0 0) 0x7f9fe80b0430 con 0x7f9fe80ad0d0 2021-07-11T03:53:47.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc8003260 con 0x7f9fe80ad0d0 2021-07-11T03:53:47.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe77fe700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe80b0b30 0x7f9fe8149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 661782838 0 0) 0x7f9fc8003260 con 0x7f9fe80ad0d0 2021-07-11T03:53:47.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe80b0b30 msgr2=0x7f9fe8149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe80b0b30 0x7f9fe8149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 >> v1:172.21.2.105:6789/0 conn(0x7f9fe80afc20 legacy=0x7f9fe813f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe80b0660 con 0x7f9fe80ad0d0 2021-07-11T03:53:47.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f9fd8001660 con 0x7f9fe80ad0d0 2021-07-11T03:53:47.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 >> v1:172.21.2.104:6789/0 conn(0x7f9fe80ad0d0 legacy=0x7f9fe813f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe80b0b30 0x7f9fc8003820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fc8003e50 0x7f9fc8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fc8004b10 0x7f9fc8004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80b5750 con 0x7f9fc8003e50 2021-07-11T03:53:47.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc800b990 con 0x7f9fc8004b10 2021-07-11T03:53:47.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc800bb50 con 0x7f9fe80b0b30 2021-07-11T03:53:47.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe67fc700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fc8003e50 0x7f9fc8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe6ffd700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe80b0b30 0x7f9fc8003820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe77fe700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fc8004b10 0x7f9fc8004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe67fc700 1 -- 172.21.2.104:0/364764659 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe80b0b30 msgr2=0x7f9fc8003820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe67fc700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe80b0b30 0x7f9fc8003820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe67fc700 1 -- 172.21.2.104:0/364764659 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fc8004b10 msgr2=0x7f9fc8004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe67fc700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fc8004b10 0x7f9fc8004f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe67fc700 1 -- 172.21.2.104:0/364764659 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc8003260 con 0x7f9fc8003e50 2021-07-11T03:53:47.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe67fc700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fc8003e50 0x7f9fc8004500 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f9fe8060b40 tx=0x7f9fd0007a00).ready entity=mon.0 client_cookie=d05691be059dc11d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fd0007d10 con 0x7f9fc8003e50 2021-07-11T03:53:47.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd0007030 con 0x7f9fc8003e50 2021-07-11T03:53:47.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe5ffb700 1 -- 172.21.2.104:0/364764659 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fd0007390 con 0x7f9fc8003e50 2021-07-11T03:53:47.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 -- 172.21.2.104:0/364764659 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fc8003e50 msgr2=0x7f9fc8004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 --2- 172.21.2.104:0/364764659 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fc8003e50 0x7f9fc8004500 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f9fe8060b40 tx=0x7f9fd0007a00).stop 2021-07-11T03:53:47.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe77fe700 1 -- 172.21.2.104:0/364764659 reap_dead start 2021-07-11T03:53:47.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 -- 172.21.2.104:0/364764659 shutdown_connections 2021-07-11T03:53:47.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 -- 172.21.2.104:0/364764659 >> 172.21.2.104:0/364764659 conn(0x7f9fe8062c30 msgr2=0x7f9fe8063010 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:47.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 -- 172.21.2.104:0/364764659 shutdown_connections 2021-07-11T03:53:47.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 -- 172.21.2.104:0/364764659 wait complete. 2021-07-11T03:53:47.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 Processor -- start 2021-07-11T03:53:47.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 -- start start 2021-07-11T03:53:47.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fe80b4e90 0x7f9fe81537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fe81396d0 0x7f9fe8153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe8154180 0x7f9fe812aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81434f0 con 0x7f9fe81396d0 2021-07-11T03:53:47.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80b4860 con 0x7f9fe80b4e90 2021-07-11T03:53:47.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fed92b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8154630 con 0x7f9fe8154180 2021-07-11T03:53:47.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fe81396d0 0x7f9fe8153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fe81396d0 0x7f9fe8153c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43012/0 (socket says 172.21.2.104:43012) 2021-07-11T03:53:47.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe67fc700 1 -- 172.21.2.104:0/739203152 learned_addr learned my addr 172.21.2.104:0/739203152 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:47.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.341+0000 7f9fe6ffd700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fe80b4e90 0x7f9fe81537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fe67fc700 1 -- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe8154180 msgr2=0x7f9fe812aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fe67fc700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe8154180 0x7f9fe812aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:47.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fe67fc700 1 -- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fe80b4e90 msgr2=0x7f9fe81537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fe67fc700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fe80b4e90 0x7f9fe81537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fe67fc700 1 -- 172.21.2.104:0/739203152 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe80b5270 con 0x7f9fe81396d0 2021-07-11T03:53:47.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fe6ffd700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fe80b4e90 0x7f9fe81537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:47.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fe67fc700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fe81396d0 0x7f9fe8153c90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f9fe8143630 tx=0x7f9fd0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fdffff700 1 -- 172.21.2.104:0/739203152 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fd0007210 con 0x7f9fe81396d0 2021-07-11T03:53:47.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fdffff700 1 -- 172.21.2.104:0/739203152 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd0007370 con 0x7f9fe81396d0 2021-07-11T03:53:47.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fed92b700 1 -- 172.21.2.104:0/739203152 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe80b4160 con 0x7f9fe81396d0 2021-07-11T03:53:47.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fdffff700 1 -- 172.21.2.104:0/739203152 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fd0012880 con 0x7f9fe81396d0 2021-07-11T03:53:47.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fed92b700 1 -- 172.21.2.104:0/739203152 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe80b3aa0 con 0x7f9fe81396d0 2021-07-11T03:53:47.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fed92b700 1 -- 172.21.2.104:0/739203152 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe80b4560 con 0x7f9fe81396d0 2021-07-11T03:53:47.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fdffff700 1 -- 172.21.2.104:0/739203152 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9fd0010070 con 0x7f9fe81396d0 2021-07-11T03:53:47.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fdffff700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fcc053330 0x7f9fcc0557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fdffff700 1 -- 172.21.2.104:0/739203152 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9fd006b8d0 con 0x7f9fe81396d0 2021-07-11T03:53:47.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fe6ffd700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fcc053330 0x7f9fcc0557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.345+0000 7f9fe6ffd700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fcc053330 0x7f9fcc0557f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9fd8005000 tx=0x7f9fd8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.349+0000 7f9fdffff700 1 -- 172.21.2.104:0/739203152 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9fd0042ea0 con 0x7f9fe81396d0 2021-07-11T03:53:47.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.457+0000 7f9fed92b700 1 -- 172.21.2.104:0/739203152 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f9fe8057350 con 0x7f9fe81396d0 2021-07-11T03:53:47.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fdffff700 1 -- 172.21.2.104:0/739203152 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v10) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9fd0024030 con 0x7f9fe81396d0 2021-07-11T03:53:47.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 -- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fcc053330 msgr2=0x7f9fcc0557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fcc053330 0x7f9fcc0557f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9fd8005000 tx=0x7f9fd8009040).stop 2021-07-11T03:53:47.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 -- 172.21.2.104:0/739203152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fe81396d0 msgr2=0x7f9fe8153c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fe81396d0 0x7f9fe8153c90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f9fe8143630 tx=0x7f9fd0015000).stop 2021-07-11T03:53:47.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 -- 172.21.2.104:0/739203152 shutdown_connections 2021-07-11T03:53:47.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fe8154180 0x7f9fe812aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:47.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fcc053330 0x7f9fcc0557f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fe81396d0 0x7f9fe8153c90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 --2- 172.21.2.104:0/739203152 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fe80b4e90 0x7f9fe81537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 -- 172.21.2.104:0/739203152 >> 172.21.2.104:0/739203152 conn(0x7f9fe8062c30 msgr2=0x7f9fe80ad640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:47.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 -- 172.21.2.104:0/739203152 shutdown_connections 2021-07-11T03:53:47.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.465+0000 7f9fed92b700 1 -- 172.21.2.104:0/739203152 wait complete. 2021-07-11T03:53:47.475 INFO:teuthology.orchestra.run.gibba004.stderr:updated 2021-07-11T03:53:47.477 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2021-07-11T03:53:47.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.521+0000 7f387671d700 1 Processor -- start 2021-07-11T03:53:47.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.521+0000 7f387671d700 1 -- start start 2021-07-11T03:53:47.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f387005f6b0 0x7f38700a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f38700a8b60 0x7f3870143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38700b14c0 con 0x7f3870143540 2021-07-11T03:53:47.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38700ab3e0 con 0x7f38700a8b60 2021-07-11T03:53:47.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38700b1b90 con 0x7f387005f6b0 2021-07-11T03:53:47.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f38700a8b60 0x7f3870143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f38700a8b60 0x7f3870143920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43018/0 (socket says 172.21.2.104:43018) 2021-07-11T03:53:47.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 -- 172.21.2.104:0/3742957798 learned_addr learned my addr 172.21.2.104:0/3742957798 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:47.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 -- 172.21.2.104:0/3742957798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f387005f6b0 msgr2=0x7f38700a9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 --2- 172.21.2.104:0/3742957798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f387005f6b0 0x7f38700a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:47.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 -- 172.21.2.104:0/3742957798 >> v1:172.21.2.105:6789/0 conn(0x7f3870143540 legacy=0x7f38700b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 -- 172.21.2.104:0/3742957798 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3870143fa0 con 0x7f38700a8b60 2021-07-11T03:53:47.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 --2- 172.21.2.104:0/3742957798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38700a8b60 0x7f3870143920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f38700611c0 tx=0x7f3868009290).ready entity=mon.0 client_cookie=6c03cf352c23bb5c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f386effd700 1 -- 172.21.2.104:0/3742957798 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f386800b860 con 0x7f38700a8b60 2021-07-11T03:53:47.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f386effd700 1 -- 172.21.2.104:0/3742957798 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f386800b9c0 con 0x7f38700a8b60 2021-07-11T03:53:47.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f386effd700 1 -- 172.21.2.104:0/3742957798 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f386800bb70 con 0x7f38700a8b60 2021-07-11T03:53:47.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- 172.21.2.104:0/3742957798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38700a8b60 msgr2=0x7f3870143920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 --2- 172.21.2.104:0/3742957798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38700a8b60 0x7f3870143920 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f38700611c0 tx=0x7f3868009290).stop 2021-07-11T03:53:47.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- 172.21.2.104:0/3742957798 shutdown_connections 2021-07-11T03:53:47.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 --2- 172.21.2.104:0/3742957798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38700a8b60 0x7f3870143920 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 --2- 172.21.2.104:0/3742957798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f387005f6b0 0x7f38700a9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:47.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- 172.21.2.104:0/3742957798 >> 172.21.2.104:0/3742957798 conn(0x7f38700b53e0 msgr2=0x7f38700b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:47.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- 172.21.2.104:0/3742957798 shutdown_connections 2021-07-11T03:53:47.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- 172.21.2.104:0/3742957798 wait complete. 2021-07-11T03:53:47.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 Processor -- start 2021-07-11T03:53:47.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- start start 2021-07-11T03:53:47.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f387005f6b0 0x7f38701576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f38700a8b60 0x7f387013de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3870143540 0x7f3870066610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3870132d80 con 0x7f3870143540 2021-07-11T03:53:47.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3870156af0 con 0x7f387005f6b0 2021-07-11T03:53:47.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3870156ce0 con 0x7f38700a8b60 2021-07-11T03:53:47.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3870143540 0x7f3870066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3870143540 0x7f3870066610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43022/0 (socket says 172.21.2.104:43022) 2021-07-11T03:53:47.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 -- 172.21.2.104:0/1306837333 learned_addr learned my addr 172.21.2.104:0/1306837333 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:47.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 -- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f38700a8b60 msgr2=0x7f387013de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f38700a8b60 0x7f387013de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:47.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 -- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f387005f6b0 msgr2=0x7f38701576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f387005f6b0 0x7f38701576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:47.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 -- 172.21.2.104:0/1306837333 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3868009530 con 0x7f3870143540 2021-07-11T03:53:47.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f3874cba700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3870143540 0x7f3870066610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f38700611c0 tx=0x7f3868016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f386d7fa700 1 -- 172.21.2.104:0/1306837333 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3868011cc0 con 0x7f3870143540 2021-07-11T03:53:47.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f386d7fa700 1 -- 172.21.2.104:0/1306837333 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3868011e20 con 0x7f3870143540 2021-07-11T03:53:47.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- 172.21.2.104:0/1306837333 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3870156f80 con 0x7f3870143540 2021-07-11T03:53:47.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f386d7fa700 1 -- 172.21.2.104:0/1306837333 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3868020620 con 0x7f3870143540 2021-07-11T03:53:47.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.525+0000 7f387671d700 1 -- 172.21.2.104:0/1306837333 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3870133780 con 0x7f3870143540 2021-07-11T03:53:47.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.529+0000 7f387671d700 1 -- 172.21.2.104:0/1306837333 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3870057350 con 0x7f3870143540 2021-07-11T03:53:47.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.529+0000 7f386d7fa700 1 -- 172.21.2.104:0/1306837333 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f386800f070 con 0x7f3870143540 2021-07-11T03:53:47.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.529+0000 7f386d7fa700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f38540531d0 0x7f3854055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.529+0000 7f386d7fa700 1 -- 172.21.2.104:0/1306837333 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f386806c3c0 con 0x7f3870143540 2021-07-11T03:53:47.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.529+0000 7f386ffff700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f38540531d0 0x7f3854055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.529+0000 7f386ffff700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f38540531d0 0x7f3854055690 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3860002f10 tx=0x7f3860006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.529+0000 7f386d7fa700 1 -- 172.21.2.104:0/1306837333 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3868043920 con 0x7f3870143540 2021-07-11T03:53:47.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.641+0000 7f387671d700 1 -- 172.21.2.104:0/1306837333 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f38700aaf50 con 0x7f3870143540 2021-07-11T03:53:47.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.645+0000 7f386d7fa700 1 -- 172.21.2.104:0/1306837333 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3868015020 con 0x7f3870143540 2021-07-11T03:53:47.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 -- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f38540531d0 msgr2=0x7f3854055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f38540531d0 0x7f3854055690 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f3860002f10 tx=0x7f3860006930).stop 2021-07-11T03:53:47.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 -- 172.21.2.104:0/1306837333 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3870143540 msgr2=0x7f3870066610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3870143540 0x7f3870066610 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f38700611c0 tx=0x7f3868016000).stop 2021-07-11T03:53:47.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 -- 172.21.2.104:0/1306837333 shutdown_connections 2021-07-11T03:53:47.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f38540531d0 0x7f3854055690 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3870143540 0x7f3870066610 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f38700a8b60 0x7f387013de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:47.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 --2- 172.21.2.104:0/1306837333 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f387005f6b0 0x7f38701576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:47.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 -- 172.21.2.104:0/1306837333 >> 172.21.2.104:0/1306837333 conn(0x7f38700b53e0 msgr2=0x7f38700b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:47.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 -- 172.21.2.104:0/1306837333 shutdown_connections 2021-07-11T03:53:47.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.649+0000 7f387671d700 1 -- 172.21.2.104:0/1306837333 wait complete. 2021-07-11T03:53:47.657 INFO:teuthology.orchestra.run.gibba004.stderr:updated 2021-07-11T03:53:47.660 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2021-07-11T03:53:47.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e119e4700 1 Processor -- start 2021-07-11T03:53:47.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e119e4700 1 -- start start 2021-07-11T03:53:47.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e119e4700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f3e0c1483a0 0x7f3e0c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e119e4700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e0c0b0350 con 0x7f3e0c0b1db0 2021-07-11T03:53:47.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e119e4700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e0c0b05f0 con 0x7f3e0c0a70c0 2021-07-11T03:53:47.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e119e4700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0c0a90c0 con 0x7f3e0c1483a0 2021-07-11T03:53:47.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e0affd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f3e0c0b1db0 0x7f3e0c0b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39798/0 (socket says 172.21.2.104:39798) 2021-07-11T03:53:47.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e0affd700 1 -- 172.21.2.104:0/203376371 learned_addr learned my addr 172.21.2.104:0/203376371 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:47.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e0a7fc700 1 --2- 172.21.2.104:0/203376371 >> v2:172.21.2.105:3300/0 conn(0x7f3e0c1483a0 0x7f3e0c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086632352 0 0) 0x7f3e0c0b0350 con 0x7f3e0c0b1db0 2021-07-11T03:53:47.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dec003260 con 0x7f3e0c0b1db0 2021-07-11T03:53:47.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1427433473 0 0) 0x7f3dec003260 con 0x7f3e0c0b1db0 2021-07-11T03:53:47.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 >> v2:172.21.2.105:3300/0 conn(0x7f3e0c1483a0 msgr2=0x7f3e0c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e09ffb700 1 --2- 172.21.2.104:0/203376371 >> v2:172.21.2.105:3300/0 conn(0x7f3e0c1483a0 0x7f3e0c0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 >> v1:172.21.2.105:6789/0 conn(0x7f3e0c0a70c0 legacy=0x7f3e0c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c0b0890 con 0x7f3e0c0b1db0 2021-07-11T03:53:47.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f3df4001660 con 0x7f3e0c0b1db0 2021-07-11T03:53:47.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 >> v1:172.21.2.104:6789/0 conn(0x7f3e0c0b1db0 legacy=0x7f3e0c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e09ffb700 1 --2- 172.21.2.104:0/203376371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dec003780 0x7f3dec003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.705+0000 7f3e09ffb700 1 --2- 172.21.2.104:0/203376371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dec004500 0x7f3dec004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e09ffb700 1 --2- 172.21.2.104:0/203376371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3dec004f90 0x7f3dec00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec00bdd0 con 0x7f3dec003780 2021-07-11T03:53:47.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec00bf10 con 0x7f3dec004f90 2021-07-11T03:53:47.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec00c0d0 con 0x7f3dec004500 2021-07-11T03:53:47.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0affd700 1 --2- 172.21.2.104:0/203376371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dec003780 0x7f3dec003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0b7fe700 1 --2- 172.21.2.104:0/203376371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3dec004f90 0x7f3dec00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0affd700 1 -- 172.21.2.104:0/203376371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dec004500 msgr2=0x7f3dec004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0affd700 1 --2- 172.21.2.104:0/203376371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dec004500 0x7f3dec004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:47.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0affd700 1 -- 172.21.2.104:0/203376371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3dec004f90 msgr2=0x7f3dec00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0affd700 1 --2- 172.21.2.104:0/203376371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3dec004f90 0x7f3dec00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0affd700 1 -- 172.21.2.104:0/203376371 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec003260 con 0x7f3dec003780 2021-07-11T03:53:47.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0b7fe700 1 --2- 172.21.2.104:0/203376371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3dec004f90 0x7f3dec00b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:47.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0b7fe700 1 -- 172.21.2.104:0/203376371 reap_dead start 2021-07-11T03:53:47.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0affd700 1 --2- 172.21.2.104:0/203376371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dec003780 0x7f3dec003ba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f3df4001370 tx=0x7f3e0c0b0a20).ready entity=mon.0 client_cookie=a70879249e736a93 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3df40070d0 con 0x7f3dec003780 2021-07-11T03:53:47.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3df4007230 con 0x7f3dec003780 2021-07-11T03:53:47.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e09ffb700 1 -- 172.21.2.104:0/203376371 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3df400c8f0 con 0x7f3dec003780 2021-07-11T03:53:47.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- 172.21.2.104:0/203376371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dec003780 msgr2=0x7f3dec003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 --2- 172.21.2.104:0/203376371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dec003780 0x7f3dec003ba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f3df4001370 tx=0x7f3e0c0b0a20).stop 2021-07-11T03:53:47.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- 172.21.2.104:0/203376371 shutdown_connections 2021-07-11T03:53:47.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 --2- 172.21.2.104:0/203376371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dec003780 0x7f3dec003ba0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- 172.21.2.104:0/203376371 >> 172.21.2.104:0/203376371 conn(0x7f3e0c05f6b0 msgr2=0x7f3e0c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:47.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- 172.21.2.104:0/203376371 shutdown_connections 2021-07-11T03:53:47.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- 172.21.2.104:0/203376371 wait complete. 2021-07-11T03:53:47.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 Processor -- start 2021-07-11T03:53:47.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- start start 2021-07-11T03:53:47.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dec003780 0x7f3e0c146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e0c14b700 0x7f3e0c13f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3e0c13fdd0 0x7f3e0c156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0afc20 con 0x7f3e0c14b700 2021-07-11T03:53:47.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0aed70 con 0x7f3e0c13fdd0 2021-07-11T03:53:47.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c0aefa0 con 0x7f3dec003780 2021-07-11T03:53:47.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0a7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e0c14b700 0x7f3e0c13f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0a7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e0c14b700 0x7f3e0c13f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43042/0 (socket says 172.21.2.104:43042) 2021-07-11T03:53:47.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0a7fc700 1 -- 172.21.2.104:0/3298242210 learned_addr learned my addr 172.21.2.104:0/3298242210 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:47.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0affd700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dec003780 0x7f3e0c146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0b7fe700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3e0c13fdd0 0x7f3e0c156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0a7fc700 1 -- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dec003780 msgr2=0x7f3e0c146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0a7fc700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dec003780 0x7f3e0c146870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0a7fc700 1 -- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3e0c13fdd0 msgr2=0x7f3e0c156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0a7fc700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3e0c13fdd0 0x7f3e0c156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0a7fc700 1 -- 172.21.2.104:0/3298242210 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c0af240 con 0x7f3e0c14b700 2021-07-11T03:53:47.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e0a7fc700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e0c14b700 0x7f3e0c13f8e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f3dfc00a000 tx=0x7f3dfc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e03fff700 1 -- 172.21.2.104:0/3298242210 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dfc016070 con 0x7f3e0c14b700 2021-07-11T03:53:47.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e03fff700 1 -- 172.21.2.104:0/3298242210 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dfc000e90 con 0x7f3e0c14b700 2021-07-11T03:53:47.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e03fff700 1 -- 172.21.2.104:0/3298242210 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dfc011970 con 0x7f3e0c14b700 2021-07-11T03:53:47.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- 172.21.2.104:0/3298242210 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0c0af4e0 con 0x7f3e0c14b700 2021-07-11T03:53:47.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- 172.21.2.104:0/3298242210 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0c140250 con 0x7f3e0c14b700 2021-07-11T03:53:47.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e119e4700 1 -- 172.21.2.104:0/3298242210 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e0c057350 con 0x7f3e0c14b700 2021-07-11T03:53:47.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.709+0000 7f3e03fff700 1 -- 172.21.2.104:0/3298242210 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3dfc011ad0 con 0x7f3e0c14b700 2021-07-11T03:53:47.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.713+0000 7f3e03fff700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3df0053380 0x7f3df0055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.713+0000 7f3e03fff700 1 -- 172.21.2.104:0/3298242210 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3dfc06aff0 con 0x7f3e0c14b700 2021-07-11T03:53:47.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.713+0000 7f3e0affd700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3df0053380 0x7f3df0055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.713+0000 7f3e0affd700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3df0053380 0x7f3df0055840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f3df4018000 tx=0x7f3df4013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.713+0000 7f3e03fff700 1 -- 172.21.2.104:0/3298242210 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3dfc042660 con 0x7f3e0c14b700 2021-07-11T03:53:47.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.825+0000 7f3e119e4700 1 -- 172.21.2.104:0/3298242210 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f3e0c148180 con 0x7f3e0c14b700 2021-07-11T03:53:47.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.829+0000 7f3e03fff700 1 -- 172.21.2.104:0/3298242210 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v12) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3dfc045c80 con 0x7f3e0c14b700 2021-07-11T03:53:47.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.829+0000 7f3e119e4700 1 -- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3df0053380 msgr2=0x7f3df0055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.829+0000 7f3e119e4700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3df0053380 0x7f3df0055840 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f3df4018000 tx=0x7f3df4013000).stop 2021-07-11T03:53:47.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.829+0000 7f3e119e4700 1 -- 172.21.2.104:0/3298242210 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e0c14b700 msgr2=0x7f3e0c13f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.829+0000 7f3e119e4700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e0c14b700 0x7f3e0c13f8e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f3dfc00a000 tx=0x7f3dfc008040).stop 2021-07-11T03:53:47.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.829+0000 7f3e119e4700 1 -- 172.21.2.104:0/3298242210 shutdown_connections 2021-07-11T03:53:47.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.829+0000 7f3e119e4700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3e0c13fdd0 0x7f3e0c156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.829+0000 7f3e119e4700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3df0053380 0x7f3df0055840 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.829+0000 7f3e119e4700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e0c14b700 0x7f3e0c13f8e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.829+0000 7f3e119e4700 1 --2- 172.21.2.104:0/3298242210 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dec003780 0x7f3e0c146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.833+0000 7f3e119e4700 1 -- 172.21.2.104:0/3298242210 >> 172.21.2.104:0/3298242210 conn(0x7f3e0c05f6b0 msgr2=0x7f3e0c0b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:47.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.833+0000 7f3e119e4700 1 -- 172.21.2.104:0/3298242210 shutdown_connections 2021-07-11T03:53:47.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.833+0000 7f3e119e4700 1 -- 172.21.2.104:0/3298242210 wait complete. 2021-07-11T03:53:47.840 INFO:teuthology.orchestra.run.gibba004.stderr:updated 2021-07-11T03:53:47.843 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:47.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fee165700 1 Processor -- start 2021-07-11T03:53:47.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fee165700 1 -- start start 2021-07-11T03:53:47.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fee165700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f4fe8067180 0x7f4fe805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fee165700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe80a7a90 con 0x7f4fe80a76b0 2021-07-11T03:53:47.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fee165700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe8147620 con 0x7f4fe80b1970 2021-07-11T03:53:47.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fee165700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe81478c0 con 0x7f4fe8067180 2021-07-11T03:53:47.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe77fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4fe80a76b0 0x7f4fe8066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39820/0 (socket says 172.21.2.104:39820) 2021-07-11T03:53:47.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe77fe700 1 -- 172.21.2.104:0/3646249478 learned_addr learned my addr 172.21.2.104:0/3646249478 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:47.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449329441 0 0) 0x7f4fe80a7a90 con 0x7f4fe80a76b0 2021-07-11T03:53:47.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fc8003260 con 0x7f4fe80a76b0 2021-07-11T03:53:47.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe6ffd700 1 --2- 172.21.2.104:0/3646249478 >> v2:172.21.2.105:3300/0 conn(0x7f4fe8067180 0x7f4fe805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3093452966 0 0) 0x7f4fc8003260 con 0x7f4fe80a76b0 2021-07-11T03:53:47.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 >> v2:172.21.2.105:3300/0 conn(0x7f4fe8067180 msgr2=0x7f4fe805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 --2- 172.21.2.104:0/3646249478 >> v2:172.21.2.105:3300/0 conn(0x7f4fe8067180 0x7f4fe805f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 >> v1:172.21.2.105:6789/0 conn(0x7f4fe80b1970 legacy=0x7f4fe805fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe8147b60 con 0x7f4fe80a76b0 2021-07-11T03:53:47.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f4fd0001660 con 0x7f4fe80a76b0 2021-07-11T03:53:47.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 >> v1:172.21.2.104:6789/0 conn(0x7f4fe80a76b0 legacy=0x7f4fe8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fc8003820 0x7f4fc8003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 0x7f4fc8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fc8004f90 0x7f4fc800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe81478c0 con 0x7f4fc8004500 2021-07-11T03:53:47.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc800be80 con 0x7f4fc8004f90 2021-07-11T03:53:47.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc800c040 con 0x7f4fc8003820 2021-07-11T03:53:47.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe6ffd700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 0x7f4fc8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe77fe700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fc8003820 0x7f4fc8003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe7fff700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fc8004f90 0x7f4fc800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe6ffd700 1 -- 172.21.2.104:0/3646249478 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fc8003820 msgr2=0x7f4fc8003c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe6ffd700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fc8003820 0x7f4fc8003c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe6ffd700 1 -- 172.21.2.104:0/3646249478 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fc8004f90 msgr2=0x7f4fc800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:47.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe6ffd700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fc8004f90 0x7f4fc800b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe7fff700 1 -- 172.21.2.104:0/3646249478 reap_dead start 2021-07-11T03:53:47.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe6ffd700 1 -- 172.21.2.104:0/3646249478 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc8003260 con 0x7f4fc8004500 2021-07-11T03:53:47.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe77fe700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fc8003820 0x7f4fc8003c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:53:47.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.889+0000 7f4fe6ffd700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 0x7f4fc8004980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f4fd800a000 tx=0x7f4fd8008040).ready entity=mon.0 client_cookie=46f98b873689b96c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd8013670 con 0x7f4fc8004500 2021-07-11T03:53:47.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd80137d0 con 0x7f4fc8004500 2021-07-11T03:53:47.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe67fc700 1 -- 172.21.2.104:0/3646249478 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd8013b30 con 0x7f4fc8004500 2021-07-11T03:53:47.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- 172.21.2.104:0/3646249478 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 msgr2=0x7f4fc8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 0x7f4fc8004980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f4fd800a000 tx=0x7f4fd8008040).stop 2021-07-11T03:53:47.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- 172.21.2.104:0/3646249478 shutdown_connections 2021-07-11T03:53:47.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 --2- 172.21.2.104:0/3646249478 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 0x7f4fc8004980 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- 172.21.2.104:0/3646249478 >> 172.21.2.104:0/3646249478 conn(0x7f4fe80b53e0 msgr2=0x7f4fe80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:47.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- 172.21.2.104:0/3646249478 shutdown_connections 2021-07-11T03:53:47.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- 172.21.2.104:0/3646249478 wait complete. 2021-07-11T03:53:47.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 Processor -- start 2021-07-11T03:53:47.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- start start 2021-07-11T03:53:47.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 0x7f4fe80ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fe80af4b0 0x7f4fe8144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fe80ae610 0x7f4fe8144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe8156510 con 0x7f4fc8004500 2021-07-11T03:53:47.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80ad5a0 con 0x7f4fe80ae610 2021-07-11T03:53:47.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe80aa530 con 0x7f4fe80af4b0 2021-07-11T03:53:47.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 0x7f4fe80ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 0x7f4fe80ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43062/0 (socket says 172.21.2.104:43062) 2021-07-11T03:53:47.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe77fe700 1 -- 172.21.2.104:0/1308056203 learned_addr learned my addr 172.21.2.104:0/1308056203 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:47.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe6ffd700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fe80af4b0 0x7f4fe8144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe7fff700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fe80ae610 0x7f4fe8144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe77fe700 1 -- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fe80af4b0 msgr2=0x7f4fe8144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe77fe700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fe80af4b0 0x7f4fe8144600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe77fe700 1 -- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fe80ae610 msgr2=0x7f4fe8144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe77fe700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fe80ae610 0x7f4fe8144af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe77fe700 1 -- 172.21.2.104:0/1308056203 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80ac6a0 con 0x7f4fc8004500 2021-07-11T03:53:47.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe77fe700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 0x7f4fe80ae120 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f4fd000b000 tx=0x7f4fe80a8c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe4ff9700 1 -- 172.21.2.104:0/1308056203 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd0010640 con 0x7f4fc8004500 2021-07-11T03:53:47.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe4ff9700 1 -- 172.21.2.104:0/1308056203 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd00107a0 con 0x7f4fc8004500 2021-07-11T03:53:47.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- 172.21.2.104:0/1308056203 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe8144fe0 con 0x7f4fc8004500 2021-07-11T03:53:47.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fe4ff9700 1 -- 172.21.2.104:0/1308056203 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fd0010b00 con 0x7f4fc8004500 2021-07-11T03:53:47.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- 172.21.2.104:0/1308056203 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe80aa2a0 con 0x7f4fc8004500 2021-07-11T03:53:47.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.893+0000 7f4fee165700 1 -- 172.21.2.104:0/1308056203 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe80aa730 con 0x7f4fc8004500 2021-07-11T03:53:47.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.897+0000 7f4fe4ff9700 1 -- 172.21.2.104:0/1308056203 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4fd0010c60 con 0x7f4fc8004500 2021-07-11T03:53:47.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.897+0000 7f4fe4ff9700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fcc053380 0x7f4fcc055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:47.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.897+0000 7f4fe4ff9700 1 -- 172.21.2.104:0/1308056203 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4fd006b3b0 con 0x7f4fc8004500 2021-07-11T03:53:47.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.897+0000 7f4fe6ffd700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fcc053380 0x7f4fcc055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:47.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.897+0000 7f4fe6ffd700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fcc053380 0x7f4fcc055840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f4fd801b000 tx=0x7f4fd8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:47.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.897+0000 7f4fe4ff9700 1 -- 172.21.2.104:0/1308056203 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4fd00428b0 con 0x7f4fc8004500 2021-07-11T03:53:47.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 -- 172.21.2.104:0/1308056203 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4fe8057350 con 0x7f4fc8004500 2021-07-11T03:53:47.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fe4ff9700 1 -- 172.21.2.104:0/1308056203 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+13181 (secure 0 0 0) 0x7f4fd0045ed0 con 0x7f4fc8004500 2021-07-11T03:53:47.986 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:47.987 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":29,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:46.448944+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:47.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 -- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fcc053380 msgr2=0x7f4fcc055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fcc053380 0x7f4fcc055840 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f4fd801b000 tx=0x7f4fd8008040).stop 2021-07-11T03:53:47.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 -- 172.21.2.104:0/1308056203 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 msgr2=0x7f4fe80ae120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:47.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 0x7f4fe80ae120 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f4fd000b000 tx=0x7f4fe80a8c80).stop 2021-07-11T03:53:47.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 -- 172.21.2.104:0/1308056203 shutdown_connections 2021-07-11T03:53:47.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fe80ae610 0x7f4fe8144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fe80af4b0 0x7f4fe8144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fcc053380 0x7f4fcc055840 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 --2- 172.21.2.104:0/1308056203 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fc8004500 0x7f4fe80ae120 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:47.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 -- 172.21.2.104:0/1308056203 >> 172.21.2.104:0/1308056203 conn(0x7f4fe80b53e0 msgr2=0x7f4fe8149b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:47.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 -- 172.21.2.104:0/1308056203 shutdown_connections 2021-07-11T03:53:47.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:47.981+0000 7f4fee165700 1 -- 172.21.2.104:0/1308056203 wait complete. 2021-07-11T03:53:47.994 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T03:53:47.995 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:53:48.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.041+0000 7fcca58b4700 1 Processor -- start 2021-07-11T03:53:48.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.041+0000 7fcca58b4700 1 -- start start 2021-07-11T03:53:48.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcca00a9c10 0x7fcca00b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca0143920 con 0x7fcca00b3710 2021-07-11T03:53:48.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca0143bc0 con 0x7fcca00b05b0 2021-07-11T03:53:48.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca0143e60 con 0x7fcca00a9c10 2021-07-11T03:53:48.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9e7fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fcca00b3710 0x7fcca00b10b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39838/0 (socket says 172.21.2.104:39838) 2021-07-11T03:53:48.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9e7fc700 1 -- 172.21.2.104:0/3152632916 learned_addr learned my addr 172.21.2.104:0/3152632916 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:48.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9f7fe700 1 --2- 172.21.2.104:0/3152632916 >> v2:172.21.2.104:3300/0 conn(0x7fcca00a9c10 0x7fcca00b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9f7fe700 1 -- 172.21.2.104:0/3152632916 >> v1:172.21.2.105:6789/0 conn(0x7fcca00b05b0 legacy=0x7fcca013d550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:48.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9f7fe700 1 -- 172.21.2.104:0/3152632916 >> v1:172.21.2.104:6789/0 conn(0x7fcca00b3710 legacy=0x7fcca00b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9f7fe700 1 -- 172.21.2.104:0/3152632916 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca0144100 con 0x7fcca00a9c10 2021-07-11T03:53:48.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9f7fe700 1 --2- 172.21.2.104:0/3152632916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca00a9c10 0x7fcca00b0990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fcca006aa90 tx=0x7fcc980091d0).ready entity=mon.0 client_cookie=6d1b4de182c09fd8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9dffb700 1 -- 172.21.2.104:0/3152632916 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc98009ea0 con 0x7fcca00a9c10 2021-07-11T03:53:48.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9dffb700 1 -- 172.21.2.104:0/3152632916 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9800b860 con 0x7fcca00a9c10 2021-07-11T03:53:48.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9dffb700 1 -- 172.21.2.104:0/3152632916 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc9800b9d0 con 0x7fcca00a9c10 2021-07-11T03:53:48.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- 172.21.2.104:0/3152632916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca00a9c10 msgr2=0x7fcca00b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 --2- 172.21.2.104:0/3152632916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca00a9c10 0x7fcca00b0990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fcca006aa90 tx=0x7fcc980091d0).stop 2021-07-11T03:53:48.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- 172.21.2.104:0/3152632916 shutdown_connections 2021-07-11T03:53:48.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 --2- 172.21.2.104:0/3152632916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca00a9c10 0x7fcca00b0990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- 172.21.2.104:0/3152632916 >> 172.21.2.104:0/3152632916 conn(0x7fcca00607c0 msgr2=0x7fcca0060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:48.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- 172.21.2.104:0/3152632916 shutdown_connections 2021-07-11T03:53:48.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- 172.21.2.104:0/3152632916 wait complete. 2021-07-11T03:53:48.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 Processor -- start 2021-07-11T03:53:48.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- start start 2021-07-11T03:53:48.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca00a9c10 0x7fcca00646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcca00b05b0 0x7fcca0136800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcca00b3710 0x7fcca0157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca015a270 con 0x7fcca00a9c10 2021-07-11T03:53:48.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca0065560 con 0x7fcca00b3710 2021-07-11T03:53:48.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca0065800 con 0x7fcca00b05b0 2021-07-11T03:53:48.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca00a9c10 0x7fcca00646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca00a9c10 0x7fcca00646f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43076/0 (socket says 172.21.2.104:43076) 2021-07-11T03:53:48.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9effd700 1 -- 172.21.2.104:0/4225410023 learned_addr learned my addr 172.21.2.104:0/4225410023 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:48.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9effd700 1 -- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcca00b05b0 msgr2=0x7fcca0136800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9effd700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcca00b05b0 0x7fcca0136800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9effd700 1 -- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcca00b3710 msgr2=0x7fcca0157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9effd700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcca00b3710 0x7fcca0157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9effd700 1 -- 172.21.2.104:0/4225410023 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc98009470 con 0x7fcca00a9c10 2021-07-11T03:53:48.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc9effd700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca00a9c10 0x7fcca00646f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fcc94002fb0 tx=0x7fcc94006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc8ffff700 1 -- 172.21.2.104:0/4225410023 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc94009ea0 con 0x7fcca00a9c10 2021-07-11T03:53:48.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc8ffff700 1 -- 172.21.2.104:0/4225410023 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc94013460 con 0x7fcca00a9c10 2021-07-11T03:53:48.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- 172.21.2.104:0/4225410023 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca0065aa0 con 0x7fcca00a9c10 2021-07-11T03:53:48.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcca58b4700 1 -- 172.21.2.104:0/4225410023 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca0156d20 con 0x7fcca00a9c10 2021-07-11T03:53:48.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.045+0000 7fcc8ffff700 1 -- 172.21.2.104:0/4225410023 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc940137c0 con 0x7fcca00a9c10 2021-07-11T03:53:48.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.049+0000 7fcca58b4700 1 -- 172.21.2.104:0/4225410023 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca015a3e0 con 0x7fcca00a9c10 2021-07-11T03:53:48.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.049+0000 7fcc8ffff700 1 -- 172.21.2.104:0/4225410023 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcc9400f070 con 0x7fcca00a9c10 2021-07-11T03:53:48.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.049+0000 7fcc8ffff700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc7c053350 0x7fcc7c055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.049+0000 7fcc8ffff700 1 -- 172.21.2.104:0/4225410023 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcc9406ac00 con 0x7fcca00a9c10 2021-07-11T03:53:48.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.049+0000 7fcc9e7fc700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc7c053350 0x7fcc7c055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.049+0000 7fcc9e7fc700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc7c053350 0x7fcc7c055810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fcca006aa50 tx=0x7fcc90007860).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.049+0000 7fcc8ffff700 1 -- 172.21.2.104:0/4225410023 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcc94042270 con 0x7fcca00a9c10 2021-07-11T03:53:48.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.133+0000 7fcca58b4700 1 -- 172.21.2.104:0/4225410023 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcca0057350 con 0x7fcca00a9c10 2021-07-11T03:53:48.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.133+0000 7fcc8ffff700 1 -- 172.21.2.104:0/4225410023 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v29) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcc94045890 con 0x7fcca00a9c10 2021-07-11T03:53:48.140 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:53:48.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 -- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc7c053350 msgr2=0x7fcc7c055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc7c053350 0x7fcc7c055810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fcca006aa50 tx=0x7fcc90007860).stop 2021-07-11T03:53:48.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 -- 172.21.2.104:0/4225410023 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca00a9c10 msgr2=0x7fcca00646f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca00a9c10 0x7fcca00646f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fcc94002fb0 tx=0x7fcc94006e10).stop 2021-07-11T03:53:48.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 -- 172.21.2.104:0/4225410023 shutdown_connections 2021-07-11T03:53:48.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcca00b3710 0x7fcca0157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcca00b05b0 0x7fcca0136800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc7c053350 0x7fcc7c055810 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 --2- 172.21.2.104:0/4225410023 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca00a9c10 0x7fcca00646f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 -- 172.21.2.104:0/4225410023 >> 172.21.2.104:0/4225410023 conn(0x7fcca00607c0 msgr2=0x7fcca00ae980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:48.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 -- 172.21.2.104:0/4225410023 shutdown_connections 2021-07-11T03:53:48.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.137+0000 7fcca58b4700 1 -- 172.21.2.104:0/4225410023 wait complete. 2021-07-11T03:53:48.147 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:53:48.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.193+0000 7f5e4b8b6700 1 Processor -- start 2021-07-11T03:53:48.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.193+0000 7f5e4b8b6700 1 -- start start 2021-07-11T03:53:48.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.193+0000 7f5e4b8b6700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5e440b1ec0 0x7f5e440b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.193+0000 7f5e4b8b6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b22a0 0x7f5e44141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e440b0420 con 0x7f5e440affb0 2021-07-11T03:53:48.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e440a77d0 con 0x7f5e440b1ec0 2021-07-11T03:53:48.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440b0650 con 0x7f5e440b22a0 2021-07-11T03:53:48.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e48e51700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f5e440affb0 0x7f5e440a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39850/0 (socket says 172.21.2.104:39850) 2021-07-11T03:53:48.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e48e51700 1 -- 172.21.2.104:0/3850976757 learned_addr learned my addr 172.21.2.104:0/3850976757 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:48.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286859141 0 0) 0x7f5e440b0420 con 0x7f5e440affb0 2021-07-11T03:53:48.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e2c003260 con 0x7f5e440affb0 2021-07-11T03:53:48.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49e53700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b22a0 0x7f5e44141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49652700 1 --2- 172.21.2.104:0/3850976757 >> v2:172.21.2.105:3300/0 conn(0x7f5e440b1ec0 0x7f5e440b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1333898023 0 0) 0x7f5e2c003260 con 0x7f5e440affb0 2021-07-11T03:53:48.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b22a0 msgr2=0x7f5e44141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:48.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b22a0 0x7f5e44141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 >> v2:172.21.2.105:3300/0 conn(0x7f5e440b1ec0 msgr2=0x7f5e440b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:48.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 --2- 172.21.2.104:0/3850976757 >> v2:172.21.2.105:3300/0 conn(0x7f5e440b1ec0 0x7f5e440b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e440b08f0 con 0x7f5e440affb0 2021-07-11T03:53:48.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f5e34001660 con 0x7f5e440affb0 2021-07-11T03:53:48.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 >> v1:172.21.2.104:6789/0 conn(0x7f5e440affb0 legacy=0x7f5e440a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e2c0037d0 0x7f5e2c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b22a0 0x7f5e2c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e2c0049f0 0x7f5e2c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440a77d0 con 0x7f5e2c0037d0 2021-07-11T03:53:48.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440b0650 con 0x7f5e2c0049f0 2021-07-11T03:53:48.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c00b880 con 0x7f5e440b22a0 2021-07-11T03:53:48.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49652700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e2c0037d0 0x7f5e2c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49e53700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e2c0049f0 0x7f5e2c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49652700 1 -- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b22a0 msgr2=0x7f5e2c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:48.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e48e51700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b22a0 0x7f5e2c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49652700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b22a0 0x7f5e2c004500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49652700 1 -- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e2c0049f0 msgr2=0x7f5e2c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:48.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49652700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e2c0049f0 0x7f5e2c004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49652700 1 -- 172.21.2.104:0/3850976757 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c003260 con 0x7f5e2c0037d0 2021-07-11T03:53:48.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49e53700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e2c0049f0 0x7f5e2c004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:48.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49652700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e2c0037d0 0x7f5e2c003bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f5e4000a000 tx=0x7f5e40008040).ready entity=mon.0 client_cookie=fb9541cd19d5021a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e40013670 con 0x7f5e2c0037d0 2021-07-11T03:53:48.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e400137d0 con 0x7f5e2c0037d0 2021-07-11T03:53:48.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e33fff700 1 -- 172.21.2.104:0/3850976757 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e40013b30 con 0x7f5e2c0037d0 2021-07-11T03:53:48.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/3850976757 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e2c0037d0 msgr2=0x7f5e2c003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 --2- 172.21.2.104:0/3850976757 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e2c0037d0 0x7f5e2c003bf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f5e4000a000 tx=0x7f5e40008040).stop 2021-07-11T03:53:48.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e49e53700 1 -- 172.21.2.104:0/3850976757 reap_dead start 2021-07-11T03:53:48.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/3850976757 shutdown_connections 2021-07-11T03:53:48.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/3850976757 >> 172.21.2.104:0/3850976757 conn(0x7f5e4405f6b0 msgr2=0x7f5e4405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:48.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/3850976757 shutdown_connections 2021-07-11T03:53:48.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/3850976757 wait complete. 2021-07-11T03:53:48.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 Processor -- start 2021-07-11T03:53:48.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- start start 2021-07-11T03:53:48.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e44139d00 0x7f5e4414a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e4414ae40 0x7f5e4415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b31f0 0x7f5e4415a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4413a0e0 con 0x7f5e4414ae40 2021-07-11T03:53:48.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440b4990 con 0x7f5e44139d00 2021-07-11T03:53:48.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e4b8b6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4414a550 con 0x7f5e440b31f0 2021-07-11T03:53:48.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.197+0000 7f5e48e51700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e4414ae40 0x7f5e4415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e48e51700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e4414ae40 0x7f5e4415a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43094/0 (socket says 172.21.2.104:43094) 2021-07-11T03:53:48.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e48e51700 1 -- 172.21.2.104:0/2611657716 learned_addr learned my addr 172.21.2.104:0/2611657716 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:48.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e48e51700 1 -- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b31f0 msgr2=0x7f5e4415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e48e51700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b31f0 0x7f5e4415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e48e51700 1 -- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e44139d00 msgr2=0x7f5e4414a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e48e51700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e44139d00 0x7f5e4414a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e48e51700 1 -- 172.21.2.104:0/2611657716 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4414a2b0 con 0x7f5e4414ae40 2021-07-11T03:53:48.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e48e51700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e4414ae40 0x7f5e4415a210 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f5e34008000 tx=0x7f5e34009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e327fc700 1 -- 172.21.2.104:0/2611657716 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e34007170 con 0x7f5e4414ae40 2021-07-11T03:53:48.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e327fc700 1 -- 172.21.2.104:0/2611657716 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e340072d0 con 0x7f5e4414ae40 2021-07-11T03:53:48.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/2611657716 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e440b4bf0 con 0x7f5e4414ae40 2021-07-11T03:53:48.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/2611657716 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e440ab6d0 con 0x7f5e4414ae40 2021-07-11T03:53:48.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e327fc700 1 -- 172.21.2.104:0/2611657716 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e340049e0 con 0x7f5e4414ae40 2021-07-11T03:53:48.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/2611657716 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e44057350 con 0x7f5e4414ae40 2021-07-11T03:53:48.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e327fc700 1 -- 172.21.2.104:0/2611657716 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5e34004b40 con 0x7f5e4414ae40 2021-07-11T03:53:48.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e327fc700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e24053270 0x7f5e24055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e327fc700 1 -- 172.21.2.104:0/2611657716 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5e3406b720 con 0x7f5e4414ae40 2021-07-11T03:53:48.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e49652700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e24053270 0x7f5e24055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.201+0000 7f5e49652700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e24053270 0x7f5e24055730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f5e4000a000 tx=0x7f5e40008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.205+0000 7f5e327fc700 1 -- 172.21.2.104:0/2611657716 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5e34042c80 con 0x7f5e4414ae40 2021-07-11T03:53:48.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/2611657716 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5e440ac760 con 0x7f5e4414ae40 2021-07-11T03:53:48.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e327fc700 1 -- 172.21.2.104:0/2611657716 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v29) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5e340462a0 con 0x7f5e4414ae40 2021-07-11T03:53:48.292 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:48.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e24053270 msgr2=0x7f5e24055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e24053270 0x7f5e24055730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f5e4000a000 tx=0x7f5e40008040).stop 2021-07-11T03:53:48.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/2611657716 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e4414ae40 msgr2=0x7f5e4415a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e4414ae40 0x7f5e4415a210 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f5e34008000 tx=0x7f5e34009040).stop 2021-07-11T03:53:48.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/2611657716 shutdown_connections 2021-07-11T03:53:48.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e440b31f0 0x7f5e4415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e24053270 0x7f5e24055730 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e4414ae40 0x7f5e4415a210 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 --2- 172.21.2.104:0/2611657716 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e44139d00 0x7f5e4414a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/2611657716 >> 172.21.2.104:0/2611657716 conn(0x7f5e4405f6b0 msgr2=0x7f5e4413d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:48.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/2611657716 shutdown_connections 2021-07-11T03:53:48.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.285+0000 7f5e4b8b6700 1 -- 172.21.2.104:0/2611657716 wait complete. 2021-07-11T03:53:48.299 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T03:53:48.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 Processor -- start 2021-07-11T03:53:48.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 -- start start 2021-07-11T03:53:48.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8f98060100 0x7f8f980b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8f98067300 0x7f8f981438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f980aabb0 con 0x7f8f980b4c40 2021-07-11T03:53:48.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f980604e0 con 0x7f8f98060100 2021-07-11T03:53:48.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f98066ea0 con 0x7f8f98067300 2021-07-11T03:53:48.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f96ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8f98060100 0x7f8f980b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f96ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8f98060100 0x7f8f980b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43100/0 (socket says 172.21.2.104:43100) 2021-07-11T03:53:48.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f96ffd700 1 -- 172.21.2.104:0/2459754567 learned_addr learned my addr 172.21.2.104:0/2459754567 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:48.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f96ffd700 1 -- 172.21.2.104:0/2459754567 >> v2:172.21.2.105:3300/0 conn(0x7f8f98067300 msgr2=0x7f8f981438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:48.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f96ffd700 1 --2- 172.21.2.104:0/2459754567 >> v2:172.21.2.105:3300/0 conn(0x7f8f98067300 0x7f8f981438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f96ffd700 1 -- 172.21.2.104:0/2459754567 >> v1:172.21.2.105:6789/0 conn(0x7f8f980b4c40 legacy=0x7f8f98144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:48.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f96ffd700 1 -- 172.21.2.104:0/2459754567 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980b33b0 con 0x7f8f98060100 2021-07-11T03:53:48.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f96ffd700 1 --2- 172.21.2.104:0/2459754567 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8f98060100 0x7f8f980b4750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f8f9805f1e0 tx=0x7f8f88006eb0).ready entity=mon.0 client_cookie=37185212df082e4e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f95ffb700 1 -- 172.21.2.104:0/2459754567 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f8800d5c0 con 0x7f8f98060100 2021-07-11T03:53:48.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f95ffb700 1 -- 172.21.2.104:0/2459754567 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88000e90 con 0x7f8f98060100 2021-07-11T03:53:48.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f95ffb700 1 -- 172.21.2.104:0/2459754567 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f8800d910 con 0x7f8f98060100 2021-07-11T03:53:48.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 -- 172.21.2.104:0/2459754567 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8f98060100 msgr2=0x7f8f980b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 --2- 172.21.2.104:0/2459754567 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8f98060100 0x7f8f980b4750 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f8f9805f1e0 tx=0x7f8f88006eb0).stop 2021-07-11T03:53:48.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 -- 172.21.2.104:0/2459754567 shutdown_connections 2021-07-11T03:53:48.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 --2- 172.21.2.104:0/2459754567 >> v2:172.21.2.105:3300/0 conn(0x7f8f98067300 0x7f8f981438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 --2- 172.21.2.104:0/2459754567 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8f98060100 0x7f8f980b4750 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 -- 172.21.2.104:0/2459754567 >> 172.21.2.104:0/2459754567 conn(0x7f8f9805cf30 msgr2=0x7f8f9805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:48.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 -- 172.21.2.104:0/2459754567 shutdown_connections 2021-07-11T03:53:48.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 -- 172.21.2.104:0/2459754567 wait complete. 2021-07-11T03:53:48.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 Processor -- start 2021-07-11T03:53:48.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.345+0000 7f8f9d97c700 1 -- start start 2021-07-11T03:53:48.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f9d97c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8f98060100 0x7f8f980619c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f9d97c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8f98067300 0x7f8f9815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f9d97c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8f980b4c40 0x7f8f980ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f9d97c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f98060c80 con 0x7f8f98060100 2021-07-11T03:53:48.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f9d97c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f980b0fa0 con 0x7f8f98067300 2021-07-11T03:53:48.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f9d97c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9815ae80 con 0x7f8f980b4c40 2021-07-11T03:53:48.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f96ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8f98060100 0x7f8f980619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f96ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8f98060100 0x7f8f980619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43110/0 (socket says 172.21.2.104:43110) 2021-07-11T03:53:48.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f96ffd700 1 -- 172.21.2.104:0/4109886806 learned_addr learned my addr 172.21.2.104:0/4109886806 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:48.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f96ffd700 1 -- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8f980b4c40 msgr2=0x7f8f980ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f96ffd700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8f980b4c40 0x7f8f980ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f96ffd700 1 -- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8f98067300 msgr2=0x7f8f9815a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f96ffd700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8f98067300 0x7f8f9815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f96ffd700 1 -- 172.21.2.104:0/4109886806 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8800b1e0 con 0x7f8f98060100 2021-07-11T03:53:48.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f96ffd700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8f98060100 0x7f8f980619c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8f980b0880 tx=0x7f8f88015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f8ffff700 1 -- 172.21.2.104:0/4109886806 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f88021070 con 0x7f8f98060100 2021-07-11T03:53:48.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f8ffff700 1 -- 172.21.2.104:0/4109886806 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f88000e90 con 0x7f8f98060100 2021-07-11T03:53:48.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f8ffff700 1 -- 172.21.2.104:0/4109886806 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8f8800de90 con 0x7f8f98060100 2021-07-11T03:53:48.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f9d97c700 1 -- 172.21.2.104:0/4109886806 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f98062640 con 0x7f8f98060100 2021-07-11T03:53:48.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f9d97c700 1 -- 172.21.2.104:0/4109886806 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f98140530 con 0x7f8f98060100 2021-07-11T03:53:48.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f9d97c700 1 -- 172.21.2.104:0/4109886806 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f981406f0 con 0x7f8f98060100 2021-07-11T03:53:48.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f8ffff700 1 -- 172.21.2.104:0/4109886806 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8f8800f070 con 0x7f8f98060100 2021-07-11T03:53:48.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f8ffff700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8f7c053270 0x7f8f7c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f8ffff700 1 -- 172.21.2.104:0/4109886806 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8f8806c670 con 0x7f8f98060100 2021-07-11T03:53:48.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f967fc700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8f7c053270 0x7f8f7c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.349+0000 7f8f967fc700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8f7c053270 0x7f8f7c055730 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f8f80002f70 tx=0x7f8f80006ab0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.353+0000 7f8f8ffff700 1 -- 172.21.2.104:0/4109886806 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8f88043bd0 con 0x7f8f98060100 2021-07-11T03:53:48.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 -- 172.21.2.104:0/4109886806 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8f98057350 con 0x7f8f98060100 2021-07-11T03:53:48.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f8ffff700 1 -- 172.21.2.104:0/4109886806 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+143 (secure 0 0 0) 0x7f8f880471f0 con 0x7f8f98060100 2021-07-11T03:53:48.443 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:48.443 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T03:53:48.443 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:48.444 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T03:53:48.444 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T03:53:48.444 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:53:48.444 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:53:48.445 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T03:53:48.445 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T03:53:48.445 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:53:48.445 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T03:53:48.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 -- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8f7c053270 msgr2=0x7f8f7c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8f7c053270 0x7f8f7c055730 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f8f80002f70 tx=0x7f8f80006ab0).stop 2021-07-11T03:53:48.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 -- 172.21.2.104:0/4109886806 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8f98060100 msgr2=0x7f8f980619c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8f98060100 0x7f8f980619c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f8f980b0880 tx=0x7f8f88015000).stop 2021-07-11T03:53:48.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 -- 172.21.2.104:0/4109886806 shutdown_connections 2021-07-11T03:53:48.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8f980b4c40 0x7f8f980ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8f98067300 0x7f8f9815a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8f7c053270 0x7f8f7c055730 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 --2- 172.21.2.104:0/4109886806 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8f98060100 0x7f8f980619c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 -- 172.21.2.104:0/4109886806 >> 172.21.2.104:0/4109886806 conn(0x7f8f9805cf30 msgr2=0x7f8f98064870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:48.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 -- 172.21.2.104:0/4109886806 shutdown_connections 2021-07-11T03:53:48.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.437+0000 7f8f9d97c700 1 -- 172.21.2.104:0/4109886806 wait complete. 2021-07-11T03:53:48.450 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T03:53:48.450 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T03:53:48.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.497+0000 7f7d47212700 1 Processor -- start 2021-07-11T03:53:48.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.497+0000 7f7d47212700 1 -- start start 2021-07-11T03:53:48.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7d400b12d0 0x7f7d400affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7d400b22a0 0x7f7d40141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d400a77d0 con 0x7f7d400b1ec0 2021-07-11T03:53:48.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d400b1740 con 0x7f7d400b22a0 2021-07-11T03:53:48.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d400b0650 con 0x7f7d400b12d0 2021-07-11T03:53:48.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7d400b22a0 0x7f7d40141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7d400b22a0 0x7f7d40141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43118/0 (socket says 172.21.2.104:43118) 2021-07-11T03:53:48.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 -- 172.21.2.104:0/1237571294 learned_addr learned my addr 172.21.2.104:0/1237571294 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:48.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 -- 172.21.2.104:0/1237571294 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7d400b12d0 msgr2=0x7f7d400affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:48.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 --2- 172.21.2.104:0/1237571294 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7d400b12d0 0x7f7d400affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 -- 172.21.2.104:0/1237571294 >> v1:172.21.2.105:6789/0 conn(0x7f7d400b1ec0 legacy=0x7f7d400a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:48.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 -- 172.21.2.104:0/1237571294 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d400b08f0 con 0x7f7d400b22a0 2021-07-11T03:53:48.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 --2- 172.21.2.104:0/1237571294 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7d400b22a0 0x7f7d40141250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f7d4006a0f0 tx=0x7f7d38009290).ready entity=mon.0 client_cookie=3c1c9b3583b1feec server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d3f7fe700 1 -- 172.21.2.104:0/1237571294 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d3800b860 con 0x7f7d400b22a0 2021-07-11T03:53:48.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d3f7fe700 1 -- 172.21.2.104:0/1237571294 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3800b9c0 con 0x7f7d400b22a0 2021-07-11T03:53:48.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d3f7fe700 1 -- 172.21.2.104:0/1237571294 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d3800bb70 con 0x7f7d400b22a0 2021-07-11T03:53:48.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- 172.21.2.104:0/1237571294 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7d400b22a0 msgr2=0x7f7d40141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 --2- 172.21.2.104:0/1237571294 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7d400b22a0 0x7f7d40141250 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f7d4006a0f0 tx=0x7f7d38009290).stop 2021-07-11T03:53:48.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- 172.21.2.104:0/1237571294 shutdown_connections 2021-07-11T03:53:48.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 --2- 172.21.2.104:0/1237571294 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7d400b22a0 0x7f7d40141250 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:48.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 --2- 172.21.2.104:0/1237571294 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7d400b12d0 0x7f7d400affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- 172.21.2.104:0/1237571294 >> 172.21.2.104:0/1237571294 conn(0x7f7d4005f6b0 msgr2=0x7f7d4005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:48.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- 172.21.2.104:0/1237571294 shutdown_connections 2021-07-11T03:53:48.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- 172.21.2.104:0/1237571294 wait complete. 2021-07-11T03:53:48.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 Processor -- start 2021-07-11T03:53:48.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- start start 2021-07-11T03:53:48.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7d400b12d0 0x7f7d4013f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7d400b1ec0 0x7f7d4014af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7d400b22a0 0x7f7d400ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d401409f0 con 0x7f7d400b22a0 2021-07-11T03:53:48.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d400ab540 con 0x7f7d400b1ec0 2021-07-11T03:53:48.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d40148b30 con 0x7f7d400b12d0 2021-07-11T03:53:48.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7d400b22a0 0x7f7d400ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7d400b22a0 0x7f7d400ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43120/0 (socket says 172.21.2.104:43120) 2021-07-11T03:53:48.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 -- 172.21.2.104:0/2261207815 learned_addr learned my addr 172.21.2.104:0/2261207815 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:48.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 -- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7d400b12d0 msgr2=0x7f7d4013f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7d400b12d0 0x7f7d4013f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 -- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7d400b1ec0 msgr2=0x7f7d4014af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:48.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7d400b1ec0 0x7f7d4014af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:48.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 -- 172.21.2.104:0/2261207815 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d38009530 con 0x7f7d400b22a0 2021-07-11T03:53:48.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d457af700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7d400b22a0 0x7f7d400ab8a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f7d4006a0f0 tx=0x7f7d38016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d3dffb700 1 -- 172.21.2.104:0/2261207815 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d38011c80 con 0x7f7d400b22a0 2021-07-11T03:53:48.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d3dffb700 1 -- 172.21.2.104:0/2261207815 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d38011de0 con 0x7f7d400b22a0 2021-07-11T03:53:48.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- 172.21.2.104:0/2261207815 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d400ab2a0 con 0x7f7d400b22a0 2021-07-11T03:53:48.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d3dffb700 1 -- 172.21.2.104:0/2261207815 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7d3801f610 con 0x7f7d400b22a0 2021-07-11T03:53:48.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.501+0000 7f7d47212700 1 -- 172.21.2.104:0/2261207815 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4013fc70 con 0x7f7d400b22a0 2021-07-11T03:53:48.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.505+0000 7f7d47212700 1 -- 172.21.2.104:0/2261207815 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d40057350 con 0x7f7d400b22a0 2021-07-11T03:53:48.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.505+0000 7f7d3dffb700 1 -- 172.21.2.104:0/2261207815 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f7d3800f070 con 0x7f7d400b22a0 2021-07-11T03:53:48.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.505+0000 7f7d3dffb700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7d24053300 0x7f7d240557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:48.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.505+0000 7f7d3dffb700 1 -- 172.21.2.104:0/2261207815 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7d3806bc50 con 0x7f7d400b22a0 2021-07-11T03:53:48.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.505+0000 7f7d44fae700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7d24053300 0x7f7d240557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:48.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.505+0000 7f7d44fae700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7d24053300 0x7f7d240557c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f7d28000f70 tx=0x7f7d280077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:48.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.505+0000 7f7d3dffb700 1 -- 172.21.2.104:0/2261207815 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7d38043230 con 0x7f7d400b22a0 2021-07-11T03:53:48.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:48.589+0000 7f7d47212700 1 -- 172.21.2.104:0/2261207815 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f7d400ac760 con 0x7f7d400b22a0 2021-07-11T03:53:49.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.413+0000 7f7d3dffb700 1 -- 172.21.2.104:0/2261207815 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f7d38015070 con 0x7f7d400b22a0 2021-07-11T03:53:49.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 -- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7d24053300 msgr2=0x7f7d240557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:49.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7d24053300 0x7f7d240557c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f7d28000f70 tx=0x7f7d280077f0).stop 2021-07-11T03:53:49.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 -- 172.21.2.104:0/2261207815 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7d400b22a0 msgr2=0x7f7d400ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:49.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7d400b22a0 0x7f7d400ab8a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f7d4006a0f0 tx=0x7f7d38016000).stop 2021-07-11T03:53:49.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 -- 172.21.2.104:0/2261207815 shutdown_connections 2021-07-11T03:53:49.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7d24053300 0x7f7d240557c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:49.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7d400b22a0 0x7f7d400ab8a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:49.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7d400b1ec0 0x7f7d4014af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:49.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 --2- 172.21.2.104:0/2261207815 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7d400b12d0 0x7f7d4013f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:49.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 -- 172.21.2.104:0/2261207815 >> 172.21.2.104:0/2261207815 conn(0x7f7d4005f6b0 msgr2=0x7f7d400af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:49.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 -- 172.21.2.104:0/2261207815 shutdown_connections 2021-07-11T03:53:49.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.417+0000 7f7d47212700 1 -- 172.21.2.104:0/2261207815 wait complete. 2021-07-11T03:53:49.426 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T03:53:49.429 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T03:53:49.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.477+0000 7f9874c90700 1 Processor -- start 2021-07-11T03:53:49.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.477+0000 7f9874c90700 1 -- start start 2021-07-11T03:53:49.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f98700b2580 0x7f987014ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:49.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f98700b0010 0x7f987014b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:49.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98700afbd0 0x7f98700a83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:49.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98700a70c0 con 0x7f98700b2580 2021-07-11T03:53:49.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98700a72f0 con 0x7f98700b0010 2021-07-11T03:53:49.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98700a7590 con 0x7f98700afbd0 2021-07-11T03:53:49.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f98700b2580 0x7f987014ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:49.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f98700b2580 0x7f987014ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43128/0 (socket says 172.21.2.104:43128) 2021-07-11T03:53:49.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986f7fe700 1 -- 172.21.2.104:0/2269667052 learned_addr learned my addr 172.21.2.104:0/2269667052 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:49.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986f7fe700 1 -- 172.21.2.104:0/2269667052 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98700afbd0 msgr2=0x7f98700a83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:49.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986f7fe700 1 --2- 172.21.2.104:0/2269667052 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98700afbd0 0x7f98700a83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:49.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986f7fe700 1 -- 172.21.2.104:0/2269667052 >> v2:172.21.2.105:3300/0 conn(0x7f98700b0010 msgr2=0x7f987014b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:49.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986f7fe700 1 --2- 172.21.2.104:0/2269667052 >> v2:172.21.2.105:3300/0 conn(0x7f98700b0010 0x7f987014b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:49.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986f7fe700 1 -- 172.21.2.104:0/2269667052 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98700a7830 con 0x7f98700b2580 2021-07-11T03:53:49.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986f7fe700 1 --2- 172.21.2.104:0/2269667052 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98700b2580 0x7f987014ae40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f9864002f70 tx=0x7f9864006eb0).ready entity=mon.0 client_cookie=14073b7b7d87ba65 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:49.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986e7fc700 1 -- 172.21.2.104:0/2269667052 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f986400d570 con 0x7f98700b2580 2021-07-11T03:53:49.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986e7fc700 1 -- 172.21.2.104:0/2269667052 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864000e90 con 0x7f98700b2580 2021-07-11T03:53:49.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986e7fc700 1 -- 172.21.2.104:0/2269667052 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f986400d900 con 0x7f98700b2580 2021-07-11T03:53:49.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- 172.21.2.104:0/2269667052 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98700b2580 msgr2=0x7f987014ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:49.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 --2- 172.21.2.104:0/2269667052 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98700b2580 0x7f987014ae40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f9864002f70 tx=0x7f9864006eb0).stop 2021-07-11T03:53:49.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- 172.21.2.104:0/2269667052 shutdown_connections 2021-07-11T03:53:49.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 --2- 172.21.2.104:0/2269667052 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98700afbd0 0x7f98700a83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:49.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 --2- 172.21.2.104:0/2269667052 >> v2:172.21.2.105:3300/0 conn(0x7f98700b0010 0x7f987014b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:49.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 --2- 172.21.2.104:0/2269667052 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98700b2580 0x7f987014ae40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:49.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- 172.21.2.104:0/2269667052 >> 172.21.2.104:0/2269667052 conn(0x7f9870060180 msgr2=0x7f98700b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:49.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- 172.21.2.104:0/2269667052 shutdown_connections 2021-07-11T03:53:49.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- 172.21.2.104:0/2269667052 wait complete. 2021-07-11T03:53:49.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 Processor -- start 2021-07-11T03:53:49.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- start start 2021-07-11T03:53:49.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98700afbd0 0x7f98700aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:49.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98700b0010 0x7f98701414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:49.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98700b2580 0x7f9870141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:49.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9870153df0 con 0x7f98700b0010 2021-07-11T03:53:49.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98700ab210 con 0x7f98700afbd0 2021-07-11T03:53:49.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98700aacd0 con 0x7f98700b2580 2021-07-11T03:53:49.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98700b0010 0x7f98701414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:49.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98700b0010 0x7f98701414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43136/0 (socket says 172.21.2.104:43136) 2021-07-11T03:53:49.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986effd700 1 -- 172.21.2.104:0/3416003519 learned_addr learned my addr 172.21.2.104:0/3416003519 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:49.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986effd700 1 -- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98700b2580 msgr2=0x7f9870141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:49.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986effd700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98700b2580 0x7f9870141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:49.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986effd700 1 -- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98700afbd0 msgr2=0x7f98700aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:49.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986f7fe700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98700afbd0 0x7f98700aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:49.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986effd700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98700afbd0 0x7f98700aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:49.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986effd700 1 -- 172.21.2.104:0/3416003519 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986400b1e0 con 0x7f98700b0010 2021-07-11T03:53:49.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986f7fe700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98700afbd0 0x7f98700aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:53:49.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986effd700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98700b0010 0x7f98701414a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f98680032f0 tx=0x7f98680091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:49.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986cff9700 1 -- 172.21.2.104:0/3416003519 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f986800f760 con 0x7f98700b0010 2021-07-11T03:53:49.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986cff9700 1 -- 172.21.2.104:0/3416003519 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986800f8c0 con 0x7f98700b0010 2021-07-11T03:53:49.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- 172.21.2.104:0/3416003519 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98700aaf70 con 0x7f98700b0010 2021-07-11T03:53:49.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f9874c90700 1 -- 172.21.2.104:0/3416003519 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9870156bc0 con 0x7f98700b0010 2021-07-11T03:53:49.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.481+0000 7f986cff9700 1 -- 172.21.2.104:0/3416003519 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f986800fc20 con 0x7f98700b0010 2021-07-11T03:53:49.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.485+0000 7f9874c90700 1 -- 172.21.2.104:0/3416003519 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98700aba30 con 0x7f98700b0010 2021-07-11T03:53:49.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.485+0000 7f986cff9700 1 -- 172.21.2.104:0/3416003519 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f986801a460 con 0x7f98700b0010 2021-07-11T03:53:49.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.485+0000 7f986cff9700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9850053350 0x7f9850055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:49.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.485+0000 7f986cff9700 1 -- 172.21.2.104:0/3416003519 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f986806b760 con 0x7f98700b0010 2021-07-11T03:53:49.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.485+0000 7f986f7fe700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9850053350 0x7f9850055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:49.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.485+0000 7f986f7fe700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9850053350 0x7f9850055810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f9864016000 tx=0x7f9864015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:49.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.485+0000 7f986cff9700 1 -- 172.21.2.104:0/3416003519 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f98700aba30 con 0x7f98700b0010 2021-07-11T03:53:49.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:49.569+0000 7f9874c90700 1 -- 172.21.2.104:0/3416003519 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f98700aba30 con 0x7f98700b0010 2021-07-11T03:53:50.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f986cff9700 1 -- 172.21.2.104:0/3416003519 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v31) v1 ==== 125+0+0 (secure 0 0 0) 0x7f98700aba30 con 0x7f98700b0010 2021-07-11T03:53:50.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 -- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9850053350 msgr2=0x7f9850055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9850053350 0x7f9850055810 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f9864016000 tx=0x7f9864015000).stop 2021-07-11T03:53:50.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 -- 172.21.2.104:0/3416003519 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98700b0010 msgr2=0x7f98701414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98700b0010 0x7f98701414a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f98680032f0 tx=0x7f98680091d0).stop 2021-07-11T03:53:50.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 -- 172.21.2.104:0/3416003519 shutdown_connections 2021-07-11T03:53:50.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98700b2580 0x7f9870141990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9850053350 0x7f9850055810 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98700b0010 0x7f98701414a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 --2- 172.21.2.104:0/3416003519 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98700afbd0 0x7f98700aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 -- 172.21.2.104:0/3416003519 >> 172.21.2.104:0/3416003519 conn(0x7f9870060180 msgr2=0x7f9870145cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:50.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 -- 172.21.2.104:0/3416003519 shutdown_connections 2021-07-11T03:53:50.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.421+0000 7f9874c90700 1 -- 172.21.2.104:0/3416003519 wait complete. 2021-07-11T03:53:50.432 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T03:53:50.436 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T03:53:50.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 Processor -- start 2021-07-11T03:53:50.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 -- start start 2021-07-11T03:53:50.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1be00a9da0 0x7f1be00b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be00ac370 con 0x7f1be00aa860 2021-07-11T03:53:50.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be00aac40 con 0x7f1be00ad150 2021-07-11T03:53:50.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be00abb50 con 0x7f1be00a9da0 2021-07-11T03:53:50.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be4998700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1be00aa860 0x7f1be00aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39910/0 (socket says 172.21.2.104:39910) 2021-07-11T03:53:50.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be4998700 1 -- 172.21.2.104:0/3704545369 learned_addr learned my addr 172.21.2.104:0/3704545369 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:50.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be5199700 1 --2- 172.21.2.104:0/3704545369 >> v2:172.21.2.104:3300/0 conn(0x7f1be00a9da0 0x7f1be00b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:50.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be5199700 1 -- 172.21.2.104:0/3704545369 >> v1:172.21.2.105:6789/0 conn(0x7f1be00ad150 legacy=0x7f1be0140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:50.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be5199700 1 -- 172.21.2.104:0/3704545369 >> v1:172.21.2.104:6789/0 conn(0x7f1be00aa860 legacy=0x7f1be00aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be5199700 1 -- 172.21.2.104:0/3704545369 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be00b16b0 con 0x7f1be00a9da0 2021-07-11T03:53:50.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be5199700 1 --2- 172.21.2.104:0/3704545369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1be00a9da0 0x7f1be00b11c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f1be00b63e0 tx=0x7f1bd800b470).ready entity=mon.0 client_cookie=2f016d17d13d8474 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:50.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1bdf7fe700 1 -- 172.21.2.104:0/3704545369 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bd800dde0 con 0x7f1be00a9da0 2021-07-11T03:53:50.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1bdf7fe700 1 -- 172.21.2.104:0/3704545369 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd8010470 con 0x7f1be00a9da0 2021-07-11T03:53:50.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1bdf7fe700 1 -- 172.21.2.104:0/3704545369 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bd80107d0 con 0x7f1be00a9da0 2021-07-11T03:53:50.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3704545369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1be00a9da0 msgr2=0x7f1be00b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 --2- 172.21.2.104:0/3704545369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1be00a9da0 0x7f1be00b11c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f1be00b63e0 tx=0x7f1bd800b470).stop 2021-07-11T03:53:50.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3704545369 shutdown_connections 2021-07-11T03:53:50.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 --2- 172.21.2.104:0/3704545369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1be00a9da0 0x7f1be00b11c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3704545369 >> 172.21.2.104:0/3704545369 conn(0x7f1be00b64d0 msgr2=0x7f1be00b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:50.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3704545369 shutdown_connections 2021-07-11T03:53:50.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.485+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3704545369 wait complete. 2021-07-11T03:53:50.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be6bfc700 1 Processor -- start 2021-07-11T03:53:50.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be6bfc700 1 -- start start 2021-07-11T03:53:50.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be6bfc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1be00a9da0 0x7f1be0143680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be6bfc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1be00aa860 0x7f1be0156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be6bfc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1be00ad150 0x7f1be0144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be6bfc700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be0155a50 con 0x7f1be00a9da0 2021-07-11T03:53:50.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be6bfc700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00b0430 con 0x7f1be00ad150 2021-07-11T03:53:50.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be6bfc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be0155cb0 con 0x7f1be00aa860 2021-07-11T03:53:50.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be4998700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1be00a9da0 0x7f1be0143680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:50.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be4998700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1be00a9da0 0x7f1be0143680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43148/0 (socket says 172.21.2.104:43148) 2021-07-11T03:53:50.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be4998700 1 -- 172.21.2.104:0/3855341809 learned_addr learned my addr 172.21.2.104:0/3855341809 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:50.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be4998700 1 -- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1be00aa860 msgr2=0x7f1be0156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be4998700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1be00aa860 0x7f1be0156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be4998700 1 -- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1be00ad150 msgr2=0x7f1be0144530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be4998700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1be00ad150 0x7f1be0144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be4998700 1 -- 172.21.2.104:0/3855341809 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd800b710 con 0x7f1be00a9da0 2021-07-11T03:53:50.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be4998700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1be00a9da0 0x7f1be0143680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f1be00b2e10 tx=0x7f1bc8007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:50.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1bddffb700 1 -- 172.21.2.104:0/3855341809 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bc800ce30 con 0x7f1be00a9da0 2021-07-11T03:53:50.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1bddffb700 1 -- 172.21.2.104:0/3855341809 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bc80072e0 con 0x7f1be00a9da0 2021-07-11T03:53:50.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1bddffb700 1 -- 172.21.2.104:0/3855341809 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1bc8005800 con 0x7f1be00a9da0 2021-07-11T03:53:50.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3855341809 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be0145730 con 0x7f1be00a9da0 2021-07-11T03:53:50.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3855341809 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be00b06d0 con 0x7f1be00a9da0 2021-07-11T03:53:50.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3855341809 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be0057350 con 0x7f1be00a9da0 2021-07-11T03:53:50.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1bddffb700 1 -- 172.21.2.104:0/3855341809 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1bc8010070 con 0x7f1be00a9da0 2021-07-11T03:53:50.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1bddffb700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1bc40531a0 0x7f1bc4055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1bddffb700 1 -- 172.21.2.104:0/3855341809 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f1bc806ac50 con 0x7f1be00a9da0 2021-07-11T03:53:50.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1bdffff700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1bc40531a0 0x7f1bc4055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:50.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.489+0000 7f1bdffff700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1bc40531a0 0x7f1bc4055660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f1bd0002f30 tx=0x7f1bd0006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:50.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.493+0000 7f1bddffb700 1 -- 172.21.2.104:0/3855341809 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1bc8041ef0 con 0x7f1be00a9da0 2021-07-11T03:53:50.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.601+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3855341809 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f1be00698e0 con 0x7f1be00a9da0 2021-07-11T03:53:50.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1bddffb700 1 -- 172.21.2.104:0/3855341809 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v10) v1 ==== 187+0+0 (secure 0 0 0) 0x7f1bc8045510 con 0x7f1be00a9da0 2021-07-11T03:53:50.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1bc40531a0 msgr2=0x7f1bc4055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1bc40531a0 0x7f1bc4055660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f1bd0002f30 tx=0x7f1bd0006930).stop 2021-07-11T03:53:50.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3855341809 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1be00a9da0 msgr2=0x7f1be0143680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1be00a9da0 0x7f1be0143680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f1be00b2e10 tx=0x7f1bc8007c40).stop 2021-07-11T03:53:50.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3855341809 shutdown_connections 2021-07-11T03:53:50.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1be00ad150 0x7f1be0144530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1be00aa860 0x7f1be0156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1bc40531a0 0x7f1bc4055660 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 --2- 172.21.2.104:0/3855341809 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1be00a9da0 0x7f1be0143680 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3855341809 >> 172.21.2.104:0/3855341809 conn(0x7f1be00b64d0 msgr2=0x7f1be00605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:50.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3855341809 shutdown_connections 2021-07-11T03:53:50.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.609+0000 7f1be6bfc700 1 -- 172.21.2.104:0/3855341809 wait complete. 2021-07-11T03:53:50.618 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 5 and data pool 6 2021-07-11T03:53:50.623 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:50.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 Processor -- start 2021-07-11T03:53:50.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 -- start start 2021-07-11T03:53:50.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f53580b7810 0x7f53581445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f53580b1170 0x7f5358144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53580b31d0 con 0x7f5358064870 2021-07-11T03:53:50.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53580b7bf0 con 0x7f53580b7810 2021-07-11T03:53:50.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53580675c0 con 0x7f53580b1170 2021-07-11T03:53:50.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f5356d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f5358064870 0x7f5358064d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39938/0 (socket says 172.21.2.104:39938) 2021-07-11T03:53:50.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f5356d9d700 1 -- 172.21.2.104:0/2863149084 learned_addr learned my addr 172.21.2.104:0/2863149084 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:50.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535659c700 1 --2- 172.21.2.104:0/2863149084 >> v2:172.21.2.104:3300/0 conn(0x7f53580b7810 0x7f53581445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:50.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535659c700 1 -- 172.21.2.104:0/2863149084 >> v2:172.21.2.105:3300/0 conn(0x7f53580b1170 msgr2=0x7f5358144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:50.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535659c700 1 --2- 172.21.2.104:0/2863149084 >> v2:172.21.2.105:3300/0 conn(0x7f53580b1170 0x7f5358144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535659c700 1 -- 172.21.2.104:0/2863149084 >> v1:172.21.2.104:6789/0 conn(0x7f5358064870 legacy=0x7f5358064d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535659c700 1 -- 172.21.2.104:0/2863149084 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53580aae70 con 0x7f53580b7810 2021-07-11T03:53:50.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535659c700 1 --2- 172.21.2.104:0/2863149084 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53580b7810 0x7f53581445a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f5348002f70 tx=0x7f5348006df0).ready entity=mon.0 client_cookie=4c1716a34f4348c3 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:50.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535559a700 1 -- 172.21.2.104:0/2863149084 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f534800d5c0 con 0x7f53580b7810 2021-07-11T03:53:50.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535559a700 1 -- 172.21.2.104:0/2863149084 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5348000e90 con 0x7f53580b7810 2021-07-11T03:53:50.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535559a700 1 -- 172.21.2.104:0/2863149084 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f534800d910 con 0x7f53580b7810 2021-07-11T03:53:50.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 -- 172.21.2.104:0/2863149084 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53580b7810 msgr2=0x7f53581445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 --2- 172.21.2.104:0/2863149084 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53580b7810 0x7f53581445a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f5348002f70 tx=0x7f5348006df0).stop 2021-07-11T03:53:50.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 -- 172.21.2.104:0/2863149084 shutdown_connections 2021-07-11T03:53:50.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 --2- 172.21.2.104:0/2863149084 >> v2:172.21.2.105:3300/0 conn(0x7f53580b1170 0x7f5358144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 --2- 172.21.2.104:0/2863149084 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53580b7810 0x7f53581445a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 -- 172.21.2.104:0/2863149084 >> 172.21.2.104:0/2863149084 conn(0x7f535805f6b0 msgr2=0x7f535805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:50.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 -- 172.21.2.104:0/2863149084 shutdown_connections 2021-07-11T03:53:50.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 -- 172.21.2.104:0/2863149084 wait complete. 2021-07-11T03:53:50.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 Processor -- start 2021-07-11T03:53:50.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.669+0000 7f535c8ff700 1 -- start start 2021-07-11T03:53:50.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f535c8ff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5358064870 0x7f53580acf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f535c8ff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53580b1170 0x7f53580b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f535c8ff700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53580b7810 0x7f53581526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f535c8ff700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5358062120 con 0x7f53580b1170 2021-07-11T03:53:50.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f535c8ff700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53580623c0 con 0x7f53580b7810 2021-07-11T03:53:50.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f535c8ff700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5358062660 con 0x7f5358064870 2021-07-11T03:53:50.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f5355d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53580b1170 0x7f53580b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:50.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f5355d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53580b1170 0x7f53580b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43176/0 (socket says 172.21.2.104:43176) 2021-07-11T03:53:50.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f5355d9b700 1 -- 172.21.2.104:0/136132590 learned_addr learned my addr 172.21.2.104:0/136132590 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:50.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f5355d9b700 1 -- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5358064870 msgr2=0x7f53580acf60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:50.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f5355d9b700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5358064870 0x7f53580acf60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f5355d9b700 1 -- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53580b7810 msgr2=0x7f53581526e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:50.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f5355d9b700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53580b7810 0x7f53581526e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f5355d9b700 1 -- 172.21.2.104:0/136132590 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534800b180 con 0x7f53580b1170 2021-07-11T03:53:50.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f5355d9b700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53580b1170 0x7f53580b5cd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f5340002f70 tx=0x7f5340006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:50.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f534f7fe700 1 -- 172.21.2.104:0/136132590 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5340013460 con 0x7f53580b1170 2021-07-11T03:53:50.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f534f7fe700 1 -- 172.21.2.104:0/136132590 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53400135c0 con 0x7f53580b1170 2021-07-11T03:53:50.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f535c8ff700 1 -- 172.21.2.104:0/136132590 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5358152240 con 0x7f53580b1170 2021-07-11T03:53:50.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f535c8ff700 1 -- 172.21.2.104:0/136132590 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53580ac400 con 0x7f53580b1170 2021-07-11T03:53:50.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f534f7fe700 1 -- 172.21.2.104:0/136132590 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5340013920 con 0x7f53580b1170 2021-07-11T03:53:50.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f535c8ff700 1 -- 172.21.2.104:0/136132590 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53581523d0 con 0x7f53580b1170 2021-07-11T03:53:50.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f534f7fe700 1 -- 172.21.2.104:0/136132590 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f534001a060 con 0x7f53580b1170 2021-07-11T03:53:50.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f534f7fe700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f533c053240 0x7f533c055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f534f7fe700 1 -- 172.21.2.104:0/136132590 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f534006b080 con 0x7f53580b1170 2021-07-11T03:53:50.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f535659c700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f533c053240 0x7f533c055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:50.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.673+0000 7f535659c700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f533c053240 0x7f533c055700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f5348002f70 tx=0x7f5348014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:50.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.677+0000 7f534f7fe700 1 -- 172.21.2.104:0/136132590 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f53400422b0 con 0x7f53580b1170 2021-07-11T03:53:50.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.757+0000 7f535c8ff700 1 -- 172.21.2.104:0/136132590 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5358057350 con 0x7f53580b1170 2021-07-11T03:53:50.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f534f7fe700 1 -- 172.21.2.104:0/136132590 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16051 (secure 0 0 0) 0x7f5340018030 con 0x7f53580b1170 2021-07-11T03:53:50.766 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:50.767 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":32,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:50.608510+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:48.603543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:49.585828+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:50.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 -- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f533c053240 msgr2=0x7f533c055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f533c053240 0x7f533c055700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f5348002f70 tx=0x7f5348014000).stop 2021-07-11T03:53:50.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 -- 172.21.2.104:0/136132590 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53580b1170 msgr2=0x7f53580b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53580b1170 0x7f53580b5cd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f5340002f70 tx=0x7f5340006ec0).stop 2021-07-11T03:53:50.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 -- 172.21.2.104:0/136132590 shutdown_connections 2021-07-11T03:53:50.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53580b7810 0x7f53581526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f533c053240 0x7f533c055700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53580b1170 0x7f53580b5cd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 --2- 172.21.2.104:0/136132590 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5358064870 0x7f53580acf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 -- 172.21.2.104:0/136132590 >> 172.21.2.104:0/136132590 conn(0x7f535805f6b0 msgr2=0x7f5358065980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:50.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 -- 172.21.2.104:0/136132590 shutdown_connections 2021-07-11T03:53:50.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.761+0000 7f535c8ff700 1 -- 172.21.2.104:0/136132590 wait complete. 2021-07-11T03:53:50.776 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:50.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 Processor -- start 2021-07-11T03:53:50.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 -- start start 2021-07-11T03:53:50.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fea840aa860 0x7fea840b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea840a9da0 0x7fea840aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea840ac370 con 0x7fea840acf30 2021-07-11T03:53:50.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea840abb50 con 0x7fea840aa860 2021-07-11T03:53:50.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea840aac40 con 0x7fea840a9da0 2021-07-11T03:53:50.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea88a88700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fea840aa860 0x7fea840b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:50.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea89289700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fea840acf30 0x7fea841434b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39952/0 (socket says 172.21.2.104:39952) 2021-07-11T03:53:50.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea89289700 1 -- 172.21.2.104:0/48615940 learned_addr learned my addr 172.21.2.104:0/48615940 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:50.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea88a88700 1 -- 172.21.2.104:0/48615940 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea840a9da0 msgr2=0x7fea840aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:50.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea88a88700 1 --2- 172.21.2.104:0/48615940 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea840a9da0 0x7fea840aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea88a88700 1 -- 172.21.2.104:0/48615940 >> v1:172.21.2.104:6789/0 conn(0x7fea840acf30 legacy=0x7fea841434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea88a88700 1 -- 172.21.2.104:0/48615940 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840b16b0 con 0x7fea840aa860 2021-07-11T03:53:50.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea88a88700 1 --2- 172.21.2.104:0/48615940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea840aa860 0x7fea840b11c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fea74002f70 tx=0x7fea74006e80).ready entity=mon.0 client_cookie=a5ddbc9d699fd55b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:50.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea837fe700 1 -- 172.21.2.104:0/48615940 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea7400ba70 con 0x7fea840aa860 2021-07-11T03:53:50.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea837fe700 1 -- 172.21.2.104:0/48615940 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea74000e90 con 0x7fea840aa860 2021-07-11T03:53:50.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea837fe700 1 -- 172.21.2.104:0/48615940 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea7400bdc0 con 0x7fea840aa860 2021-07-11T03:53:50.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 -- 172.21.2.104:0/48615940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea840aa860 msgr2=0x7fea840b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 --2- 172.21.2.104:0/48615940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea840aa860 0x7fea840b11c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fea74002f70 tx=0x7fea74006e80).stop 2021-07-11T03:53:50.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 -- 172.21.2.104:0/48615940 shutdown_connections 2021-07-11T03:53:50.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 --2- 172.21.2.104:0/48615940 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea840a9da0 0x7fea840aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 --2- 172.21.2.104:0/48615940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea840aa860 0x7fea840b11c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 -- 172.21.2.104:0/48615940 >> 172.21.2.104:0/48615940 conn(0x7fea840b64d0 msgr2=0x7fea840b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:50.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 -- 172.21.2.104:0/48615940 shutdown_connections 2021-07-11T03:53:50.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.829+0000 7fea8acec700 1 -- 172.21.2.104:0/48615940 wait complete. 2021-07-11T03:53:50.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea8acec700 1 Processor -- start 2021-07-11T03:53:50.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea8acec700 1 -- start start 2021-07-11T03:53:50.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea8acec700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea840a9da0 0x7fea84143370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea8acec700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea840aa860 0x7fea840a1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea8acec700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea840acf30 0x7fea840a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea8acec700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea840636e0 con 0x7fea840aa860 2021-07-11T03:53:50.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea8acec700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea840666c0 con 0x7fea840a9da0 2021-07-11T03:53:50.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea8acec700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8414e000 con 0x7fea840acf30 2021-07-11T03:53:50.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea83fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea840aa860 0x7fea840a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:50.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea83fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea840aa860 0x7fea840a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43190/0 (socket says 172.21.2.104:43190) 2021-07-11T03:53:50.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea83fff700 1 -- 172.21.2.104:0/3165138514 learned_addr learned my addr 172.21.2.104:0/3165138514 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:50.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea83fff700 1 -- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea840acf30 msgr2=0x7fea840a1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea83fff700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea840acf30 0x7fea840a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea83fff700 1 -- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea840a9da0 msgr2=0x7fea84143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea83fff700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea840a9da0 0x7fea84143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea83fff700 1 -- 172.21.2.104:0/3165138514 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7400d410 con 0x7fea840aa860 2021-07-11T03:53:50.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea83fff700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea840aa860 0x7fea840a1710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fea6c002f70 tx=0x7fea6c006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:50.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea81ffb700 1 -- 172.21.2.104:0/3165138514 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea6c013460 con 0x7fea840aa860 2021-07-11T03:53:50.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea81ffb700 1 -- 172.21.2.104:0/3165138514 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c0135c0 con 0x7fea840aa860 2021-07-11T03:53:50.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea8acec700 1 -- 172.21.2.104:0/3165138514 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8414e2a0 con 0x7fea840aa860 2021-07-11T03:53:50.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea81ffb700 1 -- 172.21.2.104:0/3165138514 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea6c013920 con 0x7fea840aa860 2021-07-11T03:53:50.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea8acec700 1 -- 172.21.2.104:0/3165138514 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea840a20f0 con 0x7fea840aa860 2021-07-11T03:53:50.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea8acec700 1 -- 172.21.2.104:0/3165138514 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea84063850 con 0x7fea840aa860 2021-07-11T03:53:50.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea81ffb700 1 -- 172.21.2.104:0/3165138514 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fea6c01a060 con 0x7fea840aa860 2021-07-11T03:53:50.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea81ffb700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea680531a0 0x7fea68055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea81ffb700 1 -- 172.21.2.104:0/3165138514 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fea6c06afa0 con 0x7fea840aa860 2021-07-11T03:53:50.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.833+0000 7fea88a88700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea680531a0 0x7fea68055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:50.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.837+0000 7fea88a88700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea680531a0 0x7fea68055660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fea74002f70 tx=0x7fea840ac290).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:50.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.837+0000 7fea81ffb700 1 -- 172.21.2.104:0/3165138514 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fea6c0422e0 con 0x7fea840aa860 2021-07-11T03:53:50.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.917+0000 7fea8acec700 1 -- 172.21.2.104:0/3165138514 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea84066830 con 0x7fea840aa860 2021-07-11T03:53:50.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.917+0000 7fea81ffb700 1 -- 172.21.2.104:0/3165138514 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16051 (secure 0 0 0) 0x7fea84066830 con 0x7fea840aa860 2021-07-11T03:53:50.925 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:50.926 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":32,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:50.608510+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:48.603543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:49.585828+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:50.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 -- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea680531a0 msgr2=0x7fea68055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea680531a0 0x7fea68055660 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fea74002f70 tx=0x7fea840ac290).stop 2021-07-11T03:53:50.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 -- 172.21.2.104:0/3165138514 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea840aa860 msgr2=0x7fea840a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea840aa860 0x7fea840a1710 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7fea6c002f70 tx=0x7fea6c006ec0).stop 2021-07-11T03:53:50.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 -- 172.21.2.104:0/3165138514 shutdown_connections 2021-07-11T03:53:50.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea840acf30 0x7fea840a1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea680531a0 0x7fea68055660 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea840aa860 0x7fea840a1710 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 --2- 172.21.2.104:0/3165138514 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea840a9da0 0x7fea84143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 -- 172.21.2.104:0/3165138514 >> 172.21.2.104:0/3165138514 conn(0x7fea840b64d0 msgr2=0x7fea841445b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:50.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 -- 172.21.2.104:0/3165138514 shutdown_connections 2021-07-11T03:53:50.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.921+0000 7fea8acec700 1 -- 172.21.2.104:0/3165138514 wait complete. 2021-07-11T03:53:50.932 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T03:53:50.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 Processor -- start 2021-07-11T03:53:50.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- start start 2021-07-11T03:53:50.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f55180b1ec0 0x7f55180b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f55180affb0 0x7f55180b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55180b2790 con 0x7f55180a70c0 2021-07-11T03:53:50.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55180b03f0 con 0x7f55180affb0 2021-07-11T03:53:50.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55180b0650 con 0x7f55180b1ec0 2021-07-11T03:53:50.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f55167fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f55180affb0 0x7f55180b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:50.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f55167fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f55180affb0 0x7f55180b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43198/0 (socket says 172.21.2.104:43198) 2021-07-11T03:53:50.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f55167fc700 1 -- 172.21.2.104:0/902467076 learned_addr learned my addr 172.21.2.104:0/902467076 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:50.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f55167fc700 1 -- 172.21.2.104:0/902467076 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f55180b1ec0 msgr2=0x7f55180b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:50.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f55167fc700 1 --2- 172.21.2.104:0/902467076 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f55180b1ec0 0x7f55180b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f55167fc700 1 -- 172.21.2.104:0/902467076 >> v1:172.21.2.105:6789/0 conn(0x7f55180a70c0 legacy=0x7f5518141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:50.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f55167fc700 1 -- 172.21.2.104:0/902467076 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55180b08f0 con 0x7f55180affb0 2021-07-11T03:53:50.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f55167fc700 1 --2- 172.21.2.104:0/902467076 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55180affb0 0x7f55180b22a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f551806a0f0 tx=0x7f5508006eb0).ready entity=mon.0 client_cookie=d2796f127af3fb3b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:50.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f5515ffb700 1 -- 172.21.2.104:0/902467076 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f550800d570 con 0x7f55180affb0 2021-07-11T03:53:50.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f5515ffb700 1 -- 172.21.2.104:0/902467076 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508000e90 con 0x7f55180affb0 2021-07-11T03:53:50.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f5515ffb700 1 -- 172.21.2.104:0/902467076 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f550800d8c0 con 0x7f55180affb0 2021-07-11T03:53:50.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- 172.21.2.104:0/902467076 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55180affb0 msgr2=0x7f55180b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:50.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 --2- 172.21.2.104:0/902467076 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55180affb0 0x7f55180b22a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f551806a0f0 tx=0x7f5508006eb0).stop 2021-07-11T03:53:50.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- 172.21.2.104:0/902467076 shutdown_connections 2021-07-11T03:53:50.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 --2- 172.21.2.104:0/902467076 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55180affb0 0x7f55180b22a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:50.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 --2- 172.21.2.104:0/902467076 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f55180b1ec0 0x7f55180b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:50.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- 172.21.2.104:0/902467076 >> 172.21.2.104:0/902467076 conn(0x7f551805f6b0 msgr2=0x7f551805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:50.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- 172.21.2.104:0/902467076 shutdown_connections 2021-07-11T03:53:50.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- 172.21.2.104:0/902467076 wait complete. 2021-07-11T03:53:50.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 Processor -- start 2021-07-11T03:53:50.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- start start 2021-07-11T03:53:50.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f55180a70c0 0x7f5518140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55180affb0 0x7f55180ae810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f55180b1ec0 0x7f55180aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:50.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551806aac0 con 0x7f55180affb0 2021-07-11T03:53:50.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55180ab700 con 0x7f55180a70c0 2021-07-11T03:53:51.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.981+0000 7f551d9fe700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55180adee0 con 0x7f55180b1ec0 2021-07-11T03:53:51.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f55167fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55180affb0 0x7f55180ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:51.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f55167fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55180affb0 0x7f55180ae810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43202/0 (socket says 172.21.2.104:43202) 2021-07-11T03:53:51.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f55167fc700 1 -- 172.21.2.104:0/424681232 learned_addr learned my addr 172.21.2.104:0/424681232 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:51.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f55167fc700 1 -- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f55180b1ec0 msgr2=0x7f55180aed00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:51.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f55167fc700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f55180b1ec0 0x7f55180aed00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f55167fc700 1 -- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f55180a70c0 msgr2=0x7f5518140580 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:51.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f55167fc700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f55180a70c0 0x7f5518140580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f55167fc700 1 -- 172.21.2.104:0/424681232 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550800b1e0 con 0x7f55180affb0 2021-07-11T03:53:51.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f55167fc700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55180affb0 0x7f55180ae810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f551806a0f0 tx=0x7f5508015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:51.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f5507fff700 1 -- 172.21.2.104:0/424681232 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5508021070 con 0x7f55180affb0 2021-07-11T03:53:51.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f5507fff700 1 -- 172.21.2.104:0/424681232 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5508000e90 con 0x7f55180affb0 2021-07-11T03:53:51.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f551d9fe700 1 -- 172.21.2.104:0/424681232 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55180ad500 con 0x7f55180affb0 2021-07-11T03:53:51.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f5507fff700 1 -- 172.21.2.104:0/424681232 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5508006be0 con 0x7f55180affb0 2021-07-11T03:53:51.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f551d9fe700 1 -- 172.21.2.104:0/424681232 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55180ada40 con 0x7f55180affb0 2021-07-11T03:53:51.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f551d9fe700 1 -- 172.21.2.104:0/424681232 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5518057350 con 0x7f55180affb0 2021-07-11T03:53:51.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f5507fff700 1 -- 172.21.2.104:0/424681232 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f550800f070 con 0x7f55180affb0 2021-07-11T03:53:51.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f5507fff700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54f4053270 0x7f54f4055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f5507fff700 1 -- 172.21.2.104:0/424681232 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f550806c8d0 con 0x7f55180affb0 2021-07-11T03:53:51.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f5516ffd700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54f4053270 0x7f54f4055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:51.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.985+0000 7f5516ffd700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54f4053270 0x7f54f4055730 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f550c002f70 tx=0x7f550c006990).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:51.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:50.989+0000 7f5507fff700 1 -- 172.21.2.104:0/424681232 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5508043cc0 con 0x7f55180affb0 2021-07-11T03:53:51.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.097+0000 7f551d9fe700 1 -- 172.21.2.104:0/424681232 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f55180ac760 con 0x7f55180affb0 2021-07-11T03:53:51.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.613+0000 7f5507fff700 1 -- 172.21.2.104:0/424681232 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v11) v1 ==== 118+0+0 (secure 0 0 0) 0x7f55080472e0 con 0x7f55180affb0 2021-07-11T03:53:51.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 -- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54f4053270 msgr2=0x7f54f4055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:51.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54f4053270 0x7f54f4055730 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f550c002f70 tx=0x7f550c006990).stop 2021-07-11T03:53:51.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 -- 172.21.2.104:0/424681232 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55180affb0 msgr2=0x7f55180ae810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:51.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55180affb0 0x7f55180ae810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f551806a0f0 tx=0x7f5508015000).stop 2021-07-11T03:53:51.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 -- 172.21.2.104:0/424681232 shutdown_connections 2021-07-11T03:53:51.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f55180b1ec0 0x7f55180aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54f4053270 0x7f54f4055730 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:51.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55180affb0 0x7f55180ae810 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:51.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 --2- 172.21.2.104:0/424681232 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f55180a70c0 0x7f5518140580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 -- 172.21.2.104:0/424681232 >> 172.21.2.104:0/424681232 conn(0x7f551805f6b0 msgr2=0x7f551813e300 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:51.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 -- 172.21.2.104:0/424681232 shutdown_connections 2021-07-11T03:53:51.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.617+0000 7f551d9fe700 1 -- 172.21.2.104:0/424681232 wait complete. 2021-07-11T03:53:51.629 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:53:51.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.677+0000 7f9cf1d11700 1 Processor -- start 2021-07-11T03:53:51.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.677+0000 7f9cf1d11700 1 -- start start 2021-07-11T03:53:51.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.677+0000 7f9cf1d11700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9cec0a76b0 0x7f9cec066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.677+0000 7f9cf1d11700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cec0b0060 0x7f9cec05fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.677+0000 7f9cf1d11700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cec0a7a90 con 0x7f9cec0b1970 2021-07-11T03:53:51.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.677+0000 7f9cf1d11700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cec147620 con 0x7f9cec0a76b0 2021-07-11T03:53:51.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.677+0000 7f9cf1d11700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec1478c0 con 0x7f9cec0b0060 2021-07-11T03:53:51.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.677+0000 7f9ceb7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9cec0a76b0 0x7f9cec066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:51.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.677+0000 7f9ceaffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f9cec0b1970 0x7f9cec05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:39980/0 (socket says 172.21.2.104:39980) 2021-07-11T03:53:51.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.677+0000 7f9ceaffd700 1 -- 172.21.2.104:0/1894976423 learned_addr learned my addr 172.21.2.104:0/1894976423 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:51.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceb7fe700 1 -- 172.21.2.104:0/1894976423 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cec0b0060 msgr2=0x7f9cec05fdc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:51.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceb7fe700 1 --2- 172.21.2.104:0/1894976423 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cec0b0060 0x7f9cec05fdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceb7fe700 1 -- 172.21.2.104:0/1894976423 >> v1:172.21.2.104:6789/0 conn(0x7f9cec0b1970 legacy=0x7f9cec05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:51.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceb7fe700 1 -- 172.21.2.104:0/1894976423 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec147b60 con 0x7f9cec0a76b0 2021-07-11T03:53:51.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cea7fc700 1 -- 172.21.2.104:0/1894976423 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195276176 0 0) 0x7f9cec0a7a90 con 0x7f9cec0b1970 2021-07-11T03:53:51.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceb7fe700 1 --2- 172.21.2.104:0/1894976423 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cec0a76b0 0x7f9cec066a70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f9cdc002f70 tx=0x7f9cdc006df0).ready entity=mon.0 client_cookie=5b76613483f75b17 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:51.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cea7fc700 1 -- 172.21.2.104:0/1894976423 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cdc00d570 con 0x7f9cec0a76b0 2021-07-11T03:53:51.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cea7fc700 1 -- 172.21.2.104:0/1894976423 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc000e90 con 0x7f9cec0a76b0 2021-07-11T03:53:51.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cea7fc700 1 -- 172.21.2.104:0/1894976423 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cdc00d900 con 0x7f9cec0a76b0 2021-07-11T03:53:51.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- 172.21.2.104:0/1894976423 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cec0a76b0 msgr2=0x7f9cec066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:51.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 --2- 172.21.2.104:0/1894976423 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cec0a76b0 0x7f9cec066a70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f9cdc002f70 tx=0x7f9cdc006df0).stop 2021-07-11T03:53:51.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- 172.21.2.104:0/1894976423 shutdown_connections 2021-07-11T03:53:51.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 --2- 172.21.2.104:0/1894976423 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cec0b0060 0x7f9cec05fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 --2- 172.21.2.104:0/1894976423 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cec0a76b0 0x7f9cec066a70 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:51.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- 172.21.2.104:0/1894976423 >> 172.21.2.104:0/1894976423 conn(0x7f9cec0b53e0 msgr2=0x7f9cec0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:51.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- 172.21.2.104:0/1894976423 shutdown_connections 2021-07-11T03:53:51.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- 172.21.2.104:0/1894976423 wait complete. 2021-07-11T03:53:51.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 Processor -- start 2021-07-11T03:53:51.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- start start 2021-07-11T03:53:51.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cec0a76b0 0x7f9cec0b3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cec0b0060 0x7f9cec0ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cec0b1970 0x7f9cec0aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0af240 con 0x7f9cec0b0060 2021-07-11T03:53:51.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0adf50 con 0x7f9cec0a76b0 2021-07-11T03:53:51.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec0aceb0 con 0x7f9cec0b1970 2021-07-11T03:53:51.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceaffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cec0b0060 0x7f9cec0ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:51.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceaffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cec0b0060 0x7f9cec0ae860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43216/0 (socket says 172.21.2.104:43216) 2021-07-11T03:53:51.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceaffd700 1 -- 172.21.2.104:0/2355474649 learned_addr learned my addr 172.21.2.104:0/2355474649 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:51.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceaffd700 1 -- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cec0b1970 msgr2=0x7f9cec0aed50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:51.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceaffd700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cec0b1970 0x7f9cec0aed50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceaffd700 1 -- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cec0a76b0 msgr2=0x7f9cec0b3430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:51.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceaffd700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cec0a76b0 0x7f9cec0b3430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceaffd700 1 -- 172.21.2.104:0/2355474649 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc00b180 con 0x7f9cec0b0060 2021-07-11T03:53:51.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ceaffd700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cec0b0060 0x7f9cec0ae860 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f9ce0000bc0 tx=0x7f9ce0007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:51.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ce8ff9700 1 -- 172.21.2.104:0/2355474649 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ce0001d70 con 0x7f9cec0b0060 2021-07-11T03:53:51.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ce8ff9700 1 -- 172.21.2.104:0/2355474649 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce000ce80 con 0x7f9cec0b0060 2021-07-11T03:53:51.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ce8ff9700 1 -- 172.21.2.104:0/2355474649 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ce00074d0 con 0x7f9cec0b0060 2021-07-11T03:53:51.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- 172.21.2.104:0/2355474649 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec0ad150 con 0x7f9cec0b0060 2021-07-11T03:53:51.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- 172.21.2.104:0/2355474649 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec0ab2d0 con 0x7f9cec0b0060 2021-07-11T03:53:51.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9cf1d11700 1 -- 172.21.2.104:0/2355474649 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cec057350 con 0x7f9cec0b0060 2021-07-11T03:53:51.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ce8ff9700 1 -- 172.21.2.104:0/2355474649 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9ce0010070 con 0x7f9cec0b0060 2021-07-11T03:53:51.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.681+0000 7f9ce8ff9700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ccc0531f0 0x7f9ccc0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.685+0000 7f9ce8ff9700 1 -- 172.21.2.104:0/2355474649 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9ce006ac60 con 0x7f9cec0b0060 2021-07-11T03:53:51.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.685+0000 7f9ceb7fe700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ccc0531f0 0x7f9ccc0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:51.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.685+0000 7f9ceb7fe700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ccc0531f0 0x7f9ccc0556b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f9cdc002f70 tx=0x7f9cdc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:51.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.685+0000 7f9ce8ff9700 1 -- 172.21.2.104:0/2355474649 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9ce0041e90 con 0x7f9cec0b0060 2021-07-11T03:53:51.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 -- 172.21.2.104:0/2355474649 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cec0698e0 con 0x7f9cec0b0060 2021-07-11T03:53:51.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9ce8ff9700 1 -- 172.21.2.104:0/2355474649 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f9ce00454b0 con 0x7f9cec0b0060 2021-07-11T03:53:51.798 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:51.799 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4408,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2474981992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2474981992},{"type":"v1","addr":"172.21.2.105:6835","nonce":2474981992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/1241687766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1241687766},{"type":"v1","addr":"172.21.2.104:6834","nonce":1241687766}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:50.608622+0000","modified":"2021-07-11T03:53:51.611729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/1104530072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1104530072},{"type":"v1","addr":"172.21.2.104:6835","nonce":1104530072}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-07-11T03:53:51.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 -- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ccc0531f0 msgr2=0x7f9ccc0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:51.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ccc0531f0 0x7f9ccc0556b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f9cdc002f70 tx=0x7f9cdc015000).stop 2021-07-11T03:53:51.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 -- 172.21.2.104:0/2355474649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cec0b0060 msgr2=0x7f9cec0ae860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:51.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cec0b0060 0x7f9cec0ae860 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f9ce0000bc0 tx=0x7f9ce0007c30).stop 2021-07-11T03:53:51.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 -- 172.21.2.104:0/2355474649 shutdown_connections 2021-07-11T03:53:51.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cec0b1970 0x7f9cec0aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ccc0531f0 0x7f9ccc0556b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:51.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cec0b0060 0x7f9cec0ae860 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:51.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 --2- 172.21.2.104:0/2355474649 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cec0a76b0 0x7f9cec0b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 -- 172.21.2.104:0/2355474649 >> 172.21.2.104:0/2355474649 conn(0x7f9cec0b53e0 msgr2=0x7f9cec066820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:51.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 -- 172.21.2.104:0/2355474649 shutdown_connections 2021-07-11T03:53:51.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.793+0000 7f9cf1d11700 1 -- 172.21.2.104:0/2355474649 wait complete. 2021-07-11T03:53:51.803 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 11 2021-07-11T03:53:51.806 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:51.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.849+0000 7fa9d4216700 1 Processor -- start 2021-07-11T03:53:51.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.849+0000 7fa9d4216700 1 -- start start 2021-07-11T03:53:51.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.849+0000 7fa9d4216700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa9cc1393d0 0x7fa9cc0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.849+0000 7fa9d4216700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa9cc142840 0x7fa9cc142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.849+0000 7fa9d4216700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9cc0604e0 con 0x7fa9cc060100 2021-07-11T03:53:51.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.849+0000 7fa9d4216700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9cc0b16b0 con 0x7fa9cc1393d0 2021-07-11T03:53:51.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.849+0000 7fa9d4216700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9cc0ac400 con 0x7fa9cc142840 2021-07-11T03:53:51.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d17b1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa9cc1393d0 0x7fa9cc0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:51.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d17b1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa9cc1393d0 0x7fa9cc0b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43228/0 (socket says 172.21.2.104:43228) 2021-07-11T03:53:51.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d17b1700 1 -- 172.21.2.104:0/711375202 learned_addr learned my addr 172.21.2.104:0/711375202 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:51.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d17b1700 1 -- 172.21.2.104:0/711375202 >> v2:172.21.2.105:3300/0 conn(0x7fa9cc142840 msgr2=0x7fa9cc142ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:51.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d17b1700 1 --2- 172.21.2.104:0/711375202 >> v2:172.21.2.105:3300/0 conn(0x7fa9cc142840 0x7fa9cc142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d17b1700 1 -- 172.21.2.104:0/711375202 >> v1:172.21.2.105:6789/0 conn(0x7fa9cc060100 legacy=0x7fa9cc138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:51.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d17b1700 1 -- 172.21.2.104:0/711375202 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc0ac6a0 con 0x7fa9cc1393d0 2021-07-11T03:53:51.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d17b1700 1 --2- 172.21.2.104:0/711375202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9cc1393d0 0x7fa9cc0b11c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fa9cc061a70 tx=0x7fa9c8006e40).ready entity=mon.0 client_cookie=bfbdfbb7a4bcbd0f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:51.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d0fb0700 1 -- 172.21.2.104:0/711375202 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9c800bcb0 con 0x7fa9cc1393d0 2021-07-11T03:53:51.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d0fb0700 1 -- 172.21.2.104:0/711375202 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c8000e90 con 0x7fa9cc1393d0 2021-07-11T03:53:51.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d0fb0700 1 -- 172.21.2.104:0/711375202 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9c80153f0 con 0x7fa9cc1393d0 2021-07-11T03:53:51.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 -- 172.21.2.104:0/711375202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9cc1393d0 msgr2=0x7fa9cc0b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:51.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 --2- 172.21.2.104:0/711375202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9cc1393d0 0x7fa9cc0b11c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fa9cc061a70 tx=0x7fa9c8006e40).stop 2021-07-11T03:53:51.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 -- 172.21.2.104:0/711375202 shutdown_connections 2021-07-11T03:53:51.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 --2- 172.21.2.104:0/711375202 >> v2:172.21.2.105:3300/0 conn(0x7fa9cc142840 0x7fa9cc142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 --2- 172.21.2.104:0/711375202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9cc1393d0 0x7fa9cc0b11c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:51.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 -- 172.21.2.104:0/711375202 >> 172.21.2.104:0/711375202 conn(0x7fa9cc061070 msgr2=0x7fa9cc05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:51.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 -- 172.21.2.104:0/711375202 shutdown_connections 2021-07-11T03:53:51.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 -- 172.21.2.104:0/711375202 wait complete. 2021-07-11T03:53:51.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 Processor -- start 2021-07-11T03:53:51.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 -- start start 2021-07-11T03:53:51.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9cc060100 0x7fa9cc135c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9cc1393d0 0x7fa9cc136110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9cc142840 0x7fa9cc064780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc0afc30 con 0x7fa9cc060100 2021-07-11T03:53:51.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc14e9b0 con 0x7fa9cc1393d0 2021-07-11T03:53:51.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc14ec50 con 0x7fa9cc142840 2021-07-11T03:53:51.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d1fb2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9cc060100 0x7fa9cc135c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:51.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d1fb2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9cc060100 0x7fa9cc135c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43230/0 (socket says 172.21.2.104:43230) 2021-07-11T03:53:51.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d1fb2700 1 -- 172.21.2.104:0/3155142353 learned_addr learned my addr 172.21.2.104:0/3155142353 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:51.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d1fb2700 1 -- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9cc142840 msgr2=0x7fa9cc064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:51.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d1fb2700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9cc142840 0x7fa9cc064780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d1fb2700 1 -- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9cc1393d0 msgr2=0x7fa9cc136110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:51.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d1fb2700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9cc1393d0 0x7fa9cc136110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d1fb2700 1 -- 172.21.2.104:0/3155142353 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c800d4d0 con 0x7fa9cc060100 2021-07-11T03:53:51.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d1fb2700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9cc060100 0x7fa9cc135c20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fa9bc002f90 tx=0x7fa9bc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:51.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9baffd700 1 -- 172.21.2.104:0/3155142353 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9bc013460 con 0x7fa9cc060100 2021-07-11T03:53:51.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9baffd700 1 -- 172.21.2.104:0/3155142353 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9bc0135c0 con 0x7fa9cc060100 2021-07-11T03:53:51.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 -- 172.21.2.104:0/3155142353 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc062e60 con 0x7fa9cc060100 2021-07-11T03:53:51.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9baffd700 1 -- 172.21.2.104:0/3155142353 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9bc013920 con 0x7fa9cc060100 2021-07-11T03:53:51.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.853+0000 7fa9d4216700 1 -- 172.21.2.104:0/3155142353 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc136600 con 0x7fa9cc060100 2021-07-11T03:53:51.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.857+0000 7fa9d4216700 1 -- 172.21.2.104:0/3155142353 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc14ee10 con 0x7fa9cc060100 2021-07-11T03:53:51.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.857+0000 7fa9baffd700 1 -- 172.21.2.104:0/3155142353 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa9bc01a060 con 0x7fa9cc060100 2021-07-11T03:53:51.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.857+0000 7fa9baffd700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9ac0531f0 0x7fa9ac0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:51.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.857+0000 7fa9baffd700 1 -- 172.21.2.104:0/3155142353 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa9bc06af90 con 0x7fa9cc060100 2021-07-11T03:53:51.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.857+0000 7fa9d17b1700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9ac0531f0 0x7fa9ac0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:51.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.857+0000 7fa9d17b1700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9ac0531f0 0x7fa9ac0556b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fa9c8006fc0 tx=0x7fa9c8018000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:51.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.857+0000 7fa9baffd700 1 -- 172.21.2.104:0/3155142353 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa9bc0421c0 con 0x7fa9cc060100 2021-07-11T03:53:51.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 -- 172.21.2.104:0/3155142353 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9cc057350 con 0x7fa9cc060100 2021-07-11T03:53:51.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9baffd700 1 -- 172.21.2.104:0/3155142353 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16038 (secure 0 0 0) 0x7fa9bc018020 con 0x7fa9cc060100 2021-07-11T03:53:51.946 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:51.946 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":33,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:51.612860+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:48.603543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:49.585828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:51.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 -- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9ac0531f0 msgr2=0x7fa9ac0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:51.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9ac0531f0 0x7fa9ac0556b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fa9c8006fc0 tx=0x7fa9c8018000).stop 2021-07-11T03:53:51.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 -- 172.21.2.104:0/3155142353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9cc060100 msgr2=0x7fa9cc135c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:51.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9cc060100 0x7fa9cc135c20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7fa9bc002f90 tx=0x7fa9bc006df0).stop 2021-07-11T03:53:51.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 -- 172.21.2.104:0/3155142353 shutdown_connections 2021-07-11T03:53:51.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9cc142840 0x7fa9cc064780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9cc1393d0 0x7fa9cc136110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:51.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9ac0531f0 0x7fa9ac0556b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:51.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 --2- 172.21.2.104:0/3155142353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9cc060100 0x7fa9cc135c20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:51.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 -- 172.21.2.104:0/3155142353 >> 172.21.2.104:0/3155142353 conn(0x7fa9cc061070 msgr2=0x7fa9cc142dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:51.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 -- 172.21.2.104:0/3155142353 shutdown_connections 2021-07-11T03:53:51.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:51.941+0000 7fa9d4216700 1 -- 172.21.2.104:0/3155142353 wait complete. 2021-07-11T03:53:51.954 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T03:53:52.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3eb7b0700 1 Processor -- start 2021-07-11T03:53:52.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3eb7b0700 1 -- start start 2021-07-11T03:53:52.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3eb7b0700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa3e4067180 0x7fa3e405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3eb7b0700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e40a7a90 con 0x7fa3e40b1970 2021-07-11T03:53:52.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3eb7b0700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e4147620 con 0x7fa3e40a76b0 2021-07-11T03:53:52.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3eb7b0700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e41478c0 con 0x7fa3e4067180 2021-07-11T03:53:52.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e9d4d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fa3e40b1970 0x7fa3e405fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40008/0 (socket says 172.21.2.104:40008) 2021-07-11T03:53:52.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e9d4d700 1 -- 172.21.2.104:0/1673851113 learned_addr learned my addr 172.21.2.104:0/1673851113 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:52.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770820202 0 0) 0x7fa3e40a7a90 con 0x7fa3e40b1970 2021-07-11T03:53:52.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3cc003260 con 0x7fa3e40b1970 2021-07-11T03:53:52.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e8d4b700 1 --2- 172.21.2.104:0/1673851113 >> v2:172.21.2.105:3300/0 conn(0x7fa3e4067180 0x7fa3e405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3549500654 0 0) 0x7fa3cc003260 con 0x7fa3e40b1970 2021-07-11T03:53:52.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 >> v2:172.21.2.105:3300/0 conn(0x7fa3e4067180 msgr2=0x7fa3e405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 --2- 172.21.2.104:0/1673851113 >> v2:172.21.2.105:3300/0 conn(0x7fa3e4067180 0x7fa3e405f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 >> v1:172.21.2.105:6789/0 conn(0x7fa3e40a76b0 legacy=0x7fa3e4066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e4147b60 con 0x7fa3e40b1970 2021-07-11T03:53:52.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fa3dc0019e0 con 0x7fa3e40b1970 2021-07-11T03:53:52.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 >> v1:172.21.2.104:6789/0 conn(0x7fa3e40b1970 legacy=0x7fa3e405fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 --2- 172.21.2.104:0/1673851113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3cc003750 0x7fa3cc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 --2- 172.21.2.104:0/1673851113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3cc004500 0x7fa3cc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 --2- 172.21.2.104:0/1673851113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3cc004ec0 0x7fa3cc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e41478c0 con 0x7fa3cc004ec0 2021-07-11T03:53:52.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc00bda0 con 0x7fa3cc003750 2021-07-11T03:53:52.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3cc00bf60 con 0x7fa3cc004500 2021-07-11T03:53:52.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e9d4d700 1 --2- 172.21.2.104:0/1673851113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3cc004ec0 0x7fa3cc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e954c700 1 --2- 172.21.2.104:0/1673851113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3cc003750 0x7fa3cc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e8d4b700 1 --2- 172.21.2.104:0/1673851113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3cc004500 0x7fa3cc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e9d4d700 1 -- 172.21.2.104:0/1673851113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3cc004500 msgr2=0x7fa3cc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e9d4d700 1 --2- 172.21.2.104:0/1673851113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3cc004500 0x7fa3cc004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e9d4d700 1 -- 172.21.2.104:0/1673851113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3cc003750 msgr2=0x7fa3cc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e9d4d700 1 --2- 172.21.2.104:0/1673851113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3cc003750 0x7fa3cc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e9d4d700 1 -- 172.21.2.104:0/1673851113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc003260 con 0x7fa3cc004ec0 2021-07-11T03:53:52.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e9d4d700 1 -- 172.21.2.104:0/1673851113 reap_dead start 2021-07-11T03:53:52.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3e9d4d700 1 --2- 172.21.2.104:0/1673851113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3cc004ec0 0x7fa3cc00b750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fa3e406a180 tx=0x7fa3e40a8c80).ready entity=mon.0 client_cookie=73e41466b66059e5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3dc00cc70 con 0x7fa3cc004ec0 2021-07-11T03:53:52.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc00cdd0 con 0x7fa3cc004ec0 2021-07-11T03:53:52.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3d7fff700 1 -- 172.21.2.104:0/1673851113 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3dc005b80 con 0x7fa3cc004ec0 2021-07-11T03:53:52.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/1673851113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3cc004ec0 msgr2=0x7fa3cc00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.005+0000 7fa3eb7b0700 1 --2- 172.21.2.104:0/1673851113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3cc004ec0 0x7fa3cc00b750 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fa3e406a180 tx=0x7fa3e40a8c80).stop 2021-07-11T03:53:52.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/1673851113 shutdown_connections 2021-07-11T03:53:52.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 --2- 172.21.2.104:0/1673851113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3cc004ec0 0x7fa3cc00b750 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/1673851113 >> 172.21.2.104:0/1673851113 conn(0x7fa3e40b53e0 msgr2=0x7fa3e40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:52.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/1673851113 shutdown_connections 2021-07-11T03:53:52.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/1673851113 wait complete. 2021-07-11T03:53:52.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 Processor -- start 2021-07-11T03:53:52.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 -- start start 2021-07-11T03:53:52.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3cc004ec0 0x7fa3e414d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3e40ac520 0x7fa3e4144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3e415a210 0x7fa3e4144a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e40ad120 con 0x7fa3e40ac520 2021-07-11T03:53:52.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e40aef40 con 0x7fa3e415a210 2021-07-11T03:53:52.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e40af1e0 con 0x7fa3cc004ec0 2021-07-11T03:53:52.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3e8d4b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3e40ac520 0x7fa3e4144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3e8d4b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3e40ac520 0x7fa3e4144570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43250/0 (socket says 172.21.2.104:43250) 2021-07-11T03:53:52.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3e8d4b700 1 -- 172.21.2.104:0/2696472199 learned_addr learned my addr 172.21.2.104:0/2696472199 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:52.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3e954c700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3cc004ec0 0x7fa3e414d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3e8d4b700 1 -- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3cc004ec0 msgr2=0x7fa3e414d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3e8d4b700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3cc004ec0 0x7fa3e414d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3e8d4b700 1 -- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3e415a210 msgr2=0x7fa3e4144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3e8d4b700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3e415a210 0x7fa3e4144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3e8d4b700 1 -- 172.21.2.104:0/2696472199 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e40af480 con 0x7fa3e40ac520 2021-07-11T03:53:52.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3e8d4b700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3e40ac520 0x7fa3e4144570 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fa3d8008000 tx=0x7fa3d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3d67fc700 1 -- 172.21.2.104:0/2696472199 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d8014400 con 0x7fa3e40ac520 2021-07-11T03:53:52.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3d67fc700 1 -- 172.21.2.104:0/2696472199 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d8000e90 con 0x7fa3e40ac520 2021-07-11T03:53:52.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/2696472199 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e414e720 con 0x7fa3e40ac520 2021-07-11T03:53:52.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/2696472199 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e40adbe0 con 0x7fa3e40ac520 2021-07-11T03:53:52.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3d67fc700 1 -- 172.21.2.104:0/2696472199 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa3d8014750 con 0x7fa3e40ac520 2021-07-11T03:53:52.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/2696472199 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c0003410 con 0x7fa3e40ac520 2021-07-11T03:53:52.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3d67fc700 1 -- 172.21.2.104:0/2696472199 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa3d8012070 con 0x7fa3e40ac520 2021-07-11T03:53:52.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3d67fc700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3c4053350 0x7fa3c4055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.009+0000 7fa3d67fc700 1 -- 172.21.2.104:0/2696472199 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa3d806aa50 con 0x7fa3e40ac520 2021-07-11T03:53:52.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.013+0000 7fa3e954c700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3c4053350 0x7fa3c4055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.013+0000 7fa3e954c700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3c4053350 0x7fa3c4055810 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fa3e0009000 tx=0x7fa3e0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.013+0000 7fa3d67fc700 1 -- 172.21.2.104:0/2696472199 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa3d8041c80 con 0x7fa3e40ac520 2021-07-11T03:53:52.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.121+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/2696472199 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fa3c00036c0 con 0x7fa3e40ac520 2021-07-11T03:53:52.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3d67fc700 1 -- 172.21.2.104:0/2696472199 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v13) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa3d80452a0 con 0x7fa3e40ac520 2021-07-11T03:53:52.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3c4053350 msgr2=0x7fa3c4055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3c4053350 0x7fa3c4055810 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fa3e0009000 tx=0x7fa3e0009040).stop 2021-07-11T03:53:52.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/2696472199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3e40ac520 msgr2=0x7fa3e4144570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3e40ac520 0x7fa3e4144570 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fa3d8008000 tx=0x7fa3d8008040).stop 2021-07-11T03:53:52.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/2696472199 shutdown_connections 2021-07-11T03:53:52.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa3e415a210 0x7fa3e4144a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa3c4053350 0x7fa3c4055810 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa3e40ac520 0x7fa3e4144570 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 --2- 172.21.2.104:0/2696472199 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa3cc004ec0 0x7fa3e414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/2696472199 >> 172.21.2.104:0/2696472199 conn(0x7fa3e40b53e0 msgr2=0x7fa3e4148a00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:52.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/2696472199 shutdown_connections 2021-07-11T03:53:52.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.129+0000 7fa3eb7b0700 1 -- 172.21.2.104:0/2696472199 wait complete. 2021-07-11T03:53:52.137 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T03:53:52.141 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T03:53:52.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.189+0000 7f70cf491700 1 Processor -- start 2021-07-11T03:53:52.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.189+0000 7f70cf491700 1 -- start start 2021-07-11T03:53:52.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.189+0000 7f70cf491700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f70c80b1970 0x7f70c805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.189+0000 7f70cf491700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c80a7a90 con 0x7f70c80a76b0 2021-07-11T03:53:52.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.189+0000 7f70cf491700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70c8147620 con 0x7f70c8067180 2021-07-11T03:53:52.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.189+0000 7f70cf491700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70c81478c0 con 0x7f70c80b1970 2021-07-11T03:53:52.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.189+0000 7f70cd22d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f70c80a76b0 0x7f70c8066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40028/0 (socket says 172.21.2.104:40028) 2021-07-11T03:53:52.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.189+0000 7f70cd22d700 1 -- 172.21.2.104:0/1355537203 learned_addr learned my addr 172.21.2.104:0/1355537203 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:52.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229077072 0 0) 0x7f70c80a7a90 con 0x7f70c80a76b0 2021-07-11T03:53:52.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70b0003260 con 0x7f70c80a76b0 2021-07-11T03:53:52.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 --2- 172.21.2.104:0/1355537203 >> v2:172.21.2.105:3300/0 conn(0x7f70c80b1970 0x7f70c805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2156649610 0 0) 0x7f70b0003260 con 0x7f70c80a76b0 2021-07-11T03:53:52.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 >> v2:172.21.2.105:3300/0 conn(0x7f70c80b1970 msgr2=0x7f70c805fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 --2- 172.21.2.104:0/1355537203 >> v2:172.21.2.105:3300/0 conn(0x7f70c80b1970 0x7f70c805fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 >> v1:172.21.2.105:6789/0 conn(0x7f70c8067180 legacy=0x7f70c805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c8147b60 con 0x7f70c80a76b0 2021-07-11T03:53:52.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f70c4001660 con 0x7f70c80a76b0 2021-07-11T03:53:52.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 >> v1:172.21.2.104:6789/0 conn(0x7f70c80a76b0 legacy=0x7f70c8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 --2- 172.21.2.104:0/1355537203 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f70b0003750 0x7f70b0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 --2- 172.21.2.104:0/1355537203 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70b0004500 0x7f70b0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 --2- 172.21.2.104:0/1355537203 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70b0004e70 0x7f70b000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c81478c0 con 0x7f70b0004e70 2021-07-11T03:53:52.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b000bd50 con 0x7f70b0003750 2021-07-11T03:53:52.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b000bf10 con 0x7f70b0004500 2021-07-11T03:53:52.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 --2- 172.21.2.104:0/1355537203 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70b0004e70 0x7f70b000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 -- 172.21.2.104:0/1355537203 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70b0004500 msgr2=0x7f70b0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 --2- 172.21.2.104:0/1355537203 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70b0004500 0x7f70b0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 -- 172.21.2.104:0/1355537203 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f70b0003750 msgr2=0x7f70b0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 --2- 172.21.2.104:0/1355537203 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f70b0003750 0x7f70b0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 -- 172.21.2.104:0/1355537203 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c8147b60 con 0x7f70b0004e70 2021-07-11T03:53:52.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 -- 172.21.2.104:0/1355537203 reap_dead start 2021-07-11T03:53:52.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 --2- 172.21.2.104:0/1355537203 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70b0004e70 0x7f70b000b750 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f70c000a000 tx=0x7f70c0008040).ready entity=mon.0 client_cookie=9e4c1086ae771c2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70c000c070 con 0x7f70b0004e70 2021-07-11T03:53:52.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c000e810 con 0x7f70b0004e70 2021-07-11T03:53:52.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70bbfff700 1 -- 172.21.2.104:0/1355537203 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70c000eb70 con 0x7f70b0004e70 2021-07-11T03:53:52.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 -- 172.21.2.104:0/1355537203 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70b0004e70 msgr2=0x7f70b000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 --2- 172.21.2.104:0/1355537203 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70b0004e70 0x7f70b000b750 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f70c000a000 tx=0x7f70c0008040).stop 2021-07-11T03:53:52.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 -- 172.21.2.104:0/1355537203 shutdown_connections 2021-07-11T03:53:52.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 --2- 172.21.2.104:0/1355537203 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70b0004e70 0x7f70b000b750 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 -- 172.21.2.104:0/1355537203 >> 172.21.2.104:0/1355537203 conn(0x7f70c80b53e0 msgr2=0x7f70c80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:52.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 -- 172.21.2.104:0/1355537203 shutdown_connections 2021-07-11T03:53:52.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 -- 172.21.2.104:0/1355537203 wait complete. 2021-07-11T03:53:52.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 Processor -- start 2021-07-11T03:53:52.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 -- start start 2021-07-11T03:53:52.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70b0004e70 0x7f70c80ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f70c80ac520 0x7f70c8156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70c80a9230 0x7f70c80a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c80ad1e0 con 0x7f70c80a9230 2021-07-11T03:53:52.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c814e7b0 con 0x7f70c80ac520 2021-07-11T03:53:52.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cf491700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c814ea50 con 0x7f70b0004e70 2021-07-11T03:53:52.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70c80a9230 0x7f70c80a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70c80a9230 0x7f70c80a9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43272/0 (socket says 172.21.2.104:43272) 2021-07-11T03:53:52.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 -- 172.21.2.104:0/1044830751 learned_addr learned my addr 172.21.2.104:0/1044830751 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:52.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 -- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70b0004e70 msgr2=0x7f70c80ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70b0004e70 0x7f70c80ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 -- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f70c80ac520 msgr2=0x7f70c8156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f70c80ac520 0x7f70c8156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 -- 172.21.2.104:0/1044830751 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c81566f0 con 0x7f70c80a9230 2021-07-11T03:53:52.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.193+0000 7f70cda2e700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70c80a9230 0x7f70c80a9900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f70c000a000 tx=0x7f70c0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.197+0000 7f70ba7fc700 1 -- 172.21.2.104:0/1044830751 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70c0022070 con 0x7f70c80a9230 2021-07-11T03:53:52.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.197+0000 7f70ba7fc700 1 -- 172.21.2.104:0/1044830751 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c0013620 con 0x7f70c80a9230 2021-07-11T03:53:52.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.197+0000 7f70ba7fc700 1 -- 172.21.2.104:0/1044830751 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70c0013980 con 0x7f70c80a9230 2021-07-11T03:53:52.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.197+0000 7f70cf491700 1 -- 172.21.2.104:0/1044830751 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c81563e0 con 0x7f70c80a9230 2021-07-11T03:53:52.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.197+0000 7f70cf491700 1 -- 172.21.2.104:0/1044830751 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c8157080 con 0x7f70c80a9230 2021-07-11T03:53:52.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.197+0000 7f70cf491700 1 -- 172.21.2.104:0/1044830751 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c8156880 con 0x7f70c80a9230 2021-07-11T03:53:52.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.197+0000 7f70ba7fc700 1 -- 172.21.2.104:0/1044830751 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f70c0013ae0 con 0x7f70c80a9230 2021-07-11T03:53:52.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.197+0000 7f70ba7fc700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70a8053330 0x7f70a80557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.197+0000 7f70ba7fc700 1 -- 172.21.2.104:0/1044830751 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f70c006c790 con 0x7f70c80a9230 2021-07-11T03:53:52.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.197+0000 7f70cd22d700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70a8053330 0x7f70a80557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.197+0000 7f70cd22d700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70a8053330 0x7f70a80557f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f70c4001020 tx=0x7f70c4001590).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.201+0000 7f70ba7fc700 1 -- 172.21.2.104:0/1044830751 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f70c00439c0 con 0x7f70c80a9230 2021-07-11T03:53:52.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.309+0000 7f70cf491700 1 -- 172.21.2.104:0/1044830751 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f70c8057350 con 0x7f70c80a9230 2021-07-11T03:53:52.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.313+0000 7f70ba7fc700 1 -- 172.21.2.104:0/1044830751 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v14) v1 ==== 184+0+0 (secure 0 0 0) 0x7f70c0046fe0 con 0x7f70c80a9230 2021-07-11T03:53:52.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 -- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70a8053330 msgr2=0x7f70a80557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70a8053330 0x7f70a80557f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f70c4001020 tx=0x7f70c4001590).stop 2021-07-11T03:53:52.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 -- 172.21.2.104:0/1044830751 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70c80a9230 msgr2=0x7f70c80a9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70c80a9230 0x7f70c80a9900 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f70c000a000 tx=0x7f70c0008040).stop 2021-07-11T03:53:52.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 -- 172.21.2.104:0/1044830751 shutdown_connections 2021-07-11T03:53:52.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70a8053330 0x7f70a80557f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70c80a9230 0x7f70c80a9900 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f70c80ac520 0x7f70c8156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 --2- 172.21.2.104:0/1044830751 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70b0004e70 0x7f70c80ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 -- 172.21.2.104:0/1044830751 >> 172.21.2.104:0/1044830751 conn(0x7f70c80b53e0 msgr2=0x7f70c80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:52.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 -- 172.21.2.104:0/1044830751 shutdown_connections 2021-07-11T03:53:52.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.317+0000 7f70cf491700 1 -- 172.21.2.104:0/1044830751 wait complete. 2021-07-11T03:53:52.325 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T03:53:52.328 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:53:52.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f2661e700 1 Processor -- start 2021-07-11T03:53:52.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f2661e700 1 -- start start 2021-07-11T03:53:52.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f2661e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6f200a76b0 0x7f6f20066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f2661e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f200a7a90 con 0x7f6f200b0060 2021-07-11T03:53:52.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f2661e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f20147620 con 0x7f6f200b1970 2021-07-11T03:53:52.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f2661e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f201478c0 con 0x7f6f200a76b0 2021-07-11T03:53:52.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f24bbb700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6f200b0060 0x7f6f20151f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40046/0 (socket says 172.21.2.104:40046) 2021-07-11T03:53:52.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f24bbb700 1 -- 172.21.2.104:0/680839039 learned_addr learned my addr 172.21.2.104:0/680839039 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:52.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743026975 0 0) 0x7f6f200a7a90 con 0x7f6f200b0060 2021-07-11T03:53:52.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f00003260 con 0x7f6f200b0060 2021-07-11T03:53:52.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1ffff700 1 --2- 172.21.2.104:0/680839039 >> v2:172.21.2.105:3300/0 conn(0x7f6f200a76b0 0x7f6f20066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3605533372 0 0) 0x7f6f00003260 con 0x7f6f200b0060 2021-07-11T03:53:52.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 >> v2:172.21.2.105:3300/0 conn(0x7f6f200a76b0 msgr2=0x7f6f20066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 --2- 172.21.2.104:0/680839039 >> v2:172.21.2.105:3300/0 conn(0x7f6f200a76b0 0x7f6f20066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 >> v1:172.21.2.105:6789/0 conn(0x7f6f200b1970 legacy=0x7f6f2005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f20147b60 con 0x7f6f200b0060 2021-07-11T03:53:52.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f6f18001a00 con 0x7f6f200b0060 2021-07-11T03:53:52.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 >> v1:172.21.2.104:6789/0 conn(0x7f6f200b0060 legacy=0x7f6f20151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 --2- 172.21.2.104:0/680839039 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f00003820 0x7f6f00003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 --2- 172.21.2.104:0/680839039 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 0x7f6f00004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 --2- 172.21.2.104:0/680839039 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f00004f70 0x7f6f0000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f201478c0 con 0x7f6f00004500 2021-07-11T03:53:52.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0000be50 con 0x7f6f00004f70 2021-07-11T03:53:52.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0000c010 con 0x7f6f00003820 2021-07-11T03:53:52.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.373+0000 7f6f1f7fe700 1 --2- 172.21.2.104:0/680839039 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 0x7f6f00004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1f7fe700 1 -- 172.21.2.104:0/680839039 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f00003820 msgr2=0x7f6f00003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1f7fe700 1 --2- 172.21.2.104:0/680839039 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f00003820 0x7f6f00003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1f7fe700 1 -- 172.21.2.104:0/680839039 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f00004f70 msgr2=0x7f6f0000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1f7fe700 1 --2- 172.21.2.104:0/680839039 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f00004f70 0x7f6f0000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f24bbb700 1 -- 172.21.2.104:0/680839039 reap_dead start 2021-07-11T03:53:52.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1f7fe700 1 -- 172.21.2.104:0/680839039 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f00003260 con 0x7f6f00004500 2021-07-11T03:53:52.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1f7fe700 1 --2- 172.21.2.104:0/680839039 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 0x7f6f00004980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f6f08001fa0 tx=0x7f6f08007a00).ready entity=mon.0 client_cookie=8d36e8a06aebbc4a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f08007030 con 0x7f6f00004500 2021-07-11T03:53:52.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f08002e90 con 0x7f6f00004500 2021-07-11T03:53:52.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1effd700 1 -- 172.21.2.104:0/680839039 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f08007380 con 0x7f6f00004500 2021-07-11T03:53:52.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- 172.21.2.104:0/680839039 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 msgr2=0x7f6f00004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 --2- 172.21.2.104:0/680839039 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 0x7f6f00004980 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f6f08001fa0 tx=0x7f6f08007a00).stop 2021-07-11T03:53:52.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- 172.21.2.104:0/680839039 shutdown_connections 2021-07-11T03:53:52.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 --2- 172.21.2.104:0/680839039 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 0x7f6f00004980 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- 172.21.2.104:0/680839039 >> 172.21.2.104:0/680839039 conn(0x7f6f200b53e0 msgr2=0x7f6f200b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:52.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- 172.21.2.104:0/680839039 shutdown_connections 2021-07-11T03:53:52.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- 172.21.2.104:0/680839039 wait complete. 2021-07-11T03:53:52.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 Processor -- start 2021-07-11T03:53:52.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- start start 2021-07-11T03:53:52.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 0x7f6f20066610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f200a99c0 0x7f6f20066b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f200ad760 0x7f6f200adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200ae100 con 0x7f6f00004500 2021-07-11T03:53:52.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200aa630 con 0x7f6f200a99c0 2021-07-11T03:53:52.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f200aa8d0 con 0x7f6f200ad760 2021-07-11T03:53:52.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 0x7f6f20066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 0x7f6f20066610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43288/0 (socket says 172.21.2.104:43288) 2021-07-11T03:53:52.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1ffff700 1 -- 172.21.2.104:0/431677456 learned_addr learned my addr 172.21.2.104:0/431677456 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:52.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1ffff700 1 -- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f200ad760 msgr2=0x7f6f200adc10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:52.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1ffff700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f200ad760 0x7f6f200adc10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1ffff700 1 -- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f200a99c0 msgr2=0x7f6f20066b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1ffff700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f200a99c0 0x7f6f20066b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1ffff700 1 -- 172.21.2.104:0/431677456 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2015a6d0 con 0x7f6f00004500 2021-07-11T03:53:52.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1ffff700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 0x7f6f20066610 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f6f1000a000 tx=0x7f6f10008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1d7fa700 1 -- 172.21.2.104:0/431677456 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f10016070 con 0x7f6f00004500 2021-07-11T03:53:52.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1d7fa700 1 -- 172.21.2.104:0/431677456 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f10000e90 con 0x7f6f00004500 2021-07-11T03:53:52.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- 172.21.2.104:0/431677456 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f2015a970 con 0x7f6f00004500 2021-07-11T03:53:52.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1d7fa700 1 -- 172.21.2.104:0/431677456 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f1000dea0 con 0x7f6f00004500 2021-07-11T03:53:52.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- 172.21.2.104:0/431677456 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f200aac40 con 0x7f6f00004500 2021-07-11T03:53:52.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f2661e700 1 -- 172.21.2.104:0/431677456 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f2015a530 con 0x7f6f00004500 2021-07-11T03:53:52.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1d7fa700 1 -- 172.21.2.104:0/431677456 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6f10011970 con 0x7f6f00004500 2021-07-11T03:53:52.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1d7fa700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f04053180 0x7f6f04055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.377+0000 7f6f1d7fa700 1 -- 172.21.2.104:0/431677456 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6f1006b070 con 0x7f6f00004500 2021-07-11T03:53:52.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.381+0000 7f6f1f7fe700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f04053180 0x7f6f04055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.381+0000 7f6f1f7fe700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f04053180 0x7f6f04055640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f6f08001fa0 tx=0x7f6f08015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.381+0000 7f6f1d7fa700 1 -- 172.21.2.104:0/431677456 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6f2015a530 con 0x7f6f00004500 2021-07-11T03:53:52.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 -- 172.21.2.104:0/431677456 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f2015a530 con 0x7f6f00004500 2021-07-11T03:53:52.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f1d7fa700 1 -- 172.21.2.104:0/431677456 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f6f2015a530 con 0x7f6f00004500 2021-07-11T03:53:52.496 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:52.496 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4408,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2474981992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2474981992},{"type":"v1","addr":"172.21.2.105:6835","nonce":2474981992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/1241687766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1241687766},{"type":"v1","addr":"172.21.2.104:6834","nonce":1241687766}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:50.608622+0000","modified":"2021-07-11T03:53:51.611729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/1104530072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1104530072},{"type":"v1","addr":"172.21.2.104:6835","nonce":1104530072}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-07-11T03:53:52.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 -- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f04053180 msgr2=0x7f6f04055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f04053180 0x7f6f04055640 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f6f08001fa0 tx=0x7f6f08015000).stop 2021-07-11T03:53:52.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 -- 172.21.2.104:0/431677456 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 msgr2=0x7f6f20066610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 0x7f6f20066610 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f6f1000a000 tx=0x7f6f10008040).stop 2021-07-11T03:53:52.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 -- 172.21.2.104:0/431677456 shutdown_connections 2021-07-11T03:53:52.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f200ad760 0x7f6f200adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f200a99c0 0x7f6f20066b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f04053180 0x7f6f04055640 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 --2- 172.21.2.104:0/431677456 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f00004500 0x7f6f20066610 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 -- 172.21.2.104:0/431677456 >> 172.21.2.104:0/431677456 conn(0x7f6f200b53e0 msgr2=0x7f6f201447b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:52.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 -- 172.21.2.104:0/431677456 shutdown_connections 2021-07-11T03:53:52.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.489+0000 7f6f2661e700 1 -- 172.21.2.104:0/431677456 wait complete. 2021-07-11T03:53:52.500 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 11 2021-07-11T03:53:52.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:53:50.608622+0000', 'modified': '2021-07-11T03:53:51.611729+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4863}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4863': {'gid': 4863, 'name': 'a', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/1104530072', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 1104530072}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 1104530072}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:53:52.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:53:52.806 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-07-11T03:53:52.887 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":4863,"want_state":"up:active","state":"up:active","rank_uptime":2.271356661,"mdsmap_epoch":11,"osdmap_epoch":33,"osdmap_epoch_barrier":32,"uptime":7.4452262129999998} 2021-07-11T03:53:52.888 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 4863, "mdsmap_epoch": 11, "osdmap_epoch": 33, "osdmap_epoch_barrier": 32, "rank_uptime": 2.271356661, "state": "up:active", "uptime": 7.445226213, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:53:52.889 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:53:52.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a9f4a700 1 Processor -- start 2021-07-11T03:53:52.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a9f4a700 1 -- start start 2021-07-11T03:53:52.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a9f4a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53a40a76b0 0x7f53a4066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a9f4a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f53a40b1970 0x7f53a4066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a9f4a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a4067450 con 0x7f53a40b0060 2021-07-11T03:53:52.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a9f4a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a40a7a90 con 0x7f53a40b1970 2021-07-11T03:53:52.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a9f4a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4147620 con 0x7f53a40a76b0 2021-07-11T03:53:52.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a3fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f53a40b0060 0x7f53a405f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40068/0 (socket says 172.21.2.104:40068) 2021-07-11T03:53:52.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a3fff700 1 -- 172.21.2.104:0/2567624886 learned_addr learned my addr 172.21.2.104:0/2567624886 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:52.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278944802 0 0) 0x7f53a4067450 con 0x7f53a40b0060 2021-07-11T03:53:52.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5384003260 con 0x7f53a40b0060 2021-07-11T03:53:52.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a37fe700 1 --2- 172.21.2.104:0/2567624886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53a40a76b0 0x7f53a4066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a2ffd700 1 --2- 172.21.2.104:0/2567624886 >> v2:172.21.2.105:3300/0 conn(0x7f53a40b1970 0x7f53a4066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 528247239 0 0) 0x7f5384003260 con 0x7f53a40b0060 2021-07-11T03:53:52.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53a40a76b0 msgr2=0x7f53a4066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 --2- 172.21.2.104:0/2567624886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53a40a76b0 0x7f53a4066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 >> v2:172.21.2.105:3300/0 conn(0x7f53a40b1970 msgr2=0x7f53a4066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 --2- 172.21.2.104:0/2567624886 >> v2:172.21.2.105:3300/0 conn(0x7f53a40b1970 0x7f53a4066f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:52.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a41478c0 con 0x7f53a40b0060 2021-07-11T03:53:52.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f539c0019e0 con 0x7f53a40b0060 2021-07-11T03:53:52.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 >> v1:172.21.2.104:6789/0 conn(0x7f53a40b0060 legacy=0x7f53a405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 --2- 172.21.2.104:0/2567624886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53a40a76b0 0x7f53840037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 --2- 172.21.2.104:0/2567624886 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5384003e00 0x7f5384004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 --2- 172.21.2.104:0/2567624886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53840049f0 0x7f5384004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a40a7a90 con 0x7f53840049f0 2021-07-11T03:53:52.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4147620 con 0x7f5384003e00 2021-07-11T03:53:52.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538400b8e0 con 0x7f53a40a76b0 2021-07-11T03:53:52.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a3fff700 1 --2- 172.21.2.104:0/2567624886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53840049f0 0x7f5384004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.933+0000 7f53a3fff700 1 -- 172.21.2.104:0/2567624886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53a40a76b0 msgr2=0x7f53840037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 --2- 172.21.2.104:0/2567624886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53a40a76b0 0x7f53840037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 -- 172.21.2.104:0/2567624886 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5384003e00 msgr2=0x7f5384004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 --2- 172.21.2.104:0/2567624886 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5384003e00 0x7f5384004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 -- 172.21.2.104:0/2567624886 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5384003260 con 0x7f53840049f0 2021-07-11T03:53:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 --2- 172.21.2.104:0/2567624886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53840049f0 0x7f5384004e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f539c0016f0 tx=0x7f539c002100).ready entity=mon.0 client_cookie=94fccb446d1933ab server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f539c00cb80 con 0x7f53840049f0 2021-07-11T03:53:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539c00cce0 con 0x7f53840049f0 2021-07-11T03:53:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a27fc700 1 -- 172.21.2.104:0/2567624886 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f539c005b80 con 0x7f53840049f0 2021-07-11T03:53:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- 172.21.2.104:0/2567624886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53840049f0 msgr2=0x7f5384004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 --2- 172.21.2.104:0/2567624886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53840049f0 0x7f5384004e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f539c0016f0 tx=0x7f539c002100).stop 2021-07-11T03:53:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 -- 172.21.2.104:0/2567624886 reap_dead start 2021-07-11T03:53:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- 172.21.2.104:0/2567624886 shutdown_connections 2021-07-11T03:53:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- 172.21.2.104:0/2567624886 >> 172.21.2.104:0/2567624886 conn(0x7f53a40b53e0 msgr2=0x7f53a40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- 172.21.2.104:0/2567624886 shutdown_connections 2021-07-11T03:53:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- 172.21.2.104:0/2567624886 wait complete. 2021-07-11T03:53:52.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 Processor -- start 2021-07-11T03:53:52.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- start start 2021-07-11T03:53:52.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53a414d1a0 0x7f53a41448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53a40aa560 0x7f53a4144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53a4156a40 0x7f53a4156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a4156650 con 0x7f53a4156a40 2021-07-11T03:53:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a414d580 con 0x7f53a40aa560 2021-07-11T03:53:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a40abb40 con 0x7f53a414d1a0 2021-07-11T03:53:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53a4156a40 0x7f53a4156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53a4156a40 0x7f53a4156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43310/0 (socket says 172.21.2.104:43310) 2021-07-11T03:53:52.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 -- 172.21.2.104:0/3493213335 learned_addr learned my addr 172.21.2.104:0/3493213335 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:52.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 -- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53a414d1a0 msgr2=0x7f53a41448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53a414d1a0 0x7f53a41448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 -- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53a40aa560 msgr2=0x7f53a4144d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53a40aa560 0x7f53a4144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 -- 172.21.2.104:0/3493213335 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a40abde0 con 0x7f53a4156a40 2021-07-11T03:53:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a3fff700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53a4156a40 0x7f53a4156ec0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f53a40623b0 tx=0x7f539c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a0ff9700 1 -- 172.21.2.104:0/3493213335 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f539c017020 con 0x7f53a4156a40 2021-07-11T03:53:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a0ff9700 1 -- 172.21.2.104:0/3493213335 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539c007aa0 con 0x7f53a4156a40 2021-07-11T03:53:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- 172.21.2.104:0/3493213335 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a414d8e0 con 0x7f53a4156a40 2021-07-11T03:53:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a0ff9700 1 -- 172.21.2.104:0/3493213335 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f539c004d60 con 0x7f53a4156a40 2021-07-11T03:53:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- 172.21.2.104:0/3493213335 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a40aac10 con 0x7f53a4156a40 2021-07-11T03:53:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.937+0000 7f53a9f4a700 1 -- 172.21.2.104:0/3493213335 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a4057350 con 0x7f53a4156a40 2021-07-11T03:53:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.941+0000 7f53a0ff9700 1 -- 172.21.2.104:0/3493213335 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f539c010070 con 0x7f53a4156a40 2021-07-11T03:53:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.941+0000 7f53a0ff9700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f53880532d0 0x7f5388055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.941+0000 7f53a0ff9700 1 -- 172.21.2.104:0/3493213335 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f539c06bd90 con 0x7f53a4156a40 2021-07-11T03:53:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.941+0000 7f53a37fe700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f53880532d0 0x7f5388055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.941+0000 7f53a37fe700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f53880532d0 0x7f5388055790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f538c00a000 tx=0x7f538c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:52.941+0000 7f53a0ff9700 1 -- 172.21.2.104:0/3493213335 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f539c042fc0 con 0x7f53a4156a40 2021-07-11T03:53:53.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 -- 172.21.2.104:0/3493213335 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53a40abf70 con 0x7f53a4156a40 2021-07-11T03:53:53.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a0ff9700 1 -- 172.21.2.104:0/3493213335 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f539c0465e0 con 0x7f53a4156a40 2021-07-11T03:53:53.030 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:53.031 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":33,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:53:51.612860+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:53:48.603543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2021-07-11T03:53:49.585828+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:53:53.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 -- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f53880532d0 msgr2=0x7f5388055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f53880532d0 0x7f5388055790 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f538c00a000 tx=0x7f538c008040).stop 2021-07-11T03:53:53.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 -- 172.21.2.104:0/3493213335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53a4156a40 msgr2=0x7f53a4156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53a4156a40 0x7f53a4156ec0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f53a40623b0 tx=0x7f539c015000).stop 2021-07-11T03:53:53.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 -- 172.21.2.104:0/3493213335 shutdown_connections 2021-07-11T03:53:53.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f53880532d0 0x7f5388055790 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f53a4156a40 0x7f53a4156ec0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f53a40aa560 0x7f53a4144d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 --2- 172.21.2.104:0/3493213335 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f53a414d1a0 0x7f53a41448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 -- 172.21.2.104:0/3493213335 >> 172.21.2.104:0/3493213335 conn(0x7f53a40b53e0 msgr2=0x7f53a40a7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:53.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 -- 172.21.2.104:0/3493213335 shutdown_connections 2021-07-11T03:53:53.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.025+0000 7f53a9f4a700 1 -- 172.21.2.104:0/3493213335 wait complete. 2021-07-11T03:53:53.038 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:53:48.603543+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:53:49.585828+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:53:53.039 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:53:53.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.085+0000 7f43d34e4700 1 Processor -- start 2021-07-11T03:53:53.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.085+0000 7f43d34e4700 1 -- start start 2021-07-11T03:53:53.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.085+0000 7f43d34e4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43cc0b12d0 0x7f43cc0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.085+0000 7f43d34e4700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f43cc0b22a0 0x7f43cc141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.085+0000 7f43d34e4700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43cc0a77d0 con 0x7f43cc0b1ec0 2021-07-11T03:53:53.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.085+0000 7f43d34e4700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43cc0b1740 con 0x7f43cc0b22a0 2021-07-11T03:53:53.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.085+0000 7f43d34e4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0b0650 con 0x7f43cc0b12d0 2021-07-11T03:53:53.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1280700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f43cc0b1ec0 0x7f43cc0a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40084/0 (socket says 172.21.2.104:40084) 2021-07-11T03:53:53.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1280700 1 -- 172.21.2.104:0/1906592145 learned_addr learned my addr 172.21.2.104:0/1906592145 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:53.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843262904 0 0) 0x7f43cc0a77d0 con 0x7f43cc0b1ec0 2021-07-11T03:53:53.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43b4003260 con 0x7f43cc0b1ec0 2021-07-11T03:53:53.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1a81700 1 --2- 172.21.2.104:0/1906592145 >> v2:172.21.2.105:3300/0 conn(0x7f43cc0b22a0 0x7f43cc141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d0a7f700 1 --2- 172.21.2.104:0/1906592145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43cc0b12d0 0x7f43cc0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 192134720 0 0) 0x7f43b4003260 con 0x7f43cc0b1ec0 2021-07-11T03:53:53.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43cc0b12d0 msgr2=0x7f43cc0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:53.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 --2- 172.21.2.104:0/1906592145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43cc0b12d0 0x7f43cc0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 >> v2:172.21.2.105:3300/0 conn(0x7f43cc0b22a0 msgr2=0x7f43cc141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:53.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 --2- 172.21.2.104:0/1906592145 >> v2:172.21.2.105:3300/0 conn(0x7f43cc0b22a0 0x7f43cc141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc0b08f0 con 0x7f43cc0b1ec0 2021-07-11T03:53:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f43c8001660 con 0x7f43cc0b1ec0 2021-07-11T03:53:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 >> v1:172.21.2.104:6789/0 conn(0x7f43cc0b1ec0 legacy=0x7f43cc0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 --2- 172.21.2.104:0/1906592145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43b4003750 0x7f43b4003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 --2- 172.21.2.104:0/1906592145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43cc0b12d0 0x7f43b4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 --2- 172.21.2.104:0/1906592145 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43b40049f0 0x7f43b4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0b1740 con 0x7f43b4003750 2021-07-11T03:53:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0b0650 con 0x7f43b40049f0 2021-07-11T03:53:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b400b8b0 con 0x7f43cc0b12d0 2021-07-11T03:53:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1280700 1 --2- 172.21.2.104:0/1906592145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43b4003750 0x7f43b4003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1280700 1 -- 172.21.2.104:0/1906592145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43cc0b12d0 msgr2=0x7f43b4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1280700 1 --2- 172.21.2.104:0/1906592145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43cc0b12d0 0x7f43b4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1280700 1 -- 172.21.2.104:0/1906592145 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43b40049f0 msgr2=0x7f43b4004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1280700 1 --2- 172.21.2.104:0/1906592145 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43b40049f0 0x7f43b4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1280700 1 -- 172.21.2.104:0/1906592145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b4003260 con 0x7f43b4003750 2021-07-11T03:53:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1280700 1 --2- 172.21.2.104:0/1906592145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43b4003750 0x7f43b4003b70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f43c8001370 tx=0x7f43cc0b2a00).ready entity=mon.0 client_cookie=8c7dd72101786e31 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43c80071a0 con 0x7f43b4003750 2021-07-11T03:53:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c8007300 con 0x7f43b4003750 2021-07-11T03:53:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43bffff700 1 -- 172.21.2.104:0/1906592145 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43c800cbb0 con 0x7f43b4003750 2021-07-11T03:53:53.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 -- 172.21.2.104:0/1906592145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43b4003750 msgr2=0x7f43b4003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 --2- 172.21.2.104:0/1906592145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43b4003750 0x7f43b4003b70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f43c8001370 tx=0x7f43cc0b2a00).stop 2021-07-11T03:53:53.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1a81700 1 -- 172.21.2.104:0/1906592145 reap_dead start 2021-07-11T03:53:53.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 -- 172.21.2.104:0/1906592145 shutdown_connections 2021-07-11T03:53:53.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 -- 172.21.2.104:0/1906592145 >> 172.21.2.104:0/1906592145 conn(0x7f43cc05f6b0 msgr2=0x7f43cc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:53.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 -- 172.21.2.104:0/1906592145 shutdown_connections 2021-07-11T03:53:53.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 -- 172.21.2.104:0/1906592145 wait complete. 2021-07-11T03:53:53.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 Processor -- start 2021-07-11T03:53:53.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 -- start start 2021-07-11T03:53:53.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43cc0ab950 0x7f43cc149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43cc13d180 0x7f43cc156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43cc1573b0 0x7f43cc14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc0abd30 con 0x7f43cc1573b0 2021-07-11T03:53:53.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc13dc80 con 0x7f43cc13d180 2021-07-11T03:53:53.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d34e4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43cc13df20 con 0x7f43cc0ab950 2021-07-11T03:53:53.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1a81700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43cc1573b0 0x7f43cc14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1a81700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43cc1573b0 0x7f43cc14afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43332/0 (socket says 172.21.2.104:43332) 2021-07-11T03:53:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.089+0000 7f43d1a81700 1 -- 172.21.2.104:0/1038550101 learned_addr learned my addr 172.21.2.104:0/1038550101 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43d1a81700 1 -- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43cc0ab950 msgr2=0x7f43cc149540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43d1a81700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43cc0ab950 0x7f43cc149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43d1a81700 1 -- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43cc13d180 msgr2=0x7f43cc156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43d1a81700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43cc13d180 0x7f43cc156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43d1a81700 1 -- 172.21.2.104:0/1038550101 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc13e1c0 con 0x7f43cc1573b0 2021-07-11T03:53:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43d1a81700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43cc1573b0 0x7f43cc14afe0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f43c80017c0 tx=0x7f43c4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43be7fc700 1 -- 172.21.2.104:0/1038550101 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43c4010580 con 0x7f43cc1573b0 2021-07-11T03:53:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43be7fc700 1 -- 172.21.2.104:0/1038550101 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c40106e0 con 0x7f43cc1573b0 2021-07-11T03:53:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43d34e4700 1 -- 172.21.2.104:0/1038550101 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc13e460 con 0x7f43cc1573b0 2021-07-11T03:53:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43be7fc700 1 -- 172.21.2.104:0/1038550101 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43c4010a40 con 0x7f43cc1573b0 2021-07-11T03:53:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43d34e4700 1 -- 172.21.2.104:0/1038550101 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc156a20 con 0x7f43cc1573b0 2021-07-11T03:53:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43d34e4700 1 -- 172.21.2.104:0/1038550101 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc057350 con 0x7f43cc1573b0 2021-07-11T03:53:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43be7fc700 1 -- 172.21.2.104:0/1038550101 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f43c4010ba0 con 0x7f43cc1573b0 2021-07-11T03:53:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43be7fc700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43ac053210 0x7f43ac0556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43be7fc700 1 -- 172.21.2.104:0/1038550101 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f43c406b5e0 con 0x7f43cc1573b0 2021-07-11T03:53:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43d1280700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43ac053210 0x7f43ac0556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.093+0000 7f43d1280700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43ac053210 0x7f43ac0556d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f43c8001370 tx=0x7f43c8013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.097+0000 7f43be7fc700 1 -- 172.21.2.104:0/1038550101 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f43c4042810 con 0x7f43cc1573b0 2021-07-11T03:53:53.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 -- 172.21.2.104:0/1038550101 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f43cc0ac760 con 0x7f43cc1573b0 2021-07-11T03:53:53.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43be7fc700 1 -- 172.21.2.104:0/1038550101 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v33) v1 ==== 110+0+10 (secure 0 0 0) 0x7f43c4045e30 con 0x7f43cc1573b0 2021-07-11T03:53:53.183 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:53:53.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 -- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43ac053210 msgr2=0x7f43ac0556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43ac053210 0x7f43ac0556d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f43c8001370 tx=0x7f43c8013000).stop 2021-07-11T03:53:53.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 -- 172.21.2.104:0/1038550101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43cc1573b0 msgr2=0x7f43cc14afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43cc1573b0 0x7f43cc14afe0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f43c80017c0 tx=0x7f43c4008040).stop 2021-07-11T03:53:53.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 -- 172.21.2.104:0/1038550101 shutdown_connections 2021-07-11T03:53:53.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43ac053210 0x7f43ac0556d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43cc1573b0 0x7f43cc14afe0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43cc13d180 0x7f43cc156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 --2- 172.21.2.104:0/1038550101 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43cc0ab950 0x7f43cc149540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 -- 172.21.2.104:0/1038550101 >> 172.21.2.104:0/1038550101 conn(0x7f43cc05f6b0 msgr2=0x7f43cc141020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:53.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 -- 172.21.2.104:0/1038550101 shutdown_connections 2021-07-11T03:53:53.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.177+0000 7f43d34e4700 1 -- 172.21.2.104:0/1038550101 wait complete. 2021-07-11T03:53:53.191 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:53:53.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.237+0000 7f4ff21ba700 1 Processor -- start 2021-07-11T03:53:53.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.237+0000 7f4ff21ba700 1 -- start start 2021-07-11T03:53:53.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.237+0000 7f4ff21ba700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fec060100 0x7f4fec064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.237+0000 7f4ff21ba700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f4fec144440 0x7f4fec065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.237+0000 7f4ff21ba700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4fec0b38d0 0x7f4fec144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.237+0000 7f4ff21ba700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec0656f0 con 0x7f4fec0b38d0 2021-07-11T03:53:53.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.237+0000 7f4ff21ba700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fec065c80 con 0x7f4fec144440 2021-07-11T03:53:53.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.237+0000 7f4ff21ba700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec0604e0 con 0x7f4fec060100 2021-07-11T03:53:53.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.237+0000 7f4febfff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4fec0b38d0 0x7f4fec144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4febfff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4fec0b38d0 0x7f4fec144970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43338/0 (socket says 172.21.2.104:43338) 2021-07-11T03:53:53.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4febfff700 1 -- 172.21.2.104:0/1976355499 learned_addr learned my addr 172.21.2.104:0/1976355499 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:53.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4febfff700 1 -- 172.21.2.104:0/1976355499 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fec060100 msgr2=0x7f4fec064d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:53.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4febfff700 1 --2- 172.21.2.104:0/1976355499 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fec060100 0x7f4fec064d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4febfff700 1 -- 172.21.2.104:0/1976355499 >> v2:172.21.2.105:3300/0 conn(0x7f4fec144440 msgr2=0x7f4fec065200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:53:53.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4febfff700 1 --2- 172.21.2.104:0/1976355499 >> v2:172.21.2.105:3300/0 conn(0x7f4fec144440 0x7f4fec065200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4febfff700 1 -- 172.21.2.104:0/1976355499 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fec0b23b0 con 0x7f4fec0b38d0 2021-07-11T03:53:53.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4febfff700 1 --2- 172.21.2.104:0/1976355499 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fec0b38d0 0x7f4fec144970 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f4fe40032f0 tx=0x7f4fe4009290).ready entity=mon.0 client_cookie=96e83c91f6229e8a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4fea7fc700 1 -- 172.21.2.104:0/1976355499 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fe400b870 con 0x7f4fec0b38d0 2021-07-11T03:53:53.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4fea7fc700 1 -- 172.21.2.104:0/1976355499 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe400b9d0 con 0x7f4fec0b38d0 2021-07-11T03:53:53.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4fea7fc700 1 -- 172.21.2.104:0/1976355499 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fe400bb80 con 0x7f4fec0b38d0 2021-07-11T03:53:53.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- 172.21.2.104:0/1976355499 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fec0b38d0 msgr2=0x7f4fec144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 --2- 172.21.2.104:0/1976355499 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fec0b38d0 0x7f4fec144970 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f4fe40032f0 tx=0x7f4fe4009290).stop 2021-07-11T03:53:53.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- 172.21.2.104:0/1976355499 shutdown_connections 2021-07-11T03:53:53.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 --2- 172.21.2.104:0/1976355499 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fec0b38d0 0x7f4fec144970 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 --2- 172.21.2.104:0/1976355499 >> v2:172.21.2.105:3300/0 conn(0x7f4fec144440 0x7f4fec065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 --2- 172.21.2.104:0/1976355499 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fec060100 0x7f4fec064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- 172.21.2.104:0/1976355499 >> 172.21.2.104:0/1976355499 conn(0x7f4fec05cf60 msgr2=0x7f4fec05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:53.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- 172.21.2.104:0/1976355499 shutdown_connections 2021-07-11T03:53:53.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- 172.21.2.104:0/1976355499 wait complete. 2021-07-11T03:53:53.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 Processor -- start 2021-07-11T03:53:53.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- start start 2021-07-11T03:53:53.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fec060100 0x7f4fec153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fec0b38d0 0x7f4fec143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fec142480 0x7f4fec138f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec1547f0 con 0x7f4fec0b38d0 2021-07-11T03:53:53.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec143e10 con 0x7f4fec060100 2021-07-11T03:53:53.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fec142ae0 con 0x7f4fec142480 2021-07-11T03:53:53.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4feaffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fec0b38d0 0x7f4fec143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4feaffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fec0b38d0 0x7f4fec143220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43344/0 (socket says 172.21.2.104:43344) 2021-07-11T03:53:53.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4feaffd700 1 -- 172.21.2.104:0/3472928829 learned_addr learned my addr 172.21.2.104:0/3472928829 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4feaffd700 1 -- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fec142480 msgr2=0x7f4fec138f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4feaffd700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fec142480 0x7f4fec138f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4feaffd700 1 -- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fec060100 msgr2=0x7f4fec153870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4feaffd700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fec060100 0x7f4fec153870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4feaffd700 1 -- 172.21.2.104:0/3472928829 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4009530 con 0x7f4fec0b38d0 2021-07-11T03:53:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4feaffd700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fec0b38d0 0x7f4fec143220 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f4fdc000ed0 tx=0x7f4fdc006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4fe8ff9700 1 -- 172.21.2.104:0/3472928829 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fdc012410 con 0x7f4fec0b38d0 2021-07-11T03:53:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4fe8ff9700 1 -- 172.21.2.104:0/3472928829 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc012570 con 0x7f4fec0b38d0 2021-07-11T03:53:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4fe8ff9700 1 -- 172.21.2.104:0/3472928829 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4fdc0128d0 con 0x7f4fec0b38d0 2021-07-11T03:53:53.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- 172.21.2.104:0/3472928829 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fec142d80 con 0x7f4fec0b38d0 2021-07-11T03:53:53.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- 172.21.2.104:0/3472928829 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fec139450 con 0x7f4fec0b38d0 2021-07-11T03:53:53.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.241+0000 7f4ff21ba700 1 -- 172.21.2.104:0/3472928829 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fec142f10 con 0x7f4fec0b38d0 2021-07-11T03:53:53.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.245+0000 7f4fe8ff9700 1 -- 172.21.2.104:0/3472928829 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4fdc00f030 con 0x7f4fec0b38d0 2021-07-11T03:53:53.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.245+0000 7f4fe8ff9700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fd0057760 0x7f4fd0059c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.245+0000 7f4fe8ff9700 1 -- 172.21.2.104:0/3472928829 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4fdc06b230 con 0x7f4fec0b38d0 2021-07-11T03:53:53.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.245+0000 7f4feb7fe700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fd0057760 0x7f4fd0059c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.245+0000 7f4feb7fe700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fd0057760 0x7f4fd0059c20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f4fd4002fb0 tx=0x7f4fd4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.245+0000 7f4fe8ff9700 1 -- 172.21.2.104:0/3472928829 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4fdc042460 con 0x7f4fec0b38d0 2021-07-11T03:53:53.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 -- 172.21.2.104:0/3472928829 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4fec057350 con 0x7f4fec0b38d0 2021-07-11T03:53:53.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4fe8ff9700 1 -- 172.21.2.104:0/3472928829 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4fdc045a80 con 0x7f4fec0b38d0 2021-07-11T03:53:53.335 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:53.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 -- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fd0057760 msgr2=0x7f4fd0059c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fd0057760 0x7f4fd0059c20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f4fd4002fb0 tx=0x7f4fd4006930).stop 2021-07-11T03:53:53.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 -- 172.21.2.104:0/3472928829 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fec0b38d0 msgr2=0x7f4fec143220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fec0b38d0 0x7f4fec143220 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f4fdc000ed0 tx=0x7f4fdc006d70).stop 2021-07-11T03:53:53.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 -- 172.21.2.104:0/3472928829 shutdown_connections 2021-07-11T03:53:53.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4fec142480 0x7f4fec138f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4fd0057760 0x7f4fd0059c20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4fec0b38d0 0x7f4fec143220 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 --2- 172.21.2.104:0/3472928829 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4fec060100 0x7f4fec153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 -- 172.21.2.104:0/3472928829 >> 172.21.2.104:0/3472928829 conn(0x7f4fec05cf60 msgr2=0x7f4fec13f320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:53.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 -- 172.21.2.104:0/3472928829 shutdown_connections 2021-07-11T03:53:53.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.329+0000 7f4ff21ba700 1 -- 172.21.2.104:0/3472928829 wait complete. 2021-07-11T03:53:53.342 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:53:53.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0f761700 1 Processor -- start 2021-07-11T03:53:53.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0f761700 1 -- start start 2021-07-11T03:53:53.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0f761700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f3e08144440 0x7f3e08144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0f761700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3e08065420 0x7f3e08144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0f761700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e08145350 con 0x7f3e08060100 2021-07-11T03:53:53.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0f761700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e08065c80 con 0x7f3e08144440 2021-07-11T03:53:53.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0f761700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080604e0 con 0x7f3e08065420 2021-07-11T03:53:53.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0dcfe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3e08065420 0x7f3e08144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0ccfc700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f3e08144440 0x7f3e08144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0d4fd700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f3e08060100 0x7f3e08064d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:39488/0 (socket says 172.21.2.104:39488) 2021-07-11T03:53:53.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0d4fd700 1 -- 172.21.2.104:0/3289695263 learned_addr learned my addr 172.21.2.104:0/3289695263 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:53.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.389+0000 7f3e0ccfc700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f3e08144440 0x7f3e08144970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:50914/0 (socket says 172.21.2.104:50914) 2021-07-11T03:53:53.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 -- 172.21.2.104:0/3289695263 >> v2:172.21.2.105:3300/0 conn(0x7f3e08144440 msgr2=0x7f3e08144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:53.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 --2- 172.21.2.104:0/3289695263 >> v2:172.21.2.105:3300/0 conn(0x7f3e08144440 0x7f3e08144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 -- 172.21.2.104:0/3289695263 >> v1:172.21.2.105:6789/0 conn(0x7f3e08060100 legacy=0x7f3e08064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 -- 172.21.2.104:0/3289695263 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e080b23b0 con 0x7f3e08065420 2021-07-11T03:53:53.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 --2- 172.21.2.104:0/3289695263 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3e08065420 0x7f3e08144e60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f3e000032f0 tx=0x7f3e000091d0).ready entity=mon.2 client_cookie=d76eff1392a0239b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3dfbfff700 1 -- 172.21.2.104:0/3289695263 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e0000b6c0 con 0x7f3e08065420 2021-07-11T03:53:53.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3dfbfff700 1 -- 172.21.2.104:0/3289695263 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0000b820 con 0x7f3e08065420 2021-07-11T03:53:53.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 -- 172.21.2.104:0/3289695263 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3e08065420 msgr2=0x7f3e08144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3dfbfff700 1 -- 172.21.2.104:0/3289695263 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e0000b990 con 0x7f3e08065420 2021-07-11T03:53:53.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 --2- 172.21.2.104:0/3289695263 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3e08065420 0x7f3e08144e60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f3e000032f0 tx=0x7f3e000091d0).stop 2021-07-11T03:53:53.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 -- 172.21.2.104:0/3289695263 shutdown_connections 2021-07-11T03:53:53.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 --2- 172.21.2.104:0/3289695263 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3e08065420 0x7f3e08144e60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 --2- 172.21.2.104:0/3289695263 >> v2:172.21.2.105:3300/0 conn(0x7f3e08144440 0x7f3e08144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 -- 172.21.2.104:0/3289695263 >> 172.21.2.104:0/3289695263 conn(0x7f3e0805cf60 msgr2=0x7f3e0805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:53.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 -- 172.21.2.104:0/3289695263 shutdown_connections 2021-07-11T03:53:53.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 -- 172.21.2.104:0/3289695263 wait complete. 2021-07-11T03:53:53.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 Processor -- start 2021-07-11T03:53:53.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 -- start start 2021-07-11T03:53:53.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3e08060100 0x7f3e0813f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3e08065420 0x7f3e08150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e080adc40 0x7f3e0813a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e08140b70 con 0x7f3e080adc40 2021-07-11T03:53:53.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e08140dd0 con 0x7f3e08065420 2021-07-11T03:53:53.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0f761700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e08141070 con 0x7f3e08060100 2021-07-11T03:53:53.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e080adc40 0x7f3e0813a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e080adc40 0x7f3e0813a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43360/0 (socket says 172.21.2.104:43360) 2021-07-11T03:53:53.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 -- 172.21.2.104:0/1724864494 learned_addr learned my addr 172.21.2.104:0/1724864494 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:53.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0ccfc700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3e08065420 0x7f3e08150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0d4fd700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3e08060100 0x7f3e0813f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 -- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3e08060100 msgr2=0x7f3e0813f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3e08060100 0x7f3e0813f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 -- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3e08065420 msgr2=0x7f3e08150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3e08065420 0x7f3e08150bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 -- 172.21.2.104:0/1724864494 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e00009470 con 0x7f3e080adc40 2021-07-11T03:53:53.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.393+0000 7f3e0dcfe700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e080adc40 0x7f3e0813a740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f3e000032f0 tx=0x7f3e00016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3dfa7fc700 1 -- 172.21.2.104:0/1724864494 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e00011e90 con 0x7f3e080adc40 2021-07-11T03:53:53.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3dfa7fc700 1 -- 172.21.2.104:0/1724864494 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e00021440 con 0x7f3e080adc40 2021-07-11T03:53:53.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3e0f761700 1 -- 172.21.2.104:0/1724864494 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e08141310 con 0x7f3e080adc40 2021-07-11T03:53:53.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3dfa7fc700 1 -- 172.21.2.104:0/1724864494 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3e000217a0 con 0x7f3e080adc40 2021-07-11T03:53:53.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3e0f761700 1 -- 172.21.2.104:0/1724864494 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0813ac30 con 0x7f3e080adc40 2021-07-11T03:53:53.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3e0f761700 1 -- 172.21.2.104:0/1724864494 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e081414d0 con 0x7f3e080adc40 2021-07-11T03:53:53.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3dfa7fc700 1 -- 172.21.2.104:0/1724864494 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3e00029070 con 0x7f3e080adc40 2021-07-11T03:53:53.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3dfa7fc700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3de8053330 0x7f3de80557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3dfa7fc700 1 -- 172.21.2.104:0/1724864494 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3e0006c3d0 con 0x7f3e080adc40 2021-07-11T03:53:53.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3e0d4fd700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3de8053330 0x7f3de80557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3e0d4fd700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3de8053330 0x7f3de80557f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f3e0400b000 tx=0x7f3e04009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.397+0000 7f3dfa7fc700 1 -- 172.21.2.104:0/1724864494 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3e00043750 con 0x7f3e080adc40 2021-07-11T03:53:53.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 -- 172.21.2.104:0/1724864494 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3e08057350 con 0x7f3e080adc40 2021-07-11T03:53:53.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3dfa7fc700 1 -- 172.21.2.104:0/1724864494 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3e00046d30 con 0x7f3e080adc40 2021-07-11T03:53:53.486 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:53.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 -- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3de8053330 msgr2=0x7f3de80557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3de8053330 0x7f3de80557f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f3e0400b000 tx=0x7f3e04009040).stop 2021-07-11T03:53:53.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 -- 172.21.2.104:0/1724864494 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e080adc40 msgr2=0x7f3e0813a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e080adc40 0x7f3e0813a740 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f3e000032f0 tx=0x7f3e00016000).stop 2021-07-11T03:53:53.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 -- 172.21.2.104:0/1724864494 shutdown_connections 2021-07-11T03:53:53.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3de8053330 0x7f3de80557f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3e080adc40 0x7f3e0813a740 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3e08065420 0x7f3e08150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 --2- 172.21.2.104:0/1724864494 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3e08060100 0x7f3e0813f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 -- 172.21.2.104:0/1724864494 >> 172.21.2.104:0/1724864494 conn(0x7f3e0805cf60 msgr2=0x7f3e08145500 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:53.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 -- 172.21.2.104:0/1724864494 shutdown_connections 2021-07-11T03:53:53.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.481+0000 7f3e0f761700 1 -- 172.21.2.104:0/1724864494 wait complete. 2021-07-11T03:53:53.494 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:53:53.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 Processor -- start 2021-07-11T03:53:53.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 -- start start 2021-07-11T03:53:53.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efc58060100 0x7efc58064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7efc58144440 0x7efc58065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc580604e0 con 0x7efc580b38d0 2021-07-11T03:53:53.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc580656f0 con 0x7efc58144440 2021-07-11T03:53:53.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc58065c80 con 0x7efc58060100 2021-07-11T03:53:53.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc577fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7efc580b38d0 0x7efc58144970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40134/0 (socket says 172.21.2.104:40134) 2021-07-11T03:53:53.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc577fe700 1 -- 172.21.2.104:0/3622443358 learned_addr learned my addr 172.21.2.104:0/3622443358 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:53.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc567fc700 1 --2- 172.21.2.104:0/3622443358 >> v2:172.21.2.104:3300/0 conn(0x7efc58144440 0x7efc58065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc567fc700 1 -- 172.21.2.104:0/3622443358 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efc58060100 msgr2=0x7efc58064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:53.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc567fc700 1 --2- 172.21.2.104:0/3622443358 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efc58060100 0x7efc58064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc567fc700 1 -- 172.21.2.104:0/3622443358 >> v1:172.21.2.104:6789/0 conn(0x7efc580b38d0 legacy=0x7efc58144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc567fc700 1 -- 172.21.2.104:0/3622443358 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc580b23b0 con 0x7efc58144440 2021-07-11T03:53:53.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc567fc700 1 --2- 172.21.2.104:0/3622443358 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efc58144440 0x7efc58065200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7efc48002f70 tx=0x7efc48006df0).ready entity=mon.0 client_cookie=b12bb28a1300fcd0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc55ffb700 1 -- 172.21.2.104:0/3622443358 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc4800d570 con 0x7efc58144440 2021-07-11T03:53:53.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc55ffb700 1 -- 172.21.2.104:0/3622443358 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc48000e90 con 0x7efc58144440 2021-07-11T03:53:53.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc55ffb700 1 -- 172.21.2.104:0/3622443358 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc4800d8c0 con 0x7efc58144440 2021-07-11T03:53:53.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 -- 172.21.2.104:0/3622443358 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efc58144440 msgr2=0x7efc58065200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 --2- 172.21.2.104:0/3622443358 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efc58144440 0x7efc58065200 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7efc48002f70 tx=0x7efc48006df0).stop 2021-07-11T03:53:53.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 -- 172.21.2.104:0/3622443358 shutdown_connections 2021-07-11T03:53:53.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 --2- 172.21.2.104:0/3622443358 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efc58144440 0x7efc58065200 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 --2- 172.21.2.104:0/3622443358 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efc58060100 0x7efc58064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 -- 172.21.2.104:0/3622443358 >> 172.21.2.104:0/3622443358 conn(0x7efc5805cf60 msgr2=0x7efc5805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:53.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 -- 172.21.2.104:0/3622443358 shutdown_connections 2021-07-11T03:53:53.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 -- 172.21.2.104:0/3622443358 wait complete. 2021-07-11T03:53:53.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 Processor -- start 2021-07-11T03:53:53.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.541+0000 7efc5d2e0700 1 -- start start 2021-07-11T03:53:53.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc5d2e0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efc58060100 0x7efc580b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc5d2e0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efc580b38d0 0x7efc580b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc5d2e0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efc58143290 0x7efc58143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc5d2e0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc58062120 con 0x7efc580b38d0 2021-07-11T03:53:53.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc5d2e0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc58066ad0 con 0x7efc58060100 2021-07-11T03:53:53.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc5d2e0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc580b72c0 con 0x7efc58143290 2021-07-11T03:53:53.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc567fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efc580b38d0 0x7efc580b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc567fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efc580b38d0 0x7efc580b69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43370/0 (socket says 172.21.2.104:43370) 2021-07-11T03:53:53.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc567fc700 1 -- 172.21.2.104:0/2884477605 learned_addr learned my addr 172.21.2.104:0/2884477605 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:53.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc567fc700 1 -- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efc58143290 msgr2=0x7efc58143720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc567fc700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efc58143290 0x7efc58143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc567fc700 1 -- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efc58060100 msgr2=0x7efc580b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc567fc700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efc58060100 0x7efc580b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc567fc700 1 -- 172.21.2.104:0/2884477605 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4800b180 con 0x7efc580b38d0 2021-07-11T03:53:53.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc567fc700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efc580b38d0 0x7efc580b69e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7efc48002f70 tx=0x7efc48015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc4ffff700 1 -- 172.21.2.104:0/2884477605 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc48021070 con 0x7efc580b38d0 2021-07-11T03:53:53.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc4ffff700 1 -- 172.21.2.104:0/2884477605 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc48000e90 con 0x7efc580b38d0 2021-07-11T03:53:53.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc5d2e0700 1 -- 172.21.2.104:0/2884477605 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc58062350 con 0x7efc580b38d0 2021-07-11T03:53:53.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc4ffff700 1 -- 172.21.2.104:0/2884477605 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc48011e90 con 0x7efc580b38d0 2021-07-11T03:53:53.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc5d2e0700 1 -- 172.21.2.104:0/2884477605 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc580b6ed0 con 0x7efc580b38d0 2021-07-11T03:53:53.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc5d2e0700 1 -- 172.21.2.104:0/2884477605 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc580624e0 con 0x7efc580b38d0 2021-07-11T03:53:53.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc4ffff700 1 -- 172.21.2.104:0/2884477605 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7efc4800f070 con 0x7efc580b38d0 2021-07-11T03:53:53.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc4ffff700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc3c053330 0x7efc3c0557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc4ffff700 1 -- 172.21.2.104:0/2884477605 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7efc4806c960 con 0x7efc580b38d0 2021-07-11T03:53:53.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc56ffd700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc3c053330 0x7efc3c0557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.545+0000 7efc56ffd700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc3c053330 0x7efc3c0557f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7efc40002f70 tx=0x7efc40006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.549+0000 7efc4ffff700 1 -- 172.21.2.104:0/2884477605 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7efc48043ca0 con 0x7efc580b38d0 2021-07-11T03:53:53.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 -- 172.21.2.104:0/2884477605 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efc58057350 con 0x7efc580b38d0 2021-07-11T03:53:53.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc4ffff700 1 -- 172.21.2.104:0/2884477605 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+10 (secure 0 0 0) 0x7efc480472c0 con 0x7efc580b38d0 2021-07-11T03:53:53.638 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:53:53.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 -- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc3c053330 msgr2=0x7efc3c0557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc3c053330 0x7efc3c0557f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7efc40002f70 tx=0x7efc40006a80).stop 2021-07-11T03:53:53.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 -- 172.21.2.104:0/2884477605 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efc580b38d0 msgr2=0x7efc580b69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efc580b38d0 0x7efc580b69e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7efc48002f70 tx=0x7efc48015000).stop 2021-07-11T03:53:53.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 -- 172.21.2.104:0/2884477605 shutdown_connections 2021-07-11T03:53:53.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efc58143290 0x7efc58143720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc3c053330 0x7efc3c0557f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efc580b38d0 0x7efc580b69e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 --2- 172.21.2.104:0/2884477605 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efc58060100 0x7efc580b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 -- 172.21.2.104:0/2884477605 >> 172.21.2.104:0/2884477605 conn(0x7efc5805cf60 msgr2=0x7efc580b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:53.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 -- 172.21.2.104:0/2884477605 shutdown_connections 2021-07-11T03:53:53.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.633+0000 7efc5d2e0700 1 -- 172.21.2.104:0/2884477605 wait complete. 2021-07-11T03:53:53.645 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T03:53:53.645 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:53:53.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.689+0000 7f9478bd5700 1 Processor -- start 2021-07-11T03:53:53.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.689+0000 7f9478bd5700 1 -- start start 2021-07-11T03:53:53.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94740aa860 0x7f94740b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f94740a9da0 0x7f94740aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94740ac370 con 0x7f94740acf30 2021-07-11T03:53:53.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94740abb50 con 0x7f94740a9da0 2021-07-11T03:53:53.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94740aac40 con 0x7f94740aa860 2021-07-11T03:53:53.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9472d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f94740acf30 0x7f94741434b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40146/0 (socket says 172.21.2.104:40146) 2021-07-11T03:53:53.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9472d9d700 1 -- 172.21.2.104:0/1369201908 learned_addr learned my addr 172.21.2.104:0/1369201908 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:53:53.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9471d9b700 1 --2- 172.21.2.104:0/1369201908 >> v2:172.21.2.104:3300/0 conn(0x7f94740a9da0 0x7f94740aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9471d9b700 1 -- 172.21.2.104:0/1369201908 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94740aa860 msgr2=0x7f94740b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:53:53.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9471d9b700 1 --2- 172.21.2.104:0/1369201908 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94740aa860 0x7f94740b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9471d9b700 1 -- 172.21.2.104:0/1369201908 >> v1:172.21.2.104:6789/0 conn(0x7f94740acf30 legacy=0x7f94741434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9471d9b700 1 -- 172.21.2.104:0/1369201908 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94740b16b0 con 0x7f94740a9da0 2021-07-11T03:53:53.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9471d9b700 1 --2- 172.21.2.104:0/1369201908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94740a9da0 0x7f94740aca40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f9468002f70 tx=0x7f9468006df0).ready entity=mon.0 client_cookie=7f1690814879d48a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f947159a700 1 -- 172.21.2.104:0/1369201908 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946800d570 con 0x7f94740a9da0 2021-07-11T03:53:53.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f947159a700 1 -- 172.21.2.104:0/1369201908 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468000e90 con 0x7f94740a9da0 2021-07-11T03:53:53.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f947159a700 1 -- 172.21.2.104:0/1369201908 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946800d8c0 con 0x7f94740a9da0 2021-07-11T03:53:53.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- 172.21.2.104:0/1369201908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94740a9da0 msgr2=0x7f94740aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 --2- 172.21.2.104:0/1369201908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94740a9da0 0x7f94740aca40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f9468002f70 tx=0x7f9468006df0).stop 2021-07-11T03:53:53.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- 172.21.2.104:0/1369201908 shutdown_connections 2021-07-11T03:53:53.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 --2- 172.21.2.104:0/1369201908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94740a9da0 0x7f94740aca40 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 --2- 172.21.2.104:0/1369201908 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94740aa860 0x7f94740b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- 172.21.2.104:0/1369201908 >> 172.21.2.104:0/1369201908 conn(0x7f94740b64d0 msgr2=0x7f94740b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:53.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- 172.21.2.104:0/1369201908 shutdown_connections 2021-07-11T03:53:53.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- 172.21.2.104:0/1369201908 wait complete. 2021-07-11T03:53:53.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 Processor -- start 2021-07-11T03:53:53.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- start start 2021-07-11T03:53:53.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f94740a9da0 0x7f947405f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94740aa860 0x7f94740a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94740acf30 0x7f94740a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9474060100 con 0x7f94740acf30 2021-07-11T03:53:53.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94740636e0 con 0x7f94740a9da0 2021-07-11T03:53:53.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9474065a20 con 0x7f94740aa860 2021-07-11T03:53:53.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9472d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94740acf30 0x7f94740a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9472d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94740acf30 0x7f94740a07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43388/0 (socket says 172.21.2.104:43388) 2021-07-11T03:53:53.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9472d9d700 1 -- 172.21.2.104:0/1240339910 learned_addr learned my addr 172.21.2.104:0/1240339910 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:53:53.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9472d9d700 1 -- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94740aa860 msgr2=0x7f94740a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9472d9d700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94740aa860 0x7f94740a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9472d9d700 1 -- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f94740a9da0 msgr2=0x7f947405f9f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:53:53.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9472d9d700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f94740a9da0 0x7f947405f9f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9472d9d700 1 -- 172.21.2.104:0/1240339910 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946800b180 con 0x7f94740acf30 2021-07-11T03:53:53.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9472d9d700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94740acf30 0x7f94740a07e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f9474066400 tx=0x7f946c0013a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f94637fe700 1 -- 172.21.2.104:0/1240339910 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946c005ba0 con 0x7f94740acf30 2021-07-11T03:53:53.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f94637fe700 1 -- 172.21.2.104:0/1240339910 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c005d00 con 0x7f94740acf30 2021-07-11T03:53:53.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- 172.21.2.104:0/1240339910 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9474065780 con 0x7f94740acf30 2021-07-11T03:53:53.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f94637fe700 1 -- 172.21.2.104:0/1240339910 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f946c0072c0 con 0x7f94740acf30 2021-07-11T03:53:53.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.693+0000 7f9478bd5700 1 -- 172.21.2.104:0/1240339910 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94740654b0 con 0x7f94740acf30 2021-07-11T03:53:53.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.697+0000 7f9478bd5700 1 -- 172.21.2.104:0/1240339910 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9474064d40 con 0x7f94740acf30 2021-07-11T03:53:53.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.697+0000 7f94637fe700 1 -- 172.21.2.104:0/1240339910 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f946c01a060 con 0x7f94740acf30 2021-07-11T03:53:53.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.697+0000 7f94637fe700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94500532b0 0x7f9450055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:53:53.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.697+0000 7f94637fe700 1 -- 172.21.2.104:0/1240339910 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f946c06af50 con 0x7f94740acf30 2021-07-11T03:53:53.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.697+0000 7f947259c700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94500532b0 0x7f9450055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:53:53.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.697+0000 7f947259c700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94500532b0 0x7f9450055770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f9464002f70 tx=0x7f9464006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:53:53.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.697+0000 7f94637fe700 1 -- 172.21.2.104:0/1240339910 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f946c042290 con 0x7f94740acf30 2021-07-11T03:53:53.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 -- 172.21.2.104:0/1240339910 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9474057350 con 0x7f94740acf30 2021-07-11T03:53:53.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f94637fe700 1 -- 172.21.2.104:0/1240339910 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f946c018070 con 0x7f94740acf30 2021-07-11T03:53:53.812 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:53:53.812 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4408,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2474981992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2474981992},{"type":"v1","addr":"172.21.2.105:6835","nonce":2474981992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/1241687766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1241687766},{"type":"v1","addr":"172.21.2.104:6834","nonce":1241687766}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:50.608622+0000","modified":"2021-07-11T03:53:51.611729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/1104530072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1104530072},{"type":"v1","addr":"172.21.2.104:6835","nonce":1104530072}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-07-11T03:53:53.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 -- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94500532b0 msgr2=0x7f9450055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94500532b0 0x7f9450055770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f9464002f70 tx=0x7f9464006a00).stop 2021-07-11T03:53:53.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 -- 172.21.2.104:0/1240339910 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94740acf30 msgr2=0x7f94740a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:53:53.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94740acf30 0x7f94740a07e0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f9474066400 tx=0x7f946c0013a0).stop 2021-07-11T03:53:53.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 -- 172.21.2.104:0/1240339910 shutdown_connections 2021-07-11T03:53:53.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f94500532b0 0x7f9450055770 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f94740acf30 0x7f94740a07e0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:53:53.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f94740aa860 0x7f94740a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 --2- 172.21.2.104:0/1240339910 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f94740a9da0 0x7f947405f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:53:53.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 -- 172.21.2.104:0/1240339910 >> 172.21.2.104:0/1240339910 conn(0x7f94740b64d0 msgr2=0x7f9474060fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:53:53.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 -- 172.21.2.104:0/1240339910 shutdown_connections 2021-07-11T03:53:53.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:53:53.805+0000 7f9478bd5700 1 -- 172.21.2.104:0/1240339910 wait complete. 2021-07-11T03:53:53.816 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 11 2021-07-11T03:53:54.121 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 11, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:53:50.608622+0000', 'modified': '2021-07-11T03:53:51.611729+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4863}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4863': {'gid': 4863, 'name': 'a', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/1104530072', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 1104530072}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 1104530072}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:53:54.122 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:53:54.122 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-07-11T03:53:54.187 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":4863,"want_state":"up:active","state":"up:active","rank_uptime":3.5710292410000002,"mdsmap_epoch":11,"osdmap_epoch":33,"osdmap_epoch_barrier":32,"uptime":8.7448997810000009} 2021-07-11T03:53:54.188 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 4863, "mdsmap_epoch": 11, "osdmap_epoch": 33, "osdmap_epoch_barrier": 32, "rank_uptime": 3.571029241, "state": "up:active", "uptime": 8.744899781, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:53:54.188 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T03:53:54.189 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T03:53:54.189 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T03:53:54.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:54.190 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:54.201 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:53:54.254 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:53:54.255 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:53:54.261 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:53:54.261 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:53:54.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:54.263 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:53:54.267 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T03:53:54.268 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:54.314 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:53:54.324 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:53:54.324 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:53:54.329 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:53:54.330 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:53:54.331 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:54.331 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:53:54.354 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T03:53:54.345+0000 7f842ba17080 -1 init, newargv = 0x563f2cab9dd0 newargc=9ceph-fuse[14596 2021-07-11T03:53:54.355 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:]: starting ceph client 2021-07-11T03:53:54.369 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14596]: starting fuse 2021-07-11T03:53:55.460 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:53:55.472 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:53:55.472 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:53:55.523 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:53:55.523 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:53:55.525 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:55.525 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:53:55.530 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T03:53:55.531 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T03:53:55.531 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:55.532 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T03:53:55.532 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T03:53:55.533 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T03:53:55.533 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:53:55.533 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T03:53:55.534 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:53:55.534 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T03:53:55.535 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T03:53:55.535 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T03:53:55.536 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:53:55.536 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T03:53:55.536 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T03:53:55.537 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T03:53:55.537 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T03:53:55.538 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:53:55.538 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T03:53:55.538 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T03:53:55.539 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T03:53:55.539 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T03:53:55.540 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T03:53:55.540 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:53:55.540 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T03:53:55.541 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:53:55.594 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14596.asok 2021-07-11T03:53:55.596 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14596.asok 2021-07-11T03:53:55.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:55.597 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14596.asok status 2021-07-11T03:53:55.647 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T03:53:55.647 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T03:53:55.648 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T03:53:55.648 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T03:53:55.648 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T03:53:55.648 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T03:53:55.648 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T03:53:55.649 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14596", 2021-07-11T03:53:55.649 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T03:53:55.649 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:53:55.649 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T03:53:55.650 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T03:53:55.650 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4352, 2021-07-11T03:53:55.650 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T03:53:55.650 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T03:53:55.650 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T03:53:55.651 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4352 2021-07-11T03:53:55.651 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:53:55.651 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:53:55.651 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:53:55.652 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:53:55.652 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2700850751 2021-07-11T03:53:55.652 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T03:53:55.652 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:53:55.652 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:53:55.653 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:53:55.653 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:53:55.653 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2700850751 2021-07-11T03:53:55.653 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:53:55.653 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4352 172.21.2.119:0/2700850751", 2021-07-11T03:53:55.654 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/2700850751", 2021-07-11T03:53:55.654 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T03:53:55.654 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 11, 2021-07-11T03:53:55.654 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 33, 2021-07-11T03:53:55.655 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T03:53:55.655 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T03:53:55.655 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T03:53:55.656 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T03:53:55.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:55.658 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:55.658 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:55.663 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:53:55.663 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:53:55.664 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:55.664 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:55.716 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_min_caps_per_client 2021-07-11T03:53:55.775 INFO:teuthology.orchestra.run.gibba004.stdout:{"mds_min_caps_per_client":"100"} 2021-07-11T03:53:55.775 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_min_caps_per_client": "100" } 2021-07-11T03:53:55.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:53:55.776 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:53:55.777 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:53:55.788 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:53:55.789 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:53:55.790 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-07-11T03:53:55.791 DEBUG:teuthology.orchestra.run.gibba019:> import sys 2021-07-11T03:53:55.791 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-07-11T03:53:55.792 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:53:55.792 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:53:55.792 DEBUG:teuthology.orchestra.run.gibba019:> n = 1000 2021-07-11T03:53:55.793 DEBUG:teuthology.orchestra.run.gibba019:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-07-11T03:53:55.793 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:53:55.794 DEBUG:teuthology.orchestra.run.gibba019:> if not os.path.exists(os.path.dirname(abs_path)): 2021-07-11T03:53:55.794 DEBUG:teuthology.orchestra.run.gibba019:> os.makedirs(os.path.dirname(abs_path)) 2021-07-11T03:53:55.794 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:53:55.795 DEBUG:teuthology.orchestra.run.gibba019:> for i in range(0, n): 2021-07-11T03:53:55.795 DEBUG:teuthology.orchestra.run.gibba019:> fname = "{0}_{1}".format(abs_path, i) 2021-07-11T03:53:55.795 DEBUG:teuthology.orchestra.run.gibba019:> with open(fname, '"'"'w'"'"') as f: 2021-07-11T03:53:55.796 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-07-11T03:53:55.796 DEBUG:teuthology.orchestra.run.gibba019:> if True: 2021-07-11T03:53:55.797 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-07-11T03:53:55.797 DEBUG:teuthology.orchestra.run.gibba019:> os.fsync(f.fileno()) 2021-07-11T03:53:55.797 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:54:11.800 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:54:11.806 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:54:11.808 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:54:11.819 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:54:14.113 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:54:14.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b3b4ab700 1 Processor -- start 2021-07-11T03:54:14.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b3b4ab700 1 -- start start 2021-07-11T03:54:14.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b3b4ab700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340a76b0 0x7f8b34066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b3b4ab700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8b340b1970 0x7f8b34066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b3b4ab700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b34067450 con 0x7f8b340b0060 2021-07-11T03:54:14.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b3b4ab700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b340a7a90 con 0x7f8b340b1970 2021-07-11T03:54:14.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b3b4ab700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b34147620 con 0x7f8b340a76b0 2021-07-11T03:54:14.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b38a46700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8b340b1970 0x7f8b34066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b39247700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340a76b0 0x7f8b34066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b38a46700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8b340b1970 0x7f8b34066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:50956/0 (socket says 172.21.2.104:50956) 2021-07-11T03:54:14.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b38a46700 1 -- 172.21.2.104:0/4261576069 learned_addr learned my addr 172.21.2.104:0/4261576069 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:54:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b39247700 1 -- 172.21.2.104:0/4261576069 >> v2:172.21.2.105:3300/0 conn(0x7f8b340b1970 msgr2=0x7f8b34066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b39247700 1 --2- 172.21.2.104:0/4261576069 >> v2:172.21.2.105:3300/0 conn(0x7f8b340b1970 0x7f8b34066f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b39247700 1 -- 172.21.2.104:0/4261576069 >> v1:172.21.2.105:6789/0 conn(0x7f8b340b0060 legacy=0x7f8b3405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b39247700 1 -- 172.21.2.104:0/4261576069 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b341478c0 con 0x7f8b340a76b0 2021-07-11T03:54:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b39247700 1 --2- 172.21.2.104:0/4261576069 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340a76b0 0x7f8b34066a70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8b28002f70 tx=0x7f8b28006df0).ready entity=mon.2 client_cookie=372765909a5d0924 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.169+0000 7f8b27fff700 1 -- 172.21.2.104:0/4261576069 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b2800d570 con 0x7f8b340a76b0 2021-07-11T03:54:14.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b27fff700 1 -- 172.21.2.104:0/4261576069 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b28000e90 con 0x7f8b340a76b0 2021-07-11T03:54:14.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b27fff700 1 -- 172.21.2.104:0/4261576069 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b2800d8c0 con 0x7f8b340a76b0 2021-07-11T03:54:14.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/4261576069 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340a76b0 msgr2=0x7f8b34066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 --2- 172.21.2.104:0/4261576069 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340a76b0 0x7f8b34066a70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f8b28002f70 tx=0x7f8b28006df0).stop 2021-07-11T03:54:14.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/4261576069 shutdown_connections 2021-07-11T03:54:14.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 --2- 172.21.2.104:0/4261576069 >> v2:172.21.2.105:3300/0 conn(0x7f8b340b1970 0x7f8b34066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 --2- 172.21.2.104:0/4261576069 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340a76b0 0x7f8b34066a70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/4261576069 >> 172.21.2.104:0/4261576069 conn(0x7f8b340b53e0 msgr2=0x7f8b340b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:14.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/4261576069 shutdown_connections 2021-07-11T03:54:14.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/4261576069 wait complete. 2021-07-11T03:54:14.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 Processor -- start 2021-07-11T03:54:14.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- start start 2021-07-11T03:54:14.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b340a76b0 0x7f8b341460e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b340b0060 0x7f8b34156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340b1970 0x7f8b34156d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3414ea10 con 0x7f8b340a76b0 2021-07-11T03:54:14.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340a9300 con 0x7f8b340b0060 2021-07-11T03:54:14.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340a95a0 con 0x7f8b340b1970 2021-07-11T03:54:14.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b39247700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b340a76b0 0x7f8b341460e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b39247700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b340a76b0 0x7f8b341460e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43400/0 (socket says 172.21.2.104:43400) 2021-07-11T03:54:14.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b39247700 1 -- 172.21.2.104:0/877402096 learned_addr learned my addr 172.21.2.104:0/877402096 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:54:14.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b39a48700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340b1970 0x7f8b34156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b38a46700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b340b0060 0x7f8b34156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b39247700 1 -- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340b1970 msgr2=0x7f8b34156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b39247700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340b1970 0x7f8b34156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b39247700 1 -- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b340b0060 msgr2=0x7f8b34156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b39247700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b340b0060 0x7f8b34156810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b39247700 1 -- 172.21.2.104:0/877402096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2800b180 con 0x7f8b340a76b0 2021-07-11T03:54:14.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b39a48700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340b1970 0x7f8b34156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:54:14.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b38a46700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b340b0060 0x7f8b34156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:54:14.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b39247700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b340a76b0 0x7f8b341460e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f8b28002f70 tx=0x7f8b28014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b267fc700 1 -- 172.21.2.104:0/877402096 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b28006b20 con 0x7f8b340a76b0 2021-07-11T03:54:14.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b267fc700 1 -- 172.21.2.104:0/877402096 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2801f440 con 0x7f8b340a76b0 2021-07-11T03:54:14.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b267fc700 1 -- 172.21.2.104:0/877402096 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b2801f7a0 con 0x7f8b340a76b0 2021-07-11T03:54:14.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/877402096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b340a9840 con 0x7f8b340a76b0 2021-07-11T03:54:14.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/877402096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b340a9ce0 con 0x7f8b340a76b0 2021-07-11T03:54:14.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/877402096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b340a99d0 con 0x7f8b340a76b0 2021-07-11T03:54:14.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b267fc700 1 -- 172.21.2.104:0/877402096 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8b28025020 con 0x7f8b340a76b0 2021-07-11T03:54:14.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b267fc700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b140532d0 0x7f8b14055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.173+0000 7f8b267fc700 1 -- 172.21.2.104:0/877402096 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8b2806c6e0 con 0x7f8b340a76b0 2021-07-11T03:54:14.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.177+0000 7f8b38a46700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b140532d0 0x7f8b14055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.177+0000 7f8b38a46700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b140532d0 0x7f8b14055790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f8b30008000 tx=0x7f8b30008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.177+0000 7f8b267fc700 1 -- 172.21.2.104:0/877402096 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8b28043910 con 0x7f8b340a76b0 2021-07-11T03:54:14.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/877402096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b34057350 con 0x7f8b340a76b0 2021-07-11T03:54:14.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b267fc700 1 -- 172.21.2.104:0/877402096 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f8b28046f30 con 0x7f8b340a76b0 2021-07-11T03:54:14.295 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:54:14.296 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4408,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2474981992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2474981992},{"type":"v1","addr":"172.21.2.105:6835","nonce":2474981992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/1241687766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1241687766},{"type":"v1","addr":"172.21.2.104:6834","nonce":1241687766}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:50.608622+0000","modified":"2021-07-11T03:53:51.611729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/1104530072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1104530072},{"type":"v1","addr":"172.21.2.104:6835","nonce":1104530072}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-07-11T03:54:14.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b140532d0 msgr2=0x7f8b14055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b140532d0 0x7f8b14055790 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f8b30008000 tx=0x7f8b30008040).stop 2021-07-11T03:54:14.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/877402096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b340a76b0 msgr2=0x7f8b341460e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b340a76b0 0x7f8b341460e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f8b28002f70 tx=0x7f8b28014000).stop 2021-07-11T03:54:14.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/877402096 shutdown_connections 2021-07-11T03:54:14.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b340b1970 0x7f8b34156d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b340b0060 0x7f8b34156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b140532d0 0x7f8b14055790 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 --2- 172.21.2.104:0/877402096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b340a76b0 0x7f8b341460e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/877402096 >> 172.21.2.104:0/877402096 conn(0x7f8b340b53e0 msgr2=0x7f8b34149910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:14.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/877402096 shutdown_connections 2021-07-11T03:54:14.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.289+0000 7f8b3b4ab700 1 -- 172.21.2.104:0/877402096 wait complete. 2021-07-11T03:54:14.300 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 11 2021-07-11T03:54:14.303 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_caps 20 2021-07-11T03:54:14.362 INFO:teuthology.orchestra.run.gibba004.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-07-11T03:54:14.363 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-07-11T03:54:14.364 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:54:14.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec180c6700 1 Processor -- start 2021-07-11T03:54:14.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec180c6700 1 -- start start 2021-07-11T03:54:14.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec180c6700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fec100a76b0 0x7fec10066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec180c6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fec100b1970 0x7fec10066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec180c6700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec10067450 con 0x7fec100b0060 2021-07-11T03:54:14.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec180c6700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec100a7a90 con 0x7fec100a76b0 2021-07-11T03:54:14.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec180c6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec10147620 con 0x7fec100b1970 2021-07-11T03:54:14.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec16663700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fec100b0060 0x7fec1005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40178/0 (socket says 172.21.2.104:40178) 2021-07-11T03:54:14.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec16663700 1 -- 172.21.2.104:0/1796635381 learned_addr learned my addr 172.21.2.104:0/1796635381 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:54:14.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1313062941 0 0) 0x7fec10067450 con 0x7fec100b0060 2021-07-11T03:54:14.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febf8003260 con 0x7fec100b0060 2021-07-11T03:54:14.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4091536740 0 0) 0x7febf8003260 con 0x7fec100b0060 2021-07-11T03:54:14.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fec100b1970 msgr2=0x7fec10066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 --2- 172.21.2.104:0/1796635381 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fec100b1970 0x7fec10066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 >> v2:172.21.2.105:3300/0 conn(0x7fec100a76b0 msgr2=0x7fec10066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 --2- 172.21.2.104:0/1796635381 >> v2:172.21.2.105:3300/0 conn(0x7fec100a76b0 0x7fec10066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec101478c0 con 0x7fec100b0060 2021-07-11T03:54:14.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fec080019e0 con 0x7fec100b0060 2021-07-11T03:54:14.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 >> v1:172.21.2.104:6789/0 conn(0x7fec100b0060 legacy=0x7fec1005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 --2- 172.21.2.104:0/1796635381 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fec100b1970 0x7febf80037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 --2- 172.21.2.104:0/1796635381 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febf8003ee0 0x7febf8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 --2- 172.21.2.104:0/1796635381 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7febf8004bf0 0x7febf800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec100a7a90 con 0x7febf8003ee0 2021-07-11T03:54:14.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec10147620 con 0x7febf8004bf0 2021-07-11T03:54:14.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf80053a0 con 0x7fec100b1970 2021-07-11T03:54:14.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec15661700 1 --2- 172.21.2.104:0/1796635381 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febf8003ee0 0x7febf8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec15661700 1 -- 172.21.2.104:0/1796635381 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fec100b1970 msgr2=0x7febf80037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec15661700 1 --2- 172.21.2.104:0/1796635381 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fec100b1970 0x7febf80037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec15661700 1 -- 172.21.2.104:0/1796635381 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7febf8004bf0 msgr2=0x7febf800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec15661700 1 --2- 172.21.2.104:0/1796635381 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7febf8004bf0 0x7febf800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec15661700 1 -- 172.21.2.104:0/1796635381 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf8003260 con 0x7febf8003ee0 2021-07-11T03:54:14.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec15661700 1 --2- 172.21.2.104:0/1796635381 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febf8003ee0 0x7febf8004500 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fec0c002f70 tx=0x7fec0c006ca0).ready entity=mon.0 client_cookie=98d9237acbee9ae2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.409+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0c012410 con 0x7febf8003ee0 2021-07-11T03:54:14.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec0c012570 con 0x7febf8003ee0 2021-07-11T03:54:14.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec14e60700 1 -- 172.21.2.104:0/1796635381 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec0c0128d0 con 0x7febf8003ee0 2021-07-11T03:54:14.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- 172.21.2.104:0/1796635381 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febf8003ee0 msgr2=0x7febf8004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 --2- 172.21.2.104:0/1796635381 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febf8003ee0 0x7febf8004500 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fec0c002f70 tx=0x7fec0c006ca0).stop 2021-07-11T03:54:14.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec16663700 1 -- 172.21.2.104:0/1796635381 reap_dead start 2021-07-11T03:54:14.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- 172.21.2.104:0/1796635381 shutdown_connections 2021-07-11T03:54:14.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- 172.21.2.104:0/1796635381 >> 172.21.2.104:0/1796635381 conn(0x7fec100b53e0 msgr2=0x7fec100b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:14.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- 172.21.2.104:0/1796635381 shutdown_connections 2021-07-11T03:54:14.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- 172.21.2.104:0/1796635381 wait complete. 2021-07-11T03:54:14.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 Processor -- start 2021-07-11T03:54:14.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- start start 2021-07-11T03:54:14.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fec100b42b0 0x7fec100a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fec100abd80 0x7fec100ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fec100ada00 0x7fec10144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec100b4690 con 0x7fec100ada00 2021-07-11T03:54:14.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec100ae2b0 con 0x7fec100b42b0 2021-07-11T03:54:14.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec100ae550 con 0x7fec100abd80 2021-07-11T03:54:14.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec16663700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fec100ada00 0x7fec10144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec16663700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fec100ada00 0x7fec10144690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43424/0 (socket says 172.21.2.104:43424) 2021-07-11T03:54:14.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec16663700 1 -- 172.21.2.104:0/2993311633 learned_addr learned my addr 172.21.2.104:0/2993311633 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:54:14.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec16663700 1 -- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fec100abd80 msgr2=0x7fec100ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec16663700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fec100abd80 0x7fec100ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec16663700 1 -- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fec100b42b0 msgr2=0x7fec100a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec16663700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fec100b42b0 0x7fec100a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec16663700 1 -- 172.21.2.104:0/2993311633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec100a93b0 con 0x7fec100ada00 2021-07-11T03:54:14.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec16663700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fec100ada00 0x7fec10144690 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fec08001490 tx=0x7fec08001a40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7febfeffd700 1 -- 172.21.2.104:0/2993311633 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec08004b70 con 0x7fec100ada00 2021-07-11T03:54:14.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7febfeffd700 1 -- 172.21.2.104:0/2993311633 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec08004cd0 con 0x7fec100ada00 2021-07-11T03:54:14.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- 172.21.2.104:0/2993311633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec100af520 con 0x7fec100ada00 2021-07-11T03:54:14.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7febfeffd700 1 -- 172.21.2.104:0/2993311633 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec080072c0 con 0x7fec100ada00 2021-07-11T03:54:14.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- 172.21.2.104:0/2993311633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec100ab790 con 0x7fec100ada00 2021-07-11T03:54:14.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7fec180c6700 1 -- 172.21.2.104:0/2993311633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec100a95b0 con 0x7fec100ada00 2021-07-11T03:54:14.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7febfeffd700 1 -- 172.21.2.104:0/2993311633 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fec0801c070 con 0x7fec100ada00 2021-07-11T03:54:14.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7febfeffd700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febf0053160 0x7febf0055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.413+0000 7febfeffd700 1 -- 172.21.2.104:0/2993311633 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fec0806aa20 con 0x7fec100ada00 2021-07-11T03:54:14.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.417+0000 7fec15e62700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febf0053160 0x7febf0055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.417+0000 7fec15e62700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febf0053160 0x7febf0055620 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fec00002f70 tx=0x7fec00006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.417+0000 7febfeffd700 1 -- 172.21.2.104:0/2993311633 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fec100a95b0 con 0x7fec100ada00 2021-07-11T03:54:14.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 -- 172.21.2.104:0/2993311633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec100a95b0 con 0x7fec100ada00 2021-07-11T03:54:14.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7febfeffd700 1 -- 172.21.2.104:0/2993311633 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2489 (secure 0 0 0) 0x7fec100a95b0 con 0x7fec100ada00 2021-07-11T03:54:14.535 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:54:14.535 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4408,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2474981992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2474981992},{"type":"v1","addr":"172.21.2.105:6835","nonce":2474981992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/1241687766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1241687766},{"type":"v1","addr":"172.21.2.104:6834","nonce":1241687766}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:50.608622+0000","modified":"2021-07-11T03:53:51.611729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/1104530072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1104530072},{"type":"v1","addr":"172.21.2.104:6835","nonce":1104530072}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-07-11T03:54:14.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 -- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febf0053160 msgr2=0x7febf0055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febf0053160 0x7febf0055620 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fec00002f70 tx=0x7fec00006a80).stop 2021-07-11T03:54:14.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 -- 172.21.2.104:0/2993311633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fec100ada00 msgr2=0x7fec10144690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fec100ada00 0x7fec10144690 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fec08001490 tx=0x7fec08001a40).stop 2021-07-11T03:54:14.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 -- 172.21.2.104:0/2993311633 shutdown_connections 2021-07-11T03:54:14.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febf0053160 0x7febf0055620 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fec100ada00 0x7fec10144690 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fec100abd80 0x7fec100ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 --2- 172.21.2.104:0/2993311633 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fec100b42b0 0x7fec100a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 -- 172.21.2.104:0/2993311633 >> 172.21.2.104:0/2993311633 conn(0x7fec100b53e0 msgr2=0x7fec10149cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:14.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 -- 172.21.2.104:0/2993311633 shutdown_connections 2021-07-11T03:54:14.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.529+0000 7fec180c6700 1 -- 172.21.2.104:0/2993311633 wait complete. 2021-07-11T03:54:14.539 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 11 2021-07-11T03:54:14.542 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_recall_max_decay_threshold 400 2021-07-11T03:54:14.606 INFO:teuthology.orchestra.run.gibba004.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-07-11T03:54:14.607 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-07-11T03:54:14.607 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:54:14.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 Processor -- start 2021-07-11T03:54:14.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- start start 2021-07-11T03:54:14.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f67f80b7810 0x7f67f81445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f80b31d0 con 0x7f67f8144a90 2021-07-11T03:54:14.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f80b7bf0 con 0x7f67f80b1170 2021-07-11T03:54:14.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f8144f20 con 0x7f67f80b7810 2021-07-11T03:54:14.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f67f80b7810 0x7f67f81445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f77fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f67f8144a90 0x7f67f8064f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40196/0 (socket says 172.21.2.104:40196) 2021-07-11T03:54:14.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f77fe700 1 -- 172.21.2.104:0/2850947047 learned_addr learned my addr 172.21.2.104:0/2850947047 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:54:14.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 -- 172.21.2.104:0/2850947047 >> v1:172.21.2.105:6789/0 conn(0x7f67f80b1170 legacy=0x7f67f8064870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:54:14.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 -- 172.21.2.104:0/2850947047 >> v1:172.21.2.104:6789/0 conn(0x7f67f8144a90 legacy=0x7f67f8064f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 -- 172.21.2.104:0/2850947047 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f80675c0 con 0x7f67f80b7810 2021-07-11T03:54:14.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 --2- 172.21.2.104:0/2850947047 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67f80b7810 0x7f67f81445a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f67f8060740 tx=0x7f67e8006df0).ready entity=mon.0 client_cookie=d70af620452133f7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f5ffb700 1 -- 172.21.2.104:0/2850947047 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67e800d5c0 con 0x7f67f80b7810 2021-07-11T03:54:14.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f5ffb700 1 -- 172.21.2.104:0/2850947047 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e8000e90 con 0x7f67f80b7810 2021-07-11T03:54:14.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f5ffb700 1 -- 172.21.2.104:0/2850947047 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67e800d910 con 0x7f67f80b7810 2021-07-11T03:54:14.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- 172.21.2.104:0/2850947047 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67f80b7810 msgr2=0x7f67f81445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 --2- 172.21.2.104:0/2850947047 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67f80b7810 0x7f67f81445a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f67f8060740 tx=0x7f67e8006df0).stop 2021-07-11T03:54:14.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- 172.21.2.104:0/2850947047 shutdown_connections 2021-07-11T03:54:14.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 --2- 172.21.2.104:0/2850947047 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67f80b7810 0x7f67f81445a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- 172.21.2.104:0/2850947047 >> 172.21.2.104:0/2850947047 conn(0x7f67f805f6b0 msgr2=0x7f67f805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:14.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- 172.21.2.104:0/2850947047 shutdown_connections 2021-07-11T03:54:14.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- 172.21.2.104:0/2850947047 wait complete. 2021-07-11T03:54:14.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 Processor -- start 2021-07-11T03:54:14.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- start start 2021-07-11T03:54:14.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67f80b1170 0x7f67f80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67f80b7810 0x7f67f80abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67f8144a90 0x7f67f80ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f813e940 con 0x7f67f80b1170 2021-07-11T03:54:14.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f813ebe0 con 0x7f67f8144a90 2021-07-11T03:54:14.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f813ee80 con 0x7f67f80b7810 2021-07-11T03:54:14.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67f80b1170 0x7f67f80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67f80b1170 0x7f67f80affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43434/0 (socket says 172.21.2.104:43434) 2021-07-11T03:54:14.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 -- 172.21.2.104:0/3597751737 learned_addr learned my addr 172.21.2.104:0/3597751737 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:54:14.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 -- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67f80b7810 msgr2=0x7f67f80abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67f80b7810 0x7f67f80abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 -- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67f8144a90 msgr2=0x7f67f80ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67f8144a90 0x7f67f80ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 -- 172.21.2.104:0/3597751737 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e800b180 con 0x7f67f80b1170 2021-07-11T03:54:14.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67f6ffd700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67f80b1170 0x7f67f80affb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f67f8060740 tx=0x7f67e8012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67effff700 1 -- 172.21.2.104:0/3597751737 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67e8002f50 con 0x7f67f80b1170 2021-07-11T03:54:14.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67effff700 1 -- 172.21.2.104:0/3597751737 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e8017030 con 0x7f67f80b1170 2021-07-11T03:54:14.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67fd409700 1 -- 172.21.2.104:0/3597751737 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f813e5c0 con 0x7f67f80b1170 2021-07-11T03:54:14.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.657+0000 7f67effff700 1 -- 172.21.2.104:0/3597751737 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f67e801d470 con 0x7f67f80b1170 2021-07-11T03:54:14.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.661+0000 7f67fd409700 1 -- 172.21.2.104:0/3597751737 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f80aed80 con 0x7f67f80b1170 2021-07-11T03:54:14.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.661+0000 7f67fd409700 1 -- 172.21.2.104:0/3597751737 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f813f040 con 0x7f67f80b1170 2021-07-11T03:54:14.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.661+0000 7f67effff700 1 -- 172.21.2.104:0/3597751737 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f67e8024030 con 0x7f67f80b1170 2021-07-11T03:54:14.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.661+0000 7f67effff700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67dc053160 0x7f67dc055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.661+0000 7f67effff700 1 -- 172.21.2.104:0/3597751737 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f67e806c2f0 con 0x7f67f80b1170 2021-07-11T03:54:14.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.661+0000 7f67f67fc700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67dc053160 0x7f67dc055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.661+0000 7f67f67fc700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67dc053160 0x7f67dc055620 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f67e0002fb0 tx=0x7f67e00069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.661+0000 7f67effff700 1 -- 172.21.2.104:0/3597751737 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f67e8043520 con 0x7f67f80b1170 2021-07-11T03:54:14.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.769+0000 7f67fd409700 1 -- 172.21.2.104:0/3597751737 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67f8057350 con 0x7f67f80b1170 2021-07-11T03:54:14.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67effff700 1 -- 172.21.2.104:0/3597751737 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f67e8022020 con 0x7f67f80b1170 2021-07-11T03:54:14.778 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:54:14.778 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4408,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2474981992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2474981992},{"type":"v1","addr":"172.21.2.105:6835","nonce":2474981992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/1241687766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1241687766},{"type":"v1","addr":"172.21.2.104:6834","nonce":1241687766}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:50.608622+0000","modified":"2021-07-11T03:53:51.611729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/1104530072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1104530072},{"type":"v1","addr":"172.21.2.104:6835","nonce":1104530072}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-07-11T03:54:14.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 -- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67dc053160 msgr2=0x7f67dc055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67dc053160 0x7f67dc055620 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f67e0002fb0 tx=0x7f67e00069b0).stop 2021-07-11T03:54:14.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 -- 172.21.2.104:0/3597751737 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67f80b1170 msgr2=0x7f67f80affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67f80b1170 0x7f67f80affb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f67f8060740 tx=0x7f67e8012000).stop 2021-07-11T03:54:14.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 -- 172.21.2.104:0/3597751737 shutdown_connections 2021-07-11T03:54:14.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67f8144a90 0x7f67f80ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67f80b7810 0x7f67f80abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67dc053160 0x7f67dc055620 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 --2- 172.21.2.104:0/3597751737 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67f80b1170 0x7f67f80affb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 -- 172.21.2.104:0/3597751737 >> 172.21.2.104:0/3597751737 conn(0x7f67f805f6b0 msgr2=0x7f67f80b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:14.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 -- 172.21.2.104:0/3597751737 shutdown_connections 2021-07-11T03:54:14.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.773+0000 7f67fd409700 1 -- 172.21.2.104:0/3597751737 wait complete. 2021-07-11T03:54:14.782 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 11 2021-07-11T03:54:14.785 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a cache drop 2021-07-11T03:54:14.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.829+0000 7fcac339f700 1 Processor -- start 2021-07-11T03:54:14.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.829+0000 7fcac339f700 1 -- start start 2021-07-11T03:54:14.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.829+0000 7fcac339f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcabc0aff10 0x7fcabc05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.829+0000 7fcac339f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcabc061360 con 0x7fcabc14ab80 2021-07-11T03:54:14.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.829+0000 7fcac339f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcabc14b5e0 con 0x7fcabc0ac9b0 2021-07-11T03:54:14.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.829+0000 7fcac339f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcabc14b880 con 0x7fcabc0aff10 2021-07-11T03:54:14.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.829+0000 7fcac093a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcabc0aff10 0x7fcabc05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.829+0000 7fcac113b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fcabc14ab80 0x7fcabc0af1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40210/0 (socket says 172.21.2.104:40210) 2021-07-11T03:54:14.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.829+0000 7fcac113b700 1 -- 172.21.2.104:0/3786041667 learned_addr learned my addr 172.21.2.104:0/3786041667 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:54:14.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcaaffff700 1 -- 172.21.2.104:0/3786041667 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921181682 0 0) 0x7fcabc061360 con 0x7fcabc14ab80 2021-07-11T03:54:14.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcaaffff700 1 -- 172.21.2.104:0/3786041667 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa4003260 con 0x7fcabc14ab80 2021-07-11T03:54:14.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac093a700 1 -- 172.21.2.104:0/3786041667 >> v1:172.21.2.105:6789/0 conn(0x7fcabc0ac9b0 legacy=0x7fcabc0b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac093a700 1 -- 172.21.2.104:0/3786041667 >> v1:172.21.2.104:6789/0 conn(0x7fcabc14ab80 legacy=0x7fcabc0af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac093a700 1 -- 172.21.2.104:0/3786041667 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc0aed20 con 0x7fcabc0aff10 2021-07-11T03:54:14.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac093a700 1 --2- 172.21.2.104:0/3786041667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcabc0aff10 0x7fcabc05f6b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fcabc0ac720 tx=0x7fcab8006df0).ready entity=mon.0 client_cookie=516dbdeeb1eb4c90 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcaaffff700 1 -- 172.21.2.104:0/3786041667 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab800d5c0 con 0x7fcabc0aff10 2021-07-11T03:54:14.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcaaffff700 1 -- 172.21.2.104:0/3786041667 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab8000e90 con 0x7fcabc0aff10 2021-07-11T03:54:14.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcaaffff700 1 -- 172.21.2.104:0/3786041667 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab800d910 con 0x7fcabc0aff10 2021-07-11T03:54:14.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 -- 172.21.2.104:0/3786041667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcabc0aff10 msgr2=0x7fcabc05f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 --2- 172.21.2.104:0/3786041667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcabc0aff10 0x7fcabc05f6b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fcabc0ac720 tx=0x7fcab8006df0).stop 2021-07-11T03:54:14.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 -- 172.21.2.104:0/3786041667 shutdown_connections 2021-07-11T03:54:14.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 --2- 172.21.2.104:0/3786041667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcabc0aff10 0x7fcabc05f6b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 -- 172.21.2.104:0/3786041667 >> 172.21.2.104:0/3786041667 conn(0x7fcabc0b5570 msgr2=0x7fcabc0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:14.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 -- 172.21.2.104:0/3786041667 shutdown_connections 2021-07-11T03:54:14.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 -- 172.21.2.104:0/3786041667 wait complete. 2021-07-11T03:54:14.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 Processor -- start 2021-07-11T03:54:14.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 -- start start 2021-07-11T03:54:14.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcabc0ac9b0 0x7fcabc13cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcabc0aff10 0x7fcabc156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcabc14ab80 0x7fcabc1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc157690 con 0x7fcabc0ac9b0 2021-07-11T03:54:14.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc0b4b30 con 0x7fcabc0aff10 2021-07-11T03:54:14.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc13dae0 con 0x7fcabc14ab80 2021-07-11T03:54:14.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac113b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcabc0ac9b0 0x7fcabc13cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac113b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcabc0ac9b0 0x7fcabc13cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43448/0 (socket says 172.21.2.104:43448) 2021-07-11T03:54:14.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac113b700 1 -- 172.21.2.104:0/270894455 learned_addr learned my addr 172.21.2.104:0/270894455 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:54:14.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac113b700 1 -- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcabc14ab80 msgr2=0x7fcabc1571a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:54:14.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac113b700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcabc14ab80 0x7fcabc1571a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac113b700 1 -- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcabc0aff10 msgr2=0x7fcabc156cb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:54:14.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac113b700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcabc0aff10 0x7fcabc156cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac113b700 1 -- 172.21.2.104:0/270894455 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab800b180 con 0x7fcabc0ac9b0 2021-07-11T03:54:14.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac113b700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcabc0ac9b0 0x7fcabc13cda0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fcab0001110 tx=0x7fcab0007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcaae7fc700 1 -- 172.21.2.104:0/270894455 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab0007e10 con 0x7fcabc0ac9b0 2021-07-11T03:54:14.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcaae7fc700 1 -- 172.21.2.104:0/270894455 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab00072e0 con 0x7fcabc0ac9b0 2021-07-11T03:54:14.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcaae7fc700 1 -- 172.21.2.104:0/270894455 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab0005800 con 0x7fcabc0ac9b0 2021-07-11T03:54:14.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 -- 172.21.2.104:0/270894455 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcabc0b4d60 con 0x7fcabc0ac9b0 2021-07-11T03:54:14.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.833+0000 7fcac339f700 1 -- 172.21.2.104:0/270894455 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcabc0b52a0 con 0x7fcabc0ac9b0 2021-07-11T03:54:14.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 Processor -- start 2021-07-11T03:54:14.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 -- start start 2021-07-11T03:54:14.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x145f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1495c90 0x1496110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1496600 0x149fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x13bd0d0 con 0x149a970 2021-07-11T03:54:14.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x13cfb90 con 0x1496600 2021-07-11T03:54:14.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x13cfcd0 con 0x1495c90 2021-07-11T03:54:14.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac193c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x145f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac193c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x145f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43454/0 (socket says 172.21.2.104:43454) 2021-07-11T03:54:14.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac193c700 1 -- 172.21.2.104:0/225912509 learned_addr learned my addr 172.21.2.104:0/225912509 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:54:14.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcaae7fc700 1 -- 172.21.2.104:0/270894455 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcab0010070 con 0x7fcabc0ac9b0 2021-07-11T03:54:14.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac193c700 1 -- 172.21.2.104:0/225912509 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1495c90 msgr2=0x1496110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac193c700 1 --2- 172.21.2.104:0/225912509 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1495c90 0x1496110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac193c700 1 -- 172.21.2.104:0/225912509 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1496600 msgr2=0x149fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac193c700 1 --2- 172.21.2.104:0/225912509 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1496600 0x149fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcaae7fc700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca9c053300 0x7fca9c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac193c700 1 -- 172.21.2.104:0/225912509 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1408d10 con 0x149a970 2021-07-11T03:54:14.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcaae7fc700 1 -- 172.21.2.104:0/270894455 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcab006ac90 con 0x7fcabc0ac9b0 2021-07-11T03:54:14.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac193c700 1 --2- 172.21.2.104:0/225912509 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x145f230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x145fe50 tx=0x7fcab40090f0).ready entity=mon.0 client_cookie=df24dff1df450755 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac339f700 1 -- 172.21.2.104:0/225912509 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab400b600 con 0x149a970 2021-07-11T03:54:14.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac339f700 1 -- 172.21.2.104:0/225912509 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab400b760 con 0x149a970 2021-07-11T03:54:14.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac339f700 1 -- 172.21.2.104:0/225912509 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab400bac0 con 0x149a970 2021-07-11T03:54:14.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 -- 172.21.2.104:0/225912509 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 msgr2=0x145f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/225912509 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x145f230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x145fe50 tx=0x7fcab40090f0).stop 2021-07-11T03:54:14.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac113b700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca9c053300 0x7fca9c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 -- 172.21.2.104:0/225912509 shutdown_connections 2021-07-11T03:54:14.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/225912509 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1496600 0x149fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/225912509 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1495c90 0x1496110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/225912509 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x145f230 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 -- 172.21.2.104:0/225912509 >> 172.21.2.104:0/225912509 conn(0x145d8f0 msgr2=0x145eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:14.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 -- 172.21.2.104:0/225912509 shutdown_connections 2021-07-11T03:54:14.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 -- 172.21.2.104:0/225912509 wait complete. 2021-07-11T03:54:14.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 -- start start 2021-07-11T03:54:14.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 Processor -- start 2021-07-11T03:54:14.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcac113b700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca9c053300 0x7fca9c0557c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fcab006d010 tx=0x7fcab0010000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1496600 0x1492ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.837+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x149a970 0x1492fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x14647c0 0x14a5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x13d1300 con 0x14647c0 2021-07-11T03:54:14.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x1466d80 con 0x149a970 2021-07-11T03:54:14.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x14934c0 con 0x1496600 2021-07-11T03:54:14.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 10 client.0 ll_register_callbacks cb 0x14182f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-07-11T03:54:14.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcac193c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x14647c0 0x14a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcac193c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x14647c0 0x14a5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43466/0 (socket says 172.21.2.104:43466) 2021-07-11T03:54:14.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcac193c700 1 -- 172.21.2.104:0/2986736870 learned_addr learned my addr 172.21.2.104:0/2986736870 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:54:14.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcac193c700 1 -- 172.21.2.104:0/2986736870 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1496600 msgr2=0x1492ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcac193c700 1 --2- 172.21.2.104:0/2986736870 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1496600 0x1492ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcac193c700 1 -- 172.21.2.104:0/2986736870 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x149a970 msgr2=0x1492fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcac193c700 1 --2- 172.21.2.104:0/2986736870 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x149a970 0x1492fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcac193c700 1 -- 172.21.2.104:0/2986736870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab4009390 con 0x14647c0 2021-07-11T03:54:14.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcac193c700 1 --2- 172.21.2.104:0/2986736870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x14647c0 0x14a5890 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fcab4006a20 tx=0x7fcab4016000).ready entity=mon.0 client_cookie=b02d53131e9f7e65 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fca93fff700 10 client.0 ms_handle_connect on v2:172.21.2.104:3300/0 2021-07-11T03:54:14.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fca93fff700 1 -- 172.21.2.104:0/2986736870 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab401e6f0 con 0x14647c0 2021-07-11T03:54:14.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fca93fff700 1 -- 172.21.2.104:0/2986736870 <== mon.0 v2:172.21.2.104:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcab401e850 con 0x14647c0 2021-07-11T03:54:14.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 -- 172.21.2.104:0/2986736870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1249c20 con 0x14647c0 2021-07-11T03:54:14.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fca93fff700 1 -- 172.21.2.104:0/2986736870 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcab401ea10 con 0x14647c0 2021-07-11T03:54:14.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 10 client.5100 fetch_fsmap learned FSMap version 11 2021-07-11T03:54:14.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 -- 172.21.2.104:0/2986736870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) v3 -- 0x137c7c0 con 0x14647c0 2021-07-11T03:54:14.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fca93fff700 1 -- 172.21.2.104:0/2986736870 <== mon.0 v2:172.21.2.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1249c20 con 0x14647c0 2021-07-11T03:54:14.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fca93fff700 1 -- 172.21.2.104:0/2986736870 <== mon.0 v2:172.21.2.104:3300/0 5 ==== fsmap(e 11) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fcab401eb70 con 0x14647c0 2021-07-11T03:54:14.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 10 client.5100 fetch_fsmap finished waiting for FSMap version 11 2021-07-11T03:54:14.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 10 client.5100 resolve_mds: resolved ID 'a' to GID 4863 2021-07-11T03:54:14.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 20 client.5100 populate_metadata read hostname 'gibba004' 2021-07-11T03:54:14.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/2986736870 >> [v2:172.21.2.104:6833/1104530072,v1:172.21.2.104:6835/1104530072] conn(0x14a6250 0x14628c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 4 client.5100 mds_command: new command op to 4863 tid=0[{"prefix": "get_command_descriptions"}] 2021-07-11T03:54:14.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 -- 172.21.2.104:0/2986736870 --> [v2:172.21.2.104:6833/1104530072,v1:172.21.2.104:6835/1104530072] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x125f460 con 0x14a6250 2021-07-11T03:54:14.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcac093a700 1 --2- 172.21.2.104:0/2986736870 >> [v2:172.21.2.104:6833/1104530072,v1:172.21.2.104:6835/1104530072] conn(0x14a6250 0x14628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fca93fff700 10 client.5100 ms_handle_connect on v2:172.21.2.104:6833/1104530072 2021-07-11T03:54:14.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcac093a700 1 --2- 172.21.2.104:0/2986736870 >> [v2:172.21.2.104:6833/1104530072,v1:172.21.2.104:6835/1104530072] conn(0x14a6250 0x14628c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b6163e27043eb43d server_cookie=b14e2206bc022e16 in_seq=0 out_seq=0 2021-07-11T03:54:14.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fca93fff700 1 -- 172.21.2.104:0/2986736870 <== mds.0 v2:172.21.2.104:6833/1104530072 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x125f460 con 0x14a6250 2021-07-11T03:54:14.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fca93fff700 10 client.5100 handle_command_reply: tid=0 2021-07-11T03:54:14.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 client.5100 shutdown 2021-07-11T03:54:14.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 -- 172.21.2.104:0/2986736870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x14647c0 msgr2=0x14a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/2986736870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x14647c0 0x14a5890 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fcab4006a20 tx=0x7fcab4016000).stop 2021-07-11T03:54:14.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 -- 172.21.2.104:0/2986736870 shutdown_connections 2021-07-11T03:54:14.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/2986736870 >> [v2:172.21.2.104:6833/1104530072,v1:172.21.2.104:6835/1104530072] conn(0x14a6250 0x14628c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/2986736870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x14647c0 0x14a5890 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/2986736870 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x149a970 0x1492fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/2986736870 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1496600 0x1492ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fca93fff700 0 client.5100 ms_handle_reset on v2:172.21.2.104:6833/1104530072 2021-07-11T03:54:14.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 -- 172.21.2.104:0/2986736870 >> 172.21.2.104:0/2986736870 conn(0x1495c90 msgr2=0x1497940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:14.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 -- 172.21.2.104:0/2986736870 shutdown_connections 2021-07-11T03:54:14.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 1 -- 172.21.2.104:0/2986736870 wait complete. 2021-07-11T03:54:14.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.841+0000 7fcacd5ef740 20 client.5100 trim_cache size 0 max 16384 2021-07-11T03:54:14.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 Processor -- start 2021-07-11T03:54:14.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- start start 2021-07-11T03:54:14.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x14647c0 0x1465ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1496600 0x1494d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x14a1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1496470 con 0x149a970 2021-07-11T03:54:14.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x1467330 con 0x1496600 2021-07-11T03:54:14.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x125f460 con 0x14647c0 2021-07-11T03:54:14.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcac193c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x14a1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcac193c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x14a1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43474/0 (socket says 172.21.2.104:43474) 2021-07-11T03:54:14.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcac193c700 1 -- 172.21.2.104:0/3509106908 learned_addr learned my addr 172.21.2.104:0/3509106908 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:54:14.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcac193c700 1 -- 172.21.2.104:0/3509106908 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x14647c0 msgr2=0x1465ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcac193c700 1 --2- 172.21.2.104:0/3509106908 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x14647c0 0x1465ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcac193c700 1 -- 172.21.2.104:0/3509106908 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1496600 msgr2=0x1494d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcac193c700 1 --2- 172.21.2.104:0/3509106908 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1496600 0x1494d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcac193c700 1 -- 172.21.2.104:0/3509106908 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc149ec0 con 0x149a970 2021-07-11T03:54:14.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcac193c700 1 --2- 172.21.2.104:0/3509106908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x14a1ac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fcab4000f40 tx=0x7fcab401f000).ready entity=mon.0 client_cookie=672c586b04120201 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fca9a7fc700 1 -- 172.21.2.104:0/3509106908 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab401e6f0 con 0x149a970 2021-07-11T03:54:14.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fca9a7fc700 1 -- 172.21.2.104:0/3509106908 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab401e850 con 0x149a970 2021-07-11T03:54:14.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fca9a7fc700 1 -- 172.21.2.104:0/3509106908 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab401ebf0 con 0x149a970 2021-07-11T03:54:14.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- 172.21.2.104:0/3509106908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 msgr2=0x14a1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:14.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/3509106908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x14a1ac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fcab4000f40 tx=0x7fcab401f000).stop 2021-07-11T03:54:14.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- 172.21.2.104:0/3509106908 shutdown_connections 2021-07-11T03:54:14.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/3509106908 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x149a970 0x14a1ac0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:14.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/3509106908 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1496600 0x1494d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/3509106908 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x14647c0 0x1465ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- 172.21.2.104:0/3509106908 >> 172.21.2.104:0/3509106908 conn(0x1495c90 msgr2=0x1492780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:14.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- 172.21.2.104:0/3509106908 shutdown_connections 2021-07-11T03:54:14.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- 172.21.2.104:0/3509106908 wait complete. 2021-07-11T03:54:14.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- start start 2021-07-11T03:54:14.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 Processor -- start 2021-07-11T03:54:14.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x14647c0 0x149ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1496600 0x14a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x149a970 0x14f5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x149d9c0 con 0x1496600 2021-07-11T03:54:14.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x1495610 con 0x149a970 2021-07-11T03:54:14.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x14f9990 con 0x14647c0 2021-07-11T03:54:14.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.849+0000 7fcacd5ef740 10 client.0 ll_register_callbacks cb 0x7fcab800bfa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-07-11T03:54:14.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcac093a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1496600 0x14a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcac093a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1496600 0x14a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43478/0 (socket says 172.21.2.104:43478) 2021-07-11T03:54:14.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcac093a700 1 -- 172.21.2.104:0/1174500833 learned_addr learned my addr 172.21.2.104:0/1174500833 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:54:14.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcac093a700 1 -- 172.21.2.104:0/1174500833 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x14647c0 msgr2=0x149ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcac093a700 1 --2- 172.21.2.104:0/1174500833 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x14647c0 0x149ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcac093a700 1 -- 172.21.2.104:0/1174500833 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x149a970 msgr2=0x14f5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:14.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcac093a700 1 --2- 172.21.2.104:0/1174500833 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x149a970 0x14f5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:14.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcac093a700 1 -- 172.21.2.104:0/1174500833 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x137c7c0 con 0x1496600 2021-07-11T03:54:14.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcac093a700 1 --2- 172.21.2.104:0/1174500833 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1496600 0x14a2a50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x14f8dc0 tx=0x7fcab8014040).ready entity=mon.0 client_cookie=d8675f5373acb71a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:14.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fca997fa700 10 client.0 ms_handle_connect on v2:172.21.2.104:3300/0 2021-07-11T03:54:14.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fca997fa700 1 -- 172.21.2.104:0/1174500833 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcab800db90 con 0x1496600 2021-07-11T03:54:14.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fca997fa700 1 -- 172.21.2.104:0/1174500833 <== mon.0 v2:172.21.2.104:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcab800dcf0 con 0x1496600 2021-07-11T03:54:14.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcacd5ef740 1 -- 172.21.2.104:0/1174500833 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x12da7d0 con 0x1496600 2021-07-11T03:54:14.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fca997fa700 1 -- 172.21.2.104:0/1174500833 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=11) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcab801f4a0 con 0x1496600 2021-07-11T03:54:14.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcacd5ef740 10 client.5106 fetch_fsmap learned FSMap version 11 2021-07-11T03:54:14.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcacd5ef740 1 -- 172.21.2.104:0/1174500833 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,fsmap=11,monmap=2+}) v3 -- 0x13c0060 con 0x1496600 2021-07-11T03:54:14.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fca997fa700 1 -- 172.21.2.104:0/1174500833 <== mon.0 v2:172.21.2.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x12da7d0 con 0x1496600 2021-07-11T03:54:14.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fca997fa700 1 -- 172.21.2.104:0/1174500833 <== mon.0 v2:172.21.2.104:3300/0 5 ==== fsmap(e 11) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fcab8010070 con 0x1496600 2021-07-11T03:54:14.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcacd5ef740 10 client.5106 fetch_fsmap finished waiting for FSMap version 11 2021-07-11T03:54:14.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcacd5ef740 10 client.5106 resolve_mds: resolved ID 'a' to GID 4863 2021-07-11T03:54:14.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcacd5ef740 20 client.5106 populate_metadata read hostname 'gibba004' 2021-07-11T03:54:14.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/1174500833 >> [v2:172.21.2.104:6833/1104530072,v1:172.21.2.104:6835/1104530072] conn(0x1493600 0x14f6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:14.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcacd5ef740 4 client.5106 mds_command: new command op to 4863 tid=0[{"prefix": "cache drop"}] 2021-07-11T03:54:14.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcacd5ef740 1 -- 172.21.2.104:0/1174500833 --> [v2:172.21.2.104:6833/1104530072,v1:172.21.2.104:6835/1104530072] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x1497920 con 0x1493600 2021-07-11T03:54:14.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcac193c700 1 --2- 172.21.2.104:0/1174500833 >> [v2:172.21.2.104:6833/1104530072,v1:172.21.2.104:6835/1104530072] conn(0x1493600 0x14f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:14.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fcac193c700 1 --2- 172.21.2.104:0/1174500833 >> [v2:172.21.2.104:6833/1104530072,v1:172.21.2.104:6835/1104530072] conn(0x1493600 0x14f6350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=11a84ff8bb93dcbe server_cookie=85f045fa2b457d32 in_seq=0 out_seq=0 2021-07-11T03:54:14.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:14.853+0000 7fca997fa700 10 client.5106 ms_handle_connect on v2:172.21.2.104:6833/1104530072 2021-07-11T03:54:41.899 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:54:41.903 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:54:41.907 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:54:41.911 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:54:59.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fca997fa700 1 -- 172.21.2.104:0/1174500833 <== mds.0 v2:172.21.2.104:6833/1104530072 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+386 (crc 0 0 0) 0x1497920 con 0x1493600 2021-07-11T03:54:59.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fca997fa700 10 client.5106 handle_command_reply: tid=0 2021-07-11T03:54:59.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 1 client.5106 shutdown 2021-07-11T03:54:59.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 1 -- 172.21.2.104:0/1174500833 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1496600 msgr2=0x14a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:59.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/1174500833 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1496600 0x14a2a50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x14f8dc0 tx=0x7fcab8014040).stop 2021-07-11T03:54:59.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 1 -- 172.21.2.104:0/1174500833 shutdown_connections 2021-07-11T03:54:59.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/1174500833 >> [v2:172.21.2.104:6833/1104530072,v1:172.21.2.104:6835/1104530072] conn(0x1493600 0x14f6350 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/1174500833 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x149a970 0x14f5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:59.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/1174500833 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1496600 0x14a2a50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 1 --2- 172.21.2.104:0/1174500833 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x14647c0 0x149ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:59.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 1 -- 172.21.2.104:0/1174500833 >> 172.21.2.104:0/1174500833 conn(0x1495c90 msgr2=0x1467000 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:59.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fca997fa700 0 client.5106 ms_handle_reset on v2:172.21.2.104:6833/1104530072 2021-07-11T03:54:59.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 1 -- 172.21.2.104:0/1174500833 shutdown_connections 2021-07-11T03:54:59.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 1 -- 172.21.2.104:0/1174500833 wait complete. 2021-07-11T03:54:59.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcacd5ef740 20 client.5106 trim_cache size 0 max 16384 2021-07-11T03:54:59.214 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:54:59.214 INFO:teuthology.orchestra.run.gibba004.stdout: "client_recall": { 2021-07-11T03:54:59.215 INFO:teuthology.orchestra.run.gibba004.stdout: "return_code": 0, 2021-07-11T03:54:59.215 INFO:teuthology.orchestra.run.gibba004.stdout: "message": "(0) Success", 2021-07-11T03:54:59.215 INFO:teuthology.orchestra.run.gibba004.stdout: "recalled": 920 2021-07-11T03:54:59.216 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.216 INFO:teuthology.orchestra.run.gibba004.stdout: "flush_journal": { 2021-07-11T03:54:59.217 INFO:teuthology.orchestra.run.gibba004.stdout: "return_code": 0, 2021-07-11T03:54:59.217 INFO:teuthology.orchestra.run.gibba004.stdout: "message": "" 2021-07-11T03:54:59.218 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.218 INFO:teuthology.orchestra.run.gibba004.stdout: "trim_cache": { 2021-07-11T03:54:59.219 INFO:teuthology.orchestra.run.gibba004.stdout: "trimmed": 900 2021-07-11T03:54:59.219 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.219 INFO:teuthology.orchestra.run.gibba004.stdout: "cache": { 2021-07-11T03:54:59.220 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": { 2021-07-11T03:54:59.220 INFO:teuthology.orchestra.run.gibba004.stdout: "items": 2507, 2021-07-11T03:54:59.221 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes": 431776 2021-07-11T03:54:59.221 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.222 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.222 INFO:teuthology.orchestra.run.gibba004.stdout: "duration": 44.344676579000001 2021-07-11T03:54:59.222 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:54:59.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 -- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca9c053300 msgr2=0x7fca9c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:59.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca9c053300 0x7fca9c0557c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fcab006d010 tx=0x7fcab0010000).stop 2021-07-11T03:54:59.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 -- 172.21.2.104:0/270894455 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcabc0ac9b0 msgr2=0x7fcabc13cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:59.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcabc0ac9b0 0x7fcabc13cda0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7fcab0001110 tx=0x7fcab0007ad0).stop 2021-07-11T03:54:59.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 -- 172.21.2.104:0/270894455 shutdown_connections 2021-07-11T03:54:59.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcabc14ab80 0x7fcabc1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:59.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcabc0aff10 0x7fcabc156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:59.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca9c053300 0x7fca9c0557c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 --2- 172.21.2.104:0/270894455 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcabc0ac9b0 0x7fcabc13cda0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 -- 172.21.2.104:0/270894455 >> 172.21.2.104:0/270894455 conn(0x7fcabc0b5570 msgr2=0x7fcabc1489b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:59.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 -- 172.21.2.104:0/270894455 shutdown_connections 2021-07-11T03:54:59.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.197+0000 7fcac339f700 1 -- 172.21.2.104:0/270894455 wait complete. 2021-07-11T03:54:59.232 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T03:54:59.233 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:54:59.241 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:54:59.271 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:54:59.285 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:54:59.298 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T03:54:59.299 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:54:59.299 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:54:59.300 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:54:59.331 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:54:59.332 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:54:59.332 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T03:54:59.333 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T03:54:59.333 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:54:59.343 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14596]: fuse finished with error 0 and tester_r 0 2021-07-11T03:54:59.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:54:59.346 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:54:59.346 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:54:59.352 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T03:54:59.352 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T03:54:59.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:54:59.464 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:54:59.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:54:59.469 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:54:59.469 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:54:59.479 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:54:59.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:54:59.480 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T03:54:59.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:54:59.481 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:54:59.532 INFO:teuthology.orchestra.run.gibba023.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:54:59.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:54:59.534 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T03:54:59.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04b71fb700 1 Processor -- start 2021-07-11T03:54:59.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04b71fb700 1 -- start start 2021-07-11T03:54:59.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04b71fb700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f04b00b0fd0 0x7f04b0144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:59.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04b71fb700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b00b0350 con 0x7f04b01483a0 2021-07-11T03:54:59.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04b71fb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b00b05f0 con 0x7f04b00b1db0 2021-07-11T03:54:59.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04b71fb700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b00a90c0 con 0x7f04b00b0fd0 2021-07-11T03:54:59.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04affff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f04b01483a0 0x7f04b00a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40254/0 (socket says 172.21.2.104:40254) 2021-07-11T03:54:59.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04affff700 1 -- 172.21.2.104:0/383489156 learned_addr learned my addr 172.21.2.104:0/383489156 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:54:59.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644977923 0 0) 0x7f04b00b0350 con 0x7f04b01483a0 2021-07-11T03:54:59.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0490003260 con 0x7f04b01483a0 2021-07-11T03:54:59.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04b5798700 1 --2- 172.21.2.104:0/383489156 >> v2:172.21.2.105:3300/0 conn(0x7f04b00b0fd0 0x7f04b0144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:59.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2048228132 0 0) 0x7f0490003260 con 0x7f04b01483a0 2021-07-11T03:54:59.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 >> v2:172.21.2.105:3300/0 conn(0x7f04b00b0fd0 msgr2=0x7f04b0144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:59.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 --2- 172.21.2.104:0/383489156 >> v2:172.21.2.105:3300/0 conn(0x7f04b00b0fd0 0x7f04b0144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 >> v1:172.21.2.105:6789/0 conn(0x7f04b00b1db0 legacy=0x7f04b00b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:59.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b00b0890 con 0x7f04b01483a0 2021-07-11T03:54:59.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f0498001660 con 0x7f04b01483a0 2021-07-11T03:54:59.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 >> v1:172.21.2.104:6789/0 conn(0x7f04b01483a0 legacy=0x7f04b00a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:59.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 --2- 172.21.2.104:0/383489156 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0490003820 0x7f0490003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:59.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 --2- 172.21.2.104:0/383489156 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 0x7f0490004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:59.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 --2- 172.21.2.104:0/383489156 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0490004f70 0x7f049000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:59.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049000be20 con 0x7f0490004500 2021-07-11T03:54:59.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049000bf90 con 0x7f0490003820 2021-07-11T03:54:59.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f049000c150 con 0x7f0490004f70 2021-07-11T03:54:59.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.641+0000 7f04affff700 1 --2- 172.21.2.104:0/383489156 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 0x7f0490004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:59.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04affff700 1 -- 172.21.2.104:0/383489156 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0490004f70 msgr2=0x7f049000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:59.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04affff700 1 --2- 172.21.2.104:0/383489156 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0490004f70 0x7f049000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:59.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04affff700 1 -- 172.21.2.104:0/383489156 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0490003820 msgr2=0x7f0490003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:54:59.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04affff700 1 --2- 172.21.2.104:0/383489156 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0490003820 0x7f0490003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:54:59.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04affff700 1 -- 172.21.2.104:0/383489156 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0490003260 con 0x7f0490004500 2021-07-11T03:54:59.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b5798700 1 -- 172.21.2.104:0/383489156 reap_dead start 2021-07-11T03:54:59.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04affff700 1 --2- 172.21.2.104:0/383489156 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 0x7f0490004980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f04b00b3170 tx=0x7f04b00b0780).ready entity=mon.0 client_cookie=f481d4939d52ba13 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:59.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04980070d0 con 0x7f0490004500 2021-07-11T03:54:59.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0498007230 con 0x7f0490004500 2021-07-11T03:54:59.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04af7fe700 1 -- 172.21.2.104:0/383489156 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f049800c8c0 con 0x7f0490004500 2021-07-11T03:54:59.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 -- 172.21.2.104:0/383489156 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 msgr2=0x7f0490004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:59.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 --2- 172.21.2.104:0/383489156 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 0x7f0490004980 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f04b00b3170 tx=0x7f04b00b0780).stop 2021-07-11T03:54:59.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 -- 172.21.2.104:0/383489156 shutdown_connections 2021-07-11T03:54:59.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 --2- 172.21.2.104:0/383489156 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 0x7f0490004980 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 -- 172.21.2.104:0/383489156 >> 172.21.2.104:0/383489156 conn(0x7f04b005f6b0 msgr2=0x7f04b005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:59.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 -- 172.21.2.104:0/383489156 shutdown_connections 2021-07-11T03:54:59.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 -- 172.21.2.104:0/383489156 wait complete. 2021-07-11T03:54:59.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 Processor -- start 2021-07-11T03:54:59.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 -- start start 2021-07-11T03:54:59.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 0x7f04b00af920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:59.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f04b00acc70 0x7f04b00b3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:59.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f04b0156f50 0x7f04b014cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:59.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b014e890 con 0x7f0490004500 2021-07-11T03:54:59.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b014eb30 con 0x7f04b0156f50 2021-07-11T03:54:59.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b0156570 con 0x7f04b00acc70 2021-07-11T03:54:59.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b4f97700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 0x7f04b00af920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:59.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b4f97700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 0x7f04b00af920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43500/0 (socket says 172.21.2.104:43500) 2021-07-11T03:54:59.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b4f97700 1 -- 172.21.2.104:0/3172334705 learned_addr learned my addr 172.21.2.104:0/3172334705 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:54:59.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04affff700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f04b00acc70 0x7f04b00b3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:59.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b5798700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f04b0156f50 0x7f04b014cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:59.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b4f97700 1 -- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f04b00acc70 msgr2=0x7f04b00b3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:59.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b4f97700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f04b00acc70 0x7f04b00b3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b4f97700 1 -- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f04b0156f50 msgr2=0x7f04b014cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:59.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b4f97700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f04b0156f50 0x7f04b014cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b4f97700 1 -- 172.21.2.104:0/3172334705 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b0156810 con 0x7f0490004500 2021-07-11T03:54:59.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b5798700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f04b0156f50 0x7f04b014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:54:59.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b4f97700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 0x7f04b00af920 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f04a0001fa0 tx=0x7f04a0007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:59.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04adffb700 1 -- 172.21.2.104:0/3172334705 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a000ce80 con 0x7f0490004500 2021-07-11T03:54:59.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04adffb700 1 -- 172.21.2.104:0/3172334705 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a0000e90 con 0x7f0490004500 2021-07-11T03:54:59.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 -- 172.21.2.104:0/3172334705 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b0156ab0 con 0x7f0490004500 2021-07-11T03:54:59.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04adffb700 1 -- 172.21.2.104:0/3172334705 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f04a0007430 con 0x7f0490004500 2021-07-11T03:54:59.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.645+0000 7f04b71fb700 1 -- 172.21.2.104:0/3172334705 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b014d290 con 0x7f0490004500 2021-07-11T03:54:59.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.649+0000 7f04b71fb700 1 -- 172.21.2.104:0/3172334705 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b0156c40 con 0x7f0490004500 2021-07-11T03:54:59.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.649+0000 7f04adffb700 1 -- 172.21.2.104:0/3172334705 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f04a001a060 con 0x7f0490004500 2021-07-11T03:54:59.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.649+0000 7f04adffb700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0494053270 0x7f0494055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:54:59.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.649+0000 7f04adffb700 1 -- 172.21.2.104:0/3172334705 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f04a006acf0 con 0x7f0490004500 2021-07-11T03:54:59.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.649+0000 7f04affff700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0494053270 0x7f0494055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:54:59.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.649+0000 7f04affff700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0494053270 0x7f0494055730 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f0498017000 tx=0x7f0498013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:54:59.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.649+0000 7f04adffb700 1 -- 172.21.2.104:0/3172334705 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f04a0042030 con 0x7f0490004500 2021-07-11T03:54:59.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.733+0000 7f04b71fb700 1 -- 172.21.2.104:0/3172334705 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f04b0057350 con 0x7f0490004500 2021-07-11T03:54:59.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.733+0000 7f04adffb700 1 -- 172.21.2.104:0/3172334705 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v33) v1 ==== 81+0+33645 (secure 0 0 0) 0x7f04a0018070 con 0x7f0490004500 2021-07-11T03:54:59.740 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:54:59.741 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:54:59.741 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 33, 2021-07-11T03:54:59.741 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T03:54:59.741 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T03:54:59.742 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T03:53:51.612860+0000", 2021-07-11T03:54:59.742 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:54:59.742 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:54:59.742 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T03:54:59.742 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T03:54:59.743 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T03:54:59.743 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T03:54:59.743 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T03:54:59.743 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T03:54:59.744 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T03:54:59.744 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:54:59.744 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T03:54:59.744 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T03:54:59.744 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T03:54:59.745 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T03:54:59.745 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T03:54:59.745 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 6, 2021-07-11T03:54:59.745 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T03:54:59.745 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T03:54:59.746 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T03:54:59.746 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T03:54:59.746 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T03:54:59.746 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.747 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T03:54:59.747 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T03:54:59.747 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T03:54:59.747 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:54:59.747 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:54:59.748 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:54:59.748 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:54:59.748 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:54:59.748 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:54:59.748 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:54:59.749 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:54:59.749 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T03:54:59.749 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T03:54:59.749 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T03:54:59.750 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T03:54:59.750 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T03:54:59.750 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:54:59.750 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:54:59.750 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:54:59.751 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:54:59.751 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:54:59.751 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:54:59.751 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:54:59.751 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.752 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T03:54:59.752 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:54:59.752 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:54:59.752 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:54:59.753 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:54:59.753 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:54:59.753 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:54:59.753 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:54:59.753 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:54:59.754 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:54:59.754 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:54:59.754 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:54:59.754 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:54:59.754 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:54:59.755 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:54:59.755 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:54:59.755 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:54:59.755 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:54:59.756 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:54:59.756 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:54:59.756 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:54:59.756 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:54:59.756 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:54:59.757 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:54:59.757 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:54:59.757 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:54:59.757 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:54:59.757 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.758 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:54:59.758 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:54:59.758 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:54:59.758 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:54:59.759 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:54:59.759 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:54:59.759 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:54:59.759 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:54:59.759 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:54:59.760 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:54:59.760 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:54:59.760 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:54:59.760 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T03:54:59.760 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.761 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:54:59.761 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T03:54:59.761 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.761 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.761 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.762 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T03:54:59.762 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T03:54:59.762 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T03:54:59.762 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:54:59.763 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:54:59.763 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:54:59.763 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:54:59.763 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:54:59.763 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:54:59.764 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:54:59.764 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:54:59.764 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:54:59.764 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:54:59.764 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:54:59.765 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:54:59.765 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:54:59.765 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:54:59.765 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:54:59.765 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:54:59.766 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:54:59.766 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:54:59.766 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:54:59.766 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:54:59.767 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.767 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T03:54:59.767 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:54:59.767 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:54:59.767 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:54:59.768 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:54:59.768 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:54:59.768 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:54:59.768 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:54:59.768 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:54:59.769 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:54:59.769 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:54:59.769 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:54:59.769 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:54:59.769 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:54:59.770 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:54:59.770 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:54:59.770 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:54:59.770 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:54:59.771 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:54:59.771 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:54:59.771 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:54:59.771 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:54:59.771 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:54:59.772 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:54:59.772 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:54:59.772 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:54:59.772 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:54:59.772 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.773 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:54:59.773 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:54:59.773 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:54:59.773 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:54:59.773 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:54:59.774 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:54:59.774 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:54:59.774 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:54:59.774 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:54:59.775 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:54:59.775 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:54:59.775 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:54:59.775 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:54:59.775 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T03:54:59.776 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.776 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.776 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.776 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 5, 2021-07-11T03:54:59.776 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T03:54:59.777 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:48.603543+0000", 2021-07-11T03:54:59.777 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:54:59.777 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:54:59.777 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:54:59.777 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:54:59.778 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:54:59.778 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:54:59.778 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:54:59.778 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:54:59.779 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:54:59.779 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:54:59.779 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:54:59.779 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:54:59.779 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:54:59.780 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:54:59.780 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:54:59.780 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:54:59.780 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:54:59.781 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:54:59.781 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:54:59.781 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:54:59.781 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.781 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "32", 2021-07-11T03:54:59.782 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:54:59.782 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:54:59.782 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:54:59.782 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:54:59.782 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:54:59.783 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:54:59.783 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:54:59.783 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:54:59.783 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:54:59.783 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:54:59.784 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:54:59.784 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:54:59.784 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:54:59.784 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:54:59.785 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:54:59.785 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:54:59.785 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:54:59.785 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:54:59.785 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:54:59.786 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:54:59.786 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:54:59.786 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:54:59.786 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:54:59.787 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:54:59.787 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:54:59.787 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:54:59.787 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.787 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:54:59.788 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:54:59.788 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:54:59.788 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:54:59.788 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:54:59.788 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:54:59.789 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:54:59.789 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:54:59.789 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:54:59.789 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:54:59.789 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:54:59.790 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:54:59.790 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T03:54:59.790 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T03:54:59.790 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T03:54:59.791 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.791 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:54:59.791 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T03:54:59.791 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T03:54:59.791 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.792 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.792 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.792 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.792 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 6, 2021-07-11T03:54:59.793 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T03:54:59.793 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:49.585828+0000", 2021-07-11T03:54:59.793 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:54:59.793 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:54:59.793 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:54:59.794 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:54:59.794 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:54:59.794 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:54:59.794 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:54:59.794 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:54:59.795 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:54:59.795 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:54:59.795 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:54:59.795 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:54:59.796 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:54:59.796 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:54:59.796 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:54:59.796 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:54:59.796 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:54:59.797 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:54:59.797 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:54:59.797 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:54:59.797 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.797 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "33", 2021-07-11T03:54:59.798 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:54:59.798 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:54:59.798 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:54:59.798 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:54:59.798 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:54:59.799 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:54:59.799 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:54:59.799 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:54:59.799 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:54:59.800 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:54:59.800 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:54:59.800 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:54:59.800 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:54:59.800 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:54:59.801 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:54:59.801 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:54:59.801 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:54:59.801 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:54:59.801 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:54:59.802 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:54:59.802 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:54:59.802 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:54:59.802 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:54:59.802 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:54:59.803 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:54:59.803 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:54:59.803 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.803 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:54:59.803 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:54:59.804 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:54:59.804 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:54:59.804 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:54:59.804 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:54:59.805 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:54:59.805 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:54:59.805 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:54:59.805 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:54:59.805 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:54:59.806 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:54:59.806 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:54:59.806 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T03:54:59.806 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T03:54:59.806 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.807 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.807 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.807 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:54:59.807 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T03:54:59.807 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.808 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:54:59.808 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T03:54:59.808 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:54:59.808 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:54:59.809 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:54:59.809 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:54:59.809 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:54:59.809 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:54:59.809 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:54:59.810 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:54:59.810 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:54:59.810 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:54:59.810 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:54:59.810 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.811 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.811 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.811 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T03:54:59.811 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:54:59.811 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.812 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.812 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.812 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T03:54:59.812 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:54:59.813 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.813 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.813 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.813 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:54:59.813 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.814 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.814 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.814 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T03:54:59.814 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:54:59.814 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.815 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.815 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.815 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T03:54:59.815 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:54:59.815 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.816 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.816 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.816 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:54:59.816 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.816 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.817 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.817 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T03:54:59.817 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:54:59.817 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.818 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.818 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.818 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T03:54:59.818 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:54:59.818 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.819 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.819 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.819 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:54:59.819 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.819 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.820 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.820 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T03:54:59.820 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:54:59.820 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.820 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.821 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.821 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T03:54:59.821 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:54:59.821 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.822 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.822 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.822 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T03:54:59.822 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T03:54:59.822 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T03:54:59.823 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T03:54:59.823 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:54:59.823 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:54:59.823 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:54:59.823 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.824 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.824 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.824 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:54:59.824 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T03:54:59.824 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:54:59.825 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:54:59.825 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:54:59.825 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:54:59.825 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:54:59.825 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:54:59.826 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:54:59.826 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:54:59.826 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:54:59.826 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:54:59.827 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:54:59.827 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.827 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.827 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.827 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T03:54:59.828 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:54:59.828 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.828 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.828 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.828 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T03:54:59.829 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:54:59.829 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.829 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.829 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.829 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:54:59.830 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.830 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.830 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.830 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T03:54:59.831 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:54:59.831 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.831 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.831 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.831 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T03:54:59.832 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:54:59.832 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.832 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.832 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.832 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:54:59.833 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.833 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.833 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.833 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T03:54:59.833 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:54:59.834 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.834 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.834 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.834 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T03:54:59.834 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:54:59.835 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.835 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.835 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.835 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:54:59.836 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.836 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.836 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.836 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T03:54:59.836 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:54:59.837 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.837 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.837 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.837 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T03:54:59.837 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:54:59.838 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.838 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.838 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.838 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T03:54:59.838 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T03:54:59.839 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T03:54:59.839 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T03:54:59.839 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:54:59.839 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:54:59.840 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:54:59.840 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.840 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.840 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.840 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:54:59.841 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T03:54:59.841 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:54:59.841 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:54:59.841 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:54:59.841 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:54:59.842 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:54:59.842 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:54:59.842 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:54:59.842 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 30, 2021-07-11T03:54:59.842 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:54:59.843 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:54:59.843 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:54:59.843 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.843 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.843 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.844 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T03:54:59.844 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:54:59.844 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.844 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.844 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.845 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T03:54:59.845 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:54:59.845 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.845 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.846 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.846 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:54:59.846 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.846 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.846 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.847 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T03:54:59.847 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:54:59.847 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.847 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.847 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.848 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T03:54:59.848 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:54:59.848 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.848 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.849 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.849 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:54:59.849 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.849 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.849 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.850 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T03:54:59.850 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:54:59.850 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.850 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.850 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.851 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T03:54:59.851 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:54:59.851 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.851 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.851 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.852 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:54:59.852 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.852 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.852 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.852 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T03:54:59.853 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:54:59.853 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.853 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.853 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.854 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T03:54:59.854 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:54:59.854 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.854 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.854 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.855 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T03:54:59.855 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T03:54:59.855 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T03:54:59.855 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T03:54:59.855 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:54:59.856 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:54:59.856 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:54:59.856 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.856 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.857 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.857 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:54:59.857 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T03:54:59.857 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:54:59.857 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:54:59.858 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:54:59.858 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:54:59.858 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:54:59.858 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:54:59.858 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:54:59.859 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:54:59.859 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:54:59.859 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:54:59.859 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:54:59.859 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.860 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.860 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.860 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T03:54:59.860 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:54:59.860 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.861 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.861 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.861 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T03:54:59.861 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:54:59.862 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.862 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.862 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.862 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:54:59.862 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.863 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.863 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.863 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T03:54:59.863 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:54:59.863 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.864 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.864 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.864 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T03:54:59.864 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:54:59.864 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.865 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.865 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.865 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:54:59.865 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.865 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.866 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.866 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T03:54:59.866 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:54:59.866 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.867 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.867 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.867 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T03:54:59.867 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:54:59.867 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.868 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.868 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.868 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:54:59.868 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.868 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.869 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.869 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T03:54:59.869 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:54:59.869 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.869 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.870 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.870 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T03:54:59.870 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:54:59.870 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.871 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.871 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.871 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T03:54:59.871 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T03:54:59.871 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T03:54:59.872 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T03:54:59.872 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:54:59.872 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:54:59.872 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:54:59.872 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.873 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.873 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.873 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:54:59.873 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T03:54:59.873 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:54:59.874 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:54:59.874 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:54:59.874 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:54:59.874 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:54:59.875 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:54:59.875 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:54:59.875 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:54:59.875 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:54:59.875 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:54:59.876 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:54:59.876 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.876 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.876 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.876 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T03:54:59.877 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:54:59.877 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.877 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.877 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.877 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T03:54:59.878 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:54:59.878 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.878 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.878 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.879 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:54:59.879 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.879 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.879 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.879 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T03:54:59.880 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:54:59.880 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.880 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.880 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.880 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T03:54:59.881 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:54:59.881 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.881 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.881 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.881 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:54:59.882 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.882 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.882 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.882 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T03:54:59.883 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:54:59.883 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.883 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.883 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.883 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T03:54:59.884 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:54:59.884 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.884 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.884 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.884 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:54:59.885 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.885 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.885 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.885 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T03:54:59.885 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:54:59.886 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.886 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.886 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.886 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T03:54:59.887 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:54:59.887 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.887 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.887 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.887 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T03:54:59.888 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T03:54:59.888 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T03:54:59.888 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T03:54:59.888 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:54:59.888 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:54:59.889 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:54:59.889 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.889 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.889 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.889 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:54:59.890 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T03:54:59.890 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:54:59.890 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:54:59.890 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:54:59.891 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:54:59.891 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:54:59.891 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:54:59.891 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:54:59.891 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:54:59.892 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:54:59.892 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:54:59.892 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:54:59.892 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.892 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.893 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.893 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T03:54:59.893 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:54:59.893 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.893 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.894 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.894 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T03:54:59.894 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:54:59.894 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.895 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.895 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.895 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:54:59.895 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.895 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.896 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.896 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T03:54:59.896 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:54:59.896 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.896 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.897 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.897 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T03:54:59.897 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:54:59.897 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.897 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.898 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.898 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:54:59.898 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.898 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.898 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.899 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T03:54:59.899 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:54:59.899 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.899 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.900 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.900 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T03:54:59.900 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:54:59.900 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.900 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.901 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.901 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:54:59.901 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.901 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.901 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.902 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T03:54:59.902 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:54:59.902 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.902 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.903 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.903 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T03:54:59.903 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:54:59.903 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.903 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.904 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.904 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T03:54:59.904 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T03:54:59.904 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T03:54:59.904 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T03:54:59.905 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:54:59.905 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:54:59.905 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:54:59.905 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.906 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.906 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.906 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:54:59.906 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T03:54:59.906 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:54:59.907 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:54:59.907 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:54:59.907 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:54:59.907 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:54:59.907 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:54:59.908 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:54:59.908 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 30, 2021-07-11T03:54:59.908 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:54:59.908 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:54:59.908 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:54:59.909 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.909 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.909 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.909 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T03:54:59.910 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:54:59.910 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.910 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.910 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.910 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T03:54:59.911 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:54:59.911 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.911 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.911 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.911 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:54:59.912 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.912 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.912 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.912 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T03:54:59.912 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:54:59.913 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.913 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.913 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.913 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T03:54:59.913 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:54:59.914 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.914 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.914 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.914 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:54:59.915 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.915 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.915 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.915 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T03:54:59.915 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:54:59.916 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.916 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.916 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.916 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T03:54:59.916 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:54:59.917 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.917 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.917 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.917 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:54:59.917 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.918 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.918 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.918 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T03:54:59.918 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:54:59.919 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.919 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.919 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.919 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T03:54:59.919 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:54:59.920 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.920 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.920 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.920 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T03:54:59.920 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T03:54:59.921 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T03:54:59.921 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T03:54:59.921 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:54:59.921 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:54:59.921 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:54:59.922 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.922 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.922 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.922 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:54:59.923 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T03:54:59.923 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:54:59.923 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:54:59.923 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:54:59.923 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:54:59.924 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:54:59.924 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:54:59.924 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:54:59.924 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:54:59.924 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:54:59.925 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:54:59.925 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:54:59.925 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.925 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.925 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.926 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T03:54:59.926 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:54:59.926 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.926 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.927 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.927 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T03:54:59.927 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:54:59.927 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.927 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.928 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.928 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:54:59.928 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.928 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.928 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.929 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T03:54:59.929 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:54:59.929 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.929 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.929 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.930 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T03:54:59.930 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:54:59.930 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.930 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.930 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.931 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:54:59.931 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.931 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.931 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.932 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T03:54:59.932 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:54:59.932 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.932 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.932 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.933 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T03:54:59.933 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:54:59.933 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.933 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.933 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.934 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:54:59.934 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:54:59.934 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.934 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:54:59.934 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T03:54:59.935 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:54:59.935 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.935 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.935 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:54:59.936 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T03:54:59.936 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:54:59.936 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.936 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.936 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.937 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T03:54:59.937 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T03:54:59.937 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T03:54:59.937 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T03:54:59.937 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:54:59.938 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:54:59.938 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:54:59.938 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:54:59.938 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.938 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:54:59.939 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T03:54:59.939 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.939 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:54:59.939 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:54:59.940 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:54:59.940 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:54:59.940 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:54:59.940 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:54:59.940 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T03:54:59.941 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:54:59.941 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.941 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.941 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:54:59.941 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:54:59.942 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:54:59.942 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:54:59.942 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:54:59.942 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:54:59.942 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T03:54:59.943 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:54:59.943 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.943 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.943 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:54:59.944 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:54:59.944 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:54:59.944 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:54:59.944 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:54:59.944 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:54:59.945 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T03:54:59.945 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:54:59.945 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.945 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.945 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:54:59.946 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:54:59.946 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:54:59.946 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:54:59.946 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:54:59.946 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:54:59.947 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T03:54:59.947 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:54:59.947 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.947 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.952 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:54:59.952 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:54:59.952 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:54:59.952 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:54:59.953 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:54:59.953 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:54:59.953 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T03:54:59.953 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:54:59.953 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.954 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.954 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:54:59.954 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:54:59.954 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:54:59.954 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:54:59.955 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:54:59.955 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:54:59.955 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T03:54:59.955 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:54:59.956 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.956 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.956 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:54:59.956 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:54:59.956 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:54:59.957 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:54:59.957 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:54:59.957 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:54:59.957 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T03:54:59.957 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:54:59.958 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.958 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:54:59.958 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:54:59.958 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:54:59.959 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:54:59.959 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:54:59.959 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:54:59.959 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:54:59.959 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T03:54:59.960 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:54:59.960 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.960 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:54:59.960 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T03:54:59.960 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T03:54:59.961 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T03:54:59.961 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T03:54:59.961 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": {}, 2021-07-11T03:54:59.961 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T03:54:59.961 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T03:54:59.962 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T03:54:59.962 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T03:54:59.962 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T03:54:59.962 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T03:54:59.963 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T03:54:59.963 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T03:54:59.963 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:54:59.963 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:54:59.963 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T03:54:59.964 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T03:54:59.964 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T03:54:59.964 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T03:54:59.964 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T03:54:59.964 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:54:59.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.733+0000 7f04b71fb700 1 -- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0494053270 msgr2=0x7f0494055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:59.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.733+0000 7f04b71fb700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0494053270 0x7f0494055730 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f0498017000 tx=0x7f0498013000).stop 2021-07-11T03:54:59.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.733+0000 7f04b71fb700 1 -- 172.21.2.104:0/3172334705 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 msgr2=0x7f04b00af920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:54:59.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.733+0000 7f04b71fb700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 0x7f04b00af920 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f04a0001fa0 tx=0x7f04a0007c30).stop 2021-07-11T03:54:59.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.737+0000 7f04b71fb700 1 -- 172.21.2.104:0/3172334705 shutdown_connections 2021-07-11T03:54:59.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.737+0000 7f04b71fb700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f04b0156f50 0x7f04b014cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.737+0000 7f04b71fb700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f04b00acc70 0x7f04b00b3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.737+0000 7f04b71fb700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0494053270 0x7f0494055730 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.737+0000 7f04b71fb700 1 --2- 172.21.2.104:0/3172334705 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0490004500 0x7f04b00af920 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:54:59.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.737+0000 7f04b71fb700 1 -- 172.21.2.104:0/3172334705 >> 172.21.2.104:0/3172334705 conn(0x7f04b005f6b0 msgr2=0x7f04b0147be0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:54:59.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.737+0000 7f04b71fb700 1 -- 172.21.2.104:0/3172334705 shutdown_connections 2021-07-11T03:54:59.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:54:59.737+0000 7f04b71fb700 1 -- 172.21.2.104:0/3172334705 wait complete. 2021-07-11T03:54:59.969 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:55:00.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 Processor -- start 2021-07-11T03:55:00.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 -- start start 2021-07-11T03:55:00.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b98067180 0x7f7b9805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b980a7a90 con 0x7f7b980b1970 2021-07-11T03:55:00.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b98147620 con 0x7f7b980a76b0 2021-07-11T03:55:00.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b981478c0 con 0x7f7b98067180 2021-07-11T03:55:00.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b977fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f7b980b1970 0x7f7b9805fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40278/0 (socket says 172.21.2.104:40278) 2021-07-11T03:55:00.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b977fe700 1 -- 172.21.2.104:0/88317462 learned_addr learned my addr 172.21.2.104:0/88317462 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:00.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220883266 0 0) 0x7f7b980a7a90 con 0x7f7b980b1970 2021-07-11T03:55:00.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b967fc700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b98067180 0x7f7b9805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:00.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b78003260 con 0x7f7b980b1970 2021-07-11T03:55:00.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 311445117 0 0) 0x7f7b78003260 con 0x7f7b980b1970 2021-07-11T03:55:00.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b98067180 msgr2=0x7f7b9805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:00.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b98067180 0x7f7b9805f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:00.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 >> v1:172.21.2.105:6789/0 conn(0x7f7b980a76b0 legacy=0x7f7b98066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:00.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b98147b60 con 0x7f7b980b1970 2021-07-11T03:55:00.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f7b90001a00 con 0x7f7b980b1970 2021-07-11T03:55:00.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 >> v1:172.21.2.104:6789/0 conn(0x7f7b980b1970 legacy=0x7f7b9805fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:00.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7b78003780 0x7f7b78003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7b78004500 0x7f7b78004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b98067180 0x7f7b78004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b981478c0 con 0x7f7b78004500 2021-07-11T03:55:00.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7800b800 con 0x7f7b78003780 2021-07-11T03:55:00.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7800b9c0 con 0x7f7b98067180 2021-07-11T03:55:00.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b967fc700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7b78004500 0x7f7b78004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:00.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b96ffd700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7b78003780 0x7f7b78003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:00.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b977fe700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b98067180 0x7f7b78004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:00.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b96ffd700 1 -- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b98067180 msgr2=0x7f7b78004eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:00.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b96ffd700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b98067180 0x7f7b78004eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:00.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b96ffd700 1 -- 172.21.2.104:0/88317462 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7b78004500 msgr2=0x7f7b78004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:00.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b96ffd700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7b78004500 0x7f7b78004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:00.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b96ffd700 1 -- 172.21.2.104:0/88317462 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b78003260 con 0x7f7b78003780 2021-07-11T03:55:00.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b96ffd700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7b78003780 0x7f7b78003ba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f7b980675a0 tx=0x7f7b88007a00).ready entity=mon.1 client_cookie=db8f6a7cdaf630d6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:00.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b88007d10 con 0x7f7b78003780 2021-07-11T03:55:00.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b88007030 con 0x7f7b78003780 2021-07-11T03:55:00.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b95ffb700 1 -- 172.21.2.104:0/88317462 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b88007390 con 0x7f7b78003780 2021-07-11T03:55:00.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7b78003780 msgr2=0x7f7b78003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:00.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 --2- 172.21.2.104:0/88317462 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7b78003780 0x7f7b78003ba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f7b980675a0 tx=0x7f7b88007a00).stop 2021-07-11T03:55:00.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b977fe700 1 -- 172.21.2.104:0/88317462 reap_dead start 2021-07-11T03:55:00.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/88317462 shutdown_connections 2021-07-11T03:55:00.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/88317462 >> 172.21.2.104:0/88317462 conn(0x7f7b980b53e0 msgr2=0x7f7b980b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:00.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/88317462 shutdown_connections 2021-07-11T03:55:00.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.021+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/88317462 wait complete. 2021-07-11T03:55:00.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b9d2ed700 1 Processor -- start 2021-07-11T03:55:00.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b9d2ed700 1 -- start start 2021-07-11T03:55:00.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b9d2ed700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b980ac860 0x7f7b980aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b9d2ed700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7b980aea10 0x7f7b980aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b9d2ed700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7b98156740 0x7f7b98156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b9d2ed700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980aa640 con 0x7f7b980aea10 2021-07-11T03:55:00.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b9d2ed700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b981570b0 con 0x7f7b98156740 2021-07-11T03:55:00.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b9d2ed700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980acc40 con 0x7f7b980ac860 2021-07-11T03:55:00.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b967fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7b980aea10 0x7f7b980aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:00.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b967fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7b980aea10 0x7f7b980aee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43518/0 (socket says 172.21.2.104:43518) 2021-07-11T03:55:00.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b967fc700 1 -- 172.21.2.104:0/3761281312 learned_addr learned my addr 172.21.2.104:0/3761281312 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:00.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b96ffd700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b980ac860 0x7f7b980aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:00.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b977fe700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7b98156740 0x7f7b98156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:00.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b967fc700 1 -- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b980ac860 msgr2=0x7f7b980aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:00.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b967fc700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b980ac860 0x7f7b980aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:00.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b967fc700 1 -- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7b98156740 msgr2=0x7f7b98156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:00.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b967fc700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7b98156740 0x7f7b98156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:00.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b967fc700 1 -- 172.21.2.104:0/3761281312 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b980af480 con 0x7f7b980aea10 2021-07-11T03:55:00.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b967fc700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7b980aea10 0x7f7b980aee90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f7b980acf90 tx=0x7f7b80006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:00.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b8ffff700 1 -- 172.21.2.104:0/3761281312 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b80011ea0 con 0x7f7b980aea10 2021-07-11T03:55:00.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b8ffff700 1 -- 172.21.2.104:0/3761281312 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8000f940 con 0x7f7b980aea10 2021-07-11T03:55:00.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b8ffff700 1 -- 172.21.2.104:0/3761281312 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7b8000fca0 con 0x7f7b980aea10 2021-07-11T03:55:00.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/3761281312 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b980ae2d0 con 0x7f7b980aea10 2021-07-11T03:55:00.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/3761281312 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b98144740 con 0x7f7b980aea10 2021-07-11T03:55:00.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/3761281312 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b98057350 con 0x7f7b980aea10 2021-07-11T03:55:00.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b8ffff700 1 -- 172.21.2.104:0/3761281312 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f7b80014440 con 0x7f7b980aea10 2021-07-11T03:55:00.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b8ffff700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7b7c0532b0 0x7f7b7c055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b8ffff700 1 -- 172.21.2.104:0/3761281312 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7b8006bc60 con 0x7f7b980aea10 2021-07-11T03:55:00.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b96ffd700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7b7c0532b0 0x7f7b7c055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:00.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.025+0000 7f7b96ffd700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7b7c0532b0 0x7f7b7c055770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f7b8801a000 tx=0x7f7b88015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:00.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.029+0000 7f7b8ffff700 1 -- 172.21.2.104:0/3761281312 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7b80042f40 con 0x7f7b980aea10 2021-07-11T03:55:00.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/3761281312 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b980af610 con 0x7f7b980aea10 2021-07-11T03:55:00.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b8ffff700 1 -- 172.21.2.104:0/3761281312 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f7b80018070 con 0x7f7b980aea10 2021-07-11T03:55:00.143 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:00.143 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":11,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4408,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2474981992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2474981992},{"type":"v1","addr":"172.21.2.105:6835","nonce":2474981992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/1241687766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1241687766},{"type":"v1","addr":"172.21.2.104:6834","nonce":1241687766}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9}],"filesystems":[{"mdsmap":{"epoch":11,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:50.608622+0000","modified":"2021-07-11T03:53:51.611729+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/1104530072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1104530072},{"type":"v1","addr":"172.21.2.104:6835","nonce":1104530072}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-07-11T03:55:00.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7b7c0532b0 msgr2=0x7f7b7c055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:00.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7b7c0532b0 0x7f7b7c055770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f7b8801a000 tx=0x7f7b88015000).stop 2021-07-11T03:55:00.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/3761281312 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7b980aea10 msgr2=0x7f7b980aee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:00.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7b980aea10 0x7f7b980aee90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f7b980acf90 tx=0x7f7b80006df0).stop 2021-07-11T03:55:00.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/3761281312 shutdown_connections 2021-07-11T03:55:00.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7b98156740 0x7f7b98156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:00.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7b7c0532b0 0x7f7b7c055770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:00.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7b980aea10 0x7f7b980aee90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:00.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 --2- 172.21.2.104:0/3761281312 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7b980ac860 0x7f7b980aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:00.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/3761281312 >> 172.21.2.104:0/3761281312 conn(0x7f7b980b53e0 msgr2=0x7f7b980b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:00.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/3761281312 shutdown_connections 2021-07-11T03:55:00.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.137+0000 7f7b9d2ed700 1 -- 172.21.2.104:0/3761281312 wait complete. 2021-07-11T03:55:00.147 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 11 2021-07-11T03:55:00.150 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T03:55:00.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 Processor -- start 2021-07-11T03:55:00.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 -- start start 2021-07-11T03:55:00.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe68c0a76b0 0x7fe68c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe68c0b1970 0x7fe68c066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe68c0b0060 0x7fe68c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68c067450 con 0x7fe68c0b0060 2021-07-11T03:55:00.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68c0a7a90 con 0x7fe68c0a76b0 2021-07-11T03:55:00.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c147620 con 0x7fe68c0b1970 2021-07-11T03:55:00.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe6910d8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe68c0b0060 0x7fe68c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:00.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe6910d8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe68c0b0060 0x7fe68c05f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43526/0 (socket says 172.21.2.104:43526) 2021-07-11T03:55:00.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe6910d8700 1 -- 172.21.2.104:0/3153464335 learned_addr learned my addr 172.21.2.104:0/3153464335 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:00.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe6910d8700 1 -- 172.21.2.104:0/3153464335 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe68c0b1970 msgr2=0x7fe68c066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:00.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe6910d8700 1 --2- 172.21.2.104:0/3153464335 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe68c0b1970 0x7fe68c066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:00.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe6910d8700 1 -- 172.21.2.104:0/3153464335 >> v2:172.21.2.105:3300/0 conn(0x7fe68c0a76b0 msgr2=0x7fe68c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:00.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe6910d8700 1 --2- 172.21.2.104:0/3153464335 >> v2:172.21.2.105:3300/0 conn(0x7fe68c0a76b0 0x7fe68c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:00.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe6910d8700 1 -- 172.21.2.104:0/3153464335 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c1478c0 con 0x7fe68c0b0060 2021-07-11T03:55:00.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe6910d8700 1 --2- 172.21.2.104:0/3153464335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe68c0b0060 0x7fe68c05f6b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fe6840032f0 tx=0x7fe684009290).ready entity=mon.0 client_cookie=c82926dd6acbf255 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:00.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe68b7fe700 1 -- 172.21.2.104:0/3153464335 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe68400b870 con 0x7fe68c0b0060 2021-07-11T03:55:00.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe68b7fe700 1 -- 172.21.2.104:0/3153464335 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68400b9d0 con 0x7fe68c0b0060 2021-07-11T03:55:00.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe68b7fe700 1 -- 172.21.2.104:0/3153464335 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe68400bb40 con 0x7fe68c0b0060 2021-07-11T03:55:00.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 -- 172.21.2.104:0/3153464335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe68c0b0060 msgr2=0x7fe68c05f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:00.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 --2- 172.21.2.104:0/3153464335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe68c0b0060 0x7fe68c05f6b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fe6840032f0 tx=0x7fe684009290).stop 2021-07-11T03:55:00.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 -- 172.21.2.104:0/3153464335 shutdown_connections 2021-07-11T03:55:00.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 --2- 172.21.2.104:0/3153464335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe68c0b0060 0x7fe68c05f6b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:00.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 --2- 172.21.2.104:0/3153464335 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe68c0b1970 0x7fe68c066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:00.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 --2- 172.21.2.104:0/3153464335 >> v2:172.21.2.105:3300/0 conn(0x7fe68c0a76b0 0x7fe68c066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:00.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 -- 172.21.2.104:0/3153464335 >> 172.21.2.104:0/3153464335 conn(0x7fe68c0b53e0 msgr2=0x7fe68c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:00.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 -- 172.21.2.104:0/3153464335 shutdown_connections 2021-07-11T03:55:00.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 -- 172.21.2.104:0/3153464335 wait complete. 2021-07-11T03:55:00.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 Processor -- start 2021-07-11T03:55:00.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.197+0000 7fe692b3b700 1 -- start start 2021-07-11T03:55:00.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe692b3b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe68c0a76b0 0x7fe68c0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe692b3b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe68c0b0060 0x7fe68c0b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe692b3b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe68c0b1970 0x7fe68c0aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe692b3b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0b5170 con 0x7fe68c0b1970 2021-07-11T03:55:00.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe692b3b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0abe10 con 0x7fe68c0b0060 2021-07-11T03:55:00.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe692b3b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0ab390 con 0x7fe68c0a76b0 2021-07-11T03:55:00.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe6910d8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe68c0b1970 0x7fe68c0aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:00.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe6910d8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe68c0b1970 0x7fe68c0aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43536/0 (socket says 172.21.2.104:43536) 2021-07-11T03:55:00.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe6910d8700 1 -- 172.21.2.104:0/3364167685 learned_addr learned my addr 172.21.2.104:0/3364167685 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:00.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe6910d8700 1 -- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe68c0a76b0 msgr2=0x7fe68c0adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:00.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe6910d8700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe68c0a76b0 0x7fe68c0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:00.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe6910d8700 1 -- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe68c0b0060 msgr2=0x7fe68c0b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:00.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe6910d8700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe68c0b0060 0x7fe68c0b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:00.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe6910d8700 1 -- 172.21.2.104:0/3364167685 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe684009530 con 0x7fe68c0b1970 2021-07-11T03:55:00.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe6910d8700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe68c0b1970 0x7fe68c0aef30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fe68c0b48c0 tx=0x7fe684016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:00.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe689ffb700 1 -- 172.21.2.104:0/3364167685 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe684011cb0 con 0x7fe68c0b1970 2021-07-11T03:55:00.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe689ffb700 1 -- 172.21.2.104:0/3364167685 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe684011e10 con 0x7fe68c0b1970 2021-07-11T03:55:00.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe689ffb700 1 -- 172.21.2.104:0/3364167685 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe684020660 con 0x7fe68c0b1970 2021-07-11T03:55:00.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe692b3b700 1 -- 172.21.2.104:0/3364167685 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68c0ab630 con 0x7fe68c0b1970 2021-07-11T03:55:00.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe692b3b700 1 -- 172.21.2.104:0/3364167685 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68c0abb70 con 0x7fe68c0b1970 2021-07-11T03:55:00.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe692b3b700 1 -- 172.21.2.104:0/3364167685 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68c057350 con 0x7fe68c0b1970 2021-07-11T03:55:00.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe689ffb700 1 -- 172.21.2.104:0/3364167685 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe68400f070 con 0x7fe68c0b1970 2021-07-11T03:55:00.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe689ffb700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe66c0532d0 0x7fe66c055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:00.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe689ffb700 1 -- 172.21.2.104:0/3364167685 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe68406c360 con 0x7fe68c0b1970 2021-07-11T03:55:00.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.201+0000 7fe6908d7700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe66c0532d0 0x7fe66c055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:00.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.205+0000 7fe6908d7700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe66c0532d0 0x7fe66c055790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe67c002f70 tx=0x7fe67c006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:00.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.205+0000 7fe689ffb700 1 -- 172.21.2.104:0/3364167685 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe684043590 con 0x7fe68c0b1970 2021-07-11T03:55:00.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:00.309+0000 7fe692b3b700 1 -- 172.21.2.104:0/3364167685 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe68c0698e0 con 0x7fe68c0b1970 2021-07-11T03:55:00.444 INFO:tasks.ceph.mds.a.gibba004.stderr:2021-07-11T03:55:00.437+0000 7f78c2151700 -1 MDSIOContextBase: blacklisted! Restarting... 2021-07-11T03:55:00.457 INFO:tasks.ceph.mds.a.gibba004.stderr: -1> 2021-07-11T03:55:00.437+0000 7f78c2151700 -1 MDSIOContextBase: blacklisted! Restarting... 2021-07-11T03:55:00.469 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T03:55:01.217 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T03:55:01.209+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T03:55:01.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe689ffb700 1 -- 172.21.2.104:0/3364167685 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v12) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe684015020 con 0x7fe68c0b1970 2021-07-11T03:55:01.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 -- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe66c0532d0 msgr2=0x7fe66c055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:01.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe66c0532d0 0x7fe66c055790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fe67c002f70 tx=0x7fe67c006a80).stop 2021-07-11T03:55:01.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 -- 172.21.2.104:0/3364167685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe68c0b1970 msgr2=0x7fe68c0aef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:01.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe68c0b1970 0x7fe68c0aef30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fe68c0b48c0 tx=0x7fe684016000).stop 2021-07-11T03:55:01.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 -- 172.21.2.104:0/3364167685 shutdown_connections 2021-07-11T03:55:01.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe66c0532d0 0x7fe66c055790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:01.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe68c0b1970 0x7fe68c0aef30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:01.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe68c0b0060 0x7fe68c0b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:01.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 --2- 172.21.2.104:0/3364167685 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe68c0a76b0 0x7fe68c0adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:01.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 -- 172.21.2.104:0/3364167685 >> 172.21.2.104:0/3364167685 conn(0x7fe68c0b53e0 msgr2=0x7fe68c149c60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:01.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 -- 172.21.2.104:0/3364167685 shutdown_connections 2021-07-11T03:55:01.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.225+0000 7fe692b3b700 1 -- 172.21.2.104:0/3364167685 wait complete. 2021-07-11T03:55:01.234 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T03:55:01.238 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:55:01.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.281+0000 7f1009b1f700 1 Processor -- start 2021-07-11T03:55:01.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.281+0000 7f1009b1f700 1 -- start start 2021-07-11T03:55:01.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f10040a76b0 0x7f1004066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:01.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10040a7a90 con 0x7f10040b0060 2021-07-11T03:55:01.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1004147620 con 0x7f10040b1970 2021-07-11T03:55:01.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10041478c0 con 0x7f10040a76b0 2021-07-11T03:55:01.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1003fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f10040b0060 0x7f1004151f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40326/0 (socket says 172.21.2.104:40326) 2021-07-11T03:55:01.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1003fff700 1 -- 172.21.2.104:0/1286702102 learned_addr learned my addr 172.21.2.104:0/1286702102 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:01.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f10037fe700 1 --2- 172.21.2.104:0/1286702102 >> v2:172.21.2.104:3300/0 conn(0x7f10040a76b0 0x7f1004066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:01.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f10037fe700 1 -- 172.21.2.104:0/1286702102 >> v1:172.21.2.105:6789/0 conn(0x7f10040b1970 legacy=0x7f100405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:01.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f10037fe700 1 -- 172.21.2.104:0/1286702102 >> v1:172.21.2.104:6789/0 conn(0x7f10040b0060 legacy=0x7f1004151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:01.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f10037fe700 1 -- 172.21.2.104:0/1286702102 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1004147b60 con 0x7f10040a76b0 2021-07-11T03:55:01.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f10037fe700 1 --2- 172.21.2.104:0/1286702102 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10040a76b0 0x7f1004066a70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f100406a180 tx=0x7f0fec006df0).ready entity=mon.0 client_cookie=585856d5bdf9fd14 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:01.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f10027fc700 1 -- 172.21.2.104:0/1286702102 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fec00d5c0 con 0x7f10040a76b0 2021-07-11T03:55:01.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f10027fc700 1 -- 172.21.2.104:0/1286702102 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec000e90 con 0x7f10040a76b0 2021-07-11T03:55:01.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f10027fc700 1 -- 172.21.2.104:0/1286702102 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fec00d910 con 0x7f10040a76b0 2021-07-11T03:55:01.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- 172.21.2.104:0/1286702102 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10040a76b0 msgr2=0x7f1004066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:01.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 --2- 172.21.2.104:0/1286702102 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10040a76b0 0x7f1004066a70 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f100406a180 tx=0x7f0fec006df0).stop 2021-07-11T03:55:01.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- 172.21.2.104:0/1286702102 shutdown_connections 2021-07-11T03:55:01.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 --2- 172.21.2.104:0/1286702102 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10040a76b0 0x7f1004066a70 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:01.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- 172.21.2.104:0/1286702102 >> 172.21.2.104:0/1286702102 conn(0x7f10040b53e0 msgr2=0x7f10040b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:01.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- 172.21.2.104:0/1286702102 shutdown_connections 2021-07-11T03:55:01.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- 172.21.2.104:0/1286702102 wait complete. 2021-07-11T03:55:01.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 Processor -- start 2021-07-11T03:55:01.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- start start 2021-07-11T03:55:01.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10040a76b0 0x7f100414cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:01.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10040b0060 0x7f10040a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:01.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f10040b1970 0x7f10040a9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:01.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040a9440 con 0x7f10040b0060 2021-07-11T03:55:01.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1004066500 con 0x7f10040a76b0 2021-07-11T03:55:01.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100415af40 con 0x7f10040b1970 2021-07-11T03:55:01.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1002ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10040b0060 0x7f10040a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:01.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1002ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10040b0060 0x7f10040a9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43560/0 (socket says 172.21.2.104:43560) 2021-07-11T03:55:01.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1002ffd700 1 -- 172.21.2.104:0/2853999532 learned_addr learned my addr 172.21.2.104:0/2853999532 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:01.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1002ffd700 1 -- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f10040b1970 msgr2=0x7f10040a9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:01.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1002ffd700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f10040b1970 0x7f10040a9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:01.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1002ffd700 1 -- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10040a76b0 msgr2=0x7f100414cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:01.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1002ffd700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10040a76b0 0x7f100414cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:01.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1002ffd700 1 -- 172.21.2.104:0/2853999532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec00b180 con 0x7f10040b0060 2021-07-11T03:55:01.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1002ffd700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10040b0060 0x7f10040a9ad0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f0ff4002ef0 tx=0x7f0ff4006d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:01.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1000ff9700 1 -- 172.21.2.104:0/2853999532 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ff4009e00 con 0x7f10040b0060 2021-07-11T03:55:01.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1000ff9700 1 -- 172.21.2.104:0/2853999532 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff4002320 con 0x7f10040b0060 2021-07-11T03:55:01.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1000ff9700 1 -- 172.21.2.104:0/2853999532 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ff4002680 con 0x7f10040b0060 2021-07-11T03:55:01.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- 172.21.2.104:0/2853999532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10040af070 con 0x7f10040b0060 2021-07-11T03:55:01.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.285+0000 7f1009b1f700 1 -- 172.21.2.104:0/2853999532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100415b3b0 con 0x7f10040b0060 2021-07-11T03:55:01.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.289+0000 7f1009b1f700 1 -- 172.21.2.104:0/2853999532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1004057350 con 0x7f10040b0060 2021-07-11T03:55:01.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.289+0000 7f1000ff9700 1 -- 172.21.2.104:0/2853999532 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0ff400f030 con 0x7f10040b0060 2021-07-11T03:55:01.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.289+0000 7f1000ff9700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fe8053160 0x7f0fe8055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:01.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.289+0000 7f1000ff9700 1 -- 172.21.2.104:0/2853999532 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f0ff406b210 con 0x7f10040b0060 2021-07-11T03:55:01.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.289+0000 7f10037fe700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fe8053160 0x7f0fe8055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:01.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.289+0000 7f10037fe700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fe8053160 0x7f0fe8055620 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f100406a180 tx=0x7f0fec012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:01.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.289+0000 7f1000ff9700 1 -- 172.21.2.104:0/2853999532 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0ff4042500 con 0x7f10040b0060 2021-07-11T03:55:01.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.397+0000 7f1009b1f700 1 -- 172.21.2.104:0/2853999532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f100415b0b0 con 0x7f10040b0060 2021-07-11T03:55:01.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.397+0000 7f1000ff9700 1 -- 172.21.2.104:0/2853999532 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+2481 (secure 0 0 0) 0x7f0ff4045b20 con 0x7f10040b0060 2021-07-11T03:55:01.405 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:01.405 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4408,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2474981992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2474981992},{"type":"v1","addr":"172.21.2.105:6835","nonce":2474981992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/1241687766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1241687766},{"type":"v1","addr":"172.21.2.104:6834","nonce":1241687766}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/4230356883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":4230356883},{"type":"v1","addr":"172.21.2.104:6835","nonce":4230356883}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":12}],"filesystems":[{"mdsmap":{"epoch":12,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:53:50.608622+0000","modified":"2021-07-11T03:55:01.216761+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":34,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2021-07-11T03:55:01.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 -- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fe8053160 msgr2=0x7f0fe8055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:01.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fe8053160 0x7f0fe8055620 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f100406a180 tx=0x7f0fec012000).stop 2021-07-11T03:55:01.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 -- 172.21.2.104:0/2853999532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10040b0060 msgr2=0x7f10040a9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:01.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10040b0060 0x7f10040a9ad0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f0ff4002ef0 tx=0x7f0ff4006d40).stop 2021-07-11T03:55:01.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 -- 172.21.2.104:0/2853999532 shutdown_connections 2021-07-11T03:55:01.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f10040b1970 0x7f10040a9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:01.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fe8053160 0x7f0fe8055620 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:01.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10040b0060 0x7f10040a9ad0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:01.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 --2- 172.21.2.104:0/2853999532 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10040a76b0 0x7f100414cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:01.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 -- 172.21.2.104:0/2853999532 >> 172.21.2.104:0/2853999532 conn(0x7f10040b53e0 msgr2=0x7f100414a340 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:01.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 -- 172.21.2.104:0/2853999532 shutdown_connections 2021-07-11T03:55:01.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.401+0000 7f1009b1f700 1 -- 172.21.2.104:0/2853999532 wait complete. 2021-07-11T03:55:01.411 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 12 2021-07-11T03:55:01.412 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T03:55:01.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.457+0000 7f30a0dfb700 1 Processor -- start 2021-07-11T03:55:01.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.457+0000 7f30a0dfb700 1 -- start start 2021-07-11T03:55:01.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f309c14ab80 0x7f309c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:01.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f309c0ac9b0 0x7f309c14af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:01.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309c061360 con 0x7f309c0aff10 2021-07-11T03:55:01.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c14b5e0 con 0x7f309c14ab80 2021-07-11T03:55:01.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c14b880 con 0x7f309c0ac9b0 2021-07-11T03:55:01.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f309c14ab80 0x7f309c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:01.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f309c14ab80 0x7f309c05f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43568/0 (socket says 172.21.2.104:43568) 2021-07-11T03:55:01.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 -- 172.21.2.104:0/3005542312 learned_addr learned my addr 172.21.2.104:0/3005542312 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:01.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 -- 172.21.2.104:0/3005542312 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f309c0ac9b0 msgr2=0x7f309c14af60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:01.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 --2- 172.21.2.104:0/3005542312 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f309c0ac9b0 0x7f309c14af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:01.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 -- 172.21.2.104:0/3005542312 >> v1:172.21.2.105:6789/0 conn(0x7f309c0aff10 legacy=0x7f309c0af1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:01.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 -- 172.21.2.104:0/3005542312 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0aed20 con 0x7f309c14ab80 2021-07-11T03:55:01.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 --2- 172.21.2.104:0/3005542312 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f309c14ab80 0x7f309c05f6b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f309c062060 tx=0x7f3084006eb0).ready entity=mon.0 client_cookie=cb747a81ea5f50c3 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:01.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309959a700 1 -- 172.21.2.104:0/3005542312 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f308400d570 con 0x7f309c14ab80 2021-07-11T03:55:01.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309959a700 1 -- 172.21.2.104:0/3005542312 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3084000e90 con 0x7f309c14ab80 2021-07-11T03:55:01.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309959a700 1 -- 172.21.2.104:0/3005542312 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f308400d8c0 con 0x7f309c14ab80 2021-07-11T03:55:01.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3005542312 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f309c14ab80 msgr2=0x7f309c05f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:01.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 --2- 172.21.2.104:0/3005542312 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f309c14ab80 0x7f309c05f6b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f309c062060 tx=0x7f3084006eb0).stop 2021-07-11T03:55:01.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3005542312 shutdown_connections 2021-07-11T03:55:01.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 --2- 172.21.2.104:0/3005542312 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f309c0ac9b0 0x7f309c14af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:01.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 --2- 172.21.2.104:0/3005542312 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f309c14ab80 0x7f309c05f6b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:01.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3005542312 >> 172.21.2.104:0/3005542312 conn(0x7f309c0b5570 msgr2=0x7f309c0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:01.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3005542312 shutdown_connections 2021-07-11T03:55:01.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3005542312 wait complete. 2021-07-11T03:55:01.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 Processor -- start 2021-07-11T03:55:01.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- start start 2021-07-11T03:55:01.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f309c0ac9b0 0x7f309c066e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:01.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f309c0aff10 0x7f309c156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:01.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f309c14ab80 0x7f309c156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:01.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c144f60 con 0x7f309c0ac9b0 2021-07-11T03:55:01.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c147ae0 con 0x7f309c0aff10 2021-07-11T03:55:01.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c1435e0 con 0x7f309c14ab80 2021-07-11T03:55:01.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f309c0ac9b0 0x7f309c066e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:01.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f309c0ac9b0 0x7f309c066e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43574/0 (socket says 172.21.2.104:43574) 2021-07-11T03:55:01.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 -- 172.21.2.104:0/3641026850 learned_addr learned my addr 172.21.2.104:0/3641026850 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:01.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 -- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f309c14ab80 msgr2=0x7f309c156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:01.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f309c14ab80 0x7f309c156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:01.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 -- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f309c0aff10 msgr2=0x7f309c156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:01.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f309c0aff10 0x7f309c156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:01.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 -- 172.21.2.104:0/3641026850 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308400b1e0 con 0x7f309c0ac9b0 2021-07-11T03:55:01.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f309a59c700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f309c0ac9b0 0x7f309c066e40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f309c147a00 tx=0x7f3084015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:01.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30937fe700 1 -- 172.21.2.104:0/3641026850 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3084021070 con 0x7f309c0ac9b0 2021-07-11T03:55:01.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30937fe700 1 -- 172.21.2.104:0/3641026850 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3084000e90 con 0x7f309c0ac9b0 2021-07-11T03:55:01.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3641026850 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c157820 con 0x7f309c0ac9b0 2021-07-11T03:55:01.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.461+0000 7f30937fe700 1 -- 172.21.2.104:0/3641026850 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3084002f70 con 0x7f309c0ac9b0 2021-07-11T03:55:01.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.465+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3641026850 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c157210 con 0x7f309c0ac9b0 2021-07-11T03:55:01.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.465+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3641026850 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c157670 con 0x7f309c0ac9b0 2021-07-11T03:55:01.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.465+0000 7f30937fe700 1 -- 172.21.2.104:0/3641026850 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f308400f070 con 0x7f309c0ac9b0 2021-07-11T03:55:01.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.465+0000 7f30937fe700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f30800532b0 0x7f3080055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:01.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.465+0000 7f30937fe700 1 -- 172.21.2.104:0/3641026850 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f308406c9c0 con 0x7f309c0ac9b0 2021-07-11T03:55:01.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.465+0000 7f3099d9b700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f30800532b0 0x7f3080055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:01.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.465+0000 7f3099d9b700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f30800532b0 0x7f3080055770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f308c002fb0 tx=0x7f308c0069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:01.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.465+0000 7f30937fe700 1 -- 172.21.2.104:0/3641026850 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3084043cb0 con 0x7f309c0ac9b0 2021-07-11T03:55:01.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:01.573+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3641026850 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f309c057350 con 0x7f309c0ac9b0 2021-07-11T03:55:02.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.325+0000 7f30937fe700 1 -- 172.21.2.104:0/3641026850 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f30840472d0 con 0x7f309c0ac9b0 2021-07-11T03:55:02.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f30800532b0 msgr2=0x7f3080055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:02.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f30800532b0 0x7f3080055770 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f308c002fb0 tx=0x7f308c0069d0).stop 2021-07-11T03:55:02.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3641026850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f309c0ac9b0 msgr2=0x7f309c066e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:02.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f309c0ac9b0 0x7f309c066e40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f309c147a00 tx=0x7f3084015000).stop 2021-07-11T03:55:02.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3641026850 shutdown_connections 2021-07-11T03:55:02.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f309c14ab80 0x7f309c156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:02.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f309c0aff10 0x7f309c156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:02.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f30800532b0 0x7f3080055770 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:02.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 --2- 172.21.2.104:0/3641026850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f309c0ac9b0 0x7f309c066e40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:02.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3641026850 >> 172.21.2.104:0/3641026850 conn(0x7f309c0b5570 msgr2=0x7f309c14a450 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:02.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3641026850 shutdown_connections 2021-07-11T03:55:02.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.329+0000 7f30a0dfb700 1 -- 172.21.2.104:0/3641026850 wait complete. 2021-07-11T03:55:02.342 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T03:55:02.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 Processor -- start 2021-07-11T03:55:02.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 -- start start 2021-07-11T03:55:02.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f3f401483a0 0x7f3f400b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:02.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f400b0350 con 0x7f3f400b1db0 2021-07-11T03:55:02.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f400b05f0 con 0x7f3f400a70c0 2021-07-11T03:55:02.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f400a90c0 con 0x7f3f401483a0 2021-07-11T03:55:02.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f459d3700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f3f400b1db0 0x7f3f400b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40350/0 (socket says 172.21.2.104:40350) 2021-07-11T03:55:02.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f459d3700 1 -- 172.21.2.104:0/2832344594 learned_addr learned my addr 172.21.2.104:0/2832344594 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:02.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f451d2700 1 --2- 172.21.2.104:0/2832344594 >> v2:172.21.2.105:3300/0 conn(0x7f3f401483a0 0x7f3f400b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:02.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f449d1700 1 -- 172.21.2.104:0/2832344594 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896147208 0 0) 0x7f3f400b0350 con 0x7f3f400b1db0 2021-07-11T03:55:02.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f449d1700 1 -- 172.21.2.104:0/2832344594 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f28003260 con 0x7f3f400b1db0 2021-07-11T03:55:02.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f451d2700 1 -- 172.21.2.104:0/2832344594 >> v1:172.21.2.105:6789/0 conn(0x7f3f400a70c0 legacy=0x7f3f40144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:02.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f451d2700 1 -- 172.21.2.104:0/2832344594 >> v1:172.21.2.104:6789/0 conn(0x7f3f400b1db0 legacy=0x7f3f400b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:02.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f449d1700 1 -- 172.21.2.104:0/2832344594 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2050812309 0 0) 0x7f3f28003260 con 0x7f3f400b1db0 2021-07-11T03:55:02.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f451d2700 1 -- 172.21.2.104:0/2832344594 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f400b0890 con 0x7f3f401483a0 2021-07-11T03:55:02.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f451d2700 1 --2- 172.21.2.104:0/2832344594 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3f401483a0 0x7f3f400b0fd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f3f34001800 tx=0x7f3f3c006df0).ready entity=mon.1 client_cookie=5f4e90ab40befc6f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:02.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f449d1700 1 -- 172.21.2.104:0/2832344594 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f34001660 con 0x7f3f401483a0 2021-07-11T03:55:02.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f449d1700 1 -- 172.21.2.104:0/2832344594 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c000e90 con 0x7f3f401483a0 2021-07-11T03:55:02.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 -- 172.21.2.104:0/2832344594 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3f401483a0 msgr2=0x7f3f400b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:02.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 --2- 172.21.2.104:0/2832344594 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3f401483a0 0x7f3f400b0fd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f3f34001800 tx=0x7f3f3c006df0).stop 2021-07-11T03:55:02.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 -- 172.21.2.104:0/2832344594 shutdown_connections 2021-07-11T03:55:02.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 --2- 172.21.2.104:0/2832344594 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3f401483a0 0x7f3f400b0fd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:02.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 -- 172.21.2.104:0/2832344594 >> 172.21.2.104:0/2832344594 conn(0x7f3f4005f6b0 msgr2=0x7f3f4005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:02.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 -- 172.21.2.104:0/2832344594 shutdown_connections 2021-07-11T03:55:02.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.393+0000 7f3f47c37700 1 -- 172.21.2.104:0/2832344594 wait complete. 2021-07-11T03:55:02.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f47c37700 1 Processor -- start 2021-07-11T03:55:02.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f47c37700 1 -- start start 2021-07-11T03:55:02.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f47c37700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3f400a70c0 0x7f3f400ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:02.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f47c37700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3f400b1db0 0x7f3f400627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:02.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f47c37700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3f401483a0 0x7f3f40064190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:02.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f47c37700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f400aa880 con 0x7f3f401483a0 2021-07-11T03:55:02.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f47c37700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f400aab20 con 0x7f3f400a70c0 2021-07-11T03:55:02.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f47c37700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f400aadc0 con 0x7f3f400b1db0 2021-07-11T03:55:02.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f461d4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3f401483a0 0x7f3f40064190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:02.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f461d4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3f401483a0 0x7f3f40064190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43590/0 (socket says 172.21.2.104:43590) 2021-07-11T03:55:02.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f461d4700 1 -- 172.21.2.104:0/3324452054 learned_addr learned my addr 172.21.2.104:0/3324452054 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:02.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f461d4700 1 -- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3f400b1db0 msgr2=0x7f3f400627c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:02.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f461d4700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3f400b1db0 0x7f3f400627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:02.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f461d4700 1 -- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3f400a70c0 msgr2=0x7f3f400ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:02.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f461d4700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3f400a70c0 0x7f3f400ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:02.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f461d4700 1 -- 172.21.2.104:0/3324452054 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c00b180 con 0x7f3f401483a0 2021-07-11T03:55:02.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f461d4700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3f401483a0 0x7f3f40064190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f3f400b3170 tx=0x7f3f38007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:02.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f32ffd700 1 -- 172.21.2.104:0/3324452054 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f3800e4a0 con 0x7f3f401483a0 2021-07-11T03:55:02.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f32ffd700 1 -- 172.21.2.104:0/3324452054 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3800e600 con 0x7f3f401483a0 2021-07-11T03:55:02.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f47c37700 1 -- 172.21.2.104:0/3324452054 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f400ab060 con 0x7f3f401483a0 2021-07-11T03:55:02.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f32ffd700 1 -- 172.21.2.104:0/3324452054 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3f3800e960 con 0x7f3f401483a0 2021-07-11T03:55:02.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f47c37700 1 -- 172.21.2.104:0/3324452054 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f400abd40 con 0x7f3f401483a0 2021-07-11T03:55:02.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f47c37700 1 -- 172.21.2.104:0/3324452054 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f40057350 con 0x7f3f401483a0 2021-07-11T03:55:02.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f32ffd700 1 -- 172.21.2.104:0/3324452054 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3f38019030 con 0x7f3f401483a0 2021-07-11T03:55:02.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f32ffd700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3f20053300 0x7f3f200557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:02.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f32ffd700 1 -- 172.21.2.104:0/3324452054 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f3f3806adb0 con 0x7f3f401483a0 2021-07-11T03:55:02.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f459d3700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3f20053300 0x7f3f200557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:02.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.397+0000 7f3f459d3700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3f20053300 0x7f3f200557c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f3f340018c0 tx=0x7f3f34001140).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:02.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.401+0000 7f3f32ffd700 1 -- 172.21.2.104:0/3324452054 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3f38041f90 con 0x7f3f401483a0 2021-07-11T03:55:02.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:02.481+0000 7f3f47c37700 1 -- 172.21.2.104:0/3324452054 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3f40044650 con 0x7f3f401483a0 2021-07-11T03:55:03.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f32ffd700 1 -- 172.21.2.104:0/3324452054 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 185+0+0 (secure 0 0 0) 0x7f3f38017070 con 0x7f3f401483a0 2021-07-11T03:55:03.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 -- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3f20053300 msgr2=0x7f3f200557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:03.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3f20053300 0x7f3f200557c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f3f340018c0 tx=0x7f3f34001140).stop 2021-07-11T03:55:03.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 -- 172.21.2.104:0/3324452054 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3f401483a0 msgr2=0x7f3f40064190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:03.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3f401483a0 0x7f3f40064190 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f3f400b3170 tx=0x7f3f38007f00).stop 2021-07-11T03:55:03.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 -- 172.21.2.104:0/3324452054 shutdown_connections 2021-07-11T03:55:03.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3f20053300 0x7f3f200557c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:03.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3f401483a0 0x7f3f40064190 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:03.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3f400b1db0 0x7f3f400627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:03.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 --2- 172.21.2.104:0/3324452054 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3f400a70c0 0x7f3f400ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:03.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 -- 172.21.2.104:0/3324452054 >> 172.21.2.104:0/3324452054 conn(0x7f3f4005f6b0 msgr2=0x7f3f400aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:03.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 -- 172.21.2.104:0/3324452054 shutdown_connections 2021-07-11T03:55:03.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.333+0000 7f3f47c37700 1 -- 172.21.2.104:0/3324452054 wait complete. 2021-07-11T03:55:03.344 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T03:55:03.347 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T03:55:03.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20fd628700 1 Processor -- start 2021-07-11T03:55:03.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20fd628700 1 -- start start 2021-07-11T03:55:03.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20fd628700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f20f8060100 0x7f20f80b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:03.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20fd628700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f80aabb0 con 0x7f20f80b4c40 2021-07-11T03:55:03.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20fd628700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f80604e0 con 0x7f20f8067300 2021-07-11T03:55:03.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20fd628700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f8066ea0 con 0x7f20f8060100 2021-07-11T03:55:03.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20f77fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f20f80b4c40 0x7f20f814b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40364/0 (socket says 172.21.2.104:40364) 2021-07-11T03:55:03.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20f77fe700 1 -- 172.21.2.104:0/1523732603 learned_addr learned my addr 172.21.2.104:0/1523732603 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:03.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20f6ffd700 1 --2- 172.21.2.104:0/1523732603 >> v2:172.21.2.104:3300/0 conn(0x7f20f8060100 0x7f20f80b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:03.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20f6ffd700 1 -- 172.21.2.104:0/1523732603 >> v1:172.21.2.105:6789/0 conn(0x7f20f8067300 legacy=0x7f20f81438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:03.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20f6ffd700 1 -- 172.21.2.104:0/1523732603 >> v1:172.21.2.104:6789/0 conn(0x7f20f80b4c40 legacy=0x7f20f814b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:03.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20f6ffd700 1 -- 172.21.2.104:0/1523732603 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f80b33b0 con 0x7f20f8060100 2021-07-11T03:55:03.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20f6ffd700 1 --2- 172.21.2.104:0/1523732603 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20f8060100 0x7f20f80b4750 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f20f805f220 tx=0x7f20e8006df0).ready entity=mon.0 client_cookie=78ea4750d638d4b0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:03.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20f5ffb700 1 -- 172.21.2.104:0/1523732603 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20e800d5c0 con 0x7f20f8060100 2021-07-11T03:55:03.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20f5ffb700 1 -- 172.21.2.104:0/1523732603 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8000e90 con 0x7f20f8060100 2021-07-11T03:55:03.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20f5ffb700 1 -- 172.21.2.104:0/1523732603 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20e800d910 con 0x7f20f8060100 2021-07-11T03:55:03.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20fd628700 1 -- 172.21.2.104:0/1523732603 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20f8060100 msgr2=0x7f20f80b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:03.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.397+0000 7f20fd628700 1 --2- 172.21.2.104:0/1523732603 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20f8060100 0x7f20f80b4750 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f20f805f220 tx=0x7f20e8006df0).stop 2021-07-11T03:55:03.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 -- 172.21.2.104:0/1523732603 shutdown_connections 2021-07-11T03:55:03.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 --2- 172.21.2.104:0/1523732603 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20f8060100 0x7f20f80b4750 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:03.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 -- 172.21.2.104:0/1523732603 >> 172.21.2.104:0/1523732603 conn(0x7f20f805cf30 msgr2=0x7f20f805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:03.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 -- 172.21.2.104:0/1523732603 shutdown_connections 2021-07-11T03:55:03.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 -- 172.21.2.104:0/1523732603 wait complete. 2021-07-11T03:55:03.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 Processor -- start 2021-07-11T03:55:03.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 -- start start 2021-07-11T03:55:03.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20f8060100 0x7f20f80b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:03.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20f8067300 0x7f20f80ac420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:03.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20f80b4c40 0x7f20f80ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:03.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f809f220 con 0x7f20f8060100 2021-07-11T03:55:03.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80ace00 con 0x7f20f8067300 2021-07-11T03:55:03.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f8157870 con 0x7f20f80b4c40 2021-07-11T03:55:03.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20f6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20f8060100 0x7f20f80b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:03.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20f6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20f8060100 0x7f20f80b00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43602/0 (socket says 172.21.2.104:43602) 2021-07-11T03:55:03.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20f6ffd700 1 -- 172.21.2.104:0/1417894242 learned_addr learned my addr 172.21.2.104:0/1417894242 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:03.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20f6ffd700 1 -- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20f80b4c40 msgr2=0x7f20f80ac910 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:55:03.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20f6ffd700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20f80b4c40 0x7f20f80ac910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:03.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20f6ffd700 1 -- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20f8067300 msgr2=0x7f20f80ac420 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:55:03.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20f6ffd700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20f8067300 0x7f20f80ac420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:03.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20f6ffd700 1 -- 172.21.2.104:0/1417894242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e800b180 con 0x7f20f8060100 2021-07-11T03:55:03.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20f6ffd700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20f8060100 0x7f20f80b00e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f20f805f220 tx=0x7f20e8012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:03.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20effff700 1 -- 172.21.2.104:0/1417894242 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20e8002f50 con 0x7f20f8060100 2021-07-11T03:55:03.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20effff700 1 -- 172.21.2.104:0/1417894242 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8017030 con 0x7f20f8060100 2021-07-11T03:55:03.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 -- 172.21.2.104:0/1417894242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f8156950 con 0x7f20f8060100 2021-07-11T03:55:03.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20effff700 1 -- 172.21.2.104:0/1417894242 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f20e801d470 con 0x7f20f8060100 2021-07-11T03:55:03.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 -- 172.21.2.104:0/1417894242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f8156e90 con 0x7f20f8060100 2021-07-11T03:55:03.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20fd628700 1 -- 172.21.2.104:0/1417894242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f8057350 con 0x7f20f8060100 2021-07-11T03:55:03.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20effff700 1 -- 172.21.2.104:0/1417894242 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f20e8024030 con 0x7f20f8060100 2021-07-11T03:55:03.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20effff700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20dc053270 0x7f20dc055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:03.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20effff700 1 -- 172.21.2.104:0/1417894242 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f20e806c0a0 con 0x7f20f8060100 2021-07-11T03:55:03.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.401+0000 7f20f67fc700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20dc053270 0x7f20dc055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:03.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.405+0000 7f20f67fc700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20dc053270 0x7f20dc055730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f20e0002f90 tx=0x7f20e0006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:03.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.405+0000 7f20effff700 1 -- 172.21.2.104:0/1417894242 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f20e8043430 con 0x7f20f8060100 2021-07-11T03:55:03.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:03.489+0000 7f20fd628700 1 -- 172.21.2.104:0/1417894242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f20f8157570 con 0x7f20f8060100 2021-07-11T03:55:04.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.349+0000 7f20effff700 1 -- 172.21.2.104:0/1417894242 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 173+0+0 (secure 0 0 0) 0x7f20e8022020 con 0x7f20f8060100 2021-07-11T03:55:04.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 -- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20dc053270 msgr2=0x7f20dc055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:04.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20dc053270 0x7f20dc055730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f20e0002f90 tx=0x7f20e0006930).stop 2021-07-11T03:55:04.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 -- 172.21.2.104:0/1417894242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20f8060100 msgr2=0x7f20f80b00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:04.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20f8060100 0x7f20f80b00e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f20f805f220 tx=0x7f20e8012000).stop 2021-07-11T03:55:04.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 -- 172.21.2.104:0/1417894242 shutdown_connections 2021-07-11T03:55:04.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20f80b4c40 0x7f20f80ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20f8067300 0x7f20f80ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20dc053270 0x7f20dc055730 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:04.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 --2- 172.21.2.104:0/1417894242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20f8060100 0x7f20f80b00e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:04.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 -- 172.21.2.104:0/1417894242 >> 172.21.2.104:0/1417894242 conn(0x7f20f805cf30 msgr2=0x7f20f8065f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:04.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 -- 172.21.2.104:0/1417894242 shutdown_connections 2021-07-11T03:55:04.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.353+0000 7f20fd628700 1 -- 172.21.2.104:0/1417894242 wait complete. 2021-07-11T03:55:04.366 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T03:55:04.388 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-07-11T03:55:04.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 Processor -- start 2021-07-11T03:55:04.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- start start 2021-07-11T03:55:04.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f01f4146950 0x7f01f4140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f40b4180 0x7f01f4140720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f01f40b4690 0x7f01f40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f4147510 con 0x7f01f4146950 2021-07-11T03:55:04.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f4143b60 con 0x7f01f40b4690 2021-07-11T03:55:04.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4143e00 con 0x7f01f40b4180 2021-07-11T03:55:04.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f8eb5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f01f4146950 0x7f01f4140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:04.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f8eb5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f01f4146950 0x7f01f4140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43610/0 (socket says 172.21.2.104:43610) 2021-07-11T03:55:04.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f8eb5700 1 -- 172.21.2.104:0/446541441 learned_addr learned my addr 172.21.2.104:0/446541441 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:04.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f8eb5700 1 -- 172.21.2.104:0/446541441 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f40b4180 msgr2=0x7f01f4140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:04.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f8eb5700 1 --2- 172.21.2.104:0/446541441 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f40b4180 0x7f01f4140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f8eb5700 1 -- 172.21.2.104:0/446541441 >> v2:172.21.2.105:3300/0 conn(0x7f01f40b4690 msgr2=0x7f01f40b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:04.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f8eb5700 1 --2- 172.21.2.104:0/446541441 >> v2:172.21.2.105:3300/0 conn(0x7f01f40b4690 0x7f01f40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f8eb5700 1 -- 172.21.2.104:0/446541441 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40b0c90 con 0x7f01f4146950 2021-07-11T03:55:04.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f8eb5700 1 --2- 172.21.2.104:0/446541441 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01f4146950 0x7f01f4140230 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f01dc002f70 tx=0x7f01dc006eb0).ready entity=mon.0 client_cookie=19c2f0773367051f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:04.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f37fe700 1 -- 172.21.2.104:0/446541441 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01dc00d570 con 0x7f01f4146950 2021-07-11T03:55:04.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f37fe700 1 -- 172.21.2.104:0/446541441 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc000e90 con 0x7f01f4146950 2021-07-11T03:55:04.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f37fe700 1 -- 172.21.2.104:0/446541441 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01dc00d900 con 0x7f01f4146950 2021-07-11T03:55:04.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- 172.21.2.104:0/446541441 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01f4146950 msgr2=0x7f01f4140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:04.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 --2- 172.21.2.104:0/446541441 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01f4146950 0x7f01f4140230 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f01dc002f70 tx=0x7f01dc006eb0).stop 2021-07-11T03:55:04.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- 172.21.2.104:0/446541441 shutdown_connections 2021-07-11T03:55:04.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 --2- 172.21.2.104:0/446541441 >> v2:172.21.2.105:3300/0 conn(0x7f01f40b4690 0x7f01f40b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 --2- 172.21.2.104:0/446541441 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f40b4180 0x7f01f4140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 --2- 172.21.2.104:0/446541441 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01f4146950 0x7f01f4140230 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:04.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- 172.21.2.104:0/446541441 >> 172.21.2.104:0/446541441 conn(0x7f01f405c490 msgr2=0x7f01f405c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:04.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- 172.21.2.104:0/446541441 shutdown_connections 2021-07-11T03:55:04.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- 172.21.2.104:0/446541441 wait complete. 2021-07-11T03:55:04.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 Processor -- start 2021-07-11T03:55:04.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- start start 2021-07-11T03:55:04.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f01f40b4180 0x7f01f40aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01f40b4690 0x7f01f4142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f4146950 0x7f01f413eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40afd70 con 0x7f01f40b4690 2021-07-11T03:55:04.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4142cf0 con 0x7f01f40b4180 2021-07-11T03:55:04.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01fb119700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40add70 con 0x7f01f4146950 2021-07-11T03:55:04.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01f40b4690 0x7f01f4142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:04.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01f40b4690 0x7f01f4142190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43618/0 (socket says 172.21.2.104:43618) 2021-07-11T03:55:04.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.441+0000 7f01f3fff700 1 -- 172.21.2.104:0/3435526140 learned_addr learned my addr 172.21.2.104:0/3435526140 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:04.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f3fff700 1 -- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f4146950 msgr2=0x7f01f413eae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:55:04.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f3fff700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f4146950 0x7f01f413eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f3fff700 1 -- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f01f40b4180 msgr2=0x7f01f40aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:04.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f3fff700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f01f40b4180 0x7f01f40aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f3fff700 1 -- 172.21.2.104:0/3435526140 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc00b1e0 con 0x7f01f40b4690 2021-07-11T03:55:04.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f3fff700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01f40b4690 0x7f01f4142190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f01e4002f70 tx=0x7f01e4006e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:04.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f1ffb700 1 -- 172.21.2.104:0/3435526140 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e4013460 con 0x7f01f40b4690 2021-07-11T03:55:04.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f1ffb700 1 -- 172.21.2.104:0/3435526140 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40135c0 con 0x7f01f40b4690 2021-07-11T03:55:04.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01fb119700 1 -- 172.21.2.104:0/3435526140 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f40ae010 con 0x7f01f40b4690 2021-07-11T03:55:04.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f1ffb700 1 -- 172.21.2.104:0/3435526140 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01e4013920 con 0x7f01f40b4690 2021-07-11T03:55:04.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01fb119700 1 -- 172.21.2.104:0/3435526140 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f40ae9f0 con 0x7f01f40b4690 2021-07-11T03:55:04.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01fb119700 1 -- 172.21.2.104:0/3435526140 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f40adb40 con 0x7f01f40b4690 2021-07-11T03:55:04.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f1ffb700 1 -- 172.21.2.104:0/3435526140 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f01e401a060 con 0x7f01f40b4690 2021-07-11T03:55:04.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f1ffb700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d80532d0 0x7f01d8055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f1ffb700 1 -- 172.21.2.104:0/3435526140 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f01e406ac00 con 0x7f01f40b4690 2021-07-11T03:55:04.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f8eb5700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d80532d0 0x7f01d8055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:04.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f8eb5700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d80532d0 0x7f01d8055790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f01dc002f70 tx=0x7f01dc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:04.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.445+0000 7f01f1ffb700 1 -- 172.21.2.104:0/3435526140 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f01e4042220 con 0x7f01f40b4690 2021-07-11T03:55:04.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.529+0000 7f01fb119700 1 -- 172.21.2.104:0/3435526140 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f01f40ae1a0 con 0x7f01f40b4690 2021-07-11T03:55:04.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.533+0000 7f01f1ffb700 1 -- 172.21.2.104:0/3435526140 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f01e4045840 con 0x7f01f40b4690 2021-07-11T03:55:04.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 -- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d80532d0 msgr2=0x7f01d8055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:04.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d80532d0 0x7f01d8055790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f01dc002f70 tx=0x7f01dc015000).stop 2021-07-11T03:55:04.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 -- 172.21.2.104:0/3435526140 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01f40b4690 msgr2=0x7f01f4142190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:04.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01f40b4690 0x7f01f4142190 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f01e4002f70 tx=0x7f01e4006e90).stop 2021-07-11T03:55:04.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 -- 172.21.2.104:0/3435526140 shutdown_connections 2021-07-11T03:55:04.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f4146950 0x7f01f413eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d80532d0 0x7f01d8055790 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:04.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01f40b4690 0x7f01f4142190 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:04.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 --2- 172.21.2.104:0/3435526140 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f01f40b4180 0x7f01f40aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 -- 172.21.2.104:0/3435526140 >> 172.21.2.104:0/3435526140 conn(0x7f01f405c490 msgr2=0x7f01f40b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:04.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 -- 172.21.2.104:0/3435526140 shutdown_connections 2021-07-11T03:55:04.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.537+0000 7f01fb119700 1 -- 172.21.2.104:0/3435526140 wait complete. 2021-07-11T03:55:04.548 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command' 2021-07-11T03:55:04.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 Processor -- start 2021-07-11T03:55:04.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- start start 2021-07-11T03:55:04.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6680b6580 0x7fd6680b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd6680b43c0 0x7fd6680b6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd668061d30 0x7fd6680621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6681440b0 con 0x7fd668061d30 2021-07-11T03:55:04.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6680b6e50 con 0x7fd6680b43c0 2021-07-11T03:55:04.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd668143b70 con 0x7fd6680b6580 2021-07-11T03:55:04.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd667fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd668061d30 0x7fd6680621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:04.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd667fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd668061d30 0x7fd6680621a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43624/0 (socket says 172.21.2.104:43624) 2021-07-11T03:55:04.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd667fff700 1 -- 172.21.2.104:0/2128523136 learned_addr learned my addr 172.21.2.104:0/2128523136 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:04.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd667fff700 1 -- 172.21.2.104:0/2128523136 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6680b6580 msgr2=0x7fd6680b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:04.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd667fff700 1 --2- 172.21.2.104:0/2128523136 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6680b6580 0x7fd6680b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd667fff700 1 -- 172.21.2.104:0/2128523136 >> v2:172.21.2.105:3300/0 conn(0x7fd6680b43c0 msgr2=0x7fd6680b6960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:04.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd667fff700 1 --2- 172.21.2.104:0/2128523136 >> v2:172.21.2.105:3300/0 conn(0x7fd6680b43c0 0x7fd6680b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd667fff700 1 -- 172.21.2.104:0/2128523136 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd668143e10 con 0x7fd668061d30 2021-07-11T03:55:04.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd667fff700 1 --2- 172.21.2.104:0/2128523136 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd668061d30 0x7fd6680621a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fd6600032f0 tx=0x7fd660009290).ready entity=mon.0 client_cookie=35fccbc8f75b7dc7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:04.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd6667fc700 1 -- 172.21.2.104:0/2128523136 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd66000b870 con 0x7fd668061d30 2021-07-11T03:55:04.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd6667fc700 1 -- 172.21.2.104:0/2128523136 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66000b9d0 con 0x7fd668061d30 2021-07-11T03:55:04.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd6667fc700 1 -- 172.21.2.104:0/2128523136 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd66000bb40 con 0x7fd668061d30 2021-07-11T03:55:04.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- 172.21.2.104:0/2128523136 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd668061d30 msgr2=0x7fd6680621a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:04.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 --2- 172.21.2.104:0/2128523136 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd668061d30 0x7fd6680621a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fd6600032f0 tx=0x7fd660009290).stop 2021-07-11T03:55:04.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- 172.21.2.104:0/2128523136 shutdown_connections 2021-07-11T03:55:04.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 --2- 172.21.2.104:0/2128523136 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd668061d30 0x7fd6680621a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:04.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 --2- 172.21.2.104:0/2128523136 >> v2:172.21.2.105:3300/0 conn(0x7fd6680b43c0 0x7fd6680b6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 --2- 172.21.2.104:0/2128523136 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6680b6580 0x7fd6680b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:04.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- 172.21.2.104:0/2128523136 >> 172.21.2.104:0/2128523136 conn(0x7fd66805c490 msgr2=0x7fd66805c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:04.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- 172.21.2.104:0/2128523136 shutdown_connections 2021-07-11T03:55:04.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- 172.21.2.104:0/2128523136 wait complete. 2021-07-11T03:55:04.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 Processor -- start 2021-07-11T03:55:04.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- start start 2021-07-11T03:55:04.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd668061d30 0x7fd6680ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6680b43c0 0x7fd6680b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6680b6580 0x7fd6680b8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6680b8800 con 0x7fd6680b43c0 2021-07-11T03:55:04.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd668140d10 con 0x7fd6680b6580 2021-07-11T03:55:04.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.605+0000 7fd66db22700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd668140a70 con 0x7fd668061d30 2021-07-11T03:55:04.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd666ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6680b43c0 0x7fd6680b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:04.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd666ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6680b43c0 0x7fd6680b7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43630/0 (socket says 172.21.2.104:43630) 2021-07-11T03:55:04.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd666ffd700 1 -- 172.21.2.104:0/1216180759 learned_addr learned my addr 172.21.2.104:0/1216180759 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:04.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd667fff700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6680b6580 0x7fd6680b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:04.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd6677fe700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd668061d30 0x7fd6680ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:04.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd666ffd700 1 -- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd668061d30 msgr2=0x7fd6680ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:04.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd666ffd700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd668061d30 0x7fd6680ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:04.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd666ffd700 1 -- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6680b6580 msgr2=0x7fd6680b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:04.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd666ffd700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6680b6580 0x7fd6680b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:04.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd666ffd700 1 -- 172.21.2.104:0/1216180759 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd660009530 con 0x7fd6680b43c0 2021-07-11T03:55:04.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd666ffd700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6680b43c0 0x7fd6680b7e20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fd658002f70 tx=0x7fd658006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:04.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd664ff9700 1 -- 172.21.2.104:0/1216180759 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd65800e470 con 0x7fd6680b43c0 2021-07-11T03:55:04.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd664ff9700 1 -- 172.21.2.104:0/1216180759 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd65800e5d0 con 0x7fd6680b43c0 2021-07-11T03:55:04.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd66db22700 1 -- 172.21.2.104:0/1216180759 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6680ba150 con 0x7fd6680b43c0 2021-07-11T03:55:04.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd66db22700 1 -- 172.21.2.104:0/1216180759 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6680af310 con 0x7fd6680b43c0 2021-07-11T03:55:04.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd664ff9700 1 -- 172.21.2.104:0/1216180759 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd65800e930 con 0x7fd6680b43c0 2021-07-11T03:55:04.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd66db22700 1 -- 172.21.2.104:0/1216180759 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6680b7520 con 0x7fd6680b43c0 2021-07-11T03:55:04.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd664ff9700 1 -- 172.21.2.104:0/1216180759 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd658018030 con 0x7fd6680b43c0 2021-07-11T03:55:04.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd664ff9700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd648053210 0x7fd6480556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:04.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd664ff9700 1 -- 172.21.2.104:0/1216180759 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd65806a620 con 0x7fd6680b43c0 2021-07-11T03:55:04.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd6677fe700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd648053210 0x7fd6480556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:04.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.609+0000 7fd6677fe700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd648053210 0x7fd6480556d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fd65c00a000 tx=0x7fd65c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:04.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.613+0000 7fd664ff9700 1 -- 172.21.2.104:0/1216180759 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd658041cf0 con 0x7fd6680b43c0 2021-07-11T03:55:04.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:04.721+0000 7fd66db22700 1 -- 172.21.2.104:0/1216180759 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]} v 0) v1 -- 0x7fd668057350 con 0x7fd6680b43c0 2021-07-11T03:55:05.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.357+0000 7fd664ff9700 1 -- 172.21.2.104:0/1216180759 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command"]}]=0 v86) v1 ==== 139+0+0 (secure 0 0 0) 0x7fd658045310 con 0x7fd6680b43c0 2021-07-11T03:55:05.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 -- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd648053210 msgr2=0x7fd6480556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:05.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd648053210 0x7fd6480556d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fd65c00a000 tx=0x7fd65c008040).stop 2021-07-11T03:55:05.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 -- 172.21.2.104:0/1216180759 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6680b43c0 msgr2=0x7fd6680b7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:05.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6680b43c0 0x7fd6680b7e20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fd658002f70 tx=0x7fd658006ea0).stop 2021-07-11T03:55:05.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 -- 172.21.2.104:0/1216180759 shutdown_connections 2021-07-11T03:55:05.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6680b6580 0x7fd6680b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:05.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd648053210 0x7fd6480556d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:05.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6680b43c0 0x7fd6680b7e20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:05.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 --2- 172.21.2.104:0/1216180759 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd668061d30 0x7fd6680ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:05.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 -- 172.21.2.104:0/1216180759 >> 172.21.2.104:0/1216180759 conn(0x7fd66805c490 msgr2=0x7fd668062110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:05.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 -- 172.21.2.104:0/1216180759 shutdown_connections 2021-07-11T03:55:05.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.361+0000 7fd66db22700 1 -- 172.21.2.104:0/1216180759 wait complete. 2021-07-11T03:55:05.374 INFO:tasks.cephfs_test_runner:test_drop_cache_command (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-07-11T03:55:05.375 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) 2021-07-11T03:55:05.376 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2021-07-11T03:55:05.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.421+0000 7f7125e65700 1 Processor -- start 2021-07-11T03:55:05.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.421+0000 7f7125e65700 1 -- start start 2021-07-11T03:55:05.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.421+0000 7f7125e65700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7120067300 0x7f71201438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:05.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.421+0000 7f7125e65700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7120143de0 0x7f712014b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:05.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.421+0000 7f7125e65700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71200aabb0 con 0x7f7120060100 2021-07-11T03:55:05.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.421+0000 7f7125e65700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71200604e0 con 0x7f7120067300 2021-07-11T03:55:05.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.421+0000 7f7125e65700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7120066ea0 con 0x7f7120143de0 2021-07-11T03:55:05.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711f7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f7120060100 0x7f71200b4750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40404/0 (socket says 172.21.2.104:40404) 2021-07-11T03:55:05.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711f7fe700 1 -- 172.21.2.104:0/4041681241 learned_addr learned my addr 172.21.2.104:0/4041681241 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:05.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 --2- 172.21.2.104:0/4041681241 >> v2:172.21.2.104:3300/0 conn(0x7f7120067300 0x7f71201438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:05.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 -- 172.21.2.104:0/4041681241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7120143de0 msgr2=0x7f712014b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:05.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 --2- 172.21.2.104:0/4041681241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7120143de0 0x7f712014b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:05.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 -- 172.21.2.104:0/4041681241 >> v1:172.21.2.104:6789/0 conn(0x7f7120060100 legacy=0x7f71200b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:05.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711e7fc700 1 -- 172.21.2.104:0/4041681241 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360876434 0 0) 0x7f71200aabb0 con 0x7f7120060100 2021-07-11T03:55:05.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 -- 172.21.2.104:0/4041681241 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71200b33b0 con 0x7f7120067300 2021-07-11T03:55:05.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 --2- 172.21.2.104:0/4041681241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7120067300 0x7f71201438f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f7108002f70 tx=0x7f7108006df0).ready entity=mon.0 client_cookie=ece7ebc828b544f4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:05.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711e7fc700 1 -- 172.21.2.104:0/4041681241 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f710800d570 con 0x7f7120067300 2021-07-11T03:55:05.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711e7fc700 1 -- 172.21.2.104:0/4041681241 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108000e90 con 0x7f7120067300 2021-07-11T03:55:05.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711e7fc700 1 -- 172.21.2.104:0/4041681241 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f710800d8c0 con 0x7f7120067300 2021-07-11T03:55:05.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- 172.21.2.104:0/4041681241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7120067300 msgr2=0x7f71201438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:05.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 --2- 172.21.2.104:0/4041681241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7120067300 0x7f71201438f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f7108002f70 tx=0x7f7108006df0).stop 2021-07-11T03:55:05.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- 172.21.2.104:0/4041681241 shutdown_connections 2021-07-11T03:55:05.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 --2- 172.21.2.104:0/4041681241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7120067300 0x7f71201438f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:05.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 --2- 172.21.2.104:0/4041681241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7120143de0 0x7f712014b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:05.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- 172.21.2.104:0/4041681241 >> 172.21.2.104:0/4041681241 conn(0x7f712005cf30 msgr2=0x7f712005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:05.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- 172.21.2.104:0/4041681241 shutdown_connections 2021-07-11T03:55:05.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- 172.21.2.104:0/4041681241 wait complete. 2021-07-11T03:55:05.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 Processor -- start 2021-07-11T03:55:05.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- start start 2021-07-11T03:55:05.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7120060100 0x7f712013e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:05.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7120067300 0x7f71200ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:05.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7120143de0 0x7f71200ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:05.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712013fd20 con 0x7f7120067300 2021-07-11T03:55:05.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7120140890 con 0x7f7120143de0 2021-07-11T03:55:05.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7120140b30 con 0x7f7120060100 2021-07-11T03:55:05.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7120067300 0x7f71200ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:05.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7120067300 0x7f71200ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43644/0 (socket says 172.21.2.104:43644) 2021-07-11T03:55:05.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 -- 172.21.2.104:0/2441879563 learned_addr learned my addr 172.21.2.104:0/2441879563 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:05.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 -- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7120060100 msgr2=0x7f712013e400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:55:05.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7120060100 0x7f712013e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:05.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 -- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7120143de0 msgr2=0x7f71200ad570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:55:05.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7120143de0 0x7f71200ad570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:05.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 -- 172.21.2.104:0/2441879563 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710800b180 con 0x7f7120067300 2021-07-11T03:55:05.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711effd700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7120067300 0x7f71200ad080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f7108002f70 tx=0x7f7108015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:05.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711cff9700 1 -- 172.21.2.104:0/2441879563 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7108011950 con 0x7f7120067300 2021-07-11T03:55:05.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711cff9700 1 -- 172.21.2.104:0/2441879563 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108011ab0 con 0x7f7120067300 2021-07-11T03:55:05.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- 172.21.2.104:0/2441879563 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71200ada60 con 0x7f7120067300 2021-07-11T03:55:05.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- 172.21.2.104:0/2441879563 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712013f770 con 0x7f7120067300 2021-07-11T03:55:05.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711cff9700 1 -- 172.21.2.104:0/2441879563 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7108011e10 con 0x7f7120067300 2021-07-11T03:55:05.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f7125e65700 1 -- 172.21.2.104:0/2441879563 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7120140cf0 con 0x7f7120067300 2021-07-11T03:55:05.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711cff9700 1 -- 172.21.2.104:0/2441879563 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f710800f070 con 0x7f7120067300 2021-07-11T03:55:05.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711cff9700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f71040532d0 0x7f7104055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:05.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.425+0000 7f711cff9700 1 -- 172.21.2.104:0/2441879563 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f710806cc10 con 0x7f7120067300 2021-07-11T03:55:05.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.429+0000 7f711f7fe700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f71040532d0 0x7f7104055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:05.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.429+0000 7f711f7fe700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f71040532d0 0x7f7104055790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f7110000bc0 tx=0x7f7110007630).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:05.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.429+0000 7f711cff9700 1 -- 172.21.2.104:0/2441879563 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7108044120 con 0x7f7120067300 2021-07-11T03:55:05.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:05.541+0000 7f7125e65700 1 -- 172.21.2.104:0/2441879563 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f7120057350 con 0x7f7120067300 2021-07-11T03:55:06.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f711cff9700 1 -- 172.21.2.104:0/2441879563 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v87) v1 ==== 147+0+0 (secure 0 0 0) 0x7f7108047740 con 0x7f7120067300 2021-07-11T03:55:06.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 -- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f71040532d0 msgr2=0x7f7104055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f71040532d0 0x7f7104055790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f7110000bc0 tx=0x7f7110007630).stop 2021-07-11T03:55:06.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 -- 172.21.2.104:0/2441879563 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7120067300 msgr2=0x7f71200ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7120067300 0x7f71200ad080 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f7108002f70 tx=0x7f7108015000).stop 2021-07-11T03:55:06.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 -- 172.21.2.104:0/2441879563 shutdown_connections 2021-07-11T03:55:06.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7120143de0 0x7f71200ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:06.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f71040532d0 0x7f7104055790 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7120067300 0x7f71200ad080 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 --2- 172.21.2.104:0/2441879563 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7120060100 0x7f712013e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:06.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 -- 172.21.2.104:0/2441879563 >> 172.21.2.104:0/2441879563 conn(0x7f712005cf30 msgr2=0x7f71201436c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:06.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 -- 172.21.2.104:0/2441879563 shutdown_connections 2021-07-11T03:55:06.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.365+0000 7f7125e65700 1 -- 172.21.2.104:0/2441879563 wait complete. 2021-07-11T03:55:06.379 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T03:55:06.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 Processor -- start 2021-07-11T03:55:06.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 -- start start 2021-07-11T03:55:06.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2dd80b1ec0 0x7f2dd80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2dd80b22a0 0x7f2dd8141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd80a77d0 con 0x7f2dd80affb0 2021-07-11T03:55:06.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd80b03f0 con 0x7f2dd80b22a0 2021-07-11T03:55:06.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80b0650 con 0x7f2dd80b1ec0 2021-07-11T03:55:06.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2ddcd71700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2dd80b22a0 0x7f2dd8141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:06.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2ddcd71700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2dd80b22a0 0x7f2dd8141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43650/0 (socket says 172.21.2.104:43650) 2021-07-11T03:55:06.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2ddcd71700 1 -- 172.21.2.104:0/2351153049 learned_addr learned my addr 172.21.2.104:0/2351153049 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:06.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2ddcd71700 1 -- 172.21.2.104:0/2351153049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2dd80b1ec0 msgr2=0x7f2dd80b12d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:06.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2ddcd71700 1 --2- 172.21.2.104:0/2351153049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2dd80b1ec0 0x7f2dd80b12d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:06.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2ddcd71700 1 -- 172.21.2.104:0/2351153049 >> v1:172.21.2.105:6789/0 conn(0x7f2dd80affb0 legacy=0x7f2dd80a70c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:06.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2ddcd71700 1 -- 172.21.2.104:0/2351153049 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd80b08f0 con 0x7f2dd80b22a0 2021-07-11T03:55:06.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2ddcd71700 1 --2- 172.21.2.104:0/2351153049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2dd80b22a0 0x7f2dd8141250 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f2dd806a0f0 tx=0x7f2dd0009290).ready entity=mon.0 client_cookie=acd141b4891f1a22 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:06.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dd6ffd700 1 -- 172.21.2.104:0/2351153049 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dd000b860 con 0x7f2dd80b22a0 2021-07-11T03:55:06.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dd6ffd700 1 -- 172.21.2.104:0/2351153049 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd000b9c0 con 0x7f2dd80b22a0 2021-07-11T03:55:06.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dd6ffd700 1 -- 172.21.2.104:0/2351153049 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dd000bb70 con 0x7f2dd80b22a0 2021-07-11T03:55:06.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 -- 172.21.2.104:0/2351153049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2dd80b22a0 msgr2=0x7f2dd8141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 --2- 172.21.2.104:0/2351153049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2dd80b22a0 0x7f2dd8141250 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f2dd806a0f0 tx=0x7f2dd0009290).stop 2021-07-11T03:55:06.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 -- 172.21.2.104:0/2351153049 shutdown_connections 2021-07-11T03:55:06.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 --2- 172.21.2.104:0/2351153049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2dd80b22a0 0x7f2dd8141250 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 --2- 172.21.2.104:0/2351153049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2dd80b1ec0 0x7f2dd80b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:06.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 -- 172.21.2.104:0/2351153049 >> 172.21.2.104:0/2351153049 conn(0x7f2dd805f6b0 msgr2=0x7f2dd805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:06.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 -- 172.21.2.104:0/2351153049 shutdown_connections 2021-07-11T03:55:06.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 -- 172.21.2.104:0/2351153049 wait complete. 2021-07-11T03:55:06.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 Processor -- start 2021-07-11T03:55:06.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.433+0000 7f2dde7d4700 1 -- start start 2021-07-11T03:55:06.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dde7d4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2dd80affb0 0x7f2dd81405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dde7d4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2dd80b1ec0 0x7f2dd815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dde7d4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2dd80b22a0 0x7f2dd815ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dde7d4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd815a1c0 con 0x7f2dd80affb0 2021-07-11T03:55:06.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dde7d4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80ae000 con 0x7f2dd80b22a0 2021-07-11T03:55:06.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dde7d4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd80ae2a0 con 0x7f2dd80b1ec0 2021-07-11T03:55:06.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2dd80affb0 0x7f2dd81405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:06.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2dd80affb0 0x7f2dd81405b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43654/0 (socket says 172.21.2.104:43654) 2021-07-11T03:55:06.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd7fff700 1 -- 172.21.2.104:0/1686633473 learned_addr learned my addr 172.21.2.104:0/1686633473 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:06.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd7fff700 1 -- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2dd80b1ec0 msgr2=0x7f2dd815a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd7fff700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2dd80b1ec0 0x7f2dd815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:06.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd7fff700 1 -- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2dd80b22a0 msgr2=0x7f2dd815ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd7fff700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2dd80b22a0 0x7f2dd815ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:06.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd7fff700 1 -- 172.21.2.104:0/1686633473 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd0009530 con 0x7f2dd80affb0 2021-07-11T03:55:06.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd7fff700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2dd80affb0 0x7f2dd81405b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f2dc0002fb0 tx=0x7f2dc0006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:06.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd57fa700 1 -- 172.21.2.104:0/1686633473 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dc0013460 con 0x7f2dd80affb0 2021-07-11T03:55:06.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd57fa700 1 -- 172.21.2.104:0/1686633473 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc00135c0 con 0x7f2dd80affb0 2021-07-11T03:55:06.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dde7d4700 1 -- 172.21.2.104:0/1686633473 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd815a3f0 con 0x7f2dd80affb0 2021-07-11T03:55:06.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd57fa700 1 -- 172.21.2.104:0/1686633473 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2dc0013920 con 0x7f2dd80affb0 2021-07-11T03:55:06.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dde7d4700 1 -- 172.21.2.104:0/1686633473 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd80adbd0 con 0x7f2dd80affb0 2021-07-11T03:55:06.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dde7d4700 1 -- 172.21.2.104:0/1686633473 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd80ad650 con 0x7f2dd80affb0 2021-07-11T03:55:06.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd57fa700 1 -- 172.21.2.104:0/1686633473 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2dc000f070 con 0x7f2dd80affb0 2021-07-11T03:55:06.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd57fa700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2dbc053350 0x7f2dbc055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd57fa700 1 -- 172.21.2.104:0/1686633473 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2dc006ad80 con 0x7f2dd80affb0 2021-07-11T03:55:06.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd77fe700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2dbc053350 0x7f2dbc055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:06.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.437+0000 7f2dd77fe700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2dbc053350 0x7f2dbc055810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f2dc8002fb0 tx=0x7f2dc80069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:06.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.441+0000 7f2dd57fa700 1 -- 172.21.2.104:0/1686633473 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2dd80ad650 con 0x7f2dd80affb0 2021-07-11T03:55:06.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.525+0000 7f2dde7d4700 1 -- 172.21.2.104:0/1686633473 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2dd80ad650 con 0x7f2dd80affb0 2021-07-11T03:55:06.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dd57fa700 1 -- 172.21.2.104:0/1686633473 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2dd80ad650 con 0x7f2dd80affb0 2021-07-11T03:55:06.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 -- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2dbc053350 msgr2=0x7f2dbc055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2dbc053350 0x7f2dbc055810 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f2dc8002fb0 tx=0x7f2dc80069b0).stop 2021-07-11T03:55:06.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 -- 172.21.2.104:0/1686633473 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2dd80affb0 msgr2=0x7f2dd81405b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2dd80affb0 0x7f2dd81405b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f2dc0002fb0 tx=0x7f2dc0006e10).stop 2021-07-11T03:55:06.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 -- 172.21.2.104:0/1686633473 shutdown_connections 2021-07-11T03:55:06.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2dd80b22a0 0x7f2dd815ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:06.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2dd80b1ec0 0x7f2dd815a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:06.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2dbc053350 0x7f2dbc055810 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 --2- 172.21.2.104:0/1686633473 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2dd80affb0 0x7f2dd81405b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 -- 172.21.2.104:0/1686633473 >> 172.21.2.104:0/1686633473 conn(0x7f2dd805f6b0 msgr2=0x7f2dd80b5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:06.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 -- 172.21.2.104:0/1686633473 shutdown_connections 2021-07-11T03:55:06.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.529+0000 7f2dde7d4700 1 -- 172.21.2.104:0/1686633473 wait complete. 2021-07-11T03:55:06.543 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T03:55:06.543 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:55:06.552 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:55:06.560 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:55:06.574 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:55:06.587 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T03:55:06.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:06.589 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:55:06.589 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:06.627 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T03:55:06.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:55:06.629 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T03:55:06.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:06.632 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:55:06.632 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:55:06.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:55:06.640 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:55:06.640 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T03:55:06.641 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T03:55:06.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 Processor -- start 2021-07-11T03:55:06.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 -- start start 2021-07-11T03:55:06.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2ea80b1db0 0x7f2ea80b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2ea81483a0 0x7f2ea80b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea80b0350 con 0x7f2ea80a70c0 2021-07-11T03:55:06.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea80a90c0 con 0x7f2ea80b1db0 2021-07-11T03:55:06.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea80b05f0 con 0x7f2ea81483a0 2021-07-11T03:55:06.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2ea7fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f2ea80a70c0 0x7f2ea8144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40432/0 (socket says 172.21.2.104:40432) 2021-07-11T03:55:06.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2ea7fff700 1 -- 172.21.2.104:0/3604733112 learned_addr learned my addr 172.21.2.104:0/3604733112 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:06.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2ea77fe700 1 --2- 172.21.2.104:0/3604733112 >> v2:172.21.2.104:3300/0 conn(0x7f2ea80b1db0 0x7f2ea80b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:06.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2ea77fe700 1 -- 172.21.2.104:0/3604733112 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2ea81483a0 msgr2=0x7f2ea80b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:06.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2ea77fe700 1 --2- 172.21.2.104:0/3604733112 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2ea81483a0 0x7f2ea80b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:06.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2ea77fe700 1 -- 172.21.2.104:0/3604733112 >> v1:172.21.2.104:6789/0 conn(0x7f2ea80a70c0 legacy=0x7f2ea8144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2ea77fe700 1 -- 172.21.2.104:0/3604733112 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea80b0890 con 0x7f2ea80b1db0 2021-07-11T03:55:06.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2ea77fe700 1 --2- 172.21.2.104:0/3604733112 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2ea80b1db0 0x7f2ea80b0fd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f2e90002f70 tx=0x7f2e90006df0).ready entity=mon.0 client_cookie=149c47606db35a30 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:06.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2ea67fc700 1 -- 172.21.2.104:0/3604733112 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e9000d570 con 0x7f2ea80b1db0 2021-07-11T03:55:06.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2ea67fc700 1 -- 172.21.2.104:0/3604733112 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e90000e90 con 0x7f2ea80b1db0 2021-07-11T03:55:06.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2ea67fc700 1 -- 172.21.2.104:0/3604733112 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e9000d8c0 con 0x7f2ea80b1db0 2021-07-11T03:55:06.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 -- 172.21.2.104:0/3604733112 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2ea80b1db0 msgr2=0x7f2ea80b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 --2- 172.21.2.104:0/3604733112 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2ea80b1db0 0x7f2ea80b0fd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f2e90002f70 tx=0x7f2e90006df0).stop 2021-07-11T03:55:06.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 -- 172.21.2.104:0/3604733112 shutdown_connections 2021-07-11T03:55:06.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 --2- 172.21.2.104:0/3604733112 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2ea81483a0 0x7f2ea80b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:06.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 --2- 172.21.2.104:0/3604733112 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2ea80b1db0 0x7f2ea80b0fd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 -- 172.21.2.104:0/3604733112 >> 172.21.2.104:0/3604733112 conn(0x7f2ea805f6b0 msgr2=0x7f2ea805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:06.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 -- 172.21.2.104:0/3604733112 shutdown_connections 2021-07-11T03:55:06.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.697+0000 7f2eade2b700 1 -- 172.21.2.104:0/3604733112 wait complete. 2021-07-11T03:55:06.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2eade2b700 1 Processor -- start 2021-07-11T03:55:06.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2eade2b700 1 -- start start 2021-07-11T03:55:06.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2eade2b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2ea80a70c0 0x7f2ea814e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2eade2b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2ea80b1db0 0x7f2ea813d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2eade2b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2ea81483a0 0x7f2ea813dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2eade2b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea814bae0 con 0x7f2ea80b1db0 2021-07-11T03:55:06.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2eade2b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea814b100 con 0x7f2ea81483a0 2021-07-11T03:55:06.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2eade2b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea814b3a0 con 0x7f2ea80a70c0 2021-07-11T03:55:06.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2ea80b1db0 0x7f2ea813d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:06.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2ea80b1db0 0x7f2ea813d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43672/0 (socket says 172.21.2.104:43672) 2021-07-11T03:55:06.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea6ffd700 1 -- 172.21.2.104:0/2690901873 learned_addr learned my addr 172.21.2.104:0/2690901873 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:06.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea77fe700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2ea80a70c0 0x7f2ea814e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:06.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea6ffd700 1 -- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2ea80a70c0 msgr2=0x7f2ea814e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea6ffd700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2ea80a70c0 0x7f2ea814e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea7fff700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2ea81483a0 0x7f2ea813dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:06.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea6ffd700 1 -- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2ea81483a0 msgr2=0x7f2ea813dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea6ffd700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2ea81483a0 0x7f2ea813dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea6ffd700 1 -- 172.21.2.104:0/2690901873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9000b180 con 0x7f2ea80b1db0 2021-07-11T03:55:06.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea7fff700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2ea81483a0 0x7f2ea813dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:55:06.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea6ffd700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2ea80b1db0 0x7f2ea813d850 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f2e98002f70 tx=0x7f2e98006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:06.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea4ff9700 1 -- 172.21.2.104:0/2690901873 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e98013460 con 0x7f2ea80b1db0 2021-07-11T03:55:06.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea4ff9700 1 -- 172.21.2.104:0/2690901873 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e980135c0 con 0x7f2ea80b1db0 2021-07-11T03:55:06.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2eade2b700 1 -- 172.21.2.104:0/2690901873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea814b640 con 0x7f2ea80b1db0 2021-07-11T03:55:06.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea4ff9700 1 -- 172.21.2.104:0/2690901873 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e98013920 con 0x7f2ea80b1db0 2021-07-11T03:55:06.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2eade2b700 1 -- 172.21.2.104:0/2690901873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea813e230 con 0x7f2ea80b1db0 2021-07-11T03:55:06.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2eade2b700 1 -- 172.21.2.104:0/2690901873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea814b7d0 con 0x7f2ea80b1db0 2021-07-11T03:55:06.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea4ff9700 1 -- 172.21.2.104:0/2690901873 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2e9800f070 con 0x7f2ea80b1db0 2021-07-11T03:55:06.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea4ff9700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e8c053300 0x7f2e8c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:06.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.701+0000 7f2ea4ff9700 1 -- 172.21.2.104:0/2690901873 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2e9806aad0 con 0x7f2ea80b1db0 2021-07-11T03:55:06.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.705+0000 7f2ea77fe700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e8c053300 0x7f2e8c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:06.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.705+0000 7f2ea77fe700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e8c053300 0x7f2e8c0557c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f2e90019000 tx=0x7f2e90015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:06.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.705+0000 7f2ea4ff9700 1 -- 172.21.2.104:0/2690901873 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2e980420f0 con 0x7f2ea80b1db0 2021-07-11T03:55:06.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.785+0000 7f2eade2b700 1 -- 172.21.2.104:0/2690901873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2ea8057350 con 0x7f2ea80b1db0 2021-07-11T03:55:06.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2ea4ff9700 1 -- 172.21.2.104:0/2690901873 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v36) v1 ==== 81+0+28827 (secure 0 0 0) 0x7f2e98045710 con 0x7f2ea80b1db0 2021-07-11T03:55:06.794 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:06.794 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:55:06.795 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 36, 2021-07-11T03:55:06.795 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T03:55:06.795 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T03:55:06.795 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T03:55:04.341509+0000", 2021-07-11T03:55:06.796 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:55:06.796 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:55:06.796 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T03:55:06.796 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T03:55:06.797 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T03:55:06.797 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T03:55:06.797 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T03:55:06.797 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T03:55:06.797 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T03:55:06.798 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:55:06.798 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T03:55:06.798 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T03:55:06.798 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T03:55:06.799 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T03:55:06.799 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T03:55:06.799 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 6, 2021-07-11T03:55:06.799 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T03:55:06.799 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T03:55:06.800 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T03:55:06.800 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T03:55:06.800 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T03:55:06.800 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.801 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T03:55:06.801 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T03:55:06.801 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T03:55:06.801 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:55:06.801 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:55:06.802 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:55:06.802 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:55:06.802 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:55:06.802 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:55:06.803 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:55:06.803 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:55:06.803 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T03:55:06.803 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T03:55:06.803 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T03:55:06.804 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T03:55:06.804 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T03:55:06.804 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:55:06.804 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:55:06.804 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:55:06.805 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:55:06.805 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:55:06.805 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:55:06.805 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:55:06.806 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.806 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T03:55:06.806 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:55:06.806 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:55:06.806 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:55:06.807 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:55:06.807 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:55:06.807 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:55:06.807 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:55:06.808 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:55:06.808 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:55:06.808 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:55:06.808 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:55:06.809 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:55:06.809 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:55:06.809 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:55:06.809 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:55:06.809 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:55:06.810 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:55:06.810 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:55:06.810 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:55:06.810 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:55:06.811 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:55:06.811 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:55:06.811 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:55:06.811 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:55:06.811 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:55:06.812 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:55:06.812 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.812 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:55:06.812 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:55:06.812 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:55:06.813 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:55:06.813 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:55:06.813 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:55:06.813 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:55:06.814 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:55:06.814 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:55:06.814 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:55:06.814 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:55:06.815 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:55:06.815 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T03:55:06.815 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.815 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:55:06.815 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T03:55:06.816 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.816 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.816 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.816 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T03:55:06.817 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T03:55:06.817 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T03:55:06.817 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:55:06.817 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:55:06.817 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:55:06.818 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:55:06.818 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:55:06.818 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:55:06.818 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:55:06.818 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:55:06.819 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:55:06.819 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:55:06.819 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:55:06.819 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:55:06.820 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:55:06.820 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:55:06.820 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:55:06.820 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:55:06.820 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:55:06.821 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:55:06.821 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:55:06.821 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:55:06.821 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.822 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T03:55:06.822 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:55:06.822 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:55:06.822 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:55:06.822 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:55:06.823 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:55:06.823 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:55:06.823 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:55:06.823 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:55:06.824 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:55:06.824 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:55:06.824 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:55:06.824 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:55:06.824 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:55:06.825 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:55:06.825 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:55:06.825 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:55:06.825 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:55:06.826 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:55:06.826 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:55:06.826 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:55:06.826 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:55:06.826 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:55:06.827 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:55:06.827 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:55:06.827 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:55:06.827 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:55:06.828 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.828 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:55:06.828 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:55:06.828 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:55:06.828 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:55:06.829 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:55:06.829 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:55:06.829 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:55:06.829 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:55:06.829 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:55:06.830 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:55:06.830 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:55:06.830 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:55:06.830 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:55:06.831 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T03:55:06.831 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.831 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.831 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:55:06.831 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T03:55:06.832 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.832 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:55:06.832 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T03:55:06.832 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:55:06.833 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:55:06.833 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:55:06.833 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:55:06.833 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:55:06.833 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:55:06.834 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:55:06.834 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:55:06.834 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:55:06.834 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:55:06.835 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:55:06.835 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.835 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.835 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.835 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T03:55:06.836 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:55:06.836 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.836 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.836 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.837 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T03:55:06.837 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:55:06.837 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.837 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.837 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.838 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:55:06.838 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.838 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.838 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.839 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T03:55:06.839 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:55:06.839 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.839 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.839 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.840 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T03:55:06.840 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:55:06.840 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.840 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.841 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.841 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:55:06.841 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.841 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.842 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.842 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T03:55:06.842 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:55:06.842 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.843 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.843 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.843 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T03:55:06.844 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:55:06.844 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.844 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.844 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.845 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:55:06.845 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.845 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.845 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.846 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T03:55:06.846 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:55:06.846 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.846 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.846 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.847 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T03:55:06.847 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:55:06.847 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.847 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.848 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.848 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T03:55:06.848 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T03:55:06.848 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T03:55:06.848 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T03:55:06.849 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:55:06.849 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:55:06.849 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:55:06.849 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.849 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.850 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.850 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:55:06.850 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T03:55:06.850 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:55:06.851 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:55:06.851 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:55:06.851 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:55:06.851 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:55:06.852 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:55:06.852 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:55:06.852 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:55:06.852 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:55:06.852 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:55:06.853 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:55:06.853 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.853 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.853 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.854 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T03:55:06.854 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:55:06.854 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.854 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.855 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.855 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T03:55:06.855 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:55:06.855 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.855 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.856 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.856 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:55:06.856 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.856 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.856 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.857 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T03:55:06.857 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:55:06.857 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.857 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.858 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.858 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T03:55:06.858 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:55:06.858 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.858 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.859 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.859 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:55:06.859 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.859 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.860 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.860 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T03:55:06.860 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:55:06.860 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.860 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.861 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.861 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T03:55:06.861 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:55:06.861 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.861 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.862 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.862 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:55:06.862 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.862 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.863 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.863 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T03:55:06.863 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:55:06.863 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.863 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.864 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.864 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T03:55:06.864 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:55:06.864 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.865 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.865 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.865 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T03:55:06.865 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T03:55:06.865 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T03:55:06.866 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T03:55:06.866 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:55:06.866 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:55:06.866 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:55:06.866 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.867 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.867 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.867 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:55:06.867 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T03:55:06.868 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:55:06.868 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:55:06.868 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:55:06.868 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:55:06.869 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:55:06.869 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:55:06.869 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:55:06.869 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 30, 2021-07-11T03:55:06.869 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:55:06.870 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:55:06.870 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:55:06.870 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.870 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.871 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.871 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T03:55:06.871 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:55:06.871 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.871 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.872 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.872 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T03:55:06.872 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:55:06.872 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.872 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.873 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.873 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:55:06.873 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.873 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.874 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.874 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T03:55:06.874 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:55:06.874 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.875 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.875 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.875 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T03:55:06.875 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:55:06.875 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.876 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.876 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.876 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:55:06.876 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.877 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.877 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.877 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T03:55:06.877 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:55:06.877 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.878 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.878 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.878 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T03:55:06.878 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:55:06.879 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.879 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.879 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.879 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:55:06.879 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.880 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.880 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.880 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T03:55:06.880 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:55:06.880 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.881 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.881 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.881 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T03:55:06.881 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:55:06.882 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.882 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.882 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.882 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T03:55:06.882 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T03:55:06.883 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T03:55:06.883 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T03:55:06.883 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:55:06.883 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:55:06.884 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:55:06.884 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.884 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.884 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.885 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:55:06.885 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T03:55:06.885 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:55:06.885 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:55:06.885 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:55:06.886 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:55:06.886 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:55:06.886 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:55:06.886 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:55:06.887 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:55:06.887 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:55:06.887 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:55:06.887 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:55:06.887 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.888 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.888 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.888 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T03:55:06.888 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:55:06.888 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.889 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.889 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.889 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T03:55:06.889 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:55:06.890 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.890 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.890 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.890 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:55:06.891 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.891 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.891 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.891 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T03:55:06.892 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:55:06.892 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.892 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.892 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.892 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T03:55:06.893 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:55:06.893 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.893 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.893 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.894 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:55:06.894 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.894 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.894 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.894 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T03:55:06.895 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:55:06.895 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.895 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.895 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.896 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T03:55:06.896 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:55:06.896 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.896 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.896 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.897 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:55:06.897 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.897 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.897 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.898 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T03:55:06.898 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:55:06.898 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.898 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.898 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.899 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T03:55:06.899 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:55:06.899 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.899 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.900 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.900 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T03:55:06.900 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T03:55:06.900 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T03:55:06.900 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T03:55:06.901 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:55:06.901 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:55:06.901 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:55:06.901 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.901 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.902 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.902 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:55:06.902 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T03:55:06.902 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:55:06.903 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:55:06.903 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:55:06.903 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:55:06.903 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:55:06.903 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:55:06.904 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:55:06.904 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:55:06.904 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:55:06.904 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:55:06.905 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:55:06.905 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.905 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.905 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.906 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T03:55:06.906 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:55:06.906 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.906 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.906 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.907 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T03:55:06.907 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:55:06.907 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.907 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.907 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.908 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:55:06.908 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.908 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.908 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.909 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T03:55:06.909 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:55:06.909 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.909 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.909 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.910 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T03:55:06.910 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:55:06.910 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.910 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.911 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.911 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:55:06.911 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.911 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.911 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.912 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T03:55:06.912 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:55:06.912 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.912 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.913 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.913 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T03:55:06.913 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:55:06.913 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.913 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.914 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.914 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:55:06.914 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.914 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.915 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.915 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T03:55:06.915 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:55:06.915 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.915 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.916 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.916 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T03:55:06.916 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:55:06.916 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.917 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.917 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.917 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T03:55:06.917 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T03:55:06.917 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T03:55:06.918 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T03:55:06.918 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:55:06.918 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:55:06.918 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:55:06.918 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.919 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.919 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.919 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:55:06.919 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T03:55:06.920 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:55:06.920 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:55:06.920 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:55:06.920 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:55:06.920 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:55:06.921 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:55:06.921 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:55:06.921 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:55:06.921 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:55:06.921 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:55:06.922 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:55:06.922 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.922 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.922 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.923 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T03:55:06.923 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:55:06.923 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.923 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.923 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.924 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T03:55:06.924 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:55:06.924 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.924 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.925 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.925 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:55:06.925 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.925 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.925 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.926 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T03:55:06.926 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:55:06.926 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.926 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.927 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.927 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T03:55:06.927 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:55:06.927 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.927 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.928 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.928 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:55:06.928 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.928 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.929 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.929 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T03:55:06.929 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:55:06.929 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.930 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.930 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.930 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T03:55:06.930 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:55:06.930 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.931 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.931 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.931 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:55:06.931 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.932 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.932 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.932 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T03:55:06.932 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:55:06.932 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.933 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.933 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.933 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T03:55:06.933 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:55:06.933 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.934 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.934 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.934 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T03:55:06.934 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T03:55:06.935 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T03:55:06.935 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T03:55:06.935 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:55:06.935 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:55:06.936 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:55:06.936 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.936 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.936 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.936 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:55:06.937 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T03:55:06.937 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:55:06.937 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:55:06.937 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:55:06.937 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:55:06.938 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:55:06.938 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:55:06.938 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:55:06.938 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 30, 2021-07-11T03:55:06.939 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:55:06.939 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:55:06.939 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:55:06.939 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.939 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.940 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.940 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T03:55:06.940 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:55:06.940 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.941 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.941 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.941 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T03:55:06.941 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:55:06.941 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.942 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.942 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.942 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:55:06.942 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.942 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.943 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.943 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T03:55:06.943 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:55:06.943 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.944 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.944 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.944 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T03:55:06.944 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:55:06.945 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.945 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.945 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.945 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:55:06.945 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.946 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.946 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.946 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T03:55:06.946 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:55:06.947 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.947 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.947 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.947 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T03:55:06.947 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:55:06.948 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.948 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.948 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.948 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:55:06.949 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.949 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.949 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.949 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T03:55:06.949 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:55:06.950 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.950 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.950 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.950 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T03:55:06.950 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:55:06.951 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.951 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.951 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.951 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T03:55:06.952 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T03:55:06.952 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T03:55:06.952 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T03:55:06.952 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:55:06.952 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:55:06.953 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:55:06.953 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.953 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.953 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.953 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:55:06.954 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T03:55:06.954 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:55:06.954 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:55:06.954 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:55:06.955 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:55:06.955 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:55:06.955 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:55:06.955 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:55:06.955 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 31, 2021-07-11T03:55:06.956 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:55:06.956 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:55:06.956 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:55:06.956 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.956 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.957 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.957 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T03:55:06.957 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:55:06.957 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.958 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.958 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.958 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T03:55:06.958 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:55:06.959 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.959 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.959 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.959 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:55:06.959 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.960 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.960 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T03:55:06.960 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:55:06.960 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.961 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.961 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.961 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T03:55:06.961 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:55:06.962 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.962 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.962 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.962 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:55:06.962 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.963 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.963 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.963 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T03:55:06.963 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:55:06.964 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.964 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.964 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.964 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T03:55:06.964 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:55:06.965 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.965 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.965 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.965 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:55:06.966 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:55:06.966 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.966 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:55:06.966 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T03:55:06.967 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:55:06.967 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.967 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.967 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:55:06.967 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T03:55:06.968 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:55:06.968 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.968 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.968 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.968 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T03:55:06.969 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T03:55:06.969 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T03:55:06.969 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T03:55:06.969 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:55:06.970 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:55:06.970 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:55:06.970 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:06.970 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.971 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:55:06.971 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T03:55:06.971 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.971 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:55:06.971 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:55:06.972 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:55:06.972 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:55:06.972 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:55:06.972 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:55:06.972 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T03:55:06.973 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:55:06.973 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.973 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.973 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:55:06.974 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:55:06.974 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:55:06.974 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:55:06.974 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:55:06.975 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:55:06.975 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T03:55:06.975 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:55:06.975 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.976 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.976 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:55:06.976 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:55:06.976 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:55:06.976 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:55:06.977 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:55:06.977 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:55:06.977 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T03:55:06.977 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:55:06.978 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.978 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.978 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:55:06.978 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:55:06.978 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:55:06.979 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:55:06.979 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:55:06.979 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:55:06.979 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T03:55:06.980 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:55:06.980 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.980 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.980 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:55:06.980 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:55:06.981 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:55:06.981 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:55:06.981 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:55:06.981 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:55:06.981 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T03:55:06.982 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:55:06.982 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.982 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.982 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:55:06.983 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:55:06.983 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:55:06.983 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:55:06.983 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:55:06.984 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:55:06.984 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T03:55:06.984 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:55:06.984 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.984 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.985 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:55:06.985 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:55:06.985 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:55:06.985 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:55:06.985 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:55:06.986 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:55:06.986 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T03:55:06.986 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:55:06.986 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.987 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:06.987 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:55:06.987 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:55:06.987 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:55:06.987 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:55:06.988 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:55:06.988 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:55:06.988 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T03:55:06.988 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:55:06.989 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.989 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:55:06.989 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T03:55:06.989 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T03:55:06.989 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T03:55:06.990 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T03:55:06.990 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T03:55:06.990 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6835/1104530072": "2021-07-12T03:55:00.317112+0000", 2021-07-11T03:55:06.990 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6833/1104530072": "2021-07-12T03:55:00.317112+0000" 2021-07-11T03:55:06.991 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.991 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T03:55:06.991 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T03:55:06.991 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T03:55:06.991 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T03:55:06.992 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T03:55:06.992 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T03:55:06.992 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T03:55:06.992 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T03:55:06.992 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:06.993 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:06.993 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T03:55:06.993 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T03:55:06.993 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T03:55:06.994 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T03:55:06.994 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T03:55:06.994 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:55:06.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 -- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e8c053300 msgr2=0x7f2e8c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e8c053300 0x7f2e8c0557c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f2e90019000 tx=0x7f2e90015000).stop 2021-07-11T03:55:06.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 -- 172.21.2.104:0/2690901873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2ea80b1db0 msgr2=0x7f2ea813d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:06.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2ea80b1db0 0x7f2ea813d850 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f2e98002f70 tx=0x7f2e98006df0).stop 2021-07-11T03:55:06.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 -- 172.21.2.104:0/2690901873 shutdown_connections 2021-07-11T03:55:06.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2ea81483a0 0x7f2ea813dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e8c053300 0x7f2e8c0557c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2ea80b1db0 0x7f2ea813d850 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 --2- 172.21.2.104:0/2690901873 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2ea80a70c0 0x7f2ea814e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:06.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 -- 172.21.2.104:0/2690901873 >> 172.21.2.104:0/2690901873 conn(0x7f2ea805f6b0 msgr2=0x7f2ea80acea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:06.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 -- 172.21.2.104:0/2690901873 shutdown_connections 2021-07-11T03:55:06.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:06.789+0000 7f2eade2b700 1 -- 172.21.2.104:0/2690901873 wait complete. 2021-07-11T03:55:06.999 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:55:07.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f385a19e700 1 Processor -- start 2021-07-11T03:55:07.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f385a19e700 1 -- start start 2021-07-11T03:55:07.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f385a19e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f38540b1970 0x7f385405fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f385a19e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38540a7a90 con 0x7f38540a76b0 2021-07-11T03:55:07.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f385a19e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3854147620 con 0x7f3854067180 2021-07-11T03:55:07.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f385a19e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38541478c0 con 0x7f38540b1970 2021-07-11T03:55:07.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38537fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f38540a76b0 0x7f3854066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40444/0 (socket says 172.21.2.104:40444) 2021-07-11T03:55:07.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38537fe700 1 -- 172.21.2.104:0/998606537 learned_addr learned my addr 172.21.2.104:0/998606537 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:07.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 924511640 0 0) 0x7f38540a7a90 con 0x7f38540a76b0 2021-07-11T03:55:07.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3840003260 con 0x7f38540a76b0 2021-07-11T03:55:07.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f3853fff700 1 --2- 172.21.2.104:0/998606537 >> v2:172.21.2.105:3300/0 conn(0x7f38540b1970 0x7f385405fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 203434077 0 0) 0x7f3840003260 con 0x7f38540a76b0 2021-07-11T03:55:07.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 >> v2:172.21.2.105:3300/0 conn(0x7f38540b1970 msgr2=0x7f385405fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:07.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 --2- 172.21.2.104:0/998606537 >> v2:172.21.2.105:3300/0 conn(0x7f38540b1970 0x7f385405fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 >> v1:172.21.2.105:6789/0 conn(0x7f3854067180 legacy=0x7f385405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:07.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854147b60 con 0x7f38540a76b0 2021-07-11T03:55:07.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f3844001660 con 0x7f38540a76b0 2021-07-11T03:55:07.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 >> v1:172.21.2.104:6789/0 conn(0x7f38540a76b0 legacy=0x7f3854066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 --2- 172.21.2.104:0/998606537 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38400037d0 0x7f3840003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 --2- 172.21.2.104:0/998606537 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3840004500 0x7f3840004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 --2- 172.21.2.104:0/998606537 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3840004f40 0x7f384000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38541478c0 con 0x7f38400037d0 2021-07-11T03:55:07.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384000be30 con 0x7f3840004500 2021-07-11T03:55:07.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384000bff0 con 0x7f3840004f40 2021-07-11T03:55:07.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38537fe700 1 --2- 172.21.2.104:0/998606537 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38400037d0 0x7f3840003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38537fe700 1 -- 172.21.2.104:0/998606537 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3840004f40 msgr2=0x7f384000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:07.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38537fe700 1 --2- 172.21.2.104:0/998606537 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3840004f40 0x7f384000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:07.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38537fe700 1 -- 172.21.2.104:0/998606537 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3840004500 msgr2=0x7f3840004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:07.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38537fe700 1 --2- 172.21.2.104:0/998606537 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3840004500 0x7f3840004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:07.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f3853fff700 1 -- 172.21.2.104:0/998606537 reap_dead start 2021-07-11T03:55:07.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.049+0000 7f38537fe700 1 -- 172.21.2.104:0/998606537 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3854147b60 con 0x7f38400037d0 2021-07-11T03:55:07.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f38537fe700 1 --2- 172.21.2.104:0/998606537 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38400037d0 0x7f3840003bf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f3844001020 tx=0x7f3844001e20).ready entity=mon.0 client_cookie=35848b8c4bb9f64f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:07.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f38440070d0 con 0x7f38400037d0 2021-07-11T03:55:07.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3844007230 con 0x7f38400037d0 2021-07-11T03:55:07.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f38527fc700 1 -- 172.21.2.104:0/998606537 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f384400c990 con 0x7f38400037d0 2021-07-11T03:55:07.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- 172.21.2.104:0/998606537 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38400037d0 msgr2=0x7f3840003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 --2- 172.21.2.104:0/998606537 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38400037d0 0x7f3840003bf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f3844001020 tx=0x7f3844001e20).stop 2021-07-11T03:55:07.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- 172.21.2.104:0/998606537 shutdown_connections 2021-07-11T03:55:07.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 --2- 172.21.2.104:0/998606537 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38400037d0 0x7f3840003bf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- 172.21.2.104:0/998606537 >> 172.21.2.104:0/998606537 conn(0x7f38540b53e0 msgr2=0x7f38540b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:07.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- 172.21.2.104:0/998606537 shutdown_connections 2021-07-11T03:55:07.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- 172.21.2.104:0/998606537 wait complete. 2021-07-11T03:55:07.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 Processor -- start 2021-07-11T03:55:07.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- start start 2021-07-11T03:55:07.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f38400037d0 0x7f385414e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38540ae310 0x7f385414cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3854156360 0x7f38541567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38540acff0 con 0x7f38540ae310 2021-07-11T03:55:07.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38540ab330 con 0x7f3854156360 2021-07-11T03:55:07.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38540adbd0 con 0x7f38400037d0 2021-07-11T03:55:07.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3852ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38540ae310 0x7f385414cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3852ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38540ae310 0x7f385414cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43686/0 (socket says 172.21.2.104:43686) 2021-07-11T03:55:07.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3852ffd700 1 -- 172.21.2.104:0/3265209867 learned_addr learned my addr 172.21.2.104:0/3265209867 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:07.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3852ffd700 1 -- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f38400037d0 msgr2=0x7f385414e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3852ffd700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f38400037d0 0x7f385414e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:07.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3852ffd700 1 -- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3854156360 msgr2=0x7f38541567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3852ffd700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3854156360 0x7f38541567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:07.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3852ffd700 1 -- 172.21.2.104:0/3265209867 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38540ade70 con 0x7f38540ae310 2021-07-11T03:55:07.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3852ffd700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38540ae310 0x7f385414cfb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f3848001fa0 tx=0x7f3848007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:07.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3850ff9700 1 -- 172.21.2.104:0/3265209867 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f384800ce20 con 0x7f38540ae310 2021-07-11T03:55:07.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3850ff9700 1 -- 172.21.2.104:0/3265209867 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3848000e90 con 0x7f38540ae310 2021-07-11T03:55:07.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- 172.21.2.104:0/3265209867 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385414dbd0 con 0x7f38540ae310 2021-07-11T03:55:07.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3850ff9700 1 -- 172.21.2.104:0/3265209867 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3848007430 con 0x7f38540ae310 2021-07-11T03:55:07.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- 172.21.2.104:0/3265209867 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3854144690 con 0x7f38540ae310 2021-07-11T03:55:07.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f385a19e700 1 -- 172.21.2.104:0/3265209867 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854057350 con 0x7f38540ae310 2021-07-11T03:55:07.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.053+0000 7f3850ff9700 1 -- 172.21.2.104:0/3265209867 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3848010030 con 0x7f38540ae310 2021-07-11T03:55:07.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.057+0000 7f3850ff9700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3834053270 0x7f3834055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.057+0000 7f3850ff9700 1 -- 172.21.2.104:0/3265209867 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f384806a850 con 0x7f38540ae310 2021-07-11T03:55:07.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.057+0000 7f38537fe700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3834053270 0x7f3834055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.057+0000 7f38537fe700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3834053270 0x7f3834055730 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f3844001020 tx=0x7f3844015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:07.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.057+0000 7f3850ff9700 1 -- 172.21.2.104:0/3265209867 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3848041e70 con 0x7f38540ae310 2021-07-11T03:55:07.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.165+0000 7f385a19e700 1 -- 172.21.2.104:0/3265209867 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38540698e0 con 0x7f38540ae310 2021-07-11T03:55:07.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f3850ff9700 1 -- 172.21.2.104:0/3265209867 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+1560 (secure 0 0 0) 0x7f3848045490 con 0x7f38540ae310 2021-07-11T03:55:07.174 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:07.175 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4408,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2474981992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2474981992},{"type":"v1","addr":"172.21.2.105:6835","nonce":2474981992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/1241687766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1241687766},{"type":"v1","addr":"172.21.2.104:6834","nonce":1241687766}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/4230356883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":4230356883},{"type":"v1","addr":"172.21.2.104:6835","nonce":4230356883}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":12}],"filesystems":[]} 2021-07-11T03:55:07.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 -- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3834053270 msgr2=0x7f3834055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3834053270 0x7f3834055730 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f3844001020 tx=0x7f3844015000).stop 2021-07-11T03:55:07.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 -- 172.21.2.104:0/3265209867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38540ae310 msgr2=0x7f385414cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38540ae310 0x7f385414cfb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f3848001fa0 tx=0x7f3848007c00).stop 2021-07-11T03:55:07.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 -- 172.21.2.104:0/3265209867 shutdown_connections 2021-07-11T03:55:07.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3854156360 0x7f38541567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:07.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3834053270 0x7f3834055730 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f38540ae310 0x7f385414cfb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 --2- 172.21.2.104:0/3265209867 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f38400037d0 0x7f385414e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:07.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 -- 172.21.2.104:0/3265209867 >> 172.21.2.104:0/3265209867 conn(0x7f38540b53e0 msgr2=0x7f38540b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:07.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 -- 172.21.2.104:0/3265209867 shutdown_connections 2021-07-11T03:55:07.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.169+0000 7f385a19e700 1 -- 172.21.2.104:0/3265209867 wait complete. 2021-07-11T03:55:07.179 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 13 2021-07-11T03:55:07.181 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:55:07.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f1685d1a700 1 Processor -- start 2021-07-11T03:55:07.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f1685d1a700 1 -- start start 2021-07-11T03:55:07.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f1685d1a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f16801393d0 0x7f16800b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f1685d1a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16800604e0 con 0x7f1680142840 2021-07-11T03:55:07.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f1685d1a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16800b16b0 con 0x7f1680060100 2021-07-11T03:55:07.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f1685d1a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16800ac400 con 0x7f16801393d0 2021-07-11T03:55:07.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f167effd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f16801393d0 0x7f16800b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f167ffff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1680142840 0x7f1680142ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40462/0 (socket says 172.21.2.104:40462) 2021-07-11T03:55:07.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f167ffff700 1 -- 172.21.2.104:0/803757055 learned_addr learned my addr 172.21.2.104:0/803757055 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:07.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f167effd700 1 -- 172.21.2.104:0/803757055 >> v1:172.21.2.105:6789/0 conn(0x7f1680060100 legacy=0x7f1680138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:07.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f167effd700 1 -- 172.21.2.104:0/803757055 >> v1:172.21.2.104:6789/0 conn(0x7f1680142840 legacy=0x7f1680142ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f167e7fc700 1 -- 172.21.2.104:0/803757055 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585790080 0 0) 0x7f16800604e0 con 0x7f1680142840 2021-07-11T03:55:07.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f167effd700 1 -- 172.21.2.104:0/803757055 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16800ac6a0 con 0x7f16801393d0 2021-07-11T03:55:07.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.225+0000 7f167effd700 1 --2- 172.21.2.104:0/803757055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16801393d0 0x7f16800b11c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f16800628a0 tx=0x7f1668006e80).ready entity=mon.0 client_cookie=707134b26eb24eae server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:07.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167e7fc700 1 -- 172.21.2.104:0/803757055 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f166800ba30 con 0x7f16801393d0 2021-07-11T03:55:07.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167e7fc700 1 -- 172.21.2.104:0/803757055 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1668000e90 con 0x7f16801393d0 2021-07-11T03:55:07.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167e7fc700 1 -- 172.21.2.104:0/803757055 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f166800bd80 con 0x7f16801393d0 2021-07-11T03:55:07.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- 172.21.2.104:0/803757055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16801393d0 msgr2=0x7f16800b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 --2- 172.21.2.104:0/803757055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16801393d0 0x7f16800b11c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f16800628a0 tx=0x7f1668006e80).stop 2021-07-11T03:55:07.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- 172.21.2.104:0/803757055 shutdown_connections 2021-07-11T03:55:07.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 --2- 172.21.2.104:0/803757055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16801393d0 0x7f16800b11c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- 172.21.2.104:0/803757055 >> 172.21.2.104:0/803757055 conn(0x7f1680061070 msgr2=0x7f168005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:07.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- 172.21.2.104:0/803757055 shutdown_connections 2021-07-11T03:55:07.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- 172.21.2.104:0/803757055 wait complete. 2021-07-11T03:55:07.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 Processor -- start 2021-07-11T03:55:07.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- start start 2021-07-11T03:55:07.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1680060100 0x7f168013b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16801393d0 0x7f16800ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1680142840 0x7f1680152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1680151930 con 0x7f1680060100 2021-07-11T03:55:07.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168013a1e0 con 0x7f1680142840 2021-07-11T03:55:07.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1680151b60 con 0x7f16801393d0 2021-07-11T03:55:07.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1680060100 0x7f168013b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1680060100 0x7f168013b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43700/0 (socket says 172.21.2.104:43700) 2021-07-11T03:55:07.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167f7fe700 1 -- 172.21.2.104:0/1619633723 learned_addr learned my addr 172.21.2.104:0/1619633723 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:07.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167f7fe700 1 -- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16801393d0 msgr2=0x7f16800ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167f7fe700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16801393d0 0x7f16800ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:07.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167f7fe700 1 -- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1680142840 msgr2=0x7f1680152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167f7fe700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1680142840 0x7f1680152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:07.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167f7fe700 1 -- 172.21.2.104:0/1619633723 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166800d410 con 0x7f1680060100 2021-07-11T03:55:07.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167f7fe700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1680060100 0x7f168013b300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f1670002f90 tx=0x7f1670006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:07.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167cff9700 1 -- 172.21.2.104:0/1619633723 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1670013460 con 0x7f1680060100 2021-07-11T03:55:07.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167cff9700 1 -- 172.21.2.104:0/1619633723 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16700135c0 con 0x7f1680060100 2021-07-11T03:55:07.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- 172.21.2.104:0/1619633723 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168013a480 con 0x7f1680060100 2021-07-11T03:55:07.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167cff9700 1 -- 172.21.2.104:0/1619633723 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1670013920 con 0x7f1680060100 2021-07-11T03:55:07.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- 172.21.2.104:0/1619633723 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16800ae4f0 con 0x7f1680060100 2021-07-11T03:55:07.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f1685d1a700 1 -- 172.21.2.104:0/1619633723 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1680057350 con 0x7f1680060100 2021-07-11T03:55:07.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.229+0000 7f167cff9700 1 -- 172.21.2.104:0/1619633723 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f167001a060 con 0x7f1680060100 2021-07-11T03:55:07.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.233+0000 7f167cff9700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16640531d0 0x7f1664055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.233+0000 7f167cff9700 1 -- 172.21.2.104:0/1619633723 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f167006ab40 con 0x7f1680060100 2021-07-11T03:55:07.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.233+0000 7f167effd700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16640531d0 0x7f1664055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.233+0000 7f167effd700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16640531d0 0x7f1664055690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f16800628a0 tx=0x7f168013f190).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:07.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.233+0000 7f167cff9700 1 -- 172.21.2.104:0/1619633723 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1670042160 con 0x7f1680060100 2021-07-11T03:55:07.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.337+0000 7f1685d1a700 1 -- 172.21.2.104:0/1619633723 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16800ab3a0 con 0x7f1680060100 2021-07-11T03:55:07.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f167cff9700 1 -- 172.21.2.104:0/1619633723 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+1560 (secure 0 0 0) 0x7f1670018030 con 0x7f1680060100 2021-07-11T03:55:07.346 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:07.346 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4408,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2474981992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2474981992},{"type":"v1","addr":"172.21.2.105:6835","nonce":2474981992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":4860,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/1241687766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1241687766},{"type":"v1","addr":"172.21.2.104:6834","nonce":1241687766}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":9},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/4230356883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":4230356883},{"type":"v1","addr":"172.21.2.104:6835","nonce":4230356883}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":12}],"filesystems":[]} 2021-07-11T03:55:07.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 -- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16640531d0 msgr2=0x7f1664055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16640531d0 0x7f1664055690 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f16800628a0 tx=0x7f168013f190).stop 2021-07-11T03:55:07.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 -- 172.21.2.104:0/1619633723 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1680060100 msgr2=0x7f168013b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1680060100 0x7f168013b300 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f1670002f90 tx=0x7f1670006df0).stop 2021-07-11T03:55:07.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 -- 172.21.2.104:0/1619633723 shutdown_connections 2021-07-11T03:55:07.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1680142840 0x7f1680152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:07.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16801393d0 0x7f16800ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:07.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16640531d0 0x7f1664055690 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 --2- 172.21.2.104:0/1619633723 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1680060100 0x7f168013b300 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 -- 172.21.2.104:0/1619633723 >> 172.21.2.104:0/1619633723 conn(0x7f1680061070 msgr2=0x7f1680142ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:07.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 -- 172.21.2.104:0/1619633723 shutdown_connections 2021-07-11T03:55:07.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.341+0000 7f1685d1a700 1 -- 172.21.2.104:0/1619633723 wait complete. 2021-07-11T03:55:07.350 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 13 2021-07-11T03:55:07.353 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T03:55:07.354 INFO:tasks.ceph.mds.a:Stopping old one... 2021-07-11T03:55:07.354 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T03:55:07.355 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:55:07.355 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T03:55:07.355 INFO:tasks.ceph.mds.c:Stopping old one... 2021-07-11T03:55:07.356 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T03:55:07.356 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:55:07.356 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T03:55:07.356 INFO:tasks.ceph.mds.b:Stopping old one... 2021-07-11T03:55:07.357 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T03:55:07.357 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:55:07.360 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T03:55:07.360 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T03:55:07.362 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T03:55:07.362 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T03:55:07.369 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T03:55:07.370 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T03:55:07.372 INFO:tasks.ceph.mds.b:Started 2021-07-11T03:55:07.372 DEBUG:teuthology.parallel:result is None 2021-07-11T03:55:07.404 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T03:55:07.406 INFO:tasks.ceph.mds.a:Started 2021-07-11T03:55:07.406 DEBUG:teuthology.parallel:result is None 2021-07-11T03:55:07.407 INFO:tasks.ceph.mds.c:Started 2021-07-11T03:55:07.408 DEBUG:teuthology.parallel:result is None 2021-07-11T03:55:07.408 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T03:55:07.440 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T03:55:07.441 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T03:55:07.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 Processor -- start 2021-07-11T03:55:07.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 -- start start 2021-07-11T03:55:07.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb3180b54a0 0x7fb3180af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3181475a0 0x7fb3180acc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3180b5880 con 0x7fb3180b0610 2021-07-11T03:55:07.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3181452e0 con 0x7fb3180b54a0 2021-07-11T03:55:07.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180adff0 con 0x7fb3181475a0 2021-07-11T03:55:07.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31dc4c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb3180b0610 0x7fb3180675a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40500/0 (socket says 172.21.2.104:40500) 2021-07-11T03:55:07.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31dc4c700 1 -- 172.21.2.104:0/3880884402 learned_addr learned my addr 172.21.2.104:0/3880884402 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:07.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31e44d700 1 --2- 172.21.2.104:0/3880884402 >> v2:172.21.2.104:3300/0 conn(0x7fb3180b54a0 0x7fb3180af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31ec4e700 1 --2- 172.21.2.104:0/3880884402 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3181475a0 0x7fb3180acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31e44d700 1 -- 172.21.2.104:0/3880884402 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3181475a0 msgr2=0x7fb3180acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:07.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31e44d700 1 --2- 172.21.2.104:0/3880884402 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3181475a0 0x7fb3180acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31e44d700 1 -- 172.21.2.104:0/3880884402 >> v1:172.21.2.104:6789/0 conn(0x7fb3180b0610 legacy=0x7fb3180675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31e44d700 1 -- 172.21.2.104:0/3880884402 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180ac800 con 0x7fb3180b54a0 2021-07-11T03:55:07.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31e44d700 1 --2- 172.21.2.104:0/3880884402 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3180b54a0 0x7fb3180af550 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fb314002f70 tx=0x7fb314006df0).ready entity=mon.0 client_cookie=297cc236d572f6d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:07.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31d44b700 1 -- 172.21.2.104:0/3880884402 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb31400d570 con 0x7fb3180b54a0 2021-07-11T03:55:07.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31d44b700 1 -- 172.21.2.104:0/3880884402 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb314000e90 con 0x7fb3180b54a0 2021-07-11T03:55:07.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31d44b700 1 -- 172.21.2.104:0/3880884402 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb31400d900 con 0x7fb3180b54a0 2021-07-11T03:55:07.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 -- 172.21.2.104:0/3880884402 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3180b54a0 msgr2=0x7fb3180af550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 --2- 172.21.2.104:0/3880884402 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3180b54a0 0x7fb3180af550 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fb314002f70 tx=0x7fb314006df0).stop 2021-07-11T03:55:07.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 -- 172.21.2.104:0/3880884402 shutdown_connections 2021-07-11T03:55:07.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 --2- 172.21.2.104:0/3880884402 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3181475a0 0x7fb3180acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 --2- 172.21.2.104:0/3880884402 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3180b54a0 0x7fb3180af550 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 -- 172.21.2.104:0/3880884402 >> 172.21.2.104:0/3880884402 conn(0x7fb318062560 msgr2=0x7fb318062940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:07.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 -- 172.21.2.104:0/3880884402 shutdown_connections 2021-07-11T03:55:07.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.457+0000 7fb31f44f700 1 -- 172.21.2.104:0/3880884402 wait complete. 2021-07-11T03:55:07.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31f44f700 1 Processor -- start 2021-07-11T03:55:07.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31f44f700 1 -- start start 2021-07-11T03:55:07.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31f44f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb3180b0610 0x7fb31814cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31f44f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3180b54a0 0x7fb3181569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31f44f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3181475a0 0x7fb318145420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31f44f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb318146dd0 con 0x7fb3180b54a0 2021-07-11T03:55:07.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31f44f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31813f5c0 con 0x7fb3180b0610 2021-07-11T03:55:07.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31f44f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31814d880 con 0x7fb3181475a0 2021-07-11T03:55:07.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31dc4c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3180b54a0 0x7fb3181569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31dc4c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3180b54a0 0x7fb3181569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43738/0 (socket says 172.21.2.104:43738) 2021-07-11T03:55:07.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31dc4c700 1 -- 172.21.2.104:0/3619909252 learned_addr learned my addr 172.21.2.104:0/3619909252 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:07.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31ec4e700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3181475a0 0x7fb318145420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31e44d700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb3180b0610 0x7fb31814cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31dc4c700 1 -- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3181475a0 msgr2=0x7fb318145420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31dc4c700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3181475a0 0x7fb318145420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31dc4c700 1 -- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb3180b0610 msgr2=0x7fb31814cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:07.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31dc4c700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb3180b0610 0x7fb31814cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:07.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31dc4c700 1 -- 172.21.2.104:0/3619909252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31400b180 con 0x7fb3180b54a0 2021-07-11T03:55:07.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31dc4c700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3180b54a0 0x7fb3181569e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fb318069d90 tx=0x7fb308007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:07.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb3077fe700 1 -- 172.21.2.104:0/3619909252 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb3080070d0 con 0x7fb3180b54a0 2021-07-11T03:55:07.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb3077fe700 1 -- 172.21.2.104:0/3619909252 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb308007230 con 0x7fb3180b54a0 2021-07-11T03:55:07.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb3077fe700 1 -- 172.21.2.104:0/3619909252 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb308005610 con 0x7fb3180b54a0 2021-07-11T03:55:07.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31f44f700 1 -- 172.21.2.104:0/3619909252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb318145f40 con 0x7fb3180b54a0 2021-07-11T03:55:07.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31f44f700 1 -- 172.21.2.104:0/3619909252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31814d2c0 con 0x7fb3180b54a0 2021-07-11T03:55:07.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31f44f700 1 -- 172.21.2.104:0/3619909252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31814dc90 con 0x7fb3180b54a0 2021-07-11T03:55:07.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb3077fe700 1 -- 172.21.2.104:0/3619909252 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb308010070 con 0x7fb3180b54a0 2021-07-11T03:55:07.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb3077fe700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2f80532d0 0x7fb2f8055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:07.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb3077fe700 1 -- 172.21.2.104:0/3619909252 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb30806a860 con 0x7fb3180b54a0 2021-07-11T03:55:07.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.461+0000 7fb31e44d700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2f80532d0 0x7fb2f8055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:07.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.465+0000 7fb31e44d700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2f80532d0 0x7fb2f8055790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fb314017000 tx=0x7fb314014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:07.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.465+0000 7fb3077fe700 1 -- 172.21.2.104:0/3619909252 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb308041e80 con 0x7fb3180b54a0 2021-07-11T03:55:07.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:07.557+0000 7fb31f44f700 1 -- 172.21.2.104:0/3619909252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fb318057350 con 0x7fb3180b54a0 2021-07-11T03:55:08.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.449+0000 7fb3077fe700 1 -- 172.21.2.104:0/3619909252 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v40) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb3080454a0 con 0x7fb3180b54a0 2021-07-11T03:55:08.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 -- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2f80532d0 msgr2=0x7fb2f8055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2f80532d0 0x7fb2f8055790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fb314017000 tx=0x7fb314014000).stop 2021-07-11T03:55:08.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 -- 172.21.2.104:0/3619909252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3180b54a0 msgr2=0x7fb3181569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3180b54a0 0x7fb3181569e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fb318069d90 tx=0x7fb308007c40).stop 2021-07-11T03:55:08.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 -- 172.21.2.104:0/3619909252 shutdown_connections 2021-07-11T03:55:08.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3181475a0 0x7fb318145420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2f80532d0 0x7fb2f8055790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3180b54a0 0x7fb3181569e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 --2- 172.21.2.104:0/3619909252 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb3180b0610 0x7fb31814cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 -- 172.21.2.104:0/3619909252 >> 172.21.2.104:0/3619909252 conn(0x7fb318062560 msgr2=0x7fb31814aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:08.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 -- 172.21.2.104:0/3619909252 shutdown_connections 2021-07-11T03:55:08.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.453+0000 7fb31f44f700 1 -- 172.21.2.104:0/3619909252 wait complete. 2021-07-11T03:55:08.462 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T03:55:08.470 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T03:55:08.471 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T03:55:08.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 Processor -- start 2021-07-11T03:55:08.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 -- start start 2021-07-11T03:55:08.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c1483a0 0x7fd64c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd64c0a70c0 0x7fd64c144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c0b0350 con 0x7fd64c0b1db0 2021-07-11T03:55:08.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64c0a90c0 con 0x7fd64c0a70c0 2021-07-11T03:55:08.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0b05f0 con 0x7fd64c1483a0 2021-07-11T03:55:08.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd64ad9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd64c0a70c0 0x7fd64c144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd64ad9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd64c0a70c0 0x7fd64c144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43752/0 (socket says 172.21.2.104:43752) 2021-07-11T03:55:08.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd64ad9d700 1 -- 172.21.2.104:0/3619541646 learned_addr learned my addr 172.21.2.104:0/3619541646 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:08.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd64ad9d700 1 -- 172.21.2.104:0/3619541646 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c1483a0 msgr2=0x7fd64c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:08.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd64ad9d700 1 --2- 172.21.2.104:0/3619541646 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c1483a0 0x7fd64c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:08.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd64ad9d700 1 -- 172.21.2.104:0/3619541646 >> v1:172.21.2.105:6789/0 conn(0x7fd64c0b1db0 legacy=0x7fd64c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:08.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd64ad9d700 1 -- 172.21.2.104:0/3619541646 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c0b0890 con 0x7fd64c0a70c0 2021-07-11T03:55:08.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd64ad9d700 1 --2- 172.21.2.104:0/3619541646 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c0a70c0 0x7fd64c144720 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fd64c0613a0 tx=0x7fd644009290).ready entity=mon.0 client_cookie=3fc48e7583b854d2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:08.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd64959a700 1 -- 172.21.2.104:0/3619541646 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd64400b860 con 0x7fd64c0a70c0 2021-07-11T03:55:08.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd64959a700 1 -- 172.21.2.104:0/3619541646 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64400b9c0 con 0x7fd64c0a70c0 2021-07-11T03:55:08.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd64959a700 1 -- 172.21.2.104:0/3619541646 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd64400bb70 con 0x7fd64c0a70c0 2021-07-11T03:55:08.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 -- 172.21.2.104:0/3619541646 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c0a70c0 msgr2=0x7fd64c144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 --2- 172.21.2.104:0/3619541646 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c0a70c0 0x7fd64c144720 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fd64c0613a0 tx=0x7fd644009290).stop 2021-07-11T03:55:08.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 -- 172.21.2.104:0/3619541646 shutdown_connections 2021-07-11T03:55:08.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 --2- 172.21.2.104:0/3619541646 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c0a70c0 0x7fd64c144720 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 --2- 172.21.2.104:0/3619541646 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c1483a0 0x7fd64c0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:08.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 -- 172.21.2.104:0/3619541646 >> 172.21.2.104:0/3619541646 conn(0x7fd64c05f6b0 msgr2=0x7fd64c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:08.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 -- 172.21.2.104:0/3619541646 shutdown_connections 2021-07-11T03:55:08.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 -- 172.21.2.104:0/3619541646 wait complete. 2021-07-11T03:55:08.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 Processor -- start 2021-07-11T03:55:08.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.525+0000 7fd650cf6700 1 -- start start 2021-07-11T03:55:08.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd650cf6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd64c0a70c0 0x7fd64c14adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd650cf6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c0b1db0 0x7fd64c0ac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd650cf6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c1483a0 0x7fd64c0ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd650cf6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c062940 con 0x7fd64c1483a0 2021-07-11T03:55:08.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd650cf6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0ab480 con 0x7fd64c0a70c0 2021-07-11T03:55:08.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd650cf6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0ab720 con 0x7fd64c0b1db0 2021-07-11T03:55:08.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd64ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c1483a0 0x7fd64c0ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd64ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c1483a0 0x7fd64c0ac5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43758/0 (socket says 172.21.2.104:43758) 2021-07-11T03:55:08.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd64ad9d700 1 -- 172.21.2.104:0/3387174753 learned_addr learned my addr 172.21.2.104:0/3387174753 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:08.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd64ad9d700 1 -- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c0b1db0 msgr2=0x7fd64c0ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd64ad9d700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c0b1db0 0x7fd64c0ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:08.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd64ad9d700 1 -- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd64c0a70c0 msgr2=0x7fd64c14adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd64ad9d700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd64c0a70c0 0x7fd64c14adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:08.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd64ad9d700 1 -- 172.21.2.104:0/3387174753 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd644009530 con 0x7fd64c1483a0 2021-07-11T03:55:08.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd64ad9d700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c1483a0 0x7fd64c0ac5f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fd64c0613a0 tx=0x7fd644016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:08.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd63b7fe700 1 -- 172.21.2.104:0/3387174753 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd644011c70 con 0x7fd64c1483a0 2021-07-11T03:55:08.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd63b7fe700 1 -- 172.21.2.104:0/3387174753 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd644011dd0 con 0x7fd64c1483a0 2021-07-11T03:55:08.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd650cf6700 1 -- 172.21.2.104:0/3387174753 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c064b00 con 0x7fd64c1483a0 2021-07-11T03:55:08.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd63b7fe700 1 -- 172.21.2.104:0/3387174753 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd64401f600 con 0x7fd64c1483a0 2021-07-11T03:55:08.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd650cf6700 1 -- 172.21.2.104:0/3387174753 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c156830 con 0x7fd64c1483a0 2021-07-11T03:55:08.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd650cf6700 1 -- 172.21.2.104:0/3387174753 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c057350 con 0x7fd64c1483a0 2021-07-11T03:55:08.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd63b7fe700 1 -- 172.21.2.104:0/3387174753 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd64400f070 con 0x7fd64c1483a0 2021-07-11T03:55:08.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd63b7fe700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 0x7fd6280557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd63b7fe700 1 -- 172.21.2.104:0/3387174753 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd64406bb30 con 0x7fd64c1483a0 2021-07-11T03:55:08.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd64a59c700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 0x7fd6280557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.529+0000 7fd64a59c700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 0x7fd6280557c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd63c000f70 tx=0x7fd63c0077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:08.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.533+0000 7fd63b7fe700 1 -- 172.21.2.104:0/3387174753 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd6440431a0 con 0x7fd64c1483a0 2021-07-11T03:55:08.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.641+0000 7fd650cf6700 1 -- 172.21.2.104:0/3387174753 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd64c044650 con 0x7fd64c1483a0 2021-07-11T03:55:08.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.641+0000 7fd63b7fe700 1 -- 172.21.2.104:0/3387174753 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fd6440467c0 con 0x7fd64c1483a0 2021-07-11T03:55:08.649 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:08.649 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:55:08.650 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T03:55:08.650 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.650 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T03:55:08.650 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T03:55:08.651 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.651 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:55:08.651 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.651 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.651 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.652 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.652 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.652 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.652 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T03:55:08.653 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T03:55:08.653 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.653 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:55:08.653 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.653 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.654 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.654 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.654 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.654 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.654 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T03:55:08.655 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T03:55:08.655 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.655 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:55:08.655 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.655 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.656 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.656 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.656 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.656 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.657 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T03:55:08.657 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T03:55:08.657 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.657 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.657 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.658 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.658 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.658 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.658 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.658 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T03:55:08.659 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T03:55:08.659 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.659 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.659 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.660 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.660 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.660 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.660 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.661 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T03:55:08.661 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T03:55:08.661 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.661 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.662 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.662 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.662 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.662 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.662 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.663 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T03:55:08.663 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T03:55:08.663 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.663 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.663 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.664 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.664 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.664 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.664 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.665 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T03:55:08.665 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T03:55:08.665 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.665 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.665 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.666 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.666 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.666 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.668 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.668 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T03:55:08.669 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T03:55:08.669 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.669 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.670 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.670 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.670 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.670 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.670 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.671 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T03:55:08.671 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T03:55:08.671 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.671 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.672 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.672 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.672 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.672 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.672 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.673 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T03:55:08.673 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T03:55:08.673 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.673 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.673 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.674 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.674 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.674 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.674 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.674 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T03:55:08.675 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T03:55:08.675 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.675 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:55:08.675 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T03:55:08.676 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T03:55:08.676 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.676 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.676 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.676 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T03:55:08.677 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T03:55:08.677 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.677 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:55:08.677 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T03:55:08.677 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T03:55:08.678 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.678 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.678 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.678 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T03:55:08.678 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T03:55:08.679 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.679 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:55:08.679 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:55:08.679 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:55:08.680 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.680 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.680 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.680 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.680 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T03:55:08.681 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T03:55:08.681 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.681 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:55:08.681 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T03:55:08.681 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.682 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.682 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:08.684 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:08.684 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T03:55:08.684 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T03:55:08.684 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:55:08.685 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:55:08.685 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T03:55:08.685 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:55:08.685 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.685 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:08.686 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:55:08.686 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:55:08.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 -- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 msgr2=0x7fd6280557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 0x7fd6280557c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fd63c000f70 tx=0x7fd63c0077f0).stop 2021-07-11T03:55:08.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 -- 172.21.2.104:0/3387174753 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c1483a0 msgr2=0x7fd64c0ac5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c1483a0 0x7fd64c0ac5f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7fd64c0613a0 tx=0x7fd644016000).stop 2021-07-11T03:55:08.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 -- 172.21.2.104:0/3387174753 shutdown_connections 2021-07-11T03:55:08.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd628053300 0x7fd6280557c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c1483a0 0x7fd64c0ac5f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c0b1db0 0x7fd64c0ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:08.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 --2- 172.21.2.104:0/3387174753 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd64c0a70c0 0x7fd64c14adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:08.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 -- 172.21.2.104:0/3387174753 >> 172.21.2.104:0/3387174753 conn(0x7fd64c05f6b0 msgr2=0x7fd64c0aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:08.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 -- 172.21.2.104:0/3387174753 shutdown_connections 2021-07-11T03:55:08.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.645+0000 7fd650cf6700 1 -- 172.21.2.104:0/3387174753 wait complete. 2021-07-11T03:55:08.691 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:55:08.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 Processor -- start 2021-07-11T03:55:08.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 -- start start 2021-07-11T03:55:08.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a00a76b0 0x7fc8a0066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc8a00b1970 0x7fc8a0066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc8a00b0060 0x7fc8a005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a0067450 con 0x7fc8a00b1970 2021-07-11T03:55:08.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a00a7a90 con 0x7fc8a00b0060 2021-07-11T03:55:08.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a0147620 con 0x7fc8a00a76b0 2021-07-11T03:55:08.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89effd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc8a00b1970 0x7fc8a0066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89effd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc8a00b1970 0x7fc8a0066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43762/0 (socket says 172.21.2.104:43762) 2021-07-11T03:55:08.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89effd700 1 -- 172.21.2.104:0/2095860704 learned_addr learned my addr 172.21.2.104:0/2095860704 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:08.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89f7fe700 1 --2- 172.21.2.104:0/2095860704 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a00a76b0 0x7fc8a0066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89ffff700 1 --2- 172.21.2.104:0/2095860704 >> v2:172.21.2.105:3300/0 conn(0x7fc8a00b0060 0x7fc8a005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89effd700 1 -- 172.21.2.104:0/2095860704 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a00a76b0 msgr2=0x7fc8a0066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:08.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89effd700 1 --2- 172.21.2.104:0/2095860704 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a00a76b0 0x7fc8a0066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89effd700 1 -- 172.21.2.104:0/2095860704 >> v2:172.21.2.105:3300/0 conn(0x7fc8a00b0060 msgr2=0x7fc8a005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:08.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89effd700 1 --2- 172.21.2.104:0/2095860704 >> v2:172.21.2.105:3300/0 conn(0x7fc8a00b0060 0x7fc8a005f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89effd700 1 -- 172.21.2.104:0/2095860704 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a01478c0 con 0x7fc8a00b1970 2021-07-11T03:55:08.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89ffff700 1 --2- 172.21.2.104:0/2095860704 >> v2:172.21.2.105:3300/0 conn(0x7fc8a00b0060 0x7fc8a005f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:55:08.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89effd700 1 --2- 172.21.2.104:0/2095860704 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1970 0x7fc8a0066f60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fc888002f70 tx=0x7fc888006eb0).ready entity=mon.0 client_cookie=d5a285dfeb81485 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:08.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89e7fc700 1 -- 172.21.2.104:0/2095860704 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc88800d570 con 0x7fc8a00b1970 2021-07-11T03:55:08.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89e7fc700 1 -- 172.21.2.104:0/2095860704 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc888000e90 con 0x7fc8a00b1970 2021-07-11T03:55:08.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc89e7fc700 1 -- 172.21.2.104:0/2095860704 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc88800d900 con 0x7fc8a00b1970 2021-07-11T03:55:08.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 -- 172.21.2.104:0/2095860704 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1970 msgr2=0x7fc8a0066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 --2- 172.21.2.104:0/2095860704 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1970 0x7fc8a0066f60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7fc888002f70 tx=0x7fc888006eb0).stop 2021-07-11T03:55:08.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 -- 172.21.2.104:0/2095860704 shutdown_connections 2021-07-11T03:55:08.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 --2- 172.21.2.104:0/2095860704 >> v2:172.21.2.105:3300/0 conn(0x7fc8a00b0060 0x7fc8a005f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 --2- 172.21.2.104:0/2095860704 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1970 0x7fc8a0066f60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 --2- 172.21.2.104:0/2095860704 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a00a76b0 0x7fc8a0066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 -- 172.21.2.104:0/2095860704 >> 172.21.2.104:0/2095860704 conn(0x7fc8a00b53e0 msgr2=0x7fc8a00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:08.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 -- 172.21.2.104:0/2095860704 shutdown_connections 2021-07-11T03:55:08.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.737+0000 7fc8a5deb700 1 -- 172.21.2.104:0/2095860704 wait complete. 2021-07-11T03:55:08.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc8a5deb700 1 Processor -- start 2021-07-11T03:55:08.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc8a5deb700 1 -- start start 2021-07-11T03:55:08.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc8a5deb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc8a00a76b0 0x7fc8a014d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc8a5deb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b0060 0x7fc8a00ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc8a5deb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a00b1970 0x7fc8a0144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc8a5deb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a00ab540 con 0x7fc8a00b0060 2021-07-11T03:55:08.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc8a5deb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a00aa8c0 con 0x7fc8a00a76b0 2021-07-11T03:55:08.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc8a5deb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a00aab60 con 0x7fc8a00b1970 2021-07-11T03:55:08.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b0060 0x7fc8a00ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89ffff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a00b1970 0x7fc8a0144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b0060 0x7fc8a00ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43768/0 (socket says 172.21.2.104:43768) 2021-07-11T03:55:08.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89effd700 1 -- 172.21.2.104:0/4269621049 learned_addr learned my addr 172.21.2.104:0/4269621049 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:08.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89f7fe700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc8a00a76b0 0x7fc8a014d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89effd700 1 -- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a00b1970 msgr2=0x7fc8a0144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89effd700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a00b1970 0x7fc8a0144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89effd700 1 -- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc8a00a76b0 msgr2=0x7fc8a014d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89effd700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc8a00a76b0 0x7fc8a014d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89effd700 1 -- 172.21.2.104:0/4269621049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88800b1e0 con 0x7fc8a00b0060 2021-07-11T03:55:08.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89effd700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b0060 0x7fc8a00ac520 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fc888002f70 tx=0x7fc888015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:08.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89cff9700 1 -- 172.21.2.104:0/4269621049 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc888006b80 con 0x7fc8a00b0060 2021-07-11T03:55:08.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89cff9700 1 -- 172.21.2.104:0/4269621049 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc888011c30 con 0x7fc8a00b0060 2021-07-11T03:55:08.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc8a5deb700 1 -- 172.21.2.104:0/4269621049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a00aae00 con 0x7fc8a00b0060 2021-07-11T03:55:08.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89cff9700 1 -- 172.21.2.104:0/4269621049 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc88801d450 con 0x7fc8a00b0060 2021-07-11T03:55:08.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc8a5deb700 1 -- 172.21.2.104:0/4269621049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a00aa5d0 con 0x7fc8a00b0060 2021-07-11T03:55:08.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc8a5deb700 1 -- 172.21.2.104:0/4269621049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a0057350 con 0x7fc8a00b0060 2021-07-11T03:55:08.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89cff9700 1 -- 172.21.2.104:0/4269621049 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc88800f070 con 0x7fc8a00b0060 2021-07-11T03:55:08.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89cff9700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053210 0x7fc8840556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89cff9700 1 -- 172.21.2.104:0/4269621049 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc88806c8b0 con 0x7fc8a00b0060 2021-07-11T03:55:08.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89f7fe700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053210 0x7fc8840556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.741+0000 7fc89f7fe700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053210 0x7fc8840556d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fc89000b000 tx=0x7fc890006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:08.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.745+0000 7fc89cff9700 1 -- 172.21.2.104:0/4269621049 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc888043f20 con 0x7fc8a00b0060 2021-07-11T03:55:08.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.825+0000 7fc8a5deb700 1 -- 172.21.2.104:0/4269621049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8a00afe40 con 0x7fc8a00b0060 2021-07-11T03:55:08.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.825+0000 7fc89cff9700 1 -- 172.21.2.104:0/4269621049 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13181 (secure 0 0 0) 0x7fc888047540 con 0x7fc8a00b0060 2021-07-11T03:55:08.833 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:08.833 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":40,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:55:08.449749+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:55:08.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 -- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053210 msgr2=0x7fc8840556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053210 0x7fc8840556d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fc89000b000 tx=0x7fc890006df0).stop 2021-07-11T03:55:08.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 -- 172.21.2.104:0/4269621049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b0060 msgr2=0x7fc8a00ac520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b0060 0x7fc8a00ac520 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fc888002f70 tx=0x7fc888015000).stop 2021-07-11T03:55:08.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 -- 172.21.2.104:0/4269621049 shutdown_connections 2021-07-11T03:55:08.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a00b1970 0x7fc8a0144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053210 0x7fc8840556d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b0060 0x7fc8a00ac520 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 --2- 172.21.2.104:0/4269621049 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc8a00a76b0 0x7fc8a014d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 -- 172.21.2.104:0/4269621049 >> 172.21.2.104:0/4269621049 conn(0x7fc8a00b53e0 msgr2=0x7fc8a00602e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:08.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 -- 172.21.2.104:0/4269621049 shutdown_connections 2021-07-11T03:55:08.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.829+0000 7fc8a5deb700 1 -- 172.21.2.104:0/4269621049 wait complete. 2021-07-11T03:55:08.840 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T03:55:08.841 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:55:08.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4989fb7700 1 Processor -- start 2021-07-11T03:55:08.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4989fb7700 1 -- start start 2021-07-11T03:55:08.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4989fb7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f49840b12d0 0x7f49840affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4989fb7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b22a0 0x7f4984141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4989fb7700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49840a77d0 con 0x7f49840b1ec0 2021-07-11T03:55:08.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4989fb7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49840b1740 con 0x7f49840b12d0 2021-07-11T03:55:08.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4989fb7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49840b0650 con 0x7f49840b22a0 2021-07-11T03:55:08.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4982ffd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f49840b12d0 0x7f49840affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4983fff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b22a0 0x7f4984141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4982ffd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f49840b12d0 0x7f49840affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:51340/0 (socket says 172.21.2.104:51340) 2021-07-11T03:55:08.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4983fff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b22a0 0x7f4984141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:46942/0 (socket says 172.21.2.104:46942) 2021-07-11T03:55:08.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f49837fe700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f49840b1ec0 0x7f49840a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:39914/0 (socket says 172.21.2.104:39914) 2021-07-11T03:55:08.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4982ffd700 1 -- 172.21.2.104:0/361338098 learned_addr learned my addr 172.21.2.104:0/361338098 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:08.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4982ffd700 1 -- 172.21.2.104:0/361338098 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b22a0 msgr2=0x7f4984141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:08.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4982ffd700 1 --2- 172.21.2.104:0/361338098 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b22a0 0x7f4984141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4982ffd700 1 -- 172.21.2.104:0/361338098 >> v1:172.21.2.105:6789/0 conn(0x7f49840b1ec0 legacy=0x7f49840a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4982ffd700 1 -- 172.21.2.104:0/361338098 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49840b08f0 con 0x7f49840b12d0 2021-07-11T03:55:08.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.889+0000 7f4982ffd700 1 --2- 172.21.2.104:0/361338098 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f49840b12d0 0x7f49840affb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f496c002f70 tx=0x7f496c006eb0).ready entity=mon.1 client_cookie=576626980948502f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:08.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f49827fc700 1 -- 172.21.2.104:0/361338098 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f496c00d570 con 0x7f49840b12d0 2021-07-11T03:55:08.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f49827fc700 1 -- 172.21.2.104:0/361338098 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c000e90 con 0x7f49840b12d0 2021-07-11T03:55:08.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f49827fc700 1 -- 172.21.2.104:0/361338098 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f496c00d900 con 0x7f49840b12d0 2021-07-11T03:55:08.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 -- 172.21.2.104:0/361338098 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f49840b12d0 msgr2=0x7f49840affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 --2- 172.21.2.104:0/361338098 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f49840b12d0 0x7f49840affb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f496c002f70 tx=0x7f496c006eb0).stop 2021-07-11T03:55:08.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 -- 172.21.2.104:0/361338098 shutdown_connections 2021-07-11T03:55:08.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 --2- 172.21.2.104:0/361338098 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b22a0 0x7f4984141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 --2- 172.21.2.104:0/361338098 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f49840b12d0 0x7f49840affb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 -- 172.21.2.104:0/361338098 >> 172.21.2.104:0/361338098 conn(0x7f498405f6b0 msgr2=0x7f498405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:08.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 -- 172.21.2.104:0/361338098 shutdown_connections 2021-07-11T03:55:08.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 -- 172.21.2.104:0/361338098 wait complete. 2021-07-11T03:55:08.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 Processor -- start 2021-07-11T03:55:08.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 -- start start 2021-07-11T03:55:08.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b12d0 0x7f4984148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f49840b1ec0 0x7f49840ae130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f49840b22a0 0x7f4984149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49840b5ad0 con 0x7f49840b1ec0 2021-07-11T03:55:08.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49840ae9b0 con 0x7f49840b22a0 2021-07-11T03:55:08.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49840b5830 con 0x7f49840b12d0 2021-07-11T03:55:08.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4982ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f49840b1ec0 0x7f49840ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4982ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f49840b1ec0 0x7f49840ae130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43786/0 (socket says 172.21.2.104:43786) 2021-07-11T03:55:08.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4982ffd700 1 -- 172.21.2.104:0/820673724 learned_addr learned my addr 172.21.2.104:0/820673724 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:08.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f49837fe700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b12d0 0x7f4984148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4983fff700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f49840b22a0 0x7f4984149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4982ffd700 1 -- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b12d0 msgr2=0x7f4984148280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4982ffd700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b12d0 0x7f4984148280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4982ffd700 1 -- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f49840b22a0 msgr2=0x7f4984149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4982ffd700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f49840b22a0 0x7f4984149cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4982ffd700 1 -- 172.21.2.104:0/820673724 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c00b1e0 con 0x7f49840b1ec0 2021-07-11T03:55:08.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f49837fe700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b12d0 0x7f4984148280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:55:08.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4983fff700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f49840b22a0 0x7f4984149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:55:08.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4982ffd700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f49840b1ec0 0x7f49840ae130 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f496c002f70 tx=0x7f496c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:08.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4980ff9700 1 -- 172.21.2.104:0/820673724 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f496c011860 con 0x7f49840b1ec0 2021-07-11T03:55:08.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4980ff9700 1 -- 172.21.2.104:0/820673724 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c0119c0 con 0x7f49840b1ec0 2021-07-11T03:55:08.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 -- 172.21.2.104:0/820673724 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49840af530 con 0x7f49840b1ec0 2021-07-11T03:55:08.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4980ff9700 1 -- 172.21.2.104:0/820673724 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f496c011d20 con 0x7f49840b1ec0 2021-07-11T03:55:08.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.893+0000 7f4989fb7700 1 -- 172.21.2.104:0/820673724 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49840aed50 con 0x7f49840b1ec0 2021-07-11T03:55:08.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.897+0000 7f4989fb7700 1 -- 172.21.2.104:0/820673724 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4984057350 con 0x7f49840b1ec0 2021-07-11T03:55:08.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.897+0000 7f4980ff9700 1 -- 172.21.2.104:0/820673724 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f496c00f070 con 0x7f49840b1ec0 2021-07-11T03:55:08.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.897+0000 7f4980ff9700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4968053350 0x7f4968055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:08.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.897+0000 7f4980ff9700 1 -- 172.21.2.104:0/820673724 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f496c06ca70 con 0x7f49840b1ec0 2021-07-11T03:55:08.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.897+0000 7f49837fe700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4968053350 0x7f4968055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:08.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.897+0000 7f49837fe700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4968053350 0x7f4968055810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f4974008000 tx=0x7f4974009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:08.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.897+0000 7f4980ff9700 1 -- 172.21.2.104:0/820673724 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f496c043fd0 con 0x7f49840b1ec0 2021-07-11T03:55:08.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.977+0000 7f4989fb7700 1 -- 172.21.2.104:0/820673724 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f49840aeb70 con 0x7f49840b1ec0 2021-07-11T03:55:08.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.977+0000 7f4980ff9700 1 -- 172.21.2.104:0/820673724 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v40) v1 ==== 110+0+10 (secure 0 0 0) 0x7f496c0475f0 con 0x7f49840b1ec0 2021-07-11T03:55:08.985 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:55:08.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 -- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4968053350 msgr2=0x7f4968055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4968053350 0x7f4968055810 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f4974008000 tx=0x7f4974009040).stop 2021-07-11T03:55:08.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 -- 172.21.2.104:0/820673724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f49840b1ec0 msgr2=0x7f49840ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:08.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f49840b1ec0 0x7f49840ae130 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f496c002f70 tx=0x7f496c015000).stop 2021-07-11T03:55:08.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 -- 172.21.2.104:0/820673724 shutdown_connections 2021-07-11T03:55:08.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f49840b22a0 0x7f4984149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4968053350 0x7f4968055810 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f49840b1ec0 0x7f49840ae130 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 --2- 172.21.2.104:0/820673724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f49840b12d0 0x7f4984148280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:08.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 -- 172.21.2.104:0/820673724 >> 172.21.2.104:0/820673724 conn(0x7f498405f6b0 msgr2=0x7f49840a9420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:08.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 -- 172.21.2.104:0/820673724 shutdown_connections 2021-07-11T03:55:08.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:08.981+0000 7f4989fb7700 1 -- 172.21.2.104:0/820673724 wait complete. 2021-07-11T03:55:08.993 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:55:09.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.037+0000 7f8079e68700 1 Processor -- start 2021-07-11T03:55:09.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.037+0000 7f8079e68700 1 -- start start 2021-07-11T03:55:09.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f80740b12d0 0x7f80740affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80740b22a0 0x7f8074141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80740a77d0 con 0x7f80740b1ec0 2021-07-11T03:55:09.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80740b1740 con 0x7f80740b12d0 2021-07-11T03:55:09.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80740b0650 con 0x7f80740b22a0 2021-07-11T03:55:09.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8072ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f80740b12d0 0x7f80740affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f80737fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f80740b1ec0 0x7f80740a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40558/0 (socket says 172.21.2.104:40558) 2021-07-11T03:55:09.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f80737fe700 1 -- 172.21.2.104:0/1815366886 learned_addr learned my addr 172.21.2.104:0/1815366886 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:09.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8072ffd700 1 -- 172.21.2.104:0/1815366886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80740b22a0 msgr2=0x7f8074141250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:09.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8072ffd700 1 --2- 172.21.2.104:0/1815366886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80740b22a0 0x7f8074141250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:09.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8072ffd700 1 -- 172.21.2.104:0/1815366886 >> v1:172.21.2.104:6789/0 conn(0x7f80740b1ec0 legacy=0x7f80740a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8072ffd700 1 -- 172.21.2.104:0/1815366886 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80740b08f0 con 0x7f80740b12d0 2021-07-11T03:55:09.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8072ffd700 1 --2- 172.21.2.104:0/1815366886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80740b12d0 0x7f80740affb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f8068002f70 tx=0x7f8068006df0).ready entity=mon.0 client_cookie=a155677cd6a7522a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:09.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f80727fc700 1 -- 172.21.2.104:0/1815366886 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f806800d570 con 0x7f80740b12d0 2021-07-11T03:55:09.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f80727fc700 1 -- 172.21.2.104:0/1815366886 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8068000e90 con 0x7f80740b12d0 2021-07-11T03:55:09.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f80727fc700 1 -- 172.21.2.104:0/1815366886 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f806800d900 con 0x7f80740b12d0 2021-07-11T03:55:09.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- 172.21.2.104:0/1815366886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80740b12d0 msgr2=0x7f80740affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 --2- 172.21.2.104:0/1815366886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80740b12d0 0x7f80740affb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f8068002f70 tx=0x7f8068006df0).stop 2021-07-11T03:55:09.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- 172.21.2.104:0/1815366886 shutdown_connections 2021-07-11T03:55:09.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 --2- 172.21.2.104:0/1815366886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80740b12d0 0x7f80740affb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 --2- 172.21.2.104:0/1815366886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80740b22a0 0x7f8074141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:09.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- 172.21.2.104:0/1815366886 >> 172.21.2.104:0/1815366886 conn(0x7f807405f6b0 msgr2=0x7f807405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:09.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- 172.21.2.104:0/1815366886 shutdown_connections 2021-07-11T03:55:09.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- 172.21.2.104:0/1815366886 wait complete. 2021-07-11T03:55:09.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 Processor -- start 2021-07-11T03:55:09.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- start start 2021-07-11T03:55:09.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80740b12d0 0x7f8074148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f80740b1ec0 0x7f807414afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80740b22a0 0x7f807413c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80740ab4f0 con 0x7f80740b22a0 2021-07-11T03:55:09.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80740aa550 con 0x7f80740b1ec0 2021-07-11T03:55:09.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807414a300 con 0x7f80740b12d0 2021-07-11T03:55:09.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8073fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80740b22a0 0x7f807413c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8073fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80740b22a0 0x7f807413c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43800/0 (socket says 172.21.2.104:43800) 2021-07-11T03:55:09.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8073fff700 1 -- 172.21.2.104:0/1141190403 learned_addr learned my addr 172.21.2.104:0/1141190403 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:09.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8073fff700 1 -- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80740b12d0 msgr2=0x7f8074148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8073fff700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80740b12d0 0x7f8074148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:09.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8073fff700 1 -- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f80740b1ec0 msgr2=0x7f807414afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8073fff700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f80740b1ec0 0x7f807414afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:09.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8073fff700 1 -- 172.21.2.104:0/1141190403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f806800b180 con 0x7f80740b22a0 2021-07-11T03:55:09.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8073fff700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80740b22a0 0x7f807413c7b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f806c003270 tx=0x7f806c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:09.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8070ff9700 1 -- 172.21.2.104:0/1141190403 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f806c00d4a0 con 0x7f80740b22a0 2021-07-11T03:55:09.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8070ff9700 1 -- 172.21.2.104:0/1141190403 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806c00d600 con 0x7f80740b22a0 2021-07-11T03:55:09.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- 172.21.2.104:0/1141190403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807414a5a0 con 0x7f80740b22a0 2021-07-11T03:55:09.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8070ff9700 1 -- 172.21.2.104:0/1141190403 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f806c00d9a0 con 0x7f80740b22a0 2021-07-11T03:55:09.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.041+0000 7f8079e68700 1 -- 172.21.2.104:0/1141190403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8074064890 con 0x7f80740b22a0 2021-07-11T03:55:09.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.045+0000 7f8079e68700 1 -- 172.21.2.104:0/1141190403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80740aa6c0 con 0x7f80740b22a0 2021-07-11T03:55:09.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.045+0000 7f8070ff9700 1 -- 172.21.2.104:0/1141190403 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f806c018030 con 0x7f80740b22a0 2021-07-11T03:55:09.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.045+0000 7f8070ff9700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80540531f0 0x7f80540556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.045+0000 7f8070ff9700 1 -- 172.21.2.104:0/1141190403 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f806c06aa00 con 0x7f80740b22a0 2021-07-11T03:55:09.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.045+0000 7f80737fe700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80540531f0 0x7f80540556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.045+0000 7f80737fe700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80540531f0 0x7f80540556b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f80740b2f50 tx=0x7f8064007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:09.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.049+0000 7f8070ff9700 1 -- 172.21.2.104:0/1141190403 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f806c042120 con 0x7f80740b22a0 2021-07-11T03:55:09.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 -- 172.21.2.104:0/1141190403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8074057350 con 0x7f80740b22a0 2021-07-11T03:55:09.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8070ff9700 1 -- 172.21.2.104:0/1141190403 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f806c045740 con 0x7f80740b22a0 2021-07-11T03:55:09.136 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:55:09.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 -- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80540531f0 msgr2=0x7f80540556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80540531f0 0x7f80540556b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f80740b2f50 tx=0x7f8064007800).stop 2021-07-11T03:55:09.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 -- 172.21.2.104:0/1141190403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80740b22a0 msgr2=0x7f807413c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80740b22a0 0x7f807413c7b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f806c003270 tx=0x7f806c0090f0).stop 2021-07-11T03:55:09.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 -- 172.21.2.104:0/1141190403 shutdown_connections 2021-07-11T03:55:09.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f80540531f0 0x7f80540556b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80740b22a0 0x7f807413c7b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f80740b1ec0 0x7f807414afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:09.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 --2- 172.21.2.104:0/1141190403 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80740b12d0 0x7f8074148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:09.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 -- 172.21.2.104:0/1141190403 >> 172.21.2.104:0/1141190403 conn(0x7f807405f6b0 msgr2=0x7f80740af720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:09.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 -- 172.21.2.104:0/1141190403 shutdown_connections 2021-07-11T03:55:09.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.129+0000 7f8079e68700 1 -- 172.21.2.104:0/1141190403 wait complete. 2021-07-11T03:55:09.143 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T03:55:09.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35dbe5f700 1 Processor -- start 2021-07-11T03:55:09.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35dbe5f700 1 -- start start 2021-07-11T03:55:09.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35dbe5f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35d41483a0 0x7f35d40b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35dbe5f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f35d40a70c0 0x7f35d4144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35dbe5f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d40b0350 con 0x7f35d40b1db0 2021-07-11T03:55:09.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35dbe5f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40a90c0 con 0x7f35d40a70c0 2021-07-11T03:55:09.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35dbe5f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40b05f0 con 0x7f35d41483a0 2021-07-11T03:55:09.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35da3fc700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f35d40a70c0 0x7f35d4144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35d93fa700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35d41483a0 0x7f35d40b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35d9bfb700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f35d40b1db0 0x7f35d40b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:39942/0 (socket says 172.21.2.104:39942) 2021-07-11T03:55:09.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35d9bfb700 1 -- 172.21.2.104:0/2675239471 learned_addr learned my addr 172.21.2.104:0/2675239471 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:09.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35da3fc700 1 -- 172.21.2.104:0/2675239471 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35d41483a0 msgr2=0x7f35d40b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:09.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35da3fc700 1 --2- 172.21.2.104:0/2675239471 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35d41483a0 0x7f35d40b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35da3fc700 1 -- 172.21.2.104:0/2675239471 >> v1:172.21.2.105:6789/0 conn(0x7f35d40b1db0 legacy=0x7f35d40b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35da3fc700 1 -- 172.21.2.104:0/2675239471 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40b0890 con 0x7f35d40a70c0 2021-07-11T03:55:09.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35da3fc700 1 --2- 172.21.2.104:0/2675239471 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35d40a70c0 0x7f35d4144720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f35cc0032f0 tx=0x7f35cc0091d0).ready entity=mon.1 client_cookie=bba832d950a6cd0d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:09.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35d8bf9700 1 -- 172.21.2.104:0/2675239471 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35cc00b6c0 con 0x7f35d40a70c0 2021-07-11T03:55:09.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35d8bf9700 1 -- 172.21.2.104:0/2675239471 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc00b820 con 0x7f35d40a70c0 2021-07-11T03:55:09.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2675239471 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35d40a70c0 msgr2=0x7f35d4144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.189+0000 7f35dbe5f700 1 --2- 172.21.2.104:0/2675239471 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35d40a70c0 0x7f35d4144720 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f35cc0032f0 tx=0x7f35cc0091d0).stop 2021-07-11T03:55:09.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2675239471 shutdown_connections 2021-07-11T03:55:09.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 --2- 172.21.2.104:0/2675239471 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35d40a70c0 0x7f35d4144720 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 --2- 172.21.2.104:0/2675239471 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35d41483a0 0x7f35d40b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2675239471 >> 172.21.2.104:0/2675239471 conn(0x7f35d405f6b0 msgr2=0x7f35d405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:09.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2675239471 shutdown_connections 2021-07-11T03:55:09.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2675239471 wait complete. 2021-07-11T03:55:09.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 Processor -- start 2021-07-11T03:55:09.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 -- start start 2021-07-11T03:55:09.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35d40a70c0 0x7f35d40af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35d40b1db0 0x7f35d415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35d41483a0 0x7f35d40ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4149850 con 0x7f35d40b1db0 2021-07-11T03:55:09.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4149af0 con 0x7f35d41483a0 2021-07-11T03:55:09.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4159e20 con 0x7f35d40a70c0 2021-07-11T03:55:09.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35d93fa700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35d40b1db0 0x7f35d415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35d93fa700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35d40b1db0 0x7f35d415a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43812/0 (socket says 172.21.2.104:43812) 2021-07-11T03:55:09.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35d93fa700 1 -- 172.21.2.104:0/2982817588 learned_addr learned my addr 172.21.2.104:0/2982817588 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:09.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35d9bfb700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35d40a70c0 0x7f35d40af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35da3fc700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35d41483a0 0x7f35d40ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35d93fa700 1 -- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35d40a70c0 msgr2=0x7f35d40af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35d93fa700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35d40a70c0 0x7f35d40af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35d93fa700 1 -- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35d41483a0 msgr2=0x7f35d40ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35d93fa700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35d41483a0 0x7f35d40ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35d93fa700 1 -- 172.21.2.104:0/2982817588 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc009470 con 0x7f35d40b1db0 2021-07-11T03:55:09.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35d93fa700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35d40b1db0 0x7f35d415a560 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f35d000c000 tx=0x7f35d0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:09.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35c6ffd700 1 -- 172.21.2.104:0/2982817588 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d0010780 con 0x7f35d40b1db0 2021-07-11T03:55:09.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35c6ffd700 1 -- 172.21.2.104:0/2982817588 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d00108e0 con 0x7f35d40b1db0 2021-07-11T03:55:09.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2982817588 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d415a0c0 con 0x7f35d40b1db0 2021-07-11T03:55:09.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35c6ffd700 1 -- 172.21.2.104:0/2982817588 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d0010c40 con 0x7f35d40b1db0 2021-07-11T03:55:09.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2982817588 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d40aaf10 con 0x7f35d40b1db0 2021-07-11T03:55:09.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.193+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2982817588 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d41491c0 con 0x7f35d40b1db0 2021-07-11T03:55:09.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.197+0000 7f35c6ffd700 1 -- 172.21.2.104:0/2982817588 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f35d001c460 con 0x7f35d40b1db0 2021-07-11T03:55:09.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.197+0000 7f35c6ffd700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35b4053180 0x7f35b4055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.197+0000 7f35c6ffd700 1 -- 172.21.2.104:0/2982817588 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f35d006b5c0 con 0x7f35d40b1db0 2021-07-11T03:55:09.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.197+0000 7f35d9bfb700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35b4053180 0x7f35b4055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.197+0000 7f35d9bfb700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35b4053180 0x7f35b4055640 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f35c800b000 tx=0x7f35c8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:09.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.197+0000 7f35c6ffd700 1 -- 172.21.2.104:0/2982817588 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f35d0042bd0 con 0x7f35d40b1db0 2021-07-11T03:55:09.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2982817588 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f35d415a250 con 0x7f35d40b1db0 2021-07-11T03:55:09.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35c6ffd700 1 -- 172.21.2.104:0/2982817588 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) v1 ==== 84+0+143 (secure 0 0 0) 0x7f35d00461f0 con 0x7f35d40b1db0 2021-07-11T03:55:09.287 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:09.287 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T03:55:09.288 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:09.288 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T03:55:09.288 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T03:55:09.288 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:09.289 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:55:09.289 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T03:55:09.289 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T03:55:09.289 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:09.289 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T03:55:09.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35b4053180 msgr2=0x7f35b4055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35b4053180 0x7f35b4055640 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f35c800b000 tx=0x7f35c8009040).stop 2021-07-11T03:55:09.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2982817588 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35d40b1db0 msgr2=0x7f35d415a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35d40b1db0 0x7f35d415a560 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f35d000c000 tx=0x7f35d0006df0).stop 2021-07-11T03:55:09.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2982817588 shutdown_connections 2021-07-11T03:55:09.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35d41483a0 0x7f35d40ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35b4053180 0x7f35b4055640 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35d40b1db0 0x7f35d415a560 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 --2- 172.21.2.104:0/2982817588 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35d40a70c0 0x7f35d40af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2982817588 >> 172.21.2.104:0/2982817588 conn(0x7f35d405f6b0 msgr2=0x7f35d41429e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:09.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2982817588 shutdown_connections 2021-07-11T03:55:09.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.281+0000 7f35dbe5f700 1 -- 172.21.2.104:0/2982817588 wait complete. 2021-07-11T03:55:09.294 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T03:55:09.294 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T03:55:09.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 Processor -- start 2021-07-11T03:55:09.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 -- start start 2021-07-11T03:55:09.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f20600b12d0 0x7f20600affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20600a77d0 con 0x7f20600b22a0 2021-07-11T03:55:09.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20600b0650 con 0x7f20600b1ec0 2021-07-11T03:55:09.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20600b1740 con 0x7f20600b12d0 2021-07-11T03:55:09.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f206528a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f20600b12d0 0x7f20600affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f206628c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f20600b22a0 0x7f2060141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40586/0 (socket says 172.21.2.104:40586) 2021-07-11T03:55:09.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f206628c700 1 -- 172.21.2.104:0/3652195320 learned_addr learned my addr 172.21.2.104:0/3652195320 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:09.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f206528a700 1 -- 172.21.2.104:0/3652195320 >> v1:172.21.2.105:6789/0 conn(0x7f20600b1ec0 legacy=0x7f20600a70c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:09.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f206528a700 1 -- 172.21.2.104:0/3652195320 >> v1:172.21.2.104:6789/0 conn(0x7f20600b22a0 legacy=0x7f2060141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f206528a700 1 -- 172.21.2.104:0/3652195320 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20600b08f0 con 0x7f20600b12d0 2021-07-11T03:55:09.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f206528a700 1 --2- 172.21.2.104:0/3652195320 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20600b12d0 0x7f20600affb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f20600b2f50 tx=0x7f205c006df0).ready entity=mon.0 client_cookie=b16e76af50ace06c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:09.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2064a89700 1 -- 172.21.2.104:0/3652195320 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f205c00d5c0 con 0x7f20600b12d0 2021-07-11T03:55:09.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2064a89700 1 -- 172.21.2.104:0/3652195320 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205c000e90 con 0x7f20600b12d0 2021-07-11T03:55:09.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2064a89700 1 -- 172.21.2.104:0/3652195320 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f205c00d910 con 0x7f20600b12d0 2021-07-11T03:55:09.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 -- 172.21.2.104:0/3652195320 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20600b12d0 msgr2=0x7f20600affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 --2- 172.21.2.104:0/3652195320 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20600b12d0 0x7f20600affb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f20600b2f50 tx=0x7f205c006df0).stop 2021-07-11T03:55:09.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 -- 172.21.2.104:0/3652195320 shutdown_connections 2021-07-11T03:55:09.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 --2- 172.21.2.104:0/3652195320 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20600b12d0 0x7f20600affb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:09.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 -- 172.21.2.104:0/3652195320 >> 172.21.2.104:0/3652195320 conn(0x7f206005f6b0 msgr2=0x7f206005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:09.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 -- 172.21.2.104:0/3652195320 shutdown_connections 2021-07-11T03:55:09.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.341+0000 7f2067cef700 1 -- 172.21.2.104:0/3652195320 wait complete. 2021-07-11T03:55:09.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2067cef700 1 Processor -- start 2021-07-11T03:55:09.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2067cef700 1 -- start start 2021-07-11T03:55:09.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2067cef700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20600b12d0 0x7f2060148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2067cef700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20600b1ec0 0x7f206014b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2067cef700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20600b22a0 0x7f206014a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2067cef700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2060148450 con 0x7f20600b1ec0 2021-07-11T03:55:09.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2067cef700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20600aa180 con 0x7f20600b22a0 2021-07-11T03:55:09.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2067cef700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20600aa420 con 0x7f20600b12d0 2021-07-11T03:55:09.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f206528a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20600b1ec0 0x7f206014b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f206528a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20600b1ec0 0x7f206014b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43828/0 (socket says 172.21.2.104:43828) 2021-07-11T03:55:09.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f206528a700 1 -- 172.21.2.104:0/3098711328 learned_addr learned my addr 172.21.2.104:0/3098711328 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:09.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f206528a700 1 -- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20600b12d0 msgr2=0x7f2060148e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f206528a700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20600b12d0 0x7f2060148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:09.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f206528a700 1 -- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20600b22a0 msgr2=0x7f206014a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:09.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f206528a700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20600b22a0 0x7f206014a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:09.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f206528a700 1 -- 172.21.2.104:0/3098711328 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f205c00b180 con 0x7f20600b1ec0 2021-07-11T03:55:09.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f206528a700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20600b1ec0 0x7f206014b740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f20600b2f50 tx=0x7f205c012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:09.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f204effd700 1 -- 172.21.2.104:0/3098711328 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f205c002f50 con 0x7f20600b1ec0 2021-07-11T03:55:09.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f204effd700 1 -- 172.21.2.104:0/3098711328 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205c000e90 con 0x7f20600b1ec0 2021-07-11T03:55:09.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2067cef700 1 -- 172.21.2.104:0/3098711328 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20600aa6c0 con 0x7f20600b1ec0 2021-07-11T03:55:09.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f204effd700 1 -- 172.21.2.104:0/3098711328 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f205c01d400 con 0x7f20600b1ec0 2021-07-11T03:55:09.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2067cef700 1 -- 172.21.2.104:0/3098711328 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206014b2a0 con 0x7f20600b1ec0 2021-07-11T03:55:09.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2067cef700 1 -- 172.21.2.104:0/3098711328 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206014ab90 con 0x7f20600b1ec0 2021-07-11T03:55:09.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f204effd700 1 -- 172.21.2.104:0/3098711328 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f205c01b070 con 0x7f20600b1ec0 2021-07-11T03:55:09.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f204effd700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20400532b0 0x7f2040055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:09.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f204effd700 1 -- 172.21.2.104:0/3098711328 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f205c06b910 con 0x7f20600b1ec0 2021-07-11T03:55:09.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.345+0000 7f2065a8b700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20400532b0 0x7f2040055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:09.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.349+0000 7f2065a8b700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20400532b0 0x7f2040055770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f2050002fb0 tx=0x7f20500069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:09.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.349+0000 7f204effd700 1 -- 172.21.2.104:0/3098711328 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f205c042f80 con 0x7f20600b1ec0 2021-07-11T03:55:09.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:09.433+0000 7f2067cef700 1 -- 172.21.2.104:0/3098711328 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f2060057350 con 0x7f20600b1ec0 2021-07-11T03:55:10.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.405+0000 7f204effd700 1 -- 172.21.2.104:0/3098711328 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v41) v1 ==== 133+0+0 (secure 0 0 0) 0x7f205c022070 con 0x7f20600b1ec0 2021-07-11T03:55:10.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 -- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20400532b0 msgr2=0x7f2040055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:10.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20400532b0 0x7f2040055770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f2050002fb0 tx=0x7f20500069b0).stop 2021-07-11T03:55:10.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 -- 172.21.2.104:0/3098711328 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20600b1ec0 msgr2=0x7f206014b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:10.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20600b1ec0 0x7f206014b740 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f20600b2f50 tx=0x7f205c012000).stop 2021-07-11T03:55:10.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 -- 172.21.2.104:0/3098711328 shutdown_connections 2021-07-11T03:55:10.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20600b22a0 0x7f206014a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:10.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f20400532b0 0x7f2040055770 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:10.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20600b1ec0 0x7f206014b740 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:10.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 --2- 172.21.2.104:0/3098711328 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20600b12d0 0x7f2060148e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:10.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 -- 172.21.2.104:0/3098711328 >> 172.21.2.104:0/3098711328 conn(0x7f206005f6b0 msgr2=0x7f20600b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:10.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 -- 172.21.2.104:0/3098711328 shutdown_connections 2021-07-11T03:55:10.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.409+0000 7f2067cef700 1 -- 172.21.2.104:0/3098711328 wait complete. 2021-07-11T03:55:10.418 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T03:55:10.424 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T03:55:10.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.469+0000 7f5d7468f700 1 Processor -- start 2021-07-11T03:55:10.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.469+0000 7f5d7468f700 1 -- start start 2021-07-11T03:55:10.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.469+0000 7f5d7468f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5d6c0b05b0 0x7f5d6c0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:10.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.469+0000 7f5d7468f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c0b3710 0x7f5d6c13d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:10.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.469+0000 7f5d7468f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6c143920 con 0x7f5d6c13da40 2021-07-11T03:55:10.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.469+0000 7f5d7468f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c143bc0 con 0x7f5d6c0b05b0 2021-07-11T03:55:10.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.469+0000 7f5d7468f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c143e60 con 0x7f5d6c0b3710 2021-07-11T03:55:10.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.469+0000 7f5d72c2c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f5d6c13da40 0x7f5d6c0b10b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40604/0 (socket says 172.21.2.104:40604) 2021-07-11T03:55:10.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.469+0000 7f5d72c2c700 1 -- 172.21.2.104:0/2154944097 learned_addr learned my addr 172.21.2.104:0/2154944097 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:10.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.469+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744199210 0 0) 0x7f5d6c143920 con 0x7f5d6c13da40 2021-07-11T03:55:10.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d54003260 con 0x7f5d6c13da40 2021-07-11T03:55:10.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7242b700 1 --2- 172.21.2.104:0/2154944097 >> v2:172.21.2.105:3300/0 conn(0x7f5d6c0b05b0 0x7f5d6c0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:10.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71c2a700 1 --2- 172.21.2.104:0/2154944097 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c0b3710 0x7f5d6c13d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:10.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1220626075 0 0) 0x7f5d54003260 con 0x7f5d6c13da40 2021-07-11T03:55:10.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c0b3710 msgr2=0x7f5d6c13d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:10.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 --2- 172.21.2.104:0/2154944097 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c0b3710 0x7f5d6c13d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:10.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 >> v2:172.21.2.105:3300/0 conn(0x7f5d6c0b05b0 msgr2=0x7f5d6c0b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:10.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 --2- 172.21.2.104:0/2154944097 >> v2:172.21.2.105:3300/0 conn(0x7f5d6c0b05b0 0x7f5d6c0b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:10.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c144100 con 0x7f5d6c13da40 2021-07-11T03:55:10.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f5d640019e0 con 0x7f5d6c13da40 2021-07-11T03:55:10.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7242b700 1 --2- 172.21.2.104:0/2154944097 >> v2:172.21.2.105:3300/0 conn(0x7f5d6c0b05b0 0x7f5d6c0b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:55:10.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 >> v1:172.21.2.104:6789/0 conn(0x7f5d6c13da40 legacy=0x7f5d6c0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:10.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 --2- 172.21.2.104:0/2154944097 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d540037d0 0x7f5d54003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:10.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 --2- 172.21.2.104:0/2154944097 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c0b3710 0x7f5d54004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:10.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 --2- 172.21.2.104:0/2154944097 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d540049f0 0x7f5d5400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:10.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c143bc0 con 0x7f5d540049f0 2021-07-11T03:55:10.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c143e60 con 0x7f5d540037d0 2021-07-11T03:55:10.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540050a0 con 0x7f5d6c0b3710 2021-07-11T03:55:10.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 --2- 172.21.2.104:0/2154944097 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d540049f0 0x7f5d5400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:10.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7242b700 1 --2- 172.21.2.104:0/2154944097 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d540037d0 0x7f5d54003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:10.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 -- 172.21.2.104:0/2154944097 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c0b3710 msgr2=0x7f5d54004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:10.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 --2- 172.21.2.104:0/2154944097 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c0b3710 0x7f5d54004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:10.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 -- 172.21.2.104:0/2154944097 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d540037d0 msgr2=0x7f5d54003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:10.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 --2- 172.21.2.104:0/2154944097 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d540037d0 0x7f5d54003bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:10.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 -- 172.21.2.104:0/2154944097 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d54003260 con 0x7f5d540049f0 2021-07-11T03:55:10.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 --2- 172.21.2.104:0/2154944097 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d540049f0 0x7f5d5400b750 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f5d640016f0 tx=0x7f5d6c143ab0).ready entity=mon.0 client_cookie=eef977c3bc697046 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:10.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d6400cc40 con 0x7f5d540049f0 2021-07-11T03:55:10.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6400cda0 con 0x7f5d540049f0 2021-07-11T03:55:10.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d71429700 1 -- 172.21.2.104:0/2154944097 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d64005b80 con 0x7f5d540049f0 2021-07-11T03:55:10.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 -- 172.21.2.104:0/2154944097 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d540049f0 msgr2=0x7f5d5400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:10.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 --2- 172.21.2.104:0/2154944097 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d540049f0 0x7f5d5400b750 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f5d640016f0 tx=0x7f5d6c143ab0).stop 2021-07-11T03:55:10.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 -- 172.21.2.104:0/2154944097 reap_dead start 2021-07-11T03:55:10.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 -- 172.21.2.104:0/2154944097 shutdown_connections 2021-07-11T03:55:10.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 -- 172.21.2.104:0/2154944097 >> 172.21.2.104:0/2154944097 conn(0x7f5d6c0607c0 msgr2=0x7f5d6c060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:10.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 -- 172.21.2.104:0/2154944097 shutdown_connections 2021-07-11T03:55:10.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 -- 172.21.2.104:0/2154944097 wait complete. 2021-07-11T03:55:10.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 Processor -- start 2021-07-11T03:55:10.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 -- start start 2021-07-11T03:55:10.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c139410 0x7f5d6c14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:10.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d6c1409e0 0x7f5d6c14e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:10.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d6c13f6c0 0x7f5d6c13deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:10.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c14dc10 con 0x7f5d6c13f6c0 2021-07-11T03:55:10.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c0667a0 con 0x7f5d6c1409e0 2021-07-11T03:55:10.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c1397f0 con 0x7f5d6c139410 2021-07-11T03:55:10.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d6c13f6c0 0x7f5d6c13deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:10.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d6c13f6c0 0x7f5d6c13deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43848/0 (socket says 172.21.2.104:43848) 2021-07-11T03:55:10.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 -- 172.21.2.104:0/3085359939 learned_addr learned my addr 172.21.2.104:0/3085359939 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:10.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7242b700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c139410 0x7f5d6c14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:10.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 -- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c139410 msgr2=0x7f5d6c14e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:10.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c139410 0x7f5d6c14e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:10.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 -- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d6c1409e0 msgr2=0x7f5d6c14e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:10.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d6c1409e0 0x7f5d6c14e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:10.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 -- 172.21.2.104:0/3085359939 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c13fbd0 con 0x7f5d6c13f6c0 2021-07-11T03:55:10.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d72c2c700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d6c13f6c0 0x7f5d6c13deb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f5d6c0b6620 tx=0x7f5d64013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:10.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d5b7fe700 1 -- 172.21.2.104:0/3085359939 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d64001f20 con 0x7f5d6c13f6c0 2021-07-11T03:55:10.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d5b7fe700 1 -- 172.21.2.104:0/3085359939 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d64018070 con 0x7f5d6c13f6c0 2021-07-11T03:55:10.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d5b7fe700 1 -- 172.21.2.104:0/3085359939 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5d640074d0 con 0x7f5d6c13f6c0 2021-07-11T03:55:10.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 -- 172.21.2.104:0/3085359939 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6c13fe70 con 0x7f5d6c13f6c0 2021-07-11T03:55:10.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.473+0000 7f5d7468f700 1 -- 172.21.2.104:0/3085359939 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6c139c00 con 0x7f5d6c13f6c0 2021-07-11T03:55:10.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.477+0000 7f5d7468f700 1 -- 172.21.2.104:0/3085359939 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6c057350 con 0x7f5d6c13f6c0 2021-07-11T03:55:10.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.477+0000 7f5d5b7fe700 1 -- 172.21.2.104:0/3085359939 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5d64010070 con 0x7f5d6c13f6c0 2021-07-11T03:55:10.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.477+0000 7f5d5b7fe700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d4c0531f0 0x7f5d4c0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:10.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.477+0000 7f5d5b7fe700 1 -- 172.21.2.104:0/3085359939 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f5d6406b920 con 0x7f5d6c13f6c0 2021-07-11T03:55:10.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.477+0000 7f5d7242b700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d4c0531f0 0x7f5d4c0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:10.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.477+0000 7f5d7242b700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d4c0531f0 0x7f5d4c0556b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f5d68009000 tx=0x7f5d6800b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:10.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.477+0000 7f5d5b7fe700 1 -- 172.21.2.104:0/3085359939 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5d64042e30 con 0x7f5d6c13f6c0 2021-07-11T03:55:10.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:10.561+0000 7f5d7468f700 1 -- 172.21.2.104:0/3085359939 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f5d6c044650 con 0x7f5d6c13f6c0 2021-07-11T03:55:11.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.405+0000 7f5d5b7fe700 1 -- 172.21.2.104:0/3085359939 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v42) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5d64046450 con 0x7f5d6c13f6c0 2021-07-11T03:55:11.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.405+0000 7f5d7468f700 1 -- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d4c0531f0 msgr2=0x7f5d4c0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.405+0000 7f5d7468f700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d4c0531f0 0x7f5d4c0556b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f5d68009000 tx=0x7f5d6800b040).stop 2021-07-11T03:55:11.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.405+0000 7f5d7468f700 1 -- 172.21.2.104:0/3085359939 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d6c13f6c0 msgr2=0x7f5d6c13deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.405+0000 7f5d7468f700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d6c13f6c0 0x7f5d6c13deb0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f5d6c0b6620 tx=0x7f5d64013000).stop 2021-07-11T03:55:11.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.405+0000 7f5d7468f700 1 -- 172.21.2.104:0/3085359939 shutdown_connections 2021-07-11T03:55:11.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.405+0000 7f5d7468f700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5d4c0531f0 0x7f5d4c0556b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.405+0000 7f5d7468f700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5d6c13f6c0 0x7f5d6c13deb0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.405+0000 7f5d7468f700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5d6c1409e0 0x7f5d6c14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.405+0000 7f5d7468f700 1 --2- 172.21.2.104:0/3085359939 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5d6c139410 0x7f5d6c14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.405+0000 7f5d7468f700 1 -- 172.21.2.104:0/3085359939 >> 172.21.2.104:0/3085359939 conn(0x7f5d6c0607c0 msgr2=0x7f5d6c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:11.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.409+0000 7f5d7468f700 1 -- 172.21.2.104:0/3085359939 shutdown_connections 2021-07-11T03:55:11.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.409+0000 7f5d7468f700 1 -- 172.21.2.104:0/3085359939 wait complete. 2021-07-11T03:55:11.416 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T03:55:11.422 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T03:55:11.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.469+0000 7f93fa1c7700 1 Processor -- start 2021-07-11T03:55:11.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.469+0000 7f93fa1c7700 1 -- start start 2021-07-11T03:55:11.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f93f40aca30 0x7f93f40b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f40ace10 con 0x7f93f40ad870 2021-07-11T03:55:11.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93f40b5160 con 0x7f93f40b1430 2021-07-11T03:55:11.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93f40adee0 con 0x7f93f40aca30 2021-07-11T03:55:11.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f89c4700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f93f40ad870 0x7f93f41434c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40620/0 (socket says 172.21.2.104:40620) 2021-07-11T03:55:11.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f89c4700 1 -- 172.21.2.104:0/3540650348 learned_addr learned my addr 172.21.2.104:0/3540650348 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:11.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 --2- 172.21.2.104:0/3540650348 >> v2:172.21.2.104:3300/0 conn(0x7f93f40aca30 0x7f93f40b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 -- 172.21.2.104:0/3540650348 >> v1:172.21.2.105:6789/0 conn(0x7f93f40b1430 legacy=0x7f93f4067000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:11.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 -- 172.21.2.104:0/3540650348 >> v1:172.21.2.104:6789/0 conn(0x7f93f40ad870 legacy=0x7f93f41434c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 -- 172.21.2.104:0/3540650348 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f40ae180 con 0x7f93f40aca30 2021-07-11T03:55:11.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 --2- 172.21.2.104:0/3540650348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93f40aca30 0x7f93f40b55c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f93f40b0fe0 tx=0x7f93e4006df0).ready entity=mon.0 client_cookie=593c02a380b3cfe4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93e3fff700 1 -- 172.21.2.104:0/3540650348 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e400d5c0 con 0x7f93f40aca30 2021-07-11T03:55:11.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93e3fff700 1 -- 172.21.2.104:0/3540650348 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e4000e90 con 0x7f93f40aca30 2021-07-11T03:55:11.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93e3fff700 1 -- 172.21.2.104:0/3540650348 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e400d910 con 0x7f93f40aca30 2021-07-11T03:55:11.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- 172.21.2.104:0/3540650348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93f40aca30 msgr2=0x7f93f40b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 --2- 172.21.2.104:0/3540650348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93f40aca30 0x7f93f40b55c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f93f40b0fe0 tx=0x7f93e4006df0).stop 2021-07-11T03:55:11.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- 172.21.2.104:0/3540650348 shutdown_connections 2021-07-11T03:55:11.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 --2- 172.21.2.104:0/3540650348 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93f40aca30 0x7f93f40b55c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- 172.21.2.104:0/3540650348 >> 172.21.2.104:0/3540650348 conn(0x7f93f4062560 msgr2=0x7f93f4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:11.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- 172.21.2.104:0/3540650348 shutdown_connections 2021-07-11T03:55:11.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- 172.21.2.104:0/3540650348 wait complete. 2021-07-11T03:55:11.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 Processor -- start 2021-07-11T03:55:11.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- start start 2021-07-11T03:55:11.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93f40aca30 0x7f93f40ab420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93f40ad870 0x7f93f41579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93f40b1430 0x7f93f40aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f4154950 con 0x7f93f40aca30 2021-07-11T03:55:11.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f40a92f0 con 0x7f93f40ad870 2021-07-11T03:55:11.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93f4065da0 con 0x7f93f40b1430 2021-07-11T03:55:11.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93f40aca30 0x7f93f40ab420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93f40aca30 0x7f93f40ab420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43858/0 (socket says 172.21.2.104:43858) 2021-07-11T03:55:11.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 -- 172.21.2.104:0/1484467976 learned_addr learned my addr 172.21.2.104:0/1484467976 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:11.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 -- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93f40b1430 msgr2=0x7f93f40aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93f40b1430 0x7f93f40aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 -- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93f40ad870 msgr2=0x7f93f41579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93f40ad870 0x7f93f41579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 -- 172.21.2.104:0/1484467976 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e400b180 con 0x7f93f40aca30 2021-07-11T03:55:11.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93f91c5700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93f40aca30 0x7f93f40ab420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f93f40b0fe0 tx=0x7f93e4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93e27fc700 1 -- 172.21.2.104:0/1484467976 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e400de90 con 0x7f93f40aca30 2021-07-11T03:55:11.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93e27fc700 1 -- 172.21.2.104:0/1484467976 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e4002f70 con 0x7f93f40aca30 2021-07-11T03:55:11.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- 172.21.2.104:0/1484467976 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f40ab0d0 con 0x7f93f40aca30 2021-07-11T03:55:11.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93e27fc700 1 -- 172.21.2.104:0/1484467976 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e401e620 con 0x7f93f40aca30 2021-07-11T03:55:11.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.473+0000 7f93fa1c7700 1 -- 172.21.2.104:0/1484467976 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f4066a20 con 0x7f93f40aca30 2021-07-11T03:55:11.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.477+0000 7f93fa1c7700 1 -- 172.21.2.104:0/1484467976 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f4057350 con 0x7f93f40aca30 2021-07-11T03:55:11.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.477+0000 7f93e27fc700 1 -- 172.21.2.104:0/1484467976 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f93e400f030 con 0x7f93f40aca30 2021-07-11T03:55:11.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.477+0000 7f93e27fc700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93d40531f0 0x7f93d40556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.477+0000 7f93e27fc700 1 -- 172.21.2.104:0/1484467976 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f93e406c6d0 con 0x7f93f40aca30 2021-07-11T03:55:11.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.477+0000 7f93f89c4700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93d40531f0 0x7f93d40556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.477+0000 7f93f89c4700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93d40531f0 0x7f93d40556b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f93f40af8b0 tx=0x7f93f0007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.477+0000 7f93e27fc700 1 -- 172.21.2.104:0/1484467976 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f93e4043970 con 0x7f93f40aca30 2021-07-11T03:55:11.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.589+0000 7f93fa1c7700 1 -- 172.21.2.104:0/1484467976 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f93f4057350 con 0x7f93f40aca30 2021-07-11T03:55:11.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93e27fc700 1 -- 172.21.2.104:0/1484467976 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v15) v1 ==== 187+0+0 (secure 0 0 0) 0x7f93e4046f90 con 0x7f93f40aca30 2021-07-11T03:55:11.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 -- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93d40531f0 msgr2=0x7f93d40556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93d40531f0 0x7f93d40556b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f93f40af8b0 tx=0x7f93f0007800).stop 2021-07-11T03:55:11.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 -- 172.21.2.104:0/1484467976 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93f40aca30 msgr2=0x7f93f40ab420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93f40aca30 0x7f93f40ab420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f93f40b0fe0 tx=0x7f93e4014000).stop 2021-07-11T03:55:11.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 -- 172.21.2.104:0/1484467976 shutdown_connections 2021-07-11T03:55:11.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93f40b1430 0x7f93f40aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93f40ad870 0x7f93f41579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93d40531f0 0x7f93d40556b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 --2- 172.21.2.104:0/1484467976 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93f40aca30 0x7f93f40ab420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 -- 172.21.2.104:0/1484467976 >> 172.21.2.104:0/1484467976 conn(0x7f93f4062560 msgr2=0x7f93f40b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:11.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 -- 172.21.2.104:0/1484467976 shutdown_connections 2021-07-11T03:55:11.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.597+0000 7f93fa1c7700 1 -- 172.21.2.104:0/1484467976 wait complete. 2021-07-11T03:55:11.607 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 7 and data pool 8 2021-07-11T03:55:11.612 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:55:11.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 Processor -- start 2021-07-11T03:55:11.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- start start 2021-07-11T03:55:11.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f46880ad150 0x7f46880b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f46880a9da0 0x7f4688140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46880ac370 con 0x7f46880aa860 2021-07-11T03:55:11.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880abb50 con 0x7f46880ad150 2021-07-11T03:55:11.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46880aac40 con 0x7f46880a9da0 2021-07-11T03:55:11.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468cb53700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f46880aa860 0x7f46880aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40644/0 (socket says 172.21.2.104:40644) 2021-07-11T03:55:11.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468cb53700 1 -- 172.21.2.104:0/3615063555 learned_addr learned my addr 172.21.2.104:0/3615063555 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:11.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f4687fff700 1 --2- 172.21.2.104:0/3615063555 >> v2:172.21.2.104:3300/0 conn(0x7f46880ad150 0x7f46880b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f4687fff700 1 -- 172.21.2.104:0/3615063555 >> v2:172.21.2.105:3300/0 conn(0x7f46880a9da0 msgr2=0x7f4688140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:11.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f4687fff700 1 --2- 172.21.2.104:0/3615063555 >> v2:172.21.2.105:3300/0 conn(0x7f46880a9da0 0x7f4688140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f4687fff700 1 -- 172.21.2.104:0/3615063555 >> v1:172.21.2.104:6789/0 conn(0x7f46880aa860 legacy=0x7f46880aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f4687fff700 1 -- 172.21.2.104:0/3615063555 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880b16b0 con 0x7f46880ad150 2021-07-11T03:55:11.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f46877fe700 1 -- 172.21.2.104:0/3615063555 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454134044 0 0) 0x7f46880ac370 con 0x7f46880aa860 2021-07-11T03:55:11.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f4687fff700 1 --2- 172.21.2.104:0/3615063555 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46880ad150 0x7f46880b11c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f4678002f70 tx=0x7f4678006df0).ready entity=mon.0 client_cookie=a907fb16f1ce1439 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f46877fe700 1 -- 172.21.2.104:0/3615063555 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f467800d5c0 con 0x7f46880ad150 2021-07-11T03:55:11.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f46877fe700 1 -- 172.21.2.104:0/3615063555 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4678000e90 con 0x7f46880ad150 2021-07-11T03:55:11.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f46877fe700 1 -- 172.21.2.104:0/3615063555 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f467800d910 con 0x7f46880ad150 2021-07-11T03:55:11.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- 172.21.2.104:0/3615063555 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46880ad150 msgr2=0x7f46880b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 --2- 172.21.2.104:0/3615063555 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46880ad150 0x7f46880b11c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f4678002f70 tx=0x7f4678006df0).stop 2021-07-11T03:55:11.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- 172.21.2.104:0/3615063555 shutdown_connections 2021-07-11T03:55:11.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 --2- 172.21.2.104:0/3615063555 >> v2:172.21.2.105:3300/0 conn(0x7f46880a9da0 0x7f4688140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 --2- 172.21.2.104:0/3615063555 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46880ad150 0x7f46880b11c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- 172.21.2.104:0/3615063555 >> 172.21.2.104:0/3615063555 conn(0x7f46880b64d0 msgr2=0x7f46880b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:11.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- 172.21.2.104:0/3615063555 shutdown_connections 2021-07-11T03:55:11.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- 172.21.2.104:0/3615063555 wait complete. 2021-07-11T03:55:11.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 Processor -- start 2021-07-11T03:55:11.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- start start 2021-07-11T03:55:11.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46880a9da0 0x7f46880b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46880aa860 0x7f4688151790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46880ad150 0x7f4688151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46881438e0 con 0x7f46880aa860 2021-07-11T03:55:11.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46880657a0 con 0x7f46880ad150 2021-07-11T03:55:11.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f468edb7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4688152170 con 0x7f46880a9da0 2021-07-11T03:55:11.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f4687fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46880aa860 0x7f4688151790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f4687fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46880aa860 0x7f4688151790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43886/0 (socket says 172.21.2.104:43886) 2021-07-11T03:55:11.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.657+0000 7f4687fff700 1 -- 172.21.2.104:0/1974909032 learned_addr learned my addr 172.21.2.104:0/1974909032 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:11.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4687fff700 1 -- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46880a9da0 msgr2=0x7f46880b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4687fff700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46880a9da0 0x7f46880b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4687fff700 1 -- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46880ad150 msgr2=0x7f4688151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4687fff700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46880ad150 0x7f4688151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4687fff700 1 -- 172.21.2.104:0/1974909032 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467800b180 con 0x7f46880aa860 2021-07-11T03:55:11.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4687fff700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46880aa860 0x7f4688151790 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f4678002f70 tx=0x7f4678014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4685ffb700 1 -- 172.21.2.104:0/1974909032 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4678006b20 con 0x7f46880aa860 2021-07-11T03:55:11.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4685ffb700 1 -- 172.21.2.104:0/1974909032 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467800de30 con 0x7f46880aa860 2021-07-11T03:55:11.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f468edb7700 1 -- 172.21.2.104:0/1974909032 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4688151330 con 0x7f46880aa860 2021-07-11T03:55:11.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f468edb7700 1 -- 172.21.2.104:0/1974909032 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46880660e0 con 0x7f46880aa860 2021-07-11T03:55:11.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4685ffb700 1 -- 172.21.2.104:0/1974909032 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f467801e620 con 0x7f46880aa860 2021-07-11T03:55:11.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f468edb7700 1 -- 172.21.2.104:0/1974909032 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468809f130 con 0x7f46880aa860 2021-07-11T03:55:11.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4685ffb700 1 -- 172.21.2.104:0/1974909032 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f467800f030 con 0x7f46880aa860 2021-07-11T03:55:11.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4685ffb700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f466c0531f0 0x7f466c0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f4685ffb700 1 -- 172.21.2.104:0/1974909032 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f467806c6e0 con 0x7f46880aa860 2021-07-11T03:55:11.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f468cb53700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f466c0531f0 0x7f466c0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.661+0000 7f468cb53700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f466c0531f0 0x7f466c0556b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f4670000ed0 tx=0x7f46700076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.665+0000 7f4685ffb700 1 -- 172.21.2.104:0/1974909032 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4678043910 con 0x7f46880aa860 2021-07-11T03:55:11.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 -- 172.21.2.104:0/1974909032 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46881514c0 con 0x7f46880aa860 2021-07-11T03:55:11.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f4685ffb700 1 -- 172.21.2.104:0/1974909032 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16051 (secure 0 0 0) 0x7f467802fed0 con 0x7f46880aa860 2021-07-11T03:55:11.754 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:11.755 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":43,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:55:11.595442+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:55:09.448452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-07-11T03:55:10.579101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:55:11.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 -- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f466c0531f0 msgr2=0x7f466c0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f466c0531f0 0x7f466c0556b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f4670000ed0 tx=0x7f46700076c0).stop 2021-07-11T03:55:11.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 -- 172.21.2.104:0/1974909032 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46880aa860 msgr2=0x7f4688151790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46880aa860 0x7f4688151790 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f4678002f70 tx=0x7f4678014000).stop 2021-07-11T03:55:11.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 -- 172.21.2.104:0/1974909032 shutdown_connections 2021-07-11T03:55:11.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46880ad150 0x7f4688151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f466c0531f0 0x7f466c0556b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46880aa860 0x7f4688151790 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 --2- 172.21.2.104:0/1974909032 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46880a9da0 0x7f46880b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 -- 172.21.2.104:0/1974909032 >> 172.21.2.104:0/1974909032 conn(0x7f46880b64d0 msgr2=0x7f468805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:11.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 -- 172.21.2.104:0/1974909032 shutdown_connections 2021-07-11T03:55:11.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.749+0000 7f468edb7700 1 -- 172.21.2.104:0/1974909032 wait complete. 2021-07-11T03:55:11.762 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:55:11.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d260e700 1 Processor -- start 2021-07-11T03:55:11.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d260e700 1 -- start start 2021-07-11T03:55:11.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d260e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc0a76b0 0x7fd3cc066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d260e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd3cc0b0060 0x7fd3cc05fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d260e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc0a7a90 con 0x7fd3cc0b1970 2021-07-11T03:55:11.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d260e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc147620 con 0x7fd3cc0b0060 2021-07-11T03:55:11.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d260e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc1478c0 con 0x7fd3cc0a76b0 2021-07-11T03:55:11.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3cb7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd3cc0b1970 0x7fd3cc05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40658/0 (socket says 172.21.2.104:40658) 2021-07-11T03:55:11.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3cb7fe700 1 -- 172.21.2.104:0/3511877115 learned_addr learned my addr 172.21.2.104:0/3511877115 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:11.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d0bab700 1 --2- 172.21.2.104:0/3511877115 >> v2:172.21.2.104:3300/0 conn(0x7fd3cc0b0060 0x7fd3cc05fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3caffd700 1 -- 172.21.2.104:0/3511877115 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646371139 0 0) 0x7fd3cc0a7a90 con 0x7fd3cc0b1970 2021-07-11T03:55:11.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3caffd700 1 -- 172.21.2.104:0/3511877115 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3ac003260 con 0x7fd3cc0b1970 2021-07-11T03:55:11.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d0bab700 1 -- 172.21.2.104:0/3511877115 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc0a76b0 msgr2=0x7fd3cc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:11.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d0bab700 1 --2- 172.21.2.104:0/3511877115 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc0a76b0 0x7fd3cc066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d0bab700 1 -- 172.21.2.104:0/3511877115 >> v1:172.21.2.104:6789/0 conn(0x7fd3cc0b1970 legacy=0x7fd3cc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d0bab700 1 -- 172.21.2.104:0/3511877115 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc147b60 con 0x7fd3cc0b0060 2021-07-11T03:55:11.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.805+0000 7fd3d0bab700 1 --2- 172.21.2.104:0/3511877115 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc0b0060 0x7fd3cc05fdc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fd3c40032f0 tx=0x7fd3c40091d0).ready entity=mon.0 client_cookie=9c4155fb99e53aab server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3caffd700 1 -- 172.21.2.104:0/3511877115 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3c400b6c0 con 0x7fd3cc0b0060 2021-07-11T03:55:11.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3caffd700 1 -- 172.21.2.104:0/3511877115 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c400b820 con 0x7fd3cc0b0060 2021-07-11T03:55:11.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3caffd700 1 -- 172.21.2.104:0/3511877115 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3c400b990 con 0x7fd3cc0b0060 2021-07-11T03:55:11.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- 172.21.2.104:0/3511877115 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc0b0060 msgr2=0x7fd3cc05fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 --2- 172.21.2.104:0/3511877115 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc0b0060 0x7fd3cc05fdc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fd3c40032f0 tx=0x7fd3c40091d0).stop 2021-07-11T03:55:11.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- 172.21.2.104:0/3511877115 shutdown_connections 2021-07-11T03:55:11.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 --2- 172.21.2.104:0/3511877115 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc0b0060 0x7fd3cc05fdc0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 --2- 172.21.2.104:0/3511877115 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc0a76b0 0x7fd3cc066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- 172.21.2.104:0/3511877115 >> 172.21.2.104:0/3511877115 conn(0x7fd3cc0b53e0 msgr2=0x7fd3cc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:11.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- 172.21.2.104:0/3511877115 shutdown_connections 2021-07-11T03:55:11.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- 172.21.2.104:0/3511877115 wait complete. 2021-07-11T03:55:11.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 Processor -- start 2021-07-11T03:55:11.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- start start 2021-07-11T03:55:11.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3cc0a76b0 0x7fd3cc14d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc0b0060 0x7fd3cc0a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc0b1970 0x7fd3cc0a9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0aa600 con 0x7fd3cc0b0060 2021-07-11T03:55:11.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0ac880 con 0x7fd3cc0a76b0 2021-07-11T03:55:11.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0aa830 con 0x7fd3cc0b1970 2021-07-11T03:55:11.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3cb7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc0b0060 0x7fd3cc0a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3cb7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc0b0060 0x7fd3cc0a9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43898/0 (socket says 172.21.2.104:43898) 2021-07-11T03:55:11.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3cb7fe700 1 -- 172.21.2.104:0/3646556830 learned_addr learned my addr 172.21.2.104:0/3646556830 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:11.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d0bab700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc0b1970 0x7fd3cc0a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3cb7fe700 1 -- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc0b1970 msgr2=0x7fd3cc0a9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3cb7fe700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc0b1970 0x7fd3cc0a9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3cb7fe700 1 -- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3cc0a76b0 msgr2=0x7fd3cc14d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3cb7fe700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3cc0a76b0 0x7fd3cc14d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3cb7fe700 1 -- 172.21.2.104:0/3646556830 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c4009470 con 0x7fd3cc0b0060 2021-07-11T03:55:11.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3cb7fe700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc0b0060 0x7fd3cc0a9230 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fd3b4001110 tx=0x7fd3b4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3c97fa700 1 -- 172.21.2.104:0/3646556830 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b4007e10 con 0x7fd3cc0b0060 2021-07-11T03:55:11.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3c97fa700 1 -- 172.21.2.104:0/3646556830 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b40072e0 con 0x7fd3cc0b0060 2021-07-11T03:55:11.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- 172.21.2.104:0/3646556830 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc0acb20 con 0x7fd3cc0b0060 2021-07-11T03:55:11.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- 172.21.2.104:0/3646556830 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc14e420 con 0x7fd3cc0b0060 2021-07-11T03:55:11.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3c97fa700 1 -- 172.21.2.104:0/3646556830 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3b4005800 con 0x7fd3cc0b0060 2021-07-11T03:55:11.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3d260e700 1 -- 172.21.2.104:0/3646556830 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc0aa9f0 con 0x7fd3cc0b0060 2021-07-11T03:55:11.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.809+0000 7fd3c97fa700 1 -- 172.21.2.104:0/3646556830 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd3b4010070 con 0x7fd3cc0b0060 2021-07-11T03:55:11.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.813+0000 7fd3c97fa700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3b0053330 0x7fd3b00557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.813+0000 7fd3c97fa700 1 -- 172.21.2.104:0/3646556830 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd3b406ac50 con 0x7fd3cc0b0060 2021-07-11T03:55:11.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.813+0000 7fd3cbfff700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3b0053330 0x7fd3b00557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.813+0000 7fd3cbfff700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3b0053330 0x7fd3b00557f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fd3bc002f70 tx=0x7fd3bc006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.813+0000 7fd3c97fa700 1 -- 172.21.2.104:0/3646556830 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd3b4041e80 con 0x7fd3cc0b0060 2021-07-11T03:55:11.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.893+0000 7fd3d260e700 1 -- 172.21.2.104:0/3646556830 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3cc0a9db0 con 0x7fd3cc0b0060 2021-07-11T03:55:11.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.893+0000 7fd3c97fa700 1 -- 172.21.2.104:0/3646556830 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16051 (secure 0 0 0) 0x7fd3b40454a0 con 0x7fd3cc0b0060 2021-07-11T03:55:11.901 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:11.901 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":43,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:55:11.595442+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:55:09.448452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-07-11T03:55:10.579101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:55:11.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 -- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3b0053330 msgr2=0x7fd3b00557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3b0053330 0x7fd3b00557f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fd3bc002f70 tx=0x7fd3bc006a80).stop 2021-07-11T03:55:11.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 -- 172.21.2.104:0/3646556830 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc0b0060 msgr2=0x7fd3cc0a9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc0b0060 0x7fd3cc0a9230 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fd3b4001110 tx=0x7fd3b4007ad0).stop 2021-07-11T03:55:11.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 -- 172.21.2.104:0/3646556830 shutdown_connections 2021-07-11T03:55:11.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc0b1970 0x7fd3cc0a9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3b0053330 0x7fd3b00557f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc0b0060 0x7fd3cc0a9230 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 --2- 172.21.2.104:0/3646556830 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3cc0a76b0 0x7fd3cc14d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 -- 172.21.2.104:0/3646556830 >> 172.21.2.104:0/3646556830 conn(0x7fd3cc0b53e0 msgr2=0x7fd3cc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:11.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 -- 172.21.2.104:0/3646556830 shutdown_connections 2021-07-11T03:55:11.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.897+0000 7fd3d260e700 1 -- 172.21.2.104:0/3646556830 wait complete. 2021-07-11T03:55:11.909 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T03:55:11.962 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:55:11.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 Processor -- start 2021-07-11T03:55:11.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 -- start start 2021-07-11T03:55:11.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc07c144440 0x7fc07c144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07c0604e0 con 0x7fc07c065420 2021-07-11T03:55:11.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc07c0b23b0 con 0x7fc07c060100 2021-07-11T03:55:11.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07c065c80 con 0x7fc07c144440 2021-07-11T03:55:11.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc07b7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc07c065420 0x7fc07c14b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40676/0 (socket says 172.21.2.104:40676) 2021-07-11T03:55:11.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc07b7fe700 1 -- 172.21.2.104:0/2230184497 learned_addr learned my addr 172.21.2.104:0/2230184497 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:11.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4230023204 0 0) 0x7fc07c0604e0 con 0x7fc07c065420 2021-07-11T03:55:11.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc060003260 con 0x7fc07c065420 2021-07-11T03:55:11.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3428480639 0 0) 0x7fc060003260 con 0x7fc07c065420 2021-07-11T03:55:11.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 >> v2:172.21.2.105:3300/0 conn(0x7fc07c144440 msgr2=0x7fc07c144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:11.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 --2- 172.21.2.104:0/2230184497 >> v2:172.21.2.105:3300/0 conn(0x7fc07c144440 0x7fc07c144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 >> v1:172.21.2.105:6789/0 conn(0x7fc07c060100 legacy=0x7fc07c064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:11.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0b2650 con 0x7fc07c065420 2021-07-11T03:55:11.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fc074001a00 con 0x7fc07c065420 2021-07-11T03:55:11.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 >> v1:172.21.2.104:6789/0 conn(0x7fc07c065420 legacy=0x7fc07c14b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 --2- 172.21.2.104:0/2230184497 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc060003820 0x7fc060003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 --2- 172.21.2.104:0/2230184497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 0x7fc060004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 --2- 172.21.2.104:0/2230184497 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc060005050 0x7fc06000fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c065c80 con 0x7fc060004500 2021-07-11T03:55:11.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc060010350 con 0x7fc060005050 2021-07-11T03:55:11.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc060010510 con 0x7fc060003820 2021-07-11T03:55:11.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc07a7fc700 1 --2- 172.21.2.104:0/2230184497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 0x7fc060004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc07a7fc700 1 -- 172.21.2.104:0/2230184497 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc060003820 msgr2=0x7fc060003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:11.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc07a7fc700 1 --2- 172.21.2.104:0/2230184497 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc060003820 0x7fc060003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc07a7fc700 1 -- 172.21.2.104:0/2230184497 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc060005050 msgr2=0x7fc06000fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:11.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc07a7fc700 1 --2- 172.21.2.104:0/2230184497 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc060005050 0x7fc06000fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc07a7fc700 1 -- 172.21.2.104:0/2230184497 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060003260 con 0x7fc060004500 2021-07-11T03:55:11.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc07b7fe700 1 -- 172.21.2.104:0/2230184497 reap_dead start 2021-07-11T03:55:11.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc07a7fc700 1 --2- 172.21.2.104:0/2230184497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 0x7fc060004980 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fc070002f70 tx=0x7fc070006ca0).ready entity=mon.0 client_cookie=f07692ec2f790502 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc070012410 con 0x7fc060004500 2021-07-11T03:55:11.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc070012570 con 0x7fc060004500 2021-07-11T03:55:11.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc079ffb700 1 -- 172.21.2.104:0/2230184497 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc0700128d0 con 0x7fc060004500 2021-07-11T03:55:11.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 -- 172.21.2.104:0/2230184497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 msgr2=0x7fc060004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 --2- 172.21.2.104:0/2230184497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 0x7fc060004980 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7fc070002f70 tx=0x7fc070006ca0).stop 2021-07-11T03:55:11.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 -- 172.21.2.104:0/2230184497 shutdown_connections 2021-07-11T03:55:11.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 --2- 172.21.2.104:0/2230184497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 0x7fc060004980 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:11.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 -- 172.21.2.104:0/2230184497 >> 172.21.2.104:0/2230184497 conn(0x7fc07c05cf60 msgr2=0x7fc07c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:11.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 -- 172.21.2.104:0/2230184497 shutdown_connections 2021-07-11T03:55:11.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 -- 172.21.2.104:0/2230184497 wait complete. 2021-07-11T03:55:11.975 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:55:11.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 Processor -- start 2021-07-11T03:55:11.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.957+0000 7fc08130d700 1 -- start start 2021-07-11T03:55:11.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc08130d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 0x7fc07c142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc08130d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc07c0b1280 0x7fc07c13a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc08130d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc07c13ac30 0x7fc07c151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc08130d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c0ac430 con 0x7fc060004500 2021-07-11T03:55:11.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc08130d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c0ad180 con 0x7fc07c0b1280 2021-07-11T03:55:11.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc08130d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c0ad420 con 0x7fc07c13ac30 2021-07-11T03:55:11.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc07affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 0x7fc07c142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc07affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 0x7fc07c142260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43918/0 (socket says 172.21.2.104:43918) 2021-07-11T03:55:11.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc07affd700 1 -- 172.21.2.104:0/2406933741 learned_addr learned my addr 172.21.2.104:0/2406933741 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:11.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc07affd700 1 -- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc07c13ac30 msgr2=0x7fc07c151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc07affd700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc07c13ac30 0x7fc07c151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc07affd700 1 -- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc07c0b1280 msgr2=0x7fc07c13a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:11.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc07affd700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc07c0b1280 0x7fc07c13a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:11.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc07affd700 1 -- 172.21.2.104:0/2406933741 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0ad6c0 con 0x7fc060004500 2021-07-11T03:55:11.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc07affd700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 0x7fc07c142260 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fc06c000f70 tx=0x7fc06c006d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc06bfff700 1 -- 172.21.2.104:0/2406933741 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc06c012410 con 0x7fc060004500 2021-07-11T03:55:11.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc06bfff700 1 -- 172.21.2.104:0/2406933741 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c012570 con 0x7fc060004500 2021-07-11T03:55:11.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc08130d700 1 -- 172.21.2.104:0/2406933741 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c0ad960 con 0x7fc060004500 2021-07-11T03:55:11.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc08130d700 1 -- 172.21.2.104:0/2406933741 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c0afec0 con 0x7fc060004500 2021-07-11T03:55:11.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc06bfff700 1 -- 172.21.2.104:0/2406933741 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc06c0128d0 con 0x7fc060004500 2021-07-11T03:55:11.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc08130d700 1 -- 172.21.2.104:0/2406933741 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c0adaf0 con 0x7fc060004500 2021-07-11T03:55:11.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc06bfff700 1 -- 172.21.2.104:0/2406933741 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc06c00f030 con 0x7fc060004500 2021-07-11T03:55:11.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc06bfff700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc058053260 0x7fc058055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:11.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc06bfff700 1 -- 172.21.2.104:0/2406933741 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc06c06b020 con 0x7fc060004500 2021-07-11T03:55:11.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc07a7fc700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc058053260 0x7fc058055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:11.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.961+0000 7fc07a7fc700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc058053260 0x7fc058055720 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fc070002f70 tx=0x7fc070015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:11.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:11.965+0000 7fc06bfff700 1 -- 172.21.2.104:0/2406933741 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc06c042360 con 0x7fc060004500 2021-07-11T03:55:11.986 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:55:11.989 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:55:12.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.073+0000 7fc08130d700 1 -- 172.21.2.104:0/2406933741 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc07c057350 con 0x7fc060004500 2021-07-11T03:55:12.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc06bfff700 1 -- 172.21.2.104:0/2406933741 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc06c045980 con 0x7fc060004500 2021-07-11T03:55:12.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 -- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc058053260 msgr2=0x7fc058055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc058053260 0x7fc058055720 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fc070002f70 tx=0x7fc070015000).stop 2021-07-11T03:55:12.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 -- 172.21.2.104:0/2406933741 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 msgr2=0x7fc07c142260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 0x7fc07c142260 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fc06c000f70 tx=0x7fc06c006d10).stop 2021-07-11T03:55:12.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 -- 172.21.2.104:0/2406933741 shutdown_connections 2021-07-11T03:55:12.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc07c13ac30 0x7fc07c151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:12.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc07c0b1280 0x7fc07c13a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:12.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc058053260 0x7fc058055720 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 --2- 172.21.2.104:0/2406933741 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc060004500 0x7fc07c142260 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 -- 172.21.2.104:0/2406933741 >> 172.21.2.104:0/2406933741 conn(0x7fc07c05cf60 msgr2=0x7fc07c061e60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:12.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 -- 172.21.2.104:0/2406933741 shutdown_connections 2021-07-11T03:55:12.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.609+0000 7fc08130d700 1 -- 172.21.2.104:0/2406933741 wait complete. 2021-07-11T03:55:12.623 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:55:12.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.665+0000 7fab2aa5d700 1 Processor -- start 2021-07-11T03:55:12.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.665+0000 7fab2aa5d700 1 -- start start 2021-07-11T03:55:12.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab2aa5d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fab2405f6b0 0x7fab240a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab2aa5d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab240b14c0 con 0x7fab240a8b60 2021-07-11T03:55:12.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab2aa5d700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab240b1b90 con 0x7fab24143540 2021-07-11T03:55:12.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab2aa5d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab240ab3e0 con 0x7fab2405f6b0 2021-07-11T03:55:12.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab28ffa700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fab240a8b60 0x7fab2413d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40696/0 (socket says 172.21.2.104:40696) 2021-07-11T03:55:12.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab28ffa700 1 -- 172.21.2.104:0/2523141878 learned_addr learned my addr 172.21.2.104:0/2523141878 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:12.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752034283 0 0) 0x7fab240b14c0 con 0x7fab240a8b60 2021-07-11T03:55:12.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab04003260 con 0x7fab240a8b60 2021-07-11T03:55:12.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab237fe700 1 --2- 172.21.2.104:0/2523141878 >> v2:172.21.2.105:3300/0 conn(0x7fab2405f6b0 0x7fab240a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:12.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3151575224 0 0) 0x7fab04003260 con 0x7fab240a8b60 2021-07-11T03:55:12.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 >> v2:172.21.2.105:3300/0 conn(0x7fab2405f6b0 msgr2=0x7fab240a9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:12.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 --2- 172.21.2.104:0/2523141878 >> v2:172.21.2.105:3300/0 conn(0x7fab2405f6b0 0x7fab240a9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 >> v1:172.21.2.105:6789/0 conn(0x7fab24143540 legacy=0x7fab240b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:12.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab24143fa0 con 0x7fab240a8b60 2021-07-11T03:55:12.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fab1c0019e0 con 0x7fab240a8b60 2021-07-11T03:55:12.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 >> v1:172.21.2.104:6789/0 conn(0x7fab240a8b60 legacy=0x7fab2413d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 --2- 172.21.2.104:0/2523141878 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab04003780 0x7fab04003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 --2- 172.21.2.104:0/2523141878 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab04004500 0x7fab04004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 --2- 172.21.2.104:0/2523141878 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab04004f90 0x7fab0400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab240ab3e0 con 0x7fab04003780 2021-07-11T03:55:12.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0400bdd0 con 0x7fab04004f90 2021-07-11T03:55:12.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0400bf90 con 0x7fab04004500 2021-07-11T03:55:12.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab23fff700 1 --2- 172.21.2.104:0/2523141878 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab04003780 0x7fab04003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:12.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab23fff700 1 -- 172.21.2.104:0/2523141878 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab04004500 msgr2=0x7fab04004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:12.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab23fff700 1 --2- 172.21.2.104:0/2523141878 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab04004500 0x7fab04004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:12.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab23fff700 1 -- 172.21.2.104:0/2523141878 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab04004f90 msgr2=0x7fab0400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:12.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab23fff700 1 --2- 172.21.2.104:0/2523141878 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab04004f90 0x7fab0400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:12.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab23fff700 1 -- 172.21.2.104:0/2523141878 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab04003260 con 0x7fab04003780 2021-07-11T03:55:12.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab28ffa700 1 -- 172.21.2.104:0/2523141878 reap_dead start 2021-07-11T03:55:12.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab23fff700 1 --2- 172.21.2.104:0/2523141878 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab04003780 0x7fab04003ba0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fab14001fa0 tx=0x7fab14007a00).ready entity=mon.0 client_cookie=ad359c770092de42 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:12.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab14007070 con 0x7fab04003780 2021-07-11T03:55:12.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab140071d0 con 0x7fab04003780 2021-07-11T03:55:12.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab22ffd700 1 -- 172.21.2.104:0/2523141878 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab14006510 con 0x7fab04003780 2021-07-11T03:55:12.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab2aa5d700 1 -- 172.21.2.104:0/2523141878 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab04003780 msgr2=0x7fab04003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab2aa5d700 1 --2- 172.21.2.104:0/2523141878 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab04003780 0x7fab04003ba0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fab14001fa0 tx=0x7fab14007a00).stop 2021-07-11T03:55:12.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab2aa5d700 1 -- 172.21.2.104:0/2523141878 shutdown_connections 2021-07-11T03:55:12.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab2aa5d700 1 --2- 172.21.2.104:0/2523141878 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab04003780 0x7fab04003ba0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab2aa5d700 1 -- 172.21.2.104:0/2523141878 >> 172.21.2.104:0/2523141878 conn(0x7fab240b53e0 msgr2=0x7fab240b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:12.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab2aa5d700 1 -- 172.21.2.104:0/2523141878 shutdown_connections 2021-07-11T03:55:12.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.669+0000 7fab2aa5d700 1 -- 172.21.2.104:0/2523141878 wait complete. 2021-07-11T03:55:12.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab2aa5d700 1 Processor -- start 2021-07-11T03:55:12.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab2aa5d700 1 -- start start 2021-07-11T03:55:12.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab2aa5d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab04003780 0x7fab24130c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab2aa5d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab2412fcb0 0x7fab24142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab2aa5d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab24131ad0 0x7fab24131f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab2aa5d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab24141ed0 con 0x7fab2412fcb0 2021-07-11T03:55:12.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab2aa5d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab241421c0 con 0x7fab24131ad0 2021-07-11T03:55:12.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab2aa5d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2412e570 con 0x7fab04003780 2021-07-11T03:55:12.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab237fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab2412fcb0 0x7fab24142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:12.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab237fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab2412fcb0 0x7fab24142ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43938/0 (socket says 172.21.2.104:43938) 2021-07-11T03:55:12.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab237fe700 1 -- 172.21.2.104:0/3013252970 learned_addr learned my addr 172.21.2.104:0/3013252970 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:12.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab23fff700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab04003780 0x7fab24130c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:12.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab28ffa700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab24131ad0 0x7fab24131f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:12.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab237fe700 1 -- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab04003780 msgr2=0x7fab24130c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab237fe700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab04003780 0x7fab24130c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab237fe700 1 -- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab24131ad0 msgr2=0x7fab24131f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab237fe700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab24131ad0 0x7fab24131f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab237fe700 1 -- 172.21.2.104:0/3013252970 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab24062750 con 0x7fab2412fcb0 2021-07-11T03:55:12.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab237fe700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab2412fcb0 0x7fab24142ca0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fab2412ea40 tx=0x7fab0c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:12.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab217fa700 1 -- 172.21.2.104:0/3013252970 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c016070 con 0x7fab2412fcb0 2021-07-11T03:55:12.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab217fa700 1 -- 172.21.2.104:0/3013252970 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c000e90 con 0x7fab2412fcb0 2021-07-11T03:55:12.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab2aa5d700 1 -- 172.21.2.104:0/3013252970 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2412e7a0 con 0x7fab2412fcb0 2021-07-11T03:55:12.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab217fa700 1 -- 172.21.2.104:0/3013252970 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab0c011970 con 0x7fab2412fcb0 2021-07-11T03:55:12.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab2aa5d700 1 -- 172.21.2.104:0/3013252970 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2412f540 con 0x7fab2412fcb0 2021-07-11T03:55:12.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab2aa5d700 1 -- 172.21.2.104:0/3013252970 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab24143330 con 0x7fab2412fcb0 2021-07-11T03:55:12.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab217fa700 1 -- 172.21.2.104:0/3013252970 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fab0c011ad0 con 0x7fab2412fcb0 2021-07-11T03:55:12.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab217fa700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab08053300 0x7fab080557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab217fa700 1 -- 172.21.2.104:0/3013252970 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fab0c06b420 con 0x7fab2412fcb0 2021-07-11T03:55:12.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.673+0000 7fab23fff700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab08053300 0x7fab080557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:12.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.677+0000 7fab23fff700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab08053300 0x7fab080557c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fab14019000 tx=0x7fab14015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:12.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.677+0000 7fab217fa700 1 -- 172.21.2.104:0/3013252970 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fab0c042760 con 0x7fab2412fcb0 2021-07-11T03:55:12.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.785+0000 7fab2aa5d700 1 -- 172.21.2.104:0/3013252970 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab24057350 con 0x7fab2412fcb0 2021-07-11T03:55:12.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.785+0000 7fab217fa700 1 -- 172.21.2.104:0/3013252970 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2488 (secure 0 0 0) 0x7fab0c045d80 con 0x7fab2412fcb0 2021-07-11T03:55:12.793 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:12.793 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4492,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2412138431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2412138431},{"type":"v1","addr":"172.21.2.105:6835","nonce":2412138431}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/3271417890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3271417890},{"type":"v1","addr":"172.21.2.104:6834","nonce":3271417890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:55:11.595553+0000","modified":"2021-07-11T03:55:12.600491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/795410455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":795410455},{"type":"v1","addr":"172.21.2.104:6835","nonce":795410455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-07-11T03:55:12.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.785+0000 7fab2aa5d700 1 -- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab08053300 msgr2=0x7fab080557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.785+0000 7fab2aa5d700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab08053300 0x7fab080557c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fab14019000 tx=0x7fab14015000).stop 2021-07-11T03:55:12.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.785+0000 7fab2aa5d700 1 -- 172.21.2.104:0/3013252970 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab2412fcb0 msgr2=0x7fab24142ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.785+0000 7fab2aa5d700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab2412fcb0 0x7fab24142ca0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fab2412ea40 tx=0x7fab0c008040).stop 2021-07-11T03:55:12.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.785+0000 7fab2aa5d700 1 -- 172.21.2.104:0/3013252970 shutdown_connections 2021-07-11T03:55:12.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.785+0000 7fab2aa5d700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab24131ad0 0x7fab24131f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.785+0000 7fab2aa5d700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab08053300 0x7fab080557c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.785+0000 7fab2aa5d700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab2412fcb0 0x7fab24142ca0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.785+0000 7fab2aa5d700 1 --2- 172.21.2.104:0/3013252970 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab04003780 0x7fab24130c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.789+0000 7fab2aa5d700 1 -- 172.21.2.104:0/3013252970 >> 172.21.2.104:0/3013252970 conn(0x7fab240b53e0 msgr2=0x7fab2405fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:12.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.789+0000 7fab2aa5d700 1 -- 172.21.2.104:0/3013252970 shutdown_connections 2021-07-11T03:55:12.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.789+0000 7fab2aa5d700 1 -- 172.21.2.104:0/3013252970 wait complete. 2021-07-11T03:55:12.797 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 16 2021-07-11T03:55:12.800 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:55:12.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 Processor -- start 2021-07-11T03:55:12.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 -- start start 2021-07-11T03:55:12.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f4224067180 0x7f422405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f42240b1970 0x7f422405fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42240a7a90 con 0x7f42240a76b0 2021-07-11T03:55:12.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4224147620 con 0x7f42240b1970 2021-07-11T03:55:12.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42241478c0 con 0x7f4224067180 2021-07-11T03:55:12.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f4228b19700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f42240b1970 0x7f422405fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:12.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f4228b19700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f42240b1970 0x7f422405fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43946/0 (socket says 172.21.2.104:43946) 2021-07-11T03:55:12.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f4228b19700 1 -- 172.21.2.104:0/2687620038 learned_addr learned my addr 172.21.2.104:0/2687620038 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:12.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f4228b19700 1 -- 172.21.2.104:0/2687620038 >> v2:172.21.2.105:3300/0 conn(0x7f4224067180 msgr2=0x7f422405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:12.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f4228b19700 1 --2- 172.21.2.104:0/2687620038 >> v2:172.21.2.105:3300/0 conn(0x7f4224067180 0x7f422405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:12.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f4228b19700 1 -- 172.21.2.104:0/2687620038 >> v1:172.21.2.105:6789/0 conn(0x7f42240a76b0 legacy=0x7f4224066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:12.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f4228b19700 1 -- 172.21.2.104:0/2687620038 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4224147b60 con 0x7f42240b1970 2021-07-11T03:55:12.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f4228b19700 1 --2- 172.21.2.104:0/2687620038 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42240b1970 0x7f422405fba0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f422406a0e0 tx=0x7f421c009290).ready entity=mon.0 client_cookie=921043480b23ac2e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:12.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f4222ffd700 1 -- 172.21.2.104:0/2687620038 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421c00b830 con 0x7f42240b1970 2021-07-11T03:55:12.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f4222ffd700 1 -- 172.21.2.104:0/2687620038 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421c00b990 con 0x7f42240b1970 2021-07-11T03:55:12.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f4222ffd700 1 -- 172.21.2.104:0/2687620038 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f421c00bb00 con 0x7f42240b1970 2021-07-11T03:55:12.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 -- 172.21.2.104:0/2687620038 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42240b1970 msgr2=0x7f422405fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 --2- 172.21.2.104:0/2687620038 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42240b1970 0x7f422405fba0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f422406a0e0 tx=0x7f421c009290).stop 2021-07-11T03:55:12.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 -- 172.21.2.104:0/2687620038 shutdown_connections 2021-07-11T03:55:12.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 --2- 172.21.2.104:0/2687620038 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42240b1970 0x7f422405fba0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 --2- 172.21.2.104:0/2687620038 >> v2:172.21.2.105:3300/0 conn(0x7f4224067180 0x7f422405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:12.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 -- 172.21.2.104:0/2687620038 >> 172.21.2.104:0/2687620038 conn(0x7f42240b53e0 msgr2=0x7f42240b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:12.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 -- 172.21.2.104:0/2687620038 shutdown_connections 2021-07-11T03:55:12.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.845+0000 7f422a57c700 1 -- 172.21.2.104:0/2687620038 wait complete. 2021-07-11T03:55:12.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f422a57c700 1 Processor -- start 2021-07-11T03:55:12.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f422a57c700 1 -- start start 2021-07-11T03:55:12.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f422a57c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224067180 0x7f42240ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f422a57c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f42240a76b0 0x7f4224149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f422a57c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42240b1970 0x7f422415a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f422a57c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42240b4e60 con 0x7f4224067180 2021-07-11T03:55:12.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f422a57c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4224159c60 con 0x7f42240a76b0 2021-07-11T03:55:12.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f422a57c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42240aa2e0 con 0x7f42240b1970 2021-07-11T03:55:12.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f4223fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224067180 0x7f42240ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:12.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f4223fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224067180 0x7f42240ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43950/0 (socket says 172.21.2.104:43950) 2021-07-11T03:55:12.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f4223fff700 1 -- 172.21.2.104:0/3042420863 learned_addr learned my addr 172.21.2.104:0/3042420863 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:12.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f4223fff700 1 -- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42240b1970 msgr2=0x7f422415a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f4223fff700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42240b1970 0x7f422415a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:12.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f4223fff700 1 -- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f42240a76b0 msgr2=0x7f4224149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f4223fff700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f42240a76b0 0x7f4224149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:12.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f4223fff700 1 -- 172.21.2.104:0/3042420863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421c009530 con 0x7f4224067180 2021-07-11T03:55:12.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f4223fff700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224067180 0x7f42240ad160 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f42240aaaf0 tx=0x7f420c007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:12.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f42217fa700 1 -- 172.21.2.104:0/3042420863 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420c00ce70 con 0x7f4224067180 2021-07-11T03:55:12.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f42217fa700 1 -- 172.21.2.104:0/3042420863 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c013450 con 0x7f4224067180 2021-07-11T03:55:12.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f422a57c700 1 -- 172.21.2.104:0/3042420863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42240aa580 con 0x7f4224067180 2021-07-11T03:55:12.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f42217fa700 1 -- 172.21.2.104:0/3042420863 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f420c0137b0 con 0x7f4224067180 2021-07-11T03:55:12.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f422a57c700 1 -- 172.21.2.104:0/3042420863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4224156690 con 0x7f4224067180 2021-07-11T03:55:12.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f422a57c700 1 -- 172.21.2.104:0/3042420863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42240aa050 con 0x7f4224067180 2021-07-11T03:55:12.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f42217fa700 1 -- 172.21.2.104:0/3042420863 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f420c01a060 con 0x7f4224067180 2021-07-11T03:55:12.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f42217fa700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f42080532b0 0x7f4208055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:12.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f42217fa700 1 -- 172.21.2.104:0/3042420863 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f420c06ad30 con 0x7f4224067180 2021-07-11T03:55:12.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.849+0000 7f42237fe700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f42080532b0 0x7f4208055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:12.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.853+0000 7f42237fe700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f42080532b0 0x7f4208055770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f4214002f70 tx=0x7f4214006a90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:12.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.853+0000 7f42217fa700 1 -- 172.21.2.104:0/3042420863 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f420c042070 con 0x7f4224067180 2021-07-11T03:55:12.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.933+0000 7f422a57c700 1 -- 172.21.2.104:0/3042420863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4224057350 con 0x7f4224067180 2021-07-11T03:55:12.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f42217fa700 1 -- 172.21.2.104:0/3042420863 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f420c018030 con 0x7f4224067180 2021-07-11T03:55:12.943 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:12.943 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":44,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:55:12.599036+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:55:09.448452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-07-11T03:55:10.579101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:55:12.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 -- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f42080532b0 msgr2=0x7f4208055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f42080532b0 0x7f4208055770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f4214002f70 tx=0x7f4214006a90).stop 2021-07-11T03:55:12.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 -- 172.21.2.104:0/3042420863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224067180 msgr2=0x7f42240ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:12.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224067180 0x7f42240ad160 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f42240aaaf0 tx=0x7f420c007c30).stop 2021-07-11T03:55:12.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 -- 172.21.2.104:0/3042420863 shutdown_connections 2021-07-11T03:55:12.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42240b1970 0x7f422415a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:12.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f42240a76b0 0x7f4224149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:12.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f42080532b0 0x7f4208055770 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 --2- 172.21.2.104:0/3042420863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224067180 0x7f42240ad160 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:12.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 -- 172.21.2.104:0/3042420863 >> 172.21.2.104:0/3042420863 conn(0x7f42240b53e0 msgr2=0x7f4224144740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:12.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 -- 172.21.2.104:0/3042420863 shutdown_connections 2021-07-11T03:55:12.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:12.937+0000 7f422a57c700 1 -- 172.21.2.104:0/3042420863 wait complete. 2021-07-11T03:55:12.950 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T03:55:13.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 Processor -- start 2021-07-11T03:55:13.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 -- start start 2021-07-11T03:55:13.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6d400a76b0 0x7f6d40066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d400b0060 0x7f6d4005fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d400a7a90 con 0x7f6d400b1970 2021-07-11T03:55:13.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d40147620 con 0x7f6d400a76b0 2021-07-11T03:55:13.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d401478c0 con 0x7f6d400b0060 2021-07-11T03:55:13.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3ffff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6d400b1970 0x7f6d4005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40726/0 (socket says 172.21.2.104:40726) 2021-07-11T03:55:13.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3ffff700 1 -- 172.21.2.104:0/2285579350 learned_addr learned my addr 172.21.2.104:0/2285579350 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:13.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2762711025 0 0) 0x7f6d400a7a90 con 0x7f6d400b1970 2021-07-11T03:55:13.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d24003260 con 0x7f6d400b1970 2021-07-11T03:55:13.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d44e77700 1 --2- 172.21.2.104:0/2285579350 >> v2:172.21.2.105:3300/0 conn(0x7f6d400a76b0 0x7f6d40066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d45678700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d400b0060 0x7f6d4005fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 25138895 0 0) 0x7f6d24003260 con 0x7f6d400b1970 2021-07-11T03:55:13.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d400b0060 msgr2=0x7f6d4005fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:13.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d400b0060 0x7f6d4005fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 >> v2:172.21.2.105:3300/0 conn(0x7f6d400a76b0 msgr2=0x7f6d40066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:13.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 --2- 172.21.2.104:0/2285579350 >> v2:172.21.2.105:3300/0 conn(0x7f6d400a76b0 0x7f6d40066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d40147b60 con 0x7f6d400b1970 2021-07-11T03:55:13.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f6d28001680 con 0x7f6d400b1970 2021-07-11T03:55:13.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 >> v1:172.21.2.104:6789/0 conn(0x7f6d400b1970 legacy=0x7f6d4005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d240037d0 0x7f6d24003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d24004500 0x7f6d24004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d24005070 0x7f6d2400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d40147620 con 0x7f6d24005070 2021-07-11T03:55:13.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d401478c0 con 0x7f6d24004500 2021-07-11T03:55:13.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2400bf00 con 0x7f6d240037d0 2021-07-11T03:55:13.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d45678700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d400b0060 0x7f6d4005fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:55:13.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d44e77700 1 --2- 172.21.2.104:0/2285579350 >> v2:172.21.2.105:3300/0 conn(0x7f6d400a76b0 0x7f6d40066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:55:13.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d45678700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d24005070 0x7f6d2400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3ffff700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d24004500 0x7f6d24004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d45678700 1 -- 172.21.2.104:0/2285579350 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d240037d0 msgr2=0x7f6d24003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:13.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d45678700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d240037d0 0x7f6d24003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:13.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d45678700 1 -- 172.21.2.104:0/2285579350 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d24004500 msgr2=0x7f6d24004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:13.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d45678700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d24004500 0x7f6d24004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d45678700 1 -- 172.21.2.104:0/2285579350 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d24003260 con 0x7f6d24005070 2021-07-11T03:55:13.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d45678700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d24005070 0x7f6d2400b750 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f6d38007000 tx=0x7f6d38008040).ready entity=mon.0 client_cookie=18ce1fe8a5ed24f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d38012430 con 0x7f6d24005070 2021-07-11T03:55:13.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3800c030 con 0x7f6d24005070 2021-07-11T03:55:13.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d3f7fe700 1 -- 172.21.2.104:0/2285579350 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d38012780 con 0x7f6d24005070 2021-07-11T03:55:13.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 -- 172.21.2.104:0/2285579350 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d24005070 msgr2=0x7f6d2400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 --2- 172.21.2.104:0/2285579350 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d24005070 0x7f6d2400b750 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f6d38007000 tx=0x7f6d38008040).stop 2021-07-11T03:55:13.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d45678700 1 -- 172.21.2.104:0/2285579350 reap_dead start 2021-07-11T03:55:13.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 -- 172.21.2.104:0/2285579350 shutdown_connections 2021-07-11T03:55:13.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 -- 172.21.2.104:0/2285579350 >> 172.21.2.104:0/2285579350 conn(0x7f6d400b53e0 msgr2=0x7f6d400b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:13.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 -- 172.21.2.104:0/2285579350 shutdown_connections 2021-07-11T03:55:13.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.001+0000 7f6d470db700 1 -- 172.21.2.104:0/2285579350 wait complete. 2021-07-11T03:55:13.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d470db700 1 Processor -- start 2021-07-11T03:55:13.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d470db700 1 -- start start 2021-07-11T03:55:13.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d470db700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d400b42b0 0x7f6d400a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d470db700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d400af630 0x7f6d400a9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d470db700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d400acee0 0x7f6d40159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d470db700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400aa8c0 con 0x7f6d400b42b0 2021-07-11T03:55:13.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d470db700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400aadb0 con 0x7f6d400acee0 2021-07-11T03:55:13.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d470db700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d400b4690 con 0x7f6d400af630 2021-07-11T03:55:13.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d44e77700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d400b42b0 0x7f6d400a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d44e77700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d400b42b0 0x7f6d400a9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43970/0 (socket says 172.21.2.104:43970) 2021-07-11T03:55:13.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d44e77700 1 -- 172.21.2.104:0/1819790689 learned_addr learned my addr 172.21.2.104:0/1819790689 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:13.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d44e77700 1 -- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d400af630 msgr2=0x7f6d400a9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d44e77700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d400af630 0x7f6d400a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:13.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d44e77700 1 -- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d400acee0 msgr2=0x7f6d40159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d44e77700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d400acee0 0x7f6d40159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:13.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d44e77700 1 -- 172.21.2.104:0/1819790689 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d400ae620 con 0x7f6d400b42b0 2021-07-11T03:55:13.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d44e77700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d400b42b0 0x7f6d400a9230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f6d30007000 tx=0x7f6d30008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d3dffb700 1 -- 172.21.2.104:0/1819790689 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d30015070 con 0x7f6d400b42b0 2021-07-11T03:55:13.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d3dffb700 1 -- 172.21.2.104:0/1819790689 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d30010580 con 0x7f6d400b42b0 2021-07-11T03:55:13.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d470db700 1 -- 172.21.2.104:0/1819790689 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d400ae8c0 con 0x7f6d400b42b0 2021-07-11T03:55:13.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d3dffb700 1 -- 172.21.2.104:0/1819790689 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d300108e0 con 0x7f6d400b42b0 2021-07-11T03:55:13.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d470db700 1 -- 172.21.2.104:0/1819790689 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d400aaa30 con 0x7f6d400b42b0 2021-07-11T03:55:13.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d470db700 1 -- 172.21.2.104:0/1819790689 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d40057350 con 0x7f6d400b42b0 2021-07-11T03:55:13.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d3dffb700 1 -- 172.21.2.104:0/1819790689 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6d30010a40 con 0x7f6d400b42b0 2021-07-11T03:55:13.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d3dffb700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d180532d0 0x7f6d18055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d3dffb700 1 -- 172.21.2.104:0/1819790689 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6d3006b8a0 con 0x7f6d400b42b0 2021-07-11T03:55:13.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.005+0000 7f6d3ffff700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d180532d0 0x7f6d18055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.009+0000 7f6d3ffff700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d180532d0 0x7f6d18055790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f6d28008000 tx=0x7f6d28009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.009+0000 7f6d3dffb700 1 -- 172.21.2.104:0/1819790689 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6d30042be0 con 0x7f6d400b42b0 2021-07-11T03:55:13.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.117+0000 7f6d470db700 1 -- 172.21.2.104:0/1819790689 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6d400698e0 con 0x7f6d400b42b0 2021-07-11T03:55:13.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d3dffb700 1 -- 172.21.2.104:0/1819790689 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6d30046200 con 0x7f6d400b42b0 2021-07-11T03:55:13.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 -- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d180532d0 msgr2=0x7f6d18055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d180532d0 0x7f6d18055790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f6d28008000 tx=0x7f6d28009040).stop 2021-07-11T03:55:13.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 -- 172.21.2.104:0/1819790689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d400b42b0 msgr2=0x7f6d400a9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d400b42b0 0x7f6d400a9230 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f6d30007000 tx=0x7f6d30008040).stop 2021-07-11T03:55:13.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 -- 172.21.2.104:0/1819790689 shutdown_connections 2021-07-11T03:55:13.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d400acee0 0x7f6d40159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:13.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d400af630 0x7f6d400a9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:13.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d180532d0 0x7f6d18055790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 --2- 172.21.2.104:0/1819790689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d400b42b0 0x7f6d400a9230 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 -- 172.21.2.104:0/1819790689 >> 172.21.2.104:0/1819790689 conn(0x7f6d400b53e0 msgr2=0x7f6d4014a060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:13.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 -- 172.21.2.104:0/1819790689 shutdown_connections 2021-07-11T03:55:13.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.125+0000 7f6d470db700 1 -- 172.21.2.104:0/1819790689 wait complete. 2021-07-11T03:55:13.137 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T03:55:13.138 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T03:55:13.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 Processor -- start 2021-07-11T03:55:13.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 -- start start 2021-07-11T03:55:13.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ffa340a76b0 0x7ffa34066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffa340b1970 0x7ffa34066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa34067450 con 0x7ffa340b0060 2021-07-11T03:55:13.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa340a7a90 con 0x7ffa340a76b0 2021-07-11T03:55:13.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa34147620 con 0x7ffa340b1970 2021-07-11T03:55:13.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa3259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ffa340a76b0 0x7ffa34066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa3259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ffa340a76b0 0x7ffa34066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43978/0 (socket says 172.21.2.104:43978) 2021-07-11T03:55:13.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa3259c700 1 -- 172.21.2.104:0/2979049589 learned_addr learned my addr 172.21.2.104:0/2979049589 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:13.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa3259c700 1 -- 172.21.2.104:0/2979049589 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffa340b1970 msgr2=0x7ffa34066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:13.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa3259c700 1 --2- 172.21.2.104:0/2979049589 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffa340b1970 0x7ffa34066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:13.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa3259c700 1 -- 172.21.2.104:0/2979049589 >> v1:172.21.2.105:6789/0 conn(0x7ffa340b0060 legacy=0x7ffa3405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:13.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa3259c700 1 -- 172.21.2.104:0/2979049589 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa341478c0 con 0x7ffa340a76b0 2021-07-11T03:55:13.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa3259c700 1 --2- 172.21.2.104:0/2979049589 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffa340a76b0 0x7ffa34066a70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7ffa340b6460 tx=0x7ffa1c006eb0).ready entity=mon.0 client_cookie=b6c17684d3951a19 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa3159a700 1 -- 172.21.2.104:0/2979049589 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa1c00d570 con 0x7ffa340a76b0 2021-07-11T03:55:13.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa3159a700 1 -- 172.21.2.104:0/2979049589 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1c000e90 con 0x7ffa340a76b0 2021-07-11T03:55:13.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa3159a700 1 -- 172.21.2.104:0/2979049589 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa1c00d8c0 con 0x7ffa340a76b0 2021-07-11T03:55:13.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 -- 172.21.2.104:0/2979049589 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffa340a76b0 msgr2=0x7ffa34066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 --2- 172.21.2.104:0/2979049589 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffa340a76b0 0x7ffa34066a70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7ffa340b6460 tx=0x7ffa1c006eb0).stop 2021-07-11T03:55:13.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 -- 172.21.2.104:0/2979049589 shutdown_connections 2021-07-11T03:55:13.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 --2- 172.21.2.104:0/2979049589 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffa340b1970 0x7ffa34066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:13.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 --2- 172.21.2.104:0/2979049589 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffa340a76b0 0x7ffa34066a70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 -- 172.21.2.104:0/2979049589 >> 172.21.2.104:0/2979049589 conn(0x7ffa340b53e0 msgr2=0x7ffa340b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:13.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 -- 172.21.2.104:0/2979049589 shutdown_connections 2021-07-11T03:55:13.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 -- 172.21.2.104:0/2979049589 wait complete. 2021-07-11T03:55:13.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 Processor -- start 2021-07-11T03:55:13.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 -- start start 2021-07-11T03:55:13.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffa340a76b0 0x7ffa340aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffa340b0060 0x7ffa340b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.189+0000 7ffa38a3a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffa340b1970 0x7ffa34144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa38a3a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340b4500 con 0x7ffa340b1970 2021-07-11T03:55:13.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa38a3a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340ab890 con 0x7ffa340a76b0 2021-07-11T03:55:13.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa38a3a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340ab0a0 con 0x7ffa340b0060 2021-07-11T03:55:13.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa32d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffa340b1970 0x7ffa34144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa32d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffa340b1970 0x7ffa34144690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:43984/0 (socket says 172.21.2.104:43984) 2021-07-11T03:55:13.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa32d9d700 1 -- 172.21.2.104:0/423583604 learned_addr learned my addr 172.21.2.104:0/423583604 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:13.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa3259c700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffa340a76b0 0x7ffa340aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa31d9b700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffa340b0060 0x7ffa340b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa32d9d700 1 -- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffa340b0060 msgr2=0x7ffa340b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa32d9d700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffa340b0060 0x7ffa340b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa32d9d700 1 -- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffa340a76b0 msgr2=0x7ffa340aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa32d9d700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffa340a76b0 0x7ffa340aaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa32d9d700 1 -- 172.21.2.104:0/423583604 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c00b1e0 con 0x7ffa340b1970 2021-07-11T03:55:13.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa32d9d700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffa340b1970 0x7ffa34144690 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7ffa2c0012f0 tx=0x7ffa2c0013c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa2b7fe700 1 -- 172.21.2.104:0/423583604 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa2c002490 con 0x7ffa340b1970 2021-07-11T03:55:13.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa2b7fe700 1 -- 172.21.2.104:0/423583604 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2c0025f0 con 0x7ffa340b1970 2021-07-11T03:55:13.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa38a3a700 1 -- 172.21.2.104:0/423583604 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa340abaf0 con 0x7ffa340b1970 2021-07-11T03:55:13.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa2b7fe700 1 -- 172.21.2.104:0/423583604 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffa2c002950 con 0x7ffa340b1970 2021-07-11T03:55:13.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa38a3a700 1 -- 172.21.2.104:0/423583604 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa340aa560 con 0x7ffa340b1970 2021-07-11T03:55:13.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa38a3a700 1 -- 172.21.2.104:0/423583604 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa340abf90 con 0x7ffa340b1970 2021-07-11T03:55:13.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa2b7fe700 1 -- 172.21.2.104:0/423583604 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ffa2c010030 con 0x7ffa340b1970 2021-07-11T03:55:13.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa2b7fe700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffa18053300 0x7ffa180557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa2b7fe700 1 -- 172.21.2.104:0/423583604 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ffa2c06b710 con 0x7ffa340b1970 2021-07-11T03:55:13.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa3259c700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffa18053300 0x7ffa180557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.193+0000 7ffa3259c700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffa18053300 0x7ffa180557c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7ffa1c01b000 tx=0x7ffa1c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.197+0000 7ffa2b7fe700 1 -- 172.21.2.104:0/423583604 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ffa340abf90 con 0x7ffa340b1970 2021-07-11T03:55:13.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.305+0000 7ffa38a3a700 1 -- 172.21.2.104:0/423583604 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7ffa340abf90 con 0x7ffa340b1970 2021-07-11T03:55:13.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa2b7fe700 1 -- 172.21.2.104:0/423583604 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7ffa340abf90 con 0x7ffa340b1970 2021-07-11T03:55:13.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 -- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffa18053300 msgr2=0x7ffa180557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffa18053300 0x7ffa180557c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7ffa1c01b000 tx=0x7ffa1c015000).stop 2021-07-11T03:55:13.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 -- 172.21.2.104:0/423583604 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffa340b1970 msgr2=0x7ffa34144690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffa340b1970 0x7ffa34144690 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7ffa2c0012f0 tx=0x7ffa2c0013c0).stop 2021-07-11T03:55:13.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 -- 172.21.2.104:0/423583604 shutdown_connections 2021-07-11T03:55:13.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffa18053300 0x7ffa180557c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffa340b1970 0x7ffa34144690 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffa340b0060 0x7ffa340b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 --2- 172.21.2.104:0/423583604 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffa340a76b0 0x7ffa340aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 -- 172.21.2.104:0/423583604 >> 172.21.2.104:0/423583604 conn(0x7ffa340b53e0 msgr2=0x7ffa341468b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:13.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 -- 172.21.2.104:0/423583604 shutdown_connections 2021-07-11T03:55:13.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.313+0000 7ffa38a3a700 1 -- 172.21.2.104:0/423583604 wait complete. 2021-07-11T03:55:13.324 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T03:55:13.326 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:55:13.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.369+0000 7f2d5a551700 1 Processor -- start 2021-07-11T03:55:13.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.369+0000 7f2d5a551700 1 -- start start 2021-07-11T03:55:13.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.369+0000 7f2d5a551700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d540b2660 0x7f2d5405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.369+0000 7f2d5a551700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2d54096cc0 0x7f2d54097100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.369+0000 7f2d5a551700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d540b2a40 con 0x7f2d5413f560 2021-07-11T03:55:13.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.369+0000 7f2d5a551700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d540975f0 con 0x7f2d54096cc0 2021-07-11T03:55:13.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.369+0000 7f2d5a551700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d54097b80 con 0x7f2d540b2660 2021-07-11T03:55:13.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d53fff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d540b2660 0x7f2d5405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d58aee700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f2d5413f560 0x7f2d5413f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:40130/0 (socket says 172.21.2.104:40130) 2021-07-11T03:55:13.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d58aee700 1 -- 172.21.2.104:0/4027091985 learned_addr learned my addr 172.21.2.104:0/4027091985 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:13.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d537fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2d54096cc0 0x7f2d54097100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d537fe700 1 -- 172.21.2.104:0/4027091985 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d540b2660 msgr2=0x7f2d5405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:13.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d537fe700 1 --2- 172.21.2.104:0/4027091985 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d540b2660 0x7f2d5405f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d537fe700 1 -- 172.21.2.104:0/4027091985 >> v1:172.21.2.105:6789/0 conn(0x7f2d5413f560 legacy=0x7f2d5413f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d537fe700 1 -- 172.21.2.104:0/4027091985 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d54097e20 con 0x7f2d54096cc0 2021-07-11T03:55:13.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d537fe700 1 --2- 172.21.2.104:0/4027091985 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d54096cc0 0x7f2d54097100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f2d44002f70 tx=0x7f2d44006df0).ready entity=mon.1 client_cookie=b65ac5b8a96d9ab8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d52ffd700 1 -- 172.21.2.104:0/4027091985 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d4400d570 con 0x7f2d54096cc0 2021-07-11T03:55:13.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d52ffd700 1 -- 172.21.2.104:0/4027091985 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d44000e90 con 0x7f2d54096cc0 2021-07-11T03:55:13.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d52ffd700 1 -- 172.21.2.104:0/4027091985 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d4400d8c0 con 0x7f2d54096cc0 2021-07-11T03:55:13.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 -- 172.21.2.104:0/4027091985 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d54096cc0 msgr2=0x7f2d54097100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 --2- 172.21.2.104:0/4027091985 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d54096cc0 0x7f2d54097100 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f2d44002f70 tx=0x7f2d44006df0).stop 2021-07-11T03:55:13.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 -- 172.21.2.104:0/4027091985 shutdown_connections 2021-07-11T03:55:13.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 --2- 172.21.2.104:0/4027091985 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d54096cc0 0x7f2d54097100 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 --2- 172.21.2.104:0/4027091985 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d540b2660 0x7f2d5405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 -- 172.21.2.104:0/4027091985 >> 172.21.2.104:0/4027091985 conn(0x7f2d54062f10 msgr2=0x7f2d540632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:13.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 -- 172.21.2.104:0/4027091985 shutdown_connections 2021-07-11T03:55:13.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 -- 172.21.2.104:0/4027091985 wait complete. 2021-07-11T03:55:13.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 Processor -- start 2021-07-11T03:55:13.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 -- start start 2021-07-11T03:55:13.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d54096cc0 0x7f2d540abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d540b2660 0x7f2d54066450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d5413f560 0x7f2d54063ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d540662e0 con 0x7f2d540b2660 2021-07-11T03:55:13.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d54066940 con 0x7f2d5413f560 2021-07-11T03:55:13.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d5a551700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d54064f40 con 0x7f2d54096cc0 2021-07-11T03:55:13.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d537fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d540b2660 0x7f2d54066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d537fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d540b2660 0x7f2d54066450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44000/0 (socket says 172.21.2.104:44000) 2021-07-11T03:55:13.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d537fe700 1 -- 172.21.2.104:0/3844249866 learned_addr learned my addr 172.21.2.104:0/3844249866 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:13.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d53fff700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d54096cc0 0x7f2d540abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.373+0000 7f2d58aee700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d5413f560 0x7f2d54063ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d537fe700 1 -- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d54096cc0 msgr2=0x7f2d540abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d537fe700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d54096cc0 0x7f2d540abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d537fe700 1 -- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d5413f560 msgr2=0x7f2d54063ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d537fe700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d5413f560 0x7f2d54063ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d537fe700 1 -- 172.21.2.104:0/3844249866 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4400b180 con 0x7f2d540b2660 2021-07-11T03:55:13.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d58aee700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d5413f560 0x7f2d54063ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:55:13.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d537fe700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d540b2660 0x7f2d54066450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f2d44002f70 tx=0x7f2d44014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d517fa700 1 -- 172.21.2.104:0/3844249866 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d44006b20 con 0x7f2d540b2660 2021-07-11T03:55:13.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d517fa700 1 -- 172.21.2.104:0/3844249866 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4400de40 con 0x7f2d540b2660 2021-07-11T03:55:13.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d5a551700 1 -- 172.21.2.104:0/3844249866 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5409ab80 con 0x7f2d540b2660 2021-07-11T03:55:13.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d517fa700 1 -- 172.21.2.104:0/3844249866 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d4401e620 con 0x7f2d540b2660 2021-07-11T03:55:13.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d5a551700 1 -- 172.21.2.104:0/3844249866 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5409b0c0 con 0x7f2d540b2660 2021-07-11T03:55:13.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d5a551700 1 -- 172.21.2.104:0/3844249866 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5409b280 con 0x7f2d540b2660 2021-07-11T03:55:13.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d517fa700 1 -- 172.21.2.104:0/3844249866 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2d4400f030 con 0x7f2d540b2660 2021-07-11T03:55:13.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d517fa700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d38053380 0x7f2d38055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d517fa700 1 -- 172.21.2.104:0/3844249866 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2d4406c450 con 0x7f2d540b2660 2021-07-11T03:55:13.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d53fff700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d38053380 0x7f2d38055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.377+0000 7f2d53fff700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d38053380 0x7f2d38055840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f2d3c008000 tx=0x7f2d3c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.381+0000 7f2d517fa700 1 -- 172.21.2.104:0/3844249866 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2d44043680 con 0x7f2d540b2660 2021-07-11T03:55:13.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d5a551700 1 -- 172.21.2.104:0/3844249866 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d54057350 con 0x7f2d540b2660 2021-07-11T03:55:13.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d517fa700 1 -- 172.21.2.104:0/3844249866 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f2d44046ca0 con 0x7f2d540b2660 2021-07-11T03:55:13.493 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:13.493 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4492,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2412138431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2412138431},{"type":"v1","addr":"172.21.2.105:6835","nonce":2412138431}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/3271417890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3271417890},{"type":"v1","addr":"172.21.2.104:6834","nonce":3271417890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:55:11.595553+0000","modified":"2021-07-11T03:55:12.600491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/795410455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":795410455},{"type":"v1","addr":"172.21.2.104:6835","nonce":795410455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-07-11T03:55:13.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d5a551700 1 -- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d38053380 msgr2=0x7f2d38055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d5a551700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d38053380 0x7f2d38055840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f2d3c008000 tx=0x7f2d3c008040).stop 2021-07-11T03:55:13.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d5a551700 1 -- 172.21.2.104:0/3844249866 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d540b2660 msgr2=0x7f2d54066450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d5a551700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d540b2660 0x7f2d54066450 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f2d44002f70 tx=0x7f2d44014000).stop 2021-07-11T03:55:13.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d5a551700 1 -- 172.21.2.104:0/3844249866 shutdown_connections 2021-07-11T03:55:13.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d5a551700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d5413f560 0x7f2d54063ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d5a551700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d38053380 0x7f2d38055840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d5a551700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d540b2660 0x7f2d54066450 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d5a551700 1 --2- 172.21.2.104:0/3844249866 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d54096cc0 0x7f2d540abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.485+0000 7f2d5a551700 1 -- 172.21.2.104:0/3844249866 >> 172.21.2.104:0/3844249866 conn(0x7f2d54062f10 msgr2=0x7f2d540b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:13.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.489+0000 7f2d5a551700 1 -- 172.21.2.104:0/3844249866 shutdown_connections 2021-07-11T03:55:13.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.489+0000 7f2d5a551700 1 -- 172.21.2.104:0/3844249866 wait complete. 2021-07-11T03:55:13.500 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 16 2021-07-11T03:55:13.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:55:11.595553+0000', 'modified': '2021-07-11T03:55:12.600491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5232}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5232': {'gid': 5232, 'name': 'c', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/795410455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 795410455}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 795410455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:55:13.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:55:13.807 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:55:13.871 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":5232,"want_state":"up:active","state":"up:active","rank_uptime":2.2668157920000001,"mdsmap_epoch":16,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":6.4251336080000003} 2021-07-11T03:55:13.871 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 5232, "mdsmap_epoch": 16, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 2.266815792, "state": "up:active", "uptime": 6.425133608, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:55:13.872 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:55:13.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a97cd5700 1 Processor -- start 2021-07-11T03:55:13.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a97cd5700 1 -- start start 2021-07-11T03:55:13.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a97cd5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3a900a76b0 0x7f3a90066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a97cd5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a900b0060 0x7f3a9005fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a97cd5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a900a7a90 con 0x7f3a900b1970 2021-07-11T03:55:13.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a97cd5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a90147620 con 0x7f3a900a76b0 2021-07-11T03:55:13.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a97cd5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a901478c0 con 0x7f3a900b0060 2021-07-11T03:55:13.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a95270700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f3a900b1970 0x7f3a9005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40778/0 (socket says 172.21.2.104:40778) 2021-07-11T03:55:13.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a95270700 1 -- 172.21.2.104:0/1181393101 learned_addr learned my addr 172.21.2.104:0/1181393101 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:13.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a95a71700 1 --2- 172.21.2.104:0/1181393101 >> v2:172.21.2.104:3300/0 conn(0x7f3a900a76b0 0x7f3a90066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a94a6f700 1 -- 172.21.2.104:0/1181393101 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917837926 0 0) 0x7f3a900a7a90 con 0x7f3a900b1970 2021-07-11T03:55:13.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a94a6f700 1 -- 172.21.2.104:0/1181393101 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a78003260 con 0x7f3a900b1970 2021-07-11T03:55:13.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a95a71700 1 -- 172.21.2.104:0/1181393101 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a900b0060 msgr2=0x7f3a9005fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:13.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a95a71700 1 --2- 172.21.2.104:0/1181393101 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a900b0060 0x7f3a9005fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:13.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a95a71700 1 -- 172.21.2.104:0/1181393101 >> v1:172.21.2.104:6789/0 conn(0x7f3a900b1970 legacy=0x7f3a9005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a95a71700 1 -- 172.21.2.104:0/1181393101 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a90147b60 con 0x7f3a900a76b0 2021-07-11T03:55:13.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a95a71700 1 --2- 172.21.2.104:0/1181393101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a900a76b0 0x7f3a90066a70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f3a80002f70 tx=0x7f3a80006df0).ready entity=mon.0 client_cookie=cb48e97c129152da server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a94a6f700 1 -- 172.21.2.104:0/1181393101 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a8000d570 con 0x7f3a900a76b0 2021-07-11T03:55:13.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.917+0000 7f3a94a6f700 1 -- 172.21.2.104:0/1181393101 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a80000e90 con 0x7f3a900a76b0 2021-07-11T03:55:13.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a94a6f700 1 -- 172.21.2.104:0/1181393101 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a8000d8c0 con 0x7f3a900a76b0 2021-07-11T03:55:13.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- 172.21.2.104:0/1181393101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a900a76b0 msgr2=0x7f3a90066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 --2- 172.21.2.104:0/1181393101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a900a76b0 0x7f3a90066a70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f3a80002f70 tx=0x7f3a80006df0).stop 2021-07-11T03:55:13.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- 172.21.2.104:0/1181393101 shutdown_connections 2021-07-11T03:55:13.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 --2- 172.21.2.104:0/1181393101 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a900b0060 0x7f3a9005fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:13.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 --2- 172.21.2.104:0/1181393101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a900a76b0 0x7f3a90066a70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:13.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- 172.21.2.104:0/1181393101 >> 172.21.2.104:0/1181393101 conn(0x7f3a900b53e0 msgr2=0x7f3a900b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:13.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- 172.21.2.104:0/1181393101 shutdown_connections 2021-07-11T03:55:13.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- 172.21.2.104:0/1181393101 wait complete. 2021-07-11T03:55:13.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 Processor -- start 2021-07-11T03:55:13.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- start start 2021-07-11T03:55:13.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a900a76b0 0x7f3a900ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a900b0060 0x7f3a900aab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a900b1970 0x7f3a900abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9014ce90 con 0x7f3a900b0060 2021-07-11T03:55:13.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9014d130 con 0x7f3a900b1970 2021-07-11T03:55:13.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9014d3d0 con 0x7f3a900a76b0 2021-07-11T03:55:13.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a95270700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a900b0060 0x7f3a900aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a95270700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a900b0060 0x7f3a900aab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44014/0 (socket says 172.21.2.104:44014) 2021-07-11T03:55:13.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a95270700 1 -- 172.21.2.104:0/4122409542 learned_addr learned my addr 172.21.2.104:0/4122409542 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:13.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a95270700 1 -- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a900a76b0 msgr2=0x7f3a900ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a95270700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a900a76b0 0x7f3a900ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:13.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a95270700 1 -- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a900b1970 msgr2=0x7f3a900abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:13.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a95270700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a900b1970 0x7f3a900abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:13.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a95270700 1 -- 172.21.2.104:0/4122409542 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8000b180 con 0x7f3a900b0060 2021-07-11T03:55:13.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a95270700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a900b0060 0x7f3a900aab20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f3a8c001110 tx=0x7f3a8c007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a7effd700 1 -- 172.21.2.104:0/4122409542 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a8c0070d0 con 0x7f3a900b0060 2021-07-11T03:55:13.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a7effd700 1 -- 172.21.2.104:0/4122409542 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c007230 con 0x7f3a900b0060 2021-07-11T03:55:13.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a7effd700 1 -- 172.21.2.104:0/4122409542 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a8c005610 con 0x7f3a900b0060 2021-07-11T03:55:13.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- 172.21.2.104:0/4122409542 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a900ab800 con 0x7f3a900b0060 2021-07-11T03:55:13.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- 172.21.2.104:0/4122409542 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a900ac1b0 con 0x7f3a900b0060 2021-07-11T03:55:13.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a97cd5700 1 -- 172.21.2.104:0/4122409542 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a90057350 con 0x7f3a900b0060 2021-07-11T03:55:13.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a7effd700 1 -- 172.21.2.104:0/4122409542 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3a8c010070 con 0x7f3a900b0060 2021-07-11T03:55:13.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.921+0000 7f3a7effd700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053270 0x7f3a70055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:13.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.925+0000 7f3a7effd700 1 -- 172.21.2.104:0/4122409542 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3a8c06abd0 con 0x7f3a900b0060 2021-07-11T03:55:13.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.925+0000 7f3a95a71700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053270 0x7f3a70055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:13.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.925+0000 7f3a95a71700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053270 0x7f3a70055730 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f3a80002f70 tx=0x7f3a80014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:13.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:13.925+0000 7f3a7effd700 1 -- 172.21.2.104:0/4122409542 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3a8c041f10 con 0x7f3a900b0060 2021-07-11T03:55:14.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.005+0000 7f3a97cd5700 1 -- 172.21.2.104:0/4122409542 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a900698e0 con 0x7f3a900b0060 2021-07-11T03:55:14.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a7effd700 1 -- 172.21.2.104:0/4122409542 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f3a8c045530 con 0x7f3a900b0060 2021-07-11T03:55:14.014 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:14.014 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":44,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:55:12.599036+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:55:09.448452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-07-11T03:55:10.579101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:55:14.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 -- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053270 msgr2=0x7f3a70055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053270 0x7f3a70055730 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f3a80002f70 tx=0x7f3a80014000).stop 2021-07-11T03:55:14.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 -- 172.21.2.104:0/4122409542 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a900b0060 msgr2=0x7f3a900aab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a900b0060 0x7f3a900aab20 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f3a8c001110 tx=0x7f3a8c007a80).stop 2021-07-11T03:55:14.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 -- 172.21.2.104:0/4122409542 shutdown_connections 2021-07-11T03:55:14.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a900b1970 0x7f3a900abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053270 0x7f3a70055730 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a900b0060 0x7f3a900aab20 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 --2- 172.21.2.104:0/4122409542 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a900a76b0 0x7f3a900ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 -- 172.21.2.104:0/4122409542 >> 172.21.2.104:0/4122409542 conn(0x7f3a900b53e0 msgr2=0x7f3a9005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:14.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 -- 172.21.2.104:0/4122409542 shutdown_connections 2021-07-11T03:55:14.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.009+0000 7f3a97cd5700 1 -- 172.21.2.104:0/4122409542 wait complete. 2021-07-11T03:55:14.022 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:55:09.448452+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:55:10.579101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:55:14.023 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:55:14.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f95949ca700 1 Processor -- start 2021-07-11T03:55:14.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f95949ca700 1 -- start start 2021-07-11T03:55:14.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f95949ca700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f95900b1ec0 0x7f95900b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f95949ca700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95900b0420 con 0x7f95900b22a0 2021-07-11T03:55:14.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f95949ca700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95900a77d0 con 0x7f95900affb0 2021-07-11T03:55:14.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f95949ca700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95900b0650 con 0x7f95900b1ec0 2021-07-11T03:55:14.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f958ed9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f95900b22a0 0x7f9590141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40792/0 (socket says 172.21.2.104:40792) 2021-07-11T03:55:14.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f958ed9d700 1 -- 172.21.2.104:0/2382460872 learned_addr learned my addr 172.21.2.104:0/2382460872 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:14.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f958e59c700 1 --2- 172.21.2.104:0/2382460872 >> v2:172.21.2.104:3300/0 conn(0x7f95900b1ec0 0x7f95900b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f958e59c700 1 -- 172.21.2.104:0/2382460872 >> v1:172.21.2.105:6789/0 conn(0x7f95900affb0 legacy=0x7f95900a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f958e59c700 1 -- 172.21.2.104:0/2382460872 >> v1:172.21.2.104:6789/0 conn(0x7f95900b22a0 legacy=0x7f9590141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f958e59c700 1 -- 172.21.2.104:0/2382460872 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900b08f0 con 0x7f95900b1ec0 2021-07-11T03:55:14.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f958e59c700 1 --2- 172.21.2.104:0/2382460872 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95900b1ec0 0x7f95900b12d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f95900b2f50 tx=0x7f9584006df0).ready entity=mon.0 client_cookie=f4d2a8fc9161dbc0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f958d59a700 1 -- 172.21.2.104:0/2382460872 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f958400d5c0 con 0x7f95900b1ec0 2021-07-11T03:55:14.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f958d59a700 1 -- 172.21.2.104:0/2382460872 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9584000e90 con 0x7f95900b1ec0 2021-07-11T03:55:14.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f958d59a700 1 -- 172.21.2.104:0/2382460872 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f958400d910 con 0x7f95900b1ec0 2021-07-11T03:55:14.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f95949ca700 1 -- 172.21.2.104:0/2382460872 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95900b1ec0 msgr2=0x7f95900b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.069+0000 7f95949ca700 1 --2- 172.21.2.104:0/2382460872 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95900b1ec0 0x7f95900b12d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f95900b2f50 tx=0x7f9584006df0).stop 2021-07-11T03:55:14.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 -- 172.21.2.104:0/2382460872 shutdown_connections 2021-07-11T03:55:14.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 --2- 172.21.2.104:0/2382460872 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95900b1ec0 0x7f95900b12d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 -- 172.21.2.104:0/2382460872 >> 172.21.2.104:0/2382460872 conn(0x7f959005f6b0 msgr2=0x7f959005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:14.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 -- 172.21.2.104:0/2382460872 shutdown_connections 2021-07-11T03:55:14.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 -- 172.21.2.104:0/2382460872 wait complete. 2021-07-11T03:55:14.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 Processor -- start 2021-07-11T03:55:14.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 -- start start 2021-07-11T03:55:14.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95900affb0 0x7f9590140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95900b1ec0 0x7f959015a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95900b22a0 0x7f959015a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9590145080 con 0x7f95900b22a0 2021-07-11T03:55:14.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9590145320 con 0x7f95900b1ec0 2021-07-11T03:55:14.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95900a9420 con 0x7f95900affb0 2021-07-11T03:55:14.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f958ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95900b22a0 0x7f959015a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f958ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95900b22a0 0x7f959015a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44028/0 (socket says 172.21.2.104:44028) 2021-07-11T03:55:14.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f958ed9d700 1 -- 172.21.2.104:0/4077693058 learned_addr learned my addr 172.21.2.104:0/4077693058 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:14.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f958ed9d700 1 -- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95900affb0 msgr2=0x7f9590140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f958ed9d700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95900affb0 0x7f9590140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f958ed9d700 1 -- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95900b1ec0 msgr2=0x7f959015a430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:55:14.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f958ed9d700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95900b1ec0 0x7f959015a430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f958ed9d700 1 -- 172.21.2.104:0/4077693058 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958400b180 con 0x7f95900b22a0 2021-07-11T03:55:14.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f958ed9d700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95900b22a0 0x7f959015a920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f95900ad2d0 tx=0x7f9588007f60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f957f7fe700 1 -- 172.21.2.104:0/4077693058 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9588005ba0 con 0x7f95900b22a0 2021-07-11T03:55:14.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 -- 172.21.2.104:0/4077693058 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95900a96c0 con 0x7f95900b22a0 2021-07-11T03:55:14.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f957f7fe700 1 -- 172.21.2.104:0/4077693058 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9588005d00 con 0x7f95900b22a0 2021-07-11T03:55:14.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f957f7fe700 1 -- 172.21.2.104:0/4077693058 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95880072c0 con 0x7f95900b22a0 2021-07-11T03:55:14.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 -- 172.21.2.104:0/4077693058 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95900a9b70 con 0x7f95900b22a0 2021-07-11T03:55:14.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f95949ca700 1 -- 172.21.2.104:0/4077693058 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95900a9850 con 0x7f95900b22a0 2021-07-11T03:55:14.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f957f7fe700 1 -- 172.21.2.104:0/4077693058 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9588010070 con 0x7f95900b22a0 2021-07-11T03:55:14.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f957f7fe700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f956c0532b0 0x7f956c055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.073+0000 7f957f7fe700 1 -- 172.21.2.104:0/4077693058 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f958806ade0 con 0x7f95900b22a0 2021-07-11T03:55:14.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.077+0000 7f958e59c700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f956c0532b0 0x7f956c055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.077+0000 7f958e59c700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f956c0532b0 0x7f956c055770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f95900b2f50 tx=0x7f9584012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.077+0000 7f957f7fe700 1 -- 172.21.2.104:0/4077693058 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9588042080 con 0x7f95900b22a0 2021-07-11T03:55:14.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.157+0000 7f95949ca700 1 -- 172.21.2.104:0/4077693058 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9590057350 con 0x7f95900b22a0 2021-07-11T03:55:14.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.157+0000 7f957f7fe700 1 -- 172.21.2.104:0/4077693058 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9588018030 con 0x7f95900b22a0 2021-07-11T03:55:14.165 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:55:14.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 -- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f956c0532b0 msgr2=0x7f956c055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f956c0532b0 0x7f956c055770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f95900b2f50 tx=0x7f9584012000).stop 2021-07-11T03:55:14.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 -- 172.21.2.104:0/4077693058 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95900b22a0 msgr2=0x7f959015a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95900b22a0 0x7f959015a920 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f95900ad2d0 tx=0x7f9588007f60).stop 2021-07-11T03:55:14.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 -- 172.21.2.104:0/4077693058 shutdown_connections 2021-07-11T03:55:14.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f956c0532b0 0x7f956c055770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95900b22a0 0x7f959015a920 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95900b1ec0 0x7f959015a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 --2- 172.21.2.104:0/4077693058 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95900affb0 0x7f9590140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 -- 172.21.2.104:0/4077693058 >> 172.21.2.104:0/4077693058 conn(0x7f959005f6b0 msgr2=0x7f9590146190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:14.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 -- 172.21.2.104:0/4077693058 shutdown_connections 2021-07-11T03:55:14.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.161+0000 7f95949ca700 1 -- 172.21.2.104:0/4077693058 wait complete. 2021-07-11T03:55:14.172 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:55:14.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.217+0000 7f3a738e6700 1 Processor -- start 2021-07-11T03:55:14.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.217+0000 7f3a738e6700 1 -- start start 2021-07-11T03:55:14.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a6c0b10c0 0x7f3a6c0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3a6c140ba0 0x7f3a6c0b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6c0ac7c0 con 0x7f3a6c0acb70 2021-07-11T03:55:14.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6c0ac020 con 0x7f3a6c140ba0 2021-07-11T03:55:14.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c141010 con 0x7f3a6c0b10c0 2021-07-11T03:55:14.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3a6c140ba0 0x7f3a6c0b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3a6c140ba0 0x7f3a6c0b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44034/0 (socket says 172.21.2.104:44034) 2021-07-11T03:55:14.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 -- 172.21.2.104:0/345764164 learned_addr learned my addr 172.21.2.104:0/345764164 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:14.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 -- 172.21.2.104:0/345764164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a6c0b10c0 msgr2=0x7f3a6c0b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 --2- 172.21.2.104:0/345764164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a6c0b10c0 0x7f3a6c0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 -- 172.21.2.104:0/345764164 >> v1:172.21.2.105:6789/0 conn(0x7f3a6c0acb70 legacy=0x7f3a6c13d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 -- 172.21.2.104:0/345764164 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c0ac280 con 0x7f3a6c140ba0 2021-07-11T03:55:14.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 --2- 172.21.2.104:0/345764164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c140ba0 0x7f3a6c0b24e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f3a6c140090 tx=0x7f3a64009290).ready entity=mon.0 client_cookie=f0199df0033d3fcb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a5bfff700 1 -- 172.21.2.104:0/345764164 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a6400b860 con 0x7f3a6c140ba0 2021-07-11T03:55:14.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a5bfff700 1 -- 172.21.2.104:0/345764164 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6400b9c0 con 0x7f3a6c140ba0 2021-07-11T03:55:14.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a5bfff700 1 -- 172.21.2.104:0/345764164 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a6400bb70 con 0x7f3a6c140ba0 2021-07-11T03:55:14.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- 172.21.2.104:0/345764164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c140ba0 msgr2=0x7f3a6c0b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 --2- 172.21.2.104:0/345764164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c140ba0 0x7f3a6c0b24e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f3a6c140090 tx=0x7f3a64009290).stop 2021-07-11T03:55:14.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- 172.21.2.104:0/345764164 shutdown_connections 2021-07-11T03:55:14.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 --2- 172.21.2.104:0/345764164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c140ba0 0x7f3a6c0b24e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 --2- 172.21.2.104:0/345764164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a6c0b10c0 0x7f3a6c0b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- 172.21.2.104:0/345764164 >> 172.21.2.104:0/345764164 conn(0x7f3a6c05f6b0 msgr2=0x7f3a6c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:14.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- 172.21.2.104:0/345764164 shutdown_connections 2021-07-11T03:55:14.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- 172.21.2.104:0/345764164 wait complete. 2021-07-11T03:55:14.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 Processor -- start 2021-07-11T03:55:14.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- start start 2021-07-11T03:55:14.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a6c0acb70 0x7f3a6c143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a6c0b10c0 0x7f3a6c063920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c140ba0 0x7f3a6c063e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c064300 con 0x7f3a6c140ba0 2021-07-11T03:55:14.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c144080 con 0x7f3a6c0b10c0 2021-07-11T03:55:14.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c141c60 con 0x7f3a6c0acb70 2021-07-11T03:55:14.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c140ba0 0x7f3a6c063e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c140ba0 0x7f3a6c063e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44044/0 (socket says 172.21.2.104:44044) 2021-07-11T03:55:14.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 -- 172.21.2.104:0/3789642280 learned_addr learned my addr 172.21.2.104:0/3789642280 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:14.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 -- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a6c0acb70 msgr2=0x7f3a6c143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a6c0acb70 0x7f3a6c143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 -- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a6c0b10c0 msgr2=0x7f3a6c063920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a6c0b10c0 0x7f3a6c063920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 -- 172.21.2.104:0/3789642280 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a64009530 con 0x7f3a6c140ba0 2021-07-11T03:55:14.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a71e83700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c140ba0 0x7f3a6c063e10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f3a6c140090 tx=0x7f3a64016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a5a7fc700 1 -- 172.21.2.104:0/3789642280 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a64011cc0 con 0x7f3a6c140ba0 2021-07-11T03:55:14.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a5a7fc700 1 -- 172.21.2.104:0/3789642280 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a64011e20 con 0x7f3a6c140ba0 2021-07-11T03:55:14.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a738e6700 1 -- 172.21.2.104:0/3789642280 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c141f00 con 0x7f3a6c140ba0 2021-07-11T03:55:14.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.221+0000 7f3a5a7fc700 1 -- 172.21.2.104:0/3789642280 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a6401f610 con 0x7f3a6c140ba0 2021-07-11T03:55:14.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.225+0000 7f3a738e6700 1 -- 172.21.2.104:0/3789642280 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a6c142640 con 0x7f3a6c140ba0 2021-07-11T03:55:14.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.225+0000 7f3a738e6700 1 -- 172.21.2.104:0/3789642280 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6c1423a0 con 0x7f3a6c140ba0 2021-07-11T03:55:14.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.225+0000 7f3a5a7fc700 1 -- 172.21.2.104:0/3789642280 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3a6400f070 con 0x7f3a6c140ba0 2021-07-11T03:55:14.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.225+0000 7f3a5a7fc700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a4c0531f0 0x7f3a4c0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.225+0000 7f3a5a7fc700 1 -- 172.21.2.104:0/3789642280 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f3a6406be10 con 0x7f3a6c140ba0 2021-07-11T03:55:14.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.225+0000 7f3a71682700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a4c0531f0 0x7f3a4c0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.225+0000 7f3a71682700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a4c0531f0 0x7f3a4c0556b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f3a68002f70 tx=0x7f3a68006a90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.225+0000 7f3a5a7fc700 1 -- 172.21.2.104:0/3789642280 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3a6c1423a0 con 0x7f3a6c140ba0 2021-07-11T03:55:14.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 -- 172.21.2.104:0/3789642280 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3a6c1423a0 con 0x7f3a6c140ba0 2021-07-11T03:55:14.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a5a7fc700 1 -- 172.21.2.104:0/3789642280 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3a6c1423a0 con 0x7f3a6c140ba0 2021-07-11T03:55:14.316 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:55:14.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 -- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a4c0531f0 msgr2=0x7f3a4c0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a4c0531f0 0x7f3a4c0556b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f3a68002f70 tx=0x7f3a68006a90).stop 2021-07-11T03:55:14.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 -- 172.21.2.104:0/3789642280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c140ba0 msgr2=0x7f3a6c063e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c140ba0 0x7f3a6c063e10 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f3a6c140090 tx=0x7f3a64016000).stop 2021-07-11T03:55:14.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 -- 172.21.2.104:0/3789642280 shutdown_connections 2021-07-11T03:55:14.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a4c0531f0 0x7f3a4c0556b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c140ba0 0x7f3a6c063e10 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a6c0b10c0 0x7f3a6c063920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 --2- 172.21.2.104:0/3789642280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a6c0acb70 0x7f3a6c143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 -- 172.21.2.104:0/3789642280 >> 172.21.2.104:0/3789642280 conn(0x7f3a6c05f6b0 msgr2=0x7f3a6c0afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:14.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 -- 172.21.2.104:0/3789642280 shutdown_connections 2021-07-11T03:55:14.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.309+0000 7f3a738e6700 1 -- 172.21.2.104:0/3789642280 wait complete. 2021-07-11T03:55:14.323 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:55:14.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.369+0000 7fe184957700 1 Processor -- start 2021-07-11T03:55:14.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.369+0000 7fe184957700 1 -- start start 2021-07-11T03:55:14.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.369+0000 7fe184957700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe180060100 0x7fe180064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.369+0000 7fe184957700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe180144440 0x7fe180065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.369+0000 7fe184957700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1800604e0 con 0x7fe1800b38d0 2021-07-11T03:55:14.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.369+0000 7fe184957700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1800656f0 con 0x7fe180060100 2021-07-11T03:55:14.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.369+0000 7fe184957700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe180065c80 con 0x7fe180144440 2021-07-11T03:55:14.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.369+0000 7fe17ed9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe1800b38d0 0x7fe180144970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40818/0 (socket says 172.21.2.104:40818) 2021-07-11T03:55:14.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.369+0000 7fe17ed9d700 1 -- 172.21.2.104:0/229939476 learned_addr learned my addr 172.21.2.104:0/229939476 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:14.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847607501 0 0) 0x7fe1800604e0 con 0x7fe1800b38d0 2021-07-11T03:55:14.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe160003260 con 0x7fe1800b38d0 2021-07-11T03:55:14.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3798259022 0 0) 0x7fe160003260 con 0x7fe1800b38d0 2021-07-11T03:55:14.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe180144440 msgr2=0x7fe180065200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 --2- 172.21.2.104:0/229939476 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe180144440 0x7fe180065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 >> v2:172.21.2.105:3300/0 conn(0x7fe180060100 msgr2=0x7fe180064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 --2- 172.21.2.104:0/229939476 >> v2:172.21.2.105:3300/0 conn(0x7fe180060100 0x7fe180064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800b23b0 con 0x7fe1800b38d0 2021-07-11T03:55:14.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fe1780019e0 con 0x7fe1800b38d0 2021-07-11T03:55:14.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 >> v1:172.21.2.104:6789/0 conn(0x7fe1800b38d0 legacy=0x7fe180144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 --2- 172.21.2.104:0/229939476 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe1600037d0 0x7fe160003bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 --2- 172.21.2.104:0/229939476 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe160004500 0x7fe160004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 --2- 172.21.2.104:0/229939476 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe160004f90 0x7fe16000fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1800656f0 con 0x7fe160004f90 2021-07-11T03:55:14.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe180065c80 con 0x7fe160004500 2021-07-11T03:55:14.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe160010230 con 0x7fe1600037d0 2021-07-11T03:55:14.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17ed9d700 1 --2- 172.21.2.104:0/229939476 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe160004f90 0x7fe16000fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17ed9d700 1 -- 172.21.2.104:0/229939476 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe1600037d0 msgr2=0x7fe160003bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17ed9d700 1 --2- 172.21.2.104:0/229939476 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe1600037d0 0x7fe160003bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17ed9d700 1 -- 172.21.2.104:0/229939476 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe160004500 msgr2=0x7fe160004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17ed9d700 1 --2- 172.21.2.104:0/229939476 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe160004500 0x7fe160004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17ed9d700 1 -- 172.21.2.104:0/229939476 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800b23b0 con 0x7fe160004f90 2021-07-11T03:55:14.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17ed9d700 1 --2- 172.21.2.104:0/229939476 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe160004f90 0x7fe16000fb60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fe1780016f0 tx=0x7fe1780079b0).ready entity=mon.0 client_cookie=3deed951204048e3 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe17800ccb0 con 0x7fe160004f90 2021-07-11T03:55:14.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17800ce10 con 0x7fe160004f90 2021-07-11T03:55:14.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17d59a700 1 -- 172.21.2.104:0/229939476 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe178005b80 con 0x7fe160004f90 2021-07-11T03:55:14.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 -- 172.21.2.104:0/229939476 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe160004f90 msgr2=0x7fe16000fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 --2- 172.21.2.104:0/229939476 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe160004f90 0x7fe16000fb60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7fe1780016f0 tx=0x7fe1780079b0).stop 2021-07-11T03:55:14.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17ed9d700 1 -- 172.21.2.104:0/229939476 reap_dead start 2021-07-11T03:55:14.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 -- 172.21.2.104:0/229939476 shutdown_connections 2021-07-11T03:55:14.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 -- 172.21.2.104:0/229939476 >> 172.21.2.104:0/229939476 conn(0x7fe18005cf60 msgr2=0x7fe18005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:14.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 -- 172.21.2.104:0/229939476 shutdown_connections 2021-07-11T03:55:14.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 -- 172.21.2.104:0/229939476 wait complete. 2021-07-11T03:55:14.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 Processor -- start 2021-07-11T03:55:14.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 -- start start 2021-07-11T03:55:14.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe18013e190 0x7fe18013a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe18013f420 0x7fe18013acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe18014dea0 0x7fe18014e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1801520a0 con 0x7fe18013e190 2021-07-11T03:55:14.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18013e570 con 0x7fe18014dea0 2021-07-11T03:55:14.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1800b1770 con 0x7fe18013f420 2021-07-11T03:55:14.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe18013e190 0x7fe18013a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe18013e190 0x7fe18013a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44062/0 (socket says 172.21.2.104:44062) 2021-07-11T03:55:14.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17e59c700 1 -- 172.21.2.104:0/150990652 learned_addr learned my addr 172.21.2.104:0/150990652 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:14.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17e59c700 1 -- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe18013f420 msgr2=0x7fe18013acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17e59c700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe18013f420 0x7fe18013acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17e59c700 1 -- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe18014dea0 msgr2=0x7fe18014e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17e59c700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe18014dea0 0x7fe18014e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17e59c700 1 -- 172.21.2.104:0/150990652 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe180151b90 con 0x7fe18013e190 2021-07-11T03:55:14.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe17e59c700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe18013e190 0x7fe18013a7d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fe170002f70 tx=0x7fe170006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe1777fe700 1 -- 172.21.2.104:0/150990652 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe170013460 con 0x7fe18013e190 2021-07-11T03:55:14.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe1777fe700 1 -- 172.21.2.104:0/150990652 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1700135c0 con 0x7fe18013e190 2021-07-11T03:55:14.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe1777fe700 1 -- 172.21.2.104:0/150990652 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe170013920 con 0x7fe18013e190 2021-07-11T03:55:14.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 -- 172.21.2.104:0/150990652 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe180151e00 con 0x7fe18013e190 2021-07-11T03:55:14.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.373+0000 7fe184957700 1 -- 172.21.2.104:0/150990652 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18013f0c0 con 0x7fe18013e190 2021-07-11T03:55:14.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.377+0000 7fe184957700 1 -- 172.21.2.104:0/150990652 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18014dc70 con 0x7fe18013e190 2021-07-11T03:55:14.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.377+0000 7fe1777fe700 1 -- 172.21.2.104:0/150990652 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe17000f070 con 0x7fe18013e190 2021-07-11T03:55:14.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.377+0000 7fe1777fe700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe164053270 0x7fe164055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.377+0000 7fe1777fe700 1 -- 172.21.2.104:0/150990652 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe17006ae10 con 0x7fe18013e190 2021-07-11T03:55:14.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.377+0000 7fe17dd9b700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe164053270 0x7fe164055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.377+0000 7fe17dd9b700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe164053270 0x7fe164055730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fe168002f70 tx=0x7fe168006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.377+0000 7fe1777fe700 1 -- 172.21.2.104:0/150990652 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe170042150 con 0x7fe18013e190 2021-07-11T03:55:14.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 -- 172.21.2.104:0/150990652 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe180057350 con 0x7fe18013e190 2021-07-11T03:55:14.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe1777fe700 1 -- 172.21.2.104:0/150990652 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe170045770 con 0x7fe18013e190 2021-07-11T03:55:14.468 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:55:14.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 -- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe164053270 msgr2=0x7fe164055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe164053270 0x7fe164055730 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fe168002f70 tx=0x7fe168006a10).stop 2021-07-11T03:55:14.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 -- 172.21.2.104:0/150990652 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe18013e190 msgr2=0x7fe18013a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe18013e190 0x7fe18013a7d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fe170002f70 tx=0x7fe170006ec0).stop 2021-07-11T03:55:14.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 -- 172.21.2.104:0/150990652 shutdown_connections 2021-07-11T03:55:14.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe18014dea0 0x7fe18014e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe18013f420 0x7fe18013acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe164053270 0x7fe164055730 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 --2- 172.21.2.104:0/150990652 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe18013e190 0x7fe18013a7d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 -- 172.21.2.104:0/150990652 >> 172.21.2.104:0/150990652 conn(0x7fe18005cf60 msgr2=0x7fe1800b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:14.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 -- 172.21.2.104:0/150990652 shutdown_connections 2021-07-11T03:55:14.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.461+0000 7fe184957700 1 -- 172.21.2.104:0/150990652 wait complete. 2021-07-11T03:55:14.475 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:55:14.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45c3ec700 1 Processor -- start 2021-07-11T03:55:14.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45c3ec700 1 -- start start 2021-07-11T03:55:14.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45c3ec700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe4540b1ec0 0x7fe4540b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45c3ec700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe4540affb0 0x7fe4540b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45c3ec700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4540b0420 con 0x7fe4540a70c0 2021-07-11T03:55:14.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45c3ec700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4540b2790 con 0x7fe4540b1ec0 2021-07-11T03:55:14.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45c3ec700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4540b0650 con 0x7fe4540affb0 2021-07-11T03:55:14.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45a989700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe4540a70c0 0x7fe454141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40836/0 (socket says 172.21.2.104:40836) 2021-07-11T03:55:14.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45a989700 1 -- 172.21.2.104:0/3502530500 learned_addr learned my addr 172.21.2.104:0/3502530500 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:14.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45a188700 1 --2- 172.21.2.104:0/3502530500 >> v2:172.21.2.104:3300/0 conn(0x7fe4540b1ec0 0x7fe4540b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45a188700 1 -- 172.21.2.104:0/3502530500 >> v2:172.21.2.105:3300/0 conn(0x7fe4540affb0 msgr2=0x7fe4540b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45a188700 1 --2- 172.21.2.104:0/3502530500 >> v2:172.21.2.105:3300/0 conn(0x7fe4540affb0 0x7fe4540b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45a188700 1 -- 172.21.2.104:0/3502530500 >> v1:172.21.2.104:6789/0 conn(0x7fe4540a70c0 legacy=0x7fe454141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45a188700 1 -- 172.21.2.104:0/3502530500 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4540b08f0 con 0x7fe4540b1ec0 2021-07-11T03:55:14.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45a188700 1 --2- 172.21.2.104:0/3502530500 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4540b1ec0 0x7fe4540b12d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fe448002f70 tx=0x7fe448006df0).ready entity=mon.0 client_cookie=a76ab98381caa36e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe459186700 1 -- 172.21.2.104:0/3502530500 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe44800d5c0 con 0x7fe4540b1ec0 2021-07-11T03:55:14.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe459186700 1 -- 172.21.2.104:0/3502530500 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe448000e90 con 0x7fe4540b1ec0 2021-07-11T03:55:14.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe459186700 1 -- 172.21.2.104:0/3502530500 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe44800d910 con 0x7fe4540b1ec0 2021-07-11T03:55:14.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45c3ec700 1 -- 172.21.2.104:0/3502530500 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4540b1ec0 msgr2=0x7fe4540b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.521+0000 7fe45c3ec700 1 --2- 172.21.2.104:0/3502530500 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4540b1ec0 0x7fe4540b12d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fe448002f70 tx=0x7fe448006df0).stop 2021-07-11T03:55:14.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 -- 172.21.2.104:0/3502530500 shutdown_connections 2021-07-11T03:55:14.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 --2- 172.21.2.104:0/3502530500 >> v2:172.21.2.105:3300/0 conn(0x7fe4540affb0 0x7fe4540b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 --2- 172.21.2.104:0/3502530500 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4540b1ec0 0x7fe4540b12d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 -- 172.21.2.104:0/3502530500 >> 172.21.2.104:0/3502530500 conn(0x7fe45405f6b0 msgr2=0x7fe45405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:14.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 -- 172.21.2.104:0/3502530500 shutdown_connections 2021-07-11T03:55:14.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 -- 172.21.2.104:0/3502530500 wait complete. 2021-07-11T03:55:14.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 Processor -- start 2021-07-11T03:55:14.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 -- start start 2021-07-11T03:55:14.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4540a70c0 0x7fe4540b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4540affb0 0x7fe454149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe4540b1ec0 0x7fe45414a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45414b730 con 0x7fe4540affb0 2021-07-11T03:55:14.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4540b5910 con 0x7fe4540b1ec0 2021-07-11T03:55:14.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45414b920 con 0x7fe4540a70c0 2021-07-11T03:55:14.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe459987700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4540affb0 0x7fe454149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe459987700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4540affb0 0x7fe454149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44076/0 (socket says 172.21.2.104:44076) 2021-07-11T03:55:14.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe459987700 1 -- 172.21.2.104:0/4140368962 learned_addr learned my addr 172.21.2.104:0/4140368962 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:14.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe459987700 1 -- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4540a70c0 msgr2=0x7fe4540b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe459987700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4540a70c0 0x7fe4540b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe459987700 1 -- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe4540b1ec0 msgr2=0x7fe45414a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe459987700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe4540b1ec0 0x7fe45414a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe459987700 1 -- 172.21.2.104:0/4140368962 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44800b180 con 0x7fe4540affb0 2021-07-11T03:55:14.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe459987700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4540affb0 0x7fe454149f60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fe450002f70 tx=0x7fe450006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe4477fe700 1 -- 172.21.2.104:0/4140368962 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe450013460 con 0x7fe4540affb0 2021-07-11T03:55:14.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe4477fe700 1 -- 172.21.2.104:0/4140368962 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4500135c0 con 0x7fe4540affb0 2021-07-11T03:55:14.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 -- 172.21.2.104:0/4140368962 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4540b3210 con 0x7fe4540affb0 2021-07-11T03:55:14.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe4477fe700 1 -- 172.21.2.104:0/4140368962 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe450013920 con 0x7fe4540affb0 2021-07-11T03:55:14.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 -- 172.21.2.104:0/4140368962 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4540b3bf0 con 0x7fe4540affb0 2021-07-11T03:55:14.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe45c3ec700 1 -- 172.21.2.104:0/4140368962 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe454057350 con 0x7fe4540affb0 2021-07-11T03:55:14.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe4477fe700 1 -- 172.21.2.104:0/4140368962 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe45001a060 con 0x7fe4540affb0 2021-07-11T03:55:14.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe4477fe700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4340532b0 0x7fe434055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.525+0000 7fe4477fe700 1 -- 172.21.2.104:0/4140368962 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe45006afe0 con 0x7fe4540affb0 2021-07-11T03:55:14.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.529+0000 7fe45a188700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4340532b0 0x7fe434055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.529+0000 7fe45a188700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4340532b0 0x7fe434055770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fe448002f70 tx=0x7fe448014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.529+0000 7fe4477fe700 1 -- 172.21.2.104:0/4140368962 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe450042270 con 0x7fe4540affb0 2021-07-11T03:55:14.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 -- 172.21.2.104:0/4140368962 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4540ac760 con 0x7fe4540affb0 2021-07-11T03:55:14.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe4477fe700 1 -- 172.21.2.104:0/4140368962 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe450018030 con 0x7fe4540affb0 2021-07-11T03:55:14.619 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:55:14.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 -- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4340532b0 msgr2=0x7fe434055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4340532b0 0x7fe434055770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fe448002f70 tx=0x7fe448014000).stop 2021-07-11T03:55:14.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 -- 172.21.2.104:0/4140368962 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4540affb0 msgr2=0x7fe454149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4540affb0 0x7fe454149f60 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fe450002f70 tx=0x7fe450006ec0).stop 2021-07-11T03:55:14.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 -- 172.21.2.104:0/4140368962 shutdown_connections 2021-07-11T03:55:14.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe4540b1ec0 0x7fe45414a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe4340532b0 0x7fe434055770 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe4540affb0 0x7fe454149f60 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 --2- 172.21.2.104:0/4140368962 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe4540a70c0 0x7fe4540b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 -- 172.21.2.104:0/4140368962 >> 172.21.2.104:0/4140368962 conn(0x7fe45405f6b0 msgr2=0x7fe454146390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:14.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 -- 172.21.2.104:0/4140368962 shutdown_connections 2021-07-11T03:55:14.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.613+0000 7fe45c3ec700 1 -- 172.21.2.104:0/4140368962 wait complete. 2021-07-11T03:55:14.627 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T03:55:14.627 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:55:14.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4248d7e700 1 Processor -- start 2021-07-11T03:55:14.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4248d7e700 1 -- start start 2021-07-11T03:55:14.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4248d7e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f42440ad150 0x7f42440b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4248d7e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42440ac370 con 0x7f42440a9da0 2021-07-11T03:55:14.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4248d7e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42440aac40 con 0x7f42440aa860 2021-07-11T03:55:14.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4248d7e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42440abb50 con 0x7f42440ad150 2021-07-11T03:55:14.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4242d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f42440a9da0 0x7f4244140340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40854/0 (socket says 172.21.2.104:40854) 2021-07-11T03:55:14.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4242d9d700 1 -- 172.21.2.104:0/379423540 learned_addr learned my addr 172.21.2.104:0/379423540 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:14.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 567854119 0 0) 0x7f42440ac370 con 0x7f42440a9da0 2021-07-11T03:55:14.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4224003260 con 0x7f42440a9da0 2021-07-11T03:55:14.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4241d9b700 1 --2- 172.21.2.104:0/379423540 >> v2:172.21.2.105:3300/0 conn(0x7f42440ad150 0x7f42440b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1270756276 0 0) 0x7f4224003260 con 0x7f42440a9da0 2021-07-11T03:55:14.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 >> v2:172.21.2.105:3300/0 conn(0x7f42440ad150 msgr2=0x7f42440b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 --2- 172.21.2.104:0/379423540 >> v2:172.21.2.105:3300/0 conn(0x7f42440ad150 0x7f42440b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 >> v1:172.21.2.105:6789/0 conn(0x7f42440aa860 legacy=0x7f42440aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440b16b0 con 0x7f42440a9da0 2021-07-11T03:55:14.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f423c0019e0 con 0x7f42440a9da0 2021-07-11T03:55:14.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 >> v1:172.21.2.104:6789/0 conn(0x7f42440a9da0 legacy=0x7f4244140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 --2- 172.21.2.104:0/379423540 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4224003820 0x7f4224003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 --2- 172.21.2.104:0/379423540 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4224004500 0x7f4224004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 --2- 172.21.2.104:0/379423540 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 0x7f422400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422400be20 con 0x7f4224004f70 2021-07-11T03:55:14.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422400bf90 con 0x7f4224004500 2021-07-11T03:55:14.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f422400c150 con 0x7f4224003820 2021-07-11T03:55:14.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4242d9d700 1 --2- 172.21.2.104:0/379423540 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 0x7f422400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4242d9d700 1 -- 172.21.2.104:0/379423540 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4224003820 msgr2=0x7f4224003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4242d9d700 1 --2- 172.21.2.104:0/379423540 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4224003820 0x7f4224003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4242d9d700 1 -- 172.21.2.104:0/379423540 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4224004500 msgr2=0x7f4224004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:14.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4242d9d700 1 --2- 172.21.2.104:0/379423540 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4224004500 0x7f4224004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:14.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4242d9d700 1 -- 172.21.2.104:0/379423540 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4224003260 con 0x7f4224004f70 2021-07-11T03:55:14.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4242d9d700 1 -- 172.21.2.104:0/379423540 reap_dead start 2021-07-11T03:55:14.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4242d9d700 1 --2- 172.21.2.104:0/379423540 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 0x7f422400b750 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f42440b63e0 tx=0x7f423c0020e0).ready entity=mon.0 client_cookie=9f5bed071adfc9ad server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f423c00cc60 con 0x7f4224004f70 2021-07-11T03:55:14.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c00cdc0 con 0x7f4224004f70 2021-07-11T03:55:14.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f424159a700 1 -- 172.21.2.104:0/379423540 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f423c005b80 con 0x7f4224004f70 2021-07-11T03:55:14.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4248d7e700 1 -- 172.21.2.104:0/379423540 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 msgr2=0x7f422400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4248d7e700 1 --2- 172.21.2.104:0/379423540 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 0x7f422400b750 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f42440b63e0 tx=0x7f423c0020e0).stop 2021-07-11T03:55:14.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4248d7e700 1 -- 172.21.2.104:0/379423540 shutdown_connections 2021-07-11T03:55:14.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4248d7e700 1 --2- 172.21.2.104:0/379423540 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 0x7f422400b750 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.673+0000 7f4248d7e700 1 -- 172.21.2.104:0/379423540 >> 172.21.2.104:0/379423540 conn(0x7f42440b64d0 msgr2=0x7f42440b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:14.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 -- 172.21.2.104:0/379423540 shutdown_connections 2021-07-11T03:55:14.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 -- 172.21.2.104:0/379423540 wait complete. 2021-07-11T03:55:14.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 Processor -- start 2021-07-11T03:55:14.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 -- start start 2021-07-11T03:55:14.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 0x7f42440afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4244065120 0x7f4244065580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f424413c7b0 0x7f424413cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4244065a70 con 0x7f4224004f70 2021-07-11T03:55:14.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4244143d40 con 0x7f4244065120 2021-07-11T03:55:14.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4244143b50 con 0x7f424413c7b0 2021-07-11T03:55:14.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f424259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 0x7f42440afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f424259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 0x7f42440afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44096/0 (socket says 172.21.2.104:44096) 2021-07-11T03:55:14.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f424259c700 1 -- 172.21.2.104:0/1361562310 learned_addr learned my addr 172.21.2.104:0/1361562310 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:14.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4241d9b700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4244065120 0x7f4244065580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4242d9d700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f424413c7b0 0x7f424413cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f424259c700 1 -- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f424413c7b0 msgr2=0x7f424413cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f424259c700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f424413c7b0 0x7f424413cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f424259c700 1 -- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4244065120 msgr2=0x7f4244065580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f424259c700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4244065120 0x7f4244065580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f424259c700 1 -- 172.21.2.104:0/1361562310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4244142770 con 0x7f4224004f70 2021-07-11T03:55:14.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f424259c700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 0x7f42440afa60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f4244142b70 tx=0x7f4234007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f423b7fe700 1 -- 172.21.2.104:0/1361562310 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f423400ce60 con 0x7f4224004f70 2021-07-11T03:55:14.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f423b7fe700 1 -- 172.21.2.104:0/1361562310 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4234000e90 con 0x7f4224004f70 2021-07-11T03:55:14.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 -- 172.21.2.104:0/1361562310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4244143f70 con 0x7f4224004f70 2021-07-11T03:55:14.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f423b7fe700 1 -- 172.21.2.104:0/1361562310 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4234007430 con 0x7f4224004f70 2021-07-11T03:55:14.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 -- 172.21.2.104:0/1361562310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4244066330 con 0x7f4224004f70 2021-07-11T03:55:14.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4248d7e700 1 -- 172.21.2.104:0/1361562310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424413dfb0 con 0x7f4224004f70 2021-07-11T03:55:14.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f423b7fe700 1 -- 172.21.2.104:0/1361562310 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4234010030 con 0x7f4224004f70 2021-07-11T03:55:14.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f423b7fe700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4228053300 0x7f42280557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:14.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f423b7fe700 1 -- 172.21.2.104:0/1361562310 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f423406ac50 con 0x7f4224004f70 2021-07-11T03:55:14.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.677+0000 7f4241d9b700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4228053300 0x7f42280557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:14.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.681+0000 7f4241d9b700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4228053300 0x7f42280557c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f422c00c000 tx=0x7f422c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:14.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.681+0000 7f423b7fe700 1 -- 172.21.2.104:0/1361562310 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4234041f90 con 0x7f4224004f70 2021-07-11T03:55:14.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 -- 172.21.2.104:0/1361562310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4244057350 con 0x7f4224004f70 2021-07-11T03:55:14.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f423b7fe700 1 -- 172.21.2.104:0/1361562310 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f42340455b0 con 0x7f4224004f70 2021-07-11T03:55:14.795 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:14.795 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4492,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2412138431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2412138431},{"type":"v1","addr":"172.21.2.105:6835","nonce":2412138431}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/3271417890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3271417890},{"type":"v1","addr":"172.21.2.104:6834","nonce":3271417890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:55:11.595553+0000","modified":"2021-07-11T03:55:12.600491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/795410455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":795410455},{"type":"v1","addr":"172.21.2.104:6835","nonce":795410455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-07-11T03:55:14.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 -- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4228053300 msgr2=0x7f42280557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4228053300 0x7f42280557c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f422c00c000 tx=0x7f422c008040).stop 2021-07-11T03:55:14.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 -- 172.21.2.104:0/1361562310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 msgr2=0x7f42440afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:14.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 0x7f42440afa60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f4244142b70 tx=0x7f4234007c30).stop 2021-07-11T03:55:14.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 -- 172.21.2.104:0/1361562310 shutdown_connections 2021-07-11T03:55:14.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f424413c7b0 0x7f424413cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4244065120 0x7f4244065580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4228053300 0x7f42280557c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 --2- 172.21.2.104:0/1361562310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4224004f70 0x7f42440afa60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:14.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 -- 172.21.2.104:0/1361562310 >> 172.21.2.104:0/1361562310 conn(0x7f42440b64d0 msgr2=0x7f42440b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:14.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 -- 172.21.2.104:0/1361562310 shutdown_connections 2021-07-11T03:55:14.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:14.789+0000 7f4248d7e700 1 -- 172.21.2.104:0/1361562310 wait complete. 2021-07-11T03:55:14.799 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 16 2021-07-11T03:55:15.104 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:55:11.595553+0000', 'modified': '2021-07-11T03:55:12.600491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5232}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5232': {'gid': 5232, 'name': 'c', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/795410455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 795410455}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 795410455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:55:15.105 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:55:15.106 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:55:15.171 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":5232,"want_state":"up:active","state":"up:active","rank_uptime":3.5670582350000002,"mdsmap_epoch":16,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":7.7253760250000001} 2021-07-11T03:55:15.172 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 5232, "mdsmap_epoch": 16, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 3.567058235, "state": "up:active", "uptime": 7.725376025, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:55:15.172 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T03:55:15.173 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T03:55:15.173 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T03:55:15.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:15.174 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:15.186 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:55:15.198 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:55:15.198 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:55:15.249 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:55:15.249 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:55:15.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:15.251 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:55:15.256 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T03:55:15.257 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:15.259 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:55:15.269 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:55:15.269 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:55:15.298 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T03:55:15.303+0000 7f53c5f1c080 -1 init, newargv = 0x562cfd944930 newargc=9ceph-fuse[14704 2021-07-11T03:55:15.299 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:]: starting ceph client 2021-07-11T03:55:15.304 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:55:15.305 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:55:15.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:15.306 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:55:15.312 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14704]: starting fuse 2021-07-11T03:55:15.317 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T03:55:15.317 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T03:55:15.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:15.318 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T03:55:15.319 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T03:55:15.319 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T03:55:15.319 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:55:15.320 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T03:55:15.320 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:55:15.321 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T03:55:15.321 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T03:55:15.321 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T03:55:15.322 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:55:15.322 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T03:55:15.323 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T03:55:15.323 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T03:55:15.324 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T03:55:15.324 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:55:15.324 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T03:55:15.325 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T03:55:15.325 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T03:55:15.326 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T03:55:15.326 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T03:55:15.326 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:55:15.327 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T03:55:15.327 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:55:15.384 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14704.asok 2021-07-11T03:55:15.386 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14704.asok 2021-07-11T03:55:15.387 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:15.387 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14704.asok status 2021-07-11T03:55:15.435 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T03:55:15.435 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T03:55:15.436 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T03:55:15.436 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T03:55:15.436 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T03:55:15.436 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T03:55:15.436 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T03:55:15.437 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14704", 2021-07-11T03:55:15.437 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T03:55:15.437 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:55:15.437 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T03:55:15.437 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T03:55:15.438 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 5394, 2021-07-11T03:55:15.438 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T03:55:15.438 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T03:55:15.438 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T03:55:15.439 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 5394 2021-07-11T03:55:15.439 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:55:15.439 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:55:15.439 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:55:15.439 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:55:15.440 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3926220286 2021-07-11T03:55:15.440 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T03:55:15.440 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:55:15.440 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:55:15.440 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:55:15.441 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:55:15.441 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3926220286 2021-07-11T03:55:15.441 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:55:15.442 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.5394 172.21.2.119:0/3926220286", 2021-07-11T03:55:15.442 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/3926220286", 2021-07-11T03:55:15.442 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T03:55:15.442 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 16, 2021-07-11T03:55:15.442 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 44, 2021-07-11T03:55:15.443 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T03:55:15.443 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T03:55:15.443 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T03:55:15.443 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T03:55:15.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:15.445 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:55:15.445 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:15.451 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:55:15.451 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:55:15.452 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:15.452 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:15.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:15.461 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:55:15.461 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:15.506 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:55:15.508 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:55:15.508 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-07-11T03:55:15.509 DEBUG:teuthology.orchestra.run.gibba019:> import sys 2021-07-11T03:55:15.509 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-07-11T03:55:15.510 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:55:15.510 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:55:15.511 DEBUG:teuthology.orchestra.run.gibba019:> n = 1000 2021-07-11T03:55:15.511 DEBUG:teuthology.orchestra.run.gibba019:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-07-11T03:55:15.511 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:55:15.512 DEBUG:teuthology.orchestra.run.gibba019:> if not os.path.exists(os.path.dirname(abs_path)): 2021-07-11T03:55:15.512 DEBUG:teuthology.orchestra.run.gibba019:> os.makedirs(os.path.dirname(abs_path)) 2021-07-11T03:55:15.513 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:55:15.513 DEBUG:teuthology.orchestra.run.gibba019:> for i in range(0, n): 2021-07-11T03:55:15.514 DEBUG:teuthology.orchestra.run.gibba019:> fname = "{0}_{1}".format(abs_path, i) 2021-07-11T03:55:15.514 DEBUG:teuthology.orchestra.run.gibba019:> with open(fname, '"'"'w'"'"') as f: 2021-07-11T03:55:15.515 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-07-11T03:55:15.515 DEBUG:teuthology.orchestra.run.gibba019:> if True: 2021-07-11T03:55:15.516 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-07-11T03:55:15.517 DEBUG:teuthology.orchestra.run.gibba019:> os.fsync(f.fileno()) 2021-07-11T03:55:15.517 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:55:34.017 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:55:34.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.065+0000 7fdd65857700 1 Processor -- start 2021-07-11T03:55:34.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.065+0000 7fdd65857700 1 -- start start 2021-07-11T03:55:34.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.065+0000 7fdd65857700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd600a76b0 0x7fdd60066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.065+0000 7fdd65857700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fdd600b0060 0x7fdd6005fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.065+0000 7fdd65857700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd600a7a90 con 0x7fdd600b1970 2021-07-11T03:55:34.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.065+0000 7fdd65857700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd60147620 con 0x7fdd600b0060 2021-07-11T03:55:34.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.065+0000 7fdd65857700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601478c0 con 0x7fdd600a76b0 2021-07-11T03:55:34.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5effd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd600a76b0 0x7fdd60066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5f7fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fdd600b0060 0x7fdd6005fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5f7fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fdd600b0060 0x7fdd6005fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:51666/0 (socket says 172.21.2.104:51666) 2021-07-11T03:55:34.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5f7fe700 1 -- 172.21.2.104:0/1424656804 learned_addr learned my addr 172.21.2.104:0/1424656804 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:34.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5f7fe700 1 -- 172.21.2.104:0/1424656804 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd600a76b0 msgr2=0x7fdd60066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5f7fe700 1 --2- 172.21.2.104:0/1424656804 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd600a76b0 0x7fdd60066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5f7fe700 1 -- 172.21.2.104:0/1424656804 >> v1:172.21.2.105:6789/0 conn(0x7fdd600b1970 legacy=0x7fdd6005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5f7fe700 1 -- 172.21.2.104:0/1424656804 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd60147b60 con 0x7fdd600b0060 2021-07-11T03:55:34.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5f7fe700 1 --2- 172.21.2.104:0/1424656804 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd600b0060 0x7fdd6005fdc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fdd580032f0 tx=0x7fdd58009290).ready entity=mon.1 client_cookie=ca5b43b5b20ad305 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5dffb700 1 -- 172.21.2.104:0/1424656804 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd5800b8a0 con 0x7fdd600b0060 2021-07-11T03:55:34.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5dffb700 1 -- 172.21.2.104:0/1424656804 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5800ba00 con 0x7fdd600b0060 2021-07-11T03:55:34.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd5dffb700 1 -- 172.21.2.104:0/1424656804 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd5800bb70 con 0x7fdd600b0060 2021-07-11T03:55:34.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd65857700 1 -- 172.21.2.104:0/1424656804 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd600b0060 msgr2=0x7fdd6005fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd65857700 1 --2- 172.21.2.104:0/1424656804 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd600b0060 0x7fdd6005fdc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fdd580032f0 tx=0x7fdd58009290).stop 2021-07-11T03:55:34.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd65857700 1 -- 172.21.2.104:0/1424656804 shutdown_connections 2021-07-11T03:55:34.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd65857700 1 --2- 172.21.2.104:0/1424656804 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd600b0060 0x7fdd6005fdc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd65857700 1 --2- 172.21.2.104:0/1424656804 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd600a76b0 0x7fdd60066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd65857700 1 -- 172.21.2.104:0/1424656804 >> 172.21.2.104:0/1424656804 conn(0x7fdd600b53e0 msgr2=0x7fdd600b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:34.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd65857700 1 -- 172.21.2.104:0/1424656804 shutdown_connections 2021-07-11T03:55:34.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.069+0000 7fdd65857700 1 -- 172.21.2.104:0/1424656804 wait complete. 2021-07-11T03:55:34.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd65857700 1 Processor -- start 2021-07-11T03:55:34.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd65857700 1 -- start start 2021-07-11T03:55:34.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd65857700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd600a76b0 0x7fdd600ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd65857700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd600b0060 0x7fdd600ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd65857700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd600b1970 0x7fdd6014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd65857700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd601575e0 con 0x7fdd600b1970 2021-07-11T03:55:34.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd65857700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60156960 con 0x7fdd600b0060 2021-07-11T03:55:34.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd65857700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60156c00 con 0x7fdd600a76b0 2021-07-11T03:55:34.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd600b1970 0x7fdd6014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd600b1970 0x7fdd6014cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44112/0 (socket says 172.21.2.104:44112) 2021-07-11T03:55:34.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5f7fe700 1 -- 172.21.2.104:0/4016082986 learned_addr learned my addr 172.21.2.104:0/4016082986 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:34.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5effd700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd600a76b0 0x7fdd600ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5e7fc700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd600b0060 0x7fdd600ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5f7fe700 1 -- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd600a76b0 msgr2=0x7fdd600ac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5f7fe700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd600a76b0 0x7fdd600ac5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5f7fe700 1 -- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd600b0060 msgr2=0x7fdd600ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5f7fe700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd600b0060 0x7fdd600ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5f7fe700 1 -- 172.21.2.104:0/4016082986 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd58009530 con 0x7fdd600b1970 2021-07-11T03:55:34.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5e7fc700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd600b0060 0x7fdd600ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:55:34.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5f7fe700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd600b1970 0x7fdd6014cd00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fdd580032f0 tx=0x7fdd58016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd57fff700 1 -- 172.21.2.104:0/4016082986 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd58011ca0 con 0x7fdd600b1970 2021-07-11T03:55:34.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd57fff700 1 -- 172.21.2.104:0/4016082986 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58011e00 con 0x7fdd600b1970 2021-07-11T03:55:34.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd65857700 1 -- 172.21.2.104:0/4016082986 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd60156ea0 con 0x7fdd600b1970 2021-07-11T03:55:34.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd57fff700 1 -- 172.21.2.104:0/4016082986 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd58020620 con 0x7fdd600b1970 2021-07-11T03:55:34.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd65857700 1 -- 172.21.2.104:0/4016082986 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd60156650 con 0x7fdd600b1970 2021-07-11T03:55:34.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd65857700 1 -- 172.21.2.104:0/4016082986 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd60057350 con 0x7fdd600b1970 2021-07-11T03:55:34.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd57fff700 1 -- 172.21.2.104:0/4016082986 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fdd58028070 con 0x7fdd600b1970 2021-07-11T03:55:34.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd57fff700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd440532b0 0x7fdd44055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd57fff700 1 -- 172.21.2.104:0/4016082986 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdd5806be60 con 0x7fdd600b1970 2021-07-11T03:55:34.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.073+0000 7fdd5effd700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd440532b0 0x7fdd44055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.077+0000 7fdd5effd700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd440532b0 0x7fdd44055770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fdd4800b000 tx=0x7fdd48008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.077+0000 7fdd57fff700 1 -- 172.21.2.104:0/4016082986 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fdd58043090 con 0x7fdd600b1970 2021-07-11T03:55:34.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.185+0000 7fdd65857700 1 -- 172.21.2.104:0/4016082986 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd600698e0 con 0x7fdd600b1970 2021-07-11T03:55:34.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd57fff700 1 -- 172.21.2.104:0/4016082986 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2488 (secure 0 0 0) 0x7fdd58015020 con 0x7fdd600b1970 2021-07-11T03:55:34.194 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:34.194 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4492,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2412138431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2412138431},{"type":"v1","addr":"172.21.2.105:6835","nonce":2412138431}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/3271417890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3271417890},{"type":"v1","addr":"172.21.2.104:6834","nonce":3271417890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:55:11.595553+0000","modified":"2021-07-11T03:55:12.600491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/795410455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":795410455},{"type":"v1","addr":"172.21.2.104:6835","nonce":795410455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-07-11T03:55:34.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 -- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd440532b0 msgr2=0x7fdd44055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd440532b0 0x7fdd44055770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fdd4800b000 tx=0x7fdd48008040).stop 2021-07-11T03:55:34.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 -- 172.21.2.104:0/4016082986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd600b1970 msgr2=0x7fdd6014cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd600b1970 0x7fdd6014cd00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fdd580032f0 tx=0x7fdd58016000).stop 2021-07-11T03:55:34.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 -- 172.21.2.104:0/4016082986 shutdown_connections 2021-07-11T03:55:34.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd440532b0 0x7fdd44055770 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd600b1970 0x7fdd6014cd00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd600b0060 0x7fdd600ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 --2- 172.21.2.104:0/4016082986 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd600a76b0 0x7fdd600ac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 -- 172.21.2.104:0/4016082986 >> 172.21.2.104:0/4016082986 conn(0x7fdd600b53e0 msgr2=0x7fdd60146400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:34.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 -- 172.21.2.104:0/4016082986 shutdown_connections 2021-07-11T03:55:34.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.189+0000 7fdd65857700 1 -- 172.21.2.104:0/4016082986 wait complete. 2021-07-11T03:55:34.200 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 16 2021-07-11T03:55:34.202 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_caps 20 2021-07-11T03:55:34.261 INFO:teuthology.orchestra.run.gibba004.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-07-11T03:55:34.261 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-07-11T03:55:34.262 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:55:34.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 Processor -- start 2021-07-11T03:55:34.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 -- start start 2021-07-11T03:55:34.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e0c0afd60 0x7f9e0c067860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0c144450 con 0x7f9e0c0b2850 2021-07-11T03:55:34.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0c0b2c30 con 0x7f9e0c0af130 2021-07-11T03:55:34.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c145300 con 0x7f9e0c0afd60 2021-07-11T03:55:34.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0bfff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f9e0c0b2850 0x7f9e0c067150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40886/0 (socket says 172.21.2.104:40886) 2021-07-11T03:55:34.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0bfff700 1 -- 172.21.2.104:0/2192698919 learned_addr learned my addr 172.21.2.104:0/2192698919 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:34.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049505683 0 0) 0x7f9e0c144450 con 0x7f9e0c0b2850 2021-07-11T03:55:34.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df8003260 con 0x7f9e0c0b2850 2021-07-11T03:55:34.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4077105438 0 0) 0x7f9df8003260 con 0x7f9e0c0b2850 2021-07-11T03:55:34.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e0c0afd60 msgr2=0x7f9e0c067860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 --2- 172.21.2.104:0/2192698919 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e0c0afd60 0x7f9e0c067860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 >> v1:172.21.2.105:6789/0 conn(0x7f9e0c0af130 legacy=0x7f9e0c140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c144680 con 0x7f9e0c0b2850 2021-07-11T03:55:34.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f9e04001a00 con 0x7f9e0c0b2850 2021-07-11T03:55:34.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 >> v1:172.21.2.104:6789/0 conn(0x7f9e0c0b2850 legacy=0x7f9e0c067150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 --2- 172.21.2.104:0/2192698919 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e0c0afd60 0x7f9df8003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 --2- 172.21.2.104:0/2192698919 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9df8003ef0 0x7f9df8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 --2- 172.21.2.104:0/2192698919 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9df8004bf0 0x7f9df800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c145300 con 0x7f9df8003ef0 2021-07-11T03:55:34.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df80052f0 con 0x7f9df8004bf0 2021-07-11T03:55:34.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df800bc40 con 0x7f9e0c0afd60 2021-07-11T03:55:34.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0b7fe700 1 --2- 172.21.2.104:0/2192698919 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9df8003ef0 0x7f9df8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0b7fe700 1 -- 172.21.2.104:0/2192698919 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e0c0afd60 msgr2=0x7f9df8003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0b7fe700 1 --2- 172.21.2.104:0/2192698919 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e0c0afd60 0x7f9df8003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0b7fe700 1 -- 172.21.2.104:0/2192698919 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9df8004bf0 msgr2=0x7f9df800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0b7fe700 1 --2- 172.21.2.104:0/2192698919 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9df8004bf0 0x7f9df800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0b7fe700 1 -- 172.21.2.104:0/2192698919 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8003260 con 0x7f9df8003ef0 2021-07-11T03:55:34.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0b7fe700 1 --2- 172.21.2.104:0/2192698919 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9df8003ef0 0x7f9df8004500 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f9dfc000f70 tx=0x7f9dfc0079e0).ready entity=mon.0 client_cookie=6097e3f4a58fad80 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dfc007cf0 con 0x7f9df8003ef0 2021-07-11T03:55:34.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc001d50 con 0x7f9df8003ef0 2021-07-11T03:55:34.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e0affd700 1 -- 172.21.2.104:0/2192698919 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9dfc014600 con 0x7f9df8003ef0 2021-07-11T03:55:34.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 -- 172.21.2.104:0/2192698919 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9df8003ef0 msgr2=0x7f9df8004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 --2- 172.21.2.104:0/2192698919 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9df8003ef0 0x7f9df8004500 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f9dfc000f70 tx=0x7f9dfc0079e0).stop 2021-07-11T03:55:34.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e10b54700 1 -- 172.21.2.104:0/2192698919 reap_dead start 2021-07-11T03:55:34.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 -- 172.21.2.104:0/2192698919 shutdown_connections 2021-07-11T03:55:34.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 -- 172.21.2.104:0/2192698919 >> 172.21.2.104:0/2192698919 conn(0x7f9e0c061230 msgr2=0x7f9e0c062df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:34.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 -- 172.21.2.104:0/2192698919 shutdown_connections 2021-07-11T03:55:34.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.309+0000 7f9e11355700 1 -- 172.21.2.104:0/2192698919 wait complete. 2021-07-11T03:55:34.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e11355700 1 Processor -- start 2021-07-11T03:55:34.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e11355700 1 -- start start 2021-07-11T03:55:34.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e11355700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e0c0b4960 0x7f9e0c140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e11355700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e0c1366b0 0x7f9e0c134d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e11355700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e0c1351f0 0x7f9e0c137de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e11355700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c140a20 con 0x7f9e0c1351f0 2021-07-11T03:55:34.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e11355700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0b4d40 con 0x7f9e0c0b4960 2021-07-11T03:55:34.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e11355700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c140c20 con 0x7f9e0c1366b0 2021-07-11T03:55:34.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e10b54700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e0c1351f0 0x7f9e0c137de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e10b54700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e0c1351f0 0x7f9e0c137de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44132/0 (socket says 172.21.2.104:44132) 2021-07-11T03:55:34.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e10b54700 1 -- 172.21.2.104:0/1886475934 learned_addr learned my addr 172.21.2.104:0/1886475934 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:34.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e10b54700 1 -- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e0c1366b0 msgr2=0x7f9e0c134d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e10b54700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e0c1366b0 0x7f9e0c134d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e10b54700 1 -- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e0c0b4960 msgr2=0x7f9e0c140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e10b54700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e0c0b4960 0x7f9e0c140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e10b54700 1 -- 172.21.2.104:0/1886475934 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c1412e0 con 0x7f9e0c1351f0 2021-07-11T03:55:34.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e10b54700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e0c1351f0 0x7f9e0c137de0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f9e04001490 tx=0x7f9e00007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e097fa700 1 -- 172.21.2.104:0/1886475934 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e0000ac90 con 0x7f9e0c1351f0 2021-07-11T03:55:34.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e097fa700 1 -- 172.21.2.104:0/1886475934 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0000adf0 con 0x7f9e0c1351f0 2021-07-11T03:55:34.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e11355700 1 -- 172.21.2.104:0/1886475934 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0c141580 con 0x7f9e0c1351f0 2021-07-11T03:55:34.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e11355700 1 -- 172.21.2.104:0/1886475934 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0c1356a0 con 0x7f9e0c1351f0 2021-07-11T03:55:34.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e097fa700 1 -- 172.21.2.104:0/1886475934 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e00007300 con 0x7f9e0c1351f0 2021-07-11T03:55:34.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e11355700 1 -- 172.21.2.104:0/1886475934 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0c057350 con 0x7f9e0c1351f0 2021-07-11T03:55:34.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e097fa700 1 -- 172.21.2.104:0/1886475934 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9e0001b060 con 0x7f9e0c1351f0 2021-07-11T03:55:34.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e097fa700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9dec0531a0 0x7f9dec055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e097fa700 1 -- 172.21.2.104:0/1886475934 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9e0006a9e0 con 0x7f9e0c1351f0 2021-07-11T03:55:34.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e0bfff700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9dec0531a0 0x7f9dec055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.313+0000 7f9e0bfff700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9dec0531a0 0x7f9dec055660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f9e0c062090 tx=0x7f9e0c0b1c70).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.317+0000 7f9e097fa700 1 -- 172.21.2.104:0/1886475934 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9e00041c10 con 0x7f9e0c1351f0 2021-07-11T03:55:34.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 -- 172.21.2.104:0/1886475934 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e0c13eb00 con 0x7f9e0c1351f0 2021-07-11T03:55:34.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e097fa700 1 -- 172.21.2.104:0/1886475934 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2488 (secure 0 0 0) 0x7f9e00019020 con 0x7f9e0c1351f0 2021-07-11T03:55:34.433 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:34.433 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4492,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2412138431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2412138431},{"type":"v1","addr":"172.21.2.105:6835","nonce":2412138431}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/3271417890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3271417890},{"type":"v1","addr":"172.21.2.104:6834","nonce":3271417890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:55:11.595553+0000","modified":"2021-07-11T03:55:12.600491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/795410455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":795410455},{"type":"v1","addr":"172.21.2.104:6835","nonce":795410455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-07-11T03:55:34.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 -- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9dec0531a0 msgr2=0x7f9dec055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9dec0531a0 0x7f9dec055660 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f9e0c062090 tx=0x7f9e0c0b1c70).stop 2021-07-11T03:55:34.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 -- 172.21.2.104:0/1886475934 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e0c1351f0 msgr2=0x7f9e0c137de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e0c1351f0 0x7f9e0c137de0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f9e04001490 tx=0x7f9e00007be0).stop 2021-07-11T03:55:34.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 -- 172.21.2.104:0/1886475934 shutdown_connections 2021-07-11T03:55:34.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9dec0531a0 0x7f9dec055660 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e0c1351f0 0x7f9e0c137de0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e0c1366b0 0x7f9e0c134d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 --2- 172.21.2.104:0/1886475934 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e0c0b4960 0x7f9e0c140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 -- 172.21.2.104:0/1886475934 >> 172.21.2.104:0/1886475934 conn(0x7f9e0c061230 msgr2=0x7f9e0c142280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:34.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 -- 172.21.2.104:0/1886475934 shutdown_connections 2021-07-11T03:55:34.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.425+0000 7f9e11355700 1 -- 172.21.2.104:0/1886475934 wait complete. 2021-07-11T03:55:34.440 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 16 2021-07-11T03:55:34.441 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_decay_threshold 400 2021-07-11T03:55:34.504 INFO:teuthology.orchestra.run.gibba004.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-07-11T03:55:34.504 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-07-11T03:55:34.505 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T03:55:34.521 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:55:34.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 Processor -- start 2021-07-11T03:55:34.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 -- start start 2021-07-11T03:55:34.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff69c0b1970 0x7ff69c05fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69c0a7a90 con 0x7ff69c0a76b0 2021-07-11T03:55:34.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff69c147620 con 0x7ff69c067180 2021-07-11T03:55:34.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff69c1478c0 con 0x7ff69c0b1970 2021-07-11T03:55:34.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff69a59c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff69c0a76b0 0x7ff69c066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40906/0 (socket says 172.21.2.104:40906) 2021-07-11T03:55:34.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff69a59c700 1 -- 172.21.2.104:0/2092883359 learned_addr learned my addr 172.21.2.104:0/2092883359 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:34.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff69ad9d700 1 --2- 172.21.2.104:0/2092883359 >> v2:172.21.2.104:3300/0 conn(0x7ff69c0b1970 0x7ff69c05fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff69ad9d700 1 -- 172.21.2.104:0/2092883359 >> v1:172.21.2.105:6789/0 conn(0x7ff69c067180 legacy=0x7ff69c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff69ad9d700 1 -- 172.21.2.104:0/2092883359 >> v1:172.21.2.104:6789/0 conn(0x7ff69c0a76b0 legacy=0x7ff69c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff69ad9d700 1 -- 172.21.2.104:0/2092883359 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c147b60 con 0x7ff69c0b1970 2021-07-11T03:55:34.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff69ad9d700 1 --2- 172.21.2.104:0/2092883359 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff69c0b1970 0x7ff69c05fdc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ff69c06a180 tx=0x7ff6940091d0).ready entity=mon.0 client_cookie=fe2572afa0870db4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff69959a700 1 -- 172.21.2.104:0/2092883359 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff694009ea0 con 0x7ff69c0b1970 2021-07-11T03:55:34.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff69959a700 1 -- 172.21.2.104:0/2092883359 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69400b830 con 0x7ff69c0b1970 2021-07-11T03:55:34.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff69959a700 1 -- 172.21.2.104:0/2092883359 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff69400b9e0 con 0x7ff69c0b1970 2021-07-11T03:55:34.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2092883359 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff69c0b1970 msgr2=0x7ff69c05fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 --2- 172.21.2.104:0/2092883359 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff69c0b1970 0x7ff69c05fdc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ff69c06a180 tx=0x7ff6940091d0).stop 2021-07-11T03:55:34.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2092883359 shutdown_connections 2021-07-11T03:55:34.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 --2- 172.21.2.104:0/2092883359 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff69c0b1970 0x7ff69c05fdc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2092883359 >> 172.21.2.104:0/2092883359 conn(0x7ff69c0b53e0 msgr2=0x7ff69c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:34.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2092883359 shutdown_connections 2021-07-11T03:55:34.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2092883359 wait complete. 2021-07-11T03:55:34.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.569+0000 7ff6a0c5f700 1 Processor -- start 2021-07-11T03:55:34.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6a0c5f700 1 -- start start 2021-07-11T03:55:34.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6a0c5f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff69c067180 0x7ff69c0af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6a0c5f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff69c0a76b0 0x7ff69c0aa040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6a0c5f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff69c0b1970 0x7ff69c0ab230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6a0c5f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0ae190 con 0x7ff69c0a76b0 2021-07-11T03:55:34.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6a0c5f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0ae430 con 0x7ff69c0b1970 2021-07-11T03:55:34.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6a0c5f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c0ae6d0 con 0x7ff69c067180 2021-07-11T03:55:34.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff699d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff69c0a76b0 0x7ff69c0aa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff699d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff69c0a76b0 0x7ff69c0aa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44148/0 (socket says 172.21.2.104:44148) 2021-07-11T03:55:34.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff699d9b700 1 -- 172.21.2.104:0/2609931128 learned_addr learned my addr 172.21.2.104:0/2609931128 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:34.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff699d9b700 1 -- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff69c067180 msgr2=0x7ff69c0af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff699d9b700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff69c067180 0x7ff69c0af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff699d9b700 1 -- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff69c0b1970 msgr2=0x7ff69c0ab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff699d9b700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff69c0b1970 0x7ff69c0ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff699d9b700 1 -- 172.21.2.104:0/2609931128 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff694009470 con 0x7ff69c0a76b0 2021-07-11T03:55:34.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff699d9b700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff69c0a76b0 0x7ff69c0aa040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7ff684002f90 tx=0x7ff684006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6937fe700 1 -- 172.21.2.104:0/2609931128 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff684013460 con 0x7ff69c0a76b0 2021-07-11T03:55:34.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6937fe700 1 -- 172.21.2.104:0/2609931128 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6840135c0 con 0x7ff69c0a76b0 2021-07-11T03:55:34.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6937fe700 1 -- 172.21.2.104:0/2609931128 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff684013920 con 0x7ff69c0a76b0 2021-07-11T03:55:34.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2609931128 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c0ae970 con 0x7ff69c0a76b0 2021-07-11T03:55:34.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2609931128 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c0aa530 con 0x7ff69c0a76b0 2021-07-11T03:55:34.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2609931128 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69c0aeb00 con 0x7ff69c0a76b0 2021-07-11T03:55:34.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6937fe700 1 -- 172.21.2.104:0/2609931128 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff68401a060 con 0x7ff69c0a76b0 2021-07-11T03:55:34.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6937fe700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6800531a0 0x7ff680055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff6937fe700 1 -- 172.21.2.104:0/2609931128 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff68406af60 con 0x7ff69c0a76b0 2021-07-11T03:55:34.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.573+0000 7ff69a59c700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6800531a0 0x7ff680055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.577+0000 7ff69a59c700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6800531a0 0x7ff680055660 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7ff69c06a0e0 tx=0x7ff68c0077d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.577+0000 7ff6937fe700 1 -- 172.21.2.104:0/2609931128 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff684042190 con 0x7ff69c0a76b0 2021-07-11T03:55:34.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.681+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2609931128 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff69c057350 con 0x7ff69c0a76b0 2021-07-11T03:55:34.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.681+0000 7ff6937fe700 1 -- 172.21.2.104:0/2609931128 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2488 (secure 0 0 0) 0x7ff684018030 con 0x7ff69c0a76b0 2021-07-11T03:55:34.690 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:34.690 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4492,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2412138431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2412138431},{"type":"v1","addr":"172.21.2.105:6835","nonce":2412138431}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/3271417890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3271417890},{"type":"v1","addr":"172.21.2.104:6834","nonce":3271417890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:55:11.595553+0000","modified":"2021-07-11T03:55:12.600491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/795410455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":795410455},{"type":"v1","addr":"172.21.2.104:6835","nonce":795410455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-07-11T03:55:34.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6800531a0 msgr2=0x7ff680055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6800531a0 0x7ff680055660 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7ff69c06a0e0 tx=0x7ff68c0077d0).stop 2021-07-11T03:55:34.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2609931128 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff69c0a76b0 msgr2=0x7ff69c0aa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff69c0a76b0 0x7ff69c0aa040 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7ff684002f90 tx=0x7ff684006de0).stop 2021-07-11T03:55:34.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2609931128 shutdown_connections 2021-07-11T03:55:34.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff69c0b1970 0x7ff69c0ab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6800531a0 0x7ff680055660 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff69c0a76b0 0x7ff69c0aa040 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 --2- 172.21.2.104:0/2609931128 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff69c067180 0x7ff69c0af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2609931128 >> 172.21.2.104:0/2609931128 conn(0x7ff69c0b53e0 msgr2=0x7ff69c05f950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:34.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2609931128 shutdown_connections 2021-07-11T03:55:34.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.685+0000 7ff6a0c5f700 1 -- 172.21.2.104:0/2609931128 wait complete. 2021-07-11T03:55:34.694 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 16 2021-07-11T03:55:34.697 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c cache drop 2021-07-11T03:55:34.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.741+0000 7f34915e8700 1 Processor -- start 2021-07-11T03:55:34.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.741+0000 7f34915e8700 1 -- start start 2021-07-11T03:55:34.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.741+0000 7f34915e8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f348c0aff10 0x7f348c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.741+0000 7f34915e8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f348c0ac9b0 0x7f348c14af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.741+0000 7f34915e8700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f348c061360 con 0x7f348c14ab80 2021-07-11T03:55:34.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.741+0000 7f34915e8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c14b5e0 con 0x7f348c0aff10 2021-07-11T03:55:34.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.741+0000 7f34915e8700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c14b880 con 0x7f348c0ac9b0 2021-07-11T03:55:34.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.741+0000 7f348a7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f348c0aff10 0x7f348c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.741+0000 7f348affd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f348c14ab80 0x7f348c0af1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40922/0 (socket says 172.21.2.104:40922) 2021-07-11T03:55:34.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348affd700 1 -- 172.21.2.104:0/750051113 learned_addr learned my addr 172.21.2.104:0/750051113 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:34.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f3489ffb700 1 -- 172.21.2.104:0/750051113 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403481313 0 0) 0x7f348c061360 con 0x7f348c14ab80 2021-07-11T03:55:34.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f3489ffb700 1 -- 172.21.2.104:0/750051113 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3470003260 con 0x7f348c14ab80 2021-07-11T03:55:34.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 -- 172.21.2.104:0/750051113 >> v2:172.21.2.105:3300/0 conn(0x7f348c0ac9b0 msgr2=0x7f348c14af60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:34.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 --2- 172.21.2.104:0/750051113 >> v2:172.21.2.105:3300/0 conn(0x7f348c0ac9b0 0x7f348c14af60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 -- 172.21.2.104:0/750051113 >> v1:172.21.2.104:6789/0 conn(0x7f348c14ab80 legacy=0x7f348c0af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 -- 172.21.2.104:0/750051113 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0aed20 con 0x7f348c0aff10 2021-07-11T03:55:34.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 --2- 172.21.2.104:0/750051113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f348c0aff10 0x7f348c05f6b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f34840032f0 tx=0x7f34840091d0).ready entity=mon.0 client_cookie=e08825d492ba6560 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f3489ffb700 1 -- 172.21.2.104:0/750051113 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348400b680 con 0x7f348c0aff10 2021-07-11T03:55:34.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f3489ffb700 1 -- 172.21.2.104:0/750051113 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348400b7e0 con 0x7f348c0aff10 2021-07-11T03:55:34.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f3489ffb700 1 -- 172.21.2.104:0/750051113 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f348400b990 con 0x7f348c0aff10 2021-07-11T03:55:34.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 -- 172.21.2.104:0/750051113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f348c0aff10 msgr2=0x7f348c05f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 --2- 172.21.2.104:0/750051113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f348c0aff10 0x7f348c05f6b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f34840032f0 tx=0x7f34840091d0).stop 2021-07-11T03:55:34.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 -- 172.21.2.104:0/750051113 shutdown_connections 2021-07-11T03:55:34.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 --2- 172.21.2.104:0/750051113 >> v2:172.21.2.105:3300/0 conn(0x7f348c0ac9b0 0x7f348c14af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 --2- 172.21.2.104:0/750051113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f348c0aff10 0x7f348c05f6b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 -- 172.21.2.104:0/750051113 >> 172.21.2.104:0/750051113 conn(0x7f348c0b5570 msgr2=0x7f348c0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:34.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 -- 172.21.2.104:0/750051113 shutdown_connections 2021-07-11T03:55:34.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 -- 172.21.2.104:0/750051113 wait complete. 2021-07-11T03:55:34.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 Processor -- start 2021-07-11T03:55:34.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 -- start start 2021-07-11T03:55:34.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f348c0ac9b0 0x7f348c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f348c0aff10 0x7f348c141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f348c14ab80 0x7f348c142340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c142a70 con 0x7f348c0aff10 2021-07-11T03:55:34.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c13e970 con 0x7f348c14ab80 2021-07-11T03:55:34.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c156560 con 0x7f348c0ac9b0 2021-07-11T03:55:34.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f348c0aff10 0x7f348c141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f348c0aff10 0x7f348c141e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44162/0 (socket says 172.21.2.104:44162) 2021-07-11T03:55:34.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 -- 172.21.2.104:0/837298787 learned_addr learned my addr 172.21.2.104:0/837298787 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:34.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 -- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f348c0ac9b0 msgr2=0x7f348c0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f348c0ac9b0 0x7f348c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 -- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f348c14ab80 msgr2=0x7f348c142340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f348c14ab80 0x7f348c142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 -- 172.21.2.104:0/837298787 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3484009470 con 0x7f348c0aff10 2021-07-11T03:55:34.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f348a7fc700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f348c0aff10 0x7f348c141e50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f34840032f0 tx=0x7f3484016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f347bfff700 1 -- 172.21.2.104:0/837298787 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3484006ea0 con 0x7f348c0aff10 2021-07-11T03:55:34.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f347bfff700 1 -- 172.21.2.104:0/837298787 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3484021480 con 0x7f348c0aff10 2021-07-11T03:55:34.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 -- 172.21.2.104:0/837298787 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c0b3a00 con 0x7f348c0aff10 2021-07-11T03:55:34.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f34915e8700 1 -- 172.21.2.104:0/837298787 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c0b3ed0 con 0x7f348c0aff10 2021-07-11T03:55:34.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.745+0000 7f347bfff700 1 -- 172.21.2.104:0/837298787 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34840217e0 con 0x7f348c0aff10 2021-07-11T03:55:34.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f347bfff700 1 -- 172.21.2.104:0/837298787 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f348400f070 con 0x7f348c0aff10 2021-07-11T03:55:34.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f347bfff700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34680531a0 0x7f3468055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f347bfff700 1 -- 172.21.2.104:0/837298787 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f348406c5e0 con 0x7f348c0aff10 2021-07-11T03:55:34.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 Processor -- start 2021-07-11T03:55:34.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- start start 2021-07-11T03:55:34.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348affd700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34680531a0 0x7f3468055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 0x2211230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2247c90 0x2248110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 0x2251a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x216f0d0 con 0x2247c90 2021-07-11T03:55:34.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x2181b90 con 0x2248600 2021-07-11T03:55:34.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x2181cd0 con 0x224c970 2021-07-11T03:55:34.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348a7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2247c90 0x2248110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348a7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2247c90 0x2248110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44166/0 (socket says 172.21.2.104:44166) 2021-07-11T03:55:34.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348a7fc700 1 -- 172.21.2.104:0/2095283530 learned_addr learned my addr 172.21.2.104:0/2095283530 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:34.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348a7fc700 1 -- 172.21.2.104:0/2095283530 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 msgr2=0x2211230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348a7fc700 1 --2- 172.21.2.104:0/2095283530 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 0x2211230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348a7fc700 1 -- 172.21.2.104:0/2095283530 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 msgr2=0x2251a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348a7fc700 1 --2- 172.21.2.104:0/2095283530 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 0x2251a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348a7fc700 1 -- 172.21.2.104:0/2095283530 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x21bad10 con 0x2247c90 2021-07-11T03:55:34.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348a7fc700 1 --2- 172.21.2.104:0/2095283530 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2247c90 0x2248110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x2211e50 tx=0x7f3484012000).ready entity=mon.0 client_cookie=7827aafd87f8e945 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f34915e8700 1 -- 172.21.2.104:0/2095283530 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3484047600 con 0x2247c90 2021-07-11T03:55:34.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f34915e8700 1 -- 172.21.2.104:0/2095283530 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3484045980 con 0x2247c90 2021-07-11T03:55:34.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f34915e8700 1 -- 172.21.2.104:0/2095283530 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3484046cc0 con 0x2247c90 2021-07-11T03:55:34.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- 172.21.2.104:0/2095283530 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2247c90 msgr2=0x2248110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 --2- 172.21.2.104:0/2095283530 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2247c90 0x2248110 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x2211e50 tx=0x7f3484012000).stop 2021-07-11T03:55:34.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348affd700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34680531a0 0x7f3468055660 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f3480007a80 tx=0x7f34800076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- 172.21.2.104:0/2095283530 shutdown_connections 2021-07-11T03:55:34.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 --2- 172.21.2.104:0/2095283530 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 0x2251a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 --2- 172.21.2.104:0/2095283530 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2247c90 0x2248110 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 --2- 172.21.2.104:0/2095283530 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 0x2211230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- 172.21.2.104:0/2095283530 >> 172.21.2.104:0/2095283530 conn(0x220f8f0 msgr2=0x2210b20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:34.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- 172.21.2.104:0/2095283530 shutdown_connections 2021-07-11T03:55:34.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- 172.21.2.104:0/2095283530 wait complete. 2021-07-11T03:55:34.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- start start 2021-07-11T03:55:34.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 Processor -- start 2021-07-11T03:55:34.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 0x2217a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 0x2244990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2244e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x2183300 con 0x2217f60 2021-07-11T03:55:34.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x2218cc0 con 0x2248600 2021-07-11T03:55:34.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x2216910 con 0x224c970 2021-07-11T03:55:34.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f349b838740 10 client.0 ll_register_callbacks cb 0x21ca2f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-07-11T03:55:34.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2244e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2244e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44176/0 (socket says 172.21.2.104:44176) 2021-07-11T03:55:34.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.749+0000 7f348b7fe700 1 -- 172.21.2.104:0/519615107 learned_addr learned my addr 172.21.2.104:0/519615107 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:34.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f348b7fe700 1 -- 172.21.2.104:0/519615107 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 msgr2=0x2244990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f348b7fe700 1 --2- 172.21.2.104:0/519615107 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 0x2244990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f348b7fe700 1 -- 172.21.2.104:0/519615107 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 msgr2=0x2217a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f348b7fe700 1 --2- 172.21.2.104:0/519615107 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 0x2217a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f348b7fe700 1 -- 172.21.2.104:0/519615107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c0b3ed0 con 0x2217f60 2021-07-11T03:55:34.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f348b7fe700 1 --2- 172.21.2.104:0/519615107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2244e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f347c002fb0 tx=0x7f347c006cc0).ready entity=mon.0 client_cookie=b5627b9553b800f6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f3465ffb700 10 client.0 ms_handle_connect on v2:172.21.2.104:3300/0 2021-07-11T03:55:34.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f3465ffb700 1 -- 172.21.2.104:0/519615107 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f347c009bb0 con 0x2217f60 2021-07-11T03:55:34.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f3465ffb700 1 -- 172.21.2.104:0/519615107 <== mon.0 v2:172.21.2.104:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f347c009d10 con 0x2217f60 2021-07-11T03:55:34.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 -- 172.21.2.104:0/519615107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ffbc20 con 0x2217f60 2021-07-11T03:55:34.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f3465ffb700 1 -- 172.21.2.104:0/519615107 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f347c012410 con 0x2217f60 2021-07-11T03:55:34.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 10 client.5427 fetch_fsmap learned FSMap version 16 2021-07-11T03:55:34.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 -- 172.21.2.104:0/519615107 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x212e7c0 con 0x2217f60 2021-07-11T03:55:34.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f3465ffb700 1 -- 172.21.2.104:0/519615107 <== mon.0 v2:172.21.2.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1ffbc20 con 0x2217f60 2021-07-11T03:55:34.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f3465ffb700 1 -- 172.21.2.104:0/519615107 <== mon.0 v2:172.21.2.104:3300/0 5 ==== fsmap(e 16) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f347c010070 con 0x2217f60 2021-07-11T03:55:34.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 10 client.5427 fetch_fsmap finished waiting for FSMap version 16 2021-07-11T03:55:34.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 10 client.5427 resolve_mds: resolved ID 'c' to GID 5232 2021-07-11T03:55:34.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 20 client.5427 populate_metadata read hostname 'gibba004' 2021-07-11T03:55:34.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 --2- 172.21.2.104:0/519615107 >> [v2:172.21.2.104:6833/795410455,v1:172.21.2.104:6835/795410455] conn(0x2213e30 0x2257420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 4 client.5427 mds_command: new command op to 5232 tid=0[{"prefix": "get_command_descriptions"}] 2021-07-11T03:55:34.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 -- 172.21.2.104:0/519615107 --> [v2:172.21.2.104:6833/795410455,v1:172.21.2.104:6835/795410455] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x2011460 con 0x2213e30 2021-07-11T03:55:34.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f348affd700 1 --2- 172.21.2.104:0/519615107 >> [v2:172.21.2.104:6833/795410455,v1:172.21.2.104:6835/795410455] conn(0x2213e30 0x2257420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f3465ffb700 10 client.5427 ms_handle_connect on v2:172.21.2.104:6833/795410455 2021-07-11T03:55:34.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f348affd700 1 --2- 172.21.2.104:0/519615107 >> [v2:172.21.2.104:6833/795410455,v1:172.21.2.104:6835/795410455] conn(0x2213e30 0x2257420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ea88493888e0d68a server_cookie=5613a0136c979f07 in_seq=0 out_seq=0 2021-07-11T03:55:34.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f3465ffb700 1 -- 172.21.2.104:0/519615107 <== mds.0 v2:172.21.2.104:6833/795410455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x2011460 con 0x2213e30 2021-07-11T03:55:34.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f3465ffb700 10 client.5427 handle_command_reply: tid=0 2021-07-11T03:55:34.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 client.5427 shutdown 2021-07-11T03:55:34.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 -- 172.21.2.104:0/519615107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 msgr2=0x2244e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 --2- 172.21.2.104:0/519615107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2244e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f347c002fb0 tx=0x7f347c006cc0).stop 2021-07-11T03:55:34.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 -- 172.21.2.104:0/519615107 shutdown_connections 2021-07-11T03:55:34.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 --2- 172.21.2.104:0/519615107 >> [v2:172.21.2.104:6833/795410455,v1:172.21.2.104:6835/795410455] conn(0x2213e30 0x2257420 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 --2- 172.21.2.104:0/519615107 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2244e80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 --2- 172.21.2.104:0/519615107 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 0x2244990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 --2- 172.21.2.104:0/519615107 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 0x2217a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f3465ffb700 0 client.5427 ms_handle_reset on v2:172.21.2.104:6833/795410455 2021-07-11T03:55:34.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 -- 172.21.2.104:0/519615107 >> 172.21.2.104:0/519615107 conn(0x2247c90 msgr2=0x22458c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:34.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 -- 172.21.2.104:0/519615107 shutdown_connections 2021-07-11T03:55:34.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 1 -- 172.21.2.104:0/519615107 wait complete. 2021-07-11T03:55:34.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.753+0000 7f349b838740 20 client.5427 trim_cache size 0 max 16384 2021-07-11T03:55:34.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 Processor -- start 2021-07-11T03:55:34.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 -- start start 2021-07-11T03:55:34.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2254cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 0x2248f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 0x2253560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x2215b40 con 0x2217f60 2021-07-11T03:55:34.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x2011460 con 0x2248600 2021-07-11T03:55:34.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x22163c0 con 0x224c970 2021-07-11T03:55:34.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f348affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2254cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f348affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2254cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44180/0 (socket says 172.21.2.104:44180) 2021-07-11T03:55:34.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f348affd700 1 -- 172.21.2.104:0/3310762154 learned_addr learned my addr 172.21.2.104:0/3310762154 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:34.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f348affd700 1 -- 172.21.2.104:0/3310762154 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 msgr2=0x2253560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f348affd700 1 --2- 172.21.2.104:0/3310762154 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 0x2253560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f348affd700 1 -- 172.21.2.104:0/3310762154 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 msgr2=0x2248f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f348affd700 1 --2- 172.21.2.104:0/3310762154 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 0x2248f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f348affd700 1 -- 172.21.2.104:0/3310762154 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x2217dd0 con 0x2217f60 2021-07-11T03:55:34.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f348affd700 1 --2- 172.21.2.104:0/3310762154 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2254cb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x21c70d0 tx=0x7f3480006000).ready entity=mon.0 client_cookie=f57914d56aad1425 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f3467fff700 1 -- 172.21.2.104:0/3310762154 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3480020070 con 0x2217f60 2021-07-11T03:55:34.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f3467fff700 1 -- 172.21.2.104:0/3310762154 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3480005ca0 con 0x2217f60 2021-07-11T03:55:34.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f3467fff700 1 -- 172.21.2.104:0/3310762154 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3480019420 con 0x2217f60 2021-07-11T03:55:34.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 -- 172.21.2.104:0/3310762154 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 msgr2=0x2254cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:34.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 --2- 172.21.2.104:0/3310762154 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2254cb0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x21c70d0 tx=0x7f3480006000).stop 2021-07-11T03:55:34.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 -- 172.21.2.104:0/3310762154 shutdown_connections 2021-07-11T03:55:34.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 --2- 172.21.2.104:0/3310762154 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x224c970 0x2253560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 --2- 172.21.2.104:0/3310762154 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2248600 0x2248f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 --2- 172.21.2.104:0/3310762154 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x2217f60 0x2254cb0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:34.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 -- 172.21.2.104:0/3310762154 >> 172.21.2.104:0/3310762154 conn(0x2247c90 msgr2=0x2245670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:34.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 -- 172.21.2.104:0/3310762154 shutdown_connections 2021-07-11T03:55:34.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 -- 172.21.2.104:0/3310762154 wait complete. 2021-07-11T03:55:34.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 -- start start 2021-07-11T03:55:34.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 Processor -- start 2021-07-11T03:55:34.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2217f60 0x22ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.761+0000 7f349b838740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x2248600 0x2210b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x224c970 0x220d360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x2213cc0 con 0x224c970 2021-07-11T03:55:34.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x2213a90 con 0x2217f60 2021-07-11T03:55:34.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x2245020 con 0x2248600 2021-07-11T03:55:34.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 10 client.0 ll_register_callbacks cb 0x2216870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-07-11T03:55:34.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f348b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x224c970 0x220d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f348b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x224c970 0x220d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44190/0 (socket says 172.21.2.104:44190) 2021-07-11T03:55:34.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f348b7fe700 1 -- 172.21.2.104:0/132866812 learned_addr learned my addr 172.21.2.104:0/132866812 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:34.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f348b7fe700 1 -- 172.21.2.104:0/132866812 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x2248600 msgr2=0x2210b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f348b7fe700 1 --2- 172.21.2.104:0/132866812 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x2248600 0x2210b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f348b7fe700 1 -- 172.21.2.104:0/132866812 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2217f60 msgr2=0x22ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:34.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f348b7fe700 1 --2- 172.21.2.104:0/132866812 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2217f60 0x22ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:34.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f348b7fe700 1 -- 172.21.2.104:0/132866812 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3470003260 con 0x224c970 2021-07-11T03:55:34.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f348b7fe700 1 --2- 172.21.2.104:0/132866812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x224c970 0x220d360 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f347c002fb0 tx=0x7f347c013000).ready entity=mon.0 client_cookie=69818897c858b6c2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:34.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f3466ffd700 10 client.0 ms_handle_connect on v2:172.21.2.104:3300/0 2021-07-11T03:55:34.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f3466ffd700 1 -- 172.21.2.104:0/132866812 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f347c0113b0 con 0x224c970 2021-07-11T03:55:34.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 1 -- 172.21.2.104:0/132866812 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x2178a70 con 0x224c970 2021-07-11T03:55:34.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f3466ffd700 1 -- 172.21.2.104:0/132866812 <== mon.0 v2:172.21.2.104:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f347c011510 con 0x224c970 2021-07-11T03:55:34.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f3466ffd700 1 -- 172.21.2.104:0/132866812 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f347c0116d0 con 0x224c970 2021-07-11T03:55:34.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 10 client.5433 fetch_fsmap learned FSMap version 16 2021-07-11T03:55:34.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 1 -- 172.21.2.104:0/132866812 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x7f348c060ae0 con 0x224c970 2021-07-11T03:55:34.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f3466ffd700 1 -- 172.21.2.104:0/132866812 <== mon.0 v2:172.21.2.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x2178a70 con 0x224c970 2021-07-11T03:55:34.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f3466ffd700 1 -- 172.21.2.104:0/132866812 <== mon.0 v2:172.21.2.104:3300/0 5 ==== fsmap(e 16) v1 ==== 1434+0+0 (secure 0 0 0) 0x7f347c012410 con 0x224c970 2021-07-11T03:55:34.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 10 client.5433 fetch_fsmap finished waiting for FSMap version 16 2021-07-11T03:55:34.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 10 client.5433 resolve_mds: resolved ID 'c' to GID 5232 2021-07-11T03:55:34.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 20 client.5433 populate_metadata read hostname 'gibba004' 2021-07-11T03:55:34.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 1 --2- 172.21.2.104:0/132866812 >> [v2:172.21.2.104:6833/795410455,v1:172.21.2.104:6835/795410455] conn(0x220d850 0x22aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:34.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 4 client.5433 mds_command: new command op to 5232 tid=0[{"prefix": "cache drop"}] 2021-07-11T03:55:34.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f349b838740 1 -- 172.21.2.104:0/132866812 --> [v2:172.21.2.104:6833/795410455,v1:172.21.2.104:6835/795410455] -- command(tid 0: {"prefix": "cache drop"}) v1 -- 0x208c7d0 con 0x220d850 2021-07-11T03:55:34.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f348affd700 1 --2- 172.21.2.104:0/132866812 >> [v2:172.21.2.104:6833/795410455,v1:172.21.2.104:6835/795410455] conn(0x220d850 0x22aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:34.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f348affd700 1 --2- 172.21.2.104:0/132866812 >> [v2:172.21.2.104:6833/795410455,v1:172.21.2.104:6835/795410455] conn(0x220d850 0x22aec10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=2faddfddc1d20461 server_cookie=e699e08de66751b9 in_seq=0 out_seq=0 2021-07-11T03:55:34.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:34.765+0000 7f3466ffd700 10 client.5433 ms_handle_connect on v2:172.21.2.104:6833/795410455 2021-07-11T03:55:42.183 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:55:42.187 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:55:42.191 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:55:42.195 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:55:45.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f3466ffd700 1 -- 172.21.2.104:0/132866812 <== mds.0 v2:172.21.2.104:6833/795410455 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+380 (crc 0 0 0) 0x208c7d0 con 0x220d850 2021-07-11T03:55:45.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f3466ffd700 10 client.5433 handle_command_reply: tid=0 2021-07-11T03:55:45.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 1 client.5433 shutdown 2021-07-11T03:55:45.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 1 -- 172.21.2.104:0/132866812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x224c970 msgr2=0x220d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:45.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 1 --2- 172.21.2.104:0/132866812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x224c970 0x220d360 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f347c002fb0 tx=0x7f347c013000).stop 2021-07-11T03:55:45.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 1 -- 172.21.2.104:0/132866812 shutdown_connections 2021-07-11T03:55:45.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 1 --2- 172.21.2.104:0/132866812 >> [v2:172.21.2.104:6833/795410455,v1:172.21.2.104:6835/795410455] conn(0x220d850 0x22aec10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:45.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 1 --2- 172.21.2.104:0/132866812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x224c970 0x220d360 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:45.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 1 --2- 172.21.2.104:0/132866812 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x2248600 0x2210b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:45.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 1 --2- 172.21.2.104:0/132866812 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x2217f60 0x22ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:45.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f3466ffd700 0 client.5433 ms_handle_reset on v2:172.21.2.104:6833/795410455 2021-07-11T03:55:45.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 1 -- 172.21.2.104:0/132866812 >> 172.21.2.104:0/132866812 conn(0x2247c90 msgr2=0x2254780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:45.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 1 -- 172.21.2.104:0/132866812 shutdown_connections 2021-07-11T03:55:45.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 1 -- 172.21.2.104:0/132866812 wait complete. 2021-07-11T03:55:45.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.853+0000 7f349b838740 20 client.5433 trim_cache size 0 max 16384 2021-07-11T03:55:45.869 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:55:45.870 INFO:teuthology.orchestra.run.gibba004.stdout: "client_recall": { 2021-07-11T03:55:45.870 INFO:teuthology.orchestra.run.gibba004.stdout: "return_code": 0, 2021-07-11T03:55:45.870 INFO:teuthology.orchestra.run.gibba004.stdout: "message": "(0) Success", 2021-07-11T03:55:45.871 INFO:teuthology.orchestra.run.gibba004.stdout: "recalled": 220 2021-07-11T03:55:45.871 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:45.872 INFO:teuthology.orchestra.run.gibba004.stdout: "flush_journal": { 2021-07-11T03:55:45.872 INFO:teuthology.orchestra.run.gibba004.stdout: "return_code": 0, 2021-07-11T03:55:45.873 INFO:teuthology.orchestra.run.gibba004.stdout: "message": "" 2021-07-11T03:55:45.873 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:45.874 INFO:teuthology.orchestra.run.gibba004.stdout: "trim_cache": { 2021-07-11T03:55:45.874 INFO:teuthology.orchestra.run.gibba004.stdout: "trimmed": 0 2021-07-11T03:55:45.874 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:45.875 INFO:teuthology.orchestra.run.gibba004.stdout: "cache": { 2021-07-11T03:55:45.875 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": { 2021-07-11T03:55:45.876 INFO:teuthology.orchestra.run.gibba004.stdout: "items": 19841, 2021-07-11T03:55:45.876 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes": 3090160 2021-07-11T03:55:45.877 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:55:45.877 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:55:45.877 INFO:teuthology.orchestra.run.gibba004.stdout: "duration": 11.089051719 2021-07-11T03:55:45.878 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:55:45.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 -- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34680531a0 msgr2=0x7f3468055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:45.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34680531a0 0x7f3468055660 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f3480007a80 tx=0x7f34800076c0).stop 2021-07-11T03:55:45.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 -- 172.21.2.104:0/837298787 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f348c0aff10 msgr2=0x7f348c141e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:45.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f348c0aff10 0x7f348c141e50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f34840032f0 tx=0x7f3484016000).stop 2021-07-11T03:55:45.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 -- 172.21.2.104:0/837298787 shutdown_connections 2021-07-11T03:55:45.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f348c14ab80 0x7f348c142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:45.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34680531a0 0x7f3468055660 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:45.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f348c0aff10 0x7f348c141e50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:45.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 --2- 172.21.2.104:0/837298787 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f348c0ac9b0 0x7f348c0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:45.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 -- 172.21.2.104:0/837298787 >> 172.21.2.104:0/837298787 conn(0x7f348c0b5570 msgr2=0x7f348c13f440 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:45.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 -- 172.21.2.104:0/837298787 shutdown_connections 2021-07-11T03:55:45.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:45.857+0000 7f34915e8700 1 -- 172.21.2.104:0/837298787 wait complete. 2021-07-11T03:55:45.884 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2021-07-11T03:55:45.884 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:45.885 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:55:45.885 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:45.894 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-07-11T03:55:45.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:55:45.896 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-07-11T03:55:45.896 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T03:55:45.897 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T03:55:45.897 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:45.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:45.953 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:55:45.953 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:45.957 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T03:55:45.958 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T03:55:45.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:55:45.959 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:45.963 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:55:46.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.069+0000 7f2c20b61700 1 Processor -- start 2021-07-11T03:55:46.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.069+0000 7f2c20b61700 1 -- start start 2021-07-11T03:55:46.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.069+0000 7f2c20b61700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2c1c067180 0x7f2c1c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.069+0000 7f2c20b61700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2c1c0b1970 0x7f2c1c05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.069+0000 7f2c20b61700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c1c0a7a90 con 0x7f2c1c0a76b0 2021-07-11T03:55:46.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.069+0000 7f2c20b61700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c1c147620 con 0x7f2c1c0b1970 2021-07-11T03:55:46.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.069+0000 7f2c20b61700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c1478c0 con 0x7f2c1c067180 2021-07-11T03:55:46.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.069+0000 7f2c1a59c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f2c1c0a76b0 0x7f2c1c066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40964/0 (socket says 172.21.2.104:40964) 2021-07-11T03:55:46.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.069+0000 7f2c1a59c700 1 -- 172.21.2.104:0/2288831209 learned_addr learned my addr 172.21.2.104:0/2288831209 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:46.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.069+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215548907 0 0) 0x7f2c1c0a7a90 con 0x7f2c1c0a76b0 2021-07-11T03:55:46.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.069+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bfc003260 con 0x7f2c1c0a76b0 2021-07-11T03:55:46.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c19d9b700 1 --2- 172.21.2.104:0/2288831209 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2c1c067180 0x7f2c1c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 --2- 172.21.2.104:0/2288831209 >> v2:172.21.2.105:3300/0 conn(0x7f2c1c0b1970 0x7f2c1c05fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1859314637 0 0) 0x7f2bfc003260 con 0x7f2c1c0a76b0 2021-07-11T03:55:46.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2c1c067180 msgr2=0x7f2c1c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:46.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 --2- 172.21.2.104:0/2288831209 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2c1c067180 0x7f2c1c05f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 >> v2:172.21.2.105:3300/0 conn(0x7f2c1c0b1970 msgr2=0x7f2c1c05fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:46.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 --2- 172.21.2.104:0/2288831209 >> v2:172.21.2.105:3300/0 conn(0x7f2c1c0b1970 0x7f2c1c05fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c147b60 con 0x7f2c1c0a76b0 2021-07-11T03:55:46.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f2c04001680 con 0x7f2c1c0a76b0 2021-07-11T03:55:46.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 >> v1:172.21.2.104:6789/0 conn(0x7f2c1c0a76b0 legacy=0x7f2c1c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 --2- 172.21.2.104:0/2288831209 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2c1c067180 0x7f2bfc0037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 --2- 172.21.2.104:0/2288831209 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2bfc003db0 0x7f2bfc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 --2- 172.21.2.104:0/2288831209 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bfc0049f0 0x7f2bfc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c147620 con 0x7f2bfc0049f0 2021-07-11T03:55:46.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c1478c0 con 0x7f2bfc003db0 2021-07-11T03:55:46.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc00b880 con 0x7f2c1c067180 2021-07-11T03:55:46.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 --2- 172.21.2.104:0/2288831209 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bfc0049f0 0x7f2bfc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 -- 172.21.2.104:0/2288831209 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2c1c067180 msgr2=0x7f2bfc0037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:46.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 --2- 172.21.2.104:0/2288831209 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2c1c067180 0x7f2bfc0037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 -- 172.21.2.104:0/2288831209 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2bfc003db0 msgr2=0x7f2bfc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:46.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 --2- 172.21.2.104:0/2288831209 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2bfc003db0 0x7f2bfc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 -- 172.21.2.104:0/2288831209 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c147b60 con 0x7f2bfc0049f0 2021-07-11T03:55:46.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 --2- 172.21.2.104:0/2288831209 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bfc0049f0 0x7f2bfc004e70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f2c1400a000 tx=0x7f2c14008040).ready entity=mon.0 client_cookie=6a08604defb5758e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c1400c070 con 0x7f2bfc0049f0 2021-07-11T03:55:46.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1400e5f0 con 0x7f2bfc0049f0 2021-07-11T03:55:46.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1959a700 1 -- 172.21.2.104:0/2288831209 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c1400e950 con 0x7f2bfc0049f0 2021-07-11T03:55:46.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 -- 172.21.2.104:0/2288831209 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bfc0049f0 msgr2=0x7f2bfc004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 --2- 172.21.2.104:0/2288831209 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bfc0049f0 0x7f2bfc004e70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f2c1400a000 tx=0x7f2c14008040).stop 2021-07-11T03:55:46.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 -- 172.21.2.104:0/2288831209 reap_dead start 2021-07-11T03:55:46.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 -- 172.21.2.104:0/2288831209 shutdown_connections 2021-07-11T03:55:46.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 -- 172.21.2.104:0/2288831209 >> 172.21.2.104:0/2288831209 conn(0x7f2c1c0b53e0 msgr2=0x7f2c1c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 -- 172.21.2.104:0/2288831209 shutdown_connections 2021-07-11T03:55:46.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 -- 172.21.2.104:0/2288831209 wait complete. 2021-07-11T03:55:46.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 Processor -- start 2021-07-11T03:55:46.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 -- start start 2021-07-11T03:55:46.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2c1c0b41b0 0x7f2c1c0ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2c1c156790 0x7f2c1c0ae5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2c1c0ab510 0x7f2c1c0aa240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c0b4590 con 0x7f2c1c0ab510 2021-07-11T03:55:46.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c0ac6a0 con 0x7f2c1c156790 2021-07-11T03:55:46.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1c0ac940 con 0x7f2c1c0b41b0 2021-07-11T03:55:46.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2c1c0ab510 0x7f2c1c0aa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2c1c0ab510 0x7f2c1c0aa240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44210/0 (socket says 172.21.2.104:44210) 2021-07-11T03:55:46.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 -- 172.21.2.104:0/1846141160 learned_addr learned my addr 172.21.2.104:0/1846141160 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:46.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 -- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2c1c0b41b0 msgr2=0x7f2c1c0ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2c1c0b41b0 0x7f2c1c0ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 -- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2c1c156790 msgr2=0x7f2c1c0ae5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2c1c156790 0x7f2c1c0ae5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 -- 172.21.2.104:0/1846141160 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c0acbe0 con 0x7f2c1c0ab510 2021-07-11T03:55:46.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c1ad9d700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2c1c0ab510 0x7f2c1c0aa240 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f2c1c0a8ee0 tx=0x7f2c14008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c137fe700 1 -- 172.21.2.104:0/1846141160 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c1400eee0 con 0x7f2c1c0ab510 2021-07-11T03:55:46.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c137fe700 1 -- 172.21.2.104:0/1846141160 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1401d440 con 0x7f2c1c0ab510 2021-07-11T03:55:46.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c137fe700 1 -- 172.21.2.104:0/1846141160 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2c1401d7a0 con 0x7f2c1c0ab510 2021-07-11T03:55:46.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 -- 172.21.2.104:0/1846141160 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1c0b5070 con 0x7f2c1c0ab510 2021-07-11T03:55:46.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.073+0000 7f2c20b61700 1 -- 172.21.2.104:0/1846141160 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1c0abbc0 con 0x7f2c1c0ab510 2021-07-11T03:55:46.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.077+0000 7f2c20b61700 1 -- 172.21.2.104:0/1846141160 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c1c0ac500 con 0x7f2c1c0ab510 2021-07-11T03:55:46.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.077+0000 7f2c137fe700 1 -- 172.21.2.104:0/1846141160 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2c1401d900 con 0x7f2c1c0ab510 2021-07-11T03:55:46.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.077+0000 7f2c137fe700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2c00053180 0x7f2c00055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.077+0000 7f2c137fe700 1 -- 172.21.2.104:0/1846141160 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2c1406bdc0 con 0x7f2c1c0ab510 2021-07-11T03:55:46.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.077+0000 7f2c1a59c700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2c00053180 0x7f2c00055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.077+0000 7f2c1a59c700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2c00053180 0x7f2c00055640 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f2c04001390 tx=0x7f2c04001640).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.077+0000 7f2c137fe700 1 -- 172.21.2.104:0/1846141160 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2c1c0ac500 con 0x7f2c1c0ab510 2021-07-11T03:55:46.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 -- 172.21.2.104:0/1846141160 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c1c0ac500 con 0x7f2c1c0ab510 2021-07-11T03:55:46.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c137fe700 1 -- 172.21.2.104:0/1846141160 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16038 (secure 0 0 0) 0x7f2c1c0ac500 con 0x7f2c1c0ab510 2021-07-11T03:55:46.168 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:46.169 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":44,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:55:12.599036+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:55:09.448452+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2021-07-11T03:55:10.579101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:55:46.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 -- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2c00053180 msgr2=0x7f2c00055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2c00053180 0x7f2c00055640 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f2c04001390 tx=0x7f2c04001640).stop 2021-07-11T03:55:46.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 -- 172.21.2.104:0/1846141160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2c1c0ab510 msgr2=0x7f2c1c0aa240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2c1c0ab510 0x7f2c1c0aa240 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f2c1c0a8ee0 tx=0x7f2c14008040).stop 2021-07-11T03:55:46.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 -- 172.21.2.104:0/1846141160 shutdown_connections 2021-07-11T03:55:46.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2c00053180 0x7f2c00055640 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2c1c0ab510 0x7f2c1c0aa240 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2c1c156790 0x7f2c1c0ae5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 --2- 172.21.2.104:0/1846141160 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2c1c0b41b0 0x7f2c1c0ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 -- 172.21.2.104:0/1846141160 >> 172.21.2.104:0/1846141160 conn(0x7f2c1c0b53e0 msgr2=0x7f2c1c05fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.161+0000 7f2c20b61700 1 -- 172.21.2.104:0/1846141160 shutdown_connections 2021-07-11T03:55:46.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.165+0000 7f2c20b61700 1 -- 172.21.2.104:0/1846141160 wait complete. 2021-07-11T03:55:46.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:55:09.448452+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:55:10.579101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:55:46.179 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:55:46.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e63617700 1 Processor -- start 2021-07-11T03:55:46.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e63617700 1 -- start start 2021-07-11T03:55:46.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e63617700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e5c0603d0 0x7f9e5c0aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e63617700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5c061990 con 0x7f9e5c13f790 2021-07-11T03:55:46.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e63617700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5c061c30 con 0x7f9e5c0a9ec0 2021-07-11T03:55:46.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e63617700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c060b90 con 0x7f9e5c0603d0 2021-07-11T03:55:46.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e61bb4700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f9e5c13f790 0x7f9e5c14a020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:40982/0 (socket says 172.21.2.104:40982) 2021-07-11T03:55:46.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e61bb4700 1 -- 172.21.2.104:0/1949714391 learned_addr learned my addr 172.21.2.104:0/1949714391 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:46.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267917478 0 0) 0x7f9e5c061990 con 0x7f9e5c13f790 2021-07-11T03:55:46.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e44003260 con 0x7f9e5c13f790 2021-07-11T03:55:46.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 135898862 0 0) 0x7f9e44003260 con 0x7f9e5c13f790 2021-07-11T03:55:46.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e5c0603d0 msgr2=0x7f9e5c0aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:46.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e4bfff700 1 --2- 172.21.2.104:0/1949714391 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e5c0603d0 0x7f9e5c0aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 >> v1:172.21.2.105:6789/0 conn(0x7f9e5c0a9ec0 legacy=0x7f9e5c13ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:46.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c060e30 con 0x7f9e5c13f790 2021-07-11T03:55:46.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f9e540019e0 con 0x7f9e5c13f790 2021-07-11T03:55:46.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.225+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 >> v1:172.21.2.104:6789/0 conn(0x7f9e5c13f790 legacy=0x7f9e5c14a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4bfff700 1 --2- 172.21.2.104:0/1949714391 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e44003780 0x7f9e44003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4bfff700 1 --2- 172.21.2.104:0/1949714391 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e44004500 0x7f9e44004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4bfff700 1 --2- 172.21.2.104:0/1949714391 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e5c0603d0 0x7f9e44004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c060b90 con 0x7f9e44003780 2021-07-11T03:55:46.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e440058d0 con 0x7f9e44004500 2021-07-11T03:55:46.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e44005a90 con 0x7f9e5c0603d0 2021-07-11T03:55:46.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e613b3700 1 --2- 172.21.2.104:0/1949714391 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e44003780 0x7f9e44003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e613b3700 1 -- 172.21.2.104:0/1949714391 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e5c0603d0 msgr2=0x7f9e44004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:46.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e613b3700 1 --2- 172.21.2.104:0/1949714391 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e5c0603d0 0x7f9e44004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e613b3700 1 -- 172.21.2.104:0/1949714391 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e44004500 msgr2=0x7f9e44004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:46.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e613b3700 1 --2- 172.21.2.104:0/1949714391 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e44004500 0x7f9e44004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e613b3700 1 -- 172.21.2.104:0/1949714391 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e44003260 con 0x7f9e44003780 2021-07-11T03:55:46.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e613b3700 1 --2- 172.21.2.104:0/1949714391 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e44003780 0x7f9e44003b80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f9e4c000f70 tx=0x7f9e4c0079e0).ready entity=mon.0 client_cookie=3362e0427359e1a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e4c011750 con 0x7f9e44003780 2021-07-11T03:55:46.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c016030 con 0x7f9e44003780 2021-07-11T03:55:46.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4bfff700 1 -- 172.21.2.104:0/1949714391 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e4c011aa0 con 0x7f9e44003780 2021-07-11T03:55:46.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 -- 172.21.2.104:0/1949714391 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e44003780 msgr2=0x7f9e44003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 --2- 172.21.2.104:0/1949714391 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e44003780 0x7f9e44003b80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f9e4c000f70 tx=0x7f9e4c0079e0).stop 2021-07-11T03:55:46.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e61bb4700 1 -- 172.21.2.104:0/1949714391 reap_dead start 2021-07-11T03:55:46.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 -- 172.21.2.104:0/1949714391 shutdown_connections 2021-07-11T03:55:46.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 -- 172.21.2.104:0/1949714391 >> 172.21.2.104:0/1949714391 conn(0x7f9e5c0b2970 msgr2=0x7f9e5c0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 -- 172.21.2.104:0/1949714391 shutdown_connections 2021-07-11T03:55:46.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 -- 172.21.2.104:0/1949714391 wait complete. 2021-07-11T03:55:46.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 Processor -- start 2021-07-11T03:55:46.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 -- start start 2021-07-11T03:55:46.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e5c0ad480 0x7f9e5c141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e5c0a70c0 0x7f9e5c1421e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e5c0ad860 0x7f9e5c15a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c0a94b0 con 0x7f9e5c0ad860 2021-07-11T03:55:46.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c0b0200 con 0x7f9e5c0a70c0 2021-07-11T03:55:46.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c0a8f70 con 0x7f9e5c0ad480 2021-07-11T03:55:46.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e61bb4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e5c0ad860 0x7f9e5c15a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e61bb4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e5c0ad860 0x7f9e5c15a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44228/0 (socket says 172.21.2.104:44228) 2021-07-11T03:55:46.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e61bb4700 1 -- 172.21.2.104:0/715180837 learned_addr learned my addr 172.21.2.104:0/715180837 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:46.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e61bb4700 1 -- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e5c0ad480 msgr2=0x7f9e5c141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e61bb4700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e5c0ad480 0x7f9e5c141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e61bb4700 1 -- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e5c0a70c0 msgr2=0x7f9e5c1421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e61bb4700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e5c0a70c0 0x7f9e5c1421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e61bb4700 1 -- 172.21.2.104:0/715180837 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c0a9210 con 0x7f9e5c0ad860 2021-07-11T03:55:46.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e61bb4700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e5c0ad860 0x7f9e5c15a230 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f9e540016f0 tx=0x7f9e5c060fc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4a7fc700 1 -- 172.21.2.104:0/715180837 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e54004b70 con 0x7f9e5c0ad860 2021-07-11T03:55:46.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4a7fc700 1 -- 172.21.2.104:0/715180837 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e54004cd0 con 0x7f9e5c0ad860 2021-07-11T03:55:46.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e4a7fc700 1 -- 172.21.2.104:0/715180837 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e54007220 con 0x7f9e5c0ad860 2021-07-11T03:55:46.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 -- 172.21.2.104:0/715180837 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5c13f430 con 0x7f9e5c0ad860 2021-07-11T03:55:46.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.229+0000 7f9e63617700 1 -- 172.21.2.104:0/715180837 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c15a720 con 0x7f9e5c0ad860 2021-07-11T03:55:46.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.233+0000 7f9e63617700 1 -- 172.21.2.104:0/715180837 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5c057350 con 0x7f9e5c0ad860 2021-07-11T03:55:46.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.233+0000 7f9e4a7fc700 1 -- 172.21.2.104:0/715180837 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9e54010070 con 0x7f9e5c0ad860 2021-07-11T03:55:46.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.233+0000 7f9e4a7fc700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e3c053300 0x7f9e3c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.233+0000 7f9e4a7fc700 1 -- 172.21.2.104:0/715180837 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9e5406a710 con 0x7f9e5c0ad860 2021-07-11T03:55:46.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.233+0000 7f9e613b3700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e3c053300 0x7f9e3c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.233+0000 7f9e613b3700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e3c053300 0x7f9e3c0557c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9e4c000f70 tx=0x7f9e4c017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.233+0000 7f9e4a7fc700 1 -- 172.21.2.104:0/715180837 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9e54041a50 con 0x7f9e5c0ad860 2021-07-11T03:55:46.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 -- 172.21.2.104:0/715180837 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9e5c143b40 con 0x7f9e5c0ad860 2021-07-11T03:55:46.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e4a7fc700 1 -- 172.21.2.104:0/715180837 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9e54045070 con 0x7f9e5c0ad860 2021-07-11T03:55:46.324 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:55:46.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 -- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e3c053300 msgr2=0x7f9e3c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e3c053300 0x7f9e3c0557c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f9e4c000f70 tx=0x7f9e4c017000).stop 2021-07-11T03:55:46.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 -- 172.21.2.104:0/715180837 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e5c0ad860 msgr2=0x7f9e5c15a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e5c0ad860 0x7f9e5c15a230 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f9e540016f0 tx=0x7f9e5c060fc0).stop 2021-07-11T03:55:46.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 -- 172.21.2.104:0/715180837 shutdown_connections 2021-07-11T03:55:46.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e3c053300 0x7f9e3c0557c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e5c0ad860 0x7f9e5c15a230 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e5c0a70c0 0x7f9e5c1421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 --2- 172.21.2.104:0/715180837 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e5c0ad480 0x7f9e5c141cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 -- 172.21.2.104:0/715180837 >> 172.21.2.104:0/715180837 conn(0x7f9e5c0b2970 msgr2=0x7f9e5c0b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 -- 172.21.2.104:0/715180837 shutdown_connections 2021-07-11T03:55:46.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.317+0000 7f9e63617700 1 -- 172.21.2.104:0/715180837 wait complete. 2021-07-11T03:55:46.331 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:55:46.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 Processor -- start 2021-07-11T03:55:46.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 -- start start 2021-07-11T03:55:46.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7effc80b1ec0 0x7effc80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effc80affb0 0x7effc80b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effc80b0420 con 0x7effc80a70c0 2021-07-11T03:55:46.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc80b2790 con 0x7effc80b1ec0 2021-07-11T03:55:46.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc80b0650 con 0x7effc80affb0 2021-07-11T03:55:46.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc6d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7effc80a70c0 0x7effc8141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41004/0 (socket says 172.21.2.104:41004) 2021-07-11T03:55:46.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc6d9d700 1 -- 172.21.2.104:0/3033038896 learned_addr learned my addr 172.21.2.104:0/3033038896 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:46.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc659c700 1 --2- 172.21.2.104:0/3033038896 >> v2:172.21.2.104:3300/0 conn(0x7effc80b1ec0 0x7effc80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc659c700 1 -- 172.21.2.104:0/3033038896 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effc80affb0 msgr2=0x7effc80b22a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:46.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc659c700 1 --2- 172.21.2.104:0/3033038896 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effc80affb0 0x7effc80b22a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc659c700 1 -- 172.21.2.104:0/3033038896 >> v1:172.21.2.104:6789/0 conn(0x7effc80a70c0 legacy=0x7effc8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc659c700 1 -- 172.21.2.104:0/3033038896 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc80b08f0 con 0x7effc80b1ec0 2021-07-11T03:55:46.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc559a700 1 -- 172.21.2.104:0/3033038896 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2049294548 0 0) 0x7effc80b0420 con 0x7effc80a70c0 2021-07-11T03:55:46.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc659c700 1 --2- 172.21.2.104:0/3033038896 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effc80b1ec0 0x7effc80b12d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7effb8002f70 tx=0x7effb8006df0).ready entity=mon.0 client_cookie=d2eceaa234906352 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc559a700 1 -- 172.21.2.104:0/3033038896 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb800d570 con 0x7effc80b1ec0 2021-07-11T03:55:46.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc559a700 1 -- 172.21.2.104:0/3033038896 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb8000e90 con 0x7effc80b1ec0 2021-07-11T03:55:46.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effc559a700 1 -- 172.21.2.104:0/3033038896 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb800d8c0 con 0x7effc80b1ec0 2021-07-11T03:55:46.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 -- 172.21.2.104:0/3033038896 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effc80b1ec0 msgr2=0x7effc80b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 --2- 172.21.2.104:0/3033038896 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effc80b1ec0 0x7effc80b12d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7effb8002f70 tx=0x7effb8006df0).stop 2021-07-11T03:55:46.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 -- 172.21.2.104:0/3033038896 shutdown_connections 2021-07-11T03:55:46.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 --2- 172.21.2.104:0/3033038896 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effc80affb0 0x7effc80b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 --2- 172.21.2.104:0/3033038896 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effc80b1ec0 0x7effc80b12d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 -- 172.21.2.104:0/3033038896 >> 172.21.2.104:0/3033038896 conn(0x7effc805f6b0 msgr2=0x7effc805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 -- 172.21.2.104:0/3033038896 shutdown_connections 2021-07-11T03:55:46.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.377+0000 7effccb61700 1 -- 172.21.2.104:0/3033038896 wait complete. 2021-07-11T03:55:46.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effccb61700 1 Processor -- start 2021-07-11T03:55:46.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effccb61700 1 -- start start 2021-07-11T03:55:46.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effccb61700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effc80a70c0 0x7effc80a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effccb61700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effc80affb0 0x7effc81404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effccb61700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7effc80b1ec0 0x7effc8159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effccb61700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc80aa9e0 con 0x7effc80affb0 2021-07-11T03:55:46.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effccb61700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc814b090 con 0x7effc80b1ec0 2021-07-11T03:55:46.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effccb61700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc80aa740 con 0x7effc80a70c0 2021-07-11T03:55:46.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effc5d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effc80affb0 0x7effc81404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effc5d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effc80affb0 0x7effc81404c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44242/0 (socket says 172.21.2.104:44242) 2021-07-11T03:55:46.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effc5d9b700 1 -- 172.21.2.104:0/4193843630 learned_addr learned my addr 172.21.2.104:0/4193843630 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:46.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effc5d9b700 1 -- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effc80a70c0 msgr2=0x7effc80a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effc5d9b700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effc80a70c0 0x7effc80a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effc5d9b700 1 -- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7effc80b1ec0 msgr2=0x7effc8159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effc5d9b700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7effc80b1ec0 0x7effc8159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effc5d9b700 1 -- 172.21.2.104:0/4193843630 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb800b180 con 0x7effc80affb0 2021-07-11T03:55:46.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effc5d9b700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effc80affb0 0x7effc81404c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7effb0002fb0 tx=0x7effb0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effbf7fe700 1 -- 172.21.2.104:0/4193843630 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb0013460 con 0x7effc80affb0 2021-07-11T03:55:46.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effbf7fe700 1 -- 172.21.2.104:0/4193843630 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb00135c0 con 0x7effc80affb0 2021-07-11T03:55:46.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effccb61700 1 -- 172.21.2.104:0/4193843630 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effc814b2f0 con 0x7effc80affb0 2021-07-11T03:55:46.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effbf7fe700 1 -- 172.21.2.104:0/4193843630 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effb0013920 con 0x7effc80affb0 2021-07-11T03:55:46.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effccb61700 1 -- 172.21.2.104:0/4193843630 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc814ac50 con 0x7effc80affb0 2021-07-11T03:55:46.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effccb61700 1 -- 172.21.2.104:0/4193843630 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc814b480 con 0x7effc80affb0 2021-07-11T03:55:46.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effbf7fe700 1 -- 172.21.2.104:0/4193843630 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7effb000f070 con 0x7effc80affb0 2021-07-11T03:55:46.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effbf7fe700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effac053240 0x7effac055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effbf7fe700 1 -- 172.21.2.104:0/4193843630 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7effb006b070 con 0x7effc80affb0 2021-07-11T03:55:46.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.381+0000 7effc659c700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effac053240 0x7effac055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.385+0000 7effc659c700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effac053240 0x7effac055700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7effb8002f70 tx=0x7effb8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.385+0000 7effbf7fe700 1 -- 172.21.2.104:0/4193843630 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7effb00422a0 con 0x7effc80affb0 2021-07-11T03:55:46.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.465+0000 7effccb61700 1 -- 172.21.2.104:0/4193843630 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7effc8057350 con 0x7effc80affb0 2021-07-11T03:55:46.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.465+0000 7effbf7fe700 1 -- 172.21.2.104:0/4193843630 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v44) v1 ==== 92+0+10 (secure 0 0 0) 0x7effb00458c0 con 0x7effc80affb0 2021-07-11T03:55:46.474 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:55:46.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 -- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effac053240 msgr2=0x7effac055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effac053240 0x7effac055700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7effb8002f70 tx=0x7effb8015000).stop 2021-07-11T03:55:46.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 -- 172.21.2.104:0/4193843630 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effc80affb0 msgr2=0x7effc81404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effc80affb0 0x7effc81404c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7effb0002fb0 tx=0x7effb0006df0).stop 2021-07-11T03:55:46.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 -- 172.21.2.104:0/4193843630 shutdown_connections 2021-07-11T03:55:46.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7effc80b1ec0 0x7effc8159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effac053240 0x7effac055700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effc80affb0 0x7effc81404c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 --2- 172.21.2.104:0/4193843630 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effc80a70c0 0x7effc80a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 -- 172.21.2.104:0/4193843630 >> 172.21.2.104:0/4193843630 conn(0x7effc805f6b0 msgr2=0x7effc8145ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 -- 172.21.2.104:0/4193843630 shutdown_connections 2021-07-11T03:55:46.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.469+0000 7effccb61700 1 -- 172.21.2.104:0/4193843630 wait complete. 2021-07-11T03:55:46.481 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:55:46.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8e011f5700 1 Processor -- start 2021-07-11T03:55:46.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8e011f5700 1 -- start start 2021-07-11T03:55:46.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8e011f5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8dfc0b1ec0 0x7f8dfc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8e011f5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8dfc0affb0 0x7f8dfc0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8e011f5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dfc0b2790 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8e011f5700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dfc0b03f0 con 0x7f8dfc0affb0 2021-07-11T03:55:46.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8e011f5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc0b0650 con 0x7f8dfc0b1ec0 2021-07-11T03:55:46.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8dfa59c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8dfc0affb0 0x7f8dfc0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8dfa59c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8dfc0affb0 0x7f8dfc0b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:51812/0 (socket says 172.21.2.104:51812) 2021-07-11T03:55:46.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8dfa59c700 1 -- 172.21.2.104:0/2782102426 learned_addr learned my addr 172.21.2.104:0/2782102426 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:46.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8dfb59e700 1 --1- 172.21.2.104:0/2782102426 >> v1:172.21.2.105:6789/0 conn(0x7f8dfc0a70c0 0x7f8dfc141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:40388/0 (socket says 172.21.2.104:40388) 2021-07-11T03:55:46.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8dfad9d700 1 --2- 172.21.2.104:0/2782102426 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8dfc0b1ec0 0x7f8dfc0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8dfa59c700 1 -- 172.21.2.104:0/2782102426 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8dfc0b1ec0 msgr2=0x7f8dfc0b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:55:46.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8dfa59c700 1 --2- 172.21.2.104:0/2782102426 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8dfc0b1ec0 0x7f8dfc0b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8dfa59c700 1 -- 172.21.2.104:0/2782102426 >> v1:172.21.2.105:6789/0 conn(0x7f8dfc0a70c0 legacy=0x7f8dfc141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8dfa59c700 1 -- 172.21.2.104:0/2782102426 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc0b08f0 con 0x7f8dfc0affb0 2021-07-11T03:55:46.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8dfa59c700 1 --2- 172.21.2.104:0/2782102426 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8dfc0affb0 0x7f8dfc0b22a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f8df0002f70 tx=0x7f8df0006eb0).ready entity=mon.1 client_cookie=628fdb839aeb6bdb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8df9d9b700 1 -- 172.21.2.104:0/2782102426 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df000d570 con 0x7f8dfc0affb0 2021-07-11T03:55:46.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8df9d9b700 1 -- 172.21.2.104:0/2782102426 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df0000e90 con 0x7f8dfc0affb0 2021-07-11T03:55:46.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8df9d9b700 1 -- 172.21.2.104:0/2782102426 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df000d900 con 0x7f8dfc0affb0 2021-07-11T03:55:46.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8e011f5700 1 -- 172.21.2.104:0/2782102426 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8dfc0affb0 msgr2=0x7f8dfc0b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.529+0000 7f8e011f5700 1 --2- 172.21.2.104:0/2782102426 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8dfc0affb0 0x7f8dfc0b22a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f8df0002f70 tx=0x7f8df0006eb0).stop 2021-07-11T03:55:46.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 -- 172.21.2.104:0/2782102426 shutdown_connections 2021-07-11T03:55:46.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 --2- 172.21.2.104:0/2782102426 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8dfc0affb0 0x7f8dfc0b22a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 --2- 172.21.2.104:0/2782102426 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8dfc0b1ec0 0x7f8dfc0b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 -- 172.21.2.104:0/2782102426 >> 172.21.2.104:0/2782102426 conn(0x7f8dfc05f6b0 msgr2=0x7f8dfc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 -- 172.21.2.104:0/2782102426 shutdown_connections 2021-07-11T03:55:46.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 -- 172.21.2.104:0/2782102426 wait complete. 2021-07-11T03:55:46.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 Processor -- start 2021-07-11T03:55:46.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 -- start start 2021-07-11T03:55:46.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8dfc0a70c0 0x7f8dfc0b6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8dfc0affb0 0x7f8dfc14b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8dfc0b1ec0 0x7f8dfc0ab940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc15af00 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc14bb50 con 0x7f8dfc0b1ec0 2021-07-11T03:55:46.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc14ac00 con 0x7f8dfc0affb0 2021-07-11T03:55:46.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8dfad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8dfc0a70c0 0x7f8dfc0b6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8dfad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8dfc0a70c0 0x7f8dfc0b6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44254/0 (socket says 172.21.2.104:44254) 2021-07-11T03:55:46.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8dfad9d700 1 -- 172.21.2.104:0/513189814 learned_addr learned my addr 172.21.2.104:0/513189814 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:46.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8dfad9d700 1 -- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8dfc0affb0 msgr2=0x7f8dfc14b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8dfad9d700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8dfc0affb0 0x7f8dfc14b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8dfad9d700 1 -- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8dfc0b1ec0 msgr2=0x7f8dfc0ab940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8dfad9d700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8dfc0b1ec0 0x7f8dfc0ab940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8dfad9d700 1 -- 172.21.2.104:0/513189814 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df000b1e0 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8dfad9d700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8dfc0a70c0 0x7f8dfc0b6060 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f8dec00c000 tx=0x7f8dec006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8debfff700 1 -- 172.21.2.104:0/513189814 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dec010780 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8debfff700 1 -- 172.21.2.104:0/513189814 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec0108e0 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8debfff700 1 -- 172.21.2.104:0/513189814 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8dec010c40 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 -- 172.21.2.104:0/513189814 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dfc14aea0 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 -- 172.21.2.104:0/513189814 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dfc0abe30 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8e011f5700 1 -- 172.21.2.104:0/513189814 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dfc14b400 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8debfff700 1 -- 172.21.2.104:0/513189814 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8dec01c460 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8debfff700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dd8053270 0x7f8dd8055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.533+0000 7f8debfff700 1 -- 172.21.2.104:0/513189814 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8dec06b900 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.537+0000 7f8dfa59c700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dd8053270 0x7f8dd8055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.537+0000 7f8dfa59c700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dd8053270 0x7f8dd8055730 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f8df0002f70 tx=0x7f8df0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.537+0000 7f8debfff700 1 -- 172.21.2.104:0/513189814 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8dec042be0 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.617+0000 7f8e011f5700 1 -- 172.21.2.104:0/513189814 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8dfc057350 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.617+0000 7f8debfff700 1 -- 172.21.2.104:0/513189814 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8dec046200 con 0x7f8dfc0a70c0 2021-07-11T03:55:46.625 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:55:46.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 -- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dd8053270 msgr2=0x7f8dd8055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dd8053270 0x7f8dd8055730 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f8df0002f70 tx=0x7f8df0015000).stop 2021-07-11T03:55:46.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 -- 172.21.2.104:0/513189814 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8dfc0a70c0 msgr2=0x7f8dfc0b6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8dfc0a70c0 0x7f8dfc0b6060 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f8dec00c000 tx=0x7f8dec006df0).stop 2021-07-11T03:55:46.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 -- 172.21.2.104:0/513189814 shutdown_connections 2021-07-11T03:55:46.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8dfc0b1ec0 0x7f8dfc0ab940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8dfc0affb0 0x7f8dfc14b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dd8053270 0x7f8dd8055730 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 --2- 172.21.2.104:0/513189814 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8dfc0a70c0 0x7f8dfc0b6060 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 -- 172.21.2.104:0/513189814 >> 172.21.2.104:0/513189814 conn(0x7f8dfc05f6b0 msgr2=0x7f8dfc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 -- 172.21.2.104:0/513189814 shutdown_connections 2021-07-11T03:55:46.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.621+0000 7f8e011f5700 1 -- 172.21.2.104:0/513189814 wait complete. 2021-07-11T03:55:46.632 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:55:46.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0fd74700 1 Processor -- start 2021-07-11T03:55:46.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0fd74700 1 -- start start 2021-07-11T03:55:46.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0fd74700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d080b1ec0 0x7f2d080b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0fd74700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2d080affb0 0x7f2d080b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0fd74700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2d080a70c0 0x7f2d08141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0fd74700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d080b2790 con 0x7f2d080a70c0 2021-07-11T03:55:46.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0fd74700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d080b03f0 con 0x7f2d080affb0 2021-07-11T03:55:46.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0fd74700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080b0650 con 0x7f2d080b1ec0 2021-07-11T03:55:46.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0e311700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2d080a70c0 0x7f2d08141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0e311700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2d080a70c0 0x7f2d08141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44266/0 (socket says 172.21.2.104:44266) 2021-07-11T03:55:46.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0e311700 1 -- 172.21.2.104:0/89692818 learned_addr learned my addr 172.21.2.104:0/89692818 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:46.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0e311700 1 -- 172.21.2.104:0/89692818 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d080b1ec0 msgr2=0x7f2d080b12d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:46.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0e311700 1 --2- 172.21.2.104:0/89692818 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d080b1ec0 0x7f2d080b12d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0e311700 1 -- 172.21.2.104:0/89692818 >> v2:172.21.2.105:3300/0 conn(0x7f2d080affb0 msgr2=0x7f2d080b22a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:46.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0e311700 1 --2- 172.21.2.104:0/89692818 >> v2:172.21.2.105:3300/0 conn(0x7f2d080affb0 0x7f2d080b22a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0e311700 1 -- 172.21.2.104:0/89692818 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080b08f0 con 0x7f2d080a70c0 2021-07-11T03:55:46.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0e311700 1 --2- 172.21.2.104:0/89692818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d080a70c0 0x7f2d08141250 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f2d000032f0 tx=0x7f2d00009290).ready entity=mon.0 client_cookie=19861e96826c36d6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0cb0e700 1 -- 172.21.2.104:0/89692818 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d0000b870 con 0x7f2d080a70c0 2021-07-11T03:55:46.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0cb0e700 1 -- 172.21.2.104:0/89692818 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0000b9d0 con 0x7f2d080a70c0 2021-07-11T03:55:46.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0cb0e700 1 -- 172.21.2.104:0/89692818 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d0000bb80 con 0x7f2d080a70c0 2021-07-11T03:55:46.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0fd74700 1 -- 172.21.2.104:0/89692818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d080a70c0 msgr2=0x7f2d08141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.677+0000 7f2d0fd74700 1 --2- 172.21.2.104:0/89692818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d080a70c0 0x7f2d08141250 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f2d000032f0 tx=0x7f2d00009290).stop 2021-07-11T03:55:46.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 -- 172.21.2.104:0/89692818 shutdown_connections 2021-07-11T03:55:46.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 --2- 172.21.2.104:0/89692818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d080a70c0 0x7f2d08141250 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 --2- 172.21.2.104:0/89692818 >> v2:172.21.2.105:3300/0 conn(0x7f2d080affb0 0x7f2d080b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 --2- 172.21.2.104:0/89692818 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d080b1ec0 0x7f2d080b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 -- 172.21.2.104:0/89692818 >> 172.21.2.104:0/89692818 conn(0x7f2d0805f6b0 msgr2=0x7f2d0805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 -- 172.21.2.104:0/89692818 shutdown_connections 2021-07-11T03:55:46.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 -- 172.21.2.104:0/89692818 wait complete. 2021-07-11T03:55:46.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 Processor -- start 2021-07-11T03:55:46.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 -- start start 2021-07-11T03:55:46.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d080a70c0 0x7f2d08140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d080affb0 0x7f2d0813c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d080b1ec0 0x7f2d0813cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08145320 con 0x7f2d080a70c0 2021-07-11T03:55:46.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080b3210 con 0x7f2d080b1ec0 2021-07-11T03:55:46.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d08157580 con 0x7f2d080affb0 2021-07-11T03:55:46.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0db10700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d080a70c0 0x7f2d08140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0db10700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d080a70c0 0x7f2d08140300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44270/0 (socket says 172.21.2.104:44270) 2021-07-11T03:55:46.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0db10700 1 -- 172.21.2.104:0/1858526554 learned_addr learned my addr 172.21.2.104:0/1858526554 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:46.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0db10700 1 -- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d080affb0 msgr2=0x7f2d0813c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0db10700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d080affb0 0x7f2d0813c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0db10700 1 -- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d080b1ec0 msgr2=0x7f2d0813cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0db10700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d080b1ec0 0x7f2d0813cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0db10700 1 -- 172.21.2.104:0/1858526554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d00009530 con 0x7f2d080a70c0 2021-07-11T03:55:46.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0db10700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d080a70c0 0x7f2d08140300 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f2cf8000ed0 tx=0x7f2cf8006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2cf6ffd700 1 -- 172.21.2.104:0/1858526554 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cf8012410 con 0x7f2d080a70c0 2021-07-11T03:55:46.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2cf6ffd700 1 -- 172.21.2.104:0/1858526554 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf8012570 con 0x7f2d080a70c0 2021-07-11T03:55:46.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 -- 172.21.2.104:0/1858526554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d08157820 con 0x7f2d080a70c0 2021-07-11T03:55:46.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2cf6ffd700 1 -- 172.21.2.104:0/1858526554 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cf80128d0 con 0x7f2d080a70c0 2021-07-11T03:55:46.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 -- 172.21.2.104:0/1858526554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d080b5110 con 0x7f2d080a70c0 2021-07-11T03:55:46.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2d0fd74700 1 -- 172.21.2.104:0/1858526554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d08057350 con 0x7f2d080a70c0 2021-07-11T03:55:46.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2cf6ffd700 1 -- 172.21.2.104:0/1858526554 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2cf800f030 con 0x7f2d080a70c0 2021-07-11T03:55:46.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2cf6ffd700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ce8053350 0x7f2ce8055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.681+0000 7f2cf6ffd700 1 -- 172.21.2.104:0/1858526554 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2cf806b090 con 0x7f2d080a70c0 2021-07-11T03:55:46.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.685+0000 7f2d0d30f700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ce8053350 0x7f2ce8055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.685+0000 7f2d0d30f700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ce8053350 0x7f2ce8055810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f2d04002fb0 tx=0x7f2d040069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.685+0000 7f2cf6ffd700 1 -- 172.21.2.104:0/1858526554 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2cf80423d0 con 0x7f2d080a70c0 2021-07-11T03:55:46.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.765+0000 7f2d0fd74700 1 -- 172.21.2.104:0/1858526554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2d080b3350 con 0x7f2d080a70c0 2021-07-11T03:55:46.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2cf6ffd700 1 -- 172.21.2.104:0/1858526554 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+10 (secure 0 0 0) 0x7f2cf80459f0 con 0x7f2d080a70c0 2021-07-11T03:55:46.774 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:55:46.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 -- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ce8053350 msgr2=0x7f2ce8055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ce8053350 0x7f2ce8055810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f2d04002fb0 tx=0x7f2d040069d0).stop 2021-07-11T03:55:46.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 -- 172.21.2.104:0/1858526554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d080a70c0 msgr2=0x7f2d08140300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d080a70c0 0x7f2d08140300 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f2cf8000ed0 tx=0x7f2cf8006d70).stop 2021-07-11T03:55:46.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 -- 172.21.2.104:0/1858526554 shutdown_connections 2021-07-11T03:55:46.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d080b1ec0 0x7f2d0813cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d080affb0 0x7f2d0813c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ce8053350 0x7f2ce8055810 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 --2- 172.21.2.104:0/1858526554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d080a70c0 0x7f2d08140300 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 -- 172.21.2.104:0/1858526554 >> 172.21.2.104:0/1858526554 conn(0x7f2d0805f6b0 msgr2=0x7f2d0813e540 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 -- 172.21.2.104:0/1858526554 shutdown_connections 2021-07-11T03:55:46.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.769+0000 7f2d0fd74700 1 -- 172.21.2.104:0/1858526554 wait complete. 2021-07-11T03:55:46.781 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T03:55:46.782 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:55:46.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd434809700 1 Processor -- start 2021-07-11T03:55:46.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd434809700 1 -- start start 2021-07-11T03:55:46.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd434809700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd4300a9da0 0x7fd4300b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd434809700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4300ac370 con 0x7fd4300ad150 2021-07-11T03:55:46.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd434809700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4300aac40 con 0x7fd4300aa860 2021-07-11T03:55:46.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd434809700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4300abb50 con 0x7fd4300a9da0 2021-07-11T03:55:46.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd42dd9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd4300ad150 0x7fd430140340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41046/0 (socket says 172.21.2.104:41046) 2021-07-11T03:55:46.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd42dd9b700 1 -- 172.21.2.104:0/2643530445 learned_addr learned my addr 172.21.2.104:0/2643530445 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:55:46.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd42ed9d700 1 --2- 172.21.2.104:0/2643530445 >> v2:172.21.2.104:3300/0 conn(0x7fd4300a9da0 0x7fd4300b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd42d59a700 1 -- 172.21.2.104:0/2643530445 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877499005 0 0) 0x7fd4300ac370 con 0x7fd4300ad150 2021-07-11T03:55:46.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd42d59a700 1 -- 172.21.2.104:0/2643530445 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd414003260 con 0x7fd4300ad150 2021-07-11T03:55:46.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd42ed9d700 1 -- 172.21.2.104:0/2643530445 >> v1:172.21.2.105:6789/0 conn(0x7fd4300aa860 legacy=0x7fd4300aca40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:55:46.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd42ed9d700 1 -- 172.21.2.104:0/2643530445 >> v1:172.21.2.104:6789/0 conn(0x7fd4300ad150 legacy=0x7fd430140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd42ed9d700 1 -- 172.21.2.104:0/2643530445 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4300b16b0 con 0x7fd4300a9da0 2021-07-11T03:55:46.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.825+0000 7fd42ed9d700 1 --2- 172.21.2.104:0/2643530445 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4300a9da0 0x7fd4300b11c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fd4300b63e0 tx=0x7fd4280091d0).ready entity=mon.0 client_cookie=d0e6f7ac93844422 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42d59a700 1 -- 172.21.2.104:0/2643530445 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd42800b680 con 0x7fd4300a9da0 2021-07-11T03:55:46.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42d59a700 1 -- 172.21.2.104:0/2643530445 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42800b7e0 con 0x7fd4300a9da0 2021-07-11T03:55:46.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42d59a700 1 -- 172.21.2.104:0/2643530445 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd42800b950 con 0x7fd4300a9da0 2021-07-11T03:55:46.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- 172.21.2.104:0/2643530445 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4300a9da0 msgr2=0x7fd4300b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 --2- 172.21.2.104:0/2643530445 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4300a9da0 0x7fd4300b11c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fd4300b63e0 tx=0x7fd4280091d0).stop 2021-07-11T03:55:46.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- 172.21.2.104:0/2643530445 shutdown_connections 2021-07-11T03:55:46.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 --2- 172.21.2.104:0/2643530445 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4300a9da0 0x7fd4300b11c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- 172.21.2.104:0/2643530445 >> 172.21.2.104:0/2643530445 conn(0x7fd4300b64d0 msgr2=0x7fd4300b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- 172.21.2.104:0/2643530445 shutdown_connections 2021-07-11T03:55:46.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- 172.21.2.104:0/2643530445 wait complete. 2021-07-11T03:55:46.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 Processor -- start 2021-07-11T03:55:46.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- start start 2021-07-11T03:55:46.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd4300a9da0 0x7fd4300a8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd4300aa860 0x7fd430142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4300ad150 0x7fd430142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43011e1c0 con 0x7fd4300ad150 2021-07-11T03:55:46.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43011e460 con 0x7fd4300aa860 2021-07-11T03:55:46.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43011e700 con 0x7fd4300a9da0 2021-07-11T03:55:46.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4300ad150 0x7fd430142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4300ad150 0x7fd430142c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44282/0 (socket says 172.21.2.104:44282) 2021-07-11T03:55:46.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42ed9d700 1 -- 172.21.2.104:0/2166649 learned_addr learned my addr 172.21.2.104:0/2166649 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:55:46.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42ed9d700 1 -- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd4300a9da0 msgr2=0x7fd4300a8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42ed9d700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd4300a9da0 0x7fd4300a8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42ed9d700 1 -- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd4300aa860 msgr2=0x7fd430142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42ed9d700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd4300aa860 0x7fd430142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42ed9d700 1 -- 172.21.2.104:0/2166649 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd428009470 con 0x7fd4300ad150 2021-07-11T03:55:46.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd42ed9d700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4300ad150 0x7fd430142c60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fd4300b63e0 tx=0x7fd428016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd41f7fe700 1 -- 172.21.2.104:0/2166649 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd42800bed0 con 0x7fd4300ad150 2021-07-11T03:55:46.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd41f7fe700 1 -- 172.21.2.104:0/2166649 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4280032f0 con 0x7fd4300ad150 2021-07-11T03:55:46.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd41f7fe700 1 -- 172.21.2.104:0/2166649 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd428021670 con 0x7fd4300ad150 2021-07-11T03:55:46.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- 172.21.2.104:0/2166649 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43011e9a0 con 0x7fd4300ad150 2021-07-11T03:55:46.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- 172.21.2.104:0/2166649 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd430143150 con 0x7fd4300ad150 2021-07-11T03:55:46.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd434809700 1 -- 172.21.2.104:0/2166649 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43011eb30 con 0x7fd4300ad150 2021-07-11T03:55:46.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.829+0000 7fd41f7fe700 1 -- 172.21.2.104:0/2166649 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd42800f070 con 0x7fd4300ad150 2021-07-11T03:55:46.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.833+0000 7fd41f7fe700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd40c053300 0x7fd40c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:55:46.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.833+0000 7fd41f7fe700 1 -- 172.21.2.104:0/2166649 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd42806c630 con 0x7fd4300ad150 2021-07-11T03:55:46.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.833+0000 7fd42e59c700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd40c053300 0x7fd40c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:55:46.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.833+0000 7fd42e59c700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd40c053300 0x7fd40c0557c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fd420002fb0 tx=0x7fd4200069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:55:46.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.833+0000 7fd41f7fe700 1 -- 172.21.2.104:0/2166649 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd428043970 con 0x7fd4300ad150 2021-07-11T03:55:46.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 -- 172.21.2.104:0/2166649 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd430057350 con 0x7fd4300ad150 2021-07-11T03:55:46.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd41f7fe700 1 -- 172.21.2.104:0/2166649 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2488 (secure 0 0 0) 0x7fd428026090 con 0x7fd4300ad150 2021-07-11T03:55:46.947 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:55:46.948 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4492,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2412138431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2412138431},{"type":"v1","addr":"172.21.2.105:6835","nonce":2412138431}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/3271417890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3271417890},{"type":"v1","addr":"172.21.2.104:6834","nonce":3271417890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:55:11.595553+0000","modified":"2021-07-11T03:55:12.600491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/795410455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":795410455},{"type":"v1","addr":"172.21.2.104:6835","nonce":795410455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-07-11T03:55:46.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 -- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd40c053300 msgr2=0x7fd40c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd40c053300 0x7fd40c0557c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fd420002fb0 tx=0x7fd4200069d0).stop 2021-07-11T03:55:46.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 -- 172.21.2.104:0/2166649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4300ad150 msgr2=0x7fd430142c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:55:46.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4300ad150 0x7fd430142c60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fd4300b63e0 tx=0x7fd428016000).stop 2021-07-11T03:55:46.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 -- 172.21.2.104:0/2166649 shutdown_connections 2021-07-11T03:55:46.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd40c053300 0x7fd40c0557c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4300ad150 0x7fd430142c60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:55:46.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd4300aa860 0x7fd430142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 --2- 172.21.2.104:0/2166649 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd4300a9da0 0x7fd4300a8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:55:46.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 -- 172.21.2.104:0/2166649 >> 172.21.2.104:0/2166649 conn(0x7fd4300b64d0 msgr2=0x7fd430060530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:55:46.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 -- 172.21.2.104:0/2166649 shutdown_connections 2021-07-11T03:55:46.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:55:46.941+0000 7fd434809700 1 -- 172.21.2.104:0/2166649 wait complete. 2021-07-11T03:55:46.951 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 16 2021-07-11T03:55:47.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:55:11.595553+0000', 'modified': '2021-07-11T03:55:12.600491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5232}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5232': {'gid': 5232, 'name': 'c', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/795410455', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 795410455}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 795410455}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:55:47.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:55:47.258 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:55:47.321 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":5232,"want_state":"up:active","state":"up:active","rank_uptime":35.717214904999999,"mdsmap_epoch":16,"osdmap_epoch":44,"osdmap_epoch_barrier":44,"uptime":39.875532733999997} 2021-07-11T03:55:47.322 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 5232, "mdsmap_epoch": 16, "osdmap_epoch": 44, "osdmap_epoch_barrier": 44, "rank_uptime": 35.717214905, "state": "up:active", "uptime": 39.875532734, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:55:47.322 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T03:55:47.323 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T03:55:47.323 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T03:55:47.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:47.324 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:47.349 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:55:47.363 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:55:47.363 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:55:47.369 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:55:47.370 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:55:47.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:47.371 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:55:47.418 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T03:55:47.418 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:47.421 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:55:47.430 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:55:47.431 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:55:47.435 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:55:47.436 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:55:47.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:47.437 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:55:47.465 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14783]: starting ceph client2021-07-11T03:55:47.467+0000 7f987b4d8080 -1 init, newargv = 0x56505586fa30 newargc=9 2021-07-11T03:55:47.466 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr: 2021-07-11T03:55:47.484 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14783]: starting fuse 2021-07-11T03:55:48.542 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:55:48.555 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:55:48.555 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:55:48.604 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:55:48.605 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:55:48.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:48.606 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:55:48.611 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T03:55:48.612 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T03:55:48.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:48.612 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T03:55:48.613 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T03:55:48.613 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T03:55:48.614 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:55:48.614 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T03:55:48.614 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:55:48.615 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T03:55:48.616 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T03:55:48.616 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T03:55:48.617 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:55:48.617 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T03:55:48.617 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T03:55:48.618 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T03:55:48.618 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T03:55:48.618 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:55:48.619 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T03:55:48.619 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T03:55:48.620 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T03:55:48.620 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T03:55:48.620 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T03:55:48.621 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:55:48.621 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T03:55:48.622 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:55:48.651 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14783.asok 2021-07-11T03:55:48.654 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14783.asok 2021-07-11T03:55:48.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:48.656 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14783.asok status 2021-07-11T03:55:48.747 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T03:55:48.748 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T03:55:48.748 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T03:55:48.748 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T03:55:48.749 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T03:55:48.749 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T03:55:48.749 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T03:55:48.749 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14783", 2021-07-11T03:55:48.749 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T03:55:48.750 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:55:48.750 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T03:55:48.750 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T03:55:48.750 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4439, 2021-07-11T03:55:48.750 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T03:55:48.751 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T03:55:48.751 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T03:55:48.751 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4439 2021-07-11T03:55:48.751 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:55:48.752 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:55:48.752 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:55:48.752 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:55:48.752 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2699644299 2021-07-11T03:55:48.752 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T03:55:48.753 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:55:48.753 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:55:48.753 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:55:48.753 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:55:48.753 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2699644299 2021-07-11T03:55:48.754 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:55:48.754 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4439 172.21.2.119:0/2699644299", 2021-07-11T03:55:48.754 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/2699644299", 2021-07-11T03:55:48.754 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T03:55:48.755 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 16, 2021-07-11T03:55:48.755 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 44, 2021-07-11T03:55:48.755 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T03:55:48.755 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T03:55:48.755 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T03:55:48.756 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T03:55:48.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:48.757 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:55:48.758 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:55:48.763 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:55:48.764 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:55:48.764 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:55:48.765 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:56:12.250 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:56:12.253 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:56:12.256 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:56:12.259 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:56:37.452 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T03:56:37.453 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:56:37.466 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:56:37.480 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:56:37.493 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:56:37.506 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T03:56:37.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:37.507 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:56:37.508 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:56:37.544 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:56:37.546 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:56:37.546 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T03:56:37.547 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T03:56:37.547 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:56:37.561 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14783]: fuse finished with error 0 and tester_r 0 2021-07-11T03:56:37.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:37.563 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:56:37.564 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:56:37.568 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T03:56:37.568 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T03:56:37.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:56:37.607 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:56:37.659 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:37.660 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:56:37.660 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:56:37.670 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:56:37.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:56:37.671 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T03:56:37.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:56:37.672 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:56:37.681 INFO:teuthology.orchestra.run.gibba023.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:56:37.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:56:37.683 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T03:56:37.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.729+0000 7f31e45f8700 1 Processor -- start 2021-07-11T03:56:37.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.729+0000 7f31e45f8700 1 -- start start 2021-07-11T03:56:37.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc0a8950 0x7f31dc0a8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:37.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31dc1434c0 con 0x7f31dc139ae0 2021-07-11T03:56:37.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31dc0ac470 con 0x7f31dc0b1c50 2021-07-11T03:56:37.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc0b3010 con 0x7f31dc0a8950 2021-07-11T03:56:37.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e2394700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f31dc139ae0 0x7f31dc142db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41058/0 (socket says 172.21.2.104:41058) 2021-07-11T03:56:37.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e2394700 1 -- 172.21.2.104:0/3570800565 learned_addr learned my addr 172.21.2.104:0/3570800565 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:37.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692171590 0 0) 0x7f31dc1434c0 con 0x7f31dc139ae0 2021-07-11T03:56:37.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31c4003260 con 0x7f31dc139ae0 2021-07-11T03:56:37.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 745727118 0 0) 0x7f31c4003260 con 0x7f31dc139ae0 2021-07-11T03:56:37.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc0a8950 msgr2=0x7f31dc0a8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:37.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 --2- 172.21.2.104:0/3570800565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc0a8950 0x7f31dc0a8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:37.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 >> v1:172.21.2.105:6789/0 conn(0x7f31dc0b1c50 legacy=0x7f31dc0ab530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:37.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc143730 con 0x7f31dc139ae0 2021-07-11T03:56:37.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f31cc001660 con 0x7f31dc139ae0 2021-07-11T03:56:37.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 >> v1:172.21.2.104:6789/0 conn(0x7f31dc139ae0 legacy=0x7f31dc142db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:37.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 --2- 172.21.2.104:0/3570800565 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31c4003800 0x7f31c4003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:37.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 --2- 172.21.2.104:0/3570800565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc0a8950 0x7f31c4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:37.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 --2- 172.21.2.104:0/3570800565 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31c4004bf0 0x7f31c400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:37.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc0b3010 con 0x7f31c4004bf0 2021-07-11T03:56:37.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c40052a0 con 0x7f31c4003800 2021-07-11T03:56:37.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31c400bc40 con 0x7f31dc0a8950 2021-07-11T03:56:37.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e2b95700 1 --2- 172.21.2.104:0/3570800565 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31c4004bf0 0x7f31c400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:37.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e2394700 1 --2- 172.21.2.104:0/3570800565 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31c4003800 0x7f31c4003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:37.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e2b95700 1 -- 172.21.2.104:0/3570800565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc0a8950 msgr2=0x7f31c4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:37.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1b93700 1 --2- 172.21.2.104:0/3570800565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc0a8950 0x7f31c4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:37.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e2b95700 1 --2- 172.21.2.104:0/3570800565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc0a8950 0x7f31c4004500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:37.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e2b95700 1 -- 172.21.2.104:0/3570800565 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31c4003800 msgr2=0x7f31c4003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:37.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e2b95700 1 --2- 172.21.2.104:0/3570800565 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31c4003800 0x7f31c4003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:37.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e2b95700 1 -- 172.21.2.104:0/3570800565 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c4003260 con 0x7f31c4004bf0 2021-07-11T03:56:37.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e2b95700 1 --2- 172.21.2.104:0/3570800565 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31c4004bf0 0x7f31c400b750 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f31d40032f0 tx=0x7f31d4006f90).ready entity=mon.0 client_cookie=1bad8b6c4e8330e8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:37.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31d400b460 con 0x7f31c4004bf0 2021-07-11T03:56:37.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d400b5c0 con 0x7f31c4004bf0 2021-07-11T03:56:37.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e1392700 1 -- 172.21.2.104:0/3570800565 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31d400b920 con 0x7f31c4004bf0 2021-07-11T03:56:37.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 -- 172.21.2.104:0/3570800565 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31c4004bf0 msgr2=0x7f31c400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:37.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 --2- 172.21.2.104:0/3570800565 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31c4004bf0 0x7f31c400b750 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f31d40032f0 tx=0x7f31d4006f90).stop 2021-07-11T03:56:37.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e2b95700 1 -- 172.21.2.104:0/3570800565 reap_dead start 2021-07-11T03:56:37.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 -- 172.21.2.104:0/3570800565 shutdown_connections 2021-07-11T03:56:37.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 -- 172.21.2.104:0/3570800565 >> 172.21.2.104:0/3570800565 conn(0x7f31dc05f6b0 msgr2=0x7f31dc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:37.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 -- 172.21.2.104:0/3570800565 shutdown_connections 2021-07-11T03:56:37.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 -- 172.21.2.104:0/3570800565 wait complete. 2021-07-11T03:56:37.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 Processor -- start 2021-07-11T03:56:37.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.733+0000 7f31e45f8700 1 -- start start 2021-07-11T03:56:37.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e45f8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31dc140c90 0x7f31dc152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:37.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e45f8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31dc13df40 0x7f31dc152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:37.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e45f8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc149ea0 0x7f31dc14a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:37.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e45f8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc13fe90 con 0x7f31dc13df40 2021-07-11T03:56:37.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e45f8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc141070 con 0x7f31dc140c90 2021-07-11T03:56:37.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e45f8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc13f200 con 0x7f31dc149ea0 2021-07-11T03:56:37.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e1b93700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31dc13df40 0x7f31dc152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:37.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e1b93700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31dc13df40 0x7f31dc152970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44304/0 (socket says 172.21.2.104:44304) 2021-07-11T03:56:37.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e1b93700 1 -- 172.21.2.104:0/1945881464 learned_addr learned my addr 172.21.2.104:0/1945881464 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:37.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e2394700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31dc140c90 0x7f31dc152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:37.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e2b95700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc149ea0 0x7f31dc14a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:37.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e1b93700 1 -- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc149ea0 msgr2=0x7f31dc14a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:37.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e1b93700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc149ea0 0x7f31dc14a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:37.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e1b93700 1 -- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31dc140c90 msgr2=0x7f31dc152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:37.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e1b93700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31dc140c90 0x7f31dc152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:37.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e1b93700 1 -- 172.21.2.104:0/1945881464 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc13f4a0 con 0x7f31dc13df40 2021-07-11T03:56:37.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e2394700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31dc140c90 0x7f31dc152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:56:37.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e1b93700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31dc13df40 0x7f31dc152970 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f31d8008000 tx=0x7f31d8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:37.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31cb7fe700 1 -- 172.21.2.104:0/1945881464 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31d8007e60 con 0x7f31dc13df40 2021-07-11T03:56:37.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31cb7fe700 1 -- 172.21.2.104:0/1945881464 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d800f440 con 0x7f31dc13df40 2021-07-11T03:56:37.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e45f8700 1 -- 172.21.2.104:0/1945881464 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31dc0b5ce0 con 0x7f31dc13df40 2021-07-11T03:56:37.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e45f8700 1 -- 172.21.2.104:0/1945881464 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31dc0b6420 con 0x7f31dc13df40 2021-07-11T03:56:37.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31cb7fe700 1 -- 172.21.2.104:0/1945881464 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31d800f7a0 con 0x7f31dc13df40 2021-07-11T03:56:37.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e45f8700 1 -- 172.21.2.104:0/1945881464 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31dc057350 con 0x7f31dc13df40 2021-07-11T03:56:37.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31cb7fe700 1 -- 172.21.2.104:0/1945881464 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f31d800f900 con 0x7f31dc13df40 2021-07-11T03:56:37.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31cb7fe700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f31bc053240 0x7f31bc055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:37.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31cb7fe700 1 -- 172.21.2.104:0/1945881464 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f31d806b120 con 0x7f31dc13df40 2021-07-11T03:56:37.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.737+0000 7f31e2394700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f31bc053240 0x7f31bc055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:37.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.741+0000 7f31e2394700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f31bc053240 0x7f31bc055700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f31cc006000 tx=0x7f31dc0602c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:37.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.741+0000 7f31cb7fe700 1 -- 172.21.2.104:0/1945881464 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f31d8042350 con 0x7f31dc13df40 2021-07-11T03:56:37.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.821+0000 7f31e45f8700 1 -- 172.21.2.104:0/1945881464 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f31dc044650 con 0x7f31dc13df40 2021-07-11T03:56:37.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31cb7fe700 1 -- 172.21.2.104:0/1945881464 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v44) v1 ==== 81+0+33645 (secure 0 0 0) 0x7f31d8029380 con 0x7f31dc13df40 2021-07-11T03:56:37.831 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:37.832 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:56:37.832 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 44, 2021-07-11T03:56:37.832 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T03:56:37.832 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T03:56:37.833 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T03:55:12.599036+0000", 2021-07-11T03:56:37.833 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:56:37.833 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:56:37.833 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T03:56:37.833 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T03:56:37.834 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T03:56:37.834 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T03:56:37.834 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T03:56:37.834 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T03:56:37.835 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T03:56:37.835 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:56:37.835 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T03:56:37.835 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T03:56:37.835 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T03:56:37.836 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T03:56:37.836 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T03:56:37.836 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 8, 2021-07-11T03:56:37.836 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T03:56:37.837 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T03:56:37.837 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T03:56:37.837 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T03:56:37.837 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T03:56:37.837 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.838 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T03:56:37.838 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T03:56:37.838 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T03:56:37.838 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:56:37.839 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:56:37.839 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:56:37.839 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:56:37.839 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:56:37.840 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:56:37.840 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:56:37.840 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:56:37.840 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T03:56:37.841 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T03:56:37.841 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T03:56:37.841 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T03:56:37.841 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T03:56:37.841 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:56:37.842 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:56:37.842 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:56:37.842 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:56:37.842 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:56:37.843 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:56:37.843 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:56:37.843 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.843 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T03:56:37.844 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:56:37.844 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:56:37.844 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:56:37.844 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:56:37.844 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:56:37.845 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:56:37.845 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:56:37.845 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:56:37.845 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:56:37.845 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:56:37.846 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:56:37.846 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:56:37.846 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:56:37.846 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:56:37.847 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:56:37.847 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:56:37.847 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:56:37.847 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:56:37.847 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:56:37.848 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:56:37.848 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:56:37.848 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:56:37.849 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:56:37.849 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:56:37.849 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:56:37.850 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:56:37.850 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.850 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:56:37.850 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:56:37.850 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:56:37.851 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:56:37.851 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:56:37.851 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:56:37.851 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:56:37.852 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:56:37.852 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:56:37.852 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:56:37.852 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:56:37.852 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:56:37.853 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T03:56:37.853 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.853 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:56:37.853 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T03:56:37.853 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.854 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.854 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.854 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T03:56:37.854 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T03:56:37.855 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T03:56:37.855 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:56:37.855 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:56:37.855 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:56:37.855 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:56:37.856 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:56:37.856 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:56:37.856 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:56:37.856 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:56:37.857 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:56:37.857 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:56:37.857 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:56:37.857 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:56:37.857 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:56:37.858 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:56:37.858 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:56:37.858 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:56:37.858 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:56:37.858 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:56:37.859 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:56:37.859 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:56:37.859 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.860 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T03:56:37.860 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:56:37.860 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:56:37.860 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:56:37.861 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:56:37.861 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:56:37.861 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:56:37.861 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:56:37.861 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:56:37.862 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:56:37.862 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:56:37.862 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:56:37.862 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:56:37.863 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:56:37.863 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:56:37.863 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:56:37.863 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:56:37.864 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:56:37.864 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:56:37.864 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:56:37.864 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:56:37.865 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:56:37.865 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:56:37.865 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:56:37.865 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:56:37.865 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:56:37.866 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:56:37.866 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.866 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:56:37.866 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:56:37.867 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:56:37.867 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:56:37.867 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:56:37.867 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:56:37.867 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:56:37.868 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:56:37.868 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:56:37.868 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:56:37.868 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:56:37.868 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:56:37.869 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:56:37.869 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T03:56:37.869 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.869 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.870 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.870 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 7, 2021-07-11T03:56:37.870 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T03:56:37.870 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:55:09.448452+0000", 2021-07-11T03:56:37.871 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:56:37.871 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:56:37.871 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:56:37.871 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:56:37.872 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:56:37.872 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:56:37.872 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:56:37.872 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:56:37.872 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:56:37.873 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:56:37.873 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:56:37.873 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:56:37.873 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:56:37.874 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:56:37.874 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:56:37.874 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:56:37.874 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:56:37.874 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:56:37.875 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:56:37.875 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:56:37.875 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.875 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "43", 2021-07-11T03:56:37.875 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:56:37.876 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:56:37.876 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:56:37.876 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:56:37.876 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:56:37.877 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:56:37.877 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:56:37.877 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:56:37.877 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:56:37.877 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:56:37.878 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:56:37.878 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:56:37.878 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:56:37.878 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:56:37.878 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:56:37.879 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:56:37.879 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:56:37.879 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:56:37.879 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:56:37.879 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:56:37.880 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:56:37.880 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:56:37.880 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:56:37.881 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:56:37.881 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:56:37.881 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:56:37.881 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.882 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:56:37.882 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:56:37.882 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:56:37.882 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:56:37.883 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:56:37.883 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:56:37.883 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:56:37.883 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:56:37.883 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:56:37.884 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:56:37.884 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:56:37.884 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:56:37.884 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T03:56:37.884 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T03:56:37.885 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T03:56:37.885 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.885 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:56:37.885 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T03:56:37.886 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T03:56:37.886 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.886 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.886 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.886 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.887 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 8, 2021-07-11T03:56:37.887 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T03:56:37.887 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:55:10.579101+0000", 2021-07-11T03:56:37.887 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:56:37.887 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:56:37.888 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:56:37.888 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:56:37.888 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:56:37.888 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:56:37.889 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:56:37.889 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:56:37.889 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:56:37.889 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:56:37.889 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:56:37.890 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:56:37.890 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:56:37.890 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:56:37.891 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:56:37.891 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:56:37.891 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:56:37.891 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:56:37.891 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:56:37.892 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:56:37.892 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.892 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "44", 2021-07-11T03:56:37.892 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:56:37.893 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:56:37.893 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:56:37.893 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:56:37.893 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:56:37.893 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:56:37.894 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:56:37.894 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:56:37.894 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:56:37.894 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:56:37.895 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:56:37.895 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:56:37.895 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:56:37.895 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:56:37.895 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:56:37.896 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:56:37.896 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:56:37.896 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:56:37.896 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:56:37.897 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:56:37.897 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:56:37.897 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:56:37.897 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:56:37.897 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:56:37.898 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:56:37.898 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:56:37.898 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.898 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:56:37.898 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:56:37.899 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:56:37.899 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:56:37.899 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:56:37.899 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:56:37.900 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:56:37.900 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:56:37.900 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:56:37.900 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:56:37.901 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:56:37.901 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:56:37.901 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:56:37.901 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T03:56:37.902 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T03:56:37.902 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.902 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.902 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.902 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:56:37.903 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T03:56:37.903 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.903 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:56:37.903 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T03:56:37.903 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:37.904 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:37.904 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:37.904 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:37.904 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:37.905 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:37.905 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:37.905 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:37.905 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:37.905 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:37.906 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:37.906 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.906 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.906 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.907 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T03:56:37.907 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:37.907 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.907 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.907 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.908 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T03:56:37.908 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:37.908 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.908 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.909 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.909 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:37.909 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.909 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.910 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.910 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T03:56:37.910 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:37.910 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.911 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.911 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.911 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T03:56:37.912 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:37.912 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.912 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.912 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.913 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:37.913 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.913 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.913 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.913 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T03:56:37.914 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:37.914 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.914 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.914 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.915 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T03:56:37.915 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:37.915 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.915 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.916 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.916 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:37.916 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.916 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.917 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.917 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T03:56:37.917 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:37.917 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.917 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.918 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.918 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T03:56:37.918 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:37.918 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.919 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.919 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.919 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T03:56:37.919 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T03:56:37.919 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T03:56:37.920 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T03:56:37.920 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:37.920 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:37.920 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:37.921 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.921 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.921 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.921 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:56:37.922 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T03:56:37.922 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:37.922 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:37.922 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:37.923 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:37.923 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:37.923 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:37.923 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:37.923 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:37.924 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:37.924 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:37.924 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:37.924 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.924 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.925 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.925 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T03:56:37.925 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:37.925 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.926 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.926 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.926 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T03:56:37.926 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:37.926 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.927 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.927 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.927 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:37.927 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.928 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.928 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.928 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T03:56:37.928 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:37.928 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.929 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.929 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.929 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T03:56:37.929 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:37.929 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.930 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.930 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.930 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:37.930 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.931 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.931 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.931 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T03:56:37.932 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:37.932 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.932 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.933 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.933 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T03:56:37.933 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:37.933 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.934 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.934 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.934 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:37.934 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.934 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.935 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.935 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T03:56:37.935 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:37.935 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.936 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.936 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.936 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T03:56:37.936 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:37.936 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.937 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.937 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.937 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T03:56:37.937 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T03:56:37.937 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T03:56:37.938 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T03:56:37.938 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:37.938 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:37.938 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:37.939 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.939 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.939 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.939 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:56:37.939 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T03:56:37.940 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:37.940 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:37.940 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:37.940 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:37.940 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:37.941 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:37.941 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:37.941 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 30, 2021-07-11T03:56:37.941 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:37.942 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:37.942 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:37.942 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.942 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.943 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.943 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T03:56:37.943 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:37.943 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.944 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.944 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.944 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T03:56:37.944 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:37.944 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.945 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.945 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.945 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:37.945 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.946 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.946 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.946 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T03:56:37.946 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:37.946 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.947 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.947 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.947 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T03:56:37.947 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:37.948 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.948 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.948 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.948 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:37.948 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.949 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.949 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.949 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T03:56:37.949 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:37.950 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.950 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.950 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.950 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T03:56:37.950 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:37.951 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.951 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.951 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.951 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:37.951 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.952 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.952 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.952 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T03:56:37.953 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:37.953 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.953 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.953 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.954 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T03:56:37.954 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:37.954 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.954 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.954 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.955 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T03:56:37.955 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T03:56:37.955 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T03:56:37.955 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T03:56:37.956 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:37.956 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:37.956 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:37.956 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.956 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.957 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.957 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:56:37.957 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T03:56:37.957 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:37.958 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:37.958 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:37.958 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:37.958 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:37.958 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:37.959 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:37.959 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:37.959 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:37.959 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:37.959 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:37.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.960 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.960 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T03:56:37.961 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:37.961 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.961 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.961 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.961 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T03:56:37.962 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:37.962 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.962 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.963 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.963 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:37.963 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.963 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.964 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.964 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T03:56:37.964 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:37.964 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.964 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.965 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.965 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T03:56:37.965 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:37.965 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.966 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.966 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.966 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:37.966 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.966 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.967 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.967 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T03:56:37.967 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:37.967 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.968 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.968 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.968 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T03:56:37.968 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:37.968 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.969 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.969 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.969 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:37.969 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.970 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.970 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.970 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T03:56:37.970 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:37.971 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.971 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.971 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.971 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T03:56:37.971 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:37.972 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.972 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.972 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.973 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T03:56:37.973 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T03:56:37.973 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T03:56:37.973 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T03:56:37.974 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:37.974 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:37.974 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:37.974 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.975 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.975 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.975 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:56:37.975 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T03:56:37.975 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:37.976 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:37.976 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:37.976 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:37.976 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:37.977 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:37.977 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:37.977 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:37.977 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:37.977 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:37.978 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:37.978 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.978 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.978 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.979 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T03:56:37.979 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:37.979 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.979 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.979 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.980 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T03:56:37.980 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:37.980 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.980 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.981 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.981 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:37.981 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.981 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.981 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.982 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T03:56:37.982 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:37.982 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.982 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.983 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.983 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T03:56:37.983 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:37.984 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.984 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.984 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.984 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:37.984 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.985 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.985 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.985 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T03:56:37.985 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:37.985 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.986 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.986 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.986 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T03:56:37.986 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:37.987 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.987 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.987 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.987 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:37.987 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.988 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.988 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.988 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T03:56:37.988 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:37.988 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.989 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.989 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.989 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T03:56:37.989 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:37.990 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.990 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.990 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.990 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T03:56:37.990 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T03:56:37.991 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T03:56:37.991 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T03:56:37.991 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:37.991 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:37.992 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:37.992 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.992 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.992 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.992 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:56:37.993 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T03:56:37.993 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:37.993 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:37.994 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:37.994 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:37.994 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:37.994 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:37.994 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:37.995 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:37.995 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:37.995 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:37.995 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:37.996 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.996 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.996 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.996 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T03:56:37.996 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:37.997 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.997 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.997 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:37.997 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T03:56:37.998 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:37.998 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:37.998 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:37.998 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:37.998 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:37.999 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:37.999 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:37.999 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:37.999 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T03:56:37.999 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:38.000 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.000 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.000 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:38.000 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T03:56:38.001 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:38.001 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.001 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.001 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.001 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:38.002 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:38.002 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.002 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:38.002 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T03:56:38.002 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:38.003 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.003 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.003 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:38.003 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T03:56:38.004 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:38.004 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.004 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.004 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.005 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:38.005 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:38.005 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.005 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:38.005 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T03:56:38.006 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:38.006 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.006 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.006 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:38.007 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T03:56:38.007 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:38.007 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.007 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.007 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.008 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T03:56:38.008 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T03:56:38.008 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T03:56:38.008 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T03:56:38.008 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:38.009 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:38.009 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:38.009 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.009 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.009 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.010 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:56:38.010 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T03:56:38.010 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:38.010 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:38.011 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:38.011 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:38.011 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:38.011 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:38.011 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:38.012 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:38.012 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:38.012 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:38.012 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:38.013 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:38.013 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.013 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:38.013 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T03:56:38.014 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:38.014 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.014 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.014 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:38.014 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T03:56:38.015 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:38.015 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.015 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.015 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.016 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:38.016 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:38.016 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.016 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:38.016 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T03:56:38.017 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:38.017 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.017 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.017 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:38.018 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T03:56:38.018 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:38.018 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.018 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.018 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.019 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:38.019 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:38.019 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.019 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:38.019 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T03:56:38.020 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:38.020 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.020 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.020 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:38.021 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T03:56:38.021 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:38.021 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.021 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.021 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.022 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:38.022 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:38.022 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.022 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:38.022 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T03:56:38.023 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:38.023 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.023 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.023 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:38.024 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T03:56:38.024 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:38.024 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.024 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.024 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.025 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T03:56:38.025 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T03:56:38.025 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T03:56:38.025 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T03:56:38.025 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:38.026 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:38.026 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:38.026 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.026 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.027 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.027 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:56:38.027 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T03:56:38.027 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:38.027 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:38.028 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:38.028 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:38.028 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:38.028 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:38.029 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:38.029 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 41, 2021-07-11T03:56:38.029 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:38.029 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:38.029 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:38.030 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:38.030 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.030 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:38.030 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T03:56:38.030 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:38.031 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.031 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.031 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:38.031 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T03:56:38.032 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:38.032 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.032 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.032 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.032 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:38.033 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:38.033 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.033 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:38.033 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T03:56:38.034 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:38.034 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.034 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.034 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:38.034 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T03:56:38.035 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:38.035 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.035 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.035 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.035 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:38.036 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:38.036 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.036 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:38.036 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T03:56:38.037 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:38.037 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.037 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.037 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:38.037 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T03:56:38.038 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:38.038 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.038 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.038 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.039 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:38.039 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:38.039 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.039 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:38.039 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T03:56:38.040 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:38.040 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.040 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.040 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:38.041 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T03:56:38.041 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:38.041 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.041 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.041 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.042 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T03:56:38.042 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T03:56:38.042 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T03:56:38.042 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T03:56:38.043 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:38.043 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:38.043 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:38.043 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:38.043 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.044 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:56:38.044 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T03:56:38.044 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.044 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:56:38.045 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:38.045 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:38.045 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:38.045 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:38.045 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:38.046 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T03:56:38.046 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:38.046 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.046 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.046 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:56:38.047 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:38.047 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:38.047 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:38.047 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:38.048 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:38.048 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T03:56:38.048 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:38.048 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.048 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.049 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:56:38.049 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:38.049 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:38.049 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:38.050 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:38.050 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:38.050 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T03:56:38.050 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:38.050 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.051 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.051 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:56:38.051 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:38.051 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:38.052 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:38.052 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:38.052 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:38.052 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T03:56:38.052 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:38.053 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.053 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.053 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:56:38.053 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:38.054 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:38.054 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:38.054 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:38.054 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:38.054 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T03:56:38.055 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:38.055 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.055 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.055 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:56:38.055 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:38.056 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:38.056 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:38.056 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:38.056 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:38.056 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T03:56:38.057 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:38.057 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.057 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.057 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:56:38.058 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:38.058 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:38.058 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:38.058 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:38.058 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:38.059 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T03:56:38.059 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:38.059 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.059 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:38.059 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:56:38.060 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:38.060 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:38.060 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:38.060 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:38.060 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:38.061 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T03:56:38.061 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:38.061 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.061 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:56:38.062 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T03:56:38.062 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T03:56:38.062 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T03:56:38.062 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T03:56:38.062 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": {}, 2021-07-11T03:56:38.063 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T03:56:38.063 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T03:56:38.063 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T03:56:38.063 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T03:56:38.063 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T03:56:38.064 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T03:56:38.064 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T03:56:38.064 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T03:56:38.064 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:38.065 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:38.065 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T03:56:38.065 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T03:56:38.065 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T03:56:38.065 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T03:56:38.066 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T03:56:38.066 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:56:38.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 -- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f31bc053240 msgr2=0x7f31bc055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f31bc053240 0x7f31bc055700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f31cc006000 tx=0x7f31dc0602c0).stop 2021-07-11T03:56:38.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 -- 172.21.2.104:0/1945881464 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31dc13df40 msgr2=0x7f31dc152970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31dc13df40 0x7f31dc152970 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f31d8008000 tx=0x7f31d8009040).stop 2021-07-11T03:56:38.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 -- 172.21.2.104:0/1945881464 shutdown_connections 2021-07-11T03:56:38.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31dc149ea0 0x7f31dc14a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:38.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f31bc053240 0x7f31bc055700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:38.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31dc13df40 0x7f31dc152970 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:38.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 --2- 172.21.2.104:0/1945881464 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31dc140c90 0x7f31dc152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:38.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 -- 172.21.2.104:0/1945881464 >> 172.21.2.104:0/1945881464 conn(0x7f31dc05f6b0 msgr2=0x7f31dc066ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:38.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 -- 172.21.2.104:0/1945881464 shutdown_connections 2021-07-11T03:56:38.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:37.825+0000 7f31e45f8700 1 -- 172.21.2.104:0/1945881464 wait complete. 2021-07-11T03:56:38.071 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:56:38.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe7230f0700 1 Processor -- start 2021-07-11T03:56:38.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe7230f0700 1 -- start start 2021-07-11T03:56:38.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe7230f0700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe71c0a76b0 0x7fe71c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe7230f0700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c0a7a90 con 0x7fe71c0b1970 2021-07-11T03:56:38.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe7230f0700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c147620 con 0x7fe71c0b0060 2021-07-11T03:56:38.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe7230f0700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71c1478c0 con 0x7fe71c0a76b0 2021-07-11T03:56:38.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71bfff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe71c0b1970 0x7fe71c05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41080/0 (socket says 172.21.2.104:41080) 2021-07-11T03:56:38.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71bfff700 1 -- 172.21.2.104:0/4245208770 learned_addr learned my addr 172.21.2.104:0/4245208770 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:38.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe720e8c700 1 --2- 172.21.2.104:0/4245208770 >> v2:172.21.2.105:3300/0 conn(0x7fe71c0a76b0 0x7fe71c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:38.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445185955 0 0) 0x7fe71c0a7a90 con 0x7fe71c0b1970 2021-07-11T03:56:38.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe708003260 con 0x7fe71c0b1970 2021-07-11T03:56:38.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 913343519 0 0) 0x7fe708003260 con 0x7fe71c0b1970 2021-07-11T03:56:38.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 >> v2:172.21.2.105:3300/0 conn(0x7fe71c0a76b0 msgr2=0x7fe71c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:38.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 --2- 172.21.2.104:0/4245208770 >> v2:172.21.2.105:3300/0 conn(0x7fe71c0a76b0 0x7fe71c066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:38.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 >> v1:172.21.2.105:6789/0 conn(0x7fe71c0b0060 legacy=0x7fe71c151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c147b60 con 0x7fe71c0b1970 2021-07-11T03:56:38.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fe710001680 con 0x7fe71c0b1970 2021-07-11T03:56:38.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 >> v1:172.21.2.104:6789/0 conn(0x7fe71c0b1970 legacy=0x7fe71c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 --2- 172.21.2.104:0/4245208770 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe708003750 0x7fe708003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 --2- 172.21.2.104:0/4245208770 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe708004500 0x7fe708004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 --2- 172.21.2.104:0/4245208770 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe708004e70 0x7fe70800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1478c0 con 0x7fe708003750 2021-07-11T03:56:38.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70800bcc0 con 0x7fe708004e70 2021-07-11T03:56:38.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70800be80 con 0x7fe708004500 2021-07-11T03:56:38.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.137+0000 7fe720e8c700 1 --2- 172.21.2.104:0/4245208770 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe708003750 0x7fe708003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:38.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe720e8c700 1 -- 172.21.2.104:0/4245208770 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe708004500 msgr2=0x7fe708004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:38.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe720e8c700 1 --2- 172.21.2.104:0/4245208770 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe708004500 0x7fe708004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:38.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe720e8c700 1 -- 172.21.2.104:0/4245208770 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe708004e70 msgr2=0x7fe70800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:38.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe720e8c700 1 --2- 172.21.2.104:0/4245208770 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe708004e70 0x7fe70800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:38.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe72168d700 1 -- 172.21.2.104:0/4245208770 reap_dead start 2021-07-11T03:56:38.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe720e8c700 1 -- 172.21.2.104:0/4245208770 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe708003260 con 0x7fe708003750 2021-07-11T03:56:38.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe720e8c700 1 --2- 172.21.2.104:0/4245208770 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe708003750 0x7fe708003b30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fe70c00c000 tx=0x7fe70c006df0).ready entity=mon.0 client_cookie=3e1b58b8724ed1d6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:38.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe70c00e830 con 0x7fe708003750 2021-07-11T03:56:38.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c00e990 con 0x7fe708003750 2021-07-11T03:56:38.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71b7fe700 1 -- 172.21.2.104:0/4245208770 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe70c00ecf0 con 0x7fe708003750 2021-07-11T03:56:38.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 -- 172.21.2.104:0/4245208770 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe708003750 msgr2=0x7fe708003b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 --2- 172.21.2.104:0/4245208770 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe708003750 0x7fe708003b30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fe70c00c000 tx=0x7fe70c006df0).stop 2021-07-11T03:56:38.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 -- 172.21.2.104:0/4245208770 shutdown_connections 2021-07-11T03:56:38.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 --2- 172.21.2.104:0/4245208770 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe708003750 0x7fe708003b30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:38.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 -- 172.21.2.104:0/4245208770 >> 172.21.2.104:0/4245208770 conn(0x7fe71c0b53e0 msgr2=0x7fe71c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:38.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 -- 172.21.2.104:0/4245208770 shutdown_connections 2021-07-11T03:56:38.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 -- 172.21.2.104:0/4245208770 wait complete. 2021-07-11T03:56:38.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 Processor -- start 2021-07-11T03:56:38.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 -- start start 2021-07-11T03:56:38.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe708003750 0x7fe71c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe71c0ae100 0x7fe71c0b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe71c144760 0x7fe71c144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c0634d0 con 0x7fe71c0ae100 2021-07-11T03:56:38.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c067340 con 0x7fe71c144760 2021-07-11T03:56:38.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c0af380 con 0x7fe708003750 2021-07-11T03:56:38.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe71c0ae100 0x7fe71c0b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:38.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe71c0ae100 0x7fe71c0b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44326/0 (socket says 172.21.2.104:44326) 2021-07-11T03:56:38.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71bfff700 1 -- 172.21.2.104:0/325973964 learned_addr learned my addr 172.21.2.104:0/325973964 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:38.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71bfff700 1 -- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe708003750 msgr2=0x7fe71c0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71bfff700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe708003750 0x7fe71c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:38.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71bfff700 1 -- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe71c144760 msgr2=0x7fe71c144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71bfff700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe71c144760 0x7fe71c144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:38.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71bfff700 1 -- 172.21.2.104:0/325973964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c0ad5e0 con 0x7fe71c0ae100 2021-07-11T03:56:38.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe71bfff700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe71c0ae100 0x7fe71c0b4a00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fe710001390 tx=0x7fe71c0af970).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:38.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe719ffb700 1 -- 172.21.2.104:0/325973964 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe710007170 con 0x7fe71c0ae100 2021-07-11T03:56:38.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe719ffb700 1 -- 172.21.2.104:0/325973964 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7100072d0 con 0x7fe71c0ae100 2021-07-11T03:56:38.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe719ffb700 1 -- 172.21.2.104:0/325973964 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe710004920 con 0x7fe71c0ae100 2021-07-11T03:56:38.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 -- 172.21.2.104:0/325973964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c0ad880 con 0x7fe71c0ae100 2021-07-11T03:56:38.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.141+0000 7fe7230f0700 1 -- 172.21.2.104:0/325973964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c1450d0 con 0x7fe71c0ae100 2021-07-11T03:56:38.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.145+0000 7fe7230f0700 1 -- 172.21.2.104:0/325973964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71c057350 con 0x7fe71c0ae100 2021-07-11T03:56:38.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.145+0000 7fe719ffb700 1 -- 172.21.2.104:0/325973964 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe710004a80 con 0x7fe71c0ae100 2021-07-11T03:56:38.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.145+0000 7fe719ffb700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6fc053300 0x7fe6fc0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.145+0000 7fe719ffb700 1 -- 172.21.2.104:0/325973964 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe71006b060 con 0x7fe71c0ae100 2021-07-11T03:56:38.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.145+0000 7fe720e8c700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6fc053300 0x7fe6fc0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:38.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.145+0000 7fe720e8c700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6fc053300 0x7fe6fc0557c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fe70c00c000 tx=0x7fe70c019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:38.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.145+0000 7fe719ffb700 1 -- 172.21.2.104:0/325973964 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe710042290 con 0x7fe71c0ae100 2021-07-11T03:56:38.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.253+0000 7fe7230f0700 1 -- 172.21.2.104:0/325973964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe71c0698e0 con 0x7fe71c0ae100 2021-07-11T03:56:38.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe719ffb700 1 -- 172.21.2.104:0/325973964 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+2488 (secure 0 0 0) 0x7fe7100458b0 con 0x7fe71c0ae100 2021-07-11T03:56:38.263 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:38.263 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":16,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4492,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2412138431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2412138431},{"type":"v1","addr":"172.21.2.105:6835","nonce":2412138431}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/3271417890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3271417890},{"type":"v1","addr":"172.21.2.104:6834","nonce":3271417890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:55:11.595553+0000","modified":"2021-07-11T03:55:12.600491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5232},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5232":{"gid":5232,"name":"c","rank":0,"incarnation":15,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/795410455","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":795410455},{"type":"v1","addr":"172.21.2.104:6835","nonce":795410455}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-07-11T03:56:38.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 -- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6fc053300 msgr2=0x7fe6fc0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6fc053300 0x7fe6fc0557c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fe70c00c000 tx=0x7fe70c019000).stop 2021-07-11T03:56:38.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 -- 172.21.2.104:0/325973964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe71c0ae100 msgr2=0x7fe71c0b4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe71c0ae100 0x7fe71c0b4a00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fe710001390 tx=0x7fe71c0af970).stop 2021-07-11T03:56:38.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 -- 172.21.2.104:0/325973964 shutdown_connections 2021-07-11T03:56:38.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe71c144760 0x7fe71c144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:38.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6fc053300 0x7fe6fc0557c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:38.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe71c0ae100 0x7fe71c0b4a00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:38.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 --2- 172.21.2.104:0/325973964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe708003750 0x7fe71c0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:38.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 -- 172.21.2.104:0/325973964 >> 172.21.2.104:0/325973964 conn(0x7fe71c0b53e0 msgr2=0x7fe71c149eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:38.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 -- 172.21.2.104:0/325973964 shutdown_connections 2021-07-11T03:56:38.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.257+0000 7fe7230f0700 1 -- 172.21.2.104:0/325973964 wait complete. 2021-07-11T03:56:38.267 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 16 2021-07-11T03:56:38.270 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T03:56:38.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.313+0000 7f1da0f78700 1 Processor -- start 2021-07-11T03:56:38.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.313+0000 7f1da0f78700 1 -- start start 2021-07-11T03:56:38.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.313+0000 7f1da0f78700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1d9c0a76b0 0x7f1d9c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.313+0000 7f1da0f78700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1d9c0b1970 0x7f1d9c066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.313+0000 7f1da0f78700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c0b0060 0x7f1d9c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.313+0000 7f1da0f78700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9c067450 con 0x7f1d9c0a76b0 2021-07-11T03:56:38.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.313+0000 7f1da0f78700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9c0a7a90 con 0x7f1d9c0b1970 2021-07-11T03:56:38.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.313+0000 7f1da0f78700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c147620 con 0x7f1d9c0b0060 2021-07-11T03:56:38.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9a59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1d9c0a76b0 0x7f1d9c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:38.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9a59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1d9c0a76b0 0x7f1d9c066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44332/0 (socket says 172.21.2.104:44332) 2021-07-11T03:56:38.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9a59c700 1 -- 172.21.2.104:0/293950204 learned_addr learned my addr 172.21.2.104:0/293950204 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:38.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9a59c700 1 -- 172.21.2.104:0/293950204 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c0b0060 msgr2=0x7f1d9c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:38.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9a59c700 1 --2- 172.21.2.104:0/293950204 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c0b0060 0x7f1d9c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:38.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9a59c700 1 -- 172.21.2.104:0/293950204 >> v2:172.21.2.105:3300/0 conn(0x7f1d9c0b1970 msgr2=0x7f1d9c066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:38.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9a59c700 1 --2- 172.21.2.104:0/293950204 >> v2:172.21.2.105:3300/0 conn(0x7f1d9c0b1970 0x7f1d9c066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:38.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9a59c700 1 -- 172.21.2.104:0/293950204 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c1478c0 con 0x7f1d9c0a76b0 2021-07-11T03:56:38.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9a59c700 1 --2- 172.21.2.104:0/293950204 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0a76b0 0x7f1d9c066a70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f1d90002f70 tx=0x7f1d90006eb0).ready entity=mon.0 client_cookie=4278857de327198e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:38.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9959a700 1 -- 172.21.2.104:0/293950204 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d9000d570 con 0x7f1d9c0a76b0 2021-07-11T03:56:38.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9959a700 1 -- 172.21.2.104:0/293950204 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d90000e90 con 0x7f1d9c0a76b0 2021-07-11T03:56:38.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9959a700 1 -- 172.21.2.104:0/293950204 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d9000d900 con 0x7f1d9c0a76b0 2021-07-11T03:56:38.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 -- 172.21.2.104:0/293950204 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0a76b0 msgr2=0x7f1d9c066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 --2- 172.21.2.104:0/293950204 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0a76b0 0x7f1d9c066a70 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f1d90002f70 tx=0x7f1d90006eb0).stop 2021-07-11T03:56:38.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 -- 172.21.2.104:0/293950204 shutdown_connections 2021-07-11T03:56:38.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 --2- 172.21.2.104:0/293950204 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c0b0060 0x7f1d9c05f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:38.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 --2- 172.21.2.104:0/293950204 >> v2:172.21.2.105:3300/0 conn(0x7f1d9c0b1970 0x7f1d9c066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:38.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 --2- 172.21.2.104:0/293950204 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0a76b0 0x7f1d9c066a70 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:38.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 -- 172.21.2.104:0/293950204 >> 172.21.2.104:0/293950204 conn(0x7f1d9c0b53e0 msgr2=0x7f1d9c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:38.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 -- 172.21.2.104:0/293950204 shutdown_connections 2021-07-11T03:56:38.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 -- 172.21.2.104:0/293950204 wait complete. 2021-07-11T03:56:38.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 Processor -- start 2021-07-11T03:56:38.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 -- start start 2021-07-11T03:56:38.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c0a76b0 0x7f1d9c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d9c0b0060 0x7f1d9c0ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0b1970 0x7f1d9c065f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0ac6f0 con 0x7f1d9c0b1970 2021-07-11T03:56:38.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0abcd0 con 0x7f1d9c0b0060 2021-07-11T03:56:38.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c0af180 con 0x7f1d9c0a76b0 2021-07-11T03:56:38.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0b1970 0x7f1d9c065f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:38.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0b1970 0x7f1d9c065f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44338/0 (socket says 172.21.2.104:44338) 2021-07-11T03:56:38.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9ad9d700 1 -- 172.21.2.104:0/2268363579 learned_addr learned my addr 172.21.2.104:0/2268363579 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:38.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d99d9b700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d9c0b0060 0x7f1d9c0ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:38.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9a59c700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c0a76b0 0x7f1d9c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:38.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9ad9d700 1 -- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c0a76b0 msgr2=0x7f1d9c0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9ad9d700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c0a76b0 0x7f1d9c0b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:38.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9ad9d700 1 -- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d9c0b0060 msgr2=0x7f1d9c0ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:38.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9ad9d700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d9c0b0060 0x7f1d9c0ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:38.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9ad9d700 1 -- 172.21.2.104:0/2268363579 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9000b1e0 con 0x7f1d9c0b1970 2021-07-11T03:56:38.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d99d9b700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d9c0b0060 0x7f1d9c0ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:56:38.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d9ad9d700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0b1970 0x7f1d9c065f10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f1d940032f0 tx=0x7f1d94009250).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:38.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/2268363579 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d9400f7e0 con 0x7f1d9c0b1970 2021-07-11T03:56:38.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/2268363579 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9400f940 con 0x7f1d9c0b1970 2021-07-11T03:56:38.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 -- 172.21.2.104:0/2268363579 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d9c0abf00 con 0x7f1d9c0b1970 2021-07-11T03:56:38.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/2268363579 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d9400fca0 con 0x7f1d9c0b1970 2021-07-11T03:56:38.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.317+0000 7f1da0f78700 1 -- 172.21.2.104:0/2268363579 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c159ac0 con 0x7f1d9c0b1970 2021-07-11T03:56:38.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.321+0000 7f1da0f78700 1 -- 172.21.2.104:0/2268363579 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c0af5b0 con 0x7f1d9c0b1970 2021-07-11T03:56:38.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.321+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/2268363579 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1d94010480 con 0x7f1d9c0b1970 2021-07-11T03:56:38.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.321+0000 7f1d8b7fe700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d780531f0 0x7f1d780556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:38.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.321+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/2268363579 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1d9406bc90 con 0x7f1d9c0b1970 2021-07-11T03:56:38.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.321+0000 7f1d9a59c700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d780531f0 0x7f1d780556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:38.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.321+0000 7f1d9a59c700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d780531f0 0x7f1d780556b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f1d90019000 tx=0x7f1d90015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:38.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.321+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/2268363579 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1d94042e60 con 0x7f1d9c0b1970 2021-07-11T03:56:38.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:38.429+0000 7f1da0f78700 1 -- 172.21.2.104:0/2268363579 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1d9c0ab7b0 con 0x7f1d9c0b1970 2021-07-11T03:56:39.280 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T03:56:39.273+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T03:56:39.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.285+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/2268363579 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1d9400fe00 con 0x7f1d9c0b1970 2021-07-11T03:56:39.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 -- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d780531f0 msgr2=0x7f1d780556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d780531f0 0x7f1d780556b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f1d90019000 tx=0x7f1d90015000).stop 2021-07-11T03:56:39.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 -- 172.21.2.104:0/2268363579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0b1970 msgr2=0x7f1d9c065f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0b1970 0x7f1d9c065f10 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f1d940032f0 tx=0x7f1d94009250).stop 2021-07-11T03:56:39.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 -- 172.21.2.104:0/2268363579 shutdown_connections 2021-07-11T03:56:39.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d780531f0 0x7f1d780556b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d9c0b1970 0x7f1d9c065f10 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d9c0b0060 0x7f1d9c0ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 --2- 172.21.2.104:0/2268363579 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d9c0a76b0 0x7f1d9c0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 -- 172.21.2.104:0/2268363579 >> 172.21.2.104:0/2268363579 conn(0x7f1d9c0b53e0 msgr2=0x7f1d9c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:39.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 -- 172.21.2.104:0/2268363579 shutdown_connections 2021-07-11T03:56:39.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.289+0000 7f1da0f78700 1 -- 172.21.2.104:0/2268363579 wait complete. 2021-07-11T03:56:39.298 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T03:56:39.301 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:56:39.313 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T03:56:39.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.345+0000 7fbc50a08700 1 Processor -- start 2021-07-11T03:56:39.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.345+0000 7fbc50a08700 1 -- start start 2021-07-11T03:56:39.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.345+0000 7fbc50a08700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a76b0 0x7fbc4c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.345+0000 7fbc50a08700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fbc4c0b0060 0x7fbc4c05fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.345+0000 7fbc50a08700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4c0a7a90 con 0x7fbc4c0b1970 2021-07-11T03:56:39.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.345+0000 7fbc50a08700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4c147620 con 0x7fbc4c0b0060 2021-07-11T03:56:39.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.345+0000 7fbc50a08700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c1478c0 con 0x7fbc4c0a76b0 2021-07-11T03:56:39.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.345+0000 7fbc49d9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fbc4c0b1970 0x7fbc4c05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41126/0 (socket says 172.21.2.104:41126) 2021-07-11T03:56:39.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.345+0000 7fbc49d9b700 1 -- 172.21.2.104:0/4189039825 learned_addr learned my addr 172.21.2.104:0/4189039825 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:39.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4a59c700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a76b0 0x7fbc4c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 --2- 172.21.2.104:0/4189039825 >> v2:172.21.2.105:3300/0 conn(0x7fbc4c0b0060 0x7fbc4c05fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383230203 0 0) 0x7fbc4c0a7a90 con 0x7fbc4c0b1970 2021-07-11T03:56:39.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc2c003260 con 0x7fbc4c0b1970 2021-07-11T03:56:39.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2934557239 0 0) 0x7fbc2c003260 con 0x7fbc4c0b1970 2021-07-11T03:56:39.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a76b0 msgr2=0x7fbc4c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:39.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a76b0 0x7fbc4c066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 >> v2:172.21.2.105:3300/0 conn(0x7fbc4c0b0060 msgr2=0x7fbc4c05fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:39.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 --2- 172.21.2.104:0/4189039825 >> v2:172.21.2.105:3300/0 conn(0x7fbc4c0b0060 0x7fbc4c05fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c147b60 con 0x7fbc4c0b1970 2021-07-11T03:56:39.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fbc34001660 con 0x7fbc4c0b1970 2021-07-11T03:56:39.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 >> v1:172.21.2.104:6789/0 conn(0x7fbc4c0b1970 legacy=0x7fbc4c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc2c0037d0 0x7fbc2c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a76b0 0x7fbc2c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc2c0049f0 0x7fbc2c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c147620 con 0x7fbc2c0049f0 2021-07-11T03:56:39.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c1478c0 con 0x7fbc2c0037d0 2021-07-11T03:56:39.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c00b750 con 0x7fbc4c0a76b0 2021-07-11T03:56:39.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc2c0049f0 0x7fbc2c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4a59c700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc2c0037d0 0x7fbc2c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc49d9b700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a76b0 0x7fbc2c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 -- 172.21.2.104:0/4189039825 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a76b0 msgr2=0x7fbc2c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:39.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a76b0 0x7fbc2c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 -- 172.21.2.104:0/4189039825 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc2c0037d0 msgr2=0x7fbc2c003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:39.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc2c0037d0 0x7fbc2c003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 -- 172.21.2.104:0/4189039825 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c003260 con 0x7fbc2c0049f0 2021-07-11T03:56:39.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc2c0049f0 0x7fbc2c004e70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fbc4400c000 tx=0x7fbc440091d0).ready entity=mon.0 client_cookie=b7a1d2c8e3a692a7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:39.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc4400e7e0 con 0x7fbc2c0049f0 2021-07-11T03:56:39.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4400e940 con 0x7fbc2c0049f0 2021-07-11T03:56:39.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4959a700 1 -- 172.21.2.104:0/4189039825 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc4400eca0 con 0x7fbc2c0049f0 2021-07-11T03:56:39.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 -- 172.21.2.104:0/4189039825 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc2c0049f0 msgr2=0x7fbc2c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 --2- 172.21.2.104:0/4189039825 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc2c0049f0 0x7fbc2c004e70 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fbc4400c000 tx=0x7fbc440091d0).stop 2021-07-11T03:56:39.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 -- 172.21.2.104:0/4189039825 reap_dead start 2021-07-11T03:56:39.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 -- 172.21.2.104:0/4189039825 shutdown_connections 2021-07-11T03:56:39.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 -- 172.21.2.104:0/4189039825 >> 172.21.2.104:0/4189039825 conn(0x7fbc4c0b53e0 msgr2=0x7fbc4c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:39.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 -- 172.21.2.104:0/4189039825 shutdown_connections 2021-07-11T03:56:39.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 -- 172.21.2.104:0/4189039825 wait complete. 2021-07-11T03:56:39.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 Processor -- start 2021-07-11T03:56:39.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 -- start start 2021-07-11T03:56:39.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc4c14d250 0x7fbc4c0ab260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a9230 0x7fbc4c0aea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc4c0aef60 0x7fbc4c159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c14e600 con 0x7fbc4c0aef60 2021-07-11T03:56:39.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c14d630 con 0x7fbc4c14d250 2021-07-11T03:56:39.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc50a08700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c0adfd0 con 0x7fbc4c0a9230 2021-07-11T03:56:39.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc4c0aef60 0x7fbc4c159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc4c0aef60 0x7fbc4c159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44370/0 (socket says 172.21.2.104:44370) 2021-07-11T03:56:39.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 -- 172.21.2.104:0/2937142069 learned_addr learned my addr 172.21.2.104:0/2937142069 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:39.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4a59c700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc4c14d250 0x7fbc4c0ab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc49d9b700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a9230 0x7fbc4c0aea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 -- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a9230 msgr2=0x7fbc4c0aea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a9230 0x7fbc4c0aea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 -- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc4c14d250 msgr2=0x7fbc4c0ab260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc4c14d250 0x7fbc4c0ab260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.349+0000 7fbc4ad9d700 1 -- 172.21.2.104:0/2937142069 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c0ae270 con 0x7fbc4c0aef60 2021-07-11T03:56:39.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc4ad9d700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc4c0aef60 0x7fbc4c159d60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fbc34001840 tx=0x7fbc4401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:39.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc437fe700 1 -- 172.21.2.104:0/2937142069 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc44009600 con 0x7fbc4c0aef60 2021-07-11T03:56:39.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc437fe700 1 -- 172.21.2.104:0/2937142069 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4401b640 con 0x7fbc4c0aef60 2021-07-11T03:56:39.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc50a08700 1 -- 172.21.2.104:0/2937142069 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4c0aab20 con 0x7fbc4c0aef60 2021-07-11T03:56:39.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc437fe700 1 -- 172.21.2.104:0/2937142069 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc4401b9a0 con 0x7fbc4c0aef60 2021-07-11T03:56:39.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc50a08700 1 -- 172.21.2.104:0/2937142069 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4c14de90 con 0x7fbc4c0aef60 2021-07-11T03:56:39.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc50a08700 1 -- 172.21.2.104:0/2937142069 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4c057350 con 0x7fbc4c0aef60 2021-07-11T03:56:39.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc437fe700 1 -- 172.21.2.104:0/2937142069 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbc4401bb00 con 0x7fbc4c0aef60 2021-07-11T03:56:39.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc437fe700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc30053210 0x7fbc300556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc437fe700 1 -- 172.21.2.104:0/2937142069 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fbc4406ccb0 con 0x7fbc4c0aef60 2021-07-11T03:56:39.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc4a59c700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc30053210 0x7fbc300556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc4a59c700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc30053210 0x7fbc300556d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fbc3c00b000 tx=0x7fbc3c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:39.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.353+0000 7fbc437fe700 1 -- 172.21.2.104:0/2937142069 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbc44043f20 con 0x7fbc4c0aef60 2021-07-11T03:56:39.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 -- 172.21.2.104:0/2937142069 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc4c0698e0 con 0x7fbc4c0aef60 2021-07-11T03:56:39.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc437fe700 1 -- 172.21.2.104:0/2937142069 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+2130 (secure 0 0 0) 0x7fbc44016070 con 0x7fbc4c0aef60 2021-07-11T03:56:39.471 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:39.472 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":17,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4492,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2412138431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2412138431},{"type":"v1","addr":"172.21.2.105:6835","nonce":2412138431}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/3271417890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3271417890},{"type":"v1","addr":"172.21.2.104:6834","nonce":3271417890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14}],"filesystems":[{"mdsmap":{"epoch":17,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:55:11.595553+0000","modified":"2021-07-11T03:56:39.278845+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":45,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2021-07-11T03:56:39.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 -- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc30053210 msgr2=0x7fbc300556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc30053210 0x7fbc300556d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fbc3c00b000 tx=0x7fbc3c008040).stop 2021-07-11T03:56:39.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 -- 172.21.2.104:0/2937142069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc4c0aef60 msgr2=0x7fbc4c159d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc4c0aef60 0x7fbc4c159d60 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7fbc34001840 tx=0x7fbc4401a000).stop 2021-07-11T03:56:39.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 -- 172.21.2.104:0/2937142069 shutdown_connections 2021-07-11T03:56:39.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc30053210 0x7fbc300556d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc4c0aef60 0x7fbc4c159d60 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc4c0a9230 0x7fbc4c0aea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 --2- 172.21.2.104:0/2937142069 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc4c14d250 0x7fbc4c0ab260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 -- 172.21.2.104:0/2937142069 >> 172.21.2.104:0/2937142069 conn(0x7fbc4c0b53e0 msgr2=0x7fbc4c066820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:39.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 -- 172.21.2.104:0/2937142069 shutdown_connections 2021-07-11T03:56:39.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.465+0000 7fbc50a08700 1 -- 172.21.2.104:0/2937142069 wait complete. 2021-07-11T03:56:39.475 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 17 2021-07-11T03:56:39.479 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T03:56:39.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff145fd4700 1 Processor -- start 2021-07-11T03:56:39.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff145fd4700 1 -- start start 2021-07-11T03:56:39.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff145fd4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1400aff10 0x7ff14005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff145fd4700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff1400ac9b0 0x7ff14014af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff145fd4700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff140061360 con 0x7ff14014ab80 2021-07-11T03:56:39.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff145fd4700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff14014b5e0 con 0x7ff1400ac9b0 2021-07-11T03:56:39.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff145fd4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14014b880 con 0x7ff1400aff10 2021-07-11T03:56:39.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13f7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff14014ab80 0x7ff1400af1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41144/0 (socket says 172.21.2.104:41144) 2021-07-11T03:56:39.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13f7fe700 1 -- 172.21.2.104:0/4068570276 learned_addr learned my addr 172.21.2.104:0/4068570276 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:39.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662090594 0 0) 0x7ff140061360 con 0x7ff14014ab80 2021-07-11T03:56:39.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff12c003260 con 0x7ff14014ab80 2021-07-11T03:56:39.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13ffff700 1 --2- 172.21.2.104:0/4068570276 >> v2:172.21.2.105:3300/0 conn(0x7ff1400ac9b0 0x7ff14014af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1909688619 0 0) 0x7ff12c003260 con 0x7ff14014ab80 2021-07-11T03:56:39.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1400aff10 msgr2=0x7ff14005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:39.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13effd700 1 --2- 172.21.2.104:0/4068570276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1400aff10 0x7ff14005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 --2- 172.21.2.104:0/4068570276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1400aff10 0x7ff14005f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 >> v2:172.21.2.105:3300/0 conn(0x7ff1400ac9b0 msgr2=0x7ff14014af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:39.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 --2- 172.21.2.104:0/4068570276 >> v2:172.21.2.105:3300/0 conn(0x7ff1400ac9b0 0x7ff14014af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:39.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1400aed20 con 0x7ff14014ab80 2021-07-11T03:56:39.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7ff134001660 con 0x7ff14014ab80 2021-07-11T03:56:39.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13effd700 1 --2- 172.21.2.104:0/4068570276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1400aff10 0x7ff14005f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:56:39.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13ffff700 1 --2- 172.21.2.104:0/4068570276 >> v2:172.21.2.105:3300/0 conn(0x7ff1400ac9b0 0x7ff14014af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:56:39.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 >> v1:172.21.2.104:6789/0 conn(0x7ff14014ab80 legacy=0x7ff1400af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 --2- 172.21.2.104:0/4068570276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1400aff10 0x7ff12c0037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 --2- 172.21.2.104:0/4068570276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff12c003ec0 0x7ff12c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 --2- 172.21.2.104:0/4068570276 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff12c004b40 0x7ff12c004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14014b5e0 con 0x7ff12c003ec0 2021-07-11T03:56:39.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14014b880 con 0x7ff12c004b40 2021-07-11T03:56:39.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c00ba80 con 0x7ff1400aff10 2021-07-11T03:56:39.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13effd700 1 --2- 172.21.2.104:0/4068570276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff12c003ec0 0x7ff12c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13effd700 1 -- 172.21.2.104:0/4068570276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1400aff10 msgr2=0x7ff12c0037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:39.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13effd700 1 --2- 172.21.2.104:0/4068570276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1400aff10 0x7ff12c0037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:39.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13effd700 1 -- 172.21.2.104:0/4068570276 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff12c004b40 msgr2=0x7ff12c004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:39.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13effd700 1 --2- 172.21.2.104:0/4068570276 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff12c004b40 0x7ff12c004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:39.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13effd700 1 -- 172.21.2.104:0/4068570276 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c003260 con 0x7ff12c003ec0 2021-07-11T03:56:39.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13effd700 1 --2- 172.21.2.104:0/4068570276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff12c003ec0 0x7ff12c004500 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7ff130007000 tx=0x7ff130008040).ready entity=mon.0 client_cookie=f0e00ca8c741ed11 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:39.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff130011410 con 0x7ff12c003ec0 2021-07-11T03:56:39.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff130011570 con 0x7ff12c003ec0 2021-07-11T03:56:39.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13e7fc700 1 -- 172.21.2.104:0/4068570276 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff1300118d0 con 0x7ff12c003ec0 2021-07-11T03:56:39.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff145fd4700 1 -- 172.21.2.104:0/4068570276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff12c003ec0 msgr2=0x7ff12c004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff145fd4700 1 --2- 172.21.2.104:0/4068570276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff12c003ec0 0x7ff12c004500 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7ff130007000 tx=0x7ff130008040).stop 2021-07-11T03:56:39.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.525+0000 7ff13ffff700 1 -- 172.21.2.104:0/4068570276 reap_dead start 2021-07-11T03:56:39.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 -- 172.21.2.104:0/4068570276 shutdown_connections 2021-07-11T03:56:39.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 -- 172.21.2.104:0/4068570276 >> 172.21.2.104:0/4068570276 conn(0x7ff1400b5570 msgr2=0x7ff1400b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:39.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 -- 172.21.2.104:0/4068570276 shutdown_connections 2021-07-11T03:56:39.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 -- 172.21.2.104:0/4068570276 wait complete. 2021-07-11T03:56:39.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 Processor -- start 2021-07-11T03:56:39.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 -- start start 2021-07-11T03:56:39.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff14013e070 0x7ff140145820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1400b46f0 0x7ff1400b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1401453e0 0x7ff1400b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14013cb70 con 0x7ff1400b46f0 2021-07-11T03:56:39.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1400b5200 con 0x7ff14013e070 2021-07-11T03:56:39.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14013e450 con 0x7ff1401453e0 2021-07-11T03:56:39.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1400b46f0 0x7ff1400b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1400b46f0 0x7ff1400b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44388/0 (socket says 172.21.2.104:44388) 2021-07-11T03:56:39.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13effd700 1 -- 172.21.2.104:0/3797996645 learned_addr learned my addr 172.21.2.104:0/3797996645 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:39.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13effd700 1 -- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1401453e0 msgr2=0x7ff1400b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13effd700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1401453e0 0x7ff1400b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:39.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13effd700 1 -- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff14013e070 msgr2=0x7ff140145820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:39.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13effd700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff14013e070 0x7ff140145820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:39.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13effd700 1 -- 172.21.2.104:0/3797996645 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff140148090 con 0x7ff1400b46f0 2021-07-11T03:56:39.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13effd700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1400b46f0 0x7ff1400b3170 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7ff140147b10 tx=0x7ff130008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:39.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13cff9700 1 -- 172.21.2.104:0/3797996645 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff13001a610 con 0x7ff1400b46f0 2021-07-11T03:56:39.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13cff9700 1 -- 172.21.2.104:0/3797996645 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff130000e90 con 0x7ff1400b46f0 2021-07-11T03:56:39.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 -- 172.21.2.104:0/3797996645 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff140148330 con 0x7ff1400b46f0 2021-07-11T03:56:39.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13cff9700 1 -- 172.21.2.104:0/3797996645 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff13001a960 con 0x7ff1400b46f0 2021-07-11T03:56:39.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 -- 172.21.2.104:0/3797996645 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1401425a0 con 0x7ff1400b46f0 2021-07-11T03:56:39.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff145fd4700 1 -- 172.21.2.104:0/3797996645 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff140146900 con 0x7ff1400b46f0 2021-07-11T03:56:39.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13cff9700 1 -- 172.21.2.104:0/3797996645 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff13002b070 con 0x7ff1400b46f0 2021-07-11T03:56:39.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13cff9700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff120053270 0x7ff120055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:39.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.529+0000 7ff13cff9700 1 -- 172.21.2.104:0/3797996645 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6155+0+0 (secure 0 0 0) 0x7ff13006c640 con 0x7ff1400b46f0 2021-07-11T03:56:39.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.533+0000 7ff13f7fe700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff120053270 0x7ff120055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:39.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.533+0000 7ff13f7fe700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff120053270 0x7ff120055730 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7ff134001370 tx=0x7ff1401485c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:39.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.533+0000 7ff13cff9700 1 -- 172.21.2.104:0/3797996645 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff140146900 con 0x7ff1400b46f0 2021-07-11T03:56:39.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:39.641+0000 7ff145fd4700 1 -- 172.21.2.104:0/3797996645 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff140146900 con 0x7ff1400b46f0 2021-07-11T03:56:40.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.445+0000 7ff13cff9700 1 -- 172.21.2.104:0/3797996645 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff140146900 con 0x7ff1400b46f0 2021-07-11T03:56:40.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 -- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff120053270 msgr2=0x7ff120055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:40.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff120053270 0x7ff120055730 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7ff134001370 tx=0x7ff1401485c0).stop 2021-07-11T03:56:40.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 -- 172.21.2.104:0/3797996645 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1400b46f0 msgr2=0x7ff1400b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:40.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1400b46f0 0x7ff1400b3170 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7ff140147b10 tx=0x7ff130008040).stop 2021-07-11T03:56:40.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 -- 172.21.2.104:0/3797996645 shutdown_connections 2021-07-11T03:56:40.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1401453e0 0x7ff1400b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:40.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff120053270 0x7ff120055730 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:40.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1400b46f0 0x7ff1400b3170 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:40.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 --2- 172.21.2.104:0/3797996645 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff14013e070 0x7ff140145820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:40.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 -- 172.21.2.104:0/3797996645 >> 172.21.2.104:0/3797996645 conn(0x7ff1400b5570 msgr2=0x7ff1400b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:40.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 -- 172.21.2.104:0/3797996645 shutdown_connections 2021-07-11T03:56:40.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.449+0000 7ff145fd4700 1 -- 172.21.2.104:0/3797996645 wait complete. 2021-07-11T03:56:40.461 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T03:56:40.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d90d29700 1 Processor -- start 2021-07-11T03:56:40.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d90d29700 1 -- start start 2021-07-11T03:56:40.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d90d29700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0d8c0b1db0 0x7f0d8c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:40.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d90d29700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8c0b0350 con 0x7f0d8c1483a0 2021-07-11T03:56:40.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d90d29700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8c0b05f0 con 0x7f0d8c0a70c0 2021-07-11T03:56:40.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d90d29700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8c0a90c0 con 0x7f0d8c0b1db0 2021-07-11T03:56:40.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d89d9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f0d8c1483a0 0x7f0d8c0b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41164/0 (socket says 172.21.2.104:41164) 2021-07-11T03:56:40.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d89d9b700 1 -- 172.21.2.104:0/3885994695 learned_addr learned my addr 172.21.2.104:0/3885994695 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:40.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950156414 0 0) 0x7f0d8c0b0350 con 0x7f0d8c1483a0 2021-07-11T03:56:40.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d6c003260 con 0x7f0d8c1483a0 2021-07-11T03:56:40.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d8a59c700 1 --2- 172.21.2.104:0/3885994695 >> v2:172.21.2.105:3300/0 conn(0x7f0d8c0b1db0 0x7f0d8c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:40.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3147725925 0 0) 0x7f0d6c003260 con 0x7f0d8c1483a0 2021-07-11T03:56:40.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 >> v2:172.21.2.105:3300/0 conn(0x7f0d8c0b1db0 msgr2=0x7f0d8c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:40.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d8959a700 1 --2- 172.21.2.104:0/3885994695 >> v2:172.21.2.105:3300/0 conn(0x7f0d8c0b1db0 0x7f0d8c0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:40.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 >> v1:172.21.2.105:6789/0 conn(0x7f0d8c0a70c0 legacy=0x7f0d8c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:40.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c0b0890 con 0x7f0d8c1483a0 2021-07-11T03:56:40.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f0d74001660 con 0x7f0d8c1483a0 2021-07-11T03:56:40.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 >> v1:172.21.2.104:6789/0 conn(0x7f0d8c1483a0 legacy=0x7f0d8c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:40.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.509+0000 7f0d8959a700 1 --2- 172.21.2.104:0/3885994695 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0d6c003750 0x7f0d6c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:40.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8959a700 1 --2- 172.21.2.104:0/3885994695 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0d6c004500 0x7f0d6c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:40.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8959a700 1 --2- 172.21.2.104:0/3885994695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 0x7f0d6c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:40.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c00bd70 con 0x7f0d6c004ec0 2021-07-11T03:56:40.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c00bee0 con 0x7f0d6c004500 2021-07-11T03:56:40.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c00c0a0 con 0x7f0d6c003750 2021-07-11T03:56:40.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8ad9d700 1 --2- 172.21.2.104:0/3885994695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 0x7f0d6c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:40.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d89d9b700 1 --2- 172.21.2.104:0/3885994695 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0d6c004500 0x7f0d6c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:40.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8a59c700 1 --2- 172.21.2.104:0/3885994695 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0d6c003750 0x7f0d6c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:40.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8ad9d700 1 -- 172.21.2.104:0/3885994695 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0d6c003750 msgr2=0x7f0d6c003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:40.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8ad9d700 1 --2- 172.21.2.104:0/3885994695 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0d6c003750 0x7f0d6c003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:40.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8ad9d700 1 -- 172.21.2.104:0/3885994695 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0d6c004500 msgr2=0x7f0d6c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:40.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8ad9d700 1 --2- 172.21.2.104:0/3885994695 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0d6c004500 0x7f0d6c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:40.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8ad9d700 1 -- 172.21.2.104:0/3885994695 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c003260 con 0x7f0d6c004ec0 2021-07-11T03:56:40.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8ad9d700 1 -- 172.21.2.104:0/3885994695 reap_dead start 2021-07-11T03:56:40.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8ad9d700 1 --2- 172.21.2.104:0/3885994695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 0x7f0d6c00b750 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f0d8c0b3170 tx=0x7f0d84007d80).ready entity=mon.0 client_cookie=84cddf60bcd34729 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:40.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d84001620 con 0x7f0d6c004ec0 2021-07-11T03:56:40.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d84006890 con 0x7f0d6c004ec0 2021-07-11T03:56:40.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8959a700 1 -- 172.21.2.104:0/3885994695 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d84006bf0 con 0x7f0d6c004ec0 2021-07-11T03:56:40.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- 172.21.2.104:0/3885994695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 msgr2=0x7f0d6c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:40.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 --2- 172.21.2.104:0/3885994695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 0x7f0d6c00b750 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f0d8c0b3170 tx=0x7f0d84007d80).stop 2021-07-11T03:56:40.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- 172.21.2.104:0/3885994695 shutdown_connections 2021-07-11T03:56:40.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 --2- 172.21.2.104:0/3885994695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 0x7f0d6c00b750 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:40.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- 172.21.2.104:0/3885994695 >> 172.21.2.104:0/3885994695 conn(0x7f0d8c05f6b0 msgr2=0x7f0d8c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:40.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- 172.21.2.104:0/3885994695 shutdown_connections 2021-07-11T03:56:40.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- 172.21.2.104:0/3885994695 wait complete. 2021-07-11T03:56:40.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 Processor -- start 2021-07-11T03:56:40.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- start start 2021-07-11T03:56:40.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 0x7f0d8c1493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:40.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0d8c1570b0 0x7f0d8c159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:40.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0d8c15a0f0 0x7f0d8c15a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:40.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c157630 con 0x7f0d6c004ec0 2021-07-11T03:56:40.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c15aa60 con 0x7f0d8c1570b0 2021-07-11T03:56:40.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c14b4b0 con 0x7f0d8c15a0f0 2021-07-11T03:56:40.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8a59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 0x7f0d8c1493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:40.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8a59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 0x7f0d8c1493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44408/0 (socket says 172.21.2.104:44408) 2021-07-11T03:56:40.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8a59c700 1 -- 172.21.2.104:0/1125548708 learned_addr learned my addr 172.21.2.104:0/1125548708 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:40.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d89d9b700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0d8c1570b0 0x7f0d8c159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:40.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8ad9d700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0d8c15a0f0 0x7f0d8c15a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:40.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8a59c700 1 -- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0d8c15a0f0 msgr2=0x7f0d8c15a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:40.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8a59c700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0d8c15a0f0 0x7f0d8c15a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:40.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8a59c700 1 -- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0d8c1570b0 msgr2=0x7f0d8c159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:40.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8a59c700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0d8c1570b0 0x7f0d8c159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:40.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8a59c700 1 -- 172.21.2.104:0/1125548708 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c14b750 con 0x7f0d6c004ec0 2021-07-11T03:56:40.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d8a59c700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 0x7f0d8c1493a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f0d7c0030b0 tx=0x7f0d7c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:40.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d837fe700 1 -- 172.21.2.104:0/1125548708 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d7c0135f0 con 0x7f0d6c004ec0 2021-07-11T03:56:40.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d837fe700 1 -- 172.21.2.104:0/1125548708 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c000e90 con 0x7f0d6c004ec0 2021-07-11T03:56:40.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- 172.21.2.104:0/1125548708 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8c14b9f0 con 0x7f0d6c004ec0 2021-07-11T03:56:40.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- 172.21.2.104:0/1125548708 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8c15ef90 con 0x7f0d6c004ec0 2021-07-11T03:56:40.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d837fe700 1 -- 172.21.2.104:0/1125548708 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0d7c013940 con 0x7f0d6c004ec0 2021-07-11T03:56:40.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d90d29700 1 -- 172.21.2.104:0/1125548708 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8c057350 con 0x7f0d6c004ec0 2021-07-11T03:56:40.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.513+0000 7f0d837fe700 1 -- 172.21.2.104:0/1125548708 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0d7c013aa0 con 0x7f0d6c004ec0 2021-07-11T03:56:40.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.517+0000 7f0d837fe700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0d700531f0 0x7f0d700556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:40.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.517+0000 7f0d837fe700 1 -- 172.21.2.104:0/1125548708 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f0d7c06b420 con 0x7f0d6c004ec0 2021-07-11T03:56:40.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.517+0000 7f0d89d9b700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0d700531f0 0x7f0d700556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:40.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.517+0000 7f0d89d9b700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0d700531f0 0x7f0d700556b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f0d7400d000 tx=0x7f0d74007040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:40.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.517+0000 7f0d837fe700 1 -- 172.21.2.104:0/1125548708 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0d7c042600 con 0x7f0d6c004ec0 2021-07-11T03:56:40.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:40.601+0000 7f0d90d29700 1 -- 172.21.2.104:0/1125548708 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0d8c044650 con 0x7f0d6c004ec0 2021-07-11T03:56:41.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.453+0000 7f0d837fe700 1 -- 172.21.2.104:0/1125548708 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) v1 ==== 185+0+0 (secure 0 0 0) 0x7f0d7c017020 con 0x7f0d6c004ec0 2021-07-11T03:56:41.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 -- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0d700531f0 msgr2=0x7f0d700556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:41.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0d700531f0 0x7f0d700556b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f0d7400d000 tx=0x7f0d74007040).stop 2021-07-11T03:56:41.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 -- 172.21.2.104:0/1125548708 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 msgr2=0x7f0d8c1493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:41.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 0x7f0d8c1493a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f0d7c0030b0 tx=0x7f0d7c008040).stop 2021-07-11T03:56:41.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 -- 172.21.2.104:0/1125548708 shutdown_connections 2021-07-11T03:56:41.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0d8c15a0f0 0x7f0d8c15a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:41.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0d8c1570b0 0x7f0d8c159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:41.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0d700531f0 0x7f0d700556b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:41.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 --2- 172.21.2.104:0/1125548708 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0d6c004ec0 0x7f0d8c1493a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:41.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 -- 172.21.2.104:0/1125548708 >> 172.21.2.104:0/1125548708 conn(0x7f0d8c05f6b0 msgr2=0x7f0d8c0b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:41.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 -- 172.21.2.104:0/1125548708 shutdown_connections 2021-07-11T03:56:41.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.457+0000 7f0d90d29700 1 -- 172.21.2.104:0/1125548708 wait complete. 2021-07-11T03:56:41.469 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T03:56:41.476 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T03:56:41.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.521+0000 7fb9fd7c5700 1 Processor -- start 2021-07-11T03:56:41.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.521+0000 7fb9fd7c5700 1 -- start start 2021-07-11T03:56:41.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb9f8060910 0x7fb9f80acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:41.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb9f80aa5d0 0x7fb9f813c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:41.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f80b6ce0 con 0x7fb9f80b6900 2021-07-11T03:56:41.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f80611e0 con 0x7fb9f8060910 2021-07-11T03:56:41.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f8061480 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb9f8060910 0x7fb9f80acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:41.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb9f8060910 0x7fb9f80acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44418/0 (socket says 172.21.2.104:44418) 2021-07-11T03:56:41.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 -- 172.21.2.104:0/3420688579 learned_addr learned my addr 172.21.2.104:0/3420688579 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:41.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 -- 172.21.2.104:0/3420688579 >> v2:172.21.2.105:3300/0 conn(0x7fb9f80aa5d0 msgr2=0x7fb9f813c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:41.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 --2- 172.21.2.104:0/3420688579 >> v2:172.21.2.105:3300/0 conn(0x7fb9f80aa5d0 0x7fb9f813c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:41.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 -- 172.21.2.104:0/3420688579 >> v1:172.21.2.105:6789/0 conn(0x7fb9f80b6900 legacy=0x7fb9f80a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:41.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 -- 172.21.2.104:0/3420688579 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f8061720 con 0x7fb9f8060910 2021-07-11T03:56:41.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 --2- 172.21.2.104:0/3420688579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9f8060910 0x7fb9f80acd50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fb9f8060300 tx=0x7fb9e0006eb0).ready entity=mon.0 client_cookie=c37f4c3726cf85eb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:41.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f5ffb700 1 -- 172.21.2.104:0/3420688579 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9e0010810 con 0x7fb9f8060910 2021-07-11T03:56:41.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f5ffb700 1 -- 172.21.2.104:0/3420688579 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e0000e90 con 0x7fb9f8060910 2021-07-11T03:56:41.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f5ffb700 1 -- 172.21.2.104:0/3420688579 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9e0010ba0 con 0x7fb9f8060910 2021-07-11T03:56:41.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/3420688579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9f8060910 msgr2=0x7fb9f80acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:41.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 --2- 172.21.2.104:0/3420688579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9f8060910 0x7fb9f80acd50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7fb9f8060300 tx=0x7fb9e0006eb0).stop 2021-07-11T03:56:41.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/3420688579 shutdown_connections 2021-07-11T03:56:41.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 --2- 172.21.2.104:0/3420688579 >> v2:172.21.2.105:3300/0 conn(0x7fb9f80aa5d0 0x7fb9f813c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:41.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 --2- 172.21.2.104:0/3420688579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9f8060910 0x7fb9f80acd50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:41.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/3420688579 >> 172.21.2.104:0/3420688579 conn(0x7fb9f80b3480 msgr2=0x7fb9f80b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:41.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/3420688579 shutdown_connections 2021-07-11T03:56:41.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/3420688579 wait complete. 2021-07-11T03:56:41.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 Processor -- start 2021-07-11T03:56:41.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- start start 2021-07-11T03:56:41.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9f8060910 0x7fb9f80a90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:41.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9f80aa5d0 0x7fb9f813b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:41.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9f80b6900 0x7fb9f80a85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:41.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f813bd40 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f80a7e90 con 0x7fb9f80b6900 2021-07-11T03:56:41.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f80a8130 con 0x7fb9f8060910 2021-07-11T03:56:41.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9f80aa5d0 0x7fb9f813b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:41.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9f80aa5d0 0x7fb9f813b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44424/0 (socket says 172.21.2.104:44424) 2021-07-11T03:56:41.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 -- 172.21.2.104:0/1663183613 learned_addr learned my addr 172.21.2.104:0/1663183613 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:41.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 -- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9f8060910 msgr2=0x7fb9f80a90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:41.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9f8060910 0x7fb9f80a90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:41.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 -- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9f80b6900 msgr2=0x7fb9f80a85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:41.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9f80b6900 0x7fb9f80a85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:41.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 -- 172.21.2.104:0/1663183613 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e000c2f0 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9f67fc700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9f80aa5d0 0x7fb9f813b6a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fb9f8060300 tx=0x7fb9e0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:41.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9effff700 1 -- 172.21.2.104:0/1663183613 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9e0006be0 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9effff700 1 -- 172.21.2.104:0/1663183613 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e000c030 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/1663183613 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f80add80 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9effff700 1 -- 172.21.2.104:0/1663183613 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9e0002f70 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.525+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/1663183613 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f80a9880 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.529+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/1663183613 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f8057350 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.529+0000 7fb9effff700 1 -- 172.21.2.104:0/1663183613 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb9e000e040 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.529+0000 7fb9effff700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9dc053270 0x7fb9dc055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:41.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.529+0000 7fb9effff700 1 -- 172.21.2.104:0/1663183613 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5717+0+0 (secure 0 0 0) 0x7fb9e006cca0 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.529+0000 7fb9f6ffd700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9dc053270 0x7fb9dc055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:41.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.529+0000 7fb9f6ffd700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9dc053270 0x7fb9dc055730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fb9e8002f10 tx=0x7fb9e8006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:41.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.529+0000 7fb9effff700 1 -- 172.21.2.104:0/1663183613 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb9e0071070 con 0x7fb9f80aa5d0 2021-07-11T03:56:41.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:41.613+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/1663183613 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb9f8044650 con 0x7fb9f80aa5d0 2021-07-11T03:56:42.296 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:56:42.299 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:56:42.302 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:56:42.305 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:56:42.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9effff700 1 -- 172.21.2.104:0/1663183613 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) v1 ==== 173+0+0 (secure 0 0 0) 0x7fb9e0047b70 con 0x7fb9f80aa5d0 2021-07-11T03:56:42.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9dc053270 msgr2=0x7fb9dc055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9dc053270 0x7fb9dc055730 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7fb9e8002f10 tx=0x7fb9e8006930).stop 2021-07-11T03:56:42.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/1663183613 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9f80aa5d0 msgr2=0x7fb9f813b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9f80aa5d0 0x7fb9f813b6a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7fb9f8060300 tx=0x7fb9e0017000).stop 2021-07-11T03:56:42.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/1663183613 shutdown_connections 2021-07-11T03:56:42.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9f80b6900 0x7fb9f80a85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:42.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9dc053270 0x7fb9dc055730 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:42.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9f80aa5d0 0x7fb9f813b6a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:42.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 --2- 172.21.2.104:0/1663183613 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9f8060910 0x7fb9f80a90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:42.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/1663183613 >> 172.21.2.104:0/1663183613 conn(0x7fb9f80b3480 msgr2=0x7fb9f8139820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:42.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/1663183613 shutdown_connections 2021-07-11T03:56:42.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.453+0000 7fb9fd7c5700 1 -- 172.21.2.104:0/1663183613 wait complete. 2021-07-11T03:56:42.464 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T03:56:42.468 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-07-11T03:56:42.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.569+0000 7f97fd7ad700 1 Processor -- start 2021-07-11T03:56:42.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.569+0000 7f97fd7ad700 1 -- start start 2021-07-11T03:56:42.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.569+0000 7f97fd7ad700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97f813c9c0 0x7f97f80b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.569+0000 7f97fd7ad700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80b5f40 con 0x7f97f805d760 2021-07-11T03:56:42.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.569+0000 7f97fd7ad700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80ad420 con 0x7f97f80ad040 2021-07-11T03:56:42.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.569+0000 7f97fd7ad700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80aeb60 con 0x7f97f813c9c0 2021-07-11T03:56:42.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.569+0000 7f97f77fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f97f805d760 0x7f97f813f070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41202/0 (socket says 172.21.2.104:41202) 2021-07-11T03:56:42.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.569+0000 7f97f77fe700 1 -- 172.21.2.104:0/1512690118 learned_addr learned my addr 172.21.2.104:0/1512690118 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:42.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628087596 0 0) 0x7f97f80b5f40 con 0x7f97f805d760 2021-07-11T03:56:42.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97dc003260 con 0x7f97f805d760 2021-07-11T03:56:42.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 776764791 0 0) 0x7f97dc003260 con 0x7f97f805d760 2021-07-11T03:56:42.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97f813c9c0 msgr2=0x7f97f80b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:42.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 --2- 172.21.2.104:0/1512690118 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97f813c9c0 0x7f97f80b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:42.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 >> v1:172.21.2.105:6789/0 conn(0x7f97f80ad040 legacy=0x7f97f813c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:42.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80aee00 con 0x7f97f805d760 2021-07-11T03:56:42.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f97f00019e0 con 0x7f97f805d760 2021-07-11T03:56:42.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 >> v1:172.21.2.104:6789/0 conn(0x7f97f805d760 legacy=0x7f97f813f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 --2- 172.21.2.104:0/1512690118 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97dc003800 0x7f97dc003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 --2- 172.21.2.104:0/1512690118 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97f813c9c0 0x7f97dc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 --2- 172.21.2.104:0/1512690118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97dc004bf0 0x7f97dc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80aeb60 con 0x7f97dc004bf0 2021-07-11T03:56:42.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc0052a0 con 0x7f97dc003800 2021-07-11T03:56:42.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc00bc40 con 0x7f97f813c9c0 2021-07-11T03:56:42.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f77fe700 1 --2- 172.21.2.104:0/1512690118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97dc004bf0 0x7f97dc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:42.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f6ffd700 1 --2- 172.21.2.104:0/1512690118 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97dc003800 0x7f97dc003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:42.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f77fe700 1 -- 172.21.2.104:0/1512690118 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97f813c9c0 msgr2=0x7f97dc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:42.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f77fe700 1 --2- 172.21.2.104:0/1512690118 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97f813c9c0 0x7f97dc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:42.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f77fe700 1 -- 172.21.2.104:0/1512690118 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97dc003800 msgr2=0x7f97dc003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:42.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f77fe700 1 --2- 172.21.2.104:0/1512690118 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97dc003800 0x7f97dc003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:42.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f77fe700 1 -- 172.21.2.104:0/1512690118 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc003260 con 0x7f97dc004bf0 2021-07-11T03:56:42.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f6ffd700 1 --2- 172.21.2.104:0/1512690118 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97dc003800 0x7f97dc003c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:56:42.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f77fe700 1 --2- 172.21.2.104:0/1512690118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97dc004bf0 0x7f97dc00b750 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f97f805d010 tx=0x7f97f80b1270).ready entity=mon.0 client_cookie=2f2c4084c7cf6922 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:42.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97f000cbb0 con 0x7f97dc004bf0 2021-07-11T03:56:42.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f000cd10 con 0x7f97dc004bf0 2021-07-11T03:56:42.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f5ffb700 1 -- 172.21.2.104:0/1512690118 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97f0005b80 con 0x7f97dc004bf0 2021-07-11T03:56:42.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 -- 172.21.2.104:0/1512690118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97dc004bf0 msgr2=0x7f97dc00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 --2- 172.21.2.104:0/1512690118 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97dc004bf0 0x7f97dc00b750 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f97f805d010 tx=0x7f97f80b1270).stop 2021-07-11T03:56:42.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f77fe700 1 -- 172.21.2.104:0/1512690118 reap_dead start 2021-07-11T03:56:42.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 -- 172.21.2.104:0/1512690118 shutdown_connections 2021-07-11T03:56:42.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 -- 172.21.2.104:0/1512690118 >> 172.21.2.104:0/1512690118 conn(0x7f97f805c490 msgr2=0x7f97f805c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:42.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 -- 172.21.2.104:0/1512690118 shutdown_connections 2021-07-11T03:56:42.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 -- 172.21.2.104:0/1512690118 wait complete. 2021-07-11T03:56:42.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 Processor -- start 2021-07-11T03:56:42.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 -- start start 2021-07-11T03:56:42.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97f813e260 0x7f97f813e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97f81362c0 0x7f97f81407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97f81402e0 0x7f97f8140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8142aa0 con 0x7f97f81362c0 2021-07-11T03:56:42.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8063e10 con 0x7f97f81402e0 2021-07-11T03:56:42.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97fd7ad700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8142490 con 0x7f97f813e260 2021-07-11T03:56:42.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97f81362c0 0x7f97f81407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:42.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f67fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97f81362c0 0x7f97f81407f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44444/0 (socket says 172.21.2.104:44444) 2021-07-11T03:56:42.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f67fc700 1 -- 172.21.2.104:0/3133180573 learned_addr learned my addr 172.21.2.104:0/3133180573 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:42.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f67fc700 1 -- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97f813e260 msgr2=0x7f97f813e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f67fc700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97f813e260 0x7f97f813e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:42.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f67fc700 1 -- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97f81402e0 msgr2=0x7f97f8140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f67fc700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97f81402e0 0x7f97f8140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:42.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.573+0000 7f97f67fc700 1 -- 172.21.2.104:0/3133180573 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f8064040 con 0x7f97f81362c0 2021-07-11T03:56:42.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97f67fc700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97f81362c0 0x7f97f81407f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f97ec002f70 tx=0x7f97ec006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:42.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97e7fff700 1 -- 172.21.2.104:0/3133180573 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97ec00e470 con 0x7f97f81362c0 2021-07-11T03:56:42.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97fd7ad700 1 -- 172.21.2.104:0/3133180573 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f80642e0 con 0x7f97f81362c0 2021-07-11T03:56:42.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97e7fff700 1 -- 172.21.2.104:0/3133180573 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec000e90 con 0x7f97f81362c0 2021-07-11T03:56:42.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97e7fff700 1 -- 172.21.2.104:0/3133180573 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97ec00e7c0 con 0x7f97f81362c0 2021-07-11T03:56:42.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97fd7ad700 1 -- 172.21.2.104:0/3133180573 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f8143200 con 0x7f97f81362c0 2021-07-11T03:56:42.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97fd7ad700 1 -- 172.21.2.104:0/3133180573 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f8064780 con 0x7f97f81362c0 2021-07-11T03:56:42.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97e7fff700 1 -- 172.21.2.104:0/3133180573 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f97ec00f070 con 0x7f97f81362c0 2021-07-11T03:56:42.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97e7fff700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97d4053270 0x7f97d4055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97e7fff700 1 -- 172.21.2.104:0/3133180573 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f97ec06a590 con 0x7f97f81362c0 2021-07-11T03:56:42.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97f6ffd700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97d4053270 0x7f97d4055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:42.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.577+0000 7f97f6ffd700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97d4053270 0x7f97d4055730 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f97e8008000 tx=0x7f97e8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:42.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.581+0000 7f97e7fff700 1 -- 172.21.2.104:0/3133180573 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f97f8064780 con 0x7f97f81362c0 2021-07-11T03:56:42.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.661+0000 7f97fd7ad700 1 -- 172.21.2.104:0/3133180573 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f97f8064780 con 0x7f97f81362c0 2021-07-11T03:56:42.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.665+0000 7f97e7fff700 1 -- 172.21.2.104:0/3133180573 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f97f8064780 con 0x7f97f81362c0 2021-07-11T03:56:42.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 -- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97d4053270 msgr2=0x7f97d4055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97d4053270 0x7f97d4055730 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f97e8008000 tx=0x7f97e8009040).stop 2021-07-11T03:56:42.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 -- 172.21.2.104:0/3133180573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97f81362c0 msgr2=0x7f97f81407f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97f81362c0 0x7f97f81407f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f97ec002f70 tx=0x7f97ec006e40).stop 2021-07-11T03:56:42.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 -- 172.21.2.104:0/3133180573 shutdown_connections 2021-07-11T03:56:42.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97f81402e0 0x7f97f8140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:42.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97d4053270 0x7f97d4055730 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:42.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97f81362c0 0x7f97f81407f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:42.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 --2- 172.21.2.104:0/3133180573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97f813e260 0x7f97f813e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:42.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 -- 172.21.2.104:0/3133180573 >> 172.21.2.104:0/3133180573 conn(0x7f97f805c490 msgr2=0x7f97f81385b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:42.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 -- 172.21.2.104:0/3133180573 shutdown_connections 2021-07-11T03:56:42.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.669+0000 7f97fd7ad700 1 -- 172.21.2.104:0/3133180573 wait complete. 2021-07-11T03:56:42.681 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead' 2021-07-11T03:56:42.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75c392d700 1 Processor -- start 2021-07-11T03:56:42.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75c392d700 1 -- start start 2021-07-11T03:56:42.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75c392d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f75bc060100 0x7f75bc0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75c392d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc138cc0 0x7f75bc139120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75c392d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75bc0604e0 con 0x7f75bc142840 2021-07-11T03:56:42.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75c392d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75bc139610 con 0x7f75bc060100 2021-07-11T03:56:42.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75c392d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc0b16b0 con 0x7f75bc138cc0 2021-07-11T03:56:42.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75c1eca700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f75bc142840 0x7f75bc142cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41220/0 (socket says 172.21.2.104:41220) 2021-07-11T03:56:42.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75c1eca700 1 -- 172.21.2.104:0/1691839387 learned_addr learned my addr 172.21.2.104:0/1691839387 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:42.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 783877606 0 0) 0x7f75bc0604e0 con 0x7f75bc142840 2021-07-11T03:56:42.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75a4003260 con 0x7f75bc142840 2021-07-11T03:56:42.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3197846904 0 0) 0x7f75a4003260 con 0x7f75bc142840 2021-07-11T03:56:42.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc138cc0 msgr2=0x7f75bc139120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:42.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75abfff700 1 --2- 172.21.2.104:0/1691839387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc138cc0 0x7f75bc139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:42.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 >> v2:172.21.2.105:3300/0 conn(0x7f75bc060100 msgr2=0x7f75bc0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:42.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75abfff700 1 --2- 172.21.2.104:0/1691839387 >> v2:172.21.2.105:3300/0 conn(0x7f75bc060100 0x7f75bc0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:42.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc0ac400 con 0x7f75bc142840 2021-07-11T03:56:42.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f75b4001a00 con 0x7f75bc142840 2021-07-11T03:56:42.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 >> v1:172.21.2.104:6789/0 conn(0x7f75bc142840 legacy=0x7f75bc142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.725+0000 7f75abfff700 1 --2- 172.21.2.104:0/1691839387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc138cc0 0x7f75a4003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75abfff700 1 --2- 172.21.2.104:0/1691839387 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75a4003e10 0x7f75a4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75abfff700 1 --2- 172.21.2.104:0/1691839387 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a4004b10 0x7f75a4004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc139610 con 0x7f75a4004b10 2021-07-11T03:56:42.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc0b16b0 con 0x7f75a4003e10 2021-07-11T03:56:42.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a400caf0 con 0x7f75bc138cc0 2021-07-11T03:56:42.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 --2- 172.21.2.104:0/1691839387 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a4004b10 0x7f75a4004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:42.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c16c9700 1 --2- 172.21.2.104:0/1691839387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc138cc0 0x7f75a4003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:42.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 -- 172.21.2.104:0/1691839387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc138cc0 msgr2=0x7f75a4003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:42.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 --2- 172.21.2.104:0/1691839387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc138cc0 0x7f75a4003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:42.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 -- 172.21.2.104:0/1691839387 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75a4003e10 msgr2=0x7f75a4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:42.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 --2- 172.21.2.104:0/1691839387 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75a4003e10 0x7f75a4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:42.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 -- 172.21.2.104:0/1691839387 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a4003260 con 0x7f75a4004b10 2021-07-11T03:56:42.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 --2- 172.21.2.104:0/1691839387 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a4004b10 0x7f75a4004f90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f75b4001710 tx=0x7f75b4004930).ready entity=mon.0 client_cookie=40cfc34f8d016c8f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:42.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75b4014070 con 0x7f75a4004b10 2021-07-11T03:56:42.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b400f430 con 0x7f75a4004b10 2021-07-11T03:56:42.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75abfff700 1 -- 172.21.2.104:0/1691839387 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75b400f790 con 0x7f75a4004b10 2021-07-11T03:56:42.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 -- 172.21.2.104:0/1691839387 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a4004b10 msgr2=0x7f75a4004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 --2- 172.21.2.104:0/1691839387 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a4004b10 0x7f75a4004f90 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f75b4001710 tx=0x7f75b4004930).stop 2021-07-11T03:56:42.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 -- 172.21.2.104:0/1691839387 reap_dead start 2021-07-11T03:56:42.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 -- 172.21.2.104:0/1691839387 shutdown_connections 2021-07-11T03:56:42.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 -- 172.21.2.104:0/1691839387 >> 172.21.2.104:0/1691839387 conn(0x7f75bc061070 msgr2=0x7f75bc05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:42.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 -- 172.21.2.104:0/1691839387 shutdown_connections 2021-07-11T03:56:42.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 -- 172.21.2.104:0/1691839387 wait complete. 2021-07-11T03:56:42.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 Processor -- start 2021-07-11T03:56:42.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 -- start start 2021-07-11T03:56:42.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75bc065910 0x7f75bc065cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc0afac0 0x7f75bc0add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75bc0ae220 0x7f75bc063280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc15a4a0 con 0x7f75bc0ae220 2021-07-11T03:56:42.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc0aefa0 con 0x7f75bc065910 2021-07-11T03:56:42.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75bc064af0 con 0x7f75bc0afac0 2021-07-11T03:56:42.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75bc0ae220 0x7f75bc063280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:42.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75bc0ae220 0x7f75bc063280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44462/0 (socket says 172.21.2.104:44462) 2021-07-11T03:56:42.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 -- 172.21.2.104:0/1728131562 learned_addr learned my addr 172.21.2.104:0/1728131562 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:42.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c0ec8700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc0afac0 0x7f75bc0add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:42.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c16c9700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75bc065910 0x7f75bc065cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:42.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 -- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc0afac0 msgr2=0x7f75bc0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc0afac0 0x7f75bc0add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:42.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 -- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75bc065910 msgr2=0x7f75bc065cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:42.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75bc065910 0x7f75bc065cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:42.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 -- 172.21.2.104:0/1728131562 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc0ae6a0 con 0x7f75bc0ae220 2021-07-11T03:56:42.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c1eca700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75bc0ae220 0x7f75bc063280 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f75bc0b5910 tx=0x7f75b4004930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:42.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75aa7fc700 1 -- 172.21.2.104:0/1728131562 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75b4021030 con 0x7f75bc0ae220 2021-07-11T03:56:42.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75aa7fc700 1 -- 172.21.2.104:0/1728131562 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b4001f80 con 0x7f75bc0ae220 2021-07-11T03:56:42.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 -- 172.21.2.104:0/1728131562 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75bc0ae9a0 con 0x7f75bc0ae220 2021-07-11T03:56:42.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75aa7fc700 1 -- 172.21.2.104:0/1728131562 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75b40019c0 con 0x7f75bc0ae220 2021-07-11T03:56:42.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.729+0000 7f75c392d700 1 -- 172.21.2.104:0/1728131562 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75bc15a920 con 0x7f75bc0ae220 2021-07-11T03:56:42.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.733+0000 7f75c392d700 1 -- 172.21.2.104:0/1728131562 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75bc0af330 con 0x7f75bc0ae220 2021-07-11T03:56:42.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.733+0000 7f75aa7fc700 1 -- 172.21.2.104:0/1728131562 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f75b4027470 con 0x7f75bc0ae220 2021-07-11T03:56:42.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.733+0000 7f75aa7fc700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f759c0532b0 0x7f759c055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:42.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.733+0000 7f75aa7fc700 1 -- 172.21.2.104:0/1728131562 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f75b4022070 con 0x7f75bc0ae220 2021-07-11T03:56:42.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.733+0000 7f75c16c9700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f759c0532b0 0x7f759c055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:42.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.733+0000 7f75c16c9700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f759c0532b0 0x7f759c055770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f75ac00c000 tx=0x7f75ac008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:42.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.733+0000 7f75aa7fc700 1 -- 172.21.2.104:0/1728131562 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f75b4045440 con 0x7f75bc0ae220 2021-07-11T03:56:42.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:42.845+0000 7f75c392d700 1 -- 172.21.2.104:0/1728131562 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]} v 0) v1 -- 0x7f75bc057350 con 0x7f75bc0ae220 2021-07-11T03:56:43.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75aa7fc700 1 -- 172.21.2.104:0/1728131562 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead"]}]=0 v145) v1 ==== 144+0+0 (secure 0 0 0) 0x7f75b4048a60 con 0x7f75bc0ae220 2021-07-11T03:56:43.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 -- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f759c0532b0 msgr2=0x7f759c055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:43.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f759c0532b0 0x7f759c055770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7f75ac00c000 tx=0x7f75ac008040).stop 2021-07-11T03:56:43.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 -- 172.21.2.104:0/1728131562 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75bc0ae220 msgr2=0x7f75bc063280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:43.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75bc0ae220 0x7f75bc063280 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f75bc0b5910 tx=0x7f75b4004930).stop 2021-07-11T03:56:43.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 -- 172.21.2.104:0/1728131562 shutdown_connections 2021-07-11T03:56:43.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f759c0532b0 0x7f759c055770 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:43.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75bc0ae220 0x7f75bc063280 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:43.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75bc0afac0 0x7f75bc0add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:43.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 --2- 172.21.2.104:0/1728131562 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75bc065910 0x7f75bc065cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:43.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 -- 172.21.2.104:0/1728131562 >> 172.21.2.104:0/1728131562 conn(0x7f75bc061070 msgr2=0x7f75bc0a97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:43.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 -- 172.21.2.104:0/1728131562 shutdown_connections 2021-07-11T03:56:43.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.461+0000 7f75c392d700 1 -- 172.21.2.104:0/1728131562 wait complete. 2021-07-11T03:56:43.476 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-07-11T03:56:43.477 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2021-07-11T03:56:43.477 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2021-07-11T03:56:43.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 Processor -- start 2021-07-11T03:56:43.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- start start 2021-07-11T03:56:43.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9e88060100 0x7f9e880b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:43.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9e88067300 0x7f9e881438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:43.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e880aabb0 con 0x7f9e880b4c40 2021-07-11T03:56:43.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e880604e0 con 0x7f9e88067300 2021-07-11T03:56:43.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e88066ea0 con 0x7f9e88060100 2021-07-11T03:56:43.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9e88067300 0x7f9e881438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:43.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9e88067300 0x7f9e881438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44470/0 (socket says 172.21.2.104:44470) 2021-07-11T03:56:43.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 -- 172.21.2.104:0/2318468616 learned_addr learned my addr 172.21.2.104:0/2318468616 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:43.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 -- 172.21.2.104:0/2318468616 >> v2:172.21.2.105:3300/0 conn(0x7f9e88060100 msgr2=0x7f9e880b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:43.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 --2- 172.21.2.104:0/2318468616 >> v2:172.21.2.105:3300/0 conn(0x7f9e88060100 0x7f9e880b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:43.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 -- 172.21.2.104:0/2318468616 >> v1:172.21.2.105:6789/0 conn(0x7f9e880b4c40 legacy=0x7f9e88144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:43.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 -- 172.21.2.104:0/2318468616 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e880b33b0 con 0x7f9e88067300 2021-07-11T03:56:43.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 --2- 172.21.2.104:0/2318468616 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e88067300 0x7f9e881438f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f9e8805f1e0 tx=0x7f9e70006eb0).ready entity=mon.0 client_cookie=d5e04eef9b40973 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:43.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e877fe700 1 -- 172.21.2.104:0/2318468616 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e7000d5c0 con 0x7f9e88067300 2021-07-11T03:56:43.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e877fe700 1 -- 172.21.2.104:0/2318468616 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e70000e90 con 0x7f9e88067300 2021-07-11T03:56:43.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e877fe700 1 -- 172.21.2.104:0/2318468616 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e7000d910 con 0x7f9e88067300 2021-07-11T03:56:43.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/2318468616 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e88067300 msgr2=0x7f9e881438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:43.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 --2- 172.21.2.104:0/2318468616 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e88067300 0x7f9e881438f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f9e8805f1e0 tx=0x7f9e70006eb0).stop 2021-07-11T03:56:43.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/2318468616 shutdown_connections 2021-07-11T03:56:43.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 --2- 172.21.2.104:0/2318468616 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e88067300 0x7f9e881438f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:43.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 --2- 172.21.2.104:0/2318468616 >> v2:172.21.2.105:3300/0 conn(0x7f9e88060100 0x7f9e880b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:43.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/2318468616 >> 172.21.2.104:0/2318468616 conn(0x7f9e8805cf30 msgr2=0x7f9e8805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:43.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/2318468616 shutdown_connections 2021-07-11T03:56:43.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/2318468616 wait complete. 2021-07-11T03:56:43.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 Processor -- start 2021-07-11T03:56:43.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- start start 2021-07-11T03:56:43.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e88060100 0x7f9e8814ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:43.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e88067300 0x7f9e88061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:43.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e880b4c40 0x7f9e880aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:43.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880ab7c0 con 0x7f9e88067300 2021-07-11T03:56:43.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8813e760 con 0x7f9e880b4c40 2021-07-11T03:56:43.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e88061c80 con 0x7f9e88060100 2021-07-11T03:56:43.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e88067300 0x7f9e88061790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:43.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e88067300 0x7f9e88061790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44476/0 (socket says 172.21.2.104:44476) 2021-07-11T03:56:43.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 -- 172.21.2.104:0/1078181996 learned_addr learned my addr 172.21.2.104:0/1078181996 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:43.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 -- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e88060100 msgr2=0x7f9e8814ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:43.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e88060100 0x7f9e8814ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:43.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 -- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e880b4c40 msgr2=0x7f9e880aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:43.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e880b4c40 0x7f9e880aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:43.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 -- 172.21.2.104:0/1078181996 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7000b1e0 con 0x7f9e88067300 2021-07-11T03:56:43.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e87fff700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e88067300 0x7f9e88061790 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f9e880623a0 tx=0x7f9e70015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:43.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e85ffb700 1 -- 172.21.2.104:0/1078181996 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e70021070 con 0x7f9e88067300 2021-07-11T03:56:43.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e85ffb700 1 -- 172.21.2.104:0/1078181996 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e70000e90 con 0x7f9e88067300 2021-07-11T03:56:43.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/1078181996 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e880ab520 con 0x7f9e88067300 2021-07-11T03:56:43.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e85ffb700 1 -- 172.21.2.104:0/1078181996 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e70006b80 con 0x7f9e88067300 2021-07-11T03:56:43.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.525+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/1078181996 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e88061050 con 0x7f9e88067300 2021-07-11T03:56:43.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.529+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/1078181996 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e88061360 con 0x7f9e88067300 2021-07-11T03:56:43.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.529+0000 7f9e85ffb700 1 -- 172.21.2.104:0/1078181996 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9e7000f070 con 0x7f9e88067300 2021-07-11T03:56:43.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.529+0000 7f9e85ffb700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e6c0531f0 0x7f9e6c0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:43.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.529+0000 7f9e85ffb700 1 -- 172.21.2.104:0/1078181996 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9e7006c620 con 0x7f9e88067300 2021-07-11T03:56:43.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.529+0000 7f9e8cac6700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e6c0531f0 0x7f9e6c0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:43.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.529+0000 7f9e8cac6700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e6c0531f0 0x7f9e6c0556b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f9e78002f70 tx=0x7f9e78006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:43.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.529+0000 7f9e85ffb700 1 -- 172.21.2.104:0/1078181996 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9e88061360 con 0x7f9e88067300 2021-07-11T03:56:43.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:43.641+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/1078181996 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f9e88061360 con 0x7f9e88067300 2021-07-11T03:56:44.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.465+0000 7f9e85ffb700 1 -- 172.21.2.104:0/1078181996 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v146) v1 ==== 155+0+0 (secure 0 0 0) 0x7f9e88061360 con 0x7f9e88067300 2021-07-11T03:56:44.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e6c0531f0 msgr2=0x7f9e6c0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e6c0531f0 0x7f9e6c0556b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f9e78002f70 tx=0x7f9e78006a80).stop 2021-07-11T03:56:44.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/1078181996 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e88067300 msgr2=0x7f9e88061790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e88067300 0x7f9e88061790 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f9e880623a0 tx=0x7f9e70015000).stop 2021-07-11T03:56:44.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/1078181996 shutdown_connections 2021-07-11T03:56:44.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e880b4c40 0x7f9e880aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e6c0531f0 0x7f9e6c0556b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:44.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e88067300 0x7f9e88061790 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:44.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 --2- 172.21.2.104:0/1078181996 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e88060100 0x7f9e8814ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/1078181996 >> 172.21.2.104:0/1078181996 conn(0x7f9e8805cf30 msgr2=0x7f9e88144b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:44.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/1078181996 shutdown_connections 2021-07-11T03:56:44.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.469+0000 7f9e8ed2a700 1 -- 172.21.2.104:0/1078181996 wait complete. 2021-07-11T03:56:44.481 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T03:56:44.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3b55d1700 1 Processor -- start 2021-07-11T03:56:44.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3b55d1700 1 -- start start 2021-07-11T03:56:44.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3b55d1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3b00a9ec0 0x7ff3b00aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3b55d1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff3b013f790 0x7ff3b013fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3b55d1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b0061990 con 0x7ff3b00603d0 2021-07-11T03:56:44.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3b55d1700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b0140110 con 0x7ff3b013f790 2021-07-11T03:56:44.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3b55d1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0061c30 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3ae7fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff3b00603d0 0x7ff3b013ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41256/0 (socket says 172.21.2.104:41256) 2021-07-11T03:56:44.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3ae7fc700 1 -- 172.21.2.104:0/3145159393 learned_addr learned my addr 172.21.2.104:0/3145159393 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:44.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3af7fe700 1 --2- 172.21.2.104:0/3145159393 >> v2:172.21.2.104:3300/0 conn(0x7ff3b013f790 0x7ff3b013fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:44.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3af7fe700 1 -- 172.21.2.104:0/3145159393 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3b00a9ec0 msgr2=0x7ff3b00aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:44.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3af7fe700 1 --2- 172.21.2.104:0/3145159393 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3b00a9ec0 0x7ff3b00aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3af7fe700 1 -- 172.21.2.104:0/3145159393 >> v1:172.21.2.104:6789/0 conn(0x7ff3b00603d0 legacy=0x7ff3b013ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3af7fe700 1 -- 172.21.2.104:0/3145159393 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b0060b90 con 0x7ff3b013f790 2021-07-11T03:56:44.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.529+0000 7ff3adffb700 1 -- 172.21.2.104:0/3145159393 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480082610 0 0) 0x7ff3b0061990 con 0x7ff3b00603d0 2021-07-11T03:56:44.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3af7fe700 1 --2- 172.21.2.104:0/3145159393 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3b013f790 0x7ff3b013fc20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7ff3a80032f0 tx=0x7ff3a80091d0).ready entity=mon.0 client_cookie=dc842356dc4b2ed5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:44.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3adffb700 1 -- 172.21.2.104:0/3145159393 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a8009ea0 con 0x7ff3b013f790 2021-07-11T03:56:44.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3adffb700 1 -- 172.21.2.104:0/3145159393 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a800b8a0 con 0x7ff3b013f790 2021-07-11T03:56:44.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3adffb700 1 -- 172.21.2.104:0/3145159393 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff3a800ba10 con 0x7ff3b013f790 2021-07-11T03:56:44.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- 172.21.2.104:0/3145159393 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3b013f790 msgr2=0x7ff3b013fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 --2- 172.21.2.104:0/3145159393 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3b013f790 0x7ff3b013fc20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7ff3a80032f0 tx=0x7ff3a80091d0).stop 2021-07-11T03:56:44.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- 172.21.2.104:0/3145159393 shutdown_connections 2021-07-11T03:56:44.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 --2- 172.21.2.104:0/3145159393 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3b013f790 0x7ff3b013fc20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:44.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 --2- 172.21.2.104:0/3145159393 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3b00a9ec0 0x7ff3b00aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- 172.21.2.104:0/3145159393 >> 172.21.2.104:0/3145159393 conn(0x7ff3b00b2970 msgr2=0x7ff3b00b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:44.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- 172.21.2.104:0/3145159393 shutdown_connections 2021-07-11T03:56:44.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- 172.21.2.104:0/3145159393 wait complete. 2021-07-11T03:56:44.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 Processor -- start 2021-07-11T03:56:44.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- start start 2021-07-11T03:56:44.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff3b00603d0 0x7ff3b0141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3b00a9ec0 0x7ff3b0142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3b013f790 0x7ff3b0140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0142870 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b0063e40 con 0x7ff3b00603d0 2021-07-11T03:56:44.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b013ab60 con 0x7ff3b013f790 2021-07-11T03:56:44.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3ae7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3b00a9ec0 0x7ff3b0142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:44.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3ae7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3b00a9ec0 0x7ff3b0142030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44496/0 (socket says 172.21.2.104:44496) 2021-07-11T03:56:44.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3ae7fc700 1 -- 172.21.2.104:0/361679849 learned_addr learned my addr 172.21.2.104:0/361679849 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:44.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3ae7fc700 1 -- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3b013f790 msgr2=0x7ff3b0140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3ae7fc700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3b013f790 0x7ff3b0140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3ae7fc700 1 -- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff3b00603d0 msgr2=0x7ff3b0141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3ae7fc700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff3b00603d0 0x7ff3b0141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3ae7fc700 1 -- 172.21.2.104:0/361679849 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a8009470 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3ae7fc700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3b00a9ec0 0x7ff3b0142030 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7ff398000bc0 tx=0x7ff398007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:44.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3a7fff700 1 -- 172.21.2.104:0/361679849 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff39800ce50 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3a7fff700 1 -- 172.21.2.104:0/361679849 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3980072e0 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3a7fff700 1 -- 172.21.2.104:0/361679849 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff398005800 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- 172.21.2.104:0/361679849 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b01435b0 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- 172.21.2.104:0/361679849 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b00ae970 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3b55d1700 1 -- 172.21.2.104:0/361679849 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b0057350 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.533+0000 7ff3a7fff700 1 -- 172.21.2.104:0/361679849 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff398010070 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.537+0000 7ff3a7fff700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff394053300 0x7ff3940557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.537+0000 7ff3a7fff700 1 -- 172.21.2.104:0/361679849 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff39806aa10 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.537+0000 7ff3aeffd700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff394053300 0x7ff3940557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:44.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.537+0000 7ff3aeffd700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff394053300 0x7ff3940557c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7ff3a0002f70 tx=0x7ff3a0006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:44.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.537+0000 7ff3a7fff700 1 -- 172.21.2.104:0/361679849 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff398042030 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.621+0000 7ff3b55d1700 1 -- 172.21.2.104:0/361679849 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff3b00b04f0 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3a7fff700 1 -- 172.21.2.104:0/361679849 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff398045650 con 0x7ff3b00a9ec0 2021-07-11T03:56:44.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 -- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff394053300 msgr2=0x7ff3940557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff394053300 0x7ff3940557c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7ff3a0002f70 tx=0x7ff3a0006a80).stop 2021-07-11T03:56:44.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 -- 172.21.2.104:0/361679849 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3b00a9ec0 msgr2=0x7ff3b0142030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3b00a9ec0 0x7ff3b0142030 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7ff398000bc0 tx=0x7ff398007a60).stop 2021-07-11T03:56:44.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 -- 172.21.2.104:0/361679849 shutdown_connections 2021-07-11T03:56:44.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3b013f790 0x7ff3b0140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff394053300 0x7ff3940557c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:44.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3b00a9ec0 0x7ff3b0142030 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:44.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 --2- 172.21.2.104:0/361679849 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff3b00603d0 0x7ff3b0141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 -- 172.21.2.104:0/361679849 >> 172.21.2.104:0/361679849 conn(0x7ff3b00b2970 msgr2=0x7ff3b013f030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:44.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 -- 172.21.2.104:0/361679849 shutdown_connections 2021-07-11T03:56:44.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.625+0000 7ff3b55d1700 1 -- 172.21.2.104:0/361679849 wait complete. 2021-07-11T03:56:44.639 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T03:56:44.639 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:56:44.647 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:56:44.660 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:56:44.674 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:56:44.688 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T03:56:44.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:44.689 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:56:44.689 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:56:44.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:56:44.694 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T03:56:44.695 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T03:56:44.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:44.696 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:56:44.696 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:56:44.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:56:44.701 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:56:44.701 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T03:56:44.702 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T03:56:44.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.765+0000 7f3a930f3700 1 Processor -- start 2021-07-11T03:56:44.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.765+0000 7f3a930f3700 1 -- start start 2021-07-11T03:56:44.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a8c0b0fd0 0x7f3a8c144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8c0b0350 con 0x7f3a8c1483a0 2021-07-11T03:56:44.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8c0b05f0 con 0x7f3a8c0b1db0 2021-07-11T03:56:44.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c0a90c0 con 0x7f3a8c0b0fd0 2021-07-11T03:56:44.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8bfff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f3a8c1483a0 0x7f3a8c0a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41268/0 (socket says 172.21.2.104:41268) 2021-07-11T03:56:44.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8bfff700 1 -- 172.21.2.104:0/1973389220 learned_addr learned my addr 172.21.2.104:0/1973389220 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:44.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379154647 0 0) 0x7f3a8c0b0350 con 0x7f3a8c1483a0 2021-07-11T03:56:44.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a6c003260 con 0x7f3a8c1483a0 2021-07-11T03:56:44.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2963557901 0 0) 0x7f3a6c003260 con 0x7f3a8c1483a0 2021-07-11T03:56:44.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a8c0b0fd0 msgr2=0x7f3a8c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:44.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 --2- 172.21.2.104:0/1973389220 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a8c0b0fd0 0x7f3a8c144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 >> v1:172.21.2.105:6789/0 conn(0x7f3a8c0b1db0 legacy=0x7f3a8c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:44.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c0b0890 con 0x7f3a8c1483a0 2021-07-11T03:56:44.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f3a7c001660 con 0x7f3a8c1483a0 2021-07-11T03:56:44.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 >> v1:172.21.2.104:6789/0 conn(0x7f3a8c1483a0 legacy=0x7f3a8c0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 --2- 172.21.2.104:0/1973389220 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a6c003780 0x7f3a6c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 --2- 172.21.2.104:0/1973389220 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a8c0b0fd0 0x7f3a6c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 --2- 172.21.2.104:0/1973389220 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c004b10 0x7f3a6c004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c00b8e0 con 0x7f3a6c004b10 2021-07-11T03:56:44.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c00ba50 con 0x7f3a6c003780 2021-07-11T03:56:44.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6c00bc10 con 0x7f3a8c0b0fd0 2021-07-11T03:56:44.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a91690700 1 --2- 172.21.2.104:0/1973389220 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c004b10 0x7f3a6c004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:44.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a91690700 1 -- 172.21.2.104:0/1973389220 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a8c0b0fd0 msgr2=0x7f3a6c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:44.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a91690700 1 --2- 172.21.2.104:0/1973389220 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a8c0b0fd0 0x7f3a6c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a91690700 1 -- 172.21.2.104:0/1973389220 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a6c003780 msgr2=0x7f3a6c003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:44.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a91690700 1 --2- 172.21.2.104:0/1973389220 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a6c003780 0x7f3a6c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a91690700 1 -- 172.21.2.104:0/1973389220 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6c003260 con 0x7f3a6c004b10 2021-07-11T03:56:44.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a91690700 1 --2- 172.21.2.104:0/1973389220 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c004b10 0x7f3a6c004f90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f3a840032f0 tx=0x7f3a84006f90).ready entity=mon.0 client_cookie=947add5e4acdbcaa server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:44.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a8400b460 con 0x7f3a6c004b10 2021-07-11T03:56:44.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8400b5c0 con 0x7f3a6c004b10 2021-07-11T03:56:44.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8b7fe700 1 -- 172.21.2.104:0/1973389220 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a8400b730 con 0x7f3a6c004b10 2021-07-11T03:56:44.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- 172.21.2.104:0/1973389220 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c004b10 msgr2=0x7f3a6c004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 --2- 172.21.2.104:0/1973389220 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a6c004b10 0x7f3a6c004f90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f3a840032f0 tx=0x7f3a84006f90).stop 2021-07-11T03:56:44.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a91690700 1 -- 172.21.2.104:0/1973389220 reap_dead start 2021-07-11T03:56:44.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- 172.21.2.104:0/1973389220 shutdown_connections 2021-07-11T03:56:44.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- 172.21.2.104:0/1973389220 >> 172.21.2.104:0/1973389220 conn(0x7f3a8c05f6b0 msgr2=0x7f3a8c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:44.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- 172.21.2.104:0/1973389220 shutdown_connections 2021-07-11T03:56:44.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- 172.21.2.104:0/1973389220 wait complete. 2021-07-11T03:56:44.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 Processor -- start 2021-07-11T03:56:44.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- start start 2021-07-11T03:56:44.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a8c148ca0 0x7f3a8c14b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a8c13f470 0x7f3a8c1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a8c156aa0 0x7f3a8c1419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c14aa10 con 0x7f3a8c13f470 2021-07-11T03:56:44.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c13ef20 con 0x7f3a8c148ca0 2021-07-11T03:56:44.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a930f3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c14ac40 con 0x7f3a8c156aa0 2021-07-11T03:56:44.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a8c13f470 0x7f3a8c1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:44.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a8c13f470 0x7f3a8c1565b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44510/0 (socket says 172.21.2.104:44510) 2021-07-11T03:56:44.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.769+0000 7f3a8bfff700 1 -- 172.21.2.104:0/2102844660 learned_addr learned my addr 172.21.2.104:0/2102844660 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:44.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a8bfff700 1 -- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a8c156aa0 msgr2=0x7f3a8c1419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a8bfff700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a8c156aa0 0x7f3a8c1419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a8bfff700 1 -- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a8c148ca0 msgr2=0x7f3a8c14b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:44.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a8bfff700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a8c148ca0 0x7f3a8c14b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:44.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a8bfff700 1 -- 172.21.2.104:0/2102844660 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c14aee0 con 0x7f3a8c13f470 2021-07-11T03:56:44.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a8bfff700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a8c13f470 0x7f3a8c1565b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f3a8c0613a0 tx=0x7f3a8c0b04e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:44.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a89ffb700 1 -- 172.21.2.104:0/2102844660 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a7c007170 con 0x7f3a8c13f470 2021-07-11T03:56:44.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a89ffb700 1 -- 172.21.2.104:0/2102844660 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c0072d0 con 0x7f3a8c13f470 2021-07-11T03:56:44.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a89ffb700 1 -- 172.21.2.104:0/2102844660 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3a7c0049e0 con 0x7f3a8c13f470 2021-07-11T03:56:44.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a930f3700 1 -- 172.21.2.104:0/2102844660 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8c149080 con 0x7f3a8c13f470 2021-07-11T03:56:44.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a930f3700 1 -- 172.21.2.104:0/2102844660 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c140270 con 0x7f3a8c13f470 2021-07-11T03:56:44.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a930f3700 1 -- 172.21.2.104:0/2102844660 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8c13f090 con 0x7f3a8c13f470 2021-07-11T03:56:44.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a89ffb700 1 -- 172.21.2.104:0/2102844660 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3a7c01b070 con 0x7f3a8c13f470 2021-07-11T03:56:44.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a89ffb700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053350 0x7f3a70055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:44.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a89ffb700 1 -- 172.21.2.104:0/2102844660 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3a7c06ac00 con 0x7f3a8c13f470 2021-07-11T03:56:44.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a90e8f700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053350 0x7f3a70055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:44.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.773+0000 7f3a90e8f700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053350 0x7f3a70055810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f3a74000f70 tx=0x7f3a740069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:44.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.777+0000 7f3a89ffb700 1 -- 172.21.2.104:0/2102844660 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3a8c13f090 con 0x7f3a8c13f470 2021-07-11T03:56:44.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 -- 172.21.2.104:0/2102844660 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3a8c13f090 con 0x7f3a8c13f470 2021-07-11T03:56:44.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a89ffb700 1 -- 172.21.2.104:0/2102844660 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v47) v1 ==== 81+0+28825 (secure 0 0 0) 0x7f3a8c13f090 con 0x7f3a8c13f470 2021-07-11T03:56:44.866 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:44.866 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:56:44.866 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 47, 2021-07-11T03:56:44.867 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T03:56:44.867 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T03:56:44.867 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T03:56:42.452706+0000", 2021-07-11T03:56:44.867 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:56:44.868 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:56:44.868 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T03:56:44.868 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T03:56:44.868 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T03:56:44.869 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T03:56:44.869 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T03:56:44.869 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T03:56:44.869 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T03:56:44.869 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:56:44.870 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T03:56:44.870 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T03:56:44.870 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T03:56:44.870 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T03:56:44.871 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T03:56:44.871 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 8, 2021-07-11T03:56:44.871 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T03:56:44.871 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T03:56:44.871 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T03:56:44.872 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T03:56:44.872 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T03:56:44.872 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.872 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T03:56:44.873 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T03:56:44.873 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T03:56:44.873 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:56:44.873 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:56:44.873 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:56:44.874 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:56:44.874 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:56:44.874 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:56:44.874 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:56:44.875 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:56:44.875 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T03:56:44.875 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T03:56:44.875 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T03:56:44.875 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T03:56:44.876 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T03:56:44.876 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:56:44.876 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:56:44.876 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:56:44.876 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:56:44.877 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:56:44.877 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:56:44.877 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:56:44.877 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.878 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T03:56:44.878 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:56:44.878 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:56:44.878 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:56:44.878 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:56:44.879 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:56:44.879 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:56:44.879 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:56:44.879 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:56:44.880 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:56:44.880 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:56:44.880 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:56:44.880 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:56:44.880 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:56:44.881 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:56:44.881 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:56:44.881 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:56:44.881 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:56:44.882 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:56:44.882 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:56:44.882 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:56:44.882 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:56:44.883 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:56:44.883 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:56:44.883 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:56:44.883 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:56:44.883 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:56:44.884 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.884 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:56:44.884 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:56:44.884 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:56:44.884 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:56:44.885 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:56:44.885 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:56:44.885 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:56:44.885 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:56:44.885 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:56:44.886 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:56:44.886 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:56:44.886 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:56:44.886 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T03:56:44.887 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.887 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:56:44.887 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T03:56:44.887 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.887 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.888 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.888 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T03:56:44.888 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T03:56:44.888 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T03:56:44.889 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:56:44.889 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:56:44.889 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:56:44.889 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:56:44.889 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:56:44.890 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:56:44.890 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:56:44.890 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:56:44.890 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:56:44.890 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:56:44.891 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:56:44.891 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:56:44.891 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:56:44.891 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:56:44.891 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:56:44.892 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:56:44.892 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:56:44.892 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:56:44.892 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:56:44.892 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:56:44.893 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.893 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T03:56:44.893 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:56:44.893 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:56:44.894 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:56:44.894 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:56:44.894 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:56:44.894 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:56:44.894 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:56:44.895 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:56:44.895 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:56:44.895 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:56:44.895 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:56:44.895 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:56:44.896 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:56:44.896 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:56:44.896 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:56:44.896 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:56:44.897 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:56:44.897 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:56:44.897 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:56:44.897 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:56:44.897 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:56:44.898 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:56:44.898 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:56:44.898 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:56:44.898 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:56:44.898 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:56:44.899 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.899 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:56:44.899 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:56:44.899 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:56:44.899 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:56:44.900 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:56:44.900 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:56:44.900 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:56:44.900 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:56:44.901 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:56:44.901 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:56:44.901 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:56:44.901 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:56:44.901 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:56:44.902 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T03:56:44.902 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.902 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.902 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:56:44.902 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T03:56:44.903 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.903 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:56:44.903 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T03:56:44.903 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:44.903 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:44.904 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:44.904 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:44.904 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:44.904 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:44.904 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:44.905 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:44.905 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:44.905 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:44.905 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:44.906 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.906 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.906 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.906 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T03:56:44.906 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:44.907 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.907 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.907 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.907 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T03:56:44.907 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:44.908 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.908 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.908 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.908 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:44.909 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.909 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.909 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.909 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T03:56:44.909 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:44.910 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.910 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.910 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.910 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T03:56:44.910 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:44.911 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.911 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.911 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.911 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:44.912 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.912 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.912 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.912 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T03:56:44.912 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:44.913 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.913 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.913 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.913 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T03:56:44.913 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:44.914 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.914 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.914 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.915 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:44.915 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.915 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.915 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.916 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T03:56:44.916 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:44.916 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.916 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.917 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.917 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T03:56:44.917 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:56:44.918 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.918 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.918 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.918 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T03:56:44.919 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T03:56:44.919 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T03:56:44.919 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T03:56:44.920 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:44.920 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:44.920 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:44.920 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.921 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.921 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.921 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:56:44.922 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T03:56:44.922 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:44.922 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:44.922 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:44.923 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:44.923 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:44.923 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:44.924 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:44.924 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:44.924 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:44.924 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:44.925 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:44.925 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.925 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.926 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.926 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T03:56:44.926 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:44.927 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.927 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.927 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.927 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T03:56:44.928 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:44.928 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.928 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.929 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.929 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:44.929 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.929 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.930 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.930 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T03:56:44.930 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:44.931 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.931 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.931 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.932 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T03:56:44.932 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:44.932 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.932 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.933 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.933 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:44.933 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.934 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.934 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.934 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T03:56:44.935 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:44.935 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.935 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.936 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.936 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T03:56:44.936 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:44.937 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.937 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.937 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.938 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:44.938 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.938 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.939 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.939 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T03:56:44.939 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:44.939 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.940 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.940 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.940 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T03:56:44.941 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:56:44.941 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.941 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.942 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.942 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T03:56:44.942 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T03:56:44.943 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T03:56:44.943 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T03:56:44.943 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:44.943 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:44.944 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:44.944 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.944 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.945 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.945 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:56:44.945 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T03:56:44.945 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:44.946 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:44.946 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:44.947 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:44.947 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:44.947 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:44.948 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:44.948 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 30, 2021-07-11T03:56:44.948 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:44.948 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:44.949 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:44.949 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.949 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.950 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.950 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T03:56:44.950 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:44.951 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.951 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.951 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.952 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T03:56:44.952 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:44.952 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.953 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.953 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.953 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:44.953 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.954 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.954 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.954 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T03:56:44.955 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:44.955 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.955 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.955 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.956 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T03:56:44.956 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:44.956 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.957 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.957 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.957 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:44.958 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.958 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.958 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.958 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T03:56:44.959 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:44.959 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.959 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.960 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T03:56:44.960 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:44.960 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.961 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.961 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.961 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:44.962 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.962 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.962 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.962 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T03:56:44.963 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:44.963 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.963 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.964 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.964 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T03:56:44.964 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:56:44.965 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.965 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.965 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.966 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T03:56:44.966 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T03:56:44.966 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T03:56:44.966 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T03:56:44.967 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:44.967 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:44.967 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:44.968 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.968 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.968 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.969 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:56:44.969 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T03:56:44.969 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:44.970 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:44.970 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:44.970 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:44.971 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:44.971 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:44.971 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:44.972 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:44.972 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:44.972 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:44.973 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:44.973 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.973 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.974 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.974 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T03:56:44.974 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:44.975 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.975 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.975 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.976 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T03:56:44.976 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:44.976 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.976 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.977 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.977 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:44.977 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.978 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.978 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.978 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T03:56:44.979 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:44.979 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.979 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.980 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.980 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T03:56:44.980 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:44.980 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.981 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.981 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.981 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:44.982 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.982 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.982 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.983 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T03:56:44.983 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:44.983 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.983 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.984 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.984 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T03:56:44.984 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:44.985 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.985 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.985 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.986 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:44.986 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.986 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.986 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.987 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T03:56:44.987 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:44.987 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.988 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.988 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.988 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T03:56:44.989 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:56:44.989 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:44.989 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.989 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.990 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T03:56:44.990 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T03:56:44.990 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T03:56:44.991 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T03:56:44.991 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:44.991 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:44.992 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:44.992 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:44.992 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.993 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.993 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:56:44.993 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T03:56:44.994 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:44.994 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:44.994 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:44.995 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:44.995 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:44.995 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:44.995 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:44.996 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:44.996 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:44.996 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:44.997 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:44.997 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:44.997 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.997 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:44.998 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T03:56:44.998 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:44.998 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:44.999 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:44.999 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:44.999 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T03:56:45.000 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:45.487 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.487 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.487 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.488 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:45.488 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.488 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.488 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.489 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T03:56:45.489 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:45.489 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.490 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.490 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.490 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T03:56:45.490 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:45.491 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.491 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.491 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.492 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:45.492 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.492 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.492 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.493 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T03:56:45.493 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:45.493 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.494 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.494 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.494 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T03:56:45.494 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:45.495 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.495 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.495 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.495 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:45.495 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.496 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.496 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.496 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T03:56:45.496 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:45.497 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.497 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.497 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.497 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T03:56:45.497 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:56:45.498 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.498 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.498 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.498 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T03:56:45.499 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T03:56:45.499 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T03:56:45.499 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T03:56:45.499 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:45.499 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:45.500 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:45.500 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.500 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.500 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.501 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:56:45.501 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T03:56:45.501 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:45.501 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:45.501 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:45.502 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:45.502 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:45.502 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:45.502 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:45.502 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:45.503 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:45.503 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:45.503 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:45.503 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.503 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.504 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.504 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T03:56:45.504 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:45.504 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.504 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.505 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.505 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T03:56:45.505 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:45.505 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.506 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.506 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.506 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:45.506 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.506 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.507 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.507 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T03:56:45.507 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:45.507 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.507 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.508 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.508 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T03:56:45.508 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:45.508 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.508 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.509 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.509 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:45.509 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.509 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.510 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.510 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T03:56:45.510 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:45.510 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.510 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.511 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.511 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T03:56:45.511 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:45.511 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.512 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.512 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.512 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:45.512 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.512 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.513 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.513 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T03:56:45.513 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:45.513 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.513 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.514 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.514 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T03:56:45.514 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:56:45.514 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.514 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.515 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.515 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T03:56:45.515 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T03:56:45.515 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T03:56:45.516 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T03:56:45.516 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:45.516 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:45.516 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:45.516 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.517 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.517 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.517 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:56:45.517 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T03:56:45.517 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:45.518 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:45.518 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:45.518 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:45.518 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:45.519 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:45.519 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:45.519 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 42, 2021-07-11T03:56:45.519 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:45.519 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:45.520 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:45.520 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.520 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.520 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.520 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T03:56:45.521 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:45.521 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.521 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.521 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.521 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T03:56:45.522 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:45.522 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.522 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.522 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.523 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:45.523 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.523 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.523 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.523 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T03:56:45.524 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:45.524 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.524 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.524 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.524 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T03:56:45.525 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:45.525 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.525 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.525 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.526 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:45.526 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.526 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.526 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.526 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T03:56:45.527 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:45.527 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.527 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.527 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.527 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T03:56:45.528 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:45.528 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.528 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.528 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.528 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:45.529 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.529 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.529 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.529 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T03:56:45.530 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:45.530 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.530 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.530 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.530 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T03:56:45.531 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:56:45.531 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.531 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.531 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.531 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T03:56:45.532 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T03:56:45.532 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T03:56:45.532 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T03:56:45.532 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:45.532 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:45.533 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:45.533 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.533 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.533 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.533 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:56:45.534 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T03:56:45.534 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:56:45.534 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:56:45.534 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:56:45.535 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:56:45.535 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:56:45.535 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:56:45.535 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:56:45.535 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 41, 2021-07-11T03:56:45.536 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:56:45.536 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:56:45.536 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:56:45.536 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.536 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.537 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.537 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T03:56:45.537 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:45.537 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.538 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.538 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.538 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T03:56:45.538 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:45.539 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.539 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.539 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.539 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:56:45.539 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.540 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.540 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.540 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T03:56:45.540 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:45.541 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.541 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.541 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.541 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T03:56:45.541 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:45.542 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.542 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.542 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.542 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:56:45.542 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.543 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.543 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.543 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T03:56:45.543 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:45.543 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.544 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.544 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.544 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T03:56:45.544 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:45.545 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.545 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.545 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.545 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:56:45.545 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:56:45.546 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.546 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:56:45.546 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T03:56:45.546 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:45.547 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.547 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.547 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:56:45.547 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T03:56:45.547 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:56:45.548 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.548 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.548 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.548 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T03:56:45.548 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T03:56:45.549 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T03:56:45.549 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T03:56:45.549 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:56:45.549 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:56:45.549 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:56:45.550 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:45.550 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.550 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:56:45.550 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T03:56:45.551 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.551 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:56:45.551 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:45.551 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:45.551 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:45.552 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:45.552 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:45.552 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T03:56:45.552 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:45.552 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.553 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.553 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:56:45.553 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:45.553 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:45.554 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:45.554 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:45.554 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:45.554 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T03:56:45.554 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:45.555 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.555 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.555 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:56:45.555 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:45.556 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:45.556 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:45.556 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:45.556 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:45.556 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T03:56:45.557 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:45.557 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.557 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.557 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:56:45.557 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:45.558 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:45.558 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:45.558 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:45.558 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:45.559 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T03:56:45.559 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:45.559 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.559 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.559 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:56:45.560 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:45.560 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:45.560 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:45.560 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:45.560 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:45.561 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T03:56:45.561 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:45.561 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.561 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.561 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:56:45.562 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:45.562 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:45.562 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:45.562 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:45.562 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:45.563 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T03:56:45.563 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:45.563 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.563 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.563 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:56:45.564 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:45.564 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:45.564 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:45.564 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:45.565 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:45.565 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T03:56:45.565 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:45.565 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.565 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:45.566 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:56:45.566 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:56:45.566 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:56:45.566 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:56:45.566 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:56:45.567 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:56:45.567 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T03:56:45.567 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:56:45.567 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.567 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:56:45.568 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T03:56:45.568 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T03:56:45.568 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T03:56:45.568 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T03:56:45.568 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T03:56:45.569 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6835/795410455": "2021-07-12T03:56:38.437243+0000", 2021-07-11T03:56:45.569 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6833/795410455": "2021-07-12T03:56:38.437243+0000" 2021-07-11T03:56:45.569 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.569 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T03:56:45.569 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T03:56:45.570 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T03:56:45.570 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T03:56:45.570 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T03:56:45.570 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T03:56:45.571 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T03:56:45.571 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T03:56:45.571 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:45.571 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:45.571 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T03:56:45.572 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T03:56:45.572 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T03:56:45.572 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T03:56:45.572 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T03:56:45.573 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:56:45.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 -- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053350 msgr2=0x7f3a70055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053350 0x7f3a70055810 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f3a74000f70 tx=0x7f3a740069b0).stop 2021-07-11T03:56:45.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 -- 172.21.2.104:0/2102844660 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a8c13f470 msgr2=0x7f3a8c1565b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a8c13f470 0x7f3a8c1565b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f3a8c0613a0 tx=0x7f3a8c0b04e0).stop 2021-07-11T03:56:45.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 -- 172.21.2.104:0/2102844660 shutdown_connections 2021-07-11T03:56:45.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3a8c156aa0 0x7f3a8c1419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3a70053350 0x7f3a70055810 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:45.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3a8c13f470 0x7f3a8c1565b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:45.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 --2- 172.21.2.104:0/2102844660 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3a8c148ca0 0x7f3a8c14b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 -- 172.21.2.104:0/2102844660 >> 172.21.2.104:0/2102844660 conn(0x7f3a8c05f6b0 msgr2=0x7f3a8c0a8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:45.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 -- 172.21.2.104:0/2102844660 shutdown_connections 2021-07-11T03:56:45.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:44.857+0000 7f3a930f3700 1 -- 172.21.2.104:0/2102844660 wait complete. 2021-07-11T03:56:45.577 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:56:45.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f66471f2700 1 Processor -- start 2021-07-11T03:56:45.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f66471f2700 1 -- start start 2021-07-11T03:56:45.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f66471f2700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6640067180 0x7f664005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f66471f2700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f66400b1970 0x7f664005fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f66471f2700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66400a7a90 con 0x7f66400a76b0 2021-07-11T03:56:45.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f66471f2700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6640147620 con 0x7f6640067180 2021-07-11T03:56:45.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f66471f2700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66401478c0 con 0x7f66400b1970 2021-07-11T03:56:45.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f663ffff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6640067180 0x7f664005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:45.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f663ffff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6640067180 0x7f664005f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44522/0 (socket says 172.21.2.104:44522) 2021-07-11T03:56:45.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f663ffff700 1 -- 172.21.2.104:0/3194788063 learned_addr learned my addr 172.21.2.104:0/3194788063 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:45.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f663ffff700 1 -- 172.21.2.104:0/3194788063 >> v2:172.21.2.105:3300/0 conn(0x7f66400b1970 msgr2=0x7f664005fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:45.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f663ffff700 1 --2- 172.21.2.104:0/3194788063 >> v2:172.21.2.105:3300/0 conn(0x7f66400b1970 0x7f664005fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f663ffff700 1 -- 172.21.2.104:0/3194788063 >> v1:172.21.2.105:6789/0 conn(0x7f66400a76b0 legacy=0x7f6640066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:45.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.625+0000 7f663ffff700 1 -- 172.21.2.104:0/3194788063 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6640147b60 con 0x7f6640067180 2021-07-11T03:56:45.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f663ffff700 1 --2- 172.21.2.104:0/3194788063 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6640067180 0x7f664005f6b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f664006a0e0 tx=0x7f6630006eb0).ready entity=mon.0 client_cookie=1592f498efa55702 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:45.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f663f7fe700 1 -- 172.21.2.104:0/3194788063 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663000d5c0 con 0x7f6640067180 2021-07-11T03:56:45.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f663f7fe700 1 -- 172.21.2.104:0/3194788063 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6630000e90 con 0x7f6640067180 2021-07-11T03:56:45.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f663f7fe700 1 -- 172.21.2.104:0/3194788063 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663000d910 con 0x7f6640067180 2021-07-11T03:56:45.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- 172.21.2.104:0/3194788063 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6640067180 msgr2=0x7f664005f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 --2- 172.21.2.104:0/3194788063 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6640067180 0x7f664005f6b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f664006a0e0 tx=0x7f6630006eb0).stop 2021-07-11T03:56:45.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- 172.21.2.104:0/3194788063 shutdown_connections 2021-07-11T03:56:45.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 --2- 172.21.2.104:0/3194788063 >> v2:172.21.2.105:3300/0 conn(0x7f66400b1970 0x7f664005fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 --2- 172.21.2.104:0/3194788063 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6640067180 0x7f664005f6b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:45.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- 172.21.2.104:0/3194788063 >> 172.21.2.104:0/3194788063 conn(0x7f66400b53e0 msgr2=0x7f66400b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:45.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- 172.21.2.104:0/3194788063 shutdown_connections 2021-07-11T03:56:45.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- 172.21.2.104:0/3194788063 wait complete. 2021-07-11T03:56:45.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 Processor -- start 2021-07-11T03:56:45.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- start start 2021-07-11T03:56:45.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6640067180 0x7f66400b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66400a76b0 0x7f66400af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66400b1970 0x7f6640144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66400aa740 con 0x7f66400b1970 2021-07-11T03:56:45.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66400abc50 con 0x7f66400a76b0 2021-07-11T03:56:45.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66400aaaa0 con 0x7f6640067180 2021-07-11T03:56:45.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f664578f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66400b1970 0x7f6640144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:45.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f664578f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66400b1970 0x7f6640144700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44528/0 (socket says 172.21.2.104:44528) 2021-07-11T03:56:45.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f664578f700 1 -- 172.21.2.104:0/2527052401 learned_addr learned my addr 172.21.2.104:0/2527052401 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:45.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f664578f700 1 -- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6640067180 msgr2=0x7f66400b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f664578f700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6640067180 0x7f66400b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f664578f700 1 -- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66400a76b0 msgr2=0x7f66400af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f664578f700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66400a76b0 0x7f66400af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f664578f700 1 -- 172.21.2.104:0/2527052401 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663000b1e0 con 0x7f66400b1970 2021-07-11T03:56:45.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f664578f700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66400b1970 0x7f6640144700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f66380032f0 tx=0x7f66380091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:45.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f663dffb700 1 -- 172.21.2.104:0/2527052401 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663800f760 con 0x7f66400b1970 2021-07-11T03:56:45.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f663dffb700 1 -- 172.21.2.104:0/2527052401 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663800f8c0 con 0x7f66400b1970 2021-07-11T03:56:45.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- 172.21.2.104:0/2527052401 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66400aa430 con 0x7f66400b1970 2021-07-11T03:56:45.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f663dffb700 1 -- 172.21.2.104:0/2527052401 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f663800fc20 con 0x7f66400b1970 2021-07-11T03:56:45.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- 172.21.2.104:0/2527052401 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66400aaf40 con 0x7f66400b1970 2021-07-11T03:56:45.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.629+0000 7f66471f2700 1 -- 172.21.2.104:0/2527052401 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66400ab980 con 0x7f66400b1970 2021-07-11T03:56:45.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.633+0000 7f663dffb700 1 -- 172.21.2.104:0/2527052401 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f663801a460 con 0x7f66400b1970 2021-07-11T03:56:45.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.633+0000 7f663dffb700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6624053300 0x7f66240557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.633+0000 7f663dffb700 1 -- 172.21.2.104:0/2527052401 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f663806b740 con 0x7f66400b1970 2021-07-11T03:56:45.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.633+0000 7f6644f8e700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6624053300 0x7f66240557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:45.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.633+0000 7f6644f8e700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6624053300 0x7f66240557c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f6628000f70 tx=0x7f66280077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:45.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.633+0000 7f663dffb700 1 -- 172.21.2.104:0/2527052401 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6638042bf0 con 0x7f66400b1970 2021-07-11T03:56:45.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.741+0000 7f66471f2700 1 -- 172.21.2.104:0/2527052401 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66400aac60 con 0x7f66400b1970 2021-07-11T03:56:45.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.741+0000 7f663dffb700 1 -- 172.21.2.104:0/2527052401 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f6638046210 con 0x7f66400b1970 2021-07-11T03:56:45.750 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:45.750 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4492,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2412138431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2412138431},{"type":"v1","addr":"172.21.2.105:6835","nonce":2412138431}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/3271417890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3271417890},{"type":"v1","addr":"172.21.2.104:6834","nonce":3271417890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6835/2058921436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":2058921436},{"type":"v1","addr":"172.21.2.104:6835","nonce":2058921436}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[]} 2021-07-11T03:56:45.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 -- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6624053300 msgr2=0x7f66240557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6624053300 0x7f66240557c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f6628000f70 tx=0x7f66280077f0).stop 2021-07-11T03:56:45.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 -- 172.21.2.104:0/2527052401 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66400b1970 msgr2=0x7f6640144700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66400b1970 0x7f6640144700 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f66380032f0 tx=0x7f66380091d0).stop 2021-07-11T03:56:45.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 -- 172.21.2.104:0/2527052401 shutdown_connections 2021-07-11T03:56:45.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6624053300 0x7f66240557c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:45.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66400b1970 0x7f6640144700 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:45.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66400a76b0 0x7f66400af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 --2- 172.21.2.104:0/2527052401 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6640067180 0x7f66400b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 -- 172.21.2.104:0/2527052401 >> 172.21.2.104:0/2527052401 conn(0x7f66400b53e0 msgr2=0x7f66400b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:45.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 -- 172.21.2.104:0/2527052401 shutdown_connections 2021-07-11T03:56:45.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.745+0000 7f66471f2700 1 -- 172.21.2.104:0/2527052401 wait complete. 2021-07-11T03:56:45.754 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 19 2021-07-11T03:56:45.757 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:56:45.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 Processor -- start 2021-07-11T03:56:45.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 -- start start 2021-07-11T03:56:45.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b00a76b0 0x7f33b0066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f33b00b1970 0x7f33b0066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f33b00b0060 0x7f33b005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b0067450 con 0x7f33b00b0060 2021-07-11T03:56:45.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b00a7a90 con 0x7f33b00b1970 2021-07-11T03:56:45.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b0147620 con 0x7f33b00a76b0 2021-07-11T03:56:45.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b63e0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f33b00b0060 0x7f33b005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:45.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b63e0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f33b00b0060 0x7f33b005f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44532/0 (socket says 172.21.2.104:44532) 2021-07-11T03:56:45.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b63e0700 1 -- 172.21.2.104:0/3525128499 learned_addr learned my addr 172.21.2.104:0/3525128499 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:45.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b63e0700 1 -- 172.21.2.104:0/3525128499 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b00a76b0 msgr2=0x7f33b0066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:45.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b63e0700 1 --2- 172.21.2.104:0/3525128499 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b00a76b0 0x7f33b0066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b63e0700 1 -- 172.21.2.104:0/3525128499 >> v2:172.21.2.105:3300/0 conn(0x7f33b00b1970 msgr2=0x7f33b0066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:45.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b63e0700 1 --2- 172.21.2.104:0/3525128499 >> v2:172.21.2.105:3300/0 conn(0x7f33b00b1970 0x7f33b0066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b63e0700 1 -- 172.21.2.104:0/3525128499 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b01478c0 con 0x7f33b00b0060 2021-07-11T03:56:45.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b63e0700 1 --2- 172.21.2.104:0/3525128499 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b00b0060 0x7f33b005f6b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f33a80032f0 tx=0x7f33a8009290).ready entity=mon.0 client_cookie=cab167fd5f82282e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:45.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b4bdd700 1 -- 172.21.2.104:0/3525128499 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a800b870 con 0x7f33b00b0060 2021-07-11T03:56:45.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b4bdd700 1 -- 172.21.2.104:0/3525128499 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a800b9d0 con 0x7f33b00b0060 2021-07-11T03:56:45.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b4bdd700 1 -- 172.21.2.104:0/3525128499 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a800bb40 con 0x7f33b00b0060 2021-07-11T03:56:45.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 -- 172.21.2.104:0/3525128499 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b00b0060 msgr2=0x7f33b005f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 --2- 172.21.2.104:0/3525128499 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b00b0060 0x7f33b005f6b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f33a80032f0 tx=0x7f33a8009290).stop 2021-07-11T03:56:45.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 -- 172.21.2.104:0/3525128499 shutdown_connections 2021-07-11T03:56:45.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 --2- 172.21.2.104:0/3525128499 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b00b0060 0x7f33b005f6b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:45.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 --2- 172.21.2.104:0/3525128499 >> v2:172.21.2.105:3300/0 conn(0x7f33b00b1970 0x7f33b0066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 --2- 172.21.2.104:0/3525128499 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b00a76b0 0x7f33b0066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 -- 172.21.2.104:0/3525128499 >> 172.21.2.104:0/3525128499 conn(0x7f33b00b53e0 msgr2=0x7f33b00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:45.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 -- 172.21.2.104:0/3525128499 shutdown_connections 2021-07-11T03:56:45.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.801+0000 7f33b7e43700 1 -- 172.21.2.104:0/3525128499 wait complete. 2021-07-11T03:56:45.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b7e43700 1 Processor -- start 2021-07-11T03:56:45.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b7e43700 1 -- start start 2021-07-11T03:56:45.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b7e43700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33b00a76b0 0x7f33b00abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b7e43700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b00b0060 0x7f33b00b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b7e43700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b00b1970 0x7f33b0144a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b7e43700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b00aa800 con 0x7f33b00b0060 2021-07-11T03:56:45.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b7e43700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b00af560 con 0x7f33b00a76b0 2021-07-11T03:56:45.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b7e43700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b00aaa30 con 0x7f33b00b1970 2021-07-11T03:56:45.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b53de700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b00b0060 0x7f33b00b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:45.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b53de700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b00b0060 0x7f33b00b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44538/0 (socket says 172.21.2.104:44538) 2021-07-11T03:56:45.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b53de700 1 -- 172.21.2.104:0/723347700 learned_addr learned my addr 172.21.2.104:0/723347700 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:45.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b53de700 1 -- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b00b1970 msgr2=0x7f33b0144a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b53de700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b00b1970 0x7f33b0144a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b53de700 1 -- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33b00a76b0 msgr2=0x7f33b00abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b53de700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33b00a76b0 0x7f33b00abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b53de700 1 -- 172.21.2.104:0/723347700 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a8009530 con 0x7f33b00b0060 2021-07-11T03:56:45.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b53de700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b00b0060 0x7f33b00b42b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f33a4002f70 tx=0x7f33a4006dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:45.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33a2ffd700 1 -- 172.21.2.104:0/723347700 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a400e470 con 0x7f33b00b0060 2021-07-11T03:56:45.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33a2ffd700 1 -- 172.21.2.104:0/723347700 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a400e5d0 con 0x7f33b00b0060 2021-07-11T03:56:45.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b7e43700 1 -- 172.21.2.104:0/723347700 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b00aacd0 con 0x7f33b00b0060 2021-07-11T03:56:45.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33a2ffd700 1 -- 172.21.2.104:0/723347700 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a400e930 con 0x7f33b00b0060 2021-07-11T03:56:45.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b7e43700 1 -- 172.21.2.104:0/723347700 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b014b720 con 0x7f33b00b0060 2021-07-11T03:56:45.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b7e43700 1 -- 172.21.2.104:0/723347700 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b00aae60 con 0x7f33b00b0060 2021-07-11T03:56:45.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33a2ffd700 1 -- 172.21.2.104:0/723347700 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f33a4018030 con 0x7f33b00b0060 2021-07-11T03:56:45.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33a2ffd700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f33900531f0 0x7f33900556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:45.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33a2ffd700 1 -- 172.21.2.104:0/723347700 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f33a406a6b0 con 0x7f33b00b0060 2021-07-11T03:56:45.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.805+0000 7f33b5bdf700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f33900531f0 0x7f33900556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:45.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.809+0000 7f33b5bdf700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f33900531f0 0x7f33900556b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f33ac002f70 tx=0x7f33ac006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:45.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.809+0000 7f33a2ffd700 1 -- 172.21.2.104:0/723347700 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f33a4041c70 con 0x7f33b00b0060 2021-07-11T03:56:45.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 -- 172.21.2.104:0/723347700 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33b0057350 con 0x7f33b00b0060 2021-07-11T03:56:45.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33a2ffd700 1 -- 172.21.2.104:0/723347700 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f33a4045290 con 0x7f33b00b0060 2021-07-11T03:56:45.923 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:45.924 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4492,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/2412138431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2412138431},{"type":"v1","addr":"172.21.2.105:6835","nonce":2412138431}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/3271417890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3271417890},{"type":"v1","addr":"172.21.2.104:6834","nonce":3271417890}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":14},{"gid":5505,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6835/2058921436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":2058921436},{"type":"v1","addr":"172.21.2.104:6835","nonce":2058921436}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":19}],"filesystems":[]} 2021-07-11T03:56:45.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 -- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f33900531f0 msgr2=0x7f33900556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f33900531f0 0x7f33900556b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f33ac002f70 tx=0x7f33ac006a80).stop 2021-07-11T03:56:45.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 -- 172.21.2.104:0/723347700 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b00b0060 msgr2=0x7f33b00b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:45.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b00b0060 0x7f33b00b42b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f33a4002f70 tx=0x7f33a4006dd0).stop 2021-07-11T03:56:45.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 -- 172.21.2.104:0/723347700 shutdown_connections 2021-07-11T03:56:45.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b00b1970 0x7f33b0144a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f33900531f0 0x7f33900556b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:45.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b00b0060 0x7f33b00b42b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:45.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 --2- 172.21.2.104:0/723347700 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33b00a76b0 0x7f33b00abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:45.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 -- 172.21.2.104:0/723347700 >> 172.21.2.104:0/723347700 conn(0x7f33b00b53e0 msgr2=0x7f33b00b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:45.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 -- 172.21.2.104:0/723347700 shutdown_connections 2021-07-11T03:56:45.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:45.917+0000 7f33b7e43700 1 -- 172.21.2.104:0/723347700 wait complete. 2021-07-11T03:56:45.927 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 19 2021-07-11T03:56:45.931 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T03:56:45.931 INFO:tasks.ceph.mds.a:Stopping old one... 2021-07-11T03:56:45.931 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T03:56:45.932 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:56:45.932 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T03:56:45.932 INFO:tasks.ceph.mds.c:Stopping old one... 2021-07-11T03:56:45.933 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T03:56:45.933 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:56:45.933 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T03:56:45.934 INFO:tasks.ceph.mds.b:Stopping old one... 2021-07-11T03:56:45.934 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T03:56:45.934 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:56:45.937 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T03:56:45.937 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T03:56:45.939 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T03:56:45.939 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T03:56:45.941 INFO:tasks.ceph.mds.a:Started 2021-07-11T03:56:45.941 DEBUG:teuthology.parallel:result is None 2021-07-11T03:56:45.942 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T03:56:45.943 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T03:56:45.943 INFO:tasks.ceph.mds.c:Started 2021-07-11T03:56:45.944 DEBUG:teuthology.parallel:result is None 2021-07-11T03:56:45.972 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T03:56:45.974 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T03:56:45.986 INFO:tasks.ceph.mds.b:Started 2021-07-11T03:56:45.986 DEBUG:teuthology.parallel:result is None 2021-07-11T03:56:45.987 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T03:56:46.022 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T03:56:46.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 Processor -- start 2021-07-11T03:56:46.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 -- start start 2021-07-11T03:56:46.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e840b7810 0x7f2e841445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:46.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e840b31d0 con 0x7f2e840b1170 2021-07-11T03:56:46.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e840b7bf0 con 0x7f2e84144a90 2021-07-11T03:56:46.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84144ef0 con 0x7f2e840b7810 2021-07-11T03:56:46.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e7bfff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f2e840b1170 0x7f2e84064870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41348/0 (socket says 172.21.2.104:41348) 2021-07-11T03:56:46.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e7bfff700 1 -- 172.21.2.104:0/2576617923 learned_addr learned my addr 172.21.2.104:0/2576617923 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:46.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341604540 0 0) 0x7f2e840b31d0 con 0x7f2e840b1170 2021-07-11T03:56:46.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e64003260 con 0x7f2e840b1170 2021-07-11T03:56:46.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e82ffd700 1 --2- 172.21.2.104:0/2576617923 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e840b7810 0x7f2e841445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:46.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1576031252 0 0) 0x7f2e64003260 con 0x7f2e840b1170 2021-07-11T03:56:46.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e840b7810 msgr2=0x7f2e841445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:46.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 --2- 172.21.2.104:0/2576617923 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e840b7810 0x7f2e841445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:46.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 >> v1:172.21.2.105:6789/0 conn(0x7f2e84144a90 legacy=0x7f2e84064f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:46.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e840675c0 con 0x7f2e840b1170 2021-07-11T03:56:46.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f2e6c001660 con 0x7f2e840b1170 2021-07-11T03:56:46.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 >> v1:172.21.2.104:6789/0 conn(0x7f2e840b1170 legacy=0x7f2e84064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:46.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 --2- 172.21.2.104:0/2576617923 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e64003780 0x7f2e64003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:46.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 --2- 172.21.2.104:0/2576617923 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64004500 0x7f2e64004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:46.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 --2- 172.21.2.104:0/2576617923 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e840b7810 0x7f2e64004f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:46.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84144ef0 con 0x7f2e64004500 2021-07-11T03:56:46.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6400b830 con 0x7f2e64003780 2021-07-11T03:56:46.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6400b9f0 con 0x7f2e840b7810 2021-07-11T03:56:46.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e7bfff700 1 --2- 172.21.2.104:0/2576617923 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64004500 0x7f2e64004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:46.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e7bfff700 1 -- 172.21.2.104:0/2576617923 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e840b7810 msgr2=0x7f2e64004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:46.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e7bfff700 1 --2- 172.21.2.104:0/2576617923 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e840b7810 0x7f2e64004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:46.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e7bfff700 1 -- 172.21.2.104:0/2576617923 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e64003780 msgr2=0x7f2e64003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:46.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e7bfff700 1 --2- 172.21.2.104:0/2576617923 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e64003780 0x7f2e64003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:46.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e7bfff700 1 -- 172.21.2.104:0/2576617923 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e64003260 con 0x7f2e64004500 2021-07-11T03:56:46.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e7bfff700 1 --2- 172.21.2.104:0/2576617923 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64004500 0x7f2e64004980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f2e84060260 tx=0x7f2e84066a60).ready entity=mon.0 client_cookie=c6063b4f6e2a2f18 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:46.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e6c007270 con 0x7f2e64004500 2021-07-11T03:56:46.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c0073d0 con 0x7f2e64004500 2021-07-11T03:56:46.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e827fc700 1 -- 172.21.2.104:0/2576617923 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e6c00cae0 con 0x7f2e64004500 2021-07-11T03:56:46.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 -- 172.21.2.104:0/2576617923 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64004500 msgr2=0x7f2e64004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:46.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 --2- 172.21.2.104:0/2576617923 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64004500 0x7f2e64004980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f2e84060260 tx=0x7f2e84066a60).stop 2021-07-11T03:56:46.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e837fe700 1 -- 172.21.2.104:0/2576617923 reap_dead start 2021-07-11T03:56:46.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 -- 172.21.2.104:0/2576617923 shutdown_connections 2021-07-11T03:56:46.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 -- 172.21.2.104:0/2576617923 >> 172.21.2.104:0/2576617923 conn(0x7f2e8405f6b0 msgr2=0x7f2e8405fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:46.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 -- 172.21.2.104:0/2576617923 shutdown_connections 2021-07-11T03:56:46.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.037+0000 7f2e89333700 1 -- 172.21.2.104:0/2576617923 wait complete. 2021-07-11T03:56:46.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e89333700 1 Processor -- start 2021-07-11T03:56:46.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e89333700 1 -- start start 2021-07-11T03:56:46.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e89333700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e84137f30 0x7f2e84138310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:46.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e89333700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e84061e20 0x7f2e84060530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:46.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e89333700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e84062560 0x7f2e84060a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:46.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e89333700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e840ab470 con 0x7f2e84061e20 2021-07-11T03:56:46.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e89333700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84135a70 con 0x7f2e84137f30 2021-07-11T03:56:46.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e89333700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e840ab6a0 con 0x7f2e84062560 2021-07-11T03:56:46.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e7bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e84061e20 0x7f2e84060530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:46.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e7bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e84061e20 0x7f2e84060530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44590/0 (socket says 172.21.2.104:44590) 2021-07-11T03:56:46.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e7bfff700 1 -- 172.21.2.104:0/3420243651 learned_addr learned my addr 172.21.2.104:0/3420243651 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:46.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e82ffd700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e84137f30 0x7f2e84138310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:46.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e7bfff700 1 -- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e84062560 msgr2=0x7f2e84060a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:46.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e7bfff700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e84062560 0x7f2e84060a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:46.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e7bfff700 1 -- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e84137f30 msgr2=0x7f2e84138310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:46.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e7bfff700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e84137f30 0x7f2e84138310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:46.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e7bfff700 1 -- 172.21.2.104:0/3420243651 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e840ab940 con 0x7f2e84061e20 2021-07-11T03:56:46.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e82ffd700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e84137f30 0x7f2e84138310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:56:46.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e7bfff700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e84061e20 0x7f2e84060530 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f2e84060260 tx=0x7f2e6c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:46.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e80ff9700 1 -- 172.21.2.104:0/3420243651 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e6c0049e0 con 0x7f2e84061e20 2021-07-11T03:56:46.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e80ff9700 1 -- 172.21.2.104:0/3420243651 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c004b40 con 0x7f2e84061e20 2021-07-11T03:56:46.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e89333700 1 -- 172.21.2.104:0/3420243651 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e840aecf0 con 0x7f2e84061e20 2021-07-11T03:56:46.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e80ff9700 1 -- 172.21.2.104:0/3420243651 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e6c004ea0 con 0x7f2e84061e20 2021-07-11T03:56:46.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e89333700 1 -- 172.21.2.104:0/3420243651 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e840abf70 con 0x7f2e84061e20 2021-07-11T03:56:46.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e89333700 1 -- 172.21.2.104:0/3420243651 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e84057350 con 0x7f2e84061e20 2021-07-11T03:56:46.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e80ff9700 1 -- 172.21.2.104:0/3420243651 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2e6c01d070 con 0x7f2e84061e20 2021-07-11T03:56:46.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e80ff9700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e68053210 0x7f2e680556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:46.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e80ff9700 1 -- 172.21.2.104:0/3420243651 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2e6c06be20 con 0x7f2e84061e20 2021-07-11T03:56:46.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.041+0000 7f2e82ffd700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e68053210 0x7f2e680556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:46.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.045+0000 7f2e82ffd700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e68053210 0x7f2e680556d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f2e74008000 tx=0x7f2e74008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:46.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.045+0000 7f2e80ff9700 1 -- 172.21.2.104:0/3420243651 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2e6c043440 con 0x7f2e84061e20 2021-07-11T03:56:46.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:46.137+0000 7f2e89333700 1 -- 172.21.2.104:0/3420243651 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f2e840698e0 con 0x7f2e84061e20 2021-07-11T03:56:47.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e80ff9700 1 -- 172.21.2.104:0/3420243651 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2e6c046a60 con 0x7f2e84061e20 2021-07-11T03:56:47.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 -- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e68053210 msgr2=0x7f2e680556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e68053210 0x7f2e680556d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f2e74008000 tx=0x7f2e74008040).stop 2021-07-11T03:56:47.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 -- 172.21.2.104:0/3420243651 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e84061e20 msgr2=0x7f2e84060530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e84061e20 0x7f2e84060530 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f2e84060260 tx=0x7f2e6c015000).stop 2021-07-11T03:56:47.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 -- 172.21.2.104:0/3420243651 shutdown_connections 2021-07-11T03:56:47.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e84062560 0x7f2e84060a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e68053210 0x7f2e680556d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e84061e20 0x7f2e84060530 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 --2- 172.21.2.104:0/3420243651 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e84137f30 0x7f2e84138310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 -- 172.21.2.104:0/3420243651 >> 172.21.2.104:0/3420243651 conn(0x7f2e8405f6b0 msgr2=0x7f2e8405cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 -- 172.21.2.104:0/3420243651 shutdown_connections 2021-07-11T03:56:47.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.061+0000 7f2e89333700 1 -- 172.21.2.104:0/3420243651 wait complete. 2021-07-11T03:56:47.071 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T03:56:47.075 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T03:56:47.076 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T03:56:47.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 Processor -- start 2021-07-11T03:56:47.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 -- start start 2021-07-11T03:56:47.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a6c0b1db0 0x7f5a6c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6c0b0350 con 0x7f5a6c0a70c0 2021-07-11T03:56:47.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6c0b05f0 con 0x7f5a6c1483a0 2021-07-11T03:56:47.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0a90c0 con 0x7f5a6c0b1db0 2021-07-11T03:56:47.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a71697700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f5a6c0a70c0 0x7f5a6c144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41372/0 (socket says 172.21.2.104:41372) 2021-07-11T03:56:47.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a71697700 1 -- 172.21.2.104:0/870478475 learned_addr learned my addr 172.21.2.104:0/870478475 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:47.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862546701 0 0) 0x7f5a6c0b0350 con 0x7f5a6c0a70c0 2021-07-11T03:56:47.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a58003260 con 0x7f5a6c0a70c0 2021-07-11T03:56:47.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a70e96700 1 --2- 172.21.2.104:0/870478475 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a6c0b1db0 0x7f5a6c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2231391419 0 0) 0x7f5a58003260 con 0x7f5a6c0a70c0 2021-07-11T03:56:47.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a6c0b1db0 msgr2=0x7f5a6c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:47.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 --2- 172.21.2.104:0/870478475 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a6c0b1db0 0x7f5a6c0b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 >> v1:172.21.2.105:6789/0 conn(0x7f5a6c1483a0 legacy=0x7f5a6c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:47.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c0b0890 con 0x7f5a6c0a70c0 2021-07-11T03:56:47.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f5a640019e0 con 0x7f5a6c0a70c0 2021-07-11T03:56:47.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 >> v1:172.21.2.104:6789/0 conn(0x7f5a6c0a70c0 legacy=0x7f5a6c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 --2- 172.21.2.104:0/870478475 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a6c0b1db0 0x7f5a58003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 --2- 172.21.2.104:0/870478475 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5a58003ef0 0x7f5a58004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 --2- 172.21.2.104:0/870478475 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a58004bf0 0x7f5a5800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a580052c0 con 0x7f5a58004bf0 2021-07-11T03:56:47.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5800bc40 con 0x7f5a58003ef0 2021-07-11T03:56:47.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5800bd80 con 0x7f5a6c0b1db0 2021-07-11T03:56:47.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a71697700 1 --2- 172.21.2.104:0/870478475 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a58004bf0 0x7f5a5800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a71697700 1 -- 172.21.2.104:0/870478475 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a6c0b1db0 msgr2=0x7f5a58003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:47.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a71697700 1 --2- 172.21.2.104:0/870478475 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a6c0b1db0 0x7f5a58003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a71697700 1 -- 172.21.2.104:0/870478475 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5a58003ef0 msgr2=0x7f5a58004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:47.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a71697700 1 --2- 172.21.2.104:0/870478475 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5a58003ef0 0x7f5a58004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a71697700 1 -- 172.21.2.104:0/870478475 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a58003260 con 0x7f5a58004bf0 2021-07-11T03:56:47.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a71697700 1 --2- 172.21.2.104:0/870478475 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a58004bf0 0x7f5a5800b750 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f5a6c061440 tx=0x7f5a6c0b0780).ready entity=mon.0 client_cookie=980e4c9ec559ea7a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a6400cb80 con 0x7f5a58004bf0 2021-07-11T03:56:47.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6400cce0 con 0x7f5a58004bf0 2021-07-11T03:56:47.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a6b7fe700 1 -- 172.21.2.104:0/870478475 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a64005b80 con 0x7f5a58004bf0 2021-07-11T03:56:47.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 -- 172.21.2.104:0/870478475 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a58004bf0 msgr2=0x7f5a5800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 --2- 172.21.2.104:0/870478475 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a58004bf0 0x7f5a5800b750 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f5a6c061440 tx=0x7f5a6c0b0780).stop 2021-07-11T03:56:47.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a71697700 1 -- 172.21.2.104:0/870478475 reap_dead start 2021-07-11T03:56:47.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 -- 172.21.2.104:0/870478475 shutdown_connections 2021-07-11T03:56:47.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 -- 172.21.2.104:0/870478475 >> 172.21.2.104:0/870478475 conn(0x7f5a6c05f6b0 msgr2=0x7f5a6c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 -- 172.21.2.104:0/870478475 shutdown_connections 2021-07-11T03:56:47.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 -- 172.21.2.104:0/870478475 wait complete. 2021-07-11T03:56:47.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 Processor -- start 2021-07-11T03:56:47.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.129+0000 7f5a730fa700 1 -- start start 2021-07-11T03:56:47.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a730fa700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5a6c0b4d20 0x7f5a6c1564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a730fa700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a6c0aa7f0 0x7f5a6c156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a730fa700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a6c143710 0x7f5a6c143b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a730fa700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0b5100 con 0x7f5a6c143710 2021-07-11T03:56:47.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a730fa700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c156e80 con 0x7f5a6c0b4d20 2021-07-11T03:56:47.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a730fa700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c159b80 con 0x7f5a6c0aa7f0 2021-07-11T03:56:47.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a71697700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a6c143710 0x7f5a6c143b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a71697700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a6c143710 0x7f5a6c143b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44616/0 (socket says 172.21.2.104:44616) 2021-07-11T03:56:47.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a71697700 1 -- 172.21.2.104:0/1644474853 learned_addr learned my addr 172.21.2.104:0/1644474853 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:47.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a71697700 1 -- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a6c0aa7f0 msgr2=0x7f5a6c156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a71697700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a6c0aa7f0 0x7f5a6c156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a71697700 1 -- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5a6c0b4d20 msgr2=0x7f5a6c1564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a71697700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5a6c0b4d20 0x7f5a6c1564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a71697700 1 -- 172.21.2.104:0/1644474853 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c159e20 con 0x7f5a6c143710 2021-07-11T03:56:47.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a71697700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a6c143710 0x7f5a6c143b90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f5a6c0b3170 tx=0x7f5a64013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a69ffb700 1 -- 172.21.2.104:0/1644474853 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a6401b070 con 0x7f5a6c143710 2021-07-11T03:56:47.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a69ffb700 1 -- 172.21.2.104:0/1644474853 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a64004e30 con 0x7f5a6c143710 2021-07-11T03:56:47.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a730fa700 1 -- 172.21.2.104:0/1644474853 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6c15a0c0 con 0x7f5a6c143710 2021-07-11T03:56:47.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a69ffb700 1 -- 172.21.2.104:0/1644474853 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a640072c0 con 0x7f5a6c143710 2021-07-11T03:56:47.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a730fa700 1 -- 172.21.2.104:0/1644474853 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6c064690 con 0x7f5a6c143710 2021-07-11T03:56:47.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a730fa700 1 -- 172.21.2.104:0/1644474853 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6c15a9a0 con 0x7f5a6c143710 2021-07-11T03:56:47.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a69ffb700 1 -- 172.21.2.104:0/1644474853 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5a64010070 con 0x7f5a6c143710 2021-07-11T03:56:47.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a69ffb700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a4c053160 0x7f5a4c055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a69ffb700 1 -- 172.21.2.104:0/1644474853 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5a6406b2d0 con 0x7f5a6c143710 2021-07-11T03:56:47.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a70e96700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a4c053160 0x7f5a4c055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.133+0000 7f5a70e96700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a4c053160 0x7f5a4c055620 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f5a60007000 tx=0x7f5a60008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.137+0000 7f5a69ffb700 1 -- 172.21.2.104:0/1644474853 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5a640428e0 con 0x7f5a6c143710 2021-07-11T03:56:47.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.245+0000 7f5a730fa700 1 -- 172.21.2.104:0/1644474853 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5a6c057350 con 0x7f5a6c143710 2021-07-11T03:56:47.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.245+0000 7f5a69ffb700 1 -- 172.21.2.104:0/1644474853 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f5a64045f00 con 0x7f5a6c143710 2021-07-11T03:56:47.254 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:47.255 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:56:47.255 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T03:56:47.255 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.255 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T03:56:47.256 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T03:56:47.256 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.256 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:56:47.256 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.256 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.257 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.257 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.257 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.257 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.258 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T03:56:47.258 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T03:56:47.258 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.258 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:56:47.258 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.259 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.259 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.259 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.259 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.260 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.260 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T03:56:47.260 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T03:56:47.260 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.260 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:56:47.261 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.261 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.261 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.261 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.262 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.262 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.262 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T03:56:47.262 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T03:56:47.262 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.263 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.263 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.263 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.263 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.263 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.264 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.264 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T03:56:47.264 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T03:56:47.264 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.265 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.265 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.265 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.265 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.265 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.266 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.266 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T03:56:47.266 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T03:56:47.267 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.267 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.268 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.268 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.268 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.268 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.268 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.269 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T03:56:47.269 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T03:56:47.269 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.269 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.270 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.270 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.270 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.270 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.270 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.271 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T03:56:47.271 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T03:56:47.271 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.271 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.271 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.272 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.272 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.272 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.272 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.273 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T03:56:47.273 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T03:56:47.273 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.273 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.273 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.274 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.274 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.274 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.274 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.275 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T03:56:47.275 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T03:56:47.275 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.275 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.275 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.276 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.276 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.276 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.276 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.277 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T03:56:47.277 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T03:56:47.277 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.277 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.277 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.278 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.278 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.278 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.278 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.278 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T03:56:47.279 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T03:56:47.279 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.279 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:56:47.279 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T03:56:47.280 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T03:56:47.280 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.280 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.280 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.281 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T03:56:47.281 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T03:56:47.281 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.281 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:56:47.281 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T03:56:47.282 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T03:56:47.282 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.282 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.282 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.283 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T03:56:47.283 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T03:56:47.283 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.283 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:56:47.284 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:56:47.284 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:56:47.284 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.284 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.285 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.285 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.285 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T03:56:47.285 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T03:56:47.286 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.286 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:56:47.286 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T03:56:47.286 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.287 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.287 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.287 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.287 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T03:56:47.288 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T03:56:47.288 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:56:47.288 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:56:47.288 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T03:56:47.288 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:56:47.289 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.289 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.289 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:56:47.289 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:56:47.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 -- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a4c053160 msgr2=0x7f5a4c055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a4c053160 0x7f5a4c055620 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f5a60007000 tx=0x7f5a60008040).stop 2021-07-11T03:56:47.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 -- 172.21.2.104:0/1644474853 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a6c143710 msgr2=0x7f5a6c143b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a6c143710 0x7f5a6c143b90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f5a6c0b3170 tx=0x7f5a64013000).stop 2021-07-11T03:56:47.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 -- 172.21.2.104:0/1644474853 shutdown_connections 2021-07-11T03:56:47.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a4c053160 0x7f5a4c055620 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a6c143710 0x7f5a6c143b90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a6c0aa7f0 0x7f5a6c156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 --2- 172.21.2.104:0/1644474853 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5a6c0b4d20 0x7f5a6c1564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 -- 172.21.2.104:0/1644474853 >> 172.21.2.104:0/1644474853 conn(0x7f5a6c05f6b0 msgr2=0x7f5a6c0b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 -- 172.21.2.104:0/1644474853 shutdown_connections 2021-07-11T03:56:47.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.249+0000 7f5a730fa700 1 -- 172.21.2.104:0/1644474853 wait complete. 2021-07-11T03:56:47.294 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:56:47.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 Processor -- start 2021-07-11T03:56:47.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 -- start start 2021-07-11T03:56:47.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff78060100 0x7eff780b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7eff78138cc0 0x7eff78139120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7eff78142840 0x7eff78142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff780604e0 con 0x7eff78142840 2021-07-11T03:56:47.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff78139610 con 0x7eff78138cc0 2021-07-11T03:56:47.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff780b16b0 con 0x7eff78060100 2021-07-11T03:56:47.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7e28b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7eff78142840 0x7eff78142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7e28b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7eff78142840 0x7eff78142cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44620/0 (socket says 172.21.2.104:44620) 2021-07-11T03:56:47.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7e28b700 1 -- 172.21.2.104:0/2712996893 learned_addr learned my addr 172.21.2.104:0/2712996893 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:47.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7e28b700 1 -- 172.21.2.104:0/2712996893 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff78060100 msgr2=0x7eff780b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:47.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7e28b700 1 --2- 172.21.2.104:0/2712996893 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff78060100 0x7eff780b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7e28b700 1 -- 172.21.2.104:0/2712996893 >> v2:172.21.2.105:3300/0 conn(0x7eff78138cc0 msgr2=0x7eff78139120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:47.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7e28b700 1 --2- 172.21.2.104:0/2712996893 >> v2:172.21.2.105:3300/0 conn(0x7eff78138cc0 0x7eff78139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7e28b700 1 -- 172.21.2.104:0/2712996893 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff780ac400 con 0x7eff78142840 2021-07-11T03:56:47.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7e28b700 1 --2- 172.21.2.104:0/2712996893 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff78142840 0x7eff78142cd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7eff700032f0 tx=0x7eff70008310).ready entity=mon.0 client_cookie=d4cbc18246ef21fd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7ca88700 1 -- 172.21.2.104:0/2712996893 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff70008970 con 0x7eff78142840 2021-07-11T03:56:47.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7ca88700 1 -- 172.21.2.104:0/2712996893 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff70008ad0 con 0x7eff78142840 2021-07-11T03:56:47.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7ca88700 1 -- 172.21.2.104:0/2712996893 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff70008e30 con 0x7eff78142840 2021-07-11T03:56:47.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 -- 172.21.2.104:0/2712996893 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff78142840 msgr2=0x7eff78142cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 --2- 172.21.2.104:0/2712996893 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff78142840 0x7eff78142cd0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7eff700032f0 tx=0x7eff70008310).stop 2021-07-11T03:56:47.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 -- 172.21.2.104:0/2712996893 shutdown_connections 2021-07-11T03:56:47.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 --2- 172.21.2.104:0/2712996893 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff78142840 0x7eff78142cd0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 --2- 172.21.2.104:0/2712996893 >> v2:172.21.2.105:3300/0 conn(0x7eff78138cc0 0x7eff78139120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 --2- 172.21.2.104:0/2712996893 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff78060100 0x7eff780b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 -- 172.21.2.104:0/2712996893 >> 172.21.2.104:0/2712996893 conn(0x7eff78061070 msgr2=0x7eff7805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 -- 172.21.2.104:0/2712996893 shutdown_connections 2021-07-11T03:56:47.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 -- 172.21.2.104:0/2712996893 wait complete. 2021-07-11T03:56:47.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 Processor -- start 2021-07-11T03:56:47.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.345+0000 7eff7fcee700 1 -- start start 2021-07-11T03:56:47.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7fcee700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff78060100 0x7eff780a7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7fcee700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff78138cc0 0x7eff780632e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7fcee700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff78142840 0x7eff78159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7fcee700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff780af230 con 0x7eff78142840 2021-07-11T03:56:47.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7fcee700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff780af460 con 0x7eff78138cc0 2021-07-11T03:56:47.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7fcee700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff780af9c0 con 0x7eff78060100 2021-07-11T03:56:47.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7e28b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff78142840 0x7eff78159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7e28b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff78142840 0x7eff78159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44630/0 (socket says 172.21.2.104:44630) 2021-07-11T03:56:47.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7e28b700 1 -- 172.21.2.104:0/4017758466 learned_addr learned my addr 172.21.2.104:0/4017758466 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:47.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7da8a700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff78060100 0x7eff780a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7d289700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff78138cc0 0x7eff780632e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7e28b700 1 -- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff78060100 msgr2=0x7eff780a7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7e28b700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff78060100 0x7eff780a7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7e28b700 1 -- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff78138cc0 msgr2=0x7eff780632e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7e28b700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff78138cc0 0x7eff780632e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7e28b700 1 -- 172.21.2.104:0/4017758466 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff700085b0 con 0x7eff78142840 2021-07-11T03:56:47.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7e28b700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff78142840 0x7eff78159f10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7eff700032f0 tx=0x7eff70018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff6affd700 1 -- 172.21.2.104:0/4017758466 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff7001a610 con 0x7eff78142840 2021-07-11T03:56:47.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff6affd700 1 -- 172.21.2.104:0/4017758466 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff70011030 con 0x7eff78142840 2021-07-11T03:56:47.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff6affd700 1 -- 172.21.2.104:0/4017758466 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff7001a960 con 0x7eff78142840 2021-07-11T03:56:47.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7fcee700 1 -- 172.21.2.104:0/4017758466 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff780afc60 con 0x7eff78142840 2021-07-11T03:56:47.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7fcee700 1 -- 172.21.2.104:0/4017758466 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff78065ee0 con 0x7eff78142840 2021-07-11T03:56:47.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7fcee700 1 -- 172.21.2.104:0/4017758466 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff78057350 con 0x7eff78142840 2021-07-11T03:56:47.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff6affd700 1 -- 172.21.2.104:0/4017758466 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7eff7002a070 con 0x7eff78142840 2021-07-11T03:56:47.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff6affd700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff580532d0 0x7eff58055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff6affd700 1 -- 172.21.2.104:0/4017758466 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7eff7006c4b0 con 0x7eff78142840 2021-07-11T03:56:47.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.349+0000 7eff7da8a700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff580532d0 0x7eff58055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.353+0000 7eff7da8a700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff580532d0 0x7eff58055790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7eff6c00a000 tx=0x7eff6c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.353+0000 7eff6affd700 1 -- 172.21.2.104:0/4017758466 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7eff70043a10 con 0x7eff78142840 2021-07-11T03:56:47.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.433+0000 7eff7fcee700 1 -- 172.21.2.104:0/4017758466 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff780af230 con 0x7eff78142840 2021-07-11T03:56:47.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.433+0000 7eff6affd700 1 -- 172.21.2.104:0/4017758466 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13181 (secure 0 0 0) 0x7eff7001e070 con 0x7eff78142840 2021-07-11T03:56:47.441 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:47.442 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":51,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:56:47.054176+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:56:47.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 -- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff580532d0 msgr2=0x7eff58055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff580532d0 0x7eff58055790 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7eff6c00a000 tx=0x7eff6c008040).stop 2021-07-11T03:56:47.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 -- 172.21.2.104:0/4017758466 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff78142840 msgr2=0x7eff78159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff78142840 0x7eff78159f10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7eff700032f0 tx=0x7eff70018000).stop 2021-07-11T03:56:47.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 -- 172.21.2.104:0/4017758466 shutdown_connections 2021-07-11T03:56:47.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff580532d0 0x7eff58055790 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff78142840 0x7eff78159f10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff78138cc0 0x7eff780632e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 --2- 172.21.2.104:0/4017758466 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff78060100 0x7eff780a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 -- 172.21.2.104:0/4017758466 >> 172.21.2.104:0/4017758466 conn(0x7eff78061070 msgr2=0x7eff780aa870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 -- 172.21.2.104:0/4017758466 shutdown_connections 2021-07-11T03:56:47.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.437+0000 7eff7fcee700 1 -- 172.21.2.104:0/4017758466 wait complete. 2021-07-11T03:56:47.449 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T03:56:47.450 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:56:47.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 Processor -- start 2021-07-11T03:56:47.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 -- start start 2021-07-11T03:56:47.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fad640a9ec0 0x7fad640aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fad640603d0 0x7fad6413ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad64061990 con 0x7fad6413d1c0 2021-07-11T03:56:47.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad64061c30 con 0x7fad640603d0 2021-07-11T03:56:47.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad64060b90 con 0x7fad640a9ec0 2021-07-11T03:56:47.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad63fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fad6413d1c0 0x7fad6413f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41404/0 (socket says 172.21.2.104:41404) 2021-07-11T03:56:47.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad63fff700 1 -- 172.21.2.104:0/3463935385 learned_addr learned my addr 172.21.2.104:0/3463935385 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:47.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad637fe700 1 --2- 172.21.2.104:0/3463935385 >> v2:172.21.2.105:3300/0 conn(0x7fad640a9ec0 0x7fad640aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad62ffd700 1 --2- 172.21.2.104:0/3463935385 >> v2:172.21.2.104:3300/0 conn(0x7fad640603d0 0x7fad6413ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad627fc700 1 -- 172.21.2.104:0/3463935385 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950121927 0 0) 0x7fad64061990 con 0x7fad6413d1c0 2021-07-11T03:56:47.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad62ffd700 1 -- 172.21.2.104:0/3463935385 >> v2:172.21.2.105:3300/0 conn(0x7fad640a9ec0 msgr2=0x7fad640aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:47.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad62ffd700 1 --2- 172.21.2.104:0/3463935385 >> v2:172.21.2.105:3300/0 conn(0x7fad640a9ec0 0x7fad640aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad62ffd700 1 -- 172.21.2.104:0/3463935385 >> v1:172.21.2.104:6789/0 conn(0x7fad6413d1c0 legacy=0x7fad6413f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad62ffd700 1 -- 172.21.2.104:0/3463935385 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad64060e30 con 0x7fad640603d0 2021-07-11T03:56:47.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad62ffd700 1 --2- 172.21.2.104:0/3463935385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad640603d0 0x7fad6413ccd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fad640aaaf0 tx=0x7fad54006df0).ready entity=mon.0 client_cookie=f4eeb51cf3b1fec2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad627fc700 1 -- 172.21.2.104:0/3463935385 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad5400d5c0 con 0x7fad640603d0 2021-07-11T03:56:47.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad627fc700 1 -- 172.21.2.104:0/3463935385 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad54000e90 con 0x7fad640603d0 2021-07-11T03:56:47.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad627fc700 1 -- 172.21.2.104:0/3463935385 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad5400d910 con 0x7fad640603d0 2021-07-11T03:56:47.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 -- 172.21.2.104:0/3463935385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad640603d0 msgr2=0x7fad6413ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 --2- 172.21.2.104:0/3463935385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad640603d0 0x7fad6413ccd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7fad640aaaf0 tx=0x7fad54006df0).stop 2021-07-11T03:56:47.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 -- 172.21.2.104:0/3463935385 shutdown_connections 2021-07-11T03:56:47.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 --2- 172.21.2.104:0/3463935385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad640603d0 0x7fad6413ccd0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 --2- 172.21.2.104:0/3463935385 >> v2:172.21.2.105:3300/0 conn(0x7fad640a9ec0 0x7fad640aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 -- 172.21.2.104:0/3463935385 >> 172.21.2.104:0/3463935385 conn(0x7fad640b2970 msgr2=0x7fad640b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 -- 172.21.2.104:0/3463935385 shutdown_connections 2021-07-11T03:56:47.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.497+0000 7fad6a217700 1 -- 172.21.2.104:0/3463935385 wait complete. 2021-07-11T03:56:47.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad6a217700 1 Processor -- start 2021-07-11T03:56:47.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad6a217700 1 -- start start 2021-07-11T03:56:47.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad6a217700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad640603d0 0x7fad6415a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad6a217700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad640a9ec0 0x7fad640ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad6a217700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6413d1c0 0x7fad641422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad6a217700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad640afd70 con 0x7fad6413d1c0 2021-07-11T03:56:47.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad6a217700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad64159f90 con 0x7fad640603d0 2021-07-11T03:56:47.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad6a217700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6415a230 con 0x7fad640a9ec0 2021-07-11T03:56:47.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad63fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6413d1c0 0x7fad641422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad63fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6413d1c0 0x7fad641422a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44642/0 (socket says 172.21.2.104:44642) 2021-07-11T03:56:47.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad63fff700 1 -- 172.21.2.104:0/288816852 learned_addr learned my addr 172.21.2.104:0/288816852 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:47.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad637fe700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad640603d0 0x7fad6415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad62ffd700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad640a9ec0 0x7fad640ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad63fff700 1 -- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad640a9ec0 msgr2=0x7fad640ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad63fff700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad640a9ec0 0x7fad640ad480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad63fff700 1 -- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad640603d0 msgr2=0x7fad6415a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad63fff700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad640603d0 0x7fad6415a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad63fff700 1 -- 172.21.2.104:0/288816852 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5400b180 con 0x7fad6413d1c0 2021-07-11T03:56:47.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad62ffd700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad640a9ec0 0x7fad640ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:56:47.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad63fff700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6413d1c0 0x7fad641422a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fad5c000f40 tx=0x7fad5c001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad60ff9700 1 -- 172.21.2.104:0/288816852 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad5c005ba0 con 0x7fad6413d1c0 2021-07-11T03:56:47.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad60ff9700 1 -- 172.21.2.104:0/288816852 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5c005d00 con 0x7fad6413d1c0 2021-07-11T03:56:47.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad6a217700 1 -- 172.21.2.104:0/288816852 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6415a4d0 con 0x7fad6413d1c0 2021-07-11T03:56:47.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad6a217700 1 -- 172.21.2.104:0/288816852 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad64142790 con 0x7fad6413d1c0 2021-07-11T03:56:47.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad60ff9700 1 -- 172.21.2.104:0/288816852 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad5c0072c0 con 0x7fad6413d1c0 2021-07-11T03:56:47.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad6a217700 1 -- 172.21.2.104:0/288816852 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6415a660 con 0x7fad6413d1c0 2021-07-11T03:56:47.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad60ff9700 1 -- 172.21.2.104:0/288816852 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fad5c01a060 con 0x7fad6413d1c0 2021-07-11T03:56:47.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad60ff9700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad480531a0 0x7fad48055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad60ff9700 1 -- 172.21.2.104:0/288816852 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fad5c06ac80 con 0x7fad6413d1c0 2021-07-11T03:56:47.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad637fe700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad480531a0 0x7fad48055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.501+0000 7fad637fe700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad480531a0 0x7fad48055660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fad4c00b000 tx=0x7fad4c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.505+0000 7fad60ff9700 1 -- 172.21.2.104:0/288816852 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fad5c070020 con 0x7fad6413d1c0 2021-07-11T03:56:47.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.585+0000 7fad6a217700 1 -- 172.21.2.104:0/288816852 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fad64057350 con 0x7fad6413d1c0 2021-07-11T03:56:47.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.585+0000 7fad60ff9700 1 -- 172.21.2.104:0/288816852 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v51) v1 ==== 110+0+10 (secure 0 0 0) 0x7fad5c0180b0 con 0x7fad6413d1c0 2021-07-11T03:56:47.594 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:56:47.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 -- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad480531a0 msgr2=0x7fad48055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad480531a0 0x7fad48055660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7fad4c00b000 tx=0x7fad4c006df0).stop 2021-07-11T03:56:47.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 -- 172.21.2.104:0/288816852 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6413d1c0 msgr2=0x7fad641422a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6413d1c0 0x7fad641422a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fad5c000f40 tx=0x7fad5c001360).stop 2021-07-11T03:56:47.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 -- 172.21.2.104:0/288816852 shutdown_connections 2021-07-11T03:56:47.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad480531a0 0x7fad48055660 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6413d1c0 0x7fad641422a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad640a9ec0 0x7fad640ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 --2- 172.21.2.104:0/288816852 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad640603d0 0x7fad6415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 -- 172.21.2.104:0/288816852 >> 172.21.2.104:0/288816852 conn(0x7fad640b2970 msgr2=0x7fad64143300 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 -- 172.21.2.104:0/288816852 shutdown_connections 2021-07-11T03:56:47.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.589+0000 7fad6a217700 1 -- 172.21.2.104:0/288816852 wait complete. 2021-07-11T03:56:47.601 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:56:47.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 Processor -- start 2021-07-11T03:56:47.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- start start 2021-07-11T03:56:47.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f12440b1ec0 0x7f12440b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f12440affb0 0x7f12440b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12440b0420 con 0x7f12440a70c0 2021-07-11T03:56:47.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12440b2790 con 0x7f12440b1ec0 2021-07-11T03:56:47.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12440b0650 con 0x7f12440affb0 2021-07-11T03:56:47.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f1248e32700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f12440a70c0 0x7f1244141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41418/0 (socket says 172.21.2.104:41418) 2021-07-11T03:56:47.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f1248e32700 1 -- 172.21.2.104:0/3894720068 learned_addr learned my addr 172.21.2.104:0/3894720068 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:47.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f1243fff700 1 --2- 172.21.2.104:0/3894720068 >> v2:172.21.2.104:3300/0 conn(0x7f12440b1ec0 0x7f12440b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f1243fff700 1 -- 172.21.2.104:0/3894720068 >> v2:172.21.2.105:3300/0 conn(0x7f12440affb0 msgr2=0x7f12440b22a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:56:47.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f1243fff700 1 --2- 172.21.2.104:0/3894720068 >> v2:172.21.2.105:3300/0 conn(0x7f12440affb0 0x7f12440b22a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f1243fff700 1 -- 172.21.2.104:0/3894720068 >> v1:172.21.2.104:6789/0 conn(0x7f12440a70c0 legacy=0x7f1244141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f1243fff700 1 -- 172.21.2.104:0/3894720068 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12440b08f0 con 0x7f12440b1ec0 2021-07-11T03:56:47.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f1243fff700 1 --2- 172.21.2.104:0/3894720068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12440b1ec0 0x7f12440b12d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f122c002f70 tx=0x7f122c006df0).ready entity=mon.0 client_cookie=cac03e72fb8a46b4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f1242ffd700 1 -- 172.21.2.104:0/3894720068 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f122c00d5c0 con 0x7f12440b1ec0 2021-07-11T03:56:47.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f1242ffd700 1 -- 172.21.2.104:0/3894720068 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122c000e90 con 0x7f12440b1ec0 2021-07-11T03:56:47.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f1242ffd700 1 -- 172.21.2.104:0/3894720068 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f122c00d950 con 0x7f12440b1ec0 2021-07-11T03:56:47.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- 172.21.2.104:0/3894720068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12440b1ec0 msgr2=0x7f12440b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 --2- 172.21.2.104:0/3894720068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12440b1ec0 0x7f12440b12d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f122c002f70 tx=0x7f122c006df0).stop 2021-07-11T03:56:47.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- 172.21.2.104:0/3894720068 shutdown_connections 2021-07-11T03:56:47.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 --2- 172.21.2.104:0/3894720068 >> v2:172.21.2.105:3300/0 conn(0x7f12440affb0 0x7f12440b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 --2- 172.21.2.104:0/3894720068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12440b1ec0 0x7f12440b12d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- 172.21.2.104:0/3894720068 >> 172.21.2.104:0/3894720068 conn(0x7f124405f6b0 msgr2=0x7f124405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- 172.21.2.104:0/3894720068 shutdown_connections 2021-07-11T03:56:47.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- 172.21.2.104:0/3894720068 wait complete. 2021-07-11T03:56:47.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 Processor -- start 2021-07-11T03:56:47.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- start start 2021-07-11T03:56:47.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12440a70c0 0x7f1244147000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12440affb0 0x7f124415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12440b1ec0 0x7f1244156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1244143a90 con 0x7f12440affb0 2021-07-11T03:56:47.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124415a180 con 0x7f12440b1ec0 2021-07-11T03:56:47.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f124a895700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f124415a420 con 0x7f12440a70c0 2021-07-11T03:56:47.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.649+0000 7f12437fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12440affb0 0x7f124415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12437fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12440affb0 0x7f124415ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44658/0 (socket says 172.21.2.104:44658) 2021-07-11T03:56:47.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12437fe700 1 -- 172.21.2.104:0/561902501 learned_addr learned my addr 172.21.2.104:0/561902501 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:47.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12437fe700 1 -- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12440a70c0 msgr2=0x7f1244147000 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:56:47.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12437fe700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12440a70c0 0x7f1244147000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12437fe700 1 -- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12440b1ec0 msgr2=0x7f1244156b10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:56:47.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12437fe700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12440b1ec0 0x7f1244156b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12437fe700 1 -- 172.21.2.104:0/561902501 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c00b180 con 0x7f12440affb0 2021-07-11T03:56:47.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12437fe700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12440affb0 0x7f124415ab60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f1234002fb0 tx=0x7f1234006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12417fa700 1 -- 172.21.2.104:0/561902501 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1234013460 con 0x7f12440affb0 2021-07-11T03:56:47.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12417fa700 1 -- 172.21.2.104:0/561902501 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12340135c0 con 0x7f12440affb0 2021-07-11T03:56:47.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12417fa700 1 -- 172.21.2.104:0/561902501 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1234013920 con 0x7f12440affb0 2021-07-11T03:56:47.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f124a895700 1 -- 172.21.2.104:0/561902501 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124415a6c0 con 0x7f12440affb0 2021-07-11T03:56:47.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f124a895700 1 -- 172.21.2.104:0/561902501 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12441571a0 con 0x7f12440affb0 2021-07-11T03:56:47.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f124a895700 1 -- 172.21.2.104:0/561902501 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1244057350 con 0x7f12440affb0 2021-07-11T03:56:47.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12417fa700 1 -- 172.21.2.104:0/561902501 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f123400f070 con 0x7f12440affb0 2021-07-11T03:56:47.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12417fa700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12280532d0 0x7f1228055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f12417fa700 1 -- 172.21.2.104:0/561902501 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f123406ac30 con 0x7f12440affb0 2021-07-11T03:56:47.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f1243fff700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12280532d0 0x7f1228055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.653+0000 7f1243fff700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12280532d0 0x7f1228055790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f122c002f70 tx=0x7f122c014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.657+0000 7f12417fa700 1 -- 172.21.2.104:0/561902501 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1234071080 con 0x7f12440affb0 2021-07-11T03:56:47.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 -- 172.21.2.104:0/561902501 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f12440ac760 con 0x7f12440affb0 2021-07-11T03:56:47.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f12417fa700 1 -- 172.21.2.104:0/561902501 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v51) v1 ==== 92+0+10 (secure 0 0 0) 0x7f12340422a0 con 0x7f12440affb0 2021-07-11T03:56:47.744 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:56:47.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 -- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12280532d0 msgr2=0x7f1228055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12280532d0 0x7f1228055790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f122c002f70 tx=0x7f122c014000).stop 2021-07-11T03:56:47.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 -- 172.21.2.104:0/561902501 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12440affb0 msgr2=0x7f124415ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12440affb0 0x7f124415ab60 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f1234002fb0 tx=0x7f1234006df0).stop 2021-07-11T03:56:47.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 -- 172.21.2.104:0/561902501 shutdown_connections 2021-07-11T03:56:47.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12440b1ec0 0x7f1244156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12280532d0 0x7f1228055790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12440affb0 0x7f124415ab60 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 --2- 172.21.2.104:0/561902501 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12440a70c0 0x7f1244147000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 -- 172.21.2.104:0/561902501 >> 172.21.2.104:0/561902501 conn(0x7f124405f6b0 msgr2=0x7f1244147f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 -- 172.21.2.104:0/561902501 shutdown_connections 2021-07-11T03:56:47.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.737+0000 7f124a895700 1 -- 172.21.2.104:0/561902501 wait complete. 2021-07-11T03:56:47.752 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T03:56:47.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 Processor -- start 2021-07-11T03:56:47.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 -- start start 2021-07-11T03:56:47.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97400b1db0 0x7f97400b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f97401483a0 0x7f97400b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f97400a70c0 0x7f9740144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97400a90c0 con 0x7f97400a70c0 2021-07-11T03:56:47.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97400b0350 con 0x7f97401483a0 2021-07-11T03:56:47.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97400b05f0 con 0x7f97400b1db0 2021-07-11T03:56:47.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f97400a70c0 0x7f9740144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f97400a70c0 0x7f9740144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44662/0 (socket says 172.21.2.104:44662) 2021-07-11T03:56:47.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973f7fe700 1 -- 172.21.2.104:0/1364823563 learned_addr learned my addr 172.21.2.104:0/1364823563 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:47.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973f7fe700 1 -- 172.21.2.104:0/1364823563 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97400b1db0 msgr2=0x7f97400b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:47.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973f7fe700 1 --2- 172.21.2.104:0/1364823563 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97400b1db0 0x7f97400b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973f7fe700 1 -- 172.21.2.104:0/1364823563 >> v2:172.21.2.105:3300/0 conn(0x7f97401483a0 msgr2=0x7f97400b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:47.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973f7fe700 1 --2- 172.21.2.104:0/1364823563 >> v2:172.21.2.105:3300/0 conn(0x7f97401483a0 0x7f97400b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973f7fe700 1 -- 172.21.2.104:0/1364823563 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97400b0890 con 0x7f97400a70c0 2021-07-11T03:56:47.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973f7fe700 1 --2- 172.21.2.104:0/1364823563 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97400a70c0 0x7f9740144720 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f97380032f0 tx=0x7f9738009290).ready entity=mon.0 client_cookie=d6e20a8d48c0e24f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973dffb700 1 -- 172.21.2.104:0/1364823563 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f973800b870 con 0x7f97400a70c0 2021-07-11T03:56:47.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973dffb700 1 -- 172.21.2.104:0/1364823563 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973800b9d0 con 0x7f97400a70c0 2021-07-11T03:56:47.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f973dffb700 1 -- 172.21.2.104:0/1364823563 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f973800bb40 con 0x7f97400a70c0 2021-07-11T03:56:47.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 -- 172.21.2.104:0/1364823563 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97400a70c0 msgr2=0x7f9740144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 --2- 172.21.2.104:0/1364823563 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97400a70c0 0x7f9740144720 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f97380032f0 tx=0x7f9738009290).stop 2021-07-11T03:56:47.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 -- 172.21.2.104:0/1364823563 shutdown_connections 2021-07-11T03:56:47.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 --2- 172.21.2.104:0/1364823563 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97400a70c0 0x7f9740144720 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 --2- 172.21.2.104:0/1364823563 >> v2:172.21.2.105:3300/0 conn(0x7f97401483a0 0x7f97400b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 --2- 172.21.2.104:0/1364823563 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97400b1db0 0x7f97400b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 -- 172.21.2.104:0/1364823563 >> 172.21.2.104:0/1364823563 conn(0x7f974005f6b0 msgr2=0x7f974005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 -- 172.21.2.104:0/1364823563 shutdown_connections 2021-07-11T03:56:47.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.797+0000 7f974574f700 1 -- 172.21.2.104:0/1364823563 wait complete. 2021-07-11T03:56:47.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f974574f700 1 Processor -- start 2021-07-11T03:56:47.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f974574f700 1 -- start start 2021-07-11T03:56:47.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f974574f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97400a70c0 0x7f9740065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f974574f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97400b1db0 0x7f97400ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f974574f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97401483a0 0x7f9740156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f974574f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9740154430 con 0x7f97401483a0 2021-07-11T03:56:47.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f974574f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9740067690 con 0x7f97400b1db0 2021-07-11T03:56:47.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f974574f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97400657c0 con 0x7f97400a70c0 2021-07-11T03:56:47.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f973f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97401483a0 0x7f9740156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f973f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97401483a0 0x7f9740156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44668/0 (socket says 172.21.2.104:44668) 2021-07-11T03:56:47.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f973f7fe700 1 -- 172.21.2.104:0/1130341554 learned_addr learned my addr 172.21.2.104:0/1130341554 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:47.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f973f7fe700 1 -- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97400a70c0 msgr2=0x7f9740065bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f973f7fe700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97400a70c0 0x7f9740065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f973f7fe700 1 -- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97400b1db0 msgr2=0x7f97400ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f973f7fe700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97400b1db0 0x7f97400ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f973f7fe700 1 -- 172.21.2.104:0/1130341554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9738009530 con 0x7f97401483a0 2021-07-11T03:56:47.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f973f7fe700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97401483a0 0x7f9740156710 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f97400ac290 tx=0x7f9738016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f9737fff700 1 -- 172.21.2.104:0/1130341554 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9738011cb0 con 0x7f97401483a0 2021-07-11T03:56:47.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f9737fff700 1 -- 172.21.2.104:0/1130341554 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9738011e10 con 0x7f97401483a0 2021-07-11T03:56:47.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f974574f700 1 -- 172.21.2.104:0/1130341554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97400642b0 con 0x7f97401483a0 2021-07-11T03:56:47.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f9737fff700 1 -- 172.21.2.104:0/1130341554 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9738020620 con 0x7f97401483a0 2021-07-11T03:56:47.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f974574f700 1 -- 172.21.2.104:0/1130341554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9740067310 con 0x7f97401483a0 2021-07-11T03:56:47.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f974574f700 1 -- 172.21.2.104:0/1130341554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97400659c0 con 0x7f97401483a0 2021-07-11T03:56:47.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f9737fff700 1 -- 172.21.2.104:0/1130341554 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9738026020 con 0x7f97401483a0 2021-07-11T03:56:47.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f9737fff700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9724053330 0x7f97240557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f9737fff700 1 -- 172.21.2.104:0/1130341554 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f973806be70 con 0x7f97401483a0 2021-07-11T03:56:47.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.801+0000 7f973effd700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9724053330 0x7f97240557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.805+0000 7f973effd700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9724053330 0x7f97240557f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f9728002f70 tx=0x7f9728006a90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.805+0000 7f9737fff700 1 -- 172.21.2.104:0/1130341554 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f97380434e0 con 0x7f97401483a0 2021-07-11T03:56:47.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.885+0000 7f974574f700 1 -- 172.21.2.104:0/1130341554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9740057350 con 0x7f97401483a0 2021-07-11T03:56:47.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.885+0000 7f9737fff700 1 -- 172.21.2.104:0/1130341554 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+143 (secure 0 0 0) 0x7f9738015070 con 0x7f97401483a0 2021-07-11T03:56:47.893 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:47.894 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T03:56:47.894 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.894 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T03:56:47.894 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T03:56:47.895 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:56:47.895 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:56:47.895 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T03:56:47.895 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T03:56:47.896 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:56:47.896 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T03:56:47.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 -- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9724053330 msgr2=0x7f97240557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9724053330 0x7f97240557f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f9728002f70 tx=0x7f9728006a90).stop 2021-07-11T03:56:47.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 -- 172.21.2.104:0/1130341554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97401483a0 msgr2=0x7f9740156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97401483a0 0x7f9740156710 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f97400ac290 tx=0x7f9738016000).stop 2021-07-11T03:56:47.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 -- 172.21.2.104:0/1130341554 shutdown_connections 2021-07-11T03:56:47.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9724053330 0x7f97240557f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97401483a0 0x7f9740156710 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97400b1db0 0x7f97400ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 --2- 172.21.2.104:0/1130341554 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97400a70c0 0x7f9740065bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 -- 172.21.2.104:0/1130341554 >> 172.21.2.104:0/1130341554 conn(0x7f974005f6b0 msgr2=0x7f97400aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 -- 172.21.2.104:0/1130341554 shutdown_connections 2021-07-11T03:56:47.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.889+0000 7f974574f700 1 -- 172.21.2.104:0/1130341554 wait complete. 2021-07-11T03:56:47.901 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T03:56:47.901 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T03:56:47.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.945+0000 7fc1e5fd3700 1 Processor -- start 2021-07-11T03:56:47.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.945+0000 7fc1e5fd3700 1 -- start start 2021-07-11T03:56:47.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc1e00b05b0 0x7fc1e00b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc1e00b3710 0x7fc1e013d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1e013da40 0x7fc1e00b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e0143920 con 0x7fc1e00b05b0 2021-07-11T03:56:47.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1e0143bc0 con 0x7fc1e00b3710 2021-07-11T03:56:47.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e0143e60 con 0x7fc1e013da40 2021-07-11T03:56:47.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1df7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc1e00b05b0 0x7fc1e00b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1df7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc1e00b05b0 0x7fc1e00b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44674/0 (socket says 172.21.2.104:44674) 2021-07-11T03:56:47.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1df7fe700 1 -- 172.21.2.104:0/5767515 learned_addr learned my addr 172.21.2.104:0/5767515 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:47.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1df7fe700 1 -- 172.21.2.104:0/5767515 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1e013da40 msgr2=0x7fc1e00b10b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:56:47.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1df7fe700 1 --2- 172.21.2.104:0/5767515 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1e013da40 0x7fc1e00b10b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1df7fe700 1 -- 172.21.2.104:0/5767515 >> v2:172.21.2.105:3300/0 conn(0x7fc1e00b3710 msgr2=0x7fc1e013d550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:56:47.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1df7fe700 1 --2- 172.21.2.104:0/5767515 >> v2:172.21.2.105:3300/0 conn(0x7fc1e00b3710 0x7fc1e013d550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1df7fe700 1 -- 172.21.2.104:0/5767515 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e0144100 con 0x7fc1e00b05b0 2021-07-11T03:56:47.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1df7fe700 1 --2- 172.21.2.104:0/5767515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00b05b0 0x7fc1e00b0990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fc1c8002f70 tx=0x7fc1c8006eb0).ready entity=mon.0 client_cookie=a59b08e6b4613d30 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1de7fc700 1 -- 172.21.2.104:0/5767515 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1c800d570 con 0x7fc1e00b05b0 2021-07-11T03:56:47.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1de7fc700 1 -- 172.21.2.104:0/5767515 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c8000e90 con 0x7fc1e00b05b0 2021-07-11T03:56:47.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1de7fc700 1 -- 172.21.2.104:0/5767515 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1c800d900 con 0x7fc1e00b05b0 2021-07-11T03:56:47.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/5767515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00b05b0 msgr2=0x7fc1e00b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:47.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 --2- 172.21.2.104:0/5767515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00b05b0 0x7fc1e00b0990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7fc1c8002f70 tx=0x7fc1c8006eb0).stop 2021-07-11T03:56:47.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/5767515 shutdown_connections 2021-07-11T03:56:47.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 --2- 172.21.2.104:0/5767515 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1e013da40 0x7fc1e00b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 --2- 172.21.2.104:0/5767515 >> v2:172.21.2.105:3300/0 conn(0x7fc1e00b3710 0x7fc1e013d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 --2- 172.21.2.104:0/5767515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00b05b0 0x7fc1e00b0990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:47.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/5767515 >> 172.21.2.104:0/5767515 conn(0x7fc1e00607c0 msgr2=0x7fc1e0060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:47.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/5767515 shutdown_connections 2021-07-11T03:56:47.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/5767515 wait complete. 2021-07-11T03:56:47.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 Processor -- start 2021-07-11T03:56:47.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- start start 2021-07-11T03:56:47.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1e00b05b0 0x7fc1e00a9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00b3710 0x7fc1e0131f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1e013da40 0x7fc1e0140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e00a81a0 con 0x7fc1e00b3710 2021-07-11T03:56:47.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e013e0e0 con 0x7fc1e013da40 2021-07-11T03:56:47.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e013e380 con 0x7fc1e00b05b0 2021-07-11T03:56:47.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1deffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00b3710 0x7fc1e0131f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1deffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00b3710 0x7fc1e0131f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44682/0 (socket says 172.21.2.104:44682) 2021-07-11T03:56:47.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1deffd700 1 -- 172.21.2.104:0/1515287822 learned_addr learned my addr 172.21.2.104:0/1515287822 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:47.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1deffd700 1 -- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1e00b05b0 msgr2=0x7fc1e00a9260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:56:47.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1deffd700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1e00b05b0 0x7fc1e00a9260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1deffd700 1 -- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1e013da40 msgr2=0x7fc1e0140300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:56:47.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1deffd700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1e013da40 0x7fc1e0140300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:47.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1deffd700 1 -- 172.21.2.104:0/1515287822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c800b1e0 con 0x7fc1e00b3710 2021-07-11T03:56:47.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1deffd700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00b3710 0x7fc1e0131f20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fc1d0002fb0 tx=0x7fc1d0006e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1dcff9700 1 -- 172.21.2.104:0/1515287822 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1d0013460 con 0x7fc1e00b3710 2021-07-11T03:56:47.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1dcff9700 1 -- 172.21.2.104:0/1515287822 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d00135c0 con 0x7fc1e00b3710 2021-07-11T03:56:47.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/1515287822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e00a9750 con 0x7fc1e00b3710 2021-07-11T03:56:47.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1dcff9700 1 -- 172.21.2.104:0/1515287822 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1d0013920 con 0x7fc1e00b3710 2021-07-11T03:56:47.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.949+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/1515287822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e00a72a0 con 0x7fc1e00b3710 2021-07-11T03:56:47.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.953+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/1515287822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e0057350 con 0x7fc1e00b3710 2021-07-11T03:56:47.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.953+0000 7fc1dcff9700 1 -- 172.21.2.104:0/1515287822 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc1d000f070 con 0x7fc1e00b3710 2021-07-11T03:56:47.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.953+0000 7fc1dcff9700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1c40531f0 0x7fc1c40556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:47.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.953+0000 7fc1dcff9700 1 -- 172.21.2.104:0/1515287822 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc1d006aca0 con 0x7fc1e00b3710 2021-07-11T03:56:47.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.953+0000 7fc1df7fe700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1c40531f0 0x7fc1c40556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:47.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.953+0000 7fc1df7fe700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1c40531f0 0x7fc1c40556b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fc1c8002f70 tx=0x7fc1c8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:47.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:47.953+0000 7fc1dcff9700 1 -- 172.21.2.104:0/1515287822 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc1d0042200 con 0x7fc1e00b3710 2021-07-11T03:56:48.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.037+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/1515287822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fc1e0044650 con 0x7fc1e00b3710 2021-07-11T03:56:48.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.497+0000 7fc1dcff9700 1 -- 172.21.2.104:0/1515287822 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v52) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc1d0045820 con 0x7fc1e00b3710 2021-07-11T03:56:48.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1c40531f0 msgr2=0x7fc1c40556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:48.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1c40531f0 0x7fc1c40556b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fc1c8002f70 tx=0x7fc1c8015000).stop 2021-07-11T03:56:48.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/1515287822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00b3710 msgr2=0x7fc1e0131f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:48.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00b3710 0x7fc1e0131f20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fc1d0002fb0 tx=0x7fc1d0006e00).stop 2021-07-11T03:56:48.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/1515287822 shutdown_connections 2021-07-11T03:56:48.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1e013da40 0x7fc1e0140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:48.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1c40531f0 0x7fc1c40556b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:48.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00b3710 0x7fc1e0131f20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:48.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 --2- 172.21.2.104:0/1515287822 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1e00b05b0 0x7fc1e00a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:48.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/1515287822 >> 172.21.2.104:0/1515287822 conn(0x7fc1e00607c0 msgr2=0x7fc1e0142f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:48.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/1515287822 shutdown_connections 2021-07-11T03:56:48.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.501+0000 7fc1e5fd3700 1 -- 172.21.2.104:0/1515287822 wait complete. 2021-07-11T03:56:48.510 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T03:56:48.513 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T03:56:48.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 Processor -- start 2021-07-11T03:56:48.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 -- start start 2021-07-11T03:56:48.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb1580b7370 0x7fb1580b7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:48.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb1580b3060 0x7fb15813e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:48.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb15813e710 0x7fb158064ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:48.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1580b2380 con 0x7fb1580b7370 2021-07-11T03:56:48.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb158144250 con 0x7fb1580b3060 2021-07-11T03:56:48.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1580b3570 con 0x7fb15813e710 2021-07-11T03:56:48.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15c97b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb1580b7370 0x7fb1580b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:48.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15c97b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb1580b7370 0x7fb1580b7750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44688/0 (socket says 172.21.2.104:44688) 2021-07-11T03:56:48.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15c97b700 1 -- 172.21.2.104:0/3203030429 learned_addr learned my addr 172.21.2.104:0/3203030429 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:48.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb157fff700 1 --2- 172.21.2.104:0/3203030429 >> v2:172.21.2.105:3300/0 conn(0x7fb1580b3060 0x7fb15813e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:48.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15c97b700 1 -- 172.21.2.104:0/3203030429 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb15813e710 msgr2=0x7fb158064ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:48.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15c97b700 1 --2- 172.21.2.104:0/3203030429 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb15813e710 0x7fb158064ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:48.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15c97b700 1 -- 172.21.2.104:0/3203030429 >> v2:172.21.2.105:3300/0 conn(0x7fb1580b3060 msgr2=0x7fb15813e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:48.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15c97b700 1 --2- 172.21.2.104:0/3203030429 >> v2:172.21.2.105:3300/0 conn(0x7fb1580b3060 0x7fb15813e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:48.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15c97b700 1 -- 172.21.2.104:0/3203030429 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1580b25b0 con 0x7fb1580b7370 2021-07-11T03:56:48.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15c97b700 1 --2- 172.21.2.104:0/3203030429 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1580b7370 0x7fb1580b7750 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fb14c002f70 tx=0x7fb14c006e50).ready entity=mon.0 client_cookie=22664dd4cb66e382 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:48.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb1577fe700 1 -- 172.21.2.104:0/3203030429 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb14c00d570 con 0x7fb1580b7370 2021-07-11T03:56:48.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb1577fe700 1 -- 172.21.2.104:0/3203030429 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14c000e90 con 0x7fb1580b7370 2021-07-11T03:56:48.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb1577fe700 1 -- 172.21.2.104:0/3203030429 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb14c00d900 con 0x7fb1580b7370 2021-07-11T03:56:48.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 -- 172.21.2.104:0/3203030429 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1580b7370 msgr2=0x7fb1580b7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:48.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 --2- 172.21.2.104:0/3203030429 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1580b7370 0x7fb1580b7750 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7fb14c002f70 tx=0x7fb14c006e50).stop 2021-07-11T03:56:48.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 -- 172.21.2.104:0/3203030429 shutdown_connections 2021-07-11T03:56:48.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 --2- 172.21.2.104:0/3203030429 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb15813e710 0x7fb158064ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:48.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 --2- 172.21.2.104:0/3203030429 >> v2:172.21.2.105:3300/0 conn(0x7fb1580b3060 0x7fb15813e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:48.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 --2- 172.21.2.104:0/3203030429 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1580b7370 0x7fb1580b7750 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:48.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.561+0000 7fb15d97d700 1 -- 172.21.2.104:0/3203030429 >> 172.21.2.104:0/3203030429 conn(0x7fb15805f6b0 msgr2=0x7fb15805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:48.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 -- 172.21.2.104:0/3203030429 shutdown_connections 2021-07-11T03:56:48.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 -- 172.21.2.104:0/3203030429 wait complete. 2021-07-11T03:56:48.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 Processor -- start 2021-07-11T03:56:48.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 -- start start 2021-07-11T03:56:48.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1580b3060 0x7fb1580b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:48.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1580b7370 0x7fb1580afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:48.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb15813e710 0x7fb1580b0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:48.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15814fef0 con 0x7fb15813e710 2021-07-11T03:56:48.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1581500e0 con 0x7fb1580b3060 2021-07-11T03:56:48.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb158150380 con 0x7fb1580b7370 2021-07-11T03:56:48.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d17c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb15813e710 0x7fb1580b0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:48.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d17c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb15813e710 0x7fb1580b0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44694/0 (socket says 172.21.2.104:44694) 2021-07-11T03:56:48.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d17c700 1 -- 172.21.2.104:0/4280755162 learned_addr learned my addr 172.21.2.104:0/4280755162 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:48.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d17c700 1 -- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1580b7370 msgr2=0x7fb1580afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:48.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d17c700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1580b7370 0x7fb1580afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:48.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d17c700 1 -- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1580b3060 msgr2=0x7fb1580b1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:48.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d17c700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1580b3060 0x7fb1580b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:48.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d17c700 1 -- 172.21.2.104:0/4280755162 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c00b1e0 con 0x7fb15813e710 2021-07-11T03:56:48.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d17c700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb15813e710 0x7fb1580b0050 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fb1500032f0 tx=0x7fb1500091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:48.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb155ffb700 1 -- 172.21.2.104:0/4280755162 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb15000d4a0 con 0x7fb15813e710 2021-07-11T03:56:48.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb155ffb700 1 -- 172.21.2.104:0/4280755162 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15000d600 con 0x7fb15813e710 2021-07-11T03:56:48.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb155ffb700 1 -- 172.21.2.104:0/4280755162 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb15000d960 con 0x7fb15813e710 2021-07-11T03:56:48.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 -- 172.21.2.104:0/4280755162 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb158150620 con 0x7fb15813e710 2021-07-11T03:56:48.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 -- 172.21.2.104:0/4280755162 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb158150d60 con 0x7fb15813e710 2021-07-11T03:56:48.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb15d97d700 1 -- 172.21.2.104:0/4280755162 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb158057350 con 0x7fb15813e710 2021-07-11T03:56:48.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb155ffb700 1 -- 172.21.2.104:0/4280755162 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb150018030 con 0x7fb15813e710 2021-07-11T03:56:48.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb155ffb700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1400531f0 0x7fb1400556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:48.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.565+0000 7fb155ffb700 1 -- 172.21.2.104:0/4280755162 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5603+0+0 (secure 0 0 0) 0x7fb15006acb0 con 0x7fb15813e710 2021-07-11T03:56:48.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.569+0000 7fb15c97b700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1400531f0 0x7fb1400556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:48.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.569+0000 7fb15c97b700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1400531f0 0x7fb1400556b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fb14c002f70 tx=0x7fb14c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:48.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.569+0000 7fb155ffb700 1 -- 172.21.2.104:0/4280755162 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb150042160 con 0x7fb15813e710 2021-07-11T03:56:48.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:48.653+0000 7fb15d97d700 1 -- 172.21.2.104:0/4280755162 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fb1580601a0 con 0x7fb15813e710 2021-07-11T03:56:49.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.509+0000 7fb155ffb700 1 -- 172.21.2.104:0/4280755162 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v53) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb150045780 con 0x7fb15813e710 2021-07-11T03:56:49.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.509+0000 7fb15d97d700 1 -- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1400531f0 msgr2=0x7fb1400556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.509+0000 7fb15d97d700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1400531f0 0x7fb1400556b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fb14c002f70 tx=0x7fb14c015000).stop 2021-07-11T03:56:49.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.509+0000 7fb15d97d700 1 -- 172.21.2.104:0/4280755162 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb15813e710 msgr2=0x7fb1580b0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.509+0000 7fb15d97d700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb15813e710 0x7fb1580b0050 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fb1500032f0 tx=0x7fb1500091a0).stop 2021-07-11T03:56:49.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.513+0000 7fb15d97d700 1 -- 172.21.2.104:0/4280755162 shutdown_connections 2021-07-11T03:56:49.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.513+0000 7fb15d97d700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1400531f0 0x7fb1400556b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:49.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.513+0000 7fb15d97d700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb15813e710 0x7fb1580b0050 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:49.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.513+0000 7fb15d97d700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1580b7370 0x7fb1580afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.513+0000 7fb15d97d700 1 --2- 172.21.2.104:0/4280755162 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1580b3060 0x7fb1580b1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.513+0000 7fb15d97d700 1 -- 172.21.2.104:0/4280755162 >> 172.21.2.104:0/4280755162 conn(0x7fb15805f6b0 msgr2=0x7fb15805ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:49.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.513+0000 7fb15d97d700 1 -- 172.21.2.104:0/4280755162 shutdown_connections 2021-07-11T03:56:49.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.513+0000 7fb15d97d700 1 -- 172.21.2.104:0/4280755162 wait complete. 2021-07-11T03:56:49.521 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T03:56:49.525 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T03:56:49.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576faba700 1 Processor -- start 2021-07-11T03:56:49.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576faba700 1 -- start start 2021-07-11T03:56:49.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576faba700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5768066ac0 0x7f57680b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576faba700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f576805f6b0 0x7f5768060490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576faba700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57680b2ec0 con 0x7f57680b2750 2021-07-11T03:56:49.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576faba700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57680b07b0 con 0x7f5768066ac0 2021-07-11T03:56:49.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576faba700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57680613c0 con 0x7f576805f6b0 2021-07-11T03:56:49.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576d856700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5768066ac0 0x7f57680b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:49.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576d055700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f57680b2750 0x7f576805fd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41472/0 (socket says 172.21.2.104:41472) 2021-07-11T03:56:49.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576d055700 1 -- 172.21.2.104:0/2035559712 learned_addr learned my addr 172.21.2.104:0/2035559712 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:49.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576d856700 1 -- 172.21.2.104:0/2035559712 >> v2:172.21.2.105:3300/0 conn(0x7f576805f6b0 msgr2=0x7f5768060490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:49.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576d856700 1 --2- 172.21.2.104:0/2035559712 >> v2:172.21.2.105:3300/0 conn(0x7f576805f6b0 0x7f5768060490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576d856700 1 -- 172.21.2.104:0/2035559712 >> v1:172.21.2.104:6789/0 conn(0x7f57680b2750 legacy=0x7f576805fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.573+0000 7f576d856700 1 -- 172.21.2.104:0/2035559712 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5768066ea0 con 0x7f5768066ac0 2021-07-11T03:56:49.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d856700 1 --2- 172.21.2.104:0/2035559712 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5768066ac0 0x7f57680b3590 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f57600032f0 tx=0x7f57600091d0).ready entity=mon.0 client_cookie=f8f3d6f7bd80dd30 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:49.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576c854700 1 -- 172.21.2.104:0/2035559712 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5760009ea0 con 0x7f5768066ac0 2021-07-11T03:56:49.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576c854700 1 -- 172.21.2.104:0/2035559712 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576000b870 con 0x7f5768066ac0 2021-07-11T03:56:49.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576c854700 1 -- 172.21.2.104:0/2035559712 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f576000b9e0 con 0x7f5768066ac0 2021-07-11T03:56:49.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- 172.21.2.104:0/2035559712 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5768066ac0 msgr2=0x7f57680b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 --2- 172.21.2.104:0/2035559712 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5768066ac0 0x7f57680b3590 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f57600032f0 tx=0x7f57600091d0).stop 2021-07-11T03:56:49.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- 172.21.2.104:0/2035559712 shutdown_connections 2021-07-11T03:56:49.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 --2- 172.21.2.104:0/2035559712 >> v2:172.21.2.105:3300/0 conn(0x7f576805f6b0 0x7f5768060490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 --2- 172.21.2.104:0/2035559712 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5768066ac0 0x7f57680b3590 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:49.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- 172.21.2.104:0/2035559712 >> 172.21.2.104:0/2035559712 conn(0x7f576805c490 msgr2=0x7f576805c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:49.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- 172.21.2.104:0/2035559712 shutdown_connections 2021-07-11T03:56:49.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- 172.21.2.104:0/2035559712 wait complete. 2021-07-11T03:56:49.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 Processor -- start 2021-07-11T03:56:49.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- start start 2021-07-11T03:56:49.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f576805f6b0 0x7f57680ab780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5768066ac0 0x7f57680b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57680b2750 0x7f57680a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57680ac790 con 0x7f576805f6b0 2021-07-11T03:56:49.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57680ac9c0 con 0x7f5768066ac0 2021-07-11T03:56:49.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57680acc60 con 0x7f57680b2750 2021-07-11T03:56:49.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d856700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f576805f6b0 0x7f57680ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:49.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d856700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f576805f6b0 0x7f57680ab780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44708/0 (socket says 172.21.2.104:44708) 2021-07-11T03:56:49.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d856700 1 -- 172.21.2.104:0/323072190 learned_addr learned my addr 172.21.2.104:0/323072190 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:49.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d856700 1 -- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57680b2750 msgr2=0x7f57680a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d856700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57680b2750 0x7f57680a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d856700 1 -- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5768066ac0 msgr2=0x7f57680b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d055700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5768066ac0 0x7f57680b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:49.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d856700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5768066ac0 0x7f57680b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:49.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d856700 1 -- 172.21.2.104:0/323072190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5760009470 con 0x7f576805f6b0 2021-07-11T03:56:49.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d055700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5768066ac0 0x7f57680b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:56:49.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576d856700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f576805f6b0 0x7f57680ab780 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f57600032f0 tx=0x7f5760016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:49.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f5756ffd700 1 -- 172.21.2.104:0/323072190 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5760021440 con 0x7f576805f6b0 2021-07-11T03:56:49.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f5756ffd700 1 -- 172.21.2.104:0/323072190 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57600215a0 con 0x7f576805f6b0 2021-07-11T03:56:49.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f5756ffd700 1 -- 172.21.2.104:0/323072190 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5760021900 con 0x7f576805f6b0 2021-07-11T03:56:49.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- 172.21.2.104:0/323072190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57680acf00 con 0x7f576805f6b0 2021-07-11T03:56:49.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- 172.21.2.104:0/323072190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57680a0fe0 con 0x7f576805f6b0 2021-07-11T03:56:49.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f576faba700 1 -- 172.21.2.104:0/323072190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5768057350 con 0x7f576805f6b0 2021-07-11T03:56:49.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.577+0000 7f5756ffd700 1 -- 172.21.2.104:0/323072190 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5760027020 con 0x7f576805f6b0 2021-07-11T03:56:49.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.581+0000 7f5756ffd700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5748053240 0x7f5748055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.581+0000 7f5756ffd700 1 -- 172.21.2.104:0/323072190 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f576006cab0 con 0x7f576805f6b0 2021-07-11T03:56:49.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.581+0000 7f576d055700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5748053240 0x7f5748055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:49.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.581+0000 7f576d055700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5748053240 0x7f5748055700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f5758008000 tx=0x7f5758009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:49.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.581+0000 7f5756ffd700 1 -- 172.21.2.104:0/323072190 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5760043d50 con 0x7f576805f6b0 2021-07-11T03:56:49.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.689+0000 7f576faba700 1 -- 172.21.2.104:0/323072190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f57680ad090 con 0x7f576805f6b0 2021-07-11T03:56:49.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f5756ffd700 1 -- 172.21.2.104:0/323072190 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v21) v1 ==== 188+0+0 (secure 0 0 0) 0x7f57600150d0 con 0x7f576805f6b0 2021-07-11T03:56:49.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 -- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5748053240 msgr2=0x7f5748055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5748053240 0x7f5748055700 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f5758008000 tx=0x7f5758009040).stop 2021-07-11T03:56:49.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 -- 172.21.2.104:0/323072190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f576805f6b0 msgr2=0x7f57680ab780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f576805f6b0 0x7f57680ab780 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f57600032f0 tx=0x7f5760016000).stop 2021-07-11T03:56:49.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 -- 172.21.2.104:0/323072190 shutdown_connections 2021-07-11T03:56:49.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57680b2750 0x7f57680a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5768066ac0 0x7f57680b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:49.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5748053240 0x7f5748055700 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:49.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 --2- 172.21.2.104:0/323072190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f576805f6b0 0x7f57680ab780 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:49.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 -- 172.21.2.104:0/323072190 >> 172.21.2.104:0/323072190 conn(0x7f576805c490 msgr2=0x7f576805fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:49.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 -- 172.21.2.104:0/323072190 shutdown_connections 2021-07-11T03:56:49.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.697+0000 7f576faba700 1 -- 172.21.2.104:0/323072190 wait complete. 2021-07-11T03:56:49.708 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 9 and data pool 10 2021-07-11T03:56:49.712 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:56:49.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 Processor -- start 2021-07-11T03:56:49.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 -- start start 2021-07-11T03:56:49.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6b00b7810 0x7ff6b01445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff6b00b1170 0x7ff6b0144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6b00b31d0 con 0x7ff6b0064870 2021-07-11T03:56:49.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6b00b7bf0 con 0x7ff6b00b1170 2021-07-11T03:56:49.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b00675c0 con 0x7ff6b00b7810 2021-07-11T03:56:49.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6af7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff6b0064870 0x7ff6b0064cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41492/0 (socket says 172.21.2.104:41492) 2021-07-11T03:56:49.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6af7fe700 1 -- 172.21.2.104:0/2125672179 learned_addr learned my addr 172.21.2.104:0/2125672179 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:49.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6ae7fc700 1 --2- 172.21.2.104:0/2125672179 >> v2:172.21.2.104:3300/0 conn(0x7ff6b00b1170 0x7ff6b0144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:49.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6ae7fc700 1 -- 172.21.2.104:0/2125672179 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6b00b7810 msgr2=0x7ff6b01445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:49.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6ae7fc700 1 --2- 172.21.2.104:0/2125672179 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6b00b7810 0x7ff6b01445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6ae7fc700 1 -- 172.21.2.104:0/2125672179 >> v1:172.21.2.104:6789/0 conn(0x7ff6b0064870 legacy=0x7ff6b0064cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6ae7fc700 1 -- 172.21.2.104:0/2125672179 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b00aae70 con 0x7ff6b00b1170 2021-07-11T03:56:49.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6ae7fc700 1 --2- 172.21.2.104:0/2125672179 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6b00b1170 0x7ff6b0144a90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7ff698002f70 tx=0x7ff698006df0).ready entity=mon.0 client_cookie=9aab9e2b078e23a0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:49.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6adffb700 1 -- 172.21.2.104:0/2125672179 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff69800d570 con 0x7ff6b00b1170 2021-07-11T03:56:49.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6adffb700 1 -- 172.21.2.104:0/2125672179 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff698000e90 con 0x7ff6b00b1170 2021-07-11T03:56:49.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6adffb700 1 -- 172.21.2.104:0/2125672179 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff69800d8c0 con 0x7ff6b00b1170 2021-07-11T03:56:49.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 -- 172.21.2.104:0/2125672179 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6b00b1170 msgr2=0x7ff6b0144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 --2- 172.21.2.104:0/2125672179 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6b00b1170 0x7ff6b0144a90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7ff698002f70 tx=0x7ff698006df0).stop 2021-07-11T03:56:49.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 -- 172.21.2.104:0/2125672179 shutdown_connections 2021-07-11T03:56:49.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 --2- 172.21.2.104:0/2125672179 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6b00b1170 0x7ff6b0144a90 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:49.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 --2- 172.21.2.104:0/2125672179 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6b00b7810 0x7ff6b01445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 -- 172.21.2.104:0/2125672179 >> 172.21.2.104:0/2125672179 conn(0x7ff6b005f6b0 msgr2=0x7ff6b005fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:49.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 -- 172.21.2.104:0/2125672179 shutdown_connections 2021-07-11T03:56:49.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.757+0000 7ff6b5273700 1 -- 172.21.2.104:0/2125672179 wait complete. 2021-07-11T03:56:49.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6b5273700 1 Processor -- start 2021-07-11T03:56:49.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6b5273700 1 -- start start 2021-07-11T03:56:49.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6b5273700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6b0064870 0x7ff6b00b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6b5273700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6b00b1170 0x7ff6b00ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6b5273700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6b00b7810 0x7ff6b00adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6b5273700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b013d520 con 0x7ff6b00b1170 2021-07-11T03:56:49.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6b5273700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b01444a0 con 0x7ff6b0064870 2021-07-11T03:56:49.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6b5273700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b0144700 con 0x7ff6b00b7810 2021-07-11T03:56:49.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6ae7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6b00b1170 0x7ff6b00ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:49.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6ae7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6b00b1170 0x7ff6b00ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44732/0 (socket says 172.21.2.104:44732) 2021-07-11T03:56:49.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6ae7fc700 1 -- 172.21.2.104:0/2969228576 learned_addr learned my addr 172.21.2.104:0/2969228576 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:49.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6ae7fc700 1 -- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6b00b7810 msgr2=0x7ff6b00adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6ae7fc700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6b00b7810 0x7ff6b00adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6ae7fc700 1 -- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6b0064870 msgr2=0x7ff6b00b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6ae7fc700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6b0064870 0x7ff6b00b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6ae7fc700 1 -- 172.21.2.104:0/2969228576 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69800b180 con 0x7ff6b00b1170 2021-07-11T03:56:49.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6ae7fc700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6b00b1170 0x7ff6b00ad7c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7ff698002f70 tx=0x7ff698015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:49.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6a7fff700 1 -- 172.21.2.104:0/2969228576 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff69801fa30 con 0x7ff6b00b1170 2021-07-11T03:56:49.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6a7fff700 1 -- 172.21.2.104:0/2969228576 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff698023070 con 0x7ff6b00b1170 2021-07-11T03:56:49.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6b5273700 1 -- 172.21.2.104:0/2969228576 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b0145c50 con 0x7ff6b00b1170 2021-07-11T03:56:49.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6a7fff700 1 -- 172.21.2.104:0/2969228576 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff69801fd80 con 0x7ff6b00b1170 2021-07-11T03:56:49.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6b5273700 1 -- 172.21.2.104:0/2969228576 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b0144eb0 con 0x7ff6b00b1170 2021-07-11T03:56:49.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6b5273700 1 -- 172.21.2.104:0/2969228576 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b0144a80 con 0x7ff6b00b1170 2021-07-11T03:56:49.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6a7fff700 1 -- 172.21.2.104:0/2969228576 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff69802f070 con 0x7ff6b00b1170 2021-07-11T03:56:49.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6a7fff700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff694053240 0x7ff694055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6a7fff700 1 -- 172.21.2.104:0/2969228576 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff698070c40 con 0x7ff6b00b1170 2021-07-11T03:56:49.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.761+0000 7ff6aeffd700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff694053240 0x7ff694055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:49.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.765+0000 7ff6aeffd700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff694053240 0x7ff694055700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7ff6a0002f70 tx=0x7ff6a00069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:49.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.765+0000 7ff6a7fff700 1 -- 172.21.2.104:0/2969228576 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff698047e70 con 0x7ff6b00b1170 2021-07-11T03:56:49.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.849+0000 7ff6b5273700 1 -- 172.21.2.104:0/2969228576 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6b0057350 con 0x7ff6b00b1170 2021-07-11T03:56:49.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.849+0000 7ff6a7fff700 1 -- 172.21.2.104:0/2969228576 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16053 (secure 0 0 0) 0x7ff69804b490 con 0x7ff6b00b1170 2021-07-11T03:56:49.857 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:49.857 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":54,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:56:49.697266+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:56:48.052820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-07-11T03:56:48.670109+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:56:49.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.849+0000 7ff6b5273700 1 -- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff694053240 msgr2=0x7ff694055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.849+0000 7ff6b5273700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff694053240 0x7ff694055700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7ff6a0002f70 tx=0x7ff6a00069e0).stop 2021-07-11T03:56:49.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.849+0000 7ff6b5273700 1 -- 172.21.2.104:0/2969228576 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6b00b1170 msgr2=0x7ff6b00ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.849+0000 7ff6b5273700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6b00b1170 0x7ff6b00ad7c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7ff698002f70 tx=0x7ff698015000).stop 2021-07-11T03:56:49.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.853+0000 7ff6b5273700 1 -- 172.21.2.104:0/2969228576 shutdown_connections 2021-07-11T03:56:49.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.853+0000 7ff6b5273700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6b00b7810 0x7ff6b00adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.853+0000 7ff6b5273700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff694053240 0x7ff694055700 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:49.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.853+0000 7ff6b5273700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6b00b1170 0x7ff6b00ad7c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:49.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.853+0000 7ff6b5273700 1 --2- 172.21.2.104:0/2969228576 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6b0064870 0x7ff6b00b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.853+0000 7ff6b5273700 1 -- 172.21.2.104:0/2969228576 >> 172.21.2.104:0/2969228576 conn(0x7ff6b005f6b0 msgr2=0x7ff6b0146960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:49.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.853+0000 7ff6b5273700 1 -- 172.21.2.104:0/2969228576 shutdown_connections 2021-07-11T03:56:49.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.853+0000 7ff6b5273700 1 -- 172.21.2.104:0/2969228576 wait complete. 2021-07-11T03:56:49.865 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:56:49.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 Processor -- start 2021-07-11T03:56:49.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 -- start start 2021-07-11T03:56:49.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3380b1170 0x7fc338141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc338141920 0x7fc338141d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3380b31d0 con 0x7fc3380b7810 2021-07-11T03:56:49.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3380b7bf0 con 0x7fc338141920 2021-07-11T03:56:49.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3380675c0 con 0x7fc3380b1170 2021-07-11T03:56:49.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc3377fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc338141920 0x7fc338141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:49.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc336ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc3380b7810 0x7fc3381445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41504/0 (socket says 172.21.2.104:41504) 2021-07-11T03:56:49.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc336ffd700 1 -- 172.21.2.104:0/462997363 learned_addr learned my addr 172.21.2.104:0/462997363 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:49.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc3377fe700 1 -- 172.21.2.104:0/462997363 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3380b1170 msgr2=0x7fc338141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:49.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc3377fe700 1 --2- 172.21.2.104:0/462997363 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3380b1170 0x7fc338141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc3377fe700 1 -- 172.21.2.104:0/462997363 >> v1:172.21.2.104:6789/0 conn(0x7fc3380b7810 legacy=0x7fc3381445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc3377fe700 1 -- 172.21.2.104:0/462997363 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338142270 con 0x7fc338141920 2021-07-11T03:56:49.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc3377fe700 1 --2- 172.21.2.104:0/462997363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc338141920 0x7fc338141d80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fc3300032f0 tx=0x7fc3300091d0).ready entity=mon.0 client_cookie=18c1ea1262d57be5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:49.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc335ffb700 1 -- 172.21.2.104:0/462997363 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc330009ea0 con 0x7fc338141920 2021-07-11T03:56:49.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc335ffb700 1 -- 172.21.2.104:0/462997363 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33000b8a0 con 0x7fc338141920 2021-07-11T03:56:49.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc335ffb700 1 -- 172.21.2.104:0/462997363 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc33000ba10 con 0x7fc338141920 2021-07-11T03:56:49.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 -- 172.21.2.104:0/462997363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc338141920 msgr2=0x7fc338141d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 --2- 172.21.2.104:0/462997363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc338141920 0x7fc338141d80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7fc3300032f0 tx=0x7fc3300091d0).stop 2021-07-11T03:56:49.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 -- 172.21.2.104:0/462997363 shutdown_connections 2021-07-11T03:56:49.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 --2- 172.21.2.104:0/462997363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc338141920 0x7fc338141d80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:49.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 --2- 172.21.2.104:0/462997363 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3380b1170 0x7fc338141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 -- 172.21.2.104:0/462997363 >> 172.21.2.104:0/462997363 conn(0x7fc33805f6b0 msgr2=0x7fc33805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:49.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 -- 172.21.2.104:0/462997363 shutdown_connections 2021-07-11T03:56:49.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.909+0000 7fc33d40a700 1 -- 172.21.2.104:0/462997363 wait complete. 2021-07-11T03:56:49.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc33d40a700 1 Processor -- start 2021-07-11T03:56:49.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc33d40a700 1 -- start start 2021-07-11T03:56:49.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc33d40a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3380b1170 0x7fc3380619a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc33d40a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3380b7810 0x7fc338142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc33d40a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc338141920 0x7fc338142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc33d40a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3381478b0 con 0x7fc338141920 2021-07-11T03:56:49.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc33d40a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338149be0 con 0x7fc3380b7810 2021-07-11T03:56:49.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc33d40a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc338145620 con 0x7fc3380b1170 2021-07-11T03:56:49.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc3377fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc338141920 0x7fc338142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:49.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc3377fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc338141920 0x7fc338142f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44744/0 (socket says 172.21.2.104:44744) 2021-07-11T03:56:49.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc3377fe700 1 -- 172.21.2.104:0/4009436043 learned_addr learned my addr 172.21.2.104:0/4009436043 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:49.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc3377fe700 1 -- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3380b1170 msgr2=0x7fc3380619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc3377fe700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3380b1170 0x7fc3380619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc3377fe700 1 -- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3380b7810 msgr2=0x7fc338142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:49.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc3377fe700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3380b7810 0x7fc338142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:49.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc3377fe700 1 -- 172.21.2.104:0/4009436043 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc330009470 con 0x7fc338141920 2021-07-11T03:56:49.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc3377fe700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc338141920 0x7fc338142f50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fc3381479f0 tx=0x7fc330016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:49.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc327fff700 1 -- 172.21.2.104:0/4009436043 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc330006ea0 con 0x7fc338141920 2021-07-11T03:56:49.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc327fff700 1 -- 172.21.2.104:0/4009436043 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc330021440 con 0x7fc338141920 2021-07-11T03:56:49.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc327fff700 1 -- 172.21.2.104:0/4009436043 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc3300217a0 con 0x7fc338141920 2021-07-11T03:56:49.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc33d40a700 1 -- 172.21.2.104:0/4009436043 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc338145850 con 0x7fc338141920 2021-07-11T03:56:49.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc33d40a700 1 -- 172.21.2.104:0/4009436043 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc338147170 con 0x7fc338141920 2021-07-11T03:56:49.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc33d40a700 1 -- 172.21.2.104:0/4009436043 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3381475b0 con 0x7fc338141920 2021-07-11T03:56:49.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc327fff700 1 -- 172.21.2.104:0/4009436043 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc330029070 con 0x7fc338141920 2021-07-11T03:56:49.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc327fff700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc314053270 0x7fc314055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:49.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc327fff700 1 -- 172.21.2.104:0/4009436043 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc33006c480 con 0x7fc338141920 2021-07-11T03:56:49.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.913+0000 7fc336ffd700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc314053270 0x7fc314055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:49.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.917+0000 7fc336ffd700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc314053270 0x7fc314055730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fc338060740 tx=0x7fc3280077d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:49.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:49.917+0000 7fc327fff700 1 -- 172.21.2.104:0/4009436043 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc3300436b0 con 0x7fc338141920 2021-07-11T03:56:50.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 -- 172.21.2.104:0/4009436043 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc338057350 con 0x7fc338141920 2021-07-11T03:56:50.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc327fff700 1 -- 172.21.2.104:0/4009436043 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16053 (secure 0 0 0) 0x7fc330015090 con 0x7fc338141920 2021-07-11T03:56:50.007 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:50.007 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":54,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:56:49.697266+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:56:48.052820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-07-11T03:56:48.670109+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:56:50.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 -- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc314053270 msgr2=0x7fc314055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc314053270 0x7fc314055730 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fc338060740 tx=0x7fc3280077d0).stop 2021-07-11T03:56:50.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 -- 172.21.2.104:0/4009436043 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc338141920 msgr2=0x7fc338142f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc338141920 0x7fc338142f50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fc3381479f0 tx=0x7fc330016000).stop 2021-07-11T03:56:50.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 -- 172.21.2.104:0/4009436043 shutdown_connections 2021-07-11T03:56:50.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc314053270 0x7fc314055730 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:50.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc338141920 0x7fc338142f50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:50.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3380b7810 0x7fc338142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:50.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 --2- 172.21.2.104:0/4009436043 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3380b1170 0x7fc3380619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:50.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 -- 172.21.2.104:0/4009436043 >> 172.21.2.104:0/4009436043 conn(0x7fc33805f6b0 msgr2=0x7fc33805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:50.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 -- 172.21.2.104:0/4009436043 shutdown_connections 2021-07-11T03:56:50.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.001+0000 7fc33d40a700 1 -- 172.21.2.104:0/4009436043 wait complete. 2021-07-11T03:56:50.015 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T03:56:50.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6db619700 1 Processor -- start 2021-07-11T03:56:50.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6db619700 1 -- start start 2021-07-11T03:56:50.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6db619700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd6d4144440 0x7fd6d4144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6db619700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d40604e0 con 0x7fd6d4060100 2021-07-11T03:56:50.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6db619700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d40b23b0 con 0x7fd6d4065420 2021-07-11T03:56:50.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6db619700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d4065c80 con 0x7fd6d4144440 2021-07-11T03:56:50.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6d93b5700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd6d4060100 0x7fd6d4064d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41518/0 (socket says 172.21.2.104:41518) 2021-07-11T03:56:50.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6d93b5700 1 -- 172.21.2.104:0/2975705027 learned_addr learned my addr 172.21.2.104:0/2975705027 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:50.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6d8bb4700 1 --2- 172.21.2.104:0/2975705027 >> v2:172.21.2.104:3300/0 conn(0x7fd6d4144440 0x7fd6d4144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:50.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6c7fff700 1 -- 172.21.2.104:0/2975705027 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802063948 0 0) 0x7fd6d40604e0 con 0x7fd6d4060100 2021-07-11T03:56:50.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6c7fff700 1 -- 172.21.2.104:0/2975705027 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6bc003260 con 0x7fd6d4060100 2021-07-11T03:56:50.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6d8bb4700 1 -- 172.21.2.104:0/2975705027 >> v1:172.21.2.105:6789/0 conn(0x7fd6d4065420 legacy=0x7fd6d414b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:50.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6d8bb4700 1 -- 172.21.2.104:0/2975705027 >> v1:172.21.2.104:6789/0 conn(0x7fd6d4060100 legacy=0x7fd6d4064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6d8bb4700 1 -- 172.21.2.104:0/2975705027 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d40b2650 con 0x7fd6d4144440 2021-07-11T03:56:50.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6d8bb4700 1 --2- 172.21.2.104:0/2975705027 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6d4144440 0x7fd6d4144970 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fd6d405f160 tx=0x7fd6c8006df0).ready entity=mon.0 client_cookie=f8e1007b144ce5fa server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:50.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6c7fff700 1 -- 172.21.2.104:0/2975705027 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c800d5c0 con 0x7fd6d4144440 2021-07-11T03:56:50.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6c7fff700 1 -- 172.21.2.104:0/2975705027 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c8000e90 con 0x7fd6d4144440 2021-07-11T03:56:50.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6c7fff700 1 -- 172.21.2.104:0/2975705027 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c800d910 con 0x7fd6d4144440 2021-07-11T03:56:50.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6db619700 1 -- 172.21.2.104:0/2975705027 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6d4144440 msgr2=0x7fd6d4144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.061+0000 7fd6db619700 1 --2- 172.21.2.104:0/2975705027 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6d4144440 0x7fd6d4144970 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fd6d405f160 tx=0x7fd6c8006df0).stop 2021-07-11T03:56:50.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 -- 172.21.2.104:0/2975705027 shutdown_connections 2021-07-11T03:56:50.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 --2- 172.21.2.104:0/2975705027 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6d4144440 0x7fd6d4144970 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:50.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 -- 172.21.2.104:0/2975705027 >> 172.21.2.104:0/2975705027 conn(0x7fd6d405cf60 msgr2=0x7fd6d405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:50.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 -- 172.21.2.104:0/2975705027 shutdown_connections 2021-07-11T03:56:50.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 -- 172.21.2.104:0/2975705027 wait complete. 2021-07-11T03:56:50.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 Processor -- start 2021-07-11T03:56:50.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 -- start start 2021-07-11T03:56:50.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6d4060100 0x7fd6d412f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6d4065420 0x7fd6d40b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6d40b6a10 0x7fd6d4061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d413c2d0 con 0x7fd6d4065420 2021-07-11T03:56:50.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d413bbe0 con 0x7fd6d40b6a10 2021-07-11T03:56:50.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d40b5d20 con 0x7fd6d4060100 2021-07-11T03:56:50.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6d8bb4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6d4065420 0x7fd6d40b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:50.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6d8bb4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6d4065420 0x7fd6d40b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44758/0 (socket says 172.21.2.104:44758) 2021-07-11T03:56:50.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6d8bb4700 1 -- 172.21.2.104:0/2125993155 learned_addr learned my addr 172.21.2.104:0/2125993155 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:50.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6d8bb4700 1 -- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6d4060100 msgr2=0x7fd6d412f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6d8bb4700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6d4060100 0x7fd6d412f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:50.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6d8bb4700 1 -- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6d40b6a10 msgr2=0x7fd6d4061750 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:56:50.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6d8bb4700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6d40b6a10 0x7fd6d4061750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:50.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6d8bb4700 1 -- 172.21.2.104:0/2125993155 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c800b180 con 0x7fd6d4065420 2021-07-11T03:56:50.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6d8bb4700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6d4065420 0x7fd6d40b5300 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fd6d405f160 tx=0x7fd6c8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:50.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6c67fc700 1 -- 172.21.2.104:0/2125993155 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c8006b20 con 0x7fd6d4065420 2021-07-11T03:56:50.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6c67fc700 1 -- 172.21.2.104:0/2125993155 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c8002f70 con 0x7fd6d4065420 2021-07-11T03:56:50.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 -- 172.21.2.104:0/2125993155 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d413b940 con 0x7fd6d4065420 2021-07-11T03:56:50.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 -- 172.21.2.104:0/2125993155 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d405f6b0 con 0x7fd6d4065420 2021-07-11T03:56:50.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6c67fc700 1 -- 172.21.2.104:0/2125993155 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6c801f630 con 0x7fd6d4065420 2021-07-11T03:56:50.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6db619700 1 -- 172.21.2.104:0/2125993155 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d4057350 con 0x7fd6d4065420 2021-07-11T03:56:50.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6c67fc700 1 -- 172.21.2.104:0/2125993155 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd6c8025020 con 0x7fd6d4065420 2021-07-11T03:56:50.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6c67fc700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd6b4053350 0x7fd6b4055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6c67fc700 1 -- 172.21.2.104:0/2125993155 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd6c806c640 con 0x7fd6d4065420 2021-07-11T03:56:50.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.065+0000 7fd6d93b5700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd6b4053350 0x7fd6b4055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:50.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.069+0000 7fd6d93b5700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd6b4053350 0x7fd6b4055810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fd6d0001110 tx=0x7fd6d00076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:50.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.069+0000 7fd6c67fc700 1 -- 172.21.2.104:0/2125993155 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd6c8043870 con 0x7fd6d4065420 2021-07-11T03:56:50.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.177+0000 7fd6db619700 1 -- 172.21.2.104:0/2125993155 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd6d40698e0 con 0x7fd6d4065420 2021-07-11T03:56:50.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6c67fc700 1 -- 172.21.2.104:0/2125993155 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd6c8046e90 con 0x7fd6d4065420 2021-07-11T03:56:50.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 -- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd6b4053350 msgr2=0x7fd6b4055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd6b4053350 0x7fd6b4055810 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fd6d0001110 tx=0x7fd6d00076c0).stop 2021-07-11T03:56:50.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 -- 172.21.2.104:0/2125993155 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6d4065420 msgr2=0x7fd6d40b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6d4065420 0x7fd6d40b5300 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fd6d405f160 tx=0x7fd6c8014000).stop 2021-07-11T03:56:50.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 -- 172.21.2.104:0/2125993155 shutdown_connections 2021-07-11T03:56:50.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6d40b6a10 0x7fd6d4061750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:50.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd6b4053350 0x7fd6b4055810 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:50.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd6d4065420 0x7fd6d40b5300 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:50.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 --2- 172.21.2.104:0/2125993155 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd6d4060100 0x7fd6d412f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:50.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 -- 172.21.2.104:0/2125993155 >> 172.21.2.104:0/2125993155 conn(0x7fd6d405cf60 msgr2=0x7fd6d4140120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:50.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 -- 172.21.2.104:0/2125993155 shutdown_connections 2021-07-11T03:56:50.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.705+0000 7fd6db619700 1 -- 172.21.2.104:0/2125993155 wait complete. 2021-07-11T03:56:50.725 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:56:50.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 Processor -- start 2021-07-11T03:56:50.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 -- start start 2021-07-11T03:56:50.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fde7805f6b0 0x7fde7813fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde780b2a40 con 0x7fde780973d0 2021-07-11T03:56:50.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde78097b80 con 0x7fde780b2660 2021-07-11T03:56:50.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde78097e20 con 0x7fde7805f6b0 2021-07-11T03:56:50.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde767fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fde780973d0 0x7fde7813f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41536/0 (socket says 172.21.2.104:41536) 2021-07-11T03:56:50.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde767fc700 1 -- 172.21.2.104:0/1806621199 learned_addr learned my addr 172.21.2.104:0/1806621199 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:50.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde777fe700 1 --2- 172.21.2.104:0/1806621199 >> v2:172.21.2.104:3300/0 conn(0x7fde7805f6b0 0x7fde7813fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:50.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde75ffb700 1 -- 172.21.2.104:0/1806621199 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582908979 0 0) 0x7fde780b2a40 con 0x7fde780973d0 2021-07-11T03:56:50.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde777fe700 1 -- 172.21.2.104:0/1806621199 >> v1:172.21.2.105:6789/0 conn(0x7fde780b2660 legacy=0x7fde78096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:50.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde777fe700 1 -- 172.21.2.104:0/1806621199 >> v1:172.21.2.104:6789/0 conn(0x7fde780973d0 legacy=0x7fde7813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde777fe700 1 -- 172.21.2.104:0/1806621199 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde780980c0 con 0x7fde7805f6b0 2021-07-11T03:56:50.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde777fe700 1 --2- 172.21.2.104:0/1806621199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde7805f6b0 0x7fde7813fc70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fde780b33f0 tx=0x7fde7000a760).ready entity=mon.0 client_cookie=ca475271f7d18aee server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:50.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde75ffb700 1 -- 172.21.2.104:0/1806621199 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde7000f070 con 0x7fde7805f6b0 2021-07-11T03:56:50.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde75ffb700 1 -- 172.21.2.104:0/1806621199 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7000ce20 con 0x7fde7805f6b0 2021-07-11T03:56:50.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde75ffb700 1 -- 172.21.2.104:0/1806621199 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde70013600 con 0x7fde7805f6b0 2021-07-11T03:56:50.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1806621199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde7805f6b0 msgr2=0x7fde7813fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 --2- 172.21.2.104:0/1806621199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde7805f6b0 0x7fde7813fc70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fde780b33f0 tx=0x7fde7000a760).stop 2021-07-11T03:56:50.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1806621199 shutdown_connections 2021-07-11T03:56:50.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 --2- 172.21.2.104:0/1806621199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde7805f6b0 0x7fde7813fc70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:50.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1806621199 >> 172.21.2.104:0/1806621199 conn(0x7fde78062f10 msgr2=0x7fde780632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:50.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1806621199 shutdown_connections 2021-07-11T03:56:50.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1806621199 wait complete. 2021-07-11T03:56:50.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 Processor -- start 2021-07-11T03:56:50.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.797+0000 7fde7d2b0700 1 -- start start 2021-07-11T03:56:50.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde7d2b0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde7805f6b0 0x7fde780ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde7d2b0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde780973d0 0x7fde7813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde7d2b0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde780b2660 0x7fde7813d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde7d2b0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7809a2f0 con 0x7fde780b2660 2021-07-11T03:56:50.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde7d2b0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde781516e0 con 0x7fde780973d0 2021-07-11T03:56:50.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde7d2b0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde7813b630 con 0x7fde7805f6b0 2021-07-11T03:56:50.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde777fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde780b2660 0x7fde7813d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:50.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde777fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde780b2660 0x7fde7813d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44774/0 (socket says 172.21.2.104:44774) 2021-07-11T03:56:50.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde777fe700 1 -- 172.21.2.104:0/1237589954 learned_addr learned my addr 172.21.2.104:0/1237589954 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:50.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde777fe700 1 -- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde7805f6b0 msgr2=0x7fde780ac2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde777fe700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde7805f6b0 0x7fde780ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:50.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde777fe700 1 -- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde780973d0 msgr2=0x7fde7813cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde777fe700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde780973d0 0x7fde7813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:50.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde777fe700 1 -- 172.21.2.104:0/1237589954 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7000aa00 con 0x7fde780b2660 2021-07-11T03:56:50.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde777fe700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde780b2660 0x7fde7813d1f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fde780b33f0 tx=0x7fde78096240).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:50.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde6ffff700 1 -- 172.21.2.104:0/1237589954 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde70007610 con 0x7fde780b2660 2021-07-11T03:56:50.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde6ffff700 1 -- 172.21.2.104:0/1237589954 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde70007770 con 0x7fde780b2660 2021-07-11T03:56:50.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde6ffff700 1 -- 172.21.2.104:0/1237589954 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde7001f440 con 0x7fde780b2660 2021-07-11T03:56:50.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1237589954 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7813b8d0 con 0x7fde780b2660 2021-07-11T03:56:50.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1237589954 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde78099910 con 0x7fde780b2660 2021-07-11T03:56:50.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1237589954 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde78057350 con 0x7fde780b2660 2021-07-11T03:56:50.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde6ffff700 1 -- 172.21.2.104:0/1237589954 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fde70026030 con 0x7fde780b2660 2021-07-11T03:56:50.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde6ffff700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde5c053270 0x7fde5c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde6ffff700 1 -- 172.21.2.104:0/1237589954 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fde7006cd70 con 0x7fde780b2660 2021-07-11T03:56:50.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.801+0000 7fde76ffd700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde5c053270 0x7fde5c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:50.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.805+0000 7fde76ffd700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde5c053270 0x7fde5c055730 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fde68002f10 tx=0x7fde68006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:50.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.805+0000 7fde6ffff700 1 -- 172.21.2.104:0/1237589954 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fde700440b0 con 0x7fde780b2660 2021-07-11T03:56:50.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1237589954 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde7809bbd0 con 0x7fde780b2660 2021-07-11T03:56:50.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde6ffff700 1 -- 172.21.2.104:0/1237589954 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fde70024070 con 0x7fde780b2660 2021-07-11T03:56:50.920 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:50.920 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":22,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3427300973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3427300973},{"type":"v1","addr":"172.21.2.105:6835","nonce":3427300973}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/2823255258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2823255258},{"type":"v1","addr":"172.21.2.104:6833","nonce":2823255258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:56:49.697379+0000","modified":"2021-07-11T03:56:50.700784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/2428941897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2428941897},{"type":"v1","addr":"172.21.2.104:6835","nonce":2428941897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-07-11T03:56:50.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde5c053270 msgr2=0x7fde5c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde5c053270 0x7fde5c055730 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fde68002f10 tx=0x7fde68006930).stop 2021-07-11T03:56:50.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1237589954 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde780b2660 msgr2=0x7fde7813d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:50.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde780b2660 0x7fde7813d1f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fde780b33f0 tx=0x7fde78096240).stop 2021-07-11T03:56:50.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1237589954 shutdown_connections 2021-07-11T03:56:50.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde5c053270 0x7fde5c055730 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:50.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde780b2660 0x7fde7813d1f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:50.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde780973d0 0x7fde7813cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:50.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 --2- 172.21.2.104:0/1237589954 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde7805f6b0 0x7fde780ac2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:50.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1237589954 >> 172.21.2.104:0/1237589954 conn(0x7fde78062f10 msgr2=0x7fde7813f560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:50.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1237589954 shutdown_connections 2021-07-11T03:56:50.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.913+0000 7fde7d2b0700 1 -- 172.21.2.104:0/1237589954 wait complete. 2021-07-11T03:56:50.928 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 22 2021-07-11T03:56:50.929 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:56:50.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 Processor -- start 2021-07-11T03:56:50.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 -- start start 2021-07-11T03:56:50.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9cc80ad150 0x7f9cc80b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cc80a9da0 0x7f9cc8140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:50.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc80ac370 con 0x7f9cc80aa860 2021-07-11T03:56:50.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc80abb50 con 0x7f9cc80ad150 2021-07-11T03:56:50.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80aac40 con 0x7f9cc80a9da0 2021-07-11T03:56:50.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccd8f2700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f9cc80aa860 0x7f9cc80aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41546/0 (socket says 172.21.2.104:41546) 2021-07-11T03:56:51.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccd8f2700 1 -- 172.21.2.104:0/1224728534 learned_addr learned my addr 172.21.2.104:0/1224728534 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:51.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994830035 0 0) 0x7f9cc80ac370 con 0x7f9cc80aa860 2021-07-11T03:56:51.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cb0003260 con 0x7f9cc80aa860 2021-07-11T03:56:51.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9cce0f3700 1 --2- 172.21.2.104:0/1224728534 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cc80a9da0 0x7f9cc8140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccd0f1700 1 --2- 172.21.2.104:0/1224728534 >> v2:172.21.2.105:3300/0 conn(0x7f9cc80ad150 0x7f9cc80b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 627515644 0 0) 0x7f9cb0003260 con 0x7f9cc80aa860 2021-07-11T03:56:51.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cc80a9da0 msgr2=0x7f9cc8140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:51.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 --2- 172.21.2.104:0/1224728534 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cc80a9da0 0x7f9cc8140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 >> v2:172.21.2.105:3300/0 conn(0x7f9cc80ad150 msgr2=0x7f9cc80b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:51.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 --2- 172.21.2.104:0/1224728534 >> v2:172.21.2.105:3300/0 conn(0x7f9cc80ad150 0x7f9cc80b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc80b16b0 con 0x7f9cc80aa860 2021-07-11T03:56:51.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f9cbc001660 con 0x7f9cc80aa860 2021-07-11T03:56:51.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 >> v1:172.21.2.104:6789/0 conn(0x7f9cc80aa860 legacy=0x7f9cc80aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 --2- 172.21.2.104:0/1224728534 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cb00037d0 0x7f9cb0003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 --2- 172.21.2.104:0/1224728534 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cb0004500 0x7f9cb0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 --2- 172.21.2.104:0/1224728534 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cc80a9da0 0x7f9cb0004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80aac40 con 0x7f9cb00037d0 2021-07-11T03:56:51.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb000b850 con 0x7f9cb0004500 2021-07-11T03:56:51.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb000ba10 con 0x7f9cc80a9da0 2021-07-11T03:56:51.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccd8f2700 1 --2- 172.21.2.104:0/1224728534 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cb00037d0 0x7f9cb0003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccd8f2700 1 -- 172.21.2.104:0/1224728534 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cc80a9da0 msgr2=0x7f9cb0004ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:51.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccd8f2700 1 --2- 172.21.2.104:0/1224728534 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cc80a9da0 0x7f9cb0004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccd8f2700 1 -- 172.21.2.104:0/1224728534 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cb0004500 msgr2=0x7f9cb0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:51.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccd8f2700 1 --2- 172.21.2.104:0/1224728534 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cb0004500 0x7f9cb0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccd8f2700 1 -- 172.21.2.104:0/1224728534 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb0003260 con 0x7f9cb00037d0 2021-07-11T03:56:51.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccd8f2700 1 --2- 172.21.2.104:0/1224728534 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cb00037d0 0x7f9cb0003bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f9cbc001370 tx=0x7f9cc80b7150).ready entity=mon.0 client_cookie=2a49eb060ca2b069 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cbc0071a0 con 0x7f9cb00037d0 2021-07-11T03:56:51.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc007300 con 0x7f9cb00037d0 2021-07-11T03:56:51.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccc8f0700 1 -- 172.21.2.104:0/1224728534 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cbc00cae0 con 0x7f9cb00037d0 2021-07-11T03:56:51.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 -- 172.21.2.104:0/1224728534 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cb00037d0 msgr2=0x7f9cb0003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 --2- 172.21.2.104:0/1224728534 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cb00037d0 0x7f9cb0003bf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f9cbc001370 tx=0x7f9cc80b7150).stop 2021-07-11T03:56:51.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9cce0f3700 1 -- 172.21.2.104:0/1224728534 reap_dead start 2021-07-11T03:56:51.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 -- 172.21.2.104:0/1224728534 shutdown_connections 2021-07-11T03:56:51.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 -- 172.21.2.104:0/1224728534 >> 172.21.2.104:0/1224728534 conn(0x7f9cc80b64d0 msgr2=0x7f9cc80b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:51.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 -- 172.21.2.104:0/1224728534 shutdown_connections 2021-07-11T03:56:51.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.981+0000 7f9ccfb56700 1 -- 172.21.2.104:0/1224728534 wait complete. 2021-07-11T03:56:51.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccfb56700 1 Processor -- start 2021-07-11T03:56:51.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccfb56700 1 -- start start 2021-07-11T03:56:51.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccfb56700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cc8065b30 0x7f9cc8142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccfb56700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cc813ef00 0x7f9cc813fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccfb56700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cc813c6d0 0x7f9cc813cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccfb56700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc8141890 con 0x7f9cc8065b30 2021-07-11T03:56:51.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccfb56700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc8143890 con 0x7f9cc813ef00 2021-07-11T03:56:51.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccfb56700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc8065f10 con 0x7f9cc813c6d0 2021-07-11T03:56:51.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccd8f2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cc8065b30 0x7f9cc8142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccd8f2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cc8065b30 0x7f9cc8142730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44790/0 (socket says 172.21.2.104:44790) 2021-07-11T03:56:51.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccd8f2700 1 -- 172.21.2.104:0/526742202 learned_addr learned my addr 172.21.2.104:0/526742202 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:51.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccd8f2700 1 -- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cc813c6d0 msgr2=0x7f9cc813cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccd8f2700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cc813c6d0 0x7f9cc813cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccd8f2700 1 -- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cc813ef00 msgr2=0x7f9cc813fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccd8f2700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cc813ef00 0x7f9cc813fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccd8f2700 1 -- 172.21.2.104:0/526742202 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc8142c20 con 0x7f9cc8065b30 2021-07-11T03:56:51.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccd8f2700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cc8065b30 0x7f9cc8142730 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f9cbc001370 tx=0x7f9cbc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9cbaffd700 1 -- 172.21.2.104:0/526742202 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cbc00cb60 con 0x7f9cc8065b30 2021-07-11T03:56:51.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9cbaffd700 1 -- 172.21.2.104:0/526742202 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc00ccc0 con 0x7f9cc8065b30 2021-07-11T03:56:51.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9cbaffd700 1 -- 172.21.2.104:0/526742202 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cbc0045e0 con 0x7f9cc8065b30 2021-07-11T03:56:51.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccfb56700 1 -- 172.21.2.104:0/526742202 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc8140ac0 con 0x7f9cc8065b30 2021-07-11T03:56:51.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccfb56700 1 -- 172.21.2.104:0/526742202 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc8140010 con 0x7f9cc8065b30 2021-07-11T03:56:51.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccfb56700 1 -- 172.21.2.104:0/526742202 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc8057350 con 0x7f9cc8065b30 2021-07-11T03:56:51.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9cbaffd700 1 -- 172.21.2.104:0/526742202 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9cbc010070 con 0x7f9cc8065b30 2021-07-11T03:56:51.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9cbaffd700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ca80531f0 0x7f9ca80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9cbaffd700 1 -- 172.21.2.104:0/526742202 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9cbc06bc60 con 0x7f9cc8065b30 2021-07-11T03:56:51.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccd0f1700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ca80531f0 0x7f9ca80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.985+0000 7f9ccd0f1700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ca80531f0 0x7f9ca80556b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f9cc400a000 tx=0x7f9cc4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:50.989+0000 7f9cbaffd700 1 -- 172.21.2.104:0/526742202 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9cbc042ea0 con 0x7f9cc8065b30 2021-07-11T03:56:51.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 -- 172.21.2.104:0/526742202 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cc8140c50 con 0x7f9cc8065b30 2021-07-11T03:56:51.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9cbaffd700 1 -- 172.21.2.104:0/526742202 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16040 (secure 0 0 0) 0x7f9cbc024030 con 0x7f9cc8065b30 2021-07-11T03:56:51.079 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:51.079 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":55,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:56:50.701936+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:56:48.052820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-07-11T03:56:48.670109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:56:51.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 -- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ca80531f0 msgr2=0x7f9ca80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ca80531f0 0x7f9ca80556b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f9cc400a000 tx=0x7f9cc4008040).stop 2021-07-11T03:56:51.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 -- 172.21.2.104:0/526742202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cc8065b30 msgr2=0x7f9cc8142730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cc8065b30 0x7f9cc8142730 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f9cbc001370 tx=0x7f9cbc015000).stop 2021-07-11T03:56:51.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 -- 172.21.2.104:0/526742202 shutdown_connections 2021-07-11T03:56:51.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cc813c6d0 0x7f9cc813cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cc813ef00 0x7f9cc813fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ca80531f0 0x7f9ca80556b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 --2- 172.21.2.104:0/526742202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cc8065b30 0x7f9cc8142730 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 -- 172.21.2.104:0/526742202 >> 172.21.2.104:0/526742202 conn(0x7f9cc80b64d0 msgr2=0x7f9cc8060910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:51.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 -- 172.21.2.104:0/526742202 shutdown_connections 2021-07-11T03:56:51.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.073+0000 7f9ccfb56700 1 -- 172.21.2.104:0/526742202 wait complete. 2021-07-11T03:56:51.087 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T03:56:51.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 Processor -- start 2021-07-11T03:56:51.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 -- start start 2021-07-11T03:56:51.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd7400973d0 0x7fd74005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd74013f560 0x7fd74013f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7400b2a40 con 0x7fd7400b2660 2021-07-11T03:56:51.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd740097b80 con 0x7fd7400973d0 2021-07-11T03:56:51.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd740097e20 con 0x7fd74013f560 2021-07-11T03:56:51.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd7457fd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd7400973d0 0x7fd74005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd7457fd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd7400973d0 0x7fd74005f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44800/0 (socket says 172.21.2.104:44800) 2021-07-11T03:56:51.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd7457fd700 1 -- 172.21.2.104:0/620676871 learned_addr learned my addr 172.21.2.104:0/620676871 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:51.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd7457fd700 1 -- 172.21.2.104:0/620676871 >> v2:172.21.2.105:3300/0 conn(0x7fd74013f560 msgr2=0x7fd74013f9b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:56:51.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd7457fd700 1 --2- 172.21.2.104:0/620676871 >> v2:172.21.2.105:3300/0 conn(0x7fd74013f560 0x7fd74013f9b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd7457fd700 1 -- 172.21.2.104:0/620676871 >> v1:172.21.2.105:6789/0 conn(0x7fd7400b2660 legacy=0x7fd740096cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:56:51.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd7457fd700 1 -- 172.21.2.104:0/620676871 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7400980c0 con 0x7fd7400973d0 2021-07-11T03:56:51.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd7457fd700 1 --2- 172.21.2.104:0/620676871 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7400973d0 0x7fd74005f6b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fd7400b3350 tx=0x7fd73c006e40).ready entity=mon.0 client_cookie=8fbeba84a69616ca server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd744ffc700 1 -- 172.21.2.104:0/620676871 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd73c00bcb0 con 0x7fd7400973d0 2021-07-11T03:56:51.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd744ffc700 1 -- 172.21.2.104:0/620676871 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c000e90 con 0x7fd7400973d0 2021-07-11T03:56:51.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd744ffc700 1 -- 172.21.2.104:0/620676871 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd73c0155e0 con 0x7fd7400973d0 2021-07-11T03:56:51.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 -- 172.21.2.104:0/620676871 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7400973d0 msgr2=0x7fd74005f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 --2- 172.21.2.104:0/620676871 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7400973d0 0x7fd74005f6b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fd7400b3350 tx=0x7fd73c006e40).stop 2021-07-11T03:56:51.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 -- 172.21.2.104:0/620676871 shutdown_connections 2021-07-11T03:56:51.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 --2- 172.21.2.104:0/620676871 >> v2:172.21.2.105:3300/0 conn(0x7fd74013f560 0x7fd74013f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 --2- 172.21.2.104:0/620676871 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7400973d0 0x7fd74005f6b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 -- 172.21.2.104:0/620676871 >> 172.21.2.104:0/620676871 conn(0x7fd740062f10 msgr2=0x7fd7400632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:51.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 -- 172.21.2.104:0/620676871 shutdown_connections 2021-07-11T03:56:51.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 -- 172.21.2.104:0/620676871 wait complete. 2021-07-11T03:56:51.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 Processor -- start 2021-07-11T03:56:51.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.141+0000 7fd748262700 1 -- start start 2021-07-11T03:56:51.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd748262700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7400973d0 0x7fd74009af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd748262700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7400b2660 0x7fd74013e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd748262700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74013f560 0x7fd74013ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd748262700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74014e220 con 0x7fd7400973d0 2021-07-11T03:56:51.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd748262700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400ade60 con 0x7fd7400b2660 2021-07-11T03:56:51.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd748262700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400adbc0 con 0x7fd74013f560 2021-07-11T03:56:51.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd745ffe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7400973d0 0x7fd74009af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd745ffe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7400973d0 0x7fd74009af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44804/0 (socket says 172.21.2.104:44804) 2021-07-11T03:56:51.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd745ffe700 1 -- 172.21.2.104:0/204685007 learned_addr learned my addr 172.21.2.104:0/204685007 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:51.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd745ffe700 1 -- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74013f560 msgr2=0x7fd74013ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd745ffe700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74013f560 0x7fd74013ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd745ffe700 1 -- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7400b2660 msgr2=0x7fd74013e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd745ffe700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7400b2660 0x7fd74013e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd745ffe700 1 -- 172.21.2.104:0/204685007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c00d4d0 con 0x7fd7400973d0 2021-07-11T03:56:51.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd745ffe700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7400973d0 0x7fd74009af60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fd7401517a0 tx=0x7fd734006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd732ffd700 1 -- 172.21.2.104:0/204685007 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd734010960 con 0x7fd7400973d0 2021-07-11T03:56:51.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd732ffd700 1 -- 172.21.2.104:0/204685007 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd734010ac0 con 0x7fd7400973d0 2021-07-11T03:56:51.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd732ffd700 1 -- 172.21.2.104:0/204685007 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd734010e20 con 0x7fd7400973d0 2021-07-11T03:56:51.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd748262700 1 -- 172.21.2.104:0/204685007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7400aece0 con 0x7fd7400973d0 2021-07-11T03:56:51.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd748262700 1 -- 172.21.2.104:0/204685007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74013f160 con 0x7fd7400973d0 2021-07-11T03:56:51.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd748262700 1 -- 172.21.2.104:0/204685007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7401517e0 con 0x7fd7400973d0 2021-07-11T03:56:51.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd732ffd700 1 -- 172.21.2.104:0/204685007 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd7340193d0 con 0x7fd7400973d0 2021-07-11T03:56:51.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd732ffd700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd720053260 0x7fd720055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd732ffd700 1 -- 172.21.2.104:0/204685007 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd734015020 con 0x7fd7400973d0 2021-07-11T03:56:51.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd7457fd700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd720053260 0x7fd720055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.145+0000 7fd7457fd700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd720053260 0x7fd720055720 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fd73c006fc0 tx=0x7fd73c018000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.149+0000 7fd732ffd700 1 -- 172.21.2.104:0/204685007 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd734046680 con 0x7fd7400973d0 2021-07-11T03:56:51.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.257+0000 7fd748262700 1 -- 172.21.2.104:0/204685007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd740057350 con 0x7fd7400973d0 2021-07-11T03:56:51.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.261+0000 7fd732ffd700 1 -- 172.21.2.104:0/204685007 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd734049ca0 con 0x7fd7400973d0 2021-07-11T03:56:51.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.261+0000 7fd748262700 1 -- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd720053260 msgr2=0x7fd720055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.261+0000 7fd748262700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd720053260 0x7fd720055720 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fd73c006fc0 tx=0x7fd73c018000).stop 2021-07-11T03:56:51.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.265+0000 7fd748262700 1 -- 172.21.2.104:0/204685007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7400973d0 msgr2=0x7fd74009af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.265+0000 7fd748262700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7400973d0 0x7fd74009af60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fd7401517a0 tx=0x7fd734006e10).stop 2021-07-11T03:56:51.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.265+0000 7fd748262700 1 -- 172.21.2.104:0/204685007 shutdown_connections 2021-07-11T03:56:51.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.265+0000 7fd748262700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74013f560 0x7fd74013ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.265+0000 7fd748262700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7400b2660 0x7fd74013e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.265+0000 7fd748262700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd720053260 0x7fd720055720 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.265+0000 7fd748262700 1 --2- 172.21.2.104:0/204685007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7400973d0 0x7fd74009af60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.265+0000 7fd748262700 1 -- 172.21.2.104:0/204685007 >> 172.21.2.104:0/204685007 conn(0x7fd740062f10 msgr2=0x7fd7400b5280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:51.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.265+0000 7fd748262700 1 -- 172.21.2.104:0/204685007 shutdown_connections 2021-07-11T03:56:51.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.265+0000 7fd748262700 1 -- 172.21.2.104:0/204685007 wait complete. 2021-07-11T03:56:51.273 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T03:56:51.276 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T03:56:51.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 Processor -- start 2021-07-11T03:56:51.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 -- start start 2021-07-11T03:56:51.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2e64067180 0x7f2e6405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2e640b1970 0x7f2e6405fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e640a7a90 con 0x7f2e640a76b0 2021-07-11T03:56:51.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e64147620 con 0x7f2e640b1970 2021-07-11T03:56:51.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e641478c0 con 0x7f2e64067180 2021-07-11T03:56:51.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e62ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f2e640a76b0 0x7f2e64066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41580/0 (socket says 172.21.2.104:41580) 2021-07-11T03:56:51.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e62ffd700 1 -- 172.21.2.104:0/4170392357 learned_addr learned my addr 172.21.2.104:0/4170392357 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:51.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e637fe700 1 --2- 172.21.2.104:0/4170392357 >> v2:172.21.2.104:3300/0 conn(0x7f2e640b1970 0x7f2e6405fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e627fc700 1 --2- 172.21.2.104:0/4170392357 >> v2:172.21.2.105:3300/0 conn(0x7f2e64067180 0x7f2e6405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e637fe700 1 -- 172.21.2.104:0/4170392357 >> v2:172.21.2.105:3300/0 conn(0x7f2e64067180 msgr2=0x7f2e6405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:51.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e637fe700 1 --2- 172.21.2.104:0/4170392357 >> v2:172.21.2.105:3300/0 conn(0x7f2e64067180 0x7f2e6405f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e637fe700 1 -- 172.21.2.104:0/4170392357 >> v1:172.21.2.104:6789/0 conn(0x7f2e640a76b0 legacy=0x7f2e64066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e637fe700 1 -- 172.21.2.104:0/4170392357 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e64147b60 con 0x7f2e640b1970 2021-07-11T03:56:51.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e637fe700 1 --2- 172.21.2.104:0/4170392357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e640b1970 0x7f2e6405fba0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f2e5c0032f0 tx=0x7f2e5c0091d0).ready entity=mon.0 client_cookie=f73daf6c889d0a9b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e61ffb700 1 -- 172.21.2.104:0/4170392357 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e5c009ea0 con 0x7f2e640b1970 2021-07-11T03:56:51.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e61ffb700 1 -- 172.21.2.104:0/4170392357 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c00b870 con 0x7f2e640b1970 2021-07-11T03:56:51.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e61ffb700 1 -- 172.21.2.104:0/4170392357 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e5c00ba20 con 0x7f2e640b1970 2021-07-11T03:56:51.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 -- 172.21.2.104:0/4170392357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e640b1970 msgr2=0x7f2e6405fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 --2- 172.21.2.104:0/4170392357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e640b1970 0x7f2e6405fba0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f2e5c0032f0 tx=0x7f2e5c0091d0).stop 2021-07-11T03:56:51.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 -- 172.21.2.104:0/4170392357 shutdown_connections 2021-07-11T03:56:51.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 --2- 172.21.2.104:0/4170392357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e640b1970 0x7f2e6405fba0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 --2- 172.21.2.104:0/4170392357 >> v2:172.21.2.105:3300/0 conn(0x7f2e64067180 0x7f2e6405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 -- 172.21.2.104:0/4170392357 >> 172.21.2.104:0/4170392357 conn(0x7f2e640b53e0 msgr2=0x7f2e640b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:51.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 -- 172.21.2.104:0/4170392357 shutdown_connections 2021-07-11T03:56:51.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 -- 172.21.2.104:0/4170392357 wait complete. 2021-07-11T03:56:51.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.325+0000 7f2e6933d700 1 Processor -- start 2021-07-11T03:56:51.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e6933d700 1 -- start start 2021-07-11T03:56:51.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e6933d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64067180 0x7f2e640abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e6933d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e640a76b0 0x7f2e640adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e6933d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e640b1970 0x7f2e640ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e6933d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640aa700 con 0x7f2e64067180 2021-07-11T03:56:51.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e6933d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640aa4d0 con 0x7f2e640a76b0 2021-07-11T03:56:51.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e6933d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640aa270 con 0x7f2e640b1970 2021-07-11T03:56:51.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e62ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64067180 0x7f2e640abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e62ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64067180 0x7f2e640abaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44818/0 (socket says 172.21.2.104:44818) 2021-07-11T03:56:51.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e62ffd700 1 -- 172.21.2.104:0/3367829351 learned_addr learned my addr 172.21.2.104:0/3367829351 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:51.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e62ffd700 1 -- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e640b1970 msgr2=0x7f2e640ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e62ffd700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e640b1970 0x7f2e640ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e62ffd700 1 -- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e640a76b0 msgr2=0x7f2e640adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e62ffd700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e640a76b0 0x7f2e640adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e62ffd700 1 -- 172.21.2.104:0/3367829351 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c009470 con 0x7f2e64067180 2021-07-11T03:56:51.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e62ffd700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64067180 0x7f2e640abaf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f2e6406a0e0 tx=0x7f2e58007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e53fff700 1 -- 172.21.2.104:0/3367829351 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e5800ce30 con 0x7f2e64067180 2021-07-11T03:56:51.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e53fff700 1 -- 172.21.2.104:0/3367829351 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e580072e0 con 0x7f2e64067180 2021-07-11T03:56:51.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e53fff700 1 -- 172.21.2.104:0/3367829351 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e58005800 con 0x7f2e64067180 2021-07-11T03:56:51.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e6933d700 1 -- 172.21.2.104:0/3367829351 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e640aa930 con 0x7f2e64067180 2021-07-11T03:56:51.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e6933d700 1 -- 172.21.2.104:0/3367829351 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e640ab340 con 0x7f2e64067180 2021-07-11T03:56:51.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e6933d700 1 -- 172.21.2.104:0/3367829351 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e64057350 con 0x7f2e64067180 2021-07-11T03:56:51.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e53fff700 1 -- 172.21.2.104:0/3367829351 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2e58010070 con 0x7f2e64067180 2021-07-11T03:56:51.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e53fff700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e40053270 0x7f2e40055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e53fff700 1 -- 172.21.2.104:0/3367829351 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2e5806aae0 con 0x7f2e64067180 2021-07-11T03:56:51.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e627fc700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e40053270 0x7f2e40055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.329+0000 7f2e627fc700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e40053270 0x7f2e40055730 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f2e5400a000 tx=0x7f2e54008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.333+0000 7f2e53fff700 1 -- 172.21.2.104:0/3367829351 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2e58041e20 con 0x7f2e64067180 2021-07-11T03:56:51.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.445+0000 7f2e6933d700 1 -- 172.21.2.104:0/3367829351 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2e640af210 con 0x7f2e64067180 2021-07-11T03:56:51.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e53fff700 1 -- 172.21.2.104:0/3367829351 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v18) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2e58018030 con 0x7f2e64067180 2021-07-11T03:56:51.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 -- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e40053270 msgr2=0x7f2e40055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e40053270 0x7f2e40055730 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f2e5400a000 tx=0x7f2e54008040).stop 2021-07-11T03:56:51.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 -- 172.21.2.104:0/3367829351 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64067180 msgr2=0x7f2e640abaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64067180 0x7f2e640abaf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f2e6406a0e0 tx=0x7f2e58007c00).stop 2021-07-11T03:56:51.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 -- 172.21.2.104:0/3367829351 shutdown_connections 2021-07-11T03:56:51.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e640b1970 0x7f2e640ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e640a76b0 0x7f2e640adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:51.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e40053270 0x7f2e40055730 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 --2- 172.21.2.104:0/3367829351 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e64067180 0x7f2e640abaf0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 -- 172.21.2.104:0/3367829351 >> 172.21.2.104:0/3367829351 conn(0x7f2e640b53e0 msgr2=0x7f2e6405fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:51.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 -- 172.21.2.104:0/3367829351 shutdown_connections 2021-07-11T03:56:51.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.449+0000 7f2e6933d700 1 -- 172.21.2.104:0/3367829351 wait complete. 2021-07-11T03:56:51.460 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T03:56:51.463 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:56:51.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 Processor -- start 2021-07-11T03:56:51.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 -- start start 2021-07-11T03:56:51.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f09c00b7810 0x7f09c01445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c00b1170 0x7f09c0144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c00b31d0 con 0x7f09c0064870 2021-07-11T03:56:51.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c00b7bf0 con 0x7f09c00b7810 2021-07-11T03:56:51.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00675c0 con 0x7f09c00b1170 2021-07-11T03:56:51.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c6d90700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f09c0064870 0x7f09c0064cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41598/0 (socket says 172.21.2.104:41598) 2021-07-11T03:56:51.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c6d90700 1 -- 172.21.2.104:0/3701220246 learned_addr learned my addr 172.21.2.104:0/3701220246 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:51.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313335142 0 0) 0x7f09c00b31d0 con 0x7f09c0064870 2021-07-11T03:56:51.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09a8003260 con 0x7f09c0064870 2021-07-11T03:56:51.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c5d8e700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c00b1170 0x7f09c0144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c658f700 1 --2- 172.21.2.104:0/3701220246 >> v2:172.21.2.105:3300/0 conn(0x7f09c00b7810 0x7f09c01445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3671434661 0 0) 0x7f09a8003260 con 0x7f09c0064870 2021-07-11T03:56:51.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c00b1170 msgr2=0x7f09c0144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:51.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c00b1170 0x7f09c0144a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 >> v2:172.21.2.105:3300/0 conn(0x7f09c00b7810 msgr2=0x7f09c01445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:51.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 --2- 172.21.2.104:0/3701220246 >> v2:172.21.2.105:3300/0 conn(0x7f09c00b7810 0x7f09c01445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c00aae70 con 0x7f09c0064870 2021-07-11T03:56:51.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f09b80019e0 con 0x7f09c0064870 2021-07-11T03:56:51.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 >> v1:172.21.2.104:6789/0 conn(0x7f09c0064870 legacy=0x7f09c0064cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c00b1170 0x7f09a80037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09a8003e00 0x7f09a8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09a80049f0 0x7f09a8004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00b7bf0 con 0x7f09a80049f0 2021-07-11T03:56:51.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00675c0 con 0x7f09a8003e00 2021-07-11T03:56:51.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a800b8e0 con 0x7f09c00b1170 2021-07-11T03:56:51.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c6d90700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09a80049f0 0x7f09a8004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c5d8e700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09a8003e00 0x7f09a8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c658f700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c00b1170 0x7f09a80037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c6d90700 1 -- 172.21.2.104:0/3701220246 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c00b1170 msgr2=0x7f09a80037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:51.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c6d90700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c00b1170 0x7f09a80037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c6d90700 1 -- 172.21.2.104:0/3701220246 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09a8003e00 msgr2=0x7f09a8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:51.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c6d90700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09a8003e00 0x7f09a8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c6d90700 1 -- 172.21.2.104:0/3701220246 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a8003260 con 0x7f09a80049f0 2021-07-11T03:56:51.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c6d90700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09a80049f0 0x7f09a8004e70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f09b80016f0 tx=0x7f09c0066a60).ready entity=mon.0 client_cookie=4796a47e47b95673 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09b800cc50 con 0x7f09a80049f0 2021-07-11T03:56:51.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b800cdb0 con 0x7f09a80049f0 2021-07-11T03:56:51.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c558d700 1 -- 172.21.2.104:0/3701220246 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09b8005b80 con 0x7f09a80049f0 2021-07-11T03:56:51.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 -- 172.21.2.104:0/3701220246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09a80049f0 msgr2=0x7f09a8004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 --2- 172.21.2.104:0/3701220246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09a80049f0 0x7f09a8004e70 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f09b80016f0 tx=0x7f09c0066a60).stop 2021-07-11T03:56:51.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c6d90700 1 -- 172.21.2.104:0/3701220246 reap_dead start 2021-07-11T03:56:51.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 -- 172.21.2.104:0/3701220246 shutdown_connections 2021-07-11T03:56:51.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 -- 172.21.2.104:0/3701220246 >> 172.21.2.104:0/3701220246 conn(0x7f09c005f6b0 msgr2=0x7f09c005fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:51.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 -- 172.21.2.104:0/3701220246 shutdown_connections 2021-07-11T03:56:51.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 -- 172.21.2.104:0/3701220246 wait complete. 2021-07-11T03:56:51.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 Processor -- start 2021-07-11T03:56:51.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.509+0000 7f09c87f3700 1 -- start start 2021-07-11T03:56:51.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c87f3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c0144500 0x7f09c00ac5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c87f3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09c009e8b0 0x7f09c00b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c87f3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09c00aa1a0 0x7f09c01500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c87f3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c00ad160 con 0x7f09c009e8b0 2021-07-11T03:56:51.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c87f3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01448e0 con 0x7f09c00aa1a0 2021-07-11T03:56:51.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c87f3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c01540c0 con 0x7f09c0144500 2021-07-11T03:56:51.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c5d8e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09c009e8b0 0x7f09c00b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c5d8e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09c009e8b0 0x7f09c00b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44840/0 (socket says 172.21.2.104:44840) 2021-07-11T03:56:51.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c5d8e700 1 -- 172.21.2.104:0/390738088 learned_addr learned my addr 172.21.2.104:0/390738088 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:51.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c658f700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c0144500 0x7f09c00ac5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c6d90700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09c00aa1a0 0x7f09c01500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c5d8e700 1 -- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c0144500 msgr2=0x7f09c00ac5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c5d8e700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c0144500 0x7f09c00ac5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c5d8e700 1 -- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09c00aa1a0 msgr2=0x7f09c01500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c5d8e700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09c00aa1a0 0x7f09c01500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c5d8e700 1 -- 172.21.2.104:0/390738088 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c0154360 con 0x7f09c009e8b0 2021-07-11T03:56:51.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c5d8e700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09c009e8b0 0x7f09c00b02f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f09bc00c000 tx=0x7f09bc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09af7fe700 1 -- 172.21.2.104:0/390738088 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09bc012800 con 0x7f09c009e8b0 2021-07-11T03:56:51.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09af7fe700 1 -- 172.21.2.104:0/390738088 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc012960 con 0x7f09c009e8b0 2021-07-11T03:56:51.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c87f3700 1 -- 172.21.2.104:0/390738088 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09c00ad390 con 0x7f09c009e8b0 2021-07-11T03:56:51.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c87f3700 1 -- 172.21.2.104:0/390738088 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c00b07e0 con 0x7f09c009e8b0 2021-07-11T03:56:51.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09af7fe700 1 -- 172.21.2.104:0/390738088 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09bc012cc0 con 0x7f09c009e8b0 2021-07-11T03:56:51.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c87f3700 1 -- 172.21.2.104:0/390738088 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c00ad520 con 0x7f09c009e8b0 2021-07-11T03:56:51.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09af7fe700 1 -- 172.21.2.104:0/390738088 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f09bc014470 con 0x7f09c009e8b0 2021-07-11T03:56:51.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09af7fe700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09a00531a0 0x7f09a0055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:51.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09af7fe700 1 -- 172.21.2.104:0/390738088 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f09bc06be80 con 0x7f09c009e8b0 2021-07-11T03:56:51.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c658f700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09a00531a0 0x7f09a0055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:51.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.513+0000 7f09c658f700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09a00531a0 0x7f09a0055660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f09b000c000 tx=0x7f09b0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:51.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.517+0000 7f09af7fe700 1 -- 172.21.2.104:0/390738088 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f09bc043050 con 0x7f09c009e8b0 2021-07-11T03:56:51.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 -- 172.21.2.104:0/390738088 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09c0057350 con 0x7f09c009e8b0 2021-07-11T03:56:51.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09af7fe700 1 -- 172.21.2.104:0/390738088 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f09bc046670 con 0x7f09c009e8b0 2021-07-11T03:56:51.632 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:51.632 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":22,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3427300973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3427300973},{"type":"v1","addr":"172.21.2.105:6835","nonce":3427300973}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/2823255258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2823255258},{"type":"v1","addr":"172.21.2.104:6833","nonce":2823255258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:56:49.697379+0000","modified":"2021-07-11T03:56:50.700784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/2428941897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2428941897},{"type":"v1","addr":"172.21.2.104:6835","nonce":2428941897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-07-11T03:56:51.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 -- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09a00531a0 msgr2=0x7f09a0055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09a00531a0 0x7f09a0055660 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f09b000c000 tx=0x7f09b0008040).stop 2021-07-11T03:56:51.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 -- 172.21.2.104:0/390738088 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09c009e8b0 msgr2=0x7f09c00b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:51.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09c009e8b0 0x7f09c00b02f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7f09bc00c000 tx=0x7f09bc008040).stop 2021-07-11T03:56:51.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 -- 172.21.2.104:0/390738088 shutdown_connections 2021-07-11T03:56:51.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09c00aa1a0 0x7f09c01500a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09a00531a0 0x7f09a0055660 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09c009e8b0 0x7f09c00b02f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 --2- 172.21.2.104:0/390738088 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09c0144500 0x7f09c00ac5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:51.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 -- 172.21.2.104:0/390738088 >> 172.21.2.104:0/390738088 conn(0x7f09c005f6b0 msgr2=0x7f09c005ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:51.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 -- 172.21.2.104:0/390738088 shutdown_connections 2021-07-11T03:56:51.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:51.625+0000 7f09c87f3700 1 -- 172.21.2.104:0/390738088 wait complete. 2021-07-11T03:56:51.636 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 22 2021-07-11T03:56:51.941 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:56:49.697379+0000', 'modified': '2021-07-11T03:56:50.700784+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5589': {'gid': 5589, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.104:6835/2428941897', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 2428941897}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 2428941897}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:56:51.942 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:56:51.942 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:56:52.026 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":5589,"want_state":"up:active","state":"up:active","rank_uptime":2.3200706759999998,"mdsmap_epoch":22,"osdmap_epoch":55,"osdmap_epoch_barrier":54,"uptime":6.0457841480000001} 2021-07-11T03:56:52.026 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 5589, "mdsmap_epoch": 22, "osdmap_epoch": 55, "osdmap_epoch_barrier": 54, "rank_uptime": 2.320070676, "state": "up:active", "uptime": 6.045784148, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:56:52.027 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:56:52.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.073+0000 7f79d8a69700 1 Processor -- start 2021-07-11T03:56:52.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.073+0000 7f79d8a69700 1 -- start start 2021-07-11T03:56:52.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.073+0000 7f79d8a69700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f79d40b2660 0x7f79d405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.073+0000 7f79d8a69700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f79d4096cc0 0x7f79d4097100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.073+0000 7f79d8a69700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d40b2a40 con 0x7f79d405c490 2021-07-11T03:56:52.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.073+0000 7f79d8a69700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d40975f0 con 0x7f79d40b2660 2021-07-11T03:56:52.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.073+0000 7f79d8a69700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d4097b80 con 0x7f79d4096cc0 2021-07-11T03:56:52.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.073+0000 7f79d259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f79d40b2660 0x7f79d405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.073+0000 7f79d2d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f79d405c490 0x7f79d413f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41616/0 (socket says 172.21.2.104:41616) 2021-07-11T03:56:52.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.073+0000 7f79d2d9d700 1 -- 172.21.2.104:0/1921273072 learned_addr learned my addr 172.21.2.104:0/1921273072 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:52.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 -- 172.21.2.104:0/1921273072 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f79d4096cc0 msgr2=0x7f79d4097100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:52.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 --2- 172.21.2.104:0/1921273072 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f79d4096cc0 0x7f79d4097100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 -- 172.21.2.104:0/1921273072 >> v1:172.21.2.104:6789/0 conn(0x7f79d405c490 legacy=0x7f79d413f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 -- 172.21.2.104:0/1921273072 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4097e20 con 0x7f79d40b2660 2021-07-11T03:56:52.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 --2- 172.21.2.104:0/1921273072 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f79d40b2660 0x7f79d405f6b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f79bc002f70 tx=0x7f79bc006df0).ready entity=mon.0 client_cookie=f231d872842ef32a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d159a700 1 -- 172.21.2.104:0/1921273072 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79bc00d570 con 0x7f79d40b2660 2021-07-11T03:56:52.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d159a700 1 -- 172.21.2.104:0/1921273072 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc000e90 con 0x7f79d40b2660 2021-07-11T03:56:52.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d159a700 1 -- 172.21.2.104:0/1921273072 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79bc00d8c0 con 0x7f79d40b2660 2021-07-11T03:56:52.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 -- 172.21.2.104:0/1921273072 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f79d40b2660 msgr2=0x7f79d405f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 --2- 172.21.2.104:0/1921273072 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f79d40b2660 0x7f79d405f6b0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f79bc002f70 tx=0x7f79bc006df0).stop 2021-07-11T03:56:52.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 -- 172.21.2.104:0/1921273072 shutdown_connections 2021-07-11T03:56:52.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 --2- 172.21.2.104:0/1921273072 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f79d4096cc0 0x7f79d4097100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 --2- 172.21.2.104:0/1921273072 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f79d40b2660 0x7f79d405f6b0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 -- 172.21.2.104:0/1921273072 >> 172.21.2.104:0/1921273072 conn(0x7f79d4062f10 msgr2=0x7f79d40632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 -- 172.21.2.104:0/1921273072 shutdown_connections 2021-07-11T03:56:52.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 -- 172.21.2.104:0/1921273072 wait complete. 2021-07-11T03:56:52.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 Processor -- start 2021-07-11T03:56:52.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 -- start start 2021-07-11T03:56:52.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f79d405c490 0x7f79d414a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f79d4096cc0 0x7f79d409b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f79d40b2660 0x7f79d40af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40afe50 con 0x7f79d405c490 2021-07-11T03:56:52.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40aeab0 con 0x7f79d40b2660 2021-07-11T03:56:52.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40aed50 con 0x7f79d4096cc0 2021-07-11T03:56:52.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f79d405c490 0x7f79d414a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f79d405c490 0x7f79d414a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44856/0 (socket says 172.21.2.104:44856) 2021-07-11T03:56:52.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 -- 172.21.2.104:0/3904763535 learned_addr learned my addr 172.21.2.104:0/3904763535 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:52.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 -- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f79d4096cc0 msgr2=0x7f79d409b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f79d4096cc0 0x7f79d409b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 -- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f79d40b2660 msgr2=0x7f79d40af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f79d40b2660 0x7f79d40af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 -- 172.21.2.104:0/3904763535 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc00b180 con 0x7f79d405c490 2021-07-11T03:56:52.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d259c700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f79d405c490 0x7f79d414a140 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f79bc002f70 tx=0x7f79bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79cb7fe700 1 -- 172.21.2.104:0/3904763535 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79bc021070 con 0x7f79d405c490 2021-07-11T03:56:52.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79cb7fe700 1 -- 172.21.2.104:0/3904763535 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc000e90 con 0x7f79d405c490 2021-07-11T03:56:52.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 -- 172.21.2.104:0/3904763535 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d40aeff0 con 0x7f79d405c490 2021-07-11T03:56:52.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79d8a69700 1 -- 172.21.2.104:0/3904763535 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d4148fd0 con 0x7f79d405c490 2021-07-11T03:56:52.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.077+0000 7f79cb7fe700 1 -- 172.21.2.104:0/3904763535 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f79bc011e90 con 0x7f79d405c490 2021-07-11T03:56:52.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.081+0000 7f79d8a69700 1 -- 172.21.2.104:0/3904763535 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d4057350 con 0x7f79d405c490 2021-07-11T03:56:52.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.081+0000 7f79cb7fe700 1 -- 172.21.2.104:0/3904763535 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f79bc00f070 con 0x7f79d405c490 2021-07-11T03:56:52.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.081+0000 7f79cb7fe700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f79b80532d0 0x7f79b8055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.081+0000 7f79cb7fe700 1 -- 172.21.2.104:0/3904763535 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f79bc06cab0 con 0x7f79d405c490 2021-07-11T03:56:52.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.081+0000 7f79d1d9b700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f79b80532d0 0x7f79b8055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.081+0000 7f79d1d9b700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f79b80532d0 0x7f79b8055790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f79c4002f70 tx=0x7f79c4006a90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.081+0000 7f79cb7fe700 1 -- 172.21.2.104:0/3904763535 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f79bc043ce0 con 0x7f79d405c490 2021-07-11T03:56:52.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.161+0000 7f79d8a69700 1 -- 172.21.2.104:0/3904763535 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79d40af180 con 0x7f79d405c490 2021-07-11T03:56:52.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.161+0000 7f79cb7fe700 1 -- 172.21.2.104:0/3904763535 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16040 (secure 0 0 0) 0x7f79bc047300 con 0x7f79d405c490 2021-07-11T03:56:52.170 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:52.170 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":55,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:56:50.701936+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:56:48.052820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-07-11T03:56:48.670109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:56:52.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 -- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f79b80532d0 msgr2=0x7f79b8055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f79b80532d0 0x7f79b8055790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f79c4002f70 tx=0x7f79c4006a90).stop 2021-07-11T03:56:52.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 -- 172.21.2.104:0/3904763535 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f79d405c490 msgr2=0x7f79d414a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f79d405c490 0x7f79d414a140 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f79bc002f70 tx=0x7f79bc015000).stop 2021-07-11T03:56:52.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 -- 172.21.2.104:0/3904763535 shutdown_connections 2021-07-11T03:56:52.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f79d40b2660 0x7f79d40af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f79d4096cc0 0x7f79d409b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f79b80532d0 0x7f79b8055790 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 --2- 172.21.2.104:0/3904763535 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f79d405c490 0x7f79d414a140 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 -- 172.21.2.104:0/3904763535 >> 172.21.2.104:0/3904763535 conn(0x7f79d4062f10 msgr2=0x7f79d4060210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 -- 172.21.2.104:0/3904763535 shutdown_connections 2021-07-11T03:56:52.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.165+0000 7f79d8a69700 1 -- 172.21.2.104:0/3904763535 wait complete. 2021-07-11T03:56:52.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:56:48.052820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:56:48.670109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:56:52.178 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:56:52.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 Processor -- start 2021-07-11T03:56:52.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 -- start start 2021-07-11T03:56:52.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd610144440 0x7fd610144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6100604e0 con 0x7fd610065420 2021-07-11T03:56:52.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6100b23b0 con 0x7fd610060100 2021-07-11T03:56:52.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd610065c80 con 0x7fd610144440 2021-07-11T03:56:52.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd60effd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd610144440 0x7fd610144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd60effd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd610144440 0x7fd610144970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44864/0 (socket says 172.21.2.104:44864) 2021-07-11T03:56:52.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd60ffff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd610065420 0x7fd61014b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41628/0 (socket says 172.21.2.104:41628) 2021-07-11T03:56:52.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd60effd700 1 -- 172.21.2.104:0/3466828865 learned_addr learned my addr 172.21.2.104:0/3466828865 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:52.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd60effd700 1 -- 172.21.2.104:0/3466828865 >> v1:172.21.2.105:6789/0 conn(0x7fd610060100 legacy=0x7fd610064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:52.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd60effd700 1 -- 172.21.2.104:0/3466828865 >> v1:172.21.2.104:6789/0 conn(0x7fd610065420 legacy=0x7fd61014b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd60effd700 1 -- 172.21.2.104:0/3466828865 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100b2650 con 0x7fd610144440 2021-07-11T03:56:52.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd60effd700 1 --2- 172.21.2.104:0/3466828865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd610144440 0x7fd610144970 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fd61005f160 tx=0x7fd5f8006eb0).ready entity=mon.0 client_cookie=f533f45b7713429a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd60e7fc700 1 -- 172.21.2.104:0/3466828865 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5f800d610 con 0x7fd610144440 2021-07-11T03:56:52.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd60e7fc700 1 -- 172.21.2.104:0/3466828865 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f8000e90 con 0x7fd610144440 2021-07-11T03:56:52.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd60e7fc700 1 -- 172.21.2.104:0/3466828865 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5f800d960 con 0x7fd610144440 2021-07-11T03:56:52.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 -- 172.21.2.104:0/3466828865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd610144440 msgr2=0x7fd610144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 --2- 172.21.2.104:0/3466828865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd610144440 0x7fd610144970 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fd61005f160 tx=0x7fd5f8006eb0).stop 2021-07-11T03:56:52.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 -- 172.21.2.104:0/3466828865 shutdown_connections 2021-07-11T03:56:52.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 --2- 172.21.2.104:0/3466828865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd610144440 0x7fd610144970 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 -- 172.21.2.104:0/3466828865 >> 172.21.2.104:0/3466828865 conn(0x7fd61005cf60 msgr2=0x7fd61005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 -- 172.21.2.104:0/3466828865 shutdown_connections 2021-07-11T03:56:52.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 -- 172.21.2.104:0/3466828865 wait complete. 2021-07-11T03:56:52.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 Processor -- start 2021-07-11T03:56:52.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.225+0000 7fd615f26700 1 -- start start 2021-07-11T03:56:52.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd615f26700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd610060100 0x7fd6100b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd615f26700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd610065420 0x7fd61013d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd615f26700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6100b0170 0x7fd61013d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd615f26700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61013c980 con 0x7fd610060100 2021-07-11T03:56:52.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd615f26700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61013b830 con 0x7fd6100b0170 2021-07-11T03:56:52.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd615f26700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61013bec0 con 0x7fd610065420 2021-07-11T03:56:52.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd610060100 0x7fd6100b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd610060100 0x7fd6100b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44868/0 (socket says 172.21.2.104:44868) 2021-07-11T03:56:52.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60f7fe700 1 -- 172.21.2.104:0/47579033 learned_addr learned my addr 172.21.2.104:0/47579033 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:52.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60f7fe700 1 -- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd610065420 msgr2=0x7fd61013d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60f7fe700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd610065420 0x7fd61013d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60f7fe700 1 -- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6100b0170 msgr2=0x7fd61013d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60f7fe700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6100b0170 0x7fd61013d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60f7fe700 1 -- 172.21.2.104:0/47579033 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f800b1e0 con 0x7fd610060100 2021-07-11T03:56:52.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60f7fe700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd610060100 0x7fd6100b64b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fd610152600 tx=0x7fd600006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60cff9700 1 -- 172.21.2.104:0/47579033 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd600009e60 con 0x7fd610060100 2021-07-11T03:56:52.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60cff9700 1 -- 172.21.2.104:0/47579033 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd600013460 con 0x7fd610060100 2021-07-11T03:56:52.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60cff9700 1 -- 172.21.2.104:0/47579033 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6000137c0 con 0x7fd610060100 2021-07-11T03:56:52.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd615f26700 1 -- 172.21.2.104:0/47579033 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6100b0640 con 0x7fd610060100 2021-07-11T03:56:52.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd615f26700 1 -- 172.21.2.104:0/47579033 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6100b1370 con 0x7fd610060100 2021-07-11T03:56:52.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd615f26700 1 -- 172.21.2.104:0/47579033 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6100b1760 con 0x7fd610060100 2021-07-11T03:56:52.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60cff9700 1 -- 172.21.2.104:0/47579033 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd60001a060 con 0x7fd610060100 2021-07-11T03:56:52.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60cff9700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5f40532d0 0x7fd5f4055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60cff9700 1 -- 172.21.2.104:0/47579033 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd60006ad70 con 0x7fd610060100 2021-07-11T03:56:52.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.229+0000 7fd60effd700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5f40532d0 0x7fd5f4055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.233+0000 7fd60effd700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5f40532d0 0x7fd5f4055790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fd61005f160 tx=0x7fd5f8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.233+0000 7fd60cff9700 1 -- 172.21.2.104:0/47579033 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd6000420b0 con 0x7fd610060100 2021-07-11T03:56:52.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd615f26700 1 -- 172.21.2.104:0/47579033 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd610057350 con 0x7fd610060100 2021-07-11T03:56:52.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd60cff9700 1 -- 172.21.2.104:0/47579033 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v55) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd600018070 con 0x7fd610060100 2021-07-11T03:56:52.321 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:56:52.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd615f26700 1 -- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5f40532d0 msgr2=0x7fd5f4055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd615f26700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5f40532d0 0x7fd5f4055790 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fd61005f160 tx=0x7fd5f8015000).stop 2021-07-11T03:56:52.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd615f26700 1 -- 172.21.2.104:0/47579033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd610060100 msgr2=0x7fd6100b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd615f26700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd610060100 0x7fd6100b64b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fd610152600 tx=0x7fd600006df0).stop 2021-07-11T03:56:52.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd615f26700 1 -- 172.21.2.104:0/47579033 shutdown_connections 2021-07-11T03:56:52.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd615f26700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd6100b0170 0x7fd61013d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd615f26700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd610065420 0x7fd61013d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd615f26700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5f40532d0 0x7fd5f4055790 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd615f26700 1 --2- 172.21.2.104:0/47579033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd610060100 0x7fd6100b64b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.313+0000 7fd615f26700 1 -- 172.21.2.104:0/47579033 >> 172.21.2.104:0/47579033 conn(0x7fd61005cf60 msgr2=0x7fd61013efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.317+0000 7fd615f26700 1 -- 172.21.2.104:0/47579033 shutdown_connections 2021-07-11T03:56:52.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.317+0000 7fd615f26700 1 -- 172.21.2.104:0/47579033 wait complete. 2021-07-11T03:56:52.328 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:56:52.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.381+0000 7fde24da9700 1 Processor -- start 2021-07-11T03:56:52.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.381+0000 7fde24da9700 1 -- start start 2021-07-11T03:56:52.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fde200b1ec0 0x7fde200b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde200b0420 con 0x7fde200affb0 2021-07-11T03:56:52.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde200a77d0 con 0x7fde200b22a0 2021-07-11T03:56:52.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde200b0650 con 0x7fde200b1ec0 2021-07-11T03:56:52.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fde200b1ec0 0x7fde200b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1dd9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fde200affb0 0x7fde200a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41642/0 (socket says 172.21.2.104:41642) 2021-07-11T03:56:52.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1dd9b700 1 -- 172.21.2.104:0/2274026013 learned_addr learned my addr 172.21.2.104:0/2274026013 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:52.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 -- 172.21.2.104:0/2274026013 >> v1:172.21.2.105:6789/0 conn(0x7fde200b22a0 legacy=0x7fde20141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:52.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 -- 172.21.2.104:0/2274026013 >> v1:172.21.2.104:6789/0 conn(0x7fde200affb0 legacy=0x7fde200a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 -- 172.21.2.104:0/2274026013 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde200b08f0 con 0x7fde200b1ec0 2021-07-11T03:56:52.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1d59a700 1 -- 172.21.2.104:0/2274026013 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903592433 0 0) 0x7fde200b0420 con 0x7fde200affb0 2021-07-11T03:56:52.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 --2- 172.21.2.104:0/2274026013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde200b1ec0 0x7fde200b12d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fde200b2f50 tx=0x7fde10006df0).ready entity=mon.0 client_cookie=f0a9460d2fcaafc0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1d59a700 1 -- 172.21.2.104:0/2274026013 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde1000d5c0 con 0x7fde200b1ec0 2021-07-11T03:56:52.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1d59a700 1 -- 172.21.2.104:0/2274026013 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde10000e90 con 0x7fde200b1ec0 2021-07-11T03:56:52.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1d59a700 1 -- 172.21.2.104:0/2274026013 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde1000d910 con 0x7fde200b1ec0 2021-07-11T03:56:52.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- 172.21.2.104:0/2274026013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde200b1ec0 msgr2=0x7fde200b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 --2- 172.21.2.104:0/2274026013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde200b1ec0 0x7fde200b12d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fde200b2f50 tx=0x7fde10006df0).stop 2021-07-11T03:56:52.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- 172.21.2.104:0/2274026013 shutdown_connections 2021-07-11T03:56:52.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 --2- 172.21.2.104:0/2274026013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde200b1ec0 0x7fde200b12d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- 172.21.2.104:0/2274026013 >> 172.21.2.104:0/2274026013 conn(0x7fde2005f6b0 msgr2=0x7fde2005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- 172.21.2.104:0/2274026013 shutdown_connections 2021-07-11T03:56:52.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- 172.21.2.104:0/2274026013 wait complete. 2021-07-11T03:56:52.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 Processor -- start 2021-07-11T03:56:52.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- start start 2021-07-11T03:56:52.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde200affb0 0x7fde200abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde200b1ec0 0x7fde200b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde200b22a0 0x7fde200b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde200a9420 con 0x7fde200affb0 2021-07-11T03:56:52.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde200ad920 con 0x7fde200b1ec0 2021-07-11T03:56:52.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde24da9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde200af5c0 con 0x7fde200b22a0 2021-07-11T03:56:52.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde200affb0 0x7fde200abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde200affb0 0x7fde200abcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44882/0 (socket says 172.21.2.104:44882) 2021-07-11T03:56:52.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 -- 172.21.2.104:0/3648832182 learned_addr learned my addr 172.21.2.104:0/3648832182 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:52.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 -- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde200b22a0 msgr2=0x7fde200b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde200b22a0 0x7fde200b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 -- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde200b1ec0 msgr2=0x7fde200b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde200b1ec0 0x7fde200b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 -- 172.21.2.104:0/3648832182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1000b180 con 0x7fde200affb0 2021-07-11T03:56:52.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.385+0000 7fde1e59c700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde200affb0 0x7fde200abcb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fde200b2f50 tx=0x7fde10012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde177fe700 1 -- 172.21.2.104:0/3648832182 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde10002f50 con 0x7fde200affb0 2021-07-11T03:56:52.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde177fe700 1 -- 172.21.2.104:0/3648832182 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde10000e90 con 0x7fde200affb0 2021-07-11T03:56:52.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde24da9700 1 -- 172.21.2.104:0/3648832182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde200ab680 con 0x7fde200affb0 2021-07-11T03:56:52.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde177fe700 1 -- 172.21.2.104:0/3648832182 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde1001d470 con 0x7fde200affb0 2021-07-11T03:56:52.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde24da9700 1 -- 172.21.2.104:0/3648832182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde200ac1a0 con 0x7fde200affb0 2021-07-11T03:56:52.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde24da9700 1 -- 172.21.2.104:0/3648832182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde200a9590 con 0x7fde200affb0 2021-07-11T03:56:52.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde177fe700 1 -- 172.21.2.104:0/3648832182 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fde1000f030 con 0x7fde200affb0 2021-07-11T03:56:52.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde177fe700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde04053380 0x7fde04055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde177fe700 1 -- 172.21.2.104:0/3648832182 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fde1006c230 con 0x7fde200affb0 2021-07-11T03:56:52.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde1dd9b700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde04053380 0x7fde04055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde1dd9b700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde04053380 0x7fde04055840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fde08000ed0 tx=0x7fde080076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.389+0000 7fde177fe700 1 -- 172.21.2.104:0/3648832182 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fde200a9590 con 0x7fde200affb0 2021-07-11T03:56:52.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 -- 172.21.2.104:0/3648832182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fde200af780 con 0x7fde200affb0 2021-07-11T03:56:52.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde177fe700 1 -- 172.21.2.104:0/3648832182 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7fde10043570 con 0x7fde200affb0 2021-07-11T03:56:52.479 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:56:52.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 -- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde04053380 msgr2=0x7fde04055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde04053380 0x7fde04055840 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fde08000ed0 tx=0x7fde080076c0).stop 2021-07-11T03:56:52.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 -- 172.21.2.104:0/3648832182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde200affb0 msgr2=0x7fde200abcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde200affb0 0x7fde200abcb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fde200b2f50 tx=0x7fde10012000).stop 2021-07-11T03:56:52.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 -- 172.21.2.104:0/3648832182 shutdown_connections 2021-07-11T03:56:52.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde200b22a0 0x7fde200b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde200b1ec0 0x7fde200b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde04053380 0x7fde04055840 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 --2- 172.21.2.104:0/3648832182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde200affb0 0x7fde200abcb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 -- 172.21.2.104:0/3648832182 >> 172.21.2.104:0/3648832182 conn(0x7fde2005f6b0 msgr2=0x7fde20147f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 -- 172.21.2.104:0/3648832182 shutdown_connections 2021-07-11T03:56:52.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.473+0000 7fde24da9700 1 -- 172.21.2.104:0/3648832182 wait complete. 2021-07-11T03:56:52.486 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:56:52.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 Processor -- start 2021-07-11T03:56:52.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 -- start start 2021-07-11T03:56:52.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f7bcc0b1ec0 0x7f7bcc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7bcc0affb0 0x7f7bcc0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc0b0420 con 0x7f7bcc0a70c0 2021-07-11T03:56:52.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc0b2790 con 0x7f7bcc0affb0 2021-07-11T03:56:52.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc0b0650 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bcaffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7bcc0affb0 0x7f7bcc0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bcaffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7bcc0affb0 0x7f7bcc0b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44890/0 (socket says 172.21.2.104:44890) 2021-07-11T03:56:52.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bcaffd700 1 -- 172.21.2.104:0/2359602403 learned_addr learned my addr 172.21.2.104:0/2359602403 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:52.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bcaffd700 1 -- 172.21.2.104:0/2359602403 >> v2:172.21.2.105:3300/0 conn(0x7f7bcc0b1ec0 msgr2=0x7f7bcc0b12d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:56:52.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bcaffd700 1 --2- 172.21.2.104:0/2359602403 >> v2:172.21.2.105:3300/0 conn(0x7f7bcc0b1ec0 0x7f7bcc0b12d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bcaffd700 1 -- 172.21.2.104:0/2359602403 >> v1:172.21.2.105:6789/0 conn(0x7f7bcc0a70c0 legacy=0x7f7bcc141250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:56:52.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bcaffd700 1 -- 172.21.2.104:0/2359602403 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0b08f0 con 0x7f7bcc0affb0 2021-07-11T03:56:52.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bcaffd700 1 --2- 172.21.2.104:0/2359602403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7bcc0affb0 0x7f7bcc0b22a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f7bcc0ad2d0 tx=0x7f7bbc006eb0).ready entity=mon.0 client_cookie=f44af6b479ed7a65 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bca7fc700 1 -- 172.21.2.104:0/2359602403 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bbc00d5c0 con 0x7f7bcc0affb0 2021-07-11T03:56:52.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bca7fc700 1 -- 172.21.2.104:0/2359602403 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc000e90 con 0x7f7bcc0affb0 2021-07-11T03:56:52.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bca7fc700 1 -- 172.21.2.104:0/2359602403 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bbc00d910 con 0x7f7bcc0affb0 2021-07-11T03:56:52.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/2359602403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7bcc0affb0 msgr2=0x7f7bcc0b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 --2- 172.21.2.104:0/2359602403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7bcc0affb0 0x7f7bcc0b22a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f7bcc0ad2d0 tx=0x7f7bbc006eb0).stop 2021-07-11T03:56:52.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/2359602403 shutdown_connections 2021-07-11T03:56:52.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 --2- 172.21.2.104:0/2359602403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7bcc0affb0 0x7f7bcc0b22a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 --2- 172.21.2.104:0/2359602403 >> v2:172.21.2.105:3300/0 conn(0x7f7bcc0b1ec0 0x7f7bcc0b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/2359602403 >> 172.21.2.104:0/2359602403 conn(0x7f7bcc05f6b0 msgr2=0x7f7bcc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/2359602403 shutdown_connections 2021-07-11T03:56:52.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.533+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/2359602403 wait complete. 2021-07-11T03:56:52.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bd1ed4700 1 Processor -- start 2021-07-11T03:56:52.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bd1ed4700 1 -- start start 2021-07-11T03:56:52.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bd1ed4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7bcc0a70c0 0x7f7bcc142180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bd1ed4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7bcc0affb0 0x7f7bcc1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bd1ed4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7bcc0b1ec0 0x7f7bcc143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bd1ed4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0a9f80 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bd1ed4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0aa1b0 con 0x7f7bcc0a70c0 2021-07-11T03:56:52.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bd1ed4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc0aa450 con 0x7f7bcc0affb0 2021-07-11T03:56:52.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bcbfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7bcc0b1ec0 0x7f7bcc143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bcbfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7bcc0b1ec0 0x7f7bcc143b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44896/0 (socket says 172.21.2.104:44896) 2021-07-11T03:56:52.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bcbfff700 1 -- 172.21.2.104:0/3086702172 learned_addr learned my addr 172.21.2.104:0/3086702172 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:52.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bcbfff700 1 -- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7bcc0affb0 msgr2=0x7f7bcc1575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bcbfff700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7bcc0affb0 0x7f7bcc1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bcbfff700 1 -- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7bcc0a70c0 msgr2=0x7f7bcc142180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bcbfff700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7bcc0a70c0 0x7f7bcc142180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bcbfff700 1 -- 172.21.2.104:0/3086702172 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc00b1e0 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bcbfff700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7bcc0b1ec0 0x7f7bcc143b40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f7bc4003330 tx=0x7f7bc40091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bc8ff9700 1 -- 172.21.2.104:0/3086702172 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bc4001520 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bc8ff9700 1 -- 172.21.2.104:0/3086702172 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc4001680 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bc8ff9700 1 -- 172.21.2.104:0/3086702172 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7bc40019e0 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/3086702172 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc0aa6f0 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/3086702172 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc157ae0 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/3086702172 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc142950 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bc8ff9700 1 -- 172.21.2.104:0/3086702172 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f7bc4018030 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bc8ff9700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7bb0053260 0x7f7bb0055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bc8ff9700 1 -- 172.21.2.104:0/3086702172 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7bc406aff0 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.537+0000 7f7bcb7fe700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7bb0053260 0x7f7bb0055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.541+0000 7f7bcb7fe700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7bb0053260 0x7f7bb0055720 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f7bb4000ed0 tx=0x7f7bb4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.541+0000 7f7bc8ff9700 1 -- 172.21.2.104:0/3086702172 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7bc40422d0 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/3086702172 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7bcc057350 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bc8ff9700 1 -- 172.21.2.104:0/3086702172 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7bc4020020 con 0x7f7bcc0b1ec0 2021-07-11T03:56:52.632 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:56:52.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7bb0053260 msgr2=0x7f7bb0055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7bb0053260 0x7f7bb0055720 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f7bb4000ed0 tx=0x7f7bb4006930).stop 2021-07-11T03:56:52.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/3086702172 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7bcc0b1ec0 msgr2=0x7f7bcc143b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7bcc0b1ec0 0x7f7bcc143b40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f7bc4003330 tx=0x7f7bc40091a0).stop 2021-07-11T03:56:52.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/3086702172 shutdown_connections 2021-07-11T03:56:52.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7bb0053260 0x7f7bb0055720 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7bcc0b1ec0 0x7f7bcc143b40 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7bcc0affb0 0x7f7bcc1575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 --2- 172.21.2.104:0/3086702172 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7bcc0a70c0 0x7f7bcc142180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/3086702172 >> 172.21.2.104:0/3086702172 conn(0x7f7bcc05f6b0 msgr2=0x7f7bcc13f930 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/3086702172 shutdown_connections 2021-07-11T03:56:52.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.625+0000 7f7bd1ed4700 1 -- 172.21.2.104:0/3086702172 wait complete. 2021-07-11T03:56:52.639 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:56:52.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b60269700 1 Processor -- start 2021-07-11T03:56:52.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b60269700 1 -- start start 2021-07-11T03:56:52.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b60269700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1b580b1ec0 0x7f1b580b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b60269700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1b580affb0 0x7f1b580b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b60269700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b580b0420 con 0x7f1b580a70c0 2021-07-11T03:56:52.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b60269700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b580b2790 con 0x7f1b580b1ec0 2021-07-11T03:56:52.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b60269700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b580b0650 con 0x7f1b580affb0 2021-07-11T03:56:52.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b5e806700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1b580a70c0 0x7f1b58141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41674/0 (socket says 172.21.2.104:41674) 2021-07-11T03:56:52.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b5e806700 1 -- 172.21.2.104:0/4199434414 learned_addr learned my addr 172.21.2.104:0/4199434414 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:56:52.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b5e005700 1 --2- 172.21.2.104:0/4199434414 >> v2:172.21.2.104:3300/0 conn(0x7f1b580b1ec0 0x7f1b580b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b5e005700 1 -- 172.21.2.104:0/4199434414 >> v2:172.21.2.105:3300/0 conn(0x7f1b580affb0 msgr2=0x7f1b580b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:52.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b5e005700 1 --2- 172.21.2.104:0/4199434414 >> v2:172.21.2.105:3300/0 conn(0x7f1b580affb0 0x7f1b580b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b5e005700 1 -- 172.21.2.104:0/4199434414 >> v1:172.21.2.104:6789/0 conn(0x7f1b580a70c0 legacy=0x7f1b58141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b5e005700 1 -- 172.21.2.104:0/4199434414 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b580b08f0 con 0x7f1b580b1ec0 2021-07-11T03:56:52.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b5e005700 1 --2- 172.21.2.104:0/4199434414 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1b580b1ec0 0x7f1b580b12d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f1b50001490 tx=0x7f1b4c006df0).ready entity=mon.0 client_cookie=1afaa8a590c1cdaa server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b5d003700 1 -- 172.21.2.104:0/4199434414 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b4c00d570 con 0x7f1b580b1ec0 2021-07-11T03:56:52.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b5d003700 1 -- 172.21.2.104:0/4199434414 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4c000e90 con 0x7f1b580b1ec0 2021-07-11T03:56:52.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b5d003700 1 -- 172.21.2.104:0/4199434414 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b4c00d8c0 con 0x7f1b580b1ec0 2021-07-11T03:56:52.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b60269700 1 -- 172.21.2.104:0/4199434414 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1b580b1ec0 msgr2=0x7f1b580b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.685+0000 7f1b60269700 1 --2- 172.21.2.104:0/4199434414 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1b580b1ec0 0x7f1b580b12d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f1b50001490 tx=0x7f1b4c006df0).stop 2021-07-11T03:56:52.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 -- 172.21.2.104:0/4199434414 shutdown_connections 2021-07-11T03:56:52.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 --2- 172.21.2.104:0/4199434414 >> v2:172.21.2.105:3300/0 conn(0x7f1b580affb0 0x7f1b580b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 --2- 172.21.2.104:0/4199434414 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1b580b1ec0 0x7f1b580b12d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 -- 172.21.2.104:0/4199434414 >> 172.21.2.104:0/4199434414 conn(0x7f1b5805f6b0 msgr2=0x7f1b5805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 -- 172.21.2.104:0/4199434414 shutdown_connections 2021-07-11T03:56:52.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 -- 172.21.2.104:0/4199434414 wait complete. 2021-07-11T03:56:52.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 Processor -- start 2021-07-11T03:56:52.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 -- start start 2021-07-11T03:56:52.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1b580a70c0 0x7f1b580b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1b580affb0 0x7f1b5813ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1b580b1ec0 0x7f1b5813f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580ac230 con 0x7f1b580b1ec0 2021-07-11T03:56:52.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580b3290 con 0x7f1b580a70c0 2021-07-11T03:56:52.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580b3530 con 0x7f1b580affb0 2021-07-11T03:56:52.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b5e806700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1b580b1ec0 0x7f1b5813f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b5e806700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1b580b1ec0 0x7f1b5813f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44912/0 (socket says 172.21.2.104:44912) 2021-07-11T03:56:52.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b5e806700 1 -- 172.21.2.104:0/2395994774 learned_addr learned my addr 172.21.2.104:0/2395994774 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:52.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b5e806700 1 -- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1b580affb0 msgr2=0x7f1b5813ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b5e806700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1b580affb0 0x7f1b5813ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b5e806700 1 -- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1b580a70c0 msgr2=0x7f1b580b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b5e806700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1b580a70c0 0x7f1b580b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b5e806700 1 -- 172.21.2.104:0/2395994774 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c00b180 con 0x7f1b580b1ec0 2021-07-11T03:56:52.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b5e806700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1b580b1ec0 0x7f1b5813f380 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f1b50001550 tx=0x7f1b50007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b4b7fe700 1 -- 172.21.2.104:0/2395994774 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b50005ba0 con 0x7f1b580b1ec0 2021-07-11T03:56:52.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b4b7fe700 1 -- 172.21.2.104:0/2395994774 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b50005d00 con 0x7f1b580b1ec0 2021-07-11T03:56:52.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 -- 172.21.2.104:0/2395994774 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b580b37d0 con 0x7f1b580b1ec0 2021-07-11T03:56:52.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b4b7fe700 1 -- 172.21.2.104:0/2395994774 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1b50007300 con 0x7f1b580b1ec0 2021-07-11T03:56:52.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 -- 172.21.2.104:0/2395994774 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b58140840 con 0x7f1b580b1ec0 2021-07-11T03:56:52.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b60269700 1 -- 172.21.2.104:0/2395994774 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b58057350 con 0x7f1b580b1ec0 2021-07-11T03:56:52.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b4b7fe700 1 -- 172.21.2.104:0/2395994774 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1b50010030 con 0x7f1b580b1ec0 2021-07-11T03:56:52.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b4b7fe700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1b38053210 0x7f1b380556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b4b7fe700 1 -- 172.21.2.104:0/2395994774 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1b5006b5e0 con 0x7f1b580b1ec0 2021-07-11T03:56:52.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.689+0000 7f1b5e005700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1b38053210 0x7f1b380556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.693+0000 7f1b5e005700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1b38053210 0x7f1b380556d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f1b50001490 tx=0x7f1b4c014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.693+0000 7f1b4b7fe700 1 -- 172.21.2.104:0/2395994774 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1b500428c0 con 0x7f1b580b1ec0 2021-07-11T03:56:52.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 -- 172.21.2.104:0/2395994774 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1b580ac760 con 0x7f1b580b1ec0 2021-07-11T03:56:52.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b4b7fe700 1 -- 172.21.2.104:0/2395994774 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1b50045ee0 con 0x7f1b580b1ec0 2021-07-11T03:56:52.783 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:56:52.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 -- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1b38053210 msgr2=0x7f1b380556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1b38053210 0x7f1b380556d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f1b50001490 tx=0x7f1b4c014000).stop 2021-07-11T03:56:52.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 -- 172.21.2.104:0/2395994774 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1b580b1ec0 msgr2=0x7f1b5813f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1b580b1ec0 0x7f1b5813f380 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f1b50001550 tx=0x7f1b50007f00).stop 2021-07-11T03:56:52.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 -- 172.21.2.104:0/2395994774 shutdown_connections 2021-07-11T03:56:52.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1b38053210 0x7f1b380556d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1b580b1ec0 0x7f1b5813f380 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1b580affb0 0x7f1b5813ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 --2- 172.21.2.104:0/2395994774 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1b580a70c0 0x7f1b580b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 -- 172.21.2.104:0/2395994774 >> 172.21.2.104:0/2395994774 conn(0x7f1b5805f6b0 msgr2=0x7f1b58146d30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 -- 172.21.2.104:0/2395994774 shutdown_connections 2021-07-11T03:56:52.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.777+0000 7f1b60269700 1 -- 172.21.2.104:0/2395994774 wait complete. 2021-07-11T03:56:52.790 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T03:56:52.791 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:56:52.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 Processor -- start 2021-07-11T03:56:52.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- start start 2021-07-11T03:56:52.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fccb80a76b0 0x7fccb8066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccb80b1970 0x7fccb8066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fccb80b0060 0x7fccb805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb8067450 con 0x7fccb80b0060 2021-07-11T03:56:52.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb80a7a90 con 0x7fccb80a76b0 2021-07-11T03:56:52.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb8147620 con 0x7fccb80b1970 2021-07-11T03:56:52.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fccb80b0060 0x7fccb805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fccb80b0060 0x7fccb805f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44916/0 (socket says 172.21.2.104:44916) 2021-07-11T03:56:52.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 -- 172.21.2.104:0/3213657689 learned_addr learned my addr 172.21.2.104:0/3213657689 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:52.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 -- 172.21.2.104:0/3213657689 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccb80b1970 msgr2=0x7fccb8066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:52.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 --2- 172.21.2.104:0/3213657689 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccb80b1970 0x7fccb8066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 -- 172.21.2.104:0/3213657689 >> v2:172.21.2.105:3300/0 conn(0x7fccb80a76b0 msgr2=0x7fccb8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:56:52.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 --2- 172.21.2.104:0/3213657689 >> v2:172.21.2.105:3300/0 conn(0x7fccb80a76b0 0x7fccb8066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 -- 172.21.2.104:0/3213657689 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb81478c0 con 0x7fccb80b0060 2021-07-11T03:56:52.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 --2- 172.21.2.104:0/3213657689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccb80b0060 0x7fccb805f6b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fccb00032f0 tx=0x7fccb0009290).ready entity=mon.0 client_cookie=ab7b0998877c8cab server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbca37700 1 -- 172.21.2.104:0/3213657689 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccb000b870 con 0x7fccb80b0060 2021-07-11T03:56:52.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbca37700 1 -- 172.21.2.104:0/3213657689 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb000b9d0 con 0x7fccb80b0060 2021-07-11T03:56:52.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbca37700 1 -- 172.21.2.104:0/3213657689 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccb000bb40 con 0x7fccb80b0060 2021-07-11T03:56:52.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- 172.21.2.104:0/3213657689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccb80b0060 msgr2=0x7fccb805f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 --2- 172.21.2.104:0/3213657689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccb80b0060 0x7fccb805f6b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fccb00032f0 tx=0x7fccb0009290).stop 2021-07-11T03:56:52.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- 172.21.2.104:0/3213657689 shutdown_connections 2021-07-11T03:56:52.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 --2- 172.21.2.104:0/3213657689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccb80b0060 0x7fccb805f6b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 --2- 172.21.2.104:0/3213657689 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccb80b1970 0x7fccb8066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 --2- 172.21.2.104:0/3213657689 >> v2:172.21.2.105:3300/0 conn(0x7fccb80a76b0 0x7fccb8066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- 172.21.2.104:0/3213657689 >> 172.21.2.104:0/3213657689 conn(0x7fccb80b53e0 msgr2=0x7fccb80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- 172.21.2.104:0/3213657689 shutdown_connections 2021-07-11T03:56:52.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- 172.21.2.104:0/3213657689 wait complete. 2021-07-11T03:56:52.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 Processor -- start 2021-07-11T03:56:52.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- start start 2021-07-11T03:56:52.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccb80a76b0 0x7fccb80a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccb80b0060 0x7fccb81566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccb80b1970 0x7fccb8156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80ab840 con 0x7fccb80b1970 2021-07-11T03:56:52.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80aae60 con 0x7fccb80b0060 2021-07-11T03:56:52.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80ab100 con 0x7fccb80a76b0 2021-07-11T03:56:52.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccb80b1970 0x7fccb8156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccb80b1970 0x7fccb8156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44922/0 (socket says 172.21.2.104:44922) 2021-07-11T03:56:52.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 -- 172.21.2.104:0/1069800678 learned_addr learned my addr 172.21.2.104:0/1069800678 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:56:52.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 -- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccb80a76b0 msgr2=0x7fccb80a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccb80a76b0 0x7fccb80a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 -- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccb80b0060 msgr2=0x7fccb81566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccb80b0060 0x7fccb81566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 -- 172.21.2.104:0/1069800678 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb0009530 con 0x7fccb80b1970 2021-07-11T03:56:52.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbe23a700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccb80b1970 0x7fccb8156b90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fccb00032f0 tx=0x7fccb0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccaaffd700 1 -- 172.21.2.104:0/1069800678 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccb0011cc0 con 0x7fccb80b1970 2021-07-11T03:56:52.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccaaffd700 1 -- 172.21.2.104:0/1069800678 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb0011e20 con 0x7fccb80b1970 2021-07-11T03:56:52.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- 172.21.2.104:0/1069800678 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb80ab3a0 con 0x7fccb80b1970 2021-07-11T03:56:52.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccaaffd700 1 -- 172.21.2.104:0/1069800678 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccb0020660 con 0x7fccb80b1970 2021-07-11T03:56:52.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.837+0000 7fccbfc9d700 1 -- 172.21.2.104:0/1069800678 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb80ac6c0 con 0x7fccb80b1970 2021-07-11T03:56:52.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.841+0000 7fccbfc9d700 1 -- 172.21.2.104:0/1069800678 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb80ab560 con 0x7fccb80b1970 2021-07-11T03:56:52.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.841+0000 7fccaaffd700 1 -- 172.21.2.104:0/1069800678 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fccb000f070 con 0x7fccb80b1970 2021-07-11T03:56:52.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.841+0000 7fccaaffd700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc98053160 0x7fcc98055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:56:52.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.841+0000 7fccaaffd700 1 -- 172.21.2.104:0/1069800678 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fccb006c860 con 0x7fccb80b1970 2021-07-11T03:56:52.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.841+0000 7fccbda39700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc98053160 0x7fcc98055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:56:52.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.841+0000 7fccbda39700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc98053160 0x7fcc98055620 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fccb4002f70 tx=0x7fccb4006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:56:52.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.841+0000 7fccaaffd700 1 -- 172.21.2.104:0/1069800678 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fccb0043ba0 con 0x7fccb80b1970 2021-07-11T03:56:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 -- 172.21.2.104:0/1069800678 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccb8157220 con 0x7fccb80b1970 2021-07-11T03:56:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccaaffd700 1 -- 172.21.2.104:0/1069800678 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fccb0015070 con 0x7fccb80b1970 2021-07-11T03:56:52.957 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:56:52.957 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":22,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3427300973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3427300973},{"type":"v1","addr":"172.21.2.105:6835","nonce":3427300973}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/2823255258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2823255258},{"type":"v1","addr":"172.21.2.104:6833","nonce":2823255258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:56:49.697379+0000","modified":"2021-07-11T03:56:50.700784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/2428941897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2428941897},{"type":"v1","addr":"172.21.2.104:6835","nonce":2428941897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-07-11T03:56:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 -- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc98053160 msgr2=0x7fcc98055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc98053160 0x7fcc98055620 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fccb4002f70 tx=0x7fccb4006a80).stop 2021-07-11T03:56:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 -- 172.21.2.104:0/1069800678 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccb80b1970 msgr2=0x7fccb8156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:56:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccb80b1970 0x7fccb8156b90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fccb00032f0 tx=0x7fccb0016000).stop 2021-07-11T03:56:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 -- 172.21.2.104:0/1069800678 shutdown_connections 2021-07-11T03:56:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc98053160 0x7fcc98055620 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccb80b1970 0x7fccb8156b90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:56:52.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccb80b0060 0x7fccb81566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 --2- 172.21.2.104:0/1069800678 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccb80a76b0 0x7fccb80a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:56:52.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 -- 172.21.2.104:0/1069800678 >> 172.21.2.104:0/1069800678 conn(0x7fccb80b53e0 msgr2=0x7fccb805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:56:52.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 -- 172.21.2.104:0/1069800678 shutdown_connections 2021-07-11T03:56:52.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:56:52.949+0000 7fccbfc9d700 1 -- 172.21.2.104:0/1069800678 wait complete. 2021-07-11T03:56:52.961 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 22 2021-07-11T03:56:53.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:56:49.697379+0000', 'modified': '2021-07-11T03:56:50.700784+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5589': {'gid': 5589, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.104:6835/2428941897', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 2428941897}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 2428941897}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:56:53.267 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:56:53.268 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:56:53.331 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":5589,"want_state":"up:active","state":"up:active","rank_uptime":3.6257052349999999,"mdsmap_epoch":22,"osdmap_epoch":55,"osdmap_epoch_barrier":54,"uptime":7.3514167090000004} 2021-07-11T03:56:53.332 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 5589, "mdsmap_epoch": 22, "osdmap_epoch": 55, "osdmap_epoch_barrier": 54, "rank_uptime": 3.625705235, "state": "up:active", "uptime": 7.351416709, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:56:53.333 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T03:56:53.333 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T03:56:53.333 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T03:56:53.334 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:53.334 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:56:53.345 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:56:53.396 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:56:53.397 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:56:53.403 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:56:53.404 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:56:53.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:53.405 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:56:53.409 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T03:56:53.410 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:56:53.456 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:56:53.465 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:56:53.466 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:56:53.471 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:56:53.472 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:56:53.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:53.473 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:56:53.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T03:56:53.499+0000 7f4a62469080 -1 init, newargv = 0x55b37ec5d750 newargc=9 2021-07-11T03:56:53.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14871]: starting ceph client 2021-07-11T03:56:53.511 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14871]: starting fuse 2021-07-11T03:56:54.579 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:56:54.592 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:56:54.592 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:56:54.641 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:56:54.642 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:56:54.643 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:54.643 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:56:54.648 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T03:56:54.649 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T03:56:54.649 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:54.649 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T03:56:54.650 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T03:56:54.650 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T03:56:54.651 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:56:54.651 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T03:56:54.652 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:56:54.652 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T03:56:54.652 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T03:56:54.653 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T03:56:54.653 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:56:54.654 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T03:56:54.654 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T03:56:54.654 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T03:56:54.655 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T03:56:54.655 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:56:54.656 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T03:56:54.656 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T03:56:54.656 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T03:56:54.657 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T03:56:54.657 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T03:56:54.658 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:56:54.658 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T03:56:54.658 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:56:54.725 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14871.asok 2021-07-11T03:56:54.728 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14871.asok 2021-07-11T03:56:54.728 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:54.729 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14871.asok status 2021-07-11T03:56:54.777 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T03:56:54.777 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T03:56:54.778 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T03:56:54.778 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T03:56:54.778 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T03:56:54.778 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T03:56:54.779 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T03:56:54.779 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14871", 2021-07-11T03:56:54.779 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T03:56:54.779 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:56:54.779 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T03:56:54.780 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T03:56:54.780 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4645, 2021-07-11T03:56:54.780 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T03:56:54.780 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T03:56:54.780 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T03:56:54.781 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4645 2021-07-11T03:56:54.781 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:56:54.781 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:56:54.781 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:56:54.782 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:56:54.782 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1803946025 2021-07-11T03:56:54.782 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T03:56:54.782 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:56:54.782 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:56:54.783 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:56:54.783 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:56:54.783 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1803946025 2021-07-11T03:56:54.783 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:56:54.784 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4645 172.21.2.119:0/1803946025", 2021-07-11T03:56:54.784 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/1803946025", 2021-07-11T03:56:54.784 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T03:56:54.784 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 22, 2021-07-11T03:56:54.784 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 55, 2021-07-11T03:56:54.785 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T03:56:54.785 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T03:56:54.785 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T03:56:54.785 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T03:56:54.786 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:54.787 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:56:54.787 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:56:54.793 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:56:54.793 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:56:54.793 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:54.794 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:56:54.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:56:54.844 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:56:54.844 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:56:54.850 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:56:54.852 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:56:54.853 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-07-11T03:56:54.854 DEBUG:teuthology.orchestra.run.gibba019:> import sys 2021-07-11T03:56:54.854 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-07-11T03:56:54.854 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:56:54.855 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:56:54.855 DEBUG:teuthology.orchestra.run.gibba019:> n = 1000 2021-07-11T03:56:54.855 DEBUG:teuthology.orchestra.run.gibba019:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-07-11T03:56:54.856 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:56:54.856 DEBUG:teuthology.orchestra.run.gibba019:> if not os.path.exists(os.path.dirname(abs_path)): 2021-07-11T03:56:54.857 DEBUG:teuthology.orchestra.run.gibba019:> os.makedirs(os.path.dirname(abs_path)) 2021-07-11T03:56:54.857 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:56:54.857 DEBUG:teuthology.orchestra.run.gibba019:> for i in range(0, n): 2021-07-11T03:56:54.858 DEBUG:teuthology.orchestra.run.gibba019:> fname = "{0}_{1}".format(abs_path, i) 2021-07-11T03:56:54.858 DEBUG:teuthology.orchestra.run.gibba019:> with open(fname, '"'"'w'"'"') as f: 2021-07-11T03:56:54.859 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-07-11T03:56:54.859 DEBUG:teuthology.orchestra.run.gibba019:> if True: 2021-07-11T03:56:54.859 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-07-11T03:56:54.860 DEBUG:teuthology.orchestra.run.gibba019:> os.fsync(f.fileno()) 2021-07-11T03:56:54.860 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:57:12.345 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:57:12.349 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:57:12.352 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:57:12.354 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:57:13.575 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:57:13.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 Processor -- start 2021-07-11T03:57:13.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- start start 2021-07-11T03:57:13.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f10d0147250 0x7f10d00613d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:13.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d00b3620 con 0x7f10d00b3240 2021-07-11T03:57:13.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d0146870 con 0x7f10d00b27b0 2021-07-11T03:57:13.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d0147630 con 0x7f10d0147250 2021-07-11T03:57:13.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f10d0147250 0x7f10d00613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:13.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d4cd0700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f10d00b3240 0x7f10d0060160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41696/0 (socket says 172.21.2.104:41696) 2021-07-11T03:57:13.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d4cd0700 1 -- 172.21.2.104:0/3261571304 learned_addr learned my addr 172.21.2.104:0/3261571304 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:57:13.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 --2- 172.21.2.104:0/3261571304 >> v2:172.21.2.104:3300/0 conn(0x7f10d0147250 0x7f10d00613d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44932/0 (socket says 172.21.2.104:44932) 2021-07-11T03:57:13.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 -- 172.21.2.104:0/3261571304 >> v1:172.21.2.105:6789/0 conn(0x7f10d00b27b0 legacy=0x7f10d0060cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:13.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 -- 172.21.2.104:0/3261571304 >> v1:172.21.2.104:6789/0 conn(0x7f10d00b3240 legacy=0x7f10d0060160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:13.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 -- 172.21.2.104:0/3261571304 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d0146b10 con 0x7f10d0147250 2021-07-11T03:57:13.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 --2- 172.21.2.104:0/3261571304 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10d0147250 0x7f10d00613d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f10d00b3960 tx=0x7f10c80091d0).ready entity=mon.0 client_cookie=2d636852fd1f52d9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:13.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10cf7fe700 1 -- 172.21.2.104:0/3261571304 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10c8009ea0 con 0x7f10d0147250 2021-07-11T03:57:13.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10cf7fe700 1 -- 172.21.2.104:0/3261571304 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c800b860 con 0x7f10d0147250 2021-07-11T03:57:13.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10cf7fe700 1 -- 172.21.2.104:0/3261571304 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10c800b9d0 con 0x7f10d0147250 2021-07-11T03:57:13.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- 172.21.2.104:0/3261571304 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10d0147250 msgr2=0x7f10d00613d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:13.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 --2- 172.21.2.104:0/3261571304 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10d0147250 0x7f10d00613d0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f10d00b3960 tx=0x7f10c80091d0).stop 2021-07-11T03:57:13.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- 172.21.2.104:0/3261571304 shutdown_connections 2021-07-11T03:57:13.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 --2- 172.21.2.104:0/3261571304 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10d0147250 0x7f10d00613d0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:13.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- 172.21.2.104:0/3261571304 >> 172.21.2.104:0/3261571304 conn(0x7f10d005c490 msgr2=0x7f10d005c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:13.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- 172.21.2.104:0/3261571304 shutdown_connections 2021-07-11T03:57:13.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- 172.21.2.104:0/3261571304 wait complete. 2021-07-11T03:57:13.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 Processor -- start 2021-07-11T03:57:13.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- start start 2021-07-11T03:57:13.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10d00b27b0 0x7f10d0142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:13.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f10d00b3240 0x7f10d01557a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:13.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10d0147250 0x7f10d00aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:13.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00ad220 con 0x7f10d0147250 2021-07-11T03:57:13.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d0145cf0 con 0x7f10d00b27b0 2021-07-11T03:57:13.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d6f34700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d0145a90 con 0x7f10d00b3240 2021-07-11T03:57:13.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10d0147250 0x7f10d00aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:13.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10d0147250 0x7f10d00aa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44938/0 (socket says 172.21.2.104:44938) 2021-07-11T03:57:13.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 -- 172.21.2.104:0/2144420010 learned_addr learned my addr 172.21.2.104:0/2144420010 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:13.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 -- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f10d00b3240 msgr2=0x7f10d01557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:13.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f10d00b3240 0x7f10d01557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:13.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 -- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10d00b27b0 msgr2=0x7f10d0142580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:13.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10d00b27b0 0x7f10d0142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:13.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 -- 172.21.2.104:0/2144420010 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c8009470 con 0x7f10d0147250 2021-07-11T03:57:13.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10d54d1700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10d0147250 0x7f10d00aa5f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f10d00b3960 tx=0x7f10c8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:13.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10cdffb700 1 -- 172.21.2.104:0/2144420010 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10c8021440 con 0x7f10d0147250 2021-07-11T03:57:13.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10cdffb700 1 -- 172.21.2.104:0/2144420010 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c80215a0 con 0x7f10d0147250 2021-07-11T03:57:13.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.645+0000 7f10cdffb700 1 -- 172.21.2.104:0/2144420010 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f10c8021900 con 0x7f10d0147250 2021-07-11T03:57:13.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.649+0000 7f10d6f34700 1 -- 172.21.2.104:0/2144420010 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d0144590 con 0x7f10d0147250 2021-07-11T03:57:13.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.649+0000 7f10d6f34700 1 -- 172.21.2.104:0/2144420010 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d00ab470 con 0x7f10d0147250 2021-07-11T03:57:13.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.649+0000 7f10d6f34700 1 -- 172.21.2.104:0/2144420010 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10d00aa2e0 con 0x7f10d0147250 2021-07-11T03:57:13.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.649+0000 7f10cdffb700 1 -- 172.21.2.104:0/2144420010 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f10c8027020 con 0x7f10d0147250 2021-07-11T03:57:13.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.649+0000 7f10cdffb700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10b8053160 0x7f10b8055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:13.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.649+0000 7f10cdffb700 1 -- 172.21.2.104:0/2144420010 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f10c806c8a0 con 0x7f10d0147250 2021-07-11T03:57:13.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.649+0000 7f10d4cd0700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10b8053160 0x7f10b8055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:13.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.649+0000 7f10d4cd0700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10b8053160 0x7f10b8055620 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f10d00b38c0 tx=0x7f10c4007820).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:13.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.653+0000 7f10cdffb700 1 -- 172.21.2.104:0/2144420010 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f10c8043c90 con 0x7f10d0147250 2021-07-11T03:57:13.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 -- 172.21.2.104:0/2144420010 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10d0057350 con 0x7f10d0147250 2021-07-11T03:57:13.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10cdffb700 1 -- 172.21.2.104:0/2144420010 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f10c8015090 con 0x7f10d0147250 2021-07-11T03:57:13.769 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:57:13.769 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":22,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3427300973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3427300973},{"type":"v1","addr":"172.21.2.105:6835","nonce":3427300973}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/2823255258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2823255258},{"type":"v1","addr":"172.21.2.104:6833","nonce":2823255258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:56:49.697379+0000","modified":"2021-07-11T03:56:50.700784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/2428941897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2428941897},{"type":"v1","addr":"172.21.2.104:6835","nonce":2428941897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-07-11T03:57:13.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 -- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10b8053160 msgr2=0x7f10b8055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:13.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10b8053160 0x7f10b8055620 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f10d00b38c0 tx=0x7f10c4007820).stop 2021-07-11T03:57:13.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 -- 172.21.2.104:0/2144420010 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10d0147250 msgr2=0x7f10d00aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:13.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10d0147250 0x7f10d00aa5f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f10d00b3960 tx=0x7f10c8016000).stop 2021-07-11T03:57:13.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 -- 172.21.2.104:0/2144420010 shutdown_connections 2021-07-11T03:57:13.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10b8053160 0x7f10b8055620 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:13.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10d0147250 0x7f10d00aa5f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:13.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f10d00b3240 0x7f10d01557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:13.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 --2- 172.21.2.104:0/2144420010 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10d00b27b0 0x7f10d0142580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:13.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 -- 172.21.2.104:0/2144420010 >> 172.21.2.104:0/2144420010 conn(0x7f10d005c490 msgr2=0x7f10d0065820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:13.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 -- 172.21.2.104:0/2144420010 shutdown_connections 2021-07-11T03:57:13.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.761+0000 7f10d6f34700 1 -- 172.21.2.104:0/2144420010 wait complete. 2021-07-11T03:57:13.773 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 22 2021-07-11T03:57:13.775 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_caps 20 2021-07-11T03:57:13.834 INFO:teuthology.orchestra.run.gibba004.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-07-11T03:57:13.834 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-07-11T03:57:13.835 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:57:13.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 Processor -- start 2021-07-11T03:57:13.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- start start 2021-07-11T03:57:13.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa500067180 0x7fa50005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:13.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa5000b1970 0x7fa50005fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:13.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5000a7a90 con 0x7fa5000a76b0 2021-07-11T03:57:13.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa500147620 con 0x7fa5000b1970 2021-07-11T03:57:13.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5001478c0 con 0x7fa500067180 2021-07-11T03:57:13.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa4fed9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa5000b1970 0x7fa50005fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:13.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa4fed9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa5000b1970 0x7fa50005fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44944/0 (socket says 172.21.2.104:44944) 2021-07-11T03:57:13.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa4fed9d700 1 -- 172.21.2.104:0/3174412734 learned_addr learned my addr 172.21.2.104:0/3174412734 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:13.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa4fed9d700 1 -- 172.21.2.104:0/3174412734 >> v2:172.21.2.105:3300/0 conn(0x7fa500067180 msgr2=0x7fa50005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:13.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa4fed9d700 1 --2- 172.21.2.104:0/3174412734 >> v2:172.21.2.105:3300/0 conn(0x7fa500067180 0x7fa50005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:13.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa4fed9d700 1 -- 172.21.2.104:0/3174412734 >> v1:172.21.2.105:6789/0 conn(0x7fa5000a76b0 legacy=0x7fa500066a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:57:13.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa4fed9d700 1 -- 172.21.2.104:0/3174412734 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa500147b60 con 0x7fa5000b1970 2021-07-11T03:57:13.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa4fed9d700 1 --2- 172.21.2.104:0/3174412734 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5000b1970 0x7fa50005fba0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fa50006a0e0 tx=0x7fa4f8009290).ready entity=mon.0 client_cookie=3c7ae8246053d1b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:13.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa4fd59a700 1 -- 172.21.2.104:0/3174412734 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f800b830 con 0x7fa5000b1970 2021-07-11T03:57:13.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa4fd59a700 1 -- 172.21.2.104:0/3174412734 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f800b990 con 0x7fa5000b1970 2021-07-11T03:57:13.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa4fd59a700 1 -- 172.21.2.104:0/3174412734 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4f800bb00 con 0x7fa5000b1970 2021-07-11T03:57:13.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- 172.21.2.104:0/3174412734 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5000b1970 msgr2=0x7fa50005fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:13.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 --2- 172.21.2.104:0/3174412734 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5000b1970 0x7fa50005fba0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fa50006a0e0 tx=0x7fa4f8009290).stop 2021-07-11T03:57:13.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- 172.21.2.104:0/3174412734 shutdown_connections 2021-07-11T03:57:13.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 --2- 172.21.2.104:0/3174412734 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5000b1970 0x7fa50005fba0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:13.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 --2- 172.21.2.104:0/3174412734 >> v2:172.21.2.105:3300/0 conn(0x7fa500067180 0x7fa50005f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:13.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- 172.21.2.104:0/3174412734 >> 172.21.2.104:0/3174412734 conn(0x7fa5000b53e0 msgr2=0x7fa5000b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:13.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- 172.21.2.104:0/3174412734 shutdown_connections 2021-07-11T03:57:13.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- 172.21.2.104:0/3174412734 wait complete. 2021-07-11T03:57:13.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 Processor -- start 2021-07-11T03:57:13.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- start start 2021-07-11T03:57:13.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa500067180 0x7fa5000b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:13.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5000a76b0 0x7fa5000acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:13.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa5000b1970 0x7fa5000ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:13.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa500149ca0 con 0x7fa5000a76b0 2021-07-11T03:57:13.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000aadb0 con 0x7fa500067180 2021-07-11T03:57:13.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.909+0000 7fa504e80700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5000ab050 con 0x7fa5000b1970 2021-07-11T03:57:13.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4fdd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5000a76b0 0x7fa5000acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:13.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4fdd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5000a76b0 0x7fa5000acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44952/0 (socket says 172.21.2.104:44952) 2021-07-11T03:57:13.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4fdd9b700 1 -- 172.21.2.104:0/1356490313 learned_addr learned my addr 172.21.2.104:0/1356490313 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:13.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4fdd9b700 1 -- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa5000b1970 msgr2=0x7fa5000ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:13.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4fdd9b700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa5000b1970 0x7fa5000ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:13.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4fdd9b700 1 -- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa500067180 msgr2=0x7fa5000b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:13.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4fdd9b700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa500067180 0x7fa5000b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:13.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4fdd9b700 1 -- 172.21.2.104:0/1356490313 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f8009530 con 0x7fa5000a76b0 2021-07-11T03:57:13.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4fdd9b700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5000a76b0 0x7fa5000acb40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fa4e8002f70 tx=0x7fa4e8006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:13.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4f77fe700 1 -- 172.21.2.104:0/1356490313 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e800e470 con 0x7fa5000a76b0 2021-07-11T03:57:13.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4f77fe700 1 -- 172.21.2.104:0/1356490313 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e800e5d0 con 0x7fa5000a76b0 2021-07-11T03:57:13.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4f77fe700 1 -- 172.21.2.104:0/1356490313 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4e800e930 con 0x7fa5000a76b0 2021-07-11T03:57:13.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa504e80700 1 -- 172.21.2.104:0/1356490313 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5000ac6a0 con 0x7fa5000a76b0 2021-07-11T03:57:13.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa504e80700 1 -- 172.21.2.104:0/1356490313 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5000aea30 con 0x7fa5000a76b0 2021-07-11T03:57:13.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa504e80700 1 -- 172.21.2.104:0/1356490313 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5000ac830 con 0x7fa5000a76b0 2021-07-11T03:57:13.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4f77fe700 1 -- 172.21.2.104:0/1356490313 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa4e8018030 con 0x7fa5000a76b0 2021-07-11T03:57:13.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4f77fe700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4e40531d0 0x7fa4e4055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:13.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4f77fe700 1 -- 172.21.2.104:0/1356490313 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa4e806ad00 con 0x7fa5000a76b0 2021-07-11T03:57:13.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4fe59c700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4e40531d0 0x7fa4e4055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:13.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.913+0000 7fa4fe59c700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4e40531d0 0x7fa4e4055690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fa4f0002fb0 tx=0x7fa4f00069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:13.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:13.917+0000 7fa4f77fe700 1 -- 172.21.2.104:0/1356490313 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa4e8041f30 con 0x7fa5000a76b0 2021-07-11T03:57:14.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.021+0000 7fa504e80700 1 -- 172.21.2.104:0/1356490313 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa500057350 con 0x7fa5000a76b0 2021-07-11T03:57:14.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa4f77fe700 1 -- 172.21.2.104:0/1356490313 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fa4e8045550 con 0x7fa5000a76b0 2021-07-11T03:57:14.031 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:57:14.031 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":22,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3427300973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3427300973},{"type":"v1","addr":"172.21.2.105:6835","nonce":3427300973}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/2823255258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2823255258},{"type":"v1","addr":"172.21.2.104:6833","nonce":2823255258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:56:49.697379+0000","modified":"2021-07-11T03:56:50.700784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/2428941897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2428941897},{"type":"v1","addr":"172.21.2.104:6835","nonce":2428941897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-07-11T03:57:14.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 -- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4e40531d0 msgr2=0x7fa4e4055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4e40531d0 0x7fa4e4055690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fa4f0002fb0 tx=0x7fa4f00069d0).stop 2021-07-11T03:57:14.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 -- 172.21.2.104:0/1356490313 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5000a76b0 msgr2=0x7fa5000acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5000a76b0 0x7fa5000acb40 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fa4e8002f70 tx=0x7fa4e8006ea0).stop 2021-07-11T03:57:14.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 -- 172.21.2.104:0/1356490313 shutdown_connections 2021-07-11T03:57:14.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa5000b1970 0x7fa5000ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:14.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4e40531d0 0x7fa4e4055690 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5000a76b0 0x7fa5000acb40 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 --2- 172.21.2.104:0/1356490313 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa500067180 0x7fa5000b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:14.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 -- 172.21.2.104:0/1356490313 >> 172.21.2.104:0/1356490313 conn(0x7fa5000b53e0 msgr2=0x7fa500060470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:14.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 -- 172.21.2.104:0/1356490313 shutdown_connections 2021-07-11T03:57:14.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.025+0000 7fa504e80700 1 -- 172.21.2.104:0/1356490313 wait complete. 2021-07-11T03:57:14.035 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 22 2021-07-11T03:57:14.038 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_decay_threshold 400 2021-07-11T03:57:14.095 INFO:teuthology.orchestra.run.gibba004.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-07-11T03:57:14.096 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-07-11T03:57:14.096 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T03:57:14.111 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:57:14.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.153+0000 7fd651330700 1 Processor -- start 2021-07-11T03:57:14.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.153+0000 7fd651330700 1 -- start start 2021-07-11T03:57:14.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd64c060280 0x7fd64c0668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd64c0b19a0 0x7fd64c066da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c0abea0 con 0x7fd64c05f6b0 2021-07-11T03:57:14.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64c0ab310 con 0x7fd64c060280 2021-07-11T03:57:14.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64c140300 con 0x7fd64c0b19a0 2021-07-11T03:57:14.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64b7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd64c05f6b0 0x7fd64c0a97a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41728/0 (socket says 172.21.2.104:41728) 2021-07-11T03:57:14.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64b7fe700 1 -- 172.21.2.104:0/875442732 learned_addr learned my addr 172.21.2.104:0/875442732 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:57:14.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64affd700 1 --2- 172.21.2.104:0/875442732 >> v2:172.21.2.104:3300/0 conn(0x7fd64c060280 0x7fd64c0668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd649ffb700 1 -- 172.21.2.104:0/875442732 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3172173469 0 0) 0x7fd64c0abea0 con 0x7fd64c05f6b0 2021-07-11T03:57:14.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd649ffb700 1 -- 172.21.2.104:0/875442732 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd62c003260 con 0x7fd64c05f6b0 2021-07-11T03:57:14.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64affd700 1 -- 172.21.2.104:0/875442732 >> v2:172.21.2.105:3300/0 conn(0x7fd64c0b19a0 msgr2=0x7fd64c066da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:14.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64affd700 1 --2- 172.21.2.104:0/875442732 >> v2:172.21.2.105:3300/0 conn(0x7fd64c0b19a0 0x7fd64c066da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:14.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64affd700 1 -- 172.21.2.104:0/875442732 >> v1:172.21.2.104:6789/0 conn(0x7fd64c05f6b0 legacy=0x7fd64c0a97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64affd700 1 -- 172.21.2.104:0/875442732 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c1405a0 con 0x7fd64c060280 2021-07-11T03:57:14.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64affd700 1 --2- 172.21.2.104:0/875442732 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c060280 0x7fd64c0668b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fd634002f70 tx=0x7fd634006df0).ready entity=mon.0 client_cookie=57e39fe4629f864d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:14.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd649ffb700 1 -- 172.21.2.104:0/875442732 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd63400d570 con 0x7fd64c060280 2021-07-11T03:57:14.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd649ffb700 1 -- 172.21.2.104:0/875442732 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634000e90 con 0x7fd64c060280 2021-07-11T03:57:14.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd649ffb700 1 -- 172.21.2.104:0/875442732 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd63400d8c0 con 0x7fd64c060280 2021-07-11T03:57:14.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- 172.21.2.104:0/875442732 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c060280 msgr2=0x7fd64c0668b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 --2- 172.21.2.104:0/875442732 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c060280 0x7fd64c0668b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fd634002f70 tx=0x7fd634006df0).stop 2021-07-11T03:57:14.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- 172.21.2.104:0/875442732 shutdown_connections 2021-07-11T03:57:14.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 --2- 172.21.2.104:0/875442732 >> v2:172.21.2.105:3300/0 conn(0x7fd64c0b19a0 0x7fd64c066da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:14.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 --2- 172.21.2.104:0/875442732 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c060280 0x7fd64c0668b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- 172.21.2.104:0/875442732 >> 172.21.2.104:0/875442732 conn(0x7fd64c0b53e0 msgr2=0x7fd64c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:14.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- 172.21.2.104:0/875442732 shutdown_connections 2021-07-11T03:57:14.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- 172.21.2.104:0/875442732 wait complete. 2021-07-11T03:57:14.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 Processor -- start 2021-07-11T03:57:14.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- start start 2021-07-11T03:57:14.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd64c05f6b0 0x7fd64c0b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c060280 0x7fd64c0625d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c0b19a0 0x7fd64c0af110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0ad630 con 0x7fd64c0b19a0 2021-07-11T03:57:14.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0af600 con 0x7fd64c05f6b0 2021-07-11T03:57:14.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0ad900 con 0x7fd64c060280 2021-07-11T03:57:14.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c0b19a0 0x7fd64c0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c0b19a0 0x7fd64c0af110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44966/0 (socket says 172.21.2.104:44966) 2021-07-11T03:57:14.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64b7fe700 1 -- 172.21.2.104:0/2811795176 learned_addr learned my addr 172.21.2.104:0/2811795176 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:14.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64b7fe700 1 -- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c060280 msgr2=0x7fd64c0625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64b7fe700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c060280 0x7fd64c0625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:14.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64b7fe700 1 -- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd64c05f6b0 msgr2=0x7fd64c0b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64b7fe700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd64c05f6b0 0x7fd64c0b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:14.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64b7fe700 1 -- 172.21.2.104:0/2811795176 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63400b180 con 0x7fd64c0b19a0 2021-07-11T03:57:14.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd64b7fe700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c0b19a0 0x7fd64c0af110 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fd644001490 tx=0x7fd644007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:14.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd643fff700 1 -- 172.21.2.104:0/2811795176 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd644005ba0 con 0x7fd64c0b19a0 2021-07-11T03:57:14.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd643fff700 1 -- 172.21.2.104:0/2811795176 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd644005d00 con 0x7fd64c0b19a0 2021-07-11T03:57:14.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- 172.21.2.104:0/2811795176 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c0add60 con 0x7fd64c0b19a0 2021-07-11T03:57:14.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd643fff700 1 -- 172.21.2.104:0/2811795176 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd6440072c0 con 0x7fd64c0b19a0 2021-07-11T03:57:14.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.157+0000 7fd651330700 1 -- 172.21.2.104:0/2811795176 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c13cee0 con 0x7fd64c0b19a0 2021-07-11T03:57:14.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.161+0000 7fd651330700 1 -- 172.21.2.104:0/2811795176 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c13cd40 con 0x7fd64c0b19a0 2021-07-11T03:57:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.161+0000 7fd643fff700 1 -- 172.21.2.104:0/2811795176 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd64401a060 con 0x7fd64c0b19a0 2021-07-11T03:57:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.161+0000 7fd643fff700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd630053210 0x7fd6300556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.161+0000 7fd643fff700 1 -- 172.21.2.104:0/2811795176 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd64406b060 con 0x7fd64c0b19a0 2021-07-11T03:57:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.161+0000 7fd64affd700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd630053210 0x7fd6300556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.161+0000 7fd64affd700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd630053210 0x7fd6300556d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fd634002f70 tx=0x7fd634014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:14.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.161+0000 7fd643fff700 1 -- 172.21.2.104:0/2811795176 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd64c13cd40 con 0x7fd64c0b19a0 2021-07-11T03:57:14.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd651330700 1 -- 172.21.2.104:0/2811795176 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd64c13cd40 con 0x7fd64c0b19a0 2021-07-11T03:57:14.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd643fff700 1 -- 172.21.2.104:0/2811795176 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2492 (secure 0 0 0) 0x7fd64c13cd40 con 0x7fd64c0b19a0 2021-07-11T03:57:14.277 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:57:14.278 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":22,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3427300973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3427300973},{"type":"v1","addr":"172.21.2.105:6835","nonce":3427300973}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/2823255258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2823255258},{"type":"v1","addr":"172.21.2.104:6833","nonce":2823255258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:56:49.697379+0000","modified":"2021-07-11T03:56:50.700784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/2428941897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2428941897},{"type":"v1","addr":"172.21.2.104:6835","nonce":2428941897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-07-11T03:57:14.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd651330700 1 -- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd630053210 msgr2=0x7fd6300556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd651330700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd630053210 0x7fd6300556d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fd634002f70 tx=0x7fd634014000).stop 2021-07-11T03:57:14.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd651330700 1 -- 172.21.2.104:0/2811795176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c0b19a0 msgr2=0x7fd64c0af110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd651330700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c0b19a0 0x7fd64c0af110 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fd644001490 tx=0x7fd644007e00).stop 2021-07-11T03:57:14.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd651330700 1 -- 172.21.2.104:0/2811795176 shutdown_connections 2021-07-11T03:57:14.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd651330700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd630053210 0x7fd6300556d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd651330700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd64c0b19a0 0x7fd64c0af110 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd651330700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd64c060280 0x7fd64c0625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:14.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd651330700 1 --2- 172.21.2.104:0/2811795176 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd64c05f6b0 0x7fd64c0b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:14.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.269+0000 7fd651330700 1 -- 172.21.2.104:0/2811795176 >> 172.21.2.104:0/2811795176 conn(0x7fd64c0b53e0 msgr2=0x7fd64c13e390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:14.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.273+0000 7fd651330700 1 -- 172.21.2.104:0/2811795176 shutdown_connections 2021-07-11T03:57:14.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.273+0000 7fd651330700 1 -- 172.21.2.104:0/2811795176 wait complete. 2021-07-11T03:57:14.283 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 22 2021-07-11T03:57:14.285 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c cache drop 5 2021-07-11T03:57:14.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.329+0000 7fd7d4f47700 1 Processor -- start 2021-07-11T03:57:14.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.329+0000 7fd7d4f47700 1 -- start start 2021-07-11T03:57:14.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd7d00b10c0 0x7fd7d00b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd7d0140ba0 0x7fd7d00b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d00ac7c0 con 0x7fd7d00acb70 2021-07-11T03:57:14.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d00ac020 con 0x7fd7d00b10c0 2021-07-11T03:57:14.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d0141010 con 0x7fd7d0140ba0 2021-07-11T03:57:14.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd7d00b10c0 0x7fd7d00b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd7d00b10c0 0x7fd7d00b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44970/0 (socket says 172.21.2.104:44970) 2021-07-11T03:57:14.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1441454556 learned_addr learned my addr 172.21.2.104:0/1441454556 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:14.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1441454556 >> v2:172.21.2.105:3300/0 conn(0x7fd7d0140ba0 msgr2=0x7fd7d00b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:14.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1441454556 >> v2:172.21.2.105:3300/0 conn(0x7fd7d0140ba0 0x7fd7d00b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:14.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1441454556 >> v1:172.21.2.105:6789/0 conn(0x7fd7d00acb70 legacy=0x7fd7d013d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:14.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1441454556 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d00ac280 con 0x7fd7d00b10c0 2021-07-11T03:57:14.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1441454556 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d00b10c0 0x7fd7d00b14a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fd7d00a9db0 tx=0x7fd7b8006eb0).ready entity=mon.0 client_cookie=d4c2ebb7befab5eb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:14.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7cd59a700 1 -- 172.21.2.104:0/1441454556 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7b800d5c0 con 0x7fd7d00b10c0 2021-07-11T03:57:14.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7cd59a700 1 -- 172.21.2.104:0/1441454556 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b8000e90 con 0x7fd7d00b10c0 2021-07-11T03:57:14.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7cd59a700 1 -- 172.21.2.104:0/1441454556 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7b800d910 con 0x7fd7d00b10c0 2021-07-11T03:57:14.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1441454556 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d00b10c0 msgr2=0x7fd7d00b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 --2- 172.21.2.104:0/1441454556 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d00b10c0 0x7fd7d00b14a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fd7d00a9db0 tx=0x7fd7b8006eb0).stop 2021-07-11T03:57:14.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1441454556 shutdown_connections 2021-07-11T03:57:14.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 --2- 172.21.2.104:0/1441454556 >> v2:172.21.2.105:3300/0 conn(0x7fd7d0140ba0 0x7fd7d00b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:14.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 --2- 172.21.2.104:0/1441454556 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d00b10c0 0x7fd7d00b14a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1441454556 >> 172.21.2.104:0/1441454556 conn(0x7fd7d005f6b0 msgr2=0x7fd7d005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:14.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1441454556 shutdown_connections 2021-07-11T03:57:14.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1441454556 wait complete. 2021-07-11T03:57:14.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 Processor -- start 2021-07-11T03:57:14.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- start start 2021-07-11T03:57:14.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d00acb70 0x7fd7d013e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d00b10c0 0x7fd7d013a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7d0140ba0 0x7fd7d013ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00ab030 con 0x7fd7d00acb70 2021-07-11T03:57:14.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d01465f0 con 0x7fd7d0140ba0 2021-07-11T03:57:14.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7d4f47700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0067750 con 0x7fd7d00b10c0 2021-07-11T03:57:14.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d00acb70 0x7fd7d013e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d00acb70 0x7fd7d013e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44976/0 (socket says 172.21.2.104:44976) 2021-07-11T03:57:14.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1416964151 learned_addr learned my addr 172.21.2.104:0/1416964151 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:14.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ced9d700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7d0140ba0 0x7fd7d013ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d00b10c0 0x7fd7d013a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d00b10c0 msgr2=0x7fd7d013a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d00b10c0 0x7fd7d013a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7d0140ba0 msgr2=0x7fd7d013ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7d0140ba0 0x7fd7d013ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1416964151 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b800b1e0 con 0x7fd7d00acb70 2021-07-11T03:57:14.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d00b10c0 0x7fd7d013a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:57:14.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.333+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d00acb70 0x7fd7d013e8c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fd7d00ad500 tx=0x7fd7b8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:14.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7c77fe700 1 -- 172.21.2.104:0/1416964151 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7b8021070 con 0x7fd7d00acb70 2021-07-11T03:57:14.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7c77fe700 1 -- 172.21.2.104:0/1416964151 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b8000e90 con 0x7fd7d00acb70 2021-07-11T03:57:14.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1416964151 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d00674b0 con 0x7fd7d00acb70 2021-07-11T03:57:14.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7c77fe700 1 -- 172.21.2.104:0/1416964151 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7b8006be0 con 0x7fd7d00acb70 2021-07-11T03:57:14.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1416964151 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d0146150 con 0x7fd7d00acb70 2021-07-11T03:57:14.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7c77fe700 1 -- 172.21.2.104:0/1416964151 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd7b800f070 con 0x7fd7d00acb70 2021-07-11T03:57:14.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7c77fe700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7b4053300 0x7fd7b40557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7c77fe700 1 -- 172.21.2.104:0/1416964151 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd7b806c960 con 0x7fd7d00acb70 2021-07-11T03:57:14.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7b4053300 0x7fd7b40557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 Processor -- start 2021-07-11T03:57:14.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 -- start start 2021-07-11T03:57:14.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7b4053300 0x7fd7b40557c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fd7c0007000 tx=0x7fd7c0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:14.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1553ae0 0x1557720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1556ec0 0x1553ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158caf0 0x1558e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x14c7bd0 con 0x1556ec0 2021-07-11T03:57:14.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x14c7d10 con 0x1553ae0 2021-07-11T03:57:14.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x14c93e0 con 0x158caf0 2021-07-11T03:57:14.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7cdd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1556ec0 0x1553ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7cdd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1556ec0 0x1553ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44986/0 (socket says 172.21.2.104:44986) 2021-07-11T03:57:14.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7cdd9b700 1 -- 172.21.2.104:0/2030731173 learned_addr learned my addr 172.21.2.104:0/2030731173 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:14.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/2030731173 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1553ae0 0x1557720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7ced9d700 1 --2- 172.21.2.104:0/2030731173 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158caf0 0x1558e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7cdd9b700 1 -- 172.21.2.104:0/2030731173 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158caf0 msgr2=0x1558e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:14.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/2030731173 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158caf0 0x1558e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7cdd9b700 1 -- 172.21.2.104:0/2030731173 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1553ae0 msgr2=0x1557720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:14.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/2030731173 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1553ae0 0x1557720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7cdd9b700 1 -- 172.21.2.104:0/2030731173 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1501040 con 0x1556ec0 2021-07-11T03:57:14.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/2030731173 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1556ec0 0x1553ec0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fd7c000ae00 tx=0x7fd7c0010000).ready entity=mon.0 client_cookie=85492329f424fc67 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:14.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7d4f47700 1 -- 172.21.2.104:0/2030731173 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c0019070 con 0x1556ec0 2021-07-11T03:57:14.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7d4f47700 1 -- 172.21.2.104:0/2030731173 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c0011540 con 0x1556ec0 2021-07-11T03:57:14.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7d4f47700 1 -- 172.21.2.104:0/2030731173 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c00118a0 con 0x1556ec0 2021-07-11T03:57:14.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 -- 172.21.2.104:0/2030731173 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1556ec0 msgr2=0x1553ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 --2- 172.21.2.104:0/2030731173 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1556ec0 0x1553ec0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fd7c000ae00 tx=0x7fd7c0010000).stop 2021-07-11T03:57:14.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 -- 172.21.2.104:0/2030731173 shutdown_connections 2021-07-11T03:57:14.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 --2- 172.21.2.104:0/2030731173 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158caf0 0x1558e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 --2- 172.21.2.104:0/2030731173 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1556ec0 0x1553ec0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 --2- 172.21.2.104:0/2030731173 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1553ae0 0x1557720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 -- 172.21.2.104:0/2030731173 >> 172.21.2.104:0/2030731173 conn(0x14e2950 msgr2=0x15567b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:14.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 -- 172.21.2.104:0/2030731173 shutdown_connections 2021-07-11T03:57:14.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.337+0000 7fd7df197740 1 -- 172.21.2.104:0/2030731173 wait complete. 2021-07-11T03:57:14.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- start start 2021-07-11T03:57:14.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 Processor -- start 2021-07-11T03:57:14.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1553ae0 0x155f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x158caf0 0x1589c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158f240 0x158a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x159dd90 con 0x158f240 2021-07-11T03:57:14.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x1558350 con 0x158caf0 2021-07-11T03:57:14.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x1557920 con 0x1553ae0 2021-07-11T03:57:14.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 10 client.0 ll_register_callbacks cb 0x14bc4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-07-11T03:57:14.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ced9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158f240 0x158a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ced9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158f240 0x158a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44992/0 (socket says 172.21.2.104:44992) 2021-07-11T03:57:14.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ced9d700 1 -- 172.21.2.104:0/1933700395 learned_addr learned my addr 172.21.2.104:0/1933700395 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:14.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x158caf0 0x1589c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1553ae0 0x155f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ced9d700 1 -- 172.21.2.104:0/1933700395 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1553ae0 msgr2=0x155f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:14.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ced9d700 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1553ae0 0x155f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ced9d700 1 -- 172.21.2.104:0/1933700395 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x158caf0 msgr2=0x1589c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:14.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ced9d700 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x158caf0 0x1589c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ced9d700 1 -- 172.21.2.104:0/1933700395 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c00144f0 con 0x158f240 2021-07-11T03:57:14.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1553ae0 0x155f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:57:14.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ced9d700 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158f240 0x158a130 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fd7c800bfc0 tx=0x7fd7c8008040).ready entity=mon.0 client_cookie=ab87b025045e96d5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:14.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ad7fa700 10 client.0 ms_handle_connect on v2:172.21.2.104:3300/0 2021-07-11T03:57:14.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ad7fa700 1 -- 172.21.2.104:0/1933700395 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c800fe80 con 0x158f240 2021-07-11T03:57:14.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ad7fa700 1 -- 172.21.2.104:0/1933700395 <== mon.0 v2:172.21.2.104:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd7c8013400 con 0x158f240 2021-07-11T03:57:14.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- 172.21.2.104:0/1933700395 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1341c70 con 0x158f240 2021-07-11T03:57:14.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ad7fa700 1 -- 172.21.2.104:0/1933700395 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7c80135c0 con 0x158f240 2021-07-11T03:57:14.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 10 client.5808 fetch_fsmap learned FSMap version 22 2021-07-11T03:57:14.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- 172.21.2.104:0/1933700395 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x14b80a0 con 0x158f240 2021-07-11T03:57:14.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ad7fa700 1 -- 172.21.2.104:0/1933700395 <== mon.0 v2:172.21.2.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1341c70 con 0x158f240 2021-07-11T03:57:14.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ad7fa700 1 -- 172.21.2.104:0/1933700395 <== mon.0 v2:172.21.2.104:3300/0 5 ==== fsmap(e 22) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fd7c801a020 con 0x158f240 2021-07-11T03:57:14.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 10 client.5808 fetch_fsmap finished waiting for FSMap version 22 2021-07-11T03:57:14.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 10 client.5808 resolve_mds: resolved ID 'c' to GID 5589 2021-07-11T03:57:14.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 20 client.5808 populate_metadata read hostname 'gibba004' 2021-07-11T03:57:14.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.104:6834/2428941897,v1:172.21.2.104:6835/2428941897] conn(0x155d130 0x15a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 4 client.5808 mds_command: new command op to 5589 tid=0[{"prefix": "get_command_descriptions"}] 2021-07-11T03:57:14.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- 172.21.2.104:0/1933700395 --> [v2:172.21.2.104:6834/2428941897,v1:172.21.2.104:6835/2428941897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x13574b0 con 0x155d130 2021-07-11T03:57:14.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.104:6834/2428941897,v1:172.21.2.104:6835/2428941897] conn(0x155d130 0x15a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ad7fa700 10 client.5808 ms_handle_connect on v2:172.21.2.104:6834/2428941897 2021-07-11T03:57:14.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.104:6834/2428941897,v1:172.21.2.104:6835/2428941897] conn(0x155d130 0x15a07e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=960b260093ada318 server_cookie=f181f0404d843e02 in_seq=0 out_seq=0 2021-07-11T03:57:14.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ad7fa700 1 -- 172.21.2.104:0/1933700395 <== mds.0 v2:172.21.2.104:6834/2428941897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x13574b0 con 0x155d130 2021-07-11T03:57:14.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ad7fa700 10 client.5808 handle_command_reply: tid=0 2021-07-11T03:57:14.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 client.5808 shutdown 2021-07-11T03:57:14.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- 172.21.2.104:0/1933700395 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158f240 msgr2=0x158a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158f240 0x158a130 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7fd7c800bfc0 tx=0x7fd7c8008040).stop 2021-07-11T03:57:14.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- 172.21.2.104:0/1933700395 shutdown_connections 2021-07-11T03:57:14.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.104:6834/2428941897,v1:172.21.2.104:6835/2428941897] conn(0x155d130 0x15a07e0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158f240 0x158a130 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7ad7fa700 0 client.5808 ms_handle_reset on v2:172.21.2.104:6834/2428941897 2021-07-11T03:57:14.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x158caf0 0x1589c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 --2- 172.21.2.104:0/1933700395 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1553ae0 0x155f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- 172.21.2.104:0/1933700395 >> 172.21.2.104:0/1933700395 conn(0x14e2950 msgr2=0x1553ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:14.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- 172.21.2.104:0/1933700395 shutdown_connections 2021-07-11T03:57:14.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 1 -- 172.21.2.104:0/1933700395 wait complete. 2021-07-11T03:57:14.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.341+0000 7fd7df197740 20 client.5808 trim_cache size 0 max 16384 2021-07-11T03:57:14.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 Processor -- start 2021-07-11T03:57:14.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 -- start start 2021-07-11T03:57:14.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1553ae0 0x155f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158caf0 0x159d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158f240 0x159d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x158c770 con 0x158caf0 2021-07-11T03:57:14.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x155dd80 con 0x1553ae0 2021-07-11T03:57:14.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x1515d90 con 0x158f240 2021-07-11T03:57:14.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7cdd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158caf0 0x159d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7cdd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158caf0 0x159d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:44998/0 (socket says 172.21.2.104:44998) 2021-07-11T03:57:14.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7cdd9b700 1 -- 172.21.2.104:0/3085042369 learned_addr learned my addr 172.21.2.104:0/3085042369 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:14.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/3085042369 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1553ae0 0x155f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7ced9d700 1 --2- 172.21.2.104:0/3085042369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158f240 0x159d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7cdd9b700 1 -- 172.21.2.104:0/3085042369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158f240 msgr2=0x159d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:14.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/3085042369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158f240 0x159d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7cdd9b700 1 -- 172.21.2.104:0/3085042369 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1553ae0 msgr2=0x155f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:14.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/3085042369 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1553ae0 0x155f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7cdd9b700 1 -- 172.21.2.104:0/3085042369 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d00ae130 con 0x158caf0 2021-07-11T03:57:14.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/3085042369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158caf0 0x159d1f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fd7c0021000 tx=0x7fd7c001b000).ready entity=mon.0 client_cookie=53585d79cf61cf5b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:14.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7af7fe700 1 -- 172.21.2.104:0/3085042369 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c00116b0 con 0x158caf0 2021-07-11T03:57:14.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7af7fe700 1 -- 172.21.2.104:0/3085042369 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c0026040 con 0x158caf0 2021-07-11T03:57:14.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7af7fe700 1 -- 172.21.2.104:0/3085042369 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7c0011a00 con 0x158caf0 2021-07-11T03:57:14.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 -- 172.21.2.104:0/3085042369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158caf0 msgr2=0x159d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:14.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 --2- 172.21.2.104:0/3085042369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158caf0 0x159d1f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7fd7c0021000 tx=0x7fd7c001b000).stop 2021-07-11T03:57:14.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 -- 172.21.2.104:0/3085042369 shutdown_connections 2021-07-11T03:57:14.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 --2- 172.21.2.104:0/3085042369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158f240 0x159d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 --2- 172.21.2.104:0/3085042369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x158caf0 0x159d1f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 --2- 172.21.2.104:0/3085042369 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1553ae0 0x155f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 -- 172.21.2.104:0/3085042369 >> 172.21.2.104:0/3085042369 conn(0x14e2950 msgr2=0x1556420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:14.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 -- 172.21.2.104:0/3085042369 shutdown_connections 2021-07-11T03:57:14.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.349+0000 7fd7df197740 1 -- 172.21.2.104:0/3085042369 wait complete. 2021-07-11T03:57:14.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 -- start start 2021-07-11T03:57:14.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 Processor -- start 2021-07-11T03:57:14.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1553ae0 0x15ee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x158caf0 0x15ee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158f240 0x15f2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x15eeca0 con 0x1553ae0 2021-07-11T03:57:14.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x1596e10 con 0x158caf0 2021-07-11T03:57:14.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x15f1ae0 con 0x158f240 2021-07-11T03:57:14.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 10 client.0 ll_register_callbacks cb 0x1557b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-07-11T03:57:14.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ce59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1553ae0 0x15ee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ce59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1553ae0 0x15ee2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45004/0 (socket says 172.21.2.104:45004) 2021-07-11T03:57:14.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1936847808 learned_addr learned my addr 172.21.2.104:0/1936847808 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:14.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ced9d700 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158f240 0x15f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x158caf0 0x15ee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1936847808 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158f240 msgr2=0x15f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:14.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158f240 0x15f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1936847808 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x158caf0 msgr2=0x15ee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:14.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x158caf0 0x15ee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:14.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ce59c700 1 -- 172.21.2.104:0/1936847808 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1501040 con 0x1553ae0 2021-07-11T03:57:14.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ced9d700 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158f240 0x15f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:57:14.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ce59c700 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1553ae0 0x15ee2c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fd7b8032000 tx=0x7fd7b8034040).ready entity=mon.0 client_cookie=6c0794691b245868 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:14.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ae7fc700 10 client.0 ms_handle_connect on v2:172.21.2.104:3300/0 2021-07-11T03:57:14.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ae7fc700 1 -- 172.21.2.104:0/1936847808 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd7b8037040 con 0x1553ae0 2021-07-11T03:57:14.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ae7fc700 1 -- 172.21.2.104:0/1936847808 <== mon.0 v2:172.21.2.104:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd7b8045e60 con 0x1553ae0 2021-07-11T03:57:14.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 -- 172.21.2.104:0/1936847808 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x13d2820 con 0x1553ae0 2021-07-11T03:57:14.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ae7fc700 1 -- 172.21.2.104:0/1936847808 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b8043420 con 0x1553ae0 2021-07-11T03:57:14.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 10 client.5814 fetch_fsmap learned FSMap version 22 2021-07-11T03:57:14.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 -- 172.21.2.104:0/1936847808 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x14c0c90 con 0x1553ae0 2021-07-11T03:57:14.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ae7fc700 1 -- 172.21.2.104:0/1936847808 <== mon.0 v2:172.21.2.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x13d2820 con 0x1553ae0 2021-07-11T03:57:14.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ae7fc700 1 -- 172.21.2.104:0/1936847808 <== mon.0 v2:172.21.2.104:3300/0 5 ==== fsmap(e 22) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fd7b8024020 con 0x1553ae0 2021-07-11T03:57:14.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 10 client.5814 fetch_fsmap finished waiting for FSMap version 22 2021-07-11T03:57:14.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 10 client.5814 resolve_mds: resolved ID 'c' to GID 5589 2021-07-11T03:57:14.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 20 client.5814 populate_metadata read hostname 'gibba004' 2021-07-11T03:57:14.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.104:6834/2428941897,v1:172.21.2.104:6835/2428941897] conn(0x15f2910 0x15f4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:14.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 4 client.5814 mds_command: new command op to 5589 tid=0[{"prefix": "cache drop", "timeout": 5}] 2021-07-11T03:57:14.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7df197740 1 -- 172.21.2.104:0/1936847808 --> [v2:172.21.2.104:6834/2428941897,v1:172.21.2.104:6835/2428941897] -- command(tid 0: {"prefix": "cache drop", "timeout": 5}) v1 -- 0x159ff40 con 0x15f2910 2021-07-11T03:57:14.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.104:6834/2428941897,v1:172.21.2.104:6835/2428941897] conn(0x15f2910 0x15f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:14.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7ae7fc700 10 client.5814 ms_handle_connect on v2:172.21.2.104:6834/2428941897 2021-07-11T03:57:14.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:14.353+0000 7fd7cdd9b700 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.104:6834/2428941897,v1:172.21.2.104:6835/2428941897] conn(0x15f2910 0x15f4520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d7546d14723c72ca server_cookie=bd19b56bd1478461 in_seq=0 out_seq=0 2021-07-11T03:57:19.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7ae7fc700 1 -- 172.21.2.104:0/1936847808 <== mds.0 v2:172.21.2.104:6834/2428941897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+404 (crc 0 0 0) 0x159ff40 con 0x15f2910 2021-07-11T03:57:19.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7ae7fc700 10 client.5814 handle_command_reply: tid=0 2021-07-11T03:57:19.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 1 client.5814 shutdown 2021-07-11T03:57:19.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 1 -- 172.21.2.104:0/1936847808 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1553ae0 msgr2=0x15ee2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1553ae0 0x15ee2c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fd7b8032000 tx=0x7fd7b8034040).stop 2021-07-11T03:57:19.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 1 -- 172.21.2.104:0/1936847808 shutdown_connections 2021-07-11T03:57:19.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.104:6834/2428941897,v1:172.21.2.104:6835/2428941897] conn(0x15f2910 0x15f4520 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x158f240 0x15f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x158caf0 0x15ee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 1 --2- 172.21.2.104:0/1936847808 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1553ae0 0x15ee2c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 1 -- 172.21.2.104:0/1936847808 >> 172.21.2.104:0/1936847808 conn(0x14e2950 msgr2=0x159d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:19.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7ae7fc700 0 client.5814 ms_handle_reset on v2:172.21.2.104:6834/2428941897 2021-07-11T03:57:19.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 1 -- 172.21.2.104:0/1936847808 shutdown_connections 2021-07-11T03:57:19.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 1 -- 172.21.2.104:0/1936847808 wait complete. 2021-07-11T03:57:19.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7df197740 20 client.5814 trim_cache size 0 max 16384 2021-07-11T03:57:19.455 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:57:19.456 INFO:teuthology.orchestra.run.gibba004.stdout: "client_recall": { 2021-07-11T03:57:19.456 INFO:teuthology.orchestra.run.gibba004.stdout: "return_code": -110, 2021-07-11T03:57:19.456 INFO:teuthology.orchestra.run.gibba004.stdout: "message": "(110) Connection timed out", 2021-07-11T03:57:19.456 INFO:teuthology.orchestra.run.gibba004.stdout: "recalled": 120 2021-07-11T03:57:19.457 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:57:19.457 INFO:teuthology.orchestra.run.gibba004.stdout: "flush_journal": { 2021-07-11T03:57:19.457 INFO:teuthology.orchestra.run.gibba004.stdout: "return_code": 0, 2021-07-11T03:57:19.457 INFO:teuthology.orchestra.run.gibba004.stdout: "message": "" 2021-07-11T03:57:19.458 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:57:19.458 INFO:teuthology.orchestra.run.gibba004.stdout: "trim_cache": { 2021-07-11T03:57:19.458 INFO:teuthology.orchestra.run.gibba004.stdout: "trimmed": 0 2021-07-11T03:57:19.459 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:57:19.459 INFO:teuthology.orchestra.run.gibba004.stdout: "cache": { 2021-07-11T03:57:19.459 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": { 2021-07-11T03:57:19.459 INFO:teuthology.orchestra.run.gibba004.stdout: "items": 19869, 2021-07-11T03:57:19.459 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes": 3091264 2021-07-11T03:57:19.460 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:57:19.460 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:57:19.460 INFO:teuthology.orchestra.run.gibba004.stdout: "duration": 5.0897648120000003 2021-07-11T03:57:19.460 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:57:19.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7b4053300 msgr2=0x7fd7b40557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7b4053300 0x7fd7b40557c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fd7c0007000 tx=0x7fd7c0008040).stop 2021-07-11T03:57:19.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1416964151 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d00acb70 msgr2=0x7fd7d013e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d00acb70 0x7fd7d013e8c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fd7d00ad500 tx=0x7fd7b8015000).stop 2021-07-11T03:57:19.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1416964151 shutdown_connections 2021-07-11T03:57:19.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd7d0140ba0 0x7fd7d013ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd7d00b10c0 0x7fd7d013a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7b4053300 0x7fd7b40557c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 --2- 172.21.2.104:0/1416964151 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd7d00acb70 0x7fd7d013e8c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1416964151 >> 172.21.2.104:0/1416964151 conn(0x7fd7d005f6b0 msgr2=0x7fd7d00a8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:19.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1416964151 shutdown_connections 2021-07-11T03:57:19.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.445+0000 7fd7d4f47700 1 -- 172.21.2.104:0/1416964151 wait complete. 2021-07-11T03:57:19.465 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2021-07-11T03:57:19.465 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:57:19.466 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:57:19.467 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:57:19.476 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-07-11T03:57:19.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:57:19.478 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-07-11T03:57:19.479 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T03:57:19.480 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T03:57:19.481 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:57:19.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:57:19.498 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:57:19.499 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:57:19.547 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T03:57:19.548 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T03:57:19.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:57:19.549 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:57:19.558 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:57:19.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.661+0000 7fb297e2b700 1 Processor -- start 2021-07-11T03:57:19.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.661+0000 7fb297e2b700 1 -- start start 2021-07-11T03:57:19.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb2900ab850 0x7fb2900b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb290144d50 con 0x7fb29013e950 2021-07-11T03:57:19.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb290144ff0 con 0x7fb2901445d0 2021-07-11T03:57:19.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb290145530 con 0x7fb2900ab850 2021-07-11T03:57:19.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb2963c8700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb29013e950 0x7fb2901413f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41790/0 (socket says 172.21.2.104:41790) 2021-07-11T03:57:19.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb2963c8700 1 -- 172.21.2.104:0/3055124121 learned_addr learned my addr 172.21.2.104:0/3055124121 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:57:19.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025834236 0 0) 0x7fb290144d50 con 0x7fb29013e950 2021-07-11T03:57:19.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb278003260 con 0x7fb29013e950 2021-07-11T03:57:19.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb295bc7700 1 --2- 172.21.2.104:0/3055124121 >> v2:172.21.2.105:3300/0 conn(0x7fb2900ab850 0x7fb2900b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3040730156 0 0) 0x7fb278003260 con 0x7fb29013e950 2021-07-11T03:57:19.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 >> v2:172.21.2.105:3300/0 conn(0x7fb2900ab850 msgr2=0x7fb2900b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:19.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 --2- 172.21.2.104:0/3055124121 >> v2:172.21.2.105:3300/0 conn(0x7fb2900ab850 0x7fb2900b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 >> v1:172.21.2.105:6789/0 conn(0x7fb2901445d0 legacy=0x7fb29013e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:19.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb290145290 con 0x7fb29013e950 2021-07-11T03:57:19.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fb2880019e0 con 0x7fb29013e950 2021-07-11T03:57:19.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 >> v1:172.21.2.104:6789/0 conn(0x7fb29013e950 legacy=0x7fb2901413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 --2- 172.21.2.104:0/3055124121 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb278003820 0x7fb278003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 --2- 172.21.2.104:0/3055124121 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb278004500 0x7fb278004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 --2- 172.21.2.104:0/3055124121 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb278005050 0x7fb27800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290145530 con 0x7fb278003820 2021-07-11T03:57:19.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27800bf40 con 0x7fb278004500 2021-07-11T03:57:19.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27800c100 con 0x7fb278005050 2021-07-11T03:57:19.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb295bc7700 1 --2- 172.21.2.104:0/3055124121 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb278003820 0x7fb278003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb2953c6700 1 --2- 172.21.2.104:0/3055124121 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb278004500 0x7fb278004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb295bc7700 1 -- 172.21.2.104:0/3055124121 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb278005050 msgr2=0x7fb27800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:19.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb295bc7700 1 --2- 172.21.2.104:0/3055124121 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb278005050 0x7fb27800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:19.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb295bc7700 1 -- 172.21.2.104:0/3055124121 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb278004500 msgr2=0x7fb278004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:19.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb295bc7700 1 --2- 172.21.2.104:0/3055124121 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb278004500 0x7fb278004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb295bc7700 1 -- 172.21.2.104:0/3055124121 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb278003260 con 0x7fb278003820 2021-07-11T03:57:19.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb2963c8700 1 -- 172.21.2.104:0/3055124121 reap_dead start 2021-07-11T03:57:19.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb2953c6700 1 --2- 172.21.2.104:0/3055124121 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb278004500 0x7fb278004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:57:19.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb295bc7700 1 --2- 172.21.2.104:0/3055124121 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb278003820 0x7fb278003c20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fb28c007000 tx=0x7fb28c008040).ready entity=mon.0 client_cookie=7f044e106b32cccd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:19.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb28c012470 con 0x7fb278003820 2021-07-11T03:57:19.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28c00c030 con 0x7fb278003820 2021-07-11T03:57:19.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb294bc5700 1 -- 172.21.2.104:0/3055124121 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb28c0127c0 con 0x7fb278003820 2021-07-11T03:57:19.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 -- 172.21.2.104:0/3055124121 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb278003820 msgr2=0x7fb278003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 --2- 172.21.2.104:0/3055124121 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb278003820 0x7fb278003c20 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fb28c007000 tx=0x7fb28c008040).stop 2021-07-11T03:57:19.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 -- 172.21.2.104:0/3055124121 shutdown_connections 2021-07-11T03:57:19.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 --2- 172.21.2.104:0/3055124121 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb278003820 0x7fb278003c20 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 -- 172.21.2.104:0/3055124121 >> 172.21.2.104:0/3055124121 conn(0x7fb2900b53e0 msgr2=0x7fb2900b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:19.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 -- 172.21.2.104:0/3055124121 shutdown_connections 2021-07-11T03:57:19.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 -- 172.21.2.104:0/3055124121 wait complete. 2021-07-11T03:57:19.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 Processor -- start 2021-07-11T03:57:19.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.665+0000 7fb297e2b700 1 -- start start 2021-07-11T03:57:19.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb297e2b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb278003820 0x7fb290147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb297e2b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb290140530 0x7fb290150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb297e2b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb29013f8b0 0x7fb290065f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb297e2b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290141000 con 0x7fb290140530 2021-07-11T03:57:19.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb297e2b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb290143f70 con 0x7fb278003820 2021-07-11T03:57:19.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb297e2b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb290144210 con 0x7fb29013f8b0 2021-07-11T03:57:19.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb2953c6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb290140530 0x7fb290150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb2953c6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb290140530 0x7fb290150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45032/0 (socket says 172.21.2.104:45032) 2021-07-11T03:57:19.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb2953c6700 1 -- 172.21.2.104:0/511035518 learned_addr learned my addr 172.21.2.104:0/511035518 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:19.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb2963c8700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb29013f8b0 0x7fb290065f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb295bc7700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb278003820 0x7fb290147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb2953c6700 1 -- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb29013f8b0 msgr2=0x7fb290065f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb2953c6700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb29013f8b0 0x7fb290065f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb2953c6700 1 -- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb278003820 msgr2=0x7fb290147c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb2953c6700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb278003820 0x7fb290147c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb2953c6700 1 -- 172.21.2.104:0/511035518 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29013fd30 con 0x7fb290140530 2021-07-11T03:57:19.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb2963c8700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb29013f8b0 0x7fb290065f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:57:19.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb2953c6700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb290140530 0x7fb290150020 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fb284008000 tx=0x7fb284009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:19.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb282ffd700 1 -- 172.21.2.104:0/511035518 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb284012410 con 0x7fb290140530 2021-07-11T03:57:19.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb282ffd700 1 -- 172.21.2.104:0/511035518 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb284000e90 con 0x7fb290140530 2021-07-11T03:57:19.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb282ffd700 1 -- 172.21.2.104:0/511035518 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2840127a0 con 0x7fb290140530 2021-07-11T03:57:19.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb297e2b700 1 -- 172.21.2.104:0/511035518 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb290140cf0 con 0x7fb290140530 2021-07-11T03:57:19.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb297e2b700 1 -- 172.21.2.104:0/511035518 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2901401d0 con 0x7fb290140530 2021-07-11T03:57:19.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb297e2b700 1 -- 172.21.2.104:0/511035518 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2901443d0 con 0x7fb290140530 2021-07-11T03:57:19.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb282ffd700 1 -- 172.21.2.104:0/511035518 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb284019030 con 0x7fb290140530 2021-07-11T03:57:19.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb282ffd700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb270053380 0x7fb270055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb282ffd700 1 -- 172.21.2.104:0/511035518 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb28406a960 con 0x7fb290140530 2021-07-11T03:57:19.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.669+0000 7fb295bc7700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb270053380 0x7fb270055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.673+0000 7fb295bc7700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb270053380 0x7fb270055840 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fb28c00e000 tx=0x7fb28c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:19.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.673+0000 7fb282ffd700 1 -- 172.21.2.104:0/511035518 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb284041c40 con 0x7fb290140530 2021-07-11T03:57:19.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.753+0000 7fb297e2b700 1 -- 172.21.2.104:0/511035518 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb290057350 con 0x7fb290140530 2021-07-11T03:57:19.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.753+0000 7fb282ffd700 1 -- 172.21.2.104:0/511035518 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16040 (secure 0 0 0) 0x7fb284045260 con 0x7fb290140530 2021-07-11T03:57:19.762 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:57:19.762 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":55,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:56:50.701936+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:56:48.052820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2021-07-11T03:56:48.670109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:57:19.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 -- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb270053380 msgr2=0x7fb270055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb270053380 0x7fb270055840 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fb28c00e000 tx=0x7fb28c008040).stop 2021-07-11T03:57:19.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 -- 172.21.2.104:0/511035518 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb290140530 msgr2=0x7fb290150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb290140530 0x7fb290150020 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fb284008000 tx=0x7fb284009040).stop 2021-07-11T03:57:19.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 -- 172.21.2.104:0/511035518 shutdown_connections 2021-07-11T03:57:19.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb29013f8b0 0x7fb290065f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb270053380 0x7fb270055840 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb290140530 0x7fb290150020 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 --2- 172.21.2.104:0/511035518 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb278003820 0x7fb290147c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 -- 172.21.2.104:0/511035518 >> 172.21.2.104:0/511035518 conn(0x7fb2900b53e0 msgr2=0x7fb2900b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:19.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 -- 172.21.2.104:0/511035518 shutdown_connections 2021-07-11T03:57:19.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.757+0000 7fb297e2b700 1 -- 172.21.2.104:0/511035518 wait complete. 2021-07-11T03:57:19.772 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:56:48.052820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:56:48.670109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:57:19.773 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:57:19.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.817+0000 7efcaae66700 1 Processor -- start 2021-07-11T03:57:19.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.817+0000 7efcaae66700 1 -- start start 2021-07-11T03:57:19.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.817+0000 7efcaae66700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7efca40b1ec0 0x7efca40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca40b0420 con 0x7efca40affb0 2021-07-11T03:57:19.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca40a77d0 con 0x7efca40b22a0 2021-07-11T03:57:19.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca40b0650 con 0x7efca40b1ec0 2021-07-11T03:57:19.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca3fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7efca40affb0 0x7efca40a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41808/0 (socket says 172.21.2.104:41808) 2021-07-11T03:57:19.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca3fff700 1 -- 172.21.2.104:0/4187800052 learned_addr learned my addr 172.21.2.104:0/4187800052 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:57:19.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 --2- 172.21.2.104:0/4187800052 >> v2:172.21.2.104:3300/0 conn(0x7efca40b1ec0 0x7efca40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca37fe700 1 -- 172.21.2.104:0/4187800052 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502461326 0 0) 0x7efca40b0420 con 0x7efca40affb0 2021-07-11T03:57:19.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca37fe700 1 -- 172.21.2.104:0/4187800052 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc84003260 con 0x7efca40affb0 2021-07-11T03:57:19.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 -- 172.21.2.104:0/4187800052 >> v1:172.21.2.105:6789/0 conn(0x7efca40b22a0 legacy=0x7efca4141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:19.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 -- 172.21.2.104:0/4187800052 >> v1:172.21.2.104:6789/0 conn(0x7efca40affb0 legacy=0x7efca40a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 -- 172.21.2.104:0/4187800052 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca40b08f0 con 0x7efca40b1ec0 2021-07-11T03:57:19.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 --2- 172.21.2.104:0/4187800052 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efca40b1ec0 0x7efca40b12d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7efca40b2f50 tx=0x7efc8c006df0).ready entity=mon.0 client_cookie=8059f1ae20ae357f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:19.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca37fe700 1 -- 172.21.2.104:0/4187800052 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc8c00d5c0 con 0x7efca40b1ec0 2021-07-11T03:57:19.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca37fe700 1 -- 172.21.2.104:0/4187800052 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c000e90 con 0x7efca40b1ec0 2021-07-11T03:57:19.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca37fe700 1 -- 172.21.2.104:0/4187800052 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc8c00d910 con 0x7efca40b1ec0 2021-07-11T03:57:19.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- 172.21.2.104:0/4187800052 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efca40b1ec0 msgr2=0x7efca40b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 --2- 172.21.2.104:0/4187800052 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efca40b1ec0 0x7efca40b12d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7efca40b2f50 tx=0x7efc8c006df0).stop 2021-07-11T03:57:19.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- 172.21.2.104:0/4187800052 shutdown_connections 2021-07-11T03:57:19.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 --2- 172.21.2.104:0/4187800052 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efca40b1ec0 0x7efca40b12d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- 172.21.2.104:0/4187800052 >> 172.21.2.104:0/4187800052 conn(0x7efca405f6b0 msgr2=0x7efca405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:19.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- 172.21.2.104:0/4187800052 shutdown_connections 2021-07-11T03:57:19.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- 172.21.2.104:0/4187800052 wait complete. 2021-07-11T03:57:19.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 Processor -- start 2021-07-11T03:57:19.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- start start 2021-07-11T03:57:19.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efca40affb0 0x7efca415ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efca40b1ec0 0x7efca41405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efca40b22a0 0x7efca4156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca415a1a0 con 0x7efca40affb0 2021-07-11T03:57:19.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca415a440 con 0x7efca40b22a0 2021-07-11T03:57:19.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca415a6e0 con 0x7efca40b1ec0 2021-07-11T03:57:19.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efca40affb0 0x7efca415ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efca40affb0 0x7efca415ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45044/0 (socket says 172.21.2.104:45044) 2021-07-11T03:57:19.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 -- 172.21.2.104:0/2246708071 learned_addr learned my addr 172.21.2.104:0/2246708071 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:19.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 -- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efca40b1ec0 msgr2=0x7efca41405b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:57:19.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efca40b1ec0 0x7efca41405b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:19.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 -- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efca40b22a0 msgr2=0x7efca4156ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:57:19.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efca40b22a0 0x7efca4156ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:19.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 -- 172.21.2.104:0/2246708071 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8c00b180 con 0x7efca40affb0 2021-07-11T03:57:19.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca8c02700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efca40affb0 0x7efca415ab80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7efca40b2f50 tx=0x7efc8c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:19.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca1ffb700 1 -- 172.21.2.104:0/2246708071 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc8c006b20 con 0x7efca40affb0 2021-07-11T03:57:19.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca1ffb700 1 -- 172.21.2.104:0/2246708071 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc8c002f70 con 0x7efca40affb0 2021-07-11T03:57:19.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- 172.21.2.104:0/2246708071 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca40b4470 con 0x7efca40affb0 2021-07-11T03:57:19.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efca1ffb700 1 -- 172.21.2.104:0/2246708071 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efc8c01f630 con 0x7efca40affb0 2021-07-11T03:57:19.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.821+0000 7efcaae66700 1 -- 172.21.2.104:0/2246708071 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca40ad180 con 0x7efca40affb0 2021-07-11T03:57:19.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.825+0000 7efcaae66700 1 -- 172.21.2.104:0/2246708071 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca415a8a0 con 0x7efca40affb0 2021-07-11T03:57:19.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.825+0000 7efca1ffb700 1 -- 172.21.2.104:0/2246708071 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7efc8c025020 con 0x7efca40affb0 2021-07-11T03:57:19.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.825+0000 7efca1ffb700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc880531f0 0x7efc880556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.825+0000 7efca1ffb700 1 -- 172.21.2.104:0/2246708071 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7efc8c06c710 con 0x7efca40affb0 2021-07-11T03:57:19.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.825+0000 7efca3fff700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc880531f0 0x7efc880556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.825+0000 7efca3fff700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc880531f0 0x7efc880556b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7efc94001110 tx=0x7efc94007630).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:19.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.825+0000 7efca1ffb700 1 -- 172.21.2.104:0/2246708071 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7efc8c043940 con 0x7efca40affb0 2021-07-11T03:57:19.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 -- 172.21.2.104:0/2246708071 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7efca4057350 con 0x7efca40affb0 2021-07-11T03:57:19.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efca1ffb700 1 -- 172.21.2.104:0/2246708071 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v55) v1 ==== 110+0+10 (secure 0 0 0) 0x7efc8c046f60 con 0x7efca40affb0 2021-07-11T03:57:19.916 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:57:19.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 -- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc880531f0 msgr2=0x7efc880556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc880531f0 0x7efc880556b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7efc94001110 tx=0x7efc94007630).stop 2021-07-11T03:57:19.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 -- 172.21.2.104:0/2246708071 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efca40affb0 msgr2=0x7efca415ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efca40affb0 0x7efca415ab80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7efca40b2f50 tx=0x7efc8c014000).stop 2021-07-11T03:57:19.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 -- 172.21.2.104:0/2246708071 shutdown_connections 2021-07-11T03:57:19.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efca40b22a0 0x7efca4156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:19.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efca40b1ec0 0x7efca41405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:19.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efc880531f0 0x7efc880556b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 --2- 172.21.2.104:0/2246708071 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efca40affb0 0x7efca415ab80 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 -- 172.21.2.104:0/2246708071 >> 172.21.2.104:0/2246708071 conn(0x7efca405f6b0 msgr2=0x7efca413e380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:19.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 -- 172.21.2.104:0/2246708071 shutdown_connections 2021-07-11T03:57:19.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.909+0000 7efcaae66700 1 -- 172.21.2.104:0/2246708071 wait complete. 2021-07-11T03:57:19.924 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:57:19.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.969+0000 7f52b1164700 1 Processor -- start 2021-07-11T03:57:19.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.969+0000 7f52b1164700 1 -- start start 2021-07-11T03:57:19.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.969+0000 7f52b1164700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52ac0b12d0 0x7f52ac0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.969+0000 7f52b1164700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f52ac0b22a0 0x7f52ac141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.969+0000 7f52b1164700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ac0a77d0 con 0x7f52ac0b1ec0 2021-07-11T03:57:19.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.969+0000 7f52b1164700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac0b1740 con 0x7f52ac0b22a0 2021-07-11T03:57:19.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.969+0000 7f52b1164700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0b0650 con 0x7f52ac0b12d0 2021-07-11T03:57:19.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52aa59c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52ac0b12d0 0x7f52ac0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52ab59e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f52ac0b22a0 0x7f52ac141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:19.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52aad9d700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f52ac0b1ec0 0x7f52ac0a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:41190/0 (socket says 172.21.2.104:41190) 2021-07-11T03:57:19.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52aad9d700 1 -- 172.21.2.104:0/46463271 learned_addr learned my addr 172.21.2.104:0/46463271 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:57:19.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52aa59c700 1 -- 172.21.2.104:0/46463271 >> v2:172.21.2.105:3300/0 conn(0x7f52ac0b22a0 msgr2=0x7f52ac141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:19.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52aa59c700 1 --2- 172.21.2.104:0/46463271 >> v2:172.21.2.105:3300/0 conn(0x7f52ac0b22a0 0x7f52ac141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52aa59c700 1 -- 172.21.2.104:0/46463271 >> v1:172.21.2.105:6789/0 conn(0x7f52ac0b1ec0 legacy=0x7f52ac0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52aa59c700 1 -- 172.21.2.104:0/46463271 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0b08f0 con 0x7f52ac0b12d0 2021-07-11T03:57:19.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52aa59c700 1 --2- 172.21.2.104:0/46463271 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52ac0b12d0 0x7f52ac0affb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f529c002f70 tx=0x7f529c006df0).ready entity=mon.2 client_cookie=dd29ccb051d6efde server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:19.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52a9d9b700 1 -- 172.21.2.104:0/46463271 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529c00d570 con 0x7f52ac0b12d0 2021-07-11T03:57:19.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52a9d9b700 1 -- 172.21.2.104:0/46463271 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c000e90 con 0x7f52ac0b12d0 2021-07-11T03:57:19.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52a9d9b700 1 -- 172.21.2.104:0/46463271 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529c00d8c0 con 0x7f52ac0b12d0 2021-07-11T03:57:19.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 -- 172.21.2.104:0/46463271 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52ac0b12d0 msgr2=0x7f52ac0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:19.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 --2- 172.21.2.104:0/46463271 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52ac0b12d0 0x7f52ac0affb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f529c002f70 tx=0x7f529c006df0).stop 2021-07-11T03:57:19.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 -- 172.21.2.104:0/46463271 shutdown_connections 2021-07-11T03:57:19.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 --2- 172.21.2.104:0/46463271 >> v2:172.21.2.105:3300/0 conn(0x7f52ac0b22a0 0x7f52ac141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 --2- 172.21.2.104:0/46463271 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52ac0b12d0 0x7f52ac0affb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:19.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 -- 172.21.2.104:0/46463271 >> 172.21.2.104:0/46463271 conn(0x7f52ac05f6b0 msgr2=0x7f52ac05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:19.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 -- 172.21.2.104:0/46463271 shutdown_connections 2021-07-11T03:57:19.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 -- 172.21.2.104:0/46463271 wait complete. 2021-07-11T03:57:19.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 Processor -- start 2021-07-11T03:57:19.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 -- start start 2021-07-11T03:57:19.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52ac0b12d0 0x7f52ac0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52ac0b1ec0 0x7f52ac14ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f52ac0b22a0 0x7f52ac14b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:19.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac149330 con 0x7f52ac0b12d0 2021-07-11T03:57:19.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac156960 con 0x7f52ac0b22a0 2021-07-11T03:57:19.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52b1164700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac149560 con 0x7f52ac0b1ec0 2021-07-11T03:57:20.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52aad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52ac0b12d0 0x7f52ac0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52aad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52ac0b12d0 0x7f52ac0ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45060/0 (socket says 172.21.2.104:45060) 2021-07-11T03:57:20.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.973+0000 7f52aad9d700 1 -- 172.21.2.104:0/2214744901 learned_addr learned my addr 172.21.2.104:0/2214744901 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:20.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52aad9d700 1 -- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52ac0b1ec0 msgr2=0x7f52ac14ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52aad9d700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52ac0b1ec0 0x7f52ac14ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52aad9d700 1 -- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f52ac0b22a0 msgr2=0x7f52ac14b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52aad9d700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f52ac0b22a0 0x7f52ac14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52aad9d700 1 -- 172.21.2.104:0/2214744901 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c00b180 con 0x7f52ac0b12d0 2021-07-11T03:57:20.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52aad9d700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52ac0b12d0 0x7f52ac0ae610 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f52ac06a0f0 tx=0x7f5294007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:20.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52a3fff700 1 -- 172.21.2.104:0/2214744901 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f529400ce30 con 0x7f52ac0b12d0 2021-07-11T03:57:20.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52a3fff700 1 -- 172.21.2.104:0/2214744901 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52940072e0 con 0x7f52ac0b12d0 2021-07-11T03:57:20.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52a3fff700 1 -- 172.21.2.104:0/2214744901 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5294005800 con 0x7f52ac0b12d0 2021-07-11T03:57:20.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52b1164700 1 -- 172.21.2.104:0/2214744901 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac149800 con 0x7f52ac0b12d0 2021-07-11T03:57:20.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52b1164700 1 -- 172.21.2.104:0/2214744901 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac14b6f0 con 0x7f52ac0b12d0 2021-07-11T03:57:20.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52b1164700 1 -- 172.21.2.104:0/2214744901 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac13e410 con 0x7f52ac0b12d0 2021-07-11T03:57:20.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52a3fff700 1 -- 172.21.2.104:0/2214744901 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5294010070 con 0x7f52ac0b12d0 2021-07-11T03:57:20.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52a3fff700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5290053330 0x7f52900557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52a3fff700 1 -- 172.21.2.104:0/2214744901 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f529406aae0 con 0x7f52ac0b12d0 2021-07-11T03:57:20.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52aa59c700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5290053330 0x7f52900557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52aa59c700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5290053330 0x7f52900557f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f529c002f70 tx=0x7f529c014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:20.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:19.977+0000 7f52a3fff700 1 -- 172.21.2.104:0/2214744901 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5294041e20 con 0x7f52ac0b12d0 2021-07-11T03:57:20.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 -- 172.21.2.104:0/2214744901 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f52ac057350 con 0x7f52ac0b12d0 2021-07-11T03:57:20.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52a3fff700 1 -- 172.21.2.104:0/2214744901 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v55) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5294018030 con 0x7f52ac0b12d0 2021-07-11T03:57:20.069 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:57:20.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 -- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5290053330 msgr2=0x7f52900557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5290053330 0x7f52900557f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f529c002f70 tx=0x7f529c014000).stop 2021-07-11T03:57:20.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 -- 172.21.2.104:0/2214744901 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52ac0b12d0 msgr2=0x7f52ac0ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52ac0b12d0 0x7f52ac0ae610 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f52ac06a0f0 tx=0x7f5294007c40).stop 2021-07-11T03:57:20.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 -- 172.21.2.104:0/2214744901 shutdown_connections 2021-07-11T03:57:20.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f52ac0b22a0 0x7f52ac14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52ac0b1ec0 0x7f52ac14ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5290053330 0x7f52900557f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:20.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 --2- 172.21.2.104:0/2214744901 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52ac0b12d0 0x7f52ac0ae610 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:20.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 -- 172.21.2.104:0/2214744901 >> 172.21.2.104:0/2214744901 conn(0x7f52ac05f6b0 msgr2=0x7f52ac1481a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:20.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 -- 172.21.2.104:0/2214744901 shutdown_connections 2021-07-11T03:57:20.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.061+0000 7f52b1164700 1 -- 172.21.2.104:0/2214744901 wait complete. 2021-07-11T03:57:20.076 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:57:20.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.121+0000 7f50a93e6700 1 Processor -- start 2021-07-11T03:57:20.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.121+0000 7f50a93e6700 1 -- start start 2021-07-11T03:57:20.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50a40b12d0 0x7f50a40affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f50a40b22a0 0x7f50a4141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50a40a77d0 con 0x7f50a40b1ec0 2021-07-11T03:57:20.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a40b1740 con 0x7f50a40b22a0 2021-07-11T03:57:20.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a40b0650 con 0x7f50a40b12d0 2021-07-11T03:57:20.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a2ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f50a40b1ec0 0x7f50a40a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41834/0 (socket says 172.21.2.104:41834) 2021-07-11T03:57:20.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a2ffd700 1 -- 172.21.2.104:0/1159278280 learned_addr learned my addr 172.21.2.104:0/1159278280 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:57:20.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 --2- 172.21.2.104:0/1159278280 >> v2:172.21.2.104:3300/0 conn(0x7f50a40b22a0 0x7f50a4141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 -- 172.21.2.104:0/1159278280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50a40b12d0 msgr2=0x7f50a40affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:20.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 --2- 172.21.2.104:0/1159278280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50a40b12d0 0x7f50a40affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a1ffb700 1 -- 172.21.2.104:0/1159278280 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896396285 0 0) 0x7f50a40a77d0 con 0x7f50a40b1ec0 2021-07-11T03:57:20.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 -- 172.21.2.104:0/1159278280 >> v1:172.21.2.104:6789/0 conn(0x7f50a40b1ec0 legacy=0x7f50a40a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 -- 172.21.2.104:0/1159278280 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a40b08f0 con 0x7f50a40b22a0 2021-07-11T03:57:20.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 --2- 172.21.2.104:0/1159278280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50a40b22a0 0x7f50a4141250 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f509c0032f0 tx=0x7f509c0091d0).ready entity=mon.0 client_cookie=558de35056574520 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:20.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a1ffb700 1 -- 172.21.2.104:0/1159278280 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f509c009ea0 con 0x7f50a40b22a0 2021-07-11T03:57:20.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a1ffb700 1 -- 172.21.2.104:0/1159278280 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c00b8a0 con 0x7f50a40b22a0 2021-07-11T03:57:20.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a1ffb700 1 -- 172.21.2.104:0/1159278280 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f509c00ba10 con 0x7f50a40b22a0 2021-07-11T03:57:20.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- 172.21.2.104:0/1159278280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50a40b22a0 msgr2=0x7f50a4141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 --2- 172.21.2.104:0/1159278280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50a40b22a0 0x7f50a4141250 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f509c0032f0 tx=0x7f509c0091d0).stop 2021-07-11T03:57:20.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- 172.21.2.104:0/1159278280 shutdown_connections 2021-07-11T03:57:20.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 --2- 172.21.2.104:0/1159278280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50a40b22a0 0x7f50a4141250 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:20.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 --2- 172.21.2.104:0/1159278280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50a40b12d0 0x7f50a40affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- 172.21.2.104:0/1159278280 >> 172.21.2.104:0/1159278280 conn(0x7f50a405f6b0 msgr2=0x7f50a405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:20.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- 172.21.2.104:0/1159278280 shutdown_connections 2021-07-11T03:57:20.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- 172.21.2.104:0/1159278280 wait complete. 2021-07-11T03:57:20.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 Processor -- start 2021-07-11T03:57:20.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- start start 2021-07-11T03:57:20.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f50a40b12d0 0x7f50a40aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50a40b1ec0 0x7f50a40b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50a40b22a0 0x7f50a4156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4157700 con 0x7f50a40b22a0 2021-07-11T03:57:20.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a40ae4e0 con 0x7f50a40b12d0 2021-07-11T03:57:20.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a40ae780 con 0x7f50a40b1ec0 2021-07-11T03:57:20.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50a40b22a0 0x7f50a4156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50a40b22a0 0x7f50a4156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45076/0 (socket says 172.21.2.104:45076) 2021-07-11T03:57:20.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 -- 172.21.2.104:0/4260321276 learned_addr learned my addr 172.21.2.104:0/4260321276 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:20.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 -- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50a40b1ec0 msgr2=0x7f50a40b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50a40b1ec0 0x7f50a40b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 -- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f50a40b12d0 msgr2=0x7f50a40aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f50a40b12d0 0x7f50a40aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 -- 172.21.2.104:0/4260321276 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c009470 con 0x7f50a40b22a0 2021-07-11T03:57:20.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a37fe700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50a40b22a0 0x7f50a4156cc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f509c0032f0 tx=0x7f509c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:20.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f509bfff700 1 -- 172.21.2.104:0/4260321276 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f509c021440 con 0x7f50a40b22a0 2021-07-11T03:57:20.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f509bfff700 1 -- 172.21.2.104:0/4260321276 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c0215a0 con 0x7f50a40b22a0 2021-07-11T03:57:20.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- 172.21.2.104:0/4260321276 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a40ae200 con 0x7f50a40b22a0 2021-07-11T03:57:20.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f509bfff700 1 -- 172.21.2.104:0/4260321276 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f509c021900 con 0x7f50a40b22a0 2021-07-11T03:57:20.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.125+0000 7f50a93e6700 1 -- 172.21.2.104:0/4260321276 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a40b3700 con 0x7f50a40b22a0 2021-07-11T03:57:20.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.129+0000 7f50a93e6700 1 -- 172.21.2.104:0/4260321276 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a40b39b0 con 0x7f50a40b22a0 2021-07-11T03:57:20.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.129+0000 7f509bfff700 1 -- 172.21.2.104:0/4260321276 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f509c027020 con 0x7f50a40b22a0 2021-07-11T03:57:20.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.129+0000 7f509bfff700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5088053300 0x7f50880557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.129+0000 7f509bfff700 1 -- 172.21.2.104:0/4260321276 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f509c06c930 con 0x7f50a40b22a0 2021-07-11T03:57:20.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.129+0000 7f50a2ffd700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5088053300 0x7f50880557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.129+0000 7f50a2ffd700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5088053300 0x7f50880557c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f508c000bc0 tx=0x7f508c0011a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:20.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.129+0000 7f509bfff700 1 -- 172.21.2.104:0/4260321276 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f509c043cf0 con 0x7f50a40b22a0 2021-07-11T03:57:20.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 -- 172.21.2.104:0/4260321276 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50a40ae950 con 0x7f50a40b22a0 2021-07-11T03:57:20.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f509bfff700 1 -- 172.21.2.104:0/4260321276 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v55) v1 ==== 104+0+10 (secure 0 0 0) 0x7f509c047290 con 0x7f50a40b22a0 2021-07-11T03:57:20.222 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:57:20.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 -- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5088053300 msgr2=0x7f50880557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5088053300 0x7f50880557c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f508c000bc0 tx=0x7f508c0011a0).stop 2021-07-11T03:57:20.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 -- 172.21.2.104:0/4260321276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50a40b22a0 msgr2=0x7f50a4156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50a40b22a0 0x7f50a4156cc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f509c0032f0 tx=0x7f509c016000).stop 2021-07-11T03:57:20.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 -- 172.21.2.104:0/4260321276 shutdown_connections 2021-07-11T03:57:20.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5088053300 0x7f50880557c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:20.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50a40b22a0 0x7f50a4156cc0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:20.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50a40b1ec0 0x7f50a40b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 --2- 172.21.2.104:0/4260321276 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f50a40b12d0 0x7f50a40aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 -- 172.21.2.104:0/4260321276 >> 172.21.2.104:0/4260321276 conn(0x7f50a405f6b0 msgr2=0x7f50a413e060 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:20.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 -- 172.21.2.104:0/4260321276 shutdown_connections 2021-07-11T03:57:20.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.213+0000 7f50a93e6700 1 -- 172.21.2.104:0/4260321276 wait complete. 2021-07-11T03:57:20.229 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:57:20.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 Processor -- start 2021-07-11T03:57:20.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 -- start start 2021-07-11T03:57:20.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43700a9ec0 0x7f43700aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f437013f790 0x7f437013fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4370061990 con 0x7f43700603d0 2021-07-11T03:57:20.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4370140110 con 0x7f437013f790 2021-07-11T03:57:20.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4370061c30 con 0x7f43700a9ec0 2021-07-11T03:57:20.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f436f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f437013f790 0x7f437013fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f436e7fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f43700603d0 0x7f437013ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41850/0 (socket says 172.21.2.104:41850) 2021-07-11T03:57:20.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f436e7fc700 1 -- 172.21.2.104:0/3811351102 learned_addr learned my addr 172.21.2.104:0/3811351102 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:57:20.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f436f7fe700 1 -- 172.21.2.104:0/3811351102 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43700a9ec0 msgr2=0x7f43700aa2a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:57:20.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f436f7fe700 1 --2- 172.21.2.104:0/3811351102 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43700a9ec0 0x7f43700aa2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f436f7fe700 1 -- 172.21.2.104:0/3811351102 >> v1:172.21.2.104:6789/0 conn(0x7f43700603d0 legacy=0x7f437013ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f436f7fe700 1 -- 172.21.2.104:0/3811351102 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4370060b90 con 0x7f437013f790 2021-07-11T03:57:20.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f436f7fe700 1 --2- 172.21.2.104:0/3811351102 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f437013f790 0x7f437013fc20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f43680032f0 tx=0x7f43680091d0).ready entity=mon.0 client_cookie=ee1d4ad6d1c76d88 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:20.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f436dffb700 1 -- 172.21.2.104:0/3811351102 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4368009ea0 con 0x7f437013f790 2021-07-11T03:57:20.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f436dffb700 1 -- 172.21.2.104:0/3811351102 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f436800cba0 con 0x7f437013f790 2021-07-11T03:57:20.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f436dffb700 1 -- 172.21.2.104:0/3811351102 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4368010410 con 0x7f437013f790 2021-07-11T03:57:20.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 -- 172.21.2.104:0/3811351102 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f437013f790 msgr2=0x7f437013fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 --2- 172.21.2.104:0/3811351102 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f437013f790 0x7f437013fc20 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f43680032f0 tx=0x7f43680091d0).stop 2021-07-11T03:57:20.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 -- 172.21.2.104:0/3811351102 shutdown_connections 2021-07-11T03:57:20.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 --2- 172.21.2.104:0/3811351102 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f437013f790 0x7f437013fc20 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:20.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 --2- 172.21.2.104:0/3811351102 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43700a9ec0 0x7f43700aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 -- 172.21.2.104:0/3811351102 >> 172.21.2.104:0/3811351102 conn(0x7f43700b2970 msgr2=0x7f43700b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:20.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 -- 172.21.2.104:0/3811351102 shutdown_connections 2021-07-11T03:57:20.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 -- 172.21.2.104:0/3811351102 wait complete. 2021-07-11T03:57:20.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 Processor -- start 2021-07-11T03:57:20.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.277+0000 7f43757fd700 1 -- start start 2021-07-11T03:57:20.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f43757fd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43700603d0 0x7f4370064ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f43757fd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43700a9ec0 0x7f43700666f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f43757fd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f437013f790 0x7f43700ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f43757fd700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43700639d0 con 0x7f43700a9ec0 2021-07-11T03:57:20.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f43757fd700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43700b02f0 con 0x7f43700603d0 2021-07-11T03:57:20.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f43757fd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4370063c00 con 0x7f437013f790 2021-07-11T03:57:20.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f436e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43700a9ec0 0x7f43700666f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f436e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43700a9ec0 0x7f43700666f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45084/0 (socket says 172.21.2.104:45084) 2021-07-11T03:57:20.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f436e7fc700 1 -- 172.21.2.104:0/2439257300 learned_addr learned my addr 172.21.2.104:0/2439257300 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:20.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f436e7fc700 1 -- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f437013f790 msgr2=0x7f43700ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f436e7fc700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f437013f790 0x7f43700ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f436e7fc700 1 -- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43700603d0 msgr2=0x7f4370064ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f436e7fc700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43700603d0 0x7f4370064ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f436e7fc700 1 -- 172.21.2.104:0/2439257300 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4368009470 con 0x7f43700a9ec0 2021-07-11T03:57:20.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f436e7fc700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43700a9ec0 0x7f43700666f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f437005fd90 tx=0x7f4358007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:20.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f4367fff700 1 -- 172.21.2.104:0/2439257300 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f435800ce30 con 0x7f43700a9ec0 2021-07-11T03:57:20.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f4367fff700 1 -- 172.21.2.104:0/2439257300 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43580072e0 con 0x7f43700a9ec0 2021-07-11T03:57:20.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f43757fd700 1 -- 172.21.2.104:0/2439257300 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437013eea0 con 0x7f43700a9ec0 2021-07-11T03:57:20.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f43757fd700 1 -- 172.21.2.104:0/2439257300 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4370066be0 con 0x7f43700a9ec0 2021-07-11T03:57:20.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f4367fff700 1 -- 172.21.2.104:0/2439257300 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4358005800 con 0x7f43700a9ec0 2021-07-11T03:57:20.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f43757fd700 1 -- 172.21.2.104:0/2439257300 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43700b0460 con 0x7f43700a9ec0 2021-07-11T03:57:20.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f4367fff700 1 -- 172.21.2.104:0/2439257300 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4358010070 con 0x7f43700a9ec0 2021-07-11T03:57:20.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f4367fff700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4354053240 0x7f4354055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f4367fff700 1 -- 172.21.2.104:0/2439257300 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f435806abe0 con 0x7f43700a9ec0 2021-07-11T03:57:20.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f436effd700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4354053240 0x7f4354055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.281+0000 7f436effd700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4354053240 0x7f4354055700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f4360002fb0 tx=0x7f43600069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:20.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.285+0000 7f4367fff700 1 -- 172.21.2.104:0/2439257300 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f43700b0460 con 0x7f43700a9ec0 2021-07-11T03:57:20.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.365+0000 7f43757fd700 1 -- 172.21.2.104:0/2439257300 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43700b0460 con 0x7f43700a9ec0 2021-07-11T03:57:20.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.365+0000 7f4367fff700 1 -- 172.21.2.104:0/2439257300 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v55) v1 ==== 100+0+10 (secure 0 0 0) 0x7f43700b0460 con 0x7f43700a9ec0 2021-07-11T03:57:20.375 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:57:20.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 -- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4354053240 msgr2=0x7f4354055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4354053240 0x7f4354055700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f4360002fb0 tx=0x7f43600069b0).stop 2021-07-11T03:57:20.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 -- 172.21.2.104:0/2439257300 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43700a9ec0 msgr2=0x7f43700666f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43700a9ec0 0x7f43700666f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f437005fd90 tx=0x7f4358007c40).stop 2021-07-11T03:57:20.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 -- 172.21.2.104:0/2439257300 shutdown_connections 2021-07-11T03:57:20.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f437013f790 0x7f43700ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4354053240 0x7f4354055700 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:20.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43700a9ec0 0x7f43700666f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:20.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 --2- 172.21.2.104:0/2439257300 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43700603d0 0x7f4370064ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 -- 172.21.2.104:0/2439257300 >> 172.21.2.104:0/2439257300 conn(0x7f43700b2970 msgr2=0x7f43700b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:20.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 -- 172.21.2.104:0/2439257300 shutdown_connections 2021-07-11T03:57:20.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.369+0000 7f43757fd700 1 -- 172.21.2.104:0/2439257300 wait complete. 2021-07-11T03:57:20.381 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T03:57:20.382 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:57:20.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4cb7b700 1 Processor -- start 2021-07-11T03:57:20.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4cb7b700 1 -- start start 2021-07-11T03:57:20.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4cb7b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c480b2660 0x7f8c4805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4cb7b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c48140380 con 0x7f8c48097100 2021-07-11T03:57:20.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4cb7b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c480b2a40 con 0x7f8c48096cc0 2021-07-11T03:57:20.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4cb7b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c48097b80 con 0x7f8c480b2660 2021-07-11T03:57:20.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c46d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8c48097100 0x7f8c4813fc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41864/0 (socket says 172.21.2.104:41864) 2021-07-11T03:57:20.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c46d9d700 1 -- 172.21.2.104:0/4021749445 learned_addr learned my addr 172.21.2.104:0/4021749445 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:57:20.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739925467 0 0) 0x7f8c40001490 con 0x7f8c48097100 2021-07-11T03:57:20.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c2c003260 con 0x7f8c48097100 2021-07-11T03:57:20.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4659c700 1 --2- 172.21.2.104:0/4021749445 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c480b2660 0x7f8c4805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2727514603 0 0) 0x7f8c2c003260 con 0x7f8c48097100 2021-07-11T03:57:20.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c480b2660 msgr2=0x7f8c4805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:20.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4559a700 1 --2- 172.21.2.104:0/4021749445 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c480b2660 0x7f8c4805f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:20.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 >> v1:172.21.2.105:6789/0 conn(0x7f8c48096cc0 legacy=0x7f8c4813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:20.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48097e20 con 0x7f8c48097100 2021-07-11T03:57:20.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.429+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f8c40001b70 con 0x7f8c48097100 2021-07-11T03:57:20.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 >> v1:172.21.2.104:6789/0 conn(0x7f8c48097100 legacy=0x7f8c4813fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4559a700 1 --2- 172.21.2.104:0/4021749445 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c2c003800 0x7f8c2c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4559a700 1 --2- 172.21.2.104:0/4021749445 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c2c004500 0x7f8c2c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4559a700 1 --2- 172.21.2.104:0/4021749445 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c480b2660 0x7f8c2c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c48097b80 con 0x7f8c2c003800 2021-07-11T03:57:20.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c0052b0 con 0x7f8c2c004500 2021-07-11T03:57:20.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c00bc40 con 0x7f8c480b2660 2021-07-11T03:57:20.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 --2- 172.21.2.104:0/4021749445 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c2c003800 0x7f8c2c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 -- 172.21.2.104:0/4021749445 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c480b2660 msgr2=0x7f8c2c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:20.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 --2- 172.21.2.104:0/4021749445 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c480b2660 0x7f8c2c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 -- 172.21.2.104:0/4021749445 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c2c004500 msgr2=0x7f8c2c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:57:20.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 --2- 172.21.2.104:0/4021749445 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c2c004500 0x7f8c2c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 -- 172.21.2.104:0/4021749445 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2c003260 con 0x7f8c2c003800 2021-07-11T03:57:20.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 --2- 172.21.2.104:0/4021749445 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c2c003800 0x7f8c2c003c20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f8c38007000 tx=0x7f8c38008040).ready entity=mon.0 client_cookie=10bcb61de7be517e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:20.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c38012470 con 0x7f8c2c003800 2021-07-11T03:57:20.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3800c030 con 0x7f8c2c003800 2021-07-11T03:57:20.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4559a700 1 -- 172.21.2.104:0/4021749445 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c380127c0 con 0x7f8c2c003800 2021-07-11T03:57:20.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4021749445 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c2c003800 msgr2=0x7f8c2c003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 --2- 172.21.2.104:0/4021749445 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c2c003800 0x7f8c2c003c20 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f8c38007000 tx=0x7f8c38008040).stop 2021-07-11T03:57:20.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c46d9d700 1 -- 172.21.2.104:0/4021749445 reap_dead start 2021-07-11T03:57:20.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4021749445 shutdown_connections 2021-07-11T03:57:20.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4021749445 >> 172.21.2.104:0/4021749445 conn(0x7f8c48062f10 msgr2=0x7f8c480632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:20.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4021749445 shutdown_connections 2021-07-11T03:57:20.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4021749445 wait complete. 2021-07-11T03:57:20.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 Processor -- start 2021-07-11T03:57:20.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- start start 2021-07-11T03:57:20.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c481491c0 0x7f8c480ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c480ad7d0 0x7f8c48159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c4815a2c0 0x7f8c480b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c481495a0 con 0x7f8c481491c0 2021-07-11T03:57:20.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480adf10 con 0x7f8c4815a2c0 2021-07-11T03:57:20.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c480ae1b0 con 0x7f8c480ad7d0 2021-07-11T03:57:20.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c481491c0 0x7f8c480ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c481491c0 0x7f8c480ad2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45106/0 (socket says 172.21.2.104:45106) 2021-07-11T03:57:20.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 -- 172.21.2.104:0/4214799831 learned_addr learned my addr 172.21.2.104:0/4214799831 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:57:20.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 -- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c480ad7d0 msgr2=0x7f8c48159dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c480ad7d0 0x7f8c48159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 -- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c4815a2c0 msgr2=0x7f8c480b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c4815a2c0 0x7f8c480b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 -- 172.21.2.104:0/4214799831 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c480ae450 con 0x7f8c481491c0 2021-07-11T03:57:20.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4659c700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c481491c0 0x7f8c480ad2e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f8c38007000 tx=0x7f8c38008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:20.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c377fe700 1 -- 172.21.2.104:0/4214799831 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c38011df0 con 0x7f8c481491c0 2021-07-11T03:57:20.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c377fe700 1 -- 172.21.2.104:0/4214799831 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c38000e90 con 0x7f8c481491c0 2021-07-11T03:57:20.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c377fe700 1 -- 172.21.2.104:0/4214799831 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c3801c600 con 0x7f8c481491c0 2021-07-11T03:57:20.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4214799831 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c480ae6f0 con 0x7f8c481491c0 2021-07-11T03:57:20.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4214799831 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4815a750 con 0x7f8c481491c0 2021-07-11T03:57:20.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.433+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4214799831 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c480ae880 con 0x7f8c481491c0 2021-07-11T03:57:20.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.437+0000 7f8c377fe700 1 -- 172.21.2.104:0/4214799831 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8c3802b070 con 0x7f8c481491c0 2021-07-11T03:57:20.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.437+0000 7f8c377fe700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c240532b0 0x7f8c24055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:57:20.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.437+0000 7f8c377fe700 1 -- 172.21.2.104:0/4214799831 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8c3806ce10 con 0x7f8c481491c0 2021-07-11T03:57:20.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.437+0000 7f8c45d9b700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c240532b0 0x7f8c24055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:57:20.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.437+0000 7f8c45d9b700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c240532b0 0x7f8c24055770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f8c480b20e0 tx=0x7f8c3c007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:57:20.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.437+0000 7f8c377fe700 1 -- 172.21.2.104:0/4214799831 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8c38044040 con 0x7f8c481491c0 2021-07-11T03:57:20.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.545+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4214799831 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c48057350 con 0x7f8c481491c0 2021-07-11T03:57:20.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.545+0000 7f8c377fe700 1 -- 172.21.2.104:0/4214799831 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f8c38047660 con 0x7f8c481491c0 2021-07-11T03:57:20.554 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:57:20.555 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":22,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3427300973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3427300973},{"type":"v1","addr":"172.21.2.105:6835","nonce":3427300973}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/2823255258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2823255258},{"type":"v1","addr":"172.21.2.104:6833","nonce":2823255258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:56:49.697379+0000","modified":"2021-07-11T03:56:50.700784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/2428941897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2428941897},{"type":"v1","addr":"172.21.2.104:6835","nonce":2428941897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-07-11T03:57:20.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c240532b0 msgr2=0x7f8c24055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c240532b0 0x7f8c24055770 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f8c480b20e0 tx=0x7f8c3c007810).stop 2021-07-11T03:57:20.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4214799831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c481491c0 msgr2=0x7f8c480ad2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:57:20.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c481491c0 0x7f8c480ad2e0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f8c38007000 tx=0x7f8c38008040).stop 2021-07-11T03:57:20.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4214799831 shutdown_connections 2021-07-11T03:57:20.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c4815a2c0 0x7f8c480b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c480ad7d0 0x7f8c48159dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:57:20.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c240532b0 0x7f8c24055770 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:20.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 --2- 172.21.2.104:0/4214799831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c481491c0 0x7f8c480ad2e0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:57:20.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4214799831 >> 172.21.2.104:0/4214799831 conn(0x7f8c48062f10 msgr2=0x7f8c4813e930 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:57:20.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4214799831 shutdown_connections 2021-07-11T03:57:20.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:57:20.549+0000 7f8c4cb7b700 1 -- 172.21.2.104:0/4214799831 wait complete. 2021-07-11T03:57:20.558 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 22 2021-07-11T03:57:20.863 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 22, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:56:49.697379+0000', 'modified': '2021-07-11T03:56:50.700784+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5589': {'gid': 5589, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.2.104:6835/2428941897', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 2428941897}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 2428941897}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:57:20.864 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:57:20.865 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:57:20.945 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":5589,"want_state":"up:active","state":"up:active","rank_uptime":31.239126964,"mdsmap_epoch":22,"osdmap_epoch":55,"osdmap_epoch_barrier":54,"uptime":34.964840522999999} 2021-07-11T03:57:20.946 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 5589, "mdsmap_epoch": 22, "osdmap_epoch": 55, "osdmap_epoch_barrier": 54, "rank_uptime": 31.239126964, "state": "up:active", "uptime": 34.964840523, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:57:20.946 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T03:57:20.947 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T03:57:20.947 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T03:57:20.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:57:20.948 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:57:20.959 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:57:20.970 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:57:20.971 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:57:20.977 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:57:20.978 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:57:20.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:57:20.979 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:57:20.983 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T03:57:20.984 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:57:21.028 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:57:21.037 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:57:21.037 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:57:21.042 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:57:21.043 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:57:21.044 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:57:21.044 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:57:21.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14953]: starting ceph client 2021-07-11T03:57:21.068 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T03:57:21.071+0000 7f092f927080 -1 init, newargv = 0x561061e6bbd0 newargc=9 2021-07-11T03:57:21.085 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14953]: starting fuse 2021-07-11T03:57:22.174 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:57:22.187 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:57:22.187 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:57:22.237 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:57:22.238 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:57:22.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:57:22.239 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:57:22.246 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T03:57:22.246 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T03:57:22.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:57:22.247 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T03:57:22.247 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T03:57:22.248 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T03:57:22.248 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:57:22.249 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T03:57:22.249 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:57:22.250 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T03:57:22.250 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T03:57:22.250 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T03:57:22.251 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:57:22.251 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T03:57:22.251 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T03:57:22.252 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T03:57:22.252 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T03:57:22.253 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:57:22.253 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T03:57:22.253 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T03:57:22.254 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T03:57:22.254 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T03:57:22.255 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T03:57:22.255 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:57:22.255 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T03:57:22.256 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:57:22.325 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.14953.asok 2021-07-11T03:57:22.327 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.14953.asok 2021-07-11T03:57:22.328 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:57:22.328 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.14953.asok status 2021-07-11T03:57:22.375 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T03:57:22.376 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T03:57:22.376 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T03:57:22.376 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T03:57:22.377 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T03:57:22.377 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T03:57:22.377 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T03:57:22.378 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "14953", 2021-07-11T03:57:22.378 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T03:57:22.378 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:57:22.378 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T03:57:22.379 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T03:57:22.379 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4675, 2021-07-11T03:57:22.379 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T03:57:22.379 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T03:57:22.379 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T03:57:22.380 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4675 2021-07-11T03:57:22.380 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:57:22.380 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:57:22.380 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:57:22.381 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:57:22.381 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1585485435 2021-07-11T03:57:22.381 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T03:57:22.381 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:57:22.382 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:57:22.382 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:57:22.382 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:57:22.382 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1585485435 2021-07-11T03:57:22.383 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:57:22.383 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4675 172.21.2.119:0/1585485435", 2021-07-11T03:57:22.383 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/1585485435", 2021-07-11T03:57:22.383 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T03:57:22.384 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 22, 2021-07-11T03:57:22.384 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 55, 2021-07-11T03:57:22.384 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T03:57:22.384 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T03:57:22.384 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T03:57:22.385 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T03:57:22.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:57:22.386 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:57:22.387 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:57:22.392 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:57:22.393 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:57:22.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:57:22.394 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:57:42.417 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:57:42.421 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:57:42.427 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:57:42.430 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:58:12.477 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:58:12.481 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:58:12.485 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:58:12.489 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:58:15.988 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T03:58:15.989 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:58:16.013 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:58:16.036 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:58:16.055 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:58:16.085 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T03:58:16.086 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:16.087 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:58:16.087 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:58:16.116 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:58:16.117 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:58:16.117 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T03:58:16.118 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T03:58:16.119 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:58:16.183 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[14953]: fuse finished with error 0 and tester_r 0 2021-07-11T03:58:16.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:16.185 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:58:16.186 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:58:16.275 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T03:58:16.275 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T03:58:16.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:58:16.276 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:58:16.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:16.288 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:58:16.289 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:58:16.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:58:16.308 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:58:16.308 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T03:58:16.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:58:16.309 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:58:16.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:58:16.371 INFO:teuthology.orchestra.run.gibba023.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:58:16.373 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T03:58:16.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.429+0000 7fea9b36a700 1 Processor -- start 2021-07-11T03:58:16.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.429+0000 7fea9b36a700 1 -- start start 2021-07-11T03:58:16.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.429+0000 7fea9b36a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940b1db0 0x7fea940b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.429+0000 7fea9b36a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fea941483a0 0x7fea940b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.429+0000 7fea9b36a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fea940a70c0 0x7fea94144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.429+0000 7fea9b36a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea940a90c0 con 0x7fea941483a0 2021-07-11T03:58:16.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.429+0000 7fea9b36a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea940b0350 con 0x7fea940a70c0 2021-07-11T03:58:16.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.429+0000 7fea9b36a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea940b05f0 con 0x7fea940b1db0 2021-07-11T03:58:16.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.429+0000 7fea98905700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fea941483a0 0x7fea940b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:16.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.429+0000 7fea98905700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fea941483a0 0x7fea940b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45116/0 (socket says 172.21.2.104:45116) 2021-07-11T03:58:16.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.429+0000 7fea98905700 1 -- 172.21.2.104:0/1341079955 learned_addr learned my addr 172.21.2.104:0/1341079955 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:16.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea98905700 1 -- 172.21.2.104:0/1341079955 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940b1db0 msgr2=0x7fea940b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:16.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea98905700 1 --2- 172.21.2.104:0/1341079955 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940b1db0 0x7fea940b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:16.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea98905700 1 -- 172.21.2.104:0/1341079955 >> v2:172.21.2.105:3300/0 conn(0x7fea940a70c0 msgr2=0x7fea94144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:16.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea98905700 1 --2- 172.21.2.104:0/1341079955 >> v2:172.21.2.105:3300/0 conn(0x7fea940a70c0 0x7fea94144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:16.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea98905700 1 -- 172.21.2.104:0/1341079955 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea940b0890 con 0x7fea941483a0 2021-07-11T03:58:16.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea98905700 1 --2- 172.21.2.104:0/1341079955 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea941483a0 0x7fea940b2190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fea88002f70 tx=0x7fea88006eb0).ready entity=mon.0 client_cookie=2afd84f2a85134fe server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:16.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea87fff700 1 -- 172.21.2.104:0/1341079955 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea8800d570 con 0x7fea941483a0 2021-07-11T03:58:16.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea87fff700 1 -- 172.21.2.104:0/1341079955 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea88000e90 con 0x7fea941483a0 2021-07-11T03:58:16.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea87fff700 1 -- 172.21.2.104:0/1341079955 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea8800d900 con 0x7fea941483a0 2021-07-11T03:58:16.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 -- 172.21.2.104:0/1341079955 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea941483a0 msgr2=0x7fea940b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:16.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 --2- 172.21.2.104:0/1341079955 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea941483a0 0x7fea940b2190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7fea88002f70 tx=0x7fea88006eb0).stop 2021-07-11T03:58:16.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 -- 172.21.2.104:0/1341079955 shutdown_connections 2021-07-11T03:58:16.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 --2- 172.21.2.104:0/1341079955 >> v2:172.21.2.105:3300/0 conn(0x7fea940a70c0 0x7fea94144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:16.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 --2- 172.21.2.104:0/1341079955 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea941483a0 0x7fea940b2190 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 --2- 172.21.2.104:0/1341079955 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940b1db0 0x7fea940b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:16.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 -- 172.21.2.104:0/1341079955 >> 172.21.2.104:0/1341079955 conn(0x7fea9405f6b0 msgr2=0x7fea9405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:16.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 -- 172.21.2.104:0/1341079955 shutdown_connections 2021-07-11T03:58:16.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 -- 172.21.2.104:0/1341079955 wait complete. 2021-07-11T03:58:16.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 Processor -- start 2021-07-11T03:58:16.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 -- start start 2021-07-11T03:58:16.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940a70c0 0x7fea940a9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940b1db0 0x7fea9415a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea941483a0 0x7fea94066bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea940abd20 con 0x7fea940a70c0 2021-07-11T03:58:16.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea940639b0 con 0x7fea941483a0 2021-07-11T03:58:16.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea94067810 con 0x7fea940b1db0 2021-07-11T03:58:16.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea99106700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940a70c0 0x7fea940a9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:16.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea99106700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940a70c0 0x7fea940a9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45120/0 (socket says 172.21.2.104:45120) 2021-07-11T03:58:16.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea99106700 1 -- 172.21.2.104:0/1999355478 learned_addr learned my addr 172.21.2.104:0/1999355478 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:16.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea99907700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea941483a0 0x7fea94066bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:16.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea98905700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940b1db0 0x7fea9415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:16.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea99106700 1 -- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940b1db0 msgr2=0x7fea9415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:16.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea99106700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940b1db0 0x7fea9415a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea99106700 1 -- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea941483a0 msgr2=0x7fea94066bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:16.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea99106700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea941483a0 0x7fea94066bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea99106700 1 -- 172.21.2.104:0/1999355478 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8800b1e0 con 0x7fea940a70c0 2021-07-11T03:58:16.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea99907700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea941483a0 0x7fea94066bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:58:16.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea99106700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940a70c0 0x7fea940a9780 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fea90002f70 tx=0x7fea90006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:16.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea867fc700 1 -- 172.21.2.104:0/1999355478 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea90013460 con 0x7fea940a70c0 2021-07-11T03:58:16.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea867fc700 1 -- 172.21.2.104:0/1999355478 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea900135c0 con 0x7fea940a70c0 2021-07-11T03:58:16.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 -- 172.21.2.104:0/1999355478 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9415ac70 con 0x7fea940a70c0 2021-07-11T03:58:16.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea867fc700 1 -- 172.21.2.104:0/1999355478 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea90013920 con 0x7fea940a70c0 2021-07-11T03:58:16.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.433+0000 7fea9b36a700 1 -- 172.21.2.104:0/1999355478 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea94063470 con 0x7fea940a70c0 2021-07-11T03:58:16.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.437+0000 7fea9b36a700 1 -- 172.21.2.104:0/1999355478 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea94057350 con 0x7fea940a70c0 2021-07-11T03:58:16.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.437+0000 7fea867fc700 1 -- 172.21.2.104:0/1999355478 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fea9001a060 con 0x7fea940a70c0 2021-07-11T03:58:16.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.437+0000 7fea867fc700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea74053260 0x7fea74055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.437+0000 7fea867fc700 1 -- 172.21.2.104:0/1999355478 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fea9006aff0 con 0x7fea940a70c0 2021-07-11T03:58:16.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.437+0000 7fea98905700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea74053260 0x7fea74055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:16.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.437+0000 7fea98905700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea74053260 0x7fea74055720 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fea88016000 tx=0x7fea88015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:16.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.437+0000 7fea867fc700 1 -- 172.21.2.104:0/1999355478 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fea90042240 con 0x7fea940a70c0 2021-07-11T03:58:16.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 -- 172.21.2.104:0/1999355478 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fea94044650 con 0x7fea940a70c0 2021-07-11T03:58:16.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea867fc700 1 -- 172.21.2.104:0/1999355478 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v55) v1 ==== 81+0+33647 (secure 0 0 0) 0x7fea90018070 con 0x7fea940a70c0 2021-07-11T03:58:16.533 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:16.533 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:58:16.533 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 55, 2021-07-11T03:58:16.533 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T03:58:16.534 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T03:58:16.534 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T03:56:50.701936+0000", 2021-07-11T03:58:16.534 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:58:16.535 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:58:16.535 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T03:58:16.535 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T03:58:16.535 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T03:58:16.536 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T03:58:16.536 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T03:58:16.536 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T03:58:16.536 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T03:58:16.537 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:58:16.537 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T03:58:16.537 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T03:58:16.537 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T03:58:16.538 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T03:58:16.538 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T03:58:16.538 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 10, 2021-07-11T03:58:16.538 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T03:58:16.539 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T03:58:16.539 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T03:58:16.539 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T03:58:16.539 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T03:58:16.540 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.540 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T03:58:16.540 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T03:58:16.540 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T03:58:16.540 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:58:16.541 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:58:16.541 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:58:16.541 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:58:16.541 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:58:16.542 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:58:16.542 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:58:16.542 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:58:16.542 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T03:58:16.543 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T03:58:16.543 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T03:58:16.543 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T03:58:16.543 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T03:58:16.543 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:58:16.544 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:58:16.544 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:58:16.544 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:58:16.544 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:58:16.545 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:58:16.545 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:58:16.545 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.545 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T03:58:16.546 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:58:16.546 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:58:16.546 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:58:16.550 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:58:16.551 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:58:16.551 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:58:16.551 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:58:16.551 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:58:16.552 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:58:16.552 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:58:16.552 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:58:16.552 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:58:16.553 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:58:16.553 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:58:16.553 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:58:16.553 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:58:16.554 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:58:16.554 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:58:16.554 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:58:16.555 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:58:16.556 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:58:16.556 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:58:16.556 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:58:16.556 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:58:16.557 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:58:16.557 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:58:16.557 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.557 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:58:16.557 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:58:16.558 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:58:16.558 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:58:16.558 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:58:16.558 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:58:16.559 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:58:16.559 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:58:16.559 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:58:16.559 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:58:16.560 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:58:16.560 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:58:16.560 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T03:58:16.560 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.561 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:58:16.561 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T03:58:16.561 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.561 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.561 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.562 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T03:58:16.562 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T03:58:16.562 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T03:58:16.562 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:58:16.563 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:58:16.563 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:58:16.563 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:58:16.563 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:58:16.564 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:58:16.564 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:58:16.564 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:58:16.564 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:58:16.567 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:58:16.568 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:58:16.568 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:58:16.568 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:58:16.568 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:58:16.569 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:58:16.569 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:58:16.569 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:58:16.569 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:58:16.570 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:58:16.570 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:58:16.570 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.570 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T03:58:16.571 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:58:16.571 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:58:16.571 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:58:16.571 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:58:16.572 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:58:16.572 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:58:16.572 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:58:16.572 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:58:16.572 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:58:16.573 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:58:16.573 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:58:16.573 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:58:16.573 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:58:16.574 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:58:16.574 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:58:16.574 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:58:16.574 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:58:16.575 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:58:16.575 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:58:16.575 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:58:16.575 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:58:16.576 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:58:16.576 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:58:16.576 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:58:16.576 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:58:16.577 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:58:16.577 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.577 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:58:16.577 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:58:16.577 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:58:16.578 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:58:16.578 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:58:16.578 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:58:16.583 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:58:16.583 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:58:16.583 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:58:16.584 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:58:16.584 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:58:16.584 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:58:16.584 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:58:16.585 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T03:58:16.585 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.585 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.585 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.586 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 9, 2021-07-11T03:58:16.586 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T03:58:16.586 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:56:48.052820+0000", 2021-07-11T03:58:16.587 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:58:16.587 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:58:16.587 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:58:16.588 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:58:16.588 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:58:16.588 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:58:16.589 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:58:16.589 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:58:16.589 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:58:16.589 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:58:16.590 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:58:16.590 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:58:16.590 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:58:16.591 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:58:16.591 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:58:16.591 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:58:16.591 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:58:16.592 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:58:16.592 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:58:16.592 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:58:16.593 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.593 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "54", 2021-07-11T03:58:16.603 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:58:16.603 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:58:16.604 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:58:16.604 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:58:16.604 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:58:16.604 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:58:16.605 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:58:16.605 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:58:16.605 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:58:16.605 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:58:16.605 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:58:16.606 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:58:16.606 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:58:16.606 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:58:16.606 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:58:16.607 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:58:16.607 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:58:16.607 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:58:16.607 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:58:16.608 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:58:16.608 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:58:16.608 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:58:16.608 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:58:16.609 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:58:16.609 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:58:16.609 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:58:16.609 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.609 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:58:16.610 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:58:16.610 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:58:16.610 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:58:16.610 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:58:16.611 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:58:16.611 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:58:16.611 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:58:16.611 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:58:16.611 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:58:16.612 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:58:16.612 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:58:16.612 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T03:58:16.612 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T03:58:16.613 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T03:58:16.613 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.613 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:58:16.613 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T03:58:16.614 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T03:58:16.614 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.614 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.614 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.615 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.615 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 10, 2021-07-11T03:58:16.615 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T03:58:16.615 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:56:48.670109+0000", 2021-07-11T03:58:16.616 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:58:16.616 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:58:16.616 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:58:16.616 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:58:16.616 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:58:16.617 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:58:16.617 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:58:16.617 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:58:16.617 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:58:16.618 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:58:16.618 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:58:16.618 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:58:16.618 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:58:16.619 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:58:16.619 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:58:16.619 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:58:16.619 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:58:16.620 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:58:16.620 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:58:16.620 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:58:16.620 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.620 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "55", 2021-07-11T03:58:16.621 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:58:16.621 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:58:16.621 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:58:16.622 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:58:16.622 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:58:16.622 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:58:16.623 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:58:16.623 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:58:16.623 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:58:16.623 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:58:16.624 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:58:16.624 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:58:16.624 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:58:16.624 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:58:16.625 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:58:16.625 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:58:16.625 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:58:16.626 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:58:16.626 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:58:16.626 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:58:16.627 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:58:16.627 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:58:16.627 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:58:16.627 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:58:16.628 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:58:16.628 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:58:16.628 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.628 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:58:16.629 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:58:16.629 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:58:16.629 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:58:16.630 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:58:16.630 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:58:16.630 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:58:16.630 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:58:16.631 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:58:16.631 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:58:16.631 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:58:16.631 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:58:16.632 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:58:16.632 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T03:58:16.632 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T03:58:16.632 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.633 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.633 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.633 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:58:16.633 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T03:58:16.634 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.634 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:58:16.634 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T03:58:16.634 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:16.634 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:16.635 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:16.635 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:16.635 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:16.635 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:16.636 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:16.636 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:16.636 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:16.636 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:16.637 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:16.637 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.637 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.637 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.637 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T03:58:16.638 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:16.638 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.638 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.638 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.639 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T03:58:16.639 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:16.639 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.639 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.639 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.640 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:16.640 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.640 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.640 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.641 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T03:58:16.641 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:16.641 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.641 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.642 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.642 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T03:58:16.642 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:16.642 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.644 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.644 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.644 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:16.644 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.645 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.645 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.645 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T03:58:16.645 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:16.646 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.646 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.646 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.646 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T03:58:16.647 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:16.647 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.647 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.647 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.647 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:16.648 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.648 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.648 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.648 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T03:58:16.649 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:16.649 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.649 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.649 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.650 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T03:58:16.650 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:16.650 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.650 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.650 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.651 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T03:58:16.651 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T03:58:16.651 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T03:58:16.651 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T03:58:16.652 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:16.652 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:16.652 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:16.652 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.653 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.653 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.653 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:58:16.653 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T03:58:16.653 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:16.654 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:16.654 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:16.654 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:16.654 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:16.655 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:16.655 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:16.655 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:16.655 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:16.656 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:16.656 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:16.656 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.656 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.656 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.657 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T03:58:16.657 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:16.657 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.657 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.658 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.658 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T03:58:16.658 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:16.658 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.659 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.659 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.659 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:16.659 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.660 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.660 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.660 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T03:58:16.660 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:16.661 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.661 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.661 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.662 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T03:58:16.662 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:16.662 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.662 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.663 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.663 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:16.663 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.663 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.664 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.664 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T03:58:16.664 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:16.664 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.665 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.665 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.665 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T03:58:16.665 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:16.666 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.666 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.666 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.666 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:16.666 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.667 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.667 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.667 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T03:58:16.667 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:16.668 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.668 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.668 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.668 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T03:58:16.668 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:16.669 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.669 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.669 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.669 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T03:58:16.670 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T03:58:16.670 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T03:58:16.670 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T03:58:16.670 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:16.671 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:16.671 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:16.671 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.671 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.671 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.672 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:58:16.672 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T03:58:16.672 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:16.672 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:16.673 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:16.673 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:16.673 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:16.673 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:16.674 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:16.674 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:16.674 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:16.674 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:16.675 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:16.675 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.675 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.675 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.675 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T03:58:16.676 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:16.676 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.676 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.676 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.677 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T03:58:16.677 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:16.677 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.677 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.678 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.678 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:16.678 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.678 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.678 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.679 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T03:58:16.679 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:16.679 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.679 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.680 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.680 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T03:58:16.680 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:16.680 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.681 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.681 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.681 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:16.681 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.681 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.682 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.682 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T03:58:16.682 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:16.683 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.683 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.683 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.683 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T03:58:16.684 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:16.684 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.684 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.684 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.684 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:16.685 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.685 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.685 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.685 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T03:58:16.686 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:16.686 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.686 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.686 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.687 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T03:58:16.687 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:16.687 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.687 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.687 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.688 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T03:58:16.688 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T03:58:16.688 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T03:58:16.688 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T03:58:16.689 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:16.689 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:16.689 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:16.689 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.690 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.690 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.690 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:58:16.690 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T03:58:16.690 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:16.691 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:16.691 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:16.691 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:16.691 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:16.692 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:16.692 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:16.692 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 52, 2021-07-11T03:58:16.692 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:16.692 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:16.693 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:16.693 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.693 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.693 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.694 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T03:58:16.694 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:16.694 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.694 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.695 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.695 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T03:58:16.695 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:16.695 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.695 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.696 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.696 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:16.696 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.696 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.697 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.697 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T03:58:16.697 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:16.697 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.698 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.698 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.698 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T03:58:16.698 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:16.698 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.699 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.699 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.699 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:16.700 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.700 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.700 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.700 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T03:58:16.701 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:16.701 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.701 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.701 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.702 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T03:58:16.702 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:16.702 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.702 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.702 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.703 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:16.703 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.703 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.703 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.704 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T03:58:16.704 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:16.704 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.704 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.704 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.705 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T03:58:16.705 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:16.705 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.705 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.706 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.706 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T03:58:16.706 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T03:58:16.706 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T03:58:16.707 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T03:58:16.707 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:16.707 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:16.707 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:16.707 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.708 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.708 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.708 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:58:16.708 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T03:58:16.709 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:16.709 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:16.709 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:16.709 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:16.709 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:16.710 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:16.710 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:16.710 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:16.710 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:16.711 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:16.711 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:16.711 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.711 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.712 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.712 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T03:58:16.712 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:16.712 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.713 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.713 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.713 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T03:58:16.713 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:16.713 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.714 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.714 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.714 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:16.714 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.715 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.715 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.715 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T03:58:16.715 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:16.715 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.716 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.716 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.716 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T03:58:16.717 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:16.717 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.717 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.717 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.718 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:16.718 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.718 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.718 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.719 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T03:58:16.719 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:16.719 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.719 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.719 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.720 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T03:58:16.720 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:16.720 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.720 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.721 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.721 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:16.721 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.721 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.721 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.722 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T03:58:16.722 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:16.722 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.722 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.723 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.723 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T03:58:16.723 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:16.723 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.724 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.724 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.724 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T03:58:16.724 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T03:58:16.724 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T03:58:16.725 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T03:58:16.725 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:16.725 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:16.725 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:16.726 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.726 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.726 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.726 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:58:16.726 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T03:58:16.727 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:16.727 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:16.727 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:16.727 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:16.728 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:16.728 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:16.728 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:16.728 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:16.728 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:16.729 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:16.729 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:16.729 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.729 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.730 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.730 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T03:58:16.730 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:16.730 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.731 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.731 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.731 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T03:58:16.731 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:16.731 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.732 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.732 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.732 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:16.732 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.733 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.733 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.733 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T03:58:16.733 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:16.733 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.734 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.734 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.734 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T03:58:16.734 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:16.735 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.735 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.735 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.735 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:16.736 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.736 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.736 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.736 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T03:58:16.736 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:16.737 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.737 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.737 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.737 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T03:58:16.738 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:16.738 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.738 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.738 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.739 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:16.739 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.739 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.739 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.739 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T03:58:16.740 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:16.740 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.740 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.740 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.741 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T03:58:16.741 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:16.741 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.742 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.742 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.742 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T03:58:16.742 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T03:58:16.742 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T03:58:16.743 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T03:58:16.743 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:16.743 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:16.743 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:16.744 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.744 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.744 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.744 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:58:16.745 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T03:58:16.745 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:16.745 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:16.745 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:16.745 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:16.746 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:16.746 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:16.746 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:16.746 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:16.747 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:16.747 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:16.747 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:16.747 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.747 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.748 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.748 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T03:58:16.748 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:16.748 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.749 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.749 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.749 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T03:58:16.749 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:16.750 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.750 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.750 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.750 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:16.750 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.751 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.751 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.751 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T03:58:16.751 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:16.752 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.752 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.752 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.753 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T03:58:16.753 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:16.753 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.753 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.754 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.754 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:16.754 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.754 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.754 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.755 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T03:58:16.755 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:16.755 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.755 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.756 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.756 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T03:58:16.756 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:16.756 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.756 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.757 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.757 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:16.757 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.757 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.758 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.758 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T03:58:16.758 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:16.758 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.759 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.759 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.759 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T03:58:16.759 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:16.759 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.760 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.760 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.760 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T03:58:16.760 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T03:58:16.761 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T03:58:16.761 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T03:58:16.761 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:16.761 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:16.762 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:16.762 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.762 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.762 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.762 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:58:16.763 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T03:58:16.763 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:16.763 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:16.763 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:16.764 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:16.764 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:16.764 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:16.764 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:16.764 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 52, 2021-07-11T03:58:16.765 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:16.765 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:16.765 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:16.765 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.766 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.766 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.766 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T03:58:16.766 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:16.767 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.767 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.767 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.767 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T03:58:16.768 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:16.768 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.768 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.768 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.769 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:16.769 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.769 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.769 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.769 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T03:58:16.770 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:16.770 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.770 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.770 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.771 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T03:58:16.771 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:16.771 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.772 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.772 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.772 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:16.772 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.772 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.773 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.773 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T03:58:16.773 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:16.773 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.774 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.774 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.774 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T03:58:16.774 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:16.775 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.775 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.775 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.775 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:16.775 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:16.776 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.776 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:16.776 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T03:58:16.776 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:16.777 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.777 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.777 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:16.777 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T03:58:16.778 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:16.778 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.778 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.778 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.778 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T03:58:16.779 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T03:58:16.779 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T03:58:16.779 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T03:58:16.779 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:16.780 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:16.780 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:16.780 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:16.780 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.781 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:58:16.781 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T03:58:16.781 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.781 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:58:16.781 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:16.782 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:16.782 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:16.782 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:16.782 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:16.783 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T03:58:16.783 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:16.783 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.783 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.784 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:58:16.784 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:16.784 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:16.784 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:16.784 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:16.785 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:16.785 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T03:58:16.785 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:16.785 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.786 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.786 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:58:16.786 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:16.786 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:16.787 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:16.787 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:16.787 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:16.787 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T03:58:16.787 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:16.788 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.788 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.788 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:58:16.788 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:16.789 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:16.789 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:16.789 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:16.789 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:16.790 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T03:58:16.790 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:16.790 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.790 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.791 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:58:16.791 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:16.791 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:16.791 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:16.791 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:16.792 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:16.792 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T03:58:16.792 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:16.792 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.793 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.793 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:58:16.793 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:16.793 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:16.793 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:16.794 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:16.794 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:16.794 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T03:58:16.794 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:16.795 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.795 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.795 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:58:16.795 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:16.796 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:16.796 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:16.796 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:16.796 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:16.797 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T03:58:16.797 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:16.797 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.797 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:16.797 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:58:16.798 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:16.798 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:16.798 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:16.798 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:16.799 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:16.799 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T03:58:16.799 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:16.799 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.800 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:58:16.800 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T03:58:16.800 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T03:58:16.800 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T03:58:16.800 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T03:58:16.801 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": {}, 2021-07-11T03:58:16.801 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T03:58:16.801 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T03:58:16.801 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T03:58:16.802 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T03:58:16.802 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T03:58:16.802 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T03:58:16.802 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T03:58:16.802 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T03:58:16.803 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:16.803 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:16.803 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T03:58:16.803 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T03:58:16.804 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T03:58:16.804 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T03:58:16.804 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T03:58:16.804 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:58:16.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 -- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea74053260 msgr2=0x7fea74055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:16.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea74053260 0x7fea74055720 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7fea88016000 tx=0x7fea88015000).stop 2021-07-11T03:58:16.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 -- 172.21.2.104:0/1999355478 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940a70c0 msgr2=0x7fea940a9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:16.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940a70c0 0x7fea940a9780 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fea90002f70 tx=0x7fea90006ec0).stop 2021-07-11T03:58:16.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 -- 172.21.2.104:0/1999355478 shutdown_connections 2021-07-11T03:58:16.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea941483a0 0x7fea94066bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940b1db0 0x7fea9415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea74053260 0x7fea74055720 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 --2- 172.21.2.104:0/1999355478 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940a70c0 0x7fea940a9780 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 -- 172.21.2.104:0/1999355478 >> 172.21.2.104:0/1999355478 conn(0x7fea9405f6b0 msgr2=0x7fea940a8720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:16.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 -- 172.21.2.104:0/1999355478 shutdown_connections 2021-07-11T03:58:16.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.521+0000 7fea9b36a700 1 -- 172.21.2.104:0/1999355478 wait complete. 2021-07-11T03:58:16.810 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:58:16.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 Processor -- start 2021-07-11T03:58:16.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 -- start start 2021-07-11T03:58:16.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f80a76b0 0x7f88f8066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f88f80b0060 0x7f88f805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f80a7a90 con 0x7f88f80b1970 2021-07-11T03:58:16.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f8147620 con 0x7f88f80b0060 2021-07-11T03:58:16.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f81478c0 con 0x7f88f80a76b0 2021-07-11T03:58:16.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88fe1d1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f88f80b0060 0x7f88f805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:16.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88fe1d1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f88f80b0060 0x7f88f805fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45128/0 (socket says 172.21.2.104:45128) 2021-07-11T03:58:16.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88fe1d1700 1 -- 172.21.2.104:0/906174441 learned_addr learned my addr 172.21.2.104:0/906174441 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:16.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88fe1d1700 1 -- 172.21.2.104:0/906174441 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f80a76b0 msgr2=0x7f88f8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:16.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88fe1d1700 1 --2- 172.21.2.104:0/906174441 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f80a76b0 0x7f88f8066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:16.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88fe1d1700 1 -- 172.21.2.104:0/906174441 >> v1:172.21.2.105:6789/0 conn(0x7f88f80b1970 legacy=0x7f88f805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:16.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88fe1d1700 1 -- 172.21.2.104:0/906174441 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f8147b60 con 0x7f88f80b0060 2021-07-11T03:58:16.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88fe1d1700 1 --2- 172.21.2.104:0/906174441 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b0060 0x7f88f805fdc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f88f80b6460 tx=0x7f88e8006eb0).ready entity=mon.0 client_cookie=91f4a2cfe009e162 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:16.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88fc9ce700 1 -- 172.21.2.104:0/906174441 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e800d570 con 0x7f88f80b0060 2021-07-11T03:58:16.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88fc9ce700 1 -- 172.21.2.104:0/906174441 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e8000e90 con 0x7f88f80b0060 2021-07-11T03:58:16.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88fc9ce700 1 -- 172.21.2.104:0/906174441 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e800d8c0 con 0x7f88f80b0060 2021-07-11T03:58:16.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 -- 172.21.2.104:0/906174441 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b0060 msgr2=0x7f88f805fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:16.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 --2- 172.21.2.104:0/906174441 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b0060 0x7f88f805fdc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f88f80b6460 tx=0x7f88e8006eb0).stop 2021-07-11T03:58:16.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 -- 172.21.2.104:0/906174441 shutdown_connections 2021-07-11T03:58:16.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 --2- 172.21.2.104:0/906174441 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b0060 0x7f88f805fdc0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 --2- 172.21.2.104:0/906174441 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f80a76b0 0x7f88f8066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:16.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 -- 172.21.2.104:0/906174441 >> 172.21.2.104:0/906174441 conn(0x7f88f80b53e0 msgr2=0x7f88f80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:16.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 -- 172.21.2.104:0/906174441 shutdown_connections 2021-07-11T03:58:16.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 -- 172.21.2.104:0/906174441 wait complete. 2021-07-11T03:58:16.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 Processor -- start 2021-07-11T03:58:16.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.865+0000 7f88ffc34700 1 -- start start 2021-07-11T03:58:16.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88ffc34700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f80a76b0 0x7f88f814ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88ffc34700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88f80b0060 0x7f88f8144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88ffc34700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b1970 0x7f88f8144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88ffc34700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f80ac670 con 0x7f88f80b1970 2021-07-11T03:58:16.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88ffc34700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f80ab240 con 0x7f88f80b0060 2021-07-11T03:58:16.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88ffc34700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f80ab4e0 con 0x7f88f80a76b0 2021-07-11T03:58:16.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fe1d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b1970 0x7f88f8144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:16.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fe1d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b1970 0x7f88f8144af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45134/0 (socket says 172.21.2.104:45134) 2021-07-11T03:58:16.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fe1d1700 1 -- 172.21.2.104:0/4039954781 learned_addr learned my addr 172.21.2.104:0/4039954781 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:16.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fd9d0700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f80a76b0 0x7f88f814ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:16.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fd1cf700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88f80b0060 0x7f88f8144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:16.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fe1d1700 1 -- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f80a76b0 msgr2=0x7f88f814ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:16.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fe1d1700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f80a76b0 0x7f88f814ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fe1d1700 1 -- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88f80b0060 msgr2=0x7f88f8144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:16.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fe1d1700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88f80b0060 0x7f88f8144600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fe1d1700 1 -- 172.21.2.104:0/4039954781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e800b1e0 con 0x7f88f80b1970 2021-07-11T03:58:16.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fe1d1700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b1970 0x7f88f8144af0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f88f80b6460 tx=0x7f88e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:16.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88f6ffd700 1 -- 172.21.2.104:0/4039954781 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e8021070 con 0x7f88f80b1970 2021-07-11T03:58:16.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88f6ffd700 1 -- 172.21.2.104:0/4039954781 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e8000e90 con 0x7f88f80b1970 2021-07-11T03:58:16.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88ffc34700 1 -- 172.21.2.104:0/4039954781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f80ab780 con 0x7f88f80b1970 2021-07-11T03:58:16.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88f6ffd700 1 -- 172.21.2.104:0/4039954781 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e8011a50 con 0x7f88f80b1970 2021-07-11T03:58:16.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88ffc34700 1 -- 172.21.2.104:0/4039954781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f80ae3f0 con 0x7f88f80b1970 2021-07-11T03:58:16.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88ffc34700 1 -- 172.21.2.104:0/4039954781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f8057350 con 0x7f88f80b1970 2021-07-11T03:58:16.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88f6ffd700 1 -- 172.21.2.104:0/4039954781 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f88e800f070 con 0x7f88f80b1970 2021-07-11T03:58:16.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88f6ffd700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88d80531f0 0x7f88d80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:16.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88f6ffd700 1 -- 172.21.2.104:0/4039954781 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f88e806c9c0 con 0x7f88f80b1970 2021-07-11T03:58:16.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.869+0000 7f88fd9d0700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88d80531f0 0x7f88d80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:16.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.873+0000 7f88fd9d0700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88d80531f0 0x7f88d80556b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f88f000a000 tx=0x7f88f0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:16.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.873+0000 7f88f6ffd700 1 -- 172.21.2.104:0/4039954781 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f88e8043d40 con 0x7f88f80b1970 2021-07-11T03:58:16.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.977+0000 7f88ffc34700 1 -- 172.21.2.104:0/4039954781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88f80698e0 con 0x7f88f80b1970 2021-07-11T03:58:16.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.977+0000 7f88f6ffd700 1 -- 172.21.2.104:0/4039954781 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f88e8047320 con 0x7f88f80b1970 2021-07-11T03:58:16.988 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:16.988 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":22,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3427300973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3427300973},{"type":"v1","addr":"172.21.2.105:6835","nonce":3427300973}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/2823255258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2823255258},{"type":"v1","addr":"172.21.2.104:6833","nonce":2823255258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:56:49.697379+0000","modified":"2021-07-11T03:56:50.700784+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5589":{"gid":5589,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":2,"addr":"172.21.2.104:6835/2428941897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2428941897},{"type":"v1","addr":"172.21.2.104:6835","nonce":2428941897}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-07-11T03:58:16.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 -- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88d80531f0 msgr2=0x7f88d80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:16.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88d80531f0 0x7f88d80556b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f88f000a000 tx=0x7f88f0008040).stop 2021-07-11T03:58:16.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 -- 172.21.2.104:0/4039954781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b1970 msgr2=0x7f88f8144af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:16.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b1970 0x7f88f8144af0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f88f80b6460 tx=0x7f88e8015000).stop 2021-07-11T03:58:16.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 -- 172.21.2.104:0/4039954781 shutdown_connections 2021-07-11T03:58:16.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88d80531f0 0x7f88d80556b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b1970 0x7f88f8144af0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88f80b0060 0x7f88f8144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 --2- 172.21.2.104:0/4039954781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f80a76b0 0x7f88f814ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:16.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 -- 172.21.2.104:0/4039954781 >> 172.21.2.104:0/4039954781 conn(0x7f88f80b53e0 msgr2=0x7f88f8149cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:16.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 -- 172.21.2.104:0/4039954781 shutdown_connections 2021-07-11T03:58:16.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:16.981+0000 7f88ffc34700 1 -- 172.21.2.104:0/4039954781 wait complete. 2021-07-11T03:58:16.994 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 22 2021-07-11T03:58:16.996 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T03:58:17.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 Processor -- start 2021-07-11T03:58:17.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 -- start start 2021-07-11T03:58:17.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b90067180 0x7f6b9005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6b900b1970 0x7f6b9005fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b900a7a90 con 0x7f6b900a76b0 2021-07-11T03:58:17.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b90147620 con 0x7f6b900b1970 2021-07-11T03:58:17.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b901478c0 con 0x7f6b90067180 2021-07-11T03:58:17.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b969e5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6b900b1970 0x7f6b9005fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:17.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b969e5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6b900b1970 0x7f6b9005fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45144/0 (socket says 172.21.2.104:45144) 2021-07-11T03:58:17.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b969e5700 1 -- 172.21.2.104:0/305360271 learned_addr learned my addr 172.21.2.104:0/305360271 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:17.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b969e5700 1 -- 172.21.2.104:0/305360271 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b90067180 msgr2=0x7f6b9005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:17.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b969e5700 1 --2- 172.21.2.104:0/305360271 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b90067180 0x7f6b9005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b969e5700 1 -- 172.21.2.104:0/305360271 >> v1:172.21.2.105:6789/0 conn(0x7f6b900a76b0 legacy=0x7f6b90066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:17.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b969e5700 1 -- 172.21.2.104:0/305360271 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b90147b60 con 0x7f6b900b1970 2021-07-11T03:58:17.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b969e5700 1 --2- 172.21.2.104:0/305360271 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b900b1970 0x7f6b9005fba0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f6b90148420 tx=0x7f6b88009290).ready entity=mon.0 client_cookie=a7b8e4b7987ebcc6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:17.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b951e2700 1 -- 172.21.2.104:0/305360271 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8800b830 con 0x7f6b900b1970 2021-07-11T03:58:17.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b951e2700 1 -- 172.21.2.104:0/305360271 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8800b990 con 0x7f6b900b1970 2021-07-11T03:58:17.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b951e2700 1 -- 172.21.2.104:0/305360271 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b8800bb00 con 0x7f6b900b1970 2021-07-11T03:58:17.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 -- 172.21.2.104:0/305360271 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b900b1970 msgr2=0x7f6b9005fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 --2- 172.21.2.104:0/305360271 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b900b1970 0x7f6b9005fba0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f6b90148420 tx=0x7f6b88009290).stop 2021-07-11T03:58:17.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 -- 172.21.2.104:0/305360271 shutdown_connections 2021-07-11T03:58:17.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 --2- 172.21.2.104:0/305360271 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b900b1970 0x7f6b9005fba0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:17.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 --2- 172.21.2.104:0/305360271 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b90067180 0x7f6b9005f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 -- 172.21.2.104:0/305360271 >> 172.21.2.104:0/305360271 conn(0x7f6b900b53e0 msgr2=0x7f6b900b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:17.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 -- 172.21.2.104:0/305360271 shutdown_connections 2021-07-11T03:58:17.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.041+0000 7f6b98448700 1 -- 172.21.2.104:0/305360271 wait complete. 2021-07-11T03:58:17.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b98448700 1 Processor -- start 2021-07-11T03:58:17.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b98448700 1 -- start start 2021-07-11T03:58:17.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b98448700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b90067180 0x7f6b900b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b98448700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b900a76b0 0x7f6b90156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b98448700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b900b1970 0x7f6b901571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b98448700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b90156650 con 0x7f6b900b1970 2021-07-11T03:58:17.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b98448700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900ab1d0 con 0x7f6b90067180 2021-07-11T03:58:17.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b98448700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900ac730 con 0x7f6b900a76b0 2021-07-11T03:58:17.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b969e5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b900b1970 0x7f6b901571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:17.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b969e5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b900b1970 0x7f6b901571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45150/0 (socket says 172.21.2.104:45150) 2021-07-11T03:58:17.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b969e5700 1 -- 172.21.2.104:0/282712956 learned_addr learned my addr 172.21.2.104:0/282712956 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:17.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b969e5700 1 -- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b900a76b0 msgr2=0x7f6b90156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b969e5700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b900a76b0 0x7f6b90156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b969e5700 1 -- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b90067180 msgr2=0x7f6b900b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b969e5700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b90067180 0x7f6b900b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b969e5700 1 -- 172.21.2.104:0/282712956 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b88009530 con 0x7f6b900b1970 2021-07-11T03:58:17.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b969e5700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b900b1970 0x7f6b901571d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f6b90148420 tx=0x7f6b88016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:17.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b837fe700 1 -- 172.21.2.104:0/282712956 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b88011c00 con 0x7f6b900b1970 2021-07-11T03:58:17.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b837fe700 1 -- 172.21.2.104:0/282712956 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b88011d60 con 0x7f6b900b1970 2021-07-11T03:58:17.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b98448700 1 -- 172.21.2.104:0/282712956 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b90156840 con 0x7f6b900b1970 2021-07-11T03:58:17.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b98448700 1 -- 172.21.2.104:0/282712956 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b900abff0 con 0x7f6b900b1970 2021-07-11T03:58:17.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b837fe700 1 -- 172.21.2.104:0/282712956 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b88020620 con 0x7f6b900b1970 2021-07-11T03:58:17.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b98448700 1 -- 172.21.2.104:0/282712956 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b900ac590 con 0x7f6b900b1970 2021-07-11T03:58:17.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b837fe700 1 -- 172.21.2.104:0/282712956 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6b88026020 con 0x7f6b900b1970 2021-07-11T03:58:17.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b837fe700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b70053350 0x7f6b70055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b837fe700 1 -- 172.21.2.104:0/282712956 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6b8806c270 con 0x7f6b900b1970 2021-07-11T03:58:17.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.045+0000 7f6b961e4700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b70053350 0x7f6b70055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:17.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.049+0000 7f6b961e4700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b70053350 0x7f6b70055810 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f6b84000f70 tx=0x7f6b840077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:17.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.049+0000 7f6b837fe700 1 -- 172.21.2.104:0/282712956 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6b900ac590 con 0x7f6b900b1970 2021-07-11T03:58:17.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.157+0000 7f6b98448700 1 -- 172.21.2.104:0/282712956 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6b900ac590 con 0x7f6b900b1970 2021-07-11T03:58:17.282 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T03:58:17.269+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T03:58:17.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.285+0000 7f6b837fe700 1 -- 172.21.2.104:0/282712956 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6b900ac590 con 0x7f6b900b1970 2021-07-11T03:58:17.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 -- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b70053350 msgr2=0x7f6b70055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b70053350 0x7f6b70055810 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f6b84000f70 tx=0x7f6b840077f0).stop 2021-07-11T03:58:17.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 -- 172.21.2.104:0/282712956 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b900b1970 msgr2=0x7f6b901571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b900b1970 0x7f6b901571d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f6b90148420 tx=0x7f6b88016000).stop 2021-07-11T03:58:17.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 -- 172.21.2.104:0/282712956 shutdown_connections 2021-07-11T03:58:17.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b70053350 0x7f6b70055810 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:17.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b900b1970 0x7f6b901571d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:17.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b900a76b0 0x7f6b90156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 --2- 172.21.2.104:0/282712956 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b90067180 0x7f6b900b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 -- 172.21.2.104:0/282712956 >> 172.21.2.104:0/282712956 conn(0x7f6b900b53e0 msgr2=0x7f6b9005fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:17.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 -- 172.21.2.104:0/282712956 shutdown_connections 2021-07-11T03:58:17.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.289+0000 7f6b98448700 1 -- 172.21.2.104:0/282712956 wait complete. 2021-07-11T03:58:17.302 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T03:58:17.304 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:58:17.316 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T03:58:17.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a30a1700 1 Processor -- start 2021-07-11T03:58:17.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a30a1700 1 -- start start 2021-07-11T03:58:17.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a30a1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f619c0a76b0 0x7f619c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a30a1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619c0a7a90 con 0x7f619c0b0060 2021-07-11T03:58:17.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a30a1700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f619c147620 con 0x7f619c0b1970 2021-07-11T03:58:17.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a30a1700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f619c1478c0 con 0x7f619c0a76b0 2021-07-11T03:58:17.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a163e700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f619c0b0060 0x7f619c151f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41942/0 (socket says 172.21.2.104:41942) 2021-07-11T03:58:17.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a163e700 1 -- 172.21.2.104:0/2319210146 learned_addr learned my addr 172.21.2.104:0/2319210146 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:17.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a0e3d700 1 --2- 172.21.2.104:0/2319210146 >> v2:172.21.2.104:3300/0 conn(0x7f619c0a76b0 0x7f619c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:17.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a0e3d700 1 -- 172.21.2.104:0/2319210146 >> v1:172.21.2.105:6789/0 conn(0x7f619c0b1970 legacy=0x7f619c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:17.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a0e3d700 1 -- 172.21.2.104:0/2319210146 >> v1:172.21.2.104:6789/0 conn(0x7f619c0b0060 legacy=0x7f619c151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f619b7fe700 1 -- 172.21.2.104:0/2319210146 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166353616 0 0) 0x7f619c0a7a90 con 0x7f619c0b0060 2021-07-11T03:58:17.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a0e3d700 1 -- 172.21.2.104:0/2319210146 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c147b60 con 0x7f619c0a76b0 2021-07-11T03:58:17.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a0e3d700 1 --2- 172.21.2.104:0/2319210146 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f619c0a76b0 0x7f619c066a70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f619c06a180 tx=0x7f618c006df0).ready entity=mon.0 client_cookie=853faf321b118ead server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:17.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f619b7fe700 1 -- 172.21.2.104:0/2319210146 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618c00d5c0 con 0x7f619c0a76b0 2021-07-11T03:58:17.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f619b7fe700 1 -- 172.21.2.104:0/2319210146 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c000e90 con 0x7f619c0a76b0 2021-07-11T03:58:17.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f619b7fe700 1 -- 172.21.2.104:0/2319210146 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618c00d910 con 0x7f619c0a76b0 2021-07-11T03:58:17.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a30a1700 1 -- 172.21.2.104:0/2319210146 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f619c0a76b0 msgr2=0x7f619c066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.349+0000 7f61a30a1700 1 --2- 172.21.2.104:0/2319210146 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f619c0a76b0 0x7f619c066a70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f619c06a180 tx=0x7f618c006df0).stop 2021-07-11T03:58:17.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 -- 172.21.2.104:0/2319210146 shutdown_connections 2021-07-11T03:58:17.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 --2- 172.21.2.104:0/2319210146 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f619c0a76b0 0x7f619c066a70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:17.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 -- 172.21.2.104:0/2319210146 >> 172.21.2.104:0/2319210146 conn(0x7f619c0b53e0 msgr2=0x7f619c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:17.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 -- 172.21.2.104:0/2319210146 shutdown_connections 2021-07-11T03:58:17.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 -- 172.21.2.104:0/2319210146 wait complete. 2021-07-11T03:58:17.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 Processor -- start 2021-07-11T03:58:17.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 -- start start 2021-07-11T03:58:17.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f619c0a76b0 0x7f619c0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f619c0b0060 0x7f619c066b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f619c0b1970 0x7f619c0aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c0ab610 con 0x7f619c0a76b0 2021-07-11T03:58:17.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c0ad1f0 con 0x7f619c0b0060 2021-07-11T03:58:17.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c0ad490 con 0x7f619c0b1970 2021-07-11T03:58:17.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a0e3d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f619c0a76b0 0x7f619c0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:17.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a0e3d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f619c0a76b0 0x7f619c0adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45180/0 (socket says 172.21.2.104:45180) 2021-07-11T03:58:17.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a0e3d700 1 -- 172.21.2.104:0/2303580194 learned_addr learned my addr 172.21.2.104:0/2303580194 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:17.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a0e3d700 1 -- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f619c0b1970 msgr2=0x7f619c0aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a0e3d700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f619c0b1970 0x7f619c0aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a0e3d700 1 -- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f619c0b0060 msgr2=0x7f619c066b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a0e3d700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f619c0b0060 0x7f619c066b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a0e3d700 1 -- 172.21.2.104:0/2303580194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f618c00b180 con 0x7f619c0a76b0 2021-07-11T03:58:17.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a0e3d700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f619c0a76b0 0x7f619c0adb90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f619c06a180 tx=0x7f618c012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:17.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f6199ffb700 1 -- 172.21.2.104:0/2303580194 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618c002f50 con 0x7f619c0a76b0 2021-07-11T03:58:17.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f6199ffb700 1 -- 172.21.2.104:0/2303580194 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f618c017030 con 0x7f619c0a76b0 2021-07-11T03:58:17.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f6199ffb700 1 -- 172.21.2.104:0/2303580194 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f618c01d470 con 0x7f619c0a76b0 2021-07-11T03:58:17.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 -- 172.21.2.104:0/2303580194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c0ad730 con 0x7f619c0a76b0 2021-07-11T03:58:17.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 -- 172.21.2.104:0/2303580194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c0ae810 con 0x7f619c0a76b0 2021-07-11T03:58:17.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f61a30a1700 1 -- 172.21.2.104:0/2303580194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c0a9570 con 0x7f619c0a76b0 2021-07-11T03:58:17.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f6199ffb700 1 -- 172.21.2.104:0/2303580194 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f618c00f030 con 0x7f619c0a76b0 2021-07-11T03:58:17.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f6199ffb700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61800532b0 0x7f6180055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.353+0000 7f6199ffb700 1 -- 172.21.2.104:0/2303580194 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f618c06c2d0 con 0x7f619c0a76b0 2021-07-11T03:58:17.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.357+0000 7f619bfff700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61800532b0 0x7f6180055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:17.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.357+0000 7f619bfff700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61800532b0 0x7f6180055770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f6184002f10 tx=0x7f6184006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:17.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.357+0000 7f6199ffb700 1 -- 172.21.2.104:0/2303580194 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f618c0434b0 con 0x7f619c0a76b0 2021-07-11T03:58:17.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.465+0000 7f61a30a1700 1 -- 172.21.2.104:0/2303580194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f619c0a9570 con 0x7f619c0a76b0 2021-07-11T03:58:17.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.465+0000 7f6199ffb700 1 -- 172.21.2.104:0/2303580194 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+2131 (secure 0 0 0) 0x7f619c0a9570 con 0x7f619c0a76b0 2021-07-11T03:58:17.474 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:17.474 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":23,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3427300973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3427300973},{"type":"v1","addr":"172.21.2.105:6835","nonce":3427300973}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/2823255258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2823255258},{"type":"v1","addr":"172.21.2.104:6833","nonce":2823255258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:56:49.697379+0000","modified":"2021-07-11T03:58:17.274297+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":56,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2021-07-11T03:58:17.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.465+0000 7f61a30a1700 1 -- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61800532b0 msgr2=0x7f6180055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.465+0000 7f61a30a1700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61800532b0 0x7f6180055770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f6184002f10 tx=0x7f6184006930).stop 2021-07-11T03:58:17.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.465+0000 7f61a30a1700 1 -- 172.21.2.104:0/2303580194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f619c0a76b0 msgr2=0x7f619c0adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.465+0000 7f61a30a1700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f619c0a76b0 0x7f619c0adb90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f619c06a180 tx=0x7f618c012000).stop 2021-07-11T03:58:17.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.469+0000 7f61a30a1700 1 -- 172.21.2.104:0/2303580194 shutdown_connections 2021-07-11T03:58:17.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.469+0000 7f61a30a1700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f619c0b1970 0x7f619c0aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.469+0000 7f61a30a1700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f619c0b0060 0x7f619c066b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.469+0000 7f61a30a1700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61800532b0 0x7f6180055770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:17.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.469+0000 7f61a30a1700 1 --2- 172.21.2.104:0/2303580194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f619c0a76b0 0x7f619c0adb90 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:17.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.469+0000 7f61a30a1700 1 -- 172.21.2.104:0/2303580194 >> 172.21.2.104:0/2303580194 conn(0x7f619c0b53e0 msgr2=0x7f619c14b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:17.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.469+0000 7f61a30a1700 1 -- 172.21.2.104:0/2303580194 shutdown_connections 2021-07-11T03:58:17.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.469+0000 7f61a30a1700 1 -- 172.21.2.104:0/2303580194 wait complete. 2021-07-11T03:58:17.478 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 23 2021-07-11T03:58:17.481 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T03:58:17.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.569+0000 7f7c385cd700 1 Processor -- start 2021-07-11T03:58:17.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.569+0000 7f7c385cd700 1 -- start start 2021-07-11T03:58:17.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f7c3014ab80 0x7f7c3005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7c300ac9b0 0x7f7c3014af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c30061360 con 0x7f7c300aff10 2021-07-11T03:58:17.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3014b5e0 con 0x7f7c300ac9b0 2021-07-11T03:58:17.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3014b880 con 0x7f7c3014ab80 2021-07-11T03:58:17.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36b6a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7c300ac9b0 0x7f7c3014af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:17.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36b6a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7c300ac9b0 0x7f7c3014af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45190/0 (socket says 172.21.2.104:45190) 2021-07-11T03:58:17.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36b6a700 1 -- 172.21.2.104:0/2998150914 learned_addr learned my addr 172.21.2.104:0/2998150914 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:17.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36b6a700 1 -- 172.21.2.104:0/2998150914 >> v2:172.21.2.105:3300/0 conn(0x7f7c3014ab80 msgr2=0x7f7c3005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:17.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36b6a700 1 --2- 172.21.2.104:0/2998150914 >> v2:172.21.2.105:3300/0 conn(0x7f7c3014ab80 0x7f7c3005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36b6a700 1 -- 172.21.2.104:0/2998150914 >> v1:172.21.2.105:6789/0 conn(0x7f7c300aff10 legacy=0x7f7c300af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:17.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36b6a700 1 -- 172.21.2.104:0/2998150914 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c300aed20 con 0x7f7c300ac9b0 2021-07-11T03:58:17.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36b6a700 1 --2- 172.21.2.104:0/2998150914 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7c300ac9b0 0x7f7c3014af60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f7c30062060 tx=0x7f7c28009290).ready entity=mon.0 client_cookie=111f4ff7f8f958e7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:17.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c35367700 1 -- 172.21.2.104:0/2998150914 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c2800b830 con 0x7f7c300ac9b0 2021-07-11T03:58:17.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c35367700 1 -- 172.21.2.104:0/2998150914 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2800b990 con 0x7f7c300ac9b0 2021-07-11T03:58:17.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c35367700 1 -- 172.21.2.104:0/2998150914 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c2800bb40 con 0x7f7c300ac9b0 2021-07-11T03:58:17.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- 172.21.2.104:0/2998150914 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7c300ac9b0 msgr2=0x7f7c3014af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:17.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 --2- 172.21.2.104:0/2998150914 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7c300ac9b0 0x7f7c3014af60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f7c30062060 tx=0x7f7c28009290).stop 2021-07-11T03:58:17.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- 172.21.2.104:0/2998150914 shutdown_connections 2021-07-11T03:58:17.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 --2- 172.21.2.104:0/2998150914 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7c300ac9b0 0x7f7c3014af60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:17.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 --2- 172.21.2.104:0/2998150914 >> v2:172.21.2.105:3300/0 conn(0x7f7c3014ab80 0x7f7c3005f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- 172.21.2.104:0/2998150914 >> 172.21.2.104:0/2998150914 conn(0x7f7c300b5570 msgr2=0x7f7c300b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:17.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- 172.21.2.104:0/2998150914 shutdown_connections 2021-07-11T03:58:17.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- 172.21.2.104:0/2998150914 wait complete. 2021-07-11T03:58:17.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 Processor -- start 2021-07-11T03:58:17.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- start start 2021-07-11T03:58:17.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7c300ac9b0 0x7f7c30157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7c300aff10 0x7f7c30066e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7c3014ab80 0x7f7c301421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c301485b0 con 0x7f7c300ac9b0 2021-07-11T03:58:17.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c300675f0 con 0x7f7c300aff10 2021-07-11T03:58:17.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c30067360 con 0x7f7c3014ab80 2021-07-11T03:58:17.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36369700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7c300ac9b0 0x7f7c30157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:17.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36369700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7c300ac9b0 0x7f7c30157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45194/0 (socket says 172.21.2.104:45194) 2021-07-11T03:58:17.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36369700 1 -- 172.21.2.104:0/2208093216 learned_addr learned my addr 172.21.2.104:0/2208093216 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:17.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36369700 1 -- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7c3014ab80 msgr2=0x7f7c301421b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:58:17.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36369700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7c3014ab80 0x7f7c301421b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36369700 1 -- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7c300aff10 msgr2=0x7f7c30066e70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:58:17.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36369700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7c300aff10 0x7f7c30066e70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:17.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36369700 1 -- 172.21.2.104:0/2208093216 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c28009530 con 0x7f7c300ac9b0 2021-07-11T03:58:17.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c36369700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7c300ac9b0 0x7f7c30157190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f7c2c002f70 tx=0x7f7c2c006e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:17.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c237fe700 1 -- 172.21.2.104:0/2208093216 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c2c00e470 con 0x7f7c300ac9b0 2021-07-11T03:58:17.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c237fe700 1 -- 172.21.2.104:0/2208093216 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c00e5d0 con 0x7f7c300ac9b0 2021-07-11T03:58:17.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- 172.21.2.104:0/2208093216 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c301567b0 con 0x7f7c300ac9b0 2021-07-11T03:58:17.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c237fe700 1 -- 172.21.2.104:0/2208093216 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7c2c00e930 con 0x7f7c300ac9b0 2021-07-11T03:58:17.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.573+0000 7f7c385cd700 1 -- 172.21.2.104:0/2208093216 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c30156cf0 con 0x7f7c300ac9b0 2021-07-11T03:58:17.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.577+0000 7f7c385cd700 1 -- 172.21.2.104:0/2208093216 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c30156eb0 con 0x7f7c300ac9b0 2021-07-11T03:58:17.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.577+0000 7f7c237fe700 1 -- 172.21.2.104:0/2208093216 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f7c2c018030 con 0x7f7c300ac9b0 2021-07-11T03:58:17.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.577+0000 7f7c237fe700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7c10053160 0x7f7c10055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:17.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.577+0000 7f7c237fe700 1 -- 172.21.2.104:0/2208093216 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f7c2c06a990 con 0x7f7c300ac9b0 2021-07-11T03:58:17.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.577+0000 7f7c35b68700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7c10053160 0x7f7c10055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:17.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.577+0000 7f7c35b68700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7c10053160 0x7f7c10055620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f7c24000f70 tx=0x7f7c240077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:17.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.577+0000 7f7c237fe700 1 -- 172.21.2.104:0/2208093216 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7c2c041d30 con 0x7f7c300ac9b0 2021-07-11T03:58:17.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:17.685+0000 7f7c385cd700 1 -- 172.21.2.104:0/2208093216 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7c30057350 con 0x7f7c300ac9b0 2021-07-11T03:58:18.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c237fe700 1 -- 172.21.2.104:0/2208093216 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7c2c045350 con 0x7f7c300ac9b0 2021-07-11T03:58:18.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 -- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7c10053160 msgr2=0x7f7c10055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:18.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7c10053160 0x7f7c10055620 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f7c24000f70 tx=0x7f7c240077f0).stop 2021-07-11T03:58:18.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 -- 172.21.2.104:0/2208093216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7c300ac9b0 msgr2=0x7f7c30157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:18.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7c300ac9b0 0x7f7c30157190 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f7c2c002f70 tx=0x7f7c2c006e50).stop 2021-07-11T03:58:18.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 -- 172.21.2.104:0/2208093216 shutdown_connections 2021-07-11T03:58:18.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7c3014ab80 0x7f7c301421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:18.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7c300aff10 0x7f7c30066e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:18.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7c10053160 0x7f7c10055620 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:18.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 --2- 172.21.2.104:0/2208093216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7c300ac9b0 0x7f7c30157190 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:18.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 -- 172.21.2.104:0/2208093216 >> 172.21.2.104:0/2208093216 conn(0x7f7c300b5570 msgr2=0x7f7c300acfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:18.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 -- 172.21.2.104:0/2208093216 shutdown_connections 2021-07-11T03:58:18.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.285+0000 7f7c385cd700 1 -- 172.21.2.104:0/2208093216 wait complete. 2021-07-11T03:58:18.307 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T03:58:18.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f083200d700 1 Processor -- start 2021-07-11T03:58:18.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f083200d700 1 -- start start 2021-07-11T03:58:18.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f083200d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f082c0b1db0 0x7f082c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:18.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f083200d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c0a70c0 0x7f082c144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:18.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f083200d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f082c0b0350 con 0x7f082c1483a0 2021-07-11T03:58:18.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f083200d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f082c0a90c0 con 0x7f082c0b1db0 2021-07-11T03:58:18.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f083200d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c0b05f0 con 0x7f082c0a70c0 2021-07-11T03:58:18.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082affd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f082c1483a0 0x7f082c0b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41972/0 (socket says 172.21.2.104:41972) 2021-07-11T03:58:18.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082affd700 1 -- 172.21.2.104:0/2193717869 learned_addr learned my addr 172.21.2.104:0/2193717869 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:18.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227603719 0 0) 0x7f082c0b0350 con 0x7f082c1483a0 2021-07-11T03:58:18.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f080c003260 con 0x7f082c1483a0 2021-07-11T03:58:18.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082b7fe700 1 --2- 172.21.2.104:0/2193717869 >> v2:172.21.2.105:3300/0 conn(0x7f082c0b1db0 0x7f082c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:18.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082bfff700 1 --2- 172.21.2.104:0/2193717869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c0a70c0 0x7f082c144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:18.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 266214568 0 0) 0x7f080c003260 con 0x7f082c1483a0 2021-07-11T03:58:18.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c0a70c0 msgr2=0x7f082c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:18.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 --2- 172.21.2.104:0/2193717869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c0a70c0 0x7f082c144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:18.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 >> v2:172.21.2.105:3300/0 conn(0x7f082c0b1db0 msgr2=0x7f082c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:18.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 --2- 172.21.2.104:0/2193717869 >> v2:172.21.2.105:3300/0 conn(0x7f082c0b1db0 0x7f082c0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:18.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c0b0890 con 0x7f082c1483a0 2021-07-11T03:58:18.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f081c001660 con 0x7f082c1483a0 2021-07-11T03:58:18.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082bfff700 1 --2- 172.21.2.104:0/2193717869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c0a70c0 0x7f082c144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:58:18.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 >> v1:172.21.2.104:6789/0 conn(0x7f082c1483a0 legacy=0x7f082c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:18.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 --2- 172.21.2.104:0/2193717869 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f080c003750 0x7f080c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:18.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 --2- 172.21.2.104:0/2193717869 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f080c004500 0x7f080c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:18.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 --2- 172.21.2.104:0/2193717869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c0a70c0 0x7f080c004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:18.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c0b05f0 con 0x7f080c003750 2021-07-11T03:58:18.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f080c00b800 con 0x7f080c004500 2021-07-11T03:58:18.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f080c00b9c0 con 0x7f082c0a70c0 2021-07-11T03:58:18.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.357+0000 7f082b7fe700 1 --2- 172.21.2.104:0/2193717869 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f080c003750 0x7f080c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:18.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082b7fe700 1 -- 172.21.2.104:0/2193717869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c0a70c0 msgr2=0x7f080c004eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:18.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082b7fe700 1 --2- 172.21.2.104:0/2193717869 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c0a70c0 0x7f080c004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:18.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082b7fe700 1 -- 172.21.2.104:0/2193717869 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f080c004500 msgr2=0x7f080c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:18.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082b7fe700 1 --2- 172.21.2.104:0/2193717869 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f080c004500 0x7f080c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:18.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082b7fe700 1 -- 172.21.2.104:0/2193717869 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c003260 con 0x7f080c003750 2021-07-11T03:58:18.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082b7fe700 1 --2- 172.21.2.104:0/2193717869 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f080c003750 0x7f080c003b70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f081400a000 tx=0x7f0814008040).ready entity=mon.0 client_cookie=eb163f3356c962aa server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:18.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0814013670 con 0x7f080c003750 2021-07-11T03:58:18.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08140137d0 con 0x7f080c003750 2021-07-11T03:58:18.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082a7fc700 1 -- 172.21.2.104:0/2193717869 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0814013b30 con 0x7f080c003750 2021-07-11T03:58:18.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 -- 172.21.2.104:0/2193717869 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f080c003750 msgr2=0x7f080c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:18.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 --2- 172.21.2.104:0/2193717869 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f080c003750 0x7f080c003b70 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f081400a000 tx=0x7f0814008040).stop 2021-07-11T03:58:18.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082bfff700 1 -- 172.21.2.104:0/2193717869 reap_dead start 2021-07-11T03:58:18.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 -- 172.21.2.104:0/2193717869 shutdown_connections 2021-07-11T03:58:18.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 -- 172.21.2.104:0/2193717869 >> 172.21.2.104:0/2193717869 conn(0x7f082c05f6b0 msgr2=0x7f082c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:18.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 -- 172.21.2.104:0/2193717869 shutdown_connections 2021-07-11T03:58:18.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 -- 172.21.2.104:0/2193717869 wait complete. 2021-07-11T03:58:18.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 Processor -- start 2021-07-11T03:58:18.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 -- start start 2021-07-11T03:58:18.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f082c0ad1e0 0x7f082c159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:18.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f082c1461b0 0x7f082c13f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:18.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c13fd30 0x7f082c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:18.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c146f50 con 0x7f082c1461b0 2021-07-11T03:58:18.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c0ad5c0 con 0x7f082c0ad1e0 2021-07-11T03:58:18.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f082c140670 con 0x7f082c13fd30 2021-07-11T03:58:18.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f082c1461b0 0x7f082c13f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:18.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f082c1461b0 0x7f082c13f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45218/0 (socket says 172.21.2.104:45218) 2021-07-11T03:58:18.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082affd700 1 -- 172.21.2.104:0/4030111405 learned_addr learned my addr 172.21.2.104:0/4030111405 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:18.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082affd700 1 -- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c13fd30 msgr2=0x7f082c1568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:18.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082affd700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c13fd30 0x7f082c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:18.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082affd700 1 -- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f082c0ad1e0 msgr2=0x7f082c159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:18.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082affd700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f082c0ad1e0 0x7f082c159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:18.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082affd700 1 -- 172.21.2.104:0/4030111405 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f082c146cf0 con 0x7f082c1461b0 2021-07-11T03:58:18.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f082affd700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f082c1461b0 0x7f082c13f840 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f082c147730 tx=0x7f082c0b0a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:18.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f0828ff9700 1 -- 172.21.2.104:0/4030111405 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f081c007170 con 0x7f082c1461b0 2021-07-11T03:58:18.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f0828ff9700 1 -- 172.21.2.104:0/4030111405 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081c0072d0 con 0x7f082c1461b0 2021-07-11T03:58:18.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 -- 172.21.2.104:0/4030111405 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c15a320 con 0x7f082c1461b0 2021-07-11T03:58:18.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f083200d700 1 -- 172.21.2.104:0/4030111405 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c1474c0 con 0x7f082c1461b0 2021-07-11T03:58:18.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.361+0000 7f0828ff9700 1 -- 172.21.2.104:0/4030111405 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f081c0049e0 con 0x7f082c1461b0 2021-07-11T03:58:18.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.365+0000 7f083200d700 1 -- 172.21.2.104:0/4030111405 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082c057350 con 0x7f082c1461b0 2021-07-11T03:58:18.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.365+0000 7f0828ff9700 1 -- 172.21.2.104:0/4030111405 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f081c010070 con 0x7f082c1461b0 2021-07-11T03:58:18.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.365+0000 7f0828ff9700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08100532b0 0x7f0810055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:18.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.365+0000 7f0828ff9700 1 -- 172.21.2.104:0/4030111405 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f081c06b300 con 0x7f082c1461b0 2021-07-11T03:58:18.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.365+0000 7f082b7fe700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08100532b0 0x7f0810055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:18.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.365+0000 7f082b7fe700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08100532b0 0x7f0810055770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f081400a000 tx=0x7f0814008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:18.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.365+0000 7f0828ff9700 1 -- 172.21.2.104:0/4030111405 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f081c0424e0 con 0x7f082c1461b0 2021-07-11T03:58:18.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:18.449+0000 7f083200d700 1 -- 172.21.2.104:0/4030111405 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f082c044650 con 0x7f082c1461b0 2021-07-11T03:58:19.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.289+0000 7f0828ff9700 1 -- 172.21.2.104:0/4030111405 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) v1 ==== 185+0+0 (secure 0 0 0) 0x7f081c045b00 con 0x7f082c1461b0 2021-07-11T03:58:19.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 -- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08100532b0 msgr2=0x7f0810055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:19.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08100532b0 0x7f0810055770 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f081400a000 tx=0x7f0814008040).stop 2021-07-11T03:58:19.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 -- 172.21.2.104:0/4030111405 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f082c1461b0 msgr2=0x7f082c13f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:19.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f082c1461b0 0x7f082c13f840 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f082c147730 tx=0x7f082c0b0a20).stop 2021-07-11T03:58:19.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 -- 172.21.2.104:0/4030111405 shutdown_connections 2021-07-11T03:58:19.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f082c13fd30 0x7f082c1568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:19.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08100532b0 0x7f0810055770 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:19.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f082c1461b0 0x7f082c13f840 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:19.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 --2- 172.21.2.104:0/4030111405 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f082c0ad1e0 0x7f082c159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:19.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 -- 172.21.2.104:0/4030111405 >> 172.21.2.104:0/4030111405 conn(0x7f082c05f6b0 msgr2=0x7f082c0a8720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:19.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 -- 172.21.2.104:0/4030111405 shutdown_connections 2021-07-11T03:58:19.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.293+0000 7f083200d700 1 -- 172.21.2.104:0/4030111405 wait complete. 2021-07-11T03:58:19.303 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T03:58:19.309 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T03:58:19.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 Processor -- start 2021-07-11T03:58:19.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- start start 2021-07-11T03:58:19.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f87900b3740 0x7f87900607c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:19.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87900604e0 con 0x7f8790060cb0 2021-07-11T03:58:19.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8790065aa0 con 0x7f8790060100 2021-07-11T03:58:19.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8790065d40 con 0x7f87900b3740 2021-07-11T03:58:19.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f87900b3740 0x7f87900607c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:19.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f8794998700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8790060cb0 0x7f8790061120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:41992/0 (socket says 172.21.2.104:41992) 2021-07-11T03:58:19.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f8794998700 1 -- 172.21.2.104:0/1671661126 learned_addr learned my addr 172.21.2.104:0/1671661126 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:19.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878f7fe700 1 --2- 172.21.2.104:0/1671661126 >> v2:172.21.2.104:3300/0 conn(0x7f87900b3740 0x7f87900607c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45226/0 (socket says 172.21.2.104:45226) 2021-07-11T03:58:19.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878f7fe700 1 -- 172.21.2.104:0/1671661126 >> v1:172.21.2.105:6789/0 conn(0x7f8790060100 legacy=0x7f8790066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:19.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878f7fe700 1 -- 172.21.2.104:0/1671661126 >> v1:172.21.2.104:6789/0 conn(0x7f8790060cb0 legacy=0x7f8790061120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:19.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878f7fe700 1 -- 172.21.2.104:0/1671661126 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8790065fe0 con 0x7f87900b3740 2021-07-11T03:58:19.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878f7fe700 1 --2- 172.21.2.104:0/1671661126 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87900b3740 0x7f87900607c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f879005f1e0 tx=0x7f8784006df0).ready entity=mon.0 client_cookie=1705eeeb30f9ebd8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:19.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878effd700 1 -- 172.21.2.104:0/1671661126 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f878400d5c0 con 0x7f87900b3740 2021-07-11T03:58:19.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878effd700 1 -- 172.21.2.104:0/1671661126 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8784000e90 con 0x7f87900b3740 2021-07-11T03:58:19.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878effd700 1 -- 172.21.2.104:0/1671661126 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f878400d910 con 0x7f87900b3740 2021-07-11T03:58:19.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- 172.21.2.104:0/1671661126 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87900b3740 msgr2=0x7f87900607c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:19.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 --2- 172.21.2.104:0/1671661126 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87900b3740 0x7f87900607c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f879005f1e0 tx=0x7f8784006df0).stop 2021-07-11T03:58:19.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- 172.21.2.104:0/1671661126 shutdown_connections 2021-07-11T03:58:19.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 --2- 172.21.2.104:0/1671661126 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87900b3740 0x7f87900607c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:19.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- 172.21.2.104:0/1671661126 >> 172.21.2.104:0/1671661126 conn(0x7f879005cf30 msgr2=0x7f879005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:19.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- 172.21.2.104:0/1671661126 shutdown_connections 2021-07-11T03:58:19.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- 172.21.2.104:0/1671661126 wait complete. 2021-07-11T03:58:19.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 Processor -- start 2021-07-11T03:58:19.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- start start 2021-07-11T03:58:19.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8790060100 0x7f87900b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:19.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8790060cb0 0x7f879015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:19.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87900b3740 0x7f87900b6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:19.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87900af530 con 0x7f8790060100 2021-07-11T03:58:19.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87900af7d0 con 0x7f8790060cb0 2021-07-11T03:58:19.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f87963fb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87901569c0 con 0x7f87900b3740 2021-07-11T03:58:19.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8790060100 0x7f87900b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:19.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8790060100 0x7f87900b0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45230/0 (socket says 172.21.2.104:45230) 2021-07-11T03:58:19.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.365+0000 7f878ffff700 1 -- 172.21.2.104:0/3055664758 learned_addr learned my addr 172.21.2.104:0/3055664758 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:19.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878ffff700 1 -- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87900b3740 msgr2=0x7f87900b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:19.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878ffff700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87900b3740 0x7f87900b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:19.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878ffff700 1 -- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8790060cb0 msgr2=0x7f879015a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:19.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878ffff700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8790060cb0 0x7f879015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:19.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878ffff700 1 -- 172.21.2.104:0/3055664758 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878400b180 con 0x7f8790060100 2021-07-11T03:58:19.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878ffff700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8790060100 0x7f87900b0250 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f8780002f90 tx=0x7f8780006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:19.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878d7fa700 1 -- 172.21.2.104:0/3055664758 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8780013460 con 0x7f8790060100 2021-07-11T03:58:19.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878d7fa700 1 -- 172.21.2.104:0/3055664758 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87800135c0 con 0x7f8790060100 2021-07-11T03:58:19.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878d7fa700 1 -- 172.21.2.104:0/3055664758 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8780013920 con 0x7f8790060100 2021-07-11T03:58:19.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f87963fb700 1 -- 172.21.2.104:0/3055664758 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87901566b0 con 0x7f8790060100 2021-07-11T03:58:19.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f87963fb700 1 -- 172.21.2.104:0/3055664758 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87900af040 con 0x7f8790060100 2021-07-11T03:58:19.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f87963fb700 1 -- 172.21.2.104:0/3055664758 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8790156b80 con 0x7f8790060100 2021-07-11T03:58:19.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878d7fa700 1 -- 172.21.2.104:0/3055664758 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f878001a060 con 0x7f8790060100 2021-07-11T03:58:19.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878d7fa700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8770053300 0x7f87700557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:19.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878d7fa700 1 -- 172.21.2.104:0/3055664758 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f878006adf0 con 0x7f8790060100 2021-07-11T03:58:19.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878f7fe700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8770053300 0x7f87700557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:19.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.369+0000 7f878f7fe700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8770053300 0x7f87700557c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f879005f1e0 tx=0x7f8784012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:19.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.373+0000 7f878d7fa700 1 -- 172.21.2.104:0/3055664758 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8780042180 con 0x7f8790060100 2021-07-11T03:58:19.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:19.453+0000 7f87963fb700 1 -- 172.21.2.104:0/3055664758 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87900af320 con 0x7f8790060100 2021-07-11T03:58:20.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.309+0000 7f878d7fa700 1 -- 172.21.2.104:0/3055664758 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) v1 ==== 173+0+0 (secure 0 0 0) 0x7f8780018070 con 0x7f8790060100 2021-07-11T03:58:20.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 -- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8770053300 msgr2=0x7f87700557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:20.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8770053300 0x7f87700557c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f879005f1e0 tx=0x7f8784012000).stop 2021-07-11T03:58:20.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 -- 172.21.2.104:0/3055664758 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8790060100 msgr2=0x7f87900b0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:20.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8790060100 0x7f87900b0250 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f8780002f90 tx=0x7f8780006df0).stop 2021-07-11T03:58:20.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 -- 172.21.2.104:0/3055664758 shutdown_connections 2021-07-11T03:58:20.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87900b3740 0x7f87900b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:20.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8790060cb0 0x7f879015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:20.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8770053300 0x7f87700557c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:20.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 --2- 172.21.2.104:0/3055664758 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8790060100 0x7f87900b0250 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:20.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 -- 172.21.2.104:0/3055664758 >> 172.21.2.104:0/3055664758 conn(0x7f879005cf30 msgr2=0x7f87900b3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:20.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 -- 172.21.2.104:0/3055664758 shutdown_connections 2021-07-11T03:58:20.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.313+0000 7f87963fb700 1 -- 172.21.2.104:0/3055664758 wait complete. 2021-07-11T03:58:20.327 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T03:58:20.329 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-07-11T03:58:20.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 Processor -- start 2021-07-11T03:58:20.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 -- start start 2021-07-11T03:58:20.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1890144440 0x7f1890144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1890065420 0x7f1890144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1890145350 con 0x7f1890060100 2021-07-11T03:58:20.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1890065c80 con 0x7f1890144440 2021-07-11T03:58:20.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18900604e0 con 0x7f1890065420 2021-07-11T03:58:20.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f188f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1890144440 0x7f1890144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:20.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f188f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1890144440 0x7f1890144970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45238/0 (socket says 172.21.2.104:45238) 2021-07-11T03:58:20.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f188f7fe700 1 -- 172.21.2.104:0/1831550994 learned_addr learned my addr 172.21.2.104:0/1831550994 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:20.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f188f7fe700 1 -- 172.21.2.104:0/1831550994 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1890065420 msgr2=0x7f1890144e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:58:20.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f188f7fe700 1 --2- 172.21.2.104:0/1831550994 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1890065420 0x7f1890144e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:20.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f188f7fe700 1 -- 172.21.2.104:0/1831550994 >> v1:172.21.2.105:6789/0 conn(0x7f1890060100 legacy=0x7f1890064d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:58:20.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f188f7fe700 1 -- 172.21.2.104:0/1831550994 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18900b23b0 con 0x7f1890144440 2021-07-11T03:58:20.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f188f7fe700 1 --2- 172.21.2.104:0/1831550994 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1890144440 0x7f1890144970 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f189005d460 tx=0x7f1880006eb0).ready entity=mon.0 client_cookie=c4cbd588b133ed4f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:20.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f188effd700 1 -- 172.21.2.104:0/1831550994 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188000d570 con 0x7f1890144440 2021-07-11T03:58:20.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f188effd700 1 -- 172.21.2.104:0/1831550994 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1880000e90 con 0x7f1890144440 2021-07-11T03:58:20.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f188effd700 1 -- 172.21.2.104:0/1831550994 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f188000d8c0 con 0x7f1890144440 2021-07-11T03:58:20.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 -- 172.21.2.104:0/1831550994 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1890144440 msgr2=0x7f1890144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:20.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 --2- 172.21.2.104:0/1831550994 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1890144440 0x7f1890144970 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f189005d460 tx=0x7f1880006eb0).stop 2021-07-11T03:58:20.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 -- 172.21.2.104:0/1831550994 shutdown_connections 2021-07-11T03:58:20.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 --2- 172.21.2.104:0/1831550994 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1890065420 0x7f1890144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:20.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 --2- 172.21.2.104:0/1831550994 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1890144440 0x7f1890144970 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:20.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 -- 172.21.2.104:0/1831550994 >> 172.21.2.104:0/1831550994 conn(0x7f189005cf60 msgr2=0x7f189005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:20.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 -- 172.21.2.104:0/1831550994 shutdown_connections 2021-07-11T03:58:20.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 -- 172.21.2.104:0/1831550994 wait complete. 2021-07-11T03:58:20.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 Processor -- start 2021-07-11T03:58:20.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.385+0000 7f1896a03700 1 -- start start 2021-07-11T03:58:20.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f1896a03700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1890060100 0x7f189013f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f1896a03700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1890065420 0x7f18900ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f1896a03700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f189013f610 0x7f18900ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f1896a03700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18900ad300 con 0x7f1890060100 2021-07-11T03:58:20.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f1896a03700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18900acbe0 con 0x7f1890065420 2021-07-11T03:58:20.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f1896a03700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18900aade0 con 0x7f189013f610 2021-07-11T03:58:20.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1890060100 0x7f189013f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:20.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1890060100 0x7f189013f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45242/0 (socket says 172.21.2.104:45242) 2021-07-11T03:58:20.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188ffff700 1 -- 172.21.2.104:0/1405723073 learned_addr learned my addr 172.21.2.104:0/1405723073 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:20.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188ffff700 1 -- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f189013f610 msgr2=0x7f18900ac6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:58:20.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188ffff700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f189013f610 0x7f18900ac6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:20.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188ffff700 1 -- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1890065420 msgr2=0x7f18900ac200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:58:20.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188ffff700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1890065420 0x7f18900ac200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:20.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188ffff700 1 -- 172.21.2.104:0/1405723073 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188000b1e0 con 0x7f1890060100 2021-07-11T03:58:20.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188ffff700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1890060100 0x7f189013f120 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f1878002fb0 tx=0x7f1878006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:20.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188d7fa700 1 -- 172.21.2.104:0/1405723073 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1878013460 con 0x7f1890060100 2021-07-11T03:58:20.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188d7fa700 1 -- 172.21.2.104:0/1405723073 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18780135c0 con 0x7f1890060100 2021-07-11T03:58:20.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f1896a03700 1 -- 172.21.2.104:0/1405723073 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18900ab080 con 0x7f1890060100 2021-07-11T03:58:20.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188d7fa700 1 -- 172.21.2.104:0/1405723073 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1878013920 con 0x7f1890060100 2021-07-11T03:58:20.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f1896a03700 1 -- 172.21.2.104:0/1405723073 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18900b1750 con 0x7f1890060100 2021-07-11T03:58:20.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f1896a03700 1 -- 172.21.2.104:0/1405723073 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1890057350 con 0x7f1890060100 2021-07-11T03:58:20.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188d7fa700 1 -- 172.21.2.104:0/1405723073 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f187800f070 con 0x7f1890060100 2021-07-11T03:58:20.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188d7fa700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1874053270 0x7f1874055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188d7fa700 1 -- 172.21.2.104:0/1405723073 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f187806ac60 con 0x7f1890060100 2021-07-11T03:58:20.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188f7fe700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1874053270 0x7f1874055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:20.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.389+0000 7f188f7fe700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1874053270 0x7f1874055730 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f189005d460 tx=0x7f1880015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:20.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.393+0000 7f188d7fa700 1 -- 172.21.2.104:0/1405723073 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1878068cf0 con 0x7f1890060100 2021-07-11T03:58:20.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.473+0000 7f1896a03700 1 -- 172.21.2.104:0/1405723073 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f189013db50 con 0x7f1890060100 2021-07-11T03:58:20.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f188d7fa700 1 -- 172.21.2.104:0/1405723073 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f18780470a0 con 0x7f1890060100 2021-07-11T03:58:20.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 -- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1874053270 msgr2=0x7f1874055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:20.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1874053270 0x7f1874055730 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f189005d460 tx=0x7f1880015000).stop 2021-07-11T03:58:20.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 -- 172.21.2.104:0/1405723073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1890060100 msgr2=0x7f189013f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:20.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1890060100 0x7f189013f120 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f1878002fb0 tx=0x7f1878006df0).stop 2021-07-11T03:58:20.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 -- 172.21.2.104:0/1405723073 shutdown_connections 2021-07-11T03:58:20.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f189013f610 0x7f18900ac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:20.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1890065420 0x7f18900ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:20.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1874053270 0x7f1874055730 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:20.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 --2- 172.21.2.104:0/1405723073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1890060100 0x7f189013f120 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:20.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 -- 172.21.2.104:0/1405723073 >> 172.21.2.104:0/1405723073 conn(0x7f189005cf60 msgr2=0x7f18900b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:20.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 -- 172.21.2.104:0/1405723073 shutdown_connections 2021-07-11T03:58:20.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.481+0000 7f1896a03700 1 -- 172.21.2.104:0/1405723073 wait complete. 2021-07-11T03:58:20.502 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout' 2021-07-11T03:58:20.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 Processor -- start 2021-07-11T03:58:20.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 -- start start 2021-07-11T03:58:20.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e6c060100 0x7f9e6c0b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9e6c067300 0x7f9e6c1438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6c0aabb0 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6c0604e0 con 0x7f9e6c067300 2021-07-11T03:58:20.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c066ea0 con 0x7f9e6c060100 2021-07-11T03:58:20.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6affd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9e6c067300 0x7f9e6c1438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:20.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6affd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9e6c067300 0x7f9e6c1438f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:52814/0 (socket says 172.21.2.104:52814) 2021-07-11T03:58:20.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6affd700 1 -- 172.21.2.104:0/1189885638 learned_addr learned my addr 172.21.2.104:0/1189885638 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:20.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6b7fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e6c060100 0x7f9e6c0b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:20.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6affd700 1 -- 172.21.2.104:0/1189885638 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e6c060100 msgr2=0x7f9e6c0b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:20.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6affd700 1 --2- 172.21.2.104:0/1189885638 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e6c060100 0x7f9e6c0b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:20.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6affd700 1 -- 172.21.2.104:0/1189885638 >> v1:172.21.2.105:6789/0 conn(0x7f9e6c0b4c40 legacy=0x7f9e6c144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:20.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6affd700 1 -- 172.21.2.104:0/1189885638 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6c0b33b0 con 0x7f9e6c067300 2021-07-11T03:58:20.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6affd700 1 --2- 172.21.2.104:0/1189885638 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e6c067300 0x7f9e6c1438f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f9e60002f70 tx=0x7f9e60006eb0).ready entity=mon.1 client_cookie=3b30a860823acd68 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:20.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6a7fc700 1 -- 172.21.2.104:0/1189885638 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e6000d570 con 0x7f9e6c067300 2021-07-11T03:58:20.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6a7fc700 1 -- 172.21.2.104:0/1189885638 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e60000e90 con 0x7f9e6c067300 2021-07-11T03:58:20.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e6a7fc700 1 -- 172.21.2.104:0/1189885638 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e6000d900 con 0x7f9e6c067300 2021-07-11T03:58:20.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 -- 172.21.2.104:0/1189885638 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e6c067300 msgr2=0x7f9e6c1438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:20.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 --2- 172.21.2.104:0/1189885638 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e6c067300 0x7f9e6c1438f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f9e60002f70 tx=0x7f9e60006eb0).stop 2021-07-11T03:58:20.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 -- 172.21.2.104:0/1189885638 shutdown_connections 2021-07-11T03:58:20.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 --2- 172.21.2.104:0/1189885638 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e6c067300 0x7f9e6c1438f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:20.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 --2- 172.21.2.104:0/1189885638 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e6c060100 0x7f9e6c0b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:20.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 -- 172.21.2.104:0/1189885638 >> 172.21.2.104:0/1189885638 conn(0x7f9e6c05cf30 msgr2=0x7f9e6c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:20.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 -- 172.21.2.104:0/1189885638 shutdown_connections 2021-07-11T03:58:20.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 -- 172.21.2.104:0/1189885638 wait complete. 2021-07-11T03:58:20.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 Processor -- start 2021-07-11T03:58:20.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.549+0000 7f9e71cba700 1 -- start start 2021-07-11T03:58:20.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e71cba700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e6c060100 0x7f9e6c0b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e71cba700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e6c067300 0x7f9e6c13c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e71cba700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e6c0b4c40 0x7f9e6c13a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e71cba700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c0ad3d0 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e71cba700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c13cb20 con 0x7f9e6c060100 2021-07-11T03:58:20.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e71cba700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6c13dd80 con 0x7f9e6c067300 2021-07-11T03:58:20.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e6c0b4c40 0x7f9e6c13a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:20.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e6c0b4c40 0x7f9e6c13a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45258/0 (socket says 172.21.2.104:45258) 2021-07-11T03:58:20.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6bfff700 1 -- 172.21.2.104:0/1308712415 learned_addr learned my addr 172.21.2.104:0/1308712415 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:20.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6affd700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e6c067300 0x7f9e6c13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:20.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6b7fe700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e6c060100 0x7f9e6c0b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:20.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6bfff700 1 -- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e6c067300 msgr2=0x7f9e6c13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:20.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6bfff700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e6c067300 0x7f9e6c13c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:20.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6bfff700 1 -- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e6c060100 msgr2=0x7f9e6c0b07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:20.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6bfff700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e6c060100 0x7f9e6c0b07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:20.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6bfff700 1 -- 172.21.2.104:0/1308712415 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6000b1e0 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6b7fe700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e6c060100 0x7f9e6c0b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:58:20.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6bfff700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e6c0b4c40 0x7f9e6c13a930 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f9e6c0671e0 tx=0x7f9e64007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:20.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e68ff9700 1 -- 172.21.2.104:0/1308712415 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e6400f470 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e68ff9700 1 -- 172.21.2.104:0/1308712415 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6400f5d0 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e68ff9700 1 -- 172.21.2.104:0/1308712415 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9e6400f930 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e71cba700 1 -- 172.21.2.104:0/1308712415 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6c0ac9f0 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e71cba700 1 -- 172.21.2.104:0/1308712415 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6c0acf30 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e71cba700 1 -- 172.21.2.104:0/1308712415 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e6c13cdb0 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e68ff9700 1 -- 172.21.2.104:0/1308712415 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9e64019030 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e68ff9700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e4c0532d0 0x7f9e4c055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:20.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e68ff9700 1 -- 172.21.2.104:0/1308712415 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9e6406a910 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6b7fe700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e4c0532d0 0x7f9e4c055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:20.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.553+0000 7f9e6b7fe700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e4c0532d0 0x7f9e4c055790 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f9e5c00b000 tx=0x7f9e5c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:20.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.557+0000 7f9e68ff9700 1 -- 172.21.2.104:0/1308712415 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9e64041eb0 con 0x7f9e6c0b4c40 2021-07-11T03:58:20.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:20.665+0000 7f9e71cba700 1 -- 172.21.2.104:0/1308712415 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]} v 0) v1 -- 0x7f9e6c0ad0f0 con 0x7f9e6c0b4c40 2021-07-11T03:58:21.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.313+0000 7f9e68ff9700 1 -- 172.21.2.104:0/1308712415 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_dead_timeout"]}]=0 v204) v1 ==== 152+0+0 (secure 0 0 0) 0x7f9e640454d0 con 0x7f9e6c0b4c40 2021-07-11T03:58:21.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 -- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e4c0532d0 msgr2=0x7f9e4c055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:21.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e4c0532d0 0x7f9e4c055790 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f9e5c00b000 tx=0x7f9e5c008040).stop 2021-07-11T03:58:21.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 -- 172.21.2.104:0/1308712415 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e6c0b4c40 msgr2=0x7f9e6c13a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:21.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e6c0b4c40 0x7f9e6c13a930 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f9e6c0671e0 tx=0x7f9e64007ef0).stop 2021-07-11T03:58:21.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 -- 172.21.2.104:0/1308712415 shutdown_connections 2021-07-11T03:58:21.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9e4c0532d0 0x7f9e4c055790 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:21.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9e6c0b4c40 0x7f9e6c13a930 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:21.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9e6c067300 0x7f9e6c13c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:21.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 --2- 172.21.2.104:0/1308712415 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9e6c060100 0x7f9e6c0b07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:21.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 -- 172.21.2.104:0/1308712415 >> 172.21.2.104:0/1308712415 conn(0x7f9e6c05cf30 msgr2=0x7f9e6c0b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:21.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 -- 172.21.2.104:0/1308712415 shutdown_connections 2021-07-11T03:58:21.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.317+0000 7f9e71cba700 1 -- 172.21.2.104:0/1308712415 wait complete. 2021-07-11T03:58:21.330 INFO:tasks.cephfs_test_runner:test_drop_cache_command_dead_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-07-11T03:58:21.331 INFO:tasks.cephfs_test_runner:Starting test: test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) 2021-07-11T03:58:21.331 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2021-07-11T03:58:21.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 Processor -- start 2021-07-11T03:58:21.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 -- start start 2021-07-11T03:58:21.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09b0060100 0x7f09b00b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:21.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f09b0067300 0x7f09b01438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:21.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b00aabb0 con 0x7f09b00b4c40 2021-07-11T03:58:21.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b00604e0 con 0x7f09b0067300 2021-07-11T03:58:21.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b0066ea0 con 0x7f09b0060100 2021-07-11T03:58:21.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09aeffd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f09b0067300 0x7f09b01438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:21.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09af7fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09b0060100 0x7f09b00b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:21.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09affff700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f09b00b4c40 0x7f09b0144510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:41406/0 (socket says 172.21.2.104:41406) 2021-07-11T03:58:21.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09affff700 1 -- 172.21.2.104:0/1139372421 learned_addr learned my addr 172.21.2.104:0/1139372421 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:21.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09af7fe700 1 -- 172.21.2.104:0/1139372421 >> v2:172.21.2.105:3300/0 conn(0x7f09b0067300 msgr2=0x7f09b01438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:21.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09af7fe700 1 --2- 172.21.2.104:0/1139372421 >> v2:172.21.2.105:3300/0 conn(0x7f09b0067300 0x7f09b01438f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:21.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09af7fe700 1 -- 172.21.2.104:0/1139372421 >> v1:172.21.2.105:6789/0 conn(0x7f09b00b4c40 legacy=0x7f09b0144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:21.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09af7fe700 1 -- 172.21.2.104:0/1139372421 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b00b33b0 con 0x7f09b0060100 2021-07-11T03:58:21.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09af7fe700 1 --2- 172.21.2.104:0/1139372421 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09b0060100 0x7f09b00b4750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f09a0002f70 tx=0x7f09a0006df0).ready entity=mon.2 client_cookie=b7f83ce134ebc795 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:21.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09ae7fc700 1 -- 172.21.2.104:0/1139372421 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a000d570 con 0x7f09b0060100 2021-07-11T03:58:21.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09ae7fc700 1 -- 172.21.2.104:0/1139372421 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a0000e90 con 0x7f09b0060100 2021-07-11T03:58:21.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09ae7fc700 1 -- 172.21.2.104:0/1139372421 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a000d8c0 con 0x7f09b0060100 2021-07-11T03:58:21.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 -- 172.21.2.104:0/1139372421 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09b0060100 msgr2=0x7f09b00b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:21.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 --2- 172.21.2.104:0/1139372421 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09b0060100 0x7f09b00b4750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f09a0002f70 tx=0x7f09a0006df0).stop 2021-07-11T03:58:21.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 -- 172.21.2.104:0/1139372421 shutdown_connections 2021-07-11T03:58:21.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 --2- 172.21.2.104:0/1139372421 >> v2:172.21.2.105:3300/0 conn(0x7f09b0067300 0x7f09b01438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:21.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 --2- 172.21.2.104:0/1139372421 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09b0060100 0x7f09b00b4750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:21.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.385+0000 7f09b5c30700 1 -- 172.21.2.104:0/1139372421 >> 172.21.2.104:0/1139372421 conn(0x7f09b005cf30 msgr2=0x7f09b005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:21.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 -- 172.21.2.104:0/1139372421 shutdown_connections 2021-07-11T03:58:21.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 -- 172.21.2.104:0/1139372421 wait complete. 2021-07-11T03:58:21.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 Processor -- start 2021-07-11T03:58:21.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 -- start start 2021-07-11T03:58:21.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09b0060100 0x7f09b0144650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:21.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09b0067300 0x7f09b00af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:21.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09b00b4c40 0x7f09b0156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:21.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00ae2a0 con 0x7f09b00b4c40 2021-07-11T03:58:21.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00b0270 con 0x7f09b0067300 2021-07-11T03:58:21.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b00b0510 con 0x7f09b0060100 2021-07-11T03:58:21.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09affff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09b00b4c40 0x7f09b0156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:21.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09affff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09b00b4c40 0x7f09b0156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45274/0 (socket says 172.21.2.104:45274) 2021-07-11T03:58:21.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09affff700 1 -- 172.21.2.104:0/1629144233 learned_addr learned my addr 172.21.2.104:0/1629144233 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:21.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09aeffd700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09b0067300 0x7f09b00af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:21.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09af7fe700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09b0060100 0x7f09b0144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:21.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09affff700 1 -- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09b0060100 msgr2=0x7f09b0144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:21.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09affff700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09b0060100 0x7f09b0144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:21.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09affff700 1 -- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09b0067300 msgr2=0x7f09b00af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:21.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09affff700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09b0067300 0x7f09b00af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:21.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09affff700 1 -- 172.21.2.104:0/1629144233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a000b180 con 0x7f09b00b4c40 2021-07-11T03:58:21.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09affff700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09b00b4c40 0x7f09b0156010 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f09b00671e0 tx=0x7f09a8007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:21.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09acff9700 1 -- 172.21.2.104:0/1629144233 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a8005ba0 con 0x7f09b00b4c40 2021-07-11T03:58:21.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09acff9700 1 -- 172.21.2.104:0/1629144233 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a8005d00 con 0x7f09b00b4c40 2021-07-11T03:58:21.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 -- 172.21.2.104:0/1629144233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b00b07b0 con 0x7f09b00b4c40 2021-07-11T03:58:21.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09acff9700 1 -- 172.21.2.104:0/1629144233 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f09a80072c0 con 0x7f09b00b4c40 2021-07-11T03:58:21.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 -- 172.21.2.104:0/1629144233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b00b0f20 con 0x7f09b00b4c40 2021-07-11T03:58:21.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09b5c30700 1 -- 172.21.2.104:0/1629144233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b0057350 con 0x7f09b00b4c40 2021-07-11T03:58:21.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09acff9700 1 -- 172.21.2.104:0/1629144233 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f09a8010070 con 0x7f09b00b4c40 2021-07-11T03:58:21.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09acff9700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0994053210 0x7f09940556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:21.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09acff9700 1 -- 172.21.2.104:0/1629144233 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f09a806aa60 con 0x7f09b00b4c40 2021-07-11T03:58:21.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.389+0000 7f09af7fe700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0994053210 0x7f09940556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:21.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.393+0000 7f09af7fe700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0994053210 0x7f09940556d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f09a0017000 tx=0x7f09a0014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:21.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.393+0000 7f09acff9700 1 -- 172.21.2.104:0/1629144233 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f09a8042080 con 0x7f09b00b4c40 2021-07-11T03:58:21.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:21.501+0000 7f09b5c30700 1 -- 172.21.2.104:0/1629144233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f09b01436c0 con 0x7f09b00b4c40 2021-07-11T03:58:22.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.317+0000 7f09acff9700 1 -- 172.21.2.104:0/1629144233 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v205) v1 ==== 150+0+0 (secure 0 0 0) 0x7f09a80456a0 con 0x7f09b00b4c40 2021-07-11T03:58:22.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 -- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0994053210 msgr2=0x7f09940556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:22.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0994053210 0x7f09940556d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f09a0017000 tx=0x7f09a0014000).stop 2021-07-11T03:58:22.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 -- 172.21.2.104:0/1629144233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09b00b4c40 msgr2=0x7f09b0156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:22.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09b00b4c40 0x7f09b0156010 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f09b00671e0 tx=0x7f09a8007f50).stop 2021-07-11T03:58:22.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 -- 172.21.2.104:0/1629144233 shutdown_connections 2021-07-11T03:58:22.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0994053210 0x7f09940556d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:22.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09b00b4c40 0x7f09b0156010 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:22.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09b0067300 0x7f09b00af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:22.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 --2- 172.21.2.104:0/1629144233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09b0060100 0x7f09b0144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:22.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 -- 172.21.2.104:0/1629144233 >> 172.21.2.104:0/1629144233 conn(0x7f09b005cf30 msgr2=0x7f09b01487c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:22.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 -- 172.21.2.104:0/1629144233 shutdown_connections 2021-07-11T03:58:22.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.321+0000 7f09b5c30700 1 -- 172.21.2.104:0/1629144233 wait complete. 2021-07-11T03:58:22.339 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T03:58:22.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.385+0000 7f9ff5923700 1 Processor -- start 2021-07-11T03:58:22.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.385+0000 7f9ff5923700 1 -- start start 2021-07-11T03:58:22.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.385+0000 7f9ff5923700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ff0144440 0x7f9ff0144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.385+0000 7f9ff5923700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9ff0065420 0x7f9ff0144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.385+0000 7f9ff5923700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff0145350 con 0x7f9ff0060100 2021-07-11T03:58:22.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.385+0000 7f9ff5923700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff0065c80 con 0x7f9ff0065420 2021-07-11T03:58:22.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.385+0000 7f9ff5923700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff00604e0 con 0x7f9ff0144440 2021-07-11T03:58:22.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9fef7fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9ff0065420 0x7f9ff0144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:22.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9feeffd700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f9ff0060100 0x7f9ff0064d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:41416/0 (socket says 172.21.2.104:41416) 2021-07-11T03:58:22.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9fee7fc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ff0144440 0x7f9ff0144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:22.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9feeffd700 1 -- 172.21.2.104:0/276855550 learned_addr learned my addr 172.21.2.104:0/276855550 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:22.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9fef7fe700 1 -- 172.21.2.104:0/276855550 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ff0144440 msgr2=0x7f9ff0144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:22.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9fef7fe700 1 --2- 172.21.2.104:0/276855550 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ff0144440 0x7f9ff0144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:22.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9fef7fe700 1 -- 172.21.2.104:0/276855550 >> v1:172.21.2.105:6789/0 conn(0x7f9ff0060100 legacy=0x7f9ff0064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:22.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9fef7fe700 1 -- 172.21.2.104:0/276855550 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff00b23b0 con 0x7f9ff0065420 2021-07-11T03:58:22.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9fef7fe700 1 --2- 172.21.2.104:0/276855550 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ff0065420 0x7f9ff0144e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9fe80032f0 tx=0x7f9fe80091d0).ready entity=mon.1 client_cookie=46456f2557f35e5f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:22.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9fedffb700 1 -- 172.21.2.104:0/276855550 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fe800b6c0 con 0x7f9ff0065420 2021-07-11T03:58:22.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9fedffb700 1 -- 172.21.2.104:0/276855550 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe800b820 con 0x7f9ff0065420 2021-07-11T03:58:22.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9fedffb700 1 -- 172.21.2.104:0/276855550 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fe800b990 con 0x7f9ff0065420 2021-07-11T03:58:22.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9ff5923700 1 -- 172.21.2.104:0/276855550 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ff0065420 msgr2=0x7f9ff0144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:22.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9ff5923700 1 --2- 172.21.2.104:0/276855550 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ff0065420 0x7f9ff0144e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f9fe80032f0 tx=0x7f9fe80091d0).stop 2021-07-11T03:58:22.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9ff5923700 1 -- 172.21.2.104:0/276855550 shutdown_connections 2021-07-11T03:58:22.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9ff5923700 1 --2- 172.21.2.104:0/276855550 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ff0065420 0x7f9ff0144e60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:22.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9ff5923700 1 --2- 172.21.2.104:0/276855550 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ff0144440 0x7f9ff0144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:22.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9ff5923700 1 -- 172.21.2.104:0/276855550 >> 172.21.2.104:0/276855550 conn(0x7f9ff005cf60 msgr2=0x7f9ff005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:22.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9ff5923700 1 -- 172.21.2.104:0/276855550 shutdown_connections 2021-07-11T03:58:22.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9ff5923700 1 -- 172.21.2.104:0/276855550 wait complete. 2021-07-11T03:58:22.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9ff5923700 1 Processor -- start 2021-07-11T03:58:22.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.389+0000 7f9ff5923700 1 -- start start 2021-07-11T03:58:22.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9ff5923700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ff0060100 0x7f9ff00a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9ff5923700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ff0065420 0x7f9ff013ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9ff5923700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ff00a96c0 0x7f9ff013e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9ff5923700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff00b0160 con 0x7f9ff0060100 2021-07-11T03:58:22.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9ff5923700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff013ae80 con 0x7f9ff00a96c0 2021-07-11T03:58:22.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9ff5923700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff00b0390 con 0x7f9ff0065420 2021-07-11T03:58:22.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9feeffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ff0060100 0x7f9ff00a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:22.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9feeffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ff0060100 0x7f9ff00a91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45286/0 (socket says 172.21.2.104:45286) 2021-07-11T03:58:22.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9feeffd700 1 -- 172.21.2.104:0/3179904930 learned_addr learned my addr 172.21.2.104:0/3179904930 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:22.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9feeffd700 1 -- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ff0065420 msgr2=0x7f9ff013ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:22.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9feeffd700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ff0065420 0x7f9ff013ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:22.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9feeffd700 1 -- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ff00a96c0 msgr2=0x7f9ff013e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:22.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9feeffd700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ff00a96c0 0x7f9ff013e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:22.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9feeffd700 1 -- 172.21.2.104:0/3179904930 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe8009470 con 0x7f9ff0060100 2021-07-11T03:58:22.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9feeffd700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ff0060100 0x7f9ff00a91d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f9ff0069be0 tx=0x7f9fd8007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:22.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9fe7fff700 1 -- 172.21.2.104:0/3179904930 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fd800ce30 con 0x7f9ff0060100 2021-07-11T03:58:22.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9fe7fff700 1 -- 172.21.2.104:0/3179904930 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd80072e0 con 0x7f9ff0060100 2021-07-11T03:58:22.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9fe7fff700 1 -- 172.21.2.104:0/3179904930 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fd8005800 con 0x7f9ff0060100 2021-07-11T03:58:22.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9ff5923700 1 -- 172.21.2.104:0/3179904930 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff01421b0 con 0x7f9ff0060100 2021-07-11T03:58:22.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9ff5923700 1 -- 172.21.2.104:0/3179904930 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff013e920 con 0x7f9ff0060100 2021-07-11T03:58:22.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9ff5923700 1 -- 172.21.2.104:0/3179904930 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff00b0710 con 0x7f9ff0060100 2021-07-11T03:58:22.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9fe7fff700 1 -- 172.21.2.104:0/3179904930 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9fd8010070 con 0x7f9ff0060100 2021-07-11T03:58:22.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9fe7fff700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fd4053240 0x7f9fd4055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9fe7fff700 1 -- 172.21.2.104:0/3179904930 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9fd806a9b0 con 0x7f9ff0060100 2021-07-11T03:58:22.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.393+0000 7f9fee7fc700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fd4053240 0x7f9fd4055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:22.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.397+0000 7f9fee7fc700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fd4053240 0x7f9fd4055700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f9fe000c000 tx=0x7f9fe0006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:22.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.397+0000 7f9fe7fff700 1 -- 172.21.2.104:0/3179904930 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9fd8041ec0 con 0x7f9ff0060100 2021-07-11T03:58:22.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.477+0000 7f9ff5923700 1 -- 172.21.2.104:0/3179904930 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9ff0057350 con 0x7f9ff0060100 2021-07-11T03:58:22.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.481+0000 7f9fe7fff700 1 -- 172.21.2.104:0/3179904930 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9fd80454e0 con 0x7f9ff0060100 2021-07-11T03:58:22.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.481+0000 7f9ff5923700 1 -- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fd4053240 msgr2=0x7f9fd4055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:22.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.481+0000 7f9ff5923700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fd4053240 0x7f9fd4055700 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f9fe000c000 tx=0x7f9fe0006df0).stop 2021-07-11T03:58:22.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.481+0000 7f9ff5923700 1 -- 172.21.2.104:0/3179904930 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ff0060100 msgr2=0x7f9ff00a91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:22.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.481+0000 7f9ff5923700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ff0060100 0x7f9ff00a91d0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f9ff0069be0 tx=0x7f9fd8007c00).stop 2021-07-11T03:58:22.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.481+0000 7f9ff5923700 1 -- 172.21.2.104:0/3179904930 shutdown_connections 2021-07-11T03:58:22.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.481+0000 7f9ff5923700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ff00a96c0 0x7f9ff013e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:22.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.481+0000 7f9ff5923700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ff0065420 0x7f9ff013ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:22.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.481+0000 7f9ff5923700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9fd4053240 0x7f9fd4055700 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:22.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.481+0000 7f9ff5923700 1 --2- 172.21.2.104:0/3179904930 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ff0060100 0x7f9ff00a91d0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:22.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.481+0000 7f9ff5923700 1 -- 172.21.2.104:0/3179904930 >> 172.21.2.104:0/3179904930 conn(0x7f9ff005cf60 msgr2=0x7f9ff00aa300 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:22.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.485+0000 7f9ff5923700 1 -- 172.21.2.104:0/3179904930 shutdown_connections 2021-07-11T03:58:22.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.485+0000 7f9ff5923700 1 -- 172.21.2.104:0/3179904930 wait complete. 2021-07-11T03:58:22.503 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T03:58:22.503 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:58:22.529 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:58:22.542 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:58:22.568 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:58:22.593 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T03:58:22.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:22.595 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:58:22.595 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:58:22.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:58:22.616 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T03:58:22.617 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T03:58:22.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:22.619 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:58:22.631 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:58:22.643 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:58:22.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:58:22.646 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T03:58:22.659 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T03:58:22.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f91780c8700 1 Processor -- start 2021-07-11T03:58:22.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f91780c8700 1 -- start start 2021-07-11T03:58:22.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f91780c8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f91700b1db0 0x7f91700b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f91780c8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f91701483a0 0x7f91700b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f91780c8700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91700b0350 con 0x7f91700a70c0 2021-07-11T03:58:22.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f91780c8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91700a90c0 con 0x7f91700b1db0 2021-07-11T03:58:22.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f91780c8700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91700b05f0 con 0x7f91701483a0 2021-07-11T03:58:22.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f9175e64700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f91700b1db0 0x7f91700b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:22.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f9175e64700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f91700b1db0 0x7f91700b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45290/0 (socket says 172.21.2.104:45290) 2021-07-11T03:58:22.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f9175e64700 1 -- 172.21.2.104:0/136195889 learned_addr learned my addr 172.21.2.104:0/136195889 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:22.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f9175e64700 1 -- 172.21.2.104:0/136195889 >> v2:172.21.2.105:3300/0 conn(0x7f91701483a0 msgr2=0x7f91700b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:22.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.717+0000 7f9175e64700 1 --2- 172.21.2.104:0/136195889 >> v2:172.21.2.105:3300/0 conn(0x7f91701483a0 0x7f91700b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:22.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 -- 172.21.2.104:0/136195889 >> v1:172.21.2.105:6789/0 conn(0x7f91700a70c0 legacy=0x7f9170144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:22.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 -- 172.21.2.104:0/136195889 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91700b0890 con 0x7f91700b1db0 2021-07-11T03:58:22.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 --2- 172.21.2.104:0/136195889 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91700b1db0 0x7f91700b0fd0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f91700ace60 tx=0x7f9164006eb0).ready entity=mon.0 client_cookie=470397b05daa3850 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:22.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9174e62700 1 -- 172.21.2.104:0/136195889 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f916400d5c0 con 0x7f91700b1db0 2021-07-11T03:58:22.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9174e62700 1 -- 172.21.2.104:0/136195889 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9164000e90 con 0x7f91700b1db0 2021-07-11T03:58:22.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9174e62700 1 -- 172.21.2.104:0/136195889 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f916400d950 con 0x7f91700b1db0 2021-07-11T03:58:22.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- 172.21.2.104:0/136195889 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91700b1db0 msgr2=0x7f91700b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:22.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 --2- 172.21.2.104:0/136195889 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91700b1db0 0x7f91700b0fd0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f91700ace60 tx=0x7f9164006eb0).stop 2021-07-11T03:58:22.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- 172.21.2.104:0/136195889 shutdown_connections 2021-07-11T03:58:22.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 --2- 172.21.2.104:0/136195889 >> v2:172.21.2.105:3300/0 conn(0x7f91701483a0 0x7f91700b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:22.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 --2- 172.21.2.104:0/136195889 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91700b1db0 0x7f91700b0fd0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:22.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- 172.21.2.104:0/136195889 >> 172.21.2.104:0/136195889 conn(0x7f917005f6b0 msgr2=0x7f917005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:22.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- 172.21.2.104:0/136195889 shutdown_connections 2021-07-11T03:58:22.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- 172.21.2.104:0/136195889 wait complete. 2021-07-11T03:58:22.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 Processor -- start 2021-07-11T03:58:22.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- start start 2021-07-11T03:58:22.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91700a70c0 0x7f917015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f91700b1db0 0x7f9170146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91701483a0 0x7f9170156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917015a030 con 0x7f91700a70c0 2021-07-11T03:58:22.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f917015a2d0 con 0x7f91700b1db0 2021-07-11T03:58:22.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917015a570 con 0x7f91701483a0 2021-07-11T03:58:22.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91700a70c0 0x7f917015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:22.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91700a70c0 0x7f917015aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45298/0 (socket says 172.21.2.104:45298) 2021-07-11T03:58:22.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 -- 172.21.2.104:0/71792023 learned_addr learned my addr 172.21.2.104:0/71792023 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:22.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 -- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91701483a0 msgr2=0x7f9170156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:22.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91701483a0 0x7f9170156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:22.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 -- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f91700b1db0 msgr2=0x7f9170146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:22.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f91700b1db0 0x7f9170146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:22.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 -- 172.21.2.104:0/71792023 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916400b1e0 con 0x7f91700a70c0 2021-07-11T03:58:22.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9175e64700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91700a70c0 0x7f917015aa10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f91700ace60 tx=0x7f9164015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:22.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9162ffd700 1 -- 172.21.2.104:0/71792023 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9164021070 con 0x7f91700a70c0 2021-07-11T03:58:22.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9162ffd700 1 -- 172.21.2.104:0/71792023 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9164000e90 con 0x7f91700a70c0 2021-07-11T03:58:22.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- 172.21.2.104:0/71792023 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9170146860 con 0x7f91700a70c0 2021-07-11T03:58:22.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- 172.21.2.104:0/71792023 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91700ad7e0 con 0x7f91700a70c0 2021-07-11T03:58:22.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f9162ffd700 1 -- 172.21.2.104:0/71792023 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9164006be0 con 0x7f91700a70c0 2021-07-11T03:58:22.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.721+0000 7f91780c8700 1 -- 172.21.2.104:0/71792023 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f917015a730 con 0x7f91700a70c0 2021-07-11T03:58:22.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.725+0000 7f9162ffd700 1 -- 172.21.2.104:0/71792023 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f916400f070 con 0x7f91700a70c0 2021-07-11T03:58:22.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.725+0000 7f9162ffd700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9150053350 0x7f9150055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:22.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.725+0000 7f9162ffd700 1 -- 172.21.2.104:0/71792023 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f916406c780 con 0x7f91700a70c0 2021-07-11T03:58:22.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.725+0000 7f9175663700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9150053350 0x7f9150055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:22.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.725+0000 7f9175663700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9150053350 0x7f9150055810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f916c002f70 tx=0x7f916c006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:22.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.725+0000 7f9162ffd700 1 -- 172.21.2.104:0/71792023 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9164043c90 con 0x7f91700a70c0 2021-07-11T03:58:22.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 -- 172.21.2.104:0/71792023 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f9170057350 con 0x7f91700a70c0 2021-07-11T03:58:22.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f9162ffd700 1 -- 172.21.2.104:0/71792023 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v58) v1 ==== 81+0+28828 (secure 0 0 0) 0x7f91640472b0 con 0x7f91700a70c0 2021-07-11T03:58:22.817 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:22.817 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:58:22.817 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 58, 2021-07-11T03:58:22.817 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T03:58:22.818 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T03:58:22.818 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T03:58:20.298351+0000", 2021-07-11T03:58:22.818 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:58:22.818 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:58:22.819 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T03:58:22.819 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T03:58:22.819 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T03:58:22.819 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T03:58:22.820 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T03:58:22.820 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T03:58:22.820 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T03:58:22.820 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:58:22.820 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T03:58:22.821 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T03:58:22.821 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T03:58:22.821 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T03:58:22.821 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T03:58:22.822 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 10, 2021-07-11T03:58:22.822 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T03:58:22.832 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T03:58:22.833 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T03:58:22.833 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T03:58:22.834 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T03:58:22.834 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.835 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T03:58:22.835 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T03:58:22.835 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T03:58:22.836 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:58:22.836 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:58:22.836 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:58:22.837 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:58:22.837 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:58:22.837 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:58:22.838 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:58:22.838 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:58:22.838 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T03:58:22.839 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T03:58:22.839 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T03:58:22.839 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T03:58:22.840 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T03:58:22.840 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:58:22.841 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:58:22.841 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:58:22.841 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:58:22.842 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:58:22.842 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:58:22.843 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:58:22.843 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.843 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T03:58:22.844 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:58:22.844 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:58:22.845 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:58:22.845 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:58:22.846 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:58:22.846 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:58:22.847 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:58:22.847 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:58:22.847 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:58:22.848 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:58:22.848 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:58:22.849 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:58:22.849 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:58:22.849 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:58:22.850 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:58:22.851 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:58:22.851 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:58:22.851 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:58:22.852 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:58:22.852 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:58:22.853 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:58:22.853 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:58:22.854 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:58:22.854 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:58:22.854 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:58:22.855 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:58:22.855 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.856 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:58:22.856 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:58:22.856 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:58:22.857 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:58:22.857 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:58:22.858 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:58:22.858 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:58:22.859 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:58:22.859 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:58:22.859 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:58:22.860 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:58:22.860 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:58:22.861 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T03:58:22.861 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.862 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:58:22.862 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T03:58:22.862 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.863 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.863 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.863 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T03:58:22.863 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T03:58:22.864 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T03:58:22.864 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:58:22.864 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:58:22.864 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:58:22.865 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:58:22.865 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:58:22.865 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:58:22.865 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:58:22.866 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:58:22.866 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:58:22.866 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:58:22.867 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:58:22.867 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:58:22.867 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:58:22.868 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:58:22.868 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:58:22.868 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:58:22.868 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:58:22.869 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:58:22.869 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:58:22.869 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:58:22.869 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.869 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T03:58:22.871 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:58:22.871 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:58:22.871 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:58:22.872 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:58:22.872 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:58:22.872 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:58:22.872 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:58:22.872 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:58:22.873 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:58:22.873 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:58:22.873 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:58:22.873 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:58:22.874 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:58:22.874 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:58:22.874 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:58:22.874 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:58:22.875 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:58:22.875 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:58:22.875 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:58:22.875 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:58:22.876 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:58:22.876 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:58:22.876 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:58:22.876 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:58:22.876 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:58:22.877 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:58:22.877 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.877 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:58:22.877 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:58:22.878 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:58:22.878 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:58:22.878 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:58:22.878 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:58:22.879 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:58:22.879 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:58:22.879 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:58:22.879 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:58:22.880 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:58:22.880 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:58:22.880 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:58:22.880 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T03:58:22.881 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.881 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.881 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:58:22.881 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T03:58:22.882 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.882 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:58:22.882 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T03:58:22.882 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:22.883 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:22.883 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:22.883 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:22.883 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:22.884 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:22.884 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:22.884 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:22.884 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:22.884 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:22.885 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:22.885 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.885 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.885 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.886 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T03:58:22.886 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:22.886 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.886 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.887 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.887 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T03:58:22.887 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:22.887 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.888 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.888 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.888 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:22.888 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.889 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.889 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.889 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T03:58:22.890 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:22.890 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.890 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.890 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.891 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T03:58:22.891 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:22.891 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.891 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.892 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.892 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:22.892 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.892 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.893 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.893 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T03:58:22.893 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:22.893 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.893 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.894 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.894 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T03:58:22.894 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:22.894 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.895 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.895 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.895 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:22.895 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.895 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.896 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.896 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T03:58:22.896 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:22.896 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.896 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.897 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.897 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T03:58:22.897 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:58:22.898 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.898 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.898 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.899 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T03:58:22.899 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T03:58:22.899 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T03:58:22.900 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T03:58:22.900 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:22.900 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:22.900 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:22.901 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.901 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.901 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.901 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:58:22.902 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T03:58:22.902 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:22.902 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:22.902 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:22.902 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:22.903 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:22.903 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:22.903 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:22.903 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:22.904 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:22.904 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:22.904 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:22.904 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.904 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.905 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.905 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T03:58:22.905 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:22.905 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.906 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.906 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.906 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T03:58:22.906 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:22.907 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.907 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.907 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.907 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:22.908 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.908 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.908 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.909 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T03:58:22.909 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:22.909 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.909 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.910 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.910 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T03:58:22.910 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:22.910 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.911 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.911 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.911 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:22.911 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.911 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.912 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.912 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T03:58:22.912 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:22.912 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.914 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.914 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.914 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T03:58:22.915 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:22.915 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.915 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.915 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.916 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:22.916 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.916 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.916 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.917 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T03:58:22.917 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:22.917 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.918 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.918 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.918 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T03:58:22.919 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:58:22.919 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.919 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.919 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.920 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T03:58:22.920 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T03:58:22.920 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T03:58:22.920 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T03:58:22.920 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:22.921 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:22.921 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:22.921 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.921 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.922 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.922 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:58:22.922 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T03:58:22.922 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:22.923 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:22.923 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:22.923 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:22.923 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:22.924 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:22.924 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:22.925 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:22.925 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:22.925 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:22.926 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:22.926 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.927 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.927 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.928 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T03:58:22.928 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:22.929 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.929 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.929 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.930 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T03:58:22.930 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:22.931 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.931 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.931 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.932 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:22.932 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.933 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.933 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.934 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T03:58:22.934 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:22.935 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.935 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.936 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.936 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T03:58:22.936 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:22.937 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.937 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.938 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.938 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:22.939 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.939 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.940 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.940 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T03:58:22.941 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:22.941 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.942 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.942 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.943 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T03:58:22.943 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:22.943 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.943 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.943 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.944 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:22.944 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.944 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.944 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.945 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T03:58:22.945 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:22.945 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.945 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.946 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.946 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T03:58:22.946 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:58:22.946 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.946 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.947 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.947 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T03:58:22.947 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T03:58:22.947 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T03:58:22.948 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T03:58:22.948 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:22.948 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:22.948 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:22.948 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.949 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.949 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.949 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:58:22.949 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T03:58:22.950 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:22.950 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:22.950 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:22.950 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:22.951 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:22.951 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:22.951 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:22.951 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 52, 2021-07-11T03:58:22.952 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:22.952 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:22.952 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:22.952 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.952 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.953 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.953 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T03:58:22.953 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:22.953 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.953 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.954 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.954 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T03:58:22.954 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:22.954 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.955 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.955 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.955 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:22.955 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.956 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.956 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.956 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T03:58:22.956 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:22.957 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.957 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.957 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.957 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T03:58:22.957 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:22.958 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.958 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.958 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.958 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:22.959 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.959 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.959 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.959 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T03:58:22.960 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:22.960 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.960 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.960 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T03:58:22.961 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:22.961 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.961 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.961 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.962 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:22.962 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.962 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.962 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.963 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T03:58:22.963 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:22.963 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.963 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.964 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.964 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T03:58:22.964 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:58:22.964 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.965 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.965 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.965 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T03:58:22.965 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T03:58:22.966 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T03:58:22.966 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T03:58:22.966 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:22.966 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:22.967 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:22.967 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.967 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.967 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.967 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:58:22.968 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T03:58:22.968 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:22.968 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:22.968 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:22.969 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:22.969 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:22.969 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:22.969 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:22.970 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:22.970 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:22.970 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:22.970 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:22.971 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.971 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.971 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.971 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T03:58:22.971 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:22.972 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.972 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.972 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.972 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T03:58:22.973 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:22.973 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.973 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.973 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.974 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:22.974 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.974 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.974 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.975 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T03:58:22.975 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:22.975 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.975 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.976 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.976 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T03:58:22.976 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:22.976 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.976 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.977 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.977 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:22.977 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.977 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.978 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.978 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T03:58:22.978 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:22.978 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.978 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.979 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.979 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T03:58:22.979 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:22.979 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.980 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.980 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.980 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:22.980 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.980 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.981 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.981 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T03:58:22.981 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:22.981 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.982 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.982 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.982 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T03:58:22.983 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:58:22.983 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.983 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.983 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.984 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T03:58:22.984 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T03:58:22.984 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T03:58:22.985 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T03:58:22.985 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:22.985 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:22.986 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:22.986 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.986 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.987 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.987 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:58:22.988 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T03:58:22.988 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:22.988 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:22.989 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:22.989 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:22.989 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:22.989 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:22.990 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:22.990 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:22.990 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:22.991 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:22.991 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:22.991 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.991 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.992 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:22.992 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T03:58:22.993 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:22.993 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.994 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.994 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:22.994 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T03:58:22.995 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:22.995 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:22.996 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:22.996 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:22.997 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:22.997 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:22.997 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:22.998 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:23.000 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T03:58:23.000 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:23.001 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.001 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.001 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:23.002 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T03:58:23.002 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:23.003 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.003 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.003 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.004 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:23.004 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:23.005 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.005 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:23.006 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T03:58:23.006 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:23.007 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.007 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.007 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:23.008 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T03:58:23.008 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:23.008 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.008 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.009 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.009 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:23.009 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:23.009 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.009 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:23.010 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T03:58:23.010 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:23.010 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.010 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.011 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:23.011 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T03:58:23.011 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:58:23.011 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.011 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.012 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.012 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T03:58:23.012 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T03:58:23.012 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T03:58:23.013 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T03:58:23.013 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:23.013 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:23.013 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:23.014 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.014 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.014 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.014 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:58:23.014 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T03:58:23.015 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:23.015 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:23.015 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:23.015 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:23.016 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:23.016 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:23.016 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:23.016 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 53, 2021-07-11T03:58:23.016 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:23.017 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:23.017 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:23.017 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:23.017 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.017 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:23.018 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T03:58:23.018 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:23.018 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.019 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.020 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:23.020 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T03:58:23.020 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:23.020 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.021 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.021 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.021 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:23.021 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:23.021 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.022 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:23.022 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T03:58:23.022 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:23.023 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.023 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.023 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:23.023 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T03:58:23.023 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:23.024 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.024 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.024 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.024 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:23.025 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:23.025 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.025 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:23.025 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T03:58:23.026 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:23.026 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.026 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.026 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:23.026 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T03:58:23.027 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:23.027 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.027 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.027 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.027 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:23.028 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:23.028 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.028 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:23.028 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T03:58:23.029 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:23.029 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.029 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.029 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:23.029 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T03:58:23.030 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:58:23.030 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.030 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.030 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.031 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T03:58:23.031 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T03:58:23.031 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T03:58:23.031 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T03:58:23.031 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:23.032 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:23.032 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:23.032 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.032 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.033 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.033 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:58:23.033 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T03:58:23.033 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:58:23.033 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:58:23.034 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:58:23.034 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:58:23.034 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:58:23.034 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:58:23.035 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:58:23.035 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 52, 2021-07-11T03:58:23.035 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:58:23.035 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:58:23.035 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:58:23.036 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:23.036 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.036 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:23.036 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T03:58:23.037 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:23.037 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.037 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.037 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:23.037 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T03:58:23.038 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:23.038 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.038 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.038 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.039 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:58:23.039 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:23.039 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.040 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:23.040 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T03:58:23.040 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:23.040 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.041 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.041 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:23.041 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T03:58:23.041 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:23.042 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.042 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.042 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.042 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:58:23.042 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:23.043 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.043 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:23.043 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T03:58:23.043 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:23.044 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.044 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.044 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:23.044 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T03:58:23.044 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:23.045 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.045 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.045 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.045 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:58:23.046 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:58:23.046 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.046 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:58:23.046 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T03:58:23.047 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:23.047 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.047 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.047 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:58:23.047 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T03:58:23.048 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:58:23.048 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.048 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.048 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.049 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T03:58:23.049 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T03:58:23.049 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T03:58:23.049 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T03:58:23.050 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:58:23.050 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:58:23.050 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:58:23.050 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:23.050 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.051 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:58:23.051 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T03:58:23.051 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.051 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:58:23.052 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:23.052 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:23.052 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:23.052 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:23.052 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:23.053 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T03:58:23.053 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:23.053 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.053 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.054 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:58:23.054 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:23.054 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:23.054 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:23.055 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:23.055 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:23.055 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T03:58:23.055 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:23.055 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.056 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.056 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:58:23.056 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:23.056 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:23.056 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:23.057 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:23.057 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:23.057 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T03:58:23.057 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:23.058 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.058 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.058 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:58:23.058 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:23.059 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:23.059 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:23.059 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:23.059 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:23.059 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T03:58:23.060 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:23.060 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.060 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.060 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:58:23.061 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:23.061 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:23.061 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:23.061 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:23.061 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:23.062 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T03:58:23.062 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:23.062 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.062 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.062 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:58:23.063 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:23.063 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:23.063 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:23.063 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:23.064 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:23.064 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T03:58:23.064 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:23.064 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.064 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.065 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:58:23.065 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:23.065 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:23.065 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:23.066 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:23.066 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:23.066 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T03:58:23.066 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:23.066 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.067 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:23.067 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:58:23.067 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:58:23.067 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:58:23.068 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:58:23.068 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:58:23.068 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:58:23.068 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T03:58:23.068 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:58:23.069 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.069 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:58:23.069 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T03:58:23.069 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T03:58:23.069 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T03:58:23.070 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T03:58:23.070 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T03:58:23.070 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6835/2428941897": "2021-07-12T03:58:17.163538+0000", 2021-07-11T03:58:23.070 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6834/2428941897": "2021-07-12T03:58:17.163538+0000" 2021-07-11T03:58:23.071 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.071 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T03:58:23.071 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T03:58:23.071 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T03:58:23.071 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T03:58:23.072 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T03:58:23.072 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T03:58:23.072 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T03:58:23.072 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T03:58:23.073 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:23.073 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:23.073 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T03:58:23.073 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T03:58:23.073 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T03:58:23.074 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T03:58:23.074 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T03:58:23.074 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:58:23.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 -- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9150053350 msgr2=0x7f9150055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9150053350 0x7f9150055810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f916c002f70 tx=0x7f916c006a10).stop 2021-07-11T03:58:23.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 -- 172.21.2.104:0/71792023 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91700a70c0 msgr2=0x7f917015aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91700a70c0 0x7f917015aa10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f91700ace60 tx=0x7f9164015000).stop 2021-07-11T03:58:23.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 -- 172.21.2.104:0/71792023 shutdown_connections 2021-07-11T03:58:23.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91701483a0 0x7f9170156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:23.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f91700b1db0 0x7f9170146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:23.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9150053350 0x7f9150055810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 --2- 172.21.2.104:0/71792023 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91700a70c0 0x7f917015aa10 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 -- 172.21.2.104:0/71792023 >> 172.21.2.104:0/71792023 conn(0x7f917005f6b0 msgr2=0x7f91700b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:23.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 -- 172.21.2.104:0/71792023 shutdown_connections 2021-07-11T03:58:23.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:22.809+0000 7f91780c8700 1 -- 172.21.2.104:0/71792023 wait complete. 2021-07-11T03:58:23.079 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:58:23.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.133+0000 7f86e85b1700 1 Processor -- start 2021-07-11T03:58:23.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.133+0000 7f86e85b1700 1 -- start start 2021-07-11T03:58:23.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.133+0000 7f86e85b1700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f86e00b1970 0x7f86e005fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.133+0000 7f86e85b1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e00a7a90 con 0x7f86e0067180 2021-07-11T03:58:23.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.133+0000 7f86e85b1700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e0147620 con 0x7f86e00a76b0 2021-07-11T03:58:23.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.133+0000 7f86e85b1700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e01478c0 con 0x7f86e00b1970 2021-07-11T03:58:23.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.133+0000 7f86e5b4c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f86e0067180 0x7f86e005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42072/0 (socket says 172.21.2.104:42072) 2021-07-11T03:58:23.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.133+0000 7f86e5b4c700 1 -- 172.21.2.104:0/4153097839 learned_addr learned my addr 172.21.2.104:0/4153097839 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:23.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.133+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495825120 0 0) 0x7f86e00a7a90 con 0x7f86e0067180 2021-07-11T03:58:23.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.133+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c8003260 con 0x7f86e0067180 2021-07-11T03:58:23.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.133+0000 7f86e6b4e700 1 --2- 172.21.2.104:0/4153097839 >> v2:172.21.2.105:3300/0 conn(0x7f86e00b1970 0x7f86e005fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4069805540 0 0) 0x7f86c8003260 con 0x7f86e0067180 2021-07-11T03:58:23.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 >> v2:172.21.2.105:3300/0 conn(0x7f86e00b1970 msgr2=0x7f86e005fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:23.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 --2- 172.21.2.104:0/4153097839 >> v2:172.21.2.105:3300/0 conn(0x7f86e00b1970 0x7f86e005fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 >> v1:172.21.2.105:6789/0 conn(0x7f86e00a76b0 legacy=0x7f86e0066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:23.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e0147b60 con 0x7f86e0067180 2021-07-11T03:58:23.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f86d4001660 con 0x7f86e0067180 2021-07-11T03:58:23.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 >> v1:172.21.2.104:6789/0 conn(0x7f86e0067180 legacy=0x7f86e005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 --2- 172.21.2.104:0/4153097839 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f86c8003820 0x7f86c8003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 --2- 172.21.2.104:0/4153097839 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 0x7f86c8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 --2- 172.21.2.104:0/4153097839 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f86c8004f90 0x7f86c800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e01478c0 con 0x7f86c8004500 2021-07-11T03:58:23.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c800be80 con 0x7f86c8004f90 2021-07-11T03:58:23.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c800c040 con 0x7f86c8003820 2021-07-11T03:58:23.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e5b4c700 1 --2- 172.21.2.104:0/4153097839 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 0x7f86c8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e5b4c700 1 -- 172.21.2.104:0/4153097839 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f86c8003820 msgr2=0x7f86c8003c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:23.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e5b4c700 1 --2- 172.21.2.104:0/4153097839 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f86c8003820 0x7f86c8003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:23.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e5b4c700 1 -- 172.21.2.104:0/4153097839 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f86c8004f90 msgr2=0x7f86c800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:23.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e5b4c700 1 --2- 172.21.2.104:0/4153097839 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f86c8004f90 0x7f86c800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:23.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e6b4e700 1 -- 172.21.2.104:0/4153097839 reap_dead start 2021-07-11T03:58:23.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e5b4c700 1 -- 172.21.2.104:0/4153097839 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c8003260 con 0x7f86c8004500 2021-07-11T03:58:23.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e5b4c700 1 --2- 172.21.2.104:0/4153097839 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 0x7f86c8004980 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f86e006a180 tx=0x7f86e00a8c80).ready entity=mon.0 client_cookie=728eb65ba49346b3 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:23.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86d40070d0 con 0x7f86c8004500 2021-07-11T03:58:23.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d4007230 con 0x7f86c8004500 2021-07-11T03:58:23.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e534b700 1 -- 172.21.2.104:0/4153097839 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86d400c8c0 con 0x7f86c8004500 2021-07-11T03:58:23.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 -- 172.21.2.104:0/4153097839 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 msgr2=0x7f86c8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 --2- 172.21.2.104:0/4153097839 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 0x7f86c8004980 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f86e006a180 tx=0x7f86e00a8c80).stop 2021-07-11T03:58:23.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 -- 172.21.2.104:0/4153097839 shutdown_connections 2021-07-11T03:58:23.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 --2- 172.21.2.104:0/4153097839 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 0x7f86c8004980 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 -- 172.21.2.104:0/4153097839 >> 172.21.2.104:0/4153097839 conn(0x7f86e00b53e0 msgr2=0x7f86e00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:23.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 -- 172.21.2.104:0/4153097839 shutdown_connections 2021-07-11T03:58:23.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 -- 172.21.2.104:0/4153097839 wait complete. 2021-07-11T03:58:23.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 Processor -- start 2021-07-11T03:58:23.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 -- start start 2021-07-11T03:58:23.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 0x7f86e014cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f86e00a9230 0x7f86e00ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f86e0156e50 0x7f86e00acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e00adfd0 con 0x7f86c8004500 2021-07-11T03:58:23.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e00ae270 con 0x7f86e00a9230 2021-07-11T03:58:23.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e0156710 con 0x7f86e0156e50 2021-07-11T03:58:23.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e634d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 0x7f86e014cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e634d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 0x7f86e014cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45318/0 (socket says 172.21.2.104:45318) 2021-07-11T03:58:23.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e634d700 1 -- 172.21.2.104:0/1503772545 learned_addr learned my addr 172.21.2.104:0/1503772545 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:23.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e634d700 1 -- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f86e0156e50 msgr2=0x7f86e00acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e634d700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f86e0156e50 0x7f86e00acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:23.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e5b4c700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f86e00a9230 0x7f86e00ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e634d700 1 -- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f86e00a9230 msgr2=0x7f86e00ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e634d700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f86e00a9230 0x7f86e00ac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e634d700 1 -- 172.21.2.104:0/1503772545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e01569b0 con 0x7f86c8004500 2021-07-11T03:58:23.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e634d700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 0x7f86e014cfb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f86dc001fa0 tx=0x7f86dc007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:23.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86d37fe700 1 -- 172.21.2.104:0/1503772545 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86dc00ce80 con 0x7f86c8004500 2021-07-11T03:58:23.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86d37fe700 1 -- 172.21.2.104:0/1503772545 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc000e90 con 0x7f86c8004500 2021-07-11T03:58:23.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86d37fe700 1 -- 172.21.2.104:0/1503772545 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f86dc007430 con 0x7f86c8004500 2021-07-11T03:58:23.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 -- 172.21.2.104:0/1503772545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e014e8e0 con 0x7f86c8004500 2021-07-11T03:58:23.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.137+0000 7f86e85b1700 1 -- 172.21.2.104:0/1503772545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e01572d0 con 0x7f86c8004500 2021-07-11T03:58:23.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.141+0000 7f86e85b1700 1 -- 172.21.2.104:0/1503772545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e0057350 con 0x7f86c8004500 2021-07-11T03:58:23.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.141+0000 7f86d37fe700 1 -- 172.21.2.104:0/1503772545 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f86dc010030 con 0x7f86c8004500 2021-07-11T03:58:23.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.141+0000 7f86d37fe700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f86c0053270 0x7f86c0055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.141+0000 7f86d37fe700 1 -- 172.21.2.104:0/1503772545 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f86dc06aa40 con 0x7f86c8004500 2021-07-11T03:58:23.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.141+0000 7f86e5b4c700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f86c0053270 0x7f86c0055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.141+0000 7f86e5b4c700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f86c0053270 0x7f86c0055730 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f86d4015000 tx=0x7f86d4013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:23.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.141+0000 7f86d37fe700 1 -- 172.21.2.104:0/1503772545 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f86dc041f50 con 0x7f86c8004500 2021-07-11T03:58:23.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 -- 172.21.2.104:0/1503772545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86e00698e0 con 0x7f86c8004500 2021-07-11T03:58:23.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86d37fe700 1 -- 172.21.2.104:0/1503772545 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+1562 (secure 0 0 0) 0x7f86dc045570 con 0x7f86c8004500 2021-07-11T03:58:23.256 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:23.257 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3427300973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3427300973},{"type":"v1","addr":"172.21.2.105:6835","nonce":3427300973}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/2823255258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2823255258},{"type":"v1","addr":"172.21.2.104:6833","nonce":2823255258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/1636145007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":1636145007},{"type":"v1","addr":"172.21.2.104:6835","nonce":1636145007}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[]} 2021-07-11T03:58:23.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 -- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f86c0053270 msgr2=0x7f86c0055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f86c0053270 0x7f86c0055730 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f86d4015000 tx=0x7f86d4013000).stop 2021-07-11T03:58:23.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 -- 172.21.2.104:0/1503772545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 msgr2=0x7f86e014cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 0x7f86e014cfb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f86dc001fa0 tx=0x7f86dc007c30).stop 2021-07-11T03:58:23.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 -- 172.21.2.104:0/1503772545 shutdown_connections 2021-07-11T03:58:23.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f86e0156e50 0x7f86e00acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:23.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f86e00a9230 0x7f86e00ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f86c0053270 0x7f86c0055730 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 --2- 172.21.2.104:0/1503772545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f86c8004500 0x7f86e014cfb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 -- 172.21.2.104:0/1503772545 >> 172.21.2.104:0/1503772545 conn(0x7f86e00b53e0 msgr2=0x7f86e00b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:23.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 -- 172.21.2.104:0/1503772545 shutdown_connections 2021-07-11T03:58:23.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.249+0000 7f86e85b1700 1 -- 172.21.2.104:0/1503772545 wait complete. 2021-07-11T03:58:23.261 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 24 2021-07-11T03:58:23.264 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:58:23.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.305+0000 7fc9bb830700 1 Processor -- start 2021-07-11T03:58:23.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.305+0000 7fc9bb830700 1 -- start start 2021-07-11T03:58:23.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9bb830700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9b40b1170 0x7fc9b4141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9bb830700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc9b4141920 0x7fc9b4141d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9bb830700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9b40b31d0 con 0x7fc9b40b7810 2021-07-11T03:58:23.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9bb830700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9b40b7bf0 con 0x7fc9b4141920 2021-07-11T03:58:23.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9bb830700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b40675c0 con 0x7fc9b40b1170 2021-07-11T03:58:23.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9b8dcb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9b40b1170 0x7fc9b4141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9b8dcb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9b40b1170 0x7fc9b4141430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:48490/0 (socket says 172.21.2.104:48490) 2021-07-11T03:58:23.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9b8dcb700 1 -- 172.21.2.104:0/4005127687 learned_addr learned my addr 172.21.2.104:0/4005127687 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:23.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9b9dcd700 1 --2- 172.21.2.104:0/4005127687 >> v2:172.21.2.105:3300/0 conn(0x7fc9b4141920 0x7fc9b4141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9b8dcb700 1 -- 172.21.2.104:0/4005127687 >> v2:172.21.2.105:3300/0 conn(0x7fc9b4141920 msgr2=0x7fc9b4141d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:23.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9b8dcb700 1 --2- 172.21.2.104:0/4005127687 >> v2:172.21.2.105:3300/0 conn(0x7fc9b4141920 0x7fc9b4141d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9b8dcb700 1 -- 172.21.2.104:0/4005127687 >> v1:172.21.2.105:6789/0 conn(0x7fc9b40b7810 legacy=0x7fc9b41445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9b8dcb700 1 -- 172.21.2.104:0/4005127687 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b4142270 con 0x7fc9b40b1170 2021-07-11T03:58:23.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9b8dcb700 1 --2- 172.21.2.104:0/4005127687 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9b40b1170 0x7fc9b4141430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fc9b0002f70 tx=0x7fc9b0006eb0).ready entity=mon.2 client_cookie=ea039312f94efeab server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:23.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9a3fff700 1 -- 172.21.2.104:0/4005127687 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9b000d570 con 0x7fc9b40b1170 2021-07-11T03:58:23.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9a3fff700 1 -- 172.21.2.104:0/4005127687 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b0000e90 con 0x7fc9b40b1170 2021-07-11T03:58:23.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9a3fff700 1 -- 172.21.2.104:0/4005127687 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9b000d900 con 0x7fc9b40b1170 2021-07-11T03:58:23.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9bb830700 1 -- 172.21.2.104:0/4005127687 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9b40b1170 msgr2=0x7fc9b4141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9bb830700 1 --2- 172.21.2.104:0/4005127687 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9b40b1170 0x7fc9b4141430 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fc9b0002f70 tx=0x7fc9b0006eb0).stop 2021-07-11T03:58:23.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9bb830700 1 -- 172.21.2.104:0/4005127687 shutdown_connections 2021-07-11T03:58:23.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9bb830700 1 --2- 172.21.2.104:0/4005127687 >> v2:172.21.2.105:3300/0 conn(0x7fc9b4141920 0x7fc9b4141d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9bb830700 1 --2- 172.21.2.104:0/4005127687 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9b40b1170 0x7fc9b4141430 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.309+0000 7fc9bb830700 1 -- 172.21.2.104:0/4005127687 >> 172.21.2.104:0/4005127687 conn(0x7fc9b405f6b0 msgr2=0x7fc9b405fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:23.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 -- 172.21.2.104:0/4005127687 shutdown_connections 2021-07-11T03:58:23.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 -- 172.21.2.104:0/4005127687 wait complete. 2021-07-11T03:58:23.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 Processor -- start 2021-07-11T03:58:23.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 -- start start 2021-07-11T03:58:23.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9b40b1170 0x7fc9b40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9b40b7810 0x7fc9b4140060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc9b4141920 0x7fc9b4140550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b40b02c0 con 0x7fc9b40b7810 2021-07-11T03:58:23.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b40afc50 con 0x7fc9b4141920 2021-07-11T03:58:23.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b413c850 con 0x7fc9b40b1170 2021-07-11T03:58:23.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9b8dcb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9b40b7810 0x7fc9b4140060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9b8dcb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9b40b7810 0x7fc9b4140060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45330/0 (socket says 172.21.2.104:45330) 2021-07-11T03:58:23.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9b8dcb700 1 -- 172.21.2.104:0/741873798 learned_addr learned my addr 172.21.2.104:0/741873798 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:23.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9b8dcb700 1 -- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9b40b1170 msgr2=0x7fc9b40aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9b8dcb700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9b40b1170 0x7fc9b40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:23.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9b8dcb700 1 -- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc9b4141920 msgr2=0x7fc9b4140550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9b8dcb700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc9b4141920 0x7fc9b4140550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:23.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9b8dcb700 1 -- 172.21.2.104:0/741873798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b000b1e0 con 0x7fc9b40b7810 2021-07-11T03:58:23.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9b8dcb700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9b40b7810 0x7fc9b4140060 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fc9b0002f70 tx=0x7fc9b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:23.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9a27fc700 1 -- 172.21.2.104:0/741873798 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9b0021070 con 0x7fc9b40b7810 2021-07-11T03:58:23.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9a27fc700 1 -- 172.21.2.104:0/741873798 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b0000e90 con 0x7fc9b40b7810 2021-07-11T03:58:23.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 -- 172.21.2.104:0/741873798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b413db60 con 0x7fc9b40b7810 2021-07-11T03:58:23.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 -- 172.21.2.104:0/741873798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b4140a40 con 0x7fc9b40b7810 2021-07-11T03:58:23.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9a27fc700 1 -- 172.21.2.104:0/741873798 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9b0011a50 con 0x7fc9b40b7810 2021-07-11T03:58:23.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9bb830700 1 -- 172.21.2.104:0/741873798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9b413d9c0 con 0x7fc9b40b7810 2021-07-11T03:58:23.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9a27fc700 1 -- 172.21.2.104:0/741873798 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc9b000f070 con 0x7fc9b40b7810 2021-07-11T03:58:23.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9a27fc700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc994053180 0x7fc994055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9a27fc700 1 -- 172.21.2.104:0/741873798 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc9b00477b0 con 0x7fc9b40b7810 2021-07-11T03:58:23.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.313+0000 7fc9b95cc700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc994053180 0x7fc994055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.317+0000 7fc9b95cc700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc994053180 0x7fc994055640 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7fc9b4060740 tx=0x7fc9a4007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:23.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.317+0000 7fc9a27fc700 1 -- 172.21.2.104:0/741873798 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc9b413d9c0 con 0x7fc9b40b7810 2021-07-11T03:58:23.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 -- 172.21.2.104:0/741873798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9b413d9c0 con 0x7fc9b40b7810 2021-07-11T03:58:23.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9a27fc700 1 -- 172.21.2.104:0/741873798 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+1562 (secure 0 0 0) 0x7fc9b413d9c0 con 0x7fc9b40b7810 2021-07-11T03:58:23.434 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:23.434 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":24,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4618,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3427300973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3427300973},{"type":"v1","addr":"172.21.2.105:6835","nonce":3427300973}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/2823255258","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2823255258},{"type":"v1","addr":"172.21.2.104:6833","nonce":2823255258}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":20},{"gid":5880,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/1636145007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":1636145007},{"type":"v1","addr":"172.21.2.104:6835","nonce":1636145007}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":24}],"filesystems":[]} 2021-07-11T03:58:23.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 -- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc994053180 msgr2=0x7fc994055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc994053180 0x7fc994055640 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7fc9b4060740 tx=0x7fc9a4007810).stop 2021-07-11T03:58:23.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 -- 172.21.2.104:0/741873798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9b40b7810 msgr2=0x7fc9b4140060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9b40b7810 0x7fc9b4140060 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fc9b0002f70 tx=0x7fc9b0015000).stop 2021-07-11T03:58:23.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 -- 172.21.2.104:0/741873798 shutdown_connections 2021-07-11T03:58:23.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc9b4141920 0x7fc9b4140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:23.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc994053180 0x7fc994055640 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9b40b7810 0x7fc9b4140060 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 --2- 172.21.2.104:0/741873798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9b40b1170 0x7fc9b40aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:23.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 -- 172.21.2.104:0/741873798 >> 172.21.2.104:0/741873798 conn(0x7fc9b405f6b0 msgr2=0x7fc9b40b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:23.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 -- 172.21.2.104:0/741873798 shutdown_connections 2021-07-11T03:58:23.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.425+0000 7fc9bb830700 1 -- 172.21.2.104:0/741873798 wait complete. 2021-07-11T03:58:23.438 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 24 2021-07-11T03:58:23.442 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T03:58:23.442 INFO:tasks.ceph.mds.a:Stopping old one... 2021-07-11T03:58:23.443 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T03:58:23.443 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:58:23.443 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T03:58:23.444 INFO:tasks.ceph.mds.c:Stopping old one... 2021-07-11T03:58:23.444 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T03:58:23.444 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:58:23.444 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T03:58:23.445 INFO:tasks.ceph.mds.b:Stopping old one... 2021-07-11T03:58:23.445 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T03:58:23.445 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:58:23.448 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T03:58:23.449 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T03:58:23.450 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T03:58:23.450 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T03:58:23.451 INFO:tasks.ceph.mds.a:Started 2021-07-11T03:58:23.452 DEBUG:teuthology.parallel:result is None 2021-07-11T03:58:23.453 INFO:tasks.ceph.mds.c:Started 2021-07-11T03:58:23.453 DEBUG:teuthology.parallel:result is None 2021-07-11T03:58:23.456 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T03:58:23.457 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T03:58:23.483 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T03:58:23.484 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T03:58:23.500 INFO:tasks.ceph.mds.b:Started 2021-07-11T03:58:23.500 DEBUG:teuthology.parallel:result is None 2021-07-11T03:58:23.501 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T03:58:23.533 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T03:58:23.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 Processor -- start 2021-07-11T03:58:23.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 -- start start 2021-07-11T03:58:23.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fde280ad150 0x7fde280b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde280ac370 con 0x7fde280a9da0 2021-07-11T03:58:23.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde280aac40 con 0x7fde280aa860 2021-07-11T03:58:23.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde280abb50 con 0x7fde280ad150 2021-07-11T03:58:23.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde26d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fde280a9da0 0x7fde28140340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42138/0 (socket says 172.21.2.104:42138) 2021-07-11T03:58:23.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde26d9d700 1 -- 172.21.2.104:0/2516102261 learned_addr learned my addr 172.21.2.104:0/2516102261 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:23.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915135366 0 0) 0x7fde280ac370 con 0x7fde280a9da0 2021-07-11T03:58:23.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde08003260 con 0x7fde280a9da0 2021-07-11T03:58:23.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2066681634 0 0) 0x7fde08003260 con 0x7fde280a9da0 2021-07-11T03:58:23.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 >> v2:172.21.2.105:3300/0 conn(0x7fde280ad150 msgr2=0x7fde280b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:23.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 --2- 172.21.2.104:0/2516102261 >> v2:172.21.2.105:3300/0 conn(0x7fde280ad150 0x7fde280b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:23.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 >> v1:172.21.2.105:6789/0 conn(0x7fde280aa860 legacy=0x7fde280aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:23.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde280b16b0 con 0x7fde280a9da0 2021-07-11T03:58:23.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fde200019e0 con 0x7fde280a9da0 2021-07-11T03:58:23.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 >> v1:172.21.2.104:6789/0 conn(0x7fde280a9da0 legacy=0x7fde28140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde08003820 0x7fde08003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 0x7fde08004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde08005050 0x7fde0800d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0800e120 con 0x7fde08004500 2021-07-11T03:58:23.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0800e290 con 0x7fde08003820 2021-07-11T03:58:23.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0800e450 con 0x7fde08005050 2021-07-11T03:58:23.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde25d9b700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 0x7fde08004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde26d9d700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde08005050 0x7fde0800d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde25d9b700 1 -- 172.21.2.104:0/2516102261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde08005050 msgr2=0x7fde0800d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:23.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde25d9b700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde08005050 0x7fde0800d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2659c700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde08003820 0x7fde08003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde25d9b700 1 -- 172.21.2.104:0/2516102261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde08003820 msgr2=0x7fde08003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:23.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde25d9b700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde08003820 0x7fde08003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde25d9b700 1 -- 172.21.2.104:0/2516102261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde08003260 con 0x7fde08004500 2021-07-11T03:58:23.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde26d9d700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde08005050 0x7fde0800d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:58:23.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde26d9d700 1 -- 172.21.2.104:0/2516102261 reap_dead start 2021-07-11T03:58:23.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2659c700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde08003820 0x7fde08003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:58:23.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde25d9b700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 0x7fde08004980 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fde18002f70 tx=0x7fde18006ca0).ready entity=mon.0 client_cookie=3467543b85f984c0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:23.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde18013410 con 0x7fde08004500 2021-07-11T03:58:23.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde18013570 con 0x7fde08004500 2021-07-11T03:58:23.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2559a700 1 -- 172.21.2.104:0/2516102261 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde180138d0 con 0x7fde08004500 2021-07-11T03:58:23.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2516102261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 msgr2=0x7fde08004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 0x7fde08004980 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fde18002f70 tx=0x7fde18006ca0).stop 2021-07-11T03:58:23.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2516102261 shutdown_connections 2021-07-11T03:58:23.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 --2- 172.21.2.104:0/2516102261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 0x7fde08004980 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2516102261 >> 172.21.2.104:0/2516102261 conn(0x7fde280b64d0 msgr2=0x7fde280b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:23.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2516102261 shutdown_connections 2021-07-11T03:58:23.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.573+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2516102261 wait complete. 2021-07-11T03:58:23.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2cfd9700 1 Processor -- start 2021-07-11T03:58:23.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2cfd9700 1 -- start start 2021-07-11T03:58:23.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2cfd9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 0x7fde280ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2cfd9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde28065c80 0x7fde28065240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2cfd9700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde28065730 0x7fde28151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2cfd9700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde280668f0 con 0x7fde08004500 2021-07-11T03:58:23.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2cfd9700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde280641c0 con 0x7fde28065730 2021-07-11T03:58:23.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2cfd9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2814e740 con 0x7fde28065c80 2021-07-11T03:58:23.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 0x7fde280ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 0x7fde280ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45380/0 (socket says 172.21.2.104:45380) 2021-07-11T03:58:23.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2659c700 1 -- 172.21.2.104:0/2817091365 learned_addr learned my addr 172.21.2.104:0/2817091365 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:23.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde26d9d700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde28065730 0x7fde28151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde25d9b700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde28065c80 0x7fde28065240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2659c700 1 -- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde28065c80 msgr2=0x7fde28065240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2659c700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde28065c80 0x7fde28065240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2659c700 1 -- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde28065730 msgr2=0x7fde28151170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:23.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2659c700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde28065730 0x7fde28151170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:23.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2659c700 1 -- 172.21.2.104:0/2817091365 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2814e9e0 con 0x7fde08004500 2021-07-11T03:58:23.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2659c700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 0x7fde280ae5a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fde10008000 tx=0x7fde10009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:23.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde1f7fe700 1 -- 172.21.2.104:0/2817091365 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde10013420 con 0x7fde08004500 2021-07-11T03:58:23.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde1f7fe700 1 -- 172.21.2.104:0/2817091365 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde10013580 con 0x7fde08004500 2021-07-11T03:58:23.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde1f7fe700 1 -- 172.21.2.104:0/2817091365 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fde100138e0 con 0x7fde08004500 2021-07-11T03:58:23.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2817091365 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde28064460 con 0x7fde08004500 2021-07-11T03:58:23.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2817091365 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde280aec30 con 0x7fde08004500 2021-07-11T03:58:23.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2817091365 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde28063da0 con 0x7fde08004500 2021-07-11T03:58:23.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde1f7fe700 1 -- 172.21.2.104:0/2817091365 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fde10013a40 con 0x7fde08004500 2021-07-11T03:58:23.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde1f7fe700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde0c053180 0x7fde0c055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:23.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde1f7fe700 1 -- 172.21.2.104:0/2817091365 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fde1006ace0 con 0x7fde08004500 2021-07-11T03:58:23.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde25d9b700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde0c053180 0x7fde0c055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:23.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.577+0000 7fde25d9b700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde0c053180 0x7fde0c055640 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fde18019000 tx=0x7fde18014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:23.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.581+0000 7fde1f7fe700 1 -- 172.21.2.104:0/2817091365 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fde10042300 con 0x7fde08004500 2021-07-11T03:58:23.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:23.673+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2817091365 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fde28057350 con 0x7fde08004500 2021-07-11T03:58:24.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde1f7fe700 1 -- 172.21.2.104:0/2817091365 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v60) v1 ==== 97+0+0 (secure 0 0 0) 0x7fde10021080 con 0x7fde08004500 2021-07-11T03:58:24.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde0c053180 msgr2=0x7fde0c055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde0c053180 0x7fde0c055640 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fde18019000 tx=0x7fde18014000).stop 2021-07-11T03:58:24.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2817091365 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 msgr2=0x7fde280ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 0x7fde280ae5a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7fde10008000 tx=0x7fde10009040).stop 2021-07-11T03:58:24.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2817091365 shutdown_connections 2021-07-11T03:58:24.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fde28065730 0x7fde28151170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fde28065c80 0x7fde28065240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fde0c053180 0x7fde0c055640 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 --2- 172.21.2.104:0/2817091365 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fde08004500 0x7fde280ae5a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2817091365 >> 172.21.2.104:0/2817091365 conn(0x7fde280b64d0 msgr2=0x7fde28061520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:24.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2817091365 shutdown_connections 2021-07-11T03:58:24.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.505+0000 7fde2cfd9700 1 -- 172.21.2.104:0/2817091365 wait complete. 2021-07-11T03:58:24.517 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T03:58:24.519 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T03:58:24.520 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T03:58:24.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 Processor -- start 2021-07-11T03:58:24.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 -- start start 2021-07-11T03:58:24.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0a140b0010 0x7f0a140a83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a140a70c0 con 0x7f0a140b2580 2021-07-11T03:58:24.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a140a72f0 con 0x7f0a140afbd0 2021-07-11T03:58:24.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a140a7590 con 0x7f0a140b0010 2021-07-11T03:58:24.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a18885700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0a140b0010 0x7f0a140a83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a19086700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f0a140b2580 0x7f0a1414ae40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42158/0 (socket says 172.21.2.104:42158) 2021-07-11T03:58:24.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a19086700 1 -- 172.21.2.104:0/1785534486 learned_addr learned my addr 172.21.2.104:0/1785534486 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:24.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a18885700 1 -- 172.21.2.104:0/1785534486 >> v1:172.21.2.105:6789/0 conn(0x7f0a140afbd0 legacy=0x7f0a14151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:24.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a18885700 1 -- 172.21.2.104:0/1785534486 >> v1:172.21.2.104:6789/0 conn(0x7f0a140b2580 legacy=0x7f0a1414ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a18885700 1 -- 172.21.2.104:0/1785534486 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a140a7830 con 0x7f0a140b0010 2021-07-11T03:58:24.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a18885700 1 --2- 172.21.2.104:0/1785534486 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140b0010 0x7f0a140a83b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f0a14061a80 tx=0x7f0a04006df0).ready entity=mon.0 client_cookie=26e8cbe3dfa10578 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:24.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a03fff700 1 -- 172.21.2.104:0/1785534486 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a0400d5c0 con 0x7f0a140b0010 2021-07-11T03:58:24.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a03fff700 1 -- 172.21.2.104:0/1785534486 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a04000e90 con 0x7f0a140b0010 2021-07-11T03:58:24.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a03fff700 1 -- 172.21.2.104:0/1785534486 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a0400d910 con 0x7f0a140b0010 2021-07-11T03:58:24.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 -- 172.21.2.104:0/1785534486 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140b0010 msgr2=0x7f0a140a83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 --2- 172.21.2.104:0/1785534486 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140b0010 0x7f0a140a83b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f0a14061a80 tx=0x7f0a04006df0).stop 2021-07-11T03:58:24.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 -- 172.21.2.104:0/1785534486 shutdown_connections 2021-07-11T03:58:24.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 --2- 172.21.2.104:0/1785534486 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140b0010 0x7f0a140a83b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 -- 172.21.2.104:0/1785534486 >> 172.21.2.104:0/1785534486 conn(0x7f0a14060180 msgr2=0x7f0a140b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:24.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 -- 172.21.2.104:0/1785534486 shutdown_connections 2021-07-11T03:58:24.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.565+0000 7f0a1a088700 1 -- 172.21.2.104:0/1785534486 wait complete. 2021-07-11T03:58:24.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a1a088700 1 Processor -- start 2021-07-11T03:58:24.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a1a088700 1 -- start start 2021-07-11T03:58:24.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a1a088700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140afbd0 0x7f0a140aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a1a088700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a140b0010 0x7f0a14063000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a1a088700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a140b2580 0x7f0a140634f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a1a088700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14154510 con 0x7f0a140afbd0 2021-07-11T03:58:24.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a1a088700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140ab3f0 con 0x7f0a140b0010 2021-07-11T03:58:24.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a1a088700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140abe00 con 0x7f0a140b2580 2021-07-11T03:58:24.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a19086700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140afbd0 0x7f0a140aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a19086700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140afbd0 0x7f0a140aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45396/0 (socket says 172.21.2.104:45396) 2021-07-11T03:58:24.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a19086700 1 -- 172.21.2.104:0/2304598717 learned_addr learned my addr 172.21.2.104:0/2304598717 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:24.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a19086700 1 -- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a140b2580 msgr2=0x7f0a140634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a19086700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a140b2580 0x7f0a140634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:24.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a19086700 1 -- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a140b0010 msgr2=0x7f0a14063000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a19086700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a140b0010 0x7f0a14063000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:24.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a19086700 1 -- 172.21.2.104:0/2304598717 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0400b180 con 0x7f0a140afbd0 2021-07-11T03:58:24.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a19086700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140afbd0 0x7f0a140aecb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f0a1406b7c0 tx=0x7f0a10007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:24.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a027fc700 1 -- 172.21.2.104:0/2304598717 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1000ce30 con 0x7f0a140afbd0 2021-07-11T03:58:24.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a027fc700 1 -- 172.21.2.104:0/2304598717 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a100072e0 con 0x7f0a140afbd0 2021-07-11T03:58:24.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a027fc700 1 -- 172.21.2.104:0/2304598717 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a10005800 con 0x7f0a140afbd0 2021-07-11T03:58:24.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a1a088700 1 -- 172.21.2.104:0/2304598717 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a140af790 con 0x7f0a140afbd0 2021-07-11T03:58:24.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a1a088700 1 -- 172.21.2.104:0/2304598717 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a140ab620 con 0x7f0a140afbd0 2021-07-11T03:58:24.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a1a088700 1 -- 172.21.2.104:0/2304598717 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a140abf40 con 0x7f0a140afbd0 2021-07-11T03:58:24.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a027fc700 1 -- 172.21.2.104:0/2304598717 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0a10010070 con 0x7f0a140afbd0 2021-07-11T03:58:24.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a027fc700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f40531f0 0x7f09f40556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a027fc700 1 -- 172.21.2.104:0/2304598717 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0a1006a840 con 0x7f0a140afbd0 2021-07-11T03:58:24.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.569+0000 7f0a18885700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f40531f0 0x7f09f40556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.573+0000 7f0a18885700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f40531f0 0x7f09f40556b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f0a14061a80 tx=0x7f0a04012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:24.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.573+0000 7f0a027fc700 1 -- 172.21.2.104:0/2304598717 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0a10041eb0 con 0x7f0a140afbd0 2021-07-11T03:58:24.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 -- 172.21.2.104:0/2304598717 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0a14057350 con 0x7f0a140afbd0 2021-07-11T03:58:24.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a027fc700 1 -- 172.21.2.104:0/2304598717 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f0a100454d0 con 0x7f0a140afbd0 2021-07-11T03:58:24.693 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:24.693 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:58:24.693 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T03:58:24.693 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.693 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T03:58:24.694 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T03:58:24.694 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.694 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:58:24.694 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.695 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.695 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.695 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.695 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.696 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.696 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T03:58:24.696 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T03:58:24.696 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.697 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:58:24.697 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.699 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.700 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.700 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.700 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.700 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.700 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T03:58:24.701 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T03:58:24.701 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.701 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:58:24.701 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.702 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.702 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.702 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.702 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.703 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.703 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T03:58:24.705 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T03:58:24.705 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.705 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.706 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.706 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.706 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.706 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.707 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.707 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T03:58:24.707 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T03:58:24.707 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.708 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.708 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.708 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.708 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.708 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.709 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.709 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T03:58:24.709 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T03:58:24.709 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.710 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.710 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.710 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.711 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.711 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.711 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.711 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T03:58:24.711 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T03:58:24.712 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.712 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.712 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.712 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.712 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.713 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.713 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.713 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T03:58:24.713 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T03:58:24.714 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.714 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.714 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.714 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.715 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.715 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.715 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.715 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T03:58:24.716 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T03:58:24.716 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.716 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.716 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.717 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.717 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.717 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.717 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.718 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T03:58:24.718 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T03:58:24.718 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.718 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.719 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.719 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.719 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.719 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.720 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.720 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T03:58:24.720 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T03:58:24.720 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.721 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.721 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.721 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.721 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.721 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.722 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.722 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T03:58:24.722 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T03:58:24.722 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.723 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:58:24.723 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T03:58:24.723 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T03:58:24.723 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.723 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.724 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.724 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T03:58:24.724 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T03:58:24.724 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.724 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:58:24.725 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T03:58:24.725 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T03:58:24.725 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.725 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.726 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.726 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T03:58:24.726 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T03:58:24.726 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.727 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:58:24.727 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:58:24.727 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:58:24.727 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.728 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.728 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.728 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.729 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T03:58:24.729 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T03:58:24.729 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.729 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:58:24.729 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T03:58:24.730 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.730 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.740 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:24.740 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:24.741 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T03:58:24.741 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T03:58:24.741 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:58:24.741 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:58:24.741 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T03:58:24.742 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:58:24.742 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.742 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:24.751 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:58:24.751 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:58:24.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 -- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f40531f0 msgr2=0x7f09f40556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f40531f0 0x7f09f40556b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f0a14061a80 tx=0x7f0a04012000).stop 2021-07-11T03:58:24.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 -- 172.21.2.104:0/2304598717 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140afbd0 msgr2=0x7f0a140aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140afbd0 0x7f0a140aecb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f0a1406b7c0 tx=0x7f0a10007ca0).stop 2021-07-11T03:58:24.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 -- 172.21.2.104:0/2304598717 shutdown_connections 2021-07-11T03:58:24.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a140b2580 0x7f0a140634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:24.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a140b0010 0x7f0a14063000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:24.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f40531f0 0x7f09f40556b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 --2- 172.21.2.104:0/2304598717 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140afbd0 0x7f0a140aecb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 -- 172.21.2.104:0/2304598717 >> 172.21.2.104:0/2304598717 conn(0x7f0a14060180 msgr2=0x7f0a140675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:24.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 -- 172.21.2.104:0/2304598717 shutdown_connections 2021-07-11T03:58:24.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.685+0000 7f0a1a088700 1 -- 172.21.2.104:0/2304598717 wait complete. 2021-07-11T03:58:24.760 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:58:24.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 Processor -- start 2021-07-11T03:58:24.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 -- start start 2021-07-11T03:58:24.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a0067180 0x7fe3a005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe3a00b1970 0x7fe3a005fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a00a7a90 con 0x7fe3a00a76b0 2021-07-11T03:58:24.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a0147620 con 0x7fe3a00b1970 2021-07-11T03:58:24.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a01478c0 con 0x7fe3a0067180 2021-07-11T03:58:24.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39ffff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe3a00a76b0 0x7fe3a0066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42170/0 (socket says 172.21.2.104:42170) 2021-07-11T03:58:24.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39ffff700 1 -- 172.21.2.104:0/2740230536 learned_addr learned my addr 172.21.2.104:0/2740230536 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:24.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152732220 0 0) 0x7fe3a00a7a90 con 0x7fe3a00a76b0 2021-07-11T03:58:24.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe38c003260 con 0x7fe3a00a76b0 2021-07-11T03:58:24.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a4da4700 1 --2- 172.21.2.104:0/2740230536 >> v2:172.21.2.105:3300/0 conn(0x7fe3a00b1970 0x7fe3a005fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39f7fe700 1 --2- 172.21.2.104:0/2740230536 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a0067180 0x7fe3a005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1758047739 0 0) 0x7fe38c003260 con 0x7fe3a00a76b0 2021-07-11T03:58:24.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a0067180 msgr2=0x7fe3a005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:24.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 --2- 172.21.2.104:0/2740230536 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a0067180 0x7fe3a005f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 >> v2:172.21.2.105:3300/0 conn(0x7fe3a00b1970 msgr2=0x7fe3a005fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:24.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 --2- 172.21.2.104:0/2740230536 >> v2:172.21.2.105:3300/0 conn(0x7fe3a00b1970 0x7fe3a005fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a0147b60 con 0x7fe3a00a76b0 2021-07-11T03:58:24.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fe390001680 con 0x7fe3a00a76b0 2021-07-11T03:58:24.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a4da4700 1 --2- 172.21.2.104:0/2740230536 >> v2:172.21.2.105:3300/0 conn(0x7fe3a00b1970 0x7fe3a005fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:58:24.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 >> v1:172.21.2.104:6789/0 conn(0x7fe3a00a76b0 legacy=0x7fe3a0066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 --2- 172.21.2.104:0/2740230536 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe38c0037d0 0x7fe38c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 --2- 172.21.2.104:0/2740230536 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a0067180 0x7fe38c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 --2- 172.21.2.104:0/2740230536 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe38c0049f0 0x7fe38c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0147620 con 0x7fe38c0049f0 2021-07-11T03:58:24.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a01478c0 con 0x7fe38c0037d0 2021-07-11T03:58:24.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe38c00b880 con 0x7fe3a0067180 2021-07-11T03:58:24.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a4da4700 1 --2- 172.21.2.104:0/2740230536 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe38c0049f0 0x7fe38c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a4da4700 1 -- 172.21.2.104:0/2740230536 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a0067180 msgr2=0x7fe38c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:24.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a4da4700 1 --2- 172.21.2.104:0/2740230536 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a0067180 0x7fe38c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:24.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a4da4700 1 -- 172.21.2.104:0/2740230536 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe38c0037d0 msgr2=0x7fe38c003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:24.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a4da4700 1 --2- 172.21.2.104:0/2740230536 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe38c0037d0 0x7fe38c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:24.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a4da4700 1 -- 172.21.2.104:0/2740230536 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c003260 con 0x7fe38c0049f0 2021-07-11T03:58:24.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a4da4700 1 --2- 172.21.2.104:0/2740230536 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe38c0049f0 0x7fe38c004e70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fe39800a000 tx=0x7fe398008040).ready entity=mon.0 client_cookie=2e21ac1510cb2fb0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:24.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39800c070 con 0x7fe38c0049f0 2021-07-11T03:58:24.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39800e5f0 con 0x7fe38c0049f0 2021-07-11T03:58:24.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe39effd700 1 -- 172.21.2.104:0/2740230536 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe39800e950 con 0x7fe38c0049f0 2021-07-11T03:58:24.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 -- 172.21.2.104:0/2740230536 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe38c0049f0 msgr2=0x7fe38c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 --2- 172.21.2.104:0/2740230536 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe38c0049f0 0x7fe38c004e70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7fe39800a000 tx=0x7fe398008040).stop 2021-07-11T03:58:24.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a4da4700 1 -- 172.21.2.104:0/2740230536 reap_dead start 2021-07-11T03:58:24.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 -- 172.21.2.104:0/2740230536 shutdown_connections 2021-07-11T03:58:24.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 -- 172.21.2.104:0/2740230536 >> 172.21.2.104:0/2740230536 conn(0x7fe3a00b53e0 msgr2=0x7fe3a00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:24.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 -- 172.21.2.104:0/2740230536 shutdown_connections 2021-07-11T03:58:24.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.825+0000 7fe3a6807700 1 -- 172.21.2.104:0/2740230536 wait complete. 2021-07-11T03:58:24.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a6807700 1 Processor -- start 2021-07-11T03:58:24.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a6807700 1 -- start start 2021-07-11T03:58:24.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a6807700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe3a00b4290 0x7fe3a00aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a6807700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3a00abaa0 0x7fe3a00adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a6807700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a00ae4c0 0x7fe3a0144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a6807700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00af7c0 con 0x7fe3a00abaa0 2021-07-11T03:58:24.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a6807700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00ac650 con 0x7fe3a00b4290 2021-07-11T03:58:24.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a6807700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00b4670 con 0x7fe3a00ae4c0 2021-07-11T03:58:24.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3a00abaa0 0x7fe3a00adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3a00abaa0 0x7fe3a00adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45416/0 (socket says 172.21.2.104:45416) 2021-07-11T03:58:24.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39f7fe700 1 -- 172.21.2.104:0/3814215789 learned_addr learned my addr 172.21.2.104:0/3814215789 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:24.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a4da4700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a00ae4c0 0x7fe3a0144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39ffff700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe3a00b4290 0x7fe3a00aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39f7fe700 1 -- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a00ae4c0 msgr2=0x7fe3a0144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39f7fe700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a00ae4c0 0x7fe3a0144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39f7fe700 1 -- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe3a00b4290 msgr2=0x7fe3a00aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39f7fe700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe3a00b4290 0x7fe3a00aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39f7fe700 1 -- 172.21.2.104:0/3814215789 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00aaae0 con 0x7fe3a00abaa0 2021-07-11T03:58:24.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39f7fe700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3a00abaa0 0x7fe3a00adfd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fe39400a000 tx=0x7fe394008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:24.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39d7fa700 1 -- 172.21.2.104:0/3814215789 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe394016070 con 0x7fe3a00abaa0 2021-07-11T03:58:24.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39d7fa700 1 -- 172.21.2.104:0/3814215789 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe394000e90 con 0x7fe3a00abaa0 2021-07-11T03:58:24.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a6807700 1 -- 172.21.2.104:0/3814215789 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a00aad80 con 0x7fe3a00abaa0 2021-07-11T03:58:24.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39d7fa700 1 -- 172.21.2.104:0/3814215789 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe394011970 con 0x7fe3a00abaa0 2021-07-11T03:58:24.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a6807700 1 -- 172.21.2.104:0/3814215789 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a014b810 con 0x7fe3a00abaa0 2021-07-11T03:58:24.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe3a6807700 1 -- 172.21.2.104:0/3814215789 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a0057350 con 0x7fe3a00abaa0 2021-07-11T03:58:24.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39d7fa700 1 -- 172.21.2.104:0/3814215789 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe394011ad0 con 0x7fe3a00abaa0 2021-07-11T03:58:24.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39d7fa700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe380053210 0x7fe3800556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39d7fa700 1 -- 172.21.2.104:0/3814215789 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe39406aff0 con 0x7fe3a00abaa0 2021-07-11T03:58:24.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.829+0000 7fe39ffff700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe380053210 0x7fe3800556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.833+0000 7fe39ffff700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe380053210 0x7fe3800556d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fe39000b000 tx=0x7fe3a00a8f20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:24.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.833+0000 7fe39d7fa700 1 -- 172.21.2.104:0/3814215789 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe394042660 con 0x7fe3a00abaa0 2021-07-11T03:58:24.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.913+0000 7fe3a6807700 1 -- 172.21.2.104:0/3814215789 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3a00698e0 con 0x7fe3a00abaa0 2021-07-11T03:58:24.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe39d7fa700 1 -- 172.21.2.104:0/3814215789 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+13182 (secure 0 0 0) 0x7fe394045c80 con 0x7fe3a00abaa0 2021-07-11T03:58:24.923 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:24.924 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":60,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:58:24.497594+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:58:24.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 -- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe380053210 msgr2=0x7fe3800556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe380053210 0x7fe3800556d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fe39000b000 tx=0x7fe3a00a8f20).stop 2021-07-11T03:58:24.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 -- 172.21.2.104:0/3814215789 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3a00abaa0 msgr2=0x7fe3a00adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3a00abaa0 0x7fe3a00adfd0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fe39400a000 tx=0x7fe394008040).stop 2021-07-11T03:58:24.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 -- 172.21.2.104:0/3814215789 shutdown_connections 2021-07-11T03:58:24.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3a00ae4c0 0x7fe3a0144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe380053210 0x7fe3800556d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3a00abaa0 0x7fe3a00adfd0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 --2- 172.21.2.104:0/3814215789 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe3a00b4290 0x7fe3a00aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 -- 172.21.2.104:0/3814215789 >> 172.21.2.104:0/3814215789 conn(0x7fe3a00b53e0 msgr2=0x7fe3a00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:24.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 -- 172.21.2.104:0/3814215789 shutdown_connections 2021-07-11T03:58:24.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.917+0000 7fe3a6807700 1 -- 172.21.2.104:0/3814215789 wait complete. 2021-07-11T03:58:24.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T03:58:24.932 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:58:24.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 Processor -- start 2021-07-11T03:58:24.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- start start 2021-07-11T03:58:24.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f69c00b12d0 0x7f69c00affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69c00b22a0 0x7f69c0141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:24.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c00a77d0 con 0x7f69c00b1ec0 2021-07-11T03:58:24.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c00b1740 con 0x7f69c00b12d0 2021-07-11T03:58:24.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c00b0650 con 0x7f69c00b22a0 2021-07-11T03:58:24.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c4d70700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f69c00b12d0 0x7f69c00affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c4d70700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f69c00b12d0 0x7f69c00affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45424/0 (socket says 172.21.2.104:45424) 2021-07-11T03:58:24.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c4d70700 1 -- 172.21.2.104:0/3505866369 learned_addr learned my addr 172.21.2.104:0/3505866369 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:24.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c5d72700 1 --2- 172.21.2.104:0/3505866369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69c00b22a0 0x7f69c0141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:24.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c4d70700 1 -- 172.21.2.104:0/3505866369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69c00b22a0 msgr2=0x7f69c0141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:24.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c4d70700 1 --2- 172.21.2.104:0/3505866369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69c00b22a0 0x7f69c0141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c4d70700 1 -- 172.21.2.104:0/3505866369 >> v1:172.21.2.105:6789/0 conn(0x7f69c00b1ec0 legacy=0x7f69c00a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:24.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c4d70700 1 -- 172.21.2.104:0/3505866369 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c00b08f0 con 0x7f69c00b12d0 2021-07-11T03:58:24.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c5d72700 1 --2- 172.21.2.104:0/3505866369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69c00b22a0 0x7f69c0141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:58:24.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c4d70700 1 --2- 172.21.2.104:0/3505866369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69c00b12d0 0x7f69c00affb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f69c00b2f50 tx=0x7f69bc006eb0).ready entity=mon.0 client_cookie=c1090166e855319f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:24.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69b3fff700 1 -- 172.21.2.104:0/3505866369 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69bc00d570 con 0x7f69c00b12d0 2021-07-11T03:58:24.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69b3fff700 1 -- 172.21.2.104:0/3505866369 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69bc000e90 con 0x7f69c00b12d0 2021-07-11T03:58:24.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69b3fff700 1 -- 172.21.2.104:0/3505866369 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69bc00d8c0 con 0x7f69c00b12d0 2021-07-11T03:58:24.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- 172.21.2.104:0/3505866369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69c00b12d0 msgr2=0x7f69c00affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:24.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 --2- 172.21.2.104:0/3505866369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69c00b12d0 0x7f69c00affb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f69c00b2f50 tx=0x7f69bc006eb0).stop 2021-07-11T03:58:24.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- 172.21.2.104:0/3505866369 shutdown_connections 2021-07-11T03:58:24.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 --2- 172.21.2.104:0/3505866369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69c00b22a0 0x7f69c0141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 --2- 172.21.2.104:0/3505866369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69c00b12d0 0x7f69c00affb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:24.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- 172.21.2.104:0/3505866369 >> 172.21.2.104:0/3505866369 conn(0x7f69c005f6b0 msgr2=0x7f69c005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:24.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- 172.21.2.104:0/3505866369 shutdown_connections 2021-07-11T03:58:24.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- 172.21.2.104:0/3505866369 wait complete. 2021-07-11T03:58:24.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 Processor -- start 2021-07-11T03:58:25.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- start start 2021-07-11T03:58:25.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69c00b12d0 0x7f69c00af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69c00b1ec0 0x7f69c013ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69c00b22a0 0x7f69c013cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c00a97d0 con 0x7f69c00b12d0 2021-07-11T03:58:25.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c00ab910 con 0x7f69c00b1ec0 2021-07-11T03:58:25.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c77d5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c00ab670 con 0x7f69c00b22a0 2021-07-11T03:58:25.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c5571700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69c00b12d0 0x7f69c00af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c5571700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69c00b12d0 0x7f69c00af490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45428/0 (socket says 172.21.2.104:45428) 2021-07-11T03:58:25.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.981+0000 7f69c5571700 1 -- 172.21.2.104:0/3790143357 learned_addr learned my addr 172.21.2.104:0/3790143357 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:25.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69c5571700 1 -- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69c00b22a0 msgr2=0x7f69c013cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69c5571700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69c00b22a0 0x7f69c013cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69c5571700 1 -- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69c00b1ec0 msgr2=0x7f69c013ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69c5571700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69c00b1ec0 0x7f69c013ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69c5571700 1 -- 172.21.2.104:0/3790143357 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69bc00b1e0 con 0x7f69c00b12d0 2021-07-11T03:58:25.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69c5571700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69c00b12d0 0x7f69c00af490 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f69c013de30 tx=0x7f69b4007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:25.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69b27fc700 1 -- 172.21.2.104:0/3790143357 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69b400ce70 con 0x7f69c00b12d0 2021-07-11T03:58:25.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69b27fc700 1 -- 172.21.2.104:0/3790143357 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b4007240 con 0x7f69c00b12d0 2021-07-11T03:58:25.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69c77d5700 1 -- 172.21.2.104:0/3790143357 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c00a9530 con 0x7f69c00b12d0 2021-07-11T03:58:25.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69c77d5700 1 -- 172.21.2.104:0/3790143357 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c0149d10 con 0x7f69c00b12d0 2021-07-11T03:58:25.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69b27fc700 1 -- 172.21.2.104:0/3790143357 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69b4006510 con 0x7f69c00b12d0 2021-07-11T03:58:25.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69c77d5700 1 -- 172.21.2.104:0/3790143357 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c013d610 con 0x7f69c00b12d0 2021-07-11T03:58:25.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69b27fc700 1 -- 172.21.2.104:0/3790143357 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f69b4010030 con 0x7f69c00b12d0 2021-07-11T03:58:25.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69b27fc700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69a00532d0 0x7f69a0055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69b27fc700 1 -- 172.21.2.104:0/3790143357 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f69b406afe0 con 0x7f69c00b12d0 2021-07-11T03:58:25.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69c4d70700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69a00532d0 0x7f69a0055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.985+0000 7f69c4d70700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69a00532d0 0x7f69a0055790 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f69c00b2f50 tx=0x7f69bc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:25.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:24.989+0000 7f69b27fc700 1 -- 172.21.2.104:0/3790143357 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f69b4042560 con 0x7f69c00b12d0 2021-07-11T03:58:25.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 -- 172.21.2.104:0/3790143357 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f69c0057350 con 0x7f69c00b12d0 2021-07-11T03:58:25.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69b27fc700 1 -- 172.21.2.104:0/3790143357 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v60) v1 ==== 110+0+10 (secure 0 0 0) 0x7f69b4045b80 con 0x7f69c00b12d0 2021-07-11T03:58:25.076 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:58:25.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 -- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69a00532d0 msgr2=0x7f69a0055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69a00532d0 0x7f69a0055790 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f69c00b2f50 tx=0x7f69bc015000).stop 2021-07-11T03:58:25.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 -- 172.21.2.104:0/3790143357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69c00b12d0 msgr2=0x7f69c00af490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69c00b12d0 0x7f69c00af490 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f69c013de30 tx=0x7f69b4007c30).stop 2021-07-11T03:58:25.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 -- 172.21.2.104:0/3790143357 shutdown_connections 2021-07-11T03:58:25.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69c00b22a0 0x7f69c013cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69c00b1ec0 0x7f69c013ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69a00532d0 0x7f69a0055790 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:25.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 --2- 172.21.2.104:0/3790143357 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69c00b12d0 0x7f69c00af490 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:25.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 -- 172.21.2.104:0/3790143357 >> 172.21.2.104:0/3790143357 conn(0x7f69c005f6b0 msgr2=0x7f69c00a70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:25.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 -- 172.21.2.104:0/3790143357 shutdown_connections 2021-07-11T03:58:25.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.069+0000 7f69c77d5700 1 -- 172.21.2.104:0/3790143357 wait complete. 2021-07-11T03:58:25.084 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:58:25.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 Processor -- start 2021-07-11T03:58:25.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 -- start start 2021-07-11T03:58:25.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f98540b1ec0 0x7f98540b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f98540b22a0 0x7f9854141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98540b0420 con 0x7f98540affb0 2021-07-11T03:58:25.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98540a77d0 con 0x7f98540b1ec0 2021-07-11T03:58:25.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98540b0650 con 0x7f98540b22a0 2021-07-11T03:58:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f98592fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f98540affb0 0x7f98540a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42208/0 (socket says 172.21.2.104:42208) 2021-07-11T03:58:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f98592fc700 1 -- 172.21.2.104:0/892002712 learned_addr learned my addr 172.21.2.104:0/892002712 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f9859afd700 1 --2- 172.21.2.104:0/892002712 >> v2:172.21.2.104:3300/0 conn(0x7f98540b1ec0 0x7f98540b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f9859afd700 1 -- 172.21.2.104:0/892002712 >> v2:172.21.2.105:3300/0 conn(0x7f98540b22a0 msgr2=0x7f9854141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f9859afd700 1 --2- 172.21.2.104:0/892002712 >> v2:172.21.2.105:3300/0 conn(0x7f98540b22a0 0x7f9854141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f9859afd700 1 -- 172.21.2.104:0/892002712 >> v1:172.21.2.104:6789/0 conn(0x7f98540affb0 legacy=0x7f98540a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f9859afd700 1 -- 172.21.2.104:0/892002712 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98540b08f0 con 0x7f98540b1ec0 2021-07-11T03:58:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f9859afd700 1 --2- 172.21.2.104:0/892002712 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98540b1ec0 0x7f98540b12d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f9848002f70 tx=0x7f9848006df0).ready entity=mon.0 client_cookie=2eb9239abebd0d62 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f9858afb700 1 -- 172.21.2.104:0/892002712 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984800d5c0 con 0x7f98540b1ec0 2021-07-11T03:58:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f9858afb700 1 -- 172.21.2.104:0/892002712 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9848000e90 con 0x7f98540b1ec0 2021-07-11T03:58:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f9858afb700 1 -- 172.21.2.104:0/892002712 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984800d910 con 0x7f98540b1ec0 2021-07-11T03:58:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 -- 172.21.2.104:0/892002712 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98540b1ec0 msgr2=0x7f98540b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 --2- 172.21.2.104:0/892002712 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98540b1ec0 0x7f98540b12d0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f9848002f70 tx=0x7f9848006df0).stop 2021-07-11T03:58:25.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 -- 172.21.2.104:0/892002712 shutdown_connections 2021-07-11T03:58:25.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 --2- 172.21.2.104:0/892002712 >> v2:172.21.2.105:3300/0 conn(0x7f98540b22a0 0x7f9854141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 --2- 172.21.2.104:0/892002712 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98540b1ec0 0x7f98540b12d0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:25.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 -- 172.21.2.104:0/892002712 >> 172.21.2.104:0/892002712 conn(0x7f985405f6b0 msgr2=0x7f985405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:25.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 -- 172.21.2.104:0/892002712 shutdown_connections 2021-07-11T03:58:25.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.129+0000 7f985bd61700 1 -- 172.21.2.104:0/892002712 wait complete. 2021-07-11T03:58:25.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985bd61700 1 Processor -- start 2021-07-11T03:58:25.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985bd61700 1 -- start start 2021-07-11T03:58:25.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985bd61700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98540affb0 0x7f98540b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985bd61700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98540b1ec0 0x7f985414a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985bd61700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98540b22a0 0x7f985414ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985bd61700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9854149760 con 0x7f98540b22a0 2021-07-11T03:58:25.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985bd61700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9854149e30 con 0x7f98540b1ec0 2021-07-11T03:58:25.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985bd61700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98540b3590 con 0x7f98540affb0 2021-07-11T03:58:25.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985a2fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98540b22a0 0x7f985414ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985a2fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98540b22a0 0x7f985414ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45442/0 (socket says 172.21.2.104:45442) 2021-07-11T03:58:25.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985a2fe700 1 -- 172.21.2.104:0/1807102495 learned_addr learned my addr 172.21.2.104:0/1807102495 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:25.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985a2fe700 1 -- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98540affb0 msgr2=0x7f98540b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985a2fe700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98540affb0 0x7f98540b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985a2fe700 1 -- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98540b1ec0 msgr2=0x7f985414a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985a2fe700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98540b1ec0 0x7f985414a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985a2fe700 1 -- 172.21.2.104:0/1807102495 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984800b180 con 0x7f98540b22a0 2021-07-11T03:58:25.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985a2fe700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98540b22a0 0x7f985414ae00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f984c0032f0 tx=0x7f984c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:25.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f9846ffd700 1 -- 172.21.2.104:0/1807102495 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984c00d4a0 con 0x7f98540b22a0 2021-07-11T03:58:25.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f9846ffd700 1 -- 172.21.2.104:0/1807102495 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c00d600 con 0x7f98540b22a0 2021-07-11T03:58:25.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f9846ffd700 1 -- 172.21.2.104:0/1807102495 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f984c00d960 con 0x7f98540b22a0 2021-07-11T03:58:25.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985bd61700 1 -- 172.21.2.104:0/1807102495 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9854145290 con 0x7f98540b22a0 2021-07-11T03:58:25.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985bd61700 1 -- 172.21.2.104:0/1807102495 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98540b5390 con 0x7f98540b22a0 2021-07-11T03:58:25.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f985bd61700 1 -- 172.21.2.104:0/1807102495 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9854057350 con 0x7f98540b22a0 2021-07-11T03:58:25.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f9846ffd700 1 -- 172.21.2.104:0/1807102495 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f984c018030 con 0x7f98540b22a0 2021-07-11T03:58:25.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f9846ffd700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9834053210 0x7f98340556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f9846ffd700 1 -- 172.21.2.104:0/1807102495 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f984c06b720 con 0x7f98540b22a0 2021-07-11T03:58:25.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.133+0000 7f9859afd700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9834053210 0x7f98340556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.137+0000 7f9859afd700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9834053210 0x7f98340556d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f9848002f70 tx=0x7f9848014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:25.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.137+0000 7f9846ffd700 1 -- 172.21.2.104:0/1807102495 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f984c042160 con 0x7f98540b22a0 2021-07-11T03:58:25.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.217+0000 7f985bd61700 1 -- 172.21.2.104:0/1807102495 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98540ac760 con 0x7f98540b22a0 2021-07-11T03:58:25.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f9846ffd700 1 -- 172.21.2.104:0/1807102495 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v60) v1 ==== 92+0+10 (secure 0 0 0) 0x7f984c045780 con 0x7f98540b22a0 2021-07-11T03:58:25.227 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:58:25.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 -- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9834053210 msgr2=0x7f98340556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9834053210 0x7f98340556d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f9848002f70 tx=0x7f9848014000).stop 2021-07-11T03:58:25.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 -- 172.21.2.104:0/1807102495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98540b22a0 msgr2=0x7f985414ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98540b22a0 0x7f985414ae00 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f984c0032f0 tx=0x7f984c0091a0).stop 2021-07-11T03:58:25.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 -- 172.21.2.104:0/1807102495 shutdown_connections 2021-07-11T03:58:25.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9834053210 0x7f98340556d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:25.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f98540b22a0 0x7f985414ae00 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:25.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f98540b1ec0 0x7f985414a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 --2- 172.21.2.104:0/1807102495 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f98540affb0 0x7f98540b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 -- 172.21.2.104:0/1807102495 >> 172.21.2.104:0/1807102495 conn(0x7f985405f6b0 msgr2=0x7f985405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:25.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 -- 172.21.2.104:0/1807102495 shutdown_connections 2021-07-11T03:58:25.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.221+0000 7f985bd61700 1 -- 172.21.2.104:0/1807102495 wait complete. 2021-07-11T03:58:25.234 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T03:58:25.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.277+0000 7f65df86c700 1 Processor -- start 2021-07-11T03:58:25.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.277+0000 7f65df86c700 1 -- start start 2021-07-11T03:58:25.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.277+0000 7f65df86c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65d80b1db0 0x7f65d80b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.277+0000 7f65df86c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f65d81483a0 0x7f65d80b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.277+0000 7f65df86c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d80b0350 con 0x7f65d80a70c0 2021-07-11T03:58:25.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.277+0000 7f65df86c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d80a90c0 con 0x7f65d81483a0 2021-07-11T03:58:25.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.277+0000 7f65df86c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80b05f0 con 0x7f65d80b1db0 2021-07-11T03:58:25.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65dce07700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f65d81483a0 0x7f65d80b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65dd608700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65d80b1db0 0x7f65d80b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65dde09700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f65d80a70c0 0x7f65d8144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:41590/0 (socket says 172.21.2.104:41590) 2021-07-11T03:58:25.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65dce07700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f65d81483a0 0x7f65d80b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:53012/0 (socket says 172.21.2.104:53012) 2021-07-11T03:58:25.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65dd608700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65d80b1db0 0x7f65d80b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:48614/0 (socket says 172.21.2.104:48614) 2021-07-11T03:58:25.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65dde09700 1 -- 172.21.2.104:0/2464705747 learned_addr learned my addr 172.21.2.104:0/2464705747 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:25.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65dce07700 1 -- 172.21.2.104:0/2464705747 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65d80b1db0 msgr2=0x7f65d80b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:25.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65dce07700 1 --2- 172.21.2.104:0/2464705747 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65d80b1db0 0x7f65d80b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:25.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65dce07700 1 -- 172.21.2.104:0/2464705747 >> v1:172.21.2.105:6789/0 conn(0x7f65d80a70c0 legacy=0x7f65d8144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65dce07700 1 -- 172.21.2.104:0/2464705747 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d80b0890 con 0x7f65d81483a0 2021-07-11T03:58:25.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65dce07700 1 --2- 172.21.2.104:0/2464705747 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65d81483a0 0x7f65d80b2190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f65c8002f70 tx=0x7f65c8006df0).ready entity=mon.1 client_cookie=6c7fdbfd4a89c5ab server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:25.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65c7fff700 1 -- 172.21.2.104:0/2464705747 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65c800d570 con 0x7f65d81483a0 2021-07-11T03:58:25.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65c7fff700 1 -- 172.21.2.104:0/2464705747 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c8000e90 con 0x7f65d81483a0 2021-07-11T03:58:25.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65c7fff700 1 -- 172.21.2.104:0/2464705747 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65c800d8c0 con 0x7f65d81483a0 2021-07-11T03:58:25.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65df86c700 1 -- 172.21.2.104:0/2464705747 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65d81483a0 msgr2=0x7f65d80b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65df86c700 1 --2- 172.21.2.104:0/2464705747 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65d81483a0 0x7f65d80b2190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f65c8002f70 tx=0x7f65c8006df0).stop 2021-07-11T03:58:25.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65df86c700 1 -- 172.21.2.104:0/2464705747 shutdown_connections 2021-07-11T03:58:25.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65df86c700 1 --2- 172.21.2.104:0/2464705747 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65d81483a0 0x7f65d80b2190 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:25.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65df86c700 1 --2- 172.21.2.104:0/2464705747 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65d80b1db0 0x7f65d80b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:25.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65df86c700 1 -- 172.21.2.104:0/2464705747 >> 172.21.2.104:0/2464705747 conn(0x7f65d805f6b0 msgr2=0x7f65d805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:25.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65df86c700 1 -- 172.21.2.104:0/2464705747 shutdown_connections 2021-07-11T03:58:25.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65df86c700 1 -- 172.21.2.104:0/2464705747 wait complete. 2021-07-11T03:58:25.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.281+0000 7f65df86c700 1 Processor -- start 2021-07-11T03:58:25.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65df86c700 1 -- start start 2021-07-11T03:58:25.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65df86c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65d80a70c0 0x7f65d8146c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65df86c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65d80b1db0 0x7f65d80acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65df86c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65d81483a0 0x7f65d80ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65df86c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d8140520 con 0x7f65d81483a0 2021-07-11T03:58:25.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65df86c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80adb80 con 0x7f65d80a70c0 2021-07-11T03:58:25.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65df86c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d80af090 con 0x7f65d80b1db0 2021-07-11T03:58:25.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65dde09700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65d81483a0 0x7f65d80ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65dde09700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65d81483a0 0x7f65d80ad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45460/0 (socket says 172.21.2.104:45460) 2021-07-11T03:58:25.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65dde09700 1 -- 172.21.2.104:0/3792942545 learned_addr learned my addr 172.21.2.104:0/3792942545 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:25.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65dde09700 1 -- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65d80b1db0 msgr2=0x7f65d80acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65dde09700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65d80b1db0 0x7f65d80acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65dde09700 1 -- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65d80a70c0 msgr2=0x7f65d8146c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65dde09700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65d80a70c0 0x7f65d8146c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65dde09700 1 -- 172.21.2.104:0/3792942545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c800b180 con 0x7f65d81483a0 2021-07-11T03:58:25.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65dde09700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65d81483a0 0x7f65d80ad390 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f65d80613a0 tx=0x7f65d0007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:25.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65c67fc700 1 -- 172.21.2.104:0/3792942545 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65d0005ba0 con 0x7f65d81483a0 2021-07-11T03:58:25.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65c67fc700 1 -- 172.21.2.104:0/3792942545 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d0005d00 con 0x7f65d81483a0 2021-07-11T03:58:25.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65df86c700 1 -- 172.21.2.104:0/3792942545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d80af330 con 0x7f65d81483a0 2021-07-11T03:58:25.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65c67fc700 1 -- 172.21.2.104:0/3792942545 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f65d00072c0 con 0x7f65d81483a0 2021-07-11T03:58:25.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65df86c700 1 -- 172.21.2.104:0/3792942545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d80ad900 con 0x7f65d81483a0 2021-07-11T03:58:25.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65df86c700 1 -- 172.21.2.104:0/3792942545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d8145df0 con 0x7f65d81483a0 2021-07-11T03:58:25.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65c67fc700 1 -- 172.21.2.104:0/3792942545 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f65d001a060 con 0x7f65d81483a0 2021-07-11T03:58:25.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65c67fc700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65b8053300 0x7f65b80557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65c67fc700 1 -- 172.21.2.104:0/3792942545 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f65d006acc0 con 0x7f65d81483a0 2021-07-11T03:58:25.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.285+0000 7f65dd608700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65b8053300 0x7f65b80557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.289+0000 7f65dd608700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65b8053300 0x7f65b80557c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f65d400c000 tx=0x7f65d4006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:25.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.289+0000 7f65c67fc700 1 -- 172.21.2.104:0/3792942545 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f65d0042330 con 0x7f65d81483a0 2021-07-11T03:58:25.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 -- 172.21.2.104:0/3792942545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f65d8057350 con 0x7f65d81483a0 2021-07-11T03:58:25.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65c67fc700 1 -- 172.21.2.104:0/3792942545 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v60) v1 ==== 84+0+143 (secure 0 0 0) 0x7f65d0018030 con 0x7f65d81483a0 2021-07-11T03:58:25.381 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:25.381 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T03:58:25.381 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:25.382 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T03:58:25.382 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T03:58:25.382 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:58:25.382 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:58:25.383 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T03:58:25.383 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T03:58:25.383 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:58:25.383 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T03:58:25.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 -- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65b8053300 msgr2=0x7f65b80557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65b8053300 0x7f65b80557c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f65d400c000 tx=0x7f65d4006df0).stop 2021-07-11T03:58:25.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 -- 172.21.2.104:0/3792942545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65d81483a0 msgr2=0x7f65d80ad390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65d81483a0 0x7f65d80ad390 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f65d80613a0 tx=0x7f65d0007f30).stop 2021-07-11T03:58:25.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 -- 172.21.2.104:0/3792942545 shutdown_connections 2021-07-11T03:58:25.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65b8053300 0x7f65b80557c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:25.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65d81483a0 0x7f65d80ad390 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:25.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65d80b1db0 0x7f65d80acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 --2- 172.21.2.104:0/3792942545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65d80a70c0 0x7f65d8146c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 -- 172.21.2.104:0/3792942545 >> 172.21.2.104:0/3792942545 conn(0x7f65d805f6b0 msgr2=0x7f65d8142bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:25.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 -- 172.21.2.104:0/3792942545 shutdown_connections 2021-07-11T03:58:25.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.373+0000 7f65df86c700 1 -- 172.21.2.104:0/3792942545 wait complete. 2021-07-11T03:58:25.391 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T03:58:25.392 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T03:58:25.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f442388d700 1 Processor -- start 2021-07-11T03:58:25.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f442388d700 1 -- start start 2021-07-11T03:58:25.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f442388d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f441c0b12d0 0x7f441c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f442388d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f441c0b22a0 0x7f441c141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f442388d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f441c0a77d0 con 0x7f441c0b1ec0 2021-07-11T03:58:25.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f442388d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441c0b1740 con 0x7f441c0b22a0 2021-07-11T03:58:25.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f442388d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f441c0b0650 con 0x7f441c0b12d0 2021-07-11T03:58:25.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f4421629700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f441c0b1ec0 0x7f441c0a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42232/0 (socket says 172.21.2.104:42232) 2021-07-11T03:58:25.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f4421629700 1 -- 172.21.2.104:0/4270652648 learned_addr learned my addr 172.21.2.104:0/4270652648 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:25.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f4421e2a700 1 --2- 172.21.2.104:0/4270652648 >> v2:172.21.2.104:3300/0 conn(0x7f441c0b22a0 0x7f441c141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f4421e2a700 1 -- 172.21.2.104:0/4270652648 >> v2:172.21.2.105:3300/0 conn(0x7f441c0b12d0 msgr2=0x7f441c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:25.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f4421e2a700 1 --2- 172.21.2.104:0/4270652648 >> v2:172.21.2.105:3300/0 conn(0x7f441c0b12d0 0x7f441c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f4421e2a700 1 -- 172.21.2.104:0/4270652648 >> v1:172.21.2.104:6789/0 conn(0x7f441c0b1ec0 legacy=0x7f441c0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f4421e2a700 1 -- 172.21.2.104:0/4270652648 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c0b08f0 con 0x7f441c0b22a0 2021-07-11T03:58:25.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f4421e2a700 1 --2- 172.21.2.104:0/4270652648 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f441c0b22a0 0x7f441c141250 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f44140032f0 tx=0x7f44140091d0).ready entity=mon.0 client_cookie=ce4653b19d2c2ee server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:25.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f440ffff700 1 -- 172.21.2.104:0/4270652648 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4414009ea0 con 0x7f441c0b22a0 2021-07-11T03:58:25.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f440ffff700 1 -- 172.21.2.104:0/4270652648 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441400b870 con 0x7f441c0b22a0 2021-07-11T03:58:25.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f440ffff700 1 -- 172.21.2.104:0/4270652648 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f441400b9e0 con 0x7f441c0b22a0 2021-07-11T03:58:25.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f442388d700 1 -- 172.21.2.104:0/4270652648 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f441c0b22a0 msgr2=0x7f441c141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.437+0000 7f442388d700 1 --2- 172.21.2.104:0/4270652648 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f441c0b22a0 0x7f441c141250 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f44140032f0 tx=0x7f44140091d0).stop 2021-07-11T03:58:25.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 -- 172.21.2.104:0/4270652648 shutdown_connections 2021-07-11T03:58:25.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 --2- 172.21.2.104:0/4270652648 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f441c0b22a0 0x7f441c141250 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:25.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 --2- 172.21.2.104:0/4270652648 >> v2:172.21.2.105:3300/0 conn(0x7f441c0b12d0 0x7f441c0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 -- 172.21.2.104:0/4270652648 >> 172.21.2.104:0/4270652648 conn(0x7f441c05f6b0 msgr2=0x7f441c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:25.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 -- 172.21.2.104:0/4270652648 shutdown_connections 2021-07-11T03:58:25.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 -- 172.21.2.104:0/4270652648 wait complete. 2021-07-11T03:58:25.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 Processor -- start 2021-07-11T03:58:25.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 -- start start 2021-07-11T03:58:25.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f441c0b12d0 0x7f441c147fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f441c0b1ec0 0x7f441c0a9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f441c0b22a0 0x7f441c0a9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c0a9e00 con 0x7f441c0b1ec0 2021-07-11T03:58:25.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f441c156d20 con 0x7f441c0b12d0 2021-07-11T03:58:25.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f441c156fc0 con 0x7f441c0b22a0 2021-07-11T03:58:25.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f4420e28700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f441c0b1ec0 0x7f441c0a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f4420e28700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f441c0b1ec0 0x7f441c0a9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45474/0 (socket says 172.21.2.104:45474) 2021-07-11T03:58:25.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f4420e28700 1 -- 172.21.2.104:0/3883285798 learned_addr learned my addr 172.21.2.104:0/3883285798 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:25.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f4420e28700 1 -- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f441c0b22a0 msgr2=0x7f441c0a9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f4420e28700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f441c0b22a0 0x7f441c0a9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f4420e28700 1 -- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f441c0b12d0 msgr2=0x7f441c147fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:25.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f4420e28700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f441c0b12d0 0x7f441c147fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:25.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f4420e28700 1 -- 172.21.2.104:0/3883285798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414009470 con 0x7f441c0b1ec0 2021-07-11T03:58:25.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f4420e28700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f441c0b1ec0 0x7f441c0a9420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f4418002f70 tx=0x7f4418006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:25.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f440e7fc700 1 -- 172.21.2.104:0/3883285798 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4418013460 con 0x7f441c0b1ec0 2021-07-11T03:58:25.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f440e7fc700 1 -- 172.21.2.104:0/3883285798 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44180135c0 con 0x7f441c0b1ec0 2021-07-11T03:58:25.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f440e7fc700 1 -- 172.21.2.104:0/3883285798 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4418013920 con 0x7f441c0b1ec0 2021-07-11T03:58:25.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 -- 172.21.2.104:0/3883285798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f441c157260 con 0x7f441c0b1ec0 2021-07-11T03:58:25.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 -- 172.21.2.104:0/3883285798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441c0b4f60 con 0x7f441c0b1ec0 2021-07-11T03:58:25.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f442388d700 1 -- 172.21.2.104:0/3883285798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f441c057350 con 0x7f441c0b1ec0 2021-07-11T03:58:25.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f440e7fc700 1 -- 172.21.2.104:0/3883285798 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f441801a060 con 0x7f441c0b1ec0 2021-07-11T03:58:25.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f440e7fc700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43fc0531d0 0x7f43fc055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:25.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f440e7fc700 1 -- 172.21.2.104:0/3883285798 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f441806acf0 con 0x7f441c0b1ec0 2021-07-11T03:58:25.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.441+0000 7f4421629700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43fc0531d0 0x7f43fc055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:25.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.445+0000 7f4421629700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43fc0531d0 0x7f43fc055690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f441c0ad2d0 tx=0x7f44100077d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:25.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.445+0000 7f440e7fc700 1 -- 172.21.2.104:0/3883285798 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4418042250 con 0x7f441c0b1ec0 2021-07-11T03:58:25.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:25.525+0000 7f442388d700 1 -- 172.21.2.104:0/3883285798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f441c0ac760 con 0x7f441c0b1ec0 2021-07-11T03:58:26.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f440e7fc700 1 -- 172.21.2.104:0/3883285798 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v61) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4418018030 con 0x7f441c0b1ec0 2021-07-11T03:58:26.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 -- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43fc0531d0 msgr2=0x7f43fc055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:26.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43fc0531d0 0x7f43fc055690 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f441c0ad2d0 tx=0x7f44100077d0).stop 2021-07-11T03:58:26.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 -- 172.21.2.104:0/3883285798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f441c0b1ec0 msgr2=0x7f441c0a9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:26.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f441c0b1ec0 0x7f441c0a9420 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f4418002f70 tx=0x7f4418006ec0).stop 2021-07-11T03:58:26.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 -- 172.21.2.104:0/3883285798 shutdown_connections 2021-07-11T03:58:26.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f441c0b22a0 0x7f441c0a9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:26.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43fc0531d0 0x7f43fc055690 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:26.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f441c0b1ec0 0x7f441c0a9420 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:26.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 --2- 172.21.2.104:0/3883285798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f441c0b12d0 0x7f441c147fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:26.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 -- 172.21.2.104:0/3883285798 >> 172.21.2.104:0/3883285798 conn(0x7f441c05f6b0 msgr2=0x7f441c146b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:26.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 -- 172.21.2.104:0/3883285798 shutdown_connections 2021-07-11T03:58:26.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.357+0000 7f442388d700 1 -- 172.21.2.104:0/3883285798 wait complete. 2021-07-11T03:58:26.368 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T03:58:26.372 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T03:58:26.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.437+0000 7fd88753c700 1 Processor -- start 2021-07-11T03:58:26.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.437+0000 7fd88753c700 1 -- start start 2021-07-11T03:58:26.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.437+0000 7fd88753c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd8800a9ec0 0x7fd8800aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:26.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.437+0000 7fd88753c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd88013f790 0x7fd88013fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:26.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.437+0000 7fd88753c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd880061990 con 0x7fd8800603d0 2021-07-11T03:58:26.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.437+0000 7fd88753c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8801400f0 con 0x7fd8800a9ec0 2021-07-11T03:58:26.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.437+0000 7fd88753c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd880061c30 con 0x7fd88013f790 2021-07-11T03:58:26.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.437+0000 7fd884ad7700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd8800603d0 0x7fd88013ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42248/0 (socket says 172.21.2.104:42248) 2021-07-11T03:58:26.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.437+0000 7fd884ad7700 1 -- 172.21.2.104:0/3088284224 learned_addr learned my addr 172.21.2.104:0/3088284224 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:26.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.437+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245400456 0 0) 0x7fd880061990 con 0x7fd8800603d0 2021-07-11T03:58:26.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.437+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd868003260 con 0x7fd8800603d0 2021-07-11T03:58:26.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2426771745 0 0) 0x7fd868003260 con 0x7fd8800603d0 2021-07-11T03:58:26.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd88013f790 msgr2=0x7fd88013fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:26.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 --2- 172.21.2.104:0/3088284224 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd88013f790 0x7fd88013fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:26.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 >> v2:172.21.2.105:3300/0 conn(0x7fd8800a9ec0 msgr2=0x7fd8800aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:26.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 --2- 172.21.2.104:0/3088284224 >> v2:172.21.2.105:3300/0 conn(0x7fd8800a9ec0 0x7fd8800aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:26.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd880060b90 con 0x7fd8800603d0 2021-07-11T03:58:26.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fd87c001660 con 0x7fd8800603d0 2021-07-11T03:58:26.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 >> v1:172.21.2.104:6789/0 conn(0x7fd8800603d0 legacy=0x7fd88013ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:26.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 --2- 172.21.2.104:0/3088284224 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd88013f790 0x7fd868003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:26.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 --2- 172.21.2.104:0/3088284224 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd868003e10 0x7fd868004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:26.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 --2- 172.21.2.104:0/3088284224 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd868004b10 0x7fd868004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:26.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8801400f0 con 0x7fd868004b10 2021-07-11T03:58:26.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd880061c30 con 0x7fd868003e10 2021-07-11T03:58:26.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86800caf0 con 0x7fd88013f790 2021-07-11T03:58:26.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd885ad9700 1 --2- 172.21.2.104:0/3088284224 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd868004b10 0x7fd868004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:26.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd8852d8700 1 --2- 172.21.2.104:0/3088284224 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd88013f790 0x7fd868003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:26.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd885ad9700 1 -- 172.21.2.104:0/3088284224 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd88013f790 msgr2=0x7fd868003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:26.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd885ad9700 1 --2- 172.21.2.104:0/3088284224 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd88013f790 0x7fd868003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:26.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd885ad9700 1 -- 172.21.2.104:0/3088284224 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd868003e10 msgr2=0x7fd868004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:26.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd885ad9700 1 --2- 172.21.2.104:0/3088284224 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd868003e10 0x7fd868004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:26.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd885ad9700 1 -- 172.21.2.104:0/3088284224 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868003260 con 0x7fd868004b10 2021-07-11T03:58:26.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd885ad9700 1 --2- 172.21.2.104:0/3088284224 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd868004b10 0x7fd868004f90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fd8780032f0 tx=0x7fd878006f90).ready entity=mon.0 client_cookie=26f7073e2c1d7d30 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:26.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd87800b460 con 0x7fd868004b10 2021-07-11T03:58:26.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87800b5c0 con 0x7fd868004b10 2021-07-11T03:58:26.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd873fff700 1 -- 172.21.2.104:0/3088284224 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd87800b920 con 0x7fd868004b10 2021-07-11T03:58:26.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 -- 172.21.2.104:0/3088284224 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd868004b10 msgr2=0x7fd868004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:26.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 --2- 172.21.2.104:0/3088284224 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd868004b10 0x7fd868004f90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7fd8780032f0 tx=0x7fd878006f90).stop 2021-07-11T03:58:26.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd885ad9700 1 -- 172.21.2.104:0/3088284224 reap_dead start 2021-07-11T03:58:26.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 -- 172.21.2.104:0/3088284224 shutdown_connections 2021-07-11T03:58:26.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 -- 172.21.2.104:0/3088284224 >> 172.21.2.104:0/3088284224 conn(0x7fd8800b2970 msgr2=0x7fd8800b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:26.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 -- 172.21.2.104:0/3088284224 shutdown_connections 2021-07-11T03:58:26.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 -- 172.21.2.104:0/3088284224 wait complete. 2021-07-11T03:58:26.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 Processor -- start 2021-07-11T03:58:26.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 -- start start 2021-07-11T03:58:26.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd88013e800 0x7fd8800640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:26.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd880063610 0x7fd8800645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:26.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd880064ac0 0x7fd88013b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:26.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88013ca20 con 0x7fd880063610 2021-07-11T03:58:26.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88013ebe0 con 0x7fd880064ac0 2021-07-11T03:58:26.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8800af720 con 0x7fd88013e800 2021-07-11T03:58:26.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd884ad7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd880063610 0x7fd8800645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:26.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd884ad7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd880063610 0x7fd8800645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45492/0 (socket says 172.21.2.104:45492) 2021-07-11T03:58:26.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd884ad7700 1 -- 172.21.2.104:0/2181141186 learned_addr learned my addr 172.21.2.104:0/2181141186 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:26.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd884ad7700 1 -- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd88013e800 msgr2=0x7fd8800640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:26.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd884ad7700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd88013e800 0x7fd8800640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:26.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd884ad7700 1 -- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd880064ac0 msgr2=0x7fd88013b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:26.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd884ad7700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd880064ac0 0x7fd88013b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:26.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd884ad7700 1 -- 172.21.2.104:0/2181141186 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd880066660 con 0x7fd880063610 2021-07-11T03:58:26.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd884ad7700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd880063610 0x7fd8800645d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fd87c001370 tx=0x7fd880060d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:26.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd8727fc700 1 -- 172.21.2.104:0/2181141186 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd87c007170 con 0x7fd880063610 2021-07-11T03:58:26.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd8727fc700 1 -- 172.21.2.104:0/2181141186 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c0072d0 con 0x7fd880063610 2021-07-11T03:58:26.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 -- 172.21.2.104:0/2181141186 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd880066900 con 0x7fd880063610 2021-07-11T03:58:26.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd8727fc700 1 -- 172.21.2.104:0/2181141186 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd87c0049e0 con 0x7fd880063610 2021-07-11T03:58:26.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.441+0000 7fd88753c700 1 -- 172.21.2.104:0/2181141186 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8800662e0 con 0x7fd880063610 2021-07-11T03:58:26.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.445+0000 7fd88753c700 1 -- 172.21.2.104:0/2181141186 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd880066da0 con 0x7fd880063610 2021-07-11T03:58:26.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.445+0000 7fd8727fc700 1 -- 172.21.2.104:0/2181141186 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd87c010070 con 0x7fd880063610 2021-07-11T03:58:26.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.445+0000 7fd8727fc700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd8600532b0 0x7fd860055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:26.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.445+0000 7fd8727fc700 1 -- 172.21.2.104:0/2181141186 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5603+0+0 (secure 0 0 0) 0x7fd87c06b170 con 0x7fd880063610 2021-07-11T03:58:26.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.445+0000 7fd8852d8700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd8600532b0 0x7fd860055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:26.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.445+0000 7fd8852d8700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd8600532b0 0x7fd860055770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fd87400a000 tx=0x7fd874008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:26.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.445+0000 7fd8727fc700 1 -- 172.21.2.104:0/2181141186 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd87c0425e0 con 0x7fd880063610 2021-07-11T03:58:26.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:26.529+0000 7fd88753c700 1 -- 172.21.2.104:0/2181141186 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fd880057350 con 0x7fd880063610 2021-07-11T03:58:27.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.369+0000 7fd8727fc700 1 -- 172.21.2.104:0/2181141186 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v62) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd87c045c00 con 0x7fd880063610 2021-07-11T03:58:27.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.369+0000 7fd88753c700 1 -- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd8600532b0 msgr2=0x7fd860055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.369+0000 7fd88753c700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd8600532b0 0x7fd860055770 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fd87400a000 tx=0x7fd874008040).stop 2021-07-11T03:58:27.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.369+0000 7fd88753c700 1 -- 172.21.2.104:0/2181141186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd880063610 msgr2=0x7fd8800645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.369+0000 7fd88753c700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd880063610 0x7fd8800645d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fd87c001370 tx=0x7fd880060d20).stop 2021-07-11T03:58:27.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.373+0000 7fd88753c700 1 -- 172.21.2.104:0/2181141186 shutdown_connections 2021-07-11T03:58:27.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.373+0000 7fd88753c700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd880064ac0 0x7fd88013b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.373+0000 7fd88753c700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd8600532b0 0x7fd860055770 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.373+0000 7fd88753c700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd880063610 0x7fd8800645d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.373+0000 7fd88753c700 1 --2- 172.21.2.104:0/2181141186 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd88013e800 0x7fd8800640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.373+0000 7fd88753c700 1 -- 172.21.2.104:0/2181141186 >> 172.21.2.104:0/2181141186 conn(0x7fd8800b2970 msgr2=0x7fd8800b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:27.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.373+0000 7fd88753c700 1 -- 172.21.2.104:0/2181141186 shutdown_connections 2021-07-11T03:58:27.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.373+0000 7fd88753c700 1 -- 172.21.2.104:0/2181141186 wait complete. 2021-07-11T03:58:27.382 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T03:58:27.386 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T03:58:27.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.429+0000 7f3fe2a6f700 1 Processor -- start 2021-07-11T03:58:27.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- start start 2021-07-11T03:58:27.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3fdc067180 0x7f3fdc05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3fdc0b1970 0x7f3fdc05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fdc0a7a90 con 0x7f3fdc0a76b0 2021-07-11T03:58:27.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fdc147620 con 0x7f3fdc0b1970 2021-07-11T03:58:27.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc1478c0 con 0x7f3fdc067180 2021-07-11T03:58:27.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe100c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3fdc0b1970 0x7f3fdc05fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe100c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3fdc0b1970 0x7f3fdc05fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45502/0 (socket says 172.21.2.104:45502) 2021-07-11T03:58:27.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe100c700 1 -- 172.21.2.104:0/2913027158 learned_addr learned my addr 172.21.2.104:0/2913027158 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:27.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe100c700 1 -- 172.21.2.104:0/2913027158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3fdc067180 msgr2=0x7f3fdc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:27.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe100c700 1 --2- 172.21.2.104:0/2913027158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3fdc067180 0x7f3fdc05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe100c700 1 -- 172.21.2.104:0/2913027158 >> v1:172.21.2.105:6789/0 conn(0x7f3fdc0a76b0 legacy=0x7f3fdc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:27.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe100c700 1 -- 172.21.2.104:0/2913027158 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc147b60 con 0x7f3fdc0b1970 2021-07-11T03:58:27.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe100c700 1 --2- 172.21.2.104:0/2913027158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3fdc0b1970 0x7f3fdc05fba0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f3fdc148420 tx=0x7f3fd4009290).ready entity=mon.0 client_cookie=3d4672dec326139d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fdb7fe700 1 -- 172.21.2.104:0/2913027158 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fd400b830 con 0x7f3fdc0b1970 2021-07-11T03:58:27.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fdb7fe700 1 -- 172.21.2.104:0/2913027158 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd400b990 con 0x7f3fdc0b1970 2021-07-11T03:58:27.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fdb7fe700 1 -- 172.21.2.104:0/2913027158 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fd400bb00 con 0x7f3fdc0b1970 2021-07-11T03:58:27.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/2913027158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3fdc0b1970 msgr2=0x7f3fdc05fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 --2- 172.21.2.104:0/2913027158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3fdc0b1970 0x7f3fdc05fba0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f3fdc148420 tx=0x7f3fd4009290).stop 2021-07-11T03:58:27.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/2913027158 shutdown_connections 2021-07-11T03:58:27.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 --2- 172.21.2.104:0/2913027158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3fdc0b1970 0x7f3fdc05fba0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 --2- 172.21.2.104:0/2913027158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3fdc067180 0x7f3fdc05f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/2913027158 >> 172.21.2.104:0/2913027158 conn(0x7f3fdc0b53e0 msgr2=0x7f3fdc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:27.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/2913027158 shutdown_connections 2021-07-11T03:58:27.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/2913027158 wait complete. 2021-07-11T03:58:27.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 Processor -- start 2021-07-11T03:58:27.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- start start 2021-07-11T03:58:27.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3fdc067180 0x7f3fdc0adcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3fdc0a76b0 0x7f3fdc144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3fdc0b1970 0x7f3fdc144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0ad170 con 0x7f3fdc0b1970 2021-07-11T03:58:27.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0a9da0 con 0x7f3fdc067180 2021-07-11T03:58:27.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe2a6f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc0aa000 con 0x7f3fdc0a76b0 2021-07-11T03:58:27.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe100c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3fdc0b1970 0x7f3fdc144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe100c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3fdc0b1970 0x7f3fdc144af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45504/0 (socket says 172.21.2.104:45504) 2021-07-11T03:58:27.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.433+0000 7f3fe100c700 1 -- 172.21.2.104:0/1658418565 learned_addr learned my addr 172.21.2.104:0/1658418565 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:27.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fe100c700 1 -- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3fdc0a76b0 msgr2=0x7f3fdc144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fe100c700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3fdc0a76b0 0x7f3fdc144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fe100c700 1 -- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3fdc067180 msgr2=0x7f3fdc0adcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fe100c700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3fdc067180 0x7f3fdc0adcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fe100c700 1 -- 172.21.2.104:0/1658418565 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4009530 con 0x7f3fdc0b1970 2021-07-11T03:58:27.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fe100c700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3fdc0b1970 0x7f3fdc144af0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f3fdc148420 tx=0x7f3fd4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fd9ffb700 1 -- 172.21.2.104:0/1658418565 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fd4011ca0 con 0x7f3fdc0b1970 2021-07-11T03:58:27.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fd9ffb700 1 -- 172.21.2.104:0/1658418565 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4011e00 con 0x7f3fdc0b1970 2021-07-11T03:58:27.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fd9ffb700 1 -- 172.21.2.104:0/1658418565 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3fd4020620 con 0x7f3fdc0b1970 2021-07-11T03:58:27.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/1658418565 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc0aa370 con 0x7f3fdc0b1970 2021-07-11T03:58:27.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/1658418565 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc144fe0 con 0x7f3fdc0b1970 2021-07-11T03:58:27.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/1658418565 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc057350 con 0x7f3fdc0b1970 2021-07-11T03:58:27.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fd9ffb700 1 -- 172.21.2.104:0/1658418565 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3fd4026020 con 0x7f3fdc0b1970 2021-07-11T03:58:27.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fd9ffb700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3fc0053260 0x7f3fc0055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fd9ffb700 1 -- 172.21.2.104:0/1658418565 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f3fd406c290 con 0x7f3fdc0b1970 2021-07-11T03:58:27.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fe080b700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3fc0053260 0x7f3fc0055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.437+0000 7f3fe080b700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3fc0053260 0x7f3fc0055720 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f3fcc001fa0 tx=0x7f3fcc007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.441+0000 7f3fd9ffb700 1 -- 172.21.2.104:0/1658418565 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3fd4043570 con 0x7f3fdc0b1970 2021-07-11T03:58:27.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.549+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/1658418565 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f3fdc0698e0 con 0x7f3fdc0b1970 2021-07-11T03:58:27.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.553+0000 7f3fd9ffb700 1 -- 172.21.2.104:0/1658418565 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 11 and data pool 12 v26) v1 ==== 189+0+0 (secure 0 0 0) 0x7f3fd40032d0 con 0x7f3fdc0b1970 2021-07-11T03:58:27.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3fc0053260 msgr2=0x7f3fc0055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3fc0053260 0x7f3fc0055720 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f3fcc001fa0 tx=0x7f3fcc007810).stop 2021-07-11T03:58:27.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/1658418565 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3fdc0b1970 msgr2=0x7f3fdc144af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3fdc0b1970 0x7f3fdc144af0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f3fdc148420 tx=0x7f3fd4016000).stop 2021-07-11T03:58:27.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/1658418565 shutdown_connections 2021-07-11T03:58:27.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3fc0053260 0x7f3fc0055720 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3fdc0b1970 0x7f3fdc144af0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3fdc0a76b0 0x7f3fdc144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 --2- 172.21.2.104:0/1658418565 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3fdc067180 0x7f3fdc0adcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/1658418565 >> 172.21.2.104:0/1658418565 conn(0x7f3fdc0b53e0 msgr2=0x7f3fdc05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:27.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/1658418565 shutdown_connections 2021-07-11T03:58:27.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.557+0000 7f3fe2a6f700 1 -- 172.21.2.104:0/1658418565 wait complete. 2021-07-11T03:58:27.581 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 11 and data pool 12 2021-07-11T03:58:27.583 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:58:27.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 Processor -- start 2021-07-11T03:58:27.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 -- start start 2021-07-11T03:58:27.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f558c0b2660 0x7f558c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f558c096cc0 0x7f558c097130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f558c13f560 0x7f558c13f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558c097620 con 0x7f558c0b2660 2021-07-11T03:58:27.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f558c0b2a40 con 0x7f558c096cc0 2021-07-11T03:58:27.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558c097b80 con 0x7f558c13f560 2021-07-11T03:58:27.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5591015700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f558c0b2660 0x7f558c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5591015700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f558c0b2660 0x7f558c05f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45528/0 (socket says 172.21.2.104:45528) 2021-07-11T03:58:27.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5591015700 1 -- 172.21.2.104:0/4087143065 learned_addr learned my addr 172.21.2.104:0/4087143065 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:27.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5591015700 1 -- 172.21.2.104:0/4087143065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f558c13f560 msgr2=0x7f558c13f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:27.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5591015700 1 --2- 172.21.2.104:0/4087143065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f558c13f560 0x7f558c13f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5591015700 1 -- 172.21.2.104:0/4087143065 >> v2:172.21.2.105:3300/0 conn(0x7f558c096cc0 msgr2=0x7f558c097130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:27.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5591015700 1 --2- 172.21.2.104:0/4087143065 >> v2:172.21.2.105:3300/0 conn(0x7f558c096cc0 0x7f558c097130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5591015700 1 -- 172.21.2.104:0/4087143065 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c097e20 con 0x7f558c0b2660 2021-07-11T03:58:27.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5591015700 1 --2- 172.21.2.104:0/4087143065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f558c0b2660 0x7f558c05f6b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f557c002f70 tx=0x7f557c006e40).ready entity=mon.0 client_cookie=795863534e62355a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f557bfff700 1 -- 172.21.2.104:0/4087143065 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f557c00bcf0 con 0x7f558c0b2660 2021-07-11T03:58:27.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f557bfff700 1 -- 172.21.2.104:0/4087143065 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c000e90 con 0x7f558c0b2660 2021-07-11T03:58:27.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f557bfff700 1 -- 172.21.2.104:0/4087143065 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f557c0155e0 con 0x7f558c0b2660 2021-07-11T03:58:27.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 -- 172.21.2.104:0/4087143065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f558c0b2660 msgr2=0x7f558c05f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 --2- 172.21.2.104:0/4087143065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f558c0b2660 0x7f558c05f6b0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f557c002f70 tx=0x7f557c006e40).stop 2021-07-11T03:58:27.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 -- 172.21.2.104:0/4087143065 shutdown_connections 2021-07-11T03:58:27.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 --2- 172.21.2.104:0/4087143065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f558c13f560 0x7f558c13f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 --2- 172.21.2.104:0/4087143065 >> v2:172.21.2.105:3300/0 conn(0x7f558c096cc0 0x7f558c097130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 --2- 172.21.2.104:0/4087143065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f558c0b2660 0x7f558c05f6b0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 -- 172.21.2.104:0/4087143065 >> 172.21.2.104:0/4087143065 conn(0x7f558c062f10 msgr2=0x7f558c0632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:27.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 -- 172.21.2.104:0/4087143065 shutdown_connections 2021-07-11T03:58:27.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 -- 172.21.2.104:0/4087143065 wait complete. 2021-07-11T03:58:27.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 Processor -- start 2021-07-11T03:58:27.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.629+0000 7f5593279700 1 -- start start 2021-07-11T03:58:27.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5593279700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f558c096cc0 0x7f558c13e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5593279700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f558c0b2660 0x7f558c13e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5593279700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f558c13f560 0x7f558c0643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5593279700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0acb10 con 0x7f558c13f560 2021-07-11T03:58:27.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5593279700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f558c0950a0 con 0x7f558c0b2660 2021-07-11T03:58:27.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5593279700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558c098c10 con 0x7f558c096cc0 2021-07-11T03:58:27.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5591816700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f558c13f560 0x7f558c0643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5591816700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f558c13f560 0x7f558c0643d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45534/0 (socket says 172.21.2.104:45534) 2021-07-11T03:58:27.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5591816700 1 -- 172.21.2.104:0/3095980050 learned_addr learned my addr 172.21.2.104:0/3095980050 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:27.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5591816700 1 -- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f558c096cc0 msgr2=0x7f558c13e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5591816700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f558c096cc0 0x7f558c13e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5591816700 1 -- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f558c0b2660 msgr2=0x7f558c13e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5591816700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f558c0b2660 0x7f558c13e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5591816700 1 -- 172.21.2.104:0/3095980050 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557c00d4d0 con 0x7f558c13f560 2021-07-11T03:58:27.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5591816700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f558c13f560 0x7f558c0643d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f55840032f0 tx=0x7f5584006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f557a7fc700 1 -- 172.21.2.104:0/3095980050 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f558400f510 con 0x7f558c13f560 2021-07-11T03:58:27.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f557a7fc700 1 -- 172.21.2.104:0/3095980050 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558400f670 con 0x7f558c13f560 2021-07-11T03:58:27.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f557a7fc700 1 -- 172.21.2.104:0/3095980050 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f558400f7e0 con 0x7f558c13f560 2021-07-11T03:58:27.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5593279700 1 -- 172.21.2.104:0/3095980050 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558c0952d0 con 0x7f558c13f560 2021-07-11T03:58:27.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5593279700 1 -- 172.21.2.104:0/3095980050 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558c09ac20 con 0x7f558c13f560 2021-07-11T03:58:27.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5593279700 1 -- 172.21.2.104:0/3095980050 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558c099040 con 0x7f558c13f560 2021-07-11T03:58:27.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f557a7fc700 1 -- 172.21.2.104:0/3095980050 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f558400f070 con 0x7f558c13f560 2021-07-11T03:58:27.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f557a7fc700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f556c057600 0x7f556c059ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f557a7fc700 1 -- 172.21.2.104:0/3095980050 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f558406b040 con 0x7f558c13f560 2021-07-11T03:58:27.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5591015700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f556c057600 0x7f556c059ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.633+0000 7f5591015700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f556c057600 0x7f556c059ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f557c006fc0 tx=0x7f557c018000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.637+0000 7f557a7fc700 1 -- 172.21.2.104:0/3095980050 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5584042430 con 0x7f558c13f560 2021-07-11T03:58:27.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.717+0000 7f5593279700 1 -- 172.21.2.104:0/3095980050 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f558c057350 con 0x7f558c13f560 2021-07-11T03:58:27.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f557a7fc700 1 -- 172.21.2.104:0/3095980050 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16054 (secure 0 0 0) 0x7f5584045a50 con 0x7f558c13f560 2021-07-11T03:58:27.729 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:27.730 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":65,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:58:27.556258+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:58:25.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-07-11T03:58:26.545938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:58:27.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 -- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f556c057600 msgr2=0x7f556c059ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f556c057600 0x7f556c059ac0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f557c006fc0 tx=0x7f557c018000).stop 2021-07-11T03:58:27.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 -- 172.21.2.104:0/3095980050 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f558c13f560 msgr2=0x7f558c0643d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f558c13f560 0x7f558c0643d0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f55840032f0 tx=0x7f5584006f90).stop 2021-07-11T03:58:27.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 -- 172.21.2.104:0/3095980050 shutdown_connections 2021-07-11T03:58:27.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f556c057600 0x7f556c059ac0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f558c13f560 0x7f558c0643d0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f558c0b2660 0x7f558c13e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 --2- 172.21.2.104:0/3095980050 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f558c096cc0 0x7f558c13e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 -- 172.21.2.104:0/3095980050 >> 172.21.2.104:0/3095980050 conn(0x7f558c062f10 msgr2=0x7f558c0b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:27.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 -- 172.21.2.104:0/3095980050 shutdown_connections 2021-07-11T03:58:27.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.721+0000 7f5593279700 1 -- 172.21.2.104:0/3095980050 wait complete. 2021-07-11T03:58:27.735 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:58:27.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 Processor -- start 2021-07-11T03:58:27.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 -- start start 2021-07-11T03:58:27.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82000a76b0 0x7f8200066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82000a7a90 con 0x7f82000b1970 2021-07-11T03:58:27.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8200147620 con 0x7f82000b0060 2021-07-11T03:58:27.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82001478c0 con 0x7f82000a76b0 2021-07-11T03:58:27.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f82056f3700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f82000b1970 0x7f820005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42312/0 (socket says 172.21.2.104:42312) 2021-07-11T03:58:27.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f82056f3700 1 -- 172.21.2.104:0/3211342545 learned_addr learned my addr 172.21.2.104:0/3211342545 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:27.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2453227538 0 0) 0x7f82000a7a90 con 0x7f82000b1970 2021-07-11T03:58:27.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81e8003260 con 0x7f82000b1970 2021-07-11T03:58:27.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8205ef4700 1 --2- 172.21.2.104:0/3211342545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82000a76b0 0x7f8200066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851650341 0 0) 0x7f81e8003260 con 0x7f82000b1970 2021-07-11T03:58:27.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82000a76b0 msgr2=0x7f8200066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:27.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 --2- 172.21.2.104:0/3211342545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82000a76b0 0x7f8200066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 >> v1:172.21.2.105:6789/0 conn(0x7f82000b0060 legacy=0x7f8200151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:27.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8200147b60 con 0x7f82000b1970 2021-07-11T03:58:27.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f81f4001660 con 0x7f82000b1970 2021-07-11T03:58:27.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 >> v1:172.21.2.104:6789/0 conn(0x7f82000b1970 legacy=0x7f820005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 --2- 172.21.2.104:0/3211342545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81e8003800 0x7f81e8003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 --2- 172.21.2.104:0/3211342545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81e8004500 0x7f81e8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 --2- 172.21.2.104:0/3211342545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82000a76b0 0x7f81e8004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82001478c0 con 0x7f81e8004500 2021-07-11T03:58:27.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e800b910 con 0x7f81e8003800 2021-07-11T03:58:27.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e800bad0 con 0x7f82000a76b0 2021-07-11T03:58:27.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f82056f3700 1 --2- 172.21.2.104:0/3211342545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81e8004500 0x7f81e8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f82056f3700 1 -- 172.21.2.104:0/3211342545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82000a76b0 msgr2=0x7f81e8004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:27.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f82056f3700 1 --2- 172.21.2.104:0/3211342545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82000a76b0 0x7f81e8004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f82056f3700 1 -- 172.21.2.104:0/3211342545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81e8003800 msgr2=0x7f81e8003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:27.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f82056f3700 1 --2- 172.21.2.104:0/3211342545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81e8003800 0x7f81e8003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f82056f3700 1 -- 172.21.2.104:0/3211342545 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8003260 con 0x7f81e8004500 2021-07-11T03:58:27.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f82056f3700 1 --2- 172.21.2.104:0/3211342545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81e8004500 0x7f81e8004980 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f8200148420 tx=0x7f81f4001d60).ready entity=mon.0 client_cookie=c5297f92ba4a3293 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81f40070d0 con 0x7f81e8004500 2021-07-11T03:58:27.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f4007230 con 0x7f81e8004500 2021-07-11T03:58:27.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8204ef2700 1 -- 172.21.2.104:0/3211342545 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81f400c8f0 con 0x7f81e8004500 2021-07-11T03:58:27.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 -- 172.21.2.104:0/3211342545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81e8004500 msgr2=0x7f81e8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 --2- 172.21.2.104:0/3211342545 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81e8004500 0x7f81e8004980 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f8200148420 tx=0x7f81f4001d60).stop 2021-07-11T03:58:27.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f82066f5700 1 -- 172.21.2.104:0/3211342545 reap_dead start 2021-07-11T03:58:27.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 -- 172.21.2.104:0/3211342545 shutdown_connections 2021-07-11T03:58:27.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 -- 172.21.2.104:0/3211342545 >> 172.21.2.104:0/3211342545 conn(0x7f82000b53e0 msgr2=0x7f82000b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:27.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 -- 172.21.2.104:0/3211342545 shutdown_connections 2021-07-11T03:58:27.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.781+0000 7f8208158700 1 -- 172.21.2.104:0/3211342545 wait complete. 2021-07-11T03:58:27.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8208158700 1 Processor -- start 2021-07-11T03:58:27.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8208158700 1 -- start start 2021-07-11T03:58:27.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8208158700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82000aab90 0x7f82000ab530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8208158700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82000ad1f0 0x7f82000aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8208158700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82000aaf70 0x7f82001442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8208158700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82000aa030 con 0x7f82000aaf70 2021-07-11T03:58:27.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8208158700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82000ace00 con 0x7f82000aab90 2021-07-11T03:58:27.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8208158700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82000ad920 con 0x7f82000ad1f0 2021-07-11T03:58:27.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f82066f5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82000aaf70 0x7f82001442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f82066f5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82000aaf70 0x7f82001442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45556/0 (socket says 172.21.2.104:45556) 2021-07-11T03:58:27.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f82066f5700 1 -- 172.21.2.104:0/761162817 learned_addr learned my addr 172.21.2.104:0/761162817 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:27.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f82066f5700 1 -- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82000ad1f0 msgr2=0x7f82000aba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f82066f5700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82000ad1f0 0x7f82000aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f82066f5700 1 -- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82000aab90 msgr2=0x7f82000ab530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f82066f5700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82000aab90 0x7f82000ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f82066f5700 1 -- 172.21.2.104:0/761162817 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82000af630 con 0x7f82000aaf70 2021-07-11T03:58:27.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f82066f5700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82000aaf70 0x7f82001442a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f81f4001370 tx=0x7f81f8007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f81f2ffd700 1 -- 172.21.2.104:0/761162817 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81f800ceb0 con 0x7f82000aaf70 2021-07-11T03:58:27.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f81f2ffd700 1 -- 172.21.2.104:0/761162817 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f8005a90 con 0x7f82000aaf70 2021-07-11T03:58:27.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8208158700 1 -- 172.21.2.104:0/761162817 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82000adb50 con 0x7f82000aaf70 2021-07-11T03:58:27.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8208158700 1 -- 172.21.2.104:0/761162817 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82000af040 con 0x7f82000aaf70 2021-07-11T03:58:27.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f81f2ffd700 1 -- 172.21.2.104:0/761162817 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81f8005df0 con 0x7f82000aaf70 2021-07-11T03:58:27.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8208158700 1 -- 172.21.2.104:0/761162817 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82000acf50 con 0x7f82000aaf70 2021-07-11T03:58:27.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f81f2ffd700 1 -- 172.21.2.104:0/761162817 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f81f801a060 con 0x7f82000aaf70 2021-07-11T03:58:27.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f81f2ffd700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81e0053240 0x7f81e0055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f81f2ffd700 1 -- 172.21.2.104:0/761162817 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f81f806ad50 con 0x7f82000aaf70 2021-07-11T03:58:27.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.785+0000 7f8205ef4700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81e0053240 0x7f81e0055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.789+0000 7f8205ef4700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81e0053240 0x7f81e0055700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f81fc00a000 tx=0x7f81fc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.789+0000 7f81f2ffd700 1 -- 172.21.2.104:0/761162817 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f82000acf50 con 0x7f82000aaf70 2021-07-11T03:58:27.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.869+0000 7f8208158700 1 -- 172.21.2.104:0/761162817 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82000acf50 con 0x7f82000aaf70 2021-07-11T03:58:27.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f81f2ffd700 1 -- 172.21.2.104:0/761162817 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16054 (secure 0 0 0) 0x7f82000acf50 con 0x7f82000aaf70 2021-07-11T03:58:27.879 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:27.880 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":65,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:58:27.556258+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:58:25.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-07-11T03:58:26.545938+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:58:27.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 -- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81e0053240 msgr2=0x7f81e0055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81e0053240 0x7f81e0055700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f81fc00a000 tx=0x7f81fc008040).stop 2021-07-11T03:58:27.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 -- 172.21.2.104:0/761162817 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82000aaf70 msgr2=0x7f82001442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82000aaf70 0x7f82001442a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f81f4001370 tx=0x7f81f8007dc0).stop 2021-07-11T03:58:27.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 -- 172.21.2.104:0/761162817 shutdown_connections 2021-07-11T03:58:27.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81e0053240 0x7f81e0055700 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82000aaf70 0x7f82001442a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82000ad1f0 0x7f82000aba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 --2- 172.21.2.104:0/761162817 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82000aab90 0x7f82000ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 -- 172.21.2.104:0/761162817 >> 172.21.2.104:0/761162817 conn(0x7f82000b53e0 msgr2=0x7f820005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:27.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 -- 172.21.2.104:0/761162817 shutdown_connections 2021-07-11T03:58:27.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.873+0000 7f8208158700 1 -- 172.21.2.104:0/761162817 wait complete. 2021-07-11T03:58:27.887 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T03:58:27.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e1457e700 1 Processor -- start 2021-07-11T03:58:27.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e1457e700 1 -- start start 2021-07-11T03:58:27.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e1457e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8e0c0a9ec0 0x7f8e0c0aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e1457e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8e0c0603d0 0x7f8e0c13ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e1457e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e0c061990 con 0x7f8e0c13d1c0 2021-07-11T03:58:27.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e1457e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0c061c30 con 0x7f8e0c0603d0 2021-07-11T03:58:27.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e1457e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c060b90 con 0x7f8e0c0a9ec0 2021-07-11T03:58:27.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e11b19700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8e0c0603d0 0x7f8e0c13ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e11b19700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8e0c0603d0 0x7f8e0c13ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45564/0 (socket says 172.21.2.104:45564) 2021-07-11T03:58:27.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e11b19700 1 -- 172.21.2.104:0/1186842416 learned_addr learned my addr 172.21.2.104:0/1186842416 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:27.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e11b19700 1 -- 172.21.2.104:0/1186842416 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8e0c0a9ec0 msgr2=0x7f8e0c0aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:27.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e11b19700 1 --2- 172.21.2.104:0/1186842416 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8e0c0a9ec0 0x7f8e0c0aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e11b19700 1 -- 172.21.2.104:0/1186842416 >> v1:172.21.2.105:6789/0 conn(0x7f8e0c13d1c0 legacy=0x7f8e0c13f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:27.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e11b19700 1 -- 172.21.2.104:0/1186842416 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c060e30 con 0x7f8e0c0603d0 2021-07-11T03:58:27.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e11b19700 1 --2- 172.21.2.104:0/1186842416 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8e0c0603d0 0x7f8e0c13ccd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f8e0c05f870 tx=0x7f8e00006e40).ready entity=mon.0 client_cookie=abd869fb6acd12a9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e11318700 1 -- 172.21.2.104:0/1186842416 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e0000bca0 con 0x7f8e0c0603d0 2021-07-11T03:58:27.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e11318700 1 -- 172.21.2.104:0/1186842416 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e00000e90 con 0x7f8e0c0603d0 2021-07-11T03:58:27.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e11318700 1 -- 172.21.2.104:0/1186842416 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e000155e0 con 0x7f8e0c0603d0 2021-07-11T03:58:27.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e1457e700 1 -- 172.21.2.104:0/1186842416 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8e0c0603d0 msgr2=0x7f8e0c13ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.933+0000 7f8e1457e700 1 --2- 172.21.2.104:0/1186842416 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8e0c0603d0 0x7f8e0c13ccd0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f8e0c05f870 tx=0x7f8e00006e40).stop 2021-07-11T03:58:27.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 -- 172.21.2.104:0/1186842416 shutdown_connections 2021-07-11T03:58:27.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 --2- 172.21.2.104:0/1186842416 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8e0c0603d0 0x7f8e0c13ccd0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:27.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 --2- 172.21.2.104:0/1186842416 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8e0c0a9ec0 0x7f8e0c0aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 -- 172.21.2.104:0/1186842416 >> 172.21.2.104:0/1186842416 conn(0x7f8e0c0b2970 msgr2=0x7f8e0c0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:27.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 -- 172.21.2.104:0/1186842416 shutdown_connections 2021-07-11T03:58:27.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 -- 172.21.2.104:0/1186842416 wait complete. 2021-07-11T03:58:27.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 Processor -- start 2021-07-11T03:58:27.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 -- start start 2021-07-11T03:58:27.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8e0c0603d0 0x7f8e0c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8e0c0a9ec0 0x7f8e0c13ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8e0c13d1c0 0x7f8e0c0af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c13fcd0 con 0x7f8e0c0a9ec0 2021-07-11T03:58:27.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c13fa30 con 0x7f8e0c13d1c0 2021-07-11T03:58:27.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c13f020 con 0x7f8e0c0603d0 2021-07-11T03:58:27.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e11b19700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8e0c0a9ec0 0x7f8e0c13ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e11b19700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8e0c0a9ec0 0x7f8e0c13ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45568/0 (socket says 172.21.2.104:45568) 2021-07-11T03:58:27.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e11b19700 1 -- 172.21.2.104:0/4014926185 learned_addr learned my addr 172.21.2.104:0/4014926185 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:27.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e11b19700 1 -- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8e0c0603d0 msgr2=0x7f8e0c0ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e11b19700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8e0c0603d0 0x7f8e0c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e11b19700 1 -- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8e0c13d1c0 msgr2=0x7f8e0c0af9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:27.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e11b19700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8e0c13d1c0 0x7f8e0c0af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:27.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e11b19700 1 -- 172.21.2.104:0/4014926185 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0000d4d0 con 0x7f8e0c0a9ec0 2021-07-11T03:58:27.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e11b19700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8e0c0a9ec0 0x7f8e0c13ffb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f8e00006fc0 tx=0x7f8e00018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8dff7fe700 1 -- 172.21.2.104:0/4014926185 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e00015c70 con 0x7f8e0c0a9ec0 2021-07-11T03:58:27.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8dff7fe700 1 -- 172.21.2.104:0/4014926185 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e00015dd0 con 0x7f8e0c0a9ec0 2021-07-11T03:58:27.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8dff7fe700 1 -- 172.21.2.104:0/4014926185 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8e0001e620 con 0x7f8e0c0a9ec0 2021-07-11T03:58:27.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 -- 172.21.2.104:0/4014926185 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0c13f2c0 con 0x7f8e0c0a9ec0 2021-07-11T03:58:27.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 -- 172.21.2.104:0/4014926185 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0c0afed0 con 0x7f8e0c0a9ec0 2021-07-11T03:58:27.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1457e700 1 -- 172.21.2.104:0/4014926185 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0c13e7a0 con 0x7f8e0c0a9ec0 2021-07-11T03:58:27.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8dff7fe700 1 -- 172.21.2.104:0/4014926185 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8e0001e780 con 0x7f8e0c0a9ec0 2021-07-11T03:58:27.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8dff7fe700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dec053330 0x7f8dec0557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:27.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8dff7fe700 1 -- 172.21.2.104:0/4014926185 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8e0006ecf0 con 0x7f8e0c0a9ec0 2021-07-11T03:58:27.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.937+0000 7f8e1231a700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dec053330 0x7f8dec0557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:27.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.941+0000 7f8e1231a700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dec053330 0x7f8dec0557f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f8e08002f70 tx=0x7f8e08006ab0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:27.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:27.941+0000 7f8dff7fe700 1 -- 172.21.2.104:0/4014926185 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8e00045f20 con 0x7f8e0c0a9ec0 2021-07-11T03:58:28.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.049+0000 7f8e1457e700 1 -- 172.21.2.104:0/4014926185 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8e0c13e7a0 con 0x7f8e0c0a9ec0 2021-07-11T03:58:28.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8dff7fe700 1 -- 172.21.2.104:0/4014926185 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v27) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8e00002f50 con 0x7f8e0c0a9ec0 2021-07-11T03:58:28.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 -- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dec053330 msgr2=0x7f8dec0557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dec053330 0x7f8dec0557f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f8e08002f70 tx=0x7f8e08006ab0).stop 2021-07-11T03:58:28.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 -- 172.21.2.104:0/4014926185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8e0c0a9ec0 msgr2=0x7f8e0c13ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8e0c0a9ec0 0x7f8e0c13ffb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f8e00006fc0 tx=0x7f8e00018000).stop 2021-07-11T03:58:28.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 -- 172.21.2.104:0/4014926185 shutdown_connections 2021-07-11T03:58:28.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8e0c13d1c0 0x7f8e0c0af9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:28.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8dec053330 0x7f8dec0557f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:28.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8e0c0a9ec0 0x7f8e0c13ffb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:28.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 --2- 172.21.2.104:0/4014926185 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8e0c0603d0 0x7f8e0c0ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:28.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 -- 172.21.2.104:0/4014926185 >> 172.21.2.104:0/4014926185 conn(0x7f8e0c0b2970 msgr2=0x7f8e0c13b450 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:28.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 -- 172.21.2.104:0/4014926185 shutdown_connections 2021-07-11T03:58:28.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.565+0000 7f8e1457e700 1 -- 172.21.2.104:0/4014926185 wait complete. 2021-07-11T03:58:28.584 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:58:28.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe91db7e700 1 Processor -- start 2021-07-11T03:58:28.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe91db7e700 1 -- start start 2021-07-11T03:58:28.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe91db7e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe9180b3240 0x7fe918066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe91db7e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9180b3620 con 0x7fe9180b0e70 2021-07-11T03:58:28.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe91db7e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9181436f0 con 0x7fe9180b1840 2021-07-11T03:58:28.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe91db7e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe918065ca0 con 0x7fe9180b3240 2021-07-11T03:58:28.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe917fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe9180b0e70 0x7fe91814a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42344/0 (socket says 172.21.2.104:42344) 2021-07-11T03:58:28.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe917fff700 1 -- 172.21.2.104:0/211651505 learned_addr learned my addr 172.21.2.104:0/211651505 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:28.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe9177fe700 1 --2- 172.21.2.104:0/211651505 >> v2:172.21.2.104:3300/0 conn(0x7fe9180b3240 0x7fe918066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:28.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe9167fc700 1 -- 172.21.2.104:0/211651505 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555580630 0 0) 0x7fe9180b3620 con 0x7fe9180b0e70 2021-07-11T03:58:28.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe9167fc700 1 -- 172.21.2.104:0/211651505 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f8003260 con 0x7fe9180b0e70 2021-07-11T03:58:28.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe9177fe700 1 -- 172.21.2.104:0/211651505 >> v1:172.21.2.105:6789/0 conn(0x7fe9180b1840 legacy=0x7fe918142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:28.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe9177fe700 1 -- 172.21.2.104:0/211651505 >> v1:172.21.2.104:6789/0 conn(0x7fe9180b0e70 legacy=0x7fe91814a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe9177fe700 1 -- 172.21.2.104:0/211651505 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe918065f40 con 0x7fe9180b3240 2021-07-11T03:58:28.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.649+0000 7fe9177fe700 1 --2- 172.21.2.104:0/211651505 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9180b3240 0x7fe918066680 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fe9180b7520 tx=0x7fe908006df0).ready entity=mon.0 client_cookie=8f1f3aa05a09723f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:28.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe9167fc700 1 -- 172.21.2.104:0/211651505 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe90800d5c0 con 0x7fe9180b3240 2021-07-11T03:58:28.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe9167fc700 1 -- 172.21.2.104:0/211651505 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe908000e90 con 0x7fe9180b3240 2021-07-11T03:58:28.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe9167fc700 1 -- 172.21.2.104:0/211651505 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe90800d910 con 0x7fe9180b3240 2021-07-11T03:58:28.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- 172.21.2.104:0/211651505 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9180b3240 msgr2=0x7fe918066680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 --2- 172.21.2.104:0/211651505 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9180b3240 0x7fe918066680 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fe9180b7520 tx=0x7fe908006df0).stop 2021-07-11T03:58:28.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- 172.21.2.104:0/211651505 shutdown_connections 2021-07-11T03:58:28.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 --2- 172.21.2.104:0/211651505 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9180b3240 0x7fe918066680 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:28.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- 172.21.2.104:0/211651505 >> 172.21.2.104:0/211651505 conn(0x7fe91805c490 msgr2=0x7fe91805c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:28.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- 172.21.2.104:0/211651505 shutdown_connections 2021-07-11T03:58:28.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- 172.21.2.104:0/211651505 wait complete. 2021-07-11T03:58:28.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 Processor -- start 2021-07-11T03:58:28.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- start start 2021-07-11T03:58:28.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9180b0e70 0x7fe918147a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9180b1840 0x7fe918150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9180b3240 0x7fe918150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91813a020 con 0x7fe9180b3240 2021-07-11T03:58:28.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe918137490 con 0x7fe9180b0e70 2021-07-11T03:58:28.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe918137730 con 0x7fe9180b1840 2021-07-11T03:58:28.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe917fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9180b3240 0x7fe918150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:28.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe917fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9180b3240 0x7fe918150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45584/0 (socket says 172.21.2.104:45584) 2021-07-11T03:58:28.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe917fff700 1 -- 172.21.2.104:0/107619385 learned_addr learned my addr 172.21.2.104:0/107619385 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:28.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe917fff700 1 -- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9180b1840 msgr2=0x7fe918150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe917fff700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9180b1840 0x7fe918150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:28.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe917fff700 1 -- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9180b0e70 msgr2=0x7fe918147a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe917fff700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9180b0e70 0x7fe918147a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:28.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe917fff700 1 -- 172.21.2.104:0/107619385 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90800b180 con 0x7fe9180b3240 2021-07-11T03:58:28.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe917fff700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9180b3240 0x7fe918150650 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fe910001490 tx=0x7fe910007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:28.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe914ff9700 1 -- 172.21.2.104:0/107619385 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe910005ba0 con 0x7fe9180b3240 2021-07-11T03:58:28.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe914ff9700 1 -- 172.21.2.104:0/107619385 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe910005d00 con 0x7fe9180b3240 2021-07-11T03:58:28.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- 172.21.2.104:0/107619385 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9181379d0 con 0x7fe9180b3240 2021-07-11T03:58:28.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe914ff9700 1 -- 172.21.2.104:0/107619385 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9100072c0 con 0x7fe9180b3240 2021-07-11T03:58:28.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- 172.21.2.104:0/107619385 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe918150b40 con 0x7fe9180b3240 2021-07-11T03:58:28.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe91db7e700 1 -- 172.21.2.104:0/107619385 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe918137b60 con 0x7fe9180b3240 2021-07-11T03:58:28.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe914ff9700 1 -- 172.21.2.104:0/107619385 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe91001a060 con 0x7fe9180b3240 2021-07-11T03:58:28.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe914ff9700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8fc053270 0x7fe8fc055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe914ff9700 1 -- 172.21.2.104:0/107619385 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe91006b080 con 0x7fe9180b3240 2021-07-11T03:58:28.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.653+0000 7fe9177fe700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8fc053270 0x7fe8fc055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:28.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.657+0000 7fe9177fe700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8fc053270 0x7fe8fc055730 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fe9180b7520 tx=0x7fe908014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:28.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.657+0000 7fe914ff9700 1 -- 172.21.2.104:0/107619385 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe9100423c0 con 0x7fe9180b3240 2021-07-11T03:58:28.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.765+0000 7fe91db7e700 1 -- 172.21.2.104:0/107619385 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe918057350 con 0x7fe9180b3240 2021-07-11T03:58:28.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.765+0000 7fe914ff9700 1 -- 172.21.2.104:0/107619385 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2490 (secure 0 0 0) 0x7fe910018030 con 0x7fe9180b3240 2021-07-11T03:58:28.774 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:28.775 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/2908869555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2908869555},{"type":"v1","addr":"172.21.2.105:6835","nonce":2908869555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/1110128427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1110128427},{"type":"v1","addr":"172.21.2.104:6833","nonce":1110128427}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:58:27.556328+0000","modified":"2021-07-11T03:58:28.559390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/225259368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":225259368},{"type":"v1","addr":"172.21.2.104:6835","nonce":225259368}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-07-11T03:58:28.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.765+0000 7fe91db7e700 1 -- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8fc053270 msgr2=0x7fe8fc055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.765+0000 7fe91db7e700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8fc053270 0x7fe8fc055730 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7fe9180b7520 tx=0x7fe908014000).stop 2021-07-11T03:58:28.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.765+0000 7fe91db7e700 1 -- 172.21.2.104:0/107619385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9180b3240 msgr2=0x7fe918150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.765+0000 7fe91db7e700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9180b3240 0x7fe918150650 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7fe910001490 tx=0x7fe910007dc0).stop 2021-07-11T03:58:28.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.765+0000 7fe91db7e700 1 -- 172.21.2.104:0/107619385 shutdown_connections 2021-07-11T03:58:28.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.765+0000 7fe91db7e700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8fc053270 0x7fe8fc055730 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:28.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.765+0000 7fe91db7e700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9180b3240 0x7fe918150650 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:28.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.765+0000 7fe91db7e700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9180b1840 0x7fe918150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:28.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.765+0000 7fe91db7e700 1 --2- 172.21.2.104:0/107619385 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9180b0e70 0x7fe918147a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:28.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.769+0000 7fe91db7e700 1 -- 172.21.2.104:0/107619385 >> 172.21.2.104:0/107619385 conn(0x7fe91805c490 msgr2=0x7fe918064550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:28.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.769+0000 7fe91db7e700 1 -- 172.21.2.104:0/107619385 shutdown_connections 2021-07-11T03:58:28.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.769+0000 7fe91db7e700 1 -- 172.21.2.104:0/107619385 wait complete. 2021-07-11T03:58:28.780 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 27 2021-07-11T03:58:28.781 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:58:28.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 Processor -- start 2021-07-11T03:58:28.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 -- start start 2021-07-11T03:58:28.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb8cc0a76b0 0x7fb8cc066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc0a7a90 con 0x7fb8cc0b1970 2021-07-11T03:58:28.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc147620 con 0x7fb8cc0b0060 2021-07-11T03:58:28.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8cc1478c0 con 0x7fb8cc0a76b0 2021-07-11T03:58:28.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8ca7fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb8cc0b1970 0x7fb8cc05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42356/0 (socket says 172.21.2.104:42356) 2021-07-11T03:58:28.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8ca7fc700 1 -- 172.21.2.104:0/3382841005 learned_addr learned my addr 172.21.2.104:0/3382841005 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:28.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516235893 0 0) 0x7fb8cc0a7a90 con 0x7fb8cc0b1970 2021-07-11T03:58:28.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8ac003260 con 0x7fb8cc0b1970 2021-07-11T03:58:28.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4068203603 0 0) 0x7fb8ac003260 con 0x7fb8cc0b1970 2021-07-11T03:58:28.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8caffd700 1 --2- 172.21.2.104:0/3382841005 >> v2:172.21.2.105:3300/0 conn(0x7fb8cc0a76b0 0x7fb8cc066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:28.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 >> v2:172.21.2.105:3300/0 conn(0x7fb8cc0a76b0 msgr2=0x7fb8cc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:28.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 --2- 172.21.2.104:0/3382841005 >> v2:172.21.2.105:3300/0 conn(0x7fb8cc0a76b0 0x7fb8cc066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:28.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 >> v1:172.21.2.105:6789/0 conn(0x7fb8cc0b0060 legacy=0x7fb8cc151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:28.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc147b60 con 0x7fb8cc0b1970 2021-07-11T03:58:28.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fb8b4001680 con 0x7fb8cc0b1970 2021-07-11T03:58:28.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 >> v1:172.21.2.104:6789/0 conn(0x7fb8cc0b1970 legacy=0x7fb8cc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 --2- 172.21.2.104:0/3382841005 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 0x7fb8ac003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 --2- 172.21.2.104:0/3382841005 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8ac004500 0x7fb8ac004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 --2- 172.21.2.104:0/3382841005 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8ac004f90 0x7fb8ac00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc1478c0 con 0x7fb8ac003750 2021-07-11T03:58:28.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac00be30 con 0x7fb8ac004f90 2021-07-11T03:58:28.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ac00bff0 con 0x7fb8ac004500 2021-07-11T03:58:28.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8caffd700 1 --2- 172.21.2.104:0/3382841005 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 0x7fb8ac003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:28.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8caffd700 1 -- 172.21.2.104:0/3382841005 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8ac004500 msgr2=0x7fb8ac004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:28.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8caffd700 1 --2- 172.21.2.104:0/3382841005 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8ac004500 0x7fb8ac004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:28.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8caffd700 1 -- 172.21.2.104:0/3382841005 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8ac004f90 msgr2=0x7fb8ac00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:28.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8caffd700 1 --2- 172.21.2.104:0/3382841005 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8ac004f90 0x7fb8ac00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:28.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8caffd700 1 -- 172.21.2.104:0/3382841005 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc147b60 con 0x7fb8ac003750 2021-07-11T03:58:28.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8cb7fe700 1 -- 172.21.2.104:0/3382841005 reap_dead start 2021-07-11T03:58:28.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8caffd700 1 --2- 172.21.2.104:0/3382841005 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 0x7fb8ac003b50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fb8bc007000 tx=0x7fb8bc008040).ready entity=mon.0 client_cookie=42c4f479960a22b5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:28.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8bc012470 con 0x7fb8ac003750 2021-07-11T03:58:28.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc00c030 con 0x7fb8ac003750 2021-07-11T03:58:28.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8c9ffb700 1 -- 172.21.2.104:0/3382841005 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8bc0127c0 con 0x7fb8ac003750 2021-07-11T03:58:28.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 -- 172.21.2.104:0/3382841005 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 msgr2=0x7fb8ac003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 --2- 172.21.2.104:0/3382841005 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 0x7fb8ac003b50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fb8bc007000 tx=0x7fb8bc008040).stop 2021-07-11T03:58:28.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 -- 172.21.2.104:0/3382841005 shutdown_connections 2021-07-11T03:58:28.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 --2- 172.21.2.104:0/3382841005 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 0x7fb8ac003b50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:28.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 -- 172.21.2.104:0/3382841005 >> 172.21.2.104:0/3382841005 conn(0x7fb8cc0b53e0 msgr2=0x7fb8cc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:28.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 -- 172.21.2.104:0/3382841005 shutdown_connections 2021-07-11T03:58:28.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.829+0000 7fb8d16af700 1 -- 172.21.2.104:0/3382841005 wait complete. 2021-07-11T03:58:28.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8d16af700 1 Processor -- start 2021-07-11T03:58:28.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8d16af700 1 -- start start 2021-07-11T03:58:28.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8d16af700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 0x7fb8cc146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8d16af700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8cc0af410 0x7fb8cc0ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8d16af700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8cc0a9430 0x7fb8cc144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8d16af700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc066820 con 0x7fb8ac003750 2021-07-11T03:58:28.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8d16af700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc0aca10 con 0x7fb8cc0af410 2021-07-11T03:58:28.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8d16af700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc066a50 con 0x7fb8cc0a9430 2021-07-11T03:58:28.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8caffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 0x7fb8cc146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:28.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8caffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 0x7fb8cc146790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45598/0 (socket says 172.21.2.104:45598) 2021-07-11T03:58:28.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8caffd700 1 -- 172.21.2.104:0/3114562420 learned_addr learned my addr 172.21.2.104:0/3114562420 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:28.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8ca7fc700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8cc0af410 0x7fb8cc0ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:28.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8caffd700 1 -- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8cc0a9430 msgr2=0x7fb8cc144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8caffd700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8cc0a9430 0x7fb8cc144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:28.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8caffd700 1 -- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8cc0af410 msgr2=0x7fb8cc0ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8caffd700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8cc0af410 0x7fb8cc0ac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:28.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8caffd700 1 -- 172.21.2.104:0/3114562420 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc0aba40 con 0x7fb8ac003750 2021-07-11T03:58:28.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8ca7fc700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8cc0af410 0x7fb8cc0ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:58:28.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8caffd700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 0x7fb8cc146790 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fb8bc007000 tx=0x7fb8bc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:28.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8c3fff700 1 -- 172.21.2.104:0/3114562420 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8bc01c6f0 con 0x7fb8ac003750 2021-07-11T03:58:28.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8c3fff700 1 -- 172.21.2.104:0/3114562420 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8bc01c850 con 0x7fb8ac003750 2021-07-11T03:58:28.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8c3fff700 1 -- 172.21.2.104:0/3114562420 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8bc01cbb0 con 0x7fb8ac003750 2021-07-11T03:58:28.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8d16af700 1 -- 172.21.2.104:0/3114562420 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8cc0abce0 con 0x7fb8ac003750 2021-07-11T03:58:28.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8d16af700 1 -- 172.21.2.104:0/3114562420 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8cc0adc30 con 0x7fb8ac003750 2021-07-11T03:58:28.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8d16af700 1 -- 172.21.2.104:0/3114562420 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8cc057350 con 0x7fb8ac003750 2021-07-11T03:58:28.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8c3fff700 1 -- 172.21.2.104:0/3114562420 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb8bc02b070 con 0x7fb8ac003750 2021-07-11T03:58:28.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8c3fff700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8b0053350 0x7fb8b0055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:28.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8c3fff700 1 -- 172.21.2.104:0/3114562420 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb8bc06c520 con 0x7fb8ac003750 2021-07-11T03:58:28.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.833+0000 7fb8ca7fc700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8b0053350 0x7fb8b0055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:28.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.837+0000 7fb8ca7fc700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8b0053350 0x7fb8b0055810 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fb8b4008000 tx=0x7fb8b4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:28.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.837+0000 7fb8c3fff700 1 -- 172.21.2.104:0/3114562420 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb8bc043860 con 0x7fb8ac003750 2021-07-11T03:58:28.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 -- 172.21.2.104:0/3114562420 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8cc060a40 con 0x7fb8ac003750 2021-07-11T03:58:28.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8c3fff700 1 -- 172.21.2.104:0/3114562420 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16041 (secure 0 0 0) 0x7fb8bc046e80 con 0x7fb8ac003750 2021-07-11T03:58:28.933 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:28.933 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":66,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:58:28.560561+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:58:25.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-07-11T03:58:26.545938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:58:28.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 -- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8b0053350 msgr2=0x7fb8b0055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8b0053350 0x7fb8b0055810 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fb8b4008000 tx=0x7fb8b4009040).stop 2021-07-11T03:58:28.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 -- 172.21.2.104:0/3114562420 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 msgr2=0x7fb8cc146790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:28.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 0x7fb8cc146790 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fb8bc007000 tx=0x7fb8bc008040).stop 2021-07-11T03:58:28.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 -- 172.21.2.104:0/3114562420 shutdown_connections 2021-07-11T03:58:28.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8cc0a9430 0x7fb8cc144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:28.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8cc0af410 0x7fb8cc0ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:28.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8b0053350 0x7fb8b0055810 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:28.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 --2- 172.21.2.104:0/3114562420 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8ac003750 0x7fb8cc146790 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:28.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 -- 172.21.2.104:0/3114562420 >> 172.21.2.104:0/3114562420 conn(0x7fb8cc0b53e0 msgr2=0x7fb8cc0a7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:28.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 -- 172.21.2.104:0/3114562420 shutdown_connections 2021-07-11T03:58:28.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.921+0000 7fb8d16af700 1 -- 172.21.2.104:0/3114562420 wait complete. 2021-07-11T03:58:28.939 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T03:58:29.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 Processor -- start 2021-07-11T03:58:29.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 -- start start 2021-07-11T03:58:29.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd4540973d0 0x7fd45405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4540b2a40 con 0x7fd45413f560 2021-07-11T03:58:29.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd454097b80 con 0x7fd4540b2660 2021-07-11T03:58:29.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd454097e20 con 0x7fd4540973d0 2021-07-11T03:58:29.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45942e700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd45413f560 0x7fd45413f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42374/0 (socket says 172.21.2.104:42374) 2021-07-11T03:58:29.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45942e700 1 -- 172.21.2.104:0/2177562977 learned_addr learned my addr 172.21.2.104:0/2177562977 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:29.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61422589 0 0) 0x7fd4540b2a40 con 0x7fd45413f560 2021-07-11T03:58:29.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd434004370 con 0x7fd45413f560 2021-07-11T03:58:29.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd453fff700 1 --2- 172.21.2.104:0/2177562977 >> v2:172.21.2.105:3300/0 conn(0x7fd4540973d0 0x7fd45405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3663122210 0 0) 0x7fd434004370 con 0x7fd45413f560 2021-07-11T03:58:29.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 >> v2:172.21.2.105:3300/0 conn(0x7fd4540973d0 msgr2=0x7fd45405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 --2- 172.21.2.104:0/2177562977 >> v2:172.21.2.105:3300/0 conn(0x7fd4540973d0 0x7fd45405f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 >> v1:172.21.2.105:6789/0 conn(0x7fd4540b2660 legacy=0x7fd454096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540980c0 con 0x7fd45413f560 2021-07-11T03:58:29.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fd44c001a00 con 0x7fd45413f560 2021-07-11T03:58:29.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 >> v1:172.21.2.104:6789/0 conn(0x7fd45413f560 legacy=0x7fd45413f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 --2- 172.21.2.104:0/2177562977 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd434004860 0x7fd434004c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 --2- 172.21.2.104:0/2177562977 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd434005610 0x7fd434005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 --2- 172.21.2.104:0/2177562977 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd434005f80 0x7fd43400c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd454097e20 con 0x7fd434004860 2021-07-11T03:58:29.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43400ce60 con 0x7fd434005f80 2021-07-11T03:58:29.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43400d020 con 0x7fd434005610 2021-07-11T03:58:29.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd458c2d700 1 --2- 172.21.2.104:0/2177562977 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd434004860 0x7fd434004c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd458c2d700 1 -- 172.21.2.104:0/2177562977 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd434005610 msgr2=0x7fd434005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd458c2d700 1 --2- 172.21.2.104:0/2177562977 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd434005610 0x7fd434005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd458c2d700 1 -- 172.21.2.104:0/2177562977 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd434005f80 msgr2=0x7fd43400c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd458c2d700 1 --2- 172.21.2.104:0/2177562977 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd434005f80 0x7fd43400c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd458c2d700 1 -- 172.21.2.104:0/2177562977 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540980c0 con 0x7fd434004860 2021-07-11T03:58:29.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45942e700 1 -- 172.21.2.104:0/2177562977 reap_dead start 2021-07-11T03:58:29.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd458c2d700 1 --2- 172.21.2.104:0/2177562977 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd434004860 0x7fd434004c60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fd444001fa0 tx=0x7fd444007a00).ready entity=mon.0 client_cookie=613af6d1c75a3909 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd444007070 con 0x7fd434004860 2021-07-11T03:58:29.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4440071d0 con 0x7fd434004860 2021-07-11T03:58:29.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd4537fe700 1 -- 172.21.2.104:0/2177562977 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd444006510 con 0x7fd434004860 2021-07-11T03:58:29.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 -- 172.21.2.104:0/2177562977 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd434004860 msgr2=0x7fd434004c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 --2- 172.21.2.104:0/2177562977 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd434004860 0x7fd434004c60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fd444001fa0 tx=0x7fd444007a00).stop 2021-07-11T03:58:29.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 -- 172.21.2.104:0/2177562977 shutdown_connections 2021-07-11T03:58:29.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 --2- 172.21.2.104:0/2177562977 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd434004860 0x7fd434004c60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 -- 172.21.2.104:0/2177562977 >> 172.21.2.104:0/2177562977 conn(0x7fd454062f10 msgr2=0x7fd4540632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:29.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 -- 172.21.2.104:0/2177562977 shutdown_connections 2021-07-11T03:58:29.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:28.997+0000 7fd45ae91700 1 -- 172.21.2.104:0/2177562977 wait complete. 2021-07-11T03:58:29.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd45ae91700 1 Processor -- start 2021-07-11T03:58:29.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd45ae91700 1 -- start start 2021-07-11T03:58:29.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd45ae91700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd434004860 0x7fd454065140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd45ae91700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4540abd60 0x7fd4540ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd45ae91700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd4540acf90 0x7fd4540ad410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd45ae91700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45405ef30 con 0x7fd4540abd60 2021-07-11T03:58:29.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd45ae91700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd454066510 con 0x7fd4540acf90 2021-07-11T03:58:29.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd45ae91700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4540ae3b0 con 0x7fd434004860 2021-07-11T03:58:29.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd453fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4540abd60 0x7fd4540ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd453fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4540abd60 0x7fd4540ac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45616/0 (socket says 172.21.2.104:45616) 2021-07-11T03:58:29.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd453fff700 1 -- 172.21.2.104:0/3894761051 learned_addr learned my addr 172.21.2.104:0/3894761051 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:29.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd453fff700 1 -- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd434004860 msgr2=0x7fd454065140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd453fff700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd434004860 0x7fd454065140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd453fff700 1 -- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd4540acf90 msgr2=0x7fd4540ad410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd453fff700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd4540acf90 0x7fd4540ad410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd453fff700 1 -- 172.21.2.104:0/3894761051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454066740 con 0x7fd4540abd60 2021-07-11T03:58:29.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd453fff700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4540abd60 0x7fd4540ac1c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fd4540abc40 tx=0x7fd43c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd451ffb700 1 -- 172.21.2.104:0/3894761051 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd43c016070 con 0x7fd4540abd60 2021-07-11T03:58:29.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd451ffb700 1 -- 172.21.2.104:0/3894761051 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43c000e90 con 0x7fd4540abd60 2021-07-11T03:58:29.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd45ae91700 1 -- 172.21.2.104:0/3894761051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4540669e0 con 0x7fd4540abd60 2021-07-11T03:58:29.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd451ffb700 1 -- 172.21.2.104:0/3894761051 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd43c011970 con 0x7fd4540abd60 2021-07-11T03:58:29.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd45ae91700 1 -- 172.21.2.104:0/3894761051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd454067940 con 0x7fd4540abd60 2021-07-11T03:58:29.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd45ae91700 1 -- 172.21.2.104:0/3894761051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4540657d0 con 0x7fd4540abd60 2021-07-11T03:58:29.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd451ffb700 1 -- 172.21.2.104:0/3894761051 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd43c011ad0 con 0x7fd4540abd60 2021-07-11T03:58:29.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd451ffb700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd438053200 0x7fd4380556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd451ffb700 1 -- 172.21.2.104:0/3894761051 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd43c06af40 con 0x7fd4540abd60 2021-07-11T03:58:29.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.001+0000 7fd458c2d700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd438053200 0x7fd4380556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.005+0000 7fd458c2d700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd438053200 0x7fd4380556c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fd444001fa0 tx=0x7fd444015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.005+0000 7fd451ffb700 1 -- 172.21.2.104:0/3894761051 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd43c068cf0 con 0x7fd4540abd60 2021-07-11T03:58:29.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.121+0000 7fd45ae91700 1 -- 172.21.2.104:0/3894761051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fd454057350 con 0x7fd4540abd60 2021-07-11T03:58:29.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.125+0000 7fd451ffb700 1 -- 172.21.2.104:0/3894761051 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd43c045cb0 con 0x7fd4540abd60 2021-07-11T03:58:29.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.125+0000 7fd45ae91700 1 -- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd438053200 msgr2=0x7fd4380556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.125+0000 7fd45ae91700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd438053200 0x7fd4380556c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fd444001fa0 tx=0x7fd444015000).stop 2021-07-11T03:58:29.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.125+0000 7fd45ae91700 1 -- 172.21.2.104:0/3894761051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4540abd60 msgr2=0x7fd4540ac1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.125+0000 7fd45ae91700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4540abd60 0x7fd4540ac1c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fd4540abc40 tx=0x7fd43c008040).stop 2021-07-11T03:58:29.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.129+0000 7fd45ae91700 1 -- 172.21.2.104:0/3894761051 shutdown_connections 2021-07-11T03:58:29.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.129+0000 7fd45ae91700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd4540acf90 0x7fd4540ad410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.129+0000 7fd45ae91700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd438053200 0x7fd4380556c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.129+0000 7fd45ae91700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd4540abd60 0x7fd4540ac1c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.129+0000 7fd45ae91700 1 --2- 172.21.2.104:0/3894761051 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd434004860 0x7fd454065140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.129+0000 7fd45ae91700 1 -- 172.21.2.104:0/3894761051 >> 172.21.2.104:0/3894761051 conn(0x7fd454062f10 msgr2=0x7fd4540b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:29.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.129+0000 7fd45ae91700 1 -- 172.21.2.104:0/3894761051 shutdown_connections 2021-07-11T03:58:29.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.129+0000 7fd45ae91700 1 -- 172.21.2.104:0/3894761051 wait complete. 2021-07-11T03:58:29.138 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T03:58:29.141 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T03:58:29.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 Processor -- start 2021-07-11T03:58:29.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 -- start start 2021-07-11T03:58:29.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f54c4067180 0x7f54c405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f54c40b1970 0x7f54c405fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c40a7a90 con 0x7f54c40a76b0 2021-07-11T03:58:29.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c4147620 con 0x7f54c40b1970 2021-07-11T03:58:29.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c41478c0 con 0x7f54c4067180 2021-07-11T03:58:29.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54c96b5700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f54c40a76b0 0x7f54c4066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42392/0 (socket says 172.21.2.104:42392) 2021-07-11T03:58:29.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54c96b5700 1 -- 172.21.2.104:0/3956270991 learned_addr learned my addr 172.21.2.104:0/3956270991 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:29.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54c9eb6700 1 --2- 172.21.2.104:0/3956270991 >> v2:172.21.2.104:3300/0 conn(0x7f54c40b1970 0x7f54c405fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54b3fff700 1 -- 172.21.2.104:0/3956270991 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741967390 0 0) 0x7f54c40a7a90 con 0x7f54c40a76b0 2021-07-11T03:58:29.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54b3fff700 1 -- 172.21.2.104:0/3956270991 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54ac003260 con 0x7f54c40a76b0 2021-07-11T03:58:29.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54c8eb4700 1 --2- 172.21.2.104:0/3956270991 >> v2:172.21.2.105:3300/0 conn(0x7f54c4067180 0x7f54c405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54c9eb6700 1 -- 172.21.2.104:0/3956270991 >> v2:172.21.2.105:3300/0 conn(0x7f54c4067180 msgr2=0x7f54c405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54c9eb6700 1 --2- 172.21.2.104:0/3956270991 >> v2:172.21.2.105:3300/0 conn(0x7f54c4067180 0x7f54c405f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54c9eb6700 1 -- 172.21.2.104:0/3956270991 >> v1:172.21.2.104:6789/0 conn(0x7f54c40a76b0 legacy=0x7f54c4066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54c9eb6700 1 -- 172.21.2.104:0/3956270991 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c4147b60 con 0x7f54c40b1970 2021-07-11T03:58:29.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54c9eb6700 1 --2- 172.21.2.104:0/3956270991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54c40b1970 0x7f54c405fba0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f54bc0032f0 tx=0x7f54bc0091d0).ready entity=mon.0 client_cookie=b4146876ff5ddd4e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54b3fff700 1 -- 172.21.2.104:0/3956270991 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54bc00b680 con 0x7f54c40b1970 2021-07-11T03:58:29.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54b3fff700 1 -- 172.21.2.104:0/3956270991 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54bc00b7e0 con 0x7f54c40b1970 2021-07-11T03:58:29.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54b3fff700 1 -- 172.21.2.104:0/3956270991 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54bc00b990 con 0x7f54c40b1970 2021-07-11T03:58:29.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 -- 172.21.2.104:0/3956270991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54c40b1970 msgr2=0x7f54c405fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 --2- 172.21.2.104:0/3956270991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54c40b1970 0x7f54c405fba0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f54bc0032f0 tx=0x7f54bc0091d0).stop 2021-07-11T03:58:29.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 -- 172.21.2.104:0/3956270991 shutdown_connections 2021-07-11T03:58:29.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 --2- 172.21.2.104:0/3956270991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54c40b1970 0x7f54c405fba0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 --2- 172.21.2.104:0/3956270991 >> v2:172.21.2.105:3300/0 conn(0x7f54c4067180 0x7f54c405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 -- 172.21.2.104:0/3956270991 >> 172.21.2.104:0/3956270991 conn(0x7f54c40b53e0 msgr2=0x7f54c40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:29.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 -- 172.21.2.104:0/3956270991 shutdown_connections 2021-07-11T03:58:29.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.193+0000 7f54cb919700 1 -- 172.21.2.104:0/3956270991 wait complete. 2021-07-11T03:58:29.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54cb919700 1 Processor -- start 2021-07-11T03:58:29.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54cb919700 1 -- start start 2021-07-11T03:58:29.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54cb919700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54c4067180 0x7f54c4144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54cb919700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54c40a76b0 0x7f54c4144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54cb919700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54c40b1970 0x7f54c4159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54cb919700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40ae410 con 0x7f54c4067180 2021-07-11T03:58:29.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54cb919700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40ae980 con 0x7f54c40b1970 2021-07-11T03:58:29.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54cb919700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40aec20 con 0x7f54c40a76b0 2021-07-11T03:58:29.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54c96b5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54c4067180 0x7f54c4144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54c96b5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54c4067180 0x7f54c4144600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45630/0 (socket says 172.21.2.104:45630) 2021-07-11T03:58:29.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54c96b5700 1 -- 172.21.2.104:0/3341873566 learned_addr learned my addr 172.21.2.104:0/3341873566 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:29.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54c96b5700 1 -- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54c40a76b0 msgr2=0x7f54c4144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54c96b5700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54c40a76b0 0x7f54c4144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54c96b5700 1 -- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54c40b1970 msgr2=0x7f54c4159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54c96b5700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54c40b1970 0x7f54c4159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54c96b5700 1 -- 172.21.2.104:0/3341873566 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc009470 con 0x7f54c4067180 2021-07-11T03:58:29.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54c96b5700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54c4067180 0x7f54c4144600 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f54b4001110 tx=0x7f54b4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54b27fc700 1 -- 172.21.2.104:0/3341873566 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54b40072e0 con 0x7f54c4067180 2021-07-11T03:58:29.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54b27fc700 1 -- 172.21.2.104:0/3341873566 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b4007440 con 0x7f54c4067180 2021-07-11T03:58:29.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54b27fc700 1 -- 172.21.2.104:0/3341873566 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54b4005800 con 0x7f54c4067180 2021-07-11T03:58:29.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54cb919700 1 -- 172.21.2.104:0/3341873566 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c414ec30 con 0x7f54c4067180 2021-07-11T03:58:29.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54cb919700 1 -- 172.21.2.104:0/3341873566 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c4144fe0 con 0x7f54c4067180 2021-07-11T03:58:29.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54cb919700 1 -- 172.21.2.104:0/3341873566 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c40aede0 con 0x7f54c4067180 2021-07-11T03:58:29.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54b27fc700 1 -- 172.21.2.104:0/3341873566 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f54b4010070 con 0x7f54c4067180 2021-07-11T03:58:29.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54b27fc700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54a4053210 0x7f54a40556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54b27fc700 1 -- 172.21.2.104:0/3341873566 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f54b406ac40 con 0x7f54c4067180 2021-07-11T03:58:29.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.197+0000 7f54c8eb4700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54a4053210 0x7f54a40556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.201+0000 7f54c8eb4700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54a4053210 0x7f54a40556d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f54c000a000 tx=0x7f54c0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.201+0000 7f54b27fc700 1 -- 172.21.2.104:0/3341873566 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f54b4041f20 con 0x7f54c4067180 2021-07-11T03:58:29.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.309+0000 7f54cb919700 1 -- 172.21.2.104:0/3341873566 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f54c4057350 con 0x7f54c4067180 2021-07-11T03:58:29.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54b27fc700 1 -- 172.21.2.104:0/3341873566 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v20) v1 ==== 184+0+0 (secure 0 0 0) 0x7f54b4018070 con 0x7f54c4067180 2021-07-11T03:58:29.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 -- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54a4053210 msgr2=0x7f54a40556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54a4053210 0x7f54a40556d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f54c000a000 tx=0x7f54c0008040).stop 2021-07-11T03:58:29.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 -- 172.21.2.104:0/3341873566 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54c4067180 msgr2=0x7f54c4144600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54c4067180 0x7f54c4144600 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f54b4001110 tx=0x7f54b4007ad0).stop 2021-07-11T03:58:29.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 -- 172.21.2.104:0/3341873566 shutdown_connections 2021-07-11T03:58:29.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54c40b1970 0x7f54c4159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54c40a76b0 0x7f54c4144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54a4053210 0x7f54a40556d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 --2- 172.21.2.104:0/3341873566 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54c4067180 0x7f54c4144600 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 -- 172.21.2.104:0/3341873566 >> 172.21.2.104:0/3341873566 conn(0x7f54c40b53e0 msgr2=0x7f54c40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:29.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 -- 172.21.2.104:0/3341873566 shutdown_connections 2021-07-11T03:58:29.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.317+0000 7f54cb919700 1 -- 172.21.2.104:0/3341873566 wait complete. 2021-07-11T03:58:29.335 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T03:58:29.335 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:58:29.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74d029f700 1 Processor -- start 2021-07-11T03:58:29.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74d029f700 1 -- start start 2021-07-11T03:58:29.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74d029f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c80b1970 0x7f74c805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74d029f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c80a7a90 con 0x7f74c8067180 2021-07-11T03:58:29.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74d029f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c8147620 con 0x7f74c80a76b0 2021-07-11T03:58:29.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74d029f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81478c0 con 0x7f74c80b1970 2021-07-11T03:58:29.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd83a700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f74c8067180 0x7f74c805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42408/0 (socket says 172.21.2.104:42408) 2021-07-11T03:58:29.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd83a700 1 -- 172.21.2.104:0/2848158034 learned_addr learned my addr 172.21.2.104:0/2848158034 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:29.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3312923797 0 0) 0x7f74c80a7a90 con 0x7f74c8067180 2021-07-11T03:58:29.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74b0003260 con 0x7f74c8067180 2021-07-11T03:58:29.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74ce83c700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c80b1970 0x7f74c805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2805177276 0 0) 0x7f74b0003260 con 0x7f74c8067180 2021-07-11T03:58:29.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c80b1970 msgr2=0x7f74c805fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c80b1970 0x7f74c805fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 >> v1:172.21.2.105:6789/0 conn(0x7f74c80a76b0 legacy=0x7f74c8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c8147b60 con 0x7f74c8067180 2021-07-11T03:58:29.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f74bc001660 con 0x7f74c8067180 2021-07-11T03:58:29.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 >> v1:172.21.2.104:6789/0 conn(0x7f74c8067180 legacy=0x7f74c805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74b0003800 0x7f74b0003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c80b1970 0x7f74b0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74b0004a40 0x7f74b0004ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81478c0 con 0x7f74b0004a40 2021-07-11T03:58:29.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b000b8e0 con 0x7f74b0003800 2021-07-11T03:58:29.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b000ba50 con 0x7f74c80b1970 2021-07-11T03:58:29.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74ce83c700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74b0004a40 0x7f74b0004ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74ce03b700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74b0003800 0x7f74b0003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74cd83a700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c80b1970 0x7f74b0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74ce83c700 1 -- 172.21.2.104:0/2848158034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c80b1970 msgr2=0x7f74b0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74ce83c700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c80b1970 0x7f74b0004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74ce83c700 1 -- 172.21.2.104:0/2848158034 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74b0003800 msgr2=0x7f74b0003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74ce83c700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74b0003800 0x7f74b0003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.381+0000 7f74ce83c700 1 -- 172.21.2.104:0/2848158034 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c8147b60 con 0x7f74b0004a40 2021-07-11T03:58:29.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74ce83c700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74b0004a40 0x7f74b0004ec0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f74c000a000 tx=0x7f74c0008040).ready entity=mon.0 client_cookie=18f1242f1b38ddd2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c000c070 con 0x7f74b0004a40 2021-07-11T03:58:29.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c000e620 con 0x7f74b0004a40 2021-07-11T03:58:29.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd039700 1 -- 172.21.2.104:0/2848158034 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74c000e980 con 0x7f74b0004a40 2021-07-11T03:58:29.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 -- 172.21.2.104:0/2848158034 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74b0004a40 msgr2=0x7f74b0004ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 --2- 172.21.2.104:0/2848158034 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74b0004a40 0x7f74b0004ec0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f74c000a000 tx=0x7f74c0008040).stop 2021-07-11T03:58:29.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74ce83c700 1 -- 172.21.2.104:0/2848158034 reap_dead start 2021-07-11T03:58:29.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 -- 172.21.2.104:0/2848158034 shutdown_connections 2021-07-11T03:58:29.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 -- 172.21.2.104:0/2848158034 >> 172.21.2.104:0/2848158034 conn(0x7f74c80b53e0 msgr2=0x7f74c80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:29.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 -- 172.21.2.104:0/2848158034 shutdown_connections 2021-07-11T03:58:29.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 -- 172.21.2.104:0/2848158034 wait complete. 2021-07-11T03:58:29.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 Processor -- start 2021-07-11T03:58:29.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 -- start start 2021-07-11T03:58:29.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74c80ad1f0 0x7f74c80ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74c8156430 0x7f74c81568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c8144600 0x7f74c8144a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80ad5d0 con 0x7f74c8156430 2021-07-11T03:58:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80aeaa0 con 0x7f74c80ad1f0 2021-07-11T03:58:29.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80ae120 con 0x7f74c8144600 2021-07-11T03:58:29.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd83a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74c8156430 0x7f74c81568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd83a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74c8156430 0x7f74c81568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45652/0 (socket says 172.21.2.104:45652) 2021-07-11T03:58:29.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd83a700 1 -- 172.21.2.104:0/1307522422 learned_addr learned my addr 172.21.2.104:0/1307522422 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:29.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74ce03b700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74c80ad1f0 0x7f74c80ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74ce83c700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c8144600 0x7f74c8144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd83a700 1 -- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c8144600 msgr2=0x7f74c8144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd83a700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c8144600 0x7f74c8144a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd83a700 1 -- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74c80ad1f0 msgr2=0x7f74c80ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd83a700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74c80ad1f0 0x7f74c80ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd83a700 1 -- 172.21.2.104:0/1307522422 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80aed00 con 0x7f74c8156430 2021-07-11T03:58:29.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74cd83a700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74c8156430 0x7f74c81568b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f74bc00d000 tx=0x7f74bc001e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74bb7fe700 1 -- 172.21.2.104:0/1307522422 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74bc011850 con 0x7f74c8156430 2021-07-11T03:58:29.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74bb7fe700 1 -- 172.21.2.104:0/1307522422 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc0119b0 con 0x7f74c8156430 2021-07-11T03:58:29.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 -- 172.21.2.104:0/1307522422 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c80aefa0 con 0x7f74c8156430 2021-07-11T03:58:29.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74bb7fe700 1 -- 172.21.2.104:0/1307522422 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f74bc011d10 con 0x7f74c8156430 2021-07-11T03:58:29.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.385+0000 7f74d029f700 1 -- 172.21.2.104:0/1307522422 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c80af440 con 0x7f74c8156430 2021-07-11T03:58:29.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.389+0000 7f74d029f700 1 -- 172.21.2.104:0/1307522422 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c8057350 con 0x7f74c8156430 2021-07-11T03:58:29.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.389+0000 7f74bb7fe700 1 -- 172.21.2.104:0/1307522422 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f74bc01b030 con 0x7f74c8156430 2021-07-11T03:58:29.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.389+0000 7f74bb7fe700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f74a8053350 0x7f74a8055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.389+0000 7f74bb7fe700 1 -- 172.21.2.104:0/1307522422 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f74bc06b140 con 0x7f74c8156430 2021-07-11T03:58:29.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.389+0000 7f74ce03b700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f74a8053350 0x7f74a8055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.389+0000 7f74ce03b700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f74a8053350 0x7f74a8055810 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f74c400c000 tx=0x7f74c4007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.389+0000 7f74bb7fe700 1 -- 172.21.2.104:0/1307522422 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f74bc042370 con 0x7f74c8156430 2021-07-11T03:58:29.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.497+0000 7f74d029f700 1 -- 172.21.2.104:0/1307522422 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74c80607d0 con 0x7f74c8156430 2021-07-11T03:58:29.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.497+0000 7f74bb7fe700 1 -- 172.21.2.104:0/1307522422 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f74bc019030 con 0x7f74c8156430 2021-07-11T03:58:29.507 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:29.508 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/2908869555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2908869555},{"type":"v1","addr":"172.21.2.105:6835","nonce":2908869555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/1110128427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1110128427},{"type":"v1","addr":"172.21.2.104:6833","nonce":1110128427}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:58:27.556328+0000","modified":"2021-07-11T03:58:28.559390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/225259368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":225259368},{"type":"v1","addr":"172.21.2.104:6835","nonce":225259368}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-07-11T03:58:29.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.497+0000 7f74d029f700 1 -- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f74a8053350 msgr2=0x7f74a8055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.501+0000 7f74d029f700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f74a8053350 0x7f74a8055810 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f74c400c000 tx=0x7f74c4007a00).stop 2021-07-11T03:58:29.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.501+0000 7f74d029f700 1 -- 172.21.2.104:0/1307522422 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74c8156430 msgr2=0x7f74c81568b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.501+0000 7f74d029f700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74c8156430 0x7f74c81568b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f74bc00d000 tx=0x7f74bc001e20).stop 2021-07-11T03:58:29.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.501+0000 7f74d029f700 1 -- 172.21.2.104:0/1307522422 shutdown_connections 2021-07-11T03:58:29.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.501+0000 7f74d029f700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74c8144600 0x7f74c8144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.501+0000 7f74d029f700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f74a8053350 0x7f74a8055810 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.501+0000 7f74d029f700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74c8156430 0x7f74c81568b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.501+0000 7f74d029f700 1 --2- 172.21.2.104:0/1307522422 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74c80ad1f0 0x7f74c80ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.501+0000 7f74d029f700 1 -- 172.21.2.104:0/1307522422 >> 172.21.2.104:0/1307522422 conn(0x7f74c80b53e0 msgr2=0x7f74c8148810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:29.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.501+0000 7f74d029f700 1 -- 172.21.2.104:0/1307522422 shutdown_connections 2021-07-11T03:58:29.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.501+0000 7f74d029f700 1 -- 172.21.2.104:0/1307522422 wait complete. 2021-07-11T03:58:29.511 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 27 2021-07-11T03:58:29.816 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:58:27.556328+0000', 'modified': '2021-07-11T03:58:28.559390+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5952}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5952': {'gid': 5952, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/225259368', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 225259368}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 225259368}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:58:29.817 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:58:29.818 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:58:29.883 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":5952,"want_state":"up:active","state":"up:active","rank_uptime":2.3168618900000002,"mdsmap_epoch":27,"osdmap_epoch":66,"osdmap_epoch_barrier":65,"uptime":6.3922671180000004} 2021-07-11T03:58:29.884 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 5952, "mdsmap_epoch": 27, "osdmap_epoch": 66, "osdmap_epoch_barrier": 65, "rank_uptime": 2.31686189, "state": "up:active", "uptime": 6.392267118, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:58:29.885 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:58:29.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 Processor -- start 2021-07-11T03:58:29.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 -- start start 2021-07-11T03:58:29.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80b0e70 0x7f9fa814b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa80b3620 con 0x7f9fa80b1840 2021-07-11T03:58:29.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa81436f0 con 0x7f9fa80b3240 2021-07-11T03:58:29.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8065ca0 con 0x7f9fa80b0e70 2021-07-11T03:58:29.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa6ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f9fa80b1840 0x7f9fa8142890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42428/0 (socket says 172.21.2.104:42428) 2021-07-11T03:58:29.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa6ffd700 1 -- 172.21.2.104:0/1458773696 learned_addr learned my addr 172.21.2.104:0/1458773696 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:29.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611454825 0 0) 0x7f9fa80b3620 con 0x7f9fa80b1840 2021-07-11T03:58:29.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f88003260 con 0x7f9fa80b1840 2021-07-11T03:58:29.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa7fff700 1 --2- 172.21.2.104:0/1458773696 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80b0e70 0x7f9fa814b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 270746558 0 0) 0x7f9f88003260 con 0x7f9fa80b1840 2021-07-11T03:58:29.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80b0e70 msgr2=0x7f9fa814b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 --2- 172.21.2.104:0/1458773696 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80b0e70 0x7f9fa814b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 >> v1:172.21.2.105:6789/0 conn(0x7f9fa80b3240 legacy=0x7f9fa8066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa8065f40 con 0x7f9fa80b1840 2021-07-11T03:58:29.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f9f90001680 con 0x7f9fa80b1840 2021-07-11T03:58:29.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 >> v1:172.21.2.104:6789/0 conn(0x7f9fa80b1840 legacy=0x7f9fa8142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 --2- 172.21.2.104:0/1458773696 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80b0e70 0x7f9f88003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 --2- 172.21.2.104:0/1458773696 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9f88003e10 0x7f9f88004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 --2- 172.21.2.104:0/1458773696 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9f88004b10 0x7f9f88004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8065ca0 con 0x7f9f88003e10 2021-07-11T03:58:29.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8800b960 con 0x7f9f88004b10 2021-07-11T03:58:29.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f8800bb20 con 0x7f9fa80b0e70 2021-07-11T03:58:29.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa6ffd700 1 --2- 172.21.2.104:0/1458773696 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9f88003e10 0x7f9f88004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa6ffd700 1 -- 172.21.2.104:0/1458773696 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80b0e70 msgr2=0x7f9f88003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa6ffd700 1 --2- 172.21.2.104:0/1458773696 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80b0e70 0x7f9f88003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa6ffd700 1 -- 172.21.2.104:0/1458773696 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9f88004b10 msgr2=0x7f9f88004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:29.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa6ffd700 1 --2- 172.21.2.104:0/1458773696 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9f88004b10 0x7f9f88004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:29.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa6ffd700 1 -- 172.21.2.104:0/1458773696 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f88003260 con 0x7f9f88003e10 2021-07-11T03:58:29.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa6ffd700 1 --2- 172.21.2.104:0/1458773696 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9f88003e10 0x7f9f88004500 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f9fa80b7520 tx=0x7f9f90001d80).ready entity=mon.0 client_cookie=8ac24f640acd78c8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f900070d0 con 0x7f9f88003e10 2021-07-11T03:58:29.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f90007230 con 0x7f9f88003e10 2021-07-11T03:58:29.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa67fc700 1 -- 172.21.2.104:0/1458773696 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9f9000c8c0 con 0x7f9f88003e10 2021-07-11T03:58:29.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 -- 172.21.2.104:0/1458773696 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9f88003e10 msgr2=0x7f9f88004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 --2- 172.21.2.104:0/1458773696 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9f88003e10 0x7f9f88004500 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f9fa80b7520 tx=0x7f9f90001d80).stop 2021-07-11T03:58:29.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fa7fff700 1 -- 172.21.2.104:0/1458773696 reap_dead start 2021-07-11T03:58:29.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 -- 172.21.2.104:0/1458773696 shutdown_connections 2021-07-11T03:58:29.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 -- 172.21.2.104:0/1458773696 >> 172.21.2.104:0/1458773696 conn(0x7f9fa805c490 msgr2=0x7f9fa805c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:29.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 -- 172.21.2.104:0/1458773696 shutdown_connections 2021-07-11T03:58:29.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.945+0000 7f9fae0bb700 1 -- 172.21.2.104:0/1458773696 wait complete. 2021-07-11T03:58:29.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fae0bb700 1 Processor -- start 2021-07-11T03:58:29.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fae0bb700 1 -- start start 2021-07-11T03:58:29.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fae0bb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fa80ae160 0x7f9fa80ae540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fae0bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80aec80 0x7f9fa814ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fae0bb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fa80afa70 0x7f9fa809f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fae0bb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa80af2b0 con 0x7f9fa80afa70 2021-07-11T03:58:29.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fae0bb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa814e060 con 0x7f9fa80ae160 2021-07-11T03:58:29.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fae0bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa814e300 con 0x7f9fa80aec80 2021-07-11T03:58:29.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fa80afa70 0x7f9fa809f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fa80afa70 0x7f9fa809f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45674/0 (socket says 172.21.2.104:45674) 2021-07-11T03:58:29.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa7fff700 1 -- 172.21.2.104:0/42232917 learned_addr learned my addr 172.21.2.104:0/42232917 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:29.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa77fe700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fa80ae160 0x7f9fa80ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa6ffd700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80aec80 0x7f9fa814ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa7fff700 1 -- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80aec80 msgr2=0x7f9fa814ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa7fff700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80aec80 0x7f9fa814ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa7fff700 1 -- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fa80ae160 msgr2=0x7f9fa80ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:29.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa7fff700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fa80ae160 0x7f9fa80ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:29.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa7fff700 1 -- 172.21.2.104:0/42232917 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa814e5a0 con 0x7f9fa80afa70 2021-07-11T03:58:29.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa77fe700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fa80ae160 0x7f9fa80ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:58:29.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa7fff700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fa80afa70 0x7f9fa809f800 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f9fa0000b60 tx=0x7f9fa0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa4ff9700 1 -- 172.21.2.104:0/42232917 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fa000c070 con 0x7f9fa80afa70 2021-07-11T03:58:29.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa4ff9700 1 -- 172.21.2.104:0/42232917 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa00133f0 con 0x7f9fa80afa70 2021-07-11T03:58:29.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fae0bb700 1 -- 172.21.2.104:0/42232917 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa80a9d50 con 0x7f9fa80afa70 2021-07-11T03:58:29.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa4ff9700 1 -- 172.21.2.104:0/42232917 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9fa0013750 con 0x7f9fa80afa70 2021-07-11T03:58:29.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fae0bb700 1 -- 172.21.2.104:0/42232917 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa80b0110 con 0x7f9fa80afa70 2021-07-11T03:58:29.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fae0bb700 1 -- 172.21.2.104:0/42232917 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa80af060 con 0x7f9fa80afa70 2021-07-11T03:58:29.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa4ff9700 1 -- 172.21.2.104:0/42232917 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9fa00138b0 con 0x7f9fa80afa70 2021-07-11T03:58:29.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa4ff9700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9f8c053330 0x7f9f8c0557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:29.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa4ff9700 1 -- 172.21.2.104:0/42232917 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9fa006ac10 con 0x7f9fa80afa70 2021-07-11T03:58:29.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.949+0000 7f9fa77fe700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9f8c053330 0x7f9f8c0557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:29.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.953+0000 7f9fa77fe700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9f8c053330 0x7f9f8c0557f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f9f98008000 tx=0x7f9f98009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:29.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:29.953+0000 7f9fa4ff9700 1 -- 172.21.2.104:0/42232917 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9fa0041f50 con 0x7f9fa80afa70 2021-07-11T03:58:30.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.033+0000 7f9fae0bb700 1 -- 172.21.2.104:0/42232917 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9fa80a9f40 con 0x7f9fa80afa70 2021-07-11T03:58:30.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fa4ff9700 1 -- 172.21.2.104:0/42232917 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f9fa0025ec0 con 0x7f9fa80afa70 2021-07-11T03:58:30.044 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:30.044 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":66,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:58:28.560561+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:58:25.543830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2021-07-11T03:58:26.545938+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:58:30.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 -- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9f8c053330 msgr2=0x7f9f8c0557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9f8c053330 0x7f9f8c0557f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f9f98008000 tx=0x7f9f98009040).stop 2021-07-11T03:58:30.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 -- 172.21.2.104:0/42232917 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fa80afa70 msgr2=0x7f9fa809f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fa80afa70 0x7f9fa809f800 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f9fa0000b60 tx=0x7f9fa0008040).stop 2021-07-11T03:58:30.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 -- 172.21.2.104:0/42232917 shutdown_connections 2021-07-11T03:58:30.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9f8c053330 0x7f9f8c0557f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9fa80afa70 0x7f9fa809f800 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9fa80aec80 0x7f9fa814ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 --2- 172.21.2.104:0/42232917 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9fa80ae160 0x7f9fa80ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 -- 172.21.2.104:0/42232917 >> 172.21.2.104:0/42232917 conn(0x7f9fa805c490 msgr2=0x7f9fa80b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:30.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 -- 172.21.2.104:0/42232917 shutdown_connections 2021-07-11T03:58:30.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.037+0000 7f9fae0bb700 1 -- 172.21.2.104:0/42232917 wait complete. 2021-07-11T03:58:30.054 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:58:25.543830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:58:26.545938+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:58:30.055 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:58:30.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.149+0000 7f3351e56700 1 Processor -- start 2021-07-11T03:58:30.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.149+0000 7f3351e56700 1 -- start start 2021-07-11T03:58:30.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f334c144440 0x7f334c144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c0604e0 con 0x7f334c060100 2021-07-11T03:58:30.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c0b23b0 con 0x7f334c065420 2021-07-11T03:58:30.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c065c80 con 0x7f334c144440 2021-07-11T03:58:30.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334b7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f334c060100 0x7f334c064d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42446/0 (socket says 172.21.2.104:42446) 2021-07-11T03:58:30.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334b7fe700 1 -- 172.21.2.104:0/1854473396 learned_addr learned my addr 172.21.2.104:0/1854473396 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:30.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 --2- 172.21.2.104:0/1854473396 >> v2:172.21.2.104:3300/0 conn(0x7f334c144440 0x7f334c144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 -- 172.21.2.104:0/1854473396 >> v1:172.21.2.105:6789/0 conn(0x7f334c065420 legacy=0x7f334c14b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:30.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 -- 172.21.2.104:0/1854473396 >> v1:172.21.2.104:6789/0 conn(0x7f334c060100 legacy=0x7f334c064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 -- 172.21.2.104:0/1854473396 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c0b2650 con 0x7f334c144440 2021-07-11T03:58:30.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 --2- 172.21.2.104:0/1854473396 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f334c144440 0x7f334c144970 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f334c05f160 tx=0x7f3334006df0).ready entity=mon.0 client_cookie=e18449fae21c4b5e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334a7fc700 1 -- 172.21.2.104:0/1854473396 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333400d5c0 con 0x7f334c144440 2021-07-11T03:58:30.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334a7fc700 1 -- 172.21.2.104:0/1854473396 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3334000e90 con 0x7f334c144440 2021-07-11T03:58:30.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334a7fc700 1 -- 172.21.2.104:0/1854473396 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333400d910 con 0x7f334c144440 2021-07-11T03:58:30.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- 172.21.2.104:0/1854473396 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f334c144440 msgr2=0x7f334c144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 --2- 172.21.2.104:0/1854473396 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f334c144440 0x7f334c144970 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f334c05f160 tx=0x7f3334006df0).stop 2021-07-11T03:58:30.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- 172.21.2.104:0/1854473396 shutdown_connections 2021-07-11T03:58:30.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 --2- 172.21.2.104:0/1854473396 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f334c144440 0x7f334c144970 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- 172.21.2.104:0/1854473396 >> 172.21.2.104:0/1854473396 conn(0x7f334c05cf60 msgr2=0x7f334c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:30.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- 172.21.2.104:0/1854473396 shutdown_connections 2021-07-11T03:58:30.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- 172.21.2.104:0/1854473396 wait complete. 2021-07-11T03:58:30.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 Processor -- start 2021-07-11T03:58:30.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- start start 2021-07-11T03:58:30.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f334c060100 0x7f334c0b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f334c065420 0x7f334c0ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f334c0aec10 0x7f334c13a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c13e100 con 0x7f334c065420 2021-07-11T03:58:30.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c13b9b0 con 0x7f334c0aec10 2021-07-11T03:58:30.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3351e56700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0afc40 con 0x7f334c060100 2021-07-11T03:58:30.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f334c065420 0x7f334c0ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f334c065420 0x7f334c0ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45686/0 (socket says 172.21.2.104:45686) 2021-07-11T03:58:30.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 -- 172.21.2.104:0/3430519641 learned_addr learned my addr 172.21.2.104:0/3430519641 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:30.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 -- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f334c060100 msgr2=0x7f334c0b64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f334c060100 0x7f334c0b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 -- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f334c0aec10 msgr2=0x7f334c13a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f334c0aec10 0x7f334c13a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 -- 172.21.2.104:0/3430519641 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333400b180 con 0x7f334c065420 2021-07-11T03:58:30.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f334affd700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f334c065420 0x7f334c0ae720 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f334c05f160 tx=0x7f3334012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.153+0000 7f3348ff9700 1 -- 172.21.2.104:0/3430519641 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3334002f50 con 0x7f334c065420 2021-07-11T03:58:30.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.157+0000 7f3348ff9700 1 -- 172.21.2.104:0/3430519641 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3334017030 con 0x7f334c065420 2021-07-11T03:58:30.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.157+0000 7f3351e56700 1 -- 172.21.2.104:0/3430519641 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c0afee0 con 0x7f334c065420 2021-07-11T03:58:30.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.157+0000 7f3348ff9700 1 -- 172.21.2.104:0/3430519641 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f333401d470 con 0x7f334c065420 2021-07-11T03:58:30.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.157+0000 7f3351e56700 1 -- 172.21.2.104:0/3430519641 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c0b7100 con 0x7f334c065420 2021-07-11T03:58:30.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.157+0000 7f3351e56700 1 -- 172.21.2.104:0/3430519641 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c0b0070 con 0x7f334c065420 2021-07-11T03:58:30.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.157+0000 7f3348ff9700 1 -- 172.21.2.104:0/3430519641 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3334024030 con 0x7f334c065420 2021-07-11T03:58:30.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.157+0000 7f3348ff9700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3330053330 0x7f33300557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.157+0000 7f3348ff9700 1 -- 172.21.2.104:0/3430519641 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f333406c2e0 con 0x7f334c065420 2021-07-11T03:58:30.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.157+0000 7f334b7fe700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3330053330 0x7f33300557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.157+0000 7f334b7fe700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3330053330 0x7f33300557f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f334c05d460 tx=0x7f333c007820).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.157+0000 7f3348ff9700 1 -- 172.21.2.104:0/3430519641 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3334043520 con 0x7f334c065420 2021-07-11T03:58:30.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 -- 172.21.2.104:0/3430519641 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f334c057350 con 0x7f334c065420 2021-07-11T03:58:30.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3348ff9700 1 -- 172.21.2.104:0/3430519641 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v66) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3334046b40 con 0x7f334c065420 2021-07-11T03:58:30.248 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:58:30.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 -- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3330053330 msgr2=0x7f33300557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3330053330 0x7f33300557f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f334c05d460 tx=0x7f333c007820).stop 2021-07-11T03:58:30.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 -- 172.21.2.104:0/3430519641 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f334c065420 msgr2=0x7f334c0ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f334c065420 0x7f334c0ae720 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f334c05f160 tx=0x7f3334012000).stop 2021-07-11T03:58:30.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 -- 172.21.2.104:0/3430519641 shutdown_connections 2021-07-11T03:58:30.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f334c0aec10 0x7f334c13a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3330053330 0x7f33300557f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f334c065420 0x7f334c0ae720 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 --2- 172.21.2.104:0/3430519641 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f334c060100 0x7f334c0b64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 -- 172.21.2.104:0/3430519641 >> 172.21.2.104:0/3430519641 conn(0x7f334c05cf60 msgr2=0x7f334c140240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:30.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 -- 172.21.2.104:0/3430519641 shutdown_connections 2021-07-11T03:58:30.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.241+0000 7f3351e56700 1 -- 172.21.2.104:0/3430519641 wait complete. 2021-07-11T03:58:30.260 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:58:30.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf5296b700 1 Processor -- start 2021-07-11T03:58:30.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf5296b700 1 -- start start 2021-07-11T03:58:30.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf5296b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fdf4c0ad040 0x7fdf4c0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf5296b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fdf4c0afaf0 0x7fdf4c13f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf5296b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4c0b5f40 con 0x7fdf4c05d760 2021-07-11T03:58:30.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf5296b700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4c0ad420 con 0x7fdf4c0afaf0 2021-07-11T03:58:30.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf5296b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf4c0aeb60 con 0x7fdf4c0ad040 2021-07-11T03:58:30.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf50f08700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fdf4c0afaf0 0x7fdf4c13f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf4b7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fdf4c05d760 0x7fdf4c13f070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42462/0 (socket says 172.21.2.104:42462) 2021-07-11T03:58:30.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf4b7fe700 1 -- 172.21.2.104:0/219570676 learned_addr learned my addr 172.21.2.104:0/219570676 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:30.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf50f08700 1 -- 172.21.2.104:0/219570676 >> v2:172.21.2.105:3300/0 conn(0x7fdf4c0ad040 msgr2=0x7fdf4c0b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:30.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf50f08700 1 --2- 172.21.2.104:0/219570676 >> v2:172.21.2.105:3300/0 conn(0x7fdf4c0ad040 0x7fdf4c0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf50f08700 1 -- 172.21.2.104:0/219570676 >> v1:172.21.2.104:6789/0 conn(0x7fdf4c05d760 legacy=0x7fdf4c13f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf50f08700 1 -- 172.21.2.104:0/219570676 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c0aee00 con 0x7fdf4c0afaf0 2021-07-11T03:58:30.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf50f08700 1 --2- 172.21.2.104:0/219570676 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf4c0afaf0 0x7fdf4c13f780 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fdf440032f0 tx=0x7fdf440091d0).ready entity=mon.0 client_cookie=d6ea3d92fd3b726e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf4affd700 1 -- 172.21.2.104:0/219570676 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf44009ea0 con 0x7fdf4c0afaf0 2021-07-11T03:58:30.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf4affd700 1 -- 172.21.2.104:0/219570676 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4400b870 con 0x7fdf4c0afaf0 2021-07-11T03:58:30.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.305+0000 7fdf4affd700 1 -- 172.21.2.104:0/219570676 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf4400b9e0 con 0x7fdf4c0afaf0 2021-07-11T03:58:30.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- 172.21.2.104:0/219570676 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf4c0afaf0 msgr2=0x7fdf4c13f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 --2- 172.21.2.104:0/219570676 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf4c0afaf0 0x7fdf4c13f780 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fdf440032f0 tx=0x7fdf440091d0).stop 2021-07-11T03:58:30.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- 172.21.2.104:0/219570676 shutdown_connections 2021-07-11T03:58:30.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 --2- 172.21.2.104:0/219570676 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf4c0afaf0 0x7fdf4c13f780 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 --2- 172.21.2.104:0/219570676 >> v2:172.21.2.105:3300/0 conn(0x7fdf4c0ad040 0x7fdf4c0b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- 172.21.2.104:0/219570676 >> 172.21.2.104:0/219570676 conn(0x7fdf4c05c490 msgr2=0x7fdf4c05c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:30.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- 172.21.2.104:0/219570676 shutdown_connections 2021-07-11T03:58:30.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- 172.21.2.104:0/219570676 wait complete. 2021-07-11T03:58:30.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 Processor -- start 2021-07-11T03:58:30.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- start start 2021-07-11T03:58:30.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf4c05d760 0x7fdf4c0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf4c0ad040 0x7fdf4c1424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf4c0afaf0 0x7fdf4c140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c143360 con 0x7fdf4c05d760 2021-07-11T03:58:30.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c13def0 con 0x7fdf4c0ad040 2021-07-11T03:58:30.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c1392f0 con 0x7fdf4c0afaf0 2021-07-11T03:58:30.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf4bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf4c05d760 0x7fdf4c0b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf4bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf4c05d760 0x7fdf4c0b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45698/0 (socket says 172.21.2.104:45698) 2021-07-11T03:58:30.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf4bfff700 1 -- 172.21.2.104:0/2451503152 learned_addr learned my addr 172.21.2.104:0/2451503152 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:30.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf4bfff700 1 -- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf4c0afaf0 msgr2=0x7fdf4c140320 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:58:30.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf4bfff700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf4c0afaf0 0x7fdf4c140320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf4bfff700 1 -- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf4c0ad040 msgr2=0x7fdf4c1424f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:58:30.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf4bfff700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf4c0ad040 0x7fdf4c1424f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf4bfff700 1 -- 172.21.2.104:0/2451503152 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf44009470 con 0x7fdf4c05d760 2021-07-11T03:58:30.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf4bfff700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf4c05d760 0x7fdf4c0b5030 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fdf34002f70 tx=0x7fdf34006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf497fa700 1 -- 172.21.2.104:0/2451503152 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf34013460 con 0x7fdf4c05d760 2021-07-11T03:58:30.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf497fa700 1 -- 172.21.2.104:0/2451503152 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf340135c0 con 0x7fdf4c05d760 2021-07-11T03:58:30.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- 172.21.2.104:0/2451503152 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4c13cee0 con 0x7fdf4c05d760 2021-07-11T03:58:30.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf497fa700 1 -- 172.21.2.104:0/2451503152 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf34013920 con 0x7fdf4c05d760 2021-07-11T03:58:30.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- 172.21.2.104:0/2451503152 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c12c490 con 0x7fdf4c05d760 2021-07-11T03:58:30.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf5296b700 1 -- 172.21.2.104:0/2451503152 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4c139460 con 0x7fdf4c05d760 2021-07-11T03:58:30.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf497fa700 1 -- 172.21.2.104:0/2451503152 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fdf3401a060 con 0x7fdf4c05d760 2021-07-11T03:58:30.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf497fa700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf30053350 0x7fdf30055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.309+0000 7fdf497fa700 1 -- 172.21.2.104:0/2451503152 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdf3406afc0 con 0x7fdf4c05d760 2021-07-11T03:58:30.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.313+0000 7fdf4b7fe700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf30053350 0x7fdf30055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.313+0000 7fdf4b7fe700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf30053350 0x7fdf30055810 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fdf4c062900 tx=0x7fdf3c007790).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.313+0000 7fdf497fa700 1 -- 172.21.2.104:0/2451503152 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fdf4c139460 con 0x7fdf4c05d760 2021-07-11T03:58:30.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 -- 172.21.2.104:0/2451503152 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdf4c139460 con 0x7fdf4c05d760 2021-07-11T03:58:30.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf497fa700 1 -- 172.21.2.104:0/2451503152 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v66) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdf4c139460 con 0x7fdf4c05d760 2021-07-11T03:58:30.405 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:58:30.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 -- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf30053350 msgr2=0x7fdf30055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf30053350 0x7fdf30055810 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7fdf4c062900 tx=0x7fdf3c007790).stop 2021-07-11T03:58:30.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 -- 172.21.2.104:0/2451503152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf4c05d760 msgr2=0x7fdf4c0b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf4c05d760 0x7fdf4c0b5030 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fdf34002f70 tx=0x7fdf34006ec0).stop 2021-07-11T03:58:30.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 -- 172.21.2.104:0/2451503152 shutdown_connections 2021-07-11T03:58:30.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf4c0afaf0 0x7fdf4c140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf4c0ad040 0x7fdf4c1424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf30053350 0x7fdf30055810 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 --2- 172.21.2.104:0/2451503152 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf4c05d760 0x7fdf4c0b5030 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 -- 172.21.2.104:0/2451503152 >> 172.21.2.104:0/2451503152 conn(0x7fdf4c05c490 msgr2=0x7fdf4c13a660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:30.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 -- 172.21.2.104:0/2451503152 shutdown_connections 2021-07-11T03:58:30.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.397+0000 7fdf5296b700 1 -- 172.21.2.104:0/2451503152 wait complete. 2021-07-11T03:58:30.412 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:58:30.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f658bd700 1 Processor -- start 2021-07-11T03:58:30.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f658bd700 1 -- start start 2021-07-11T03:58:30.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f658bd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6f600b12d0 0x7f6f600affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f658bd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f600b22a0 0x7f6f60141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f658bd700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f600a77d0 con 0x7f6f600b1ec0 2021-07-11T03:58:30.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f658bd700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f600b1740 con 0x7f6f600b12d0 2021-07-11T03:58:30.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f658bd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600b0650 con 0x7f6f600b22a0 2021-07-11T03:58:30.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5effd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6f600b1ec0 0x7f6f600a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42474/0 (socket says 172.21.2.104:42474) 2021-07-11T03:58:30.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5effd700 1 -- 172.21.2.104:0/2180190166 learned_addr learned my addr 172.21.2.104:0/2180190166 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:30.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057663669 0 0) 0x7f6f600a77d0 con 0x7f6f600b1ec0 2021-07-11T03:58:30.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f40003260 con 0x7f6f600b1ec0 2021-07-11T03:58:30.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2666500183 0 0) 0x7f6f40003260 con 0x7f6f600b1ec0 2021-07-11T03:58:30.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f600b22a0 msgr2=0x7f6f60141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:30.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 --2- 172.21.2.104:0/2180190166 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f600b22a0 0x7f6f60141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 >> v2:172.21.2.105:3300/0 conn(0x7f6f600b12d0 msgr2=0x7f6f600affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:30.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 --2- 172.21.2.104:0/2180190166 >> v2:172.21.2.105:3300/0 conn(0x7f6f600b12d0 0x7f6f600affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f600b08f0 con 0x7f6f600b1ec0 2021-07-11T03:58:30.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f6f50001660 con 0x7f6f600b1ec0 2021-07-11T03:58:30.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 >> v1:172.21.2.104:6789/0 conn(0x7f6f600b1ec0 legacy=0x7f6f600a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 --2- 172.21.2.104:0/2180190166 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f400037d0 0x7f6f40003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 --2- 172.21.2.104:0/2180190166 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f40004500 0x7f6f40004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 --2- 172.21.2.104:0/2180190166 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f40005070 0x7f6f4000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600b1740 con 0x7f6f40004500 2021-07-11T03:58:30.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600b0650 con 0x7f6f40005070 2021-07-11T03:58:30.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4000bf00 con 0x7f6f400037d0 2021-07-11T03:58:30.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5e7fc700 1 --2- 172.21.2.104:0/2180190166 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f40004500 0x7f6f40004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5e7fc700 1 -- 172.21.2.104:0/2180190166 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f400037d0 msgr2=0x7f6f40003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:30.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5e7fc700 1 --2- 172.21.2.104:0/2180190166 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f400037d0 0x7f6f40003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5e7fc700 1 -- 172.21.2.104:0/2180190166 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f40005070 msgr2=0x7f6f4000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:30.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.457+0000 7f6f5e7fc700 1 --2- 172.21.2.104:0/2180190166 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f40005070 0x7f6f4000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5e7fc700 1 -- 172.21.2.104:0/2180190166 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f40003260 con 0x7f6f40004500 2021-07-11T03:58:30.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5e7fc700 1 --2- 172.21.2.104:0/2180190166 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f40004500 0x7f6f40004980 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f6f48002f70 tx=0x7f6f48006ca0).ready entity=mon.0 client_cookie=d7cfd80dbda70be8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f48012410 con 0x7f6f40004500 2021-07-11T03:58:30.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f48012570 con 0x7f6f40004500 2021-07-11T03:58:30.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5dffb700 1 -- 172.21.2.104:0/2180190166 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f480128d0 con 0x7f6f40004500 2021-07-11T03:58:30.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- 172.21.2.104:0/2180190166 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f40004500 msgr2=0x7f6f40004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 --2- 172.21.2.104:0/2180190166 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f40004500 0x7f6f40004980 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f6f48002f70 tx=0x7f6f48006ca0).stop 2021-07-11T03:58:30.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5f7fe700 1 -- 172.21.2.104:0/2180190166 reap_dead start 2021-07-11T03:58:30.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- 172.21.2.104:0/2180190166 shutdown_connections 2021-07-11T03:58:30.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- 172.21.2.104:0/2180190166 >> 172.21.2.104:0/2180190166 conn(0x7f6f6005f6b0 msgr2=0x7f6f6005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:30.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- 172.21.2.104:0/2180190166 shutdown_connections 2021-07-11T03:58:30.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- 172.21.2.104:0/2180190166 wait complete. 2021-07-11T03:58:30.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 Processor -- start 2021-07-11T03:58:30.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- start start 2021-07-11T03:58:30.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f600ab0b0 0x7f6f6015ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f600b3a70 0x7f6f600b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f60148af0 0x7f6f60148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f60156c90 con 0x7f6f600b3a70 2021-07-11T03:58:30.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600ab490 con 0x7f6f600ab0b0 2021-07-11T03:58:30.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f600b5d00 con 0x7f6f60148af0 2021-07-11T03:58:30.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f600b3a70 0x7f6f600b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f600b3a70 0x7f6f600b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45718/0 (socket says 172.21.2.104:45718) 2021-07-11T03:58:30.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5e7fc700 1 -- 172.21.2.104:0/1106414570 learned_addr learned my addr 172.21.2.104:0/1106414570 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:30.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5e7fc700 1 -- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f60148af0 msgr2=0x7f6f60148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5e7fc700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f60148af0 0x7f6f60148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5e7fc700 1 -- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f600ab0b0 msgr2=0x7f6f6015ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5e7fc700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f600ab0b0 0x7f6f6015ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5e7fc700 1 -- 172.21.2.104:0/1106414570 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f600b5fa0 con 0x7f6f600b3a70 2021-07-11T03:58:30.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f5e7fc700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f600b3a70 0x7f6f600b3ed0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f6f48002f70 tx=0x7f6f48015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f57fff700 1 -- 172.21.2.104:0/1106414570 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f48011880 con 0x7f6f600b3a70 2021-07-11T03:58:30.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f57fff700 1 -- 172.21.2.104:0/1106414570 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f480119e0 con 0x7f6f600b3a70 2021-07-11T03:58:30.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- 172.21.2.104:0/1106414570 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f600b6240 con 0x7f6f600b3a70 2021-07-11T03:58:30.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- 172.21.2.104:0/1106414570 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f600b5a30 con 0x7f6f600b3a70 2021-07-11T03:58:30.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f57fff700 1 -- 172.21.2.104:0/1106414570 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f48011d40 con 0x7f6f600b3a70 2021-07-11T03:58:30.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f658bd700 1 -- 172.21.2.104:0/1106414570 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f600b3780 con 0x7f6f600b3a70 2021-07-11T03:58:30.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f57fff700 1 -- 172.21.2.104:0/1106414570 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6f4800f030 con 0x7f6f600b3a70 2021-07-11T03:58:30.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f57fff700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f440531d0 0x7f6f44055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.461+0000 7f6f57fff700 1 -- 172.21.2.104:0/1106414570 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6f4806bf90 con 0x7f6f600b3a70 2021-07-11T03:58:30.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.465+0000 7f6f5effd700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f440531d0 0x7f6f44055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.465+0000 7f6f5effd700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f440531d0 0x7f6f44055690 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f6f50001370 tx=0x7f6f600b0a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.465+0000 7f6f57fff700 1 -- 172.21.2.104:0/1106414570 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6f48043320 con 0x7f6f600b3a70 2021-07-11T03:58:30.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 -- 172.21.2.104:0/1106414570 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f600b4560 con 0x7f6f600b3a70 2021-07-11T03:58:30.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f57fff700 1 -- 172.21.2.104:0/1106414570 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6f48022020 con 0x7f6f600b3a70 2021-07-11T03:58:30.557 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:58:30.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 -- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f440531d0 msgr2=0x7f6f44055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f440531d0 0x7f6f44055690 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f6f50001370 tx=0x7f6f600b0a80).stop 2021-07-11T03:58:30.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 -- 172.21.2.104:0/1106414570 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f600b3a70 msgr2=0x7f6f600b3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f600b3a70 0x7f6f600b3ed0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f6f48002f70 tx=0x7f6f48015000).stop 2021-07-11T03:58:30.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 -- 172.21.2.104:0/1106414570 shutdown_connections 2021-07-11T03:58:30.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f60148af0 0x7f6f60148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f440531d0 0x7f6f44055690 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f600b3a70 0x7f6f600b3ed0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 --2- 172.21.2.104:0/1106414570 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f600ab0b0 0x7f6f6015ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 -- 172.21.2.104:0/1106414570 >> 172.21.2.104:0/1106414570 conn(0x7f6f6005f6b0 msgr2=0x7f6f600abad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:30.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 -- 172.21.2.104:0/1106414570 shutdown_connections 2021-07-11T03:58:30.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.549+0000 7f6f658bd700 1 -- 172.21.2.104:0/1106414570 wait complete. 2021-07-11T03:58:30.564 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:58:30.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc846965700 1 Processor -- start 2021-07-11T03:58:30.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc846965700 1 -- start start 2021-07-11T03:58:30.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc846965700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840141960 0x7fc8400affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc846965700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8400a77d0 con 0x7fc8400b12d0 2021-07-11T03:58:30.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc846965700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8400b0650 con 0x7fc8400b1ec0 2021-07-11T03:58:30.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc846965700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400b1740 con 0x7fc840141960 2021-07-11T03:58:30.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83f7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc8400b12d0 0x7fc840141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42494/0 (socket says 172.21.2.104:42494) 2021-07-11T03:58:30.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83f7fe700 1 -- 172.21.2.104:0/3118609168 learned_addr learned my addr 172.21.2.104:0/3118609168 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:30.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120332019 0 0) 0x7fc8400a77d0 con 0x7fc8400b12d0 2021-07-11T03:58:30.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc844f02700 1 --2- 172.21.2.104:0/3118609168 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840141960 0x7fc8400affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc820003260 con 0x7fc8400b12d0 2021-07-11T03:58:30.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3689922504 0 0) 0x7fc820003260 con 0x7fc8400b12d0 2021-07-11T03:58:30.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840141960 msgr2=0x7fc8400affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:30.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83effd700 1 --2- 172.21.2.104:0/3118609168 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840141960 0x7fc8400affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 >> v1:172.21.2.105:6789/0 conn(0x7fc8400b1ec0 legacy=0x7fc8400a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8400b08f0 con 0x7fc8400b12d0 2021-07-11T03:58:30.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fc830001660 con 0x7fc8400b12d0 2021-07-11T03:58:30.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 >> v1:172.21.2.104:6789/0 conn(0x7fc8400b12d0 legacy=0x7fc840141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.609+0000 7fc83effd700 1 --2- 172.21.2.104:0/3118609168 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc820003780 0x7fc820003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83effd700 1 --2- 172.21.2.104:0/3118609168 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc820004500 0x7fc820004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83effd700 1 --2- 172.21.2.104:0/3118609168 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840141960 0x7fc820004f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400b1740 con 0x7fc820004500 2021-07-11T03:58:30.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82000b830 con 0x7fc820003780 2021-07-11T03:58:30.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82000b9f0 con 0x7fc840141960 2021-07-11T03:58:30.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83f7fe700 1 --2- 172.21.2.104:0/3118609168 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc820004500 0x7fc820004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83f7fe700 1 -- 172.21.2.104:0/3118609168 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840141960 msgr2=0x7fc820004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:30.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83f7fe700 1 --2- 172.21.2.104:0/3118609168 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840141960 0x7fc820004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83f7fe700 1 -- 172.21.2.104:0/3118609168 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc820003780 msgr2=0x7fc820003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:30.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83f7fe700 1 --2- 172.21.2.104:0/3118609168 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc820003780 0x7fc820003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83f7fe700 1 -- 172.21.2.104:0/3118609168 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820003260 con 0x7fc820004500 2021-07-11T03:58:30.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83f7fe700 1 --2- 172.21.2.104:0/3118609168 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc820004500 0x7fc820004980 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fc830001370 tx=0x7fc8400b2a00).ready entity=mon.0 client_cookie=874ec6646c4d1cf server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc8300070d0 con 0x7fc820004500 2021-07-11T03:58:30.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc830007230 con 0x7fc820004500 2021-07-11T03:58:30.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83effd700 1 -- 172.21.2.104:0/3118609168 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc83000c8f0 con 0x7fc820004500 2021-07-11T03:58:30.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- 172.21.2.104:0/3118609168 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc820004500 msgr2=0x7fc820004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 --2- 172.21.2.104:0/3118609168 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc820004500 0x7fc820004980 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7fc830001370 tx=0x7fc8400b2a00).stop 2021-07-11T03:58:30.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc844f02700 1 -- 172.21.2.104:0/3118609168 reap_dead start 2021-07-11T03:58:30.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- 172.21.2.104:0/3118609168 shutdown_connections 2021-07-11T03:58:30.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- 172.21.2.104:0/3118609168 >> 172.21.2.104:0/3118609168 conn(0x7fc84005f6b0 msgr2=0x7fc84005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:30.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- 172.21.2.104:0/3118609168 shutdown_connections 2021-07-11T03:58:30.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- 172.21.2.104:0/3118609168 wait complete. 2021-07-11T03:58:30.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 Processor -- start 2021-07-11T03:58:30.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- start start 2021-07-11T03:58:30.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc84013f580 0x7fc8400abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840149000 0x7fc840149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc84015a4b0 0x7fc84015a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8400aa240 con 0x7fc84013f580 2021-07-11T03:58:30.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84014a8b0 con 0x7fc84015a4b0 2021-07-11T03:58:30.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc84013f960 con 0x7fc840149000 2021-07-11T03:58:30.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc84013f580 0x7fc8400abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc84013f580 0x7fc8400abb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45738/0 (socket says 172.21.2.104:45738) 2021-07-11T03:58:30.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83ffff700 1 -- 172.21.2.104:0/1403149873 learned_addr learned my addr 172.21.2.104:0/1403149873 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:30.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83f7fe700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840149000 0x7fc840149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83ffff700 1 -- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840149000 msgr2=0x7fc840149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83ffff700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840149000 0x7fc840149460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83ffff700 1 -- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc84015a4b0 msgr2=0x7fc84015a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83ffff700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc84015a4b0 0x7fc84015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83ffff700 1 -- 172.21.2.104:0/1403149873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc840149fb0 con 0x7fc84013f580 2021-07-11T03:58:30.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83ffff700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc84013f580 0x7fc8400abb60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fc84006ac60 tx=0x7fc828007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83d7fa700 1 -- 172.21.2.104:0/1403149873 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc82800ce30 con 0x7fc84013f580 2021-07-11T03:58:30.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83d7fa700 1 -- 172.21.2.104:0/1403149873 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82800f470 con 0x7fc84013f580 2021-07-11T03:58:30.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- 172.21.2.104:0/1403149873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8400b31f0 con 0x7fc84013f580 2021-07-11T03:58:30.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83d7fa700 1 -- 172.21.2.104:0/1403149873 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc82800f7d0 con 0x7fc84013f580 2021-07-11T03:58:30.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- 172.21.2.104:0/1403149873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84014aae0 con 0x7fc84013f580 2021-07-11T03:58:30.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc846965700 1 -- 172.21.2.104:0/1403149873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc840057350 con 0x7fc84013f580 2021-07-11T03:58:30.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.613+0000 7fc83d7fa700 1 -- 172.21.2.104:0/1403149873 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc828010070 con 0x7fc84013f580 2021-07-11T03:58:30.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.617+0000 7fc83d7fa700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc824053210 0x7fc8240556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.617+0000 7fc83d7fa700 1 -- 172.21.2.104:0/1403149873 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc82806a650 con 0x7fc84013f580 2021-07-11T03:58:30.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.617+0000 7fc83f7fe700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc824053210 0x7fc8240556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.617+0000 7fc83f7fe700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc824053210 0x7fc8240556d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fc830015000 tx=0x7fc830013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.617+0000 7fc83d7fa700 1 -- 172.21.2.104:0/1403149873 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc828041990 con 0x7fc84013f580 2021-07-11T03:58:30.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 -- 172.21.2.104:0/1403149873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc84014bb60 con 0x7fc84013f580 2021-07-11T03:58:30.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc83d7fa700 1 -- 172.21.2.104:0/1403149873 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc828044fb0 con 0x7fc84013f580 2021-07-11T03:58:30.709 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:58:30.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 -- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc824053210 msgr2=0x7fc8240556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc824053210 0x7fc8240556d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7fc830015000 tx=0x7fc830013000).stop 2021-07-11T03:58:30.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 -- 172.21.2.104:0/1403149873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc84013f580 msgr2=0x7fc8400abb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc84013f580 0x7fc8400abb60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7fc84006ac60 tx=0x7fc828007bd0).stop 2021-07-11T03:58:30.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 -- 172.21.2.104:0/1403149873 shutdown_connections 2021-07-11T03:58:30.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc84015a4b0 0x7fc84015a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc840149000 0x7fc840149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc824053210 0x7fc8240556d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 --2- 172.21.2.104:0/1403149873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc84013f580 0x7fc8400abb60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 -- 172.21.2.104:0/1403149873 >> 172.21.2.104:0/1403149873 conn(0x7fc84005f6b0 msgr2=0x7fc8400b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:30.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 -- 172.21.2.104:0/1403149873 shutdown_connections 2021-07-11T03:58:30.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.701+0000 7fc846965700 1 -- 172.21.2.104:0/1403149873 wait complete. 2021-07-11T03:58:30.716 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T03:58:30.716 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:58:30.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 Processor -- start 2021-07-11T03:58:30.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 -- start start 2021-07-11T03:58:30.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f88c40b2510 0x7f88c4144850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88c40a70c0 0x7f88c4144d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c40a7560 con 0x7f88c414ab50 2021-07-11T03:58:30.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c40a7800 con 0x7f88c40b2510 2021-07-11T03:58:30.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c40a7aa0 con 0x7f88c40a70c0 2021-07-11T03:58:30.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c2ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f88c40b2510 0x7f88c4144850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c2ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f88c40b2510 0x7f88c4144850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45744/0 (socket says 172.21.2.104:45744) 2021-07-11T03:58:30.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c2ffd700 1 -- 172.21.2.104:0/3334361309 learned_addr learned my addr 172.21.2.104:0/3334361309 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:30.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c2ffd700 1 -- 172.21.2.104:0/3334361309 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88c40a70c0 msgr2=0x7f88c4144d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:58:30.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c2ffd700 1 --2- 172.21.2.104:0/3334361309 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88c40a70c0 0x7f88c4144d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c2ffd700 1 -- 172.21.2.104:0/3334361309 >> v1:172.21.2.105:6789/0 conn(0x7f88c414ab50 legacy=0x7f88c40b0330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:58:30.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c2ffd700 1 -- 172.21.2.104:0/3334361309 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c40a7d40 con 0x7f88c40b2510 2021-07-11T03:58:30.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c2ffd700 1 --2- 172.21.2.104:0/3334361309 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88c40b2510 0x7f88c4144850 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f88c4062870 tx=0x7f88ac006eb0).ready entity=mon.0 client_cookie=23292fd2bd6131b6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c1ffb700 1 -- 172.21.2.104:0/3334361309 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88ac00d570 con 0x7f88c40b2510 2021-07-11T03:58:30.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c1ffb700 1 -- 172.21.2.104:0/3334361309 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac000e90 con 0x7f88c40b2510 2021-07-11T03:58:30.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c1ffb700 1 -- 172.21.2.104:0/3334361309 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88ac00d8c0 con 0x7f88c40b2510 2021-07-11T03:58:30.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 -- 172.21.2.104:0/3334361309 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88c40b2510 msgr2=0x7f88c4144850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 --2- 172.21.2.104:0/3334361309 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88c40b2510 0x7f88c4144850 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f88c4062870 tx=0x7f88ac006eb0).stop 2021-07-11T03:58:30.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 -- 172.21.2.104:0/3334361309 shutdown_connections 2021-07-11T03:58:30.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 --2- 172.21.2.104:0/3334361309 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88c40a70c0 0x7f88c4144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 --2- 172.21.2.104:0/3334361309 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88c40b2510 0x7f88c4144850 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 -- 172.21.2.104:0/3334361309 >> 172.21.2.104:0/3334361309 conn(0x7f88c4061070 msgr2=0x7f88c405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:30.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 -- 172.21.2.104:0/3334361309 shutdown_connections 2021-07-11T03:58:30.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.761+0000 7f88c9516700 1 -- 172.21.2.104:0/3334361309 wait complete. 2021-07-11T03:58:30.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c9516700 1 Processor -- start 2021-07-11T03:58:30.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c9516700 1 -- start start 2021-07-11T03:58:30.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c9516700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88c40a70c0 0x7f88c40677c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c9516700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88c40b2510 0x7f88c4064200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c9516700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88c414ab50 0x7f88c40646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c9516700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c4063b00 con 0x7f88c40a70c0 2021-07-11T03:58:30.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c9516700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c40b6700 con 0x7f88c40b2510 2021-07-11T03:58:30.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c9516700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c4063560 con 0x7f88c414ab50 2021-07-11T03:58:30.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88c40a70c0 0x7f88c40677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88c40a70c0 0x7f88c40677c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45752/0 (socket says 172.21.2.104:45752) 2021-07-11T03:58:30.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c2ffd700 1 -- 172.21.2.104:0/3007730541 learned_addr learned my addr 172.21.2.104:0/3007730541 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:30.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c2ffd700 1 -- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88c414ab50 msgr2=0x7f88c40646f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:58:30.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c2ffd700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88c414ab50 0x7f88c40646f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c2ffd700 1 -- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88c40b2510 msgr2=0x7f88c4064200 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:58:30.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c2ffd700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88c40b2510 0x7f88c4064200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c2ffd700 1 -- 172.21.2.104:0/3007730541 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac00b1e0 con 0x7f88c40a70c0 2021-07-11T03:58:30.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c2ffd700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88c40a70c0 0x7f88c40677c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f88c4062870 tx=0x7f88ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88bbfff700 1 -- 172.21.2.104:0/3007730541 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88ac002f70 con 0x7f88c40a70c0 2021-07-11T03:58:30.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88bbfff700 1 -- 172.21.2.104:0/3007730541 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ac00de10 con 0x7f88c40a70c0 2021-07-11T03:58:30.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c9516700 1 -- 172.21.2.104:0/3007730541 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c40b5aa0 con 0x7f88c40a70c0 2021-07-11T03:58:30.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c9516700 1 -- 172.21.2.104:0/3007730541 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c4063d60 con 0x7f88c40a70c0 2021-07-11T03:58:30.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88bbfff700 1 -- 172.21.2.104:0/3007730541 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88ac01d450 con 0x7f88c40a70c0 2021-07-11T03:58:30.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c9516700 1 -- 172.21.2.104:0/3007730541 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c4063f20 con 0x7f88c40a70c0 2021-07-11T03:58:30.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88bbfff700 1 -- 172.21.2.104:0/3007730541 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f88ac00f070 con 0x7f88c40a70c0 2021-07-11T03:58:30.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88bbfff700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88a8053350 0x7f88a8055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:30.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88bbfff700 1 -- 172.21.2.104:0/3007730541 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f88ac06cf40 con 0x7f88c40a70c0 2021-07-11T03:58:30.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.765+0000 7f88c27fc700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88a8053350 0x7f88a8055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:30.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.769+0000 7f88c27fc700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88a8053350 0x7f88a8055810 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f88b4002fb0 tx=0x7f88b4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:30.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.769+0000 7f88bbfff700 1 -- 172.21.2.104:0/3007730541 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f88ac044170 con 0x7f88c40a70c0 2021-07-11T03:58:30.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.873+0000 7f88c9516700 1 -- 172.21.2.104:0/3007730541 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88c414a6d0 con 0x7f88c40a70c0 2021-07-11T03:58:30.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88bbfff700 1 -- 172.21.2.104:0/3007730541 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f88ac047790 con 0x7f88c40a70c0 2021-07-11T03:58:30.884 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:30.884 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/2908869555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2908869555},{"type":"v1","addr":"172.21.2.105:6835","nonce":2908869555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/1110128427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1110128427},{"type":"v1","addr":"172.21.2.104:6833","nonce":1110128427}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:58:27.556328+0000","modified":"2021-07-11T03:58:28.559390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/225259368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":225259368},{"type":"v1","addr":"172.21.2.104:6835","nonce":225259368}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-07-11T03:58:30.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 -- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88a8053350 msgr2=0x7f88a8055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88a8053350 0x7f88a8055810 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f88b4002fb0 tx=0x7f88b4006930).stop 2021-07-11T03:58:30.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 -- 172.21.2.104:0/3007730541 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88c40a70c0 msgr2=0x7f88c40677c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:30.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88c40a70c0 0x7f88c40677c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f88c4062870 tx=0x7f88ac015000).stop 2021-07-11T03:58:30.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 -- 172.21.2.104:0/3007730541 shutdown_connections 2021-07-11T03:58:30.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88c414ab50 0x7f88c40646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88c40b2510 0x7f88c4064200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:30.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88a8053350 0x7f88a8055810 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 --2- 172.21.2.104:0/3007730541 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88c40a70c0 0x7f88c40677c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:30.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 -- 172.21.2.104:0/3007730541 >> 172.21.2.104:0/3007730541 conn(0x7f88c4061070 msgr2=0x7f88c4147570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:30.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 -- 172.21.2.104:0/3007730541 shutdown_connections 2021-07-11T03:58:30.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:30.877+0000 7f88c9516700 1 -- 172.21.2.104:0/3007730541 wait complete. 2021-07-11T03:58:30.888 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 27 2021-07-11T03:58:31.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:58:27.556328+0000', 'modified': '2021-07-11T03:58:28.559390+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5952}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5952': {'gid': 5952, 'name': 'c', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/225259368', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 225259368}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 225259368}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:58:31.199 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:58:31.199 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:58:31.264 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":5952,"want_state":"up:active","state":"up:active","rank_uptime":3.699030509,"mdsmap_epoch":27,"osdmap_epoch":66,"osdmap_epoch_barrier":65,"uptime":7.7744363160000001} 2021-07-11T03:58:31.265 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 5952, "mdsmap_epoch": 27, "osdmap_epoch": 66, "osdmap_epoch_barrier": 65, "rank_uptime": 3.699030509, "state": "up:active", "uptime": 7.774436316, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:58:31.265 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T03:58:31.266 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T03:58:31.266 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T03:58:31.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:31.267 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:58:31.278 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:58:31.292 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:58:31.293 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:58:31.342 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:58:31.342 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:58:31.344 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:31.344 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:58:31.350 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T03:58:31.351 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:58:31.359 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:58:31.371 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:58:31.371 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:58:31.398 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T03:58:31.399+0000 7f9471bcb080 -1 init, newargv = 0x5603d2a14ba0 newargc=9 2021-07-11T03:58:31.399 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15049]: starting ceph client 2021-07-11T03:58:31.404 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:58:31.405 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:58:31.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:31.406 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:58:31.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15049]: starting fuse 2021-07-11T03:58:31.415 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T03:58:31.415 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T03:58:31.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:31.416 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T03:58:31.417 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T03:58:31.417 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T03:58:31.418 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:58:31.418 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T03:58:31.418 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:58:31.419 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T03:58:31.419 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T03:58:31.420 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T03:58:31.420 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:58:31.421 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T03:58:31.421 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T03:58:31.422 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T03:58:31.431 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T03:58:31.431 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:58:31.432 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T03:58:31.432 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T03:58:31.433 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T03:58:31.433 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T03:58:31.433 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T03:58:31.441 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:58:31.442 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T03:58:31.442 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:58:31.476 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15049.asok 2021-07-11T03:58:31.479 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15049.asok 2021-07-11T03:58:31.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:31.480 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15049.asok status 2021-07-11T03:58:31.532 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T03:58:31.532 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T03:58:31.532 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T03:58:31.533 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T03:58:31.533 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T03:58:31.533 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T03:58:31.533 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T03:58:31.534 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15049", 2021-07-11T03:58:31.534 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T03:58:31.534 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:58:31.534 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T03:58:31.534 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T03:58:31.535 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4756, 2021-07-11T03:58:31.535 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T03:58:31.535 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T03:58:31.535 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T03:58:31.536 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4756 2021-07-11T03:58:31.543 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:58:31.543 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:58:31.543 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:58:31.543 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:58:31.543 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3964280495 2021-07-11T03:58:31.544 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T03:58:31.544 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:58:31.544 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:58:31.544 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:58:31.545 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:58:31.545 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3964280495 2021-07-11T03:58:31.545 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:58:31.545 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4756 172.21.2.119:0/3964280495", 2021-07-11T03:58:31.546 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/3964280495", 2021-07-11T03:58:31.546 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T03:58:31.546 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 27, 2021-07-11T03:58:31.546 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 66, 2021-07-11T03:58:31.546 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T03:58:31.547 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T03:58:31.547 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T03:58:31.547 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T03:58:31.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:31.549 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:58:31.550 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:58:31.555 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:58:31.556 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:58:31.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:31.557 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:58:31.570 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:58:31.571 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:58:31.571 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:58:31.619 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:58:31.620 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:58:31.621 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-07-11T03:58:31.622 DEBUG:teuthology.orchestra.run.gibba019:> import sys 2021-07-11T03:58:31.622 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-07-11T03:58:31.623 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:58:31.623 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:58:31.623 DEBUG:teuthology.orchestra.run.gibba019:> n = 1000 2021-07-11T03:58:31.624 DEBUG:teuthology.orchestra.run.gibba019:> abs_path = "/home/ubuntu/cephtest/mnt.0/dc-dir/dc-file" 2021-07-11T03:58:31.624 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:58:31.625 DEBUG:teuthology.orchestra.run.gibba019:> if not os.path.exists(os.path.dirname(abs_path)): 2021-07-11T03:58:31.625 DEBUG:teuthology.orchestra.run.gibba019:> os.makedirs(os.path.dirname(abs_path)) 2021-07-11T03:58:31.625 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:58:31.626 DEBUG:teuthology.orchestra.run.gibba019:> for i in range(0, n): 2021-07-11T03:58:31.636 DEBUG:teuthology.orchestra.run.gibba019:> fname = "{0}_{1}".format(abs_path, i) 2021-07-11T03:58:31.637 DEBUG:teuthology.orchestra.run.gibba019:> with open(fname, '"'"'w'"'"') as f: 2021-07-11T03:58:31.637 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-07-11T03:58:31.638 DEBUG:teuthology.orchestra.run.gibba019:> if True: 2021-07-11T03:58:31.638 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-07-11T03:58:31.638 DEBUG:teuthology.orchestra.run.gibba019:> os.fsync(f.fileno()) 2021-07-11T03:58:31.639 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:58:42.529 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:58:42.532 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:58:42.537 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:58:42.540 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:58:50.126 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:58:50.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 Processor -- start 2021-07-11T03:58:50.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 -- start start 2021-07-11T03:58:50.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2260066e00 0x7f22600afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f22600aaf10 0x7f2260143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22600671e0 con 0x7f2260060d00 2021-07-11T03:58:50.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22600ab340 con 0x7f22600aaf10 2021-07-11T03:58:50.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22600a9450 con 0x7f2260066e00 2021-07-11T03:58:50.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f226570f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f22600aaf10 0x7f2260143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f226570f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f22600aaf10 0x7f2260143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45758/0 (socket says 172.21.2.104:45758) 2021-07-11T03:58:50.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f226570f700 1 -- 172.21.2.104:0/3480552470 learned_addr learned my addr 172.21.2.104:0/3480552470 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:50.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f226570f700 1 -- 172.21.2.104:0/3480552470 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2260066e00 msgr2=0x7f22600afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f226570f700 1 --2- 172.21.2.104:0/3480552470 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2260066e00 0x7f22600afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f226570f700 1 -- 172.21.2.104:0/3480552470 >> v1:172.21.2.105:6789/0 conn(0x7f2260060d00 legacy=0x7f22600aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f226570f700 1 -- 172.21.2.104:0/3480552470 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22600b03b0 con 0x7f22600aaf10 2021-07-11T03:58:50.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f226570f700 1 --2- 172.21.2.104:0/3480552470 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22600aaf10 0x7f2260143460 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f226005f320 tx=0x7f2258009290).ready entity=mon.0 client_cookie=9839297640c00771 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f225f7fe700 1 -- 172.21.2.104:0/3480552470 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f225800b830 con 0x7f22600aaf10 2021-07-11T03:58:50.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f225f7fe700 1 -- 172.21.2.104:0/3480552470 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225800b990 con 0x7f22600aaf10 2021-07-11T03:58:50.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f225f7fe700 1 -- 172.21.2.104:0/3480552470 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f225800bb00 con 0x7f22600aaf10 2021-07-11T03:58:50.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 -- 172.21.2.104:0/3480552470 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22600aaf10 msgr2=0x7f2260143460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 --2- 172.21.2.104:0/3480552470 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22600aaf10 0x7f2260143460 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f226005f320 tx=0x7f2258009290).stop 2021-07-11T03:58:50.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 -- 172.21.2.104:0/3480552470 shutdown_connections 2021-07-11T03:58:50.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 --2- 172.21.2.104:0/3480552470 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22600aaf10 0x7f2260143460 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 --2- 172.21.2.104:0/3480552470 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2260066e00 0x7f22600afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 -- 172.21.2.104:0/3480552470 >> 172.21.2.104:0/3480552470 conn(0x7f22600b53e0 msgr2=0x7f22600b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:50.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 -- 172.21.2.104:0/3480552470 shutdown_connections 2021-07-11T03:58:50.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 -- 172.21.2.104:0/3480552470 wait complete. 2021-07-11T03:58:50.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 Processor -- start 2021-07-11T03:58:50.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.173+0000 7f2267172700 1 -- start start 2021-07-11T03:58:50.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f2267172700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2260060d00 0x7f226013ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f2267172700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2260066e00 0x7f22600b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f2267172700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22600aaf10 0x7f22600b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f2267172700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2260145af0 con 0x7f2260066e00 2021-07-11T03:58:50.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f2267172700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22600b4400 con 0x7f22600aaf10 2021-07-11T03:58:50.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f2267172700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22601430d0 con 0x7f2260060d00 2021-07-11T03:58:50.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2260066e00 0x7f22600b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2260066e00 0x7f22600b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45764/0 (socket says 172.21.2.104:45764) 2021-07-11T03:58:50.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225ffff700 1 -- 172.21.2.104:0/4198363825 learned_addr learned my addr 172.21.2.104:0/4198363825 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:50.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225ffff700 1 -- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2260060d00 msgr2=0x7f226013ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225ffff700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2260060d00 0x7f226013ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225ffff700 1 -- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22600aaf10 msgr2=0x7f22600b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225ffff700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22600aaf10 0x7f22600b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225ffff700 1 -- 172.21.2.104:0/4198363825 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2258009530 con 0x7f2260066e00 2021-07-11T03:58:50.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225ffff700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2260066e00 0x7f22600b35e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f2250002f90 tx=0x7f2250006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225dffb700 1 -- 172.21.2.104:0/4198363825 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2250013460 con 0x7f2260066e00 2021-07-11T03:58:50.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225dffb700 1 -- 172.21.2.104:0/4198363825 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22500135c0 con 0x7f2260066e00 2021-07-11T03:58:50.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f2267172700 1 -- 172.21.2.104:0/4198363825 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2260143370 con 0x7f2260066e00 2021-07-11T03:58:50.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225dffb700 1 -- 172.21.2.104:0/4198363825 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2250013920 con 0x7f2260066e00 2021-07-11T03:58:50.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f2267172700 1 -- 172.21.2.104:0/4198363825 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2260143940 con 0x7f2260066e00 2021-07-11T03:58:50.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f2267172700 1 -- 172.21.2.104:0/4198363825 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2260143500 con 0x7f2260066e00 2021-07-11T03:58:50.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225dffb700 1 -- 172.21.2.104:0/4198363825 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f225001a060 con 0x7f2260066e00 2021-07-11T03:58:50.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225dffb700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22440531a0 0x7f2244055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f225dffb700 1 -- 172.21.2.104:0/4198363825 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f225006ae20 con 0x7f2260066e00 2021-07-11T03:58:50.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f2264f0e700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22440531a0 0x7f2244055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.177+0000 7f2264f0e700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22440531a0 0x7f2244055660 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f2248002f70 tx=0x7f2248006a90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.181+0000 7f225dffb700 1 -- 172.21.2.104:0/4198363825 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2250042050 con 0x7f2260066e00 2021-07-11T03:58:50.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 -- 172.21.2.104:0/4198363825 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2260057350 con 0x7f2260066e00 2021-07-11T03:58:50.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f225dffb700 1 -- 172.21.2.104:0/4198363825 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f2250018030 con 0x7f2260066e00 2021-07-11T03:58:50.296 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:50.296 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/2908869555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2908869555},{"type":"v1","addr":"172.21.2.105:6835","nonce":2908869555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/1110128427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1110128427},{"type":"v1","addr":"172.21.2.104:6833","nonce":1110128427}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:58:27.556328+0000","modified":"2021-07-11T03:58:28.559390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/225259368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":225259368},{"type":"v1","addr":"172.21.2.104:6835","nonce":225259368}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-07-11T03:58:50.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 -- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22440531a0 msgr2=0x7f2244055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22440531a0 0x7f2244055660 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f2248002f70 tx=0x7f2248006a90).stop 2021-07-11T03:58:50.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 -- 172.21.2.104:0/4198363825 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2260066e00 msgr2=0x7f22600b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2260066e00 0x7f22600b35e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f2250002f90 tx=0x7f2250006df0).stop 2021-07-11T03:58:50.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 -- 172.21.2.104:0/4198363825 shutdown_connections 2021-07-11T03:58:50.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22600aaf10 0x7f22600b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22440531a0 0x7f2244055660 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2260066e00 0x7f22600b35e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 --2- 172.21.2.104:0/4198363825 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2260060d00 0x7f226013ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 -- 172.21.2.104:0/4198363825 >> 172.21.2.104:0/4198363825 conn(0x7f22600b53e0 msgr2=0x7f226013df60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:50.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 -- 172.21.2.104:0/4198363825 shutdown_connections 2021-07-11T03:58:50.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.289+0000 7f2267172700 1 -- 172.21.2.104:0/4198363825 wait complete. 2021-07-11T03:58:50.300 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 27 2021-07-11T03:58:50.303 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_caps 20 2021-07-11T03:58:50.365 INFO:teuthology.orchestra.run.gibba004.stdout:{"success":"mds_recall_max_caps = '20' (not observed, change may require restart) "} 2021-07-11T03:58:50.366 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_caps = '20' (not observed, change may require restart) " } 2021-07-11T03:58:50.367 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:58:50.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 Processor -- start 2021-07-11T03:58:50.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 -- start start 2021-07-11T03:58:50.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f56200ab160 0x7f5620067300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5620065960 con 0x7f56200aad80 2021-07-11T03:58:50.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5620064f10 con 0x7f56200b4d50 2021-07-11T03:58:50.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56200b3ce0 con 0x7f56200ab160 2021-07-11T03:58:50.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561effd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f56200aad80 0x7f5620064800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42540/0 (socket says 172.21.2.104:42540) 2021-07-11T03:58:50.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561effd700 1 -- 172.21.2.104:0/3880465432 learned_addr learned my addr 172.21.2.104:0/3880465432 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:50.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51479122 0 0) 0x7f5620065960 con 0x7f56200aad80 2021-07-11T03:58:50.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5604003260 con 0x7f56200aad80 2021-07-11T03:58:50.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4068443664 0 0) 0x7f5604003260 con 0x7f56200aad80 2021-07-11T03:58:50.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 >> v2:172.21.2.105:3300/0 conn(0x7f56200ab160 msgr2=0x7f5620067300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 --2- 172.21.2.104:0/3880465432 >> v2:172.21.2.105:3300/0 conn(0x7f56200ab160 0x7f5620067300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 >> v1:172.21.2.105:6789/0 conn(0x7f56200b4d50 legacy=0x7f56200b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5620066e60 con 0x7f56200aad80 2021-07-11T03:58:50.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f5614001680 con 0x7f56200aad80 2021-07-11T03:58:50.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 >> v1:172.21.2.104:6789/0 conn(0x7f56200aad80 legacy=0x7f5620064800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 --2- 172.21.2.104:0/3880465432 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5604003750 0x7f5604003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 --2- 172.21.2.104:0/3880465432 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5604004500 0x7f5604004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 --2- 172.21.2.104:0/3880465432 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5604004f90 0x7f560400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200b3ce0 con 0x7f5604003750 2021-07-11T03:58:50.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560400be80 con 0x7f5604004500 2021-07-11T03:58:50.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560400c040 con 0x7f5604004f90 2021-07-11T03:58:50.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561effd700 1 --2- 172.21.2.104:0/3880465432 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5604003750 0x7f5604003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561effd700 1 -- 172.21.2.104:0/3880465432 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5604004f90 msgr2=0x7f560400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561effd700 1 --2- 172.21.2.104:0/3880465432 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5604004f90 0x7f560400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561effd700 1 -- 172.21.2.104:0/3880465432 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5604004500 msgr2=0x7f5604004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561effd700 1 --2- 172.21.2.104:0/3880465432 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5604004500 0x7f5604004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561effd700 1 -- 172.21.2.104:0/3880465432 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5604003260 con 0x7f5604003750 2021-07-11T03:58:50.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561f7fe700 1 -- 172.21.2.104:0/3880465432 reap_dead start 2021-07-11T03:58:50.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561effd700 1 --2- 172.21.2.104:0/3880465432 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5604003750 0x7f5604003b50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f5620063560 tx=0x7f5620066ff0).ready entity=mon.0 client_cookie=28769c5be44cf366 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f56140071a0 con 0x7f5604003750 2021-07-11T03:58:50.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5614007300 con 0x7f5604003750 2021-07-11T03:58:50.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561dffb700 1 -- 172.21.2.104:0/3880465432 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f561400cb80 con 0x7f5604003750 2021-07-11T03:58:50.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 -- 172.21.2.104:0/3880465432 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5604003750 msgr2=0x7f5604003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 --2- 172.21.2.104:0/3880465432 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5604003750 0x7f5604003b50 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f5620063560 tx=0x7f5620066ff0).stop 2021-07-11T03:58:50.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 -- 172.21.2.104:0/3880465432 shutdown_connections 2021-07-11T03:58:50.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 --2- 172.21.2.104:0/3880465432 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5604003750 0x7f5604003b50 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 -- 172.21.2.104:0/3880465432 >> 172.21.2.104:0/3880465432 conn(0x7f5620062be0 msgr2=0x7f562005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:50.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 -- 172.21.2.104:0/3880465432 shutdown_connections 2021-07-11T03:58:50.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 -- 172.21.2.104:0/3880465432 wait complete. 2021-07-11T03:58:50.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 Processor -- start 2021-07-11T03:58:50.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.413+0000 7f561ffff700 1 -- start start 2021-07-11T03:58:50.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561ffff700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5604003750 0x7f56200af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f56200b1740 0x7f56200b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561ffff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f56200ac1f0 0x7f56200ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561ffff700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5620150a60 con 0x7f56200b1740 2021-07-11T03:58:50.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561ffff700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5620144a80 con 0x7f5604003750 2021-07-11T03:58:50.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561ffff700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5620144f20 con 0x7f56200ac1f0 2021-07-11T03:58:50.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f56200b1740 0x7f56200b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f56200b1740 0x7f56200b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45786/0 (socket says 172.21.2.104:45786) 2021-07-11T03:58:50.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561e7fc700 1 -- 172.21.2.104:0/593458972 learned_addr learned my addr 172.21.2.104:0/593458972 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:50.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561e7fc700 1 -- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f56200ac1f0 msgr2=0x7f56200ac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561e7fc700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f56200ac1f0 0x7f56200ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561e7fc700 1 -- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5604003750 msgr2=0x7f56200af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561e7fc700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5604003750 0x7f56200af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561e7fc700 1 -- 172.21.2.104:0/593458972 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5620150780 con 0x7f56200b1740 2021-07-11T03:58:50.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561e7fc700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f56200b1740 0x7f56200b1bc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f56180032f0 tx=0x7f5618009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f560ffff700 1 -- 172.21.2.104:0/593458972 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f561800d790 con 0x7f56200b1740 2021-07-11T03:58:50.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f560ffff700 1 -- 172.21.2.104:0/593458972 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561800d8f0 con 0x7f56200b1740 2021-07-11T03:58:50.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f560ffff700 1 -- 172.21.2.104:0/593458972 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f561800dc90 con 0x7f56200b1740 2021-07-11T03:58:50.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561ffff700 1 -- 172.21.2.104:0/593458972 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56200ae6c0 con 0x7f56200b1740 2021-07-11T03:58:50.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561ffff700 1 -- 172.21.2.104:0/593458972 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5620150d00 con 0x7f56200b1740 2021-07-11T03:58:50.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561ffff700 1 -- 172.21.2.104:0/593458972 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5620057350 con 0x7f56200b1740 2021-07-11T03:58:50.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f560ffff700 1 -- 172.21.2.104:0/593458972 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5618019460 con 0x7f56200b1740 2021-07-11T03:58:50.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f560ffff700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f55fc053350 0x7f55fc055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f560ffff700 1 -- 172.21.2.104:0/593458972 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f561806ba50 con 0x7f56200b1740 2021-07-11T03:58:50.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561effd700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f55fc053350 0x7f55fc055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.417+0000 7f561effd700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f55fc053350 0x7f55fc055810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f5620063560 tx=0x7f5614013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.421+0000 7f560ffff700 1 -- 172.21.2.104:0/593458972 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5618042d30 con 0x7f56200b1740 2021-07-11T03:58:50.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 -- 172.21.2.104:0/593458972 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f562005d9d0 con 0x7f56200b1740 2021-07-11T03:58:50.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f560ffff700 1 -- 172.21.2.104:0/593458972 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f5618046350 con 0x7f56200b1740 2021-07-11T03:58:50.538 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:50.538 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/2908869555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2908869555},{"type":"v1","addr":"172.21.2.105:6835","nonce":2908869555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/1110128427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1110128427},{"type":"v1","addr":"172.21.2.104:6833","nonce":1110128427}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:58:27.556328+0000","modified":"2021-07-11T03:58:28.559390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/225259368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":225259368},{"type":"v1","addr":"172.21.2.104:6835","nonce":225259368}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-07-11T03:58:50.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 -- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f55fc053350 msgr2=0x7f55fc055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f55fc053350 0x7f55fc055810 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f5620063560 tx=0x7f5614013000).stop 2021-07-11T03:58:50.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 -- 172.21.2.104:0/593458972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f56200b1740 msgr2=0x7f56200b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f56200b1740 0x7f56200b1bc0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f56180032f0 tx=0x7f5618009290).stop 2021-07-11T03:58:50.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 -- 172.21.2.104:0/593458972 shutdown_connections 2021-07-11T03:58:50.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f56200ac1f0 0x7f56200ac680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f55fc053350 0x7f55fc055810 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f56200b1740 0x7f56200b1bc0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 --2- 172.21.2.104:0/593458972 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5604003750 0x7f56200af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 -- 172.21.2.104:0/593458972 >> 172.21.2.104:0/593458972 conn(0x7f5620062be0 msgr2=0x7f56200b6800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:50.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 -- 172.21.2.104:0/593458972 shutdown_connections 2021-07-11T03:58:50.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.529+0000 7f561ffff700 1 -- 172.21.2.104:0/593458972 wait complete. 2021-07-11T03:58:50.546 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 27 2021-07-11T03:58:50.548 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_recall_max_decay_threshold 400 2021-07-11T03:58:50.606 INFO:teuthology.orchestra.run.gibba004.stdout:{"success":"mds_recall_max_decay_threshold = '400' (not observed, change may require restart) "} 2021-07-11T03:58:50.606 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_recall_max_decay_threshold = '400' (not observed, change may require restart) " } 2021-07-11T03:58:50.607 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:58:50.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f9108a11700 1 Processor -- start 2021-07-11T03:58:50.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f9108a11700 1 -- start start 2021-07-11T03:58:50.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f9108a11700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9104060280 0x7f91040668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f9108a11700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91040b19a0 0x7f9104066da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f9108a11700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91040abea0 con 0x7f910405f6b0 2021-07-11T03:58:50.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f9108a11700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91040ab310 con 0x7f9104060280 2021-07-11T03:58:50.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f9108a11700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9104140300 con 0x7f91040b19a0 2021-07-11T03:58:50.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f910259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9104060280 0x7f91040668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f910259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9104060280 0x7f91040668b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45790/0 (socket says 172.21.2.104:45790) 2021-07-11T03:58:50.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f910259c700 1 -- 172.21.2.104:0/2023623983 learned_addr learned my addr 172.21.2.104:0/2023623983 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:50.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f910259c700 1 -- 172.21.2.104:0/2023623983 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91040b19a0 msgr2=0x7f9104066da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f910259c700 1 --2- 172.21.2.104:0/2023623983 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91040b19a0 0x7f9104066da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f910259c700 1 -- 172.21.2.104:0/2023623983 >> v1:172.21.2.105:6789/0 conn(0x7f910405f6b0 legacy=0x7f91040a97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f910259c700 1 -- 172.21.2.104:0/2023623983 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91041405a0 con 0x7f9104060280 2021-07-11T03:58:50.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f910259c700 1 --2- 172.21.2.104:0/2023623983 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9104060280 0x7f91040668b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f9104060d70 tx=0x7f90ec006eb0).ready entity=mon.0 client_cookie=9e06efe2bac4ee42 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f910159a700 1 -- 172.21.2.104:0/2023623983 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90ec00d570 con 0x7f9104060280 2021-07-11T03:58:50.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f910159a700 1 -- 172.21.2.104:0/2023623983 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ec000e90 con 0x7f9104060280 2021-07-11T03:58:50.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f910159a700 1 -- 172.21.2.104:0/2023623983 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90ec00d8c0 con 0x7f9104060280 2021-07-11T03:58:50.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f9108a11700 1 -- 172.21.2.104:0/2023623983 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9104060280 msgr2=0x7f91040668b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.649+0000 7f9108a11700 1 --2- 172.21.2.104:0/2023623983 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9104060280 0x7f91040668b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f9104060d70 tx=0x7f90ec006eb0).stop 2021-07-11T03:58:50.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 -- 172.21.2.104:0/2023623983 shutdown_connections 2021-07-11T03:58:50.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 --2- 172.21.2.104:0/2023623983 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91040b19a0 0x7f9104066da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 --2- 172.21.2.104:0/2023623983 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9104060280 0x7f91040668b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 -- 172.21.2.104:0/2023623983 >> 172.21.2.104:0/2023623983 conn(0x7f91040b53e0 msgr2=0x7f91040b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:50.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 -- 172.21.2.104:0/2023623983 shutdown_connections 2021-07-11T03:58:50.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 -- 172.21.2.104:0/2023623983 wait complete. 2021-07-11T03:58:50.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 Processor -- start 2021-07-11T03:58:50.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 -- start start 2021-07-11T03:58:50.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f910405f6b0 0x7f91041412f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9104060280 0x7f9104159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91040b19a0 0x7f910415a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9104157220 con 0x7f9104060280 2021-07-11T03:58:50.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9104142470 con 0x7f910405f6b0 2021-07-11T03:58:50.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9104141d80 con 0x7f91040b19a0 2021-07-11T03:58:50.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9101d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9104060280 0x7f9104159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9101d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9104060280 0x7f9104159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45798/0 (socket says 172.21.2.104:45798) 2021-07-11T03:58:50.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9101d9b700 1 -- 172.21.2.104:0/2502034916 learned_addr learned my addr 172.21.2.104:0/2502034916 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:50.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9101d9b700 1 -- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91040b19a0 msgr2=0x7f910415a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9101d9b700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91040b19a0 0x7f910415a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9101d9b700 1 -- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f910405f6b0 msgr2=0x7f91041412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9101d9b700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f910405f6b0 0x7f91041412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9101d9b700 1 -- 172.21.2.104:0/2502034916 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec00b1e0 con 0x7f9104060280 2021-07-11T03:58:50.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9101d9b700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9104060280 0x7f9104159e60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f90f4002f70 tx=0x7f90f4006e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f90fb7fe700 1 -- 172.21.2.104:0/2502034916 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90f400e470 con 0x7f9104060280 2021-07-11T03:58:50.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f90fb7fe700 1 -- 172.21.2.104:0/2502034916 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f400e5d0 con 0x7f9104060280 2021-07-11T03:58:50.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 -- 172.21.2.104:0/2502034916 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9104157490 con 0x7f9104060280 2021-07-11T03:58:50.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f90fb7fe700 1 -- 172.21.2.104:0/2502034916 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90f400e930 con 0x7f9104060280 2021-07-11T03:58:50.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 -- 172.21.2.104:0/2502034916 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910413cb70 con 0x7f9104060280 2021-07-11T03:58:50.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f9108a11700 1 -- 172.21.2.104:0/2502034916 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9104157620 con 0x7f9104060280 2021-07-11T03:58:50.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.653+0000 7f90fb7fe700 1 -- 172.21.2.104:0/2502034916 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f90f4018030 con 0x7f9104060280 2021-07-11T03:58:50.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.657+0000 7f90fb7fe700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90e8053270 0x7f90e8055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.657+0000 7f90fb7fe700 1 -- 172.21.2.104:0/2502034916 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f90f406a970 con 0x7f9104060280 2021-07-11T03:58:50.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.657+0000 7f910259c700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90e8053270 0x7f90e8055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.657+0000 7f910259c700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90e8053270 0x7f90e8055730 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f9104060d70 tx=0x7f90ec015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.657+0000 7f90fb7fe700 1 -- 172.21.2.104:0/2502034916 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f90f4041d60 con 0x7f9104060280 2021-07-11T03:58:50.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.765+0000 7f9108a11700 1 -- 172.21.2.104:0/2502034916 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9104141ec0 con 0x7f9104060280 2021-07-11T03:58:50.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.765+0000 7f90fb7fe700 1 -- 172.21.2.104:0/2502034916 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f90f4045380 con 0x7f9104060280 2021-07-11T03:58:50.776 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:58:50.776 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/2908869555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2908869555},{"type":"v1","addr":"172.21.2.105:6835","nonce":2908869555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/1110128427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1110128427},{"type":"v1","addr":"172.21.2.104:6833","nonce":1110128427}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:58:27.556328+0000","modified":"2021-07-11T03:58:28.559390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/225259368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":225259368},{"type":"v1","addr":"172.21.2.104:6835","nonce":225259368}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-07-11T03:58:50.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 -- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90e8053270 msgr2=0x7f90e8055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90e8053270 0x7f90e8055730 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f9104060d70 tx=0x7f90ec015000).stop 2021-07-11T03:58:50.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 -- 172.21.2.104:0/2502034916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9104060280 msgr2=0x7f9104159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9104060280 0x7f9104159e60 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f90f4002f70 tx=0x7f90f4006e50).stop 2021-07-11T03:58:50.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 -- 172.21.2.104:0/2502034916 shutdown_connections 2021-07-11T03:58:50.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91040b19a0 0x7f910415a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90e8053270 0x7f90e8055730 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9104060280 0x7f9104159e60 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 --2- 172.21.2.104:0/2502034916 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f910405f6b0 0x7f91041412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 -- 172.21.2.104:0/2502034916 >> 172.21.2.104:0/2502034916 conn(0x7f91040b53e0 msgr2=0x7f9104066820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:50.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 -- 172.21.2.104:0/2502034916 shutdown_connections 2021-07-11T03:58:50.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.769+0000 7f9108a11700 1 -- 172.21.2.104:0/2502034916 wait complete. 2021-07-11T03:58:50.779 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 27 2021-07-11T03:58:50.784 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c cache drop 10 2021-07-11T03:58:50.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 Processor -- start 2021-07-11T03:58:50.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 -- start start 2021-07-11T03:58:50.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce84141960 0x7fce840affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce840a77d0 con 0x7fce840b1ec0 2021-07-11T03:58:50.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce840b0650 con 0x7fce840b12d0 2021-07-11T03:58:50.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce840b1740 con 0x7fce84141960 2021-07-11T03:58:50.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce837fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fce840b1ec0 0x7fce840a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42572/0 (socket says 172.21.2.104:42572) 2021-07-11T03:58:50.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce837fe700 1 -- 172.21.2.104:0/2774801600 learned_addr learned my addr 172.21.2.104:0/2774801600 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:58:50.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068170472 0 0) 0x7fce840a77d0 con 0x7fce840b1ec0 2021-07-11T03:58:50.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce64003260 con 0x7fce840b1ec0 2021-07-11T03:58:50.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce83fff700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce84141960 0x7fce840affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 441117864 0 0) 0x7fce64003260 con 0x7fce840b1ec0 2021-07-11T03:58:50.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce84141960 msgr2=0x7fce840affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce84141960 0x7fce840affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 >> v1:172.21.2.105:6789/0 conn(0x7fce840b12d0 legacy=0x7fce84141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce840b08f0 con 0x7fce840b1ec0 2021-07-11T03:58:50.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fce6c001660 con 0x7fce840b1ec0 2021-07-11T03:58:50.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 >> v1:172.21.2.104:6789/0 conn(0x7fce840b1ec0 legacy=0x7fce840a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce84141960 0x7fce64003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce64003d40 0x7fce64004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce640049f0 0x7fce64004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce840b1740 con 0x7fce640049f0 2021-07-11T03:58:50.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6400b880 con 0x7fce64003d40 2021-07-11T03:58:50.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6400ba40 con 0x7fce84141960 2021-07-11T03:58:50.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce83fff700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce640049f0 0x7fce64004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce837fe700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce84141960 0x7fce64003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce83fff700 1 -- 172.21.2.104:0/2774801600 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce84141960 msgr2=0x7fce64003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce83fff700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce84141960 0x7fce64003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce83fff700 1 -- 172.21.2.104:0/2774801600 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce64003d40 msgr2=0x7fce64004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce83fff700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce64003d40 0x7fce64004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce83fff700 1 -- 172.21.2.104:0/2774801600 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce64003260 con 0x7fce640049f0 2021-07-11T03:58:50.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce837fe700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce84141960 0x7fce64003780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:58:50.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce83fff700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce640049f0 0x7fce64004e70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fce7c00a000 tx=0x7fce7c008040).ready entity=mon.0 client_cookie=e745ce03929c93a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7c00c070 con 0x7fce640049f0 2021-07-11T03:58:50.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c00e620 con 0x7fce640049f0 2021-07-11T03:58:50.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce827fc700 1 -- 172.21.2.104:0/2774801600 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7c00e980 con 0x7fce640049f0 2021-07-11T03:58:50.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 -- 172.21.2.104:0/2774801600 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce640049f0 msgr2=0x7fce64004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 --2- 172.21.2.104:0/2774801600 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce640049f0 0x7fce64004e70 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fce7c00a000 tx=0x7fce7c008040).stop 2021-07-11T03:58:50.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce83fff700 1 -- 172.21.2.104:0/2774801600 reap_dead start 2021-07-11T03:58:50.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 -- 172.21.2.104:0/2774801600 shutdown_connections 2021-07-11T03:58:50.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 -- 172.21.2.104:0/2774801600 >> 172.21.2.104:0/2774801600 conn(0x7fce8405f6b0 msgr2=0x7fce8405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:50.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 -- 172.21.2.104:0/2774801600 shutdown_connections 2021-07-11T03:58:50.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.829+0000 7fce8a0c0700 1 -- 172.21.2.104:0/2774801600 wait complete. 2021-07-11T03:58:50.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce8a0c0700 1 Processor -- start 2021-07-11T03:58:50.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce8a0c0700 1 -- start start 2021-07-11T03:58:50.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce8a0c0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce84140350 0x7fce840aa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce8a0c0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce840a9420 0x7fce840aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce8a0c0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce840ad310 0x7fce840ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce8a0c0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce840ac140 con 0x7fce840ad310 2021-07-11T03:58:50.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce8a0c0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce840af9d0 con 0x7fce84140350 2021-07-11T03:58:50.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce8a0c0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce84140730 con 0x7fce840a9420 2021-07-11T03:58:50.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce83fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce840ad310 0x7fce840ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce83fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce840ad310 0x7fce840ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45816/0 (socket says 172.21.2.104:45816) 2021-07-11T03:58:50.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce83fff700 1 -- 172.21.2.104:0/3187044127 learned_addr learned my addr 172.21.2.104:0/3187044127 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:50.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce83fff700 1 -- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce840a9420 msgr2=0x7fce840aaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce83fff700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce840a9420 0x7fce840aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce83fff700 1 -- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce84140350 msgr2=0x7fce840aa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce83fff700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce84140350 0x7fce840aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce83fff700 1 -- 172.21.2.104:0/3187044127 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce840aaf40 con 0x7fce840ad310 2021-07-11T03:58:50.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce83fff700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce840ad310 0x7fce840ad790 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fce6c001370 tx=0x7fce7c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce80ff9700 1 -- 172.21.2.104:0/3187044127 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7c00ee90 con 0x7fce840ad310 2021-07-11T03:58:50.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce80ff9700 1 -- 172.21.2.104:0/3187044127 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c01d440 con 0x7fce840ad310 2021-07-11T03:58:50.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce8a0c0700 1 -- 172.21.2.104:0/3187044127 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce840af490 con 0x7fce840ad310 2021-07-11T03:58:50.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce80ff9700 1 -- 172.21.2.104:0/3187044127 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7c01d5b0 con 0x7fce840ad310 2021-07-11T03:58:50.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce8a0c0700 1 -- 172.21.2.104:0/3187044127 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce840a9e00 con 0x7fce840ad310 2021-07-11T03:58:50.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce80ff9700 1 -- 172.21.2.104:0/3187044127 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fce7c01d710 con 0x7fce840ad310 2021-07-11T03:58:50.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce80ff9700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce680532b0 0x7fce68055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce80ff9700 1 -- 172.21.2.104:0/3187044127 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fce7c06b940 con 0x7fce840ad310 2021-07-11T03:58:50.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce837fe700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce680532b0 0x7fce68055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce94310740 1 Processor -- start 2021-07-11T03:58:50.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.833+0000 7fce94310740 1 -- start start 2021-07-11T03:58:50.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1eb6ae0 0x1eba720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eb9ec0 0x1eb6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1ebbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e2abd0 con 0x1eefaf0 2021-07-11T03:58:50.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e2ad10 con 0x1eb6ae0 2021-07-11T03:58:50.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x1e2c3e0 con 0x1eb9ec0 2021-07-11T03:58:50.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce83fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1ebbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce83fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1ebbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45824/0 (socket says 172.21.2.104:45824) 2021-07-11T03:58:50.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce83fff700 1 -- 172.21.2.104:0/4152236988 learned_addr learned my addr 172.21.2.104:0/4152236988 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:50.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce83fff700 1 -- 172.21.2.104:0/4152236988 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eb9ec0 msgr2=0x1eb6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce83fff700 1 --2- 172.21.2.104:0/4152236988 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eb9ec0 0x1eb6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce83fff700 1 -- 172.21.2.104:0/4152236988 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1eb6ae0 msgr2=0x1eba720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce83fff700 1 --2- 172.21.2.104:0/4152236988 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1eb6ae0 0x1eba720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce83fff700 1 -- 172.21.2.104:0/4152236988 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x1e64040 con 0x1eefaf0 2021-07-11T03:58:50.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce837fe700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce680532b0 0x7fce68055770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fce6c008000 tx=0x7fce6c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce83fff700 1 --2- 172.21.2.104:0/4152236988 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1ebbe80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fce7c06bb50 tx=0x7fce7c017000).ready entity=mon.0 client_cookie=65928a125904cfa2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce8a0c0700 1 -- 172.21.2.104:0/4152236988 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7c044c80 con 0x1eefaf0 2021-07-11T03:58:50.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce8a0c0700 1 -- 172.21.2.104:0/4152236988 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c042ff0 con 0x1eefaf0 2021-07-11T03:58:50.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce8a0c0700 1 -- 172.21.2.104:0/4152236988 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7c045fc0 con 0x1eefaf0 2021-07-11T03:58:50.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- 172.21.2.104:0/4152236988 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 msgr2=0x1ebbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 --2- 172.21.2.104:0/4152236988 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1ebbe80 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fce7c06bb50 tx=0x7fce7c017000).stop 2021-07-11T03:58:50.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- 172.21.2.104:0/4152236988 shutdown_connections 2021-07-11T03:58:50.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 --2- 172.21.2.104:0/4152236988 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1ebbe80 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 --2- 172.21.2.104:0/4152236988 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eb9ec0 0x1eb6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 --2- 172.21.2.104:0/4152236988 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1eb6ae0 0x1eba720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- 172.21.2.104:0/4152236988 >> 172.21.2.104:0/4152236988 conn(0x1e45950 msgr2=0x1eb97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:50.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- 172.21.2.104:0/4152236988 shutdown_connections 2021-07-11T03:58:50.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- 172.21.2.104:0/4152236988 wait complete. 2021-07-11T03:58:50.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- start start 2021-07-11T03:58:50.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 Processor -- start 2021-07-11T03:58:50.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eefaf0 0x1ef3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1ef6180 0x1eed970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ef4c40 con 0x1eb6ae0 2021-07-11T03:58:50.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ef4e70 con 0x1ef6180 2021-07-11T03:58:50.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x1ef52b0 con 0x1eefaf0 2021-07-11T03:58:50.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 10 client.0 ll_register_callbacks cb 0x1e1f4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-07-11T03:58:50.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce837fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce837fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45834/0 (socket says 172.21.2.104:45834) 2021-07-11T03:58:50.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce837fe700 1 -- 172.21.2.104:0/1892058403 learned_addr learned my addr 172.21.2.104:0/1892058403 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:50.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce837fe700 1 -- 172.21.2.104:0/1892058403 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eefaf0 msgr2=0x1ef3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce83fff700 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1ef6180 0x1eed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce837fe700 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eefaf0 0x1ef3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce837fe700 1 -- 172.21.2.104:0/1892058403 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1ef6180 msgr2=0x1eed970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce837fe700 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1ef6180 0x1eed970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce837fe700 1 -- 172.21.2.104:0/1892058403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce840a9e00 con 0x1eb6ae0 2021-07-11T03:58:50.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce837fe700 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef3270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fce6c007f80 tx=0x7fce6c007000).ready entity=mon.0 client_cookie=b5bc0b4f4227bb16 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce627fc700 10 client.0 ms_handle_connect on v2:172.21.2.104:3300/0 2021-07-11T03:58:50.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce627fc700 1 -- 172.21.2.104:0/1892058403 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce6c017e30 con 0x1eb6ae0 2021-07-11T03:58:50.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce627fc700 1 -- 172.21.2.104:0/1892058403 <== mon.0 v2:172.21.2.104:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fce6c012ba0 con 0x1eb6ae0 2021-07-11T03:58:50.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- 172.21.2.104:0/1892058403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1ca4c70 con 0x1eb6ae0 2021-07-11T03:58:50.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce627fc700 1 -- 172.21.2.104:0/1892058403 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce6c012d60 con 0x1eb6ae0 2021-07-11T03:58:50.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 10 client.6141 fetch_fsmap learned FSMap version 27 2021-07-11T03:58:50.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- 172.21.2.104:0/1892058403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x1e1b0a0 con 0x1eb6ae0 2021-07-11T03:58:50.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce627fc700 1 -- 172.21.2.104:0/1892058403 <== mon.0 v2:172.21.2.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c025030 con 0x1eb6ae0 2021-07-11T03:58:50.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce627fc700 1 -- 172.21.2.104:0/1892058403 <== mon.0 v2:172.21.2.104:3300/0 5 ==== fsmap(e 27) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fce6c014a10 con 0x1eb6ae0 2021-07-11T03:58:50.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 10 client.6141 fetch_fsmap finished waiting for FSMap version 27 2021-07-11T03:58:50.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 10 client.6141 resolve_mds: resolved ID 'c' to GID 5952 2021-07-11T03:58:50.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 20 client.6141 populate_metadata read hostname 'gibba004' 2021-07-11T03:58:50.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.104:6834/225259368,v1:172.21.2.104:6835/225259368] conn(0x1ef5c90 0x1f03370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 4 client.6141 mds_command: new command op to 5952 tid=0[{"prefix": "get_command_descriptions"}] 2021-07-11T03:58:50.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce94310740 1 -- 172.21.2.104:0/1892058403 --> [v2:172.21.2.104:6834/225259368,v1:172.21.2.104:6835/225259368] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x1cba4b0 con 0x1ef5c90 2021-07-11T03:58:50.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.837+0000 7fce82ffd700 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.104:6834/225259368,v1:172.21.2.104:6835/225259368] conn(0x1ef5c90 0x1f03370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce82ffd700 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.104:6834/225259368,v1:172.21.2.104:6835/225259368] conn(0x1ef5c90 0x1f03370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=fb3789530956fed0 server_cookie=4b41dd30f53b56f5 in_seq=0 out_seq=0 2021-07-11T03:58:50.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce627fc700 10 client.6141 ms_handle_connect on v2:172.21.2.104:6834/225259368 2021-07-11T03:58:50.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce627fc700 1 -- 172.21.2.104:0/1892058403 <== mds.0 v2:172.21.2.104:6834/225259368 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17002 (crc 0 0 0) 0x1cba4b0 con 0x1ef5c90 2021-07-11T03:58:50.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce627fc700 10 client.6141 handle_command_reply: tid=0 2021-07-11T03:58:50.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 1 client.6141 shutdown 2021-07-11T03:58:50.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 1 -- 172.21.2.104:0/1892058403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 msgr2=0x1ef3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef3270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fce6c007f80 tx=0x7fce6c007000).stop 2021-07-11T03:58:50.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 1 -- 172.21.2.104:0/1892058403 shutdown_connections 2021-07-11T03:58:50.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.104:6834/225259368,v1:172.21.2.104:6835/225259368] conn(0x1ef5c90 0x1f03370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1ef6180 0x1eed970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eefaf0 0x1ef3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 1 --2- 172.21.2.104:0/1892058403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef3270 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 1 -- 172.21.2.104:0/1892058403 >> 172.21.2.104:0/1892058403 conn(0x1e45950 msgr2=0x1eba4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:50.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce627fc700 0 client.6141 ms_handle_reset on v2:172.21.2.104:6834/225259368 2021-07-11T03:58:50.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 1 -- 172.21.2.104:0/1892058403 shutdown_connections 2021-07-11T03:58:50.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 1 -- 172.21.2.104:0/1892058403 wait complete. 2021-07-11T03:58:50.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.841+0000 7fce94310740 20 client.6141 trim_cache size 0 max 16384 2021-07-11T03:58:50.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.845+0000 7fce94310740 1 Processor -- start 2021-07-11T03:58:50.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.845+0000 7fce94310740 1 -- start start 2021-07-11T03:58:50.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1eefaf0 0x1ec0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1ef6180 0x1f03280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1e2c3e0 con 0x1eb6ae0 2021-07-11T03:58:50.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eba2b0 con 0x1eefaf0 2021-07-11T03:58:50.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x1eef9a0 con 0x1ef6180 2021-07-11T03:58:50.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce837fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce837fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45840/0 (socket says 172.21.2.104:45840) 2021-07-11T03:58:50.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce837fe700 1 -- 172.21.2.104:0/1453853066 learned_addr learned my addr 172.21.2.104:0/1453853066 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:50.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce837fe700 1 -- 172.21.2.104:0/1453853066 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1ef6180 msgr2=0x1f03280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce837fe700 1 --2- 172.21.2.104:0/1453853066 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1ef6180 0x1f03280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce837fe700 1 -- 172.21.2.104:0/1453853066 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1eefaf0 msgr2=0x1ec0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce837fe700 1 --2- 172.21.2.104:0/1453853066 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1eefaf0 0x1ec0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce837fe700 1 -- 172.21.2.104:0/1453853066 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8414d3d0 con 0x1eb6ae0 2021-07-11T03:58:50.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce837fe700 1 --2- 172.21.2.104:0/1453853066 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef1230 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x1ef6140 tx=0x7fce6c014000).ready entity=mon.0 client_cookie=970a348d61d9ffe0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce78ff9700 1 -- 172.21.2.104:0/1453853066 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce6c017e30 con 0x1eb6ae0 2021-07-11T03:58:50.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce78ff9700 1 -- 172.21.2.104:0/1453853066 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c025070 con 0x1eb6ae0 2021-07-11T03:58:50.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce78ff9700 1 -- 172.21.2.104:0/1453853066 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce6c0145f0 con 0x1eb6ae0 2021-07-11T03:58:50.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- 172.21.2.104:0/1453853066 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 msgr2=0x1ef1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:58:50.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 --2- 172.21.2.104:0/1453853066 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef1230 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x1ef6140 tx=0x7fce6c014000).stop 2021-07-11T03:58:50.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- 172.21.2.104:0/1453853066 shutdown_connections 2021-07-11T03:58:50.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 --2- 172.21.2.104:0/1453853066 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1ef6180 0x1f03280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 --2- 172.21.2.104:0/1453853066 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1eefaf0 0x1ec0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 --2- 172.21.2.104:0/1453853066 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eb6ae0 0x1ef1230 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:58:50.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- 172.21.2.104:0/1453853066 >> 172.21.2.104:0/1453853066 conn(0x1e45950 msgr2=0x1eed430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:58:50.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- 172.21.2.104:0/1453853066 shutdown_connections 2021-07-11T03:58:50.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- 172.21.2.104:0/1453853066 wait complete. 2021-07-11T03:58:50.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- start start 2021-07-11T03:58:50.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 Processor -- start 2021-07-11T03:58:50.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eb6ae0 0x1f554a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1f55990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1ef6180 0x1f575a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x1eb79c0 con 0x1eefaf0 2021-07-11T03:58:50.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x1ebadb0 con 0x1ef6180 2021-07-11T03:58:50.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x1f54cf0 con 0x1eb6ae0 2021-07-11T03:58:50.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 10 client.0 ll_register_callbacks cb 0x7fce740020f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2021-07-11T03:58:50.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce82ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1f55990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce82ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1f55990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45844/0 (socket says 172.21.2.104:45844) 2021-07-11T03:58:50.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce82ffd700 1 -- 172.21.2.104:0/2345805789 learned_addr learned my addr 172.21.2.104:0/2345805789 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:58:50.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce82ffd700 1 -- 172.21.2.104:0/2345805789 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eb6ae0 msgr2=0x1f554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce82ffd700 1 --2- 172.21.2.104:0/2345805789 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eb6ae0 0x1f554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce82ffd700 1 -- 172.21.2.104:0/2345805789 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1ef6180 msgr2=0x1f575a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:58:50.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce82ffd700 1 --2- 172.21.2.104:0/2345805789 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1ef6180 0x1f575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:58:50.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce82ffd700 1 -- 172.21.2.104:0/2345805789 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x1e1b0a0 con 0x1eefaf0 2021-07-11T03:58:50.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce82ffd700 1 --2- 172.21.2.104:0/2345805789 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1f55990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7fce74007000 tx=0x7fce74002190).ready entity=mon.0 client_cookie=6b6d4429f778e894 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:58:50.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce637fe700 10 client.0 ms_handle_connect on v2:172.21.2.104:3300/0 2021-07-11T03:58:50.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce637fe700 1 -- 172.21.2.104:0/2345805789 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce74017030 con 0x1eefaf0 2021-07-11T03:58:50.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce637fe700 1 -- 172.21.2.104:0/2345805789 <== mon.0 v2:172.21.2.104:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fce74018070 con 0x1eefaf0 2021-07-11T03:58:50.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- 172.21.2.104:0/2345805789 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x1d35820 con 0x1eefaf0 2021-07-11T03:58:50.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce637fe700 1 -- 172.21.2.104:0/2345805789 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce74006920 con 0x1eefaf0 2021-07-11T03:58:50.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 10 client.6147 fetch_fsmap learned FSMap version 27 2021-07-11T03:58:50.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- 172.21.2.104:0/2345805789 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=2+}) v3 -- 0x1e23c90 con 0x1eefaf0 2021-07-11T03:58:50.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce637fe700 1 -- 172.21.2.104:0/2345805789 <== mon.0 v2:172.21.2.104:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x1d35820 con 0x1eefaf0 2021-07-11T03:58:50.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce637fe700 1 -- 172.21.2.104:0/2345805789 <== mon.0 v2:172.21.2.104:3300/0 5 ==== fsmap(e 27) v1 ==== 1434+0+0 (secure 0 0 0) 0x7fce74013a80 con 0x1eefaf0 2021-07-11T03:58:50.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 10 client.6147 fetch_fsmap finished waiting for FSMap version 27 2021-07-11T03:58:50.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 10 client.6147 resolve_mds: resolved ID 'c' to GID 5952 2021-07-11T03:58:50.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 20 client.6147 populate_metadata read hostname 'gibba004' 2021-07-11T03:58:50.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 --2- 172.21.2.104:0/2345805789 >> [v2:172.21.2.104:6834/225259368,v1:172.21.2.104:6835/225259368] conn(0x1f1d190 0x1f57a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:58:50.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 4 client.6147 mds_command: new command op to 5952 tid=0[{"prefix": "cache drop", "timeout": 10}] 2021-07-11T03:58:50.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce94310740 1 -- 172.21.2.104:0/2345805789 --> [v2:172.21.2.104:6834/225259368,v1:172.21.2.104:6835/225259368] -- command(tid 0: {"prefix": "cache drop", "timeout": 10}) v1 -- 0x1eee450 con 0x1f1d190 2021-07-11T03:58:50.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.849+0000 7fce837fe700 1 --2- 172.21.2.104:0/2345805789 >> [v2:172.21.2.104:6834/225259368,v1:172.21.2.104:6835/225259368] conn(0x1f1d190 0x1f57a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:58:50.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.853+0000 7fce837fe700 1 --2- 172.21.2.104:0/2345805789 >> [v2:172.21.2.104:6834/225259368,v1:172.21.2.104:6835/225259368] conn(0x1f1d190 0x1f57a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=48bf5ee0f4dfe142 server_cookie=c3c527e57a6b5224 in_seq=0 out_seq=0 2021-07-11T03:58:50.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:58:50.853+0000 7fce637fe700 10 client.6147 ms_handle_connect on v2:172.21.2.104:6834/225259368 2021-07-11T03:59:01.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce637fe700 1 -- 172.21.2.104:0/2345805789 <== mds.0 v2:172.21.2.104:6834/225259368 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+400 (crc 0 0 0) 0x7fce6c0251a0 con 0x1f1d190 2021-07-11T03:59:01.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce637fe700 10 client.6147 handle_command_reply: tid=0 2021-07-11T03:59:01.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 1 client.6147 shutdown 2021-07-11T03:59:01.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 1 -- 172.21.2.104:0/2345805789 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 msgr2=0x1f55990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:01.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 1 --2- 172.21.2.104:0/2345805789 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1f55990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7fce74007000 tx=0x7fce74002190).stop 2021-07-11T03:59:01.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 1 -- 172.21.2.104:0/2345805789 shutdown_connections 2021-07-11T03:59:01.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 1 --2- 172.21.2.104:0/2345805789 >> [v2:172.21.2.104:6834/225259368,v1:172.21.2.104:6835/225259368] conn(0x1f1d190 0x1f57a90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:01.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 1 --2- 172.21.2.104:0/2345805789 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x1ef6180 0x1f575a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:01.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce637fe700 0 client.6147 ms_handle_reset on v2:172.21.2.104:6834/225259368 2021-07-11T03:59:01.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 1 --2- 172.21.2.104:0/2345805789 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x1eefaf0 0x1f55990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:01.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 1 --2- 172.21.2.104:0/2345805789 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x1eb6ae0 0x1f554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:01.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 1 -- 172.21.2.104:0/2345805789 >> 172.21.2.104:0/2345805789 conn(0x1e45950 msgr2=0x1eed430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:01.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 1 -- 172.21.2.104:0/2345805789 shutdown_connections 2021-07-11T03:59:01.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 1 -- 172.21.2.104:0/2345805789 wait complete. 2021-07-11T03:59:01.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.029+0000 7fce94310740 20 client.6147 trim_cache size 0 max 16384 2021-07-11T03:59:01.044 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:59:01.045 INFO:teuthology.orchestra.run.gibba004.stdout: "client_recall": { 2021-07-11T03:59:01.045 INFO:teuthology.orchestra.run.gibba004.stdout: "return_code": -110, 2021-07-11T03:59:01.045 INFO:teuthology.orchestra.run.gibba004.stdout: "message": "(110) Connection timed out", 2021-07-11T03:59:01.045 INFO:teuthology.orchestra.run.gibba004.stdout: "recalled": 240 2021-07-11T03:59:01.045 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:01.046 INFO:teuthology.orchestra.run.gibba004.stdout: "flush_journal": { 2021-07-11T03:59:01.046 INFO:teuthology.orchestra.run.gibba004.stdout: "return_code": 0, 2021-07-11T03:59:01.046 INFO:teuthology.orchestra.run.gibba004.stdout: "message": "" 2021-07-11T03:59:01.046 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:01.047 INFO:teuthology.orchestra.run.gibba004.stdout: "trim_cache": { 2021-07-11T03:59:01.047 INFO:teuthology.orchestra.run.gibba004.stdout: "trimmed": 220 2021-07-11T03:59:01.047 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:01.047 INFO:teuthology.orchestra.run.gibba004.stdout: "cache": { 2021-07-11T03:59:01.048 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": { 2021-07-11T03:59:01.048 INFO:teuthology.orchestra.run.gibba004.stdout: "items": 15428, 2021-07-11T03:59:01.048 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes": 2428304 2021-07-11T03:59:01.048 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:01.049 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:01.049 INFO:teuthology.orchestra.run.gibba004.stdout: "duration": 10.177662928 2021-07-11T03:59:01.049 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:59:01.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 -- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce680532b0 msgr2=0x7fce68055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:01.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce680532b0 0x7fce68055770 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fce6c008000 tx=0x7fce6c009040).stop 2021-07-11T03:59:01.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 -- 172.21.2.104:0/3187044127 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce840ad310 msgr2=0x7fce840ad790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:01.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce840ad310 0x7fce840ad790 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fce6c001370 tx=0x7fce7c008040).stop 2021-07-11T03:59:01.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 -- 172.21.2.104:0/3187044127 shutdown_connections 2021-07-11T03:59:01.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce680532b0 0x7fce68055770 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:01.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce840ad310 0x7fce840ad790 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:01.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce840a9420 0x7fce840aaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:01.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 --2- 172.21.2.104:0/3187044127 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce84140350 0x7fce840aa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:01.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 -- 172.21.2.104:0/3187044127 >> 172.21.2.104:0/3187044127 conn(0x7fce8405f6b0 msgr2=0x7fce8413a640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:01.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 -- 172.21.2.104:0/3187044127 shutdown_connections 2021-07-11T03:59:01.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.033+0000 7fce8a0c0700 1 -- 172.21.2.104:0/3187044127 wait complete. 2021-07-11T03:59:01.056 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T03:59:01.056 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:59:01.070 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:59:01.098 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:59:01.116 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:59:01.638 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T03:59:01.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:01.640 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:59:01.641 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:59:01.654 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:59:01.656 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:59:01.656 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T03:59:01.657 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T03:59:01.657 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:59:01.672 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15049]: fuse finished with error 0 and tester_r 0 2021-07-11T03:59:01.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:01.674 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:59:01.674 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:59:01.717 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T03:59:01.718 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T03:59:01.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:59:01.839 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:59:01.850 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:01.851 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:59:01.851 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:59:01.861 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:59:01.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:59:01.863 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T03:59:01.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T03:59:01.864 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:59:01.873 INFO:teuthology.orchestra.run.gibba023.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:59:01.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:59:01.874 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T03:59:01.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 Processor -- start 2021-07-11T03:59:01.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 -- start start 2021-07-11T03:59:01.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad840b1db0 0x7fad840b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:01.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad840b0350 con 0x7fad840a70c0 2021-07-11T03:59:01.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad840b05f0 con 0x7fad841483a0 2021-07-11T03:59:01.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840a90c0 con 0x7fad840b1db0 2021-07-11T03:59:01.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8a393700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fad840a70c0 0x7fad84144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42626/0 (socket says 172.21.2.104:42626) 2021-07-11T03:59:01.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8a393700 1 -- 172.21.2.104:0/3311854275 learned_addr learned my addr 172.21.2.104:0/3311854275 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:01.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582598719 0 0) 0x7fad840b0350 con 0x7fad840a70c0 2021-07-11T03:59:01.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad6c003260 con 0x7fad840a70c0 2021-07-11T03:59:01.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad89b92700 1 --2- 172.21.2.104:0/3311854275 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad840b1db0 0x7fad840b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:01.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2266615821 0 0) 0x7fad6c003260 con 0x7fad840a70c0 2021-07-11T03:59:01.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad840b1db0 msgr2=0x7fad840b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:01.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 --2- 172.21.2.104:0/3311854275 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad840b1db0 0x7fad840b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:01.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 >> v1:172.21.2.105:6789/0 conn(0x7fad841483a0 legacy=0x7fad840b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:01.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840b0890 con 0x7fad840a70c0 2021-07-11T03:59:01.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fad7c0019e0 con 0x7fad840a70c0 2021-07-11T03:59:01.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 >> v1:172.21.2.104:6789/0 conn(0x7fad840a70c0 legacy=0x7fad84144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:01.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 --2- 172.21.2.104:0/3311854275 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad840b1db0 0x7fad6c003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:01.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 --2- 172.21.2.104:0/3311854275 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad6c003e30 0x7fad6c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:01.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 --2- 172.21.2.104:0/3311854275 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6c004b10 0x7fad6c004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:01.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00b960 con 0x7fad6c004b10 2021-07-11T03:59:01.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00bad0 con 0x7fad6c003e30 2021-07-11T03:59:01.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c00bc90 con 0x7fad840b1db0 2021-07-11T03:59:01.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8a393700 1 --2- 172.21.2.104:0/3311854275 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6c004b10 0x7fad6c004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:01.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8a393700 1 -- 172.21.2.104:0/3311854275 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad840b1db0 msgr2=0x7fad6c003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:01.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8a393700 1 --2- 172.21.2.104:0/3311854275 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad840b1db0 0x7fad6c003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:01.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8a393700 1 -- 172.21.2.104:0/3311854275 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad6c003e30 msgr2=0x7fad6c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:01.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8a393700 1 --2- 172.21.2.104:0/3311854275 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad6c003e30 0x7fad6c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:01.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8a393700 1 -- 172.21.2.104:0/3311854275 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c003260 con 0x7fad6c004b10 2021-07-11T03:59:01.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8a393700 1 --2- 172.21.2.104:0/3311854275 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6c004b10 0x7fad6c004f90 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fad7c0016f0 tx=0x7fad840b0780).ready entity=mon.0 client_cookie=ada2cde6806f376d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:01.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad7c00cba0 con 0x7fad6c004b10 2021-07-11T03:59:01.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c00cd00 con 0x7fad6c004b10 2021-07-11T03:59:01.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad88b90700 1 -- 172.21.2.104:0/3311854275 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad7c005b80 con 0x7fad6c004b10 2021-07-11T03:59:01.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 -- 172.21.2.104:0/3311854275 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6c004b10 msgr2=0x7fad6c004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:01.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 --2- 172.21.2.104:0/3311854275 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad6c004b10 0x7fad6c004f90 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fad7c0016f0 tx=0x7fad840b0780).stop 2021-07-11T03:59:01.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8a393700 1 -- 172.21.2.104:0/3311854275 reap_dead start 2021-07-11T03:59:01.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 -- 172.21.2.104:0/3311854275 shutdown_connections 2021-07-11T03:59:01.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 -- 172.21.2.104:0/3311854275 >> 172.21.2.104:0/3311854275 conn(0x7fad8405f6b0 msgr2=0x7fad8405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:01.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 -- 172.21.2.104:0/3311854275 shutdown_connections 2021-07-11T03:59:01.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.925+0000 7fad8bdf6700 1 -- 172.21.2.104:0/3311854275 wait complete. 2021-07-11T03:59:01.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad8bdf6700 1 Processor -- start 2021-07-11T03:59:01.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad8bdf6700 1 -- start start 2021-07-11T03:59:01.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad8bdf6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad8414af50 0x7fad84156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:01.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad8bdf6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad840627c0 0x7fad840ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:01.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad8bdf6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad840660d0 0x7fad84066580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:01.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad8bdf6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8414b330 con 0x7fad840627c0 2021-07-11T03:59:01.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad8bdf6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad84067850 con 0x7fad8414af50 2021-07-11T03:59:01.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad8bdf6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad840ab280 con 0x7fad840660d0 2021-07-11T03:59:01.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad89391700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad840627c0 0x7fad840ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:01.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad89391700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad840627c0 0x7fad840ac4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45870/0 (socket says 172.21.2.104:45870) 2021-07-11T03:59:01.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad89391700 1 -- 172.21.2.104:0/2127600635 learned_addr learned my addr 172.21.2.104:0/2127600635 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:01.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad89391700 1 -- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad840660d0 msgr2=0x7fad84066580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:01.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad89391700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad840660d0 0x7fad84066580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:01.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad89391700 1 -- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad8414af50 msgr2=0x7fad84156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:01.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad89391700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad8414af50 0x7fad84156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:01.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad89391700 1 -- 172.21.2.104:0/2127600635 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad840ac000 con 0x7fad840627c0 2021-07-11T03:59:01.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad89391700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad840627c0 0x7fad840ac4a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fad84061440 tx=0x7fad74007ba0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:01.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad72ffd700 1 -- 172.21.2.104:0/2127600635 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad7400ce30 con 0x7fad840627c0 2021-07-11T03:59:01.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad72ffd700 1 -- 172.21.2.104:0/2127600635 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7400f470 con 0x7fad840627c0 2021-07-11T03:59:01.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad8bdf6700 1 -- 172.21.2.104:0/2127600635 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad84156af0 con 0x7fad840627c0 2021-07-11T03:59:01.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad8bdf6700 1 -- 172.21.2.104:0/2127600635 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad840aa980 con 0x7fad840627c0 2021-07-11T03:59:01.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad72ffd700 1 -- 172.21.2.104:0/2127600635 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fad7400f7d0 con 0x7fad840627c0 2021-07-11T03:59:01.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad8bdf6700 1 -- 172.21.2.104:0/2127600635 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad840abe40 con 0x7fad840627c0 2021-07-11T03:59:01.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad72ffd700 1 -- 172.21.2.104:0/2127600635 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fad74010070 con 0x7fad840627c0 2021-07-11T03:59:01.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad72ffd700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad640532d0 0x7fad64055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:01.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.929+0000 7fad72ffd700 1 -- 172.21.2.104:0/2127600635 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fad7406a790 con 0x7fad840627c0 2021-07-11T03:59:01.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.933+0000 7fad89b92700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad640532d0 0x7fad64055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:01.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.933+0000 7fad89b92700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad640532d0 0x7fad64055790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fad8000a000 tx=0x7fad80008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:01.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:01.933+0000 7fad72ffd700 1 -- 172.21.2.104:0/2127600635 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fad740419c0 con 0x7fad840627c0 2021-07-11T03:59:02.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.013+0000 7fad8bdf6700 1 -- 172.21.2.104:0/2127600635 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fad840ac190 con 0x7fad840627c0 2021-07-11T03:59:02.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.013+0000 7fad72ffd700 1 -- 172.21.2.104:0/2127600635 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v66) v1 ==== 81+0+33648 (secure 0 0 0) 0x7fad74044fe0 con 0x7fad840627c0 2021-07-11T03:59:02.022 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:02.023 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:59:02.023 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 66, 2021-07-11T03:59:02.023 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T03:59:02.023 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T03:59:02.023 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T03:58:28.560561+0000", 2021-07-11T03:59:02.024 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:59:02.024 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:59:02.024 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T03:59:02.024 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T03:59:02.025 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T03:59:02.025 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T03:59:02.025 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T03:59:02.025 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T03:59:02.025 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T03:59:02.026 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:59:02.026 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T03:59:02.026 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T03:59:02.026 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T03:59:02.027 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T03:59:02.027 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T03:59:02.027 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 12, 2021-07-11T03:59:02.027 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T03:59:02.027 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T03:59:02.028 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T03:59:02.028 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T03:59:02.028 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T03:59:02.028 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.029 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T03:59:02.029 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T03:59:02.029 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T03:59:02.029 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:59:02.029 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:59:02.030 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:59:02.030 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:59:02.030 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:59:02.030 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:59:02.030 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:59:02.031 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:59:02.031 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T03:59:02.031 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T03:59:02.031 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T03:59:02.032 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T03:59:02.032 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T03:59:02.032 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:59:02.032 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:59:02.032 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:59:02.033 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:59:02.033 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:59:02.033 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:59:02.033 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:59:02.034 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.034 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T03:59:02.034 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:59:02.034 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:59:02.034 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:59:02.035 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:59:02.035 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:59:02.035 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:59:02.035 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:59:02.035 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:59:02.036 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:59:02.036 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:59:02.036 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:59:02.036 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:59:02.037 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:59:02.037 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:59:02.037 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:59:02.037 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:59:02.037 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:59:02.038 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:59:02.038 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:59:02.038 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:59:02.038 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:59:02.038 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:59:02.039 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:59:02.039 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:59:02.039 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:59:02.039 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:59:02.040 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.040 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:59:02.040 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:59:02.040 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:59:02.040 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:59:02.041 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:59:02.041 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:59:02.041 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:59:02.041 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:59:02.041 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:59:02.042 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:59:02.042 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:59:02.042 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:59:02.042 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T03:59:02.043 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.043 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:59:02.043 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T03:59:02.043 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.044 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.044 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.044 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T03:59:02.044 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T03:59:02.044 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T03:59:02.045 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:59:02.045 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:59:02.045 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:59:02.045 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:59:02.045 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:59:02.046 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:59:02.046 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:59:02.046 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:59:02.046 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:59:02.047 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:59:02.047 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:59:02.047 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:59:02.047 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:59:02.047 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:59:02.048 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:59:02.048 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:59:02.048 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:59:02.048 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:59:02.048 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:59:02.049 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:59:02.049 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.049 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T03:59:02.049 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:59:02.050 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:59:02.050 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:59:02.050 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:59:02.050 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:59:02.050 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:59:02.051 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:59:02.051 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:59:02.051 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:59:02.051 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:59:02.052 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:59:02.052 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:59:02.052 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:59:02.052 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:59:02.052 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:59:02.053 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:59:02.053 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:59:02.053 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:59:02.053 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:59:02.053 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:59:02.054 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:59:02.054 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:59:02.054 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:59:02.054 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:59:02.055 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:59:02.055 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:59:02.055 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.055 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:59:02.055 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:59:02.056 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:59:02.056 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:59:02.056 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:59:02.056 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:59:02.057 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:59:02.057 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:59:02.057 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:59:02.057 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:59:02.057 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:59:02.058 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:59:02.058 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:59:02.058 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T03:59:02.058 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.058 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.059 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.059 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 11, 2021-07-11T03:59:02.059 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T03:59:02.059 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:58:25.543830+0000", 2021-07-11T03:59:02.060 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:59:02.060 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:59:02.060 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:59:02.060 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:59:02.060 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:59:02.061 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:59:02.061 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:59:02.061 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:59:02.061 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:59:02.061 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:59:02.062 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:59:02.062 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:59:02.062 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:59:02.062 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:59:02.063 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:59:02.063 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:59:02.063 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:59:02.063 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:59:02.063 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:59:02.064 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:59:02.064 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.064 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "65", 2021-07-11T03:59:02.064 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:59:02.065 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:59:02.065 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:59:02.065 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:59:02.065 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:59:02.065 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:59:02.066 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:59:02.066 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:59:02.066 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:59:02.066 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:59:02.067 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:59:02.067 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:59:02.067 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:59:02.067 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:59:02.067 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:59:02.068 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:59:02.068 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:59:02.068 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:59:02.068 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:59:02.069 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:59:02.069 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:59:02.069 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:59:02.069 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:59:02.069 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:59:02.070 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:59:02.070 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:59:02.070 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.070 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:59:02.071 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:59:02.071 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:59:02.071 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:59:02.071 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:59:02.071 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:59:02.072 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:59:02.072 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:59:02.072 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:59:02.072 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:59:02.072 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:59:02.073 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:59:02.073 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T03:59:02.073 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T03:59:02.073 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T03:59:02.073 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.074 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:59:02.074 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T03:59:02.074 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T03:59:02.074 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.075 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.075 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.075 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.075 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 12, 2021-07-11T03:59:02.075 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T03:59:02.076 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:58:26.545938+0000", 2021-07-11T03:59:02.076 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:59:02.076 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:59:02.076 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:59:02.077 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:59:02.077 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:59:02.077 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:59:02.077 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:59:02.077 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:59:02.078 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:59:02.078 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:59:02.078 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:59:02.078 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:59:02.078 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:59:02.079 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:59:02.079 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:59:02.079 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:59:02.079 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:59:02.080 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:59:02.080 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:59:02.080 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:59:02.080 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.080 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "66", 2021-07-11T03:59:02.081 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:59:02.081 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:59:02.081 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:59:02.081 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:59:02.081 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:59:02.082 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:59:02.082 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:59:02.082 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:59:02.082 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:59:02.083 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:59:02.083 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:59:02.083 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:59:02.083 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:59:02.084 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:59:02.084 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:59:02.084 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:59:02.084 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:59:02.084 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:59:02.085 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:59:02.085 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:59:02.085 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:59:02.085 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:59:02.086 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:59:02.086 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:59:02.086 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:59:02.086 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:59:02.086 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.087 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:59:02.087 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:59:02.087 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:59:02.087 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:59:02.087 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:59:02.088 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:59:02.088 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:59:02.088 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:59:02.088 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:59:02.089 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:59:02.089 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:59:02.089 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:59:02.089 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:59:02.089 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T03:59:02.090 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T03:59:02.090 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.090 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.090 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.090 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:59:02.091 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T03:59:02.091 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.091 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:59:02.091 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T03:59:02.092 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:02.092 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:02.092 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:02.092 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:02.092 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:02.093 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:02.093 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:02.093 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 62, 2021-07-11T03:59:02.093 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:02.093 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:02.094 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:02.094 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.094 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.094 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.095 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T03:59:02.095 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:02.095 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.095 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.095 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.096 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T03:59:02.096 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:02.096 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.096 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.097 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.097 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:02.097 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.097 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.097 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.098 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T03:59:02.098 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:02.098 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.098 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.098 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.099 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T03:59:02.099 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:02.099 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.099 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.100 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.100 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:02.100 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.100 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.100 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.101 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T03:59:02.101 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:02.101 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.101 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.101 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.102 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T03:59:02.102 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:02.102 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.102 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.103 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.103 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:02.103 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.103 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.103 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.104 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T03:59:02.104 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:02.104 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.104 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.104 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.105 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T03:59:02.105 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:02.105 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.105 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.106 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.106 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T03:59:02.106 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T03:59:02.106 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T03:59:02.106 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T03:59:02.107 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:02.107 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:02.107 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:02.107 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.108 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.108 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.108 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:59:02.108 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T03:59:02.108 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:02.109 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:02.109 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:02.109 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:02.109 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:02.109 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:02.110 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:02.110 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 62, 2021-07-11T03:59:02.110 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:02.110 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:02.111 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:02.111 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.111 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.111 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.111 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T03:59:02.112 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:02.112 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.112 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.112 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.113 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T03:59:02.113 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:02.113 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.113 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.113 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.114 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:02.114 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.114 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.114 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.114 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T03:59:02.115 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:02.115 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.115 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.115 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.116 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T03:59:02.116 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:02.116 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.116 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.116 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.117 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:02.117 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.117 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.117 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.117 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T03:59:02.118 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:02.118 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.118 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.118 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.119 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T03:59:02.119 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:02.119 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.119 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.120 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.120 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:02.120 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.120 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.120 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.121 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T03:59:02.121 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:02.121 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.121 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.121 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.122 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T03:59:02.122 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:02.122 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.122 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.123 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.123 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T03:59:02.123 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T03:59:02.123 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T03:59:02.123 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T03:59:02.124 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:02.124 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:02.124 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:02.124 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.124 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.125 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.125 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:59:02.125 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T03:59:02.125 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:02.126 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:02.126 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:02.126 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:02.126 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:02.126 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:02.127 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:02.127 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 61, 2021-07-11T03:59:02.127 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:02.127 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:02.127 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:02.128 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.128 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.128 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.128 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T03:59:02.128 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:02.129 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.129 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.129 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.129 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T03:59:02.130 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:02.130 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.130 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.130 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.130 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:02.131 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.131 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.131 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.131 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T03:59:02.132 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:02.132 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.132 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.132 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.132 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T03:59:02.133 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:02.133 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.133 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.133 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.133 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:02.134 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.134 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.134 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.134 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T03:59:02.135 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:02.135 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.135 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.135 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.135 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T03:59:02.136 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:02.136 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.136 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.136 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.136 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:02.137 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.137 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.137 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.137 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T03:59:02.138 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:02.138 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.138 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.138 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.138 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T03:59:02.139 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:02.139 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.139 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.139 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.139 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T03:59:02.140 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T03:59:02.140 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T03:59:02.140 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T03:59:02.140 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:02.141 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:02.141 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:02.141 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.141 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.141 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.142 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:59:02.142 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T03:59:02.142 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:02.142 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:02.143 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:02.143 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:02.143 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:02.143 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:02.143 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:02.144 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 52, 2021-07-11T03:59:02.144 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:02.144 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:02.144 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:02.144 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.145 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.145 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.145 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T03:59:02.145 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:02.146 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.146 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.146 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.146 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T03:59:02.146 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:02.147 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.147 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.147 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.147 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:02.147 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.148 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.148 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.148 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T03:59:02.148 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:02.149 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.149 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.149 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.149 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T03:59:02.149 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:02.150 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.150 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.150 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.150 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:02.151 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.151 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.151 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.151 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T03:59:02.151 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:02.152 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.152 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.152 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.152 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T03:59:02.152 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:02.153 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.153 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.153 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.153 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:02.154 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.154 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.154 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.154 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T03:59:02.154 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:02.155 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.155 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.155 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.155 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T03:59:02.155 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:02.156 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.156 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.156 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.156 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T03:59:02.157 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T03:59:02.157 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T03:59:02.157 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T03:59:02.157 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:02.157 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:02.158 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:02.158 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.158 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.158 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.158 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:59:02.159 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T03:59:02.159 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:02.159 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:02.159 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:02.160 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:02.160 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:02.160 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:02.160 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:02.160 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 63, 2021-07-11T03:59:02.161 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:02.161 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:02.161 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:02.161 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.161 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.162 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.162 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T03:59:02.162 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:02.162 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.163 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.163 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.163 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T03:59:02.163 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:02.163 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.164 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.164 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.164 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:02.164 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.165 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.165 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.165 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T03:59:02.165 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:02.165 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.166 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.166 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.166 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T03:59:02.166 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:02.166 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.167 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.167 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.167 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:02.167 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.168 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.168 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.168 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T03:59:02.168 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:02.168 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.169 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.169 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.169 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T03:59:02.169 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:02.170 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.170 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.170 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.170 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:02.170 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.171 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.171 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.171 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T03:59:02.171 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:02.171 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.172 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.172 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.172 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T03:59:02.172 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:02.173 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.173 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.173 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.173 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T03:59:02.173 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T03:59:02.174 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T03:59:02.174 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T03:59:02.174 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:02.174 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:02.175 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:02.175 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.175 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.175 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.175 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:59:02.176 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T03:59:02.176 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:02.176 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:02.176 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:02.176 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:02.177 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:02.177 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:02.177 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:02.177 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 61, 2021-07-11T03:59:02.177 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:02.178 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:02.178 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:02.178 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.178 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.179 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.179 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T03:59:02.179 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:02.179 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.179 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.180 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.180 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T03:59:02.180 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:02.180 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.181 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.181 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.181 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:02.181 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.181 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.182 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.182 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T03:59:02.182 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:02.182 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.182 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.183 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.183 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T03:59:02.183 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:02.183 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.184 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.184 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.184 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:02.184 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.184 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.185 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.185 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T03:59:02.185 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:02.185 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.185 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.186 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.186 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T03:59:02.186 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:02.186 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.187 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.187 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.187 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:02.187 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.187 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.188 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.188 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T03:59:02.188 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:02.188 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.188 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.189 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.189 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T03:59:02.189 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:02.189 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.190 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.190 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.190 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T03:59:02.190 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T03:59:02.190 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T03:59:02.191 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T03:59:02.191 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:02.191 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:02.191 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:02.192 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.192 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.192 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.192 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:59:02.192 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T03:59:02.193 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:02.193 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:02.193 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:02.193 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:02.193 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:02.194 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:02.194 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:02.194 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 62, 2021-07-11T03:59:02.194 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:02.195 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:02.195 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:02.195 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.195 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.195 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.196 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T03:59:02.196 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:02.196 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.196 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.196 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.197 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T03:59:02.197 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:02.197 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.197 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.197 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.198 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:02.198 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.198 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.198 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.199 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T03:59:02.199 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:02.199 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.199 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.199 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.200 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T03:59:02.200 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:02.200 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.200 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.200 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.201 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:02.201 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.201 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.201 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.202 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T03:59:02.202 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:02.202 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.202 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.202 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.203 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T03:59:02.203 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:02.203 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.203 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.203 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.204 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:02.204 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.204 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.204 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.205 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T03:59:02.205 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:02.205 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.205 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.205 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.206 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T03:59:02.206 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:02.206 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.206 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.206 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.207 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T03:59:02.207 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T03:59:02.207 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T03:59:02.207 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T03:59:02.208 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:02.208 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:02.208 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:02.208 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.208 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.209 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.209 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:59:02.209 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T03:59:02.209 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:02.209 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:02.210 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:02.210 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:02.210 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:02.210 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:02.211 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:02.211 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 61, 2021-07-11T03:59:02.211 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:02.211 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:02.211 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:02.212 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.212 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.212 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.212 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T03:59:02.213 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:02.213 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.213 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.213 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.213 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T03:59:02.214 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:02.214 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.214 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.214 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.214 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:02.215 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.215 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.215 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.215 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T03:59:02.216 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:02.216 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.216 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.216 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.217 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T03:59:02.217 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:02.217 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.217 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.218 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.218 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:02.218 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.218 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.218 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.219 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T03:59:02.219 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:02.219 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.219 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.220 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.220 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T03:59:02.220 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:02.220 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.220 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.221 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.221 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:02.221 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:02.221 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.222 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:02.222 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T03:59:02.222 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:02.222 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.222 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.223 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:02.223 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T03:59:02.223 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:02.223 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.223 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.224 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.224 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T03:59:02.224 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T03:59:02.224 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T03:59:02.225 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T03:59:02.225 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:02.225 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:02.225 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:02.225 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:02.226 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.226 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:59:02.226 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T03:59:02.226 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.226 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:59:02.227 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:02.227 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:02.227 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:02.227 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:02.228 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:02.228 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T03:59:02.228 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:02.228 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.228 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.229 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:59:02.229 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:02.229 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:02.229 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:02.230 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:02.230 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:02.230 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T03:59:02.230 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:02.230 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.231 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.231 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:59:02.231 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:02.231 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:02.232 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:02.232 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:02.232 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:02.232 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T03:59:02.232 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:02.233 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.233 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.233 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:59:02.233 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:02.233 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:02.234 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:02.234 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:02.234 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:02.234 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T03:59:02.235 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:02.235 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.235 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.235 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:59:02.235 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:02.236 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:02.236 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:02.236 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:02.236 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:02.236 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T03:59:02.237 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:02.237 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.237 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.237 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:59:02.237 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:02.238 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:02.238 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:02.238 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:02.238 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:02.239 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T03:59:02.239 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:02.239 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.239 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.239 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:59:02.240 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:02.240 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:02.240 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:02.240 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:02.240 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:02.241 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T03:59:02.241 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:02.241 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.241 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:02.242 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:59:02.242 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:02.242 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:02.242 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:02.242 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:02.243 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:02.243 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T03:59:02.243 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:02.243 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.243 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:59:02.244 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T03:59:02.244 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T03:59:02.244 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T03:59:02.244 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T03:59:02.245 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": {}, 2021-07-11T03:59:02.245 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T03:59:02.245 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T03:59:02.245 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T03:59:02.245 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T03:59:02.246 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T03:59:02.246 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T03:59:02.246 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T03:59:02.246 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T03:59:02.246 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:02.247 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:02.247 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T03:59:02.247 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T03:59:02.247 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T03:59:02.248 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T03:59:02.248 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T03:59:02.248 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:59:02.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.013+0000 7fad8bdf6700 1 -- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad640532d0 msgr2=0x7fad64055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.013+0000 7fad8bdf6700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad640532d0 0x7fad64055790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fad8000a000 tx=0x7fad80008040).stop 2021-07-11T03:59:02.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.013+0000 7fad8bdf6700 1 -- 172.21.2.104:0/2127600635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad840627c0 msgr2=0x7fad840ac4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.013+0000 7fad8bdf6700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad840627c0 0x7fad840ac4a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fad84061440 tx=0x7fad74007ba0).stop 2021-07-11T03:59:02.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.017+0000 7fad8bdf6700 1 -- 172.21.2.104:0/2127600635 shutdown_connections 2021-07-11T03:59:02.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.017+0000 7fad8bdf6700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fad840660d0 0x7fad84066580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:02.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.017+0000 7fad8bdf6700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fad640532d0 0x7fad64055790 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.017+0000 7fad8bdf6700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fad840627c0 0x7fad840ac4a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.017+0000 7fad8bdf6700 1 --2- 172.21.2.104:0/2127600635 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fad8414af50 0x7fad84156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:02.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.017+0000 7fad8bdf6700 1 -- 172.21.2.104:0/2127600635 >> 172.21.2.104:0/2127600635 conn(0x7fad8405f6b0 msgr2=0x7fad840a8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:02.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.017+0000 7fad8bdf6700 1 -- 172.21.2.104:0/2127600635 shutdown_connections 2021-07-11T03:59:02.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.017+0000 7fad8bdf6700 1 -- 172.21.2.104:0/2127600635 wait complete. 2021-07-11T03:59:02.253 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:59:02.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.357+0000 7f14837f2700 1 Processor -- start 2021-07-11T03:59:02.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.357+0000 7f14837f2700 1 -- start start 2021-07-11T03:59:02.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f14837f2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 0x7f147c0b0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f14837f2700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f147c0a9410 0x7f147c0b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f14837f2700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f147c146860 con 0x7f147c0b1cc0 2021-07-11T03:59:02.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f14837f2700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147c05f750 con 0x7f147c0a9410 2021-07-11T03:59:02.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f14837f2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147c05f9f0 con 0x7f147c0b2810 2021-07-11T03:59:02.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f1480d8d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f147c0a9410 0x7f147c0b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:02.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f148158e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 0x7f147c0b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:02.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f1481d8f700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f147c0b1cc0 0x7f147c097970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:42016/0 (socket says 172.21.2.104:42016) 2021-07-11T03:59:02.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f1480d8d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f147c0a9410 0x7f147c0b17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:53434/0 (socket says 172.21.2.104:53434) 2021-07-11T03:59:02.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f148158e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 0x7f147c0b0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:49040/0 (socket says 172.21.2.104:49040) 2021-07-11T03:59:02.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f1481d8f700 1 -- 172.21.2.104:0/3985425472 learned_addr learned my addr 172.21.2.104:0/3985425472 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:02.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f148158e700 1 -- 172.21.2.104:0/3985425472 >> v2:172.21.2.105:3300/0 conn(0x7f147c0a9410 msgr2=0x7f147c0b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:02.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f148158e700 1 --2- 172.21.2.104:0/3985425472 >> v2:172.21.2.105:3300/0 conn(0x7f147c0a9410 0x7f147c0b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f148158e700 1 -- 172.21.2.104:0/3985425472 >> v1:172.21.2.105:6789/0 conn(0x7f147c0b1cc0 legacy=0x7f147c097970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f148158e700 1 -- 172.21.2.104:0/3985425472 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147c05fc90 con 0x7f147c0b2810 2021-07-11T03:59:02.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f1480d8d700 1 --2- 172.21.2.104:0/3985425472 >> v2:172.21.2.105:3300/0 conn(0x7f147c0a9410 0x7f147c0b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T03:59:02.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f148158e700 1 --2- 172.21.2.104:0/3985425472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 0x7f147c0b0810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1470002f70 tx=0x7f1470006df0).ready entity=mon.2 client_cookie=9eef2735d234662e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:02.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f146ffff700 1 -- 172.21.2.104:0/3985425472 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f147000d570 con 0x7f147c0b2810 2021-07-11T03:59:02.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f146ffff700 1 -- 172.21.2.104:0/3985425472 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1470000e90 con 0x7f147c0b2810 2021-07-11T03:59:02.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f146ffff700 1 -- 172.21.2.104:0/3985425472 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f147000d8c0 con 0x7f147c0b2810 2021-07-11T03:59:02.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f14837f2700 1 -- 172.21.2.104:0/3985425472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 msgr2=0x7f147c0b0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f14837f2700 1 --2- 172.21.2.104:0/3985425472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 0x7f147c0b0810 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1470002f70 tx=0x7f1470006df0).stop 2021-07-11T03:59:02.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f14837f2700 1 -- 172.21.2.104:0/3985425472 shutdown_connections 2021-07-11T03:59:02.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f14837f2700 1 --2- 172.21.2.104:0/3985425472 >> v2:172.21.2.105:3300/0 conn(0x7f147c0a9410 0x7f147c0b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f14837f2700 1 --2- 172.21.2.104:0/3985425472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 0x7f147c0b0810 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.361+0000 7f14837f2700 1 -- 172.21.2.104:0/3985425472 >> 172.21.2.104:0/3985425472 conn(0x7f147c062f10 msgr2=0x7f147c0632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:02.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 -- 172.21.2.104:0/3985425472 shutdown_connections 2021-07-11T03:59:02.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 -- 172.21.2.104:0/3985425472 wait complete. 2021-07-11T03:59:02.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 Processor -- start 2021-07-11T03:59:02.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 -- start start 2021-07-11T03:59:02.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f147c0a9410 0x7f147c0aba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f147c0b1cc0 0x7f147c140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 0x7f147c1412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c152b70 con 0x7f147c0a9410 2021-07-11T03:59:02.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147c0ac350 con 0x7f147c0b1cc0 2021-07-11T03:59:02.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147c152df0 con 0x7f147c0b2810 2021-07-11T03:59:02.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f148158e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f147c0a9410 0x7f147c0aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:02.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f148158e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f147c0a9410 0x7f147c0aba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45880/0 (socket says 172.21.2.104:45880) 2021-07-11T03:59:02.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f148158e700 1 -- 172.21.2.104:0/1118350524 learned_addr learned my addr 172.21.2.104:0/1118350524 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:02.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f1480d8d700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f147c0b1cc0 0x7f147c140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:02.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f1481d8f700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 0x7f147c1412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:02.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f148158e700 1 -- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 msgr2=0x7f147c1412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f148158e700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 0x7f147c1412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f148158e700 1 -- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f147c0b1cc0 msgr2=0x7f147c140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f148158e700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f147c0b1cc0 0x7f147c140dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f148158e700 1 -- 172.21.2.104:0/1118350524 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147800b180 con 0x7f147c0a9410 2021-07-11T03:59:02.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f148158e700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f147c0a9410 0x7f147c0aba30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f1470002f70 tx=0x7f1470014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:02.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f146e7fc700 1 -- 172.21.2.104:0/1118350524 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1470006b20 con 0x7f147c0a9410 2021-07-11T03:59:02.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f146e7fc700 1 -- 172.21.2.104:0/1118350524 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1470012e40 con 0x7f147c0a9410 2021-07-11T03:59:02.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 -- 172.21.2.104:0/1118350524 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147000b180 con 0x7f147c0a9410 2021-07-11T03:59:02.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f146e7fc700 1 -- 172.21.2.104:0/1118350524 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f147001f630 con 0x7f147c0a9410 2021-07-11T03:59:02.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 -- 172.21.2.104:0/1118350524 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147c0aa8d0 con 0x7f147c0a9410 2021-07-11T03:59:02.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f14837f2700 1 -- 172.21.2.104:0/1118350524 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f147c057350 con 0x7f147c0a9410 2021-07-11T03:59:02.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f146e7fc700 1 -- 172.21.2.104:0/1118350524 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1470025020 con 0x7f147c0a9410 2021-07-11T03:59:02.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f146e7fc700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f145c053160 0x7f145c055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f146e7fc700 1 -- 172.21.2.104:0/1118350524 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f147006c530 con 0x7f147c0a9410 2021-07-11T03:59:02.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.365+0000 7f1480d8d700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f145c053160 0x7f145c055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:02.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.369+0000 7f1480d8d700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f145c053160 0x7f145c055620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f147800b000 tx=0x7f1478006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:02.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.369+0000 7f146e7fc700 1 -- 172.21.2.104:0/1118350524 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f14700437e0 con 0x7f147c0a9410 2021-07-11T03:59:02.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.473+0000 7f14837f2700 1 -- 172.21.2.104:0/1118350524 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f147c0698e0 con 0x7f147c0a9410 2021-07-11T03:59:02.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.473+0000 7f146e7fc700 1 -- 172.21.2.104:0/1118350524 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+2490 (secure 0 0 0) 0x7f147000bea0 con 0x7f147c0a9410 2021-07-11T03:59:02.483 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:02.483 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":27,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/2908869555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2908869555},{"type":"v1","addr":"172.21.2.105:6835","nonce":2908869555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/1110128427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1110128427},{"type":"v1","addr":"172.21.2.104:6833","nonce":1110128427}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:58:27.556328+0000","modified":"2021-07-11T03:58:28.559390+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5952},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5952":{"gid":5952,"name":"c","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/225259368","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":225259368},{"type":"v1","addr":"172.21.2.104:6835","nonce":225259368}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-07-11T03:59:02.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 -- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f145c053160 msgr2=0x7f145c055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f145c053160 0x7f145c055620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f147800b000 tx=0x7f1478006df0).stop 2021-07-11T03:59:02.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 -- 172.21.2.104:0/1118350524 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f147c0a9410 msgr2=0x7f147c0aba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f147c0a9410 0x7f147c0aba30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f1470002f70 tx=0x7f1470014000).stop 2021-07-11T03:59:02.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 -- 172.21.2.104:0/1118350524 shutdown_connections 2021-07-11T03:59:02.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f147c0b2810 0x7f147c1412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f147c0b1cc0 0x7f147c140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f145c053160 0x7f145c055620 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 --2- 172.21.2.104:0/1118350524 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f147c0a9410 0x7f147c0aba30 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 -- 172.21.2.104:0/1118350524 >> 172.21.2.104:0/1118350524 conn(0x7f147c062f10 msgr2=0x7f147c094b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:02.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 -- 172.21.2.104:0/1118350524 shutdown_connections 2021-07-11T03:59:02.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.477+0000 7f14837f2700 1 -- 172.21.2.104:0/1118350524 wait complete. 2021-07-11T03:59:02.487 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 27 2021-07-11T03:59:02.490 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T03:59:02.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f054da2b700 1 Processor -- start 2021-07-11T03:59:02.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f054da2b700 1 -- start start 2021-07-11T03:59:02.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f054da2b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f05480a76b0 0x7f0548066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f054da2b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f05480b1970 0x7f0548066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f054da2b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0548067450 con 0x7f05480b0060 2021-07-11T03:59:02.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f054da2b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05480a7a90 con 0x7f05480b1970 2021-07-11T03:59:02.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f054da2b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0548147620 con 0x7f05480a76b0 2021-07-11T03:59:02.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f05477fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f05480b0060 0x7f054805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42660/0 (socket says 172.21.2.104:42660) 2021-07-11T03:59:02.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f05477fe700 1 -- 172.21.2.104:0/2328889966 learned_addr learned my addr 172.21.2.104:0/2328889966 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:02.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f05467fc700 1 --2- 172.21.2.104:0/2328889966 >> v2:172.21.2.105:3300/0 conn(0x7f05480b1970 0x7f0548066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:02.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336377527 0 0) 0x7f0548067450 con 0x7f05480b0060 2021-07-11T03:59:02.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f052c003260 con 0x7f05480b0060 2021-07-11T03:59:02.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0546ffd700 1 --2- 172.21.2.104:0/2328889966 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f05480a76b0 0x7f0548066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:02.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 756250276 0 0) 0x7f052c003260 con 0x7f05480b0060 2021-07-11T03:59:02.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f05480a76b0 msgr2=0x7f0548066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:02.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 --2- 172.21.2.104:0/2328889966 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f05480a76b0 0x7f0548066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 >> v2:172.21.2.105:3300/0 conn(0x7f05480b1970 msgr2=0x7f0548066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:02.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 --2- 172.21.2.104:0/2328889966 >> v2:172.21.2.105:3300/0 conn(0x7f05480b1970 0x7f0548066f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:02.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05481478c0 con 0x7f05480b0060 2021-07-11T03:59:02.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f05400019e0 con 0x7f05480b0060 2021-07-11T03:59:02.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f05467fc700 1 --2- 172.21.2.104:0/2328889966 >> v2:172.21.2.105:3300/0 conn(0x7f05480b1970 0x7f0548066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:59:02.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 >> v1:172.21.2.104:6789/0 conn(0x7f05480b0060 legacy=0x7f054805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 --2- 172.21.2.104:0/2328889966 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f05480a76b0 0x7f052c003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 --2- 172.21.2.104:0/2328889966 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f052c003d30 0x7f052c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 --2- 172.21.2.104:0/2328889966 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f052c0049f0 0x7f052c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05480a7a90 con 0x7f052c0049f0 2021-07-11T03:59:02.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548147620 con 0x7f052c003d30 2021-07-11T03:59:02.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052c00b800 con 0x7f05480a76b0 2021-07-11T03:59:02.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.533+0000 7f05477fe700 1 --2- 172.21.2.104:0/2328889966 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f052c0049f0 0x7f052c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:02.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f05477fe700 1 -- 172.21.2.104:0/2328889966 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f05480a76b0 msgr2=0x7f052c003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:02.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f05477fe700 1 --2- 172.21.2.104:0/2328889966 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f05480a76b0 0x7f052c003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:02.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f05477fe700 1 -- 172.21.2.104:0/2328889966 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f052c003d30 msgr2=0x7f052c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:02.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f05477fe700 1 --2- 172.21.2.104:0/2328889966 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f052c003d30 0x7f052c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:02.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f05477fe700 1 -- 172.21.2.104:0/2328889966 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c003260 con 0x7f052c0049f0 2021-07-11T03:59:02.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f05477fe700 1 --2- 172.21.2.104:0/2328889966 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f052c0049f0 0x7f052c004e70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f05400016f0 tx=0x7f05400021d0).ready entity=mon.0 client_cookie=4ebfafadda5029ac server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:02.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f054000cc80 con 0x7f052c0049f0 2021-07-11T03:59:02.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054000cde0 con 0x7f052c0049f0 2021-07-11T03:59:02.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0545ffb700 1 -- 172.21.2.104:0/2328889966 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0540005b80 con 0x7f052c0049f0 2021-07-11T03:59:02.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- 172.21.2.104:0/2328889966 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f052c0049f0 msgr2=0x7f052c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 --2- 172.21.2.104:0/2328889966 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f052c0049f0 0x7f052c004e70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f05400016f0 tx=0x7f05400021d0).stop 2021-07-11T03:59:02.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f05477fe700 1 -- 172.21.2.104:0/2328889966 reap_dead start 2021-07-11T03:59:02.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- 172.21.2.104:0/2328889966 shutdown_connections 2021-07-11T03:59:02.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- 172.21.2.104:0/2328889966 >> 172.21.2.104:0/2328889966 conn(0x7f05480b53e0 msgr2=0x7f05480b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:02.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- 172.21.2.104:0/2328889966 shutdown_connections 2021-07-11T03:59:02.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- 172.21.2.104:0/2328889966 wait complete. 2021-07-11T03:59:02.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 Processor -- start 2021-07-11T03:59:02.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- start start 2021-07-11T03:59:02.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f05480b4780 0x7f05481570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f05480ac600 0x7f0548159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f054815a030 0x7f0548144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548146960 con 0x7f05480b4780 2021-07-11T03:59:02.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05480b4b60 con 0x7f054815a030 2021-07-11T03:59:02.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0548146050 con 0x7f05480ac600 2021-07-11T03:59:02.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0546ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f05480b4780 0x7f05481570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:02.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0546ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f05480b4780 0x7f05481570f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45900/0 (socket says 172.21.2.104:45900) 2021-07-11T03:59:02.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0546ffd700 1 -- 172.21.2.104:0/3827095358 learned_addr learned my addr 172.21.2.104:0/3827095358 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:02.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0546ffd700 1 -- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f05480ac600 msgr2=0x7f0548159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0546ffd700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f05480ac600 0x7f0548159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:02.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0546ffd700 1 -- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f054815a030 msgr2=0x7f0548144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:02.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0546ffd700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f054815a030 0x7f0548144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:02.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0546ffd700 1 -- 172.21.2.104:0/3827095358 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0548156710 con 0x7f05480b4780 2021-07-11T03:59:02.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0546ffd700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f05480b4780 0x7f05481570f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f053800a000 tx=0x7f0538008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:02.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0537fff700 1 -- 172.21.2.104:0/3827095358 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0538016070 con 0x7f05480b4780 2021-07-11T03:59:02.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0537fff700 1 -- 172.21.2.104:0/3827095358 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0538000e90 con 0x7f05480b4780 2021-07-11T03:59:02.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- 172.21.2.104:0/3827095358 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05481569b0 con 0x7f05480b4780 2021-07-11T03:59:02.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0537fff700 1 -- 172.21.2.104:0/3827095358 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0538011970 con 0x7f05480b4780 2021-07-11T03:59:02.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- 172.21.2.104:0/3827095358 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05480ae710 con 0x7f05480b4780 2021-07-11T03:59:02.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f054da2b700 1 -- 172.21.2.104:0/3827095358 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0548057350 con 0x7f05480b4780 2021-07-11T03:59:02.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0537fff700 1 -- 172.21.2.104:0/3827095358 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0538011ad0 con 0x7f05480b4780 2021-07-11T03:59:02.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0537fff700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0524053210 0x7f05240556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:02.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.537+0000 7f0537fff700 1 -- 172.21.2.104:0/3827095358 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f053806b070 con 0x7f05480b4780 2021-07-11T03:59:02.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.541+0000 7f05467fc700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0524053210 0x7f05240556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:02.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.541+0000 7f05467fc700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0524053210 0x7f05240556d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f053c00a000 tx=0x7f053c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:02.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.541+0000 7f0537fff700 1 -- 172.21.2.104:0/3827095358 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f05380423b0 con 0x7f05480b4780 2021-07-11T03:59:02.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:02.645+0000 7f054da2b700 1 -- 172.21.2.104:0/3827095358 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f05480698e0 con 0x7f05480b4780 2021-07-11T03:59:03.323 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T03:59:03.317+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T03:59:03.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f0537fff700 1 -- 172.21.2.104:0/3827095358 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) v1 ==== 161+0+0 (secure 0 0 0) 0x7f05380459d0 con 0x7f05480b4780 2021-07-11T03:59:03.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 -- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0524053210 msgr2=0x7f05240556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:03.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0524053210 0x7f05240556d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f053c00a000 tx=0x7f053c008040).stop 2021-07-11T03:59:03.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 -- 172.21.2.104:0/3827095358 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f05480b4780 msgr2=0x7f05481570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:03.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f05480b4780 0x7f05481570f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f053800a000 tx=0x7f0538008040).stop 2021-07-11T03:59:03.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 -- 172.21.2.104:0/3827095358 shutdown_connections 2021-07-11T03:59:03.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f054815a030 0x7f0548144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:03.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f05480ac600 0x7f0548159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:03.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0524053210 0x7f05240556d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 --2- 172.21.2.104:0/3827095358 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f05480b4780 0x7f05481570f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 -- 172.21.2.104:0/3827095358 >> 172.21.2.104:0/3827095358 conn(0x7f05480b53e0 msgr2=0x7f0548066820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:03.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 -- 172.21.2.104:0/3827095358 shutdown_connections 2021-07-11T03:59:03.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.325+0000 7f054da2b700 1 -- 172.21.2.104:0/3827095358 wait complete. 2021-07-11T03:59:03.338 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T03:59:03.341 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:59:03.355 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T03:59:03.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 Processor -- start 2021-07-11T03:59:03.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- start start 2021-07-11T03:59:03.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f4c1c0b2660 0x7f4c1c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c1c096cc0 0x7f4c1c097100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4c1c13f560 0x7f4c1c13f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c0b2a40 con 0x7f4c1c13f560 2021-07-11T03:59:03.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c0975f0 con 0x7f4c1c0b2660 2021-07-11T03:59:03.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c097b80 con 0x7f4c1c096cc0 2021-07-11T03:59:03.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4c1c13f560 0x7f4c1c13f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:03.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4c1c13f560 0x7f4c1c13f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45928/0 (socket says 172.21.2.104:45928) 2021-07-11T03:59:03.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 -- 172.21.2.104:0/1469468590 learned_addr learned my addr 172.21.2.104:0/1469468590 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:03.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 -- 172.21.2.104:0/1469468590 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c1c096cc0 msgr2=0x7f4c1c097100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:03.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 --2- 172.21.2.104:0/1469468590 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c1c096cc0 0x7f4c1c097100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:03.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 -- 172.21.2.104:0/1469468590 >> v2:172.21.2.105:3300/0 conn(0x7f4c1c0b2660 msgr2=0x7f4c1c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:03.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 --2- 172.21.2.104:0/1469468590 >> v2:172.21.2.105:3300/0 conn(0x7f4c1c0b2660 0x7f4c1c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:03.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 -- 172.21.2.104:0/1469468590 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c097e20 con 0x7f4c1c13f560 2021-07-11T03:59:03.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 --2- 172.21.2.104:0/1469468590 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c1c13f560 0x7f4c1c13f9f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f4c140032f0 tx=0x7f4c14009290).ready entity=mon.0 client_cookie=5cc7cf2090c1284f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:03.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c1b7fe700 1 -- 172.21.2.104:0/1469468590 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c1400cb70 con 0x7f4c1c13f560 2021-07-11T03:59:03.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c1b7fe700 1 -- 172.21.2.104:0/1469468590 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1400ccd0 con 0x7f4c1c13f560 2021-07-11T03:59:03.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c1b7fe700 1 -- 172.21.2.104:0/1469468590 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c14010600 con 0x7f4c1c13f560 2021-07-11T03:59:03.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- 172.21.2.104:0/1469468590 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c1c13f560 msgr2=0x7f4c1c13f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:03.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 --2- 172.21.2.104:0/1469468590 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c1c13f560 0x7f4c1c13f9f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f4c140032f0 tx=0x7f4c14009290).stop 2021-07-11T03:59:03.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- 172.21.2.104:0/1469468590 shutdown_connections 2021-07-11T03:59:03.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 --2- 172.21.2.104:0/1469468590 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c1c13f560 0x7f4c1c13f9f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 --2- 172.21.2.104:0/1469468590 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c1c096cc0 0x7f4c1c097100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:03.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 --2- 172.21.2.104:0/1469468590 >> v2:172.21.2.105:3300/0 conn(0x7f4c1c0b2660 0x7f4c1c05f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:03.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- 172.21.2.104:0/1469468590 >> 172.21.2.104:0/1469468590 conn(0x7f4c1c062f10 msgr2=0x7f4c1c0632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:03.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- 172.21.2.104:0/1469468590 shutdown_connections 2021-07-11T03:59:03.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- 172.21.2.104:0/1469468590 wait complete. 2021-07-11T03:59:03.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 Processor -- start 2021-07-11T03:59:03.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- start start 2021-07-11T03:59:03.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c1c096cc0 0x7f4c1c063bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c1c0b2660 0x7f4c1c151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c1c13f560 0x7f4c1c0aa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1524c0 con 0x7f4c1c13f560 2021-07-11T03:59:03.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c151df0 con 0x7f4c1c0b2660 2021-07-11T03:59:03.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0ac5b0 con 0x7f4c1c096cc0 2021-07-11T03:59:03.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c1c13f560 0x7f4c1c0aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:03.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c1c13f560 0x7f4c1c0aa860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45930/0 (socket says 172.21.2.104:45930) 2021-07-11T03:59:03.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 -- 172.21.2.104:0/358086449 learned_addr learned my addr 172.21.2.104:0/358086449 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:03.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c1bfff700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c1c0b2660 0x7f4c1c151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:03.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c20ec1700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c1c096cc0 0x7f4c1c063bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:03.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 -- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c1c096cc0 msgr2=0x7f4c1c063bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:03.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c1c096cc0 0x7f4c1c063bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 -- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c1c0b2660 msgr2=0x7f4c1c151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:03.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c1c0b2660 0x7f4c1c151900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 -- 172.21.2.104:0/358086449 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c14009530 con 0x7f4c1c13f560 2021-07-11T03:59:03.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c1bfff700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c1c0b2660 0x7f4c1c151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:59:03.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c216c2700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c1c13f560 0x7f4c1c0aa860 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f4c140032f0 tx=0x7f4c14017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:03.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c19ffb700 1 -- 172.21.2.104:0/358086449 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c1401a450 con 0x7f4c1c13f560 2021-07-11T03:59:03.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c19ffb700 1 -- 172.21.2.104:0/358086449 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1401a5b0 con 0x7f4c1c13f560 2021-07-11T03:59:03.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- 172.21.2.104:0/358086449 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c0ac850 con 0x7f4c1c13f560 2021-07-11T03:59:03.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c19ffb700 1 -- 172.21.2.104:0/358086449 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c1401a910 con 0x7f4c1c13f560 2021-07-11T03:59:03.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.389+0000 7f4c23125700 1 -- 172.21.2.104:0/358086449 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c0aad50 con 0x7f4c1c13f560 2021-07-11T03:59:03.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.393+0000 7f4c23125700 1 -- 172.21.2.104:0/358086449 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c1c0ac9e0 con 0x7f4c1c13f560 2021-07-11T03:59:03.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.393+0000 7f4c19ffb700 1 -- 172.21.2.104:0/358086449 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4c14029020 con 0x7f4c1c13f560 2021-07-11T03:59:03.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.393+0000 7f4c19ffb700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c00053210 0x7f4c000556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.393+0000 7f4c19ffb700 1 -- 172.21.2.104:0/358086449 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f4c1406d500 con 0x7f4c1c13f560 2021-07-11T03:59:03.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.393+0000 7f4c20ec1700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c00053210 0x7f4c000556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:03.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.393+0000 7f4c20ec1700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c00053210 0x7f4c000556d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f4c0400a000 tx=0x7f4c04008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:03.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.393+0000 7f4c19ffb700 1 -- 172.21.2.104:0/358086449 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4c140447f0 con 0x7f4c1c13f560 2021-07-11T03:59:03.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 -- 172.21.2.104:0/358086449 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c1c057350 con 0x7f4c1c13f560 2021-07-11T03:59:03.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c19ffb700 1 -- 172.21.2.104:0/358086449 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2132 (secure 0 0 0) 0x7f4c14047e10 con 0x7f4c1c13f560 2021-07-11T03:59:03.513 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:03.514 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":28,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/2908869555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2908869555},{"type":"v1","addr":"172.21.2.105:6835","nonce":2908869555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/1110128427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1110128427},{"type":"v1","addr":"172.21.2.104:6833","nonce":1110128427}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26}],"filesystems":[{"mdsmap":{"epoch":28,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:58:27.556328+0000","modified":"2021-07-11T03:59:03.319774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":67,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2021-07-11T03:59:03.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 -- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c00053210 msgr2=0x7f4c000556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:03.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c00053210 0x7f4c000556d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f4c0400a000 tx=0x7f4c04008040).stop 2021-07-11T03:59:03.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 -- 172.21.2.104:0/358086449 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c1c13f560 msgr2=0x7f4c1c0aa860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:03.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c1c13f560 0x7f4c1c0aa860 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f4c140032f0 tx=0x7f4c14017000).stop 2021-07-11T03:59:03.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 -- 172.21.2.104:0/358086449 shutdown_connections 2021-07-11T03:59:03.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c00053210 0x7f4c000556d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c1c13f560 0x7f4c1c0aa860 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c1c0b2660 0x7f4c1c151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 --2- 172.21.2.104:0/358086449 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c1c096cc0 0x7f4c1c063bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 -- 172.21.2.104:0/358086449 >> 172.21.2.104:0/358086449 conn(0x7f4c1c062f10 msgr2=0x7f4c1c09ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:03.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 -- 172.21.2.104:0/358086449 shutdown_connections 2021-07-11T03:59:03.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.505+0000 7f4c23125700 1 -- 172.21.2.104:0/358086449 wait complete. 2021-07-11T03:59:03.518 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 28 2021-07-11T03:59:03.520 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T03:59:03.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 Processor -- start 2021-07-11T03:59:03.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- start start 2021-07-11T03:59:03.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbdc405fc00 0x7fbdc40b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbdc40b5990 0x7fbdc40b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc4143f60 con 0x7fbdc40b6560 2021-07-11T03:59:03.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc40ba6b0 con 0x7fbdc405fc00 2021-07-11T03:59:03.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc405ffe0 con 0x7fbdc40b5990 2021-07-11T03:59:03.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc8b3b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbdc405fc00 0x7fbdc40b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:03.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc8b3b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbdc405fc00 0x7fbdc40b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45938/0 (socket says 172.21.2.104:45938) 2021-07-11T03:59:03.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc8b3b700 1 -- 172.21.2.104:0/1629678378 learned_addr learned my addr 172.21.2.104:0/1629678378 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:03.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc8b3b700 1 -- 172.21.2.104:0/1629678378 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbdc40b5990 msgr2=0x7fbdc40b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:03.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc8b3b700 1 --2- 172.21.2.104:0/1629678378 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbdc40b5990 0x7fbdc40b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:03.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc8b3b700 1 -- 172.21.2.104:0/1629678378 >> v1:172.21.2.105:6789/0 conn(0x7fbdc40b6560 legacy=0x7fbdc40b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:03.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc8b3b700 1 -- 172.21.2.104:0/1629678378 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc4143a50 con 0x7fbdc405fc00 2021-07-11T03:59:03.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc8b3b700 1 --2- 172.21.2.104:0/1629678378 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbdc405fc00 0x7fbdc40b41b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fbdc40b19e0 tx=0x7fbdb4006eb0).ready entity=mon.0 client_cookie=d6a59d8ccb85324 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:03.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc37fe700 1 -- 172.21.2.104:0/1629678378 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdb400d570 con 0x7fbdc405fc00 2021-07-11T03:59:03.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc37fe700 1 -- 172.21.2.104:0/1629678378 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb4000e90 con 0x7fbdc405fc00 2021-07-11T03:59:03.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc37fe700 1 -- 172.21.2.104:0/1629678378 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdb400d8c0 con 0x7fbdc405fc00 2021-07-11T03:59:03.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- 172.21.2.104:0/1629678378 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbdc405fc00 msgr2=0x7fbdc40b41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:03.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 --2- 172.21.2.104:0/1629678378 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbdc405fc00 0x7fbdc40b41b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fbdc40b19e0 tx=0x7fbdb4006eb0).stop 2021-07-11T03:59:03.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- 172.21.2.104:0/1629678378 shutdown_connections 2021-07-11T03:59:03.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 --2- 172.21.2.104:0/1629678378 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbdc40b5990 0x7fbdc40b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:03.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 --2- 172.21.2.104:0/1629678378 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbdc405fc00 0x7fbdc40b41b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- 172.21.2.104:0/1629678378 >> 172.21.2.104:0/1629678378 conn(0x7fbdc40623a0 msgr2=0x7fbdc4066de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:03.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- 172.21.2.104:0/1629678378 shutdown_connections 2021-07-11T03:59:03.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- 172.21.2.104:0/1629678378 wait complete. 2021-07-11T03:59:03.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 Processor -- start 2021-07-11T03:59:03.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- start start 2021-07-11T03:59:03.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbdc405fc00 0x7fbdc412acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbdc40b5990 0x7fbdc40ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbdc40b6560 0x7fbdc4141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc413cf80 con 0x7fbdc40b6560 2021-07-11T03:59:03.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc40b3660 con 0x7fbdc405fc00 2021-07-11T03:59:03.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdcad9f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc413d160 con 0x7fbdc40b5990 2021-07-11T03:59:03.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc933c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbdc40b6560 0x7fbdc4141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:03.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc933c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbdc40b6560 0x7fbdc4141e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45942/0 (socket says 172.21.2.104:45942) 2021-07-11T03:59:03.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.577+0000 7fbdc933c700 1 -- 172.21.2.104:0/2945828964 learned_addr learned my addr 172.21.2.104:0/2945828964 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:03.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc8b3b700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbdc405fc00 0x7fbdc412acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:03.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc3fff700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbdc40b5990 0x7fbdc40ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:03.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc933c700 1 -- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbdc40b5990 msgr2=0x7fbdc40ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:03.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc933c700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbdc40b5990 0x7fbdc40ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc933c700 1 -- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbdc405fc00 msgr2=0x7fbdc412acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:03.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc933c700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbdc405fc00 0x7fbdc412acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:03.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc933c700 1 -- 172.21.2.104:0/2945828964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb400b1e0 con 0x7fbdc40b6560 2021-07-11T03:59:03.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc3fff700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbdc40b5990 0x7fbdc40ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:59:03.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc933c700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbdc40b6560 0x7fbdc4141e40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fbdbc0032f0 tx=0x7fbdbc009250).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:03.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc1ffb700 1 -- 172.21.2.104:0/2945828964 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdbc00f7e0 con 0x7fbdc40b6560 2021-07-11T03:59:03.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc1ffb700 1 -- 172.21.2.104:0/2945828964 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdbc00f940 con 0x7fbdc40b6560 2021-07-11T03:59:03.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc1ffb700 1 -- 172.21.2.104:0/2945828964 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbdbc00fca0 con 0x7fbdc40b6560 2021-07-11T03:59:03.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdcad9f700 1 -- 172.21.2.104:0/2945828964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc40b1ab0 con 0x7fbdc40b6560 2021-07-11T03:59:03.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdcad9f700 1 -- 172.21.2.104:0/2945828964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc40b33c0 con 0x7fbdc40b6560 2021-07-11T03:59:03.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdcad9f700 1 -- 172.21.2.104:0/2945828964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc40b3b70 con 0x7fbdc40b6560 2021-07-11T03:59:03.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc1ffb700 1 -- 172.21.2.104:0/2945828964 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbdbc010480 con 0x7fbdc40b6560 2021-07-11T03:59:03.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc1ffb700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbda4053240 0x7fbda4055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:03.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc1ffb700 1 -- 172.21.2.104:0/2945828964 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fbdbc06bc90 con 0x7fbdc40b6560 2021-07-11T03:59:03.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc8b3b700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbda4053240 0x7fbda4055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:03.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.581+0000 7fbdc8b3b700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbda4053240 0x7fbda4055700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fbdb4019000 tx=0x7fbdb4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:03.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.585+0000 7fbdc1ffb700 1 -- 172.21.2.104:0/2945828964 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbdbc042e10 con 0x7fbdc40b6560 2021-07-11T03:59:03.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:03.689+0000 7fbdcad9f700 1 -- 172.21.2.104:0/2945828964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbdc413d320 con 0x7fbdc40b6560 2021-07-11T03:59:04.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.329+0000 7fbdc1ffb700 1 -- 172.21.2.104:0/2945828964 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbdbc00fe00 con 0x7fbdc40b6560 2021-07-11T03:59:04.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 -- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbda4053240 msgr2=0x7fbda4055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:04.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbda4053240 0x7fbda4055700 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7fbdb4019000 tx=0x7fbdb4015000).stop 2021-07-11T03:59:04.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 -- 172.21.2.104:0/2945828964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbdc40b6560 msgr2=0x7fbdc4141e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:04.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbdc40b6560 0x7fbdc4141e40 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fbdbc0032f0 tx=0x7fbdbc009250).stop 2021-07-11T03:59:04.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 -- 172.21.2.104:0/2945828964 shutdown_connections 2021-07-11T03:59:04.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbda4053240 0x7fbda4055700 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:04.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbdc40b6560 0x7fbdc4141e40 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:04.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbdc40b5990 0x7fbdc40ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:04.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 --2- 172.21.2.104:0/2945828964 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbdc405fc00 0x7fbdc412acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:04.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 -- 172.21.2.104:0/2945828964 >> 172.21.2.104:0/2945828964 conn(0x7fbdc40623a0 msgr2=0x7fbdc412b900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:04.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 -- 172.21.2.104:0/2945828964 shutdown_connections 2021-07-11T03:59:04.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.333+0000 7fbdcad9f700 1 -- 172.21.2.104:0/2945828964 wait complete. 2021-07-11T03:59:04.346 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T03:59:04.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.393+0000 7f2eba952700 1 Processor -- start 2021-07-11T03:59:04.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.393+0000 7f2eba952700 1 -- start start 2021-07-11T03:59:04.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.393+0000 7f2eba952700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2eb4060100 0x7f2eb40b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:04.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.393+0000 7f2eba952700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb40aabb0 con 0x7f2eb40b4c40 2021-07-11T03:59:04.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.393+0000 7f2eba952700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb40604e0 con 0x7f2eb4067300 2021-07-11T03:59:04.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.393+0000 7f2eba952700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb4066ea0 con 0x7f2eb4060100 2021-07-11T03:59:04.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.393+0000 7f2eb8eef700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f2eb40b4c40 0x7f2eb414b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42722/0 (socket says 172.21.2.104:42722) 2021-07-11T03:59:04.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.393+0000 7f2eb8eef700 1 -- 172.21.2.104:0/618444022 learned_addr learned my addr 172.21.2.104:0/618444022 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:04.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.393+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767916983 0 0) 0x7f2eb40aabb0 con 0x7f2eb40b4c40 2021-07-11T03:59:04.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.393+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e94003260 con 0x7f2eb40b4c40 2021-07-11T03:59:04.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb3fff700 1 --2- 172.21.2.104:0/618444022 >> v2:172.21.2.105:3300/0 conn(0x7f2eb4060100 0x7f2eb40b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:04.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 270477930 0 0) 0x7f2e94003260 con 0x7f2eb40b4c40 2021-07-11T03:59:04.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 >> v2:172.21.2.105:3300/0 conn(0x7f2eb4060100 msgr2=0x7f2eb40b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:04.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 --2- 172.21.2.104:0/618444022 >> v2:172.21.2.105:3300/0 conn(0x7f2eb4060100 0x7f2eb40b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:04.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 >> v1:172.21.2.105:6789/0 conn(0x7f2eb4067300 legacy=0x7f2eb41438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:04.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb40b33b0 con 0x7f2eb40b4c40 2021-07-11T03:59:04.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f2eac0019e0 con 0x7f2eb40b4c40 2021-07-11T03:59:04.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 >> v1:172.21.2.104:6789/0 conn(0x7f2eb40b4c40 legacy=0x7f2eb414b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:04.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 --2- 172.21.2.104:0/618444022 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e94003820 0x7f2e94003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:04.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 --2- 172.21.2.104:0/618444022 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e94004500 0x7f2e94004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:04.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 --2- 172.21.2.104:0/618444022 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e94004f70 0x7f2e9400fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:04.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb4066ea0 con 0x7f2e94004f70 2021-07-11T03:59:04.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e94010260 con 0x7f2e94003820 2021-07-11T03:59:04.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e94010420 con 0x7f2e94004500 2021-07-11T03:59:04.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb8eef700 1 --2- 172.21.2.104:0/618444022 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e94004f70 0x7f2e9400fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:04.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb3fff700 1 --2- 172.21.2.104:0/618444022 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e94003820 0x7f2e94003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:04.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb8eef700 1 -- 172.21.2.104:0/618444022 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e94004500 msgr2=0x7f2e94004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:04.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb8eef700 1 --2- 172.21.2.104:0/618444022 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e94004500 0x7f2e94004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:04.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb8eef700 1 -- 172.21.2.104:0/618444022 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e94003820 msgr2=0x7f2e94003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:04.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb8eef700 1 --2- 172.21.2.104:0/618444022 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e94003820 0x7f2e94003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:04.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb8eef700 1 -- 172.21.2.104:0/618444022 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e94003260 con 0x7f2e94004f70 2021-07-11T03:59:04.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb8eef700 1 -- 172.21.2.104:0/618444022 reap_dead start 2021-07-11T03:59:04.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb8eef700 1 --2- 172.21.2.104:0/618444022 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e94004f70 0x7f2e9400fb60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f2eb405f220 tx=0x7f2eac0020e0).ready entity=mon.0 client_cookie=b2507c213acbca48 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:04.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2eac00cc00 con 0x7f2e94004f70 2021-07-11T03:59:04.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac00cd60 con 0x7f2e94004f70 2021-07-11T03:59:04.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb2ffd700 1 -- 172.21.2.104:0/618444022 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2eac005b80 con 0x7f2e94004f70 2021-07-11T03:59:04.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 -- 172.21.2.104:0/618444022 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e94004f70 msgr2=0x7f2e9400fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:04.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 --2- 172.21.2.104:0/618444022 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e94004f70 0x7f2e9400fb60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f2eb405f220 tx=0x7f2eac0020e0).stop 2021-07-11T03:59:04.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 -- 172.21.2.104:0/618444022 shutdown_connections 2021-07-11T03:59:04.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 --2- 172.21.2.104:0/618444022 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e94004f70 0x7f2e9400fb60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:04.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 -- 172.21.2.104:0/618444022 >> 172.21.2.104:0/618444022 conn(0x7f2eb405cf30 msgr2=0x7f2eb405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:04.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 -- 172.21.2.104:0/618444022 shutdown_connections 2021-07-11T03:59:04.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 -- 172.21.2.104:0/618444022 wait complete. 2021-07-11T03:59:04.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 Processor -- start 2021-07-11T03:59:04.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 -- start start 2021-07-11T03:59:04.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e94004f70 0x7f2eb41520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:04.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2eb40a4070 0x7f2eb40a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:04.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2eb40b0b20 0x7f2eb409e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:04.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb41516d0 con 0x7f2eb40a4070 2021-07-11T03:59:04.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb4151970 con 0x7f2e94004f70 2021-07-11T03:59:04.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb4151c10 con 0x7f2eb40b0b20 2021-07-11T03:59:04.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb37fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2eb40a4070 0x7f2eb40a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:04.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb37fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2eb40a4070 0x7f2eb40a4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45966/0 (socket says 172.21.2.104:45966) 2021-07-11T03:59:04.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb37fe700 1 -- 172.21.2.104:0/1452648793 learned_addr learned my addr 172.21.2.104:0/1452648793 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:04.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb37fe700 1 -- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2eb40b0b20 msgr2=0x7f2eb409e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:04.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb37fe700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2eb40b0b20 0x7f2eb409e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:04.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb37fe700 1 -- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e94004f70 msgr2=0x7f2eb41520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:04.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb37fe700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e94004f70 0x7f2eb41520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:04.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb37fe700 1 -- 172.21.2.104:0/1452648793 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb40a0dc0 con 0x7f2eb40a4070 2021-07-11T03:59:04.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb37fe700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2eb40a4070 0x7f2eb40a4630 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f2e9c000f70 tx=0x7f2e9c006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:04.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb17fa700 1 -- 172.21.2.104:0/1452648793 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e9c013460 con 0x7f2eb40a4070 2021-07-11T03:59:04.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb17fa700 1 -- 172.21.2.104:0/1452648793 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9c0135c0 con 0x7f2eb40a4070 2021-07-11T03:59:04.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 -- 172.21.2.104:0/1452648793 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb409eda0 con 0x7f2eb40a4070 2021-07-11T03:59:04.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eba952700 1 -- 172.21.2.104:0/1452648793 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb40a9f60 con 0x7f2eb40a4070 2021-07-11T03:59:04.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.397+0000 7f2eb17fa700 1 -- 172.21.2.104:0/1452648793 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e9c013920 con 0x7f2eb40a4070 2021-07-11T03:59:04.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.401+0000 7f2eba952700 1 -- 172.21.2.104:0/1452648793 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb4057350 con 0x7f2eb40a4070 2021-07-11T03:59:04.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.401+0000 7f2eb17fa700 1 -- 172.21.2.104:0/1452648793 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2e9c01a060 con 0x7f2eb40a4070 2021-07-11T03:59:04.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.401+0000 7f2eb17fa700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e980532d0 0x7f2e98055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:04.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.401+0000 7f2eb17fa700 1 -- 172.21.2.104:0/1452648793 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f2e9c06af00 con 0x7f2eb40a4070 2021-07-11T03:59:04.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.401+0000 7f2eb3fff700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e980532d0 0x7f2e98055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:04.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.401+0000 7f2eb3fff700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e980532d0 0x7f2e98055790 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f2ea400c000 tx=0x7f2ea4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:04.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.401+0000 7f2eb17fa700 1 -- 172.21.2.104:0/1452648793 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2e9c0420e0 con 0x7f2eb40a4070 2021-07-11T03:59:04.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:04.481+0000 7f2eba952700 1 -- 172.21.2.104:0/1452648793 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2eb40af600 con 0x7f2eb40a4070 2021-07-11T03:59:05.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eb17fa700 1 -- 172.21.2.104:0/1452648793 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v68) v1 ==== 185+0+0 (secure 0 0 0) 0x7f2e9c018070 con 0x7f2eb40a4070 2021-07-11T03:59:05.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 -- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e980532d0 msgr2=0x7f2e98055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:05.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e980532d0 0x7f2e98055790 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f2ea400c000 tx=0x7f2ea4008040).stop 2021-07-11T03:59:05.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 -- 172.21.2.104:0/1452648793 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2eb40a4070 msgr2=0x7f2eb40a4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:05.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2eb40a4070 0x7f2eb40a4630 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f2e9c000f70 tx=0x7f2e9c006dc0).stop 2021-07-11T03:59:05.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 -- 172.21.2.104:0/1452648793 shutdown_connections 2021-07-11T03:59:05.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2eb40b0b20 0x7f2eb409e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:05.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e980532d0 0x7f2e98055790 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:05.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2eb40a4070 0x7f2eb40a4630 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:05.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 --2- 172.21.2.104:0/1452648793 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e94004f70 0x7f2eb41520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:05.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 -- 172.21.2.104:0/1452648793 >> 172.21.2.104:0/1452648793 conn(0x7f2eb405cf30 msgr2=0x7f2eb40b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:05.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 -- 172.21.2.104:0/1452648793 shutdown_connections 2021-07-11T03:59:05.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.341+0000 7f2eba952700 1 -- 172.21.2.104:0/1452648793 wait complete. 2021-07-11T03:59:05.354 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T03:59:05.357 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T03:59:05.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 Processor -- start 2021-07-11T03:59:05.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 -- start start 2021-07-11T03:59:05.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f905c060100 0x7f905c0b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:05.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f905c0b4c40 0x7f905c14b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:05.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f905c0aabb0 con 0x7f905c067300 2021-07-11T03:59:05.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905c0604e0 con 0x7f905c060100 2021-07-11T03:59:05.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f905c066ea0 con 0x7f905c0b4c40 2021-07-11T03:59:05.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f905affd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f905c060100 0x7f905c0b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:05.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f905affd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f905c060100 0x7f905c0b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45974/0 (socket says 172.21.2.104:45974) 2021-07-11T03:59:05.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f905affd700 1 -- 172.21.2.104:0/799845176 learned_addr learned my addr 172.21.2.104:0/799845176 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:05.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f905affd700 1 -- 172.21.2.104:0/799845176 >> v2:172.21.2.105:3300/0 conn(0x7f905c0b4c40 msgr2=0x7f905c14b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:05.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f905affd700 1 --2- 172.21.2.104:0/799845176 >> v2:172.21.2.105:3300/0 conn(0x7f905c0b4c40 0x7f905c14b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:05.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f905affd700 1 -- 172.21.2.104:0/799845176 >> v1:172.21.2.105:6789/0 conn(0x7f905c067300 legacy=0x7f905c1438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:05.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f905affd700 1 -- 172.21.2.104:0/799845176 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c0b33b0 con 0x7f905c060100 2021-07-11T03:59:05.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f905affd700 1 --2- 172.21.2.104:0/799845176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f905c060100 0x7f905c0b4750 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f905c05f1e0 tx=0x7f9050006eb0).ready entity=mon.0 client_cookie=e9de58d4e1be6e99 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:05.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f9059ffb700 1 -- 172.21.2.104:0/799845176 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f905000d5c0 con 0x7f905c060100 2021-07-11T03:59:05.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f9059ffb700 1 -- 172.21.2.104:0/799845176 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050000e90 con 0x7f905c060100 2021-07-11T03:59:05.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f9059ffb700 1 -- 172.21.2.104:0/799845176 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f905000d910 con 0x7f905c060100 2021-07-11T03:59:05.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 -- 172.21.2.104:0/799845176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f905c060100 msgr2=0x7f905c0b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:05.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 --2- 172.21.2.104:0/799845176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f905c060100 0x7f905c0b4750 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f905c05f1e0 tx=0x7f9050006eb0).stop 2021-07-11T03:59:05.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 -- 172.21.2.104:0/799845176 shutdown_connections 2021-07-11T03:59:05.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 --2- 172.21.2.104:0/799845176 >> v2:172.21.2.105:3300/0 conn(0x7f905c0b4c40 0x7f905c14b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:05.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 --2- 172.21.2.104:0/799845176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f905c060100 0x7f905c0b4750 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:05.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 -- 172.21.2.104:0/799845176 >> 172.21.2.104:0/799845176 conn(0x7f905c05cf30 msgr2=0x7f905c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:05.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 -- 172.21.2.104:0/799845176 shutdown_connections 2021-07-11T03:59:05.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.405+0000 7f90618e6700 1 -- 172.21.2.104:0/799845176 wait complete. 2021-07-11T03:59:05.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f90618e6700 1 Processor -- start 2021-07-11T03:59:05.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f90618e6700 1 -- start start 2021-07-11T03:59:05.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f90618e6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f905c060100 0x7f905c13e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:05.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f90618e6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f905c067300 0x7f905c13f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:05.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f90618e6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f905c0b4c40 0x7f905c13f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:05.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f90618e6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c140a60 con 0x7f905c067300 2021-07-11T03:59:05.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f90618e6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c145aa0 con 0x7f905c060100 2021-07-11T03:59:05.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f90618e6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905c146c00 con 0x7f905c0b4c40 2021-07-11T03:59:05.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f905a7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f905c067300 0x7f905c13f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:05.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f905a7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f905c067300 0x7f905c13f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45978/0 (socket says 172.21.2.104:45978) 2021-07-11T03:59:05.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f905a7fc700 1 -- 172.21.2.104:0/3703009351 learned_addr learned my addr 172.21.2.104:0/3703009351 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:05.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f905a7fc700 1 -- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f905c0b4c40 msgr2=0x7f905c13f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:05.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f905a7fc700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f905c0b4c40 0x7f905c13f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:05.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f905a7fc700 1 -- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f905c060100 msgr2=0x7f905c13e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:05.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f905a7fc700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f905c060100 0x7f905c13e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:05.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f905a7fc700 1 -- 172.21.2.104:0/3703009351 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905000b1e0 con 0x7f905c067300 2021-07-11T03:59:05.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f905a7fc700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f905c067300 0x7f905c13f270 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f904c000f70 tx=0x7f904c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:05.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f904bfff700 1 -- 172.21.2.104:0/3703009351 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f904c00ce80 con 0x7f905c067300 2021-07-11T03:59:05.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f904bfff700 1 -- 172.21.2.104:0/3703009351 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904c013450 con 0x7f905c067300 2021-07-11T03:59:05.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f904bfff700 1 -- 172.21.2.104:0/3703009351 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f904c0137b0 con 0x7f905c067300 2021-07-11T03:59:05.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f90618e6700 1 -- 172.21.2.104:0/3703009351 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c1470d0 con 0x7f905c067300 2021-07-11T03:59:05.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f90618e6700 1 -- 172.21.2.104:0/3703009351 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c146880 con 0x7f905c067300 2021-07-11T03:59:05.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f90618e6700 1 -- 172.21.2.104:0/3703009351 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905c147480 con 0x7f905c067300 2021-07-11T03:59:05.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f904bfff700 1 -- 172.21.2.104:0/3703009351 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f904c010070 con 0x7f905c067300 2021-07-11T03:59:05.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f904bfff700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9038053270 0x7f9038055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:05.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f904bfff700 1 -- 172.21.2.104:0/3703009351 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f904c06ab80 con 0x7f905c067300 2021-07-11T03:59:05.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f905affd700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9038053270 0x7f9038055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:05.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.409+0000 7f905affd700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9038053270 0x7f9038055730 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f905c05f1e0 tx=0x7f9050015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:05.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.413+0000 7f904bfff700 1 -- 172.21.2.104:0/3703009351 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f905c147480 con 0x7f905c067300 2021-07-11T03:59:05.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:05.493+0000 7f90618e6700 1 -- 172.21.2.104:0/3703009351 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f905c147480 con 0x7f905c067300 2021-07-11T03:59:06.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f904bfff700 1 -- 172.21.2.104:0/3703009351 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v69) v1 ==== 173+0+0 (secure 0 0 0) 0x7f905c147480 con 0x7f905c067300 2021-07-11T03:59:06.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 -- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9038053270 msgr2=0x7f9038055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:06.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9038053270 0x7f9038055730 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f905c05f1e0 tx=0x7f9050015000).stop 2021-07-11T03:59:06.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 -- 172.21.2.104:0/3703009351 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f905c067300 msgr2=0x7f905c13f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:06.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f905c067300 0x7f905c13f270 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f904c000f70 tx=0x7f904c007c00).stop 2021-07-11T03:59:06.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 -- 172.21.2.104:0/3703009351 shutdown_connections 2021-07-11T03:59:06.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f905c0b4c40 0x7f905c13f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:06.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9038053270 0x7f9038055730 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:06.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f905c067300 0x7f905c13f270 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:06.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 --2- 172.21.2.104:0/3703009351 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f905c060100 0x7f905c13e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:06.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 -- 172.21.2.104:0/3703009351 >> 172.21.2.104:0/3703009351 conn(0x7f905c05cf30 msgr2=0x7f905c140f30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:06.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 -- 172.21.2.104:0/3703009351 shutdown_connections 2021-07-11T03:59:06.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.353+0000 7f90618e6700 1 -- 172.21.2.104:0/3703009351 wait complete. 2021-07-11T03:59:06.368 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T03:59:06.370 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-07-11T03:59:06.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 Processor -- start 2021-07-11T03:59:06.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- start start 2021-07-11T03:59:06.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f76040b12d0 0x7f76040affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:06.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76040a77d0 con 0x7f76040b1ec0 2021-07-11T03:59:06.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76040b0650 con 0x7f76040b22a0 2021-07-11T03:59:06.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76040b1740 con 0x7f76040b12d0 2021-07-11T03:59:06.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760a1a8700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f76040b1ec0 0x7f76040a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42752/0 (socket says 172.21.2.104:42752) 2021-07-11T03:59:06.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f76040b12d0 0x7f76040affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:06.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760a1a8700 1 -- 172.21.2.104:0/644276109 learned_addr learned my addr 172.21.2.104:0/644276109 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:06.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 -- 172.21.2.104:0/644276109 >> v1:172.21.2.105:6789/0 conn(0x7f76040b22a0 legacy=0x7f7604141250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:59:06.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 -- 172.21.2.104:0/644276109 >> v1:172.21.2.104:6789/0 conn(0x7f76040b1ec0 legacy=0x7f76040a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:06.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 -- 172.21.2.104:0/644276109 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76040b08f0 con 0x7f76040b12d0 2021-07-11T03:59:06.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 --2- 172.21.2.104:0/644276109 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f76040b12d0 0x7f76040affb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f76040b2f50 tx=0x7f75f4006df0).ready entity=mon.0 client_cookie=eb9c872bb46e18e3 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:06.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76091a6700 1 -- 172.21.2.104:0/644276109 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75f400d5c0 con 0x7f76040b12d0 2021-07-11T03:59:06.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76091a6700 1 -- 172.21.2.104:0/644276109 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f4000e90 con 0x7f76040b12d0 2021-07-11T03:59:06.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76091a6700 1 -- 172.21.2.104:0/644276109 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75f400d910 con 0x7f76040b12d0 2021-07-11T03:59:06.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- 172.21.2.104:0/644276109 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f76040b12d0 msgr2=0x7f76040affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:06.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 --2- 172.21.2.104:0/644276109 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f76040b12d0 0x7f76040affb0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f76040b2f50 tx=0x7f75f4006df0).stop 2021-07-11T03:59:06.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- 172.21.2.104:0/644276109 shutdown_connections 2021-07-11T03:59:06.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 --2- 172.21.2.104:0/644276109 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f76040b12d0 0x7f76040affb0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:06.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- 172.21.2.104:0/644276109 >> 172.21.2.104:0/644276109 conn(0x7f760405f6b0 msgr2=0x7f760405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:06.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- 172.21.2.104:0/644276109 shutdown_connections 2021-07-11T03:59:06.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- 172.21.2.104:0/644276109 wait complete. 2021-07-11T03:59:06.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 Processor -- start 2021-07-11T03:59:06.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- start start 2021-07-11T03:59:06.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f76040b12d0 0x7f760414a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:06.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f76040b1ec0 0x7f76040ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:06.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f76040b22a0 0x7f76040abdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:06.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76040aef80 con 0x7f76040b1ec0 2021-07-11T03:59:06.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76040aaf00 con 0x7f76040b22a0 2021-07-11T03:59:06.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76040ab1a0 con 0x7f76040b12d0 2021-07-11T03:59:06.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f76040b1ec0 0x7f76040ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:06.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f76040b1ec0 0x7f76040ae440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:45992/0 (socket says 172.21.2.104:45992) 2021-07-11T03:59:06.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 -- 172.21.2.104:0/789101730 learned_addr learned my addr 172.21.2.104:0/789101730 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:06.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 -- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f76040b12d0 msgr2=0x7f760414a880 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:59:06.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f76040b12d0 0x7f760414a880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:06.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 -- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f76040b22a0 msgr2=0x7f76040abdb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:59:06.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f76040b22a0 0x7f76040abdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:06.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 -- 172.21.2.104:0/789101730 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f400b180 con 0x7f76040b1ec0 2021-07-11T03:59:06.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f76099a7700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f76040b1ec0 0x7f76040ae440 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f76040b2f50 tx=0x7f75f4012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:06.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f75f37fe700 1 -- 172.21.2.104:0/789101730 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75f4002f50 con 0x7f76040b1ec0 2021-07-11T03:59:06.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f75f37fe700 1 -- 172.21.2.104:0/789101730 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f4017030 con 0x7f76040b1ec0 2021-07-11T03:59:06.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- 172.21.2.104:0/789101730 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76040ab440 con 0x7f76040b1ec0 2021-07-11T03:59:06.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f75f37fe700 1 -- 172.21.2.104:0/789101730 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75f401d470 con 0x7f76040b1ec0 2021-07-11T03:59:06.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.417+0000 7f760c40c700 1 -- 172.21.2.104:0/789101730 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76040ae930 con 0x7f76040b1ec0 2021-07-11T03:59:06.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.421+0000 7f760c40c700 1 -- 172.21.2.104:0/789101730 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76040abae0 con 0x7f76040b1ec0 2021-07-11T03:59:06.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.421+0000 7f75f37fe700 1 -- 172.21.2.104:0/789101730 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f75f4024030 con 0x7f76040b1ec0 2021-07-11T03:59:06.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.421+0000 7f75f37fe700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75e4053270 0x7f75e4055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:06.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.421+0000 7f75f37fe700 1 -- 172.21.2.104:0/789101730 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f75f406c0a0 con 0x7f76040b1ec0 2021-07-11T03:59:06.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.421+0000 7f760a1a8700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75e4053270 0x7f75e4055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:06.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.421+0000 7f760a1a8700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75e4053270 0x7f75e4055730 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f76040ad2d0 tx=0x7f7600007790).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:06.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.421+0000 7f75f37fe700 1 -- 172.21.2.104:0/789101730 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f75f40435b0 con 0x7f76040b1ec0 2021-07-11T03:59:06.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.505+0000 7f760c40c700 1 -- 172.21.2.104:0/789101730 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7604057350 con 0x7f76040b1ec0 2021-07-11T03:59:06.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f75f37fe700 1 -- 172.21.2.104:0/789101730 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f75f40220a0 con 0x7f76040b1ec0 2021-07-11T03:59:06.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 -- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75e4053270 msgr2=0x7f75e4055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:06.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75e4053270 0x7f75e4055730 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f76040ad2d0 tx=0x7f7600007790).stop 2021-07-11T03:59:06.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 -- 172.21.2.104:0/789101730 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f76040b1ec0 msgr2=0x7f76040ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:06.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f76040b1ec0 0x7f76040ae440 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f76040b2f50 tx=0x7f75f4012000).stop 2021-07-11T03:59:06.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 -- 172.21.2.104:0/789101730 shutdown_connections 2021-07-11T03:59:06.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f76040b22a0 0x7f76040abdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:06.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75e4053270 0x7f75e4055730 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:06.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f76040b1ec0 0x7f76040ae440 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:06.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 --2- 172.21.2.104:0/789101730 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f76040b12d0 0x7f760414a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:06.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 -- 172.21.2.104:0/789101730 >> 172.21.2.104:0/789101730 conn(0x7f760405f6b0 msgr2=0x7f76040b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:06.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 -- 172.21.2.104:0/789101730 shutdown_connections 2021-07-11T03:59:06.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.513+0000 7f760c40c700 1 -- 172.21.2.104:0/789101730 wait complete. 2021-07-11T03:59:06.528 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout' 2021-07-11T03:59:06.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f4094aa1700 1 Processor -- start 2021-07-11T03:59:06.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f4094aa1700 1 -- start start 2021-07-11T03:59:06.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f4094aa1700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f40900b1db0 0x7f40900b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:06.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f4094aa1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40901483a0 0x7f40900b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:06.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f4094aa1700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40900b0350 con 0x7f40900a70c0 2021-07-11T03:59:06.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f4094aa1700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40900a90c0 con 0x7f40900b1db0 2021-07-11T03:59:06.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f4094aa1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40900b05f0 con 0x7f40901483a0 2021-07-11T03:59:06.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f408e59c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f40900b1db0 0x7f40900b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:06.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f408dd9b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40901483a0 0x7f40900b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:06.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f408e59c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f40900b1db0 0x7f40900b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:53558/0 (socket says 172.21.2.104:53558) 2021-07-11T03:59:06.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f408e59c700 1 -- 172.21.2.104:0/2806169040 learned_addr learned my addr 172.21.2.104:0/2806169040 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:06.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f408e59c700 1 -- 172.21.2.104:0/2806169040 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40901483a0 msgr2=0x7f40900b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:06.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f408e59c700 1 --2- 172.21.2.104:0/2806169040 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40901483a0 0x7f40900b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:06.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f408e59c700 1 -- 172.21.2.104:0/2806169040 >> v1:172.21.2.105:6789/0 conn(0x7f40900a70c0 legacy=0x7f4090144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:06.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.613+0000 7f408e59c700 1 -- 172.21.2.104:0/2806169040 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900b0890 con 0x7f40900b1db0 2021-07-11T03:59:06.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408e59c700 1 --2- 172.21.2.104:0/2806169040 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40900b1db0 0x7f40900b0fd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f4078002f70 tx=0x7f4078006eb0).ready entity=mon.1 client_cookie=44b2896fb7f115cb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:06.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408d59a700 1 -- 172.21.2.104:0/2806169040 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407800d570 con 0x7f40900b1db0 2021-07-11T03:59:06.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408d59a700 1 -- 172.21.2.104:0/2806169040 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4078000e90 con 0x7f40900b1db0 2021-07-11T03:59:06.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408d59a700 1 -- 172.21.2.104:0/2806169040 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407800d900 con 0x7f40900b1db0 2021-07-11T03:59:06.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 -- 172.21.2.104:0/2806169040 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40900b1db0 msgr2=0x7f40900b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:06.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 --2- 172.21.2.104:0/2806169040 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40900b1db0 0x7f40900b0fd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f4078002f70 tx=0x7f4078006eb0).stop 2021-07-11T03:59:06.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 -- 172.21.2.104:0/2806169040 shutdown_connections 2021-07-11T03:59:06.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 --2- 172.21.2.104:0/2806169040 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40901483a0 0x7f40900b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:06.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 --2- 172.21.2.104:0/2806169040 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40900b1db0 0x7f40900b0fd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:06.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 -- 172.21.2.104:0/2806169040 >> 172.21.2.104:0/2806169040 conn(0x7f409005f6b0 msgr2=0x7f409005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:06.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 -- 172.21.2.104:0/2806169040 shutdown_connections 2021-07-11T03:59:06.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 -- 172.21.2.104:0/2806169040 wait complete. 2021-07-11T03:59:06.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 Processor -- start 2021-07-11T03:59:06.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 -- start start 2021-07-11T03:59:06.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40900a70c0 0x7f4090142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:06.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40900b1db0 0x7f409015a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:06.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40901483a0 0x7f409015a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:06.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409014a030 con 0x7f40901483a0 2021-07-11T03:59:06.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409015ac70 con 0x7f40900a70c0 2021-07-11T03:59:06.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4090141780 con 0x7f40900b1db0 2021-07-11T03:59:06.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40901483a0 0x7f409015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:06.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40901483a0 0x7f409015a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46008/0 (socket says 172.21.2.104:46008) 2021-07-11T03:59:06.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408ed9d700 1 -- 172.21.2.104:0/2199995155 learned_addr learned my addr 172.21.2.104:0/2199995155 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:06.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408e59c700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40900a70c0 0x7f4090142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:06.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408dd9b700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40900b1db0 0x7f409015a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:06.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408ed9d700 1 -- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40900b1db0 msgr2=0x7f409015a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:06.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408ed9d700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40900b1db0 0x7f409015a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:06.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408ed9d700 1 -- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40900a70c0 msgr2=0x7f4090142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:06.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408ed9d700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40900a70c0 0x7f4090142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:06.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408ed9d700 1 -- 172.21.2.104:0/2199995155 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407800b1e0 con 0x7f40901483a0 2021-07-11T03:59:06.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f408ed9d700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40901483a0 0x7f409015a780 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f4090143650 tx=0x7f4088007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:06.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f40877fe700 1 -- 172.21.2.104:0/2199995155 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408800e4a0 con 0x7f40901483a0 2021-07-11T03:59:06.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f40877fe700 1 -- 172.21.2.104:0/2199995155 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408800e600 con 0x7f40901483a0 2021-07-11T03:59:06.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f40877fe700 1 -- 172.21.2.104:0/2199995155 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f408800e960 con 0x7f40901483a0 2021-07-11T03:59:06.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 -- 172.21.2.104:0/2199995155 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4090141a20 con 0x7f40901483a0 2021-07-11T03:59:06.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.617+0000 7f4094aa1700 1 -- 172.21.2.104:0/2199995155 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4090159df0 con 0x7f40901483a0 2021-07-11T03:59:06.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.621+0000 7f4094aa1700 1 -- 172.21.2.104:0/2199995155 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4090159fb0 con 0x7f40901483a0 2021-07-11T03:59:06.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.621+0000 7f40877fe700 1 -- 172.21.2.104:0/2199995155 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4088019030 con 0x7f40901483a0 2021-07-11T03:59:06.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.621+0000 7f40877fe700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4074053300 0x7f40740557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:06.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.621+0000 7f40877fe700 1 -- 172.21.2.104:0/2199995155 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f408806aaa0 con 0x7f40901483a0 2021-07-11T03:59:06.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.621+0000 7f408e59c700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4074053300 0x7f40740557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:06.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.621+0000 7f408e59c700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4074053300 0x7f40740557c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f4078019000 tx=0x7f4078015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:06.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.621+0000 7f40877fe700 1 -- 172.21.2.104:0/2199995155 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4088041f50 con 0x7f40901483a0 2021-07-11T03:59:06.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:06.734+0000 7f4094aa1700 1 -- 172.21.2.104:0/2199995155 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]} v 0) v1 -- 0x7f4090057350 con 0x7f40901483a0 2021-07-11T03:59:07.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f40877fe700 1 -- 172.21.2.104:0/2199995155 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestCacheDrop.test_drop_cache_command_timeout"]}]=0 v235) v1 ==== 147+0+0 (secure 0 0 0) 0x7f4088017070 con 0x7f40901483a0 2021-07-11T03:59:07.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 -- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4074053300 msgr2=0x7f40740557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:07.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4074053300 0x7f40740557c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f4078019000 tx=0x7f4078015000).stop 2021-07-11T03:59:07.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 -- 172.21.2.104:0/2199995155 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40901483a0 msgr2=0x7f409015a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:07.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40901483a0 0x7f409015a780 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f4090143650 tx=0x7f4088007f40).stop 2021-07-11T03:59:07.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 -- 172.21.2.104:0/2199995155 shutdown_connections 2021-07-11T03:59:07.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4074053300 0x7f40740557c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:07.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40901483a0 0x7f409015a780 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:07.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40900b1db0 0x7f409015a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:07.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 --2- 172.21.2.104:0/2199995155 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40900a70c0 0x7f4090142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:07.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 -- 172.21.2.104:0/2199995155 >> 172.21.2.104:0/2199995155 conn(0x7f409005f6b0 msgr2=0x7f40900b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:07.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 -- 172.21.2.104:0/2199995155 shutdown_connections 2021-07-11T03:59:07.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.366+0000 7f4094aa1700 1 -- 172.21.2.104:0/2199995155 wait complete. 2021-07-11T03:59:07.383 INFO:tasks.cephfs_test_runner:test_drop_cache_command_timeout (tasks.cephfs.test_misc.TestCacheDrop) ... ok 2021-07-11T03:59:07.384 INFO:tasks.cephfs_test_runner:Starting test: test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) 2021-07-11T03:59:07.384 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2021-07-11T03:59:07.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe4957e700 1 Processor -- start 2021-07-11T03:59:07.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe4957e700 1 -- start start 2021-07-11T03:59:07.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe4957e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fbe441393d0 0x7fbe440b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:07.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe4957e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbe44142840 0x7fbe44142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:07.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe4957e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe440604e0 con 0x7fbe44060100 2021-07-11T03:59:07.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe4957e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe440b16b0 con 0x7fbe441393d0 2021-07-11T03:59:07.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe4957e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440ac400 con 0x7fbe44142840 2021-07-11T03:59:07.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe427fc700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fbe441393d0 0x7fbe440b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:07.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe437fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbe44142840 0x7fbe44142ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:07.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe42ffd700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fbe44060100 0x7fbe44138cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:42150/0 (socket says 172.21.2.104:42150) 2021-07-11T03:59:07.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe42ffd700 1 -- 172.21.2.104:0/2117578876 learned_addr learned my addr 172.21.2.104:0/2117578876 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:07.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe437fe700 1 -- 172.21.2.104:0/2117578876 >> v2:172.21.2.105:3300/0 conn(0x7fbe441393d0 msgr2=0x7fbe440b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:07.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe437fe700 1 --2- 172.21.2.104:0/2117578876 >> v2:172.21.2.105:3300/0 conn(0x7fbe441393d0 0x7fbe440b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:07.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe437fe700 1 -- 172.21.2.104:0/2117578876 >> v1:172.21.2.105:6789/0 conn(0x7fbe44060100 legacy=0x7fbe44138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:07.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe437fe700 1 -- 172.21.2.104:0/2117578876 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe440ac6a0 con 0x7fbe44142840 2021-07-11T03:59:07.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe427fc700 1 --2- 172.21.2.104:0/2117578876 >> v2:172.21.2.105:3300/0 conn(0x7fbe441393d0 0x7fbe440b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T03:59:07.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.446+0000 7fbe437fe700 1 --2- 172.21.2.104:0/2117578876 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbe44142840 0x7fbe44142ca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbe3c0032f0 tx=0x7fbe3c0060d0).ready entity=mon.2 client_cookie=a21d1ad4c1273267 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:07.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe41ffb700 1 -- 172.21.2.104:0/2117578876 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe3c00f070 con 0x7fbe44142840 2021-07-11T03:59:07.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe41ffb700 1 -- 172.21.2.104:0/2117578876 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c00a9a0 con 0x7fbe44142840 2021-07-11T03:59:07.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe41ffb700 1 -- 172.21.2.104:0/2117578876 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe3c00ad00 con 0x7fbe44142840 2021-07-11T03:59:07.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- 172.21.2.104:0/2117578876 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbe44142840 msgr2=0x7fbe44142ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:07.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 --2- 172.21.2.104:0/2117578876 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbe44142840 0x7fbe44142ca0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbe3c0032f0 tx=0x7fbe3c0060d0).stop 2021-07-11T03:59:07.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- 172.21.2.104:0/2117578876 shutdown_connections 2021-07-11T03:59:07.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 --2- 172.21.2.104:0/2117578876 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbe44142840 0x7fbe44142ca0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:07.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 --2- 172.21.2.104:0/2117578876 >> v2:172.21.2.105:3300/0 conn(0x7fbe441393d0 0x7fbe440b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:07.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- 172.21.2.104:0/2117578876 >> 172.21.2.104:0/2117578876 conn(0x7fbe44061070 msgr2=0x7fbe4405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:07.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- 172.21.2.104:0/2117578876 shutdown_connections 2021-07-11T03:59:07.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- 172.21.2.104:0/2117578876 wait complete. 2021-07-11T03:59:07.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 Processor -- start 2021-07-11T03:59:07.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- start start 2021-07-11T03:59:07.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbe44060100 0x7fbe44067380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:07.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbe441393d0 0x7fbe44156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:07.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbe44142840 0x7fbe44152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:07.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44141e80 con 0x7fbe44060100 2021-07-11T03:59:07.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44142120 con 0x7fbe441393d0 2021-07-11T03:59:07.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441558e0 con 0x7fbe44142840 2021-07-11T03:59:07.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe42ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbe44060100 0x7fbe44067380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:07.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe42ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbe44060100 0x7fbe44067380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46018/0 (socket says 172.21.2.104:46018) 2021-07-11T03:59:07.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe42ffd700 1 -- 172.21.2.104:0/2828929813 learned_addr learned my addr 172.21.2.104:0/2828929813 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:07.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe427fc700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbe441393d0 0x7fbe44156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:07.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe42ffd700 1 -- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbe44142840 msgr2=0x7fbe44152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:07.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe42ffd700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbe44142840 0x7fbe44152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:07.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe42ffd700 1 -- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbe441393d0 msgr2=0x7fbe44156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:07.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe42ffd700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbe441393d0 0x7fbe44156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:07.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe42ffd700 1 -- 172.21.2.104:0/2828929813 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3400d410 con 0x7fbe44060100 2021-07-11T03:59:07.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe42ffd700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbe44060100 0x7fbe44067380 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fbe44061a70 tx=0x7fbe2c007cd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:07.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe3bfff700 1 -- 172.21.2.104:0/2828929813 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe2c00ce30 con 0x7fbe44060100 2021-07-11T03:59:07.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe3bfff700 1 -- 172.21.2.104:0/2828929813 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c00f470 con 0x7fbe44060100 2021-07-11T03:59:07.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- 172.21.2.104:0/2828929813 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c0074b0 con 0x7fbe44060100 2021-07-11T03:59:07.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- 172.21.2.104:0/2828929813 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe441423d0 con 0x7fbe44060100 2021-07-11T03:59:07.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe3bfff700 1 -- 172.21.2.104:0/2828929813 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbe2c00f7d0 con 0x7fbe44060100 2021-07-11T03:59:07.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe4957e700 1 -- 172.21.2.104:0/2828929813 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe44057350 con 0x7fbe44060100 2021-07-11T03:59:07.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.450+0000 7fbe3bfff700 1 -- 172.21.2.104:0/2828929813 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbe2c019030 con 0x7fbe44060100 2021-07-11T03:59:07.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.454+0000 7fbe3bfff700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbe28053240 0x7fbe28055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:07.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.454+0000 7fbe3bfff700 1 -- 172.21.2.104:0/2828929813 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbe2c06a710 con 0x7fbe44060100 2021-07-11T03:59:07.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.454+0000 7fbe427fc700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbe28053240 0x7fbe28055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:07.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.454+0000 7fbe427fc700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbe28053240 0x7fbe28055700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fbe34008000 tx=0x7fbe34008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:07.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.454+0000 7fbe3bfff700 1 -- 172.21.2.104:0/2828929813 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbe2c041c20 con 0x7fbe44060100 2021-07-11T03:59:07.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:07.566+0000 7fbe4957e700 1 -- 172.21.2.104:0/2828929813 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7fbe4405f2f0 con 0x7fbe44060100 2021-07-11T03:59:08.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe3bfff700 1 -- 172.21.2.104:0/2828929813 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v236) v1 ==== 142+0+0 (secure 0 0 0) 0x7fbe2c045240 con 0x7fbe44060100 2021-07-11T03:59:08.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 -- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbe28053240 msgr2=0x7fbe28055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbe28053240 0x7fbe28055700 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fbe34008000 tx=0x7fbe34008040).stop 2021-07-11T03:59:08.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 -- 172.21.2.104:0/2828929813 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbe44060100 msgr2=0x7fbe44067380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbe44060100 0x7fbe44067380 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fbe44061a70 tx=0x7fbe2c007cd0).stop 2021-07-11T03:59:08.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 -- 172.21.2.104:0/2828929813 shutdown_connections 2021-07-11T03:59:08.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbe44142840 0x7fbe44152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:08.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbe441393d0 0x7fbe44156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:08.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbe28053240 0x7fbe28055700 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:08.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 --2- 172.21.2.104:0/2828929813 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbe44060100 0x7fbe44067380 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:08.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 -- 172.21.2.104:0/2828929813 >> 172.21.2.104:0/2828929813 conn(0x7fbe44061070 msgr2=0x7fbe440a9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:08.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 -- 172.21.2.104:0/2828929813 shutdown_connections 2021-07-11T03:59:08.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.370+0000 7fbe4957e700 1 -- 172.21.2.104:0/2828929813 wait complete. 2021-07-11T03:59:08.385 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T03:59:08.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 Processor -- start 2021-07-11T03:59:08.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- start start 2021-07-11T03:59:08.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9d78060100 0x7f9d78064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:08.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d780604e0 con 0x7f9d78144440 2021-07-11T03:59:08.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d780b23b0 con 0x7f9d780b38d0 2021-07-11T03:59:08.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d78065c80 con 0x7f9d78060100 2021-07-11T03:59:08.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7659c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f9d78144440 0x7f9d78144970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42798/0 (socket says 172.21.2.104:42798) 2021-07-11T03:59:08.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7659c700 1 -- 172.21.2.104:0/1972625495 learned_addr learned my addr 172.21.2.104:0/1972625495 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:08.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d76d9d700 1 --2- 172.21.2.104:0/1972625495 >> v2:172.21.2.104:3300/0 conn(0x7f9d78060100 0x7f9d78064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:08.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d76d9d700 1 -- 172.21.2.104:0/1972625495 >> v1:172.21.2.105:6789/0 conn(0x7f9d780b38d0 legacy=0x7f9d7814b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:08.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d76d9d700 1 -- 172.21.2.104:0/1972625495 >> v1:172.21.2.104:6789/0 conn(0x7f9d78144440 legacy=0x7f9d78144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d76d9d700 1 -- 172.21.2.104:0/1972625495 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d780b2650 con 0x7f9d78060100 2021-07-11T03:59:08.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d76d9d700 1 --2- 172.21.2.104:0/1972625495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d78060100 0x7f9d78064d10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f9d7805f160 tx=0x7f9d60006df0).ready entity=mon.0 client_cookie=24fb420a436cde1f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:08.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d75d9b700 1 -- 172.21.2.104:0/1972625495 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d6000d5c0 con 0x7f9d78060100 2021-07-11T03:59:08.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d75d9b700 1 -- 172.21.2.104:0/1972625495 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d60000e90 con 0x7f9d78060100 2021-07-11T03:59:08.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d75d9b700 1 -- 172.21.2.104:0/1972625495 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d6000d910 con 0x7f9d78060100 2021-07-11T03:59:08.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- 172.21.2.104:0/1972625495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d78060100 msgr2=0x7f9d78064d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 --2- 172.21.2.104:0/1972625495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d78060100 0x7f9d78064d10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f9d7805f160 tx=0x7f9d60006df0).stop 2021-07-11T03:59:08.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- 172.21.2.104:0/1972625495 shutdown_connections 2021-07-11T03:59:08.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 --2- 172.21.2.104:0/1972625495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d78060100 0x7f9d78064d10 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:08.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- 172.21.2.104:0/1972625495 >> 172.21.2.104:0/1972625495 conn(0x7f9d7805cf60 msgr2=0x7f9d7805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:08.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- 172.21.2.104:0/1972625495 shutdown_connections 2021-07-11T03:59:08.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- 172.21.2.104:0/1972625495 wait complete. 2021-07-11T03:59:08.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 Processor -- start 2021-07-11T03:59:08.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- start start 2021-07-11T03:59:08.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9d78060100 0x7f9d78143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:08.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9d780b38d0 0x7f9d78153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:08.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d78153df0 0x7f9d78138880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:08.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780ae720 con 0x7f9d78153df0 2021-07-11T03:59:08.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7813d430 con 0x7f9d780b38d0 2021-07-11T03:59:08.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7d133700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780ae910 con 0x7f9d78060100 2021-07-11T03:59:08.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7759e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d78153df0 0x7f9d78138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:08.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7759e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d78153df0 0x7f9d78138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46036/0 (socket says 172.21.2.104:46036) 2021-07-11T03:59:08.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.434+0000 7f9d7759e700 1 -- 172.21.2.104:0/399388952 learned_addr learned my addr 172.21.2.104:0/399388952 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:08.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d7759e700 1 -- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9d78060100 msgr2=0x7f9d78143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d7759e700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9d78060100 0x7f9d78143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:08.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d7759e700 1 -- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9d780b38d0 msgr2=0x7f9d78153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d7759e700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9d780b38d0 0x7f9d78153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:08.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d7759e700 1 -- 172.21.2.104:0/399388952 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6000b180 con 0x7f9d78153df0 2021-07-11T03:59:08.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d7759e700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d78153df0 0x7f9d78138880 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f9d70003310 tx=0x7f9d70009120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:08.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d6ffff700 1 -- 172.21.2.104:0/399388952 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d70001500 con 0x7f9d78153df0 2021-07-11T03:59:08.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d6ffff700 1 -- 172.21.2.104:0/399388952 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d70001660 con 0x7f9d78153df0 2021-07-11T03:59:08.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d6ffff700 1 -- 172.21.2.104:0/399388952 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d700019c0 con 0x7f9d78153df0 2021-07-11T03:59:08.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d7d133700 1 -- 172.21.2.104:0/399388952 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d780aebb0 con 0x7f9d78153df0 2021-07-11T03:59:08.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d7d133700 1 -- 172.21.2.104:0/399388952 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7813e760 con 0x7f9d78153df0 2021-07-11T03:59:08.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d7d133700 1 -- 172.21.2.104:0/399388952 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d780aed40 con 0x7f9d78153df0 2021-07-11T03:59:08.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d6ffff700 1 -- 172.21.2.104:0/399388952 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9d70018030 con 0x7f9d78153df0 2021-07-11T03:59:08.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d6ffff700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d5c053300 0x7f9d5c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:08.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d6ffff700 1 -- 172.21.2.104:0/399388952 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9d7006aca0 con 0x7f9d78153df0 2021-07-11T03:59:08.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d76d9d700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d5c053300 0x7f9d5c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:08.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.438+0000 7f9d76d9d700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d5c053300 0x7f9d5c0557c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f9d7805f160 tx=0x7f9d60012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:08.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.442+0000 7f9d6ffff700 1 -- 172.21.2.104:0/399388952 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9d70042260 con 0x7f9d78153df0 2021-07-11T03:59:08.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.522+0000 7f9d7d133700 1 -- 172.21.2.104:0/399388952 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9d78057350 con 0x7f9d78153df0 2021-07-11T03:59:08.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d6ffff700 1 -- 172.21.2.104:0/399388952 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9d70016030 con 0x7f9d78153df0 2021-07-11T03:59:08.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 -- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d5c053300 msgr2=0x7f9d5c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d5c053300 0x7f9d5c0557c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f9d7805f160 tx=0x7f9d60012000).stop 2021-07-11T03:59:08.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 -- 172.21.2.104:0/399388952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d78153df0 msgr2=0x7f9d78138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d78153df0 0x7f9d78138880 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f9d70003310 tx=0x7f9d70009120).stop 2021-07-11T03:59:08.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 -- 172.21.2.104:0/399388952 shutdown_connections 2021-07-11T03:59:08.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d5c053300 0x7f9d5c0557c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:08.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d78153df0 0x7f9d78138880 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:08.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9d780b38d0 0x7f9d78153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:08.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 --2- 172.21.2.104:0/399388952 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9d78060100 0x7f9d78143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:08.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 -- 172.21.2.104:0/399388952 >> 172.21.2.104:0/399388952 conn(0x7f9d7805cf60 msgr2=0x7f9d78139c60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:08.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 -- 172.21.2.104:0/399388952 shutdown_connections 2021-07-11T03:59:08.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.530+0000 7f9d7d133700 1 -- 172.21.2.104:0/399388952 wait complete. 2021-07-11T03:59:08.543 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T03:59:08.544 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:59:08.552 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:59:08.562 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:59:08.576 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T03:59:08.589 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T03:59:08.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:08.590 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:59:08.591 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:59:08.601 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T03:59:08.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:59:08.603 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T03:59:08.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:08.604 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:59:08.604 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:59:08.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T03:59:08.615 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T03:59:08.616 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T03:59:08.617 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T03:59:08.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.666+0000 7fd20a448700 1 Processor -- start 2021-07-11T03:59:08.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.666+0000 7fd20a448700 1 -- start start 2021-07-11T03:59:08.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd2040b1db0 0x7fd2040b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:08.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd2040a70c0 0x7fd204144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:08.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2040b0350 con 0x7fd2041483a0 2021-07-11T03:59:08.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2040a90c0 con 0x7fd2040a70c0 2021-07-11T03:59:08.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040b05f0 con 0x7fd2040b1db0 2021-07-11T03:59:08.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2089e5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd2040a70c0 0x7fd204144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:08.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2089e5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd2040a70c0 0x7fd204144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46044/0 (socket says 172.21.2.104:46044) 2021-07-11T03:59:08.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2089e5700 1 -- 172.21.2.104:0/4218492517 learned_addr learned my addr 172.21.2.104:0/4218492517 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:08.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd203fff700 1 --2- 172.21.2.104:0/4218492517 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd2040b1db0 0x7fd2040b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:08.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2089e5700 1 -- 172.21.2.104:0/4218492517 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd2040b1db0 msgr2=0x7fd2040b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:08.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2089e5700 1 --2- 172.21.2.104:0/4218492517 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd2040b1db0 0x7fd2040b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:08.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2089e5700 1 -- 172.21.2.104:0/4218492517 >> v1:172.21.2.105:6789/0 conn(0x7fd2041483a0 legacy=0x7fd2040b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:08.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2089e5700 1 -- 172.21.2.104:0/4218492517 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2040b0890 con 0x7fd2040a70c0 2021-07-11T03:59:08.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd203fff700 1 --2- 172.21.2.104:0/4218492517 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd2040b1db0 0x7fd2040b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:59:08.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2089e5700 1 --2- 172.21.2.104:0/4218492517 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd2040a70c0 0x7fd204144720 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fd2040613a0 tx=0x7fd1fc009290).ready entity=mon.0 client_cookie=e7f242e904485268 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:08.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd202ffd700 1 -- 172.21.2.104:0/4218492517 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1fc00b860 con 0x7fd2040a70c0 2021-07-11T03:59:08.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd202ffd700 1 -- 172.21.2.104:0/4218492517 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc00b9c0 con 0x7fd2040a70c0 2021-07-11T03:59:08.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd202ffd700 1 -- 172.21.2.104:0/4218492517 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1fc00bb70 con 0x7fd2040a70c0 2021-07-11T03:59:08.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- 172.21.2.104:0/4218492517 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd2040a70c0 msgr2=0x7fd204144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 --2- 172.21.2.104:0/4218492517 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd2040a70c0 0x7fd204144720 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fd2040613a0 tx=0x7fd1fc009290).stop 2021-07-11T03:59:08.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- 172.21.2.104:0/4218492517 shutdown_connections 2021-07-11T03:59:08.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 --2- 172.21.2.104:0/4218492517 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd2040a70c0 0x7fd204144720 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:08.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 --2- 172.21.2.104:0/4218492517 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd2040b1db0 0x7fd2040b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:08.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- 172.21.2.104:0/4218492517 >> 172.21.2.104:0/4218492517 conn(0x7fd20405f6b0 msgr2=0x7fd20405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:08.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- 172.21.2.104:0/4218492517 shutdown_connections 2021-07-11T03:59:08.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- 172.21.2.104:0/4218492517 wait complete. 2021-07-11T03:59:08.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 Processor -- start 2021-07-11T03:59:08.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- start start 2021-07-11T03:59:08.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd2040a70c0 0x7fd2040ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:08.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd2040b1db0 0x7fd2040abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:08.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd2041483a0 0x7fd2040aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:08.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040aafb0 con 0x7fd2040b1db0 2021-07-11T03:59:08.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2040ac330 con 0x7fd2040a70c0 2021-07-11T03:59:08.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20415a8f0 con 0x7fd2041483a0 2021-07-11T03:59:08.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2037fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd2040b1db0 0x7fd2040abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:08.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2037fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd2040b1db0 0x7fd2040abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46046/0 (socket says 172.21.2.104:46046) 2021-07-11T03:59:08.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2037fe700 1 -- 172.21.2.104:0/3044826491 learned_addr learned my addr 172.21.2.104:0/3044826491 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:08.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2037fe700 1 -- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd2041483a0 msgr2=0x7fd2040aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2037fe700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd2041483a0 0x7fd2040aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:08.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2037fe700 1 -- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd2040a70c0 msgr2=0x7fd2040ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2037fe700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd2040a70c0 0x7fd2040ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:08.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2037fe700 1 -- 172.21.2.104:0/3044826491 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc009530 con 0x7fd2040b1db0 2021-07-11T03:59:08.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2037fe700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd2040b1db0 0x7fd2040abe40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fd2040ac4c0 tx=0x7fd1f4006d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:08.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2017fa700 1 -- 172.21.2.104:0/3044826491 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1f4012410 con 0x7fd2040b1db0 2021-07-11T03:59:08.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2017fa700 1 -- 172.21.2.104:0/3044826491 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f4012570 con 0x7fd2040b1db0 2021-07-11T03:59:08.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- 172.21.2.104:0/3044826491 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2040ab1e0 con 0x7fd2040b1db0 2021-07-11T03:59:08.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd2017fa700 1 -- 172.21.2.104:0/3044826491 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd1f40128d0 con 0x7fd2040b1db0 2021-07-11T03:59:08.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.670+0000 7fd20a448700 1 -- 172.21.2.104:0/3044826491 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2040aa980 con 0x7fd2040b1db0 2021-07-11T03:59:08.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.674+0000 7fd20a448700 1 -- 172.21.2.104:0/3044826491 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd204062960 con 0x7fd2040b1db0 2021-07-11T03:59:08.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.674+0000 7fd2017fa700 1 -- 172.21.2.104:0/3044826491 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd1f400f030 con 0x7fd2040b1db0 2021-07-11T03:59:08.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.674+0000 7fd2017fa700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd1e8053240 0x7fd1e8055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:08.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.674+0000 7fd2017fa700 1 -- 172.21.2.104:0/3044826491 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd1f406afe0 con 0x7fd2040b1db0 2021-07-11T03:59:08.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.674+0000 7fd203fff700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd1e8053240 0x7fd1e8055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:08.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.674+0000 7fd203fff700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd1e8053240 0x7fd1e8055700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fd1ec007000 tx=0x7fd1ec008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:08.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.674+0000 7fd2017fa700 1 -- 172.21.2.104:0/3044826491 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd1f40424f0 con 0x7fd2040b1db0 2021-07-11T03:59:08.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.754+0000 7fd20a448700 1 -- 172.21.2.104:0/3044826491 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd204057350 con 0x7fd2040b1db0 2021-07-11T03:59:08.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd2017fa700 1 -- 172.21.2.104:0/3044826491 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v69) v1 ==== 81+0+28826 (secure 0 0 0) 0x7fd1f4045b10 con 0x7fd2040b1db0 2021-07-11T03:59:08.764 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:08.764 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:59:08.765 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 69, 2021-07-11T03:59:08.765 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T03:59:08.765 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T03:59:08.765 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T03:59:06.345891+0000", 2021-07-11T03:59:08.765 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:59:08.766 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T03:59:08.766 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T03:59:08.766 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T03:59:08.767 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T03:59:08.767 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T03:59:08.767 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T03:59:08.768 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T03:59:08.768 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T03:59:08.768 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:59:08.768 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T03:59:08.769 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T03:59:08.769 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T03:59:08.769 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T03:59:08.769 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T03:59:08.769 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 12, 2021-07-11T03:59:08.770 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T03:59:08.770 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T03:59:08.770 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T03:59:08.770 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T03:59:08.771 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T03:59:08.771 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.771 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T03:59:08.771 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T03:59:08.771 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T03:59:08.772 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:59:08.772 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:59:08.772 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:59:08.772 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:59:08.772 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:59:08.773 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:59:08.773 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:59:08.773 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:59:08.773 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T03:59:08.773 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T03:59:08.774 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T03:59:08.774 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T03:59:08.774 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T03:59:08.774 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:59:08.775 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:59:08.775 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:59:08.775 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:59:08.775 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:59:08.775 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:59:08.776 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:59:08.776 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.776 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T03:59:08.776 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:59:08.776 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:59:08.777 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:59:08.777 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:59:08.778 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:59:08.778 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:59:08.778 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:59:08.778 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:59:08.779 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:59:08.779 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:59:08.779 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:59:08.779 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:59:08.779 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:59:08.780 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:59:08.780 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:59:08.780 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:59:08.780 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:59:08.780 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:59:08.781 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:59:08.781 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:59:08.781 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:59:08.781 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:59:08.782 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:59:08.782 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:59:08.782 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:59:08.782 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:59:08.782 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.783 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:59:08.783 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:59:08.783 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:59:08.783 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:59:08.783 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:59:08.784 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:59:08.784 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:59:08.784 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:59:08.784 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:59:08.785 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:59:08.785 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:59:08.785 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T03:59:08.785 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T03:59:08.786 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.786 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:59:08.786 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T03:59:08.786 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.787 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.787 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.787 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T03:59:08.788 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T03:59:08.788 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T03:59:08.788 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T03:59:08.788 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T03:59:08.788 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T03:59:08.789 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T03:59:08.789 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T03:59:08.789 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T03:59:08.789 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T03:59:08.789 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T03:59:08.790 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T03:59:08.790 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T03:59:08.790 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T03:59:08.790 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T03:59:08.791 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T03:59:08.791 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T03:59:08.791 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T03:59:08.791 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T03:59:08.791 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T03:59:08.792 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T03:59:08.792 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T03:59:08.792 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T03:59:08.792 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.792 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T03:59:08.793 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T03:59:08.793 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T03:59:08.793 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T03:59:08.793 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T03:59:08.794 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T03:59:08.794 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T03:59:08.794 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T03:59:08.795 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T03:59:08.795 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T03:59:08.795 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T03:59:08.795 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T03:59:08.795 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T03:59:08.796 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T03:59:08.796 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T03:59:08.796 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T03:59:08.796 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T03:59:08.796 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T03:59:08.797 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T03:59:08.797 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T03:59:08.797 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T03:59:08.797 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T03:59:08.797 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T03:59:08.798 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T03:59:08.798 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T03:59:08.798 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T03:59:08.799 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T03:59:08.799 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.799 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T03:59:08.799 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T03:59:08.800 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T03:59:08.800 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T03:59:08.800 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T03:59:08.800 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T03:59:08.800 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T03:59:08.801 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T03:59:08.801 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T03:59:08.801 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T03:59:08.801 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T03:59:08.801 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T03:59:08.802 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T03:59:08.802 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T03:59:08.802 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.803 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.803 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:59:08.803 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T03:59:08.803 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.803 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:59:08.804 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T03:59:08.804 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:08.804 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:08.804 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:08.805 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:08.805 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:08.805 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:08.805 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:08.805 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 62, 2021-07-11T03:59:08.806 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:08.806 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:08.806 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:08.806 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.807 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.807 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.807 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T03:59:08.807 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:08.807 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.808 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.808 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.808 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T03:59:08.809 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:08.809 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.809 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.809 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.810 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:08.810 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.810 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.811 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.811 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T03:59:08.811 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:08.811 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.811 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.812 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.812 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T03:59:08.812 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:08.812 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.813 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.813 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.813 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:08.813 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.813 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.814 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.814 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T03:59:08.814 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:08.814 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.815 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.815 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.815 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T03:59:08.815 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:08.815 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.816 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.816 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.816 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:08.816 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.816 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.817 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.817 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T03:59:08.817 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:08.817 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.817 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.818 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.818 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T03:59:08.818 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T03:59:08.819 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.819 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.819 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.820 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T03:59:08.820 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T03:59:08.820 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T03:59:08.820 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T03:59:08.821 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:08.821 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:08.821 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:08.821 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.822 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.822 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.822 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:59:08.822 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T03:59:08.822 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:08.823 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:08.823 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:08.823 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:08.823 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:08.824 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:08.824 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:08.824 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 62, 2021-07-11T03:59:08.824 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:08.824 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:08.825 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:08.825 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.825 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.825 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.826 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T03:59:08.826 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:08.826 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.826 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.826 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.827 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T03:59:08.827 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:08.827 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.827 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.827 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.828 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:08.828 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.828 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.829 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.829 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T03:59:08.830 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:08.830 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.830 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.830 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.830 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T03:59:08.831 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:08.831 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.831 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.831 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.832 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:08.832 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.832 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.832 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.832 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T03:59:08.833 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:08.833 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.833 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.833 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.834 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T03:59:08.834 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:08.834 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.834 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.834 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.835 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:08.835 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.835 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.835 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.836 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T03:59:08.836 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:08.836 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.836 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.837 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.837 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T03:59:08.837 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T03:59:08.837 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.838 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.838 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.838 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T03:59:08.838 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T03:59:08.839 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T03:59:08.839 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T03:59:08.840 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:08.840 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:08.840 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:08.841 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.841 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.841 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.841 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:59:08.842 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T03:59:08.842 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:08.842 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:08.842 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:08.842 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:08.843 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:08.843 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:08.843 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:08.843 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 61, 2021-07-11T03:59:08.843 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:08.844 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:08.844 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:08.844 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.844 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.845 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.845 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T03:59:08.845 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:08.845 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.846 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.846 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.846 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T03:59:08.846 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:08.847 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.847 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.847 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.847 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:08.847 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.848 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.848 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.848 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T03:59:08.848 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:08.848 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.849 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.849 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.849 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T03:59:08.849 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:08.850 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.850 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.850 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.851 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:08.851 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.851 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.851 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.852 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T03:59:08.852 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:08.852 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.852 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.852 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.853 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T03:59:08.853 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:08.853 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.853 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.853 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.854 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:08.854 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.854 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.855 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.855 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T03:59:08.855 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:08.855 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.855 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.856 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.856 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T03:59:08.856 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T03:59:08.856 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.857 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.857 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.857 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T03:59:08.857 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T03:59:08.857 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T03:59:08.858 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T03:59:08.858 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:08.858 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:08.858 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:08.858 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.859 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.859 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.859 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:59:08.859 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T03:59:08.860 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:08.860 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:08.860 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:08.860 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:08.861 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:08.861 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:08.861 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:08.862 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 52, 2021-07-11T03:59:08.862 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:08.862 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:08.863 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:08.863 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.863 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.863 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.863 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T03:59:08.864 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:08.864 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.864 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.864 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.865 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T03:59:08.865 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:08.865 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.865 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.865 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.866 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:08.866 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.866 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.866 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.866 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T03:59:08.867 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:08.867 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.867 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.867 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.867 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T03:59:08.868 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:08.868 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.868 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.868 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.869 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:08.869 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.869 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.869 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.869 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T03:59:08.870 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:08.870 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.870 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.870 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.870 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T03:59:08.871 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:08.871 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.872 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.872 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.872 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:08.872 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.873 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.873 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.873 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T03:59:08.873 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:08.873 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.874 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.874 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.874 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T03:59:08.874 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T03:59:08.875 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.875 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.875 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.875 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T03:59:08.875 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T03:59:08.876 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T03:59:08.876 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T03:59:08.876 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:08.876 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:08.876 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:08.877 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.877 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.877 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.877 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:59:08.877 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T03:59:08.878 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:08.878 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:08.878 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:08.878 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:08.879 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:08.879 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:08.879 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:08.879 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 63, 2021-07-11T03:59:08.879 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:08.880 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:08.880 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:08.880 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.880 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.881 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.881 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T03:59:08.881 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:08.882 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.882 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.882 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.882 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T03:59:08.883 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:08.883 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.883 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.883 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.884 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:08.884 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.884 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.884 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.884 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T03:59:08.885 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:08.885 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.885 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.885 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.885 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T03:59:08.886 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:08.886 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.886 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.886 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.886 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:08.887 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.887 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.887 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.887 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T03:59:08.888 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:08.888 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.888 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.888 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.889 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T03:59:08.889 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:08.889 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.889 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.890 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.890 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:08.890 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.890 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.890 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.891 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T03:59:08.891 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:08.891 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.891 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.891 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.892 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T03:59:08.892 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T03:59:08.893 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.893 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.893 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.893 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T03:59:08.893 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T03:59:08.894 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T03:59:08.894 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T03:59:08.894 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:08.894 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:08.895 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:08.895 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.895 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.895 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.895 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:59:08.896 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T03:59:08.896 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:08.896 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:08.896 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:08.896 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:08.897 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:08.897 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:08.897 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:08.898 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 61, 2021-07-11T03:59:08.898 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:08.898 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:08.898 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:08.899 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.899 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.899 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.899 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T03:59:08.899 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:08.900 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.900 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.900 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.900 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T03:59:08.900 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:08.901 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.901 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.901 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.901 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:08.901 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.902 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.902 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.902 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T03:59:08.903 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:08.903 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.903 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.903 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.904 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T03:59:08.904 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:08.904 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.904 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.904 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.905 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:08.905 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.905 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.905 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.905 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T03:59:08.906 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:08.906 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.906 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.907 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.907 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T03:59:08.907 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:08.907 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.908 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.908 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.908 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:08.908 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.908 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.909 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.909 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T03:59:08.909 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:08.909 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.910 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.910 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.910 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T03:59:08.910 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T03:59:08.910 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.911 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.911 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.911 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T03:59:08.911 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T03:59:08.911 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T03:59:08.912 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T03:59:08.912 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:08.912 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:08.912 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:08.912 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.913 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.913 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.913 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:59:08.914 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T03:59:08.914 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:08.914 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:08.914 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:08.915 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:08.915 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:08.915 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:08.915 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:08.916 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 62, 2021-07-11T03:59:08.916 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:08.916 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:08.916 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:08.916 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.917 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.917 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.917 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T03:59:08.917 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:08.918 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.918 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.918 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.918 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T03:59:08.918 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:08.919 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.919 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.919 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.919 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:08.919 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.920 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.920 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.920 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T03:59:08.920 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:08.920 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.921 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.921 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.921 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T03:59:08.921 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:08.921 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.922 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.922 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.922 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:08.922 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.923 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.923 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.923 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T03:59:08.924 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:08.924 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.924 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.924 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.925 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T03:59:08.925 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:08.925 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.925 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.925 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.926 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:08.926 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.926 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.926 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.927 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T03:59:08.927 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:08.927 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.927 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.928 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.928 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T03:59:08.928 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T03:59:08.928 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.928 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.929 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.929 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T03:59:08.929 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T03:59:08.929 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T03:59:08.929 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T03:59:08.930 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:08.930 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:08.930 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:08.930 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.931 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.931 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.931 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:59:08.931 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T03:59:08.931 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T03:59:08.932 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T03:59:08.932 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T03:59:08.932 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T03:59:08.932 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T03:59:08.933 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T03:59:08.933 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T03:59:08.933 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 61, 2021-07-11T03:59:08.934 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T03:59:08.934 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T03:59:08.934 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T03:59:08.934 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.935 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.935 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.935 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T03:59:08.935 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:08.935 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.936 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.936 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.936 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T03:59:08.936 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:08.936 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.937 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.937 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.937 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T03:59:08.937 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.937 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.938 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.938 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T03:59:08.938 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:08.938 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.939 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.939 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.939 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T03:59:08.939 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:08.939 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.940 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.940 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.940 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T03:59:08.941 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.941 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.941 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.941 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T03:59:08.941 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:08.942 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.942 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.942 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.942 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T03:59:08.943 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:08.943 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.943 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.943 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.944 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T03:59:08.944 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T03:59:08.944 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.945 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T03:59:08.945 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T03:59:08.945 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:08.945 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.945 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.946 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T03:59:08.946 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T03:59:08.946 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T03:59:08.946 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.946 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.947 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.947 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T03:59:08.947 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T03:59:08.947 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T03:59:08.948 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T03:59:08.948 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T03:59:08.948 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T03:59:08.948 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T03:59:08.948 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:08.949 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.949 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:59:08.949 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T03:59:08.950 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.950 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T03:59:08.950 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:08.950 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:08.950 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:08.951 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:08.951 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:08.951 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T03:59:08.951 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:08.951 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.952 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.952 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T03:59:08.952 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:08.952 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:08.953 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:08.953 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:08.953 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:08.953 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T03:59:08.953 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:08.954 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.954 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.954 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T03:59:08.955 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:08.955 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:08.955 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:08.955 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:08.956 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:08.956 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T03:59:08.956 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:08.956 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.956 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.957 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T03:59:08.957 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:08.957 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:08.957 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:08.957 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:08.958 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:08.958 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T03:59:08.958 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:08.959 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.959 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.959 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T03:59:08.959 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:08.960 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:08.960 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:08.960 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:08.960 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:08.960 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T03:59:08.961 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:08.961 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.961 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.961 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T03:59:08.961 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:08.962 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:08.962 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:08.962 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:08.962 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:08.963 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T03:59:08.963 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:08.963 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.963 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.963 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T03:59:08.964 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:08.964 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:08.964 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:08.965 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:08.965 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:08.965 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T03:59:08.965 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:08.966 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.966 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:08.966 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T03:59:08.967 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T03:59:08.967 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T03:59:08.967 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T03:59:08.967 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T03:59:08.967 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T03:59:08.968 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T03:59:08.968 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T03:59:08.968 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.968 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T03:59:08.968 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T03:59:08.969 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T03:59:08.969 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T03:59:08.969 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T03:59:08.969 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T03:59:08.970 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6835/225259368": "2021-07-12T03:59:02.653299+0000", 2021-07-11T03:59:08.970 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6834/225259368": "2021-07-12T03:59:02.653299+0000" 2021-07-11T03:59:08.970 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.970 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T03:59:08.970 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T03:59:08.971 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T03:59:08.971 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T03:59:08.971 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T03:59:08.971 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T03:59:08.971 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T03:59:08.972 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T03:59:08.972 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:08.972 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:08.972 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T03:59:08.972 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T03:59:08.973 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T03:59:08.973 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T03:59:08.973 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T03:59:08.973 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:59:08.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 -- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd1e8053240 msgr2=0x7fd1e8055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd1e8053240 0x7fd1e8055700 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fd1ec007000 tx=0x7fd1ec008040).stop 2021-07-11T03:59:08.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 -- 172.21.2.104:0/3044826491 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd2040b1db0 msgr2=0x7fd2040abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:08.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd2040b1db0 0x7fd2040abe40 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fd2040ac4c0 tx=0x7fd1f4006d40).stop 2021-07-11T03:59:08.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 -- 172.21.2.104:0/3044826491 shutdown_connections 2021-07-11T03:59:08.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd2041483a0 0x7fd2040aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:08.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd1e8053240 0x7fd1e8055700 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:08.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd2040b1db0 0x7fd2040abe40 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:08.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 --2- 172.21.2.104:0/3044826491 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd2040a70c0 0x7fd2040ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:08.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 -- 172.21.2.104:0/3044826491 >> 172.21.2.104:0/3044826491 conn(0x7fd20405f6b0 msgr2=0x7fd204147770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:08.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 -- 172.21.2.104:0/3044826491 shutdown_connections 2021-07-11T03:59:08.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:08.758+0000 7fd20a448700 1 -- 172.21.2.104:0/3044826491 wait complete. 2021-07-11T03:59:08.978 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:59:09.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.026+0000 7fcf7e370700 1 Processor -- start 2021-07-11T03:59:09.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.026+0000 7fcf7e370700 1 -- start start 2021-07-11T03:59:09.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcf780a76b0 0x7fcf78066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf780b1970 0x7fcf78066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf78067450 con 0x7fcf780b0060 2021-07-11T03:59:09.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf780a7a90 con 0x7fcf780a76b0 2021-07-11T03:59:09.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78147620 con 0x7fcf780b1970 2021-07-11T03:59:09.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf777fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf780b1970 0x7fcf78066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf77fff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcf780a76b0 0x7fcf78066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7c90d700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fcf780b0060 0x7fcf7805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:42192/0 (socket says 172.21.2.104:42192) 2021-07-11T03:59:09.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7c90d700 1 -- 172.21.2.104:0/341491897 learned_addr learned my addr 172.21.2.104:0/341491897 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:09.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf76ffd700 1 -- 172.21.2.104:0/341491897 <== mon.1 v1:172.21.2.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618175813 0 0) 0x7fcf78067450 con 0x7fcf780b0060 2021-07-11T03:59:09.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf76ffd700 1 -- 172.21.2.104:0/341491897 --> v1:172.21.2.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf58003260 con 0x7fcf780b0060 2021-07-11T03:59:09.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf777fe700 1 -- 172.21.2.104:0/341491897 >> v2:172.21.2.105:3300/0 conn(0x7fcf780a76b0 msgr2=0x7fcf78066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:09.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf777fe700 1 --2- 172.21.2.104:0/341491897 >> v2:172.21.2.105:3300/0 conn(0x7fcf780a76b0 0x7fcf78066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf777fe700 1 -- 172.21.2.104:0/341491897 >> v1:172.21.2.105:6789/0 conn(0x7fcf780b0060 legacy=0x7fcf7805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf777fe700 1 -- 172.21.2.104:0/341491897 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf781478c0 con 0x7fcf780b1970 2021-07-11T03:59:09.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf77fff700 1 --2- 172.21.2.104:0/341491897 >> v2:172.21.2.105:3300/0 conn(0x7fcf780a76b0 0x7fcf78066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T03:59:09.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf777fe700 1 --2- 172.21.2.104:0/341491897 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf780b1970 0x7fcf78066f60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fcf68002f70 tx=0x7fcf68006df0).ready entity=mon.2 client_cookie=11c476edaba63d07 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:09.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf76ffd700 1 -- 172.21.2.104:0/341491897 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6800d4a0 con 0x7fcf780b1970 2021-07-11T03:59:09.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf76ffd700 1 -- 172.21.2.104:0/341491897 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf68000e90 con 0x7fcf780b1970 2021-07-11T03:59:09.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf76ffd700 1 -- 172.21.2.104:0/341491897 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6800d7f0 con 0x7fcf780b1970 2021-07-11T03:59:09.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 -- 172.21.2.104:0/341491897 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf780b1970 msgr2=0x7fcf78066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 --2- 172.21.2.104:0/341491897 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf780b1970 0x7fcf78066f60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fcf68002f70 tx=0x7fcf68006df0).stop 2021-07-11T03:59:09.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 -- 172.21.2.104:0/341491897 shutdown_connections 2021-07-11T03:59:09.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 --2- 172.21.2.104:0/341491897 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf780b1970 0x7fcf78066f60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 --2- 172.21.2.104:0/341491897 >> v2:172.21.2.105:3300/0 conn(0x7fcf780a76b0 0x7fcf78066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 -- 172.21.2.104:0/341491897 >> 172.21.2.104:0/341491897 conn(0x7fcf780b53e0 msgr2=0x7fcf780b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:09.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 -- 172.21.2.104:0/341491897 shutdown_connections 2021-07-11T03:59:09.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.030+0000 7fcf7e370700 1 -- 172.21.2.104:0/341491897 wait complete. 2021-07-11T03:59:09.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf7e370700 1 Processor -- start 2021-07-11T03:59:09.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf7e370700 1 -- start start 2021-07-11T03:59:09.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf7e370700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf780a76b0 0x7fcf780a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf7e370700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf780b0060 0x7fcf780ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf7e370700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf780b1970 0x7fcf780ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf7e370700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78157190 con 0x7fcf780b0060 2021-07-11T03:59:09.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf7e370700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78157660 con 0x7fcf780b1970 2021-07-11T03:59:09.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf7e370700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf780ade50 con 0x7fcf780a76b0 2021-07-11T03:59:09.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf777fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf780b0060 0x7fcf780ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf777fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf780b0060 0x7fcf780ab2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46060/0 (socket says 172.21.2.104:46060) 2021-07-11T03:59:09.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf777fe700 1 -- 172.21.2.104:0/2322828888 learned_addr learned my addr 172.21.2.104:0/2322828888 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:09.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf77fff700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf780a76b0 0x7fcf780a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf777fe700 1 -- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf780a76b0 msgr2=0x7fcf780a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf777fe700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf780a76b0 0x7fcf780a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf777fe700 1 -- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf780b1970 msgr2=0x7fcf780ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf777fe700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf780b1970 0x7fcf780ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:09.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf777fe700 1 -- 172.21.2.104:0/2322828888 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6000b180 con 0x7fcf780b0060 2021-07-11T03:59:09.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf777fe700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf780b0060 0x7fcf780ab2c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fcf68002f70 tx=0x7fcf68014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:09.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf757fa700 1 -- 172.21.2.104:0/2322828888 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf68012a30 con 0x7fcf780b0060 2021-07-11T03:59:09.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf757fa700 1 -- 172.21.2.104:0/2322828888 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf68012b90 con 0x7fcf780b0060 2021-07-11T03:59:09.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf757fa700 1 -- 172.21.2.104:0/2322828888 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6801f440 con 0x7fcf780b0060 2021-07-11T03:59:09.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf7e370700 1 -- 172.21.2.104:0/2322828888 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6800b180 con 0x7fcf780b0060 2021-07-11T03:59:09.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf7e370700 1 -- 172.21.2.104:0/2322828888 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf780a9320 con 0x7fcf780b0060 2021-07-11T03:59:09.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf7e370700 1 -- 172.21.2.104:0/2322828888 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf780a95e0 con 0x7fcf780b0060 2021-07-11T03:59:09.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf757fa700 1 -- 172.21.2.104:0/2322828888 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcf68025020 con 0x7fcf780b0060 2021-07-11T03:59:09.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf757fa700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf5c053210 0x7fcf5c0556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf757fa700 1 -- 172.21.2.104:0/2322828888 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcf6806c350 con 0x7fcf780b0060 2021-07-11T03:59:09.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf77fff700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf5c053210 0x7fcf5c0556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.034+0000 7fcf77fff700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf5c053210 0x7fcf5c0556d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fcf6000b000 tx=0x7fcf60008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:09.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.038+0000 7fcf757fa700 1 -- 172.21.2.104:0/2322828888 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcf780a95e0 con 0x7fcf780b0060 2021-07-11T03:59:09.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 -- 172.21.2.104:0/2322828888 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf780a95e0 con 0x7fcf780b0060 2021-07-11T03:59:09.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf757fa700 1 -- 172.21.2.104:0/2322828888 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fcf780a95e0 con 0x7fcf780b0060 2021-07-11T03:59:09.152 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:09.153 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/2908869555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2908869555},{"type":"v1","addr":"172.21.2.105:6835","nonce":2908869555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/1110128427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1110128427},{"type":"v1","addr":"172.21.2.104:6833","nonce":1110128427}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/214455898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":214455898},{"type":"v1","addr":"172.21.2.104:6835","nonce":214455898}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29}],"filesystems":[]} 2021-07-11T03:59:09.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 -- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf5c053210 msgr2=0x7fcf5c0556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf5c053210 0x7fcf5c0556d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fcf6000b000 tx=0x7fcf60008040).stop 2021-07-11T03:59:09.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 -- 172.21.2.104:0/2322828888 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf780b0060 msgr2=0x7fcf780ab2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf780b0060 0x7fcf780ab2c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fcf68002f70 tx=0x7fcf68014000).stop 2021-07-11T03:59:09.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 -- 172.21.2.104:0/2322828888 shutdown_connections 2021-07-11T03:59:09.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf780b1970 0x7fcf780ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:09.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf5c053210 0x7fcf5c0556d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf780b0060 0x7fcf780ab2c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 --2- 172.21.2.104:0/2322828888 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf780a76b0 0x7fcf780a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 -- 172.21.2.104:0/2322828888 >> 172.21.2.104:0/2322828888 conn(0x7fcf780b53e0 msgr2=0x7fcf780b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:09.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 -- 172.21.2.104:0/2322828888 shutdown_connections 2021-07-11T03:59:09.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.146+0000 7fcf7e370700 1 -- 172.21.2.104:0/2322828888 wait complete. 2021-07-11T03:59:09.156 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 29 2021-07-11T03:59:09.160 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:59:09.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.202+0000 7f698443f700 1 Processor -- start 2021-07-11T03:59:09.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.202+0000 7f698443f700 1 -- start start 2021-07-11T03:59:09.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f698443f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f697c060280 0x7f697c0668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f698443f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c0abea0 con 0x7f697c0b19a0 2021-07-11T03:59:09.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f698443f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c140300 con 0x7f697c0a97a0 2021-07-11T03:59:09.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f698443f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0ab310 con 0x7f697c060280 2021-07-11T03:59:09.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69819da700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f697c0b19a0 0x7f697c066da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42836/0 (socket says 172.21.2.104:42836) 2021-07-11T03:59:09.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69819da700 1 -- 172.21.2.104:0/1958400818 learned_addr learned my addr 172.21.2.104:0/1958400818 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:09.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707526052 0 0) 0x7f697c0abea0 con 0x7f697c0b19a0 2021-07-11T03:59:09.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6964003260 con 0x7f697c0b19a0 2021-07-11T03:59:09.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2013602244 0 0) 0x7f6964003260 con 0x7f697c0b19a0 2021-07-11T03:59:09.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69821db700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f697c060280 0x7f697c0668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f697c060280 msgr2=0x7f697c0668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:09.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f697c060280 0x7f697c0668b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 >> v1:172.21.2.105:6789/0 conn(0x7f697c0a97a0 legacy=0x7f697c0a9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c1405a0 con 0x7f697c0b19a0 2021-07-11T03:59:09.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f6978001660 con 0x7f697c0b19a0 2021-07-11T03:59:09.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 >> v1:172.21.2.104:6789/0 conn(0x7f697c0b19a0 legacy=0x7f697c066da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6964003800 0x7f6964003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6964004500 0x7f6964004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6964005070 0x7f6964005720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0ab310 con 0x7f6964003800 2021-07-11T03:59:09.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6964005ec0 con 0x7f6964005070 2021-07-11T03:59:09.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6964006080 con 0x7f6964004500 2021-07-11T03:59:09.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69821db700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f697c060280 0x7f697c0668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:59:09.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69821db700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6964003800 0x7f6964003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69829dc700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6964005070 0x7f6964005720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69821db700 1 -- 172.21.2.104:0/1958400818 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6964004500 msgr2=0x7f6964004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:09.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69821db700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6964004500 0x7f6964004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:09.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69821db700 1 -- 172.21.2.104:0/1958400818 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6964005070 msgr2=0x7f6964005720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:09.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69821db700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6964005070 0x7f6964005720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69821db700 1 -- 172.21.2.104:0/1958400818 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6964003260 con 0x7f6964003800 2021-07-11T03:59:09.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69821db700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6964003800 0x7f6964003c20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f6970007000 tx=0x7f6970008040).ready entity=mon.0 client_cookie=c6d441da1710f374 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:09.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6970012430 con 0x7f6964003800 2021-07-11T03:59:09.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697000c030 con 0x7f6964003800 2021-07-11T03:59:09.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69811d9700 1 -- 172.21.2.104:0/1958400818 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6970012780 con 0x7f6964003800 2021-07-11T03:59:09.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f698443f700 1 -- 172.21.2.104:0/1958400818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6964003800 msgr2=0x7f6964003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f698443f700 1 --2- 172.21.2.104:0/1958400818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6964003800 0x7f6964003c20 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f6970007000 tx=0x7f6970008040).stop 2021-07-11T03:59:09.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f69829dc700 1 -- 172.21.2.104:0/1958400818 reap_dead start 2021-07-11T03:59:09.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f698443f700 1 -- 172.21.2.104:0/1958400818 shutdown_connections 2021-07-11T03:59:09.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f698443f700 1 -- 172.21.2.104:0/1958400818 >> 172.21.2.104:0/1958400818 conn(0x7f697c0b53e0 msgr2=0x7f697c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:09.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f698443f700 1 -- 172.21.2.104:0/1958400818 shutdown_connections 2021-07-11T03:59:09.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f698443f700 1 -- 172.21.2.104:0/1958400818 wait complete. 2021-07-11T03:59:09.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.206+0000 7f698443f700 1 Processor -- start 2021-07-11T03:59:09.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f698443f700 1 -- start start 2021-07-11T03:59:09.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f698443f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f697c0632f0 0x7f697c0636d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f698443f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f697c14f030 0x7f697c065cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f698443f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f697c0661c0 0x7f697c0adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f698443f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0ac9f0 con 0x7f697c14f030 2021-07-11T03:59:09.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f698443f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c14f500 con 0x7f697c0632f0 2021-07-11T03:59:09.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f698443f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c1433f0 con 0x7f697c0661c0 2021-07-11T03:59:09.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69819da700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f697c14f030 0x7f697c065cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69819da700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f697c14f030 0x7f697c065cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46078/0 (socket says 172.21.2.104:46078) 2021-07-11T03:59:09.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69819da700 1 -- 172.21.2.104:0/3631722972 learned_addr learned my addr 172.21.2.104:0/3631722972 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:09.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69821db700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f697c0632f0 0x7f697c0636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69819da700 1 -- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f697c0661c0 msgr2=0x7f697c0adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69819da700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f697c0661c0 0x7f697c0adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:09.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69819da700 1 -- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f697c0632f0 msgr2=0x7f697c0636d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69819da700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f697c0632f0 0x7f697c0636d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69819da700 1 -- 172.21.2.104:0/3631722972 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c0acc50 con 0x7f697c14f030 2021-07-11T03:59:09.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69821db700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f697c0632f0 0x7f697c0636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:59:09.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69819da700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f697c14f030 0x7f697c065cd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f6978001370 tx=0x7f697c140490).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:09.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f696f7fe700 1 -- 172.21.2.104:0/3631722972 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6978007170 con 0x7f697c14f030 2021-07-11T03:59:09.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f696f7fe700 1 -- 172.21.2.104:0/3631722972 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69780072d0 con 0x7f697c14f030 2021-07-11T03:59:09.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f698443f700 1 -- 172.21.2.104:0/3631722972 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c0acef0 con 0x7f697c14f030 2021-07-11T03:59:09.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f698443f700 1 -- 172.21.2.104:0/3631722972 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c066650 con 0x7f697c14f030 2021-07-11T03:59:09.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f696f7fe700 1 -- 172.21.2.104:0/3631722972 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69780049e0 con 0x7f697c14f030 2021-07-11T03:59:09.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f696f7fe700 1 -- 172.21.2.104:0/3631722972 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f697801b070 con 0x7f697c14f030 2021-07-11T03:59:09.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f698443f700 1 -- 172.21.2.104:0/3631722972 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c057350 con 0x7f697c14f030 2021-07-11T03:59:09.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f696f7fe700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f695c053350 0x7f695c055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f696f7fe700 1 -- 172.21.2.104:0/3631722972 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f697806aaa0 con 0x7f697c14f030 2021-07-11T03:59:09.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.210+0000 7f69821db700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f695c053350 0x7f695c055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.214+0000 7f69821db700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f695c053350 0x7f695c055810 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f697000e000 tx=0x7f6970008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:09.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.214+0000 7f696f7fe700 1 -- 172.21.2.104:0/3631722972 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f69780420c0 con 0x7f697c14f030 2021-07-11T03:59:09.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.318+0000 7f698443f700 1 -- 172.21.2.104:0/3631722972 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f697c0aa6a0 con 0x7f697c14f030 2021-07-11T03:59:09.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f696f7fe700 1 -- 172.21.2.104:0/3631722972 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f6978019020 con 0x7f697c14f030 2021-07-11T03:59:09.328 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:09.328 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":29,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4532,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/2908869555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":2908869555},{"type":"v1","addr":"172.21.2.105:6835","nonce":2908869555}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":5949,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/1110128427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":1110128427},{"type":"v1","addr":"172.21.2.104:6833","nonce":1110128427}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":26},{"gid":6168,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/214455898","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":214455898},{"type":"v1","addr":"172.21.2.104:6835","nonce":214455898}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":29}],"filesystems":[]} 2021-07-11T03:59:09.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 -- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f695c053350 msgr2=0x7f695c055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f695c053350 0x7f695c055810 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f697000e000 tx=0x7f6970008040).stop 2021-07-11T03:59:09.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 -- 172.21.2.104:0/3631722972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f697c14f030 msgr2=0x7f697c065cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f697c14f030 0x7f697c065cd0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f6978001370 tx=0x7f697c140490).stop 2021-07-11T03:59:09.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 -- 172.21.2.104:0/3631722972 shutdown_connections 2021-07-11T03:59:09.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f697c0661c0 0x7f697c0adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:09.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f695c053350 0x7f695c055810 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f697c14f030 0x7f697c065cd0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 --2- 172.21.2.104:0/3631722972 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f697c0632f0 0x7f697c0636d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 -- 172.21.2.104:0/3631722972 >> 172.21.2.104:0/3631722972 conn(0x7f697c0b53e0 msgr2=0x7f697c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:09.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 -- 172.21.2.104:0/3631722972 shutdown_connections 2021-07-11T03:59:09.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.322+0000 7f698443f700 1 -- 172.21.2.104:0/3631722972 wait complete. 2021-07-11T03:59:09.332 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 29 2021-07-11T03:59:09.335 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T03:59:09.336 INFO:tasks.ceph.mds.a:Stopping old one... 2021-07-11T03:59:09.336 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T03:59:09.336 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:59:09.337 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T03:59:09.337 INFO:tasks.ceph.mds.c:Stopping old one... 2021-07-11T03:59:09.337 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T03:59:09.338 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:59:09.338 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T03:59:09.338 INFO:tasks.ceph.mds.b:Stopping old one... 2021-07-11T03:59:09.339 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T03:59:09.339 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T03:59:09.342 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T03:59:09.342 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T03:59:09.344 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T03:59:09.344 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T03:59:09.350 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T03:59:09.350 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T03:59:09.352 INFO:tasks.ceph.mds.b:Started 2021-07-11T03:59:09.353 DEBUG:teuthology.parallel:result is None 2021-07-11T03:59:09.386 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T03:59:09.387 INFO:tasks.ceph.mds.a:Started 2021-07-11T03:59:09.388 DEBUG:teuthology.parallel:result is None 2021-07-11T03:59:09.389 INFO:tasks.ceph.mds.c:Started 2021-07-11T03:59:09.389 DEBUG:teuthology.parallel:result is None 2021-07-11T03:59:09.390 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T03:59:09.425 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T03:59:09.426 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T03:59:09.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.438+0000 7f1284104700 1 Processor -- start 2021-07-11T03:59:09.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.438+0000 7f1284104700 1 -- start start 2021-07-11T03:59:09.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f127c0ac600 0x7f127c0ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f127c060d60 0x7f127c066c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f127c067130 0x7f127c1411e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f127c0619b0 con 0x7f127c0ac600 2021-07-11T03:59:09.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f127c0a9ef0 con 0x7f127c060d60 2021-07-11T03:59:09.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0b20b0 con 0x7f127c067130 2021-07-11T03:59:09.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1281ea0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f127c0ac600 0x7f127c0ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1281ea0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f127c0ac600 0x7f127c0ac9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46110/0 (socket says 172.21.2.104:46110) 2021-07-11T03:59:09.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1281ea0700 1 -- 172.21.2.104:0/3836325978 learned_addr learned my addr 172.21.2.104:0/3836325978 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:09.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f12826a1700 1 --2- 172.21.2.104:0/3836325978 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f127c067130 0x7f127c1411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f128169f700 1 --2- 172.21.2.104:0/3836325978 >> v2:172.21.2.105:3300/0 conn(0x7f127c060d60 0x7f127c066c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1281ea0700 1 -- 172.21.2.104:0/3836325978 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f127c067130 msgr2=0x7f127c1411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:09.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1281ea0700 1 --2- 172.21.2.104:0/3836325978 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f127c067130 0x7f127c1411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1281ea0700 1 -- 172.21.2.104:0/3836325978 >> v2:172.21.2.105:3300/0 conn(0x7f127c060d60 msgr2=0x7f127c066c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:09.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1281ea0700 1 --2- 172.21.2.104:0/3836325978 >> v2:172.21.2.105:3300/0 conn(0x7f127c060d60 0x7f127c066c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1281ea0700 1 -- 172.21.2.104:0/3836325978 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c0b2350 con 0x7f127c0ac600 2021-07-11T03:59:09.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f128169f700 1 --2- 172.21.2.104:0/3836325978 >> v2:172.21.2.105:3300/0 conn(0x7f127c060d60 0x7f127c066c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:59:09.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1281ea0700 1 --2- 172.21.2.104:0/3836325978 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f127c0ac600 0x7f127c0ac9e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f127c140f40 tx=0x7f1278006eb0).ready entity=mon.0 client_cookie=8710395d580d81f7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:09.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1280e9e700 1 -- 172.21.2.104:0/3836325978 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f127800d570 con 0x7f127c0ac600 2021-07-11T03:59:09.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1280e9e700 1 -- 172.21.2.104:0/3836325978 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1278000e90 con 0x7f127c0ac600 2021-07-11T03:59:09.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1280e9e700 1 -- 172.21.2.104:0/3836325978 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f127800d900 con 0x7f127c0ac600 2021-07-11T03:59:09.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 -- 172.21.2.104:0/3836325978 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f127c0ac600 msgr2=0x7f127c0ac9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 --2- 172.21.2.104:0/3836325978 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f127c0ac600 0x7f127c0ac9e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f127c140f40 tx=0x7f1278006eb0).stop 2021-07-11T03:59:09.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 -- 172.21.2.104:0/3836325978 shutdown_connections 2021-07-11T03:59:09.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 --2- 172.21.2.104:0/3836325978 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f127c067130 0x7f127c1411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 --2- 172.21.2.104:0/3836325978 >> v2:172.21.2.105:3300/0 conn(0x7f127c060d60 0x7f127c066c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 --2- 172.21.2.104:0/3836325978 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f127c0ac600 0x7f127c0ac9e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 -- 172.21.2.104:0/3836325978 >> 172.21.2.104:0/3836325978 conn(0x7f127c0b64d0 msgr2=0x7f127c0b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:09.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 -- 172.21.2.104:0/3836325978 shutdown_connections 2021-07-11T03:59:09.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 -- 172.21.2.104:0/3836325978 wait complete. 2021-07-11T03:59:09.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 Processor -- start 2021-07-11T03:59:09.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.442+0000 7f1284104700 1 -- start start 2021-07-11T03:59:09.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1284104700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f127c060d60 0x7f127c141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1284104700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f127c067130 0x7f127c143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1284104700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f127c0ac600 0x7f127c143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1284104700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c064a40 con 0x7f127c060d60 2021-07-11T03:59:09.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1284104700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0657d0 con 0x7f127c067130 2021-07-11T03:59:09.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1284104700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c066550 con 0x7f127c0ac600 2021-07-11T03:59:09.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1281ea0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f127c060d60 0x7f127c141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1281ea0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f127c060d60 0x7f127c141830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46116/0 (socket says 172.21.2.104:46116) 2021-07-11T03:59:09.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1281ea0700 1 -- 172.21.2.104:0/1303572781 learned_addr learned my addr 172.21.2.104:0/1303572781 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:09.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f128169f700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f127c067130 0x7f127c143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f12826a1700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f127c0ac600 0x7f127c143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1281ea0700 1 -- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f127c0ac600 msgr2=0x7f127c143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1281ea0700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f127c0ac600 0x7f127c143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1281ea0700 1 -- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f127c067130 msgr2=0x7f127c143810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:09.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1281ea0700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f127c067130 0x7f127c143810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:09.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1281ea0700 1 -- 172.21.2.104:0/1303572781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127800b1e0 con 0x7f127c060d60 2021-07-11T03:59:09.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f128169f700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f127c067130 0x7f127c143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:59:09.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1281ea0700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f127c060d60 0x7f127c141830 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f127c140f40 tx=0x7f1278015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:09.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f126effd700 1 -- 172.21.2.104:0/1303572781 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1278020070 con 0x7f127c060d60 2021-07-11T03:59:09.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f126effd700 1 -- 172.21.2.104:0/1303572781 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1278011760 con 0x7f127c060d60 2021-07-11T03:59:09.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1284104700 1 -- 172.21.2.104:0/1303572781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127c063950 con 0x7f127c060d60 2021-07-11T03:59:09.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f126effd700 1 -- 172.21.2.104:0/1303572781 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1278011ac0 con 0x7f127c060d60 2021-07-11T03:59:09.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1284104700 1 -- 172.21.2.104:0/1303572781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127c064300 con 0x7f127c060d60 2021-07-11T03:59:09.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f1284104700 1 -- 172.21.2.104:0/1303572781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127c057350 con 0x7f127c060d60 2021-07-11T03:59:09.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f126effd700 1 -- 172.21.2.104:0/1303572781 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f127800f070 con 0x7f127c060d60 2021-07-11T03:59:09.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f126effd700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f125c0531d0 0x7f125c055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:09.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f126effd700 1 -- 172.21.2.104:0/1303572781 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f127806c080 con 0x7f127c060d60 2021-07-11T03:59:09.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f128169f700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f125c0531d0 0x7f125c055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:09.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.446+0000 7f128169f700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f125c0531d0 0x7f125c055690 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f127000c000 tx=0x7f1270008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:09.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.450+0000 7f126effd700 1 -- 172.21.2.104:0/1303572781 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1278043750 con 0x7f127c060d60 2021-07-11T03:59:09.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:09.542+0000 7f1284104700 1 -- 172.21.2.104:0/1303572781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f127c0ae350 con 0x7f127c060d60 2021-07-11T03:59:10.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.458+0000 7f126effd700 1 -- 172.21.2.104:0/1303572781 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v72) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1278046d70 con 0x7f127c060d60 2021-07-11T03:59:10.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 -- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f125c0531d0 msgr2=0x7f125c055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f125c0531d0 0x7f125c055690 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f127000c000 tx=0x7f1270008040).stop 2021-07-11T03:59:10.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 -- 172.21.2.104:0/1303572781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f127c060d60 msgr2=0x7f127c141830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f127c060d60 0x7f127c141830 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f127c140f40 tx=0x7f1278015000).stop 2021-07-11T03:59:10.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 -- 172.21.2.104:0/1303572781 shutdown_connections 2021-07-11T03:59:10.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f127c0ac600 0x7f127c143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f127c067130 0x7f127c143810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f125c0531d0 0x7f125c055690 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 --2- 172.21.2.104:0/1303572781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f127c060d60 0x7f127c141830 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 -- 172.21.2.104:0/1303572781 >> 172.21.2.104:0/1303572781 conn(0x7f127c0b64d0 msgr2=0x7f127c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:10.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 -- 172.21.2.104:0/1303572781 shutdown_connections 2021-07-11T03:59:10.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.466+0000 7f1284104700 1 -- 172.21.2.104:0/1303572781 wait complete. 2021-07-11T03:59:10.475 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T03:59:10.485 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T03:59:10.485 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T03:59:10.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 Processor -- start 2021-07-11T03:59:10.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 -- start start 2021-07-11T03:59:10.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93b00b2580 0x7f93b014ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f93b00b0010 0x7f93b014b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f93b00afbd0 0x7f93b00a83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b00a70c0 con 0x7f93b00afbd0 2021-07-11T03:59:10.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b00a72f0 con 0x7f93b00b0010 2021-07-11T03:59:10.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b00a7590 con 0x7f93b00b2580 2021-07-11T03:59:10.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93af7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f93b00afbd0 0x7f93b00a83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93af7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f93b00afbd0 0x7f93b00a83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46130/0 (socket says 172.21.2.104:46130) 2021-07-11T03:59:10.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93af7fe700 1 -- 172.21.2.104:0/4249688511 learned_addr learned my addr 172.21.2.104:0/4249688511 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:10.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93af7fe700 1 -- 172.21.2.104:0/4249688511 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93b00b2580 msgr2=0x7f93b014ae40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:59:10.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93af7fe700 1 --2- 172.21.2.104:0/4249688511 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93b00b2580 0x7f93b014ae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93af7fe700 1 -- 172.21.2.104:0/4249688511 >> v2:172.21.2.105:3300/0 conn(0x7f93b00b0010 msgr2=0x7f93b014b330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:59:10.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93af7fe700 1 --2- 172.21.2.104:0/4249688511 >> v2:172.21.2.105:3300/0 conn(0x7f93b00b0010 0x7f93b014b330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93af7fe700 1 -- 172.21.2.104:0/4249688511 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b00a7830 con 0x7f93b00afbd0 2021-07-11T03:59:10.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93af7fe700 1 --2- 172.21.2.104:0/4249688511 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93b00afbd0 0x7f93b00a83b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f93a80032f0 tx=0x7f93a8009290).ready entity=mon.0 client_cookie=635c50faf62187b8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:10.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93adffb700 1 -- 172.21.2.104:0/4249688511 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93a800b870 con 0x7f93b00afbd0 2021-07-11T03:59:10.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93adffb700 1 -- 172.21.2.104:0/4249688511 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a800b9d0 con 0x7f93b00afbd0 2021-07-11T03:59:10.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93adffb700 1 -- 172.21.2.104:0/4249688511 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93a800bb80 con 0x7f93b00afbd0 2021-07-11T03:59:10.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 -- 172.21.2.104:0/4249688511 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93b00afbd0 msgr2=0x7f93b00a83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 --2- 172.21.2.104:0/4249688511 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93b00afbd0 0x7f93b00a83b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f93a80032f0 tx=0x7f93a8009290).stop 2021-07-11T03:59:10.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 -- 172.21.2.104:0/4249688511 shutdown_connections 2021-07-11T03:59:10.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 --2- 172.21.2.104:0/4249688511 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93b00afbd0 0x7f93b00a83b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 --2- 172.21.2.104:0/4249688511 >> v2:172.21.2.105:3300/0 conn(0x7f93b00b0010 0x7f93b014b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 --2- 172.21.2.104:0/4249688511 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93b00b2580 0x7f93b014ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 -- 172.21.2.104:0/4249688511 >> 172.21.2.104:0/4249688511 conn(0x7f93b0060180 msgr2=0x7f93b00b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:10.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 -- 172.21.2.104:0/4249688511 shutdown_connections 2021-07-11T03:59:10.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.534+0000 7f93affff700 1 -- 172.21.2.104:0/4249688511 wait complete. 2021-07-11T03:59:10.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93affff700 1 Processor -- start 2021-07-11T03:59:10.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93affff700 1 -- start start 2021-07-11T03:59:10.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93affff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93b00afbd0 0x7f93b00b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93affff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93b00b0010 0x7f93b014af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93affff700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93b00b2580 0x7f93b0159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93affff700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b014a330 con 0x7f93b00b0010 2021-07-11T03:59:10.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93affff700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b014a810 con 0x7f93b00b2580 2021-07-11T03:59:10.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93affff700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b014aab0 con 0x7f93b00afbd0 2021-07-11T03:59:10.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93ae7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93b00b0010 0x7f93b014af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93ae7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93b00b0010 0x7f93b014af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46136/0 (socket says 172.21.2.104:46136) 2021-07-11T03:59:10.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93ae7fc700 1 -- 172.21.2.104:0/2825809397 learned_addr learned my addr 172.21.2.104:0/2825809397 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:10.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93ae7fc700 1 -- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93b00afbd0 msgr2=0x7f93b00b3110 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:59:10.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93ae7fc700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93b00afbd0 0x7f93b00b3110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93ae7fc700 1 -- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93b00b2580 msgr2=0x7f93b0159980 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:59:10.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93ae7fc700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93b00b2580 0x7f93b0159980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93ae7fc700 1 -- 172.21.2.104:0/2825809397 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a8009530 con 0x7f93b00b0010 2021-07-11T03:59:10.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93ae7fc700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93b00b0010 0x7f93b014af80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f93a0002fb0 tx=0x7f93a0006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:10.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93a7fff700 1 -- 172.21.2.104:0/2825809397 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93a0013460 con 0x7f93b00b0010 2021-07-11T03:59:10.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93a7fff700 1 -- 172.21.2.104:0/2825809397 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a00135c0 con 0x7f93b00b0010 2021-07-11T03:59:10.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93a7fff700 1 -- 172.21.2.104:0/2825809397 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93a0013920 con 0x7f93b00b0010 2021-07-11T03:59:10.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93affff700 1 -- 172.21.2.104:0/2825809397 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b0149bf0 con 0x7f93b00b0010 2021-07-11T03:59:10.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93affff700 1 -- 172.21.2.104:0/2825809397 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b014a500 con 0x7f93b00b0010 2021-07-11T03:59:10.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93affff700 1 -- 172.21.2.104:0/2825809397 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b0057350 con 0x7f93b00b0010 2021-07-11T03:59:10.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93a7fff700 1 -- 172.21.2.104:0/2825809397 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f93a000f070 con 0x7f93b00b0010 2021-07-11T03:59:10.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93a7fff700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9394053260 0x7f9394055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93a7fff700 1 -- 172.21.2.104:0/2825809397 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f93a006ac90 con 0x7f93b00b0010 2021-07-11T03:59:10.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.538+0000 7f93aeffd700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9394053260 0x7f9394055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.542+0000 7f93aeffd700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9394053260 0x7f9394055720 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f9398000ed0 tx=0x7f93980068b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:10.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.542+0000 7f93a7fff700 1 -- 172.21.2.104:0/2825809397 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f93a00421f0 con 0x7f93b00b0010 2021-07-11T03:59:10.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.650+0000 7f93affff700 1 -- 172.21.2.104:0/2825809397 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f93b00698e0 con 0x7f93b00b0010 2021-07-11T03:59:10.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93a7fff700 1 -- 172.21.2.104:0/2825809397 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f93a0045810 con 0x7f93b00b0010 2021-07-11T03:59:10.659 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:10.660 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T03:59:10.660 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T03:59:10.660 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.660 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T03:59:10.661 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T03:59:10.661 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.661 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:59:10.661 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.661 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.662 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.662 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.663 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.663 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.663 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T03:59:10.663 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T03:59:10.664 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.664 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:59:10.664 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.664 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.665 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.665 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.665 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.665 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.665 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T03:59:10.666 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T03:59:10.666 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.666 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:59:10.666 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.667 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.667 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.667 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.667 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.667 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.668 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T03:59:10.668 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T03:59:10.668 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.668 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.669 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.669 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.669 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.669 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.669 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.670 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T03:59:10.670 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T03:59:10.670 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.670 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.671 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.671 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.671 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.671 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.671 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.672 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T03:59:10.672 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T03:59:10.672 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.672 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.673 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.673 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.674 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.674 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.674 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.674 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T03:59:10.674 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T03:59:10.675 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.675 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.675 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.675 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.676 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.676 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.676 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.676 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T03:59:10.676 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T03:59:10.677 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.677 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.677 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.677 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.678 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.678 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.678 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.678 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T03:59:10.678 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T03:59:10.679 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.679 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.679 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.679 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.680 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.680 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.680 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.680 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T03:59:10.680 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T03:59:10.681 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.681 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.681 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.682 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.682 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.682 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.682 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.683 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T03:59:10.683 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T03:59:10.684 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.684 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.684 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.685 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.685 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.685 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.685 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.686 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T03:59:10.686 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T03:59:10.686 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.686 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:59:10.686 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T03:59:10.687 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T03:59:10.687 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.687 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.687 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.688 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T03:59:10.688 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T03:59:10.688 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.688 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T03:59:10.688 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T03:59:10.689 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T03:59:10.689 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.689 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.689 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.690 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T03:59:10.690 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T03:59:10.690 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.691 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:59:10.691 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T03:59:10.691 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T03:59:10.691 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.692 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.692 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.692 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.692 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T03:59:10.692 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T03:59:10.693 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.693 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:59:10.693 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T03:59:10.693 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.695 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.695 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:10.695 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:10.695 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T03:59:10.695 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T03:59:10.696 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T03:59:10.696 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T03:59:10.696 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T03:59:10.696 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T03:59:10.697 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.697 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:10.697 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T03:59:10.697 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T03:59:10.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 -- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9394053260 msgr2=0x7f9394055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9394053260 0x7f9394055720 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f9398000ed0 tx=0x7f93980068b0).stop 2021-07-11T03:59:10.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 -- 172.21.2.104:0/2825809397 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93b00b0010 msgr2=0x7f93b014af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93b00b0010 0x7f93b014af80 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f93a0002fb0 tx=0x7f93a0006e10).stop 2021-07-11T03:59:10.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 -- 172.21.2.104:0/2825809397 shutdown_connections 2021-07-11T03:59:10.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93b00b2580 0x7f93b0159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9394053260 0x7f9394055720 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93b00b0010 0x7f93b014af80 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 --2- 172.21.2.104:0/2825809397 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93b00afbd0 0x7f93b00b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 -- 172.21.2.104:0/2825809397 >> 172.21.2.104:0/2825809397 conn(0x7f93b0060180 msgr2=0x7f93b005bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:10.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 -- 172.21.2.104:0/2825809397 shutdown_connections 2021-07-11T03:59:10.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.654+0000 7f93affff700 1 -- 172.21.2.104:0/2825809397 wait complete. 2021-07-11T03:59:10.702 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:59:10.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 Processor -- start 2021-07-11T03:59:10.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 -- start start 2021-07-11T03:59:10.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f97580b1970 0x7f975805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97580a7a90 con 0x7f97580a76b0 2021-07-11T03:59:10.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9758147620 con 0x7f9758067180 2021-07-11T03:59:10.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97581478c0 con 0x7f97580b1970 2021-07-11T03:59:10.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975659c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f97580a76b0 0x7f9758066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42910/0 (socket says 172.21.2.104:42910) 2021-07-11T03:59:10.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975659c700 1 -- 172.21.2.104:0/897835371 learned_addr learned my addr 172.21.2.104:0/897835371 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:10.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968718689 0 0) 0x7f97580a7a90 con 0x7f97580a76b0 2021-07-11T03:59:10.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9738003260 con 0x7f97580a76b0 2021-07-11T03:59:10.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f9756d9d700 1 --2- 172.21.2.104:0/897835371 >> v2:172.21.2.105:3300/0 conn(0x7f97580b1970 0x7f975805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2611976959 0 0) 0x7f9738003260 con 0x7f97580a76b0 2021-07-11T03:59:10.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 >> v2:172.21.2.105:3300/0 conn(0x7f97580b1970 msgr2=0x7f975805fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:10.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 --2- 172.21.2.104:0/897835371 >> v2:172.21.2.105:3300/0 conn(0x7f97580b1970 0x7f975805fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 >> v1:172.21.2.105:6789/0 conn(0x7f9758067180 legacy=0x7f975805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9758147b60 con 0x7f97580a76b0 2021-07-11T03:59:10.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f9740001660 con 0x7f97580a76b0 2021-07-11T03:59:10.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 >> v1:172.21.2.104:6789/0 conn(0x7f97580a76b0 legacy=0x7f9758066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 --2- 172.21.2.104:0/897835371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 0x7f9738003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 --2- 172.21.2.104:0/897835371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9738004500 0x7f9738004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 --2- 172.21.2.104:0/897835371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9738004f90 0x7f973800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97581478c0 con 0x7f97380037a0 2021-07-11T03:59:10.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973800be00 con 0x7f9738004500 2021-07-11T03:59:10.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973800bfc0 con 0x7f9738004f90 2021-07-11T03:59:10.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975659c700 1 --2- 172.21.2.104:0/897835371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 0x7f9738003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975659c700 1 -- 172.21.2.104:0/897835371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9738004f90 msgr2=0x7f973800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:10.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975659c700 1 --2- 172.21.2.104:0/897835371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9738004f90 0x7f973800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975659c700 1 -- 172.21.2.104:0/897835371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9738004500 msgr2=0x7f9738004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:10.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975659c700 1 --2- 172.21.2.104:0/897835371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9738004500 0x7f9738004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f9756d9d700 1 -- 172.21.2.104:0/897835371 reap_dead start 2021-07-11T03:59:10.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975659c700 1 -- 172.21.2.104:0/897835371 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9738003260 con 0x7f97380037a0 2021-07-11T03:59:10.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975659c700 1 --2- 172.21.2.104:0/897835371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 0x7f9738003bc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f9740001370 tx=0x7f97580a8c80).ready entity=mon.0 client_cookie=437e3a4d1a538d16 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:10.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f97400070d0 con 0x7f97380037a0 2021-07-11T03:59:10.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9740007230 con 0x7f97380037a0 2021-07-11T03:59:10.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975559a700 1 -- 172.21.2.104:0/897835371 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f974000c8c0 con 0x7f97380037a0 2021-07-11T03:59:10.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 -- 172.21.2.104:0/897835371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 msgr2=0x7f9738003bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 --2- 172.21.2.104:0/897835371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 0x7f9738003bc0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f9740001370 tx=0x7f97580a8c80).stop 2021-07-11T03:59:10.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 -- 172.21.2.104:0/897835371 shutdown_connections 2021-07-11T03:59:10.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 --2- 172.21.2.104:0/897835371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 0x7f9738003bc0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 -- 172.21.2.104:0/897835371 >> 172.21.2.104:0/897835371 conn(0x7f97580b53e0 msgr2=0x7f97580b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:10.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 -- 172.21.2.104:0/897835371 shutdown_connections 2021-07-11T03:59:10.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.750+0000 7f975ca4b700 1 -- 172.21.2.104:0/897835371 wait complete. 2021-07-11T03:59:10.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975ca4b700 1 Processor -- start 2021-07-11T03:59:10.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975ca4b700 1 -- start start 2021-07-11T03:59:10.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975ca4b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 0x7f97580aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975ca4b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97580ae120 0x7f97580abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975ca4b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97580ac520 0x7f97580ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975ca4b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97580ae580 con 0x7f97380037a0 2021-07-11T03:59:10.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975ca4b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97580aef30 con 0x7f97580ae120 2021-07-11T03:59:10.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975ca4b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97580af1d0 con 0x7f97580ac520 2021-07-11T03:59:10.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 0x7f97580aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 0x7f97580aaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46156/0 (socket says 172.21.2.104:46156) 2021-07-11T03:59:10.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975659c700 1 -- 172.21.2.104:0/534552186 learned_addr learned my addr 172.21.2.104:0/534552186 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:10.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975659c700 1 -- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97580ac520 msgr2=0x7f97580ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975659c700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97580ac520 0x7f97580ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975659c700 1 -- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97580ae120 msgr2=0x7f97580abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975659c700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97580ae120 0x7f97580abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975659c700 1 -- 172.21.2.104:0/534552186 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580af470 con 0x7f97380037a0 2021-07-11T03:59:10.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975659c700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 0x7f97580aaaf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f9740001370 tx=0x7f9740013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:10.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f974f7fe700 1 -- 172.21.2.104:0/534552186 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9740017030 con 0x7f97380037a0 2021-07-11T03:59:10.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f974f7fe700 1 -- 172.21.2.104:0/534552186 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9740004520 con 0x7f97380037a0 2021-07-11T03:59:10.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975ca4b700 1 -- 172.21.2.104:0/534552186 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97580af710 con 0x7f97380037a0 2021-07-11T03:59:10.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f974f7fe700 1 -- 172.21.2.104:0/534552186 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f974001a040 con 0x7f97380037a0 2021-07-11T03:59:10.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975ca4b700 1 -- 172.21.2.104:0/534552186 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97580ac2b0 con 0x7f97380037a0 2021-07-11T03:59:10.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f975ca4b700 1 -- 172.21.2.104:0/534552186 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9758057350 con 0x7f97380037a0 2021-07-11T03:59:10.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f974f7fe700 1 -- 172.21.2.104:0/534552186 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9740010070 con 0x7f97380037a0 2021-07-11T03:59:10.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f974f7fe700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f973c053210 0x7f973c0556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f974f7fe700 1 -- 172.21.2.104:0/534552186 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f974006bc80 con 0x7f97380037a0 2021-07-11T03:59:10.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.754+0000 7f9755d9b700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f973c053210 0x7f973c0556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.758+0000 7f9755d9b700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f973c053210 0x7f973c0556d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f975806a180 tx=0x7f9748007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:10.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.758+0000 7f974f7fe700 1 -- 172.21.2.104:0/534552186 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f97400432f0 con 0x7f97380037a0 2021-07-11T03:59:10.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 -- 172.21.2.104:0/534552186 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97580698e0 con 0x7f97380037a0 2021-07-11T03:59:10.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f974f7fe700 1 -- 172.21.2.104:0/534552186 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+13182 (secure 0 0 0) 0x7f9740046910 con 0x7f97380037a0 2021-07-11T03:59:10.852 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:10.852 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":72,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:59:10.453649+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:59:10.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 -- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f973c053210 msgr2=0x7f973c0556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f973c053210 0x7f973c0556d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f975806a180 tx=0x7f9748007810).stop 2021-07-11T03:59:10.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 -- 172.21.2.104:0/534552186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 msgr2=0x7f97580aaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 0x7f97580aaaf0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f9740001370 tx=0x7f9740013000).stop 2021-07-11T03:59:10.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 -- 172.21.2.104:0/534552186 shutdown_connections 2021-07-11T03:59:10.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97580ac520 0x7f97580ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97580ae120 0x7f97580abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:10.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f973c053210 0x7f973c0556d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 --2- 172.21.2.104:0/534552186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f97380037a0 0x7f97580aaaf0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 -- 172.21.2.104:0/534552186 >> 172.21.2.104:0/534552186 conn(0x7f97580b53e0 msgr2=0x7f9758148c00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:10.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 -- 172.21.2.104:0/534552186 shutdown_connections 2021-07-11T03:59:10.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.846+0000 7f975ca4b700 1 -- 172.21.2.104:0/534552186 wait complete. 2021-07-11T03:59:10.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T03:59:10.860 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:59:10.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.906+0000 7fd5cbf10700 1 Processor -- start 2021-07-11T03:59:10.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.906+0000 7fd5cbf10700 1 -- start start 2021-07-11T03:59:10.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.906+0000 7fd5cbf10700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd5c40b12d0 0x7fd5c40affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.906+0000 7fd5cbf10700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b22a0 0x7fd5c4141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.906+0000 7fd5cbf10700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c40a77d0 con 0x7fd5c40b1ec0 2021-07-11T03:59:10.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.906+0000 7fd5cbf10700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c40b1740 con 0x7fd5c40b12d0 2021-07-11T03:59:10.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.906+0000 7fd5cbf10700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c40b0650 con 0x7fd5c40b22a0 2021-07-11T03:59:10.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5c94ab700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd5c40b12d0 0x7fd5c40affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5ca4ad700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b22a0 0x7fd5c4141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5c9cac700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fd5c40b1ec0 0x7fd5c40a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:42300/0 (socket says 172.21.2.104:42300) 2021-07-11T03:59:10.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5c9cac700 1 -- 172.21.2.104:0/504380746 learned_addr learned my addr 172.21.2.104:0/504380746 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:10.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5ca4ad700 1 -- 172.21.2.104:0/504380746 >> v2:172.21.2.105:3300/0 conn(0x7fd5c40b12d0 msgr2=0x7fd5c40affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:10.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5ca4ad700 1 --2- 172.21.2.104:0/504380746 >> v2:172.21.2.105:3300/0 conn(0x7fd5c40b12d0 0x7fd5c40affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5ca4ad700 1 -- 172.21.2.104:0/504380746 >> v1:172.21.2.105:6789/0 conn(0x7fd5c40b1ec0 legacy=0x7fd5c40a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5ca4ad700 1 -- 172.21.2.104:0/504380746 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c40b08f0 con 0x7fd5c40b22a0 2021-07-11T03:59:10.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5ca4ad700 1 --2- 172.21.2.104:0/504380746 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b22a0 0x7fd5c4141250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd5bc0032f0 tx=0x7fd5bc0091d0).ready entity=mon.2 client_cookie=50bab7d6e20d828e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:10.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5c8caa700 1 -- 172.21.2.104:0/504380746 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5bc00b6c0 con 0x7fd5c40b22a0 2021-07-11T03:59:10.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5c8caa700 1 -- 172.21.2.104:0/504380746 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc00b820 con 0x7fd5c40b22a0 2021-07-11T03:59:10.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5c8caa700 1 -- 172.21.2.104:0/504380746 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5bc00b990 con 0x7fd5c40b22a0 2021-07-11T03:59:10.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5cbf10700 1 -- 172.21.2.104:0/504380746 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b22a0 msgr2=0x7fd5c4141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5cbf10700 1 --2- 172.21.2.104:0/504380746 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b22a0 0x7fd5c4141250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fd5bc0032f0 tx=0x7fd5bc0091d0).stop 2021-07-11T03:59:10.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5cbf10700 1 -- 172.21.2.104:0/504380746 shutdown_connections 2021-07-11T03:59:10.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5cbf10700 1 --2- 172.21.2.104:0/504380746 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b22a0 0x7fd5c4141250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5cbf10700 1 --2- 172.21.2.104:0/504380746 >> v2:172.21.2.105:3300/0 conn(0x7fd5c40b12d0 0x7fd5c40affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5cbf10700 1 -- 172.21.2.104:0/504380746 >> 172.21.2.104:0/504380746 conn(0x7fd5c405f6b0 msgr2=0x7fd5c405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:10.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5cbf10700 1 -- 172.21.2.104:0/504380746 shutdown_connections 2021-07-11T03:59:10.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5cbf10700 1 -- 172.21.2.104:0/504380746 wait complete. 2021-07-11T03:59:10.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5cbf10700 1 Processor -- start 2021-07-11T03:59:10.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.910+0000 7fd5cbf10700 1 -- start start 2021-07-11T03:59:10.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5cbf10700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd5c40b12d0 0x7fd5c4149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5cbf10700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b1ec0 0x7fd5c40afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5cbf10700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5c40b22a0 0x7fd5c413c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5cbf10700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c40af9c0 con 0x7fd5c40b22a0 2021-07-11T03:59:10.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5cbf10700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c40ada30 con 0x7fd5c40b12d0 2021-07-11T03:59:10.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5cbf10700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c40adcd0 con 0x7fd5c40b1ec0 2021-07-11T03:59:10.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5ca4ad700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5c40b22a0 0x7fd5c413c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5ca4ad700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5c40b22a0 0x7fd5c413c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46172/0 (socket says 172.21.2.104:46172) 2021-07-11T03:59:10.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5ca4ad700 1 -- 172.21.2.104:0/3294772487 learned_addr learned my addr 172.21.2.104:0/3294772487 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:10.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5ca4ad700 1 -- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b1ec0 msgr2=0x7fd5c40afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5c9cac700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd5c40b12d0 0x7fd5c4149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5c94ab700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b1ec0 0x7fd5c40afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5ca4ad700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b1ec0 0x7fd5c40afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5ca4ad700 1 -- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd5c40b12d0 msgr2=0x7fd5c4149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:10.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5ca4ad700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd5c40b12d0 0x7fd5c4149cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:10.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5ca4ad700 1 -- 172.21.2.104:0/3294772487 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc009470 con 0x7fd5c40b22a0 2021-07-11T03:59:10.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5c94ab700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b1ec0 0x7fd5c40afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:59:10.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5ca4ad700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5c40b22a0 0x7fd5c413c840 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fd5bc0032f0 tx=0x7fd5bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:10.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5b6ffd700 1 -- 172.21.2.104:0/3294772487 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5bc006ea0 con 0x7fd5c40b22a0 2021-07-11T03:59:10.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5b6ffd700 1 -- 172.21.2.104:0/3294772487 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc011e90 con 0x7fd5c40b22a0 2021-07-11T03:59:10.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5cbf10700 1 -- 172.21.2.104:0/3294772487 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c40adf70 con 0x7fd5c40b22a0 2021-07-11T03:59:10.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5b6ffd700 1 -- 172.21.2.104:0/3294772487 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5bc021670 con 0x7fd5c40b22a0 2021-07-11T03:59:10.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5cbf10700 1 -- 172.21.2.104:0/3294772487 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c40b02e0 con 0x7fd5c40b22a0 2021-07-11T03:59:10.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5cbf10700 1 -- 172.21.2.104:0/3294772487 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c40aed50 con 0x7fd5c40b22a0 2021-07-11T03:59:10.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5b6ffd700 1 -- 172.21.2.104:0/3294772487 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd5bc00f070 con 0x7fd5c40b22a0 2021-07-11T03:59:10.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5b6ffd700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5a40532d0 0x7fd5a4055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:10.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5b6ffd700 1 -- 172.21.2.104:0/3294772487 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd5bc06c550 con 0x7fd5c40b22a0 2021-07-11T03:59:10.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.914+0000 7fd5c9cac700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5a40532d0 0x7fd5a4055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:10.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.918+0000 7fd5c9cac700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5a40532d0 0x7fd5a4055790 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fd5c0008000 tx=0x7fd5c0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:10.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.918+0000 7fd5b6ffd700 1 -- 172.21.2.104:0/3294772487 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd5c40aed50 con 0x7fd5c40b22a0 2021-07-11T03:59:11.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.998+0000 7fd5cbf10700 1 -- 172.21.2.104:0/3294772487 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd5c40aed50 con 0x7fd5c40b22a0 2021-07-11T03:59:11.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:10.998+0000 7fd5b6ffd700 1 -- 172.21.2.104:0/3294772487 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v72) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd5c40aed50 con 0x7fd5c40b22a0 2021-07-11T03:59:11.007 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:59:11.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 -- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5a40532d0 msgr2=0x7fd5a4055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5a40532d0 0x7fd5a4055790 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fd5c0008000 tx=0x7fd5c0009040).stop 2021-07-11T03:59:11.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 -- 172.21.2.104:0/3294772487 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5c40b22a0 msgr2=0x7fd5c413c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5c40b22a0 0x7fd5c413c840 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fd5bc0032f0 tx=0x7fd5bc016000).stop 2021-07-11T03:59:11.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 -- 172.21.2.104:0/3294772487 shutdown_connections 2021-07-11T03:59:11.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5a40532d0 0x7fd5a4055790 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5c40b22a0 0x7fd5c413c840 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5c40b1ec0 0x7fd5c40afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 --2- 172.21.2.104:0/3294772487 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd5c40b12d0 0x7fd5c4149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 -- 172.21.2.104:0/3294772487 >> 172.21.2.104:0/3294772487 conn(0x7fd5c405f6b0 msgr2=0x7fd5c40aa8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:11.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 -- 172.21.2.104:0/3294772487 shutdown_connections 2021-07-11T03:59:11.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.002+0000 7fd5cbf10700 1 -- 172.21.2.104:0/3294772487 wait complete. 2021-07-11T03:59:11.013 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:59:11.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf2404a700 1 Processor -- start 2021-07-11T03:59:11.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf2404a700 1 -- start start 2021-07-11T03:59:11.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf2404a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcf1c0b3710 0x7fcf1c0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf2404a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcf1c0a9c10 0x7fcf1c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf2404a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1c143920 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf2404a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1c143bc0 con 0x7fcf1c0a9c10 2021-07-11T03:59:11.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf2404a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1c143e60 con 0x7fcf1c0b3710 2021-07-11T03:59:11.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf21de6700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fcf1c0b05b0 0x7fcf1c13d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42944/0 (socket says 172.21.2.104:42944) 2021-07-11T03:59:11.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf21de6700 1 -- 172.21.2.104:0/487254103 learned_addr learned my addr 172.21.2.104:0/487254103 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:11.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf225e7700 1 --2- 172.21.2.104:0/487254103 >> v2:172.21.2.104:3300/0 conn(0x7fcf1c0a9c10 0x7fcf1c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf225e7700 1 -- 172.21.2.104:0/487254103 >> v2:172.21.2.105:3300/0 conn(0x7fcf1c0b3710 msgr2=0x7fcf1c0b0990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:59:11.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf225e7700 1 --2- 172.21.2.104:0/487254103 >> v2:172.21.2.105:3300/0 conn(0x7fcf1c0b3710 0x7fcf1c0b0990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:11.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf225e7700 1 -- 172.21.2.104:0/487254103 >> v1:172.21.2.104:6789/0 conn(0x7fcf1c0b05b0 legacy=0x7fcf1c13d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf225e7700 1 -- 172.21.2.104:0/487254103 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c144100 con 0x7fcf1c0a9c10 2021-07-11T03:59:11.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf225e7700 1 --2- 172.21.2.104:0/487254103 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf1c0a9c10 0x7fcf1c0b10b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7fcf140032f0 tx=0x7fcf140091d0).ready entity=mon.0 client_cookie=71cc638bfc072771 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:11.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf20de4700 1 -- 172.21.2.104:0/487254103 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf14009ea0 con 0x7fcf1c0a9c10 2021-07-11T03:59:11.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf20de4700 1 -- 172.21.2.104:0/487254103 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1400b870 con 0x7fcf1c0a9c10 2021-07-11T03:59:11.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.062+0000 7fcf20de4700 1 -- 172.21.2.104:0/487254103 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf1400ba20 con 0x7fcf1c0a9c10 2021-07-11T03:59:11.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- 172.21.2.104:0/487254103 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf1c0a9c10 msgr2=0x7fcf1c0b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 --2- 172.21.2.104:0/487254103 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf1c0a9c10 0x7fcf1c0b10b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7fcf140032f0 tx=0x7fcf140091d0).stop 2021-07-11T03:59:11.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- 172.21.2.104:0/487254103 shutdown_connections 2021-07-11T03:59:11.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 --2- 172.21.2.104:0/487254103 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf1c0a9c10 0x7fcf1c0b10b0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 --2- 172.21.2.104:0/487254103 >> v2:172.21.2.105:3300/0 conn(0x7fcf1c0b3710 0x7fcf1c0b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:11.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- 172.21.2.104:0/487254103 >> 172.21.2.104:0/487254103 conn(0x7fcf1c0607c0 msgr2=0x7fcf1c060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:11.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- 172.21.2.104:0/487254103 shutdown_connections 2021-07-11T03:59:11.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- 172.21.2.104:0/487254103 wait complete. 2021-07-11T03:59:11.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 Processor -- start 2021-07-11T03:59:11.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- start start 2021-07-11T03:59:11.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf1c0a9c10 0x7fcf1c13e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf1c0b05b0 0x7fcf1c137fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf1c0b3710 0x7fcf1c1384d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c1389c0 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c1356a0 con 0x7fcf1c0a9c10 2021-07-11T03:59:11.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c153330 con 0x7fcf1c0b3710 2021-07-11T03:59:11.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf215e5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf1c0b05b0 0x7fcf1c137fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf215e5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf1c0b05b0 0x7fcf1c137fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46184/0 (socket says 172.21.2.104:46184) 2021-07-11T03:59:11.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf215e5700 1 -- 172.21.2.104:0/1074945282 learned_addr learned my addr 172.21.2.104:0/1074945282 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:11.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf225e7700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf1c0b3710 0x7fcf1c1384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf21de6700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf1c0a9c10 0x7fcf1c13e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf215e5700 1 -- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf1c0b3710 msgr2=0x7fcf1c1384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf215e5700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf1c0b3710 0x7fcf1c1384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf215e5700 1 -- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf1c0a9c10 msgr2=0x7fcf1c13e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf215e5700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf1c0a9c10 0x7fcf1c13e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf215e5700 1 -- 172.21.2.104:0/1074945282 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf14009470 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf21de6700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf1c0a9c10 0x7fcf1c13e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:59:11.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf215e5700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf1c0b05b0 0x7fcf1c137fe0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7fcf18002ef0 tx=0x7fcf18006d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:11.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf0effd700 1 -- 172.21.2.104:0/1074945282 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf18012410 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf0effd700 1 -- 172.21.2.104:0/1074945282 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf18012570 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf0effd700 1 -- 172.21.2.104:0/1074945282 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf180128d0 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- 172.21.2.104:0/1074945282 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c1535d0 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- 172.21.2.104:0/1074945282 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c1526c0 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf2404a700 1 -- 172.21.2.104:0/1074945282 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1c057350 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf0effd700 1 -- 172.21.2.104:0/1074945282 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcf1800f030 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf0effd700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcefc053330 0x7fcefc0557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.066+0000 7fcf0effd700 1 -- 172.21.2.104:0/1074945282 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcf1806baf0 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.070+0000 7fcf21de6700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcefc053330 0x7fcefc0557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.070+0000 7fcf21de6700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcefc053330 0x7fcefc0557f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fcf1000b000 tx=0x7fcf10009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:11.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.070+0000 7fcf0effd700 1 -- 172.21.2.104:0/1074945282 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcf180423a0 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 -- 172.21.2.104:0/1074945282 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcf1c044650 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf0effd700 1 -- 172.21.2.104:0/1074945282 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v72) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcf180459c0 con 0x7fcf1c0b05b0 2021-07-11T03:59:11.161 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:59:11.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 -- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcefc053330 msgr2=0x7fcefc0557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcefc053330 0x7fcefc0557f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fcf1000b000 tx=0x7fcf10009040).stop 2021-07-11T03:59:11.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 -- 172.21.2.104:0/1074945282 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf1c0b05b0 msgr2=0x7fcf1c137fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf1c0b05b0 0x7fcf1c137fe0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7fcf18002ef0 tx=0x7fcf18006d40).stop 2021-07-11T03:59:11.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 -- 172.21.2.104:0/1074945282 shutdown_connections 2021-07-11T03:59:11.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf1c0b3710 0x7fcf1c1384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcefc053330 0x7fcefc0557f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf1c0b05b0 0x7fcf1c137fe0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 --2- 172.21.2.104:0/1074945282 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf1c0a9c10 0x7fcf1c13e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 -- 172.21.2.104:0/1074945282 >> 172.21.2.104:0/1074945282 conn(0x7fcf1c0607c0 msgr2=0x7fcf1c0ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:11.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 -- 172.21.2.104:0/1074945282 shutdown_connections 2021-07-11T03:59:11.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.154+0000 7fcf2404a700 1 -- 172.21.2.104:0/1074945282 wait complete. 2021-07-11T03:59:11.168 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T03:59:11.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc14ab93700 1 Processor -- start 2021-07-11T03:59:11.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc14ab93700 1 -- start start 2021-07-11T03:59:11.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc14ab93700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc1440b0fd0 0x7fc144144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc14ab93700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1440b0350 con 0x7fc1441483a0 2021-07-11T03:59:11.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc14ab93700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1440b05f0 con 0x7fc1440b1db0 2021-07-11T03:59:11.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc14ab93700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1440a90c0 con 0x7fc1440b0fd0 2021-07-11T03:59:11.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc143fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc1441483a0 0x7fc1440a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42960/0 (socket says 172.21.2.104:42960) 2021-07-11T03:59:11.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc143fff700 1 -- 172.21.2.104:0/1420516991 learned_addr learned my addr 172.21.2.104:0/1420516991 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:11.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1660789503 0 0) 0x7fc1440b0350 con 0x7fc1441483a0 2021-07-11T03:59:11.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc124003260 con 0x7fc1441483a0 2021-07-11T03:59:11.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc149130700 1 --2- 172.21.2.104:0/1420516991 >> v2:172.21.2.105:3300/0 conn(0x7fc1440b0fd0 0x7fc144144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449139899 0 0) 0x7fc124003260 con 0x7fc1441483a0 2021-07-11T03:59:11.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 >> v2:172.21.2.105:3300/0 conn(0x7fc1440b0fd0 msgr2=0x7fc144144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:11.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 --2- 172.21.2.104:0/1420516991 >> v2:172.21.2.105:3300/0 conn(0x7fc1440b0fd0 0x7fc144144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 >> v1:172.21.2.105:6789/0 conn(0x7fc1440b1db0 legacy=0x7fc1440b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:11.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1440b0890 con 0x7fc1441483a0 2021-07-11T03:59:11.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fc134001660 con 0x7fc1441483a0 2021-07-11T03:59:11.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 >> v1:172.21.2.104:6789/0 conn(0x7fc1441483a0 legacy=0x7fc1440a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 --2- 172.21.2.104:0/1420516991 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc124003820 0x7fc124003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 --2- 172.21.2.104:0/1420516991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc124004500 0x7fc124004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 --2- 172.21.2.104:0/1420516991 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc124004f70 0x7fc12400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12400be20 con 0x7fc124004500 2021-07-11T03:59:11.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12400bf90 con 0x7fc124004f70 2021-07-11T03:59:11.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12400c150 con 0x7fc124003820 2021-07-11T03:59:11.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc143fff700 1 --2- 172.21.2.104:0/1420516991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc124004500 0x7fc124004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc143fff700 1 -- 172.21.2.104:0/1420516991 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc124003820 msgr2=0x7fc124003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:11.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc143fff700 1 --2- 172.21.2.104:0/1420516991 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc124003820 0x7fc124003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:11.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc143fff700 1 -- 172.21.2.104:0/1420516991 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc124004f70 msgr2=0x7fc12400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:11.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc143fff700 1 --2- 172.21.2.104:0/1420516991 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc124004f70 0x7fc12400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:11.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc143fff700 1 -- 172.21.2.104:0/1420516991 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc124003260 con 0x7fc124004500 2021-07-11T03:59:11.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc149130700 1 -- 172.21.2.104:0/1420516991 reap_dead start 2021-07-11T03:59:11.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc143fff700 1 --2- 172.21.2.104:0/1420516991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc124004500 0x7fc124004980 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7fc1440b3170 tx=0x7fc1440b0780).ready entity=mon.0 client_cookie=e4396db1dbbf460b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:11.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1340070d0 con 0x7fc124004500 2021-07-11T03:59:11.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc134007230 con 0x7fc124004500 2021-07-11T03:59:11.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc1437fe700 1 -- 172.21.2.104:0/1420516991 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc13400c8c0 con 0x7fc124004500 2021-07-11T03:59:11.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc14ab93700 1 -- 172.21.2.104:0/1420516991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc124004500 msgr2=0x7fc124004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc14ab93700 1 --2- 172.21.2.104:0/1420516991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc124004500 0x7fc124004980 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7fc1440b3170 tx=0x7fc1440b0780).stop 2021-07-11T03:59:11.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc14ab93700 1 -- 172.21.2.104:0/1420516991 shutdown_connections 2021-07-11T03:59:11.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc14ab93700 1 --2- 172.21.2.104:0/1420516991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc124004500 0x7fc124004980 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.214+0000 7fc14ab93700 1 -- 172.21.2.104:0/1420516991 >> 172.21.2.104:0/1420516991 conn(0x7fc14405f6b0 msgr2=0x7fc14405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:11.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 -- 172.21.2.104:0/1420516991 shutdown_connections 2021-07-11T03:59:11.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 -- 172.21.2.104:0/1420516991 wait complete. 2021-07-11T03:59:11.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 Processor -- start 2021-07-11T03:59:11.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 -- start start 2021-07-11T03:59:11.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc124004500 0x7fc1440ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc144156b00 0x7fc144156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc144148d00 0x7fc144149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc144156640 con 0x7fc144156b00 2021-07-11T03:59:11.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc144141680 con 0x7fc144148d00 2021-07-11T03:59:11.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14414daa0 con 0x7fc124004500 2021-07-11T03:59:11.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc143fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc144156b00 0x7fc144156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc143fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc144156b00 0x7fc144156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46204/0 (socket says 172.21.2.104:46204) 2021-07-11T03:59:11.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc143fff700 1 -- 172.21.2.104:0/4137305007 learned_addr learned my addr 172.21.2.104:0/4137305007 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:11.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14892f700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc124004500 0x7fc1440ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc149130700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc144148d00 0x7fc144149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc143fff700 1 -- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc124004500 msgr2=0x7fc1440ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc143fff700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc124004500 0x7fc1440ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc143fff700 1 -- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc144148d00 msgr2=0x7fc144149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc143fff700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc144148d00 0x7fc144149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc143fff700 1 -- 172.21.2.104:0/4137305007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14414e4c0 con 0x7fc144156b00 2021-07-11T03:59:11.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc149130700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc144148d00 0x7fc144149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:59:11.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc143fff700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc144156b00 0x7fc144156f80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7fc1441417c0 tx=0x7fc134013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:11.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc141ffb700 1 -- 172.21.2.104:0/4137305007 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc13401a070 con 0x7fc144156b00 2021-07-11T03:59:11.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc141ffb700 1 -- 172.21.2.104:0/4137305007 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13400cdc0 con 0x7fc144156b00 2021-07-11T03:59:11.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 -- 172.21.2.104:0/4137305007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc14414e760 con 0x7fc144156b00 2021-07-11T03:59:11.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc141ffb700 1 -- 172.21.2.104:0/4137305007 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc134004710 con 0x7fc144156b00 2021-07-11T03:59:11.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 -- 172.21.2.104:0/4137305007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14414b6f0 con 0x7fc144156b00 2021-07-11T03:59:11.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14ab93700 1 -- 172.21.2.104:0/4137305007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc144057350 con 0x7fc144156b00 2021-07-11T03:59:11.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc141ffb700 1 -- 172.21.2.104:0/4137305007 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc134010070 con 0x7fc144156b00 2021-07-11T03:59:11.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc141ffb700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc128053330 0x7fc1280557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc141ffb700 1 -- 172.21.2.104:0/4137305007 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc13406aed0 con 0x7fc144156b00 2021-07-11T03:59:11.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.218+0000 7fc14892f700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc128053330 0x7fc1280557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.222+0000 7fc14892f700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc128053330 0x7fc1280557f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fc12c00b000 tx=0x7fc12c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:11.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.222+0000 7fc141ffb700 1 -- 172.21.2.104:0/4137305007 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc134042540 con 0x7fc144156b00 2021-07-11T03:59:11.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 -- 172.21.2.104:0/4137305007 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc144044650 con 0x7fc144156b00 2021-07-11T03:59:11.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc141ffb700 1 -- 172.21.2.104:0/4137305007 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc134045b60 con 0x7fc144156b00 2021-07-11T03:59:11.313 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:11.313 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T03:59:11.314 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:11.314 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T03:59:11.314 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T03:59:11.314 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T03:59:11.315 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T03:59:11.315 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T03:59:11.315 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T03:59:11.315 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T03:59:11.315 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T03:59:11.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 -- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc128053330 msgr2=0x7fc1280557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc128053330 0x7fc1280557f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fc12c00b000 tx=0x7fc12c009040).stop 2021-07-11T03:59:11.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 -- 172.21.2.104:0/4137305007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc144156b00 msgr2=0x7fc144156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc144156b00 0x7fc144156f80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7fc1441417c0 tx=0x7fc134013000).stop 2021-07-11T03:59:11.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 -- 172.21.2.104:0/4137305007 shutdown_connections 2021-07-11T03:59:11.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc144148d00 0x7fc144149180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc128053330 0x7fc1280557f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc144156b00 0x7fc144156f80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 --2- 172.21.2.104:0/4137305007 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc124004500 0x7fc1440ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 -- 172.21.2.104:0/4137305007 >> 172.21.2.104:0/4137305007 conn(0x7fc14405f6b0 msgr2=0x7fc1440b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:11.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 -- 172.21.2.104:0/4137305007 shutdown_connections 2021-07-11T03:59:11.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.306+0000 7fc14ab93700 1 -- 172.21.2.104:0/4137305007 wait complete. 2021-07-11T03:59:11.321 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T03:59:11.322 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T03:59:11.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 Processor -- start 2021-07-11T03:59:11.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 -- start start 2021-07-11T03:59:11.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffad40b1ec0 0x7ffad40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad40a77d0 con 0x7ffad40affb0 2021-07-11T03:59:11.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad40b03f0 con 0x7ffad40b22a0 2021-07-11T03:59:11.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40b0650 con 0x7ffad40b1ec0 2021-07-11T03:59:11.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad3fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ffad40affb0 0x7ffad40a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42980/0 (socket says 172.21.2.104:42980) 2021-07-11T03:59:11.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad3fff700 1 -- 172.21.2.104:0/2461825091 learned_addr learned my addr 172.21.2.104:0/2461825091 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:11.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267701844 0 0) 0x7ffad40a77d0 con 0x7ffad40affb0 2021-07-11T03:59:11.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffab4003260 con 0x7ffad40affb0 2021-07-11T03:59:11.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad88b6700 1 --2- 172.21.2.104:0/2461825091 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffad40b1ec0 0x7ffad40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2761905964 0 0) 0x7ffab4003260 con 0x7ffad40affb0 2021-07-11T03:59:11.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffad40b1ec0 msgr2=0x7ffad40b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:11.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 --2- 172.21.2.104:0/2461825091 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffad40b1ec0 0x7ffad40b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:11.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 >> v1:172.21.2.105:6789/0 conn(0x7ffad40b22a0 legacy=0x7ffad4141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:11.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad40b08f0 con 0x7ffad40affb0 2021-07-11T03:59:11.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7ffac4001660 con 0x7ffad40affb0 2021-07-11T03:59:11.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 >> v1:172.21.2.104:6789/0 conn(0x7ffad40affb0 legacy=0x7ffad40a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 --2- 172.21.2.104:0/2461825091 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffab4003800 0x7ffab4003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 --2- 172.21.2.104:0/2461825091 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffad40b1ec0 0x7ffab4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 --2- 172.21.2.104:0/2461825091 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffab4004a40 0x7ffab4004ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40b0650 con 0x7ffab4004a40 2021-07-11T03:59:11.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab400b8e0 con 0x7ffab4003800 2021-07-11T03:59:11.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab400ba50 con 0x7ffad40b1ec0 2021-07-11T03:59:11.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad90b7700 1 --2- 172.21.2.104:0/2461825091 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffab4004a40 0x7ffab4004ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad90b7700 1 -- 172.21.2.104:0/2461825091 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffad40b1ec0 msgr2=0x7ffab4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:11.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad90b7700 1 --2- 172.21.2.104:0/2461825091 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffad40b1ec0 0x7ffab4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:11.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad90b7700 1 -- 172.21.2.104:0/2461825091 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffab4003800 msgr2=0x7ffab4003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:11.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad90b7700 1 --2- 172.21.2.104:0/2461825091 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffab4003800 0x7ffab4003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:11.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad90b7700 1 -- 172.21.2.104:0/2461825091 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad40b08f0 con 0x7ffab4004a40 2021-07-11T03:59:11.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad90b7700 1 --2- 172.21.2.104:0/2461825091 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffab4004a40 0x7ffab4004ec0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7ffacc002320 tx=0x7ffacc007d80).ready entity=mon.0 client_cookie=1fba63280c9ec568 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:11.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffacc002510 con 0x7ffab4004a40 2021-07-11T03:59:11.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc006890 con 0x7ffab4004a40 2021-07-11T03:59:11.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad37fe700 1 -- 172.21.2.104:0/2461825091 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffacc006bf0 con 0x7ffab4004a40 2021-07-11T03:59:11.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 -- 172.21.2.104:0/2461825091 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffab4004a40 msgr2=0x7ffab4004ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 --2- 172.21.2.104:0/2461825091 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffab4004a40 0x7ffab4004ec0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7ffacc002320 tx=0x7ffacc007d80).stop 2021-07-11T03:59:11.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffad90b7700 1 -- 172.21.2.104:0/2461825091 reap_dead start 2021-07-11T03:59:11.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 -- 172.21.2.104:0/2461825091 shutdown_connections 2021-07-11T03:59:11.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 -- 172.21.2.104:0/2461825091 >> 172.21.2.104:0/2461825091 conn(0x7ffad405f6b0 msgr2=0x7ffad405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:11.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 -- 172.21.2.104:0/2461825091 shutdown_connections 2021-07-11T03:59:11.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.370+0000 7ffadab1a700 1 -- 172.21.2.104:0/2461825091 wait complete. 2021-07-11T03:59:11.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffadab1a700 1 Processor -- start 2021-07-11T03:59:11.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffadab1a700 1 -- start start 2021-07-11T03:59:11.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffadab1a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffad40ae630 0x7ffad40aa950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffadab1a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffad40b5750 0x7ffad40ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffadab1a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffad40b5f30 0x7ffad40ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffadab1a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40aae40 con 0x7ffad40b5750 2021-07-11T03:59:11.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffadab1a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40ab0d0 con 0x7ffad40ae630 2021-07-11T03:59:11.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffadab1a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad40b5010 con 0x7ffad40b5f30 2021-07-11T03:59:11.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffad40b5750 0x7ffad40ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffad40b5750 0x7ffad40ab500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46226/0 (socket says 172.21.2.104:46226) 2021-07-11T03:59:11.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad3fff700 1 -- 172.21.2.104:0/1730992145 learned_addr learned my addr 172.21.2.104:0/1730992145 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:11.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad3fff700 1 -- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffad40b5f30 msgr2=0x7ffad40ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad3fff700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffad40b5f30 0x7ffad40ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:11.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad3fff700 1 -- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffad40ae630 msgr2=0x7ffad40aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:11.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad3fff700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffad40ae630 0x7ffad40aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:11.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad3fff700 1 -- 172.21.2.104:0/1730992145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad40b52b0 con 0x7ffad40b5750 2021-07-11T03:59:11.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad3fff700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffad40b5750 0x7ffad40ab500 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7ffac4001020 tx=0x7ffac4001b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:11.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad1ffb700 1 -- 172.21.2.104:0/1730992145 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffac4007170 con 0x7ffad40b5750 2021-07-11T03:59:11.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad1ffb700 1 -- 172.21.2.104:0/1730992145 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac40072d0 con 0x7ffad40b5750 2021-07-11T03:59:11.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffadab1a700 1 -- 172.21.2.104:0/1730992145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad40aea10 con 0x7ffad40b5750 2021-07-11T03:59:11.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffadab1a700 1 -- 172.21.2.104:0/1730992145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad4159f90 con 0x7ffad40b5750 2021-07-11T03:59:11.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad1ffb700 1 -- 172.21.2.104:0/1730992145 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffac40049e0 con 0x7ffad40b5750 2021-07-11T03:59:11.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffadab1a700 1 -- 172.21.2.104:0/1730992145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad40b5440 con 0x7ffad40b5750 2021-07-11T03:59:11.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad1ffb700 1 -- 172.21.2.104:0/1730992145 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ffac401b060 con 0x7ffad40b5750 2021-07-11T03:59:11.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad1ffb700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffab8053350 0x7ffab8055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:11.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad1ffb700 1 -- 172.21.2.104:0/1730992145 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ffac406af90 con 0x7ffad40b5750 2021-07-11T03:59:11.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.374+0000 7ffad88b6700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffab8053350 0x7ffab8055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:11.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.378+0000 7ffad88b6700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffab8053350 0x7ffab8055810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7ffabc00a000 tx=0x7ffabc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:11.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.378+0000 7ffad1ffb700 1 -- 172.21.2.104:0/1730992145 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ffac4042600 con 0x7ffad40b5750 2021-07-11T03:59:11.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:11.462+0000 7ffadab1a700 1 -- 172.21.2.104:0/1730992145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7ffad40ab290 con 0x7ffad40b5750 2021-07-11T03:59:12.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.470+0000 7ffad1ffb700 1 -- 172.21.2.104:0/1730992145 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v73) v1 ==== 133+0+0 (secure 0 0 0) 0x7ffac4019020 con 0x7ffad40b5750 2021-07-11T03:59:12.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.470+0000 7ffadab1a700 1 -- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffab8053350 msgr2=0x7ffab8055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:12.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.470+0000 7ffadab1a700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffab8053350 0x7ffab8055810 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7ffabc00a000 tx=0x7ffabc008040).stop 2021-07-11T03:59:12.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.470+0000 7ffadab1a700 1 -- 172.21.2.104:0/1730992145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffad40b5750 msgr2=0x7ffad40ab500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:12.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.470+0000 7ffadab1a700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffad40b5750 0x7ffad40ab500 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7ffac4001020 tx=0x7ffac4001b70).stop 2021-07-11T03:59:12.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.474+0000 7ffadab1a700 1 -- 172.21.2.104:0/1730992145 shutdown_connections 2021-07-11T03:59:12.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.474+0000 7ffadab1a700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffad40b5f30 0x7ffad40ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:12.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.474+0000 7ffadab1a700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffab8053350 0x7ffab8055810 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:12.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.474+0000 7ffadab1a700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffad40b5750 0x7ffad40ab500 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:12.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.474+0000 7ffadab1a700 1 --2- 172.21.2.104:0/1730992145 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffad40ae630 0x7ffad40aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:12.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.474+0000 7ffadab1a700 1 -- 172.21.2.104:0/1730992145 >> 172.21.2.104:0/1730992145 conn(0x7ffad405f6b0 msgr2=0x7ffad413f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:12.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.474+0000 7ffadab1a700 1 -- 172.21.2.104:0/1730992145 shutdown_connections 2021-07-11T03:59:12.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.474+0000 7ffadab1a700 1 -- 172.21.2.104:0/1730992145 wait complete. 2021-07-11T03:59:12.484 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T03:59:12.486 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T03:59:12.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 Processor -- start 2021-07-11T03:59:12.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- start start 2021-07-11T03:59:12.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f66540b1ec0 0x7f66540b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:12.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66540b0420 con 0x7f66540b22a0 2021-07-11T03:59:12.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66540a77d0 con 0x7f66540affb0 2021-07-11T03:59:12.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66540b0650 con 0x7f66540b1ec0 2021-07-11T03:59:12.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f66588cd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f66540b22a0 0x7f6654141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:42998/0 (socket says 172.21.2.104:42998) 2021-07-11T03:59:12.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f66540b1ec0 0x7f66540b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:12.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f66588cd700 1 -- 172.21.2.104:0/2926000515 learned_addr learned my addr 172.21.2.104:0/2926000515 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:12.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 -- 172.21.2.104:0/2926000515 >> v1:172.21.2.105:6789/0 conn(0x7f66540affb0 legacy=0x7f66540a70c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:59:12.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 -- 172.21.2.104:0/2926000515 >> v1:172.21.2.104:6789/0 conn(0x7f66540b22a0 legacy=0x7f6654141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:12.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 -- 172.21.2.104:0/2926000515 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66540b08f0 con 0x7f66540b1ec0 2021-07-11T03:59:12.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 --2- 172.21.2.104:0/2926000515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66540b1ec0 0x7f66540b12d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f66540ad2d0 tx=0x7f6644006df0).ready entity=mon.0 client_cookie=9216a482d0bcf843 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:12.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6652ffd700 1 -- 172.21.2.104:0/2926000515 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664400d570 con 0x7f66540b1ec0 2021-07-11T03:59:12.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6652ffd700 1 -- 172.21.2.104:0/2926000515 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6644000e90 con 0x7f66540b1ec0 2021-07-11T03:59:12.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6652ffd700 1 -- 172.21.2.104:0/2926000515 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664400d8c0 con 0x7f66540b1ec0 2021-07-11T03:59:12.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- 172.21.2.104:0/2926000515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66540b1ec0 msgr2=0x7f66540b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:12.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 --2- 172.21.2.104:0/2926000515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66540b1ec0 0x7f66540b12d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f66540ad2d0 tx=0x7f6644006df0).stop 2021-07-11T03:59:12.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- 172.21.2.104:0/2926000515 shutdown_connections 2021-07-11T03:59:12.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 --2- 172.21.2.104:0/2926000515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66540b1ec0 0x7f66540b12d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:12.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- 172.21.2.104:0/2926000515 >> 172.21.2.104:0/2926000515 conn(0x7f665405f6b0 msgr2=0x7f665405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:12.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- 172.21.2.104:0/2926000515 shutdown_connections 2021-07-11T03:59:12.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- 172.21.2.104:0/2926000515 wait complete. 2021-07-11T03:59:12.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 Processor -- start 2021-07-11T03:59:12.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- start start 2021-07-11T03:59:12.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66540affb0 0x7f665414c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:12.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66540b1ec0 0x7f665415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:12.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66540b22a0 0x7f665415aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:12.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665415b090 con 0x7f66540affb0 2021-07-11T03:59:12.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66540aaa10 con 0x7f66540b1ec0 2021-07-11T03:59:12.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66540aacb0 con 0x7f66540b22a0 2021-07-11T03:59:12.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66540affb0 0x7f665414c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:12.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66540affb0 0x7f665414c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46234/0 (socket says 172.21.2.104:46234) 2021-07-11T03:59:12.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 -- 172.21.2.104:0/1315531717 learned_addr learned my addr 172.21.2.104:0/1315531717 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:12.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 -- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66540b22a0 msgr2=0x7f665415aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:12.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66540b22a0 0x7f665415aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:12.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 -- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66540b1ec0 msgr2=0x7f665415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:12.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66540b1ec0 0x7f665415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:12.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 -- 172.21.2.104:0/1315531717 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664400b180 con 0x7f66540affb0 2021-07-11T03:59:12.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f6653fff700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66540affb0 0x7f665414c780 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f66540ad2d0 tx=0x7f6644014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:12.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f66517fa700 1 -- 172.21.2.104:0/1315531717 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6644002f50 con 0x7f66540affb0 2021-07-11T03:59:12.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f66517fa700 1 -- 172.21.2.104:0/1315531717 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664400de20 con 0x7f66540affb0 2021-07-11T03:59:12.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f66517fa700 1 -- 172.21.2.104:0/1315531717 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664401e620 con 0x7f66540affb0 2021-07-11T03:59:12.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.538+0000 7f665a330700 1 -- 172.21.2.104:0/1315531717 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665415a210 con 0x7f66540affb0 2021-07-11T03:59:12.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.542+0000 7f665a330700 1 -- 172.21.2.104:0/1315531717 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66540af4c0 con 0x7f66540affb0 2021-07-11T03:59:12.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.542+0000 7f665a330700 1 -- 172.21.2.104:0/1315531717 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66540af4c0 con 0x7f66540affb0 2021-07-11T03:59:12.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.542+0000 7f66517fa700 1 -- 172.21.2.104:0/1315531717 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f664400f030 con 0x7f66540affb0 2021-07-11T03:59:12.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.542+0000 7f66517fa700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6634053210 0x7f66340556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:12.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.542+0000 7f66517fa700 1 -- 172.21.2.104:0/1315531717 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f664406c580 con 0x7f66540affb0 2021-07-11T03:59:12.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.542+0000 7f66537fe700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6634053210 0x7f66340556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:12.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.542+0000 7f66537fe700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6634053210 0x7f66340556d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f6648002fb0 tx=0x7f66480069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:12.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.542+0000 7f66517fa700 1 -- 172.21.2.104:0/1315531717 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6644043980 con 0x7f66540affb0 2021-07-11T03:59:12.587 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:59:12.589 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:59:12.592 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:59:12.598 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:59:12.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:12.626+0000 7f665a330700 1 -- 172.21.2.104:0/1315531717 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f66540af4c0 con 0x7f66540affb0 2021-07-11T03:59:13.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f66517fa700 1 -- 172.21.2.104:0/1315531717 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v74) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6644046fa0 con 0x7f66540affb0 2021-07-11T03:59:13.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 -- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6634053210 msgr2=0x7f66340556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6634053210 0x7f66340556d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f6648002fb0 tx=0x7f66480069b0).stop 2021-07-11T03:59:13.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 -- 172.21.2.104:0/1315531717 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66540affb0 msgr2=0x7f665414c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66540affb0 0x7f665414c780 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f66540ad2d0 tx=0x7f6644014000).stop 2021-07-11T03:59:13.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 -- 172.21.2.104:0/1315531717 shutdown_connections 2021-07-11T03:59:13.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66540b22a0 0x7f665415aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66540b1ec0 0x7f665415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6634053210 0x7f66340556d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 --2- 172.21.2.104:0/1315531717 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66540affb0 0x7f665414c780 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 -- 172.21.2.104:0/1315531717 >> 172.21.2.104:0/1315531717 conn(0x7f665405f6b0 msgr2=0x7f66540b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:13.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 -- 172.21.2.104:0/1315531717 shutdown_connections 2021-07-11T03:59:13.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.434+0000 7f665a330700 1 -- 172.21.2.104:0/1315531717 wait complete. 2021-07-11T03:59:13.445 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T03:59:13.450 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T03:59:13.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cc13b1700 1 Processor -- start 2021-07-11T03:59:13.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cc13b1700 1 -- start start 2021-07-11T03:59:13.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cc13b1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2cbc1393d0 0x7f2cbc0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cc13b1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2cbc142840 0x7f2cbc142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cc13b1700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc0604e0 con 0x7f2cbc060100 2021-07-11T03:59:13.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cc13b1700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0b16b0 con 0x7f2cbc142840 2021-07-11T03:59:13.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cc13b1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0ac400 con 0x7f2cbc1393d0 2021-07-11T03:59:13.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cbb7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2cbc142840 0x7f2cbc142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cbb7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2cbc142840 0x7f2cbc142cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46246/0 (socket says 172.21.2.104:46246) 2021-07-11T03:59:13.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cbb7fe700 1 -- 172.21.2.104:0/1125255338 learned_addr learned my addr 172.21.2.104:0/1125255338 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:13.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cba7fc700 1 --2- 172.21.2.104:0/1125255338 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2cbc1393d0 0x7f2cbc0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cbb7fe700 1 -- 172.21.2.104:0/1125255338 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2cbc1393d0 msgr2=0x7f2cbc0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:13.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cbb7fe700 1 --2- 172.21.2.104:0/1125255338 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2cbc1393d0 0x7f2cbc0b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cbb7fe700 1 -- 172.21.2.104:0/1125255338 >> v1:172.21.2.105:6789/0 conn(0x7f2cbc060100 legacy=0x7f2cbc138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cbb7fe700 1 -- 172.21.2.104:0/1125255338 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc0ac6a0 con 0x7f2cbc142840 2021-07-11T03:59:13.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.498+0000 7f2cbb7fe700 1 --2- 172.21.2.104:0/1125255338 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2cbc142840 0x7f2cbc142cd0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f2cb40032f0 tx=0x7f2cb400acc0).ready entity=mon.0 client_cookie=6207ab1642c56328 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:13.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cb9ffb700 1 -- 172.21.2.104:0/1125255338 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cb400f070 con 0x7f2cbc142840 2021-07-11T03:59:13.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cb9ffb700 1 -- 172.21.2.104:0/1125255338 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb4008de0 con 0x7f2cbc142840 2021-07-11T03:59:13.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cb9ffb700 1 -- 172.21.2.104:0/1125255338 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cb400c8c0 con 0x7f2cbc142840 2021-07-11T03:59:13.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- 172.21.2.104:0/1125255338 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2cbc142840 msgr2=0x7f2cbc142cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 --2- 172.21.2.104:0/1125255338 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2cbc142840 0x7f2cbc142cd0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f2cb40032f0 tx=0x7f2cb400acc0).stop 2021-07-11T03:59:13.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- 172.21.2.104:0/1125255338 shutdown_connections 2021-07-11T03:59:13.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 --2- 172.21.2.104:0/1125255338 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2cbc142840 0x7f2cbc142cd0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 --2- 172.21.2.104:0/1125255338 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2cbc1393d0 0x7f2cbc0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- 172.21.2.104:0/1125255338 >> 172.21.2.104:0/1125255338 conn(0x7f2cbc061070 msgr2=0x7f2cbc05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:13.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- 172.21.2.104:0/1125255338 shutdown_connections 2021-07-11T03:59:13.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- 172.21.2.104:0/1125255338 wait complete. 2021-07-11T03:59:13.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 Processor -- start 2021-07-11T03:59:13.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- start start 2021-07-11T03:59:13.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2cbc060100 0x7f2cbc137b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2cbc1393d0 0x7f2cbc14e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2cbc142840 0x7f2cbc14e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc1438a0 con 0x7f2cbc142840 2021-07-11T03:59:13.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0ace80 con 0x7f2cbc060100 2021-07-11T03:59:13.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0ad120 con 0x7f2cbc1393d0 2021-07-11T03:59:13.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cbaffd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2cbc060100 0x7f2cbc137b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cbb7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2cbc142840 0x7f2cbc14e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cbaffd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2cbc060100 0x7f2cbc137b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:53808/0 (socket says 172.21.2.104:53808) 2021-07-11T03:59:13.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cbaffd700 1 -- 172.21.2.104:0/31464040 learned_addr learned my addr 172.21.2.104:0/31464040 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:13.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cba7fc700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2cbc1393d0 0x7f2cbc14e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cbb7fe700 1 -- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2cbc1393d0 msgr2=0x7f2cbc14e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cbb7fe700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2cbc1393d0 0x7f2cbc14e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cbb7fe700 1 -- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2cbc060100 msgr2=0x7f2cbc137b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cbb7fe700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2cbc060100 0x7f2cbc137b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cbb7fe700 1 -- 172.21.2.104:0/31464040 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4007040 con 0x7f2cbc142840 2021-07-11T03:59:13.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cbb7fe700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2cbc142840 0x7f2cbc14e760 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f2cb40032f0 tx=0x7f2cbc0b0f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:13.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cb3fff700 1 -- 172.21.2.104:0/31464040 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cb400cab0 con 0x7f2cbc142840 2021-07-11T03:59:13.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cb3fff700 1 -- 172.21.2.104:0/31464040 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb4019070 con 0x7f2cbc142840 2021-07-11T03:59:13.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cb3fff700 1 -- 172.21.2.104:0/31464040 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2cb400ce00 con 0x7f2cbc142840 2021-07-11T03:59:13.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- 172.21.2.104:0/31464040 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc0ad3c0 con 0x7f2cbc142840 2021-07-11T03:59:13.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- 172.21.2.104:0/31464040 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc14ec50 con 0x7f2cbc142840 2021-07-11T03:59:13.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cc13b1700 1 -- 172.21.2.104:0/31464040 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc1466d0 con 0x7f2cbc142840 2021-07-11T03:59:13.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cb3fff700 1 -- 172.21.2.104:0/31464040 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2cb4008040 con 0x7f2cbc142840 2021-07-11T03:59:13.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cb3fff700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ca0053160 0x7f2ca0055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cb3fff700 1 -- 172.21.2.104:0/31464040 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f2cb406c4a0 con 0x7f2cbc142840 2021-07-11T03:59:13.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.502+0000 7f2cbaffd700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ca0053160 0x7f2ca0055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.506+0000 7f2cbaffd700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ca0053160 0x7f2ca0055620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f2cac00d000 tx=0x7f2cac007be0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:13.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.506+0000 7f2cb3fff700 1 -- 172.21.2.104:0/31464040 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2cb4043780 con 0x7f2cbc142840 2021-07-11T03:59:13.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.614+0000 7f2cc13b1700 1 -- 172.21.2.104:0/31464040 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f2cbc1439e0 con 0x7f2cbc142840 2021-07-11T03:59:13.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cb3fff700 1 -- 172.21.2.104:0/31464040 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 13 and data pool 14 v31) v1 ==== 189+0+0 (secure 0 0 0) 0x7f2cb4046da0 con 0x7f2cbc142840 2021-07-11T03:59:13.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 -- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ca0053160 msgr2=0x7f2ca0055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ca0053160 0x7f2ca0055620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f2cac00d000 tx=0x7f2cac007be0).stop 2021-07-11T03:59:13.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 -- 172.21.2.104:0/31464040 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2cbc142840 msgr2=0x7f2cbc14e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2cbc142840 0x7f2cbc14e760 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f2cb40032f0 tx=0x7f2cbc0b0f50).stop 2021-07-11T03:59:13.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 -- 172.21.2.104:0/31464040 shutdown_connections 2021-07-11T03:59:13.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2ca0053160 0x7f2ca0055620 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2cbc142840 0x7f2cbc14e760 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2cbc1393d0 0x7f2cbc14e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 --2- 172.21.2.104:0/31464040 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2cbc060100 0x7f2cbc137b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 -- 172.21.2.104:0/31464040 >> 172.21.2.104:0/31464040 conn(0x7f2cbc061070 msgr2=0x7f2cbc0aa470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:13.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 -- 172.21.2.104:0/31464040 shutdown_connections 2021-07-11T03:59:13.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.622+0000 7f2cc13b1700 1 -- 172.21.2.104:0/31464040 wait complete. 2021-07-11T03:59:13.633 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 13 and data pool 14 2021-07-11T03:59:13.639 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:59:13.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9c10be700 1 Processor -- start 2021-07-11T03:59:13.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9c10be700 1 -- start start 2021-07-11T03:59:13.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9c10be700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd9bc067180 0x7fd9bc05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9c10be700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9bc0b1970 0x7fd9bc05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9c10be700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9bc0a7a90 con 0x7fd9bc0a76b0 2021-07-11T03:59:13.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9c10be700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9bc147620 con 0x7fd9bc067180 2021-07-11T03:59:13.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9c10be700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc1478c0 con 0x7fd9bc0b1970 2021-07-11T03:59:13.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9ba59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd9bc067180 0x7fd9bc05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9ba59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd9bc067180 0x7fd9bc05f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46268/0 (socket says 172.21.2.104:46268) 2021-07-11T03:59:13.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9ba59c700 1 -- 172.21.2.104:0/4161758816 learned_addr learned my addr 172.21.2.104:0/4161758816 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:13.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9ba59c700 1 -- 172.21.2.104:0/4161758816 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9bc0b1970 msgr2=0x7fd9bc05fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:13.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9ba59c700 1 --2- 172.21.2.104:0/4161758816 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9bc0b1970 0x7fd9bc05fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9ba59c700 1 -- 172.21.2.104:0/4161758816 >> v1:172.21.2.105:6789/0 conn(0x7fd9bc0a76b0 legacy=0x7fd9bc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:13.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9ba59c700 1 -- 172.21.2.104:0/4161758816 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc147b60 con 0x7fd9bc067180 2021-07-11T03:59:13.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9ba59c700 1 --2- 172.21.2.104:0/4161758816 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9bc067180 0x7fd9bc05f6b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fd9bc06a0e0 tx=0x7fd9a4006eb0).ready entity=mon.0 client_cookie=7736177aa3a87d15 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:13.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9b9d9b700 1 -- 172.21.2.104:0/4161758816 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9a400d570 con 0x7fd9bc067180 2021-07-11T03:59:13.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9b9d9b700 1 -- 172.21.2.104:0/4161758816 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a4000e90 con 0x7fd9bc067180 2021-07-11T03:59:13.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9b9d9b700 1 -- 172.21.2.104:0/4161758816 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9a400d8c0 con 0x7fd9bc067180 2021-07-11T03:59:13.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9c10be700 1 -- 172.21.2.104:0/4161758816 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9bc067180 msgr2=0x7fd9bc05f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.682+0000 7fd9c10be700 1 --2- 172.21.2.104:0/4161758816 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9bc067180 0x7fd9bc05f6b0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fd9bc06a0e0 tx=0x7fd9a4006eb0).stop 2021-07-11T03:59:13.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 -- 172.21.2.104:0/4161758816 shutdown_connections 2021-07-11T03:59:13.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 --2- 172.21.2.104:0/4161758816 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9bc0b1970 0x7fd9bc05fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 --2- 172.21.2.104:0/4161758816 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9bc067180 0x7fd9bc05f6b0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 -- 172.21.2.104:0/4161758816 >> 172.21.2.104:0/4161758816 conn(0x7fd9bc0b53e0 msgr2=0x7fd9bc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:13.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 -- 172.21.2.104:0/4161758816 shutdown_connections 2021-07-11T03:59:13.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 -- 172.21.2.104:0/4161758816 wait complete. 2021-07-11T03:59:13.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 Processor -- start 2021-07-11T03:59:13.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 -- start start 2021-07-11T03:59:13.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9bc067180 0x7fd9bc0ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9bc0a76b0 0x7fd9bc0ad940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9bc0b1970 0x7fd9bc0aa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc0ade30 con 0x7fd9bc067180 2021-07-11T03:59:13.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc0a9680 con 0x7fd9bc0b1970 2021-07-11T03:59:13.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc0a98e0 con 0x7fd9bc0a76b0 2021-07-11T03:59:13.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9bad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9bc067180 0x7fd9bc0ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9bad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9bc067180 0x7fd9bc0ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46276/0 (socket says 172.21.2.104:46276) 2021-07-11T03:59:13.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9bad9d700 1 -- 172.21.2.104:0/1562552435 learned_addr learned my addr 172.21.2.104:0/1562552435 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:13.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9bad9d700 1 -- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9bc0a76b0 msgr2=0x7fd9bc0ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9bad9d700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9bc0a76b0 0x7fd9bc0ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9bad9d700 1 -- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9bc0b1970 msgr2=0x7fd9bc0aa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9bad9d700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9bc0b1970 0x7fd9bc0aa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9bad9d700 1 -- 172.21.2.104:0/1562552435 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a400b1e0 con 0x7fd9bc067180 2021-07-11T03:59:13.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9bad9d700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9bc067180 0x7fd9bc0ad450 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7fd9ac000f70 tx=0x7fd9ac007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:13.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9b3fff700 1 -- 172.21.2.104:0/1562552435 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9ac00ce80 con 0x7fd9bc067180 2021-07-11T03:59:13.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9b3fff700 1 -- 172.21.2.104:0/1562552435 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac013450 con 0x7fd9bc067180 2021-07-11T03:59:13.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 -- 172.21.2.104:0/1562552435 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc0a9b80 con 0x7fd9bc067180 2021-07-11T03:59:13.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 -- 172.21.2.104:0/1562552435 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc0aad50 con 0x7fd9bc067180 2021-07-11T03:59:13.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9b3fff700 1 -- 172.21.2.104:0/1562552435 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9ac0137b0 con 0x7fd9bc067180 2021-07-11T03:59:13.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9c10be700 1 -- 172.21.2.104:0/1562552435 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc057350 con 0x7fd9bc067180 2021-07-11T03:59:13.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9b3fff700 1 -- 172.21.2.104:0/1562552435 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd9ac01a060 con 0x7fd9bc067180 2021-07-11T03:59:13.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9b3fff700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9a00531a0 0x7fd9a0055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9b3fff700 1 -- 172.21.2.104:0/1562552435 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd9ac06af70 con 0x7fd9bc067180 2021-07-11T03:59:13.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.686+0000 7fd9ba59c700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9a00531a0 0x7fd9a0055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.690+0000 7fd9ba59c700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9a00531a0 0x7fd9a0055660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fd9bc06a0e0 tx=0x7fd9a4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:13.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.690+0000 7fd9b3fff700 1 -- 172.21.2.104:0/1562552435 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd9ac0421a0 con 0x7fd9bc067180 2021-07-11T03:59:13.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.774+0000 7fd9c10be700 1 -- 172.21.2.104:0/1562552435 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9bc0698e0 con 0x7fd9bc067180 2021-07-11T03:59:13.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9b3fff700 1 -- 172.21.2.104:0/1562552435 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16054 (secure 0 0 0) 0x7fd9ac018020 con 0x7fd9bc067180 2021-07-11T03:59:13.783 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:13.783 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":75,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:59:13.620301+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:59:11.477806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-07-11T03:59:12.643109+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:59:13.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 -- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9a00531a0 msgr2=0x7fd9a0055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9a00531a0 0x7fd9a0055660 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fd9bc06a0e0 tx=0x7fd9a4015000).stop 2021-07-11T03:59:13.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 -- 172.21.2.104:0/1562552435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9bc067180 msgr2=0x7fd9bc0ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9bc067180 0x7fd9bc0ad450 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7fd9ac000f70 tx=0x7fd9ac007c30).stop 2021-07-11T03:59:13.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 -- 172.21.2.104:0/1562552435 shutdown_connections 2021-07-11T03:59:13.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9bc0b1970 0x7fd9bc0aa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9bc0a76b0 0x7fd9bc0ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9a00531a0 0x7fd9a0055660 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 --2- 172.21.2.104:0/1562552435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9bc067180 0x7fd9bc0ad450 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 -- 172.21.2.104:0/1562552435 >> 172.21.2.104:0/1562552435 conn(0x7fd9bc0b53e0 msgr2=0x7fd9bc1447b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:13.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 -- 172.21.2.104:0/1562552435 shutdown_connections 2021-07-11T03:59:13.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.778+0000 7fd9c10be700 1 -- 172.21.2.104:0/1562552435 wait complete. 2021-07-11T03:59:13.791 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:59:13.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.834+0000 7f14c89ce700 1 Processor -- start 2021-07-11T03:59:13.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.834+0000 7f14c89ce700 1 -- start start 2021-07-11T03:59:13.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f14c40a76b0 0x7f14c4066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f14c40b0060 0x7f14c405fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c40a7a90 con 0x7f14c40b1970 2021-07-11T03:59:13.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c4147620 con 0x7f14c40b0060 2021-07-11T03:59:13.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c41478c0 con 0x7f14c40a76b0 2021-07-11T03:59:13.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c1d9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f14c40b1970 0x7f14c405f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43050/0 (socket says 172.21.2.104:43050) 2021-07-11T03:59:13.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c1d9b700 1 -- 172.21.2.104:0/1291723836 learned_addr learned my addr 172.21.2.104:0/1291723836 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:13.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c2d9d700 1 --2- 172.21.2.104:0/1291723836 >> v2:172.21.2.104:3300/0 conn(0x7f14c40b0060 0x7f14c405fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c2d9d700 1 -- 172.21.2.104:0/1291723836 >> v2:172.21.2.105:3300/0 conn(0x7f14c40a76b0 msgr2=0x7f14c4066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:13.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c2d9d700 1 --2- 172.21.2.104:0/1291723836 >> v2:172.21.2.105:3300/0 conn(0x7f14c40a76b0 0x7f14c4066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c2d9d700 1 -- 172.21.2.104:0/1291723836 >> v1:172.21.2.104:6789/0 conn(0x7f14c40b1970 legacy=0x7f14c405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c159a700 1 -- 172.21.2.104:0/1291723836 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521372007 0 0) 0x7f14c40a7a90 con 0x7f14c40b1970 2021-07-11T03:59:13.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c2d9d700 1 -- 172.21.2.104:0/1291723836 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c4147b60 con 0x7f14c40b0060 2021-07-11T03:59:13.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c2d9d700 1 --2- 172.21.2.104:0/1291723836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14c40b0060 0x7f14c405fdc0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f14bc0032f0 tx=0x7f14bc0091d0).ready entity=mon.0 client_cookie=5b34663a2d76e090 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:13.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c159a700 1 -- 172.21.2.104:0/1291723836 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14bc009ea0 con 0x7f14c40b0060 2021-07-11T03:59:13.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c159a700 1 -- 172.21.2.104:0/1291723836 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14bc00b870 con 0x7f14c40b0060 2021-07-11T03:59:13.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c159a700 1 -- 172.21.2.104:0/1291723836 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14bc00b9e0 con 0x7f14c40b0060 2021-07-11T03:59:13.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- 172.21.2.104:0/1291723836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14c40b0060 msgr2=0x7f14c405fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 --2- 172.21.2.104:0/1291723836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14c40b0060 0x7f14c405fdc0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f14bc0032f0 tx=0x7f14bc0091d0).stop 2021-07-11T03:59:13.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- 172.21.2.104:0/1291723836 shutdown_connections 2021-07-11T03:59:13.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 --2- 172.21.2.104:0/1291723836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14c40b0060 0x7f14c405fdc0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 --2- 172.21.2.104:0/1291723836 >> v2:172.21.2.105:3300/0 conn(0x7f14c40a76b0 0x7f14c4066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- 172.21.2.104:0/1291723836 >> 172.21.2.104:0/1291723836 conn(0x7f14c40b53e0 msgr2=0x7f14c40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:13.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- 172.21.2.104:0/1291723836 shutdown_connections 2021-07-11T03:59:13.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- 172.21.2.104:0/1291723836 wait complete. 2021-07-11T03:59:13.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 Processor -- start 2021-07-11T03:59:13.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- start start 2021-07-11T03:59:13.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14c40a76b0 0x7f14c414c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14c40b0060 0x7f14c40ab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14c40b1970 0x7f14c40ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c4141640 con 0x7f14c40a76b0 2021-07-11T03:59:13.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c40aad50 con 0x7f14c40b0060 2021-07-11T03:59:13.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c40aa570 con 0x7f14c40b1970 2021-07-11T03:59:13.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14c40a76b0 0x7f14c414c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14c40a76b0 0x7f14c414c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46288/0 (socket says 172.21.2.104:46288) 2021-07-11T03:59:13.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c259c700 1 -- 172.21.2.104:0/4195548325 learned_addr learned my addr 172.21.2.104:0/4195548325 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:13.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c259c700 1 -- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14c40b1970 msgr2=0x7f14c40ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c259c700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14c40b1970 0x7f14c40ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c259c700 1 -- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14c40b0060 msgr2=0x7f14c40ab270 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:59:13.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c259c700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14c40b0060 0x7f14c40ab270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c259c700 1 -- 172.21.2.104:0/4195548325 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc009470 con 0x7f14c40a76b0 2021-07-11T03:59:13.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c259c700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14c40a76b0 0x7f14c414c530 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f14c40a9ae0 tx=0x7f14b4006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:13.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14bb7fe700 1 -- 172.21.2.104:0/4195548325 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14b4013460 con 0x7f14c40a76b0 2021-07-11T03:59:13.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14bb7fe700 1 -- 172.21.2.104:0/4195548325 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b40135c0 con 0x7f14c40a76b0 2021-07-11T03:59:13.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- 172.21.2.104:0/4195548325 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c40aa810 con 0x7f14c40a76b0 2021-07-11T03:59:13.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14c89ce700 1 -- 172.21.2.104:0/4195548325 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c414ca20 con 0x7f14c40a76b0 2021-07-11T03:59:13.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.838+0000 7f14bb7fe700 1 -- 172.21.2.104:0/4195548325 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14b4013920 con 0x7f14c40a76b0 2021-07-11T03:59:13.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.842+0000 7f14c89ce700 1 -- 172.21.2.104:0/4195548325 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c4057350 con 0x7f14c40a76b0 2021-07-11T03:59:13.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.842+0000 7f14bb7fe700 1 -- 172.21.2.104:0/4195548325 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f14b401a060 con 0x7f14c40a76b0 2021-07-11T03:59:13.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.842+0000 7f14bb7fe700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14a80532d0 0x7f14a8055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.842+0000 7f14bb7fe700 1 -- 172.21.2.104:0/4195548325 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f14b406af70 con 0x7f14c40a76b0 2021-07-11T03:59:13.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.842+0000 7f14c1d9b700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14a80532d0 0x7f14a8055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:13.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.842+0000 7f14c1d9b700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14a80532d0 0x7f14a8055790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f14ac000bc0 tx=0x7f14ac007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:13.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.842+0000 7f14bb7fe700 1 -- 172.21.2.104:0/4195548325 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f14b40421a0 con 0x7f14c40a76b0 2021-07-11T03:59:13.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.926+0000 7f14c89ce700 1 -- 172.21.2.104:0/4195548325 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14c414a910 con 0x7f14c40a76b0 2021-07-11T03:59:13.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.926+0000 7f14bb7fe700 1 -- 172.21.2.104:0/4195548325 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16054 (secure 0 0 0) 0x7f14b4018070 con 0x7f14c40a76b0 2021-07-11T03:59:13.935 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:13.935 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":75,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:59:13.620301+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:59:11.477806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-07-11T03:59:12.643109+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:59:13.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 -- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14a80532d0 msgr2=0x7f14a8055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14a80532d0 0x7f14a8055790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f14ac000bc0 tx=0x7f14ac007810).stop 2021-07-11T03:59:13.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 -- 172.21.2.104:0/4195548325 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14c40a76b0 msgr2=0x7f14c414c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:13.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14c40a76b0 0x7f14c414c530 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f14c40a9ae0 tx=0x7f14b4006ec0).stop 2021-07-11T03:59:13.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 -- 172.21.2.104:0/4195548325 shutdown_connections 2021-07-11T03:59:13.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14c40b1970 0x7f14c40ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14c40b0060 0x7f14c40ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:13.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14a80532d0 0x7f14a8055790 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 --2- 172.21.2.104:0/4195548325 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14c40a76b0 0x7f14c414c530 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:13.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 -- 172.21.2.104:0/4195548325 >> 172.21.2.104:0/4195548325 conn(0x7f14c40b53e0 msgr2=0x7f14c41460f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:13.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 -- 172.21.2.104:0/4195548325 shutdown_connections 2021-07-11T03:59:13.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.930+0000 7f14c89ce700 1 -- 172.21.2.104:0/4195548325 wait complete. 2021-07-11T03:59:13.942 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T03:59:13.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b2465e700 1 Processor -- start 2021-07-11T03:59:13.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b2465e700 1 -- start start 2021-07-11T03:59:13.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b2465e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b1c0b12d0 0x7f4b1c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:13.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b2465e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1c0a77d0 con 0x7f4b1c0b1ec0 2021-07-11T03:59:13.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b2465e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1c0b0650 con 0x7f4b1c0b22a0 2021-07-11T03:59:13.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b2465e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0b1740 con 0x7f4b1c0b12d0 2021-07-11T03:59:13.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b223fa700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4b1c0b1ec0 0x7f4b1c0a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43060/0 (socket says 172.21.2.104:43060) 2021-07-11T03:59:13.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b223fa700 1 -- 172.21.2.104:0/1925779850 learned_addr learned my addr 172.21.2.104:0/1925779850 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:14.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996473911 0 0) 0x7f4b1c0a77d0 con 0x7f4b1c0b1ec0 2021-07-11T03:59:14.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b04003260 con 0x7f4b1c0b1ec0 2021-07-11T03:59:14.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b21bf9700 1 --2- 172.21.2.104:0/1925779850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b1c0b12d0 0x7f4b1c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034819385 0 0) 0x7f4b04003260 con 0x7f4b1c0b1ec0 2021-07-11T03:59:14.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b1c0b12d0 msgr2=0x7f4b1c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 --2- 172.21.2.104:0/1925779850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b1c0b12d0 0x7f4b1c0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 >> v1:172.21.2.105:6789/0 conn(0x7f4b1c0b22a0 legacy=0x7f4b1c141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c0b08f0 con 0x7f4b1c0b1ec0 2021-07-11T03:59:14.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f4b10001660 con 0x7f4b1c0b1ec0 2021-07-11T03:59:14.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 >> v1:172.21.2.104:6789/0 conn(0x7f4b1c0b1ec0 legacy=0x7f4b1c0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 --2- 172.21.2.104:0/1925779850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b04003800 0x7f4b04003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 --2- 172.21.2.104:0/1925779850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b1c0b12d0 0x7f4b04004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 --2- 172.21.2.104:0/1925779850 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b04004bf0 0x7f4b0400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0b1740 con 0x7f4b04003800 2021-07-11T03:59:14.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b04005280 con 0x7f4b04004bf0 2021-07-11T03:59:14.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0400bc40 con 0x7f4b1c0b12d0 2021-07-11T03:59:14.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b223fa700 1 --2- 172.21.2.104:0/1925779850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b04003800 0x7f4b04003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b223fa700 1 -- 172.21.2.104:0/1925779850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b1c0b12d0 msgr2=0x7f4b04004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b223fa700 1 --2- 172.21.2.104:0/1925779850 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b1c0b12d0 0x7f4b04004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b223fa700 1 -- 172.21.2.104:0/1925779850 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b04004bf0 msgr2=0x7f4b0400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b223fa700 1 --2- 172.21.2.104:0/1925779850 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b04004bf0 0x7f4b0400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b223fa700 1 -- 172.21.2.104:0/1925779850 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c0b08f0 con 0x7f4b04003800 2021-07-11T03:59:14.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b223fa700 1 --2- 172.21.2.104:0/1925779850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b04003800 0x7f4b04003c00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f4b10001020 tx=0x7f4b10001e20).ready entity=mon.0 client_cookie=434acbcc39e99897 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:14.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b100070d0 con 0x7f4b04003800 2021-07-11T03:59:14.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b10007230 con 0x7f4b04003800 2021-07-11T03:59:14.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b213f8700 1 -- 172.21.2.104:0/1925779850 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b1000c9c0 con 0x7f4b04003800 2021-07-11T03:59:14.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b2465e700 1 -- 172.21.2.104:0/1925779850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b04003800 msgr2=0x7f4b04003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b2465e700 1 --2- 172.21.2.104:0/1925779850 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b04003800 0x7f4b04003c00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f4b10001020 tx=0x7f4b10001e20).stop 2021-07-11T03:59:14.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.990+0000 7f4b22bfb700 1 -- 172.21.2.104:0/1925779850 reap_dead start 2021-07-11T03:59:14.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 -- 172.21.2.104:0/1925779850 shutdown_connections 2021-07-11T03:59:14.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 -- 172.21.2.104:0/1925779850 >> 172.21.2.104:0/1925779850 conn(0x7f4b1c05f6b0 msgr2=0x7f4b1c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:14.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 -- 172.21.2.104:0/1925779850 shutdown_connections 2021-07-11T03:59:14.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 -- 172.21.2.104:0/1925779850 wait complete. 2021-07-11T03:59:14.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 Processor -- start 2021-07-11T03:59:14.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 -- start start 2021-07-11T03:59:14.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b1c0ae190 0x7f4b1c0abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b1c0aa7c0 0x7f4b1c13d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b1c159ea0 0x7f4b1c0a9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c13e790 con 0x7f4b1c159ea0 2021-07-11T03:59:14.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c0ae570 con 0x7f4b1c0ae190 2021-07-11T03:59:14.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c13e250 con 0x7f4b1c0aa7c0 2021-07-11T03:59:14.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b22bfb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b1c159ea0 0x7f4b1c0a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b22bfb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b1c159ea0 0x7f4b1c0a9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46306/0 (socket says 172.21.2.104:46306) 2021-07-11T03:59:14.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b22bfb700 1 -- 172.21.2.104:0/1126875113 learned_addr learned my addr 172.21.2.104:0/1126875113 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:14.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b22bfb700 1 -- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b1c0aa7c0 msgr2=0x7f4b1c13d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b22bfb700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b1c0aa7c0 0x7f4b1c13d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b22bfb700 1 -- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b1c0ae190 msgr2=0x7f4b1c0abfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b22bfb700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b1c0ae190 0x7f4b1c0abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b22bfb700 1 -- 172.21.2.104:0/1126875113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1c13e4f0 con 0x7f4b1c159ea0 2021-07-11T03:59:14.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b22bfb700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b1c159ea0 0x7f4b1c0a9420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f4b10001370 tx=0x7f4b14007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:14.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b0f7fe700 1 -- 172.21.2.104:0/1126875113 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b14005880 con 0x7f4b1c159ea0 2021-07-11T03:59:14.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b0f7fe700 1 -- 172.21.2.104:0/1126875113 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b140059e0 con 0x7f4b1c159ea0 2021-07-11T03:59:14.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 -- 172.21.2.104:0/1126875113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1c13d550 con 0x7f4b1c159ea0 2021-07-11T03:59:14.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b0f7fe700 1 -- 172.21.2.104:0/1126875113 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b14005d40 con 0x7f4b1c159ea0 2021-07-11T03:59:14.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 -- 172.21.2.104:0/1126875113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1c13c1e0 con 0x7f4b1c159ea0 2021-07-11T03:59:14.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b2465e700 1 -- 172.21.2.104:0/1126875113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1c13d6e0 con 0x7f4b1c159ea0 2021-07-11T03:59:14.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b0f7fe700 1 -- 172.21.2.104:0/1126875113 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4b1401a060 con 0x7f4b1c159ea0 2021-07-11T03:59:14.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b0f7fe700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4afc0531f0 0x7f4afc0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.994+0000 7f4b0f7fe700 1 -- 172.21.2.104:0/1126875113 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4b1406ad80 con 0x7f4b1c159ea0 2021-07-11T03:59:14.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.998+0000 7f4b223fa700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4afc0531f0 0x7f4afc0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.998+0000 7f4b223fa700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4afc0531f0 0x7f4afc0556b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f4b10001020 tx=0x7f4b10015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:14.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:13.998+0000 7f4b0f7fe700 1 -- 172.21.2.104:0/1126875113 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4b14041fb0 con 0x7f4b1c159ea0 2021-07-11T03:59:14.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.106+0000 7f4b2465e700 1 -- 172.21.2.104:0/1126875113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4b1c057350 con 0x7f4b1c159ea0 2021-07-11T03:59:14.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b0f7fe700 1 -- 172.21.2.104:0/1126875113 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4b14018070 con 0x7f4b1c159ea0 2021-07-11T03:59:14.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 -- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4afc0531f0 msgr2=0x7f4afc0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4afc0531f0 0x7f4afc0556b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f4b10001020 tx=0x7f4b10015000).stop 2021-07-11T03:59:14.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 -- 172.21.2.104:0/1126875113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b1c159ea0 msgr2=0x7f4b1c0a9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b1c159ea0 0x7f4b1c0a9420 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f4b10001370 tx=0x7f4b14007dc0).stop 2021-07-11T03:59:14.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 -- 172.21.2.104:0/1126875113 shutdown_connections 2021-07-11T03:59:14.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4afc0531f0 0x7f4afc0556b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b1c159ea0 0x7f4b1c0a9420 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b1c0aa7c0 0x7f4b1c13d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 --2- 172.21.2.104:0/1126875113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b1c0ae190 0x7f4b1c0abfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 -- 172.21.2.104:0/1126875113 >> 172.21.2.104:0/1126875113 conn(0x7f4b1c05f6b0 msgr2=0x7f4b1c13f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:14.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 -- 172.21.2.104:0/1126875113 shutdown_connections 2021-07-11T03:59:14.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.630+0000 7f4b2465e700 1 -- 172.21.2.104:0/1126875113 wait complete. 2021-07-11T03:59:14.648 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:59:14.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.694+0000 7f6f7da70700 1 Processor -- start 2021-07-11T03:59:14.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.694+0000 7f6f7da70700 1 -- start start 2021-07-11T03:59:14.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.694+0000 7f6f7da70700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6f78067180 0x7f6f7805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.694+0000 7f6f7da70700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f780b1970 0x7f6f7805fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.694+0000 7f6f7da70700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f780a7a90 con 0x7f6f780a76b0 2021-07-11T03:59:14.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.694+0000 7f6f7da70700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f78147620 con 0x7f6f78067180 2021-07-11T03:59:14.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.694+0000 7f6f7da70700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f781478c0 con 0x7f6f780b1970 2021-07-11T03:59:14.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.694+0000 7f6f777fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6f780a76b0 0x7f6f78066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43084/0 (socket says 172.21.2.104:43084) 2021-07-11T03:59:14.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.694+0000 7f6f777fe700 1 -- 172.21.2.104:0/1784881999 learned_addr learned my addr 172.21.2.104:0/1784881999 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:14.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114041242 0 0) 0x7f6f780a7a90 con 0x7f6f780a76b0 2021-07-11T03:59:14.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f58003260 con 0x7f6f780a76b0 2021-07-11T03:59:14.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f77fff700 1 --2- 172.21.2.104:0/1784881999 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f780b1970 0x7f6f7805fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 734618328 0 0) 0x7f6f58003260 con 0x7f6f780a76b0 2021-07-11T03:59:14.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f76ffd700 1 --2- 172.21.2.104:0/1784881999 >> v2:172.21.2.105:3300/0 conn(0x7f6f78067180 0x7f6f7805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f780b1970 msgr2=0x7f6f7805fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 --2- 172.21.2.104:0/1784881999 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f780b1970 0x7f6f7805fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 >> v2:172.21.2.105:3300/0 conn(0x7f6f78067180 msgr2=0x7f6f7805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 --2- 172.21.2.104:0/1784881999 >> v2:172.21.2.105:3300/0 conn(0x7f6f78067180 0x7f6f7805f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f78147b60 con 0x7f6f780a76b0 2021-07-11T03:59:14.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f6f68001660 con 0x7f6f780a76b0 2021-07-11T03:59:14.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 >> v1:172.21.2.104:6789/0 conn(0x7f6f780a76b0 legacy=0x7f6f78066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 --2- 172.21.2.104:0/1784881999 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f58003750 0x7f6f58003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 --2- 172.21.2.104:0/1784881999 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f58004500 0x7f6f58004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 --2- 172.21.2.104:0/1784881999 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f58004f90 0x7f6f5800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78147620 con 0x7f6f58003750 2021-07-11T03:59:14.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f781478c0 con 0x7f6f58004500 2021-07-11T03:59:14.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5800be50 con 0x7f6f58004f90 2021-07-11T03:59:14.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f77fff700 1 --2- 172.21.2.104:0/1784881999 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f780b1970 0x7f6f7805fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:59:14.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f76ffd700 1 --2- 172.21.2.104:0/1784881999 >> v2:172.21.2.105:3300/0 conn(0x7f6f78067180 0x7f6f7805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:59:14.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f777fe700 1 --2- 172.21.2.104:0/1784881999 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f58003750 0x7f6f58003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f777fe700 1 -- 172.21.2.104:0/1784881999 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f58004f90 msgr2=0x7f6f5800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f777fe700 1 --2- 172.21.2.104:0/1784881999 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f58004f90 0x7f6f5800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f777fe700 1 -- 172.21.2.104:0/1784881999 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f58004500 msgr2=0x7f6f58004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f777fe700 1 --2- 172.21.2.104:0/1784881999 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f58004500 0x7f6f58004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f777fe700 1 -- 172.21.2.104:0/1784881999 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f58003260 con 0x7f6f58003750 2021-07-11T03:59:14.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f777fe700 1 --2- 172.21.2.104:0/1784881999 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f58003750 0x7f6f58003b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f6f68001370 tx=0x7f6f68001e50).ready entity=mon.0 client_cookie=c79559939e05bc59 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:14.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f680070d0 con 0x7f6f58003750 2021-07-11T03:59:14.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f68007230 con 0x7f6f58003750 2021-07-11T03:59:14.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f767fc700 1 -- 172.21.2.104:0/1784881999 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f6800c9c0 con 0x7f6f58003750 2021-07-11T03:59:14.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 -- 172.21.2.104:0/1784881999 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f58003750 msgr2=0x7f6f58003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 --2- 172.21.2.104:0/1784881999 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f58003750 0x7f6f58003b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f6f68001370 tx=0x7f6f68001e50).stop 2021-07-11T03:59:14.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f77fff700 1 -- 172.21.2.104:0/1784881999 reap_dead start 2021-07-11T03:59:14.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 -- 172.21.2.104:0/1784881999 shutdown_connections 2021-07-11T03:59:14.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 -- 172.21.2.104:0/1784881999 >> 172.21.2.104:0/1784881999 conn(0x7f6f780b53e0 msgr2=0x7f6f780b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:14.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 -- 172.21.2.104:0/1784881999 shutdown_connections 2021-07-11T03:59:14.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 -- 172.21.2.104:0/1784881999 wait complete. 2021-07-11T03:59:14.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 Processor -- start 2021-07-11T03:59:14.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 -- start start 2021-07-11T03:59:14.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f780aa710 0x7f6f780a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f7815a650 0x7f6f780aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f780ab3c0 0x7f6f780ab820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780aaaf0 con 0x7f6f7815a650 2021-07-11T03:59:14.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780a9510 con 0x7f6f780ab3c0 2021-07-11T03:59:14.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7805fdf0 con 0x7f6f780aa710 2021-07-11T03:59:14.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f76ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f7815a650 0x7f6f780aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f76ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f7815a650 0x7f6f780aaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46328/0 (socket says 172.21.2.104:46328) 2021-07-11T03:59:14.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f76ffd700 1 -- 172.21.2.104:0/3056512046 learned_addr learned my addr 172.21.2.104:0/3056512046 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:14.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f76ffd700 1 -- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f780aa710 msgr2=0x7f6f780a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f76ffd700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f780aa710 0x7f6f780a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f76ffd700 1 -- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f780ab3c0 msgr2=0x7f6f780ab820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f76ffd700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f780ab3c0 0x7f6f780ab820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f76ffd700 1 -- 172.21.2.104:0/3056512046 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f78060090 con 0x7f6f7815a650 2021-07-11T03:59:14.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f76ffd700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f7815a650 0x7f6f780aaed0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f6f60007000 tx=0x7f6f60008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:14.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f74ff9700 1 -- 172.21.2.104:0/3056512046 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f60015070 con 0x7f6f7815a650 2021-07-11T03:59:14.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f74ff9700 1 -- 172.21.2.104:0/3056512046 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f60000e90 con 0x7f6f7815a650 2021-07-11T03:59:14.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 -- 172.21.2.104:0/3056512046 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f78060330 con 0x7f6f7815a650 2021-07-11T03:59:14.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f74ff9700 1 -- 172.21.2.104:0/3056512046 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f60010770 con 0x7f6f7815a650 2021-07-11T03:59:14.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.698+0000 7f6f7da70700 1 -- 172.21.2.104:0/3056512046 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f780b3460 con 0x7f6f7815a650 2021-07-11T03:59:14.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.702+0000 7f6f7da70700 1 -- 172.21.2.104:0/3056512046 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f780604c0 con 0x7f6f7815a650 2021-07-11T03:59:14.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.702+0000 7f6f74ff9700 1 -- 172.21.2.104:0/3056512046 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6f600108d0 con 0x7f6f7815a650 2021-07-11T03:59:14.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.702+0000 7f6f74ff9700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f5c053330 0x7f6f5c0557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.702+0000 7f6f74ff9700 1 -- 172.21.2.104:0/3056512046 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6f6006b1f0 con 0x7f6f7815a650 2021-07-11T03:59:14.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.702+0000 7f6f777fe700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f5c053330 0x7f6f5c0557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.702+0000 7f6f777fe700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f5c053330 0x7f6f5c0557f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f6f68001370 tx=0x7f6f68015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:14.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.702+0000 7f6f74ff9700 1 -- 172.21.2.104:0/3056512046 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6f60042530 con 0x7f6f7815a650 2021-07-11T03:59:14.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 -- 172.21.2.104:0/3056512046 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f78057350 con 0x7f6f7815a650 2021-07-11T03:59:14.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f74ff9700 1 -- 172.21.2.104:0/3056512046 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f6f60045b50 con 0x7f6f7815a650 2021-07-11T03:59:14.817 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:14.817 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/140397159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":140397159},{"type":"v1","addr":"172.21.2.105:6835","nonce":140397159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/2616903361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2616903361},{"type":"v1","addr":"172.21.2.104:6834","nonce":2616903361}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:59:13.620413+0000","modified":"2021-07-11T03:59:14.624165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/611806767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":611806767},{"type":"v1","addr":"172.21.2.104:6835","nonce":611806767}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-07-11T03:59:14.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 -- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f5c053330 msgr2=0x7f6f5c0557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f5c053330 0x7f6f5c0557f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f6f68001370 tx=0x7f6f68015000).stop 2021-07-11T03:59:14.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 -- 172.21.2.104:0/3056512046 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f7815a650 msgr2=0x7f6f780aaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f7815a650 0x7f6f780aaed0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f6f60007000 tx=0x7f6f60008040).stop 2021-07-11T03:59:14.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 -- 172.21.2.104:0/3056512046 shutdown_connections 2021-07-11T03:59:14.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f780ab3c0 0x7f6f780ab820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f5c053330 0x7f6f5c0557f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f7815a650 0x7f6f780aaed0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 --2- 172.21.2.104:0/3056512046 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f780aa710 0x7f6f780a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 -- 172.21.2.104:0/3056512046 >> 172.21.2.104:0/3056512046 conn(0x7f6f780b53e0 msgr2=0x7f6f78149160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:14.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 -- 172.21.2.104:0/3056512046 shutdown_connections 2021-07-11T03:59:14.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.810+0000 7f6f7da70700 1 -- 172.21.2.104:0/3056512046 wait complete. 2021-07-11T03:59:14.821 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 32 2021-07-11T03:59:14.824 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:59:14.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 Processor -- start 2021-07-11T03:59:14.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 -- start start 2021-07-11T03:59:14.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff754067180 0x7ff75405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7540a7a90 con 0x7ff7540a76b0 2021-07-11T03:59:14.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff754147620 con 0x7ff7540b1970 2021-07-11T03:59:14.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7541478c0 con 0x7ff754067180 2021-07-11T03:59:14.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff752ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff7540a76b0 0x7ff754066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43100/0 (socket says 172.21.2.104:43100) 2021-07-11T03:59:14.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff752ffd700 1 -- 172.21.2.104:0/57061698 learned_addr learned my addr 172.21.2.104:0/57061698 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:14.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1932254594 0 0) 0x7ff7540a7a90 con 0x7ff7540a76b0 2021-07-11T03:59:14.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff734003260 con 0x7ff7540a76b0 2021-07-11T03:59:14.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4135353706 0 0) 0x7ff734003260 con 0x7ff7540a76b0 2021-07-11T03:59:14.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 >> v2:172.21.2.105:3300/0 conn(0x7ff754067180 msgr2=0x7ff75405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 --2- 172.21.2.104:0/57061698 >> v2:172.21.2.105:3300/0 conn(0x7ff754067180 0x7ff75405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 >> v1:172.21.2.105:6789/0 conn(0x7ff7540b1970 legacy=0x7ff75405fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754147b60 con 0x7ff7540a76b0 2021-07-11T03:59:14.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7ff744001660 con 0x7ff7540a76b0 2021-07-11T03:59:14.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 >> v1:172.21.2.104:6789/0 conn(0x7ff7540a76b0 legacy=0x7ff754066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 --2- 172.21.2.104:0/57061698 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff734003820 0x7ff734003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 --2- 172.21.2.104:0/57061698 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff734004500 0x7ff734004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 --2- 172.21.2.104:0/57061698 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff734005070 0x7ff73400b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7541478c0 con 0x7ff734003820 2021-07-11T03:59:14.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73400bf60 con 0x7ff734004500 2021-07-11T03:59:14.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73400c120 con 0x7ff734005070 2021-07-11T03:59:14.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff752ffd700 1 --2- 172.21.2.104:0/57061698 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff734003820 0x7ff734003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff752ffd700 1 -- 172.21.2.104:0/57061698 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff734005070 msgr2=0x7ff73400b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff752ffd700 1 --2- 172.21.2.104:0/57061698 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff734005070 0x7ff73400b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff752ffd700 1 -- 172.21.2.104:0/57061698 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff734004500 msgr2=0x7ff734004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:14.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff752ffd700 1 --2- 172.21.2.104:0/57061698 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff734004500 0x7ff734004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:14.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff752ffd700 1 -- 172.21.2.104:0/57061698 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff734003260 con 0x7ff734003820 2021-07-11T03:59:14.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7537fe700 1 -- 172.21.2.104:0/57061698 reap_dead start 2021-07-11T03:59:14.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff752ffd700 1 --2- 172.21.2.104:0/57061698 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff734003820 0x7ff734003c40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7ff75406a180 tx=0x7ff7540a8c80).ready entity=mon.0 client_cookie=18137d5aeec827a7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:14.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff7440070d0 con 0x7ff734003820 2021-07-11T03:59:14.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff744007230 con 0x7ff734003820 2021-07-11T03:59:14.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff751ffb700 1 -- 172.21.2.104:0/57061698 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff74400c8f0 con 0x7ff734003820 2021-07-11T03:59:14.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 -- 172.21.2.104:0/57061698 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff734003820 msgr2=0x7ff734003c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 --2- 172.21.2.104:0/57061698 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff734003820 0x7ff734003c40 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7ff75406a180 tx=0x7ff7540a8c80).stop 2021-07-11T03:59:14.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 -- 172.21.2.104:0/57061698 shutdown_connections 2021-07-11T03:59:14.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 --2- 172.21.2.104:0/57061698 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff734003820 0x7ff734003c40 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 -- 172.21.2.104:0/57061698 >> 172.21.2.104:0/57061698 conn(0x7ff7540b53e0 msgr2=0x7ff7540b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:14.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 -- 172.21.2.104:0/57061698 shutdown_connections 2021-07-11T03:59:14.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.870+0000 7ff7597a4700 1 -- 172.21.2.104:0/57061698 wait complete. 2021-07-11T03:59:14.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7597a4700 1 Processor -- start 2021-07-11T03:59:14.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7597a4700 1 -- start start 2021-07-11T03:59:14.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7597a4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff734003820 0x7ff7540ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7597a4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff75414cdc0 0x7ff7541447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7597a4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff754144cc0 0x7ff7540aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7597a4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7540ace90 con 0x7ff754144cc0 2021-07-11T03:59:14.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7597a4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff754145170 con 0x7ff734003820 2021-07-11T03:59:14.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7597a4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff754156550 con 0x7ff75414cdc0 2021-07-11T03:59:14.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7537fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff754144cc0 0x7ff7540aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7537fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff754144cc0 0x7ff7540aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46346/0 (socket says 172.21.2.104:46346) 2021-07-11T03:59:14.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7537fe700 1 -- 172.21.2.104:0/2461515474 learned_addr learned my addr 172.21.2.104:0/2461515474 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:14.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7527fc700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff75414cdc0 0x7ff7541447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff752ffd700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff734003820 0x7ff7540ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7537fe700 1 -- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff75414cdc0 msgr2=0x7ff7541447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7537fe700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff75414cdc0 0x7ff7541447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7537fe700 1 -- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff734003820 msgr2=0x7ff7540ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7537fe700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff734003820 0x7ff7540ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7537fe700 1 -- 172.21.2.104:0/2461515474 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75414d3e0 con 0x7ff754144cc0 2021-07-11T03:59:14.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7537fe700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff754144cc0 0x7ff7540aebf0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7ff75414d990 tx=0x7ff74c007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:14.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff74bfff700 1 -- 172.21.2.104:0/2461515474 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff74c00cec0 con 0x7ff754144cc0 2021-07-11T03:59:14.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff74bfff700 1 -- 172.21.2.104:0/2461515474 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c004850 con 0x7ff754144cc0 2021-07-11T03:59:14.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7597a4700 1 -- 172.21.2.104:0/2461515474 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75414d610 con 0x7ff754144cc0 2021-07-11T03:59:14.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7597a4700 1 -- 172.21.2.104:0/2461515474 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75414caa0 con 0x7ff754144cc0 2021-07-11T03:59:14.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff74bfff700 1 -- 172.21.2.104:0/2461515474 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff74c004bb0 con 0x7ff754144cc0 2021-07-11T03:59:14.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff74bfff700 1 -- 172.21.2.104:0/2461515474 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff74c01c070 con 0x7ff754144cc0 2021-07-11T03:59:14.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff7597a4700 1 -- 172.21.2.104:0/2461515474 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75414d7a0 con 0x7ff754144cc0 2021-07-11T03:59:14.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff74bfff700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff738053210 0x7ff7380556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:14.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff74bfff700 1 -- 172.21.2.104:0/2461515474 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff74c06a520 con 0x7ff754144cc0 2021-07-11T03:59:14.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.874+0000 7ff752ffd700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff738053210 0x7ff7380556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:14.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.878+0000 7ff752ffd700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff738053210 0x7ff7380556d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7ff744017000 tx=0x7ff744013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:14.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.878+0000 7ff74bfff700 1 -- 172.21.2.104:0/2461515474 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff74c041860 con 0x7ff754144cc0 2021-07-11T03:59:14.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.958+0000 7ff7597a4700 1 -- 172.21.2.104:0/2461515474 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff754057350 con 0x7ff754144cc0 2021-07-11T03:59:14.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.958+0000 7ff74bfff700 1 -- 172.21.2.104:0/2461515474 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16041 (secure 0 0 0) 0x7ff74c044e80 con 0x7ff754144cc0 2021-07-11T03:59:14.967 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:14.967 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":76,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:59:14.625343+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:59:11.477806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-07-11T03:59:12.643109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:59:14.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 -- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff738053210 msgr2=0x7ff7380556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff738053210 0x7ff7380556d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7ff744017000 tx=0x7ff744013000).stop 2021-07-11T03:59:14.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 -- 172.21.2.104:0/2461515474 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff754144cc0 msgr2=0x7ff7540aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:14.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff754144cc0 0x7ff7540aebf0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7ff75414d990 tx=0x7ff74c007dc0).stop 2021-07-11T03:59:14.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 -- 172.21.2.104:0/2461515474 shutdown_connections 2021-07-11T03:59:14.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff738053210 0x7ff7380556d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff754144cc0 0x7ff7540aebf0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff75414cdc0 0x7ff7541447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 --2- 172.21.2.104:0/2461515474 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff734003820 0x7ff7540ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:14.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 -- 172.21.2.104:0/2461515474 >> 172.21.2.104:0/2461515474 conn(0x7ff7540b53e0 msgr2=0x7ff75405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:14.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 -- 172.21.2.104:0/2461515474 shutdown_connections 2021-07-11T03:59:14.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:14.962+0000 7ff7597a4700 1 -- 172.21.2.104:0/2461515474 wait complete. 2021-07-11T03:59:14.974 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T03:59:15.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 Processor -- start 2021-07-11T03:59:15.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 -- start start 2021-07-11T03:59:15.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6eec0ad150 0x7f6eec0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6eec0a9da0 0x7f6eec140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eec0ac370 con 0x7f6eec0aa860 2021-07-11T03:59:15.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eec0abb50 con 0x7f6eec0a9da0 2021-07-11T03:59:15.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eec0aac40 con 0x7f6eec0ad150 2021-07-11T03:59:15.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef236e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6eec0a9da0 0x7f6eec140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef1b6d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6eec0aa860 0x7f6eec0aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43120/0 (socket says 172.21.2.104:43120) 2021-07-11T03:59:15.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef1b6d700 1 -- 172.21.2.104:0/1766273579 learned_addr learned my addr 172.21.2.104:0/1766273579 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:15.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef236e700 1 -- 172.21.2.104:0/1766273579 >> v2:172.21.2.105:3300/0 conn(0x7f6eec0ad150 msgr2=0x7f6eec0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:15.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef236e700 1 --2- 172.21.2.104:0/1766273579 >> v2:172.21.2.105:3300/0 conn(0x7f6eec0ad150 0x7f6eec0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:15.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef236e700 1 -- 172.21.2.104:0/1766273579 >> v1:172.21.2.104:6789/0 conn(0x7f6eec0aa860 legacy=0x7f6eec0aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef236e700 1 -- 172.21.2.104:0/1766273579 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec0b16b0 con 0x7f6eec0a9da0 2021-07-11T03:59:15.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef236e700 1 --2- 172.21.2.104:0/1766273579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6eec0a9da0 0x7f6eec140340 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f6ee0001110 tx=0x7f6ee40091d0).ready entity=mon.0 client_cookie=ab16d4e5238d75e1 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef0b6b700 1 -- 172.21.2.104:0/1766273579 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ee4009ea0 con 0x7f6eec0a9da0 2021-07-11T03:59:15.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef0b6b700 1 -- 172.21.2.104:0/1766273579 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee400b730 con 0x7f6eec0a9da0 2021-07-11T03:59:15.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef0b6b700 1 -- 172.21.2.104:0/1766273579 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ee400b8a0 con 0x7f6eec0a9da0 2021-07-11T03:59:15.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/1766273579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6eec0a9da0 msgr2=0x7f6eec140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 --2- 172.21.2.104:0/1766273579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6eec0a9da0 0x7f6eec140340 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f6ee0001110 tx=0x7f6ee40091d0).stop 2021-07-11T03:59:15.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/1766273579 shutdown_connections 2021-07-11T03:59:15.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 --2- 172.21.2.104:0/1766273579 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6eec0a9da0 0x7f6eec140340 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 --2- 172.21.2.104:0/1766273579 >> v2:172.21.2.105:3300/0 conn(0x7f6eec0ad150 0x7f6eec0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:15.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/1766273579 >> 172.21.2.104:0/1766273579 conn(0x7f6eec0b64d0 msgr2=0x7f6eec0b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:15.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/1766273579 shutdown_connections 2021-07-11T03:59:15.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.026+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/1766273579 wait complete. 2021-07-11T03:59:15.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef3dd1700 1 Processor -- start 2021-07-11T03:59:15.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef3dd1700 1 -- start start 2021-07-11T03:59:15.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef3dd1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6eec0a9da0 0x7f6eec13bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef3dd1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6eec0aa860 0x7f6eec143850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef3dd1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6eec0ad150 0x7f6eec0af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef3dd1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0b6180 con 0x7f6eec0a9da0 2021-07-11T03:59:15.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef3dd1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec0ad790 con 0x7f6eec0ad150 2021-07-11T03:59:15.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef3dd1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec142560 con 0x7f6eec0aa860 2021-07-11T03:59:15.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef1b6d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6eec0a9da0 0x7f6eec13bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef1b6d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6eec0a9da0 0x7f6eec13bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46358/0 (socket says 172.21.2.104:46358) 2021-07-11T03:59:15.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef1b6d700 1 -- 172.21.2.104:0/2076279249 learned_addr learned my addr 172.21.2.104:0/2076279249 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:15.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef1b6d700 1 -- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6eec0aa860 msgr2=0x7f6eec143850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef1b6d700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6eec0aa860 0x7f6eec143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:15.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef1b6d700 1 -- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6eec0ad150 msgr2=0x7f6eec0af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef1b6d700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6eec0ad150 0x7f6eec0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:15.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef1b6d700 1 -- 172.21.2.104:0/2076279249 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee4009470 con 0x7f6eec0a9da0 2021-07-11T03:59:15.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef1b6d700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6eec0a9da0 0x7f6eec13bd40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f6ee00011d0 tx=0x7f6ee0007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6edeffd700 1 -- 172.21.2.104:0/2076279249 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ee00072e0 con 0x7f6eec0a9da0 2021-07-11T03:59:15.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6edeffd700 1 -- 172.21.2.104:0/2076279249 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee0007440 con 0x7f6eec0a9da0 2021-07-11T03:59:15.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/2076279249 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eec0adfe0 con 0x7f6eec0a9da0 2021-07-11T03:59:15.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/2076279249 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eec0afbb0 con 0x7f6eec0a9da0 2021-07-11T03:59:15.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6edeffd700 1 -- 172.21.2.104:0/2076279249 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6ee0005800 con 0x7f6eec0a9da0 2021-07-11T03:59:15.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/2076279249 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eec0ad990 con 0x7f6eec0a9da0 2021-07-11T03:59:15.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6edeffd700 1 -- 172.21.2.104:0/2076279249 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6ee0010070 con 0x7f6eec0a9da0 2021-07-11T03:59:15.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6edeffd700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6ecc053260 0x7f6ecc055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6edeffd700 1 -- 172.21.2.104:0/2076279249 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6ee006ade0 con 0x7f6eec0a9da0 2021-07-11T03:59:15.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.030+0000 7f6ef136c700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6ecc053260 0x7f6ecc055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.034+0000 7f6ef136c700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6ecc053260 0x7f6ecc055720 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f6ee8002f70 tx=0x7f6ee80069f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.034+0000 7f6edeffd700 1 -- 172.21.2.104:0/2076279249 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6ee0042010 con 0x7f6eec0a9da0 2021-07-11T03:59:15.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.146+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/2076279249 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6eec0b5e90 con 0x7f6eec0a9da0 2021-07-11T03:59:15.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.150+0000 7f6edeffd700 1 -- 172.21.2.104:0/2076279249 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v21) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6ee0045630 con 0x7f6eec0a9da0 2021-07-11T03:59:15.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6ecc053260 msgr2=0x7f6ecc055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6ecc053260 0x7f6ecc055720 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f6ee8002f70 tx=0x7f6ee80069f0).stop 2021-07-11T03:59:15.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/2076279249 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6eec0a9da0 msgr2=0x7f6eec13bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6eec0a9da0 0x7f6eec13bd40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f6ee00011d0 tx=0x7f6ee0007bc0).stop 2021-07-11T03:59:15.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/2076279249 shutdown_connections 2021-07-11T03:59:15.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6eec0ad150 0x7f6eec0af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:15.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6eec0aa860 0x7f6eec143850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:15.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6ecc053260 0x7f6ecc055720 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 --2- 172.21.2.104:0/2076279249 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6eec0a9da0 0x7f6eec13bd40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/2076279249 >> 172.21.2.104:0/2076279249 conn(0x7f6eec0b64d0 msgr2=0x7f6eec140130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:15.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/2076279249 shutdown_connections 2021-07-11T03:59:15.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.154+0000 7f6ef3dd1700 1 -- 172.21.2.104:0/2076279249 wait complete. 2021-07-11T03:59:15.162 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T03:59:15.166 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T03:59:15.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.214+0000 7f6d23f87700 1 Processor -- start 2021-07-11T03:59:15.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.214+0000 7f6d23f87700 1 -- start start 2021-07-11T03:59:15.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.214+0000 7f6d23f87700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c067120 0x7f6d1c0b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.214+0000 7f6d23f87700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6d1c13fff0 0x7f6d1c140480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.214+0000 7f6d23f87700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c0b9bd0 con 0x7f6d1c1439b0 2021-07-11T03:59:15.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.214+0000 7f6d23f87700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c143080 con 0x7f6d1c13fff0 2021-07-11T03:59:15.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.214+0000 7f6d23f87700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c067500 con 0x7f6d1c067120 2021-07-11T03:59:15.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.214+0000 7f6d21522700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6d1c1439b0 0x7f6d1c0aec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43138/0 (socket says 172.21.2.104:43138) 2021-07-11T03:59:15.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.214+0000 7f6d21522700 1 -- 172.21.2.104:0/2320822960 learned_addr learned my addr 172.21.2.104:0/2320822960 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:15.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.214+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060097838 0 0) 0x7f6d10001110 con 0x7f6d1c1439b0 2021-07-11T03:59:15.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.214+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d04003260 con 0x7f6d1c1439b0 2021-07-11T03:59:15.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d22524700 1 --2- 172.21.2.104:0/2320822960 >> v2:172.21.2.105:3300/0 conn(0x7f6d1c13fff0 0x7f6d1c140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21d23700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c067120 0x7f6d1c0b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3501911789 0 0) 0x7f6d04003260 con 0x7f6d1c1439b0 2021-07-11T03:59:15.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c067120 msgr2=0x7f6d1c0b96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:15.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c067120 0x7f6d1c0b96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 >> v2:172.21.2.105:3300/0 conn(0x7f6d1c13fff0 msgr2=0x7f6d1c140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:15.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 --2- 172.21.2.104:0/2320822960 >> v2:172.21.2.105:3300/0 conn(0x7f6d1c13fff0 0x7f6d1c140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c0af4d0 con 0x7f6d1c1439b0 2021-07-11T03:59:15.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f6d100017f0 con 0x7f6d1c1439b0 2021-07-11T03:59:15.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d22524700 1 --2- 172.21.2.104:0/2320822960 >> v2:172.21.2.105:3300/0 conn(0x7f6d1c13fff0 0x7f6d1c140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:59:15.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 >> v1:172.21.2.104:6789/0 conn(0x7f6d1c1439b0 legacy=0x7f6d1c0aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c067120 0x7f6d040037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d04003e00 0x7f6d04004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d040049f0 0x7f6d04004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c067500 con 0x7f6d04003e00 2021-07-11T03:59:15.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0400b860 con 0x7f6d040049f0 2021-07-11T03:59:15.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0400ba20 con 0x7f6d1c067120 2021-07-11T03:59:15.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21522700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d04003e00 0x7f6d04004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21d23700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c067120 0x7f6d040037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d22524700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d040049f0 0x7f6d04004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21522700 1 -- 172.21.2.104:0/2320822960 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c067120 msgr2=0x7f6d040037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:15.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21522700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c067120 0x7f6d040037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21522700 1 -- 172.21.2.104:0/2320822960 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d040049f0 msgr2=0x7f6d04004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:15.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21522700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d040049f0 0x7f6d04004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21522700 1 -- 172.21.2.104:0/2320822960 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d04003260 con 0x7f6d04003e00 2021-07-11T03:59:15.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21522700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d04003e00 0x7f6d04004500 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f6d10001500 tx=0x7f6d1c0b6870).ready entity=mon.0 client_cookie=c418c056d9973623 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d100070d0 con 0x7f6d04003e00 2021-07-11T03:59:15.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d10007230 con 0x7f6d04003e00 2021-07-11T03:59:15.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d20d21700 1 -- 172.21.2.104:0/2320822960 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d1000c9c0 con 0x7f6d04003e00 2021-07-11T03:59:15.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 -- 172.21.2.104:0/2320822960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d04003e00 msgr2=0x7f6d04004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 --2- 172.21.2.104:0/2320822960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d04003e00 0x7f6d04004500 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f6d10001500 tx=0x7f6d1c0b6870).stop 2021-07-11T03:59:15.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d22524700 1 -- 172.21.2.104:0/2320822960 reap_dead start 2021-07-11T03:59:15.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 -- 172.21.2.104:0/2320822960 shutdown_connections 2021-07-11T03:59:15.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 -- 172.21.2.104:0/2320822960 >> 172.21.2.104:0/2320822960 conn(0x7f6d1c05f9f0 msgr2=0x7f6d1c0616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:15.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 -- 172.21.2.104:0/2320822960 shutdown_connections 2021-07-11T03:59:15.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 -- 172.21.2.104:0/2320822960 wait complete. 2021-07-11T03:59:15.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 Processor -- start 2021-07-11T03:59:15.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 -- start start 2021-07-11T03:59:15.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d1c13d9a0 0x7f6d1c0b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d1c142470 0x7f6d1c0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c0b8900 0x7f6d1c159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c142250 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c142a80 con 0x7f6d1c142470 2021-07-11T03:59:15.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0b8310 con 0x7f6d1c0b8900 2021-07-11T03:59:15.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21d23700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d1c13d9a0 0x7f6d1c0b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21d23700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d1c13d9a0 0x7f6d1c0b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46378/0 (socket says 172.21.2.104:46378) 2021-07-11T03:59:15.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21d23700 1 -- 172.21.2.104:0/4145555583 learned_addr learned my addr 172.21.2.104:0/4145555583 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:15.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d22524700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c0b8900 0x7f6d1c159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21522700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d1c142470 0x7f6d1c0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21d23700 1 -- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c0b8900 msgr2=0x7f6d1c159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21d23700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c0b8900 0x7f6d1c159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21d23700 1 -- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d1c142470 msgr2=0x7f6d1c0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21d23700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d1c142470 0x7f6d1c0b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21d23700 1 -- 172.21.2.104:0/4145555583 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c05c8f0 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d21d23700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d1c13d9a0 0x7f6d1c0b2f50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f6d1c05c790 tx=0x7f6d18008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d0effd700 1 -- 172.21.2.104:0/4145555583 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d18013800 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d0effd700 1 -- 172.21.2.104:0/4145555583 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d18000e90 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d0effd700 1 -- 172.21.2.104:0/4145555583 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6d18013b50 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 -- 172.21.2.104:0/4145555583 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c0b8570 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.218+0000 7f6d23f87700 1 -- 172.21.2.104:0/4145555583 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c0b3930 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.222+0000 7f6d23f87700 1 -- 172.21.2.104:0/4145555583 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c057350 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.222+0000 7f6d0effd700 1 -- 172.21.2.104:0/4145555583 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6d18013cb0 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.222+0000 7f6d0effd700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6cfc053180 0x7f6cfc055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.222+0000 7f6d0effd700 1 -- 172.21.2.104:0/4145555583 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6d1806b700 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.222+0000 7f6d21522700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6cfc053180 0x7f6cfc055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.222+0000 7f6d21522700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6cfc053180 0x7f6cfc055640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f6d10017000 tx=0x7f6d10015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.222+0000 7f6d0effd700 1 -- 172.21.2.104:0/4145555583 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6d18042930 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.334+0000 7f6d23f87700 1 -- 172.21.2.104:0/4145555583 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f6d1c0ada40 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.338+0000 7f6d0effd700 1 -- 172.21.2.104:0/4145555583 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6d18017020 con 0x7f6d1c13d9a0 2021-07-11T03:59:15.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 -- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6cfc053180 msgr2=0x7f6cfc055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6cfc053180 0x7f6cfc055640 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f6d10017000 tx=0x7f6d10015000).stop 2021-07-11T03:59:15.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 -- 172.21.2.104:0/4145555583 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d1c13d9a0 msgr2=0x7f6d1c0b2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d1c13d9a0 0x7f6d1c0b2f50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7f6d1c05c790 tx=0x7f6d18008040).stop 2021-07-11T03:59:15.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 -- 172.21.2.104:0/4145555583 shutdown_connections 2021-07-11T03:59:15.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6d1c0b8900 0x7f6d1c159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6d1c142470 0x7f6d1c0b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6cfc053180 0x7f6cfc055640 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 --2- 172.21.2.104:0/4145555583 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6d1c13d9a0 0x7f6d1c0b2f50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 -- 172.21.2.104:0/4145555583 >> 172.21.2.104:0/4145555583 conn(0x7f6d1c05f9f0 msgr2=0x7f6d1c1457b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:15.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 -- 172.21.2.104:0/4145555583 shutdown_connections 2021-07-11T03:59:15.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.342+0000 7f6d23f87700 1 -- 172.21.2.104:0/4145555583 wait complete. 2021-07-11T03:59:15.350 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T03:59:15.353 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:59:15.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ff53e700 1 Processor -- start 2021-07-11T03:59:15.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ff53e700 1 -- start start 2021-07-11T03:59:15.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ff53e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb8f80a76b0 0x7fb8f8066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ff53e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80b1970 0x7fb8f8066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ff53e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f8067450 con 0x7fb8f80b0060 2021-07-11T03:59:15.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ff53e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f80a7a90 con 0x7fb8f80a76b0 2021-07-11T03:59:15.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ff53e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f8147620 con 0x7fb8f80b1970 2021-07-11T03:59:15.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8fdadb700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb8f80b0060 0x7fb8f805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43154/0 (socket says 172.21.2.104:43154) 2021-07-11T03:59:15.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8fdadb700 1 -- 172.21.2.104:0/979675158 learned_addr learned my addr 172.21.2.104:0/979675158 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:15.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608181558 0 0) 0x7fb8f8067450 con 0x7fb8f80b0060 2021-07-11T03:59:15.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8e0003260 con 0x7fb8f80b0060 2021-07-11T03:59:15.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8fcad9700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80b1970 0x7fb8f8066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8fd2da700 1 --2- 172.21.2.104:0/979675158 >> v2:172.21.2.105:3300/0 conn(0x7fb8f80a76b0 0x7fb8f8066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1975045503 0 0) 0x7fb8e0003260 con 0x7fb8f80b0060 2021-07-11T03:59:15.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80b1970 msgr2=0x7fb8f8066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:15.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ebfff700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80b1970 0x7fb8f8066f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 >> v2:172.21.2.105:3300/0 conn(0x7fb8f80a76b0 msgr2=0x7fb8f8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:15.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ebfff700 1 --2- 172.21.2.104:0/979675158 >> v2:172.21.2.105:3300/0 conn(0x7fb8f80a76b0 0x7fb8f8066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f81478c0 con 0x7fb8f80b0060 2021-07-11T03:59:15.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fb8f00019e0 con 0x7fb8f80b0060 2021-07-11T03:59:15.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 >> v1:172.21.2.104:6789/0 conn(0x7fb8f80b0060 legacy=0x7fb8f805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.398+0000 7fb8ebfff700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8e00037d0 0x7fb8e0003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ebfff700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80b1970 0x7fb8e0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ebfff700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8e00049f0 0x7fb8e0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f80a7a90 con 0x7fb8e00049f0 2021-07-11T03:59:15.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f8147620 con 0x7fb8e00037d0 2021-07-11T03:59:15.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e000b880 con 0x7fb8f80b1970 2021-07-11T03:59:15.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fdadb700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8e00049f0 0x7fb8e0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fd2da700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8e00037d0 0x7fb8e0003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fcad9700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80b1970 0x7fb8e0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fdadb700 1 -- 172.21.2.104:0/979675158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80b1970 msgr2=0x7fb8e0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:15.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fdadb700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80b1970 0x7fb8e0004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fdadb700 1 -- 172.21.2.104:0/979675158 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8e00037d0 msgr2=0x7fb8e0003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:15.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fdadb700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8e00037d0 0x7fb8e0003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fdadb700 1 -- 172.21.2.104:0/979675158 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e0003260 con 0x7fb8e00049f0 2021-07-11T03:59:15.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fdadb700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8e00049f0 0x7fb8e0004e70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fb8f00016f0 tx=0x7fb8f0002100).ready entity=mon.0 client_cookie=bfb16d97b46147c2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8f000cb80 con 0x7fb8e00049f0 2021-07-11T03:59:15.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f000cce0 con 0x7fb8e00049f0 2021-07-11T03:59:15.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ebfff700 1 -- 172.21.2.104:0/979675158 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8f0005b80 con 0x7fb8e00049f0 2021-07-11T03:59:15.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 -- 172.21.2.104:0/979675158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8e00049f0 msgr2=0x7fb8e0004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 --2- 172.21.2.104:0/979675158 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8e00049f0 0x7fb8e0004e70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7fb8f00016f0 tx=0x7fb8f0002100).stop 2021-07-11T03:59:15.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fdadb700 1 -- 172.21.2.104:0/979675158 reap_dead start 2021-07-11T03:59:15.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 -- 172.21.2.104:0/979675158 shutdown_connections 2021-07-11T03:59:15.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 -- 172.21.2.104:0/979675158 >> 172.21.2.104:0/979675158 conn(0x7fb8f80b53e0 msgr2=0x7fb8f80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:15.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 -- 172.21.2.104:0/979675158 shutdown_connections 2021-07-11T03:59:15.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 -- 172.21.2.104:0/979675158 wait complete. 2021-07-11T03:59:15.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 Processor -- start 2021-07-11T03:59:15.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 -- start start 2021-07-11T03:59:15.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80adf60 0x7fb8f80ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8f8159ae0 0x7fb8f8159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8f8144600 0x7fb8f8144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f814d690 con 0x7fb8f8159ae0 2021-07-11T03:59:15.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f814d880 con 0x7fb8f8144600 2021-07-11T03:59:15.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f814c940 con 0x7fb8f80adf60 2021-07-11T03:59:15.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fcad9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8f8159ae0 0x7fb8f8159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fcad9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8f8159ae0 0x7fb8f8159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46402/0 (socket says 172.21.2.104:46402) 2021-07-11T03:59:15.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fcad9700 1 -- 172.21.2.104:0/204873247 learned_addr learned my addr 172.21.2.104:0/204873247 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:15.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fd2da700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80adf60 0x7fb8f80ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fcad9700 1 -- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80adf60 msgr2=0x7fb8f80ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fcad9700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80adf60 0x7fb8f80ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fdadb700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8f8144600 0x7fb8f8144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fcad9700 1 -- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8f8144600 msgr2=0x7fb8f8144a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fcad9700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8f8144600 0x7fb8f8144a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fcad9700 1 -- 172.21.2.104:0/204873247 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f814d3f0 con 0x7fb8f8159ae0 2021-07-11T03:59:15.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fd2da700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80adf60 0x7fb8f80ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:59:15.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fdadb700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8f8144600 0x7fb8f8144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T03:59:15.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8fcad9700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8f8159ae0 0x7fb8f8159f60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fb8ec0030b0 tx=0x7fb8ec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ea7fc700 1 -- 172.21.2.104:0/204873247 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8ec013800 con 0x7fb8f8159ae0 2021-07-11T03:59:15.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ea7fc700 1 -- 172.21.2.104:0/204873247 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec000e90 con 0x7fb8f8159ae0 2021-07-11T03:59:15.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ea7fc700 1 -- 172.21.2.104:0/204873247 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb8ec013b50 con 0x7fb8f8159ae0 2021-07-11T03:59:15.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 -- 172.21.2.104:0/204873247 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f814cda0 con 0x7fb8f8159ae0 2021-07-11T03:59:15.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.402+0000 7fb8ff53e700 1 -- 172.21.2.104:0/204873247 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f815a450 con 0x7fb8f8159ae0 2021-07-11T03:59:15.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.406+0000 7fb8ff53e700 1 -- 172.21.2.104:0/204873247 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f8145110 con 0x7fb8f8159ae0 2021-07-11T03:59:15.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.406+0000 7fb8ea7fc700 1 -- 172.21.2.104:0/204873247 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb8ec013cb0 con 0x7fb8f8159ae0 2021-07-11T03:59:15.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.406+0000 7fb8ea7fc700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8d8053180 0x7fb8d8055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.406+0000 7fb8ea7fc700 1 -- 172.21.2.104:0/204873247 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb8ec06b690 con 0x7fb8f8159ae0 2021-07-11T03:59:15.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.406+0000 7fb8fd2da700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8d8053180 0x7fb8d8055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.406+0000 7fb8fd2da700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8d8053180 0x7fb8d8055640 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fb8f400c000 tx=0x7fb8f400b040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.406+0000 7fb8ea7fc700 1 -- 172.21.2.104:0/204873247 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb8f8145110 con 0x7fb8f8159ae0 2021-07-11T03:59:15.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.518+0000 7fb8ff53e700 1 -- 172.21.2.104:0/204873247 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8f8145110 con 0x7fb8f8159ae0 2021-07-11T03:59:15.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.518+0000 7fb8ea7fc700 1 -- 172.21.2.104:0/204873247 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2487 (secure 0 0 0) 0x7fb8f8145110 con 0x7fb8f8159ae0 2021-07-11T03:59:15.528 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:15.528 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/140397159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":140397159},{"type":"v1","addr":"172.21.2.105:6835","nonce":140397159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/2616903361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2616903361},{"type":"v1","addr":"172.21.2.104:6834","nonce":2616903361}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:59:13.620413+0000","modified":"2021-07-11T03:59:14.624165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/611806767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":611806767},{"type":"v1","addr":"172.21.2.104:6835","nonce":611806767}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-07-11T03:59:15.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 -- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8d8053180 msgr2=0x7fb8d8055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8d8053180 0x7fb8d8055640 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fb8f400c000 tx=0x7fb8f400b040).stop 2021-07-11T03:59:15.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 -- 172.21.2.104:0/204873247 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8f8159ae0 msgr2=0x7fb8f8159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8f8159ae0 0x7fb8f8159f60 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fb8ec0030b0 tx=0x7fb8ec008040).stop 2021-07-11T03:59:15.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 -- 172.21.2.104:0/204873247 shutdown_connections 2021-07-11T03:59:15.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb8f8144600 0x7fb8f8144a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb8d8053180 0x7fb8d8055640 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb8f8159ae0 0x7fb8f8159f60 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 --2- 172.21.2.104:0/204873247 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb8f80adf60 0x7fb8f80ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 -- 172.21.2.104:0/204873247 >> 172.21.2.104:0/204873247 conn(0x7fb8f80b53e0 msgr2=0x7fb8f8066740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:15.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 -- 172.21.2.104:0/204873247 shutdown_connections 2021-07-11T03:59:15.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.522+0000 7fb8ff53e700 1 -- 172.21.2.104:0/204873247 wait complete. 2021-07-11T03:59:15.536 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 32 2021-07-11T03:59:15.840 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:59:13.620413+0000', 'modified': '2021-07-11T03:59:14.624165+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6246}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6246': {'gid': 6246, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/611806767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 611806767}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 611806767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:59:15.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:59:15.841 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:59:15.906 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":6246,"want_state":"up:active","state":"up:active","rank_uptime":2.2764775959999999,"mdsmap_epoch":32,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":6.4759601079999998} 2021-07-11T03:59:15.907 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 6246, "mdsmap_epoch": 32, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 2.276477596, "state": "up:active", "uptime": 6.475960108, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:59:15.908 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:59:15.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecb4a58700 1 Processor -- start 2021-07-11T03:59:15.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecb4a58700 1 -- start start 2021-07-11T03:59:15.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecb4a58700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b7810 0x7fecb01445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecb4a58700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fecb0144a90 0x7fecb0064f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecb4a58700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb00b31d0 con 0x7fecb00b1170 2021-07-11T03:59:15.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecb4a58700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb0144ef0 con 0x7fecb0144a90 2021-07-11T03:59:15.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecb4a58700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00b7bf0 con 0x7fecb00b7810 2021-07-11T03:59:15.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecae59c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b7810 0x7fecb01445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecaed9d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fecb0144a90 0x7fecb0064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecaed9d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fecb0144a90 0x7fecb0064f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:53968/0 (socket says 172.21.2.104:53968) 2021-07-11T03:59:15.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecaed9d700 1 -- 172.21.2.104:0/2796628110 learned_addr learned my addr 172.21.2.104:0/2796628110 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:15.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecae59c700 1 -- 172.21.2.104:0/2796628110 >> v2:172.21.2.105:3300/0 conn(0x7fecb0144a90 msgr2=0x7fecb0064f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:15.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecae59c700 1 --2- 172.21.2.104:0/2796628110 >> v2:172.21.2.105:3300/0 conn(0x7fecb0144a90 0x7fecb0064f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecae59c700 1 -- 172.21.2.104:0/2796628110 >> v1:172.21.2.105:6789/0 conn(0x7fecb00b1170 legacy=0x7fecb0064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.954+0000 7fecae59c700 1 -- 172.21.2.104:0/2796628110 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00675c0 con 0x7fecb00b7810 2021-07-11T03:59:15.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecae59c700 1 --2- 172.21.2.104:0/2796628110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b7810 0x7fecb01445a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fec98002f70 tx=0x7fec98006e80).ready entity=mon.2 client_cookie=45330c5ea0173c65 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecad59a700 1 -- 172.21.2.104:0/2796628110 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec98012070 con 0x7fecb00b7810 2021-07-11T03:59:15.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecad59a700 1 -- 172.21.2.104:0/2796628110 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec98000e90 con 0x7fecb00b7810 2021-07-11T03:59:15.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecad59a700 1 -- 172.21.2.104:0/2796628110 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fec9800f7f0 con 0x7fecb00b7810 2021-07-11T03:59:15.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 -- 172.21.2.104:0/2796628110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b7810 msgr2=0x7fecb01445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 --2- 172.21.2.104:0/2796628110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b7810 0x7fecb01445a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fec98002f70 tx=0x7fec98006e80).stop 2021-07-11T03:59:15.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 -- 172.21.2.104:0/2796628110 shutdown_connections 2021-07-11T03:59:15.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 --2- 172.21.2.104:0/2796628110 >> v2:172.21.2.105:3300/0 conn(0x7fecb0144a90 0x7fecb0064f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 --2- 172.21.2.104:0/2796628110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b7810 0x7fecb01445a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 -- 172.21.2.104:0/2796628110 >> 172.21.2.104:0/2796628110 conn(0x7fecb005f6b0 msgr2=0x7fecb005fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:15.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 -- 172.21.2.104:0/2796628110 shutdown_connections 2021-07-11T03:59:15.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 -- 172.21.2.104:0/2796628110 wait complete. 2021-07-11T03:59:15.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 Processor -- start 2021-07-11T03:59:15.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 -- start start 2021-07-11T03:59:15.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b1170 0x7fecb014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fecb00b7810 0x7fecb00abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fecb0144a90 0x7fecb00ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00b0290 con 0x7fecb00b7810 2021-07-11T03:59:15.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb0062290 con 0x7fecb0144a90 2021-07-11T03:59:15.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecb4a58700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00aa1a0 con 0x7fecb00b1170 2021-07-11T03:59:15.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecadd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fecb00b7810 0x7fecb00abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecadd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fecb00b7810 0x7fecb00abfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46414/0 (socket says 172.21.2.104:46414) 2021-07-11T03:59:15.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecadd9b700 1 -- 172.21.2.104:0/1202985143 learned_addr learned my addr 172.21.2.104:0/1202985143 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:15.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecae59c700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b1170 0x7fecb014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecaed9d700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fecb0144a90 0x7fecb00ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecadd9b700 1 -- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b1170 msgr2=0x7fecb014e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecadd9b700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b1170 0x7fecb014e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecadd9b700 1 -- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fecb0144a90 msgr2=0x7fecb00ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:15.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecadd9b700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fecb0144a90 0x7fecb00ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:15.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecadd9b700 1 -- 172.21.2.104:0/1202985143 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9800d590 con 0x7fecb00b7810 2021-07-11T03:59:15.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecae59c700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b1170 0x7fecb014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T03:59:15.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7fecadd9b700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fecb00b7810 0x7fecb00abfc0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7fecb00afbf0 tx=0x7feca0007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.958+0000 7feca77fe700 1 -- 172.21.2.104:0/1202985143 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feca000ce20 con 0x7fecb00b7810 2021-07-11T03:59:15.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.962+0000 7feca77fe700 1 -- 172.21.2.104:0/1202985143 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca000f470 con 0x7fecb00b7810 2021-07-11T03:59:15.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.962+0000 7fecb4a58700 1 -- 172.21.2.104:0/1202985143 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb00aa440 con 0x7fecb00b7810 2021-07-11T03:59:15.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.962+0000 7feca77fe700 1 -- 172.21.2.104:0/1202985143 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7feca000f7d0 con 0x7fecb00b7810 2021-07-11T03:59:15.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.962+0000 7fecb4a58700 1 -- 172.21.2.104:0/1202985143 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb014e690 con 0x7fecb00b7810 2021-07-11T03:59:15.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.962+0000 7fecb4a58700 1 -- 172.21.2.104:0/1202985143 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb00b4ea0 con 0x7fecb00b7810 2021-07-11T03:59:15.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.962+0000 7feca77fe700 1 -- 172.21.2.104:0/1202985143 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7feca0019030 con 0x7fecb00b7810 2021-07-11T03:59:15.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.962+0000 7feca77fe700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fec940531f0 0x7fec940556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:15.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.962+0000 7feca77fe700 1 -- 172.21.2.104:0/1202985143 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7feca006d3a0 con 0x7fecb00b7810 2021-07-11T03:59:15.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.962+0000 7fecae59c700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fec940531f0 0x7fec940556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:15.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.962+0000 7fecae59c700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fec940531f0 0x7fec940556b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fec9801b000 tx=0x7fecb0066a60).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:15.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:15.966+0000 7feca77fe700 1 -- 172.21.2.104:0/1202985143 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7feca0041bf0 con 0x7fecb00b7810 2021-07-11T03:59:16.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.046+0000 7fecb4a58700 1 -- 172.21.2.104:0/1202985143 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecb0057350 con 0x7fecb00b7810 2021-07-11T03:59:16.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.046+0000 7feca77fe700 1 -- 172.21.2.104:0/1202985143 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16041 (secure 0 0 0) 0x7feca0045210 con 0x7fecb00b7810 2021-07-11T03:59:16.053 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:16.054 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":76,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:59:14.625343+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:59:11.477806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-07-11T03:59:12.643109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:59:16.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 -- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fec940531f0 msgr2=0x7fec940556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fec940531f0 0x7fec940556b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7fec9801b000 tx=0x7fecb0066a60).stop 2021-07-11T03:59:16.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 -- 172.21.2.104:0/1202985143 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fecb00b7810 msgr2=0x7fecb00abfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fecb00b7810 0x7fecb00abfc0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7fecb00afbf0 tx=0x7feca0007c50).stop 2021-07-11T03:59:16.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 -- 172.21.2.104:0/1202985143 shutdown_connections 2021-07-11T03:59:16.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fecb0144a90 0x7fecb00ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fec940531f0 0x7fec940556b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fecb00b7810 0x7fecb00abfc0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 --2- 172.21.2.104:0/1202985143 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fecb00b1170 0x7fecb014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 -- 172.21.2.104:0/1202985143 >> 172.21.2.104:0/1202985143 conn(0x7fecb005f6b0 msgr2=0x7fecb00b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:16.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 -- 172.21.2.104:0/1202985143 shutdown_connections 2021-07-11T03:59:16.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.050+0000 7fecb4a58700 1 -- 172.21.2.104:0/1202985143 wait complete. 2021-07-11T03:59:16.061 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:59:11.477806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:59:12.643109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:59:16.062 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:59:16.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 Processor -- start 2021-07-11T03:59:16.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 -- start start 2021-07-11T03:59:16.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa260146950 0x7fa260140230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa2600b4690 0x7fa2600b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa260143b60 con 0x7fa2600b4180 2021-07-11T03:59:16.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa260147510 con 0x7fa2600b4690 2021-07-11T03:59:16.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa260143e00 con 0x7fa260146950 2021-07-11T03:59:16.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa264d7e700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fa2600b4180 0x7fa260140720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43188/0 (socket says 172.21.2.104:43188) 2021-07-11T03:59:16.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa264d7e700 1 -- 172.21.2.104:0/412000586 learned_addr learned my addr 172.21.2.104:0/412000586 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:16.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa265d80700 1 --2- 172.21.2.104:0/412000586 >> v2:172.21.2.104:3300/0 conn(0x7fa2600b4690 0x7fa2600b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa265d80700 1 -- 172.21.2.104:0/412000586 >> v2:172.21.2.105:3300/0 conn(0x7fa260146950 msgr2=0x7fa260140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:16.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa265d80700 1 --2- 172.21.2.104:0/412000586 >> v2:172.21.2.105:3300/0 conn(0x7fa260146950 0x7fa260140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa265d80700 1 -- 172.21.2.104:0/412000586 >> v1:172.21.2.104:6789/0 conn(0x7fa2600b4180 legacy=0x7fa260140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa265d80700 1 -- 172.21.2.104:0/412000586 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600b0c90 con 0x7fa2600b4690 2021-07-11T03:59:16.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa265d80700 1 --2- 172.21.2.104:0/412000586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa2600b4690 0x7fa2600b4ed0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7fa2580032f0 tx=0x7fa2580091d0).ready entity=mon.0 client_cookie=e90f1b0399345ab5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa253fff700 1 -- 172.21.2.104:0/412000586 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa258009ea0 con 0x7fa2600b4690 2021-07-11T03:59:16.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa253fff700 1 -- 172.21.2.104:0/412000586 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25800b870 con 0x7fa2600b4690 2021-07-11T03:59:16.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa253fff700 1 -- 172.21.2.104:0/412000586 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa25800b9e0 con 0x7fa2600b4690 2021-07-11T03:59:16.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 -- 172.21.2.104:0/412000586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa2600b4690 msgr2=0x7fa2600b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 --2- 172.21.2.104:0/412000586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa2600b4690 0x7fa2600b4ed0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7fa2580032f0 tx=0x7fa2580091d0).stop 2021-07-11T03:59:16.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 -- 172.21.2.104:0/412000586 shutdown_connections 2021-07-11T03:59:16.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 --2- 172.21.2.104:0/412000586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa2600b4690 0x7fa2600b4ed0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 --2- 172.21.2.104:0/412000586 >> v2:172.21.2.105:3300/0 conn(0x7fa260146950 0x7fa260140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 -- 172.21.2.104:0/412000586 >> 172.21.2.104:0/412000586 conn(0x7fa26005c490 msgr2=0x7fa26005c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:16.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 -- 172.21.2.104:0/412000586 shutdown_connections 2021-07-11T03:59:16.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 -- 172.21.2.104:0/412000586 wait complete. 2021-07-11T03:59:16.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 Processor -- start 2021-07-11T03:59:16.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.110+0000 7fa2677e3700 1 -- start start 2021-07-11T03:59:16.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2677e3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa2600b4180 0x7fa2600b9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2677e3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa2600b4690 0x7fa2600b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2677e3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa260146950 0x7fa2600b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2677e3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600b75f0 con 0x7fa260146950 2021-07-11T03:59:16.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2677e3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600b69c0 con 0x7fa2600b4180 2021-07-11T03:59:16.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2677e3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2600b6c60 con 0x7fa2600b4690 2021-07-11T03:59:16.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa265d80700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa260146950 0x7fa2600b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa265d80700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa260146950 0x7fa2600b7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46428/0 (socket says 172.21.2.104:46428) 2021-07-11T03:59:16.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa265d80700 1 -- 172.21.2.104:0/4237026633 learned_addr learned my addr 172.21.2.104:0/4237026633 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:16.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa265d80700 1 -- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa2600b4690 msgr2=0x7fa2600b7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa265d80700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa2600b4690 0x7fa2600b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa265d80700 1 -- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa2600b4180 msgr2=0x7fa2600b9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa265d80700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa2600b4180 0x7fa2600b9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa265d80700 1 -- 172.21.2.104:0/4237026633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa258009470 con 0x7fa260146950 2021-07-11T03:59:16.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa265d80700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa260146950 0x7fa2600b7f10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7fa2580032f0 tx=0x7fa258016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2527fc700 1 -- 172.21.2.104:0/4237026633 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa258021440 con 0x7fa260146950 2021-07-11T03:59:16.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2527fc700 1 -- 172.21.2.104:0/4237026633 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2580215a0 con 0x7fa260146950 2021-07-11T03:59:16.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2677e3700 1 -- 172.21.2.104:0/4237026633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2600b6f00 con 0x7fa260146950 2021-07-11T03:59:16.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2677e3700 1 -- 172.21.2.104:0/4237026633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26013ef00 con 0x7fa260146950 2021-07-11T03:59:16.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2527fc700 1 -- 172.21.2.104:0/4237026633 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa258021900 con 0x7fa260146950 2021-07-11T03:59:16.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2677e3700 1 -- 172.21.2.104:0/4237026633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2600b7760 con 0x7fa260146950 2021-07-11T03:59:16.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2527fc700 1 -- 172.21.2.104:0/4237026633 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa258027020 con 0x7fa260146950 2021-07-11T03:59:16.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2527fc700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2400532b0 0x7fa240055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa2527fc700 1 -- 172.21.2.104:0/4237026633 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa25806c960 con 0x7fa260146950 2021-07-11T03:59:16.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.114+0000 7fa26557f700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2400532b0 0x7fa240055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.118+0000 7fa26557f700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2400532b0 0x7fa240055770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fa254002f70 tx=0x7fa2540069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.118+0000 7fa2527fc700 1 -- 172.21.2.104:0/4237026633 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa2600b7760 con 0x7fa260146950 2021-07-11T03:59:16.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 -- 172.21.2.104:0/4237026633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa260057350 con 0x7fa260146950 2021-07-11T03:59:16.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2527fc700 1 -- 172.21.2.104:0/4237026633 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa2580150b0 con 0x7fa260146950 2021-07-11T03:59:16.205 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:59:16.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 -- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2400532b0 msgr2=0x7fa240055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2400532b0 0x7fa240055770 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fa254002f70 tx=0x7fa2540069e0).stop 2021-07-11T03:59:16.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 -- 172.21.2.104:0/4237026633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa260146950 msgr2=0x7fa2600b7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa260146950 0x7fa2600b7f10 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7fa2580032f0 tx=0x7fa258016000).stop 2021-07-11T03:59:16.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 -- 172.21.2.104:0/4237026633 shutdown_connections 2021-07-11T03:59:16.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa2400532b0 0x7fa240055770 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa260146950 0x7fa2600b7f10 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa2600b4690 0x7fa2600b7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 --2- 172.21.2.104:0/4237026633 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa2600b4180 0x7fa2600b9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 -- 172.21.2.104:0/4237026633 >> 172.21.2.104:0/4237026633 conn(0x7fa26005c490 msgr2=0x7fa2600b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:16.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 -- 172.21.2.104:0/4237026633 shutdown_connections 2021-07-11T03:59:16.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.198+0000 7fa2677e3700 1 -- 172.21.2.104:0/4237026633 wait complete. 2021-07-11T03:59:16.212 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:59:16.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff061da2700 1 Processor -- start 2021-07-11T03:59:16.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff061da2700 1 -- start start 2021-07-11T03:59:16.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff061da2700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff05c0b05b0 0x7ff05c0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff061da2700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05c143920 con 0x7ff05c0b3710 2021-07-11T03:59:16.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff061da2700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05c143bc0 con 0x7ff05c0a9c10 2021-07-11T03:59:16.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff061da2700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05c143e60 con 0x7ff05c0b05b0 2021-07-11T03:59:16.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05affd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff05c0b3710 0x7ff05c13d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43206/0 (socket says 172.21.2.104:43206) 2021-07-11T03:59:16.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05affd700 1 -- 172.21.2.104:0/11484324 learned_addr learned my addr 172.21.2.104:0/11484324 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:16.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129197099 0 0) 0x7ff05c143920 con 0x7ff05c0b3710 2021-07-11T03:59:16.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff03c003260 con 0x7ff05c0b3710 2021-07-11T03:59:16.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05b7fe700 1 --2- 172.21.2.104:0/11484324 >> v2:172.21.2.105:3300/0 conn(0x7ff05c0b05b0 0x7ff05c0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 921691730 0 0) 0x7ff03c003260 con 0x7ff05c0b3710 2021-07-11T03:59:16.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 >> v2:172.21.2.105:3300/0 conn(0x7ff05c0b05b0 msgr2=0x7ff05c0b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:16.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 --2- 172.21.2.104:0/11484324 >> v2:172.21.2.105:3300/0 conn(0x7ff05c0b05b0 0x7ff05c0b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 >> v1:172.21.2.105:6789/0 conn(0x7ff05c0a9c10 legacy=0x7ff05c0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:16.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c144100 con 0x7ff05c0b3710 2021-07-11T03:59:16.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7ff044001660 con 0x7ff05c0b3710 2021-07-11T03:59:16.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 >> v1:172.21.2.104:6789/0 conn(0x7ff05c0b3710 legacy=0x7ff05c13d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 --2- 172.21.2.104:0/11484324 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff03c003820 0x7ff03c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 --2- 172.21.2.104:0/11484324 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff03c004500 0x7ff03c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 --2- 172.21.2.104:0/11484324 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff03c004f70 0x7ff03c00d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c143e60 con 0x7ff03c004f70 2021-07-11T03:59:16.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c00e100 con 0x7ff03c003820 2021-07-11T03:59:16.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c00e2c0 con 0x7ff03c004500 2021-07-11T03:59:16.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05bfff700 1 --2- 172.21.2.104:0/11484324 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff03c004f70 0x7ff03c00d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05bfff700 1 -- 172.21.2.104:0/11484324 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff03c004500 msgr2=0x7ff03c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:16.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05bfff700 1 --2- 172.21.2.104:0/11484324 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff03c004500 0x7ff03c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05bfff700 1 -- 172.21.2.104:0/11484324 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff03c003820 msgr2=0x7ff03c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:16.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05bfff700 1 --2- 172.21.2.104:0/11484324 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff03c003820 0x7ff03c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.258+0000 7ff05bfff700 1 -- 172.21.2.104:0/11484324 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c003260 con 0x7ff03c004f70 2021-07-11T03:59:16.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05bfff700 1 -- 172.21.2.104:0/11484324 reap_dead start 2021-07-11T03:59:16.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05bfff700 1 --2- 172.21.2.104:0/11484324 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff03c004f70 0x7ff03c00d960 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7ff054002320 tx=0x7ff054007d80).ready entity=mon.0 client_cookie=f39654ff56f71c5a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff054002510 con 0x7ff03c004f70 2021-07-11T03:59:16.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054006890 con 0x7ff03c004f70 2021-07-11T03:59:16.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05a7fc700 1 -- 172.21.2.104:0/11484324 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff054006bf0 con 0x7ff03c004f70 2021-07-11T03:59:16.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- 172.21.2.104:0/11484324 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff03c004f70 msgr2=0x7ff03c00d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 --2- 172.21.2.104:0/11484324 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff03c004f70 0x7ff03c00d960 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7ff054002320 tx=0x7ff054007d80).stop 2021-07-11T03:59:16.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- 172.21.2.104:0/11484324 shutdown_connections 2021-07-11T03:59:16.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 --2- 172.21.2.104:0/11484324 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff03c004f70 0x7ff03c00d960 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- 172.21.2.104:0/11484324 >> 172.21.2.104:0/11484324 conn(0x7ff05c0607c0 msgr2=0x7ff05c060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:16.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- 172.21.2.104:0/11484324 shutdown_connections 2021-07-11T03:59:16.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- 172.21.2.104:0/11484324 wait complete. 2021-07-11T03:59:16.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 Processor -- start 2021-07-11T03:59:16.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- start start 2021-07-11T03:59:16.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff03c004f70 0x7ff05c144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff05c0b60d0 0x7ff05c13b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff05c13bc50 0x7ff05c139400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0b44d0 con 0x7ff05c0b60d0 2021-07-11T03:59:16.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0b70a0 con 0x7ff03c004f70 2021-07-11T03:59:16.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05c0b5990 con 0x7ff05c13bc50 2021-07-11T03:59:16.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff05c0b60d0 0x7ff05c13b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff05c0b60d0 0x7ff05c13b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46450/0 (socket says 172.21.2.104:46450) 2021-07-11T03:59:16.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05affd700 1 -- 172.21.2.104:0/3871011559 learned_addr learned my addr 172.21.2.104:0/3871011559 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:16.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05bfff700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff05c13bc50 0x7ff05c139400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05b7fe700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff03c004f70 0x7ff05c144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05affd700 1 -- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff05c13bc50 msgr2=0x7ff05c139400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05affd700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff05c13bc50 0x7ff05c139400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05affd700 1 -- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff03c004f70 msgr2=0x7ff05c144f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05affd700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff03c004f70 0x7ff05c144f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05affd700 1 -- 172.21.2.104:0/3871011559 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05c0b5c30 con 0x7ff05c0b60d0 2021-07-11T03:59:16.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff05affd700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff05c0b60d0 0x7ff05c13b760 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7ff05c06aa90 tx=0x7ff05c143d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff058ff9700 1 -- 172.21.2.104:0/3871011559 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff044007170 con 0x7ff05c0b60d0 2021-07-11T03:59:16.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff058ff9700 1 -- 172.21.2.104:0/3871011559 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0440072d0 con 0x7ff05c0b60d0 2021-07-11T03:59:16.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- 172.21.2.104:0/3871011559 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff05c13d4a0 con 0x7ff05c0b60d0 2021-07-11T03:59:16.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff058ff9700 1 -- 172.21.2.104:0/3871011559 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff044004920 con 0x7ff05c0b60d0 2021-07-11T03:59:16.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- 172.21.2.104:0/3871011559 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff05c1456a0 con 0x7ff05c0b60d0 2021-07-11T03:59:16.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.262+0000 7ff061da2700 1 -- 172.21.2.104:0/3871011559 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05c0b72a0 con 0x7ff05c0b60d0 2021-07-11T03:59:16.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.266+0000 7ff058ff9700 1 -- 172.21.2.104:0/3871011559 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff04401b070 con 0x7ff05c0b60d0 2021-07-11T03:59:16.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.266+0000 7ff058ff9700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff040053160 0x7ff040055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.266+0000 7ff058ff9700 1 -- 172.21.2.104:0/3871011559 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff04406abd0 con 0x7ff05c0b60d0 2021-07-11T03:59:16.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.266+0000 7ff05b7fe700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff040053160 0x7ff040055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.266+0000 7ff05b7fe700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff040053160 0x7ff040055620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7ff04c00c000 tx=0x7ff04c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.266+0000 7ff058ff9700 1 -- 172.21.2.104:0/3871011559 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff044041f10 con 0x7ff05c0b60d0 2021-07-11T03:59:16.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 -- 172.21.2.104:0/3871011559 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff05c0b5dc0 con 0x7ff05c0b60d0 2021-07-11T03:59:16.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff058ff9700 1 -- 172.21.2.104:0/3871011559 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff044045530 con 0x7ff05c0b60d0 2021-07-11T03:59:16.356 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:59:16.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 -- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff040053160 msgr2=0x7ff040055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff040053160 0x7ff040055620 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7ff04c00c000 tx=0x7ff04c008040).stop 2021-07-11T03:59:16.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 -- 172.21.2.104:0/3871011559 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff05c0b60d0 msgr2=0x7ff05c13b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff05c0b60d0 0x7ff05c13b760 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7ff05c06aa90 tx=0x7ff05c143d50).stop 2021-07-11T03:59:16.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 -- 172.21.2.104:0/3871011559 shutdown_connections 2021-07-11T03:59:16.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff05c13bc50 0x7ff05c139400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff040053160 0x7ff040055620 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff05c0b60d0 0x7ff05c13b760 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 --2- 172.21.2.104:0/3871011559 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff03c004f70 0x7ff05c144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 -- 172.21.2.104:0/3871011559 >> 172.21.2.104:0/3871011559 conn(0x7ff05c0607c0 msgr2=0x7ff05c143090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:16.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 -- 172.21.2.104:0/3871011559 shutdown_connections 2021-07-11T03:59:16.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.350+0000 7ff061da2700 1 -- 172.21.2.104:0/3871011559 wait complete. 2021-07-11T03:59:16.363 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:59:16.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f8766e2e700 1 Processor -- start 2021-07-11T03:59:16.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f8766e2e700 1 -- start start 2021-07-11T03:59:16.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f8766e2e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f87600affb0 0x7f87600b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f8766e2e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87600ae4c0 con 0x7f87600ad030 2021-07-11T03:59:16.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f8766e2e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87600b1a00 con 0x7f87600b2120 2021-07-11T03:59:16.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f8766e2e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87600ae6f0 con 0x7f87600affb0 2021-07-11T03:59:16.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875ffff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f87600ad030 0x7f87600b0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43222/0 (socket says 172.21.2.104:43222) 2021-07-11T03:59:16.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875ffff700 1 -- 172.21.2.104:0/3670479695 learned_addr learned my addr 172.21.2.104:0/3670479695 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:16.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458705259 0 0) 0x7f87600ae4c0 con 0x7f87600ad030 2021-07-11T03:59:16.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8740003260 con 0x7f87600ad030 2021-07-11T03:59:16.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1256484088 0 0) 0x7f8740003260 con 0x7f87600ad030 2021-07-11T03:59:16.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 >> v2:172.21.2.105:3300/0 conn(0x7f87600affb0 msgr2=0x7f87600b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:16.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 --2- 172.21.2.104:0/3670479695 >> v2:172.21.2.105:3300/0 conn(0x7f87600affb0 0x7f87600b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 >> v1:172.21.2.105:6789/0 conn(0x7f87600b2120 legacy=0x7f8760143bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:16.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87600ae990 con 0x7f87600ad030 2021-07-11T03:59:16.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f8750001660 con 0x7f87600ad030 2021-07-11T03:59:16.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 >> v1:172.21.2.104:6789/0 conn(0x7f87600ad030 legacy=0x7f87600b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 --2- 172.21.2.104:0/3670479695 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8740003820 0x7f8740003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 --2- 172.21.2.104:0/3670479695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8740004500 0x7f8740004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 --2- 172.21.2.104:0/3670479695 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8740005050 0x7f874000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87600ae6f0 con 0x7f8740004500 2021-07-11T03:59:16.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874000bf40 con 0x7f8740005050 2021-07-11T03:59:16.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874000c100 con 0x7f8740003820 2021-07-11T03:59:16.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875ffff700 1 --2- 172.21.2.104:0/3670479695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8740004500 0x7f8740004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.410+0000 7f875ffff700 1 -- 172.21.2.104:0/3670479695 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8740003820 msgr2=0x7f8740003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:16.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 --2- 172.21.2.104:0/3670479695 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8740003820 0x7f8740003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 -- 172.21.2.104:0/3670479695 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8740005050 msgr2=0x7f874000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:16.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 --2- 172.21.2.104:0/3670479695 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8740005050 0x7f874000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 -- 172.21.2.104:0/3670479695 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8740003260 con 0x7f8740004500 2021-07-11T03:59:16.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f87653cb700 1 -- 172.21.2.104:0/3670479695 reap_dead start 2021-07-11T03:59:16.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 --2- 172.21.2.104:0/3670479695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8740004500 0x7f8740004980 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f87600615c0 tx=0x7f87600afe20).ready entity=mon.0 client_cookie=83df7cd2fb7c60ce server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87500070d0 con 0x7f8740004500 2021-07-11T03:59:16.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8750007230 con 0x7f8740004500 2021-07-11T03:59:16.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875f7fe700 1 -- 172.21.2.104:0/3670479695 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f875000c8f0 con 0x7f8740004500 2021-07-11T03:59:16.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 -- 172.21.2.104:0/3670479695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8740004500 msgr2=0x7f8740004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 --2- 172.21.2.104:0/3670479695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8740004500 0x7f8740004980 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f87600615c0 tx=0x7f87600afe20).stop 2021-07-11T03:59:16.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 -- 172.21.2.104:0/3670479695 shutdown_connections 2021-07-11T03:59:16.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 --2- 172.21.2.104:0/3670479695 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8740004500 0x7f8740004980 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 -- 172.21.2.104:0/3670479695 >> 172.21.2.104:0/3670479695 conn(0x7f876005f6b0 msgr2=0x7f876005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:16.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 -- 172.21.2.104:0/3670479695 shutdown_connections 2021-07-11T03:59:16.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 -- 172.21.2.104:0/3670479695 wait complete. 2021-07-11T03:59:16.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 Processor -- start 2021-07-11T03:59:16.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 -- start start 2021-07-11T03:59:16.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8740004500 0x7f8760140b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8760156410 0x7f8760156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8760156d60 0x7f876013cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8760139610 con 0x7f8760156410 2021-07-11T03:59:16.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8760139370 con 0x7f8740004500 2021-07-11T03:59:16.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87600ac300 con 0x7f8760156d60 2021-07-11T03:59:16.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8760156410 0x7f8760156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8760156410 0x7f8760156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46466/0 (socket says 172.21.2.104:46466) 2021-07-11T03:59:16.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 -- 172.21.2.104:0/1086540211 learned_addr learned my addr 172.21.2.104:0/1086540211 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:16.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 -- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8760156d60 msgr2=0x7f876013cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8760156d60 0x7f876013cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 -- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8740004500 msgr2=0x7f8760140b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8740004500 0x7f8760140b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 -- 172.21.2.104:0/1086540211 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876013abf0 con 0x7f8760156410 2021-07-11T03:59:16.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875ffff700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8760156410 0x7f8760156870 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f87600615c0 tx=0x7f8750015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875dffb700 1 -- 172.21.2.104:0/1086540211 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8750017070 con 0x7f8760156410 2021-07-11T03:59:16.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875dffb700 1 -- 172.21.2.104:0/1086540211 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8750001b20 con 0x7f8760156410 2021-07-11T03:59:16.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 -- 172.21.2.104:0/1086540211 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87600abbc0 con 0x7f8760156410 2021-07-11T03:59:16.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f8766e2e700 1 -- 172.21.2.104:0/1086540211 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876013eb60 con 0x7f8760156410 2021-07-11T03:59:16.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.414+0000 7f875dffb700 1 -- 172.21.2.104:0/1086540211 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8750004920 con 0x7f8760156410 2021-07-11T03:59:16.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.418+0000 7f8766e2e700 1 -- 172.21.2.104:0/1086540211 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8760057350 con 0x7f8760156410 2021-07-11T03:59:16.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.418+0000 7f875dffb700 1 -- 172.21.2.104:0/1086540211 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8750010070 con 0x7f8760156410 2021-07-11T03:59:16.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.418+0000 7f875dffb700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8744053300 0x7f87440557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.418+0000 7f875dffb700 1 -- 172.21.2.104:0/1086540211 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f875006c1a0 con 0x7f8760156410 2021-07-11T03:59:16.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.418+0000 7f8764bca700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8744053300 0x7f87440557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.418+0000 7f8764bca700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8744053300 0x7f87440557c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f8748000f70 tx=0x7f87480077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.418+0000 7f875dffb700 1 -- 172.21.2.104:0/1086540211 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f87500433d0 con 0x7f8760156410 2021-07-11T03:59:16.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.502+0000 7f8766e2e700 1 -- 172.21.2.104:0/1086540211 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87600acad0 con 0x7f8760156410 2021-07-11T03:59:16.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.502+0000 7f875dffb700 1 -- 172.21.2.104:0/1086540211 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+10 (secure 0 0 0) 0x7f87500469f0 con 0x7f8760156410 2021-07-11T03:59:16.510 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:59:16.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 -- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8744053300 msgr2=0x7f87440557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8744053300 0x7f87440557c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f8748000f70 tx=0x7f87480077f0).stop 2021-07-11T03:59:16.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 -- 172.21.2.104:0/1086540211 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8760156410 msgr2=0x7f8760156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8760156410 0x7f8760156870 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f87600615c0 tx=0x7f8750015000).stop 2021-07-11T03:59:16.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 -- 172.21.2.104:0/1086540211 shutdown_connections 2021-07-11T03:59:16.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8760156d60 0x7f876013cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8744053300 0x7f87440557c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8760156410 0x7f8760156870 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 --2- 172.21.2.104:0/1086540211 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8740004500 0x7f8760140b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 -- 172.21.2.104:0/1086540211 >> 172.21.2.104:0/1086540211 conn(0x7f876005f6b0 msgr2=0x7f876013fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:16.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 -- 172.21.2.104:0/1086540211 shutdown_connections 2021-07-11T03:59:16.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.506+0000 7f8766e2e700 1 -- 172.21.2.104:0/1086540211 wait complete. 2021-07-11T03:59:16.517 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:59:16.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 Processor -- start 2021-07-11T03:59:16.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- start start 2021-07-11T03:59:16.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe538060100 0x7fe538064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe538144440 0x7fe538065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5380604e0 con 0x7fe5380b38d0 2021-07-11T03:59:16.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5380656f0 con 0x7fe538060100 2021-07-11T03:59:16.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe538065c80 con 0x7fe538144440 2021-07-11T03:59:16.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53659c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe538060100 0x7fe538064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe536d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe5380b38d0 0x7fe538144970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43244/0 (socket says 172.21.2.104:43244) 2021-07-11T03:59:16.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe536d9d700 1 -- 172.21.2.104:0/4267052607 learned_addr learned my addr 172.21.2.104:0/4267052607 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:16.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53659c700 1 -- 172.21.2.104:0/4267052607 >> v2:172.21.2.105:3300/0 conn(0x7fe538144440 msgr2=0x7fe538065200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T03:59:16.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53659c700 1 --2- 172.21.2.104:0/4267052607 >> v2:172.21.2.105:3300/0 conn(0x7fe538144440 0x7fe538065200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53659c700 1 -- 172.21.2.104:0/4267052607 >> v1:172.21.2.104:6789/0 conn(0x7fe5380b38d0 legacy=0x7fe538144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53659c700 1 -- 172.21.2.104:0/4267052607 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5380b23b0 con 0x7fe538060100 2021-07-11T03:59:16.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53659c700 1 --2- 172.21.2.104:0/4267052607 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe538060100 0x7fe538064d10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7fe528002f70 tx=0x7fe528006df0).ready entity=mon.0 client_cookie=d52176de3b83a2ed server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53559a700 1 -- 172.21.2.104:0/4267052607 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe52800d5c0 con 0x7fe538060100 2021-07-11T03:59:16.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53559a700 1 -- 172.21.2.104:0/4267052607 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe528000e90 con 0x7fe538060100 2021-07-11T03:59:16.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53559a700 1 -- 172.21.2.104:0/4267052607 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe52800d950 con 0x7fe538060100 2021-07-11T03:59:16.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- 172.21.2.104:0/4267052607 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe538060100 msgr2=0x7fe538064d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 --2- 172.21.2.104:0/4267052607 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe538060100 0x7fe538064d10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7fe528002f70 tx=0x7fe528006df0).stop 2021-07-11T03:59:16.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- 172.21.2.104:0/4267052607 shutdown_connections 2021-07-11T03:59:16.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 --2- 172.21.2.104:0/4267052607 >> v2:172.21.2.105:3300/0 conn(0x7fe538144440 0x7fe538065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 --2- 172.21.2.104:0/4267052607 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe538060100 0x7fe538064d10 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- 172.21.2.104:0/4267052607 >> 172.21.2.104:0/4267052607 conn(0x7fe53805cf60 msgr2=0x7fe53805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:16.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- 172.21.2.104:0/4267052607 shutdown_connections 2021-07-11T03:59:16.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- 172.21.2.104:0/4267052607 wait complete. 2021-07-11T03:59:16.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 Processor -- start 2021-07-11T03:59:16.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- start start 2021-07-11T03:59:16.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe538060100 0x7fe53814d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5380b38d0 0x7fe5380b6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe538060cd0 0x7fe53813a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380b5b80 con 0x7fe538060cd0 2021-07-11T03:59:16.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380b51e0 con 0x7fe538060100 2021-07-11T03:59:16.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe53cdc5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5380b6980 con 0x7fe5380b38d0 2021-07-11T03:59:16.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe536d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe538060cd0 0x7fe53813a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe536d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe538060cd0 0x7fe53813a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46484/0 (socket says 172.21.2.104:46484) 2021-07-11T03:59:16.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe536d9d700 1 -- 172.21.2.104:0/1291306963 learned_addr learned my addr 172.21.2.104:0/1291306963 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:16.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe536d9d700 1 -- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5380b38d0 msgr2=0x7fe5380b6db0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:59:16.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe536d9d700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5380b38d0 0x7fe5380b6db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe536d9d700 1 -- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe538060100 msgr2=0x7fe53814d3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:59:16.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.566+0000 7fe536d9d700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe538060100 0x7fe53814d3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe536d9d700 1 -- 172.21.2.104:0/1291306963 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe52800b180 con 0x7fe538060cd0 2021-07-11T03:59:16.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe536d9d700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe538060cd0 0x7fe53813a3e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7fe53805d460 tx=0x7fe530007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe52f7fe700 1 -- 172.21.2.104:0/1291306963 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe530005ba0 con 0x7fe538060cd0 2021-07-11T03:59:16.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe52f7fe700 1 -- 172.21.2.104:0/1291306963 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe530005d00 con 0x7fe538060cd0 2021-07-11T03:59:16.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe53cdc5700 1 -- 172.21.2.104:0/1291306963 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53814b110 con 0x7fe538060cd0 2021-07-11T03:59:16.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe52f7fe700 1 -- 172.21.2.104:0/1291306963 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5300072c0 con 0x7fe538060cd0 2021-07-11T03:59:16.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe53cdc5700 1 -- 172.21.2.104:0/1291306963 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5380b72a0 con 0x7fe538060cd0 2021-07-11T03:59:16.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe53cdc5700 1 -- 172.21.2.104:0/1291306963 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe538060710 con 0x7fe538060cd0 2021-07-11T03:59:16.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe52f7fe700 1 -- 172.21.2.104:0/1291306963 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe53001a060 con 0x7fe538060cd0 2021-07-11T03:59:16.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe52f7fe700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe51c053300 0x7fe51c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe52f7fe700 1 -- 172.21.2.104:0/1291306963 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe53006afe0 con 0x7fe538060cd0 2021-07-11T03:59:16.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe53659c700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe51c053300 0x7fe51c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe53659c700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe51c053300 0x7fe51c0557c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fe528002f70 tx=0x7fe528014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.570+0000 7fe52f7fe700 1 -- 172.21.2.104:0/1291306963 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe530042320 con 0x7fe538060cd0 2021-07-11T03:59:16.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.654+0000 7fe53cdc5700 1 -- 172.21.2.104:0/1291306963 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe5380b6ac0 con 0x7fe538060cd0 2021-07-11T03:59:16.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.654+0000 7fe52f7fe700 1 -- 172.21.2.104:0/1291306963 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe530018030 con 0x7fe538060cd0 2021-07-11T03:59:16.661 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:59:16.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.654+0000 7fe53cdc5700 1 -- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe51c053300 msgr2=0x7fe51c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.654+0000 7fe53cdc5700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe51c053300 0x7fe51c0557c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fe528002f70 tx=0x7fe528014000).stop 2021-07-11T03:59:16.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.654+0000 7fe53cdc5700 1 -- 172.21.2.104:0/1291306963 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe538060cd0 msgr2=0x7fe53813a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.654+0000 7fe53cdc5700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe538060cd0 0x7fe53813a3e0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7fe53805d460 tx=0x7fe530007f10).stop 2021-07-11T03:59:16.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.658+0000 7fe53cdc5700 1 -- 172.21.2.104:0/1291306963 shutdown_connections 2021-07-11T03:59:16.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.658+0000 7fe53cdc5700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe51c053300 0x7fe51c0557c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.658+0000 7fe53cdc5700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe538060cd0 0x7fe53813a3e0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.658+0000 7fe53cdc5700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5380b38d0 0x7fe5380b6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.658+0000 7fe53cdc5700 1 --2- 172.21.2.104:0/1291306963 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe538060100 0x7fe53814d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.658+0000 7fe53cdc5700 1 -- 172.21.2.104:0/1291306963 >> 172.21.2.104:0/1291306963 conn(0x7fe53805cf60 msgr2=0x7fe53813ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:16.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.658+0000 7fe53cdc5700 1 -- 172.21.2.104:0/1291306963 shutdown_connections 2021-07-11T03:59:16.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.658+0000 7fe53cdc5700 1 -- 172.21.2.104:0/1291306963 wait complete. 2021-07-11T03:59:16.669 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T03:59:16.669 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:59:16.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f6431f1e700 1 Processor -- start 2021-07-11T03:59:16.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f6431f1e700 1 -- start start 2021-07-11T03:59:16.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f6431f1e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f642c0a76b0 0x7f642c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f6431f1e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f642c0b1970 0x7f642c066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f6431f1e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f642c067450 con 0x7f642c0b0060 2021-07-11T03:59:16.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f6431f1e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f642c0a7a90 con 0x7f642c0a76b0 2021-07-11T03:59:16.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f6431f1e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c147620 con 0x7f642c0b1970 2021-07-11T03:59:16.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642bfff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f642c0b0060 0x7f642c05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43258/0 (socket says 172.21.2.104:43258) 2021-07-11T03:59:16.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642bfff700 1 -- 172.21.2.104:0/1401329036 learned_addr learned my addr 172.21.2.104:0/1401329036 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:16.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642b7fe700 1 --2- 172.21.2.104:0/1401329036 >> v2:172.21.2.104:3300/0 conn(0x7f642c0a76b0 0x7f642c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642a7fc700 1 -- 172.21.2.104:0/1401329036 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1288575009 0 0) 0x7f642c067450 con 0x7f642c0b0060 2021-07-11T03:59:16.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642a7fc700 1 -- 172.21.2.104:0/1401329036 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f640c003260 con 0x7f642c0b0060 2021-07-11T03:59:16.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642b7fe700 1 -- 172.21.2.104:0/1401329036 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f642c0b1970 msgr2=0x7f642c066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:16.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642b7fe700 1 --2- 172.21.2.104:0/1401329036 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f642c0b1970 0x7f642c066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642b7fe700 1 -- 172.21.2.104:0/1401329036 >> v1:172.21.2.104:6789/0 conn(0x7f642c0b0060 legacy=0x7f642c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642b7fe700 1 -- 172.21.2.104:0/1401329036 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c1478c0 con 0x7f642c0a76b0 2021-07-11T03:59:16.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642b7fe700 1 --2- 172.21.2.104:0/1401329036 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f642c0a76b0 0x7f642c066a70 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f641c002f70 tx=0x7f641c006df0).ready entity=mon.0 client_cookie=4fbfbae0aa0e3ecb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642a7fc700 1 -- 172.21.2.104:0/1401329036 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f641c00d570 con 0x7f642c0a76b0 2021-07-11T03:59:16.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642a7fc700 1 -- 172.21.2.104:0/1401329036 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c000e90 con 0x7f642c0a76b0 2021-07-11T03:59:16.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f642a7fc700 1 -- 172.21.2.104:0/1401329036 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f641c00d8c0 con 0x7f642c0a76b0 2021-07-11T03:59:16.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f6431f1e700 1 -- 172.21.2.104:0/1401329036 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f642c0a76b0 msgr2=0x7f642c066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.714+0000 7f6431f1e700 1 --2- 172.21.2.104:0/1401329036 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f642c0a76b0 0x7f642c066a70 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7f641c002f70 tx=0x7f641c006df0).stop 2021-07-11T03:59:16.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 -- 172.21.2.104:0/1401329036 shutdown_connections 2021-07-11T03:59:16.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 --2- 172.21.2.104:0/1401329036 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f642c0b1970 0x7f642c066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 --2- 172.21.2.104:0/1401329036 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f642c0a76b0 0x7f642c066a70 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 -- 172.21.2.104:0/1401329036 >> 172.21.2.104:0/1401329036 conn(0x7f642c0b53e0 msgr2=0x7f642c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:16.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 -- 172.21.2.104:0/1401329036 shutdown_connections 2021-07-11T03:59:16.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 -- 172.21.2.104:0/1401329036 wait complete. 2021-07-11T03:59:16.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 Processor -- start 2021-07-11T03:59:16.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 -- start start 2021-07-11T03:59:16.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f642c0a76b0 0x7f642c14d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f642c0b0060 0x7f642c159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f642c0b1970 0x7f642c15a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c0ab3e0 con 0x7f642c0b0060 2021-07-11T03:59:16.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c0ab680 con 0x7f642c0b1970 2021-07-11T03:59:16.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642c0ab920 con 0x7f642c0a76b0 2021-07-11T03:59:16.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f642affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f642c0b0060 0x7f642c159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f642affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f642c0b0060 0x7f642c159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46496/0 (socket says 172.21.2.104:46496) 2021-07-11T03:59:16.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f642affd700 1 -- 172.21.2.104:0/985261346 learned_addr learned my addr 172.21.2.104:0/985261346 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:16.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f642affd700 1 -- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f642c0a76b0 msgr2=0x7f642c14d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f642affd700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f642c0a76b0 0x7f642c14d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f642affd700 1 -- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f642c0b1970 msgr2=0x7f642c15a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f642affd700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f642c0b1970 0x7f642c15a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f642affd700 1 -- 172.21.2.104:0/985261346 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c00b180 con 0x7f642c0b0060 2021-07-11T03:59:16.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f642affd700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f642c0b0060 0x7f642c159b80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f6414002f70 tx=0x7f6414006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6428ff9700 1 -- 172.21.2.104:0/985261346 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6414013460 con 0x7f642c0b0060 2021-07-11T03:59:16.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6428ff9700 1 -- 172.21.2.104:0/985261346 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64140135c0 con 0x7f642c0b0060 2021-07-11T03:59:16.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6428ff9700 1 -- 172.21.2.104:0/985261346 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6414013920 con 0x7f642c0b0060 2021-07-11T03:59:16.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 -- 172.21.2.104:0/985261346 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c0abbc0 con 0x7f642c0b0060 2021-07-11T03:59:16.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 -- 172.21.2.104:0/985261346 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c0ad8c0 con 0x7f642c0b0060 2021-07-11T03:59:16.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6431f1e700 1 -- 172.21.2.104:0/985261346 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642c0abd50 con 0x7f642c0b0060 2021-07-11T03:59:16.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6428ff9700 1 -- 172.21.2.104:0/985261346 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f641400f070 con 0x7f642c0b0060 2021-07-11T03:59:16.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6428ff9700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64100531f0 0x7f64100556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:16.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.718+0000 7f6428ff9700 1 -- 172.21.2.104:0/985261346 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f641406ae90 con 0x7f642c0b0060 2021-07-11T03:59:16.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.722+0000 7f642b7fe700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64100531f0 0x7f64100556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:16.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.722+0000 7f642b7fe700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64100531f0 0x7f64100556b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f641c002f70 tx=0x7f641c014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:16.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.722+0000 7f6428ff9700 1 -- 172.21.2.104:0/985261346 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f64140420c0 con 0x7f642c0b0060 2021-07-11T03:59:16.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 -- 172.21.2.104:0/985261346 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f642c057350 con 0x7f642c0b0060 2021-07-11T03:59:16.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6428ff9700 1 -- 172.21.2.104:0/985261346 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f64140456e0 con 0x7f642c0b0060 2021-07-11T03:59:16.835 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:16.836 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/140397159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":140397159},{"type":"v1","addr":"172.21.2.105:6835","nonce":140397159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/2616903361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2616903361},{"type":"v1","addr":"172.21.2.104:6834","nonce":2616903361}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:59:13.620413+0000","modified":"2021-07-11T03:59:14.624165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/611806767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":611806767},{"type":"v1","addr":"172.21.2.104:6835","nonce":611806767}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-07-11T03:59:16.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 -- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64100531f0 msgr2=0x7f64100556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64100531f0 0x7f64100556b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f641c002f70 tx=0x7f641c014000).stop 2021-07-11T03:59:16.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 -- 172.21.2.104:0/985261346 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f642c0b0060 msgr2=0x7f642c159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:16.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f642c0b0060 0x7f642c159b80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f6414002f70 tx=0x7f6414006e20).stop 2021-07-11T03:59:16.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 -- 172.21.2.104:0/985261346 shutdown_connections 2021-07-11T03:59:16.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f642c0b1970 0x7f642c15a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f64100531f0 0x7f64100556b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f642c0b0060 0x7f642c159b80 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:16.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 --2- 172.21.2.104:0/985261346 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f642c0a76b0 0x7f642c14d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:16.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 -- 172.21.2.104:0/985261346 >> 172.21.2.104:0/985261346 conn(0x7f642c0b53e0 msgr2=0x7f642c14a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:16.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 -- 172.21.2.104:0/985261346 shutdown_connections 2021-07-11T03:59:16.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:16.830+0000 7f6431f1e700 1 -- 172.21.2.104:0/985261346 wait complete. 2021-07-11T03:59:16.839 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 32 2021-07-11T03:59:17.145 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:59:13.620413+0000', 'modified': '2021-07-11T03:59:14.624165+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6246}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6246': {'gid': 6246, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/611806767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 611806767}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 611806767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:59:17.145 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:59:17.146 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:59:17.211 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":6246,"want_state":"up:active","state":"up:active","rank_uptime":3.5814507870000001,"mdsmap_epoch":32,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":7.7809330289999998} 2021-07-11T03:59:17.212 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 6246, "mdsmap_epoch": 32, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 3.581450787, "state": "up:active", "uptime": 7.780933029, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:59:17.212 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T03:59:17.212 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T03:59:17.213 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T03:59:17.213 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:17.214 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:59:17.226 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:59:17.277 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:59:17.277 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:59:17.284 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:59:17.285 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:59:17.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:17.287 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:59:17.291 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T03:59:17.292 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:59:17.337 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:59:17.349 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:59:17.350 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:59:17.356 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:59:17.357 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:59:17.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:17.358 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:59:17.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T03:59:17.379+0000 7fb6d716e080 -1 init, newargv = 0x557b56869480 newargc=9 2021-07-11T03:59:17.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15145]: starting ceph client 2021-07-11T03:59:17.390 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15145]: starting fuse 2021-07-11T03:59:18.484 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T03:59:18.498 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:59:18.499 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:59:18.553 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:59:18.554 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:59:18.555 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:18.555 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T03:59:18.562 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T03:59:18.562 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T03:59:18.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:18.563 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T03:59:18.564 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T03:59:18.564 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T03:59:18.564 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T03:59:18.565 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T03:59:18.565 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:59:18.566 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T03:59:18.566 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T03:59:18.567 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T03:59:18.567 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:59:18.568 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T03:59:18.568 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T03:59:18.569 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T03:59:18.569 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T03:59:18.570 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:59:18.571 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T03:59:18.571 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T03:59:18.571 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T03:59:18.572 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T03:59:18.572 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T03:59:18.573 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:59:18.573 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T03:59:18.573 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:59:18.642 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15145.asok 2021-07-11T03:59:18.647 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15145.asok 2021-07-11T03:59:18.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:18.649 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15145.asok status 2021-07-11T03:59:18.697 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T03:59:18.697 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T03:59:18.697 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T03:59:18.697 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T03:59:18.698 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T03:59:18.698 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T03:59:18.698 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T03:59:18.698 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15145", 2021-07-11T03:59:18.699 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T03:59:18.699 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:59:18.699 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T03:59:18.699 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T03:59:18.699 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 6405, 2021-07-11T03:59:18.700 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T03:59:18.700 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T03:59:18.700 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T03:59:18.700 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 6405 2021-07-11T03:59:18.701 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:59:18.701 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:59:18.701 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:59:18.701 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:59:18.701 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1304352204 2021-07-11T03:59:18.702 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T03:59:18.702 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:59:18.702 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T03:59:18.702 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T03:59:18.702 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T03:59:18.703 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 1304352204 2021-07-11T03:59:18.703 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T03:59:18.703 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.6405 172.21.2.119:0/1304352204", 2021-07-11T03:59:18.703 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/1304352204", 2021-07-11T03:59:18.704 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T03:59:18.704 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 32, 2021-07-11T03:59:18.704 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 76, 2021-07-11T03:59:18.704 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T03:59:18.704 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T03:59:18.705 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T03:59:18.705 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T03:59:18.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:18.710 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:59:18.711 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:59:18.720 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:59:18.721 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:59:18.721 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:18.722 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:59:18.738 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T03:59:18.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.787+0000 7f555af93700 1 Processor -- start 2021-07-11T03:59:18.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.787+0000 7f555af93700 1 -- start start 2021-07-11T03:59:18.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5554143540 0x7f55540a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f55540a8b60 0x7f5554143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55540b14c0 con 0x7f555405f6b0 2021-07-11T03:59:18.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55540ab3e0 con 0x7f55540a8b60 2021-07-11T03:59:18.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55540b1b90 con 0x7f5554143540 2021-07-11T03:59:18.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f5559530700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f55540a8b60 0x7f5554143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:18.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f5559530700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f55540a8b60 0x7f5554143920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46506/0 (socket says 172.21.2.104:46506) 2021-07-11T03:59:18.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f5559530700 1 -- 172.21.2.104:0/2711318899 learned_addr learned my addr 172.21.2.104:0/2711318899 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:18.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f5559530700 1 -- 172.21.2.104:0/2711318899 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5554143540 msgr2=0x7f55540a9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:18.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f5559530700 1 --2- 172.21.2.104:0/2711318899 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5554143540 0x7f55540a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:18.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f5559530700 1 -- 172.21.2.104:0/2711318899 >> v1:172.21.2.104:6789/0 conn(0x7f555405f6b0 legacy=0x7f55540b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:18.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f5559530700 1 -- 172.21.2.104:0/2711318899 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5554143fa0 con 0x7f55540a8b60 2021-07-11T03:59:18.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f5559530700 1 --2- 172.21.2.104:0/2711318899 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55540a8b60 0x7f5554143920 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f554c0032f0 tx=0x7f554c009290).ready entity=mon.0 client_cookie=edd3cab99ff94c35 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:18.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f55537fe700 1 -- 172.21.2.104:0/2711318899 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f554c00b8a0 con 0x7f55540a8b60 2021-07-11T03:59:18.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f55537fe700 1 -- 172.21.2.104:0/2711318899 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554c00ba00 con 0x7f55540a8b60 2021-07-11T03:59:18.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f55537fe700 1 -- 172.21.2.104:0/2711318899 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f554c00bb70 con 0x7f55540a8b60 2021-07-11T03:59:18.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- 172.21.2.104:0/2711318899 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55540a8b60 msgr2=0x7f5554143920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:18.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 --2- 172.21.2.104:0/2711318899 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55540a8b60 0x7f5554143920 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7f554c0032f0 tx=0x7f554c009290).stop 2021-07-11T03:59:18.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- 172.21.2.104:0/2711318899 shutdown_connections 2021-07-11T03:59:18.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 --2- 172.21.2.104:0/2711318899 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f55540a8b60 0x7f5554143920 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:18.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 --2- 172.21.2.104:0/2711318899 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5554143540 0x7f55540a9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:18.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- 172.21.2.104:0/2711318899 >> 172.21.2.104:0/2711318899 conn(0x7f55540b53e0 msgr2=0x7f55540b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:18.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- 172.21.2.104:0/2711318899 shutdown_connections 2021-07-11T03:59:18.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- 172.21.2.104:0/2711318899 wait complete. 2021-07-11T03:59:18.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 Processor -- start 2021-07-11T03:59:18.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- start start 2021-07-11T03:59:18.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f555405f6b0 0x7f55540aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f55540a8b60 0x7f555415a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5554143540 0x7f55540ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55540ae0e0 con 0x7f555405f6b0 2021-07-11T03:59:18.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55540ae380 con 0x7f55540a8b60 2021-07-11T03:59:18.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f555af93700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55540ae620 con 0x7f5554143540 2021-07-11T03:59:18.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f5558d2f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f555405f6b0 0x7f55540aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:18.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f5558d2f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f555405f6b0 0x7f55540aa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46508/0 (socket says 172.21.2.104:46508) 2021-07-11T03:59:18.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.791+0000 7f5558d2f700 1 -- 172.21.2.104:0/2020179120 learned_addr learned my addr 172.21.2.104:0/2020179120 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:18.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5558d2f700 1 -- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5554143540 msgr2=0x7f55540ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:18.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5558d2f700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5554143540 0x7f55540ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:18.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5558d2f700 1 -- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f55540a8b60 msgr2=0x7f555415a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:18.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5558d2f700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f55540a8b60 0x7f555415a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:18.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5558d2f700 1 -- 172.21.2.104:0/2020179120 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c009530 con 0x7f555405f6b0 2021-07-11T03:59:18.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5558d2f700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f555405f6b0 0x7f55540aa150 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f5544002f70 tx=0x7f5544006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:18.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5551ffb700 1 -- 172.21.2.104:0/2020179120 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5544013460 con 0x7f555405f6b0 2021-07-11T03:59:18.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5551ffb700 1 -- 172.21.2.104:0/2020179120 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55440135c0 con 0x7f555405f6b0 2021-07-11T03:59:18.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f555af93700 1 -- 172.21.2.104:0/2020179120 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f555415a790 con 0x7f555405f6b0 2021-07-11T03:59:18.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f555af93700 1 -- 172.21.2.104:0/2020179120 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55541412a0 con 0x7f555405f6b0 2021-07-11T03:59:18.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5551ffb700 1 -- 172.21.2.104:0/2020179120 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5544013920 con 0x7f555405f6b0 2021-07-11T03:59:18.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f555af93700 1 -- 172.21.2.104:0/2020179120 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55540ae7e0 con 0x7f555405f6b0 2021-07-11T03:59:18.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5551ffb700 1 -- 172.21.2.104:0/2020179120 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f554401a060 con 0x7f555405f6b0 2021-07-11T03:59:18.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5551ffb700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5538053240 0x7f5538055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5551ffb700 1 -- 172.21.2.104:0/2020179120 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f554406af90 con 0x7f555405f6b0 2021-07-11T03:59:18.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5553fff700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5538053240 0x7f5538055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:18.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.795+0000 7f5553fff700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5538053240 0x7f5538055700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f55540611c0 tx=0x7f553c007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:18.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.799+0000 7f5551ffb700 1 -- 172.21.2.104:0/2020179120 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f55440421c0 con 0x7f555405f6b0 2021-07-11T03:59:18.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 -- 172.21.2.104:0/2020179120 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5554057350 con 0x7f555405f6b0 2021-07-11T03:59:18.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f5551ffb700 1 -- 172.21.2.104:0/2020179120 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f5544018020 con 0x7f555405f6b0 2021-07-11T03:59:18.883 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:18.884 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":76,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:59:14.625343+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:59:11.477806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-07-11T03:59:12.643109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T03:59:18.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 -- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5538053240 msgr2=0x7f5538055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:18.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5538053240 0x7f5538055700 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f55540611c0 tx=0x7f553c007810).stop 2021-07-11T03:59:18.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 -- 172.21.2.104:0/2020179120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f555405f6b0 msgr2=0x7f55540aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:18.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f555405f6b0 0x7f55540aa150 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f5544002f70 tx=0x7f5544006e20).stop 2021-07-11T03:59:18.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 -- 172.21.2.104:0/2020179120 shutdown_connections 2021-07-11T03:59:18.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5554143540 0x7f55540ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:18.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f55540a8b60 0x7f555415a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:18.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5538053240 0x7f5538055700 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:18.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 --2- 172.21.2.104:0/2020179120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f555405f6b0 0x7f55540aa150 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:18.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 -- 172.21.2.104:0/2020179120 >> 172.21.2.104:0/2020179120 conn(0x7f55540b53e0 msgr2=0x7f55540b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:18.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 -- 172.21.2.104:0/2020179120 shutdown_connections 2021-07-11T03:59:18.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.879+0000 7f555af93700 1 -- 172.21.2.104:0/2020179120 wait complete. 2021-07-11T03:59:18.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:59:11.477806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:59:12.643109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T03:59:18.893 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T03:59:18.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 Processor -- start 2021-07-11T03:59:18.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- start start 2021-07-11T03:59:18.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f81780b05b0 0x7f81780b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f81780b3710 0x7f817813d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8178143920 con 0x7f817813da40 2021-07-11T03:59:18.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8178143bc0 con 0x7f81780b3710 2021-07-11T03:59:18.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8178143e60 con 0x7f81780b05b0 2021-07-11T03:59:18.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f81767fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f81780b3710 0x7f817813d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:18.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f81767fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f81780b3710 0x7f817813d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46516/0 (socket says 172.21.2.104:46516) 2021-07-11T03:59:18.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f81767fc700 1 -- 172.21.2.104:0/332679655 learned_addr learned my addr 172.21.2.104:0/332679655 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:18.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f81767fc700 1 -- 172.21.2.104:0/332679655 >> v2:172.21.2.105:3300/0 conn(0x7f81780b05b0 msgr2=0x7f81780b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:18.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f81767fc700 1 --2- 172.21.2.104:0/332679655 >> v2:172.21.2.105:3300/0 conn(0x7f81780b05b0 0x7f81780b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:18.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f81767fc700 1 -- 172.21.2.104:0/332679655 >> v1:172.21.2.105:6789/0 conn(0x7f817813da40 legacy=0x7f81780b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:18.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f81767fc700 1 -- 172.21.2.104:0/332679655 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8178144100 con 0x7f81780b3710 2021-07-11T03:59:18.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f81767fc700 1 --2- 172.21.2.104:0/332679655 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81780b3710 0x7f817813d550 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f817806aa50 tx=0x7f8168006eb0).ready entity=mon.0 client_cookie=f8ac0ef7ed690ce6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:18.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f8175ffb700 1 -- 172.21.2.104:0/332679655 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f816800f9c0 con 0x7f81780b3710 2021-07-11T03:59:18.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f8175ffb700 1 -- 172.21.2.104:0/332679655 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8168000e90 con 0x7f81780b3710 2021-07-11T03:59:18.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f8175ffb700 1 -- 172.21.2.104:0/332679655 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f816800fd10 con 0x7f81780b3710 2021-07-11T03:59:18.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- 172.21.2.104:0/332679655 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81780b3710 msgr2=0x7f817813d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:18.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 --2- 172.21.2.104:0/332679655 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81780b3710 0x7f817813d550 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f817806aa50 tx=0x7f8168006eb0).stop 2021-07-11T03:59:18.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- 172.21.2.104:0/332679655 shutdown_connections 2021-07-11T03:59:18.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 --2- 172.21.2.104:0/332679655 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81780b3710 0x7f817813d550 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:18.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 --2- 172.21.2.104:0/332679655 >> v2:172.21.2.105:3300/0 conn(0x7f81780b05b0 0x7f81780b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:18.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- 172.21.2.104:0/332679655 >> 172.21.2.104:0/332679655 conn(0x7f81780607c0 msgr2=0x7f8178060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:18.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- 172.21.2.104:0/332679655 shutdown_connections 2021-07-11T03:59:18.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- 172.21.2.104:0/332679655 wait complete. 2021-07-11T03:59:18.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 Processor -- start 2021-07-11T03:59:18.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- start start 2021-07-11T03:59:18.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81780b05b0 0x7f8178148d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81780b3710 0x7f8178156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f817813da40 0x7f8178156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817813df60 con 0x7f81780b05b0 2021-07-11T03:59:18.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f817813e190 con 0x7f81780b3710 2021-07-11T03:59:18.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f817d4e8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f817813e430 con 0x7f817813da40 2021-07-11T03:59:18.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f8176ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81780b05b0 0x7f8178148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:18.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f8176ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81780b05b0 0x7f8178148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46526/0 (socket says 172.21.2.104:46526) 2021-07-11T03:59:18.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f8176ffd700 1 -- 172.21.2.104:0/3590698051 learned_addr learned my addr 172.21.2.104:0/3590698051 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:18.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f8176ffd700 1 -- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f817813da40 msgr2=0x7f8178156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:18.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f8176ffd700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f817813da40 0x7f8178156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:18.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f8176ffd700 1 -- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81780b3710 msgr2=0x7f8178156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:18.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f8176ffd700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81780b3710 0x7f8178156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:18.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.943+0000 7f8176ffd700 1 -- 172.21.2.104:0/3590698051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816800b1e0 con 0x7f81780b05b0 2021-07-11T03:59:18.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f8176ffd700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81780b05b0 0x7f8178148d30 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f8160002f70 tx=0x7f8160006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:18.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f816ffff700 1 -- 172.21.2.104:0/3590698051 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8160013460 con 0x7f81780b05b0 2021-07-11T03:59:18.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f816ffff700 1 -- 172.21.2.104:0/3590698051 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81600135c0 con 0x7f81780b05b0 2021-07-11T03:59:18.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f816ffff700 1 -- 172.21.2.104:0/3590698051 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8160013920 con 0x7f81780b05b0 2021-07-11T03:59:18.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f817d4e8700 1 -- 172.21.2.104:0/3590698051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8178155c90 con 0x7f81780b05b0 2021-07-11T03:59:18.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f817d4e8700 1 -- 172.21.2.104:0/3590698051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f817813b950 con 0x7f81780b05b0 2021-07-11T03:59:18.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f817d4e8700 1 -- 172.21.2.104:0/3590698051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f817813ebb0 con 0x7f81780b05b0 2021-07-11T03:59:18.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f816ffff700 1 -- 172.21.2.104:0/3590698051 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f816000f070 con 0x7f81780b05b0 2021-07-11T03:59:18.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f816ffff700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f815c053210 0x7f815c0556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:18.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f816ffff700 1 -- 172.21.2.104:0/3590698051 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f816006af00 con 0x7f81780b05b0 2021-07-11T03:59:18.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f81767fc700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f815c053210 0x7f815c0556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:18.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f81767fc700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f815c053210 0x7f815c0556d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f817806aa50 tx=0x7f8168017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:18.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:18.947+0000 7f816ffff700 1 -- 172.21.2.104:0/3590698051 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f817813ebb0 con 0x7f81780b05b0 2021-07-11T03:59:19.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.031+0000 7f817d4e8700 1 -- 172.21.2.104:0/3590698051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f817813ebb0 con 0x7f81780b05b0 2021-07-11T03:59:19.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.031+0000 7f816ffff700 1 -- 172.21.2.104:0/3590698051 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v76) v1 ==== 110+0+10 (secure 0 0 0) 0x7f817813ebb0 con 0x7f81780b05b0 2021-07-11T03:59:19.038 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T03:59:19.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 -- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f815c053210 msgr2=0x7f815c0556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f815c053210 0x7f815c0556d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f817806aa50 tx=0x7f8168017000).stop 2021-07-11T03:59:19.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 -- 172.21.2.104:0/3590698051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81780b05b0 msgr2=0x7f8178148d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81780b05b0 0x7f8178148d30 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f8160002f70 tx=0x7f8160006ec0).stop 2021-07-11T03:59:19.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 -- 172.21.2.104:0/3590698051 shutdown_connections 2021-07-11T03:59:19.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f817813da40 0x7f8178156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81780b3710 0x7f8178156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f815c053210 0x7f815c0556d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 --2- 172.21.2.104:0/3590698051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81780b05b0 0x7f8178148d30 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 -- 172.21.2.104:0/3590698051 >> 172.21.2.104:0/3590698051 conn(0x7f81780607c0 msgr2=0x7f81780b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:19.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 -- 172.21.2.104:0/3590698051 shutdown_connections 2021-07-11T03:59:19.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.035+0000 7f817d4e8700 1 -- 172.21.2.104:0/3590698051 wait complete. 2021-07-11T03:59:19.046 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T03:59:19.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 Processor -- start 2021-07-11T03:59:19.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- start start 2021-07-11T03:59:19.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0680b12d0 0x7fd0680affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd0680b22a0 0x7fd068141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0680a77d0 con 0x7fd0680b1ec0 2021-07-11T03:59:19.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0680b1740 con 0x7fd0680b22a0 2021-07-11T03:59:19.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680b0650 con 0x7fd0680b12d0 2021-07-11T03:59:19.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06d559700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd0680b22a0 0x7fd068141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06d559700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd0680b22a0 0x7fd068141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46530/0 (socket says 172.21.2.104:46530) 2021-07-11T03:59:19.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06d559700 1 -- 172.21.2.104:0/1946217672 learned_addr learned my addr 172.21.2.104:0/1946217672 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:19.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06d559700 1 -- 172.21.2.104:0/1946217672 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0680b12d0 msgr2=0x7fd0680affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:19.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06d559700 1 --2- 172.21.2.104:0/1946217672 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0680b12d0 0x7fd0680affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06d559700 1 -- 172.21.2.104:0/1946217672 >> v1:172.21.2.104:6789/0 conn(0x7fd0680b1ec0 legacy=0x7fd0680a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd0677fe700 1 -- 172.21.2.104:0/1946217672 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268180236 0 0) 0x7fd0680a77d0 con 0x7fd0680b1ec0 2021-07-11T03:59:19.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06d559700 1 -- 172.21.2.104:0/1946217672 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0680b08f0 con 0x7fd0680b22a0 2021-07-11T03:59:19.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06d559700 1 --2- 172.21.2.104:0/1946217672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0680b22a0 0x7fd068141250 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7fd0600032f0 tx=0x7fd060009290).ready entity=mon.0 client_cookie=5927c4f49f49afdf server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd0677fe700 1 -- 172.21.2.104:0/1946217672 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd06000b8a0 con 0x7fd0680b22a0 2021-07-11T03:59:19.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd0677fe700 1 -- 172.21.2.104:0/1946217672 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06000ba00 con 0x7fd0680b22a0 2021-07-11T03:59:19.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd0677fe700 1 -- 172.21.2.104:0/1946217672 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd06000bb70 con 0x7fd0680b22a0 2021-07-11T03:59:19.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- 172.21.2.104:0/1946217672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0680b22a0 msgr2=0x7fd068141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 --2- 172.21.2.104:0/1946217672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0680b22a0 0x7fd068141250 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7fd0600032f0 tx=0x7fd060009290).stop 2021-07-11T03:59:19.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- 172.21.2.104:0/1946217672 shutdown_connections 2021-07-11T03:59:19.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 --2- 172.21.2.104:0/1946217672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0680b22a0 0x7fd068141250 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 --2- 172.21.2.104:0/1946217672 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0680b12d0 0x7fd0680affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- 172.21.2.104:0/1946217672 >> 172.21.2.104:0/1946217672 conn(0x7fd06805f6b0 msgr2=0x7fd06805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:19.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- 172.21.2.104:0/1946217672 shutdown_connections 2021-07-11T03:59:19.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- 172.21.2.104:0/1946217672 wait complete. 2021-07-11T03:59:19.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 Processor -- start 2021-07-11T03:59:19.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- start start 2021-07-11T03:59:19.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0680b12d0 0x7fd06813d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0680b1ec0 0x7fd068138410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0680b22a0 0x7fd068138900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd068140d40 con 0x7fd0680b22a0 2021-07-11T03:59:19.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd068140170 con 0x7fd0680b1ec0 2021-07-11T03:59:19.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06efbc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd068138df0 con 0x7fd0680b12d0 2021-07-11T03:59:19.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06d559700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0680b22a0 0x7fd068138900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06d559700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0680b22a0 0x7fd068138900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46534/0 (socket says 172.21.2.104:46534) 2021-07-11T03:59:19.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.095+0000 7fd06d559700 1 -- 172.21.2.104:0/3392155653 learned_addr learned my addr 172.21.2.104:0/3392155653 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:19.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd06d559700 1 -- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0680b12d0 msgr2=0x7fd06813d7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:59:19.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd06d559700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0680b12d0 0x7fd06813d7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd06d559700 1 -- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0680b1ec0 msgr2=0x7fd068138410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:59:19.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd06d559700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0680b1ec0 0x7fd068138410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd06d559700 1 -- 172.21.2.104:0/3392155653 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd060009530 con 0x7fd0680b22a0 2021-07-11T03:59:19.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd06d559700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0680b22a0 0x7fd068138900 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7fd06813ff30 tx=0x7fd060016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd065ffb700 1 -- 172.21.2.104:0/3392155653 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd060011c40 con 0x7fd0680b22a0 2021-07-11T03:59:19.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd065ffb700 1 -- 172.21.2.104:0/3392155653 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd060011da0 con 0x7fd0680b22a0 2021-07-11T03:59:19.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd06efbc700 1 -- 172.21.2.104:0/3392155653 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd068140aa0 con 0x7fd0680b22a0 2021-07-11T03:59:19.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd06efbc700 1 -- 172.21.2.104:0/3392155653 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0680b44f0 con 0x7fd0680b22a0 2021-07-11T03:59:19.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd065ffb700 1 -- 172.21.2.104:0/3392155653 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd06001f610 con 0x7fd0680b22a0 2021-07-11T03:59:19.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd06efbc700 1 -- 172.21.2.104:0/3392155653 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0680b33b0 con 0x7fd0680b22a0 2021-07-11T03:59:19.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd065ffb700 1 -- 172.21.2.104:0/3392155653 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd060026030 con 0x7fd0680b22a0 2021-07-11T03:59:19.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd065ffb700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd04c053270 0x7fd04c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd065ffb700 1 -- 172.21.2.104:0/3392155653 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd06006c110 con 0x7fd0680b22a0 2021-07-11T03:59:19.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd06cd58700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd04c053270 0x7fd04c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.099+0000 7fd06cd58700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd04c053270 0x7fd04c055730 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7fd050000bc0 tx=0x7fd0500075b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.103+0000 7fd065ffb700 1 -- 172.21.2.104:0/3392155653 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd060043380 con 0x7fd0680b22a0 2021-07-11T03:59:19.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 -- 172.21.2.104:0/3392155653 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd068057350 con 0x7fd0680b22a0 2021-07-11T03:59:19.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd065ffb700 1 -- 172.21.2.104:0/3392155653 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v76) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd060024020 con 0x7fd0680b22a0 2021-07-11T03:59:19.188 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:59:19.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 -- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd04c053270 msgr2=0x7fd04c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd04c053270 0x7fd04c055730 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7fd050000bc0 tx=0x7fd0500075b0).stop 2021-07-11T03:59:19.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 -- 172.21.2.104:0/3392155653 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0680b22a0 msgr2=0x7fd068138900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0680b22a0 0x7fd068138900 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7fd06813ff30 tx=0x7fd060016000).stop 2021-07-11T03:59:19.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 -- 172.21.2.104:0/3392155653 shutdown_connections 2021-07-11T03:59:19.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd04c053270 0x7fd04c055730 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0680b22a0 0x7fd068138900 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0680b1ec0 0x7fd068138410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 --2- 172.21.2.104:0/3392155653 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0680b12d0 0x7fd06813d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 -- 172.21.2.104:0/3392155653 >> 172.21.2.104:0/3392155653 conn(0x7fd06805f6b0 msgr2=0x7fd0681477a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:19.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 -- 172.21.2.104:0/3392155653 shutdown_connections 2021-07-11T03:59:19.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.183+0000 7fd06efbc700 1 -- 172.21.2.104:0/3392155653 wait complete. 2021-07-11T03:59:19.197 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T03:59:19.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.243+0000 7fdd48fb1700 1 Processor -- start 2021-07-11T03:59:19.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.243+0000 7fdd48fb1700 1 -- start start 2021-07-11T03:59:19.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fdd440ac500 0x7fdd441407b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4405fe80 con 0x7fdd440607d0 2021-07-11T03:59:19.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd440acc60 con 0x7fdd4405faa0 2021-07-11T03:59:19.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd44143240 con 0x7fdd440ac500 2021-07-11T03:59:19.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd41d9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fdd440607d0 0x7fdd441400a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43312/0 (socket says 172.21.2.104:43312) 2021-07-11T03:59:19.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd41d9b700 1 -- 172.21.2.104:0/3210363940 learned_addr learned my addr 172.21.2.104:0/3210363940 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:19.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624532195 0 0) 0x7fdd4405fe80 con 0x7fdd440607d0 2021-07-11T03:59:19.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd24003260 con 0x7fdd440607d0 2021-07-11T03:59:19.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2458845150 0 0) 0x7fdd24003260 con 0x7fdd440607d0 2021-07-11T03:59:19.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 >> v2:172.21.2.105:3300/0 conn(0x7fdd440ac500 msgr2=0x7fdd441407b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:19.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 --2- 172.21.2.104:0/3210363940 >> v2:172.21.2.105:3300/0 conn(0x7fdd440ac500 0x7fdd441407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 >> v1:172.21.2.105:6789/0 conn(0x7fdd4405faa0 legacy=0x7fdd441438d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:19.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd440acf00 con 0x7fdd440607d0 2021-07-11T03:59:19.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fdd2c001660 con 0x7fdd440607d0 2021-07-11T03:59:19.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 >> v1:172.21.2.104:6789/0 conn(0x7fdd440607d0 legacy=0x7fdd441400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 --2- 172.21.2.104:0/3210363940 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd240037d0 0x7fdd24003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 --2- 172.21.2.104:0/3210363940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd24004500 0x7fdd24004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 --2- 172.21.2.104:0/3210363940 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd24005070 0x7fdd2400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2400bec0 con 0x7fdd24004500 2021-07-11T03:59:19.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2400c030 con 0x7fdd24005070 2021-07-11T03:59:19.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2400c1f0 con 0x7fdd240037d0 2021-07-11T03:59:19.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd41d9b700 1 --2- 172.21.2.104:0/3210363940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd24004500 0x7fdd24004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd41d9b700 1 -- 172.21.2.104:0/3210363940 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd240037d0 msgr2=0x7fdd24003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:19.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd41d9b700 1 --2- 172.21.2.104:0/3210363940 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd240037d0 0x7fdd24003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd41d9b700 1 -- 172.21.2.104:0/3210363940 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd24005070 msgr2=0x7fdd2400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:19.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd41d9b700 1 --2- 172.21.2.104:0/3210363940 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd24005070 0x7fdd2400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd41d9b700 1 -- 172.21.2.104:0/3210363940 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd440acf00 con 0x7fdd24004500 2021-07-11T03:59:19.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd42d9d700 1 -- 172.21.2.104:0/3210363940 reap_dead start 2021-07-11T03:59:19.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd41d9b700 1 --2- 172.21.2.104:0/3210363940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd24004500 0x7fdd24004980 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7fdd2c001020 tx=0x7fdd2c001e20).ready entity=mon.0 client_cookie=f6f38863813426b5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd2c0070d0 con 0x7fdd24004500 2021-07-11T03:59:19.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2c007230 con 0x7fdd24004500 2021-07-11T03:59:19.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd4159a700 1 -- 172.21.2.104:0/3210363940 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd2c00c9c0 con 0x7fdd24004500 2021-07-11T03:59:19.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 -- 172.21.2.104:0/3210363940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd24004500 msgr2=0x7fdd24004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 --2- 172.21.2.104:0/3210363940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd24004500 0x7fdd24004980 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7fdd2c001020 tx=0x7fdd2c001e20).stop 2021-07-11T03:59:19.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 -- 172.21.2.104:0/3210363940 shutdown_connections 2021-07-11T03:59:19.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 --2- 172.21.2.104:0/3210363940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd24004500 0x7fdd24004980 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 -- 172.21.2.104:0/3210363940 >> 172.21.2.104:0/3210363940 conn(0x7fdd440b2970 msgr2=0x7fdd440b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:19.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 -- 172.21.2.104:0/3210363940 shutdown_connections 2021-07-11T03:59:19.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 -- 172.21.2.104:0/3210363940 wait complete. 2021-07-11T03:59:19.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 Processor -- start 2021-07-11T03:59:19.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.247+0000 7fdd48fb1700 1 -- start start 2021-07-11T03:59:19.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd48fb1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd24004500 0x7fdd440aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd48fb1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd440aef00 0x7fdd440ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd48fb1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd441558b0 0x7fdd44155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd48fb1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44146580 con 0x7fdd441558b0 2021-07-11T03:59:19.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd48fb1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44145da0 con 0x7fdd440aef00 2021-07-11T03:59:19.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd48fb1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd44146040 con 0x7fdd24004500 2021-07-11T03:59:19.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd42d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd441558b0 0x7fdd44155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd42d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd441558b0 0x7fdd44155d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46558/0 (socket says 172.21.2.104:46558) 2021-07-11T03:59:19.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd42d9d700 1 -- 172.21.2.104:0/1531981318 learned_addr learned my addr 172.21.2.104:0/1531981318 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:19.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd42d9d700 1 -- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd24004500 msgr2=0x7fdd440aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd42d9d700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd24004500 0x7fdd440aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd42d9d700 1 -- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd440aef00 msgr2=0x7fdd440ad3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd42d9d700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd440aef00 0x7fdd440ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd42d9d700 1 -- 172.21.2.104:0/1531981318 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd441462e0 con 0x7fdd441558b0 2021-07-11T03:59:19.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd42d9d700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd441558b0 0x7fdd44155d60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fdd440b5da0 tx=0x7fdd3c007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd3b7fe700 1 -- 172.21.2.104:0/1531981318 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd3c005a20 con 0x7fdd441558b0 2021-07-11T03:59:19.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd3b7fe700 1 -- 172.21.2.104:0/1531981318 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c005b80 con 0x7fdd441558b0 2021-07-11T03:59:19.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd3b7fe700 1 -- 172.21.2.104:0/1531981318 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdd3c007070 con 0x7fdd441558b0 2021-07-11T03:59:19.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd48fb1700 1 -- 172.21.2.104:0/1531981318 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd44147480 con 0x7fdd441558b0 2021-07-11T03:59:19.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd48fb1700 1 -- 172.21.2.104:0/1531981318 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd440ada60 con 0x7fdd441558b0 2021-07-11T03:59:19.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd48fb1700 1 -- 172.21.2.104:0/1531981318 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd44057350 con 0x7fdd441558b0 2021-07-11T03:59:19.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd3b7fe700 1 -- 172.21.2.104:0/1531981318 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fdd3c010030 con 0x7fdd441558b0 2021-07-11T03:59:19.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd3b7fe700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd28053160 0x7fdd28055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd3b7fe700 1 -- 172.21.2.104:0/1531981318 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdd3c06a6a0 con 0x7fdd441558b0 2021-07-11T03:59:19.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.251+0000 7fdd4259c700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd28053160 0x7fdd28055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.255+0000 7fdd4259c700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd28053160 0x7fdd28055620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fdd34000f70 tx=0x7fdd340069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.255+0000 7fdd3b7fe700 1 -- 172.21.2.104:0/1531981318 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fdd3c0418d0 con 0x7fdd441558b0 2021-07-11T03:59:19.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.335+0000 7fdd48fb1700 1 -- 172.21.2.104:0/1531981318 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd440ac0d0 con 0x7fdd441558b0 2021-07-11T03:59:19.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd3b7fe700 1 -- 172.21.2.104:0/1531981318 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+10 (secure 0 0 0) 0x7fdd3c044ef0 con 0x7fdd441558b0 2021-07-11T03:59:19.342 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:59:19.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 -- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd28053160 msgr2=0x7fdd28055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd28053160 0x7fdd28055620 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fdd34000f70 tx=0x7fdd340069b0).stop 2021-07-11T03:59:19.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 -- 172.21.2.104:0/1531981318 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd441558b0 msgr2=0x7fdd44155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd441558b0 0x7fdd44155d60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7fdd440b5da0 tx=0x7fdd3c007f00).stop 2021-07-11T03:59:19.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 -- 172.21.2.104:0/1531981318 shutdown_connections 2021-07-11T03:59:19.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdd28053160 0x7fdd28055620 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdd441558b0 0x7fdd44155d60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdd440aef00 0x7fdd440ad3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 --2- 172.21.2.104:0/1531981318 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdd24004500 0x7fdd440aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 -- 172.21.2.104:0/1531981318 >> 172.21.2.104:0/1531981318 conn(0x7fdd440b2970 msgr2=0x7fdd440b07a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:19.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 -- 172.21.2.104:0/1531981318 shutdown_connections 2021-07-11T03:59:19.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.339+0000 7fdd48fb1700 1 -- 172.21.2.104:0/1531981318 wait complete. 2021-07-11T03:59:19.350 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T03:59:19.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe67e106700 1 Processor -- start 2021-07-11T03:59:19.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe67e106700 1 -- start start 2021-07-11T03:59:19.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe67e106700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe678144440 0x7fe678144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe67e106700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6780604e0 con 0x7fe678060100 2021-07-11T03:59:19.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe67e106700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6780b23b0 con 0x7fe678065420 2021-07-11T03:59:19.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe67e106700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe678065c80 con 0x7fe678144440 2021-07-11T03:59:19.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6777fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe678060100 0x7fe678064d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43330/0 (socket says 172.21.2.104:43330) 2021-07-11T03:59:19.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6777fe700 1 -- 172.21.2.104:0/2907826194 learned_addr learned my addr 172.21.2.104:0/2907826194 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:19.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012049843 0 0) 0x7fe6780604e0 con 0x7fe678060100 2021-07-11T03:59:19.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe664003260 con 0x7fe678060100 2021-07-11T03:59:19.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3369996284 0 0) 0x7fe664003260 con 0x7fe678060100 2021-07-11T03:59:19.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe678144440 msgr2=0x7fe678144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:19.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 --2- 172.21.2.104:0/2907826194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe678144440 0x7fe678144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 >> v1:172.21.2.105:6789/0 conn(0x7fe678065420 legacy=0x7fe67814b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:19.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6780b2650 con 0x7fe678060100 2021-07-11T03:59:19.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fe668001660 con 0x7fe678060100 2021-07-11T03:59:19.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 >> v1:172.21.2.104:6789/0 conn(0x7fe678060100 legacy=0x7fe678064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 --2- 172.21.2.104:0/2907826194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe664003800 0x7fe664003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 --2- 172.21.2.104:0/2907826194 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe664004500 0x7fe664004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 --2- 172.21.2.104:0/2907826194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6640050a0 0x7fe66400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe678065c80 con 0x7fe664003800 2021-07-11T03:59:19.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66400be00 con 0x7fe664004500 2021-07-11T03:59:19.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66400bfc0 con 0x7fe6640050a0 2021-07-11T03:59:19.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6777fe700 1 --2- 172.21.2.104:0/2907826194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe664003800 0x7fe664003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6777fe700 1 -- 172.21.2.104:0/2907826194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6640050a0 msgr2=0x7fe66400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:19.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6777fe700 1 --2- 172.21.2.104:0/2907826194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6640050a0 0x7fe66400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6777fe700 1 -- 172.21.2.104:0/2907826194 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe664004500 msgr2=0x7fe664004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:19.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6777fe700 1 --2- 172.21.2.104:0/2907826194 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe664004500 0x7fe664004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6777fe700 1 -- 172.21.2.104:0/2907826194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe664003260 con 0x7fe664003800 2021-07-11T03:59:19.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6777fe700 1 --2- 172.21.2.104:0/2907826194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe664003800 0x7fe664003c00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7fe6780b5480 tx=0x7fe668001d60).ready entity=mon.0 client_cookie=bc79c07348af8a68 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe6680070d0 con 0x7fe664003800 2021-07-11T03:59:19.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe668007230 con 0x7fe664003800 2021-07-11T03:59:19.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe6767fc700 1 -- 172.21.2.104:0/2907826194 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe66800c8f0 con 0x7fe664003800 2021-07-11T03:59:19.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe67e106700 1 -- 172.21.2.104:0/2907826194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe664003800 msgr2=0x7fe664003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe67e106700 1 --2- 172.21.2.104:0/2907826194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe664003800 0x7fe664003c00 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7fe6780b5480 tx=0x7fe668001d60).stop 2021-07-11T03:59:19.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.399+0000 7fe677fff700 1 -- 172.21.2.104:0/2907826194 reap_dead start 2021-07-11T03:59:19.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 -- 172.21.2.104:0/2907826194 shutdown_connections 2021-07-11T03:59:19.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 -- 172.21.2.104:0/2907826194 >> 172.21.2.104:0/2907826194 conn(0x7fe67805cf60 msgr2=0x7fe67805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:19.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 -- 172.21.2.104:0/2907826194 shutdown_connections 2021-07-11T03:59:19.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 -- 172.21.2.104:0/2907826194 wait complete. 2021-07-11T03:59:19.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 Processor -- start 2021-07-11T03:59:19.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 -- start start 2021-07-11T03:59:19.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe67814ccd0 0x7fe678159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6780b02c0 0x7fe678159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe67815a470 0x7fe6780ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780ad0d0 con 0x7fe67814ccd0 2021-07-11T03:59:19.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67814d0b0 con 0x7fe67815a470 2021-07-11T03:59:19.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe678144240 con 0x7fe6780b02c0 2021-07-11T03:59:19.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe6777fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe67814ccd0 0x7fe678159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe6777fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe67814ccd0 0x7fe678159a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46574/0 (socket says 172.21.2.104:46574) 2021-07-11T03:59:19.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe6777fe700 1 -- 172.21.2.104:0/2442254402 learned_addr learned my addr 172.21.2.104:0/2442254402 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:19.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe6777fe700 1 -- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6780b02c0 msgr2=0x7fe678159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe6777fe700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6780b02c0 0x7fe678159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe6777fe700 1 -- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe67815a470 msgr2=0x7fe6780ae720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe6777fe700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe67815a470 0x7fe6780ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe6777fe700 1 -- 172.21.2.104:0/2442254402 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6780b0950 con 0x7fe67814ccd0 2021-07-11T03:59:19.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe6777fe700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe67814ccd0 0x7fe678159a90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fe6780b5480 tx=0x7fe668015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe674ff9700 1 -- 172.21.2.104:0/2442254402 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe668001b40 con 0x7fe67814ccd0 2021-07-11T03:59:19.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe674ff9700 1 -- 172.21.2.104:0/2442254402 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6680049e0 con 0x7fe67814ccd0 2021-07-11T03:59:19.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe674ff9700 1 -- 172.21.2.104:0/2442254402 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe668004d40 con 0x7fe67814ccd0 2021-07-11T03:59:19.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 -- 172.21.2.104:0/2442254402 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6780b0bf0 con 0x7fe67814ccd0 2021-07-11T03:59:19.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 -- 172.21.2.104:0/2442254402 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6780afec0 con 0x7fe67814ccd0 2021-07-11T03:59:19.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe67e106700 1 -- 172.21.2.104:0/2442254402 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6780b0d80 con 0x7fe67814ccd0 2021-07-11T03:59:19.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe674ff9700 1 -- 172.21.2.104:0/2442254402 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe668010070 con 0x7fe67814ccd0 2021-07-11T03:59:19.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe674ff9700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6580532d0 0x7fe658055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.403+0000 7fe674ff9700 1 -- 172.21.2.104:0/2442254402 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe66806c260 con 0x7fe67814ccd0 2021-07-11T03:59:19.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.407+0000 7fe676ffd700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6580532d0 0x7fe658055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.407+0000 7fe676ffd700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6580532d0 0x7fe658055790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fe66c002f70 tx=0x7fe66c006a30).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.407+0000 7fe674ff9700 1 -- 172.21.2.104:0/2442254402 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe668043550 con 0x7fe67814ccd0 2021-07-11T03:59:19.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.487+0000 7fe67e106700 1 -- 172.21.2.104:0/2442254402 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe6780b0d80 con 0x7fe67814ccd0 2021-07-11T03:59:19.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe674ff9700 1 -- 172.21.2.104:0/2442254402 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe668001d60 con 0x7fe67814ccd0 2021-07-11T03:59:19.495 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T03:59:19.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 -- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6580532d0 msgr2=0x7fe658055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6580532d0 0x7fe658055790 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fe66c002f70 tx=0x7fe66c006a30).stop 2021-07-11T03:59:19.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 -- 172.21.2.104:0/2442254402 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe67814ccd0 msgr2=0x7fe678159a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe67814ccd0 0x7fe678159a90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7fe6780b5480 tx=0x7fe668015000).stop 2021-07-11T03:59:19.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 -- 172.21.2.104:0/2442254402 shutdown_connections 2021-07-11T03:59:19.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe67815a470 0x7fe6780ae720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6780b02c0 0x7fe678159f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe6580532d0 0x7fe658055790 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 --2- 172.21.2.104:0/2442254402 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe67814ccd0 0x7fe678159a90 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 -- 172.21.2.104:0/2442254402 >> 172.21.2.104:0/2442254402 conn(0x7fe67805cf60 msgr2=0x7fe6780b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:19.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 -- 172.21.2.104:0/2442254402 shutdown_connections 2021-07-11T03:59:19.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.491+0000 7fe67e106700 1 -- 172.21.2.104:0/2442254402 wait complete. 2021-07-11T03:59:19.507 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T03:59:19.508 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:59:19.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 Processor -- start 2021-07-11T03:59:19.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 -- start start 2021-07-11T03:59:19.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f47a00a76b0 0x7f47a0066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f47a00b1970 0x7f47a0066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a00b0060 0x7f47a005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a0067450 con 0x7f47a00b1970 2021-07-11T03:59:19.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47a00a7a90 con 0x7f47a00a76b0 2021-07-11T03:59:19.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a0147620 con 0x7f47a00b0060 2021-07-11T03:59:19.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479e7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f47a00b1970 0x7f47a0066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479e7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f47a00b1970 0x7f47a0066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46586/0 (socket says 172.21.2.104:46586) 2021-07-11T03:59:19.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479e7fc700 1 -- 172.21.2.104:0/3319366347 learned_addr learned my addr 172.21.2.104:0/3319366347 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:19.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479e7fc700 1 -- 172.21.2.104:0/3319366347 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a00b0060 msgr2=0x7f47a005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:19.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479e7fc700 1 --2- 172.21.2.104:0/3319366347 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a00b0060 0x7f47a005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479e7fc700 1 -- 172.21.2.104:0/3319366347 >> v2:172.21.2.105:3300/0 conn(0x7f47a00a76b0 msgr2=0x7f47a0066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:19.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479e7fc700 1 --2- 172.21.2.104:0/3319366347 >> v2:172.21.2.105:3300/0 conn(0x7f47a00a76b0 0x7f47a0066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479e7fc700 1 -- 172.21.2.104:0/3319366347 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a01478c0 con 0x7f47a00b1970 2021-07-11T03:59:19.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479e7fc700 1 --2- 172.21.2.104:0/3319366347 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a00b1970 0x7f47a0066f60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f4790002f70 tx=0x7f4790006eb0).ready entity=mon.0 client_cookie=6e6c20e7c019b2ed server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479dffb700 1 -- 172.21.2.104:0/3319366347 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f479000d570 con 0x7f47a00b1970 2021-07-11T03:59:19.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479dffb700 1 -- 172.21.2.104:0/3319366347 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4790000e90 con 0x7f47a00b1970 2021-07-11T03:59:19.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f479dffb700 1 -- 172.21.2.104:0/3319366347 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f479000d900 con 0x7f47a00b1970 2021-07-11T03:59:19.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 -- 172.21.2.104:0/3319366347 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a00b1970 msgr2=0x7f47a0066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 --2- 172.21.2.104:0/3319366347 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a00b1970 0x7f47a0066f60 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f4790002f70 tx=0x7f4790006eb0).stop 2021-07-11T03:59:19.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 -- 172.21.2.104:0/3319366347 shutdown_connections 2021-07-11T03:59:19.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 --2- 172.21.2.104:0/3319366347 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a00b0060 0x7f47a005f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 --2- 172.21.2.104:0/3319366347 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a00b1970 0x7f47a0066f60 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 --2- 172.21.2.104:0/3319366347 >> v2:172.21.2.105:3300/0 conn(0x7f47a00a76b0 0x7f47a0066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 -- 172.21.2.104:0/3319366347 >> 172.21.2.104:0/3319366347 conn(0x7f47a00b53e0 msgr2=0x7f47a00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:19.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 -- 172.21.2.104:0/3319366347 shutdown_connections 2021-07-11T03:59:19.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.559+0000 7f47a5505700 1 -- 172.21.2.104:0/3319366347 wait complete. 2021-07-11T03:59:19.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f47a5505700 1 Processor -- start 2021-07-11T03:59:19.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f47a5505700 1 -- start start 2021-07-11T03:59:19.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f47a5505700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a00a76b0 0x7f47a00abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f47a5505700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f47a00b0060 0x7f47a0144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f47a5505700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a00b1970 0x7f47a0144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f47a5505700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a00a9ba0 con 0x7f47a00b1970 2021-07-11T03:59:19.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f47a5505700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a00a9910 con 0x7f47a00b0060 2021-07-11T03:59:19.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f47a5505700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a00af310 con 0x7f47a00a76b0 2021-07-11T03:59:19.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f479f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a00b1970 0x7f47a0144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f479f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a00b1970 0x7f47a0144af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46588/0 (socket says 172.21.2.104:46588) 2021-07-11T03:59:19.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f479f7fe700 1 -- 172.21.2.104:0/3858226934 learned_addr learned my addr 172.21.2.104:0/3858226934 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:19.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f479f7fe700 1 -- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a00a76b0 msgr2=0x7f47a00abee0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:59:19.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f479f7fe700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a00a76b0 0x7f47a00abee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f479f7fe700 1 -- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f47a00b0060 msgr2=0x7f47a0144600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T03:59:19.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f479f7fe700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f47a00b0060 0x7f47a0144600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f479f7fe700 1 -- 172.21.2.104:0/3858226934 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479000b1e0 con 0x7f47a00b1970 2021-07-11T03:59:19.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f479f7fe700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a00b1970 0x7f47a0144af0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f47a00aecf0 tx=0x7f47980091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f4797fff700 1 -- 172.21.2.104:0/3858226934 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4798005c30 con 0x7f47a00b1970 2021-07-11T03:59:19.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f47a5505700 1 -- 172.21.2.104:0/3858226934 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a00aedd0 con 0x7f47a00b1970 2021-07-11T03:59:19.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f4797fff700 1 -- 172.21.2.104:0/3858226934 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4798005d90 con 0x7f47a00b1970 2021-07-11T03:59:19.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f4797fff700 1 -- 172.21.2.104:0/3858226934 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4798006220 con 0x7f47a00b1970 2021-07-11T03:59:19.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f47a5505700 1 -- 172.21.2.104:0/3858226934 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a00ab870 con 0x7f47a00b1970 2021-07-11T03:59:19.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f47a5505700 1 -- 172.21.2.104:0/3858226934 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a0057350 con 0x7f47a00b1970 2021-07-11T03:59:19.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f4797fff700 1 -- 172.21.2.104:0/3858226934 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f479800f030 con 0x7f47a00b1970 2021-07-11T03:59:19.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f4797fff700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4784053240 0x7f4784055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:19.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f4797fff700 1 -- 172.21.2.104:0/3858226934 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f479806b780 con 0x7f47a00b1970 2021-07-11T03:59:19.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.563+0000 7f479effd700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4784053240 0x7f4784055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:19.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.567+0000 7f479effd700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4784053240 0x7f4784055700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f4788002f70 tx=0x7f4788006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:19.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.567+0000 7f4797fff700 1 -- 172.21.2.104:0/3858226934 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f47980429b0 con 0x7f47a00b1970 2021-07-11T03:59:19.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.671+0000 7f47a5505700 1 -- 172.21.2.104:0/3858226934 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47a00698e0 con 0x7f47a00b1970 2021-07-11T03:59:19.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.671+0000 7f4797fff700 1 -- 172.21.2.104:0/3858226934 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f4798045fd0 con 0x7f47a00b1970 2021-07-11T03:59:19.678 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:19.678 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/140397159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":140397159},{"type":"v1","addr":"172.21.2.105:6835","nonce":140397159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/2616903361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2616903361},{"type":"v1","addr":"172.21.2.104:6834","nonce":2616903361}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:59:13.620413+0000","modified":"2021-07-11T03:59:14.624165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/611806767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":611806767},{"type":"v1","addr":"172.21.2.104:6835","nonce":611806767}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-07-11T03:59:19.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 -- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4784053240 msgr2=0x7f4784055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4784053240 0x7f4784055700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f4788002f70 tx=0x7f4788006a00).stop 2021-07-11T03:59:19.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 -- 172.21.2.104:0/3858226934 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a00b1970 msgr2=0x7f47a0144af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:19.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a00b1970 0x7f47a0144af0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f47a00aecf0 tx=0x7f47980091a0).stop 2021-07-11T03:59:19.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 -- 172.21.2.104:0/3858226934 shutdown_connections 2021-07-11T03:59:19.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4784053240 0x7f4784055700 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a00b1970 0x7f47a0144af0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:19.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f47a00b0060 0x7f47a0144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 --2- 172.21.2.104:0/3858226934 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a00a76b0 0x7f47a00abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:19.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 -- 172.21.2.104:0/3858226934 >> 172.21.2.104:0/3858226934 conn(0x7f47a00b53e0 msgr2=0x7f47a01499e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:19.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 -- 172.21.2.104:0/3858226934 shutdown_connections 2021-07-11T03:59:19.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:19.675+0000 7f47a5505700 1 -- 172.21.2.104:0/3858226934 wait complete. 2021-07-11T03:59:19.683 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 32 2021-07-11T03:59:19.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:59:13.620413+0000', 'modified': '2021-07-11T03:59:14.624165+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6246}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6246': {'gid': 6246, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/611806767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 611806767}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 611806767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T03:59:19.988 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T03:59:19.989 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T03:59:20.059 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":6246,"want_state":"up:active","state":"up:active","rank_uptime":6.4293107340000004,"mdsmap_epoch":32,"osdmap_epoch":76,"osdmap_epoch_barrier":75,"uptime":10.628793236} 2021-07-11T03:59:20.060 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 6246, "mdsmap_epoch": 32, "osdmap_epoch": 76, "osdmap_epoch_barrier": 75, "rank_uptime": 6.429310734, "state": "up:active", "uptime": 10.628793236, "want_state": "up:active", "whoami": 0 } 2021-07-11T03:59:20.061 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T03:59:20.061 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-07-11T03:59:20.061 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba023.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-07-11T03:59:20.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:20.062 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:59:20.071 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T03:59:20.081 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:59:20.081 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:59:20.090 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:59:20.091 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:59:20.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:20.094 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T03:59:20.144 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T03:59:20.144 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:59:20.148 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T03:59:20.159 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T03:59:20.160 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T03:59:20.186 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:2021-07-11T03:59:20.183+0000 7f7c8f073080 -1 init, newargv = 0x5625c6405a70 newargc=9ceph-fuse[14657]: starting ceph client 2021-07-11T03:59:20.187 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr: 2021-07-11T03:59:20.192 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T03:59:20.193 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T03:59:20.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:20.195 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T03:59:20.201 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14657]: starting fuse 2021-07-11T03:59:20.206 INFO:teuthology.orchestra.run.gibba023.stdout:52 2021-07-11T03:59:20.207 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T03:59:20.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:20.209 DEBUG:teuthology.orchestra.run.gibba023:> sudo python3 -c ' 2021-07-11T03:59:20.209 DEBUG:teuthology.orchestra.run.gibba023:> import glob 2021-07-11T03:59:20.211 DEBUG:teuthology.orchestra.run.gibba023:> import re 2021-07-11T03:59:20.212 DEBUG:teuthology.orchestra.run.gibba023:> import os 2021-07-11T03:59:20.212 DEBUG:teuthology.orchestra.run.gibba023:> import subprocess 2021-07-11T03:59:20.213 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T03:59:20.214 DEBUG:teuthology.orchestra.run.gibba023:> def find_socket(client_name): 2021-07-11T03:59:20.215 DEBUG:teuthology.orchestra.run.gibba023:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-07-11T03:59:20.215 DEBUG:teuthology.orchestra.run.gibba023:> files = glob.glob(asok_path) 2021-07-11T03:59:20.217 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T03:59:20.218 DEBUG:teuthology.orchestra.run.gibba023:> # Given a non-glob path, it better be there 2021-07-11T03:59:20.218 DEBUG:teuthology.orchestra.run.gibba023:> if "*" not in asok_path: 2021-07-11T03:59:20.218 DEBUG:teuthology.orchestra.run.gibba023:> assert(len(files) == 1) 2021-07-11T03:59:20.219 DEBUG:teuthology.orchestra.run.gibba023:> return files[0] 2021-07-11T03:59:20.219 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T03:59:20.220 DEBUG:teuthology.orchestra.run.gibba023:> for f in files: 2021-07-11T03:59:20.220 DEBUG:teuthology.orchestra.run.gibba023:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T03:59:20.220 DEBUG:teuthology.orchestra.run.gibba023:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T03:59:20.221 DEBUG:teuthology.orchestra.run.gibba023:> return f 2021-07-11T03:59:20.221 DEBUG:teuthology.orchestra.run.gibba023:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T03:59:20.222 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T03:59:20.222 DEBUG:teuthology.orchestra.run.gibba023:> print(find_socket("client.1")) 2021-07-11T03:59:20.222 DEBUG:teuthology.orchestra.run.gibba023:> ' 2021-07-11T03:59:20.269 INFO:teuthology.orchestra.run.gibba023.stdout:/var/run/ceph/ceph-client.1.14657.asok 2021-07-11T03:59:20.271 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14657.asok 2021-07-11T03:59:20.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:20.273 DEBUG:teuthology.orchestra.run.gibba023:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14657.asok status 2021-07-11T03:59:20.326 INFO:teuthology.orchestra.run.gibba023.stdout:{ 2021-07-11T03:59:20.327 INFO:teuthology.orchestra.run.gibba023.stdout: "metadata": { 2021-07-11T03:59:20.327 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T03:59:20.327 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T03:59:20.328 INFO:teuthology.orchestra.run.gibba023.stdout: "entity_id": "1", 2021-07-11T03:59:20.328 INFO:teuthology.orchestra.run.gibba023.stdout: "hostname": "gibba023", 2021-07-11T03:59:20.328 INFO:teuthology.orchestra.run.gibba023.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-07-11T03:59:20.329 INFO:teuthology.orchestra.run.gibba023.stdout: "pid": "14657", 2021-07-11T03:59:20.329 INFO:teuthology.orchestra.run.gibba023.stdout: "root": "/" 2021-07-11T03:59:20.330 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T03:59:20.330 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_count": 0, 2021-07-11T03:59:20.330 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_pinned_count": 0, 2021-07-11T03:59:20.331 INFO:teuthology.orchestra.run.gibba023.stdout: "id": 4661, 2021-07-11T03:59:20.331 INFO:teuthology.orchestra.run.gibba023.stdout: "inst": { 2021-07-11T03:59:20.331 INFO:teuthology.orchestra.run.gibba023.stdout: "name": { 2021-07-11T03:59:20.331 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "client", 2021-07-11T03:59:20.332 INFO:teuthology.orchestra.run.gibba023.stdout: "num": 4661 2021-07-11T03:59:20.332 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T03:59:20.332 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T03:59:20.332 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T03:59:20.332 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T03:59:20.333 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 4129153972 2021-07-11T03:59:20.333 INFO:teuthology.orchestra.run.gibba023.stdout: } 2021-07-11T03:59:20.333 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T03:59:20.333 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T03:59:20.333 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T03:59:20.334 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T03:59:20.334 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 4129153972 2021-07-11T03:59:20.334 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T03:59:20.334 INFO:teuthology.orchestra.run.gibba023.stdout: "inst_str": "client.4661 172.21.2.123:0/4129153972", 2021-07-11T03:59:20.335 INFO:teuthology.orchestra.run.gibba023.stdout: "addr_str": "172.21.2.123:0/4129153972", 2021-07-11T03:59:20.335 INFO:teuthology.orchestra.run.gibba023.stdout: "inode_count": 1, 2021-07-11T03:59:20.335 INFO:teuthology.orchestra.run.gibba023.stdout: "mds_epoch": 32, 2021-07-11T03:59:20.335 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch": 76, 2021-07-11T03:59:20.335 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch_barrier": 0, 2021-07-11T03:59:20.336 INFO:teuthology.orchestra.run.gibba023.stdout: "blacklisted": false, 2021-07-11T03:59:20.336 INFO:teuthology.orchestra.run.gibba023.stdout: "fs_name": "cephfs" 2021-07-11T03:59:20.336 INFO:teuthology.orchestra.run.gibba023.stdout:} 2021-07-11T03:59:20.338 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:20.338 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:59:20.339 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:59:20.345 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T03:59:20.346 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T03:59:20.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:20.348 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:59:20.395 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:59:20.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0793d06700 1 Processor -- start 2021-07-11T03:59:20.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0793d06700 1 -- start start 2021-07-11T03:59:20.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0793d06700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f078c0aa860 0x7f078c0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0793d06700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078c0ac370 con 0x7f078c0a9da0 2021-07-11T03:59:20.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0793d06700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078c0aac40 con 0x7f078c0ad150 2021-07-11T03:59:20.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0793d06700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c0abb50 con 0x7f078c0aa860 2021-07-11T03:59:20.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f07912a1700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f078c0a9da0 0x7f078c0aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43368/0 (socket says 172.21.2.104:43368) 2021-07-11T03:59:20.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f07912a1700 1 -- 172.21.2.104:0/3634385176 learned_addr learned my addr 172.21.2.104:0/3634385176 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:20.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453266376 0 0) 0x7f078c0ac370 con 0x7f078c0a9da0 2021-07-11T03:59:20.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0774003260 con 0x7f078c0a9da0 2021-07-11T03:59:20.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0791aa2700 1 --2- 172.21.2.104:0/3634385176 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f078c0aa860 0x7f078c0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:20.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1942837121 0 0) 0x7f0774003260 con 0x7f078c0a9da0 2021-07-11T03:59:20.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f078c0aa860 msgr2=0x7f078c0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:20.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 --2- 172.21.2.104:0/3634385176 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f078c0aa860 0x7f078c0b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:20.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 >> v1:172.21.2.105:6789/0 conn(0x7f078c0ad150 legacy=0x7f078c140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c0b16b0 con 0x7f078c0a9da0 2021-07-11T03:59:20.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f0788001660 con 0x7f078c0a9da0 2021-07-11T03:59:20.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 >> v1:172.21.2.104:6789/0 conn(0x7f078c0a9da0 legacy=0x7f078c0aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 --2- 172.21.2.104:0/3634385176 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0774003780 0x7f0774003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 --2- 172.21.2.104:0/3634385176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07740040b0 0x7f0774004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 --2- 172.21.2.104:0/3634385176 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0774004c70 0x7f077400fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07740053c0 con 0x7f07740040b0 2021-07-11T03:59:20.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0774010280 con 0x7f0774004c70 2021-07-11T03:59:20.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0774010440 con 0x7f0774003780 2021-07-11T03:59:20.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f07912a1700 1 --2- 172.21.2.104:0/3634385176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07740040b0 0x7f0774004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:20.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f07912a1700 1 -- 172.21.2.104:0/3634385176 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0774003780 msgr2=0x7f0774003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:20.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f07912a1700 1 --2- 172.21.2.104:0/3634385176 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0774003780 0x7f0774003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f07912a1700 1 -- 172.21.2.104:0/3634385176 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0774004c70 msgr2=0x7f077400fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:20.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f07912a1700 1 --2- 172.21.2.104:0/3634385176 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0774004c70 0x7f077400fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f07912a1700 1 -- 172.21.2.104:0/3634385176 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c0b16b0 con 0x7f07740040b0 2021-07-11T03:59:20.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.447+0000 7f07912a1700 1 --2- 172.21.2.104:0/3634385176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07740040b0 0x7f0774004500 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f0788001370 tx=0x7f0788001dc0).ready entity=mon.0 client_cookie=9e47fc1616d0cfe8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:20.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07880071a0 con 0x7f07740040b0 2021-07-11T03:59:20.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788007300 con 0x7f07740040b0 2021-07-11T03:59:20.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0790aa0700 1 -- 172.21.2.104:0/3634385176 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f078800cae0 con 0x7f07740040b0 2021-07-11T03:59:20.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- 172.21.2.104:0/3634385176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07740040b0 msgr2=0x7f0774004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 --2- 172.21.2.104:0/3634385176 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f07740040b0 0x7f0774004500 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f0788001370 tx=0x7f0788001dc0).stop 2021-07-11T03:59:20.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f07922a3700 1 -- 172.21.2.104:0/3634385176 reap_dead start 2021-07-11T03:59:20.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- 172.21.2.104:0/3634385176 shutdown_connections 2021-07-11T03:59:20.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- 172.21.2.104:0/3634385176 >> 172.21.2.104:0/3634385176 conn(0x7f078c0b64d0 msgr2=0x7f078c0b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:20.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- 172.21.2.104:0/3634385176 shutdown_connections 2021-07-11T03:59:20.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- 172.21.2.104:0/3634385176 wait complete. 2021-07-11T03:59:20.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 Processor -- start 2021-07-11T03:59:20.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- start start 2021-07-11T03:59:20.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f078c060640 0x7f078c060a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f078c14dea0 0x7f078c140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f078c140830 0x7f078c09e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c14eeb0 con 0x7f078c14dea0 2021-07-11T03:59:20.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c151db0 con 0x7f078c140830 2021-07-11T03:59:20.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c152030 con 0x7f078c060640 2021-07-11T03:59:20.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f07912a1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f078c14dea0 0x7f078c140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:20.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f07912a1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f078c14dea0 0x7f078c140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46610/0 (socket says 172.21.2.104:46610) 2021-07-11T03:59:20.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f07912a1700 1 -- 172.21.2.104:0/968998961 learned_addr learned my addr 172.21.2.104:0/968998961 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:20.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f07912a1700 1 -- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f078c060640 msgr2=0x7f078c060a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f07912a1700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f078c060640 0x7f078c060a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f07912a1700 1 -- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f078c140830 msgr2=0x7f078c09e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f07912a1700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f078c140830 0x7f078c09e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f07912a1700 1 -- 172.21.2.104:0/968998961 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c0af6b0 con 0x7f078c14dea0 2021-07-11T03:59:20.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f07912a1700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f078c14dea0 0x7f078c140340 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f078c0af360 tx=0x7f0788015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:20.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f077effd700 1 -- 172.21.2.104:0/968998961 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f07880049e0 con 0x7f078c14dea0 2021-07-11T03:59:20.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f077effd700 1 -- 172.21.2.104:0/968998961 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0788004b40 con 0x7f078c14dea0 2021-07-11T03:59:20.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- 172.21.2.104:0/968998961 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c0af950 con 0x7f078c14dea0 2021-07-11T03:59:20.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f077effd700 1 -- 172.21.2.104:0/968998961 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0788004ea0 con 0x7f078c14dea0 2021-07-11T03:59:20.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- 172.21.2.104:0/968998961 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c0663b0 con 0x7f078c14dea0 2021-07-11T03:59:20.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f0793d06700 1 -- 172.21.2.104:0/968998961 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c0b03e0 con 0x7f078c14dea0 2021-07-11T03:59:20.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f077effd700 1 -- 172.21.2.104:0/968998961 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0788027070 con 0x7f078c14dea0 2021-07-11T03:59:20.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f077effd700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f076c053180 0x7f076c055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.451+0000 7f077effd700 1 -- 172.21.2.104:0/968998961 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f078806bf80 con 0x7f078c14dea0 2021-07-11T03:59:20.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.455+0000 7f0791aa2700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f076c053180 0x7f076c055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:20.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.455+0000 7f0791aa2700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f076c053180 0x7f076c055640 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f078000a000 tx=0x7f0780008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:20.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.455+0000 7f077effd700 1 -- 172.21.2.104:0/968998961 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f078c0b03e0 con 0x7f078c14dea0 2021-07-11T03:59:20.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.563+0000 7f0793d06700 1 -- 172.21.2.104:0/968998961 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f078c0b03e0 con 0x7f078c14dea0 2021-07-11T03:59:20.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.563+0000 7f077effd700 1 -- 172.21.2.104:0/968998961 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f078c0b03e0 con 0x7f078c14dea0 2021-07-11T03:59:20.571 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:20.571 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/140397159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":140397159},{"type":"v1","addr":"172.21.2.105:6835","nonce":140397159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/2616903361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2616903361},{"type":"v1","addr":"172.21.2.104:6834","nonce":2616903361}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:59:13.620413+0000","modified":"2021-07-11T03:59:14.624165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/611806767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":611806767},{"type":"v1","addr":"172.21.2.104:6835","nonce":611806767}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-07-11T03:59:20.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 -- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f076c053180 msgr2=0x7f076c055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f076c053180 0x7f076c055640 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f078000a000 tx=0x7f0780008040).stop 2021-07-11T03:59:20.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 -- 172.21.2.104:0/968998961 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f078c14dea0 msgr2=0x7f078c140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f078c14dea0 0x7f078c140340 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f078c0af360 tx=0x7f0788015000).stop 2021-07-11T03:59:20.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 -- 172.21.2.104:0/968998961 shutdown_connections 2021-07-11T03:59:20.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f078c140830 0x7f078c09e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f076c053180 0x7f076c055640 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:20.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f078c14dea0 0x7f078c140340 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:20.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 --2- 172.21.2.104:0/968998961 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f078c060640 0x7f078c060a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 -- 172.21.2.104:0/968998961 >> 172.21.2.104:0/968998961 conn(0x7f078c0b64d0 msgr2=0x7f078c066920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:20.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 -- 172.21.2.104:0/968998961 shutdown_connections 2021-07-11T03:59:20.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.567+0000 7f0793d06700 1 -- 172.21.2.104:0/968998961 wait complete. 2021-07-11T03:59:20.575 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 32 2021-07-11T03:59:20.577 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T03:59:20.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0faf5700 1 Processor -- start 2021-07-11T03:59:20.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0faf5700 1 -- start start 2021-07-11T03:59:20.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0faf5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8a080aa860 0x7f8a080b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0faf5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a080ad150 0x7f8a08140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0faf5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a080ac370 con 0x7f8a080a9da0 2021-07-11T03:59:20.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0faf5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a080abb50 con 0x7f8a080aa860 2021-07-11T03:59:20.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0faf5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080aac40 con 0x7f8a080ad150 2021-07-11T03:59:20.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0d090700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8a080a9da0 0x7f8a080aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43386/0 (socket says 172.21.2.104:43386) 2021-07-11T03:59:20.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0d891700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8a080aa860 0x7f8a080b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:20.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0d090700 1 -- 172.21.2.104:0/1548202280 learned_addr learned my addr 172.21.2.104:0/1548202280 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T03:59:20.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0d891700 1 -- 172.21.2.104:0/1548202280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a080ad150 msgr2=0x7f8a08140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T03:59:20.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0d891700 1 --2- 172.21.2.104:0/1548202280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a080ad150 0x7f8a08140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0d891700 1 -- 172.21.2.104:0/1548202280 >> v1:172.21.2.104:6789/0 conn(0x7f8a080a9da0 legacy=0x7f8a080aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0d891700 1 -- 172.21.2.104:0/1548202280 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a080b16b0 con 0x7f8a080aa860 2021-07-11T03:59:20.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0d891700 1 --2- 172.21.2.104:0/1548202280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a080aa860 0x7f8a080b11c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f89f8002f70 tx=0x7f89f8006df0).ready entity=mon.0 client_cookie=b04c7e189aa5be9c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:20.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.623+0000 7f8a0c88f700 1 -- 172.21.2.104:0/1548202280 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89f800d570 con 0x7f8a080aa860 2021-07-11T03:59:20.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0c88f700 1 -- 172.21.2.104:0/1548202280 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f8000e90 con 0x7f8a080aa860 2021-07-11T03:59:20.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0c88f700 1 -- 172.21.2.104:0/1548202280 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89f800d8c0 con 0x7f8a080aa860 2021-07-11T03:59:20.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- 172.21.2.104:0/1548202280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a080aa860 msgr2=0x7f8a080b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 --2- 172.21.2.104:0/1548202280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a080aa860 0x7f8a080b11c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f89f8002f70 tx=0x7f89f8006df0).stop 2021-07-11T03:59:20.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- 172.21.2.104:0/1548202280 shutdown_connections 2021-07-11T03:59:20.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 --2- 172.21.2.104:0/1548202280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a080ad150 0x7f8a08140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 --2- 172.21.2.104:0/1548202280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a080aa860 0x7f8a080b11c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:20.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- 172.21.2.104:0/1548202280 >> 172.21.2.104:0/1548202280 conn(0x7f8a080b64d0 msgr2=0x7f8a080b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:20.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- 172.21.2.104:0/1548202280 shutdown_connections 2021-07-11T03:59:20.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- 172.21.2.104:0/1548202280 wait complete. 2021-07-11T03:59:20.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 Processor -- start 2021-07-11T03:59:20.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- start start 2021-07-11T03:59:20.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a080a9da0 0x7f8a08142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a080aa860 0x7f8a0814ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a080ad150 0x7f8a0814f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08066640 con 0x7f8a080a9da0 2021-07-11T03:59:20.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a080663a0 con 0x7f8a080aa860 2021-07-11T03:59:20.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a08064910 con 0x7f8a080ad150 2021-07-11T03:59:20.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0d891700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a080a9da0 0x7f8a08142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:20.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0d891700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a080a9da0 0x7f8a08142fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46624/0 (socket says 172.21.2.104:46624) 2021-07-11T03:59:20.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0d891700 1 -- 172.21.2.104:0/2295344598 learned_addr learned my addr 172.21.2.104:0/2295344598 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T03:59:20.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0d891700 1 -- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a080ad150 msgr2=0x7f8a0814f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0d891700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a080ad150 0x7f8a0814f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0d891700 1 -- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a080aa860 msgr2=0x7f8a0814ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0d891700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a080aa860 0x7f8a0814ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0d891700 1 -- 172.21.2.104:0/2295344598 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f800b180 con 0x7f8a080a9da0 2021-07-11T03:59:20.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0d891700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a080a9da0 0x7f8a08142fe0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f89f8002f70 tx=0x7f89f8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:20.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f89f6ffd700 1 -- 172.21.2.104:0/2295344598 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89f801f070 con 0x7f8a080a9da0 2021-07-11T03:59:20.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f89f6ffd700 1 -- 172.21.2.104:0/2295344598 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f800d8f0 con 0x7f8a080a9da0 2021-07-11T03:59:20.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f89f6ffd700 1 -- 172.21.2.104:0/2295344598 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89f801ace0 con 0x7f8a080a9da0 2021-07-11T03:59:20.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- 172.21.2.104:0/2295344598 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0814f920 con 0x7f8a080a9da0 2021-07-11T03:59:20.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- 172.21.2.104:0/2295344598 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a08063ce0 con 0x7f8a080a9da0 2021-07-11T03:59:20.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f8a0faf5700 1 -- 172.21.2.104:0/2295344598 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a08064ad0 con 0x7f8a080a9da0 2021-07-11T03:59:20.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f89f6ffd700 1 -- 172.21.2.104:0/2295344598 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f89f802b070 con 0x7f8a080a9da0 2021-07-11T03:59:20.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f89f6ffd700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f89e8053300 0x7f89e80557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T03:59:20.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.627+0000 7f89f6ffd700 1 -- 172.21.2.104:0/2295344598 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f89f806dd40 con 0x7f8a080a9da0 2021-07-11T03:59:20.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.631+0000 7f8a0d090700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f89e8053300 0x7f89e80557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T03:59:20.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.631+0000 7f8a0d090700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f89e8053300 0x7f89e80557c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f8a080b63e0 tx=0x7f8a04007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T03:59:20.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.631+0000 7f89f6ffd700 1 -- 172.21.2.104:0/2295344598 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f89f8048e60 con 0x7f8a080a9da0 2021-07-11T03:59:20.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f8a0faf5700 1 -- 172.21.2.104:0/2295344598 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a08057350 con 0x7f8a080a9da0 2021-07-11T03:59:20.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f89f6ffd700 1 -- 172.21.2.104:0/2295344598 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f89f8074080 con 0x7f8a080a9da0 2021-07-11T03:59:20.746 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T03:59:20.746 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/140397159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":140397159},{"type":"v1","addr":"172.21.2.105:6835","nonce":140397159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/2616903361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2616903361},{"type":"v1","addr":"172.21.2.104:6834","nonce":2616903361}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:59:13.620413+0000","modified":"2021-07-11T03:59:14.624165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/611806767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":611806767},{"type":"v1","addr":"172.21.2.104:6835","nonce":611806767}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-07-11T03:59:20.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f8a0faf5700 1 -- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f89e8053300 msgr2=0x7f89e80557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f8a0faf5700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f89e8053300 0x7f89e80557c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f8a080b63e0 tx=0x7f8a04007800).stop 2021-07-11T03:59:20.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f8a0faf5700 1 -- 172.21.2.104:0/2295344598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a080a9da0 msgr2=0x7f8a08142fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T03:59:20.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f8a0faf5700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a080a9da0 0x7f8a08142fe0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f89f8002f70 tx=0x7f89f8015000).stop 2021-07-11T03:59:20.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f8a0faf5700 1 -- 172.21.2.104:0/2295344598 shutdown_connections 2021-07-11T03:59:20.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f8a0faf5700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a080ad150 0x7f8a0814f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f8a0faf5700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a080aa860 0x7f8a0814ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T03:59:20.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f8a0faf5700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f89e8053300 0x7f89e80557c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:20.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f8a0faf5700 1 --2- 172.21.2.104:0/2295344598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a080a9da0 0x7f8a08142fe0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T03:59:20.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.739+0000 7f8a0faf5700 1 -- 172.21.2.104:0/2295344598 >> 172.21.2.104:0/2295344598 conn(0x7f8a080b64d0 msgr2=0x7f8a080b40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T03:59:20.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.743+0000 7f8a0faf5700 1 -- 172.21.2.104:0/2295344598 shutdown_connections 2021-07-11T03:59:20.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T03:59:20.743+0000 7f8a0faf5700 1 -- 172.21.2.104:0/2295344598 wait complete. 2021-07-11T03:59:20.754 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 32 2021-07-11T03:59:20.757 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json config set mds_cap_revoke_eviction_timeout 30.0 2021-07-11T03:59:20.814 INFO:teuthology.orchestra.run.gibba004.stdout:{"success":"mds_cap_revoke_eviction_timeout = '30.000000' "} 2021-07-11T03:59:20.815 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_cap_revoke_eviction_timeout = '30.000000' " } 2021-07-11T03:59:20.815 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:20.816 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:59:20.816 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T03:59:20.827 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T03:59:20.828 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T03:59:20.829 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-07-11T03:59:20.829 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-07-11T03:59:20.830 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T03:59:20.830 DEBUG:teuthology.orchestra.run.gibba019:> with open("/home/ubuntu/cephtest/mnt.0/background_file", '"'"'w'"'"') as f: 2021-07-11T03:59:20.831 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-07-11T03:59:20.831 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-07-11T03:59:20.831 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content2'"'"') 2021-07-11T03:59:20.832 DEBUG:teuthology.orchestra.run.gibba019:> while True: 2021-07-11T03:59:20.832 DEBUG:teuthology.orchestra.run.gibba019:> time.sleep(1) 2021-07-11T03:59:20.832 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T03:59:20.841 DEBUG:teuthology.orchestra.run.gibba019:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2021-07-11T03:59:20.856 INFO:teuthology.orchestra.run.gibba019.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/background_file': No such file or directory 2021-07-11T03:59:20.856 DEBUG:teuthology.orchestra.run:got remote process result: 2 2021-07-11T03:59:21.858 DEBUG:teuthology.orchestra.run.gibba019:> sudo ls /home/ubuntu/cephtest/mnt.0/background_file 2021-07-11T03:59:21.870 INFO:teuthology.orchestra.run.gibba019.stdout:/home/ubuntu/cephtest/mnt.0/background_file 2021-07-11T03:59:21.871 DEBUG:tasks.cephfs.mount:File background_file became visible from 0 after 1s 2021-07-11T03:59:21.871 DEBUG:teuthology.orchestra.run.gibba023:> sudo ls /home/ubuntu/cephtest/mnt.1/background_file 2021-07-11T03:59:21.918 INFO:teuthology.orchestra.run.gibba023.stdout:/home/ubuntu/cephtest/mnt.1/background_file 2021-07-11T03:59:21.919 DEBUG:tasks.cephfs.mount:File background_file became visible from 1 after 0s 2021-07-11T03:59:21.920 INFO:tasks.cephfs.fuse_mount:Killing ceph-fuse connection on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T03:59:21.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T03:59:21.931 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T03:59:21.931 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T03:59:21.969 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T03:59:21.970 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T03:59:21.971 DEBUG:teuthology.orchestra.run.gibba023:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-07-11T03:59:21.971 DEBUG:teuthology.orchestra.run.gibba023:> import os 2021-07-11T03:59:21.972 DEBUG:teuthology.orchestra.run.gibba023:> import time 2021-07-11T03:59:21.972 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T03:59:21.972 DEBUG:teuthology.orchestra.run.gibba023:> fd = os.open("/home/ubuntu/cephtest/mnt.1/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2021-07-11T03:59:21.973 DEBUG:teuthology.orchestra.run.gibba023:> try: 2021-07-11T03:59:21.973 DEBUG:teuthology.orchestra.run.gibba023:> while True: 2021-07-11T03:59:21.974 DEBUG:teuthology.orchestra.run.gibba023:> os.write(fd, b'"'"'content'"'"') 2021-07-11T03:59:21.974 DEBUG:teuthology.orchestra.run.gibba023:> time.sleep(1) 2021-07-11T03:59:21.974 DEBUG:teuthology.orchestra.run.gibba023:> if not False: 2021-07-11T03:59:21.975 DEBUG:teuthology.orchestra.run.gibba023:> break 2021-07-11T03:59:21.975 DEBUG:teuthology.orchestra.run.gibba023:> except IOError as e: 2021-07-11T03:59:21.976 DEBUG:teuthology.orchestra.run.gibba023:> pass 2021-07-11T03:59:21.976 DEBUG:teuthology.orchestra.run.gibba023:> os.close(fd) 2021-07-11T03:59:21.976 DEBUG:teuthology.orchestra.run.gibba023:> ' 2021-07-11T03:59:42.653 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:59:42.657 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:59:42.660 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T03:59:42.664 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:00:00.546 INFO:tasks.cephfs.test_misc:cap_waiter waited 38.562819957733154s 2021-07-11T04:00:00.547 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist ls --format=json 2021-07-11T04:00:00.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.601+0000 7fb08eb1a700 1 Processor -- start 2021-07-11T04:00:00.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.601+0000 7fb08eb1a700 1 -- start start 2021-07-11T04:00:00.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.601+0000 7fb08eb1a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb0880b6580 0x7fb0880b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.601+0000 7fb08eb1a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb0880b6960 0x7fb08812ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.601+0000 7fb08eb1a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0881440b0 con 0x7fb0880b43c0 2021-07-11T04:00:00.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.601+0000 7fb08eb1a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb088143b70 con 0x7fb0880b6580 2021-07-11T04:00:00.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.601+0000 7fb08eb1a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb088143e10 con 0x7fb0880b6960 2021-07-11T04:00:00.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.601+0000 7fb08c8b6700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb0880b6580 0x7fb0880b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:00.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.601+0000 7fb087fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb0880b43c0 0x7fb088061d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43402/0 (socket says 172.21.2.104:43402) 2021-07-11T04:00:00.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.601+0000 7fb087fff700 1 -- 172.21.2.104:0/2512905069 learned_addr learned my addr 172.21.2.104:0/2512905069 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:00.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb0877fe700 1 -- 172.21.2.104:0/2512905069 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369177884 0 0) 0x7fb0881440b0 con 0x7fb0880b43c0 2021-07-11T04:00:00.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb0877fe700 1 -- 172.21.2.104:0/2512905069 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb068003260 con 0x7fb0880b43c0 2021-07-11T04:00:00.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08c8b6700 1 -- 172.21.2.104:0/2512905069 >> v2:172.21.2.105:3300/0 conn(0x7fb0880b6960 msgr2=0x7fb08812ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:00.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08c8b6700 1 --2- 172.21.2.104:0/2512905069 >> v2:172.21.2.105:3300/0 conn(0x7fb0880b6960 0x7fb08812ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:00.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08c8b6700 1 -- 172.21.2.104:0/2512905069 >> v1:172.21.2.104:6789/0 conn(0x7fb0880b43c0 legacy=0x7fb088061d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08c8b6700 1 -- 172.21.2.104:0/2512905069 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0880615f0 con 0x7fb0880b6580 2021-07-11T04:00:00.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08c8b6700 1 --2- 172.21.2.104:0/2512905069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0880b6580 0x7fb0880b5bf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fb088143120 tx=0x7fb078006df0).ready entity=mon.0 client_cookie=7ab55b70c63907c7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:00.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb0877fe700 1 -- 172.21.2.104:0/2512905069 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb07800d570 con 0x7fb0880b6580 2021-07-11T04:00:00.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb0877fe700 1 -- 172.21.2.104:0/2512905069 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb078000e90 con 0x7fb0880b6580 2021-07-11T04:00:00.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb0877fe700 1 -- 172.21.2.104:0/2512905069 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb07800d8c0 con 0x7fb0880b6580 2021-07-11T04:00:00.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 -- 172.21.2.104:0/2512905069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0880b6580 msgr2=0x7fb0880b5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 --2- 172.21.2.104:0/2512905069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0880b6580 0x7fb0880b5bf0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fb088143120 tx=0x7fb078006df0).stop 2021-07-11T04:00:00.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 -- 172.21.2.104:0/2512905069 shutdown_connections 2021-07-11T04:00:00.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 --2- 172.21.2.104:0/2512905069 >> v2:172.21.2.105:3300/0 conn(0x7fb0880b6960 0x7fb08812ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:00.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 --2- 172.21.2.104:0/2512905069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0880b6580 0x7fb0880b5bf0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 -- 172.21.2.104:0/2512905069 >> 172.21.2.104:0/2512905069 conn(0x7fb08805c490 msgr2=0x7fb08805c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:00.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 -- 172.21.2.104:0/2512905069 shutdown_connections 2021-07-11T04:00:00.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 -- 172.21.2.104:0/2512905069 wait complete. 2021-07-11T04:00:00.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 Processor -- start 2021-07-11T04:00:00.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 -- start start 2021-07-11T04:00:00.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0880b43c0 0x7fb088152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0880b6580 0x7fb088152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0880b6960 0x7fb0880b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08812c7a0 con 0x7fb0880b6580 2021-07-11T04:00:00.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb088153190 con 0x7fb0880b43c0 2021-07-11T04:00:00.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0881457f0 con 0x7fb0880b6960 2021-07-11T04:00:00.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb087fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0880b6580 0x7fb088152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:00.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb087fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0880b6580 0x7fb088152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46638/0 (socket says 172.21.2.104:46638) 2021-07-11T04:00:00.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb087fff700 1 -- 172.21.2.104:0/1565091870 learned_addr learned my addr 172.21.2.104:0/1565091870 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:00.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb087fff700 1 -- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0880b6960 msgr2=0x7fb0880b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb087fff700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0880b6960 0x7fb0880b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:00.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb087fff700 1 -- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0880b43c0 msgr2=0x7fb088152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb087fff700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0880b43c0 0x7fb088152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:00.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb087fff700 1 -- 172.21.2.104:0/1565091870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07800b180 con 0x7fb0880b6580 2021-07-11T04:00:00.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb087fff700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0880b6580 0x7fb088152a00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fb070000ed0 tx=0x7fb070007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:00.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb085ffb700 1 -- 172.21.2.104:0/1565091870 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb07000ce50 con 0x7fb0880b6580 2021-07-11T04:00:00.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb085ffb700 1 -- 172.21.2.104:0/1565091870 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0700072e0 con 0x7fb0880b6580 2021-07-11T04:00:00.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 -- 172.21.2.104:0/1565091870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb088145a90 con 0x7fb0880b6580 2021-07-11T04:00:00.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb08eb1a700 1 -- 172.21.2.104:0/1565091870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb088152ef0 con 0x7fb0880b6580 2021-07-11T04:00:00.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.605+0000 7fb085ffb700 1 -- 172.21.2.104:0/1565091870 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb070005800 con 0x7fb0880b6580 2021-07-11T04:00:00.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.609+0000 7fb08eb1a700 1 -- 172.21.2.104:0/1565091870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0880b7e60 con 0x7fb0880b6580 2021-07-11T04:00:00.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.609+0000 7fb085ffb700 1 -- 172.21.2.104:0/1565091870 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb07001a060 con 0x7fb0880b6580 2021-07-11T04:00:00.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.609+0000 7fb085ffb700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb06c053350 0x7fb06c055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.609+0000 7fb085ffb700 1 -- 172.21.2.104:0/1565091870 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fb07006aec0 con 0x7fb0880b6580 2021-07-11T04:00:00.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.609+0000 7fb08c8b6700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb06c053350 0x7fb06c055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:00.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.609+0000 7fb08c8b6700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb06c053350 0x7fb06c055810 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fb088143120 tx=0x7fb078014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:00.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.609+0000 7fb085ffb700 1 -- 172.21.2.104:0/1565091870 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb0700420d0 con 0x7fb0880b6580 2021-07-11T04:00:00.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 -- 172.21.2.104:0/1565091870 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist ls", "format": "json"} v 0) v1 -- 0x7fb088057350 con 0x7fb0880b6580 2021-07-11T04:00:00.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb085ffb700 1 -- 172.21.2.104:0/1565091870 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist ls", "format": "json"}]=0 listed 1 entries v77) v1 ==== 98+0+80 (secure 0 0 0) 0x7fb070018070 con 0x7fb0880b6580 2021-07-11T04:00:00.702 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:00.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 -- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb06c053350 msgr2=0x7fb06c055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb06c053350 0x7fb06c055810 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fb088143120 tx=0x7fb078014000).stop 2021-07-11T04:00:00.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 -- 172.21.2.104:0/1565091870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0880b6580 msgr2=0x7fb088152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0880b6580 0x7fb088152a00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fb070000ed0 tx=0x7fb070007a80).stop 2021-07-11T04:00:00.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 -- 172.21.2.104:0/1565091870 shutdown_connections 2021-07-11T04:00:00.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0880b6960 0x7fb0880b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:00.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb06c053350 0x7fb06c055810 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0880b6580 0x7fb088152a00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 --2- 172.21.2.104:0/1565091870 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0880b43c0 0x7fb088152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:00.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 -- 172.21.2.104:0/1565091870 >> 172.21.2.104:0/1565091870 conn(0x7fb08805c490 msgr2=0x7fb088141c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:00.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 -- 172.21.2.104:0/1565091870 shutdown_connections 2021-07-11T04:00:00.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.701+0000 7fb08eb1a700 1 -- 172.21.2.104:0/1565091870 wait complete. 2021-07-11T04:00:00.705 INFO:teuthology.orchestra.run.gibba004.stderr:listed 1 entries 2021-07-11T04:00:00.708 INFO:teuthology.orchestra.run.gibba004.stdout:[{"addr":"172.21.2.119:0/1304352204","until":"2021-07-11T04:59:54.428687+0000"}] 2021-07-11T04:00:00.723 INFO:tasks.cephfs.fuse_mount:Cleaning up killed ceph-fuse connection 2021-07-11T04:00:00.724 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:00.725 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:00.725 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:00.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:00:00.730 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': Transport endpoint is not connected 2021-07-11T04:00:00.730 WARNING:tasks.cephfs.fuse_mount:Found stale moutn point at /home/ubuntu/cephtest/mnt.0 2021-07-11T04:00:00.731 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T04:00:00.731 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:00:00.732 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:00.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:00.740 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:00.740 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:00.786 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T04:00:00.788 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:00:00.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:00:00.788 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:00.796 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:00.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.853+0000 7ff0c1ade700 1 Processor -- start 2021-07-11T04:00:00.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.853+0000 7ff0c1ade700 1 -- start start 2021-07-11T04:00:00.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.853+0000 7ff0c1ade700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 0x7ff0bc066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.853+0000 7ff0c1ade700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff0bc0b0060 0x7ff0bc05fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.853+0000 7ff0c1ade700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc0a7a90 con 0x7ff0bc0b1970 2021-07-11T04:00:00.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.853+0000 7ff0c1ade700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0bc147620 con 0x7ff0bc0b0060 2021-07-11T04:00:00.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.853+0000 7ff0c1ade700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc1478c0 con 0x7ff0bc0a76b0 2021-07-11T04:00:00.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0bbfff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff0bc0b0060 0x7ff0bc05fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:00.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0bb7fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 0x7ff0bc066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:00.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0bb7fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 0x7ff0bc066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:49812/0 (socket says 172.21.2.104:49812) 2021-07-11T04:00:00.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0bbfff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff0bc0b0060 0x7ff0bc05fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:54206/0 (socket says 172.21.2.104:54206) 2021-07-11T04:00:00.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0bb7fe700 1 -- 172.21.2.104:0/1321874565 learned_addr learned my addr 172.21.2.104:0/1321874565 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:00.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0bbfff700 1 -- 172.21.2.104:0/1321874565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 msgr2=0x7ff0bc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:00.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0bbfff700 1 --2- 172.21.2.104:0/1321874565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 0x7ff0bc066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0bbfff700 1 -- 172.21.2.104:0/1321874565 >> v1:172.21.2.105:6789/0 conn(0x7ff0bc0b1970 legacy=0x7ff0bc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0bbfff700 1 -- 172.21.2.104:0/1321874565 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc147b60 con 0x7ff0bc0b0060 2021-07-11T04:00:00.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0bbfff700 1 --2- 172.21.2.104:0/1321874565 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0bc0b0060 0x7ff0bc05fdc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7ff0b40032f0 tx=0x7ff0b40091d0).ready entity=mon.1 client_cookie=446bca8b6fcf3b9b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:00.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0ba7fc700 1 -- 172.21.2.104:0/1321874565 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0b400b6c0 con 0x7ff0bc0b0060 2021-07-11T04:00:00.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0ba7fc700 1 -- 172.21.2.104:0/1321874565 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b400b820 con 0x7ff0bc0b0060 2021-07-11T04:00:00.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0ba7fc700 1 -- 172.21.2.104:0/1321874565 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0b400b990 con 0x7ff0bc0b0060 2021-07-11T04:00:00.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0c1ade700 1 -- 172.21.2.104:0/1321874565 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0bc0b0060 msgr2=0x7ff0bc05fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0c1ade700 1 --2- 172.21.2.104:0/1321874565 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0bc0b0060 0x7ff0bc05fdc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7ff0b40032f0 tx=0x7ff0b40091d0).stop 2021-07-11T04:00:00.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0c1ade700 1 -- 172.21.2.104:0/1321874565 shutdown_connections 2021-07-11T04:00:00.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0c1ade700 1 --2- 172.21.2.104:0/1321874565 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0bc0b0060 0x7ff0bc05fdc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0c1ade700 1 --2- 172.21.2.104:0/1321874565 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 0x7ff0bc066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0c1ade700 1 -- 172.21.2.104:0/1321874565 >> 172.21.2.104:0/1321874565 conn(0x7ff0bc0b53e0 msgr2=0x7ff0bc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:00.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0c1ade700 1 -- 172.21.2.104:0/1321874565 shutdown_connections 2021-07-11T04:00:00.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0c1ade700 1 -- 172.21.2.104:0/1321874565 wait complete. 2021-07-11T04:00:00.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0c1ade700 1 Processor -- start 2021-07-11T04:00:00.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.857+0000 7ff0c1ade700 1 -- start start 2021-07-11T04:00:00.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0c1ade700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 0x7ff0bc14c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0c1ade700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0bc0b0060 0x7ff0bc0ab270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0c1ade700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0bc0b1970 0x7ff0bc0ab760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0c1ade700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc1436b0 con 0x7ff0bc0b1970 2021-07-11T04:00:00.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0c1ade700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0aa5a0 con 0x7ff0bc0b0060 2021-07-11T04:00:00.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0c1ade700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc0aa840 con 0x7ff0bc0a76b0 2021-07-11T04:00:00.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bbfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0bc0b1970 0x7ff0bc0ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:00.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bbfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0bc0b1970 0x7ff0bc0ab760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46656/0 (socket says 172.21.2.104:46656) 2021-07-11T04:00:00.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bbfff700 1 -- 172.21.2.104:0/2530532724 learned_addr learned my addr 172.21.2.104:0/2530532724 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:00.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bb7fe700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 0x7ff0bc14c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:00.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0baffd700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0bc0b0060 0x7ff0bc0ab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:00.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bbfff700 1 -- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 msgr2=0x7ff0bc14c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bbfff700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 0x7ff0bc14c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bbfff700 1 -- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0bc0b0060 msgr2=0x7ff0bc0ab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bbfff700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0bc0b0060 0x7ff0bc0ab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bbfff700 1 -- 172.21.2.104:0/2530532724 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4009470 con 0x7ff0bc0b1970 2021-07-11T04:00:00.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bb7fe700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 0x7ff0bc14c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:00.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0baffd700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0bc0b0060 0x7ff0bc0ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:00.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bbfff700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0bc0b1970 0x7ff0bc0ab760 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7ff0b40032f0 tx=0x7ff0b4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:00.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0b8ff9700 1 -- 172.21.2.104:0/2530532724 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0b4021440 con 0x7ff0bc0b1970 2021-07-11T04:00:00.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0b8ff9700 1 -- 172.21.2.104:0/2530532724 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b40215a0 con 0x7ff0bc0b1970 2021-07-11T04:00:00.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0b8ff9700 1 -- 172.21.2.104:0/2530532724 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0b4021900 con 0x7ff0bc0b1970 2021-07-11T04:00:00.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0c1ade700 1 -- 172.21.2.104:0/2530532724 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0bc0aaae0 con 0x7ff0bc0b1970 2021-07-11T04:00:00.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0c1ade700 1 -- 172.21.2.104:0/2530532724 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0bc14ce90 con 0x7ff0bc0b1970 2021-07-11T04:00:00.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0c1ade700 1 -- 172.21.2.104:0/2530532724 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0bc057350 con 0x7ff0bc0b1970 2021-07-11T04:00:00.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0b8ff9700 1 -- 172.21.2.104:0/2530532724 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff0b4027020 con 0x7ff0bc0b1970 2021-07-11T04:00:00.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0b8ff9700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0a00531d0 0x7ff0a0055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:00.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0b8ff9700 1 -- 172.21.2.104:0/2530532724 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7ff0b406c8f0 con 0x7ff0bc0b1970 2021-07-11T04:00:00.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.861+0000 7ff0bb7fe700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0a00531d0 0x7ff0a0055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:00.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.865+0000 7ff0bb7fe700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0a00531d0 0x7ff0a0055690 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7ff0ac008000 tx=0x7ff0ac008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:00.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.865+0000 7ff0b8ff9700 1 -- 172.21.2.104:0/2530532724 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff0b4043b40 con 0x7ff0bc0b1970 2021-07-11T04:00:00.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.945+0000 7ff0c1ade700 1 -- 172.21.2.104:0/2530532724 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0bc0698e0 con 0x7ff0bc0b1970 2021-07-11T04:00:00.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.945+0000 7ff0b8ff9700 1 -- 172.21.2.104:0/2530532724 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16102 (secure 0 0 0) 0x7ff0b4015090 con 0x7ff0bc0b1970 2021-07-11T04:00:00.948 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:00.949 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":77,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T03:59:55.331085+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2021-07-11T03:59:11.477806+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2021-07-11T03:59:12.643109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.119:0/1304352204":"2021-07-11T04:59:54.428687+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:00.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 -- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0a00531d0 msgr2=0x7ff0a0055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0a00531d0 0x7ff0a0055690 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7ff0ac008000 tx=0x7ff0ac008040).stop 2021-07-11T04:00:00.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 -- 172.21.2.104:0/2530532724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0bc0b1970 msgr2=0x7ff0bc0ab760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:00.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0bc0b1970 0x7ff0bc0ab760 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7ff0b40032f0 tx=0x7ff0b4016000).stop 2021-07-11T04:00:00.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 -- 172.21.2.104:0/2530532724 shutdown_connections 2021-07-11T04:00:00.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0a00531d0 0x7ff0a0055690 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0bc0b1970 0x7ff0bc0ab760 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0bc0b0060 0x7ff0bc0ab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 --2- 172.21.2.104:0/2530532724 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0bc0a76b0 0x7ff0bc14c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:00.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 -- 172.21.2.104:0/2530532724 >> 172.21.2.104:0/2530532724 conn(0x7ff0bc0b53e0 msgr2=0x7ff0bc146040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:00.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 -- 172.21.2.104:0/2530532724 shutdown_connections 2021-07-11T04:00:00.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:00.949+0000 7ff0c1ade700 1 -- 172.21.2.104:0/2530532724 wait complete. 2021-07-11T04:00:00.955 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T03:59:11.477806+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T03:59:12.643109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:00:00.956 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:00:01.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.005+0000 7f5282815700 1 Processor -- start 2021-07-11T04:00:01.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.005+0000 7f5282815700 1 -- start start 2021-07-11T04:00:01.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f527c0afaf0 0x7f527c0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527c0b5f40 con 0x7f527c13c9c0 2021-07-11T04:00:01.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527c0ad420 con 0x7f527c0ad040 2021-07-11T04:00:01.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527c0aeb60 con 0x7f527c0afaf0 2021-07-11T04:00:01.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527b7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f527c13c9c0 0x7f527c13f070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43430/0 (socket says 172.21.2.104:43430) 2021-07-11T04:00:01.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527b7fe700 1 -- 172.21.2.104:0/874652179 learned_addr learned my addr 172.21.2.104:0/874652179 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:01.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5280db2700 1 --2- 172.21.2.104:0/874652179 >> v2:172.21.2.104:3300/0 conn(0x7f527c0afaf0 0x7f527c0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527affd700 1 -- 172.21.2.104:0/874652179 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150000435 0 0) 0x7f527c0b5f40 con 0x7f527c13c9c0 2021-07-11T04:00:01.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527affd700 1 -- 172.21.2.104:0/874652179 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f525c003260 con 0x7f527c13c9c0 2021-07-11T04:00:01.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5280db2700 1 -- 172.21.2.104:0/874652179 >> v1:172.21.2.105:6789/0 conn(0x7f527c0ad040 legacy=0x7f527c13c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5280db2700 1 -- 172.21.2.104:0/874652179 >> v1:172.21.2.104:6789/0 conn(0x7f527c13c9c0 legacy=0x7f527c13f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5280db2700 1 -- 172.21.2.104:0/874652179 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527c0aee00 con 0x7f527c0afaf0 2021-07-11T04:00:01.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5280db2700 1 --2- 172.21.2.104:0/874652179 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f527c0afaf0 0x7f527c0b5a50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f527c05d010 tx=0x7f52740091d0).ready entity=mon.0 client_cookie=92ce4613fd7edfe9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527affd700 1 -- 172.21.2.104:0/874652179 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f527400c980 con 0x7f527c0afaf0 2021-07-11T04:00:01.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527affd700 1 -- 172.21.2.104:0/874652179 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527400cae0 con 0x7f527c0afaf0 2021-07-11T04:00:01.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527affd700 1 -- 172.21.2.104:0/874652179 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f527400ce40 con 0x7f527c0afaf0 2021-07-11T04:00:01.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- 172.21.2.104:0/874652179 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f527c0afaf0 msgr2=0x7f527c0b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 --2- 172.21.2.104:0/874652179 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f527c0afaf0 0x7f527c0b5a50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f527c05d010 tx=0x7f52740091d0).stop 2021-07-11T04:00:01.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- 172.21.2.104:0/874652179 shutdown_connections 2021-07-11T04:00:01.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 --2- 172.21.2.104:0/874652179 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f527c0afaf0 0x7f527c0b5a50 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- 172.21.2.104:0/874652179 >> 172.21.2.104:0/874652179 conn(0x7f527c05c490 msgr2=0x7f527c05c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:01.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- 172.21.2.104:0/874652179 shutdown_connections 2021-07-11T04:00:01.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- 172.21.2.104:0/874652179 wait complete. 2021-07-11T04:00:01.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 Processor -- start 2021-07-11T04:00:01.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- start start 2021-07-11T04:00:01.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f527c0ad040 0x7f527c13e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f527c0afaf0 0x7f527c142930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f527c13c9c0 0x7f527c142e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c064070 con 0x7f527c0ad040 2021-07-11T04:00:01.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527c063c10 con 0x7f527c0afaf0 2021-07-11T04:00:01.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527c0af8e0 con 0x7f527c13c9c0 2021-07-11T04:00:01.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f527c0ad040 0x7f527c13e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f527c0ad040 0x7f527c13e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46668/0 (socket says 172.21.2.104:46668) 2021-07-11T04:00:01.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527bfff700 1 -- 172.21.2.104:0/3744902821 learned_addr learned my addr 172.21.2.104:0/3744902821 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:01.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527bfff700 1 -- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f527c13c9c0 msgr2=0x7f527c142e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527bfff700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f527c13c9c0 0x7f527c142e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527bfff700 1 -- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f527c0afaf0 msgr2=0x7f527c142930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527bfff700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f527c0afaf0 0x7f527c142930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527bfff700 1 -- 172.21.2.104:0/3744902821 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5274009470 con 0x7f527c0ad040 2021-07-11T04:00:01.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f527bfff700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f527c0ad040 0x7f527c13e3e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f526c002f90 tx=0x7f526c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f52797fa700 1 -- 172.21.2.104:0/3744902821 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f526c013460 con 0x7f527c0ad040 2021-07-11T04:00:01.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f52797fa700 1 -- 172.21.2.104:0/3744902821 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c0135c0 con 0x7f527c0ad040 2021-07-11T04:00:01.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- 172.21.2.104:0/3744902821 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527c0639b0 con 0x7f527c0ad040 2021-07-11T04:00:01.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f52797fa700 1 -- 172.21.2.104:0/3744902821 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f526c013920 con 0x7f527c0ad040 2021-07-11T04:00:01.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.009+0000 7f5282815700 1 -- 172.21.2.104:0/3744902821 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527c141570 con 0x7f527c0ad040 2021-07-11T04:00:01.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.013+0000 7f5282815700 1 -- 172.21.2.104:0/3744902821 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527c057350 con 0x7f527c0ad040 2021-07-11T04:00:01.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.013+0000 7f52797fa700 1 -- 172.21.2.104:0/3744902821 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f526c01a060 con 0x7f527c0ad040 2021-07-11T04:00:01.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.013+0000 7f52797fa700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f52600531f0 0x7f52600556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.013+0000 7f52797fa700 1 -- 172.21.2.104:0/3744902821 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f526c06af70 con 0x7f527c0ad040 2021-07-11T04:00:01.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.013+0000 7f527b7fe700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f52600531f0 0x7f52600556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.013+0000 7f527b7fe700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f52600531f0 0x7f52600556b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f5264001110 tx=0x7f52640076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.013+0000 7f52797fa700 1 -- 172.21.2.104:0/3744902821 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f526c042180 con 0x7f527c0ad040 2021-07-11T04:00:01.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 -- 172.21.2.104:0/3744902821 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f527c13fca0 con 0x7f527c0ad040 2021-07-11T04:00:01.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f52797fa700 1 -- 172.21.2.104:0/3744902821 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v77) v1 ==== 110+0+10 (secure 0 0 0) 0x7f526c018070 con 0x7f527c0ad040 2021-07-11T04:00:01.097 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:00:01.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 -- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f52600531f0 msgr2=0x7f52600556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f52600531f0 0x7f52600556b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f5264001110 tx=0x7f52640076c0).stop 2021-07-11T04:00:01.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 -- 172.21.2.104:0/3744902821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f527c0ad040 msgr2=0x7f527c13e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f527c0ad040 0x7f527c13e3e0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f526c002f90 tx=0x7f526c006df0).stop 2021-07-11T04:00:01.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 -- 172.21.2.104:0/3744902821 shutdown_connections 2021-07-11T04:00:01.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f527c13c9c0 0x7f527c142e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f527c0afaf0 0x7f527c142930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f52600531f0 0x7f52600556b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 --2- 172.21.2.104:0/3744902821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f527c0ad040 0x7f527c13e3e0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 -- 172.21.2.104:0/3744902821 >> 172.21.2.104:0/3744902821 conn(0x7f527c05c490 msgr2=0x7f527c13a840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:01.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 -- 172.21.2.104:0/3744902821 shutdown_connections 2021-07-11T04:00:01.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.097+0000 7f5282815700 1 -- 172.21.2.104:0/3744902821 wait complete. 2021-07-11T04:00:01.104 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:00:01.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa81ba3e700 1 Processor -- start 2021-07-11T04:00:01.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa81ba3e700 1 -- start start 2021-07-11T04:00:01.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa81ba3e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa814141960 0x7fa8140affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa81ba3e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8140a77d0 con 0x7fa8140b1ec0 2021-07-11T04:00:01.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa81ba3e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8140b0650 con 0x7fa8140b12d0 2021-07-11T04:00:01.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa81ba3e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b1740 con 0x7fa814141960 2021-07-11T04:00:01.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa8197da700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fa8140b1ec0 0x7fa8140a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43442/0 (socket says 172.21.2.104:43442) 2021-07-11T04:00:01.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa8197da700 1 -- 172.21.2.104:0/1073614941 learned_addr learned my addr 172.21.2.104:0/1073614941 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:01.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281616370 0 0) 0x7fa8140a77d0 con 0x7fa8140b1ec0 2021-07-11T04:00:01.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7fc003260 con 0x7fa8140b1ec0 2021-07-11T04:00:01.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa819fdb700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa814141960 0x7fa8140affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3658922609 0 0) 0x7fa7fc003260 con 0x7fa8140b1ec0 2021-07-11T04:00:01.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa814141960 msgr2=0x7fa8140affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa814141960 0x7fa8140affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 >> v1:172.21.2.105:6789/0 conn(0x7fa8140b12d0 legacy=0x7fa814141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140b08f0 con 0x7fa8140b1ec0 2021-07-11T04:00:01.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fa804001660 con 0x7fa8140b1ec0 2021-07-11T04:00:01.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 >> v1:172.21.2.104:6789/0 conn(0x7fa8140b1ec0 legacy=0x7fa8140a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7fc003800 0x7fa7fc003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa7fc004500 0x7fa7fc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7fc004ec0 0x7fa7fc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b1740 con 0x7fa7fc004ec0 2021-07-11T04:00:01.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc00bd70 con 0x7fa7fc004500 2021-07-11T04:00:01.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7fc00bf30 con 0x7fa7fc003800 2021-07-11T04:00:01.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa819fdb700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7fc004ec0 0x7fa7fc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa8197da700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7fc003800 0x7fa7fc003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa818fd9700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa7fc004500 0x7fa7fc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa819fdb700 1 -- 172.21.2.104:0/1073614941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7fc003800 msgr2=0x7fa7fc003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa819fdb700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7fc003800 0x7fa7fc003c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa819fdb700 1 -- 172.21.2.104:0/1073614941 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa7fc004500 msgr2=0x7fa7fc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa819fdb700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa7fc004500 0x7fa7fc004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa819fdb700 1 -- 172.21.2.104:0/1073614941 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140b08f0 con 0x7fa7fc004ec0 2021-07-11T04:00:01.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa819fdb700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7fc004ec0 0x7fa7fc00b750 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7fa80c00a000 tx=0x7fa80c008040).ready entity=mon.0 client_cookie=d9632a4f955458f4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa80c00c070 con 0x7fa7fc004ec0 2021-07-11T04:00:01.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c00e4d0 con 0x7fa7fc004ec0 2021-07-11T04:00:01.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa803fff700 1 -- 172.21.2.104:0/1073614941 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa80c00e830 con 0x7fa7fc004ec0 2021-07-11T04:00:01.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1073614941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7fc004ec0 msgr2=0x7fa7fc00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa81ba3e700 1 --2- 172.21.2.104:0/1073614941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7fc004ec0 0x7fa7fc00b750 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7fa80c00a000 tx=0x7fa80c008040).stop 2021-07-11T04:00:01.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.157+0000 7fa819fdb700 1 -- 172.21.2.104:0/1073614941 reap_dead start 2021-07-11T04:00:01.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1073614941 shutdown_connections 2021-07-11T04:00:01.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1073614941 >> 172.21.2.104:0/1073614941 conn(0x7fa81405f6b0 msgr2=0x7fa81405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:01.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1073614941 shutdown_connections 2021-07-11T04:00:01.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1073614941 wait complete. 2021-07-11T04:00:01.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 Processor -- start 2021-07-11T04:00:01.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 -- start start 2021-07-11T04:00:01.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa81414c690 0x7fa8140a9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa8140a9910 0x7fa814148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa814148fd0 0x7fa814149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81414bb30 con 0x7fa814148fd0 2021-07-11T04:00:01.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa814149940 con 0x7fa8140a9910 2021-07-11T04:00:01.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa81414ca70 con 0x7fa81414c690 2021-07-11T04:00:01.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa819fdb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa814148fd0 0x7fa814149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa819fdb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa814148fd0 0x7fa814149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46684/0 (socket says 172.21.2.104:46684) 2021-07-11T04:00:01.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa819fdb700 1 -- 172.21.2.104:0/1780762330 learned_addr learned my addr 172.21.2.104:0/1780762330 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:01.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa819fdb700 1 -- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa81414c690 msgr2=0x7fa8140a9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa819fdb700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa81414c690 0x7fa8140a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa819fdb700 1 -- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa8140a9910 msgr2=0x7fa814148ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa819fdb700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa8140a9910 0x7fa814148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa819fdb700 1 -- 172.21.2.104:0/1780762330 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa814149cf0 con 0x7fa814148fd0 2021-07-11T04:00:01.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa819fdb700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa814148fd0 0x7fa814149450 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fa8140ad260 tx=0x7fa80c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa8027fc700 1 -- 172.21.2.104:0/1780762330 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa80c013c30 con 0x7fa814148fd0 2021-07-11T04:00:01.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa8027fc700 1 -- 172.21.2.104:0/1780762330 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80c013d90 con 0x7fa814148fd0 2021-07-11T04:00:01.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1780762330 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81414b890 con 0x7fa814148fd0 2021-07-11T04:00:01.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1780762330 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8140a9d70 con 0x7fa814148fd0 2021-07-11T04:00:01.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa8027fc700 1 -- 172.21.2.104:0/1780762330 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa80c01d630 con 0x7fa814148fd0 2021-07-11T04:00:01.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1780762330 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa814057350 con 0x7fa814148fd0 2021-07-11T04:00:01.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa8027fc700 1 -- 172.21.2.104:0/1780762330 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa80c024030 con 0x7fa814148fd0 2021-07-11T04:00:01.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa8027fc700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7f4053160 0x7fa7f4055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa8027fc700 1 -- 172.21.2.104:0/1780762330 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fa80c06b7d0 con 0x7fa814148fd0 2021-07-11T04:00:01.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.161+0000 7fa8197da700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7f4053160 0x7fa7f4055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.165+0000 7fa8197da700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7f4053160 0x7fa7f4055620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fa804008000 tx=0x7fa804009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.165+0000 7fa8027fc700 1 -- 172.21.2.104:0/1780762330 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa80c042af0 con 0x7fa814148fd0 2021-07-11T04:00:01.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1780762330 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa8140ac760 con 0x7fa814148fd0 2021-07-11T04:00:01.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa8027fc700 1 -- 172.21.2.104:0/1780762330 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v77) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa80c022020 con 0x7fa814148fd0 2021-07-11T04:00:01.249 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:01.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7f4053160 msgr2=0x7fa7f4055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7f4053160 0x7fa7f4055620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fa804008000 tx=0x7fa804009040).stop 2021-07-11T04:00:01.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1780762330 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa814148fd0 msgr2=0x7fa814149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa814148fd0 0x7fa814149450 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7fa8140ad260 tx=0x7fa80c008040).stop 2021-07-11T04:00:01.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1780762330 shutdown_connections 2021-07-11T04:00:01.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7f4053160 0x7fa7f4055620 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa814148fd0 0x7fa814149450 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa8140a9910 0x7fa814148ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 --2- 172.21.2.104:0/1780762330 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa81414c690 0x7fa8140a9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1780762330 >> 172.21.2.104:0/1780762330 conn(0x7fa81405f6b0 msgr2=0x7fa81414cf30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:01.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1780762330 shutdown_connections 2021-07-11T04:00:01.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.249+0000 7fa81ba3e700 1 -- 172.21.2.104:0/1780762330 wait complete. 2021-07-11T04:00:01.256 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:00:01.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 Processor -- start 2021-07-11T04:00:01.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 -- start start 2021-07-11T04:00:01.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8ebc0b1ec0 0x7f8ebc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ebc0b0420 con 0x7f8ebc0b22a0 2021-07-11T04:00:01.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ebc0a77d0 con 0x7f8ebc0affb0 2021-07-11T04:00:01.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ebc0b0650 con 0x7f8ebc0b1ec0 2021-07-11T04:00:01.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec1ca5700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8ebc0b22a0 0x7f8ebc141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43462/0 (socket says 172.21.2.104:43462) 2021-07-11T04:00:01.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec1ca5700 1 -- 172.21.2.104:0/549488888 learned_addr learned my addr 172.21.2.104:0/549488888 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:01.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949640501 0 0) 0x7f8ebc0b0420 con 0x7f8ebc0b22a0 2021-07-11T04:00:01.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ea4003260 con 0x7f8ebc0b22a0 2021-07-11T04:00:01.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec14a4700 1 --2- 172.21.2.104:0/549488888 >> v2:172.21.2.105:3300/0 conn(0x7f8ebc0b1ec0 0x7f8ebc0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2755478204 0 0) 0x7f8ea4003260 con 0x7f8ebc0b22a0 2021-07-11T04:00:01.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 >> v2:172.21.2.105:3300/0 conn(0x7f8ebc0b1ec0 msgr2=0x7f8ebc0b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 --2- 172.21.2.104:0/549488888 >> v2:172.21.2.105:3300/0 conn(0x7f8ebc0b1ec0 0x7f8ebc0b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 >> v1:172.21.2.105:6789/0 conn(0x7f8ebc0affb0 legacy=0x7f8ebc0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc0b08f0 con 0x7f8ebc0b22a0 2021-07-11T04:00:01.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f8eb40019e0 con 0x7f8ebc0b22a0 2021-07-11T04:00:01.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 >> v1:172.21.2.104:6789/0 conn(0x7f8ebc0b22a0 legacy=0x7f8ebc141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 --2- 172.21.2.104:0/549488888 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ea4003820 0x7f8ea4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 --2- 172.21.2.104:0/549488888 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 0x7f8ea4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 --2- 172.21.2.104:0/549488888 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ea4005050 0x7f8ea400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0b0650 con 0x7f8ea4004500 2021-07-11T04:00:01.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea400bf40 con 0x7f8ea4003820 2021-07-11T04:00:01.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea400c100 con 0x7f8ea4005050 2021-07-11T04:00:01.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec0ca3700 1 --2- 172.21.2.104:0/549488888 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 0x7f8ea4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec0ca3700 1 -- 172.21.2.104:0/549488888 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ea4005050 msgr2=0x7f8ea400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec0ca3700 1 --2- 172.21.2.104:0/549488888 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ea4005050 0x7f8ea400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec0ca3700 1 -- 172.21.2.104:0/549488888 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ea4003820 msgr2=0x7f8ea4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec0ca3700 1 --2- 172.21.2.104:0/549488888 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ea4003820 0x7f8ea4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec0ca3700 1 -- 172.21.2.104:0/549488888 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea4003260 con 0x7f8ea4004500 2021-07-11T04:00:01.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec1ca5700 1 -- 172.21.2.104:0/549488888 reap_dead start 2021-07-11T04:00:01.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec0ca3700 1 --2- 172.21.2.104:0/549488888 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 0x7f8ea4004980 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f8eb0000f70 tx=0x7f8eb00079e0).ready entity=mon.0 client_cookie=312609cbac02ad09 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb0007cf0 con 0x7f8ea4004500 2021-07-11T04:00:01.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb0014410 con 0x7f8ea4004500 2021-07-11T04:00:01.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8eaffff700 1 -- 172.21.2.104:0/549488888 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb0014770 con 0x7f8ea4004500 2021-07-11T04:00:01.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 -- 172.21.2.104:0/549488888 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 msgr2=0x7f8ea4004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 --2- 172.21.2.104:0/549488888 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 0x7f8ea4004980 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f8eb0000f70 tx=0x7f8eb00079e0).stop 2021-07-11T04:00:01.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 -- 172.21.2.104:0/549488888 shutdown_connections 2021-07-11T04:00:01.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 --2- 172.21.2.104:0/549488888 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 0x7f8ea4004980 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 -- 172.21.2.104:0/549488888 >> 172.21.2.104:0/549488888 conn(0x7f8ebc05f6b0 msgr2=0x7f8ebc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:01.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 -- 172.21.2.104:0/549488888 shutdown_connections 2021-07-11T04:00:01.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.309+0000 7f8ec3708700 1 -- 172.21.2.104:0/549488888 wait complete. 2021-07-11T04:00:01.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec3708700 1 Processor -- start 2021-07-11T04:00:01.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec3708700 1 -- start start 2021-07-11T04:00:01.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec3708700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 0x7f8ebc063f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec3708700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ebc064480 0x7f8ebc156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec3708700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ebc156e10 0x7f8ebc14b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec3708700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0b4240 con 0x7f8ea4004500 2021-07-11T04:00:01.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec3708700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0ac310 con 0x7f8ebc156e10 2021-07-11T04:00:01.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec3708700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0b46a0 con 0x7f8ebc064480 2021-07-11T04:00:01.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec14a4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 0x7f8ebc063f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec14a4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 0x7f8ebc063f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46706/0 (socket says 172.21.2.104:46706) 2021-07-11T04:00:01.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec14a4700 1 -- 172.21.2.104:0/1559499611 learned_addr learned my addr 172.21.2.104:0/1559499611 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:01.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec14a4700 1 -- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ebc064480 msgr2=0x7f8ebc156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec14a4700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ebc064480 0x7f8ebc156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec14a4700 1 -- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ebc156e10 msgr2=0x7f8ebc14b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec14a4700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ebc156e10 0x7f8ebc14b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec14a4700 1 -- 172.21.2.104:0/1559499611 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc157290 con 0x7f8ea4004500 2021-07-11T04:00:01.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec14a4700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 0x7f8ebc063f90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f8eb8007000 tx=0x7f8eb8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8eae7fc700 1 -- 172.21.2.104:0/1559499611 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb800f4e0 con 0x7f8ea4004500 2021-07-11T04:00:01.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8eae7fc700 1 -- 172.21.2.104:0/1559499611 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb8000e90 con 0x7f8ea4004500 2021-07-11T04:00:01.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec3708700 1 -- 172.21.2.104:0/1559499611 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc15a7a0 con 0x7f8ea4004500 2021-07-11T04:00:01.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec3708700 1 -- 172.21.2.104:0/1559499611 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc0b3ec0 con 0x7f8ea4004500 2021-07-11T04:00:01.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8eae7fc700 1 -- 172.21.2.104:0/1559499611 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8eb800f830 con 0x7f8ea4004500 2021-07-11T04:00:01.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec3708700 1 -- 172.21.2.104:0/1559499611 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ebc057350 con 0x7f8ea4004500 2021-07-11T04:00:01.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8eae7fc700 1 -- 172.21.2.104:0/1559499611 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8eb800f990 con 0x7f8ea4004500 2021-07-11T04:00:01.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8eae7fc700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8e9c053300 0x7f8e9c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8eae7fc700 1 -- 172.21.2.104:0/1559499611 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f8eb806b160 con 0x7f8ea4004500 2021-07-11T04:00:01.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.313+0000 7f8ec0ca3700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8e9c053300 0x7f8e9c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.317+0000 7f8ec0ca3700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8e9c053300 0x7f8e9c0557c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f8eb0000f70 tx=0x7f8eb0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.317+0000 7f8eae7fc700 1 -- 172.21.2.104:0/1559499611 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8eb8042370 con 0x7f8ea4004500 2021-07-11T04:00:01.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.397+0000 7f8ec3708700 1 -- 172.21.2.104:0/1559499611 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8ebc0b4ca0 con 0x7f8ea4004500 2021-07-11T04:00:01.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.397+0000 7f8eae7fc700 1 -- 172.21.2.104:0/1559499611 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+10 (secure 0 0 0) 0x7f8eb8045990 con 0x7f8ea4004500 2021-07-11T04:00:01.401 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:01.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 -- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8e9c053300 msgr2=0x7f8e9c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8e9c053300 0x7f8e9c0557c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f8eb0000f70 tx=0x7f8eb0015000).stop 2021-07-11T04:00:01.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 -- 172.21.2.104:0/1559499611 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 msgr2=0x7f8ebc063f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 0x7f8ebc063f90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f8eb8007000 tx=0x7f8eb8008040).stop 2021-07-11T04:00:01.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 -- 172.21.2.104:0/1559499611 shutdown_connections 2021-07-11T04:00:01.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ebc156e10 0x7f8ebc14b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ebc064480 0x7f8ebc156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8e9c053300 0x7f8e9c0557c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 --2- 172.21.2.104:0/1559499611 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ea4004500 0x7f8ebc063f90 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 -- 172.21.2.104:0/1559499611 >> 172.21.2.104:0/1559499611 conn(0x7f8ebc05f6b0 msgr2=0x7f8ebc0aef40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:01.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 -- 172.21.2.104:0/1559499611 shutdown_connections 2021-07-11T04:00:01.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.401+0000 7f8ec3708700 1 -- 172.21.2.104:0/1559499611 wait complete. 2021-07-11T04:00:01.408 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:00:01.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 Processor -- start 2021-07-11T04:00:01.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- start start 2021-07-11T04:00:01.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f60540b1ec0 0x7f60540b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f60540b22a0 0x7f6054141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60540b0420 con 0x7f60540affb0 2021-07-11T04:00:01.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60540a77d0 con 0x7f60540b22a0 2021-07-11T04:00:01.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60540b0650 con 0x7f60540b1ec0 2021-07-11T04:00:01.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f60540b22a0 0x7f6054141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f60540b22a0 0x7f6054141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46712/0 (socket says 172.21.2.104:46712) 2021-07-11T04:00:01.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 -- 172.21.2.104:0/2404217166 learned_addr learned my addr 172.21.2.104:0/2404217166 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:01.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 -- 172.21.2.104:0/2404217166 >> v2:172.21.2.105:3300/0 conn(0x7f60540b1ec0 msgr2=0x7f60540b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 --2- 172.21.2.104:0/2404217166 >> v2:172.21.2.105:3300/0 conn(0x7f60540b1ec0 0x7f60540b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 -- 172.21.2.104:0/2404217166 >> v1:172.21.2.105:6789/0 conn(0x7f60540affb0 legacy=0x7f60540a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 -- 172.21.2.104:0/2404217166 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60540b08f0 con 0x7f60540b22a0 2021-07-11T04:00:01.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 --2- 172.21.2.104:0/2404217166 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60540b22a0 0x7f6054141250 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f60540ad2d0 tx=0x7f604c009290).ready entity=mon.0 client_cookie=2fb800c57395fdd6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f60590e0700 1 -- 172.21.2.104:0/2404217166 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f604c00b830 con 0x7f60540b22a0 2021-07-11T04:00:01.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f60590e0700 1 -- 172.21.2.104:0/2404217166 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604c00b990 con 0x7f60540b22a0 2021-07-11T04:00:01.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f60590e0700 1 -- 172.21.2.104:0/2404217166 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f604c00bb40 con 0x7f60540b22a0 2021-07-11T04:00:01.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- 172.21.2.104:0/2404217166 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60540b22a0 msgr2=0x7f6054141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 --2- 172.21.2.104:0/2404217166 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60540b22a0 0x7f6054141250 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f60540ad2d0 tx=0x7f604c009290).stop 2021-07-11T04:00:01.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- 172.21.2.104:0/2404217166 shutdown_connections 2021-07-11T04:00:01.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 --2- 172.21.2.104:0/2404217166 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60540b22a0 0x7f6054141250 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 --2- 172.21.2.104:0/2404217166 >> v2:172.21.2.105:3300/0 conn(0x7f60540b1ec0 0x7f60540b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- 172.21.2.104:0/2404217166 >> 172.21.2.104:0/2404217166 conn(0x7f605405f6b0 msgr2=0x7f605405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:01.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- 172.21.2.104:0/2404217166 shutdown_connections 2021-07-11T04:00:01.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- 172.21.2.104:0/2404217166 wait complete. 2021-07-11T04:00:01.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 Processor -- start 2021-07-11T04:00:01.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- start start 2021-07-11T04:00:01.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f60540affb0 0x7f60540ad6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60540b1ec0 0x7f60540aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60540b22a0 0x7f60540aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60540b5310 con 0x7f60540b22a0 2021-07-11T04:00:01.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60540ae2f0 con 0x7f60540affb0 2021-07-11T04:00:01.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605c346700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60540a94c0 con 0x7f60540b1ec0 2021-07-11T04:00:01.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60540b22a0 0x7f60540aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60540b22a0 0x7f60540aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46722/0 (socket says 172.21.2.104:46722) 2021-07-11T04:00:01.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 -- 172.21.2.104:0/2261348462 learned_addr learned my addr 172.21.2.104:0/2261348462 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:01.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 -- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60540b1ec0 msgr2=0x7f60540aadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60540b1ec0 0x7f60540aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 -- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f60540affb0 msgr2=0x7f60540ad6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f60540affb0 0x7f60540ad6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 -- 172.21.2.104:0/2261348462 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c009530 con 0x7f60540b22a0 2021-07-11T04:00:01.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.461+0000 7f605a8e3700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60540b22a0 0x7f60540aece0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f60540ad2d0 tx=0x7f604c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f60477fe700 1 -- 172.21.2.104:0/2261348462 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f604c011cc0 con 0x7f60540b22a0 2021-07-11T04:00:01.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f60477fe700 1 -- 172.21.2.104:0/2261348462 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604c011e20 con 0x7f60540b22a0 2021-07-11T04:00:01.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f605c346700 1 -- 172.21.2.104:0/2261348462 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60540abfa0 con 0x7f60540b22a0 2021-07-11T04:00:01.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f605c346700 1 -- 172.21.2.104:0/2261348462 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60540b4c70 con 0x7f60540b22a0 2021-07-11T04:00:01.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f60477fe700 1 -- 172.21.2.104:0/2261348462 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f604c01f610 con 0x7f60540b22a0 2021-07-11T04:00:01.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f605c346700 1 -- 172.21.2.104:0/2261348462 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6054057350 con 0x7f60540b22a0 2021-07-11T04:00:01.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f60477fe700 1 -- 172.21.2.104:0/2261348462 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f604c00f070 con 0x7f60540b22a0 2021-07-11T04:00:01.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f60477fe700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6034053300 0x7f60340557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f60477fe700 1 -- 172.21.2.104:0/2261348462 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f604c06c040 con 0x7f60540b22a0 2021-07-11T04:00:01.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f605a0e2700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6034053300 0x7f60340557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f605a0e2700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6034053300 0x7f60340557c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f6048002f70 tx=0x7f6048006a90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.465+0000 7f60477fe700 1 -- 172.21.2.104:0/2261348462 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f604c043250 con 0x7f60540b22a0 2021-07-11T04:00:01.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 -- 172.21.2.104:0/2261348462 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f60540a75b0 con 0x7f60540b22a0 2021-07-11T04:00:01.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f60477fe700 1 -- 172.21.2.104:0/2261348462 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+10 (secure 0 0 0) 0x7f604c0150a0 con 0x7f60540b22a0 2021-07-11T04:00:01.551 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:01.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 -- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6034053300 msgr2=0x7f60340557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6034053300 0x7f60340557c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f6048002f70 tx=0x7f6048006a90).stop 2021-07-11T04:00:01.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 -- 172.21.2.104:0/2261348462 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60540b22a0 msgr2=0x7f60540aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60540b22a0 0x7f60540aece0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f60540ad2d0 tx=0x7f604c016000).stop 2021-07-11T04:00:01.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 -- 172.21.2.104:0/2261348462 shutdown_connections 2021-07-11T04:00:01.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6034053300 0x7f60340557c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60540b22a0 0x7f60540aece0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60540b1ec0 0x7f60540aadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 --2- 172.21.2.104:0/2261348462 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f60540affb0 0x7f60540ad6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 -- 172.21.2.104:0/2261348462 >> 172.21.2.104:0/2261348462 conn(0x7f605405f6b0 msgr2=0x7f6054147760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:01.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 -- 172.21.2.104:0/2261348462 shutdown_connections 2021-07-11T04:00:01.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.549+0000 7f605c346700 1 -- 172.21.2.104:0/2261348462 wait complete. 2021-07-11T04:00:01.559 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:00:01.559 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:01.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024e5d1700 1 Processor -- start 2021-07-11T04:00:01.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024e5d1700 1 -- start start 2021-07-11T04:00:01.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024e5d1700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f02480ad150 0x7f02480b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024e5d1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f02480a9da0 0x7f0248140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024e5d1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02480ac370 con 0x7f02480aa860 2021-07-11T04:00:01.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024e5d1700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480abb50 con 0x7f02480a9da0 2021-07-11T04:00:01.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024e5d1700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02480aac40 con 0x7f02480ad150 2021-07-11T04:00:01.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024cb6e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f02480a9da0 0x7f0248140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f0247fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f02480aa860 0x7f02480aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43494/0 (socket says 172.21.2.104:43494) 2021-07-11T04:00:01.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f0247fff700 1 -- 172.21.2.104:0/569655363 learned_addr learned my addr 172.21.2.104:0/569655363 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:01.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024cb6e700 1 -- 172.21.2.104:0/569655363 >> v2:172.21.2.105:3300/0 conn(0x7f02480ad150 msgr2=0x7f02480b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:01.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024cb6e700 1 --2- 172.21.2.104:0/569655363 >> v2:172.21.2.105:3300/0 conn(0x7f02480ad150 0x7f02480b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024cb6e700 1 -- 172.21.2.104:0/569655363 >> v1:172.21.2.104:6789/0 conn(0x7f02480aa860 legacy=0x7f02480aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024cb6e700 1 -- 172.21.2.104:0/569655363 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480b16b0 con 0x7f02480a9da0 2021-07-11T04:00:01.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024cb6e700 1 --2- 172.21.2.104:0/569655363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f02480a9da0 0x7f0248140340 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f02400032f0 tx=0x7f024000b470).ready entity=mon.0 client_cookie=e314956d2c7085c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f0246ffd700 1 -- 172.21.2.104:0/569655363 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f024000ddf0 con 0x7f02480a9da0 2021-07-11T04:00:01.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f0246ffd700 1 -- 172.21.2.104:0/569655363 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0240010470 con 0x7f02480a9da0 2021-07-11T04:00:01.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f0246ffd700 1 -- 172.21.2.104:0/569655363 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f02400107d0 con 0x7f02480a9da0 2021-07-11T04:00:01.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024e5d1700 1 -- 172.21.2.104:0/569655363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f02480a9da0 msgr2=0x7f0248140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.609+0000 7f024e5d1700 1 --2- 172.21.2.104:0/569655363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f02480a9da0 0x7f0248140340 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f02400032f0 tx=0x7f024000b470).stop 2021-07-11T04:00:01.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 -- 172.21.2.104:0/569655363 shutdown_connections 2021-07-11T04:00:01.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 --2- 172.21.2.104:0/569655363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f02480a9da0 0x7f0248140340 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 --2- 172.21.2.104:0/569655363 >> v2:172.21.2.105:3300/0 conn(0x7f02480ad150 0x7f02480b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 -- 172.21.2.104:0/569655363 >> 172.21.2.104:0/569655363 conn(0x7f02480b64d0 msgr2=0x7f02480b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:01.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 -- 172.21.2.104:0/569655363 shutdown_connections 2021-07-11T04:00:01.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 -- 172.21.2.104:0/569655363 wait complete. 2021-07-11T04:00:01.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 Processor -- start 2021-07-11T04:00:01.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 -- start start 2021-07-11T04:00:01.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f02480a9da0 0x7f02480afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02480aa860 0x7f024809e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02480ad150 0x7f024809eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0248066880 con 0x7f02480a9da0 2021-07-11T04:00:01.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0248065cd0 con 0x7f02480ad150 2021-07-11T04:00:01.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0248065f70 con 0x7f02480aa860 2021-07-11T04:00:01.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f0247fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f02480a9da0 0x7f02480afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f0247fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f02480a9da0 0x7f02480afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46732/0 (socket says 172.21.2.104:46732) 2021-07-11T04:00:01.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f0247fff700 1 -- 172.21.2.104:0/1210740799 learned_addr learned my addr 172.21.2.104:0/1210740799 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:01.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f0247fff700 1 -- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02480aa860 msgr2=0x7f024809e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f0247fff700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02480aa860 0x7f024809e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f0247fff700 1 -- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02480ad150 msgr2=0x7f024809eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f0247fff700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02480ad150 0x7f024809eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f0247fff700 1 -- 172.21.2.104:0/1210740799 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024000b710 con 0x7f02480a9da0 2021-07-11T04:00:01.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f0247fff700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f02480a9da0 0x7f02480afa60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7f02480b2e10 tx=0x7f0230007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f02457fa700 1 -- 172.21.2.104:0/1210740799 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f023000ce30 con 0x7f02480a9da0 2021-07-11T04:00:01.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f02457fa700 1 -- 172.21.2.104:0/1210740799 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02300072e0 con 0x7f02480a9da0 2021-07-11T04:00:01.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 -- 172.21.2.104:0/1210740799 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02480a12b0 con 0x7f02480a9da0 2021-07-11T04:00:01.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f02457fa700 1 -- 172.21.2.104:0/1210740799 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0230005800 con 0x7f02480a9da0 2021-07-11T04:00:01.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 -- 172.21.2.104:0/1210740799 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024809f430 con 0x7f02480a9da0 2021-07-11T04:00:01.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f024e5d1700 1 -- 172.21.2.104:0/1210740799 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0248066170 con 0x7f02480a9da0 2021-07-11T04:00:01.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f02457fa700 1 -- 172.21.2.104:0/1210740799 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0230010070 con 0x7f02480a9da0 2021-07-11T04:00:01.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f02457fa700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f022c053160 0x7f022c055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:01.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.613+0000 7f02457fa700 1 -- 172.21.2.104:0/1210740799 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f023006aa70 con 0x7f02480a9da0 2021-07-11T04:00:01.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.617+0000 7f02477fe700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f022c053160 0x7f022c055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:01.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.617+0000 7f02477fe700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f022c053160 0x7f022c055620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f0238002f70 tx=0x7f02380069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:01.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.617+0000 7f02457fa700 1 -- 172.21.2.104:0/1210740799 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0248066170 con 0x7f02480a9da0 2021-07-11T04:00:01.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.725+0000 7f024e5d1700 1 -- 172.21.2.104:0/1210740799 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0248066170 con 0x7f02480a9da0 2021-07-11T04:00:01.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.725+0000 7f02457fa700 1 -- 172.21.2.104:0/1210740799 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f0248066170 con 0x7f02480a9da0 2021-07-11T04:00:01.728 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:01.728 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/140397159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":140397159},{"type":"v1","addr":"172.21.2.105:6835","nonce":140397159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/2616903361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2616903361},{"type":"v1","addr":"172.21.2.104:6834","nonce":2616903361}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:59:13.620413+0000","modified":"2021-07-11T03:59:14.624165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/611806767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":611806767},{"type":"v1","addr":"172.21.2.104:6835","nonce":611806767}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-07-11T04:00:01.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 -- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f022c053160 msgr2=0x7f022c055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f022c053160 0x7f022c055620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f0238002f70 tx=0x7f02380069e0).stop 2021-07-11T04:00:01.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 -- 172.21.2.104:0/1210740799 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f02480a9da0 msgr2=0x7f02480afa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:01.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f02480a9da0 0x7f02480afa60 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7f02480b2e10 tx=0x7f0230007c00).stop 2021-07-11T04:00:01.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 -- 172.21.2.104:0/1210740799 shutdown_connections 2021-07-11T04:00:01.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02480ad150 0x7f024809eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02480aa860 0x7f024809e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:01.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f022c053160 0x7f022c055620 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 --2- 172.21.2.104:0/1210740799 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f02480a9da0 0x7f02480afa60 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:01.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 -- 172.21.2.104:0/1210740799 >> 172.21.2.104:0/1210740799 conn(0x7f02480b64d0 msgr2=0x7f0248143150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:01.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 -- 172.21.2.104:0/1210740799 shutdown_connections 2021-07-11T04:00:01.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:01.729+0000 7f024e5d1700 1 -- 172.21.2.104:0/1210740799 wait complete. 2021-07-11T04:00:01.732 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 32 2021-07-11T04:00:02.038 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T03:59:13.620413+0000', 'modified': '2021-07-11T03:59:14.624165+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6246}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6246': {'gid': 6246, 'name': 'c', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/611806767', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 611806767}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 611806767}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:00:02.039 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:00:02.039 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:00:02.107 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":6246,"want_state":"up:active","state":"up:active","rank_uptime":48.476931952000001,"mdsmap_epoch":32,"osdmap_epoch":77,"osdmap_epoch_barrier":77,"uptime":52.676414569999999} 2021-07-11T04:00:02.108 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 6246, "mdsmap_epoch": 32, "osdmap_epoch": 77, "osdmap_epoch_barrier": 77, "rank_uptime": 48.476931952, "state": "up:active", "uptime": 52.67641457, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:00:02.109 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:00:02.109 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T04:00:02.109 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T04:00:02.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:02.110 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:02.122 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:00:02.177 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:02.177 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:02.187 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:02.188 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:02.189 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:02.189 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:00:02.198 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:00:02.198 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:02.244 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:00:02.254 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:02.254 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:02.260 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:02.261 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:02.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:02.262 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:00:02.283 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T04:00:02.280+0000 7f95d3f7d080 -1 init, newargv = 0x564cf7251b40 newargc=9 2021-07-11T04:00:02.284 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15229]: starting ceph client 2021-07-11T04:00:02.299 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15229]: starting fuse 2021-07-11T04:00:03.391 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:00:03.403 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:03.404 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:03.454 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:03.454 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:03.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:03.456 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:00:03.461 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T04:00:03.462 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:00:03.462 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:03.463 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T04:00:03.463 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T04:00:03.464 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T04:00:03.464 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T04:00:03.464 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T04:00:03.465 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:03.465 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T04:00:03.466 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T04:00:03.466 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T04:00:03.466 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:03.467 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T04:00:03.467 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T04:00:03.468 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T04:00:03.468 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T04:00:03.468 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:03.469 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T04:00:03.469 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:00:03.470 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:00:03.471 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T04:00:03.471 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:00:03.471 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:03.472 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T04:00:03.472 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T04:00:03.503 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15229.asok 2021-07-11T04:00:03.506 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15229.asok 2021-07-11T04:00:03.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:03.507 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15229.asok status 2021-07-11T04:00:03.599 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T04:00:03.599 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T04:00:03.600 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:00:03.600 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:00:03.600 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T04:00:03.600 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T04:00:03.601 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T04:00:03.601 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15229", 2021-07-11T04:00:03.601 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T04:00:03.601 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:03.602 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T04:00:03.606 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T04:00:03.607 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4676, 2021-07-11T04:00:03.607 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T04:00:03.607 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T04:00:03.607 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T04:00:03.608 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4676 2021-07-11T04:00:03.608 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:03.608 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:00:03.608 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:00:03.608 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:00:03.609 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2695675706 2021-07-11T04:00:03.609 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T04:00:03.609 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:03.610 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:00:03.610 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:00:03.610 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:00:03.610 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2695675706 2021-07-11T04:00:03.610 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:03.611 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4676 172.21.2.119:0/2695675706", 2021-07-11T04:00:03.611 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/2695675706", 2021-07-11T04:00:03.611 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T04:00:03.611 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 32, 2021-07-11T04:00:03.612 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 77, 2021-07-11T04:00:03.612 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:00:03.612 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T04:00:03.612 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T04:00:03.612 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T04:00:03.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:03.615 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:03.615 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:03.624 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:00:03.625 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:00:03.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:03.626 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:03.678 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:00:03.679 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:03.692 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:03.708 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:03.736 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:03.751 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:00:03.751 INFO:tasks.cephfs.mount:Terminating background process 2021-07-11T04:00:03.752 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:03.752 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:03.753 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:03.766 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:00:03.766 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:00:03.767 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T04:00:03.767 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:00:03.768 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:03.823 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15229]: fuse finished with error 0 and tester_r 0 2021-07-11T04:00:03.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:03.825 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:03.825 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:03.835 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T04:00:03.836 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:00:03.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:00:03.882 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:03.888 INFO:tasks.cephfs.mount:Terminating background process 2021-07-11T04:00:03.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:03.891 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:03.892 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:03.904 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:00:03.906 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:00:03.906 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba023.front.sepia.ceph.com... 2021-07-11T04:00:03.907 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:00:03.907 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:03.962 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14657]: fuse finished with error 0 and tester_r 0 2021-07-11T04:00:03.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:03.964 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:03.964 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:03.969 INFO:teuthology.orchestra.run.gibba023.stdout:ext2/ext3 2021-07-11T04:00:03.970 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:00:04.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:00:04.086 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:04.133 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:00:04.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 Processor -- start 2021-07-11T04:00:04.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- start start 2021-07-11T04:00:04.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f13f40b0fd0 0x7f13f4144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f40b0350 con 0x7f13f40b1db0 2021-07-11T04:00:04.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f40b05f0 con 0x7f13f41483a0 2021-07-11T04:00:04.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f40a90c0 con 0x7f13f40b0fd0 2021-07-11T04:00:04.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f13f40b1db0 0x7f13f40b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43508/0 (socket says 172.21.2.104:43508) 2021-07-11T04:00:04.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 -- 172.21.2.104:0/2418016823 learned_addr learned my addr 172.21.2.104:0/2418016823 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:04.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651006032 0 0) 0x7f13f40b0350 con 0x7f13f40b1db0 2021-07-11T04:00:04.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d4003260 con 0x7f13f40b1db0 2021-07-11T04:00:04.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f9462700 1 --2- 172.21.2.104:0/2418016823 >> v2:172.21.2.105:3300/0 conn(0x7f13f40b0fd0 0x7f13f4144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 493429509 0 0) 0x7f13d4003260 con 0x7f13f40b1db0 2021-07-11T04:00:04.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 >> v2:172.21.2.105:3300/0 conn(0x7f13f40b0fd0 msgr2=0x7f13f4144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:04.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 --2- 172.21.2.104:0/2418016823 >> v2:172.21.2.105:3300/0 conn(0x7f13f40b0fd0 0x7f13f4144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 >> v1:172.21.2.105:6789/0 conn(0x7f13f41483a0 legacy=0x7f13f40a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:04.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f40b0890 con 0x7f13f40b1db0 2021-07-11T04:00:04.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f13dc001660 con 0x7f13f40b1db0 2021-07-11T04:00:04.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 >> v1:172.21.2.104:6789/0 conn(0x7f13f40b1db0 legacy=0x7f13f40b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 --2- 172.21.2.104:0/2418016823 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13d4003820 0x7f13d4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 --2- 172.21.2.104:0/2418016823 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13d4004500 0x7f13d4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 --2- 172.21.2.104:0/2418016823 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 0x7f13d400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d400be20 con 0x7f13d4004f70 2021-07-11T04:00:04.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d400bf90 con 0x7f13d4004500 2021-07-11T04:00:04.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d400c150 con 0x7f13d4003820 2021-07-11T04:00:04.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f9462700 1 --2- 172.21.2.104:0/2418016823 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 0x7f13d400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 --2- 172.21.2.104:0/2418016823 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13d4003820 0x7f13d4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f9462700 1 -- 172.21.2.104:0/2418016823 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13d4003820 msgr2=0x7f13d4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:04.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f9462700 1 --2- 172.21.2.104:0/2418016823 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13d4003820 0x7f13d4003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f9462700 1 -- 172.21.2.104:0/2418016823 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13d4004500 msgr2=0x7f13d4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:04.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f9462700 1 --2- 172.21.2.104:0/2418016823 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13d4004500 0x7f13d4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:04.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f9462700 1 -- 172.21.2.104:0/2418016823 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4003260 con 0x7f13d4004f70 2021-07-11T04:00:04.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f9462700 1 -- 172.21.2.104:0/2418016823 reap_dead start 2021-07-11T04:00:04.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f9462700 1 --2- 172.21.2.104:0/2418016823 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 0x7f13d400b750 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f13ec00a000 tx=0x7f13ec008040).ready entity=mon.0 client_cookie=f4de474806be401f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:04.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13ec00c070 con 0x7f13d4004f70 2021-07-11T04:00:04.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec00e810 con 0x7f13d4004f70 2021-07-11T04:00:04.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f37fe700 1 -- 172.21.2.104:0/2418016823 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13ec00eb70 con 0x7f13d4004f70 2021-07-11T04:00:04.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- 172.21.2.104:0/2418016823 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 msgr2=0x7f13d400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 --2- 172.21.2.104:0/2418016823 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 0x7f13d400b750 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f13ec00a000 tx=0x7f13ec008040).stop 2021-07-11T04:00:04.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- 172.21.2.104:0/2418016823 shutdown_connections 2021-07-11T04:00:04.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 --2- 172.21.2.104:0/2418016823 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 0x7f13d400b750 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- 172.21.2.104:0/2418016823 >> 172.21.2.104:0/2418016823 conn(0x7f13f405f6b0 msgr2=0x7f13f405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:04.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- 172.21.2.104:0/2418016823 shutdown_connections 2021-07-11T04:00:04.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- 172.21.2.104:0/2418016823 wait complete. 2021-07-11T04:00:04.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 Processor -- start 2021-07-11T04:00:04.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- start start 2021-07-11T04:00:04.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 0x7f13f4143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13f4149010 0x7f13f40adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13f4148b00 0x7f13f41565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40ae070 con 0x7f13d4004f70 2021-07-11T04:00:04.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f415a840 con 0x7f13f4149010 2021-07-11T04:00:04.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40ad6e0 con 0x7f13f4148b00 2021-07-11T04:00:04.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 0x7f13f4143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 0x7f13f4143100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46752/0 (socket says 172.21.2.104:46752) 2021-07-11T04:00:04.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 -- 172.21.2.104:0/2227909317 learned_addr learned my addr 172.21.2.104:0/2227909317 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:04.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 -- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13f4148b00 msgr2=0x7f13f41565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13f4148b00 0x7f13f41565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:04.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 -- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13f4149010 msgr2=0x7f13f40adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13f4149010 0x7f13f40adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:04.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 -- 172.21.2.104:0/2227909317 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f415a490 con 0x7f13d4004f70 2021-07-11T04:00:04.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f8c61700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 0x7f13f4143100 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7f13dc00b000 tx=0x7f13f40b0780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:04.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f1ffb700 1 -- 172.21.2.104:0/2227909317 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13dc010680 con 0x7f13d4004f70 2021-07-11T04:00:04.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f1ffb700 1 -- 172.21.2.104:0/2227909317 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13dc010040 con 0x7f13d4004f70 2021-07-11T04:00:04.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f1ffb700 1 -- 172.21.2.104:0/2227909317 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13dc0109d0 con 0x7f13d4004f70 2021-07-11T04:00:04.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- 172.21.2.104:0/2227909317 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f4156ad0 con 0x7f13d4004f70 2021-07-11T04:00:04.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- 172.21.2.104:0/2227909317 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f414ec10 con 0x7f13d4004f70 2021-07-11T04:00:04.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13faec5700 1 -- 172.21.2.104:0/2227909317 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f415aa40 con 0x7f13d4004f70 2021-07-11T04:00:04.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f1ffb700 1 -- 172.21.2.104:0/2227909317 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f13dc010b30 con 0x7f13d4004f70 2021-07-11T04:00:04.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f1ffb700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13d80532d0 0x7f13d8055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f1ffb700 1 -- 172.21.2.104:0/2227909317 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f13dc06b060 con 0x7f13d4004f70 2021-07-11T04:00:04.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.189+0000 7f13f3fff700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13d80532d0 0x7f13d8055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.193+0000 7f13f3fff700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13d80532d0 0x7f13d8055790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f13e4001fa0 tx=0x7f13e40077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:04.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.193+0000 7f13f1ffb700 1 -- 172.21.2.104:0/2227909317 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f13dc042270 con 0x7f13d4004f70 2021-07-11T04:00:04.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 -- 172.21.2.104:0/2227909317 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f13f40ad8a0 con 0x7f13d4004f70 2021-07-11T04:00:04.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13f1ffb700 1 -- 172.21.2.104:0/2227909317 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v77) v1 ==== 81+0+33724 (secure 0 0 0) 0x7f13dc045890 con 0x7f13d4004f70 2021-07-11T04:00:04.280 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:04.281 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:00:04.281 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 77, 2021-07-11T04:00:04.281 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:00:04.282 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:00:04.282 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T03:59:55.331085+0000", 2021-07-11T04:00:04.282 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:04.282 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:04.283 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:00:04.284 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:00:04.284 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:00:04.284 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:00:04.284 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:00:04.285 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:00:04.285 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:00:04.285 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:04.285 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:00:04.286 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:00:04.286 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:00:04.286 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:00:04.286 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:00:04.287 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 14, 2021-07-11T04:00:04.287 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:00:04.287 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:00:04.287 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:00:04.287 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:00:04.288 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:00:04.288 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.288 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:00:04.288 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:00:04.289 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:00:04.289 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:04.289 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:04.290 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:04.290 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:04.290 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:04.291 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:04.291 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:04.291 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:04.291 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:00:04.292 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:00:04.292 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:00:04.292 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:00:04.292 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:00:04.292 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:04.293 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:04.293 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:04.294 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:04.294 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:04.294 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:04.295 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:04.295 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.295 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:00:04.295 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:04.296 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:04.296 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:04.296 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:04.296 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:04.296 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:04.297 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:04.297 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:04.297 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:04.297 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:04.298 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:04.298 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:04.298 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:04.299 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:04.299 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:04.299 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:04.299 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:04.300 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:04.300 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:04.301 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:04.301 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:04.301 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:04.301 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:04.302 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:04.302 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:04.302 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:04.302 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.303 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:04.303 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:04.303 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:04.303 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:04.303 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:04.304 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:04.304 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:04.305 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:04.305 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:04.305 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:04.305 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:04.306 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:00:04.306 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:00:04.306 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.306 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:04.307 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:00:04.307 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.307 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.307 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.307 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:00:04.308 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:00:04.308 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:00:04.308 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:04.308 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:04.309 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:04.309 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:04.309 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:04.309 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:04.310 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:04.310 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:04.310 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:04.311 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:04.311 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:04.312 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:04.312 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:04.312 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:04.312 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:04.313 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:04.313 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:04.313 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:04.313 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:04.313 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:04.314 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.314 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:00:04.314 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:04.315 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:04.315 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:04.315 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:04.315 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:04.316 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:04.316 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:04.317 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:04.317 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:04.317 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:04.317 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:04.318 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:04.318 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:04.319 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:04.319 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:04.319 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:04.320 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:04.320 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:04.320 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:04.322 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:04.322 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:04.322 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:04.322 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:04.323 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:04.323 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:04.323 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:04.324 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.324 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:04.324 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:04.325 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:04.325 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:04.325 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:04.325 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:04.326 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:04.326 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:04.327 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:04.327 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:04.327 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:04.327 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:00:04.328 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:04.328 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:00:04.328 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.328 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.329 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.329 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 13, 2021-07-11T04:00:04.330 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T04:00:04.330 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:59:11.477806+0000", 2021-07-11T04:00:04.330 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:04.331 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:04.331 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:04.331 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:04.332 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:04.332 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:04.332 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:04.332 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:04.333 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:04.333 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:04.333 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:04.334 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:04.334 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:04.334 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:04.334 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:04.335 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:04.335 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:04.335 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:04.335 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:04.336 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:04.336 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.336 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "75", 2021-07-11T04:00:04.337 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:04.337 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:04.337 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:04.338 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:04.338 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:04.338 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:04.339 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:04.339 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:04.339 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:04.339 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:04.340 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:04.341 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:04.341 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:04.341 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:04.342 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:04.342 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:04.342 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:04.343 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:04.343 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:04.343 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:04.344 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:04.344 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:04.344 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:04.344 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:04.345 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:04.345 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:04.345 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.345 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:04.347 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:04.347 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:04.347 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:04.348 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:04.348 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:04.348 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:04.348 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:04.348 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:04.349 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:04.349 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:04.349 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:00:04.349 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T04:00:04.349 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T04:00:04.350 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T04:00:04.350 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.350 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:04.350 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:00:04.351 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T04:00:04.352 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.353 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.353 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.354 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.354 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 14, 2021-07-11T04:00:04.354 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T04:00:04.354 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:59:12.643109+0000", 2021-07-11T04:00:04.358 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:04.358 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:04.358 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:04.358 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:04.359 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:04.359 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:04.359 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:04.359 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:04.360 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:04.360 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:04.360 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:04.360 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:04.361 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:04.361 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:04.362 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:04.362 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:04.363 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:04.363 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:04.363 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:04.363 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:04.364 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.364 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "76", 2021-07-11T04:00:04.364 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:04.364 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:04.365 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:04.365 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:04.366 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:04.366 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:04.366 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:04.366 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:04.367 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:04.367 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:04.367 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:04.367 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:04.368 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:04.368 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:04.368 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:04.368 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:04.369 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:04.369 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:04.369 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:04.369 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:04.370 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:04.370 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:04.370 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:04.370 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:04.371 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:04.371 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:04.371 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.371 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:04.372 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:04.374 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:04.374 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:04.374 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:04.374 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:04.375 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:04.375 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:04.375 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:04.375 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:04.376 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:04.376 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:00:04.376 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:04.377 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:00:04.377 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T04:00:04.377 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.377 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.377 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.378 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:04.378 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:00:04.378 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.378 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:04.379 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:00:04.379 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:04.379 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:04.379 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:04.379 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:04.380 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:04.380 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:04.380 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:04.380 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 74, 2021-07-11T04:00:04.380 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:04.381 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:04.381 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:04.381 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.381 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.382 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.382 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:00:04.382 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:04.383 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.383 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.384 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.384 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:00:04.384 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:04.384 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.384 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.385 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.385 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:04.385 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.385 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.386 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.386 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:00:04.386 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:04.386 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.387 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.387 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.387 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:00:04.387 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:04.387 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.388 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.388 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.388 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:04.388 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.389 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.389 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.389 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:00:04.389 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:04.390 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.390 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.390 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.390 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:00:04.391 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:04.391 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.391 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.391 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.391 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:04.392 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.392 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.392 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.395 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:00:04.395 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:04.395 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.396 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.396 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.396 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:00:04.396 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:04.396 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.397 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.397 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.397 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:00:04.397 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:00:04.398 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:00:04.398 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:00:04.398 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:04.398 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:04.399 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:04.399 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.399 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.399 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.400 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:04.400 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:00:04.400 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:04.401 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:04.401 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:04.401 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:04.401 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:04.402 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:04.402 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:04.402 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 62, 2021-07-11T04:00:04.402 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:04.403 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:04.403 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:04.403 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.403 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.403 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.404 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:00:04.404 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:04.404 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.404 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.405 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.405 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:00:04.405 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:04.405 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.405 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.406 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.406 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:04.406 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.407 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.407 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.407 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:00:04.407 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:04.407 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.408 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.408 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.408 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:00:04.408 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:04.409 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.409 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.409 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.409 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:04.410 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.410 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.410 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.410 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:00:04.411 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:04.411 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.411 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.411 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.412 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:00:04.412 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:04.412 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.412 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.412 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.413 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:04.413 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.413 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.413 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.414 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:00:04.414 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:04.414 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.414 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.414 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.415 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:00:04.415 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:04.415 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.416 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.416 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.416 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:00:04.416 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:00:04.417 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:00:04.417 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:00:04.417 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:04.417 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:04.417 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:04.418 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.418 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.418 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.418 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:04.419 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:00:04.419 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:04.419 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:04.419 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:04.420 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:04.420 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:04.420 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:04.420 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:04.420 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 74, 2021-07-11T04:00:04.421 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:04.421 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:04.421 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:04.421 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.421 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.422 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.422 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:00:04.422 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:04.422 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.423 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.423 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.423 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:00:04.423 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:04.424 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.424 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.424 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.424 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:04.424 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.425 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.425 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.425 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:00:04.425 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:04.426 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.426 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.426 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.426 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:00:04.427 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:04.427 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.427 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.427 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.427 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:04.428 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.428 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.428 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.428 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:00:04.429 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:04.429 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.429 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.429 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.429 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:00:04.430 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:04.430 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.430 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.430 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.431 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:04.431 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.431 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.431 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.431 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:00:04.432 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:04.432 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.432 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.433 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.433 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:00:04.433 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:04.433 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.433 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.434 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.434 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:00:04.434 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:00:04.434 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:00:04.435 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:00:04.435 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:04.435 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:04.435 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:04.435 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.436 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.436 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.436 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:04.436 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:00:04.437 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:04.437 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:04.437 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:04.437 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:04.438 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:04.438 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:04.438 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:04.438 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 74, 2021-07-11T04:00:04.438 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:04.439 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:04.439 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:04.439 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.439 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.440 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.440 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:00:04.440 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:04.440 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.441 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.441 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.441 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:00:04.441 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:04.441 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.442 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.442 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.442 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:04.443 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.443 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.443 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.444 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:00:04.444 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:04.444 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.444 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.445 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.445 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:00:04.445 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:04.445 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.445 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.446 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.446 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:04.446 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.446 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.446 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.447 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:00:04.447 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:04.447 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.447 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.448 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.448 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:00:04.448 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:04.448 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.448 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.449 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.449 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:04.449 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.449 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.450 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.450 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:00:04.450 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:04.451 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.451 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.451 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.451 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:00:04.451 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:04.452 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.452 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.452 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.452 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:00:04.453 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:00:04.453 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:00:04.453 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:00:04.453 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:04.453 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:04.454 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:04.454 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.454 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.454 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.455 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:04.455 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:00:04.455 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:04.455 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:04.455 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:04.456 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:04.456 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:04.456 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:04.456 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:04.456 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 73, 2021-07-11T04:00:04.457 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:04.457 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:04.457 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:04.457 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.458 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.458 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.458 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:00:04.458 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:04.458 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.459 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.459 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.459 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:00:04.459 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:04.460 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.460 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.460 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.460 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:04.460 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.461 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.461 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.461 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:00:04.461 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:04.461 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.462 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.462 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.462 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:00:04.462 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:04.463 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.463 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.463 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.463 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:04.464 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.464 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.464 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.464 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:00:04.464 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:04.465 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.465 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.465 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.465 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:00:04.466 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:04.466 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.466 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.466 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.466 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:04.467 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.467 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.467 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.468 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:00:04.468 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:04.468 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.468 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.469 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.469 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:00:04.469 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:04.469 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.469 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.470 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.470 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:00:04.470 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:00:04.470 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:00:04.471 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:00:04.471 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:04.471 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:04.471 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:04.471 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.472 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.472 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.472 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:04.472 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:00:04.472 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:04.473 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:04.473 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:04.473 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:04.473 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:04.474 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:04.474 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:04.474 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 74, 2021-07-11T04:00:04.474 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:04.474 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:04.475 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:04.475 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.475 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.475 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.476 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:00:04.476 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:04.476 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.476 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.476 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.477 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:00:04.477 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:04.477 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.477 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.478 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.478 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:04.478 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.478 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.478 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.479 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:00:04.479 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:04.479 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.479 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.480 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.480 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:00:04.480 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:04.480 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.480 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.481 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.481 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:04.481 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.481 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.482 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.482 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:00:04.482 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:04.482 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.483 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.483 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.483 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:00:04.483 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:04.484 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.484 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.484 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.485 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:04.485 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.485 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.485 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.485 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:00:04.486 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:04.486 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.486 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.486 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.487 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:00:04.487 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:04.487 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.487 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.488 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.488 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:00:04.488 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:00:04.488 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:00:04.489 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:00:04.489 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:04.489 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:04.489 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:04.489 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.490 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.490 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.490 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:04.490 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:00:04.490 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:04.491 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:04.491 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:04.491 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:04.491 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:04.492 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:04.492 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:04.492 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 74, 2021-07-11T04:00:04.492 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:04.492 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:04.493 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:04.493 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.493 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.493 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.494 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:00:04.494 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:04.494 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.494 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.495 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.495 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:00:04.495 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:04.495 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.495 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.496 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.496 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:04.496 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.496 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.497 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.497 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:00:04.497 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:04.497 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.497 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.498 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.498 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:00:04.498 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:04.498 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.498 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.499 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.499 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:04.499 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.499 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.500 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.500 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:00:04.500 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:04.500 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.500 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.501 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.501 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:00:04.501 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:04.501 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.502 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.502 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.502 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:04.502 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.503 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.503 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.503 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:00:04.503 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:04.503 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.504 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.504 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.504 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:00:04.504 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:04.505 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.505 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.505 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.505 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:00:04.505 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:00:04.506 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:00:04.506 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:00:04.506 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:04.506 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:04.507 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:04.507 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.507 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.507 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.508 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:04.508 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:00:04.508 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:04.508 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:04.508 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:04.509 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:04.509 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:04.509 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:04.509 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:04.509 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 73, 2021-07-11T04:00:04.510 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:04.510 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:04.510 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:04.510 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.511 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.511 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.511 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:00:04.511 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:04.511 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.512 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.512 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.512 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:00:04.512 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:04.513 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.513 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.513 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.513 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:04.513 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.514 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.514 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.514 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:00:04.514 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:04.515 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.515 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.515 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.515 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:00:04.515 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:04.516 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.516 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.516 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.516 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:04.517 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.517 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.517 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.517 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:00:04.518 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:04.518 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.518 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.518 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.518 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:00:04.519 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:04.519 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.519 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.520 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.520 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:04.520 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:04.520 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.520 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:04.521 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:00:04.521 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:04.521 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.521 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.521 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:04.522 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:00:04.522 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:04.522 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.522 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.523 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.523 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:00:04.523 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:00:04.523 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:00:04.523 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:00:04.524 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:04.524 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:04.524 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:04.524 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:04.525 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.525 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:04.525 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:00:04.526 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.526 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:04.527 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:04.527 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:04.527 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:04.527 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:04.527 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:04.528 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:00:04.528 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:04.528 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.528 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.529 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:04.529 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:04.529 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:04.529 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:04.530 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:04.530 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:04.530 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:00:04.531 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:04.531 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.531 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.531 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:04.531 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:04.532 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:04.532 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:04.533 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:04.533 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:04.533 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:00:04.534 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:04.534 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.534 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.534 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:04.534 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:04.535 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:04.535 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:04.535 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:04.535 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:04.536 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:00:04.536 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:04.536 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.536 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.536 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:04.537 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:04.537 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:04.537 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:04.537 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:04.538 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:04.538 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:00:04.538 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:04.538 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.539 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.539 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:04.539 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:04.539 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:04.539 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:04.540 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:04.540 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:04.540 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:00:04.540 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:04.541 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.541 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.541 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:04.541 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:04.541 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:04.542 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:04.542 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:04.542 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:04.542 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:00:04.543 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:04.543 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.543 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:04.543 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:04.544 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:04.544 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:04.544 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:04.544 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:04.545 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:04.545 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:00:04.545 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:04.545 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.545 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:04.546 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:00:04.546 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:00:04.546 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:00:04.546 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:00:04.546 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T04:00:04.547 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.119:0/1304352204": "2021-07-11T04:59:54.428687+0000" 2021-07-11T04:00:04.547 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.547 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:00:04.547 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:00:04.548 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:00:04.548 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:00:04.548 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:00:04.548 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:00:04.548 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:00:04.549 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:00:04.549 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:04.549 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:04.549 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:00:04.550 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:00:04.550 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:00:04.550 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:00:04.550 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:00:04.550 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:00:04.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 -- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13d80532d0 msgr2=0x7f13d8055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13d80532d0 0x7f13d8055790 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f13e4001fa0 tx=0x7f13e40077f0).stop 2021-07-11T04:00:04.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 -- 172.21.2.104:0/2227909317 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 msgr2=0x7f13f4143100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 0x7f13f4143100 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7f13dc00b000 tx=0x7f13f40b0780).stop 2021-07-11T04:00:04.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 -- 172.21.2.104:0/2227909317 shutdown_connections 2021-07-11T04:00:04.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13f4148b00 0x7f13f41565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:04.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13f4149010 0x7f13f40adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:04.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13d80532d0 0x7f13d8055790 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 --2- 172.21.2.104:0/2227909317 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13d4004f70 0x7f13f4143100 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 -- 172.21.2.104:0/2227909317 >> 172.21.2.104:0/2227909317 conn(0x7f13f405f6b0 msgr2=0x7f13f40b3670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:04.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 -- 172.21.2.104:0/2227909317 shutdown_connections 2021-07-11T04:00:04.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.273+0000 7f13faec5700 1 -- 172.21.2.104:0/2227909317 wait complete. 2021-07-11T04:00:04.556 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:04.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8987aaf700 1 Processor -- start 2021-07-11T04:00:04.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8987aaf700 1 -- start start 2021-07-11T04:00:04.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8987aaf700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f89800a76b0 0x7f8980066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8987aaf700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89800a7a90 con 0x7f89800b1970 2021-07-11T04:00:04.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8987aaf700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8980147620 con 0x7f89800b0060 2021-07-11T04:00:04.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8987aaf700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89801478c0 con 0x7f89800a76b0 2021-07-11T04:00:04.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f898504a700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f89800b1970 0x7f898005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43532/0 (socket says 172.21.2.104:43532) 2021-07-11T04:00:04.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f898504a700 1 -- 172.21.2.104:0/4199936375 learned_addr learned my addr 172.21.2.104:0/4199936375 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:04.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f898584b700 1 --2- 172.21.2.104:0/4199936375 >> v2:172.21.2.104:3300/0 conn(0x7f89800a76b0 0x7f8980066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8984849700 1 -- 172.21.2.104:0/4199936375 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4246152514 0 0) 0x7f89800a7a90 con 0x7f89800b1970 2021-07-11T04:00:04.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8984849700 1 -- 172.21.2.104:0/4199936375 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8968003260 con 0x7f89800b1970 2021-07-11T04:00:04.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f898584b700 1 -- 172.21.2.104:0/4199936375 >> v1:172.21.2.105:6789/0 conn(0x7f89800b0060 legacy=0x7f8980151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:04.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f898584b700 1 -- 172.21.2.104:0/4199936375 >> v1:172.21.2.104:6789/0 conn(0x7f89800b1970 legacy=0x7f898005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f898584b700 1 -- 172.21.2.104:0/4199936375 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8980147b60 con 0x7f89800a76b0 2021-07-11T04:00:04.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f898584b700 1 --2- 172.21.2.104:0/4199936375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89800a76b0 0x7f8980066a70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f898006a180 tx=0x7f897c006df0).ready entity=mon.0 client_cookie=e8d5ce34b65639a4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:04.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8984849700 1 -- 172.21.2.104:0/4199936375 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f897c00d5c0 con 0x7f89800a76b0 2021-07-11T04:00:04.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8984849700 1 -- 172.21.2.104:0/4199936375 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c000e90 con 0x7f89800a76b0 2021-07-11T04:00:04.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8984849700 1 -- 172.21.2.104:0/4199936375 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f897c00d910 con 0x7f89800a76b0 2021-07-11T04:00:04.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8987aaf700 1 -- 172.21.2.104:0/4199936375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89800a76b0 msgr2=0x7f8980066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.609+0000 7f8987aaf700 1 --2- 172.21.2.104:0/4199936375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89800a76b0 0x7f8980066a70 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f898006a180 tx=0x7f897c006df0).stop 2021-07-11T04:00:04.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 -- 172.21.2.104:0/4199936375 shutdown_connections 2021-07-11T04:00:04.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 --2- 172.21.2.104:0/4199936375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89800a76b0 0x7f8980066a70 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 -- 172.21.2.104:0/4199936375 >> 172.21.2.104:0/4199936375 conn(0x7f89800b53e0 msgr2=0x7f89800b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:04.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 -- 172.21.2.104:0/4199936375 shutdown_connections 2021-07-11T04:00:04.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 -- 172.21.2.104:0/4199936375 wait complete. 2021-07-11T04:00:04.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 Processor -- start 2021-07-11T04:00:04.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 -- start start 2021-07-11T04:00:04.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89800a76b0 0x7f89800b3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f89800b0060 0x7f89800ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89800b1970 0x7f89800aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800ae960 con 0x7f89800b1970 2021-07-11T04:00:04.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800ab600 con 0x7f89800b0060 2021-07-11T04:00:04.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89800ab8a0 con 0x7f89800a76b0 2021-07-11T04:00:04.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f898604c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89800b1970 0x7f89800aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f898604c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89800b1970 0x7f89800aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46770/0 (socket says 172.21.2.104:46770) 2021-07-11T04:00:04.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f898604c700 1 -- 172.21.2.104:0/3213569828 learned_addr learned my addr 172.21.2.104:0/3213569828 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:04.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f898604c700 1 -- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89800a76b0 msgr2=0x7f89800b3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f898604c700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89800a76b0 0x7f89800b3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:04.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f898604c700 1 -- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f89800b0060 msgr2=0x7f89800ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f898604c700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f89800b0060 0x7f89800ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:04.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f898604c700 1 -- 172.21.2.104:0/3213569828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c00b180 con 0x7f89800b1970 2021-07-11T04:00:04.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f898604c700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89800b1970 0x7f89800aeb50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f8978003310 tx=0x7f89780091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:04.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8972ffd700 1 -- 172.21.2.104:0/3213569828 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8978001500 con 0x7f89800b1970 2021-07-11T04:00:04.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8972ffd700 1 -- 172.21.2.104:0/3213569828 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8978001660 con 0x7f89800b1970 2021-07-11T04:00:04.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 -- 172.21.2.104:0/3213569828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89800adfd0 con 0x7f89800b1970 2021-07-11T04:00:04.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8972ffd700 1 -- 172.21.2.104:0/3213569828 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89780019c0 con 0x7f89800b1970 2021-07-11T04:00:04.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 -- 172.21.2.104:0/3213569828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89800af040 con 0x7f89800b1970 2021-07-11T04:00:04.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8987aaf700 1 -- 172.21.2.104:0/3213569828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89800abaa0 con 0x7f89800b1970 2021-07-11T04:00:04.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8972ffd700 1 -- 172.21.2.104:0/3213569828 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8978018030 con 0x7f89800b1970 2021-07-11T04:00:04.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8972ffd700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8960053260 0x7f8960055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.613+0000 7f8972ffd700 1 -- 172.21.2.104:0/3213569828 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7f897806afd0 con 0x7f89800b1970 2021-07-11T04:00:04.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.617+0000 7f898584b700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8960053260 0x7f8960055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.617+0000 7f898584b700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8960053260 0x7f8960055720 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f898006a180 tx=0x7f897c014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:04.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.617+0000 7f8972ffd700 1 -- 172.21.2.104:0/3213569828 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8978042290 con 0x7f89800b1970 2021-07-11T04:00:04.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.721+0000 7f8987aaf700 1 -- 172.21.2.104:0/3213569828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89800ae190 con 0x7f89800b1970 2021-07-11T04:00:04.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8972ffd700 1 -- 172.21.2.104:0/3213569828 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f89780458b0 con 0x7f89800b1970 2021-07-11T04:00:04.729 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:04.729 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":32,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/140397159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":140397159},{"type":"v1","addr":"172.21.2.105:6835","nonce":140397159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/2616903361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2616903361},{"type":"v1","addr":"172.21.2.104:6834","nonce":2616903361}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:59:13.620413+0000","modified":"2021-07-11T03:59:14.624165+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"c","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/611806767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":611806767},{"type":"v1","addr":"172.21.2.104:6835","nonce":611806767}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-07-11T04:00:04.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 -- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8960053260 msgr2=0x7f8960055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8960053260 0x7f8960055720 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f898006a180 tx=0x7f897c014000).stop 2021-07-11T04:00:04.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 -- 172.21.2.104:0/3213569828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89800b1970 msgr2=0x7f89800aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89800b1970 0x7f89800aeb50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f8978003310 tx=0x7f89780091a0).stop 2021-07-11T04:00:04.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 -- 172.21.2.104:0/3213569828 shutdown_connections 2021-07-11T04:00:04.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8960053260 0x7f8960055720 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89800b1970 0x7f89800aeb50 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f89800b0060 0x7f89800ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:04.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 --2- 172.21.2.104:0/3213569828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89800a76b0 0x7f89800b3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:04.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 -- 172.21.2.104:0/3213569828 >> 172.21.2.104:0/3213569828 conn(0x7f89800b53e0 msgr2=0x7f8980148990 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:04.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 -- 172.21.2.104:0/3213569828 shutdown_connections 2021-07-11T04:00:04.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.725+0000 7f8987aaf700 1 -- 172.21.2.104:0/3213569828 wait complete. 2021-07-11T04:00:04.733 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 32 2021-07-11T04:00:04.736 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T04:00:04.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.781+0000 7fcf0166b700 1 Processor -- start 2021-07-11T04:00:04.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.781+0000 7fcf0166b700 1 -- start start 2021-07-11T04:00:04.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.781+0000 7fcf0166b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcefc067180 0x7fcefc05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.781+0000 7fcf0166b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc0b1970 0x7fcefc05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.781+0000 7fcf0166b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcefc0a7a90 con 0x7fcefc0a76b0 2021-07-11T04:00:04.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.781+0000 7fcf0166b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcefc147620 con 0x7fcefc067180 2021-07-11T04:00:04.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.781+0000 7fcf0166b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc1478c0 con 0x7fcefc0b1970 2021-07-11T04:00:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.781+0000 7fcefaffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fcefc0a76b0 0x7fcefc066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43542/0 (socket says 172.21.2.104:43542) 2021-07-11T04:00:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.781+0000 7fcefaffd700 1 -- 172.21.2.104:0/2959630039 learned_addr learned my addr 172.21.2.104:0/2959630039 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201611655 0 0) 0x7fcefc0a7a90 con 0x7fcefc0a76b0 2021-07-11T04:00:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcedc003260 con 0x7fcefc0a76b0 2021-07-11T04:00:04.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefb7fe700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc0b1970 0x7fcefc05fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefa7fc700 1 --2- 172.21.2.104:0/2959630039 >> v2:172.21.2.105:3300/0 conn(0x7fcefc067180 0x7fcefc05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2353061976 0 0) 0x7fcedc003260 con 0x7fcefc0a76b0 2021-07-11T04:00:04.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc0b1970 msgr2=0x7fcefc05fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:04.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc0b1970 0x7fcefc05fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 >> v2:172.21.2.105:3300/0 conn(0x7fcefc067180 msgr2=0x7fcefc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:04.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 --2- 172.21.2.104:0/2959630039 >> v2:172.21.2.105:3300/0 conn(0x7fcefc067180 0x7fcefc05f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc147b60 con 0x7fcefc0a76b0 2021-07-11T04:00:04.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fceec001660 con 0x7fcefc0a76b0 2021-07-11T04:00:04.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 >> v1:172.21.2.104:6789/0 conn(0x7fcefc0a76b0 legacy=0x7fcefc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcedc0037d0 0x7fcedc003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc0b1970 0x7fcedc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcedc0049f0 0x7fcedc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc147620 con 0x7fcedc0049f0 2021-07-11T04:00:04.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc1478c0 con 0x7fcedc0037d0 2021-07-11T04:00:04.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc00b880 con 0x7fcefc0b1970 2021-07-11T04:00:04.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefb7fe700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcedc0049f0 0x7fcedc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefaffd700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcedc0037d0 0x7fcedc003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefb7fe700 1 -- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc0b1970 msgr2=0x7fcedc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:04.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefa7fc700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc0b1970 0x7fcedc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefb7fe700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc0b1970 0x7fcedc004500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefb7fe700 1 -- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcedc0037d0 msgr2=0x7fcedc003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:04.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefb7fe700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcedc0037d0 0x7fcedc003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefb7fe700 1 -- 172.21.2.104:0/2959630039 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc003260 con 0x7fcedc0049f0 2021-07-11T04:00:04.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefaffd700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcedc0037d0 0x7fcedc003bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefb7fe700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcedc0049f0 0x7fcedc004e70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7fcef400a000 tx=0x7fcef4008040).ready entity=mon.0 client_cookie=1c0a61786ae134bf server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef400c070 con 0x7fcedc0049f0 2021-07-11T04:00:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef400e5f0 con 0x7fcedc0049f0 2021-07-11T04:00:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcef9ffb700 1 -- 172.21.2.104:0/2959630039 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcef400e950 con 0x7fcedc0049f0 2021-07-11T04:00:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 -- 172.21.2.104:0/2959630039 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcedc0049f0 msgr2=0x7fcedc004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 --2- 172.21.2.104:0/2959630039 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcedc0049f0 0x7fcedc004e70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7fcef400a000 tx=0x7fcef4008040).stop 2021-07-11T04:00:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefb7fe700 1 -- 172.21.2.104:0/2959630039 reap_dead start 2021-07-11T04:00:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 -- 172.21.2.104:0/2959630039 shutdown_connections 2021-07-11T04:00:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 -- 172.21.2.104:0/2959630039 >> 172.21.2.104:0/2959630039 conn(0x7fcefc0b53e0 msgr2=0x7fcefc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 -- 172.21.2.104:0/2959630039 shutdown_connections 2021-07-11T04:00:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 -- 172.21.2.104:0/2959630039 wait complete. 2021-07-11T04:00:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 Processor -- start 2021-07-11T04:00:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 -- start start 2021-07-11T04:00:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcefc0adf60 0x7fcefc0ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcefc0aa710 0x7fcefc159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc15a020 0x7fcefc0ab190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0af6e0 con 0x7fcefc0aa710 2021-07-11T04:00:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0a9310 con 0x7fcefc0adf60 2021-07-11T04:00:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcf0166b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0af440 con 0x7fcefc15a020 2021-07-11T04:00:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefa7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcefc0aa710 0x7fcefc159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefa7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcefc0aa710 0x7fcefc159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46788/0 (socket says 172.21.2.104:46788) 2021-07-11T04:00:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefa7fc700 1 -- 172.21.2.104:0/2425432233 learned_addr learned my addr 172.21.2.104:0/2425432233 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefaffd700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcefc0adf60 0x7fcefc0ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefa7fc700 1 -- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc15a020 msgr2=0x7fcefc0ab190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefa7fc700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc15a020 0x7fcefc0ab190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefa7fc700 1 -- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcefc0adf60 msgr2=0x7fcefc0ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.785+0000 7fcefa7fc700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcefc0adf60 0x7fcefc0ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:04.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcefa7fc700 1 -- 172.21.2.104:0/2425432233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcefc0a9d00 con 0x7fcefc0aa710 2021-07-11T04:00:04.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcefa7fc700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcefc0aa710 0x7fcefc159b30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7fcee4008000 tx=0x7fcee4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:04.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcef3fff700 1 -- 172.21.2.104:0/2425432233 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcee4014400 con 0x7fcefc0aa710 2021-07-11T04:00:04.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcef3fff700 1 -- 172.21.2.104:0/2425432233 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee4000e90 con 0x7fcefc0aa710 2021-07-11T04:00:04.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcf0166b700 1 -- 172.21.2.104:0/2425432233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc15a4b0 con 0x7fcefc0aa710 2021-07-11T04:00:04.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcf0166b700 1 -- 172.21.2.104:0/2425432233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc0ac520 con 0x7fcefc0aa710 2021-07-11T04:00:04.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcef3fff700 1 -- 172.21.2.104:0/2425432233 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcee4014750 con 0x7fcefc0aa710 2021-07-11T04:00:04.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcf0166b700 1 -- 172.21.2.104:0/2425432233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcefc0a9e90 con 0x7fcefc0aa710 2021-07-11T04:00:04.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcef3fff700 1 -- 172.21.2.104:0/2425432233 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcee4012070 con 0x7fcefc0aa710 2021-07-11T04:00:04.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcef3fff700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcee0053160 0x7fcee0055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:04.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcef3fff700 1 -- 172.21.2.104:0/2425432233 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6112+0+0 (secure 0 0 0) 0x7fcee406a950 con 0x7fcefc0aa710 2021-07-11T04:00:04.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcefaffd700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcee0053160 0x7fcee0055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:04.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.789+0000 7fcefaffd700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcee0053160 0x7fcee0055620 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fceec009000 tx=0x7fceec009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:04.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.793+0000 7fcef3fff700 1 -- 172.21.2.104:0/2425432233 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcee4041b00 con 0x7fcefc0aa710 2021-07-11T04:00:04.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:04.897+0000 7fcf0166b700 1 -- 172.21.2.104:0/2425432233 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcefc057350 con 0x7fcefc0aa710 2021-07-11T04:00:05.314 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T04:00:05.313+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T04:00:05.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.321+0000 7fcef3fff700 1 -- 172.21.2.104:0/2425432233 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcee4045120 con 0x7fcefc0aa710 2021-07-11T04:00:05.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 -- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcee0053160 msgr2=0x7fcee0055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:05.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcee0053160 0x7fcee0055620 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7fceec009000 tx=0x7fceec009040).stop 2021-07-11T04:00:05.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 -- 172.21.2.104:0/2425432233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcefc0aa710 msgr2=0x7fcefc159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:05.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcefc0aa710 0x7fcefc159b30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7fcee4008000 tx=0x7fcee4008040).stop 2021-07-11T04:00:05.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 -- 172.21.2.104:0/2425432233 shutdown_connections 2021-07-11T04:00:05.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcefc15a020 0x7fcefc0ab190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:05.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcee0053160 0x7fcee0055620 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:05.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcefc0aa710 0x7fcefc159b30 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:05.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 --2- 172.21.2.104:0/2425432233 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcefc0adf60 0x7fcefc0ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:05.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 -- 172.21.2.104:0/2425432233 >> 172.21.2.104:0/2425432233 conn(0x7fcefc0b53e0 msgr2=0x7fcefc149c50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:05.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 -- 172.21.2.104:0/2425432233 shutdown_connections 2021-07-11T04:00:05.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.325+0000 7fcf0166b700 1 -- 172.21.2.104:0/2425432233 wait complete. 2021-07-11T04:00:05.332 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T04:00:05.336 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:05.344 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T04:00:05.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 Processor -- start 2021-07-11T04:00:05.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 -- start start 2021-07-11T04:00:05.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0a700aa860 0x7f0a700b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0a700a9da0 0x7f0a700aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a700ac370 con 0x7f0a700acf30 2021-07-11T04:00:05.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a700abb50 con 0x7f0a700aa860 2021-07-11T04:00:05.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a700aac40 con 0x7f0a700a9da0 2021-07-11T04:00:05.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a6effd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0a700aa860 0x7f0a700b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:05.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a6effd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0a700aa860 0x7f0a700b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46806/0 (socket says 172.21.2.104:46806) 2021-07-11T04:00:05.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a6effd700 1 -- 172.21.2.104:0/2755887079 learned_addr learned my addr 172.21.2.104:0/2755887079 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:05.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a6effd700 1 -- 172.21.2.104:0/2755887079 >> v2:172.21.2.105:3300/0 conn(0x7f0a700a9da0 msgr2=0x7f0a700aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:05.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a6effd700 1 --2- 172.21.2.104:0/2755887079 >> v2:172.21.2.105:3300/0 conn(0x7f0a700a9da0 0x7f0a700aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:05.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a6effd700 1 -- 172.21.2.104:0/2755887079 >> v1:172.21.2.105:6789/0 conn(0x7f0a700acf30 legacy=0x7f0a701434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:05.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a6effd700 1 -- 172.21.2.104:0/2755887079 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a700b16b0 con 0x7f0a700aa860 2021-07-11T04:00:05.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a6effd700 1 --2- 172.21.2.104:0/2755887079 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a700aa860 0x7f0a700b11c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f0a700b63e0 tx=0x7f0a60006eb0).ready entity=mon.0 client_cookie=339a7ffe7a28f34c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:05.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a6dffb700 1 -- 172.21.2.104:0/2755887079 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a6000d5c0 con 0x7f0a700aa860 2021-07-11T04:00:05.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a6dffb700 1 -- 172.21.2.104:0/2755887079 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a60000e90 con 0x7f0a700aa860 2021-07-11T04:00:05.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a6dffb700 1 -- 172.21.2.104:0/2755887079 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a6000d910 con 0x7f0a700aa860 2021-07-11T04:00:05.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 -- 172.21.2.104:0/2755887079 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a700aa860 msgr2=0x7f0a700b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:05.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 --2- 172.21.2.104:0/2755887079 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a700aa860 0x7f0a700b11c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f0a700b63e0 tx=0x7f0a60006eb0).stop 2021-07-11T04:00:05.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 -- 172.21.2.104:0/2755887079 shutdown_connections 2021-07-11T04:00:05.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 --2- 172.21.2.104:0/2755887079 >> v2:172.21.2.105:3300/0 conn(0x7f0a700a9da0 0x7f0a700aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:05.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 --2- 172.21.2.104:0/2755887079 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a700aa860 0x7f0a700b11c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:05.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 -- 172.21.2.104:0/2755887079 >> 172.21.2.104:0/2755887079 conn(0x7f0a700b64d0 msgr2=0x7f0a700b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:05.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 -- 172.21.2.104:0/2755887079 shutdown_connections 2021-07-11T04:00:05.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 -- 172.21.2.104:0/2755887079 wait complete. 2021-07-11T04:00:05.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 Processor -- start 2021-07-11T04:00:05.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.385+0000 7f0a75307700 1 -- start start 2021-07-11T04:00:05.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a75307700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a700a9da0 0x7f0a700a9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a75307700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a700aa860 0x7f0a7005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a75307700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a700acf30 0x7f0a7015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a75307700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a700a8950 con 0x7f0a700acf30 2021-07-11T04:00:05.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a75307700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a700a8bf0 con 0x7f0a700a9da0 2021-07-11T04:00:05.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a75307700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a700a8e90 con 0x7f0a700aa860 2021-07-11T04:00:05.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a700acf30 0x7f0a7015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:05.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a700acf30 0x7f0a7015a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46814/0 (socket says 172.21.2.104:46814) 2021-07-11T04:00:05.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6f7fe700 1 -- 172.21.2.104:0/1465549096 learned_addr learned my addr 172.21.2.104:0/1465549096 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:05.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6effd700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a700a9da0 0x7f0a700a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:05.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6f7fe700 1 -- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a700aa860 msgr2=0x7f0a7005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:05.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6f7fe700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a700aa860 0x7f0a7005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:05.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6f7fe700 1 -- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a700a9da0 msgr2=0x7f0a700a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:05.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6f7fe700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a700a9da0 0x7f0a700a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:05.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6f7fe700 1 -- 172.21.2.104:0/1465549096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6000b1e0 con 0x7f0a700acf30 2021-07-11T04:00:05.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6effd700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a700a9da0 0x7f0a700a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:05.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6f7fe700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a700acf30 0x7f0a7015a550 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f0a68003290 tx=0x7f0a680090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:05.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a5ffff700 1 -- 172.21.2.104:0/1465549096 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a6800f510 con 0x7f0a700acf30 2021-07-11T04:00:05.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a5ffff700 1 -- 172.21.2.104:0/1465549096 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6800f670 con 0x7f0a700acf30 2021-07-11T04:00:05.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a75307700 1 -- 172.21.2.104:0/1465549096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a700a9130 con 0x7f0a700acf30 2021-07-11T04:00:05.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a5ffff700 1 -- 172.21.2.104:0/1465549096 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a6800f7e0 con 0x7f0a700acf30 2021-07-11T04:00:05.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a75307700 1 -- 172.21.2.104:0/1465549096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a70066470 con 0x7f0a700acf30 2021-07-11T04:00:05.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a75307700 1 -- 172.21.2.104:0/1465549096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a70057350 con 0x7f0a700acf30 2021-07-11T04:00:05.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a5ffff700 1 -- 172.21.2.104:0/1465549096 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0a6800f070 con 0x7f0a700acf30 2021-07-11T04:00:05.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a5ffff700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a4c053210 0x7f0a4c0556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a5ffff700 1 -- 172.21.2.104:0/1465549096 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6198+0+0 (secure 0 0 0) 0x7f0a6806b1b0 con 0x7f0a700acf30 2021-07-11T04:00:05.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6effd700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a4c053210 0x7f0a4c0556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:05.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.389+0000 7f0a6effd700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a4c053210 0x7f0a4c0556d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f0a60016000 tx=0x7f0a60015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:05.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.393+0000 7f0a5ffff700 1 -- 172.21.2.104:0/1465549096 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0a68042360 con 0x7f0a700acf30 2021-07-11T04:00:05.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 -- 172.21.2.104:0/1465549096 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a700acd10 con 0x7f0a700acf30 2021-07-11T04:00:05.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a5ffff700 1 -- 172.21.2.104:0/1465549096 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+2129 (secure 0 0 0) 0x7f0a68045980 con 0x7f0a700acf30 2021-07-11T04:00:05.507 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:05.507 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":33,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/140397159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":140397159},{"type":"v1","addr":"172.21.2.105:6835","nonce":140397159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/2616903361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2616903361},{"type":"v1","addr":"172.21.2.104:6834","nonce":2616903361}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T03:59:13.620413+0000","modified":"2021-07-11T04:00:05.315055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":78,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2021-07-11T04:00:05.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 -- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a4c053210 msgr2=0x7f0a4c0556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:05.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a4c053210 0x7f0a4c0556d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f0a60016000 tx=0x7f0a60015000).stop 2021-07-11T04:00:05.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 -- 172.21.2.104:0/1465549096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a700acf30 msgr2=0x7f0a7015a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:05.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a700acf30 0x7f0a7015a550 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f0a68003290 tx=0x7f0a680090f0).stop 2021-07-11T04:00:05.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 -- 172.21.2.104:0/1465549096 shutdown_connections 2021-07-11T04:00:05.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a4c053210 0x7f0a4c0556d0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:05.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a700acf30 0x7f0a7015a550 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:05.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a700aa860 0x7f0a7005f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:05.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 --2- 172.21.2.104:0/1465549096 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a700a9da0 0x7f0a700a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:05.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 -- 172.21.2.104:0/1465549096 >> 172.21.2.104:0/1465549096 conn(0x7f0a700b64d0 msgr2=0x7f0a70140920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:05.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 -- 172.21.2.104:0/1465549096 shutdown_connections 2021-07-11T04:00:05.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.501+0000 7f0a75307700 1 -- 172.21.2.104:0/1465549096 wait complete. 2021-07-11T04:00:05.511 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 33 2021-07-11T04:00:05.514 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T04:00:05.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 Processor -- start 2021-07-11T04:00:05.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 -- start start 2021-07-11T04:00:05.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa9d40aff10 0x7fa9d405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa9d40ac9b0 0x7fa9d414af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d4061360 con 0x7fa9d414ab80 2021-07-11T04:00:05.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d414b5e0 con 0x7fa9d40ac9b0 2021-07-11T04:00:05.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d414b880 con 0x7fa9d40aff10 2021-07-11T04:00:05.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9da126700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fa9d414ab80 0x7fa9d40af1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43588/0 (socket says 172.21.2.104:43588) 2021-07-11T04:00:05.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9da126700 1 -- 172.21.2.104:0/2073544965 learned_addr learned my addr 172.21.2.104:0/2073544965 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:05.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9da927700 1 --2- 172.21.2.104:0/2073544965 >> v2:172.21.2.104:3300/0 conn(0x7fa9d40ac9b0 0x7fa9d414af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:05.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9da927700 1 -- 172.21.2.104:0/2073544965 >> v2:172.21.2.105:3300/0 conn(0x7fa9d40aff10 msgr2=0x7fa9d405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:05.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9da927700 1 --2- 172.21.2.104:0/2073544965 >> v2:172.21.2.105:3300/0 conn(0x7fa9d40aff10 0x7fa9d405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:05.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9da927700 1 -- 172.21.2.104:0/2073544965 >> v1:172.21.2.104:6789/0 conn(0x7fa9d414ab80 legacy=0x7fa9d40af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:05.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9da927700 1 -- 172.21.2.104:0/2073544965 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d40aed20 con 0x7fa9d40ac9b0 2021-07-11T04:00:05.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9da927700 1 --2- 172.21.2.104:0/2073544965 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9d40ac9b0 0x7fa9d414af60 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7fa9cc0032f0 tx=0x7fa9cc0091d0).ready entity=mon.0 client_cookie=36ed0da4ae821c56 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:05.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9d9124700 1 -- 172.21.2.104:0/2073544965 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9cc009ea0 con 0x7fa9d40ac9b0 2021-07-11T04:00:05.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9d9124700 1 -- 172.21.2.104:0/2073544965 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc00b870 con 0x7fa9d40ac9b0 2021-07-11T04:00:05.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9d9124700 1 -- 172.21.2.104:0/2073544965 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9cc00b9e0 con 0x7fa9d40ac9b0 2021-07-11T04:00:05.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 -- 172.21.2.104:0/2073544965 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9d40ac9b0 msgr2=0x7fa9d414af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:05.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 --2- 172.21.2.104:0/2073544965 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9d40ac9b0 0x7fa9d414af60 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7fa9cc0032f0 tx=0x7fa9cc0091d0).stop 2021-07-11T04:00:05.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 -- 172.21.2.104:0/2073544965 shutdown_connections 2021-07-11T04:00:05.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 --2- 172.21.2.104:0/2073544965 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9d40ac9b0 0x7fa9d414af60 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:05.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 --2- 172.21.2.104:0/2073544965 >> v2:172.21.2.105:3300/0 conn(0x7fa9d40aff10 0x7fa9d405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:05.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 -- 172.21.2.104:0/2073544965 >> 172.21.2.104:0/2073544965 conn(0x7fa9d40b5570 msgr2=0x7fa9d40b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:05.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 -- 172.21.2.104:0/2073544965 shutdown_connections 2021-07-11T04:00:05.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 -- 172.21.2.104:0/2073544965 wait complete. 2021-07-11T04:00:05.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.605+0000 7fa9dc38a700 1 Processor -- start 2021-07-11T04:00:05.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9dc38a700 1 -- start start 2021-07-11T04:00:05.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9dc38a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9d40ac9b0 0x7fa9d4156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9dc38a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9d40aff10 0x7fa9d4142930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9dc38a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9d414ab80 0x7fa9d41449a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9dc38a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d4157460 con 0x7fa9d414ab80 2021-07-11T04:00:05.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9dc38a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d41480b0 con 0x7fa9d40aff10 2021-07-11T04:00:05.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9dc38a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d4142e20 con 0x7fa9d40ac9b0 2021-07-11T04:00:05.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9da927700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9d414ab80 0x7fa9d41449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:05.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9da927700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9d414ab80 0x7fa9d41449a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46830/0 (socket says 172.21.2.104:46830) 2021-07-11T04:00:05.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9da927700 1 -- 172.21.2.104:0/1261493414 learned_addr learned my addr 172.21.2.104:0/1261493414 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:05.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9d9925700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9d40aff10 0x7fa9d4142930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:05.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9da126700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9d40ac9b0 0x7fa9d4156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:05.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9da927700 1 -- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9d40ac9b0 msgr2=0x7fa9d4156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:05.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9da927700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9d40ac9b0 0x7fa9d4156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:05.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9da927700 1 -- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9d40aff10 msgr2=0x7fa9d4142930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:05.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9da927700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9d40aff10 0x7fa9d4142930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:05.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9da927700 1 -- 172.21.2.104:0/1261493414 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc009470 con 0x7fa9d414ab80 2021-07-11T04:00:05.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9da927700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9d414ab80 0x7fa9d41449a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7fa9cc0032f0 tx=0x7fa9cc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:05.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9c77fe700 1 -- 172.21.2.104:0/1261493414 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9cc021480 con 0x7fa9d414ab80 2021-07-11T04:00:05.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9c77fe700 1 -- 172.21.2.104:0/1261493414 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc0215e0 con 0x7fa9d414ab80 2021-07-11T04:00:05.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9dc38a700 1 -- 172.21.2.104:0/1261493414 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d4147e10 con 0x7fa9d414ab80 2021-07-11T04:00:05.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9c77fe700 1 -- 172.21.2.104:0/1261493414 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9cc021940 con 0x7fa9d414ab80 2021-07-11T04:00:05.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9dc38a700 1 -- 172.21.2.104:0/1261493414 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d4144e90 con 0x7fa9d414ab80 2021-07-11T04:00:05.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9dc38a700 1 -- 172.21.2.104:0/1261493414 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d4057350 con 0x7fa9d414ab80 2021-07-11T04:00:05.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9c77fe700 1 -- 172.21.2.104:0/1261493414 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa9cc02a070 con 0x7fa9d414ab80 2021-07-11T04:00:05.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9c77fe700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9b4053210 0x7fa9b40556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:05.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9c77fe700 1 -- 172.21.2.104:0/1261493414 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6198+0+0 (secure 0 0 0) 0x7fa9cc06c780 con 0x7fa9d414ab80 2021-07-11T04:00:05.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.609+0000 7fa9da126700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9b4053210 0x7fa9b40556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:05.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.613+0000 7fa9da126700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9b4053210 0x7fa9b40556d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fa9d000b000 tx=0x7fa9d0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:05.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.613+0000 7fa9c77fe700 1 -- 172.21.2.104:0/1261493414 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa9cc043a40 con 0x7fa9d414ab80 2021-07-11T04:00:05.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:05.717+0000 7fa9dc38a700 1 -- 172.21.2.104:0/1261493414 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa9d40675c0 con 0x7fa9d414ab80 2021-07-11T04:00:06.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9c77fe700 1 -- 172.21.2.104:0/1261493414 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa9cc0150a0 con 0x7fa9d414ab80 2021-07-11T04:00:06.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 -- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9b4053210 msgr2=0x7fa9b40556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:06.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9b4053210 0x7fa9b40556d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fa9d000b000 tx=0x7fa9d0009040).stop 2021-07-11T04:00:06.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 -- 172.21.2.104:0/1261493414 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9d414ab80 msgr2=0x7fa9d41449a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:06.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9d414ab80 0x7fa9d41449a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7fa9cc0032f0 tx=0x7fa9cc016000).stop 2021-07-11T04:00:06.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 -- 172.21.2.104:0/1261493414 shutdown_connections 2021-07-11T04:00:06.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9b4053210 0x7fa9b40556d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:06.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9d414ab80 0x7fa9d41449a0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:06.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9d40aff10 0x7fa9d4142930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:06.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 --2- 172.21.2.104:0/1261493414 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9d40ac9b0 0x7fa9d4156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:06.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 -- 172.21.2.104:0/1261493414 >> 172.21.2.104:0/1261493414 conn(0x7fa9d40b5570 msgr2=0x7fa9d40acfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:06.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 -- 172.21.2.104:0/1261493414 shutdown_connections 2021-07-11T04:00:06.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.329+0000 7fa9dc38a700 1 -- 172.21.2.104:0/1261493414 wait complete. 2021-07-11T04:00:06.342 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T04:00:06.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.389+0000 7fb0a9291700 1 Processor -- start 2021-07-11T04:00:06.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.389+0000 7fb0a9291700 1 -- start start 2021-07-11T04:00:06.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.389+0000 7fb0a9291700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0a41483a0 0x7fb0a40b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:06.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.389+0000 7fb0a9291700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb0a40a70c0 0x7fb0a4144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:06.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.389+0000 7fb0a9291700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a40b0350 con 0x7fb0a40b1db0 2021-07-11T04:00:06.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.389+0000 7fb0a9291700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a40a90c0 con 0x7fb0a40a70c0 2021-07-11T04:00:06.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.389+0000 7fb0a9291700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a40b05f0 con 0x7fb0a41483a0 2021-07-11T04:00:06.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a37fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb0a40a70c0 0x7fb0a4144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:06.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a37fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb0a40a70c0 0x7fb0a4144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46838/0 (socket says 172.21.2.104:46838) 2021-07-11T04:00:06.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a37fe700 1 -- 172.21.2.104:0/1953662784 learned_addr learned my addr 172.21.2.104:0/1953662784 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:06.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a37fe700 1 -- 172.21.2.104:0/1953662784 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0a41483a0 msgr2=0x7fb0a40b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:06.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a37fe700 1 --2- 172.21.2.104:0/1953662784 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0a41483a0 0x7fb0a40b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:06.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a37fe700 1 -- 172.21.2.104:0/1953662784 >> v1:172.21.2.105:6789/0 conn(0x7fb0a40b1db0 legacy=0x7fb0a40b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:06.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a37fe700 1 -- 172.21.2.104:0/1953662784 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a40b0890 con 0x7fb0a40a70c0 2021-07-11T04:00:06.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a37fe700 1 --2- 172.21.2.104:0/1953662784 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0a40a70c0 0x7fb0a4144720 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7fb0a40613a0 tx=0x7fb09c009290).ready entity=mon.0 client_cookie=92dd3ee0ab237095 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:06.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a1ffb700 1 -- 172.21.2.104:0/1953662784 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb09c00b860 con 0x7fb0a40a70c0 2021-07-11T04:00:06.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a1ffb700 1 -- 172.21.2.104:0/1953662784 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c00b9c0 con 0x7fb0a40a70c0 2021-07-11T04:00:06.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a1ffb700 1 -- 172.21.2.104:0/1953662784 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb09c00bb70 con 0x7fb0a40a70c0 2021-07-11T04:00:06.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 -- 172.21.2.104:0/1953662784 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0a40a70c0 msgr2=0x7fb0a4144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:06.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 --2- 172.21.2.104:0/1953662784 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0a40a70c0 0x7fb0a4144720 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7fb0a40613a0 tx=0x7fb09c009290).stop 2021-07-11T04:00:06.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 -- 172.21.2.104:0/1953662784 shutdown_connections 2021-07-11T04:00:06.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 --2- 172.21.2.104:0/1953662784 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0a40a70c0 0x7fb0a4144720 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:06.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 --2- 172.21.2.104:0/1953662784 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0a41483a0 0x7fb0a40b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:06.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 -- 172.21.2.104:0/1953662784 >> 172.21.2.104:0/1953662784 conn(0x7fb0a405f6b0 msgr2=0x7fb0a405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:06.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 -- 172.21.2.104:0/1953662784 shutdown_connections 2021-07-11T04:00:06.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 -- 172.21.2.104:0/1953662784 wait complete. 2021-07-11T04:00:06.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 Processor -- start 2021-07-11T04:00:06.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 -- start start 2021-07-11T04:00:06.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0a40a70c0 0x7fb0a40abba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:06.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0a40b1db0 0x7fb0a414af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:06.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0a41483a0 0x7fb0a4064040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:06.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a4064530 con 0x7fb0a40a70c0 2021-07-11T04:00:06.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a40647c0 con 0x7fb0a41483a0 2021-07-11T04:00:06.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a4064a60 con 0x7fb0a40b1db0 2021-07-11T04:00:06.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0a40a70c0 0x7fb0a40abba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:06.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0a40a70c0 0x7fb0a40abba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46842/0 (socket says 172.21.2.104:46842) 2021-07-11T04:00:06.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a2ffd700 1 -- 172.21.2.104:0/210345261 learned_addr learned my addr 172.21.2.104:0/210345261 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:06.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a37fe700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0a41483a0 0x7fb0a4064040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:06.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a27fc700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0a40b1db0 0x7fb0a414af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:06.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a2ffd700 1 -- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0a40b1db0 msgr2=0x7fb0a414af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:06.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a2ffd700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0a40b1db0 0x7fb0a414af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:06.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a2ffd700 1 -- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0a41483a0 msgr2=0x7fb0a4064040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:06.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a2ffd700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0a41483a0 0x7fb0a4064040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:06.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a2ffd700 1 -- 172.21.2.104:0/210345261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c009530 con 0x7fb0a40a70c0 2021-07-11T04:00:06.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a27fc700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0a40b1db0 0x7fb0a414af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:00:06.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a2ffd700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0a40a70c0 0x7fb0a40abba0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7fb098000f70 tx=0x7fb098007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:06.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb093fff700 1 -- 172.21.2.104:0/210345261 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb098013450 con 0x7fb0a40a70c0 2021-07-11T04:00:06.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb093fff700 1 -- 172.21.2.104:0/210345261 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0980135b0 con 0x7fb0a40a70c0 2021-07-11T04:00:06.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 -- 172.21.2.104:0/210345261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a4159dc0 con 0x7fb0a40a70c0 2021-07-11T04:00:06.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb093fff700 1 -- 172.21.2.104:0/210345261 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb098013910 con 0x7fb0a40a70c0 2021-07-11T04:00:06.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.393+0000 7fb0a9291700 1 -- 172.21.2.104:0/210345261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a4062960 con 0x7fb0a40a70c0 2021-07-11T04:00:06.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.397+0000 7fb0a9291700 1 -- 172.21.2.104:0/210345261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a4057350 con 0x7fb0a40a70c0 2021-07-11T04:00:06.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.397+0000 7fb093fff700 1 -- 172.21.2.104:0/210345261 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb098010070 con 0x7fb0a40a70c0 2021-07-11T04:00:06.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.397+0000 7fb093fff700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb080053270 0x7fb080055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:06.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.397+0000 7fb093fff700 1 -- 172.21.2.104:0/210345261 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6198+0+0 (secure 0 0 0) 0x7fb09806ae10 con 0x7fb0a40a70c0 2021-07-11T04:00:06.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.397+0000 7fb0a27fc700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb080053270 0x7fb080055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:06.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.397+0000 7fb0a27fc700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb080053270 0x7fb080055730 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fb09400a000 tx=0x7fb094008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:06.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.397+0000 7fb093fff700 1 -- 172.21.2.104:0/210345261 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb098041fc0 con 0x7fb0a40a70c0 2021-07-11T04:00:06.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:06.481+0000 7fb0a9291700 1 -- 172.21.2.104:0/210345261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb0a4044650 con 0x7fb0a40a70c0 2021-07-11T04:00:07.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.337+0000 7fb093fff700 1 -- 172.21.2.104:0/210345261 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) v1 ==== 185+0+0 (secure 0 0 0) 0x7fb0980455e0 con 0x7fb0a40a70c0 2021-07-11T04:00:07.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 -- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb080053270 msgr2=0x7fb080055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:07.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb080053270 0x7fb080055730 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7fb09400a000 tx=0x7fb094008040).stop 2021-07-11T04:00:07.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 -- 172.21.2.104:0/210345261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0a40a70c0 msgr2=0x7fb0a40abba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:07.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0a40a70c0 0x7fb0a40abba0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7fb098000f70 tx=0x7fb098007c30).stop 2021-07-11T04:00:07.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 -- 172.21.2.104:0/210345261 shutdown_connections 2021-07-11T04:00:07.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb0a41483a0 0x7fb0a4064040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:07.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0a40b1db0 0x7fb0a414af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:07.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb080053270 0x7fb080055730 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:07.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 --2- 172.21.2.104:0/210345261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0a40a70c0 0x7fb0a40abba0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:07.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 -- 172.21.2.104:0/210345261 >> 172.21.2.104:0/210345261 conn(0x7fb0a405f6b0 msgr2=0x7fb0a40aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:07.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 -- 172.21.2.104:0/210345261 shutdown_connections 2021-07-11T04:00:07.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.341+0000 7fb0a9291700 1 -- 172.21.2.104:0/210345261 wait complete. 2021-07-11T04:00:07.348 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T04:00:07.360 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T04:00:07.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 Processor -- start 2021-07-11T04:00:07.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 -- start start 2021-07-11T04:00:07.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f27640b6900 0x7f27640acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:07.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27640b6ce0 con 0x7f27640aa5d0 2021-07-11T04:00:07.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27640611e0 con 0x7f2764060910 2021-07-11T04:00:07.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2764061480 con 0x7f27640b6900 2021-07-11T04:00:07.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f2769f9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f27640aa5d0 0x7f276413c470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43618/0 (socket says 172.21.2.104:43618) 2021-07-11T04:00:07.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f2769f9d700 1 -- 172.21.2.104:0/4287326960 learned_addr learned my addr 172.21.2.104:0/4287326960 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:07.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276979c700 1 --2- 172.21.2.104:0/4287326960 >> v2:172.21.2.104:3300/0 conn(0x7f27640b6900 0x7f27640acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:07.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276979c700 1 -- 172.21.2.104:0/4287326960 >> v1:172.21.2.105:6789/0 conn(0x7f2764060910 legacy=0x7f27640a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:07.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276979c700 1 -- 172.21.2.104:0/4287326960 >> v1:172.21.2.104:6789/0 conn(0x7f27640aa5d0 legacy=0x7f276413c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:07.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f2753fff700 1 -- 172.21.2.104:0/4287326960 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1361816210 0 0) 0x7f27640b6ce0 con 0x7f27640aa5d0 2021-07-11T04:00:07.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276979c700 1 -- 172.21.2.104:0/4287326960 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2764061720 con 0x7f27640b6900 2021-07-11T04:00:07.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276979c700 1 --2- 172.21.2.104:0/4287326960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27640b6900 0x7f27640acd50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f276405dc30 tx=0x7f2760007f00).ready entity=mon.0 client_cookie=c1688def68ddaaf6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:07.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f2753fff700 1 -- 172.21.2.104:0/4287326960 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27600075d0 con 0x7f27640b6900 2021-07-11T04:00:07.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f2753fff700 1 -- 172.21.2.104:0/4287326960 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2760000e90 con 0x7f27640b6900 2021-07-11T04:00:07.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f2753fff700 1 -- 172.21.2.104:0/4287326960 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27600058e0 con 0x7f27640b6900 2021-07-11T04:00:07.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 -- 172.21.2.104:0/4287326960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27640b6900 msgr2=0x7f27640acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:07.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 --2- 172.21.2.104:0/4287326960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27640b6900 0x7f27640acd50 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f276405dc30 tx=0x7f2760007f00).stop 2021-07-11T04:00:07.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 -- 172.21.2.104:0/4287326960 shutdown_connections 2021-07-11T04:00:07.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 --2- 172.21.2.104:0/4287326960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27640b6900 0x7f27640acd50 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:07.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 -- 172.21.2.104:0/4287326960 >> 172.21.2.104:0/4287326960 conn(0x7f27640b3480 msgr2=0x7f27640b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:07.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 -- 172.21.2.104:0/4287326960 shutdown_connections 2021-07-11T04:00:07.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 -- 172.21.2.104:0/4287326960 wait complete. 2021-07-11T04:00:07.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 Processor -- start 2021-07-11T04:00:07.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 -- start start 2021-07-11T04:00:07.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2764060910 0x7f276413cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:07.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27640aa5d0 0x7f27640b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:07.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.413+0000 7f276ba00700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27640b6900 0x7f27640b19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:07.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276ba00700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276413eb80 con 0x7f2764060910 2021-07-11T04:00:07.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276ba00700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276413e060 con 0x7f27640b6900 2021-07-11T04:00:07.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276ba00700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276413edb0 con 0x7f27640aa5d0 2021-07-11T04:00:07.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276979c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2764060910 0x7f276413cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:07.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276979c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2764060910 0x7f276413cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46856/0 (socket says 172.21.2.104:46856) 2021-07-11T04:00:07.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276979c700 1 -- 172.21.2.104:0/4029897330 learned_addr learned my addr 172.21.2.104:0/4029897330 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:07.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276979c700 1 -- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27640aa5d0 msgr2=0x7f27640b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:07.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276979c700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27640aa5d0 0x7f27640b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:07.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276979c700 1 -- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27640b6900 msgr2=0x7f27640b19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:07.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276979c700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27640b6900 0x7f27640b19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:07.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276979c700 1 -- 172.21.2.104:0/4029897330 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276000c200 con 0x7f2764060910 2021-07-11T04:00:07.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276979c700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2764060910 0x7f276413cd80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f276405dc30 tx=0x7f2760015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:07.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f27527fc700 1 -- 172.21.2.104:0/4029897330 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2760010710 con 0x7f2764060910 2021-07-11T04:00:07.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f27527fc700 1 -- 172.21.2.104:0/4029897330 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2760010870 con 0x7f2764060910 2021-07-11T04:00:07.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276ba00700 1 -- 172.21.2.104:0/4029897330 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276413e290 con 0x7f2764060910 2021-07-11T04:00:07.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f27527fc700 1 -- 172.21.2.104:0/4029897330 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2760010bd0 con 0x7f2764060910 2021-07-11T04:00:07.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276ba00700 1 -- 172.21.2.104:0/4029897330 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276413da60 con 0x7f2764060910 2021-07-11T04:00:07.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f276ba00700 1 -- 172.21.2.104:0/4029897330 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2764057350 con 0x7f2764060910 2021-07-11T04:00:07.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f27527fc700 1 -- 172.21.2.104:0/4029897330 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f276002a070 con 0x7f2764060910 2021-07-11T04:00:07.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f27527fc700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2744053270 0x7f2744055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:07.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f27527fc700 1 -- 172.21.2.104:0/4029897330 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5760+0+0 (secure 0 0 0) 0x7f276006c4f0 con 0x7f2764060910 2021-07-11T04:00:07.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f2768f9b700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2744053270 0x7f2744055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:07.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.417+0000 7f2768f9b700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2744053270 0x7f2744055730 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f2754002f90 tx=0x7f27540069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:07.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.421+0000 7f27527fc700 1 -- 172.21.2.104:0/4029897330 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2760043970 con 0x7f2764060910 2021-07-11T04:00:07.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:07.501+0000 7f276ba00700 1 -- 172.21.2.104:0/4029897330 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f27640b63a0 con 0x7f2764060910 2021-07-11T04:00:08.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.341+0000 7f27527fc700 1 -- 172.21.2.104:0/4029897330 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2760046f90 con 0x7f2764060910 2021-07-11T04:00:08.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.341+0000 7f276ba00700 1 -- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2744053270 msgr2=0x7f2744055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:08.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.341+0000 7f276ba00700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2744053270 0x7f2744055730 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f2754002f90 tx=0x7f27540069b0).stop 2021-07-11T04:00:08.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.345+0000 7f276ba00700 1 -- 172.21.2.104:0/4029897330 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2764060910 msgr2=0x7f276413cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:08.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.345+0000 7f276ba00700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2764060910 0x7f276413cd80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f276405dc30 tx=0x7f2760015000).stop 2021-07-11T04:00:08.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.345+0000 7f276ba00700 1 -- 172.21.2.104:0/4029897330 shutdown_connections 2021-07-11T04:00:08.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.345+0000 7f276ba00700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27640b6900 0x7f27640b19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:08.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.345+0000 7f276ba00700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27640aa5d0 0x7f27640b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:08.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.345+0000 7f276ba00700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2744053270 0x7f2744055730 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.345+0000 7f276ba00700 1 --2- 172.21.2.104:0/4029897330 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2764060910 0x7f276413cd80 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.345+0000 7f276ba00700 1 -- 172.21.2.104:0/4029897330 >> 172.21.2.104:0/4029897330 conn(0x7f27640b3480 msgr2=0x7f27640b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:08.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.345+0000 7f276ba00700 1 -- 172.21.2.104:0/4029897330 shutdown_connections 2021-07-11T04:00:08.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.345+0000 7f276ba00700 1 -- 172.21.2.104:0/4029897330 wait complete. 2021-07-11T04:00:08.352 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T04:00:08.357 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-07-11T04:00:08.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 Processor -- start 2021-07-11T04:00:08.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 -- start start 2021-07-11T04:00:08.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3b7c0ad040 0x7f3b7c0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3b7c13c9c0 0x7f3b7c13f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7c0b5f40 con 0x7f3b7c0afaf0 2021-07-11T04:00:08.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7c0ad420 con 0x7f3b7c0ad040 2021-07-11T04:00:08.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c0aeb60 con 0x7f3b7c13c9c0 2021-07-11T04:00:08.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b7b7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f3b7c0afaf0 0x7f3b7c13c2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43630/0 (socket says 172.21.2.104:43630) 2021-07-11T04:00:08.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b7b7fe700 1 -- 172.21.2.104:0/4214788240 learned_addr learned my addr 172.21.2.104:0/4214788240 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:08.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b7bfff700 1 --2- 172.21.2.104:0/4214788240 >> v2:172.21.2.104:3300/0 conn(0x7f3b7c0ad040 0x7f3b7c0b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:08.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b80b6d700 1 --2- 172.21.2.104:0/4214788240 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3b7c13c9c0 0x7f3b7c13f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:08.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b7bfff700 1 -- 172.21.2.104:0/4214788240 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3b7c13c9c0 msgr2=0x7f3b7c13f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:08.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b7bfff700 1 --2- 172.21.2.104:0/4214788240 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3b7c13c9c0 0x7f3b7c13f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b7bfff700 1 -- 172.21.2.104:0/4214788240 >> v1:172.21.2.104:6789/0 conn(0x7f3b7c0afaf0 legacy=0x7f3b7c13c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:08.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b7bfff700 1 -- 172.21.2.104:0/4214788240 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c0aee00 con 0x7f3b7c0ad040 2021-07-11T04:00:08.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b7bfff700 1 --2- 172.21.2.104:0/4214788240 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3b7c0ad040 0x7f3b7c0b5a50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f3b64002f70 tx=0x7f3b64006df0).ready entity=mon.0 client_cookie=4aefbe6457bbca04 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:08.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b7affd700 1 -- 172.21.2.104:0/4214788240 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b6400f940 con 0x7f3b7c0ad040 2021-07-11T04:00:08.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b7affd700 1 -- 172.21.2.104:0/4214788240 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b64000e90 con 0x7f3b7c0ad040 2021-07-11T04:00:08.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b7affd700 1 -- 172.21.2.104:0/4214788240 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b6400fc90 con 0x7f3b7c0ad040 2021-07-11T04:00:08.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 -- 172.21.2.104:0/4214788240 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3b7c0ad040 msgr2=0x7f3b7c0b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:08.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 --2- 172.21.2.104:0/4214788240 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3b7c0ad040 0x7f3b7c0b5a50 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f3b64002f70 tx=0x7f3b64006df0).stop 2021-07-11T04:00:08.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 -- 172.21.2.104:0/4214788240 shutdown_connections 2021-07-11T04:00:08.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 --2- 172.21.2.104:0/4214788240 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3b7c13c9c0 0x7f3b7c13f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 --2- 172.21.2.104:0/4214788240 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3b7c0ad040 0x7f3b7c0b5a50 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 -- 172.21.2.104:0/4214788240 >> 172.21.2.104:0/4214788240 conn(0x7f3b7c05c490 msgr2=0x7f3b7c05c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:08.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 -- 172.21.2.104:0/4214788240 shutdown_connections 2021-07-11T04:00:08.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.405+0000 7f3b825d0700 1 -- 172.21.2.104:0/4214788240 wait complete. 2021-07-11T04:00:08.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b825d0700 1 Processor -- start 2021-07-11T04:00:08.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b825d0700 1 -- start start 2021-07-11T04:00:08.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b825d0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3b7c0ad040 0x7f3b7c0b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b825d0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3b7c0afaf0 0x7f3b7c13efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b825d0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3b7c13c9c0 0x7f3b7c13f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b825d0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c13f990 con 0x7f3b7c0ad040 2021-07-11T04:00:08.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b825d0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c1450b0 con 0x7f3b7c0afaf0 2021-07-11T04:00:08.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b825d0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c145630 con 0x7f3b7c13c9c0 2021-07-11T04:00:08.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3b7c0ad040 0x7f3b7c0b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:08.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3b7c0ad040 0x7f3b7c0b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46868/0 (socket says 172.21.2.104:46868) 2021-07-11T04:00:08.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7bfff700 1 -- 172.21.2.104:0/2199440853 learned_addr learned my addr 172.21.2.104:0/2199440853 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:08.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b80b6d700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3b7c13c9c0 0x7f3b7c13f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:08.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7b7fe700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3b7c0afaf0 0x7f3b7c13efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:08.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7bfff700 1 -- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3b7c13c9c0 msgr2=0x7f3b7c13f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:08.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7bfff700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3b7c13c9c0 0x7f3b7c13f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7bfff700 1 -- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3b7c0afaf0 msgr2=0x7f3b7c13efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:08.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7bfff700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3b7c0afaf0 0x7f3b7c13efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7bfff700 1 -- 172.21.2.104:0/2199440853 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6400c290 con 0x7f3b7c0ad040 2021-07-11T04:00:08.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7b7fe700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3b7c0afaf0 0x7f3b7c13efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:08.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7bfff700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3b7c0ad040 0x7f3b7c0b4c40 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f3b7c13fad0 tx=0x7f3b64017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:08.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b797fa700 1 -- 172.21.2.104:0/2199440853 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b64006b20 con 0x7f3b7c0ad040 2021-07-11T04:00:08.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b797fa700 1 -- 172.21.2.104:0/2199440853 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b64017c60 con 0x7f3b7c0ad040 2021-07-11T04:00:08.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b825d0700 1 -- 172.21.2.104:0/2199440853 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c1452e0 con 0x7f3b7c0ad040 2021-07-11T04:00:08.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b797fa700 1 -- 172.21.2.104:0/2199440853 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3b64023450 con 0x7f3b7c0ad040 2021-07-11T04:00:08.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b825d0700 1 -- 172.21.2.104:0/2199440853 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c13d7a0 con 0x7f3b7c0ad040 2021-07-11T04:00:08.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b825d0700 1 -- 172.21.2.104:0/2199440853 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7c057350 con 0x7f3b7c0ad040 2021-07-11T04:00:08.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b797fa700 1 -- 172.21.2.104:0/2199440853 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3b64020070 con 0x7f3b7c0ad040 2021-07-11T04:00:08.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b797fa700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3b60053300 0x7f3b600557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b797fa700 1 -- 172.21.2.104:0/2199440853 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f3b6406e070 con 0x7f3b7c0ad040 2021-07-11T04:00:08.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7b7fe700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3b60053300 0x7f3b600557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:08.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.409+0000 7f3b7b7fe700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3b60053300 0x7f3b600557c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f3b6c008000 tx=0x7f3b6c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:08.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.413+0000 7f3b797fa700 1 -- 172.21.2.104:0/2199440853 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3b64045550 con 0x7f3b7c0ad040 2021-07-11T04:00:08.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.497+0000 7f3b825d0700 1 -- 172.21.2.104:0/2199440853 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3b7c0ac990 con 0x7f3b7c0ad040 2021-07-11T04:00:08.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.501+0000 7f3b797fa700 1 -- 172.21.2.104:0/2199440853 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3b64028020 con 0x7f3b7c0ad040 2021-07-11T04:00:08.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.501+0000 7f3b825d0700 1 -- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3b60053300 msgr2=0x7f3b600557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:08.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.501+0000 7f3b825d0700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3b60053300 0x7f3b600557c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f3b6c008000 tx=0x7f3b6c009040).stop 2021-07-11T04:00:08.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.501+0000 7f3b825d0700 1 -- 172.21.2.104:0/2199440853 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3b7c0ad040 msgr2=0x7f3b7c0b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:08.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.501+0000 7f3b825d0700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3b7c0ad040 0x7f3b7c0b4c40 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f3b7c13fad0 tx=0x7f3b64017000).stop 2021-07-11T04:00:08.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.501+0000 7f3b825d0700 1 -- 172.21.2.104:0/2199440853 shutdown_connections 2021-07-11T04:00:08.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.501+0000 7f3b825d0700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3b7c13c9c0 0x7f3b7c13f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.501+0000 7f3b825d0700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3b7c0afaf0 0x7f3b7c13efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.501+0000 7f3b825d0700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3b60053300 0x7f3b600557c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.501+0000 7f3b825d0700 1 --2- 172.21.2.104:0/2199440853 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3b7c0ad040 0x7f3b7c0b4c40 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.501+0000 7f3b825d0700 1 -- 172.21.2.104:0/2199440853 >> 172.21.2.104:0/2199440853 conn(0x7f3b7c05c490 msgr2=0x7f3b7c13a890 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:08.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.505+0000 7f3b825d0700 1 -- 172.21.2.104:0/2199440853 shutdown_connections 2021-07-11T04:00:08.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.505+0000 7f3b825d0700 1 -- 172.21.2.104:0/2199440853 wait complete. 2021-07-11T04:00:08.514 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder' 2021-07-11T04:00:08.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 Processor -- start 2021-07-11T04:00:08.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 -- start start 2021-07-11T04:00:08.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8cc40b6900 0x7f8cc40acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8cc40aa5d0 0x7f8cc413c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc40b6ce0 con 0x7f8cc4060910 2021-07-11T04:00:08.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc40611e0 con 0x7f8cc40b6900 2021-07-11T04:00:08.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc4061480 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8cca2c8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8cc40b6900 0x7f8cc40acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:08.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8cca2c8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8cc40b6900 0x7f8cc40acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46876/0 (socket says 172.21.2.104:46876) 2021-07-11T04:00:08.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8cca2c8700 1 -- 172.21.2.104:0/3878605332 learned_addr learned my addr 172.21.2.104:0/3878605332 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:08.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8cca2c8700 1 -- 172.21.2.104:0/3878605332 >> v2:172.21.2.105:3300/0 conn(0x7f8cc40aa5d0 msgr2=0x7f8cc413c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:08.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8cca2c8700 1 --2- 172.21.2.104:0/3878605332 >> v2:172.21.2.105:3300/0 conn(0x7f8cc40aa5d0 0x7f8cc413c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:08.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8cca2c8700 1 -- 172.21.2.104:0/3878605332 >> v1:172.21.2.105:6789/0 conn(0x7f8cc4060910 legacy=0x7f8cc40a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:08.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8cca2c8700 1 -- 172.21.2.104:0/3878605332 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4061720 con 0x7f8cc40b6900 2021-07-11T04:00:08.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8cca2c8700 1 --2- 172.21.2.104:0/3878605332 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cc40b6900 0x7f8cc40acd50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f8cc4060300 tx=0x7f8cb8006eb0).ready entity=mon.0 client_cookie=61c5c93c7927cd86 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:08.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8cc92c6700 1 -- 172.21.2.104:0/3878605332 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cb8010810 con 0x7f8cc40b6900 2021-07-11T04:00:08.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8cc92c6700 1 -- 172.21.2.104:0/3878605332 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb8000e90 con 0x7f8cc40b6900 2021-07-11T04:00:08.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8cc92c6700 1 -- 172.21.2.104:0/3878605332 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cb8010ba0 con 0x7f8cc40b6900 2021-07-11T04:00:08.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 -- 172.21.2.104:0/3878605332 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cc40b6900 msgr2=0x7f8cc40acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:08.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 --2- 172.21.2.104:0/3878605332 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cc40b6900 0x7f8cc40acd50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f8cc4060300 tx=0x7f8cb8006eb0).stop 2021-07-11T04:00:08.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 -- 172.21.2.104:0/3878605332 shutdown_connections 2021-07-11T04:00:08.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 --2- 172.21.2.104:0/3878605332 >> v2:172.21.2.105:3300/0 conn(0x7f8cc40aa5d0 0x7f8cc413c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:08.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 --2- 172.21.2.104:0/3878605332 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cc40b6900 0x7f8cc40acd50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:08.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 -- 172.21.2.104:0/3878605332 >> 172.21.2.104:0/3878605332 conn(0x7f8cc40b3480 msgr2=0x7f8cc40b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:08.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 -- 172.21.2.104:0/3878605332 shutdown_connections 2021-07-11T04:00:08.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.561+0000 7f8ccc52c700 1 -- 172.21.2.104:0/3878605332 wait complete. 2021-07-11T04:00:08.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8ccc52c700 1 Processor -- start 2021-07-11T04:00:08.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8ccc52c700 1 -- start start 2021-07-11T04:00:08.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8ccc52c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8cc4060910 0x7f8cc4136650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8ccc52c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cc40aa5d0 0x7f8cc4159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8ccc52c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8cc40b6900 0x7f8cc415a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8ccc52c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc41566d0 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8ccc52c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc4156970 con 0x7f8cc40b6900 2021-07-11T04:00:08.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8ccc52c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc4156c10 con 0x7f8cc4060910 2021-07-11T04:00:08.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cc9ac7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cc40aa5d0 0x7f8cc4159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:08.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cc9ac7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cc40aa5d0 0x7f8cc4159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46884/0 (socket says 172.21.2.104:46884) 2021-07-11T04:00:08.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cc9ac7700 1 -- 172.21.2.104:0/615077668 learned_addr learned my addr 172.21.2.104:0/615077668 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:08.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cc9ac7700 1 -- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8cc4060910 msgr2=0x7f8cc4136650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:08.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cc9ac7700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8cc4060910 0x7f8cc4136650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:08.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cc9ac7700 1 -- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8cc40b6900 msgr2=0x7f8cc415a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:08.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cc9ac7700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8cc40b6900 0x7f8cc415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:08.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cc9ac7700 1 -- 172.21.2.104:0/615077668 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb800c2f0 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cc9ac7700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cc40aa5d0 0x7f8cc4159c80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f8cc0002f90 tx=0x7f8cc0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:08.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cb77fe700 1 -- 172.21.2.104:0/615077668 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cc0013460 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8ccc52c700 1 -- 172.21.2.104:0/615077668 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc4156eb0 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cb77fe700 1 -- 172.21.2.104:0/615077668 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc00135c0 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8ccc52c700 1 -- 172.21.2.104:0/615077668 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc413bf90 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cb77fe700 1 -- 172.21.2.104:0/615077668 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8cc0013920 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8ccc52c700 1 -- 172.21.2.104:0/615077668 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc4157040 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cb77fe700 1 -- 172.21.2.104:0/615077668 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8cc000f070 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cb77fe700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ca4053330 0x7f8ca40557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:08.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cb77fe700 1 -- 172.21.2.104:0/615077668 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f8cc006aa50 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.565+0000 7f8cca2c8700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ca4053330 0x7f8ca40557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:08.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.569+0000 7f8cca2c8700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ca4053330 0x7f8ca40557f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8cc4060300 tx=0x7f8cb8017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:08.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.569+0000 7f8cb77fe700 1 -- 172.21.2.104:0/615077668 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8cc0041fa0 con 0x7f8cc40aa5d0 2021-07-11T04:00:08.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:08.677+0000 7f8ccc52c700 1 -- 172.21.2.104:0/615077668 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]} v 0) v1 -- 0x7f8cc4057350 con 0x7f8cc40aa5d0 2021-07-11T04:00:09.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.353+0000 7f8cb77fe700 1 -- 172.21.2.104:0/615077668 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_cap_revoke_nonresponder"]}]=0 v278) v1 ==== 139+0+0 (secure 0 0 0) 0x7f8cc00455c0 con 0x7f8cc40aa5d0 2021-07-11T04:00:09.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 -- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ca4053330 msgr2=0x7f8ca40557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:09.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ca4053330 0x7f8ca40557f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f8cc4060300 tx=0x7f8cb8017000).stop 2021-07-11T04:00:09.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 -- 172.21.2.104:0/615077668 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cc40aa5d0 msgr2=0x7f8cc4159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:09.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cc40aa5d0 0x7f8cc4159c80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f8cc0002f90 tx=0x7f8cc0006df0).stop 2021-07-11T04:00:09.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 -- 172.21.2.104:0/615077668 shutdown_connections 2021-07-11T04:00:09.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8cc40b6900 0x7f8cc415a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:09.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ca4053330 0x7f8ca40557f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:09.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cc40aa5d0 0x7f8cc4159c80 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:09.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 --2- 172.21.2.104:0/615077668 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8cc4060910 0x7f8cc4136650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:09.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 -- 172.21.2.104:0/615077668 >> 172.21.2.104:0/615077668 conn(0x7f8cc40b3480 msgr2=0x7f8cc40aca90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:09.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 -- 172.21.2.104:0/615077668 shutdown_connections 2021-07-11T04:00:09.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.357+0000 7f8ccc52c700 1 -- 172.21.2.104:0/615077668 wait complete. 2021-07-11T04:00:09.370 INFO:tasks.cephfs_test_runner:test_cap_revoke_nonresponder (tasks.cephfs.test_misc.TestMisc) ... ok 2021-07-11T04:00:09.371 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:00:09.372 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2021-07-11T04:00:09.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f114b33a700 1 Processor -- start 2021-07-11T04:00:09.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f114b33a700 1 -- start start 2021-07-11T04:00:09.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f114b33a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f11440b1170 0x7f1144141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:09.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f114b33a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1144141920 0x7f1144141d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:09.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f114b33a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11440b31d0 con 0x7f11440b7810 2021-07-11T04:00:09.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f114b33a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440b7bf0 con 0x7f11440b1170 2021-07-11T04:00:09.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f114b33a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11440675c0 con 0x7f1144141920 2021-07-11T04:00:09.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11490d6700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f11440b7810 0x7f11441445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43658/0 (socket says 172.21.2.104:43658) 2021-07-11T04:00:09.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11490d6700 1 -- 172.21.2.104:0/1961481757 learned_addr learned my addr 172.21.2.104:0/1961481757 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:09.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326630481 0 0) 0x7f11440b31d0 con 0x7f11440b7810 2021-07-11T04:00:09.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f112c003260 con 0x7f11440b7810 2021-07-11T04:00:09.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11498d7700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1144141920 0x7f1144141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:09.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11488d5700 1 --2- 172.21.2.104:0/1961481757 >> v2:172.21.2.105:3300/0 conn(0x7f11440b1170 0x7f1144141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:09.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2408989126 0 0) 0x7f112c003260 con 0x7f11440b7810 2021-07-11T04:00:09.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1144141920 msgr2=0x7f1144141d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:09.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1144141920 0x7f1144141d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:09.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 >> v2:172.21.2.105:3300/0 conn(0x7f11440b1170 msgr2=0x7f1144141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:09.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 --2- 172.21.2.104:0/1961481757 >> v2:172.21.2.105:3300/0 conn(0x7f11440b1170 0x7f1144141430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:09.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1144142270 con 0x7f11440b7810 2021-07-11T04:00:09.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f1138001660 con 0x7f11440b7810 2021-07-11T04:00:09.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 >> v1:172.21.2.104:6789/0 conn(0x7f11440b7810 legacy=0x7f11441445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:09.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f112c0037d0 0x7f112c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:09.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f112c004500 0x7f112c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:09.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1144141920 0x7f112c004ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:09.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440b7bf0 con 0x7f112c004500 2021-07-11T04:00:09.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11440675c0 con 0x7f112c0037d0 2021-07-11T04:00:09.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112c00b8d0 con 0x7f1144141920 2021-07-11T04:00:09.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11488d5700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f112c004500 0x7f112c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:09.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11498d7700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1144141920 0x7f112c004ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:09.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11488d5700 1 -- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1144141920 msgr2=0x7f112c004ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:09.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11490d6700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f112c0037d0 0x7f112c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:09.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11488d5700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1144141920 0x7f112c004ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:09.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11488d5700 1 -- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f112c0037d0 msgr2=0x7f112c003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:09.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11488d5700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f112c0037d0 0x7f112c003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:09.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11488d5700 1 -- 172.21.2.104:0/1961481757 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c003260 con 0x7f112c004500 2021-07-11T04:00:09.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11490d6700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f112c0037d0 0x7f112c003bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:09.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11488d5700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f112c004500 0x7f112c004980 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f114000a000 tx=0x7f1140008040).ready entity=mon.0 client_cookie=ff17138d8bb128d8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:09.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f11400128a0 con 0x7f112c004500 2021-07-11T04:00:09.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1140012a00 con 0x7f112c004500 2021-07-11T04:00:09.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f1137fff700 1 -- 172.21.2.104:0/1961481757 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1140012d60 con 0x7f112c004500 2021-07-11T04:00:09.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f114b33a700 1 -- 172.21.2.104:0/1961481757 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f112c004500 msgr2=0x7f112c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:09.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f114b33a700 1 --2- 172.21.2.104:0/1961481757 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f112c004500 0x7f112c004980 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f114000a000 tx=0x7f1140008040).stop 2021-07-11T04:00:09.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.421+0000 7f11498d7700 1 -- 172.21.2.104:0/1961481757 reap_dead start 2021-07-11T04:00:09.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 -- 172.21.2.104:0/1961481757 shutdown_connections 2021-07-11T04:00:09.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 -- 172.21.2.104:0/1961481757 >> 172.21.2.104:0/1961481757 conn(0x7f114405f6b0 msgr2=0x7f114405fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:09.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 -- 172.21.2.104:0/1961481757 shutdown_connections 2021-07-11T04:00:09.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 -- 172.21.2.104:0/1961481757 wait complete. 2021-07-11T04:00:09.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 Processor -- start 2021-07-11T04:00:09.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 -- start start 2021-07-11T04:00:09.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f11440ae110 0x7f11440ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:09.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1144144da0 0x7f11440ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:09.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f11440abe90 0x7f1144151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:09.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1144144920 con 0x7f1144144da0 2021-07-11T04:00:09.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1144145390 con 0x7f11440abe90 2021-07-11T04:00:09.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114414ec00 con 0x7f11440ae110 2021-07-11T04:00:09.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11488d5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1144144da0 0x7f11440ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:09.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11488d5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1144144da0 0x7f11440ab9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46904/0 (socket says 172.21.2.104:46904) 2021-07-11T04:00:09.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11488d5700 1 -- 172.21.2.104:0/857254785 learned_addr learned my addr 172.21.2.104:0/857254785 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:09.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11498d7700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f11440abe90 0x7f1144151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:09.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11490d6700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f11440ae110 0x7f11440ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:09.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11488d5700 1 -- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f11440ae110 msgr2=0x7f11440ae4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:09.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11488d5700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f11440ae110 0x7f11440ae4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:09.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11488d5700 1 -- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f11440abe90 msgr2=0x7f1144151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:09.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11488d5700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f11440abe90 0x7f1144151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:09.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11488d5700 1 -- 172.21.2.104:0/857254785 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114414eea0 con 0x7f1144144da0 2021-07-11T04:00:09.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11498d7700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f11440abe90 0x7f1144151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:09.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11490d6700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f11440ae110 0x7f11440ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:09.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11488d5700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1144144da0 0x7f11440ab9a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f114000a000 tx=0x7f114001c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:09.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11367fc700 1 -- 172.21.2.104:0/857254785 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f114001ba70 con 0x7f1144144da0 2021-07-11T04:00:09.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11367fc700 1 -- 172.21.2.104:0/857254785 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1140000e90 con 0x7f1144144da0 2021-07-11T04:00:09.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 -- 172.21.2.104:0/857254785 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11441441e0 con 0x7f1144144da0 2021-07-11T04:00:09.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11367fc700 1 -- 172.21.2.104:0/857254785 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f114001bdc0 con 0x7f1144144da0 2021-07-11T04:00:09.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 -- 172.21.2.104:0/857254785 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11440ac310 con 0x7f1144144da0 2021-07-11T04:00:09.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f114b33a700 1 -- 172.21.2.104:0/857254785 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1144057350 con 0x7f1144144da0 2021-07-11T04:00:09.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11367fc700 1 -- 172.21.2.104:0/857254785 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1140023430 con 0x7f1144144da0 2021-07-11T04:00:09.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11367fc700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f11240531f0 0x7f11240556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:09.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.425+0000 7f11367fc700 1 -- 172.21.2.104:0/857254785 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f11400702a0 con 0x7f1144144da0 2021-07-11T04:00:09.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.429+0000 7f11490d6700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f11240531f0 0x7f11240556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:09.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.429+0000 7f11490d6700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f11240531f0 0x7f11240556b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f1138009000 tx=0x7f1138009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:09.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.429+0000 7f11367fc700 1 -- 172.21.2.104:0/857254785 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1140047830 con 0x7f1144144da0 2021-07-11T04:00:09.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:09.541+0000 7f114b33a700 1 -- 172.21.2.104:0/857254785 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f11440698e0 con 0x7f1144144da0 2021-07-11T04:00:10.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.357+0000 7f11367fc700 1 -- 172.21.2.104:0/857254785 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v279) v1 ==== 129+0+0 (secure 0 0 0) 0x7f1140028020 con 0x7f1144144da0 2021-07-11T04:00:10.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 -- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f11240531f0 msgr2=0x7f11240556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f11240531f0 0x7f11240556b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f1138009000 tx=0x7f1138009040).stop 2021-07-11T04:00:10.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 -- 172.21.2.104:0/857254785 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1144144da0 msgr2=0x7f11440ab9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1144144da0 0x7f11440ab9a0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f114000a000 tx=0x7f114001c000).stop 2021-07-11T04:00:10.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 -- 172.21.2.104:0/857254785 shutdown_connections 2021-07-11T04:00:10.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f11440abe90 0x7f1144151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:10.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f11240531f0 0x7f11240556b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:10.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1144144da0 0x7f11440ab9a0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:10.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 --2- 172.21.2.104:0/857254785 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f11440ae110 0x7f11440ae4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:10.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 -- 172.21.2.104:0/857254785 >> 172.21.2.104:0/857254785 conn(0x7f114405f6b0 msgr2=0x7f114405bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:10.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 -- 172.21.2.104:0/857254785 shutdown_connections 2021-07-11T04:00:10.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.361+0000 7f114b33a700 1 -- 172.21.2.104:0/857254785 wait complete. 2021-07-11T04:00:10.373 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T04:00:10.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 Processor -- start 2021-07-11T04:00:10.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- start start 2021-07-11T04:00:10.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9b680b1d60 0x7f9b680b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9b681472c0 0x7f9b681435b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b680ae650 con 0x7f9b680b13f0 2021-07-11T04:00:10.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b680b17d0 con 0x7f9b680b1d60 2021-07-11T04:00:10.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b680b22e0 con 0x7f9b681472c0 2021-07-11T04:00:10.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6ccad700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9b680b1d60 0x7f9b680b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:10.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6ccad700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9b680b1d60 0x7f9b680b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46912/0 (socket says 172.21.2.104:46912) 2021-07-11T04:00:10.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6ccad700 1 -- 172.21.2.104:0/527687257 learned_addr learned my addr 172.21.2.104:0/527687257 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:10.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6ccad700 1 -- 172.21.2.104:0/527687257 >> v2:172.21.2.105:3300/0 conn(0x7f9b681472c0 msgr2=0x7f9b681435b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:10.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6ccad700 1 --2- 172.21.2.104:0/527687257 >> v2:172.21.2.105:3300/0 conn(0x7f9b681472c0 0x7f9b681435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6ccad700 1 -- 172.21.2.104:0/527687257 >> v1:172.21.2.104:6789/0 conn(0x7f9b680b13f0 legacy=0x7f9b68146bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6ccad700 1 -- 172.21.2.104:0/527687257 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b680b2580 con 0x7f9b680b1d60 2021-07-11T04:00:10.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6ccad700 1 --2- 172.21.2.104:0/527687257 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9b680b1d60 0x7f9b680b3080 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f9b600032f0 tx=0x7f9b60009290).ready entity=mon.0 client_cookie=91ac3a81bdbbd08c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:10.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b67fff700 1 -- 172.21.2.104:0/527687257 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b6000b870 con 0x7f9b680b1d60 2021-07-11T04:00:10.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b67fff700 1 -- 172.21.2.104:0/527687257 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6000b9d0 con 0x7f9b680b1d60 2021-07-11T04:00:10.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b67fff700 1 -- 172.21.2.104:0/527687257 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b6000bb40 con 0x7f9b680b1d60 2021-07-11T04:00:10.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- 172.21.2.104:0/527687257 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9b680b1d60 msgr2=0x7f9b680b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 --2- 172.21.2.104:0/527687257 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9b680b1d60 0x7f9b680b3080 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f9b600032f0 tx=0x7f9b60009290).stop 2021-07-11T04:00:10.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- 172.21.2.104:0/527687257 shutdown_connections 2021-07-11T04:00:10.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 --2- 172.21.2.104:0/527687257 >> v2:172.21.2.105:3300/0 conn(0x7f9b681472c0 0x7f9b681435b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 --2- 172.21.2.104:0/527687257 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9b680b1d60 0x7f9b680b3080 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:10.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- 172.21.2.104:0/527687257 >> 172.21.2.104:0/527687257 conn(0x7f9b680607c0 msgr2=0x7f9b68060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:10.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- 172.21.2.104:0/527687257 shutdown_connections 2021-07-11T04:00:10.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- 172.21.2.104:0/527687257 wait complete. 2021-07-11T04:00:10.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 Processor -- start 2021-07-11T04:00:10.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- start start 2021-07-11T04:00:10.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9b680b13f0 0x7f9b68064510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9b680b1d60 0x7f9b68067260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9b681472c0 0x7f9b681439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b680acbe0 con 0x7f9b681472c0 2021-07-11T04:00:10.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b680638d0 con 0x7f9b680b13f0 2021-07-11T04:00:10.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.425+0000 7f9b6f712700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b68153220 con 0x7f9b680b1d60 2021-07-11T04:00:10.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6dcaf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9b681472c0 0x7f9b681439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:10.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6dcaf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9b681472c0 0x7f9b681439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46922/0 (socket says 172.21.2.104:46922) 2021-07-11T04:00:10.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6dcaf700 1 -- 172.21.2.104:0/859261245 learned_addr learned my addr 172.21.2.104:0/859261245 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:10.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6dcaf700 1 -- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9b680b1d60 msgr2=0x7f9b68067260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6dcaf700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9b680b1d60 0x7f9b68067260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6dcaf700 1 -- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9b680b13f0 msgr2=0x7f9b68064510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6dcaf700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9b680b13f0 0x7f9b68064510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6dcaf700 1 -- 172.21.2.104:0/859261245 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b60009530 con 0x7f9b681472c0 2021-07-11T04:00:10.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6dcaf700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9b681472c0 0x7f9b681439a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f9b680acd20 tx=0x7f9b58006da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:10.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b667fc700 1 -- 172.21.2.104:0/859261245 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b58012410 con 0x7f9b681472c0 2021-07-11T04:00:10.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b667fc700 1 -- 172.21.2.104:0/859261245 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b58012570 con 0x7f9b681472c0 2021-07-11T04:00:10.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6f712700 1 -- 172.21.2.104:0/859261245 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b68063b00 con 0x7f9b681472c0 2021-07-11T04:00:10.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b667fc700 1 -- 172.21.2.104:0/859261245 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9b580128d0 con 0x7f9b681472c0 2021-07-11T04:00:10.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6f712700 1 -- 172.21.2.104:0/859261245 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b68152ed0 con 0x7f9b681472c0 2021-07-11T04:00:10.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6f712700 1 -- 172.21.2.104:0/859261245 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b68057350 con 0x7f9b681472c0 2021-07-11T04:00:10.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b667fc700 1 -- 172.21.2.104:0/859261245 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9b5800f030 con 0x7f9b681472c0 2021-07-11T04:00:10.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b667fc700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b48053270 0x7f9b48055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b667fc700 1 -- 172.21.2.104:0/859261245 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f9b5806aff0 con 0x7f9b681472c0 2021-07-11T04:00:10.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6d4ae700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b48053270 0x7f9b48055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:10.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.429+0000 7f9b6d4ae700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b48053270 0x7f9b48055730 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9b68146a90 tx=0x7f9b5c0077d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:10.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.433+0000 7f9b667fc700 1 -- 172.21.2.104:0/859261245 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9b580424d0 con 0x7f9b681472c0 2021-07-11T04:00:10.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.513+0000 7f9b6f712700 1 -- 172.21.2.104:0/859261245 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9b680b3810 con 0x7f9b681472c0 2021-07-11T04:00:10.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.517+0000 7f9b667fc700 1 -- 172.21.2.104:0/859261245 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9b58045af0 con 0x7f9b681472c0 2021-07-11T04:00:10.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 -- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b48053270 msgr2=0x7f9b48055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b48053270 0x7f9b48055730 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9b68146a90 tx=0x7f9b5c0077d0).stop 2021-07-11T04:00:10.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 -- 172.21.2.104:0/859261245 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9b681472c0 msgr2=0x7f9b681439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9b681472c0 0x7f9b681439a0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f9b680acd20 tx=0x7f9b58006da0).stop 2021-07-11T04:00:10.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 -- 172.21.2.104:0/859261245 shutdown_connections 2021-07-11T04:00:10.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9b48053270 0x7f9b48055730 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:10.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9b681472c0 0x7f9b681439a0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:10.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9b680b1d60 0x7f9b68067260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 --2- 172.21.2.104:0/859261245 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9b680b13f0 0x7f9b68064510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 -- 172.21.2.104:0/859261245 >> 172.21.2.104:0/859261245 conn(0x7f9b680607c0 msgr2=0x7f9b68060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:10.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 -- 172.21.2.104:0/859261245 shutdown_connections 2021-07-11T04:00:10.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.521+0000 7f9b6f712700 1 -- 172.21.2.104:0/859261245 wait complete. 2021-07-11T04:00:10.533 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:00:10.537 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:10.549 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:10.560 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:10.574 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:10.589 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:00:10.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:10.590 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:10.590 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:10.602 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T04:00:10.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:00:10.606 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T04:00:10.606 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:10.607 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:10.607 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:10.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:00:10.612 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T04:00:10.613 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T04:00:10.613 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:00:10.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.661+0000 7f62ae13e700 1 Processor -- start 2021-07-11T04:00:10.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.661+0000 7f62ae13e700 1 -- start start 2021-07-11T04:00:10.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.661+0000 7f62ae13e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f62a80b1db0 0x7f62a80b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.661+0000 7f62ae13e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a80b0350 con 0x7f62a80a70c0 2021-07-11T04:00:10.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.661+0000 7f62ae13e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a80b05f0 con 0x7f62a81483a0 2021-07-11T04:00:10.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.661+0000 7f62ae13e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a80a90c0 con 0x7f62a80b1db0 2021-07-11T04:00:10.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.661+0000 7f62a7fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f62a80a70c0 0x7f62a8144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43696/0 (socket says 172.21.2.104:43696) 2021-07-11T04:00:10.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.661+0000 7f62a7fff700 1 -- 172.21.2.104:0/2349601587 learned_addr learned my addr 172.21.2.104:0/2349601587 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:10.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.661+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190459184 0 0) 0x7f62a80b0350 con 0x7f62a80a70c0 2021-07-11T04:00:10.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.661+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6288003260 con 0x7f62a80a70c0 2021-07-11T04:00:10.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2666367197 0 0) 0x7f6288003260 con 0x7f62a80a70c0 2021-07-11T04:00:10.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 >> v2:172.21.2.105:3300/0 conn(0x7f62a80b1db0 msgr2=0x7f62a80b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:10.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 --2- 172.21.2.104:0/2349601587 >> v2:172.21.2.105:3300/0 conn(0x7f62a80b1db0 0x7f62a80b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 >> v1:172.21.2.105:6789/0 conn(0x7f62a81483a0 legacy=0x7f62a80b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:10.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a80b0890 con 0x7f62a80a70c0 2021-07-11T04:00:10.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f62a00019e0 con 0x7f62a80a70c0 2021-07-11T04:00:10.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 >> v1:172.21.2.104:6789/0 conn(0x7f62a80a70c0 legacy=0x7f62a8144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 --2- 172.21.2.104:0/2349601587 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6288003750 0x7f6288003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 --2- 172.21.2.104:0/2349601587 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6288004500 0x7f6288004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 --2- 172.21.2.104:0/2349601587 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6288004f90 0x7f628800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628800be50 con 0x7f6288004f90 2021-07-11T04:00:10.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628800bfc0 con 0x7f6288004500 2021-07-11T04:00:10.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628800c180 con 0x7f6288003750 2021-07-11T04:00:10.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a7fff700 1 --2- 172.21.2.104:0/2349601587 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6288004f90 0x7f628800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:10.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a7fff700 1 -- 172.21.2.104:0/2349601587 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6288003750 msgr2=0x7f6288003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:10.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a7fff700 1 --2- 172.21.2.104:0/2349601587 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6288003750 0x7f6288003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a7fff700 1 -- 172.21.2.104:0/2349601587 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6288004500 msgr2=0x7f6288004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:10.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a7fff700 1 --2- 172.21.2.104:0/2349601587 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6288004500 0x7f6288004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a7fff700 1 -- 172.21.2.104:0/2349601587 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6288003260 con 0x7f6288004f90 2021-07-11T04:00:10.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a7fff700 1 -- 172.21.2.104:0/2349601587 reap_dead start 2021-07-11T04:00:10.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a7fff700 1 --2- 172.21.2.104:0/2349601587 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6288004f90 0x7f628800b750 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f62a80b3170 tx=0x7f62a80b0a20).ready entity=mon.0 client_cookie=abe1d3b0be9d4004 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:10.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62a000cc90 con 0x7f6288004f90 2021-07-11T04:00:10.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a000cdf0 con 0x7f6288004f90 2021-07-11T04:00:10.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a67fc700 1 -- 172.21.2.104:0/2349601587 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f62a0005b80 con 0x7f6288004f90 2021-07-11T04:00:10.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 -- 172.21.2.104:0/2349601587 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6288004f90 msgr2=0x7f628800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 --2- 172.21.2.104:0/2349601587 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6288004f90 0x7f628800b750 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f62a80b3170 tx=0x7f62a80b0a20).stop 2021-07-11T04:00:10.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 -- 172.21.2.104:0/2349601587 shutdown_connections 2021-07-11T04:00:10.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 --2- 172.21.2.104:0/2349601587 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6288004f90 0x7f628800b750 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:10.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 -- 172.21.2.104:0/2349601587 >> 172.21.2.104:0/2349601587 conn(0x7f62a805f6b0 msgr2=0x7f62a805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:10.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 -- 172.21.2.104:0/2349601587 shutdown_connections 2021-07-11T04:00:10.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 -- 172.21.2.104:0/2349601587 wait complete. 2021-07-11T04:00:10.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 Processor -- start 2021-07-11T04:00:10.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 -- start start 2021-07-11T04:00:10.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6288004f90 0x7f62a80abc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62a80655a0 0x7f62a8144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f62a8144cf0 0x7f62a8066040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8064520 con 0x7f62a80655a0 2021-07-11T04:00:10.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80647c0 con 0x7f62a8144cf0 2021-07-11T04:00:10.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8064a60 con 0x7f6288004f90 2021-07-11T04:00:10.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62a80655a0 0x7f62a8144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:10.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62a80655a0 0x7f62a8144800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46942/0 (socket says 172.21.2.104:46942) 2021-07-11T04:00:10.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a6ffd700 1 -- 172.21.2.104:0/1902368028 learned_addr learned my addr 172.21.2.104:0/1902368028 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:10.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a6ffd700 1 -- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6288004f90 msgr2=0x7f62a80abc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a6ffd700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6288004f90 0x7f62a80abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a6ffd700 1 -- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f62a8144cf0 msgr2=0x7f62a8066040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a6ffd700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f62a8144cf0 0x7f62a8066040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a6ffd700 1 -- 172.21.2.104:0/1902368028 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a8145170 con 0x7f62a80655a0 2021-07-11T04:00:10.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a6ffd700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62a80655a0 0x7f62a8144800 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f6290000f70 tx=0x7f6290006dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:10.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a4ff9700 1 -- 172.21.2.104:0/1902368028 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6290013460 con 0x7f62a80655a0 2021-07-11T04:00:10.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a4ff9700 1 -- 172.21.2.104:0/1902368028 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62900135c0 con 0x7f62a80655a0 2021-07-11T04:00:10.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 -- 172.21.2.104:0/1902368028 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a80ac130 con 0x7f62a80655a0 2021-07-11T04:00:10.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62ae13e700 1 -- 172.21.2.104:0/1902368028 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a81564c0 con 0x7f62a80655a0 2021-07-11T04:00:10.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.665+0000 7f62a4ff9700 1 -- 172.21.2.104:0/1902368028 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6290013920 con 0x7f62a80655a0 2021-07-11T04:00:10.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.669+0000 7f62ae13e700 1 -- 172.21.2.104:0/1902368028 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a8064c20 con 0x7f62a80655a0 2021-07-11T04:00:10.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.669+0000 7f62a4ff9700 1 -- 172.21.2.104:0/1902368028 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f629001a060 con 0x7f62a80655a0 2021-07-11T04:00:10.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.669+0000 7f62a4ff9700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f628c0532b0 0x7f628c055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:10.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.669+0000 7f62a4ff9700 1 -- 172.21.2.104:0/1902368028 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f629006ac00 con 0x7f62a80655a0 2021-07-11T04:00:10.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.669+0000 7f62a77fe700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f628c0532b0 0x7f628c055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:10.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.669+0000 7f62a77fe700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f628c0532b0 0x7f628c055770 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f6298002f70 tx=0x7f6298006a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:10.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.669+0000 7f62a4ff9700 1 -- 172.21.2.104:0/1902368028 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6290042080 con 0x7f62a80655a0 2021-07-11T04:00:10.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 -- 172.21.2.104:0/1902368028 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f62a8057350 con 0x7f62a80655a0 2021-07-11T04:00:10.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62a4ff9700 1 -- 172.21.2.104:0/1902368028 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v80) v1 ==== 81+0+28898 (secure 0 0 0) 0x7f6290018020 con 0x7f62a80655a0 2021-07-11T04:00:10.759 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:10.759 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:00:10.760 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 80, 2021-07-11T04:00:10.760 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:00:10.760 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:00:10.760 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:00:08.342868+0000", 2021-07-11T04:00:10.761 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:10.761 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:10.761 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:00:10.761 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:00:10.762 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:00:10.762 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:00:10.762 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:00:10.762 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:00:10.763 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:00:10.763 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:10.763 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:00:10.763 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:00:10.763 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:00:10.764 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:00:10.764 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:00:10.764 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 14, 2021-07-11T04:00:10.764 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:00:10.765 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:00:10.765 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:00:10.765 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:00:10.765 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:00:10.766 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.766 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:00:10.766 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:00:10.768 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:00:10.768 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:10.769 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:10.769 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:10.769 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:10.769 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:10.770 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:10.770 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:10.770 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:10.770 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:00:10.771 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:00:10.771 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:00:10.771 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:00:10.771 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:00:10.771 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:10.772 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:10.772 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:10.772 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:10.772 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:10.773 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:10.773 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:10.773 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.773 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:00:10.773 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:10.774 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:10.774 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:10.774 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:10.774 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:10.775 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:10.775 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:10.775 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:10.775 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:10.775 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:10.776 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:10.776 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:10.776 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:10.776 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:10.777 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:10.777 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:10.777 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:10.777 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:10.778 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:10.778 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:10.778 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:10.778 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:10.779 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:10.779 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:10.779 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:10.779 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:10.779 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.780 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:10.780 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:10.780 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:10.780 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:10.781 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:10.781 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:10.781 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:10.781 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:10.781 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:10.782 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:10.782 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:10.782 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:00:10.782 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:00:10.783 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.783 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:10.783 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:00:10.783 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.784 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.784 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.784 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:00:10.784 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:00:10.784 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:00:10.785 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:10.785 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:10.785 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:10.785 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:10.785 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:10.786 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:10.786 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:10.786 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:10.787 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:10.787 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:10.787 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:10.787 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:10.788 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:10.788 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:10.788 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:10.788 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:10.788 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:10.789 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:10.789 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:10.789 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:10.789 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.790 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:00:10.790 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:10.790 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:10.790 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:10.791 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:10.791 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:10.791 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:10.791 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:10.792 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:10.792 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:10.792 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:10.792 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:10.792 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:10.793 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:10.793 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:10.793 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:10.793 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:10.794 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:10.794 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:10.794 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:10.794 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:10.794 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:10.795 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:10.795 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:10.795 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:10.795 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:10.795 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:10.796 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.796 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:10.796 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:10.796 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:10.797 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:10.797 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:10.797 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:10.797 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:10.798 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:10.798 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:10.798 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:10.798 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:10.799 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:00:10.799 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:10.799 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:00:10.799 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.799 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.800 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:10.800 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:00:10.800 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.801 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:10.801 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:00:10.801 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:10.801 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:10.802 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:10.802 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:10.802 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:10.802 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:10.803 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:10.803 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 74, 2021-07-11T04:00:10.803 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:10.803 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:10.803 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:10.804 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.804 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.804 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.805 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:00:10.805 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:10.805 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.805 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.805 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.806 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:00:10.806 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:10.806 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.806 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.807 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.807 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:10.807 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.807 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.807 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.808 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:00:10.808 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:10.808 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.808 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.809 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.809 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:00:10.809 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:10.809 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.810 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.810 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.810 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:10.811 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.811 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.811 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.811 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:00:10.812 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:10.812 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.812 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.812 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.812 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:00:10.813 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:10.813 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.813 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.813 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.814 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:10.814 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.814 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.814 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.815 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:00:10.815 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:10.815 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.815 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.815 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.816 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:00:10.816 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:10.816 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.816 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.816 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.817 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:00:10.817 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:00:10.817 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:00:10.818 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:00:10.818 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:10.818 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:10.818 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:10.818 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.819 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.819 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.819 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:10.819 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:00:10.820 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:10.820 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:10.820 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:10.820 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:10.820 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:10.821 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:10.821 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:10.821 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 62, 2021-07-11T04:00:10.823 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:10.823 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:10.823 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:10.824 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.824 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.824 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.824 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:00:10.825 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:10.825 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.825 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.825 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.825 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:00:10.826 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:10.826 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.826 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.826 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.827 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:10.827 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.827 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.827 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.828 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:00:10.828 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:10.828 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.828 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.828 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.829 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:00:10.829 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:10.829 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.829 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.830 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.830 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:10.830 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.830 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.831 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.831 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:00:10.831 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:10.832 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.832 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.832 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.832 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:00:10.833 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:10.833 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.833 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.833 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.833 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:10.834 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.834 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.835 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.835 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:00:10.835 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:10.835 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.836 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.836 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.836 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:00:10.836 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:10.837 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.837 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.837 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.837 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:00:10.838 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:00:10.838 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:00:10.838 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:00:10.838 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:10.839 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:10.839 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:10.839 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.839 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.840 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.840 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:10.840 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:00:10.840 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:10.840 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:10.841 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:10.841 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:10.841 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:10.841 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:10.842 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:10.842 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 74, 2021-07-11T04:00:10.842 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:10.843 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:10.843 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:10.843 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.843 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.843 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.844 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:00:10.844 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:10.844 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.844 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.853 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.853 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:00:10.853 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:10.853 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.854 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.854 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.854 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:10.854 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.855 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.855 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.855 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:00:10.855 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:10.855 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.856 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.856 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.856 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:00:10.856 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:10.856 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.857 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.857 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.857 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:10.857 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.858 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.858 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.858 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:00:10.858 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:10.859 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.859 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.859 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.859 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:00:10.859 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:10.860 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.860 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.860 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.860 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:10.861 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.861 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.861 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.861 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:00:10.861 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:10.862 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.862 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.862 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.862 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:00:10.863 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:10.863 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.863 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.863 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.863 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:00:10.864 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:00:10.864 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:00:10.864 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:00:10.864 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:10.865 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:10.865 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:10.865 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.865 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.866 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.866 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:10.866 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:00:10.866 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:10.867 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:10.867 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:10.867 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:10.867 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:10.867 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:10.868 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:10.868 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 74, 2021-07-11T04:00:10.868 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:10.868 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:10.868 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:10.869 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.869 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.869 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.869 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:00:10.870 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:10.870 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.870 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.870 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.870 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:00:10.871 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:10.871 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.871 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.871 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.872 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:10.872 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.872 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.872 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.873 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:00:10.873 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:10.873 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.873 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.873 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.874 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:00:10.874 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:10.874 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.874 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.875 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.875 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:10.875 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.875 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.875 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.876 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:00:10.876 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:10.876 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.877 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.877 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.877 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:00:10.877 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:10.878 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.878 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.878 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.878 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:10.878 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.879 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.879 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.879 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:00:10.880 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:10.880 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.880 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.880 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.881 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:00:10.881 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:10.881 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.881 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.882 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.882 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:00:10.882 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:00:10.882 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:00:10.882 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:00:10.883 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:10.883 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:10.883 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:10.883 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.883 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.884 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.884 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:10.884 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:00:10.884 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:10.885 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:10.885 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:10.885 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:10.885 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:10.885 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:10.886 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:10.886 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 73, 2021-07-11T04:00:10.886 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:10.886 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:10.887 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:10.887 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.887 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.887 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.888 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:00:10.888 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:10.888 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.888 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.888 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.889 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:00:10.889 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:10.889 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.889 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.890 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.890 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:10.890 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.890 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.890 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.891 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:00:10.891 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:10.891 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.891 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.891 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.892 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:00:10.892 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:10.892 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.892 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.893 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.893 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:10.893 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.893 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.894 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.894 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:00:10.894 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:10.894 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.895 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.895 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.895 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:00:10.895 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:10.895 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.896 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.896 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.896 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:10.896 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.896 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.897 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.897 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:00:10.897 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:10.897 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.898 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.898 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.898 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:00:10.898 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:10.898 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.899 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.899 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.899 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:00:10.899 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:00:10.900 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:00:10.900 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:00:10.900 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:10.900 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:10.901 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:10.901 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.901 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.901 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.902 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:10.902 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:00:10.902 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:10.903 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:10.903 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:10.903 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:10.904 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:10.904 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:10.904 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:10.904 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 74, 2021-07-11T04:00:10.904 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:10.905 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:10.905 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:10.905 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.905 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.906 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.906 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:00:10.906 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:10.907 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.908 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.908 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.908 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:00:10.908 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:10.908 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.909 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.909 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.909 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:10.909 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.910 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.910 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.910 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:00:10.910 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:10.910 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.911 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.911 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.911 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:00:10.911 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:10.911 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.912 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.912 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.912 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:10.912 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.913 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.913 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.913 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:00:10.913 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:10.913 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.914 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.914 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.914 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:00:10.915 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:10.915 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.915 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.915 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.915 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:10.916 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.916 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.916 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.916 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:00:10.917 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:10.917 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.917 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.918 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.918 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:00:10.918 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:10.918 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.918 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.919 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.919 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:00:10.919 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:00:10.919 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:00:10.920 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:00:10.920 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:10.920 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:10.920 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:10.921 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.921 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.921 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.921 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:10.922 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:00:10.922 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:10.922 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:10.923 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:10.932 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:10.932 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:10.932 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:10.932 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:10.933 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 74, 2021-07-11T04:00:10.933 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:10.933 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:10.933 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:10.933 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.934 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.934 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.934 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:00:10.935 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:10.935 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.935 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.935 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.936 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:00:10.936 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:10.937 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.937 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.937 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.937 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:10.938 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.938 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.938 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.938 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:00:10.939 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:10.939 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.939 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.939 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.939 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:00:10.940 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:10.940 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.940 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.940 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.941 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:10.941 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.941 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.942 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.942 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:00:10.942 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:10.942 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.942 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.943 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.943 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:00:10.943 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:10.943 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.944 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.944 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.944 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:10.944 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.945 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.945 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.945 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:00:10.945 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:10.945 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.946 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.946 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.946 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:00:10.947 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:10.947 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.947 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.948 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.948 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:00:10.948 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:00:10.948 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:00:10.949 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:00:10.949 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:10.949 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:10.949 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:10.949 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.950 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.950 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.950 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:10.950 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:00:10.950 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:10.951 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:10.951 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:10.951 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:10.951 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:10.952 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:10.952 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:10.952 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 73, 2021-07-11T04:00:10.952 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:10.953 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:10.953 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:10.953 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.953 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.953 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.954 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:00:10.954 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:10.954 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.954 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.955 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.955 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:00:10.955 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:10.955 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.956 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.956 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.956 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:10.956 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.956 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.957 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.957 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:00:10.957 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:10.957 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.957 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.958 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.958 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:00:10.958 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:10.958 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.959 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.959 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.959 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:10.959 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.959 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.960 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:00:10.960 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:10.960 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.960 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.961 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.961 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:00:10.961 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:10.962 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.962 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.962 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.962 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:10.962 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:10.963 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.963 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:10.963 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:00:10.963 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:10.964 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.964 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.964 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:10.964 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:00:10.964 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:10.965 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.965 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.965 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.965 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:00:10.965 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:00:10.966 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:00:10.966 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:00:10.966 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:10.966 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:10.967 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:10.967 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:10.967 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.967 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:10.967 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:00:10.968 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.968 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:10.968 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:10.968 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:10.969 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:10.969 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:10.969 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:10.969 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:00:10.970 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:10.970 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.970 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.970 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:10.971 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:10.971 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:10.971 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:10.971 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:10.971 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:10.972 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:00:10.972 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:10.972 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.972 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.973 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:10.973 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:10.973 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:10.973 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:10.973 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:10.974 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:10.974 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:00:10.974 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:10.975 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.975 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.975 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:10.976 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:10.976 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:10.976 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:10.976 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:10.977 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:10.977 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:00:10.977 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:10.977 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.977 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.978 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:10.978 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:10.978 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:10.978 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:10.978 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:10.979 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:10.979 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:00:10.979 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:10.979 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.980 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.980 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:10.980 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:10.980 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:10.980 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:10.981 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:10.981 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:10.981 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:00:10.981 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:10.982 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.982 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.982 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:10.982 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:10.983 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:10.983 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:10.983 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:10.983 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:10.984 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:00:10.984 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:10.984 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.984 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:10.985 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:10.985 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:10.985 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:10.985 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:10.985 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:10.986 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:10.986 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:00:10.986 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:10.986 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.987 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:10.987 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:00:10.987 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:00:10.987 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:00:10.988 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:00:10.988 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T04:00:10.988 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6835/611806767": "2021-07-12T04:00:04.904477+0000", 2021-07-11T04:00:10.988 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6833/611806767": "2021-07-12T04:00:04.904477+0000", 2021-07-11T04:00:10.988 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.119:0/1304352204": "2021-07-11T04:59:54.428687+0000" 2021-07-11T04:00:10.989 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.989 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:00:10.989 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:00:10.990 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:00:10.990 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:00:10.990 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:00:10.990 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:00:10.990 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:00:10.991 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:00:10.991 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:10.991 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:10.991 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:00:10.991 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:00:10.992 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:00:10.992 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:00:10.992 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:00:10.992 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:00:10.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 -- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f628c0532b0 msgr2=0x7f628c055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f628c0532b0 0x7f628c055770 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f6298002f70 tx=0x7f6298006a00).stop 2021-07-11T04:00:10.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 -- 172.21.2.104:0/1902368028 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62a80655a0 msgr2=0x7f62a8144800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:10.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62a80655a0 0x7f62a8144800 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f6290000f70 tx=0x7f6290006dc0).stop 2021-07-11T04:00:10.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 -- 172.21.2.104:0/1902368028 shutdown_connections 2021-07-11T04:00:10.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f62a8144cf0 0x7f62a8066040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f628c0532b0 0x7f628c055770 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:10.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f62a80655a0 0x7f62a8144800 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:10.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 --2- 172.21.2.104:0/1902368028 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6288004f90 0x7f62a80abc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:10.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 -- 172.21.2.104:0/1902368028 >> 172.21.2.104:0/1902368028 conn(0x7f62a805f6b0 msgr2=0x7f62a80b3950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:10.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 -- 172.21.2.104:0/1902368028 shutdown_connections 2021-07-11T04:00:10.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:10.753+0000 7f62ae13e700 1 -- 172.21.2.104:0/1902368028 wait complete. 2021-07-11T04:00:10.997 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:11.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b13f8e700 1 Processor -- start 2021-07-11T04:00:11.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b13f8e700 1 -- start start 2021-07-11T04:00:11.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b13f8e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5b0c0a76b0 0x7f5b0c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b13f8e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b0c0b1970 0x7f5b0c066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b13f8e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b0c067450 con 0x7f5b0c0b0060 2021-07-11T04:00:11.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b13f8e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b0c0a7a90 con 0x7f5b0c0a76b0 2021-07-11T04:00:11.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b13f8e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c147620 con 0x7f5b0c0b1970 2021-07-11T04:00:11.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b1252b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f5b0c0b0060 0x7f5b0c05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43714/0 (socket says 172.21.2.104:43714) 2021-07-11T04:00:11.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b1252b700 1 -- 172.21.2.104:0/2560719355 learned_addr learned my addr 172.21.2.104:0/2560719355 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:11.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792067113 0 0) 0x7f5b0c067450 con 0x7f5b0c0b0060 2021-07-11T04:00:11.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af4003260 con 0x7f5b0c0b0060 2021-07-11T04:00:11.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b11529700 1 --2- 172.21.2.104:0/2560719355 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b0c0b1970 0x7f5b0c066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:11.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 439273909 0 0) 0x7f5af4003260 con 0x7f5b0c0b0060 2021-07-11T04:00:11.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b11d2a700 1 --2- 172.21.2.104:0/2560719355 >> v2:172.21.2.105:3300/0 conn(0x7f5b0c0a76b0 0x7f5b0c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:11.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b0c0b1970 msgr2=0x7f5b0c066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:11.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 --2- 172.21.2.104:0/2560719355 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b0c0b1970 0x7f5b0c066f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:11.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 >> v2:172.21.2.105:3300/0 conn(0x7f5b0c0a76b0 msgr2=0x7f5b0c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:11.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 --2- 172.21.2.104:0/2560719355 >> v2:172.21.2.105:3300/0 conn(0x7f5b0c0a76b0 0x7f5b0c066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:11.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c1478c0 con 0x7f5b0c0b0060 2021-07-11T04:00:11.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f5b040019e0 con 0x7f5b0c0b0060 2021-07-11T04:00:11.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 >> v1:172.21.2.104:6789/0 conn(0x7f5b0c0b0060 legacy=0x7f5b0c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 --2- 172.21.2.104:0/2560719355 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5af40037d0 0x7f5af4003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 --2- 172.21.2.104:0/2560719355 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5af4004500 0x7f5af4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 --2- 172.21.2.104:0/2560719355 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5af4005070 0x7f5af400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0a7a90 con 0x7f5af4005070 2021-07-11T04:00:11.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c147620 con 0x7f5af40037d0 2021-07-11T04:00:11.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af400bf00 con 0x7f5af4004500 2021-07-11T04:00:11.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b11d2a700 1 --2- 172.21.2.104:0/2560719355 >> v2:172.21.2.105:3300/0 conn(0x7f5b0c0a76b0 0x7f5b0c066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:11.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b11529700 1 --2- 172.21.2.104:0/2560719355 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b0c0b1970 0x7f5b0c066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:11.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.049+0000 7f5b1252b700 1 --2- 172.21.2.104:0/2560719355 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5af4005070 0x7f5af400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:11.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 -- 172.21.2.104:0/2560719355 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5af4004500 msgr2=0x7f5af4004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:11.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 --2- 172.21.2.104:0/2560719355 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5af4004500 0x7f5af4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 -- 172.21.2.104:0/2560719355 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5af40037d0 msgr2=0x7f5af4003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:11.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 --2- 172.21.2.104:0/2560719355 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5af40037d0 0x7f5af4003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 -- 172.21.2.104:0/2560719355 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af4003260 con 0x7f5af4005070 2021-07-11T04:00:11.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 --2- 172.21.2.104:0/2560719355 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5af4005070 0x7f5af400b750 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f5b040016f0 tx=0x7f5b04002100).ready entity=mon.0 client_cookie=d43b9befda0a4d90 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:11.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b0400cbb0 con 0x7f5af4005070 2021-07-11T04:00:11.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0400cd10 con 0x7f5af4005070 2021-07-11T04:00:11.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b10d28700 1 -- 172.21.2.104:0/2560719355 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b04005b80 con 0x7f5af4005070 2021-07-11T04:00:11.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- 172.21.2.104:0/2560719355 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5af4005070 msgr2=0x7f5af400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 --2- 172.21.2.104:0/2560719355 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5af4005070 0x7f5af400b750 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f5b040016f0 tx=0x7f5b04002100).stop 2021-07-11T04:00:11.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 -- 172.21.2.104:0/2560719355 reap_dead start 2021-07-11T04:00:11.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- 172.21.2.104:0/2560719355 shutdown_connections 2021-07-11T04:00:11.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- 172.21.2.104:0/2560719355 >> 172.21.2.104:0/2560719355 conn(0x7f5b0c0b53e0 msgr2=0x7f5b0c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:11.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- 172.21.2.104:0/2560719355 shutdown_connections 2021-07-11T04:00:11.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- 172.21.2.104:0/2560719355 wait complete. 2021-07-11T04:00:11.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 Processor -- start 2021-07-11T04:00:11.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- start start 2021-07-11T04:00:11.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b0c0a9de0 0x7f5b0c0abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b0c0ae770 0x7f5b0c159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b0c0aebf0 0x7f5b0c15a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0aa1c0 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0aa940 con 0x7f5b0c0a9de0 2021-07-11T04:00:11.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c0aabe0 con 0x7f5b0c0ae770 2021-07-11T04:00:11.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b0c0aebf0 0x7f5b0c15a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:11.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b0c0aebf0 0x7f5b0c15a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46960/0 (socket says 172.21.2.104:46960) 2021-07-11T04:00:11.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 -- 172.21.2.104:0/565671638 learned_addr learned my addr 172.21.2.104:0/565671638 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:11.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 -- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b0c0ae770 msgr2=0x7f5b0c159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b0c0ae770 0x7f5b0c159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 -- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b0c0a9de0 msgr2=0x7f5b0c0abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b0c0a9de0 0x7f5b0c0abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 -- 172.21.2.104:0/565671638 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c0ab3a0 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b1252b700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b0c0aebf0 0x7f5b0c15a130 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f5b0c0a8ee0 tx=0x7f5b04015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:11.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5afeffd700 1 -- 172.21.2.104:0/565671638 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b04001ee0 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5afeffd700 1 -- 172.21.2.104:0/565671638 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b040074d0 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5afeffd700 1 -- 172.21.2.104:0/565671638 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b04007830 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- 172.21.2.104:0/565671638 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c0ab640 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- 172.21.2.104:0/565671638 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c0ac520 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.053+0000 7f5b13f8e700 1 -- 172.21.2.104:0/565671638 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0c0ac220 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.057+0000 7f5afeffd700 1 -- 172.21.2.104:0/565671638 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5b04010070 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.057+0000 7f5afeffd700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5aec0531f0 0x7f5aec0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.057+0000 7f5afeffd700 1 -- 172.21.2.104:0/565671638 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f5b0406c000 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.057+0000 7f5b11d2a700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5aec0531f0 0x7f5aec0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:11.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.057+0000 7f5b11d2a700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5aec0531f0 0x7f5aec0556b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f5b00007000 tx=0x7f5b00008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:11.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.057+0000 7f5afeffd700 1 -- 172.21.2.104:0/565671638 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5b040434e0 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.165+0000 7f5b13f8e700 1 -- 172.21.2.104:0/565671638 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b0c0aa1c0 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5afeffd700 1 -- 172.21.2.104:0/565671638 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f5b04025030 con 0x7f5b0c0aebf0 2021-07-11T04:00:11.174 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:11.175 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/140397159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":140397159},{"type":"v1","addr":"172.21.2.105:6835","nonce":140397159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/2616903361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2616903361},{"type":"v1","addr":"172.21.2.104:6834","nonce":2616903361}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":6546,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6835/2101865821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":2101865821},{"type":"v1","addr":"172.21.2.104:6835","nonce":2101865821}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":35}],"filesystems":[]} 2021-07-11T04:00:11.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 -- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5aec0531f0 msgr2=0x7f5aec0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5aec0531f0 0x7f5aec0556b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f5b00007000 tx=0x7f5b00008040).stop 2021-07-11T04:00:11.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 -- 172.21.2.104:0/565671638 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b0c0aebf0 msgr2=0x7f5b0c15a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b0c0aebf0 0x7f5b0c15a130 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f5b0c0a8ee0 tx=0x7f5b04015000).stop 2021-07-11T04:00:11.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 -- 172.21.2.104:0/565671638 shutdown_connections 2021-07-11T04:00:11.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5aec0531f0 0x7f5aec0556b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:11.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b0c0aebf0 0x7f5b0c15a130 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:11.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b0c0ae770 0x7f5b0c159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 --2- 172.21.2.104:0/565671638 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b0c0a9de0 0x7f5b0c0abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 -- 172.21.2.104:0/565671638 >> 172.21.2.104:0/565671638 conn(0x7f5b0c0b53e0 msgr2=0x7f5b0c1497a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:11.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 -- 172.21.2.104:0/565671638 shutdown_connections 2021-07-11T04:00:11.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.169+0000 7f5b13f8e700 1 -- 172.21.2.104:0/565671638 wait complete. 2021-07-11T04:00:11.179 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 35 2021-07-11T04:00:11.181 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:11.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40bb54a700 1 Processor -- start 2021-07-11T04:00:11.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40bb54a700 1 -- start start 2021-07-11T04:00:11.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40bb54a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f40b40a76b0 0x7f40b4066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40bb54a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40b40b1970 0x7f40b4066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40bb54a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40b40b0060 0x7f40b405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40bb54a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b4067450 con 0x7f40b40b1970 2021-07-11T04:00:11.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40bb54a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b40a7a90 con 0x7f40b40a76b0 2021-07-11T04:00:11.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40bb54a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b4147620 con 0x7f40b40b0060 2021-07-11T04:00:11.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40b8ae5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40b40b1970 0x7f40b4066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:11.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40b8ae5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40b40b1970 0x7f40b4066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46966/0 (socket says 172.21.2.104:46966) 2021-07-11T04:00:11.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40b8ae5700 1 -- 172.21.2.104:0/2960704754 learned_addr learned my addr 172.21.2.104:0/2960704754 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:11.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40b8ae5700 1 -- 172.21.2.104:0/2960704754 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40b40b0060 msgr2=0x7f40b405f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:11.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40b8ae5700 1 --2- 172.21.2.104:0/2960704754 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40b40b0060 0x7f40b405f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40b8ae5700 1 -- 172.21.2.104:0/2960704754 >> v2:172.21.2.105:3300/0 conn(0x7f40b40a76b0 msgr2=0x7f40b4066a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:11.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40b8ae5700 1 --2- 172.21.2.104:0/2960704754 >> v2:172.21.2.105:3300/0 conn(0x7f40b40a76b0 0x7f40b4066a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40b8ae5700 1 -- 172.21.2.104:0/2960704754 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b41478c0 con 0x7f40b40b1970 2021-07-11T04:00:11.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.229+0000 7f40b8ae5700 1 --2- 172.21.2.104:0/2960704754 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40b40b1970 0x7f40b4066f60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f40b0002f70 tx=0x7f40b0006eb0).ready entity=mon.0 client_cookie=1edea04c5861dcee server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:11.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40a7fff700 1 -- 172.21.2.104:0/2960704754 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40b000d570 con 0x7f40b40b1970 2021-07-11T04:00:11.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40a7fff700 1 -- 172.21.2.104:0/2960704754 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b0000e90 con 0x7f40b40b1970 2021-07-11T04:00:11.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40a7fff700 1 -- 172.21.2.104:0/2960704754 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40b000d900 con 0x7f40b40b1970 2021-07-11T04:00:11.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- 172.21.2.104:0/2960704754 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40b40b1970 msgr2=0x7f40b4066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 --2- 172.21.2.104:0/2960704754 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40b40b1970 0x7f40b4066f60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f40b0002f70 tx=0x7f40b0006eb0).stop 2021-07-11T04:00:11.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- 172.21.2.104:0/2960704754 shutdown_connections 2021-07-11T04:00:11.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 --2- 172.21.2.104:0/2960704754 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40b40b0060 0x7f40b405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 --2- 172.21.2.104:0/2960704754 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40b40b1970 0x7f40b4066f60 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:11.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 --2- 172.21.2.104:0/2960704754 >> v2:172.21.2.105:3300/0 conn(0x7f40b40a76b0 0x7f40b4066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- 172.21.2.104:0/2960704754 >> 172.21.2.104:0/2960704754 conn(0x7f40b40b53e0 msgr2=0x7f40b40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:11.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- 172.21.2.104:0/2960704754 shutdown_connections 2021-07-11T04:00:11.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- 172.21.2.104:0/2960704754 wait complete. 2021-07-11T04:00:11.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 Processor -- start 2021-07-11T04:00:11.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- start start 2021-07-11T04:00:11.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40b40a76b0 0x7f40b4148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40b40b0060 0x7f40b4159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40b40b1970 0x7f40b40ab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40abdd0 con 0x7f40b40b0060 2021-07-11T04:00:11.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40ac070 con 0x7f40b40a76b0 2021-07-11T04:00:11.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b40ab020 con 0x7f40b40b1970 2021-07-11T04:00:11.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40b8ae5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40b40b0060 0x7f40b4159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:11.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40b8ae5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40b40b0060 0x7f40b4159e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:46972/0 (socket says 172.21.2.104:46972) 2021-07-11T04:00:11.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40b8ae5700 1 -- 172.21.2.104:0/2655270671 learned_addr learned my addr 172.21.2.104:0/2655270671 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:11.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40b8ae5700 1 -- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40b40b1970 msgr2=0x7f40b40ab4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40b8ae5700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40b40b1970 0x7f40b40ab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40b8ae5700 1 -- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40b40a76b0 msgr2=0x7f40b4148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40b8ae5700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40b40a76b0 0x7f40b4148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40b8ae5700 1 -- 172.21.2.104:0/2655270671 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b000b1e0 con 0x7f40b40b0060 2021-07-11T04:00:11.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40b8ae5700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40b40b0060 0x7f40b4159e40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f40b0002f70 tx=0x7f40b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:11.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40a67fc700 1 -- 172.21.2.104:0/2655270671 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40b0021070 con 0x7f40b40b0060 2021-07-11T04:00:11.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40a67fc700 1 -- 172.21.2.104:0/2655270671 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b0000e90 con 0x7f40b40b0060 2021-07-11T04:00:11.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- 172.21.2.104:0/2655270671 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b415a330 con 0x7f40b40b0060 2021-07-11T04:00:11.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40a67fc700 1 -- 172.21.2.104:0/2655270671 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f40b0011b80 con 0x7f40b40b0060 2021-07-11T04:00:11.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- 172.21.2.104:0/2655270671 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b40abb50 con 0x7f40b40b0060 2021-07-11T04:00:11.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40bb54a700 1 -- 172.21.2.104:0/2655270671 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b40ab1e0 con 0x7f40b40b0060 2021-07-11T04:00:11.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40a67fc700 1 -- 172.21.2.104:0/2655270671 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f40b000f070 con 0x7f40b40b0060 2021-07-11T04:00:11.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.233+0000 7f40a67fc700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4094053240 0x7f4094055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.237+0000 7f40a67fc700 1 -- 172.21.2.104:0/2655270671 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f40b006c740 con 0x7f40b40b0060 2021-07-11T04:00:11.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.237+0000 7f40b92e6700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4094053240 0x7f4094055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:11.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.237+0000 7f40b92e6700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4094053240 0x7f4094055700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f40a8002fb0 tx=0x7f40a80069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:11.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.237+0000 7f40a67fc700 1 -- 172.21.2.104:0/2655270671 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f40b0043c20 con 0x7f40b40b0060 2021-07-11T04:00:11.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 -- 172.21.2.104:0/2655270671 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40b4057350 con 0x7f40b40b0060 2021-07-11T04:00:11.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40a67fc700 1 -- 172.21.2.104:0/2655270671 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f40b0047240 con 0x7f40b40b0060 2021-07-11T04:00:11.350 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:11.351 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4807,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/140397159","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":140397159},{"type":"v1","addr":"172.21.2.105:6835","nonce":140397159}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":31},{"gid":6243,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/2616903361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":2616903361},{"type":"v1","addr":"172.21.2.104:6834","nonce":2616903361}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":30},{"gid":6546,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6835/2101865821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":2101865821},{"type":"v1","addr":"172.21.2.104:6835","nonce":2101865821}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":35}],"filesystems":[]} 2021-07-11T04:00:11.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 -- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4094053240 msgr2=0x7f4094055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4094053240 0x7f4094055700 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f40a8002fb0 tx=0x7f40a80069b0).stop 2021-07-11T04:00:11.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 -- 172.21.2.104:0/2655270671 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40b40b0060 msgr2=0x7f40b4159e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40b40b0060 0x7f40b4159e40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f40b0002f70 tx=0x7f40b0015000).stop 2021-07-11T04:00:11.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 -- 172.21.2.104:0/2655270671 shutdown_connections 2021-07-11T04:00:11.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40b40b1970 0x7f40b40ab4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4094053240 0x7f4094055700 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:11.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40b40b0060 0x7f40b4159e40 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:11.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 --2- 172.21.2.104:0/2655270671 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40b40a76b0 0x7f40b4148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 -- 172.21.2.104:0/2655270671 >> 172.21.2.104:0/2655270671 conn(0x7f40b40b53e0 msgr2=0x7f40b4149e60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:11.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 -- 172.21.2.104:0/2655270671 shutdown_connections 2021-07-11T04:00:11.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.345+0000 7f40bb54a700 1 -- 172.21.2.104:0/2655270671 wait complete. 2021-07-11T04:00:11.356 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 35 2021-07-11T04:00:11.359 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T04:00:11.359 INFO:tasks.ceph.mds.a:Stopping old one... 2021-07-11T04:00:11.360 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T04:00:11.360 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:00:11.360 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T04:00:11.360 INFO:tasks.ceph.mds.c:Stopping old one... 2021-07-11T04:00:11.361 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T04:00:11.361 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:00:11.362 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T04:00:11.362 INFO:tasks.ceph.mds.b:Stopping old one... 2021-07-11T04:00:11.362 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T04:00:11.363 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:00:11.366 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T04:00:11.367 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T04:00:11.370 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T04:00:11.370 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T04:00:11.372 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T04:00:11.372 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T04:00:11.374 INFO:tasks.ceph.mds.b:Started 2021-07-11T04:00:11.374 DEBUG:teuthology.parallel:result is None 2021-07-11T04:00:11.409 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T04:00:11.414 INFO:tasks.ceph.mds.a:Started 2021-07-11T04:00:11.414 INFO:tasks.ceph.mds.c:Started 2021-07-11T04:00:11.415 DEBUG:teuthology.parallel:result is None 2021-07-11T04:00:11.415 DEBUG:teuthology.parallel:result is None 2021-07-11T04:00:11.416 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T04:00:11.450 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T04:00:11.459 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T04:00:11.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf8551700 1 Processor -- start 2021-07-11T04:00:11.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf8551700 1 -- start start 2021-07-11T04:00:11.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf8551700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fccf0067120 0x7fccf00b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf8551700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf01439b0 0x7fccf00aec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf8551700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf00b9bd0 con 0x7fccf013fff0 2021-07-11T04:00:11.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf8551700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf0143080 con 0x7fccf0067120 2021-07-11T04:00:11.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf8551700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf0067500 con 0x7fccf01439b0 2021-07-11T04:00:11.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf62ed700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fccf0067120 0x7fccf00b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:11.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf62ed700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fccf0067120 0x7fccf00b96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47014/0 (socket says 172.21.2.104:47014) 2021-07-11T04:00:11.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf62ed700 1 -- 172.21.2.104:0/1808445063 learned_addr learned my addr 172.21.2.104:0/1808445063 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:11.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf62ed700 1 -- 172.21.2.104:0/1808445063 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf01439b0 msgr2=0x7fccf00aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:11.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf62ed700 1 --2- 172.21.2.104:0/1808445063 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf01439b0 0x7fccf00aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf62ed700 1 -- 172.21.2.104:0/1808445063 >> v1:172.21.2.105:6789/0 conn(0x7fccf013fff0 legacy=0x7fccf0140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:11.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf62ed700 1 -- 172.21.2.104:0/1808445063 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf00af4d0 con 0x7fccf0067120 2021-07-11T04:00:11.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf62ed700 1 --2- 172.21.2.104:0/1808445063 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf0067120 0x7fccf00b96e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fccf00b0b40 tx=0x7fccec006eb0).ready entity=mon.0 client_cookie=bbaee387beff2b05 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:11.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf52eb700 1 -- 172.21.2.104:0/1808445063 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccec00d570 con 0x7fccf0067120 2021-07-11T04:00:11.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf52eb700 1 -- 172.21.2.104:0/1808445063 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec000e90 con 0x7fccf0067120 2021-07-11T04:00:11.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf52eb700 1 -- 172.21.2.104:0/1808445063 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccec00d8c0 con 0x7fccf0067120 2021-07-11T04:00:11.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf8551700 1 -- 172.21.2.104:0/1808445063 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf0067120 msgr2=0x7fccf00b96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.477+0000 7fccf8551700 1 --2- 172.21.2.104:0/1808445063 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf0067120 0x7fccf00b96e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7fccf00b0b40 tx=0x7fccec006eb0).stop 2021-07-11T04:00:11.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 -- 172.21.2.104:0/1808445063 shutdown_connections 2021-07-11T04:00:11.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 --2- 172.21.2.104:0/1808445063 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf01439b0 0x7fccf00aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 --2- 172.21.2.104:0/1808445063 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf0067120 0x7fccf00b96e0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:11.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 -- 172.21.2.104:0/1808445063 >> 172.21.2.104:0/1808445063 conn(0x7fccf005f9f0 msgr2=0x7fccf00616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:11.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 -- 172.21.2.104:0/1808445063 shutdown_connections 2021-07-11T04:00:11.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 -- 172.21.2.104:0/1808445063 wait complete. 2021-07-11T04:00:11.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 Processor -- start 2021-07-11T04:00:11.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 -- start start 2021-07-11T04:00:11.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf0067120 0x7fccf00b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccf013fff0 0x7fccf01417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf01439b0 0x7fccf013f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00b8aa0 con 0x7fccf0067120 2021-07-11T04:00:11.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00b1b70 con 0x7fccf013fff0 2021-07-11T04:00:11.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00b1dc0 con 0x7fccf01439b0 2021-07-11T04:00:11.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf62ed700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf0067120 0x7fccf00b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:11.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf62ed700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf0067120 0x7fccf00b1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47020/0 (socket says 172.21.2.104:47020) 2021-07-11T04:00:11.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf62ed700 1 -- 172.21.2.104:0/404744020 learned_addr learned my addr 172.21.2.104:0/404744020 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:11.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf62ed700 1 -- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf01439b0 msgr2=0x7fccf013f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf62ed700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf01439b0 0x7fccf013f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf62ed700 1 -- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccf013fff0 msgr2=0x7fccf01417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:11.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf62ed700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccf013fff0 0x7fccf01417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:11.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf62ed700 1 -- 172.21.2.104:0/404744020 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec00b1e0 con 0x7fccf0067120 2021-07-11T04:00:11.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf62ed700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf0067120 0x7fccf00b1570 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fccf00b0b40 tx=0x7fccec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:11.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccdf7fe700 1 -- 172.21.2.104:0/404744020 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccec021070 con 0x7fccf0067120 2021-07-11T04:00:11.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccdf7fe700 1 -- 172.21.2.104:0/404744020 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec000e90 con 0x7fccf0067120 2021-07-11T04:00:11.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccdf7fe700 1 -- 172.21.2.104:0/404744020 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccec006be0 con 0x7fccf0067120 2021-07-11T04:00:11.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 -- 172.21.2.104:0/404744020 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf00b9270 con 0x7fccf0067120 2021-07-11T04:00:11.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 -- 172.21.2.104:0/404744020 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf00b9510 con 0x7fccf0067120 2021-07-11T04:00:11.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf8551700 1 -- 172.21.2.104:0/404744020 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf00b8c10 con 0x7fccf0067120 2021-07-11T04:00:11.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccdf7fe700 1 -- 172.21.2.104:0/404744020 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fccec00f070 con 0x7fccf0067120 2021-07-11T04:00:11.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccdf7fe700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053300 0x7fccd00557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:11.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccdf7fe700 1 -- 172.21.2.104:0/404744020 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fccec06c660 con 0x7fccf0067120 2021-07-11T04:00:11.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.481+0000 7fccf5aec700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053300 0x7fccd00557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:11.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.485+0000 7fccf5aec700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053300 0x7fccd00557c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fcce0002f70 tx=0x7fcce0006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:11.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.485+0000 7fccdf7fe700 1 -- 172.21.2.104:0/404744020 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fccf00b8c10 con 0x7fccf0067120 2021-07-11T04:00:11.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:11.577+0000 7fccf8551700 1 -- 172.21.2.104:0/404744020 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fccf00b9bd0 con 0x7fccf0067120 2021-07-11T04:00:12.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.533+0000 7fccdf7fe700 1 -- 172.21.2.104:0/404744020 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v82) v1 ==== 97+0+0 (secure 0 0 0) 0x7fccf00b9bd0 con 0x7fccf0067120 2021-07-11T04:00:12.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 -- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053300 msgr2=0x7fccd00557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053300 0x7fccd00557c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fcce0002f70 tx=0x7fcce0006a10).stop 2021-07-11T04:00:12.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 -- 172.21.2.104:0/404744020 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf0067120 msgr2=0x7fccf00b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf0067120 0x7fccf00b1570 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7fccf00b0b40 tx=0x7fccec015000).stop 2021-07-11T04:00:12.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 -- 172.21.2.104:0/404744020 shutdown_connections 2021-07-11T04:00:12.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf01439b0 0x7fccf013f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccf013fff0 0x7fccf01417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053300 0x7fccd00557c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:12.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 --2- 172.21.2.104:0/404744020 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf0067120 0x7fccf00b1570 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:12.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 -- 172.21.2.104:0/404744020 >> 172.21.2.104:0/404744020 conn(0x7fccf005f9f0 msgr2=0x7fccf005cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:12.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 -- 172.21.2.104:0/404744020 shutdown_connections 2021-07-11T04:00:12.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.537+0000 7fccf8551700 1 -- 172.21.2.104:0/404744020 wait complete. 2021-07-11T04:00:12.550 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T04:00:12.555 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T04:00:12.555 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T04:00:12.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e9fde700 1 Processor -- start 2021-07-11T04:00:12.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e9fde700 1 -- start start 2021-07-11T04:00:12.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e9fde700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f14e40b6900 0x7f14e40acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e9fde700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14e4060910 0x7f14e40a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e9fde700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e40b6ce0 con 0x7f14e40aa3b0 2021-07-11T04:00:12.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e9fde700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e40611e0 con 0x7f14e40b6900 2021-07-11T04:00:12.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e9fde700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e4061480 con 0x7f14e4060910 2021-07-11T04:00:12.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e3fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f14e40aa3b0 0x7f14e40b1880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43798/0 (socket says 172.21.2.104:43798) 2021-07-11T04:00:12.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e3fff700 1 -- 172.21.2.104:0/1583130183 learned_addr learned my addr 172.21.2.104:0/1583130183 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:12.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e37fe700 1 --2- 172.21.2.104:0/1583130183 >> v2:172.21.2.104:3300/0 conn(0x7f14e40b6900 0x7f14e40acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:12.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e37fe700 1 -- 172.21.2.104:0/1583130183 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14e4060910 msgr2=0x7f14e40a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:12.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e27fc700 1 -- 172.21.2.104:0/1583130183 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408969978 0 0) 0x7f14e40b6ce0 con 0x7f14e40aa3b0 2021-07-11T04:00:12.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e37fe700 1 --2- 172.21.2.104:0/1583130183 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14e4060910 0x7f14e40a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e37fe700 1 -- 172.21.2.104:0/1583130183 >> v1:172.21.2.104:6789/0 conn(0x7f14e40aa3b0 legacy=0x7f14e40b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e37fe700 1 -- 172.21.2.104:0/1583130183 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e4061720 con 0x7f14e40b6900 2021-07-11T04:00:12.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e37fe700 1 --2- 172.21.2.104:0/1583130183 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14e40b6900 0x7f14e40acd50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f14d4002f70 tx=0x7f14d4007f00).ready entity=mon.0 client_cookie=1cb199eead1c2b2c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:12.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e27fc700 1 -- 172.21.2.104:0/1583130183 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14d400a500 con 0x7f14e40b6900 2021-07-11T04:00:12.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e27fc700 1 -- 172.21.2.104:0/1583130183 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d4000e90 con 0x7f14e40b6900 2021-07-11T04:00:12.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e27fc700 1 -- 172.21.2.104:0/1583130183 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14d400a850 con 0x7f14e40b6900 2021-07-11T04:00:12.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e9fde700 1 -- 172.21.2.104:0/1583130183 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14e40b6900 msgr2=0x7f14e40acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.645+0000 7f14e9fde700 1 --2- 172.21.2.104:0/1583130183 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14e40b6900 0x7f14e40acd50 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f14d4002f70 tx=0x7f14d4007f00).stop 2021-07-11T04:00:12.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 -- 172.21.2.104:0/1583130183 shutdown_connections 2021-07-11T04:00:12.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 --2- 172.21.2.104:0/1583130183 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14e4060910 0x7f14e40a9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 --2- 172.21.2.104:0/1583130183 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14e40b6900 0x7f14e40acd50 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:12.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 -- 172.21.2.104:0/1583130183 >> 172.21.2.104:0/1583130183 conn(0x7f14e40b3480 msgr2=0x7f14e40b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:12.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 -- 172.21.2.104:0/1583130183 shutdown_connections 2021-07-11T04:00:12.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 -- 172.21.2.104:0/1583130183 wait complete. 2021-07-11T04:00:12.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 Processor -- start 2021-07-11T04:00:12.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 -- start start 2021-07-11T04:00:12.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14e4060910 0x7f14e413ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14e40aa3b0 0x7f14e413e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14e40b6900 0x7f14e41564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e413eca0 con 0x7f14e40b6900 2021-07-11T04:00:12.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e413c4e0 con 0x7f14e40aa3b0 2021-07-11T04:00:12.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e413c780 con 0x7f14e4060910 2021-07-11T04:00:12.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14e40b6900 0x7f14e41564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:12.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14e40b6900 0x7f14e41564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47038/0 (socket says 172.21.2.104:47038) 2021-07-11T04:00:12.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e3fff700 1 -- 172.21.2.104:0/3582683658 learned_addr learned my addr 172.21.2.104:0/3582683658 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:12.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e3fff700 1 -- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14e4060910 msgr2=0x7f14e413ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e3fff700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14e4060910 0x7f14e413ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e3fff700 1 -- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14e40aa3b0 msgr2=0x7f14e413e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e3fff700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14e40aa3b0 0x7f14e413e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e3fff700 1 -- 172.21.2.104:0/3582683658 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d400c200 con 0x7f14e40b6900 2021-07-11T04:00:12.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e3fff700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14e40b6900 0x7f14e41564a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f14dc001250 tx=0x7f14dc007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:12.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e0ff9700 1 -- 172.21.2.104:0/3582683658 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14dc005b40 con 0x7f14e40b6900 2021-07-11T04:00:12.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e0ff9700 1 -- 172.21.2.104:0/3582683658 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc005ca0 con 0x7f14e40b6900 2021-07-11T04:00:12.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 -- 172.21.2.104:0/3582683658 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e413e310 con 0x7f14e40b6900 2021-07-11T04:00:12.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e0ff9700 1 -- 172.21.2.104:0/3582683658 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14dc0048a0 con 0x7f14e40b6900 2021-07-11T04:00:12.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 -- 172.21.2.104:0/3582683658 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e4156990 con 0x7f14e40b6900 2021-07-11T04:00:12.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e9fde700 1 -- 172.21.2.104:0/3582683658 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e413c940 con 0x7f14e40b6900 2021-07-11T04:00:12.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e0ff9700 1 -- 172.21.2.104:0/3582683658 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f14dc01b070 con 0x7f14e40b6900 2021-07-11T04:00:12.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e0ff9700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14c8053270 0x7f14c8055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.649+0000 7f14e0ff9700 1 -- 172.21.2.104:0/3582683658 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f14dc06aaf0 con 0x7f14e40b6900 2021-07-11T04:00:12.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.653+0000 7f14e37fe700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14c8053270 0x7f14c8055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:12.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.653+0000 7f14e37fe700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14c8053270 0x7f14c8055730 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f14d4002f70 tx=0x7f14d4016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:12.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.653+0000 7f14e0ff9700 1 -- 172.21.2.104:0/3582683658 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f14dc042160 con 0x7f14e40b6900 2021-07-11T04:00:12.720 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:00:12.722 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:00:12.725 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:00:12.729 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:00:12.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.765+0000 7f14e9fde700 1 -- 172.21.2.104:0/3582683658 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f14e413e4a0 con 0x7f14e40b6900 2021-07-11T04:00:12.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.765+0000 7f14e0ff9700 1 -- 172.21.2.104:0/3582683658 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f14dc045780 con 0x7f14e40b6900 2021-07-11T04:00:12.771 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:12.771 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:00:12.772 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T04:00:12.772 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.772 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T04:00:12.772 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T04:00:12.772 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.773 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:12.773 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.773 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.773 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.773 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.774 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.774 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.774 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T04:00:12.774 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T04:00:12.775 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.775 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:12.775 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.775 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.776 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.776 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.776 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.776 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.776 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T04:00:12.777 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T04:00:12.781 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.781 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:12.781 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.782 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.782 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.782 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.782 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.782 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.783 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T04:00:12.783 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T04:00:12.783 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.783 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.784 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.784 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.784 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.784 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.784 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.785 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T04:00:12.785 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T04:00:12.785 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.785 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.785 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.786 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.786 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.786 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.786 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.787 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T04:00:12.787 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T04:00:12.787 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.787 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.787 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.788 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.788 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.788 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.788 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.788 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T04:00:12.789 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T04:00:12.789 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.789 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.789 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.790 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.790 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.790 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.790 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.791 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T04:00:12.814 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T04:00:12.815 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.815 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.815 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.815 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.816 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.816 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.816 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.816 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T04:00:12.816 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T04:00:12.817 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.817 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.817 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.817 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.817 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.818 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.818 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.818 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T04:00:12.818 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T04:00:12.819 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.819 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.819 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.819 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.819 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.820 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.820 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.820 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T04:00:12.820 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T04:00:12.820 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.821 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.821 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.821 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.821 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.821 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.822 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.822 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T04:00:12.822 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T04:00:12.823 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.823 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:12.823 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:00:12.823 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:00:12.824 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.824 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.825 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.825 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T04:00:12.825 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T04:00:12.825 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.826 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:12.826 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:00:12.826 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:00:12.826 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.827 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.827 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.827 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T04:00:12.827 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T04:00:12.827 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.828 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:00:12.828 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:12.828 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:12.829 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.829 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.829 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.829 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.829 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T04:00:12.830 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T04:00:12.830 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.830 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:00:12.830 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:00:12.830 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.831 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.831 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:12.831 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:12.831 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T04:00:12.832 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T04:00:12.832 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:12.832 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:00:12.832 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:00:12.832 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:12.833 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.833 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:12.833 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:12.833 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:00:12.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.765+0000 7f14e9fde700 1 -- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14c8053270 msgr2=0x7f14c8055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.765+0000 7f14e9fde700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14c8053270 0x7f14c8055730 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f14d4002f70 tx=0x7f14d4016000).stop 2021-07-11T04:00:12.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.765+0000 7f14e9fde700 1 -- 172.21.2.104:0/3582683658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14e40b6900 msgr2=0x7f14e41564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.765+0000 7f14e9fde700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14e40b6900 0x7f14e41564a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f14dc001250 tx=0x7f14dc007da0).stop 2021-07-11T04:00:12.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.769+0000 7f14e9fde700 1 -- 172.21.2.104:0/3582683658 shutdown_connections 2021-07-11T04:00:12.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.769+0000 7f14e9fde700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f14c8053270 0x7f14c8055730 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:12.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.769+0000 7f14e9fde700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14e40b6900 0x7f14e41564a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:12.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.769+0000 7f14e9fde700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14e40aa3b0 0x7f14e413e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.769+0000 7f14e9fde700 1 --2- 172.21.2.104:0/3582683658 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f14e4060910 0x7f14e413ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.769+0000 7f14e9fde700 1 -- 172.21.2.104:0/3582683658 >> 172.21.2.104:0/3582683658 conn(0x7f14e40b3480 msgr2=0x7f14e40b3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:12.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.769+0000 7f14e9fde700 1 -- 172.21.2.104:0/3582683658 shutdown_connections 2021-07-11T04:00:12.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.769+0000 7f14e9fde700 1 -- 172.21.2.104:0/3582683658 wait complete. 2021-07-11T04:00:12.840 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:12.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 Processor -- start 2021-07-11T04:00:12.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 -- start start 2021-07-11T04:00:12.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc7c80b2660 0x7fc7c805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc7c8097130 0x7fc7c805c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80b2a40 con 0x7fc7c8096cc0 2021-07-11T04:00:12.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c8097b80 con 0x7fc7c80b2660 2021-07-11T04:00:12.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c8097e20 con 0x7fc7c8097130 2021-07-11T04:00:12.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c6ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc7c8096cc0 0x7fc7c813f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43812/0 (socket says 172.21.2.104:43812) 2021-07-11T04:00:12.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c6ffd700 1 -- 172.21.2.104:0/1343146628 learned_addr learned my addr 172.21.2.104:0/1343146628 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:12.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c77fe700 1 --2- 172.21.2.104:0/1343146628 >> v2:172.21.2.104:3300/0 conn(0x7fc7c80b2660 0x7fc7c805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:12.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c7fff700 1 --2- 172.21.2.104:0/1343146628 >> v2:172.21.2.105:3300/0 conn(0x7fc7c8097130 0x7fc7c805c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:12.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c67fc700 1 -- 172.21.2.104:0/1343146628 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998015676 0 0) 0x7fc7c80b2a40 con 0x7fc7c8096cc0 2021-07-11T04:00:12.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c67fc700 1 -- 172.21.2.104:0/1343146628 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a8003260 con 0x7fc7c8096cc0 2021-07-11T04:00:12.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c77fe700 1 -- 172.21.2.104:0/1343146628 >> v2:172.21.2.105:3300/0 conn(0x7fc7c8097130 msgr2=0x7fc7c805c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:12.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c77fe700 1 --2- 172.21.2.104:0/1343146628 >> v2:172.21.2.105:3300/0 conn(0x7fc7c8097130 0x7fc7c805c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:12.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c77fe700 1 -- 172.21.2.104:0/1343146628 >> v1:172.21.2.104:6789/0 conn(0x7fc7c8096cc0 legacy=0x7fc7c813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c77fe700 1 -- 172.21.2.104:0/1343146628 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80980c0 con 0x7fc7c80b2660 2021-07-11T04:00:12.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c77fe700 1 --2- 172.21.2.104:0/1343146628 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc7c80b2660 0x7fc7c805f6b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7fc7b8001110 tx=0x7fc7b0006df0).ready entity=mon.0 client_cookie=bfb08e2d41cd3378 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:12.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c67fc700 1 -- 172.21.2.104:0/1343146628 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7b000d570 con 0x7fc7c80b2660 2021-07-11T04:00:12.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c67fc700 1 -- 172.21.2.104:0/1343146628 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b0000e90 con 0x7fc7c80b2660 2021-07-11T04:00:12.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7c67fc700 1 -- 172.21.2.104:0/1343146628 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7b000d8c0 con 0x7fc7c80b2660 2021-07-11T04:00:12.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 -- 172.21.2.104:0/1343146628 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc7c80b2660 msgr2=0x7fc7c805f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 --2- 172.21.2.104:0/1343146628 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc7c80b2660 0x7fc7c805f6b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7fc7b8001110 tx=0x7fc7b0006df0).stop 2021-07-11T04:00:12.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 -- 172.21.2.104:0/1343146628 shutdown_connections 2021-07-11T04:00:12.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 --2- 172.21.2.104:0/1343146628 >> v2:172.21.2.105:3300/0 conn(0x7fc7c8097130 0x7fc7c805c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:12.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 --2- 172.21.2.104:0/1343146628 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc7c80b2660 0x7fc7c805f6b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:12.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 -- 172.21.2.104:0/1343146628 >> 172.21.2.104:0/1343146628 conn(0x7fc7c8062f10 msgr2=0x7fc7c80632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:12.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 -- 172.21.2.104:0/1343146628 shutdown_connections 2021-07-11T04:00:12.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 -- 172.21.2.104:0/1343146628 wait complete. 2021-07-11T04:00:12.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 Processor -- start 2021-07-11T04:00:12.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.897+0000 7fc7ce140700 1 -- start start 2021-07-11T04:00:12.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7ce140700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc7c8096cc0 0x7fc7c80b4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7ce140700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc7c8097130 0x7fc7c80aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7ce140700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc7c80b2660 0x7fc7c8159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7ce140700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c815ac00 con 0x7fc7c8096cc0 2021-07-11T04:00:12.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7ce140700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80af1a0 con 0x7fc7c80b2660 2021-07-11T04:00:12.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7ce140700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80af920 con 0x7fc7c8097130 2021-07-11T04:00:12.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc7c8096cc0 0x7fc7c80b4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:12.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc7c8096cc0 0x7fc7c80b4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47048/0 (socket says 172.21.2.104:47048) 2021-07-11T04:00:12.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c77fe700 1 -- 172.21.2.104:0/2873969793 learned_addr learned my addr 172.21.2.104:0/2873969793 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:12.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c77fe700 1 -- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc7c8097130 msgr2=0x7fc7c80aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c77fe700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc7c8097130 0x7fc7c80aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c77fe700 1 -- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc7c80b2660 msgr2=0x7fc7c8159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c77fe700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc7c80b2660 0x7fc7c8159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c77fe700 1 -- 172.21.2.104:0/2873969793 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b000b180 con 0x7fc7c8096cc0 2021-07-11T04:00:12.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c77fe700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc7c8096cc0 0x7fc7c80b4c30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7fc7c80af330 tx=0x7fc7b0014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:12.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c4ff9700 1 -- 172.21.2.104:0/2873969793 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7b0006b20 con 0x7fc7c8096cc0 2021-07-11T04:00:12.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c4ff9700 1 -- 172.21.2.104:0/2873969793 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b0002f70 con 0x7fc7c8096cc0 2021-07-11T04:00:12.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7ce140700 1 -- 172.21.2.104:0/2873969793 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c80afbc0 con 0x7fc7c8096cc0 2021-07-11T04:00:12.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7ce140700 1 -- 172.21.2.104:0/2873969793 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c815a1d0 con 0x7fc7c8096cc0 2021-07-11T04:00:12.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c4ff9700 1 -- 172.21.2.104:0/2873969793 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc7b000dea0 con 0x7fc7c8096cc0 2021-07-11T04:00:12.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7ce140700 1 -- 172.21.2.104:0/2873969793 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c80afd80 con 0x7fc7c8096cc0 2021-07-11T04:00:12.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c4ff9700 1 -- 172.21.2.104:0/2873969793 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc7b0025020 con 0x7fc7c8096cc0 2021-07-11T04:00:12.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c4ff9700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc7ac0531a0 0x7fc7ac055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:12.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c4ff9700 1 -- 172.21.2.104:0/2873969793 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc7b006c370 con 0x7fc7c8096cc0 2021-07-11T04:00:12.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.901+0000 7fc7c6ffd700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc7ac0531a0 0x7fc7ac055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:12.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.905+0000 7fc7c6ffd700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc7ac0531a0 0x7fc7ac055660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fc7b8000ed0 tx=0x7fc7b80076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:12.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.905+0000 7fc7c4ff9700 1 -- 172.21.2.104:0/2873969793 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc7b00439c0 con 0x7fc7c8096cc0 2021-07-11T04:00:12.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.985+0000 7fc7ce140700 1 -- 172.21.2.104:0/2873969793 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7c8057350 con 0x7fc7c8096cc0 2021-07-11T04:00:12.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.985+0000 7fc7c4ff9700 1 -- 172.21.2.104:0/2873969793 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+13182 (secure 0 0 0) 0x7fc7b0046fa0 con 0x7fc7c8096cc0 2021-07-11T04:00:12.992 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:12.992 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":82,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:12.525950+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:12.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 -- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc7ac0531a0 msgr2=0x7fc7ac055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc7ac0531a0 0x7fc7ac055660 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fc7b8000ed0 tx=0x7fc7b80076c0).stop 2021-07-11T04:00:12.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 -- 172.21.2.104:0/2873969793 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc7c8096cc0 msgr2=0x7fc7c80b4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:12.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc7c8096cc0 0x7fc7c80b4c30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7fc7c80af330 tx=0x7fc7b0014000).stop 2021-07-11T04:00:12.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 -- 172.21.2.104:0/2873969793 shutdown_connections 2021-07-11T04:00:12.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc7c80b2660 0x7fc7c8159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc7c8097130 0x7fc7c80aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:12.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc7ac0531a0 0x7fc7ac055660 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:12.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 --2- 172.21.2.104:0/2873969793 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc7c8096cc0 0x7fc7c80b4c30 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:12.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 -- 172.21.2.104:0/2873969793 >> 172.21.2.104:0/2873969793 conn(0x7fc7c8062f10 msgr2=0x7fc7c809b560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:12.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 -- 172.21.2.104:0/2873969793 shutdown_connections 2021-07-11T04:00:12.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:12.989+0000 7fc7ce140700 1 -- 172.21.2.104:0/2873969793 wait complete. 2021-07-11T04:00:12.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T04:00:13.000 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:00:13.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 Processor -- start 2021-07-11T04:00:13.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- start start 2021-07-11T04:00:13.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f858c060100 0x7f858c064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f858c0b38d0 0x7f858c065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858c0604e0 con 0x7f858c144440 2021-07-11T04:00:13.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858c0656f0 con 0x7f858c060100 2021-07-11T04:00:13.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858c065c80 con 0x7f858c0b38d0 2021-07-11T04:00:13.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f858c060100 0x7f858c064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f858c060100 0x7f858c064d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47056/0 (socket says 172.21.2.104:47056) 2021-07-11T04:00:13.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 -- 172.21.2.104:0/1213796246 learned_addr learned my addr 172.21.2.104:0/1213796246 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:13.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 -- 172.21.2.104:0/1213796246 >> v2:172.21.2.105:3300/0 conn(0x7f858c0b38d0 msgr2=0x7f858c065200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:13.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 --2- 172.21.2.104:0/1213796246 >> v2:172.21.2.105:3300/0 conn(0x7f858c0b38d0 0x7f858c065200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 -- 172.21.2.104:0/1213796246 >> v1:172.21.2.105:6789/0 conn(0x7f858c144440 legacy=0x7f858c144970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:13.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 -- 172.21.2.104:0/1213796246 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c0b23b0 con 0x7f858c060100 2021-07-11T04:00:13.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 --2- 172.21.2.104:0/1213796246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f858c060100 0x7f858c064d10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f858c05d460 tx=0x7f8578006eb0).ready entity=mon.0 client_cookie=5d767f26a7531cbe server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f858b7fe700 1 -- 172.21.2.104:0/1213796246 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857800d5c0 con 0x7f858c060100 2021-07-11T04:00:13.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f858b7fe700 1 -- 172.21.2.104:0/1213796246 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8578000e90 con 0x7f858c060100 2021-07-11T04:00:13.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- 172.21.2.104:0/1213796246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f858c060100 msgr2=0x7f858c064d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 --2- 172.21.2.104:0/1213796246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f858c060100 0x7f858c064d10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f858c05d460 tx=0x7f8578006eb0).stop 2021-07-11T04:00:13.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f858b7fe700 1 -- 172.21.2.104:0/1213796246 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f857800d910 con 0x7f858c060100 2021-07-11T04:00:13.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- 172.21.2.104:0/1213796246 shutdown_connections 2021-07-11T04:00:13.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 --2- 172.21.2.104:0/1213796246 >> v2:172.21.2.105:3300/0 conn(0x7f858c0b38d0 0x7f858c065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 --2- 172.21.2.104:0/1213796246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f858c060100 0x7f858c064d10 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- 172.21.2.104:0/1213796246 >> 172.21.2.104:0/1213796246 conn(0x7f858c05cf60 msgr2=0x7f858c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:13.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- 172.21.2.104:0/1213796246 shutdown_connections 2021-07-11T04:00:13.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- 172.21.2.104:0/1213796246 wait complete. 2021-07-11T04:00:13.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 Processor -- start 2021-07-11T04:00:13.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- start start 2021-07-11T04:00:13.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f858c060100 0x7f858c0b64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f858c0b38d0 0x7f858c0afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f858c0b14f0 0x7f858c13a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c13c7c0 con 0x7f858c060100 2021-07-11T04:00:13.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c05ef30 con 0x7f858c0b14f0 2021-07-11T04:00:13.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c13b940 con 0x7f858c0b38d0 2021-07-11T04:00:13.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f858c060100 0x7f858c0b64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f858c060100 0x7f858c0b64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47062/0 (socket says 172.21.2.104:47062) 2021-07-11T04:00:13.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 -- 172.21.2.104:0/2633444751 learned_addr learned my addr 172.21.2.104:0/2633444751 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:13.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 -- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f858c0b38d0 msgr2=0x7f858c0afbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:13.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f858c0b38d0 0x7f858c0afbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 -- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f858c0b14f0 msgr2=0x7f858c13a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f858c0b14f0 0x7f858c13a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 -- 172.21.2.104:0/2633444751 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857800b1e0 con 0x7f858c060100 2021-07-11T04:00:13.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8590892700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f858c060100 0x7f858c0b64b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f858c05d460 tx=0x7f8578015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8589ffb700 1 -- 172.21.2.104:0/2633444751 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8578021070 con 0x7f858c060100 2021-07-11T04:00:13.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8589ffb700 1 -- 172.21.2.104:0/2633444751 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8578000e90 con 0x7f858c060100 2021-07-11T04:00:13.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8592af6700 1 -- 172.21.2.104:0/2633444751 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858c0b0fc0 con 0x7f858c060100 2021-07-11T04:00:13.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.053+0000 7f8589ffb700 1 -- 172.21.2.104:0/2633444751 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8578002f50 con 0x7f858c060100 2021-07-11T04:00:13.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.057+0000 7f8592af6700 1 -- 172.21.2.104:0/2633444751 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858c13bba0 con 0x7f858c060100 2021-07-11T04:00:13.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.057+0000 7f8592af6700 1 -- 172.21.2.104:0/2633444751 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858c13bd60 con 0x7f858c060100 2021-07-11T04:00:13.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.057+0000 7f8589ffb700 1 -- 172.21.2.104:0/2633444751 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f857800f070 con 0x7f858c060100 2021-07-11T04:00:13.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.057+0000 7f8589ffb700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8574053300 0x7f85740557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.057+0000 7f8589ffb700 1 -- 172.21.2.104:0/2633444751 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f857806c590 con 0x7f858c060100 2021-07-11T04:00:13.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.057+0000 7f858bfff700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8574053300 0x7f85740557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.057+0000 7f858bfff700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8574053300 0x7f85740557c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f8580000ed0 tx=0x7f85800068b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.057+0000 7f8589ffb700 1 -- 172.21.2.104:0/2633444751 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8578043c00 con 0x7f858c060100 2021-07-11T04:00:13.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 -- 172.21.2.104:0/2633444751 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f858c057350 con 0x7f858c060100 2021-07-11T04:00:13.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8589ffb700 1 -- 172.21.2.104:0/2633444751 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v82) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8578047220 con 0x7f858c060100 2021-07-11T04:00:13.146 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:00:13.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 -- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8574053300 msgr2=0x7f85740557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8574053300 0x7f85740557c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f8580000ed0 tx=0x7f85800068b0).stop 2021-07-11T04:00:13.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 -- 172.21.2.104:0/2633444751 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f858c060100 msgr2=0x7f858c0b64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f858c060100 0x7f858c0b64b0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f858c05d460 tx=0x7f8578015000).stop 2021-07-11T04:00:13.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 -- 172.21.2.104:0/2633444751 shutdown_connections 2021-07-11T04:00:13.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f858c0b14f0 0x7f858c13a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f858c0b38d0 0x7f858c0afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8574053300 0x7f85740557c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 --2- 172.21.2.104:0/2633444751 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f858c060100 0x7f858c0b64b0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 -- 172.21.2.104:0/2633444751 >> 172.21.2.104:0/2633444751 conn(0x7f858c05cf60 msgr2=0x7f858c0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:13.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 -- 172.21.2.104:0/2633444751 shutdown_connections 2021-07-11T04:00:13.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.141+0000 7f8592af6700 1 -- 172.21.2.104:0/2633444751 wait complete. 2021-07-11T04:00:13.153 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:00:13.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 Processor -- start 2021-07-11T04:00:13.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 -- start start 2021-07-11T04:00:13.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcd440b05b0 0x7fcd440b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd44143920 con 0x7fcd440b3710 2021-07-11T04:00:13.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd44143bc0 con 0x7fcd440a9c10 2021-07-11T04:00:13.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd44143e60 con 0x7fcd440b05b0 2021-07-11T04:00:13.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4965f700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fcd440b3710 0x7fcd4413d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43840/0 (socket says 172.21.2.104:43840) 2021-07-11T04:00:13.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4965f700 1 -- 172.21.2.104:0/2045765865 learned_addr learned my addr 172.21.2.104:0/2045765865 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:13.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd49e60700 1 --2- 172.21.2.104:0/2045765865 >> v2:172.21.2.104:3300/0 conn(0x7fcd440b05b0 0x7fcd440b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd49e60700 1 -- 172.21.2.104:0/2045765865 >> v1:172.21.2.105:6789/0 conn(0x7fcd440a9c10 legacy=0x7fcd440b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:13.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd49e60700 1 -- 172.21.2.104:0/2045765865 >> v1:172.21.2.104:6789/0 conn(0x7fcd440b3710 legacy=0x7fcd4413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd49e60700 1 -- 172.21.2.104:0/2045765865 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd44144100 con 0x7fcd440b05b0 2021-07-11T04:00:13.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd49e60700 1 --2- 172.21.2.104:0/2045765865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd440b05b0 0x7fcd440b0990 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fcd4406aa90 tx=0x7fcd38006df0).ready entity=mon.0 client_cookie=4c8d142c521bef7c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd48e5e700 1 -- 172.21.2.104:0/2045765865 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd38014070 con 0x7fcd440b05b0 2021-07-11T04:00:13.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd48e5e700 1 -- 172.21.2.104:0/2045765865 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd38000e90 con 0x7fcd440b05b0 2021-07-11T04:00:13.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd48e5e700 1 -- 172.21.2.104:0/2045765865 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd3800fbb0 con 0x7fcd440b05b0 2021-07-11T04:00:13.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/2045765865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd440b05b0 msgr2=0x7fcd440b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 --2- 172.21.2.104:0/2045765865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd440b05b0 0x7fcd440b0990 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fcd4406aa90 tx=0x7fcd38006df0).stop 2021-07-11T04:00:13.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/2045765865 shutdown_connections 2021-07-11T04:00:13.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 --2- 172.21.2.104:0/2045765865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd440b05b0 0x7fcd440b0990 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/2045765865 >> 172.21.2.104:0/2045765865 conn(0x7fcd440607c0 msgr2=0x7fcd44060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:13.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/2045765865 shutdown_connections 2021-07-11T04:00:13.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.205+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/2045765865 wait complete. 2021-07-11T04:00:13.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4c0c4700 1 Processor -- start 2021-07-11T04:00:13.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4c0c4700 1 -- start start 2021-07-11T04:00:13.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4c0c4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd440a9c10 0x7fcd44141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4c0c4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd440b05b0 0x7fcd4414dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4c0c4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd440b3710 0x7fcd4414e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4c0c4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4414e9b0 con 0x7fcd440a9c10 2021-07-11T04:00:13.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4c0c4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd44140030 con 0x7fcd440b3710 2021-07-11T04:00:13.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4c0c4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd441407c0 con 0x7fcd440b05b0 2021-07-11T04:00:13.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd49e60700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd440a9c10 0x7fcd44141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd49e60700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd440a9c10 0x7fcd44141710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47076/0 (socket says 172.21.2.104:47076) 2021-07-11T04:00:13.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd49e60700 1 -- 172.21.2.104:0/482853147 learned_addr learned my addr 172.21.2.104:0/482853147 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:13.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd49e60700 1 -- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd440b05b0 msgr2=0x7fcd4414dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd49e60700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd440b05b0 0x7fcd4414dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd49e60700 1 -- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd440b3710 msgr2=0x7fcd4414e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd49e60700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd440b3710 0x7fcd4414e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd49e60700 1 -- 172.21.2.104:0/482853147 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3800b180 con 0x7fcd440a9c10 2021-07-11T04:00:13.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd49e60700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd440a9c10 0x7fcd44141710 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fcd4406aa90 tx=0x7fcd38014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd36ffd700 1 -- 172.21.2.104:0/482853147 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd38002f70 con 0x7fcd440a9c10 2021-07-11T04:00:13.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd36ffd700 1 -- 172.21.2.104:0/482853147 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd38010d30 con 0x7fcd440a9c10 2021-07-11T04:00:13.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd36ffd700 1 -- 172.21.2.104:0/482853147 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd38010ea0 con 0x7fcd440a9c10 2021-07-11T04:00:13.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/482853147 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd440ad790 con 0x7fcd440a9c10 2021-07-11T04:00:13.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/482853147 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd44140c30 con 0x7fcd440a9c10 2021-07-11T04:00:13.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/482853147 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd441401f0 con 0x7fcd440a9c10 2021-07-11T04:00:13.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd36ffd700 1 -- 172.21.2.104:0/482853147 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcd3801b430 con 0x7fcd440a9c10 2021-07-11T04:00:13.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd36ffd700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd24053380 0x7fcd24055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd36ffd700 1 -- 172.21.2.104:0/482853147 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcd3806c5c0 con 0x7fcd440a9c10 2021-07-11T04:00:13.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.209+0000 7fcd4965f700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd24053380 0x7fcd24055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.213+0000 7fcd4965f700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd24053380 0x7fcd24055840 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fcd4406aa50 tx=0x7fcd40007860).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.213+0000 7fcd36ffd700 1 -- 172.21.2.104:0/482853147 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcd38043ac0 con 0x7fcd440a9c10 2021-07-11T04:00:13.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.293+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/482853147 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcd44140980 con 0x7fcd440a9c10 2021-07-11T04:00:13.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd36ffd700 1 -- 172.21.2.104:0/482853147 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v82) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcd3801f0a0 con 0x7fcd440a9c10 2021-07-11T04:00:13.303 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:13.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd24053380 msgr2=0x7fcd24055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd24053380 0x7fcd24055840 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fcd4406aa50 tx=0x7fcd40007860).stop 2021-07-11T04:00:13.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/482853147 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd440a9c10 msgr2=0x7fcd44141710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd440a9c10 0x7fcd44141710 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7fcd4406aa90 tx=0x7fcd38014000).stop 2021-07-11T04:00:13.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/482853147 shutdown_connections 2021-07-11T04:00:13.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd440b3710 0x7fcd4414e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd440b05b0 0x7fcd4414dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd24053380 0x7fcd24055840 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 --2- 172.21.2.104:0/482853147 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd440a9c10 0x7fcd44141710 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/482853147 >> 172.21.2.104:0/482853147 conn(0x7fcd440607c0 msgr2=0x7fcd440b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:13.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/482853147 shutdown_connections 2021-07-11T04:00:13.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.297+0000 7fcd4c0c4700 1 -- 172.21.2.104:0/482853147 wait complete. 2021-07-11T04:00:13.314 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T04:00:13.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 Processor -- start 2021-07-11T04:00:13.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 -- start start 2021-07-11T04:00:13.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1d1c1483a0 0x7f1d1c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 0x7f1d1c144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1c0b0350 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c0a90c0 con 0x7f1d1c1483a0 2021-07-11T04:00:13.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c0b05f0 con 0x7f1d1c0a70c0 2021-07-11T04:00:13.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d1ad9d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 0x7f1d1c144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d1ad9d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 0x7f1d1c144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:50248/0 (socket says 172.21.2.104:50248) 2021-07-11T04:00:13.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d19d9b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1d1c1483a0 0x7f1d1c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d1ad9d700 1 -- 172.21.2.104:0/1447615380 learned_addr learned my addr 172.21.2.104:0/1447615380 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:13.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d1ad9d700 1 -- 172.21.2.104:0/1447615380 >> v2:172.21.2.105:3300/0 conn(0x7f1d1c1483a0 msgr2=0x7f1d1c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:13.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d1ad9d700 1 --2- 172.21.2.104:0/1447615380 >> v2:172.21.2.105:3300/0 conn(0x7f1d1c1483a0 0x7f1d1c0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d1ad9d700 1 -- 172.21.2.104:0/1447615380 >> v1:172.21.2.105:6789/0 conn(0x7f1d1c0b1db0 legacy=0x7f1d1c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d1ad9d700 1 -- 172.21.2.104:0/1447615380 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c0b0890 con 0x7f1d1c0a70c0 2021-07-11T04:00:13.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d1ad9d700 1 --2- 172.21.2.104:0/1447615380 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 0x7f1d1c144720 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f1d1c147730 tx=0x7f1d14009290).ready entity=mon.2 client_cookie=1cd570f48e80650d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d1959a700 1 -- 172.21.2.104:0/1447615380 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1400b860 con 0x7f1d1c0a70c0 2021-07-11T04:00:13.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d1959a700 1 -- 172.21.2.104:0/1447615380 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1400b9c0 con 0x7f1d1c0a70c0 2021-07-11T04:00:13.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d1959a700 1 -- 172.21.2.104:0/1447615380 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d1400bb30 con 0x7f1d1c0a70c0 2021-07-11T04:00:13.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 -- 172.21.2.104:0/1447615380 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 msgr2=0x7f1d1c144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 --2- 172.21.2.104:0/1447615380 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 0x7f1d1c144720 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f1d1c147730 tx=0x7f1d14009290).stop 2021-07-11T04:00:13.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 -- 172.21.2.104:0/1447615380 shutdown_connections 2021-07-11T04:00:13.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 --2- 172.21.2.104:0/1447615380 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 0x7f1d1c144720 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 --2- 172.21.2.104:0/1447615380 >> v2:172.21.2.105:3300/0 conn(0x7f1d1c1483a0 0x7f1d1c0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.361+0000 7f1d20f1c700 1 -- 172.21.2.104:0/1447615380 >> 172.21.2.104:0/1447615380 conn(0x7f1d1c05f6b0 msgr2=0x7f1d1c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:13.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 -- 172.21.2.104:0/1447615380 shutdown_connections 2021-07-11T04:00:13.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 -- 172.21.2.104:0/1447615380 wait complete. 2021-07-11T04:00:13.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 Processor -- start 2021-07-11T04:00:13.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 -- start start 2021-07-11T04:00:13.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 0x7f1d1c13fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d1c0b1db0 0x7f1d1c13d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d1c1483a0 0x7f1d1c13ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c146af0 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c13e2c0 con 0x7f1d1c1483a0 2021-07-11T04:00:13.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c147460 con 0x7f1d1c0a70c0 2021-07-11T04:00:13.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d19d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d1c0b1db0 0x7f1d1c13d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d19d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d1c0b1db0 0x7f1d1c13d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47092/0 (socket says 172.21.2.104:47092) 2021-07-11T04:00:13.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d19d9b700 1 -- 172.21.2.104:0/87855607 learned_addr learned my addr 172.21.2.104:0/87855607 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:13.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d1ad9d700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d1c1483a0 0x7f1d1c13ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d1a59c700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 0x7f1d1c13fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d19d9b700 1 -- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 msgr2=0x7f1d1c13fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d19d9b700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 0x7f1d1c13fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d19d9b700 1 -- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d1c1483a0 msgr2=0x7f1d1c13ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d19d9b700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d1c1483a0 0x7f1d1c13ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d19d9b700 1 -- 172.21.2.104:0/87855607 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d14009530 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d19d9b700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d1c0b1db0 0x7f1d1c13d8e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f1d1000c000 tx=0x7f1d10006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d0b7fe700 1 -- 172.21.2.104:0/87855607 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d10011750 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d0b7fe700 1 -- 172.21.2.104:0/87855607 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d100118b0 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 -- 172.21.2.104:0/87855607 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c146040 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d0b7fe700 1 -- 172.21.2.104:0/87855607 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d10011c10 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 -- 172.21.2.104:0/87855607 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1c146fc0 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d20f1c700 1 -- 172.21.2.104:0/87855607 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d1c057350 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d0b7fe700 1 -- 172.21.2.104:0/87855607 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1d10011d70 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d0b7fe700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf8053300 0x7f1cf80557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d0b7fe700 1 -- 172.21.2.104:0/87855607 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1d1006b4d0 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d1a59c700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf8053300 0x7f1cf80557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.365+0000 7f1d1a59c700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf8053300 0x7f1cf80557c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f1d0c00d000 tx=0x7f1d0c007b90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.369+0000 7f1d0b7fe700 1 -- 172.21.2.104:0/87855607 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1d10042b40 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 -- 172.21.2.104:0/87855607 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1d1c044650 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d0b7fe700 1 -- 172.21.2.104:0/87855607 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+143 (secure 0 0 0) 0x7f1d10046160 con 0x7f1d1c0b1db0 2021-07-11T04:00:13.458 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:13.459 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T04:00:13.459 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:13.459 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T04:00:13.459 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T04:00:13.460 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:13.460 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:13.460 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T04:00:13.460 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T04:00:13.461 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:13.461 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T04:00:13.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 -- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf8053300 msgr2=0x7f1cf80557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf8053300 0x7f1cf80557c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f1d0c00d000 tx=0x7f1d0c007b90).stop 2021-07-11T04:00:13.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 -- 172.21.2.104:0/87855607 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d1c0b1db0 msgr2=0x7f1d1c13d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d1c0b1db0 0x7f1d1c13d8e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f1d1000c000 tx=0x7f1d10006df0).stop 2021-07-11T04:00:13.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 -- 172.21.2.104:0/87855607 shutdown_connections 2021-07-11T04:00:13.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d1c1483a0 0x7f1d1c13ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cf8053300 0x7f1cf80557c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d1c0b1db0 0x7f1d1c13d8e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 --2- 172.21.2.104:0/87855607 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d1c0a70c0 0x7f1d1c13fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 -- 172.21.2.104:0/87855607 >> 172.21.2.104:0/87855607 conn(0x7f1d1c05f6b0 msgr2=0x7f1d1c0b4980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:13.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 -- 172.21.2.104:0/87855607 shutdown_connections 2021-07-11T04:00:13.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.453+0000 7f1d20f1c700 1 -- 172.21.2.104:0/87855607 wait complete. 2021-07-11T04:00:13.473 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T04:00:13.474 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T04:00:13.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 Processor -- start 2021-07-11T04:00:13.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- start start 2021-07-11T04:00:13.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fccf00b1ec0 0x7fccf00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf00affb0 0x7fccf00b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccf00b0420 con 0x7fccf00a70c0 2021-07-11T04:00:13.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccf00b2790 con 0x7fccf00b1ec0 2021-07-11T04:00:13.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf00b0650 con 0x7fccf00affb0 2021-07-11T04:00:13.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccef7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fccf00b1ec0 0x7fccf00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccef7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fccf00b1ec0 0x7fccf00b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47098/0 (socket says 172.21.2.104:47098) 2021-07-11T04:00:13.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccef7fe700 1 -- 172.21.2.104:0/810046297 learned_addr learned my addr 172.21.2.104:0/810046297 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:13.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccef7fe700 1 -- 172.21.2.104:0/810046297 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf00affb0 msgr2=0x7fccf00b22a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:13.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccef7fe700 1 --2- 172.21.2.104:0/810046297 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf00affb0 0x7fccf00b22a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccef7fe700 1 -- 172.21.2.104:0/810046297 >> v1:172.21.2.105:6789/0 conn(0x7fccf00a70c0 legacy=0x7fccf0141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:13.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccef7fe700 1 -- 172.21.2.104:0/810046297 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf00b08f0 con 0x7fccf00b1ec0 2021-07-11T04:00:13.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccef7fe700 1 --2- 172.21.2.104:0/810046297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf00b1ec0 0x7fccf00b12d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7fccf006a0f0 tx=0x7fcce4006eb0).ready entity=mon.0 client_cookie=6458b90fe85a1c27 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccee7fc700 1 -- 172.21.2.104:0/810046297 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcce400d570 con 0x7fccf00b1ec0 2021-07-11T04:00:13.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccee7fc700 1 -- 172.21.2.104:0/810046297 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce4000e90 con 0x7fccf00b1ec0 2021-07-11T04:00:13.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccee7fc700 1 -- 172.21.2.104:0/810046297 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcce400d8c0 con 0x7fccf00b1ec0 2021-07-11T04:00:13.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- 172.21.2.104:0/810046297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf00b1ec0 msgr2=0x7fccf00b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 --2- 172.21.2.104:0/810046297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf00b1ec0 0x7fccf00b12d0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7fccf006a0f0 tx=0x7fcce4006eb0).stop 2021-07-11T04:00:13.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- 172.21.2.104:0/810046297 shutdown_connections 2021-07-11T04:00:13.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 --2- 172.21.2.104:0/810046297 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf00affb0 0x7fccf00b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 --2- 172.21.2.104:0/810046297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf00b1ec0 0x7fccf00b12d0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:13.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- 172.21.2.104:0/810046297 >> 172.21.2.104:0/810046297 conn(0x7fccf005f6b0 msgr2=0x7fccf005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:13.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- 172.21.2.104:0/810046297 shutdown_connections 2021-07-11T04:00:13.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- 172.21.2.104:0/810046297 wait complete. 2021-07-11T04:00:13.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 Processor -- start 2021-07-11T04:00:13.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- start start 2021-07-11T04:00:13.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf00a70c0 0x7fccf01400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccf00affb0 0x7fccf013b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf00b1ec0 0x7fccf013bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf015b0c0 con 0x7fccf00b1ec0 2021-07-11T04:00:13.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccf015abb0 con 0x7fccf00affb0 2021-07-11T04:00:13.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fccf600e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccf015ae20 con 0x7fccf00a70c0 2021-07-11T04:00:13.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fcceffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf00b1ec0 0x7fccf013bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fcceffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf00b1ec0 0x7fccf013bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47104/0 (socket says 172.21.2.104:47104) 2021-07-11T04:00:13.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fcceffff700 1 -- 172.21.2.104:0/1988398144 learned_addr learned my addr 172.21.2.104:0/1988398144 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:13.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fcceffff700 1 -- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf00a70c0 msgr2=0x7fccf01400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fcceffff700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf00a70c0 0x7fccf01400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fcceffff700 1 -- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccf00affb0 msgr2=0x7fccf013b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:13.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fcceffff700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccf00affb0 0x7fccf013b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:13.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fcceffff700 1 -- 172.21.2.104:0/1988398144 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce400b1e0 con 0x7fccf00b1ec0 2021-07-11T04:00:13.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.525+0000 7fcceffff700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf00b1ec0 0x7fccf013bd90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7fcce8003310 tx=0x7fcce80091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.529+0000 7fccecff9700 1 -- 172.21.2.104:0/1988398144 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcce8001500 con 0x7fccf00b1ec0 2021-07-11T04:00:13.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.529+0000 7fccecff9700 1 -- 172.21.2.104:0/1988398144 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce8001660 con 0x7fccf00b1ec0 2021-07-11T04:00:13.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.529+0000 7fccecff9700 1 -- 172.21.2.104:0/1988398144 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcce80019c0 con 0x7fccf00b1ec0 2021-07-11T04:00:13.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.529+0000 7fccf600e700 1 -- 172.21.2.104:0/1988398144 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccf01422f0 con 0x7fccf00b1ec0 2021-07-11T04:00:13.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.529+0000 7fccf600e700 1 -- 172.21.2.104:0/1988398144 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccf0142a30 con 0x7fccf00b1ec0 2021-07-11T04:00:13.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.529+0000 7fccf600e700 1 -- 172.21.2.104:0/1988398144 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf0057350 con 0x7fccf00b1ec0 2021-07-11T04:00:13.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.529+0000 7fccecff9700 1 -- 172.21.2.104:0/1988398144 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcce8018030 con 0x7fccf00b1ec0 2021-07-11T04:00:13.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.529+0000 7fccecff9700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053380 0x7fccd0055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:13.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.529+0000 7fccecff9700 1 -- 172.21.2.104:0/1988398144 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcce80694e0 con 0x7fccf00b1ec0 2021-07-11T04:00:13.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.529+0000 7fccef7fe700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053380 0x7fccd0055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:13.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.529+0000 7fccef7fe700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053380 0x7fccd0055840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fccf006a0f0 tx=0x7fcce4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:13.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.533+0000 7fccecff9700 1 -- 172.21.2.104:0/1988398144 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcce8042240 con 0x7fccf00b1ec0 2021-07-11T04:00:13.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:13.613+0000 7fccf600e700 1 -- 172.21.2.104:0/1988398144 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fccf00b6430 con 0x7fccf00b1ec0 2021-07-11T04:00:14.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccecff9700 1 -- 172.21.2.104:0/1988398144 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v83) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcce8045860 con 0x7fccf00b1ec0 2021-07-11T04:00:14.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 -- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053380 msgr2=0x7fccd0055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:14.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053380 0x7fccd0055840 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7fccf006a0f0 tx=0x7fcce4015000).stop 2021-07-11T04:00:14.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 -- 172.21.2.104:0/1988398144 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf00b1ec0 msgr2=0x7fccf013bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:14.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf00b1ec0 0x7fccf013bd90 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7fcce8003310 tx=0x7fcce80091a0).stop 2021-07-11T04:00:14.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 -- 172.21.2.104:0/1988398144 shutdown_connections 2021-07-11T04:00:14.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccd0053380 0x7fccd0055840 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:14.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccf00b1ec0 0x7fccf013bd90 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:14.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccf00affb0 0x7fccf013b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:14.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 --2- 172.21.2.104:0/1988398144 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccf00a70c0 0x7fccf01400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:14.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 -- 172.21.2.104:0/1988398144 >> 172.21.2.104:0/1988398144 conn(0x7fccf005f6b0 msgr2=0x7fccf01450c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:14.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 -- 172.21.2.104:0/1988398144 shutdown_connections 2021-07-11T04:00:14.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.541+0000 7fccf600e700 1 -- 172.21.2.104:0/1988398144 wait complete. 2021-07-11T04:00:14.553 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T04:00:14.555 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T04:00:14.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e69954700 1 Processor -- start 2021-07-11T04:00:14.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e69954700 1 -- start start 2021-07-11T04:00:14.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e69954700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2e64060100 0x7f2e64064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:14.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e69954700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e64144440 0x7f2e64065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:14.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e69954700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e640604e0 con 0x7f2e640b38d0 2021-07-11T04:00:14.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e69954700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e640656f0 con 0x7f2e64060100 2021-07-11T04:00:14.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e69954700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64065c80 con 0x7f2e64144440 2021-07-11T04:00:14.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e637fe700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f2e640b38d0 0x7f2e64144970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:43250/0 (socket says 172.21.2.104:43250) 2021-07-11T04:00:14.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e637fe700 1 -- 172.21.2.104:0/528475055 learned_addr learned my addr 172.21.2.104:0/528475055 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:14.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e627fc700 1 --2- 172.21.2.104:0/528475055 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e64144440 0x7f2e64065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:14.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e62ffd700 1 --2- 172.21.2.104:0/528475055 >> v2:172.21.2.105:3300/0 conn(0x7f2e64060100 0x7f2e64064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:14.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e627fc700 1 -- 172.21.2.104:0/528475055 >> v2:172.21.2.105:3300/0 conn(0x7f2e64060100 msgr2=0x7f2e64064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:14.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e627fc700 1 --2- 172.21.2.104:0/528475055 >> v2:172.21.2.105:3300/0 conn(0x7f2e64060100 0x7f2e64064d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:14.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e627fc700 1 -- 172.21.2.104:0/528475055 >> v1:172.21.2.105:6789/0 conn(0x7f2e640b38d0 legacy=0x7f2e64144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:14.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.617+0000 7f2e627fc700 1 -- 172.21.2.104:0/528475055 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e640b23b0 con 0x7f2e64144440 2021-07-11T04:00:14.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e627fc700 1 --2- 172.21.2.104:0/528475055 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e64144440 0x7f2e64065200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f2e4c002f70 tx=0x7f2e4c006df0).ready entity=mon.2 client_cookie=2a0e4c6f793fd52b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:14.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e61ffb700 1 -- 172.21.2.104:0/528475055 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e4c00d570 con 0x7f2e64144440 2021-07-11T04:00:14.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e61ffb700 1 -- 172.21.2.104:0/528475055 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c000e90 con 0x7f2e64144440 2021-07-11T04:00:14.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e61ffb700 1 -- 172.21.2.104:0/528475055 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e4c00d8c0 con 0x7f2e64144440 2021-07-11T04:00:14.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 -- 172.21.2.104:0/528475055 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e64144440 msgr2=0x7f2e64065200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:14.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 --2- 172.21.2.104:0/528475055 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e64144440 0x7f2e64065200 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f2e4c002f70 tx=0x7f2e4c006df0).stop 2021-07-11T04:00:14.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 -- 172.21.2.104:0/528475055 shutdown_connections 2021-07-11T04:00:14.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 --2- 172.21.2.104:0/528475055 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e64144440 0x7f2e64065200 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:14.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 --2- 172.21.2.104:0/528475055 >> v2:172.21.2.105:3300/0 conn(0x7f2e64060100 0x7f2e64064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:14.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 -- 172.21.2.104:0/528475055 >> 172.21.2.104:0/528475055 conn(0x7f2e6405cf60 msgr2=0x7f2e6405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:14.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 -- 172.21.2.104:0/528475055 shutdown_connections 2021-07-11T04:00:14.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 -- 172.21.2.104:0/528475055 wait complete. 2021-07-11T04:00:14.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 Processor -- start 2021-07-11T04:00:14.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 -- start start 2021-07-11T04:00:14.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e64060100 0x7f2e64143a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:14.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e640b38d0 0x7f2e641411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:14.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e6413e280 0x7f2e640afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:14.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640aab20 con 0x7f2e6413e280 2021-07-11T04:00:14.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640ade10 con 0x7f2e64060100 2021-07-11T04:00:14.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6413e800 con 0x7f2e640b38d0 2021-07-11T04:00:14.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e637fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e6413e280 0x7f2e640afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:14.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e637fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e6413e280 0x7f2e640afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47120/0 (socket says 172.21.2.104:47120) 2021-07-11T04:00:14.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e637fe700 1 -- 172.21.2.104:0/3589907562 learned_addr learned my addr 172.21.2.104:0/3589907562 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:14.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e637fe700 1 -- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e640b38d0 msgr2=0x7f2e641411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:14.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e637fe700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e640b38d0 0x7f2e641411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:14.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e637fe700 1 -- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e64060100 msgr2=0x7f2e64143a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:14.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e637fe700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e64060100 0x7f2e64143a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:14.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e637fe700 1 -- 172.21.2.104:0/3589907562 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c00b180 con 0x7f2e6413e280 2021-07-11T04:00:14.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e637fe700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e6413e280 0x7f2e640afe70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f2e64069be0 tx=0x7f2e5c007e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:14.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e5bfff700 1 -- 172.21.2.104:0/3589907562 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e5c005ba0 con 0x7f2e6413e280 2021-07-11T04:00:14.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e5bfff700 1 -- 172.21.2.104:0/3589907562 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5c005d00 con 0x7f2e6413e280 2021-07-11T04:00:14.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e5bfff700 1 -- 172.21.2.104:0/3589907562 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e5c0072c0 con 0x7f2e6413e280 2021-07-11T04:00:14.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 -- 172.21.2.104:0/3589907562 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e640ae060 con 0x7f2e6413e280 2021-07-11T04:00:14.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.621+0000 7f2e69954700 1 -- 172.21.2.104:0/3589907562 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e640b0d60 con 0x7f2e6413e280 2021-07-11T04:00:14.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.625+0000 7f2e69954700 1 -- 172.21.2.104:0/3589907562 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e64057350 con 0x7f2e6413e280 2021-07-11T04:00:14.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.625+0000 7f2e5bfff700 1 -- 172.21.2.104:0/3589907562 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2e5c01a060 con 0x7f2e6413e280 2021-07-11T04:00:14.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.625+0000 7f2e5bfff700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e48053260 0x7f2e48055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:14.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.625+0000 7f2e5bfff700 1 -- 172.21.2.104:0/3589907562 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f2e5c06ae60 con 0x7f2e6413e280 2021-07-11T04:00:14.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.625+0000 7f2e62ffd700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e48053260 0x7f2e48055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:14.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.625+0000 7f2e62ffd700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e48053260 0x7f2e48055720 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f2e5400c000 tx=0x7f2e54006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:14.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.625+0000 7f2e5bfff700 1 -- 172.21.2.104:0/3589907562 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2e5c042260 con 0x7f2e6413e280 2021-07-11T04:00:14.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:14.709+0000 7f2e69954700 1 -- 172.21.2.104:0/3589907562 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f2e640698e0 con 0x7f2e6413e280 2021-07-11T04:00:15.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.481+0000 7f2e5bfff700 1 -- 172.21.2.104:0/3589907562 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v84) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2e5c018030 con 0x7f2e6413e280 2021-07-11T04:00:15.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 -- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e48053260 msgr2=0x7f2e48055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e48053260 0x7f2e48055720 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f2e5400c000 tx=0x7f2e54006df0).stop 2021-07-11T04:00:15.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 -- 172.21.2.104:0/3589907562 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e6413e280 msgr2=0x7f2e640afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e6413e280 0x7f2e640afe70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f2e64069be0 tx=0x7f2e5c007e70).stop 2021-07-11T04:00:15.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 -- 172.21.2.104:0/3589907562 shutdown_connections 2021-07-11T04:00:15.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e48053260 0x7f2e48055720 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e6413e280 0x7f2e640afe70 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e640b38d0 0x7f2e641411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 --2- 172.21.2.104:0/3589907562 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e64060100 0x7f2e64143a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 -- 172.21.2.104:0/3589907562 >> 172.21.2.104:0/3589907562 conn(0x7f2e6405cf60 msgr2=0x7f2e6413ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:15.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 -- 172.21.2.104:0/3589907562 shutdown_connections 2021-07-11T04:00:15.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.485+0000 7f2e69954700 1 -- 172.21.2.104:0/3589907562 wait complete. 2021-07-11T04:00:15.493 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T04:00:15.497 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T04:00:15.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff516a72700 1 Processor -- start 2021-07-11T04:00:15.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff516a72700 1 -- start start 2021-07-11T04:00:15.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff516a72700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff510067120 0x7ff5100b96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff516a72700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff51013fff0 0x7ff510140480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff516a72700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5100b9bd0 con 0x7ff5101439b0 2021-07-11T04:00:15.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff516a72700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff510143080 con 0x7ff510067120 2021-07-11T04:00:15.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff516a72700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff510067500 con 0x7ff51013fff0 2021-07-11T04:00:15.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff50ffff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff5101439b0 0x7ff5100aec00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43898/0 (socket says 172.21.2.104:43898) 2021-07-11T04:00:15.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff50ffff700 1 -- 172.21.2.104:0/4217797040 learned_addr learned my addr 172.21.2.104:0/4217797040 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:15.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff51480e700 1 --2- 172.21.2.104:0/4217797040 >> v2:172.21.2.104:3300/0 conn(0x7ff510067120 0x7ff5100b96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff51500f700 1 --2- 172.21.2.104:0/4217797040 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff51013fff0 0x7ff510140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff51480e700 1 -- 172.21.2.104:0/4217797040 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff51013fff0 msgr2=0x7ff510140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:15.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff51480e700 1 --2- 172.21.2.104:0/4217797040 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff51013fff0 0x7ff510140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff51480e700 1 -- 172.21.2.104:0/4217797040 >> v1:172.21.2.104:6789/0 conn(0x7ff5101439b0 legacy=0x7ff5100aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff51480e700 1 -- 172.21.2.104:0/4217797040 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100af4d0 con 0x7ff510067120 2021-07-11T04:00:15.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff51480e700 1 --2- 172.21.2.104:0/4217797040 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff510067120 0x7ff5100b96e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7ff504002f70 tx=0x7ff504006df0).ready entity=mon.0 client_cookie=17ac007f5bf9bd4a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:15.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff50f7fe700 1 -- 172.21.2.104:0/4217797040 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff50400d570 con 0x7ff510067120 2021-07-11T04:00:15.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff50f7fe700 1 -- 172.21.2.104:0/4217797040 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff504000e90 con 0x7ff510067120 2021-07-11T04:00:15.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff50f7fe700 1 -- 172.21.2.104:0/4217797040 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff50400d8c0 con 0x7ff510067120 2021-07-11T04:00:15.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff516a72700 1 -- 172.21.2.104:0/4217797040 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff510067120 msgr2=0x7ff5100b96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.549+0000 7ff516a72700 1 --2- 172.21.2.104:0/4217797040 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff510067120 0x7ff5100b96e0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7ff504002f70 tx=0x7ff504006df0).stop 2021-07-11T04:00:15.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 -- 172.21.2.104:0/4217797040 shutdown_connections 2021-07-11T04:00:15.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 --2- 172.21.2.104:0/4217797040 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff51013fff0 0x7ff510140480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 --2- 172.21.2.104:0/4217797040 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff510067120 0x7ff5100b96e0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 -- 172.21.2.104:0/4217797040 >> 172.21.2.104:0/4217797040 conn(0x7ff51005f9f0 msgr2=0x7ff5100616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:15.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 -- 172.21.2.104:0/4217797040 shutdown_connections 2021-07-11T04:00:15.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 -- 172.21.2.104:0/4217797040 wait complete. 2021-07-11T04:00:15.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 Processor -- start 2021-07-11T04:00:15.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 -- start start 2021-07-11T04:00:15.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff510067120 0x7ff510145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff51013fff0 0x7ff5100b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff5101439b0 0x7ff510152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100b0430 con 0x7ff5101439b0 2021-07-11T04:00:15.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff510145830 con 0x7ff510067120 2021-07-11T04:00:15.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5101522e0 con 0x7ff51013fff0 2021-07-11T04:00:15.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff51500f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff5101439b0 0x7ff510152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff51500f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff5101439b0 0x7ff510152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47138/0 (socket says 172.21.2.104:47138) 2021-07-11T04:00:15.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff51500f700 1 -- 172.21.2.104:0/3119993521 learned_addr learned my addr 172.21.2.104:0/3119993521 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:15.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff51500f700 1 -- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff51013fff0 msgr2=0x7ff5100b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff51500f700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff51013fff0 0x7ff5100b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff51500f700 1 -- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff510067120 msgr2=0x7ff510145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff51500f700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff510067120 0x7ff510145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff51500f700 1 -- 172.21.2.104:0/3119993521 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50400b180 con 0x7ff5101439b0 2021-07-11T04:00:15.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff51500f700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff5101439b0 0x7ff510152cc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7ff508006e00 tx=0x7ff5080091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:15.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff50dffb700 1 -- 172.21.2.104:0/3119993521 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff50800c850 con 0x7ff5101439b0 2021-07-11T04:00:15.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff50dffb700 1 -- 172.21.2.104:0/3119993521 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50800c9b0 con 0x7ff5101439b0 2021-07-11T04:00:15.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff50dffb700 1 -- 172.21.2.104:0/3119993521 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff50800cd10 con 0x7ff5101439b0 2021-07-11T04:00:15.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 -- 172.21.2.104:0/3119993521 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff510152580 con 0x7ff5101439b0 2021-07-11T04:00:15.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 -- 172.21.2.104:0/3119993521 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5101454d0 con 0x7ff5101439b0 2021-07-11T04:00:15.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff516a72700 1 -- 172.21.2.104:0/3119993521 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff510057350 con 0x7ff5101439b0 2021-07-11T04:00:15.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff50dffb700 1 -- 172.21.2.104:0/3119993521 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff508022070 con 0x7ff5101439b0 2021-07-11T04:00:15.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.553+0000 7ff50dffb700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4f8053270 0x7ff4f8055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.557+0000 7ff50dffb700 1 -- 172.21.2.104:0/3119993521 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5953+0+0 (secure 0 0 0) 0x7ff50806bc80 con 0x7ff5101439b0 2021-07-11T04:00:15.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.557+0000 7ff51480e700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4f8053270 0x7ff4f8055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.557+0000 7ff51480e700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4f8053270 0x7ff4f8055730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7ff504002f70 tx=0x7ff504014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:15.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.557+0000 7ff50dffb700 1 -- 172.21.2.104:0/3119993521 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff508072080 con 0x7ff5101439b0 2021-07-11T04:00:15.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.669+0000 7ff516a72700 1 -- 172.21.2.104:0/3119993521 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7ff5100698e0 con 0x7ff5101439b0 2021-07-11T04:00:15.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.673+0000 7ff50dffb700 1 -- 172.21.2.104:0/3119993521 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v38) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff508042f20 con 0x7ff5101439b0 2021-07-11T04:00:15.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 -- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4f8053270 msgr2=0x7ff4f8055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4f8053270 0x7ff4f8055730 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7ff504002f70 tx=0x7ff504014000).stop 2021-07-11T04:00:15.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 -- 172.21.2.104:0/3119993521 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff5101439b0 msgr2=0x7ff510152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff5101439b0 0x7ff510152cc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7ff508006e00 tx=0x7ff5080091d0).stop 2021-07-11T04:00:15.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 -- 172.21.2.104:0/3119993521 shutdown_connections 2021-07-11T04:00:15.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4f8053270 0x7ff4f8055730 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff5101439b0 0x7ff510152cc0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff51013fff0 0x7ff5100b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 --2- 172.21.2.104:0/3119993521 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff510067120 0x7ff510145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 -- 172.21.2.104:0/3119993521 >> 172.21.2.104:0/3119993521 conn(0x7ff51005f9f0 msgr2=0x7ff5100616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:15.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 -- 172.21.2.104:0/3119993521 shutdown_connections 2021-07-11T04:00:15.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.677+0000 7ff516a72700 1 -- 172.21.2.104:0/3119993521 wait complete. 2021-07-11T04:00:15.688 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 15 and data pool 16 2021-07-11T04:00:15.689 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:15.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.733+0000 7fbefe75e700 1 Processor -- start 2021-07-11T04:00:15.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.733+0000 7fbefe75e700 1 -- start start 2021-07-11T04:00:15.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80973d0 0x7fbef805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef80b2a40 con 0x7fbef813f560 2021-07-11T04:00:15.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef8097b80 con 0x7fbef80b2660 2021-07-11T04:00:15.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef8097e20 con 0x7fbef80973d0 2021-07-11T04:00:15.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefccfb700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fbef813f560 0x7fbef813fa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43922/0 (socket says 172.21.2.104:43922) 2021-07-11T04:00:15.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefccfb700 1 -- 172.21.2.104:0/445358388 learned_addr learned my addr 172.21.2.104:0/445358388 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:15.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061375764 0 0) 0x7fbef80b2a40 con 0x7fbef813f560 2021-07-11T04:00:15.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbed8003260 con 0x7fbef813f560 2021-07-11T04:00:15.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef77fe700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80973d0 0x7fbef805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3160382159 0 0) 0x7fbed8003260 con 0x7fbef813f560 2021-07-11T04:00:15.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80973d0 msgr2=0x7fbef805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:15.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80973d0 0x7fbef805f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 >> v1:172.21.2.105:6789/0 conn(0x7fbef80b2660 legacy=0x7fbef8096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef80980c0 con 0x7fbef813f560 2021-07-11T04:00:15.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fbef00019e0 con 0x7fbef813f560 2021-07-11T04:00:15.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 >> v1:172.21.2.104:6789/0 conn(0x7fbef813f560 legacy=0x7fbef813fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbed8003780 0x7fbed8003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbed8004500 0x7fbed8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80973d0 0x7fbed8004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef8097e20 con 0x7fbed8004500 2021-07-11T04:00:15.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed800b800 con 0x7fbed8003780 2021-07-11T04:00:15.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed800b9c0 con 0x7fbef80973d0 2021-07-11T04:00:15.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef77fe700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbed8004500 0x7fbed8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef7fff700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbed8003780 0x7fbed8003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefccfb700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80973d0 0x7fbed8004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef77fe700 1 -- 172.21.2.104:0/445358388 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80973d0 msgr2=0x7fbed8004eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:15.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef77fe700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80973d0 0x7fbed8004eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef77fe700 1 -- 172.21.2.104:0/445358388 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbed8003780 msgr2=0x7fbed8003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:15.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef77fe700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbed8003780 0x7fbed8003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef77fe700 1 -- 172.21.2.104:0/445358388 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed8003260 con 0x7fbed8004500 2021-07-11T04:00:15.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef77fe700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbed8004500 0x7fbed8004980 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fbee000c000 tx=0x7fbee0006df0).ready entity=mon.0 client_cookie=b175c3110eded922 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:15.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbee000e830 con 0x7fbed8004500 2021-07-11T04:00:15.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee000e990 con 0x7fbed8004500 2021-07-11T04:00:15.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbef6ffd700 1 -- 172.21.2.104:0/445358388 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbee000ecf0 con 0x7fbed8004500 2021-07-11T04:00:15.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 -- 172.21.2.104:0/445358388 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbed8004500 msgr2=0x7fbed8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 --2- 172.21.2.104:0/445358388 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbed8004500 0x7fbed8004980 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fbee000c000 tx=0x7fbee0006df0).stop 2021-07-11T04:00:15.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefccfb700 1 -- 172.21.2.104:0/445358388 reap_dead start 2021-07-11T04:00:15.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 -- 172.21.2.104:0/445358388 shutdown_connections 2021-07-11T04:00:15.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 -- 172.21.2.104:0/445358388 >> 172.21.2.104:0/445358388 conn(0x7fbef8062f10 msgr2=0x7fbef80632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:15.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 -- 172.21.2.104:0/445358388 shutdown_connections 2021-07-11T04:00:15.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 -- 172.21.2.104:0/445358388 wait complete. 2021-07-11T04:00:15.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 Processor -- start 2021-07-11T04:00:15.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.737+0000 7fbefe75e700 1 -- start start 2021-07-11T04:00:15.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbefe75e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbef81332b0 0x7fbef81568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbefe75e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbef80ab170 0x7fbef8130710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbefe75e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80ab5f0 0x7fbef8130c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbefe75e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef80a9c60 con 0x7fbef81332b0 2021-07-11T04:00:15.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbefe75e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef8133690 con 0x7fbef80ab170 2021-07-11T04:00:15.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbefe75e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef8155f10 con 0x7fbef80ab5f0 2021-07-11T04:00:15.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbef81332b0 0x7fbef81568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbef81332b0 0x7fbef81568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47164/0 (socket says 172.21.2.104:47164) 2021-07-11T04:00:15.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef7fff700 1 -- 172.21.2.104:0/3700906083 learned_addr learned my addr 172.21.2.104:0/3700906083 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:15.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbefccfb700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80ab5f0 0x7fbef8130c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef77fe700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbef80ab170 0x7fbef8130710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef7fff700 1 -- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80ab5f0 msgr2=0x7fbef8130c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef7fff700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80ab5f0 0x7fbef8130c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef7fff700 1 -- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbef80ab170 msgr2=0x7fbef8130710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef7fff700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbef80ab170 0x7fbef8130710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef7fff700 1 -- 172.21.2.104:0/3700906083 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef81561b0 con 0x7fbef81332b0 2021-07-11T04:00:15.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbefccfb700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80ab5f0 0x7fbef8130c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:00:15.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef77fe700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbef80ab170 0x7fbef8130710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:00:15.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef7fff700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbef81332b0 0x7fbef81568f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fbee800d000 tx=0x7fbee8007a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:15.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef57fa700 1 -- 172.21.2.104:0/3700906083 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbee8011590 con 0x7fbef81332b0 2021-07-11T04:00:15.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef57fa700 1 -- 172.21.2.104:0/3700906083 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee80116f0 con 0x7fbef81332b0 2021-07-11T04:00:15.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef57fa700 1 -- 172.21.2.104:0/3700906083 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbee8011a50 con 0x7fbef81332b0 2021-07-11T04:00:15.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbefe75e700 1 -- 172.21.2.104:0/3700906083 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef8156450 con 0x7fbef81332b0 2021-07-11T04:00:15.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbefe75e700 1 -- 172.21.2.104:0/3700906083 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef8060560 con 0x7fbef81332b0 2021-07-11T04:00:15.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbefe75e700 1 -- 172.21.2.104:0/3700906083 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef80af520 con 0x7fbef81332b0 2021-07-11T04:00:15.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef57fa700 1 -- 172.21.2.104:0/3700906083 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbee801a030 con 0x7fbef81332b0 2021-07-11T04:00:15.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef57fa700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbedc0532b0 0x7fbedc055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef57fa700 1 -- 172.21.2.104:0/3700906083 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbee806add0 con 0x7fbef81332b0 2021-07-11T04:00:15.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef77fe700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbedc0532b0 0x7fbedc055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.741+0000 7fbef77fe700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbedc0532b0 0x7fbedc055770 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7fbee001a000 tx=0x7fbee0019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:15.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.745+0000 7fbef57fa700 1 -- 172.21.2.104:0/3700906083 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbee80421c0 con 0x7fbef81332b0 2021-07-11T04:00:15.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 -- 172.21.2.104:0/3700906083 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbef81565e0 con 0x7fbef81332b0 2021-07-11T04:00:15.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbef57fa700 1 -- 172.21.2.104:0/3700906083 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16054 (secure 0 0 0) 0x7fbee8022090 con 0x7fbef81332b0 2021-07-11T04:00:15.834 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:15.834 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":86,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:15.674412+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:13.631492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:14.726743+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:15.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 -- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbedc0532b0 msgr2=0x7fbedc055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbedc0532b0 0x7fbedc055770 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7fbee001a000 tx=0x7fbee0019000).stop 2021-07-11T04:00:15.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 -- 172.21.2.104:0/3700906083 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbef81332b0 msgr2=0x7fbef81568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbef81332b0 0x7fbef81568f0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7fbee800d000 tx=0x7fbee8007a00).stop 2021-07-11T04:00:15.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 -- 172.21.2.104:0/3700906083 shutdown_connections 2021-07-11T04:00:15.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbef80ab5f0 0x7fbef8130c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbef80ab170 0x7fbef8130710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbedc0532b0 0x7fbedc055770 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 --2- 172.21.2.104:0/3700906083 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbef81332b0 0x7fbef81568f0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 -- 172.21.2.104:0/3700906083 >> 172.21.2.104:0/3700906083 conn(0x7fbef8062f10 msgr2=0x7fbef80b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:15.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 -- 172.21.2.104:0/3700906083 shutdown_connections 2021-07-11T04:00:15.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.829+0000 7fbefe75e700 1 -- 172.21.2.104:0/3700906083 wait complete. 2021-07-11T04:00:15.842 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:15.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 Processor -- start 2021-07-11T04:00:15.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 -- start start 2021-07-11T04:00:15.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff6fc0b7810 0x7ff6fc1445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6fc0b1170 0x7ff6fc144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff6fc064870 0x7ff6fc064cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6fc0b31d0 con 0x7ff6fc064870 2021-07-11T04:00:15.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6fc0b7bf0 con 0x7ff6fc0b7810 2021-07-11T04:00:15.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0675c0 con 0x7ff6fc0b1170 2021-07-11T04:00:15.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6fb59e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff6fc064870 0x7ff6fc064cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6fb59e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff6fc064870 0x7ff6fc064cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47176/0 (socket says 172.21.2.104:47176) 2021-07-11T04:00:15.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6fb59e700 1 -- 172.21.2.104:0/3737198573 learned_addr learned my addr 172.21.2.104:0/3737198573 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:15.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6fb59e700 1 -- 172.21.2.104:0/3737198573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6fc0b1170 msgr2=0x7ff6fc144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:15.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6fb59e700 1 --2- 172.21.2.104:0/3737198573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6fc0b1170 0x7ff6fc144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6fb59e700 1 -- 172.21.2.104:0/3737198573 >> v2:172.21.2.105:3300/0 conn(0x7ff6fc0b7810 msgr2=0x7ff6fc1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:15.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6fb59e700 1 --2- 172.21.2.104:0/3737198573 >> v2:172.21.2.105:3300/0 conn(0x7ff6fc0b7810 0x7ff6fc1445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6fb59e700 1 -- 172.21.2.104:0/3737198573 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc0aae70 con 0x7ff6fc064870 2021-07-11T04:00:15.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6fb59e700 1 --2- 172.21.2.104:0/3737198573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6fc064870 0x7ff6fc064cd0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7ff6f40032f0 tx=0x7ff6f4009290).ready entity=mon.0 client_cookie=c8f4083676c1a1df server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:15.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6f9d9b700 1 -- 172.21.2.104:0/3737198573 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f400ba60 con 0x7ff6fc064870 2021-07-11T04:00:15.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6f9d9b700 1 -- 172.21.2.104:0/3737198573 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f400bbc0 con 0x7ff6fc064870 2021-07-11T04:00:15.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff6f9d9b700 1 -- 172.21.2.104:0/3737198573 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6f4010560 con 0x7ff6fc064870 2021-07-11T04:00:15.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 -- 172.21.2.104:0/3737198573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6fc064870 msgr2=0x7ff6fc064cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 --2- 172.21.2.104:0/3737198573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6fc064870 0x7ff6fc064cd0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7ff6f40032f0 tx=0x7ff6f4009290).stop 2021-07-11T04:00:15.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 -- 172.21.2.104:0/3737198573 shutdown_connections 2021-07-11T04:00:15.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 --2- 172.21.2.104:0/3737198573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6fc064870 0x7ff6fc064cd0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 --2- 172.21.2.104:0/3737198573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6fc0b1170 0x7ff6fc144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 --2- 172.21.2.104:0/3737198573 >> v2:172.21.2.105:3300/0 conn(0x7ff6fc0b7810 0x7ff6fc1445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 -- 172.21.2.104:0/3737198573 >> 172.21.2.104:0/3737198573 conn(0x7ff6fc05f6b0 msgr2=0x7ff6fc05fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:15.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 -- 172.21.2.104:0/3737198573 shutdown_connections 2021-07-11T04:00:15.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.889+0000 7ff70115a700 1 -- 172.21.2.104:0/3737198573 wait complete. 2021-07-11T04:00:15.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff70115a700 1 Processor -- start 2021-07-11T04:00:15.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff70115a700 1 -- start start 2021-07-11T04:00:15.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff70115a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6fc064870 0x7ff6fc0a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff70115a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6fc0b1170 0x7ff6fc13fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff70115a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6fc0b7810 0x7ff6fc0ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff70115a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc13fc40 con 0x7ff6fc0b1170 2021-07-11T04:00:15.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff70115a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0a9e60 con 0x7ff6fc064870 2021-07-11T04:00:15.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff70115a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc0aa0c0 con 0x7ff6fc0b7810 2021-07-11T04:00:15.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6fa59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6fc0b1170 0x7ff6fc13fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6fa59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6fc0b1170 0x7ff6fc13fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47178/0 (socket says 172.21.2.104:47178) 2021-07-11T04:00:15.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6fa59c700 1 -- 172.21.2.104:0/3020226306 learned_addr learned my addr 172.21.2.104:0/3020226306 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:15.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6fa59c700 1 -- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6fc0b7810 msgr2=0x7ff6fc0ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6fa59c700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6fc0b7810 0x7ff6fc0ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6fa59c700 1 -- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6fc064870 msgr2=0x7ff6fc0a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6fa59c700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6fc064870 0x7ff6fc0a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6fa59c700 1 -- 172.21.2.104:0/3020226306 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f4009530 con 0x7ff6fc0b1170 2021-07-11T04:00:15.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6fa59c700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6fc0b1170 0x7ff6fc13fe70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7ff6e4002f70 tx=0x7ff6e4006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:15.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6f3fff700 1 -- 172.21.2.104:0/3020226306 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e4013460 con 0x7ff6fc0b1170 2021-07-11T04:00:15.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6f3fff700 1 -- 172.21.2.104:0/3020226306 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e40135c0 con 0x7ff6fc0b1170 2021-07-11T04:00:15.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6f3fff700 1 -- 172.21.2.104:0/3020226306 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff6e4013920 con 0x7ff6fc0b1170 2021-07-11T04:00:15.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff70115a700 1 -- 172.21.2.104:0/3020226306 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6fc0a83a0 con 0x7ff6fc0b1170 2021-07-11T04:00:15.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff70115a700 1 -- 172.21.2.104:0/3020226306 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6fc0a8890 con 0x7ff6fc0b1170 2021-07-11T04:00:15.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff70115a700 1 -- 172.21.2.104:0/3020226306 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6fc0b3b20 con 0x7ff6fc0b1170 2021-07-11T04:00:15.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6f3fff700 1 -- 172.21.2.104:0/3020226306 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff6e401a060 con 0x7ff6fc0b1170 2021-07-11T04:00:15.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6f3fff700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6e00531a0 0x7ff6e0055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:15.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6f3fff700 1 -- 172.21.2.104:0/3020226306 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff6e406b030 con 0x7ff6fc0b1170 2021-07-11T04:00:15.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.893+0000 7ff6fad9d700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6e00531a0 0x7ff6e0055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:15.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.897+0000 7ff6fad9d700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6e00531a0 0x7ff6e0055660 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7ff6ec002f70 tx=0x7ff6ec006990).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:15.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.897+0000 7ff6f3fff700 1 -- 172.21.2.104:0/3020226306 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff6e4042260 con 0x7ff6fc0b1170 2021-07-11T04:00:15.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.977+0000 7ff70115a700 1 -- 172.21.2.104:0/3020226306 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6fc057350 con 0x7ff6fc0b1170 2021-07-11T04:00:15.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.977+0000 7ff6f3fff700 1 -- 172.21.2.104:0/3020226306 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16054 (secure 0 0 0) 0x7ff6e4018030 con 0x7ff6fc0b1170 2021-07-11T04:00:15.984 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:15.984 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":86,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:15.674412+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:13.631492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:14.726743+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:15.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 -- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6e00531a0 msgr2=0x7ff6e0055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6e00531a0 0x7ff6e0055660 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7ff6ec002f70 tx=0x7ff6ec006990).stop 2021-07-11T04:00:15.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 -- 172.21.2.104:0/3020226306 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6fc0b1170 msgr2=0x7ff6fc13fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:15.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6fc0b1170 0x7ff6fc13fe70 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7ff6e4002f70 tx=0x7ff6e4006ec0).stop 2021-07-11T04:00:15.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 -- 172.21.2.104:0/3020226306 shutdown_connections 2021-07-11T04:00:15.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6fc0b7810 0x7ff6fc0ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff6e00531a0 0x7ff6e0055660 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6fc0b1170 0x7ff6fc13fe70 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:15.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 --2- 172.21.2.104:0/3020226306 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6fc064870 0x7ff6fc0a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:15.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 -- 172.21.2.104:0/3020226306 >> 172.21.2.104:0/3020226306 conn(0x7ff6fc05f6b0 msgr2=0x7ff6fc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:15.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 -- 172.21.2.104:0/3020226306 shutdown_connections 2021-07-11T04:00:15.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:15.981+0000 7ff70115a700 1 -- 172.21.2.104:0/3020226306 wait complete. 2021-07-11T04:00:15.991 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T04:00:16.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.037+0000 7f80ae3bd700 1 Processor -- start 2021-07-11T04:00:16.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.037+0000 7f80ae3bd700 1 -- start start 2021-07-11T04:00:16.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80ae3bd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80a80ac260 0x7f80a80b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80ae3bd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f80a80b06c0 0x7f80a80aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80ae3bd700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a80b1c60 con 0x7f80a80a9960 2021-07-11T04:00:16.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80ae3bd700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a80a9d40 con 0x7f80a80b06c0 2021-07-11T04:00:16.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80ae3bd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80b2940 con 0x7f80a80ac260 2021-07-11T04:00:16.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80ac95a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f80a80b06c0 0x7f80a80aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:16.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80a77fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80a80ac260 0x7f80a80b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:16.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80ac95a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f80a80b06c0 0x7f80a80aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:54748/0 (socket says 172.21.2.104:54748) 2021-07-11T04:00:16.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80a77fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80a80ac260 0x7f80a80b1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:50354/0 (socket says 172.21.2.104:50354) 2021-07-11T04:00:16.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80ac95a700 1 -- 172.21.2.104:0/2734833920 learned_addr learned my addr 172.21.2.104:0/2734833920 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:16.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80a7fff700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f80a80a9960 0x7f80a80aec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:43324/0 (socket says 172.21.2.104:43324) 2021-07-11T04:00:16.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80a77fe700 1 -- 172.21.2.104:0/2734833920 >> v2:172.21.2.105:3300/0 conn(0x7f80a80b06c0 msgr2=0x7f80a80aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:16.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80a77fe700 1 --2- 172.21.2.104:0/2734833920 >> v2:172.21.2.105:3300/0 conn(0x7f80a80b06c0 0x7f80a80aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80a77fe700 1 -- 172.21.2.104:0/2734833920 >> v1:172.21.2.105:6789/0 conn(0x7f80a80a9960 legacy=0x7f80a80aec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80a6ffd700 1 -- 172.21.2.104:0/2734833920 <== mon.1 v1:172.21.2.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176200361 0 0) 0x7f8098001110 con 0x7f80a80a9960 2021-07-11T04:00:16.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80a77fe700 1 -- 172.21.2.104:0/2734833920 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a80b00c0 con 0x7f80a80ac260 2021-07-11T04:00:16.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80ac95a700 1 --2- 172.21.2.104:0/2734833920 >> v2:172.21.2.105:3300/0 conn(0x7f80a80b06c0 0x7f80a80aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:00:16.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.041+0000 7f80a77fe700 1 --2- 172.21.2.104:0/2734833920 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80a80ac260 0x7f80a80b1770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f809c002f70 tx=0x7f809c006df0).ready entity=mon.2 client_cookie=c09b995f4c7b55c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:16.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80a6ffd700 1 -- 172.21.2.104:0/2734833920 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f809c00d570 con 0x7f80a80ac260 2021-07-11T04:00:16.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80a6ffd700 1 -- 172.21.2.104:0/2734833920 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c000e90 con 0x7f80a80ac260 2021-07-11T04:00:16.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80a6ffd700 1 -- 172.21.2.104:0/2734833920 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f809c00d8c0 con 0x7f80a80ac260 2021-07-11T04:00:16.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 -- 172.21.2.104:0/2734833920 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80a80ac260 msgr2=0x7f80a80b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 --2- 172.21.2.104:0/2734833920 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80a80ac260 0x7f80a80b1770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f809c002f70 tx=0x7f809c006df0).stop 2021-07-11T04:00:16.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 -- 172.21.2.104:0/2734833920 shutdown_connections 2021-07-11T04:00:16.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 --2- 172.21.2.104:0/2734833920 >> v2:172.21.2.105:3300/0 conn(0x7f80a80b06c0 0x7f80a80aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 --2- 172.21.2.104:0/2734833920 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80a80ac260 0x7f80a80b1770 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 -- 172.21.2.104:0/2734833920 >> 172.21.2.104:0/2734833920 conn(0x7f80a805f6b0 msgr2=0x7f80a805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:16.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 -- 172.21.2.104:0/2734833920 shutdown_connections 2021-07-11T04:00:16.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 -- 172.21.2.104:0/2734833920 wait complete. 2021-07-11T04:00:16.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 Processor -- start 2021-07-11T04:00:16.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 -- start start 2021-07-11T04:00:16.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80a80a9960 0x7f80a8146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f80a80ac260 0x7f80a8157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80a80b06c0 0x7f80a80add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80ad2a0 con 0x7f80a80b06c0 2021-07-11T04:00:16.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a81575f0 con 0x7f80a80ac260 2021-07-11T04:00:16.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a80ae5f0 con 0x7f80a80a9960 2021-07-11T04:00:16.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ac95a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80a80b06c0 0x7f80a80add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:16.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ac95a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80a80b06c0 0x7f80a80add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47196/0 (socket says 172.21.2.104:47196) 2021-07-11T04:00:16.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ac95a700 1 -- 172.21.2.104:0/910629261 learned_addr learned my addr 172.21.2.104:0/910629261 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:16.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ac95a700 1 -- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80a80a9960 msgr2=0x7f80a8146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ac95a700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80a80a9960 0x7f80a8146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:16.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ac95a700 1 -- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f80a80ac260 msgr2=0x7f80a8157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ac95a700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f80a80ac260 0x7f80a8157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:16.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ac95a700 1 -- 172.21.2.104:0/910629261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a0009530 con 0x7f80a80b06c0 2021-07-11T04:00:16.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ac95a700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80a80b06c0 0x7f80a80add50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f80a815a680 tx=0x7f80a0009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:16.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80a57fa700 1 -- 172.21.2.104:0/910629261 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80a000c970 con 0x7f80a80b06c0 2021-07-11T04:00:16.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80a57fa700 1 -- 172.21.2.104:0/910629261 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a000cad0 con 0x7f80a80b06c0 2021-07-11T04:00:16.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80a57fa700 1 -- 172.21.2.104:0/910629261 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f80a000ce30 con 0x7f80a80b06c0 2021-07-11T04:00:16.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 -- 172.21.2.104:0/910629261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f809c00b180 con 0x7f80a80b06c0 2021-07-11T04:00:16.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.045+0000 7f80ae3bd700 1 -- 172.21.2.104:0/910629261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a815aa00 con 0x7f80a80b06c0 2021-07-11T04:00:16.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.049+0000 7f80ae3bd700 1 -- 172.21.2.104:0/910629261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a815abc0 con 0x7f80a80b06c0 2021-07-11T04:00:16.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.049+0000 7f80a57fa700 1 -- 172.21.2.104:0/910629261 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f80a0022070 con 0x7f80a80b06c0 2021-07-11T04:00:16.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.049+0000 7f80a57fa700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8088053180 0x7f8088055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.049+0000 7f80a57fa700 1 -- 172.21.2.104:0/910629261 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f80a006bdd0 con 0x7f80a80b06c0 2021-07-11T04:00:16.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.049+0000 7f80a7fff700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8088053180 0x7f8088055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:16.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.049+0000 7f80a7fff700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8088053180 0x7f8088055640 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f8098000bc0 tx=0x7f80980012f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:16.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.049+0000 7f80a57fa700 1 -- 172.21.2.104:0/910629261 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f80a006f030 con 0x7f80a80b06c0 2021-07-11T04:00:16.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.157+0000 7f80ae3bd700 1 -- 172.21.2.104:0/910629261 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f80a80ae7b0 con 0x7f80a80b06c0 2021-07-11T04:00:16.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.681+0000 7f80a57fa700 1 -- 172.21.2.104:0/910629261 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7f80a0016020 con 0x7f80a80b06c0 2021-07-11T04:00:16.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 -- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8088053180 msgr2=0x7f8088055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8088053180 0x7f8088055640 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f8098000bc0 tx=0x7f80980012f0).stop 2021-07-11T04:00:16.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 -- 172.21.2.104:0/910629261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80a80b06c0 msgr2=0x7f80a80add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80a80b06c0 0x7f80a80add50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f80a815a680 tx=0x7f80a0009290).stop 2021-07-11T04:00:16.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 -- 172.21.2.104:0/910629261 shutdown_connections 2021-07-11T04:00:16.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8088053180 0x7f8088055640 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f80a80b06c0 0x7f80a80add50 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f80a80ac260 0x7f80a8157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:16.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 --2- 172.21.2.104:0/910629261 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f80a80a9960 0x7f80a8146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:16.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 -- 172.21.2.104:0/910629261 >> 172.21.2.104:0/910629261 conn(0x7f80a805f6b0 msgr2=0x7f80a80aaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:16.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 -- 172.21.2.104:0/910629261 shutdown_connections 2021-07-11T04:00:16.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.685+0000 7f80ae3bd700 1 -- 172.21.2.104:0/910629261 wait complete. 2021-07-11T04:00:16.699 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:16.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.753+0000 7fcb69c2f700 1 Processor -- start 2021-07-11T04:00:16.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.753+0000 7fcb69c2f700 1 -- start start 2021-07-11T04:00:16.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcb640b3240 0x7fcb64066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb640b3620 con 0x7fcb640b0e70 2021-07-11T04:00:16.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb641436f0 con 0x7fcb640b1840 2021-07-11T04:00:16.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb64065ca0 con 0x7fcb640b3240 2021-07-11T04:00:16.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb63fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fcb640b0e70 0x7fcb6414a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43970/0 (socket says 172.21.2.104:43970) 2021-07-11T04:00:16.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb63fff700 1 -- 172.21.2.104:0/4006933988 learned_addr learned my addr 172.21.2.104:0/4006933988 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:16.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118685514 0 0) 0x7fcb640b3620 con 0x7fcb640b0e70 2021-07-11T04:00:16.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb44003260 con 0x7fcb640b0e70 2021-07-11T04:00:16.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb637fe700 1 --2- 172.21.2.104:0/4006933988 >> v2:172.21.2.105:3300/0 conn(0x7fcb640b3240 0x7fcb64066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:16.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4027469866 0 0) 0x7fcb44003260 con 0x7fcb640b0e70 2021-07-11T04:00:16.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 >> v2:172.21.2.105:3300/0 conn(0x7fcb640b3240 msgr2=0x7fcb64066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:16.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 --2- 172.21.2.104:0/4006933988 >> v2:172.21.2.105:3300/0 conn(0x7fcb640b3240 0x7fcb64066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 >> v1:172.21.2.105:6789/0 conn(0x7fcb640b1840 legacy=0x7fcb64142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:16.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb64065f40 con 0x7fcb640b0e70 2021-07-11T04:00:16.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fcb54001680 con 0x7fcb640b0e70 2021-07-11T04:00:16.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 >> v1:172.21.2.104:6789/0 conn(0x7fcb640b0e70 legacy=0x7fcb6414a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 --2- 172.21.2.104:0/4006933988 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb44003820 0x7fcb44003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 --2- 172.21.2.104:0/4006933988 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb44004500 0x7fcb44004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 --2- 172.21.2.104:0/4006933988 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb44004f70 0x7fcb4400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb64065ca0 con 0x7fcb44003820 2021-07-11T04:00:16.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4400be50 con 0x7fcb44004f70 2021-07-11T04:00:16.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4400c010 con 0x7fcb44004500 2021-07-11T04:00:16.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb637fe700 1 --2- 172.21.2.104:0/4006933988 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb44003820 0x7fcb44003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:16.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb637fe700 1 -- 172.21.2.104:0/4006933988 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb44004500 msgr2=0x7fcb44004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:16.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb637fe700 1 --2- 172.21.2.104:0/4006933988 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb44004500 0x7fcb44004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:16.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb637fe700 1 -- 172.21.2.104:0/4006933988 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb44004f70 msgr2=0x7fcb4400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:16.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb637fe700 1 --2- 172.21.2.104:0/4006933988 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb44004f70 0x7fcb4400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:16.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb637fe700 1 -- 172.21.2.104:0/4006933988 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb44003260 con 0x7fcb44003820 2021-07-11T04:00:16.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb63fff700 1 -- 172.21.2.104:0/4006933988 reap_dead start 2021-07-11T04:00:16.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb637fe700 1 --2- 172.21.2.104:0/4006933988 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb44003820 0x7fcb44003c20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fcb4c00c000 tx=0x7fcb4c006ea0).ready entity=mon.0 client_cookie=fcae184fa0c7cc73 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:16.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb4c00fa10 con 0x7fcb44003820 2021-07-11T04:00:16.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c00fb70 con 0x7fcb44003820 2021-07-11T04:00:16.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb627fc700 1 -- 172.21.2.104:0/4006933988 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb4c011450 con 0x7fcb44003820 2021-07-11T04:00:16.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 -- 172.21.2.104:0/4006933988 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb44003820 msgr2=0x7fcb44003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 --2- 172.21.2.104:0/4006933988 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb44003820 0x7fcb44003c20 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7fcb4c00c000 tx=0x7fcb4c006ea0).stop 2021-07-11T04:00:16.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 -- 172.21.2.104:0/4006933988 shutdown_connections 2021-07-11T04:00:16.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 --2- 172.21.2.104:0/4006933988 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb44003820 0x7fcb44003c20 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 -- 172.21.2.104:0/4006933988 >> 172.21.2.104:0/4006933988 conn(0x7fcb6405c490 msgr2=0x7fcb6405c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:16.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 -- 172.21.2.104:0/4006933988 shutdown_connections 2021-07-11T04:00:16.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 -- 172.21.2.104:0/4006933988 wait complete. 2021-07-11T04:00:16.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 Processor -- start 2021-07-11T04:00:16.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 -- start start 2021-07-11T04:00:16.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb44003820 0x7fcb640abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.757+0000 7fcb69c2f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb64151340 0x7fcb641517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb69c2f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb640ae9c0 0x7fcb640aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb69c2f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640aabb0 con 0x7fcb640ae9c0 2021-07-11T04:00:16.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb69c2f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640b0320 con 0x7fcb44003820 2021-07-11T04:00:16.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb69c2f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640aade0 con 0x7fcb64151340 2021-07-11T04:00:16.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb63fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb640ae9c0 0x7fcb640aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:16.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb63fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb640ae9c0 0x7fcb640aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47214/0 (socket says 172.21.2.104:47214) 2021-07-11T04:00:16.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb63fff700 1 -- 172.21.2.104:0/1610942242 learned_addr learned my addr 172.21.2.104:0/1610942242 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:16.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb62ffd700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb64151340 0x7fcb641517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:16.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb637fe700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb44003820 0x7fcb640abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:16.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb63fff700 1 -- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb64151340 msgr2=0x7fcb641517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb63fff700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb64151340 0x7fcb641517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb63fff700 1 -- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb44003820 msgr2=0x7fcb640abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb63fff700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb44003820 0x7fcb640abb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb63fff700 1 -- 172.21.2.104:0/1610942242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb640ab080 con 0x7fcb640ae9c0 2021-07-11T04:00:16.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb637fe700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb44003820 0x7fcb640abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:16.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb63fff700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb640ae9c0 0x7fcb640aee50 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fcb64065ae0 tx=0x7fcb640671d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:16.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb60ff9700 1 -- 172.21.2.104:0/1610942242 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb5400cec0 con 0x7fcb640ae9c0 2021-07-11T04:00:16.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb60ff9700 1 -- 172.21.2.104:0/1610942242 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb540070d0 con 0x7fcb640ae9c0 2021-07-11T04:00:16.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb69c2f700 1 -- 172.21.2.104:0/1610942242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb640ab520 con 0x7fcb640ae9c0 2021-07-11T04:00:16.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb60ff9700 1 -- 172.21.2.104:0/1610942242 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb54004520 con 0x7fcb640ae9c0 2021-07-11T04:00:16.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb69c2f700 1 -- 172.21.2.104:0/1610942242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb640b06b0 con 0x7fcb640ae9c0 2021-07-11T04:00:16.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb69c2f700 1 -- 172.21.2.104:0/1610942242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb640ab210 con 0x7fcb640ae9c0 2021-07-11T04:00:16.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb60ff9700 1 -- 172.21.2.104:0/1610942242 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcb5401b070 con 0x7fcb640ae9c0 2021-07-11T04:00:16.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb60ff9700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb48053210 0x7fcb480556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb60ff9700 1 -- 172.21.2.104:0/1610942242 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcb5406a7c0 con 0x7fcb640ae9c0 2021-07-11T04:00:16.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb637fe700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb48053210 0x7fcb480556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:16.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.761+0000 7fcb637fe700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb48053210 0x7fcb480556d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fcb4c002000 tx=0x7fcb4c006ea0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:16.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.765+0000 7fcb60ff9700 1 -- 172.21.2.104:0/1610942242 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcb54041aa0 con 0x7fcb640ae9c0 2021-07-11T04:00:16.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.873+0000 7fcb69c2f700 1 -- 172.21.2.104:0/1610942242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb6405d760 con 0x7fcb640ae9c0 2021-07-11T04:00:16.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb60ff9700 1 -- 172.21.2.104:0/1610942242 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2487 (secure 0 0 0) 0x7fcb540450c0 con 0x7fcb640ae9c0 2021-07-11T04:00:16.881 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:16.882 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/43570850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":43570850},{"type":"v1","addr":"172.21.2.105:6835","nonce":43570850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/3334136782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3334136782},{"type":"v1","addr":"172.21.2.104:6833","nonce":3334136782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:15.674524+0000","modified":"2021-07-11T04:00:16.678211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/4286166182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":4286166182},{"type":"v1","addr":"172.21.2.104:6835","nonce":4286166182}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-07-11T04:00:16.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 -- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb48053210 msgr2=0x7fcb480556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb48053210 0x7fcb480556d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fcb4c002000 tx=0x7fcb4c006ea0).stop 2021-07-11T04:00:16.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 -- 172.21.2.104:0/1610942242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb640ae9c0 msgr2=0x7fcb640aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb640ae9c0 0x7fcb640aee50 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fcb64065ae0 tx=0x7fcb640671d0).stop 2021-07-11T04:00:16.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 -- 172.21.2.104:0/1610942242 shutdown_connections 2021-07-11T04:00:16.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb48053210 0x7fcb480556d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb640ae9c0 0x7fcb640aee50 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb64151340 0x7fcb641517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 --2- 172.21.2.104:0/1610942242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb44003820 0x7fcb640abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 -- 172.21.2.104:0/1610942242 >> 172.21.2.104:0/1610942242 conn(0x7fcb6405c490 msgr2=0x7fcb640616c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:16.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 -- 172.21.2.104:0/1610942242 shutdown_connections 2021-07-11T04:00:16.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.877+0000 7fcb69c2f700 1 -- 172.21.2.104:0/1610942242 wait complete. 2021-07-11T04:00:16.890 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 39 2021-07-11T04:00:16.900 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:16.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 Processor -- start 2021-07-11T04:00:16.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 -- start start 2021-07-11T04:00:16.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f89680973d0 0x7f896805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89680b2a40 con 0x7f896805c490 2021-07-11T04:00:16.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8968097b80 con 0x7f89680b2660 2021-07-11T04:00:16.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8968097e20 con 0x7f89680973d0 2021-07-11T04:00:16.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896df1d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f896805c490 0x7f896813f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:43990/0 (socket says 172.21.2.104:43990) 2021-07-11T04:00:16.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896df1d700 1 -- 172.21.2.104:0/2402730950 learned_addr learned my addr 172.21.2.104:0/2402730950 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:16.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896cf1b700 1 --2- 172.21.2.104:0/2402730950 >> v2:172.21.2.104:3300/0 conn(0x7f89680973d0 0x7f896805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:16.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896cf1b700 1 -- 172.21.2.104:0/2402730950 >> v1:172.21.2.105:6789/0 conn(0x7f89680b2660 legacy=0x7f8968096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:16.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896cf1b700 1 -- 172.21.2.104:0/2402730950 >> v1:172.21.2.104:6789/0 conn(0x7f896805c490 legacy=0x7f896813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896cf1b700 1 -- 172.21.2.104:0/2402730950 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89680980c0 con 0x7f89680973d0 2021-07-11T04:00:16.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896cf1b700 1 --2- 172.21.2.104:0/2402730950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89680973d0 0x7f896805f6b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f89680b33f0 tx=0x7f8958006df0).ready entity=mon.0 client_cookie=338b07550af155bd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:16.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f8957fff700 1 -- 172.21.2.104:0/2402730950 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f895800d5c0 con 0x7f89680973d0 2021-07-11T04:00:16.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f8957fff700 1 -- 172.21.2.104:0/2402730950 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8958000e90 con 0x7f89680973d0 2021-07-11T04:00:16.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f8957fff700 1 -- 172.21.2.104:0/2402730950 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f895800d910 con 0x7f89680973d0 2021-07-11T04:00:16.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 -- 172.21.2.104:0/2402730950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89680973d0 msgr2=0x7f896805f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:16.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 --2- 172.21.2.104:0/2402730950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89680973d0 0x7f896805f6b0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7f89680b33f0 tx=0x7f8958006df0).stop 2021-07-11T04:00:16.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 -- 172.21.2.104:0/2402730950 shutdown_connections 2021-07-11T04:00:16.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 --2- 172.21.2.104:0/2402730950 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f89680973d0 0x7f896805f6b0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:16.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 -- 172.21.2.104:0/2402730950 >> 172.21.2.104:0/2402730950 conn(0x7f8968062f10 msgr2=0x7f89680632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:16.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 -- 172.21.2.104:0/2402730950 shutdown_connections 2021-07-11T04:00:16.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.949+0000 7f896f980700 1 -- 172.21.2.104:0/2402730950 wait complete. 2021-07-11T04:00:16.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896f980700 1 Processor -- start 2021-07-11T04:00:16.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896f980700 1 -- start start 2021-07-11T04:00:16.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896f980700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f896805c490 0x7f89680b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:16.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896f980700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89680973d0 0x7f8968066800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896f980700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f89680b2660 0x7f8968066cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896f980700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8968065bc0 con 0x7f896805c490 2021-07-11T04:00:17.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896f980700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8968065e60 con 0x7f89680b2660 2021-07-11T04:00:17.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896f980700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8968066100 con 0x7f89680973d0 2021-07-11T04:00:17.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896d71c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f896805c490 0x7f89680b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896d71c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f896805c490 0x7f89680b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47226/0 (socket says 172.21.2.104:47226) 2021-07-11T04:00:17.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896d71c700 1 -- 172.21.2.104:0/2863650157 learned_addr learned my addr 172.21.2.104:0/2863650157 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:17.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896d71c700 1 -- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89680973d0 msgr2=0x7f8968066800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896d71c700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89680973d0 0x7f8968066800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896d71c700 1 -- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f89680b2660 msgr2=0x7f8968066cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896d71c700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f89680b2660 0x7f8968066cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896d71c700 1 -- 172.21.2.104:0/2863650157 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895800b180 con 0x7f896805c490 2021-07-11T04:00:17.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896d71c700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f896805c490 0x7f89680b4a60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f8964002ef0 tx=0x7f8964006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:17.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f89567fc700 1 -- 172.21.2.104:0/2863650157 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8964012410 con 0x7f896805c490 2021-07-11T04:00:17.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f89567fc700 1 -- 172.21.2.104:0/2863650157 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8964012570 con 0x7f896805c490 2021-07-11T04:00:17.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896f980700 1 -- 172.21.2.104:0/2863650157 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89680663a0 con 0x7f896805c490 2021-07-11T04:00:17.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896f980700 1 -- 172.21.2.104:0/2863650157 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89680646a0 con 0x7f896805c490 2021-07-11T04:00:17.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f89567fc700 1 -- 172.21.2.104:0/2863650157 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89640128d0 con 0x7f896805c490 2021-07-11T04:00:17.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896f980700 1 -- 172.21.2.104:0/2863650157 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8968057350 con 0x7f896805c490 2021-07-11T04:00:17.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f89567fc700 1 -- 172.21.2.104:0/2863650157 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f896400f030 con 0x7f896805c490 2021-07-11T04:00:17.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f89567fc700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8948053240 0x7f8948055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f89567fc700 1 -- 172.21.2.104:0/2863650157 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f896406b380 con 0x7f896805c490 2021-07-11T04:00:17.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.953+0000 7f896cf1b700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8948053240 0x7f8948055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.957+0000 7f896cf1b700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8948053240 0x7f8948055700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f89680b33f0 tx=0x7f8958012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:17.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:16.957+0000 7f89567fc700 1 -- 172.21.2.104:0/2863650157 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f89640425b0 con 0x7f896805c490 2021-07-11T04:00:17.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 -- 172.21.2.104:0/2863650157 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89680aca80 con 0x7f896805c490 2021-07-11T04:00:17.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f89567fc700 1 -- 172.21.2.104:0/2863650157 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f8964045bd0 con 0x7f896805c490 2021-07-11T04:00:17.046 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:17.047 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":87,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:16.679409+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:13.631492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:14.726743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:17.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 -- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8948053240 msgr2=0x7f8948055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8948053240 0x7f8948055700 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f89680b33f0 tx=0x7f8958012000).stop 2021-07-11T04:00:17.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 -- 172.21.2.104:0/2863650157 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f896805c490 msgr2=0x7f89680b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f896805c490 0x7f89680b4a60 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7f8964002ef0 tx=0x7f8964006d50).stop 2021-07-11T04:00:17.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 -- 172.21.2.104:0/2863650157 shutdown_connections 2021-07-11T04:00:17.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f89680b2660 0x7f8968066cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89680973d0 0x7f8968066800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8948053240 0x7f8948055700 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 --2- 172.21.2.104:0/2863650157 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f896805c490 0x7f89680b4a60 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 -- 172.21.2.104:0/2863650157 >> 172.21.2.104:0/2863650157 conn(0x7f8968062f10 msgr2=0x7f89680b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:17.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 -- 172.21.2.104:0/2863650157 shutdown_connections 2021-07-11T04:00:17.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.041+0000 7f896f980700 1 -- 172.21.2.104:0/2863650157 wait complete. 2021-07-11T04:00:17.064 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:00:17.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 Processor -- start 2021-07-11T04:00:17.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 -- start start 2021-07-11T04:00:17.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f70e80b7810 0x7f70e81445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f70e80b1170 0x7f70e8144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e80b31d0 con 0x7f70e8064870 2021-07-11T04:00:17.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e80b7bf0 con 0x7f70e80b7810 2021-07-11T04:00:17.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e80675c0 con 0x7f70e80b1170 2021-07-11T04:00:17.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ecd0d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f70e8064870 0x7f70e8064d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44004/0 (socket says 172.21.2.104:44004) 2021-07-11T04:00:17.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ecd0d700 1 -- 172.21.2.104:0/4099111622 learned_addr learned my addr 172.21.2.104:0/4099111622 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:17.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70e7fff700 1 --2- 172.21.2.104:0/4099111622 >> v2:172.21.2.104:3300/0 conn(0x7f70e80b7810 0x7f70e81445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70e7fff700 1 -- 172.21.2.104:0/4099111622 >> v2:172.21.2.105:3300/0 conn(0x7f70e80b1170 msgr2=0x7f70e8144a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:17.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70e7fff700 1 --2- 172.21.2.104:0/4099111622 >> v2:172.21.2.105:3300/0 conn(0x7f70e80b1170 0x7f70e8144a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70e7fff700 1 -- 172.21.2.104:0/4099111622 >> v1:172.21.2.104:6789/0 conn(0x7f70e8064870 legacy=0x7f70e8064d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70e7fff700 1 -- 172.21.2.104:0/4099111622 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e80aae70 con 0x7f70e80b7810 2021-07-11T04:00:17.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70e7fff700 1 --2- 172.21.2.104:0/4099111622 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70e80b7810 0x7f70e81445a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f70d8002f70 tx=0x7f70d8006df0).ready entity=mon.0 client_cookie=21fc327260b7911c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:17.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70e6ffd700 1 -- 172.21.2.104:0/4099111622 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70d800d5c0 con 0x7f70e80b7810 2021-07-11T04:00:17.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70e6ffd700 1 -- 172.21.2.104:0/4099111622 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d8000e90 con 0x7f70e80b7810 2021-07-11T04:00:17.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70e6ffd700 1 -- 172.21.2.104:0/4099111622 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70d800d950 con 0x7f70e80b7810 2021-07-11T04:00:17.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 -- 172.21.2.104:0/4099111622 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70e80b7810 msgr2=0x7f70e81445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 --2- 172.21.2.104:0/4099111622 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70e80b7810 0x7f70e81445a0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f70d8002f70 tx=0x7f70d8006df0).stop 2021-07-11T04:00:17.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 -- 172.21.2.104:0/4099111622 shutdown_connections 2021-07-11T04:00:17.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 --2- 172.21.2.104:0/4099111622 >> v2:172.21.2.105:3300/0 conn(0x7f70e80b1170 0x7f70e8144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 --2- 172.21.2.104:0/4099111622 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70e80b7810 0x7f70e81445a0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 -- 172.21.2.104:0/4099111622 >> 172.21.2.104:0/4099111622 conn(0x7f70e805f6b0 msgr2=0x7f70e805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:17.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 -- 172.21.2.104:0/4099111622 shutdown_connections 2021-07-11T04:00:17.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 -- 172.21.2.104:0/4099111622 wait complete. 2021-07-11T04:00:17.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 Processor -- start 2021-07-11T04:00:17.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.125+0000 7f70ee770700 1 -- start start 2021-07-11T04:00:17.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70ee770700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70e8064870 0x7f70e80aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70ee770700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70e80b1170 0x7f70e80ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70ee770700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f70e80b7810 0x7f70e80a8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70ee770700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e8159d70 con 0x7f70e8064870 2021-07-11T04:00:17.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70ee770700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e8159fd0 con 0x7f70e80b7810 2021-07-11T04:00:17.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70ee770700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e815a270 con 0x7f70e80b1170 2021-07-11T04:00:17.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70e8064870 0x7f70e80aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70e8064870 0x7f70e80aa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47240/0 (socket says 172.21.2.104:47240) 2021-07-11T04:00:17.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e7fff700 1 -- 172.21.2.104:0/1214354760 learned_addr learned my addr 172.21.2.104:0/1214354760 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:17.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e7fff700 1 -- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70e80b1170 msgr2=0x7f70e80ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e7fff700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70e80b1170 0x7f70e80ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e7fff700 1 -- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f70e80b7810 msgr2=0x7f70e80a8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e7fff700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f70e80b7810 0x7f70e80a8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e7fff700 1 -- 172.21.2.104:0/1214354760 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d800b180 con 0x7f70e8064870 2021-07-11T04:00:17.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e7fff700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70e8064870 0x7f70e80aa410 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f70d8002f70 tx=0x7f70d8014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:17.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e57fa700 1 -- 172.21.2.104:0/1214354760 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70d8006b20 con 0x7f70e8064870 2021-07-11T04:00:17.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e57fa700 1 -- 172.21.2.104:0/1214354760 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d8000e90 con 0x7f70e8064870 2021-07-11T04:00:17.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70ee770700 1 -- 172.21.2.104:0/1214354760 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e815a510 con 0x7f70e8064870 2021-07-11T04:00:17.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70ee770700 1 -- 172.21.2.104:0/1214354760 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e80abe80 con 0x7f70e8064870 2021-07-11T04:00:17.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e57fa700 1 -- 172.21.2.104:0/1214354760 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f70d801e630 con 0x7f70e8064870 2021-07-11T04:00:17.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70ee770700 1 -- 172.21.2.104:0/1214354760 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e8057350 con 0x7f70e8064870 2021-07-11T04:00:17.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e57fa700 1 -- 172.21.2.104:0/1214354760 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f70d8024020 con 0x7f70e8064870 2021-07-11T04:00:17.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e57fa700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70cc0532d0 0x7f70cc055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e57fa700 1 -- 172.21.2.104:0/1214354760 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f70d806c7e0 con 0x7f70e8064870 2021-07-11T04:00:17.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.129+0000 7f70e77fe700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70cc0532d0 0x7f70cc055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.133+0000 7f70e77fe700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70cc0532d0 0x7f70cc055790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f70d0002fb0 tx=0x7f70d00069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:17.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.133+0000 7f70e57fa700 1 -- 172.21.2.104:0/1214354760 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f70d8043a10 con 0x7f70e8064870 2021-07-11T04:00:17.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.241+0000 7f70ee770700 1 -- 172.21.2.104:0/1214354760 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f70e815a6a0 con 0x7f70e8064870 2021-07-11T04:00:17.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70e57fa700 1 -- 172.21.2.104:0/1214354760 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v23) v1 ==== 184+0+0 (secure 0 0 0) 0x7f70d8047030 con 0x7f70e8064870 2021-07-11T04:00:17.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 -- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70cc0532d0 msgr2=0x7f70cc055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70cc0532d0 0x7f70cc055790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f70d0002fb0 tx=0x7f70d00069b0).stop 2021-07-11T04:00:17.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 -- 172.21.2.104:0/1214354760 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70e8064870 msgr2=0x7f70e80aa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70e8064870 0x7f70e80aa410 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f70d8002f70 tx=0x7f70d8014000).stop 2021-07-11T04:00:17.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 -- 172.21.2.104:0/1214354760 shutdown_connections 2021-07-11T04:00:17.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f70e80b7810 0x7f70e80a8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70e80b1170 0x7f70e80ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70cc0532d0 0x7f70cc055790 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 --2- 172.21.2.104:0/1214354760 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f70e8064870 0x7f70e80aa410 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 -- 172.21.2.104:0/1214354760 >> 172.21.2.104:0/1214354760 conn(0x7f70e805f6b0 msgr2=0x7f70e8146f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:17.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 -- 172.21.2.104:0/1214354760 shutdown_connections 2021-07-11T04:00:17.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.249+0000 7f70ee770700 1 -- 172.21.2.104:0/1214354760 wait complete. 2021-07-11T04:00:17.258 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T04:00:17.260 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:00:17.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.309+0000 7f2823d3e700 1 Processor -- start 2021-07-11T04:00:17.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.309+0000 7f2823d3e700 1 -- start start 2021-07-11T04:00:17.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f281c0ad150 0x7f281c0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281c0ac370 con 0x7f281c0aa860 2021-07-11T04:00:17.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281c0aac40 con 0x7f281c0a9da0 2021-07-11T04:00:17.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f281c0abb50 con 0x7f281c0ad150 2021-07-11T04:00:17.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2821ada700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f281c0aa860 0x7f281c0aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44014/0 (socket says 172.21.2.104:44014) 2021-07-11T04:00:17.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2821ada700 1 -- 172.21.2.104:0/915267184 learned_addr learned my addr 172.21.2.104:0/915267184 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:17.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975917876 0 0) 0x7f281c0ac370 con 0x7f281c0aa860 2021-07-11T04:00:17.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2800003260 con 0x7f281c0aa860 2021-07-11T04:00:17.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 50573204 0 0) 0x7f2800003260 con 0x7f281c0aa860 2021-07-11T04:00:17.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 >> v2:172.21.2.105:3300/0 conn(0x7f281c0ad150 msgr2=0x7f281c0b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:17.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 --2- 172.21.2.104:0/915267184 >> v2:172.21.2.105:3300/0 conn(0x7f281c0ad150 0x7f281c0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 >> v1:172.21.2.105:6789/0 conn(0x7f281c0a9da0 legacy=0x7f281c140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:17.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c0b16b0 con 0x7f281c0aa860 2021-07-11T04:00:17.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f2810001660 con 0x7f281c0aa860 2021-07-11T04:00:17.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 >> v1:172.21.2.104:6789/0 conn(0x7f281c0aa860 legacy=0x7f281c0aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2800003820 0x7f2800003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2800004500 0x7f2800004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2800004ec0 0x7f280000fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2800010330 con 0x7f2800004ec0 2021-07-11T04:00:17.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28000104a0 con 0x7f2800003820 2021-07-11T04:00:17.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2800010660 con 0x7f2800004500 2021-07-11T04:00:17.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f28222db700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2800004ec0 0x7f280000fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f28212d9700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2800004500 0x7f2800004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2821ada700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2800003820 0x7f2800003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f28222db700 1 -- 172.21.2.104:0/915267184 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2800004500 msgr2=0x7f2800004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:17.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f28222db700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2800004500 0x7f2800004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f28222db700 1 -- 172.21.2.104:0/915267184 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2800003820 msgr2=0x7f2800003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:17.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f28222db700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2800003820 0x7f2800003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f28222db700 1 -- 172.21.2.104:0/915267184 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800003260 con 0x7f2800004ec0 2021-07-11T04:00:17.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f28212d9700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2800004500 0x7f2800004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:17.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f28222db700 1 -- 172.21.2.104:0/915267184 reap_dead start 2021-07-11T04:00:17.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f28222db700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2800004ec0 0x7f280000fd60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f28140012f0 tx=0x7f2814007d60).ready entity=mon.0 client_cookie=29c40192df775ba2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:17.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28140014c0 con 0x7f2800004ec0 2021-07-11T04:00:17.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2814006870 con 0x7f2800004ec0 2021-07-11T04:00:17.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2820ad8700 1 -- 172.21.2.104:0/915267184 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2814006bd0 con 0x7f2800004ec0 2021-07-11T04:00:17.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 -- 172.21.2.104:0/915267184 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2800004ec0 msgr2=0x7f280000fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2800004ec0 0x7f280000fd60 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f28140012f0 tx=0x7f2814007d60).stop 2021-07-11T04:00:17.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 -- 172.21.2.104:0/915267184 shutdown_connections 2021-07-11T04:00:17.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 --2- 172.21.2.104:0/915267184 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2800004ec0 0x7f280000fd60 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 -- 172.21.2.104:0/915267184 >> 172.21.2.104:0/915267184 conn(0x7f281c0b64d0 msgr2=0x7f281c0b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:17.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 -- 172.21.2.104:0/915267184 shutdown_connections 2021-07-11T04:00:17.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 -- 172.21.2.104:0/915267184 wait complete. 2021-07-11T04:00:17.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 Processor -- start 2021-07-11T04:00:17.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.313+0000 7f2823d3e700 1 -- start start 2021-07-11T04:00:17.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2823d3e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2800004ec0 0x7f281c142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2823d3e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f281c140060 0x7f281c156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2823d3e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f281c156f50 0x7f281c159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2823d3e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c1564b0 con 0x7f281c140060 2021-07-11T04:00:17.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2823d3e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0b0290 con 0x7f2800004ec0 2021-07-11T04:00:17.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2823d3e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281c1573d0 con 0x7f281c156f50 2021-07-11T04:00:17.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f28212d9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f281c140060 0x7f281c156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f28212d9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f281c140060 0x7f281c156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47258/0 (socket says 172.21.2.104:47258) 2021-07-11T04:00:17.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f28212d9700 1 -- 172.21.2.104:0/2039358110 learned_addr learned my addr 172.21.2.104:0/2039358110 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:17.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f28222db700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f281c156f50 0x7f281c159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2821ada700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2800004ec0 0x7f281c142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f28212d9700 1 -- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f281c156f50 msgr2=0x7f281c159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f28212d9700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f281c156f50 0x7f281c159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f28212d9700 1 -- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2800004ec0 msgr2=0x7f281c142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f28212d9700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2800004ec0 0x7f281c142190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f28212d9700 1 -- 172.21.2.104:0/2039358110 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c0af7d0 con 0x7f281c140060 2021-07-11T04:00:17.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f28212d9700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f281c140060 0x7f281c156a60 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f281c0b0420 tx=0x7f2818008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:17.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f280effd700 1 -- 172.21.2.104:0/2039358110 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f281800e440 con 0x7f281c140060 2021-07-11T04:00:17.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f280effd700 1 -- 172.21.2.104:0/2039358110 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281800c030 con 0x7f281c140060 2021-07-11T04:00:17.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f280effd700 1 -- 172.21.2.104:0/2039358110 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f281800e790 con 0x7f281c140060 2021-07-11T04:00:17.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2823d3e700 1 -- 172.21.2.104:0/2039358110 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281c0afff0 con 0x7f281c140060 2021-07-11T04:00:17.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2823d3e700 1 -- 172.21.2.104:0/2039358110 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281c0ae790 con 0x7f281c140060 2021-07-11T04:00:17.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2823d3e700 1 -- 172.21.2.104:0/2039358110 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281c057350 con 0x7f281c140060 2021-07-11T04:00:17.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f280effd700 1 -- 172.21.2.104:0/2039358110 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f281800e8f0 con 0x7f281c140060 2021-07-11T04:00:17.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f280effd700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2804053330 0x7f28040557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f280effd700 1 -- 172.21.2.104:0/2039358110 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f281806b4c0 con 0x7f281c140060 2021-07-11T04:00:17.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2821ada700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2804053330 0x7f28040557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.317+0000 7f2821ada700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2804053330 0x7f28040557f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f281000d000 tx=0x7f2810009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:17.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.321+0000 7f280effd700 1 -- 172.21.2.104:0/2039358110 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f28180426f0 con 0x7f281c140060 2021-07-11T04:00:17.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.429+0000 7f2823d3e700 1 -- 172.21.2.104:0/2039358110 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f281c0698e0 con 0x7f281c140060 2021-07-11T04:00:17.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f280effd700 1 -- 172.21.2.104:0/2039358110 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v24) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2818045d10 con 0x7f281c140060 2021-07-11T04:00:17.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 -- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2804053330 msgr2=0x7f28040557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2804053330 0x7f28040557f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f281000d000 tx=0x7f2810009040).stop 2021-07-11T04:00:17.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 -- 172.21.2.104:0/2039358110 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f281c140060 msgr2=0x7f281c156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f281c140060 0x7f281c156a60 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f281c0b0420 tx=0x7f2818008040).stop 2021-07-11T04:00:17.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 -- 172.21.2.104:0/2039358110 shutdown_connections 2021-07-11T04:00:17.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f281c156f50 0x7f281c159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2804053330 0x7f28040557f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f281c140060 0x7f281c156a60 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 --2- 172.21.2.104:0/2039358110 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2800004ec0 0x7f281c142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 -- 172.21.2.104:0/2039358110 >> 172.21.2.104:0/2039358110 conn(0x7f281c0b64d0 msgr2=0x7f281c0b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:17.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 -- 172.21.2.104:0/2039358110 shutdown_connections 2021-07-11T04:00:17.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.437+0000 7f2823d3e700 1 -- 172.21.2.104:0/2039358110 wait complete. 2021-07-11T04:00:17.445 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T04:00:17.455 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:17.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f898e58c700 1 Processor -- start 2021-07-11T04:00:17.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f898e58c700 1 -- start start 2021-07-11T04:00:17.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f898e58c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89880b11c0 0x7f8988142f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f898e58c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89880604e0 con 0x7f8988060100 2021-07-11T04:00:17.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f898e58c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89880ac400 con 0x7f89881393d0 2021-07-11T04:00:17.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f898e58c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89880ac6a0 con 0x7f89880b11c0 2021-07-11T04:00:17.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8987fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8988060100 0x7f8988138cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44034/0 (socket says 172.21.2.104:44034) 2021-07-11T04:00:17.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8987fff700 1 -- 172.21.2.104:0/1263248628 learned_addr learned my addr 172.21.2.104:0/1263248628 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:17.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824100613 0 0) 0x7f89880604e0 con 0x7f8988060100 2021-07-11T04:00:17.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8968004370 con 0x7f8988060100 2021-07-11T04:00:17.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 603597398 0 0) 0x7f8968004370 con 0x7f8988060100 2021-07-11T04:00:17.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89880b11c0 msgr2=0x7f8988142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:17.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 --2- 172.21.2.104:0/1263248628 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89880b11c0 0x7f8988142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 >> v1:172.21.2.105:6789/0 conn(0x7f89881393d0 legacy=0x7f8988142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:17.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89880ac940 con 0x7f8988060100 2021-07-11T04:00:17.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f8978004a20 con 0x7f8988060100 2021-07-11T04:00:17.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 >> v1:172.21.2.104:6789/0 conn(0x7f8988060100 legacy=0x7f8988138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 --2- 172.21.2.104:0/1263248628 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8968004910 0x7f8968004d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 --2- 172.21.2.104:0/1263248628 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89880b11c0 0x7f8968005610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 --2- 172.21.2.104:0/1263248628 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8968005d00 0x7f896800c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89880ac6a0 con 0x7f8968004910 2021-07-11T04:00:17.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89680063e0 con 0x7f8968005d00 2021-07-11T04:00:17.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8968003260 con 0x7f89880b11c0 2021-07-11T04:00:17.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8987fff700 1 --2- 172.21.2.104:0/1263248628 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8968004910 0x7f8968004d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8987fff700 1 -- 172.21.2.104:0/1263248628 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89880b11c0 msgr2=0x7f8968005610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:17.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8987fff700 1 --2- 172.21.2.104:0/1263248628 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f89880b11c0 0x7f8968005610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:17.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8987fff700 1 -- 172.21.2.104:0/1263248628 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8968005d00 msgr2=0x7f896800c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:17.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f898cb29700 1 --2- 172.21.2.104:0/1263248628 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8968005d00 0x7f896800c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8987fff700 1 --2- 172.21.2.104:0/1263248628 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8968005d00 0x7f896800c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8987fff700 1 -- 172.21.2.104:0/1263248628 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8968004370 con 0x7f8968004910 2021-07-11T04:00:17.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f898cb29700 1 --2- 172.21.2.104:0/1263248628 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8968005d00 0x7f896800c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:17.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.501+0000 7f8987fff700 1 --2- 172.21.2.104:0/1263248628 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8968004910 0x7f8968004d10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f89880628a0 tx=0x7f898813f190).ready entity=mon.0 client_cookie=c81b82a2ebd089de server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:17.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f897800c9a0 con 0x7f8968004910 2021-07-11T04:00:17.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897800cb00 con 0x7f8968004910 2021-07-11T04:00:17.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8986ffd700 1 -- 172.21.2.104:0/1263248628 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f897800ce60 con 0x7f8968004910 2021-07-11T04:00:17.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- 172.21.2.104:0/1263248628 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8968004910 msgr2=0x7f8968004d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 --2- 172.21.2.104:0/1263248628 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8968004910 0x7f8968004d10 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f89880628a0 tx=0x7f898813f190).stop 2021-07-11T04:00:17.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898cb29700 1 -- 172.21.2.104:0/1263248628 reap_dead start 2021-07-11T04:00:17.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- 172.21.2.104:0/1263248628 shutdown_connections 2021-07-11T04:00:17.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- 172.21.2.104:0/1263248628 >> 172.21.2.104:0/1263248628 conn(0x7f8988061070 msgr2=0x7f898805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:17.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- 172.21.2.104:0/1263248628 shutdown_connections 2021-07-11T04:00:17.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- 172.21.2.104:0/1263248628 wait complete. 2021-07-11T04:00:17.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 Processor -- start 2021-07-11T04:00:17.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- start start 2021-07-11T04:00:17.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8988146200 0x7f898813cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f898813c3e0 0x7f898814dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f898814e390 0x7f8988136f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898813ba70 con 0x7f8988146200 2021-07-11T04:00:17.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89881465e0 con 0x7f898813c3e0 2021-07-11T04:00:17.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8988137ae0 con 0x7f898814e390 2021-07-11T04:00:17.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8987fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8988146200 0x7f898813cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8987fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8988146200 0x7f898813cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47278/0 (socket says 172.21.2.104:47278) 2021-07-11T04:00:17.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8987fff700 1 -- 172.21.2.104:0/2755514484 learned_addr learned my addr 172.21.2.104:0/2755514484 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:17.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898cb29700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f898814e390 0x7f8988136f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f89877fe700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f898813c3e0 0x7f898814dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8987fff700 1 -- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f898814e390 msgr2=0x7f8988136f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8987fff700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f898814e390 0x7f8988136f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8987fff700 1 -- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f898813c3e0 msgr2=0x7f898814dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8987fff700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f898813c3e0 0x7f898814dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8987fff700 1 -- 172.21.2.104:0/2755514484 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8988137d40 con 0x7f8988146200 2021-07-11T04:00:17.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f8987fff700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8988146200 0x7f898813cb30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f89880628a0 tx=0x7f8978013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:17.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f89857fa700 1 -- 172.21.2.104:0/2755514484 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8978001840 con 0x7f8988146200 2021-07-11T04:00:17.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f89857fa700 1 -- 172.21.2.104:0/2755514484 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8978014b50 con 0x7f8988146200 2021-07-11T04:00:17.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f89857fa700 1 -- 172.21.2.104:0/2755514484 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f89780013a0 con 0x7f8988146200 2021-07-11T04:00:17.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- 172.21.2.104:0/2755514484 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898813bca0 con 0x7f8988146200 2021-07-11T04:00:17.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- 172.21.2.104:0/2755514484 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898813ffe0 con 0x7f8988146200 2021-07-11T04:00:17.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f898e58c700 1 -- 172.21.2.104:0/2755514484 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898813c140 con 0x7f8988146200 2021-07-11T04:00:17.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.505+0000 7f89857fa700 1 -- 172.21.2.104:0/2755514484 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8978027070 con 0x7f8988146200 2021-07-11T04:00:17.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.509+0000 7f89857fa700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f896c053300 0x7f896c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:17.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.509+0000 7f89857fa700 1 -- 172.21.2.104:0/2755514484 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f897806bd40 con 0x7f8988146200 2021-07-11T04:00:17.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.509+0000 7f89877fe700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f896c053300 0x7f896c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:17.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.509+0000 7f89877fe700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f896c053300 0x7f896c0557c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f897000c000 tx=0x7f8970008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:17.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.509+0000 7f89857fa700 1 -- 172.21.2.104:0/2755514484 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f898813c140 con 0x7f8988146200 2021-07-11T04:00:17.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.617+0000 7f898e58c700 1 -- 172.21.2.104:0/2755514484 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f898813c140 con 0x7f8988146200 2021-07-11T04:00:17.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.617+0000 7f89857fa700 1 -- 172.21.2.104:0/2755514484 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f898813c140 con 0x7f8988146200 2021-07-11T04:00:17.624 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:17.624 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/43570850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":43570850},{"type":"v1","addr":"172.21.2.105:6835","nonce":43570850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/3334136782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3334136782},{"type":"v1","addr":"172.21.2.104:6833","nonce":3334136782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:15.674524+0000","modified":"2021-07-11T04:00:16.678211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/4286166182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":4286166182},{"type":"v1","addr":"172.21.2.104:6835","nonce":4286166182}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-07-11T04:00:17.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 -- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f896c053300 msgr2=0x7f896c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f896c053300 0x7f896c0557c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f897000c000 tx=0x7f8970008040).stop 2021-07-11T04:00:17.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 -- 172.21.2.104:0/2755514484 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8988146200 msgr2=0x7f898813cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:17.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8988146200 0x7f898813cb30 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f89880628a0 tx=0x7f8978013000).stop 2021-07-11T04:00:17.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 -- 172.21.2.104:0/2755514484 shutdown_connections 2021-07-11T04:00:17.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f898814e390 0x7f8988136f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f898813c3e0 0x7f898814dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f896c053300 0x7f896c0557c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 --2- 172.21.2.104:0/2755514484 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8988146200 0x7f898813cb30 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:17.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 -- 172.21.2.104:0/2755514484 >> 172.21.2.104:0/2755514484 conn(0x7f8988061070 msgr2=0x7f89880b4e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:17.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 -- 172.21.2.104:0/2755514484 shutdown_connections 2021-07-11T04:00:17.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:17.621+0000 7f898e58c700 1 -- 172.21.2.104:0/2755514484 wait complete. 2021-07-11T04:00:17.630 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 39 2021-07-11T04:00:17.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:00:15.674524+0000', 'modified': '2021-07-11T04:00:16.678211+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6636}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6636': {'gid': 6636, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/4286166182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 4286166182}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 4286166182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:00:17.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:00:17.935 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-07-11T04:00:18.006 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":6636,"want_state":"up:active","state":"up:active","rank_uptime":2.3242063229999999,"mdsmap_epoch":39,"osdmap_epoch":87,"osdmap_epoch_barrier":86,"uptime":6.5437849300000002} 2021-07-11T04:00:18.006 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 6636, "mdsmap_epoch": 39, "osdmap_epoch": 87, "osdmap_epoch_barrier": 86, "rank_uptime": 2.324206323, "state": "up:active", "uptime": 6.54378493, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:00:18.007 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:18.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee9135e700 1 Processor -- start 2021-07-11T04:00:18.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee9135e700 1 -- start start 2021-07-11T04:00:18.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee9135e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fee8c0a76b0 0x7fee8c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee9135e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee8c0b1970 0x7fee8c066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee9135e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee8c067450 con 0x7fee8c0b0060 2021-07-11T04:00:18.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee9135e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee8c0a7a90 con 0x7fee8c0a76b0 2021-07-11T04:00:18.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee9135e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c147620 con 0x7fee8c0b1970 2021-07-11T04:00:18.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee8affd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fee8c0a76b0 0x7fee8c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee8affd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fee8c0a76b0 0x7fee8c066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47286/0 (socket says 172.21.2.104:47286) 2021-07-11T04:00:18.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee8affd700 1 -- 172.21.2.104:0/1654644598 learned_addr learned my addr 172.21.2.104:0/1654644598 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:18.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee8affd700 1 -- 172.21.2.104:0/1654644598 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee8c0b1970 msgr2=0x7fee8c066f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:18.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee8affd700 1 --2- 172.21.2.104:0/1654644598 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee8c0b1970 0x7fee8c066f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee8affd700 1 -- 172.21.2.104:0/1654644598 >> v1:172.21.2.105:6789/0 conn(0x7fee8c0b0060 legacy=0x7fee8c05f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:18.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee8affd700 1 -- 172.21.2.104:0/1654644598 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c1478c0 con 0x7fee8c0a76b0 2021-07-11T04:00:18.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee8affd700 1 --2- 172.21.2.104:0/1654644598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee8c0a76b0 0x7fee8c066a70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fee8c0b6460 tx=0x7fee74006eb0).ready entity=mon.0 client_cookie=246dc6c53472ec9c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee89ffb700 1 -- 172.21.2.104:0/1654644598 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee7400d570 con 0x7fee8c0a76b0 2021-07-11T04:00:18.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.053+0000 7fee89ffb700 1 -- 172.21.2.104:0/1654644598 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee74000e90 con 0x7fee8c0a76b0 2021-07-11T04:00:18.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee89ffb700 1 -- 172.21.2.104:0/1654644598 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee7400d8c0 con 0x7fee8c0a76b0 2021-07-11T04:00:18.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- 172.21.2.104:0/1654644598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee8c0a76b0 msgr2=0x7fee8c066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 --2- 172.21.2.104:0/1654644598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee8c0a76b0 0x7fee8c066a70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fee8c0b6460 tx=0x7fee74006eb0).stop 2021-07-11T04:00:18.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- 172.21.2.104:0/1654644598 shutdown_connections 2021-07-11T04:00:18.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 --2- 172.21.2.104:0/1654644598 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee8c0b1970 0x7fee8c066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 --2- 172.21.2.104:0/1654644598 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee8c0a76b0 0x7fee8c066a70 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- 172.21.2.104:0/1654644598 >> 172.21.2.104:0/1654644598 conn(0x7fee8c0b53e0 msgr2=0x7fee8c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- 172.21.2.104:0/1654644598 shutdown_connections 2021-07-11T04:00:18.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- 172.21.2.104:0/1654644598 wait complete. 2021-07-11T04:00:18.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 Processor -- start 2021-07-11T04:00:18.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- start start 2021-07-11T04:00:18.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee8c0a76b0 0x7fee8c0b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee8c0b0060 0x7fee8c0aa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fee8c0b1970 0x7fee8c156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c0af040 con 0x7fee8c0a76b0 2021-07-11T04:00:18.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c0acfa0 con 0x7fee8c0b1970 2021-07-11T04:00:18.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8c0ae3c0 con 0x7fee8c0b0060 2021-07-11T04:00:18.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee8affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee8c0a76b0 0x7fee8c0b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee8affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee8c0a76b0 0x7fee8c0b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47292/0 (socket says 172.21.2.104:47292) 2021-07-11T04:00:18.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee8affd700 1 -- 172.21.2.104:0/3795016026 learned_addr learned my addr 172.21.2.104:0/3795016026 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:18.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee8affd700 1 -- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee8c0b0060 msgr2=0x7fee8c0aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee8affd700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee8c0b0060 0x7fee8c0aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee8affd700 1 -- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fee8c0b1970 msgr2=0x7fee8c156810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee8affd700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fee8c0b1970 0x7fee8c156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee8affd700 1 -- 172.21.2.104:0/3795016026 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7400b1e0 con 0x7fee8c0a76b0 2021-07-11T04:00:18.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee8affd700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee8c0a76b0 0x7fee8c0b44a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fee8c0b6460 tx=0x7fee74015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee83fff700 1 -- 172.21.2.104:0/3795016026 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee74021070 con 0x7fee8c0a76b0 2021-07-11T04:00:18.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee83fff700 1 -- 172.21.2.104:0/3795016026 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee74000e90 con 0x7fee8c0a76b0 2021-07-11T04:00:18.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- 172.21.2.104:0/3795016026 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8c0ae660 con 0x7fee8c0a76b0 2021-07-11T04:00:18.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- 172.21.2.104:0/3795016026 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee8c0aeba0 con 0x7fee8c0a76b0 2021-07-11T04:00:18.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee83fff700 1 -- 172.21.2.104:0/3795016026 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee74006b80 con 0x7fee8c0a76b0 2021-07-11T04:00:18.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee9135e700 1 -- 172.21.2.104:0/3795016026 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee8c0af6a0 con 0x7fee8c0a76b0 2021-07-11T04:00:18.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee83fff700 1 -- 172.21.2.104:0/3795016026 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fee7400f070 con 0x7fee8c0a76b0 2021-07-11T04:00:18.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.057+0000 7fee83fff700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fee70053240 0x7fee70055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.061+0000 7fee83fff700 1 -- 172.21.2.104:0/3795016026 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fee7406ca60 con 0x7fee8c0a76b0 2021-07-11T04:00:18.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.061+0000 7fee8a7fc700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fee70053240 0x7fee70055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.061+0000 7fee8a7fc700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fee70053240 0x7fee70055700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fee7c000ed0 tx=0x7fee7c006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.061+0000 7fee83fff700 1 -- 172.21.2.104:0/3795016026 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fee74043d40 con 0x7fee8c0a76b0 2021-07-11T04:00:18.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.145+0000 7fee9135e700 1 -- 172.21.2.104:0/3795016026 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee8c0aaad0 con 0x7fee8c0a76b0 2021-07-11T04:00:18.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.145+0000 7fee83fff700 1 -- 172.21.2.104:0/3795016026 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16041 (secure 0 0 0) 0x7fee74047360 con 0x7fee8c0a76b0 2021-07-11T04:00:18.152 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:18.153 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":87,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:16.679409+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:13.631492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:14.726743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:18.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 -- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fee70053240 msgr2=0x7fee70055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fee70053240 0x7fee70055700 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7fee7c000ed0 tx=0x7fee7c006930).stop 2021-07-11T04:00:18.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 -- 172.21.2.104:0/3795016026 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee8c0a76b0 msgr2=0x7fee8c0b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee8c0a76b0 0x7fee8c0b44a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fee8c0b6460 tx=0x7fee74015000).stop 2021-07-11T04:00:18.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 -- 172.21.2.104:0/3795016026 shutdown_connections 2021-07-11T04:00:18.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fee8c0b1970 0x7fee8c156810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee8c0b0060 0x7fee8c0aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fee70053240 0x7fee70055700 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 --2- 172.21.2.104:0/3795016026 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee8c0a76b0 0x7fee8c0b44a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 -- 172.21.2.104:0/3795016026 >> 172.21.2.104:0/3795016026 conn(0x7fee8c0b53e0 msgr2=0x7fee8c145f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 -- 172.21.2.104:0/3795016026 shutdown_connections 2021-07-11T04:00:18.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.149+0000 7fee9135e700 1 -- 172.21.2.104:0/3795016026 wait complete. 2021-07-11T04:00:18.161 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:00:13.631492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:00:14.726743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:00:18.164 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:00:18.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.221+0000 7f21ffc12700 1 Processor -- start 2021-07-11T04:00:18.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.221+0000 7f21ffc12700 1 -- start start 2021-07-11T04:00:18.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f21f8141960 0x7f21f80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f80a77d0 con 0x7f21f80b12d0 2021-07-11T04:00:18.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f80b0650 con 0x7f21f80b1ec0 2021-07-11T04:00:18.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80b1740 con 0x7f21f8141960 2021-07-11T04:00:18.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fd1ad700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f21f80b12d0 0x7f21f8141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44068/0 (socket says 172.21.2.104:44068) 2021-07-11T04:00:18.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fd1ad700 1 -- 172.21.2.104:0/2770285047 learned_addr learned my addr 172.21.2.104:0/2770285047 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:18.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672947978 0 0) 0x7f21f80a77d0 con 0x7f21f80b12d0 2021-07-11T04:00:18.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e0003260 con 0x7f21f80b12d0 2021-07-11T04:00:18.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1070857493 0 0) 0x7f21e0003260 con 0x7f21f80b12d0 2021-07-11T04:00:18.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f21f8141960 msgr2=0x7f21f80affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:18.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 --2- 172.21.2.104:0/2770285047 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f21f8141960 0x7f21f80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 >> v1:172.21.2.105:6789/0 conn(0x7f21f80b1ec0 legacy=0x7f21f80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:18.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f80b08f0 con 0x7f21f80b12d0 2021-07-11T04:00:18.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f21e8001660 con 0x7f21f80b12d0 2021-07-11T04:00:18.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 >> v1:172.21.2.104:6789/0 conn(0x7f21f80b12d0 legacy=0x7f21f8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 --2- 172.21.2.104:0/2770285047 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f21e0003800 0x7f21e0003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 --2- 172.21.2.104:0/2770285047 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21e0004500 0x7f21e0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 --2- 172.21.2.104:0/2770285047 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f21f8141960 0x7f21e000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80b1740 con 0x7f21e0004500 2021-07-11T04:00:18.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e0005290 con 0x7f21e0003800 2021-07-11T04:00:18.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e000bc40 con 0x7f21f8141960 2021-07-11T04:00:18.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fd1ad700 1 --2- 172.21.2.104:0/2770285047 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21e0004500 0x7f21e0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fd1ad700 1 -- 172.21.2.104:0/2770285047 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f21f8141960 msgr2=0x7f21e000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:18.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fd1ad700 1 --2- 172.21.2.104:0/2770285047 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f21f8141960 0x7f21e000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fd1ad700 1 -- 172.21.2.104:0/2770285047 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f21e0003800 msgr2=0x7f21e0003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:18.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fd1ad700 1 --2- 172.21.2.104:0/2770285047 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f21e0003800 0x7f21e0003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fd1ad700 1 -- 172.21.2.104:0/2770285047 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e0003260 con 0x7f21e0004500 2021-07-11T04:00:18.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fd1ad700 1 --2- 172.21.2.104:0/2770285047 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21e0004500 0x7f21e0004980 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f21f806ac60 tx=0x7f21f80b2a00).ready entity=mon.0 client_cookie=adef872459677da8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21e80070d0 con 0x7f21e0004500 2021-07-11T04:00:18.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e8007230 con 0x7f21e0004500 2021-07-11T04:00:18.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fc9ac700 1 -- 172.21.2.104:0/2770285047 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21e800c8f0 con 0x7f21e0004500 2021-07-11T04:00:18.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- 172.21.2.104:0/2770285047 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21e0004500 msgr2=0x7f21e0004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 --2- 172.21.2.104:0/2770285047 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21e0004500 0x7f21e0004980 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f21f806ac60 tx=0x7f21f80b2a00).stop 2021-07-11T04:00:18.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fe1af700 1 -- 172.21.2.104:0/2770285047 reap_dead start 2021-07-11T04:00:18.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- 172.21.2.104:0/2770285047 shutdown_connections 2021-07-11T04:00:18.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- 172.21.2.104:0/2770285047 >> 172.21.2.104:0/2770285047 conn(0x7f21f805f6b0 msgr2=0x7f21f805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- 172.21.2.104:0/2770285047 shutdown_connections 2021-07-11T04:00:18.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- 172.21.2.104:0/2770285047 wait complete. 2021-07-11T04:00:18.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 Processor -- start 2021-07-11T04:00:18.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- start start 2021-07-11T04:00:18.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f21f80b3530 0x7f21f80b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f21f80aada0 0x7f21f813a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21f80b3910 0x7f21f813acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f8149d60 con 0x7f21f80b3910 2021-07-11T04:00:18.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80aa5b0 con 0x7f21f80aada0 2021-07-11T04:00:18.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21ffc12700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80abd50 con 0x7f21f80b3530 2021-07-11T04:00:18.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fe1af700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21f80b3910 0x7f21f813acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fe1af700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21f80b3910 0x7f21f813acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47316/0 (socket says 172.21.2.104:47316) 2021-07-11T04:00:18.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.225+0000 7f21fe1af700 1 -- 172.21.2.104:0/2574530588 learned_addr learned my addr 172.21.2.104:0/2574530588 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:18.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21fe1af700 1 -- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f21f80b3530 msgr2=0x7f21f80b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21fe1af700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f21f80b3530 0x7f21f80b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21fe1af700 1 -- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f21f80aada0 msgr2=0x7f21f813a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21fe1af700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f21f80aada0 0x7f21f813a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21fe1af700 1 -- 172.21.2.104:0/2574530588 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f80b5560 con 0x7f21f80b3910 2021-07-11T04:00:18.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21fe1af700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21f80b3910 0x7f21f813acf0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f21e8001370 tx=0x7f21f0007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21e6ffd700 1 -- 172.21.2.104:0/2574530588 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21f000ce40 con 0x7f21f80b3910 2021-07-11T04:00:18.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21e6ffd700 1 -- 172.21.2.104:0/2574530588 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f0005c30 con 0x7f21f80b3910 2021-07-11T04:00:18.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21ffc12700 1 -- 172.21.2.104:0/2574530588 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f813b1e0 con 0x7f21f80b3910 2021-07-11T04:00:18.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21ffc12700 1 -- 172.21.2.104:0/2574530588 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f80aa110 con 0x7f21f80b3910 2021-07-11T04:00:18.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21e6ffd700 1 -- 172.21.2.104:0/2574530588 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21f0007110 con 0x7f21f80b3910 2021-07-11T04:00:18.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21ffc12700 1 -- 172.21.2.104:0/2574530588 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f80abf10 con 0x7f21f80b3910 2021-07-11T04:00:18.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21e6ffd700 1 -- 172.21.2.104:0/2574530588 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f21f001b060 con 0x7f21f80b3910 2021-07-11T04:00:18.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21e6ffd700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21d80532b0 0x7f21d8055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21e6ffd700 1 -- 172.21.2.104:0/2574530588 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f21f006abc0 con 0x7f21f80b3910 2021-07-11T04:00:18.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21fd9ae700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21d80532b0 0x7f21d8055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.229+0000 7f21fd9ae700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21d80532b0 0x7f21d8055770 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f21f4000f70 tx=0x7f21f40069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.233+0000 7f21e6ffd700 1 -- 172.21.2.104:0/2574530588 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f21f0041f00 con 0x7f21f80b3910 2021-07-11T04:00:18.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.313+0000 7f21ffc12700 1 -- 172.21.2.104:0/2574530588 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f21f80aa770 con 0x7f21f80b3910 2021-07-11T04:00:18.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21e6ffd700 1 -- 172.21.2.104:0/2574530588 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v87) v1 ==== 110+0+10 (secure 0 0 0) 0x7f21f0019030 con 0x7f21f80b3910 2021-07-11T04:00:18.321 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:00:18.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 -- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21d80532b0 msgr2=0x7f21d8055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21d80532b0 0x7f21d8055770 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f21f4000f70 tx=0x7f21f40069b0).stop 2021-07-11T04:00:18.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 -- 172.21.2.104:0/2574530588 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21f80b3910 msgr2=0x7f21f813acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21f80b3910 0x7f21f813acf0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f21e8001370 tx=0x7f21f0007f40).stop 2021-07-11T04:00:18.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 -- 172.21.2.104:0/2574530588 shutdown_connections 2021-07-11T04:00:18.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21d80532b0 0x7f21d8055770 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f21f80b3910 0x7f21f813acf0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f21f80aada0 0x7f21f813a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 --2- 172.21.2.104:0/2574530588 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f21f80b3530 0x7f21f80b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 -- 172.21.2.104:0/2574530588 >> 172.21.2.104:0/2574530588 conn(0x7f21f805f6b0 msgr2=0x7f21f813db40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 -- 172.21.2.104:0/2574530588 shutdown_connections 2021-07-11T04:00:18.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.317+0000 7f21ffc12700 1 -- 172.21.2.104:0/2574530588 wait complete. 2021-07-11T04:00:18.328 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:00:18.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.377+0000 7f348590c700 1 Processor -- start 2021-07-11T04:00:18.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.377+0000 7f348590c700 1 -- start start 2021-07-11T04:00:18.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.377+0000 7f348590c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f34800603d0 0x7f34800aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.377+0000 7f348590c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f348013f790 0x7f348013fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.377+0000 7f348590c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3480061990 con 0x7f34800a9ec0 2021-07-11T04:00:18.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.377+0000 7f348590c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34801400f0 con 0x7f34800603d0 2021-07-11T04:00:18.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.377+0000 7f348590c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3480061c30 con 0x7f348013f790 2021-07-11T04:00:18.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.377+0000 7f347e7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f34800603d0 0x7f34800aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.377+0000 7f347effd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f34800a9ec0 0x7f348013ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44088/0 (socket says 172.21.2.104:44088) 2021-07-11T04:00:18.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.377+0000 7f347effd700 1 -- 172.21.2.104:0/501856898 learned_addr learned my addr 172.21.2.104:0/501856898 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:18.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 -- 172.21.2.104:0/501856898 >> v2:172.21.2.105:3300/0 conn(0x7f348013f790 msgr2=0x7f348013fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:18.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 --2- 172.21.2.104:0/501856898 >> v2:172.21.2.105:3300/0 conn(0x7f348013f790 0x7f348013fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 -- 172.21.2.104:0/501856898 >> v1:172.21.2.104:6789/0 conn(0x7f34800a9ec0 legacy=0x7f348013ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 -- 172.21.2.104:0/501856898 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480060b90 con 0x7f34800603d0 2021-07-11T04:00:18.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 --2- 172.21.2.104:0/501856898 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34800603d0 0x7f34800aa2a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f34800aaaf0 tx=0x7f3470006df0).ready entity=mon.0 client_cookie=7eac68b2d76ba6ec server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347dffb700 1 -- 172.21.2.104:0/501856898 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f347000d5c0 con 0x7f34800603d0 2021-07-11T04:00:18.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347dffb700 1 -- 172.21.2.104:0/501856898 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3470000e90 con 0x7f34800603d0 2021-07-11T04:00:18.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347dffb700 1 -- 172.21.2.104:0/501856898 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f347000d910 con 0x7f34800603d0 2021-07-11T04:00:18.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 -- 172.21.2.104:0/501856898 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34800603d0 msgr2=0x7f34800aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 --2- 172.21.2.104:0/501856898 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34800603d0 0x7f34800aa2a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f34800aaaf0 tx=0x7f3470006df0).stop 2021-07-11T04:00:18.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 -- 172.21.2.104:0/501856898 shutdown_connections 2021-07-11T04:00:18.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 --2- 172.21.2.104:0/501856898 >> v2:172.21.2.105:3300/0 conn(0x7f348013f790 0x7f348013fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 --2- 172.21.2.104:0/501856898 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34800603d0 0x7f34800aa2a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 -- 172.21.2.104:0/501856898 >> 172.21.2.104:0/501856898 conn(0x7f34800b2970 msgr2=0x7f34800b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 -- 172.21.2.104:0/501856898 shutdown_connections 2021-07-11T04:00:18.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 -- 172.21.2.104:0/501856898 wait complete. 2021-07-11T04:00:18.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 Processor -- start 2021-07-11T04:00:18.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 -- start start 2021-07-11T04:00:18.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34800603d0 0x7f34800b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34800a9ec0 0x7f348013ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f348013f790 0x7f348013b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34800aeea0 con 0x7f34800a9ec0 2021-07-11T04:00:18.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34800af3f0 con 0x7f348013f790 2021-07-11T04:00:18.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34800af620 con 0x7f34800603d0 2021-07-11T04:00:18.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34800a9ec0 0x7f348013ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34800a9ec0 0x7f348013ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47328/0 (socket says 172.21.2.104:47328) 2021-07-11T04:00:18.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 -- 172.21.2.104:0/662803711 learned_addr learned my addr 172.21.2.104:0/662803711 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:18.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 -- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34800603d0 msgr2=0x7f34800b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34800603d0 0x7f34800b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 -- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f348013f790 msgr2=0x7f348013b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f348013f790 0x7f348013b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 -- 172.21.2.104:0/662803711 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347000b180 con 0x7f34800a9ec0 2021-07-11T04:00:18.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f347e7fc700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34800a9ec0 0x7f348013ada0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f34800aaaf0 tx=0x7f3470012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f3477fff700 1 -- 172.21.2.104:0/662803711 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3470006b20 con 0x7f34800a9ec0 2021-07-11T04:00:18.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f3477fff700 1 -- 172.21.2.104:0/662803711 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3470017030 con 0x7f34800a9ec0 2021-07-11T04:00:18.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 -- 172.21.2.104:0/662803711 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34800af8c0 con 0x7f34800a9ec0 2021-07-11T04:00:18.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f3477fff700 1 -- 172.21.2.104:0/662803711 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3470002f70 con 0x7f34800a9ec0 2021-07-11T04:00:18.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.381+0000 7f348590c700 1 -- 172.21.2.104:0/662803711 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348013b780 con 0x7f34800a9ec0 2021-07-11T04:00:18.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.385+0000 7f348590c700 1 -- 172.21.2.104:0/662803711 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3480057350 con 0x7f34800a9ec0 2021-07-11T04:00:18.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.385+0000 7f3477fff700 1 -- 172.21.2.104:0/662803711 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3470024030 con 0x7f34800a9ec0 2021-07-11T04:00:18.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.385+0000 7f3477fff700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3464053300 0x7f34640557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.385+0000 7f3477fff700 1 -- 172.21.2.104:0/662803711 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f347006c320 con 0x7f34800a9ec0 2021-07-11T04:00:18.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.385+0000 7f347effd700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3464053300 0x7f34640557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.385+0000 7f347effd700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3464053300 0x7f34640557c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f34800abc40 tx=0x7f3468007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.385+0000 7f3477fff700 1 -- 172.21.2.104:0/662803711 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3470043590 con 0x7f34800a9ec0 2021-07-11T04:00:18.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 -- 172.21.2.104:0/662803711 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3480044650 con 0x7f34800a9ec0 2021-07-11T04:00:18.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f3477fff700 1 -- 172.21.2.104:0/662803711 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3470046b70 con 0x7f34800a9ec0 2021-07-11T04:00:18.473 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:18.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 -- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3464053300 msgr2=0x7f34640557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3464053300 0x7f34640557c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7f34800abc40 tx=0x7f3468007800).stop 2021-07-11T04:00:18.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 -- 172.21.2.104:0/662803711 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34800a9ec0 msgr2=0x7f348013ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34800a9ec0 0x7f348013ada0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f34800aaaf0 tx=0x7f3470012000).stop 2021-07-11T04:00:18.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 -- 172.21.2.104:0/662803711 shutdown_connections 2021-07-11T04:00:18.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f348013f790 0x7f348013b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3464053300 0x7f34640557c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34800a9ec0 0x7f348013ada0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 --2- 172.21.2.104:0/662803711 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34800603d0 0x7f34800b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 -- 172.21.2.104:0/662803711 >> 172.21.2.104:0/662803711 conn(0x7f34800b2970 msgr2=0x7f34800aab30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 -- 172.21.2.104:0/662803711 shutdown_connections 2021-07-11T04:00:18.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.469+0000 7f348590c700 1 -- 172.21.2.104:0/662803711 wait complete. 2021-07-11T04:00:18.484 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:00:18.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0aa0d42700 1 Processor -- start 2021-07-11T04:00:18.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0aa0d42700 1 -- start start 2021-07-11T04:00:18.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0aa0d42700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0a9c0b1ec0 0x7f0a9c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0aa0d42700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a9c0affb0 0x7f0a9c0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0aa0d42700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a9c0b0420 con 0x7f0a9c0a70c0 2021-07-11T04:00:18.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0aa0d42700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a9c0b2790 con 0x7f0a9c0b1ec0 2021-07-11T04:00:18.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0aa0d42700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c0b0650 con 0x7f0a9c0affb0 2021-07-11T04:00:18.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0a9ad9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f0a9c0a70c0 0x7f0a9c141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44106/0 (socket says 172.21.2.104:44106) 2021-07-11T04:00:18.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0a9ad9d700 1 -- 172.21.2.104:0/2230022697 learned_addr learned my addr 172.21.2.104:0/2230022697 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:18.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0a9a59c700 1 --2- 172.21.2.104:0/2230022697 >> v2:172.21.2.104:3300/0 conn(0x7f0a9c0b1ec0 0x7f0a9c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0a9a59c700 1 -- 172.21.2.104:0/2230022697 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a9c0affb0 msgr2=0x7f0a9c0b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:18.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0a9a59c700 1 --2- 172.21.2.104:0/2230022697 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a9c0affb0 0x7f0a9c0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0a9a59c700 1 -- 172.21.2.104:0/2230022697 >> v1:172.21.2.104:6789/0 conn(0x7f0a9c0a70c0 legacy=0x7f0a9c141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0a9a59c700 1 -- 172.21.2.104:0/2230022697 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0b08f0 con 0x7f0a9c0b1ec0 2021-07-11T04:00:18.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0a9a59c700 1 --2- 172.21.2.104:0/2230022697 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a9c0b1ec0 0x7f0a9c0b12d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f0a84002f70 tx=0x7f0a84006df0).ready entity=mon.0 client_cookie=210974f16396d9b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.533+0000 7f0a9959a700 1 -- 172.21.2.104:0/2230022697 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a8400d570 con 0x7f0a9c0b1ec0 2021-07-11T04:00:18.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a9959a700 1 -- 172.21.2.104:0/2230022697 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a84000e90 con 0x7f0a9c0b1ec0 2021-07-11T04:00:18.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a9959a700 1 -- 172.21.2.104:0/2230022697 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a8400d8c0 con 0x7f0a9c0b1ec0 2021-07-11T04:00:18.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- 172.21.2.104:0/2230022697 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a9c0b1ec0 msgr2=0x7f0a9c0b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 --2- 172.21.2.104:0/2230022697 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a9c0b1ec0 0x7f0a9c0b12d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f0a84002f70 tx=0x7f0a84006df0).stop 2021-07-11T04:00:18.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- 172.21.2.104:0/2230022697 shutdown_connections 2021-07-11T04:00:18.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 --2- 172.21.2.104:0/2230022697 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a9c0affb0 0x7f0a9c0b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 --2- 172.21.2.104:0/2230022697 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a9c0b1ec0 0x7f0a9c0b12d0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- 172.21.2.104:0/2230022697 >> 172.21.2.104:0/2230022697 conn(0x7f0a9c05f6b0 msgr2=0x7f0a9c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- 172.21.2.104:0/2230022697 shutdown_connections 2021-07-11T04:00:18.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- 172.21.2.104:0/2230022697 wait complete. 2021-07-11T04:00:18.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 Processor -- start 2021-07-11T04:00:18.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- start start 2021-07-11T04:00:18.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a9c0a70c0 0x7f0a9c0aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a9c0affb0 0x7f0a9c149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a9c0b1ec0 0x7f0a9c0aab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c14a720 con 0x7f0a9c0affb0 2021-07-11T04:00:18.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c1575e0 con 0x7f0a9c0a70c0 2021-07-11T04:00:18.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9c157880 con 0x7f0a9c0b1ec0 2021-07-11T04:00:18.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a99d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a9c0affb0 0x7f0a9c149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a99d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a9c0affb0 0x7f0a9c149f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47344/0 (socket says 172.21.2.104:47344) 2021-07-11T04:00:18.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a99d9b700 1 -- 172.21.2.104:0/385869968 learned_addr learned my addr 172.21.2.104:0/385869968 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:18.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a99d9b700 1 -- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a9c0b1ec0 msgr2=0x7f0a9c0aab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a99d9b700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a9c0b1ec0 0x7f0a9c0aab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a99d9b700 1 -- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a9c0a70c0 msgr2=0x7f0a9c0aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a99d9b700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a9c0a70c0 0x7f0a9c0aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a99d9b700 1 -- 172.21.2.104:0/385869968 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8400b180 con 0x7f0a9c0affb0 2021-07-11T04:00:18.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a99d9b700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a9c0affb0 0x7f0a9c149f50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f0a8c002f70 tx=0x7f0a8c006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a937fe700 1 -- 172.21.2.104:0/385869968 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a8c013460 con 0x7f0a9c0affb0 2021-07-11T04:00:18.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a937fe700 1 -- 172.21.2.104:0/385869968 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8c0135c0 con 0x7f0a9c0affb0 2021-07-11T04:00:18.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a937fe700 1 -- 172.21.2.104:0/385869968 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a8c013920 con 0x7f0a9c0affb0 2021-07-11T04:00:18.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- 172.21.2.104:0/385869968 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a9c0a9c90 con 0x7f0a9c0affb0 2021-07-11T04:00:18.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- 172.21.2.104:0/385869968 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9c1572a0 con 0x7f0a9c0affb0 2021-07-11T04:00:18.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0aa0d42700 1 -- 172.21.2.104:0/385869968 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c057350 con 0x7f0a9c0affb0 2021-07-11T04:00:18.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a937fe700 1 -- 172.21.2.104:0/385869968 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0a8c01a060 con 0x7f0a9c0affb0 2021-07-11T04:00:18.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a937fe700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a80053350 0x7f0a80055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.537+0000 7f0a937fe700 1 -- 172.21.2.104:0/385869968 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0a8c06b0b0 con 0x7f0a9c0affb0 2021-07-11T04:00:18.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.541+0000 7f0a9a59c700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a80053350 0x7f0a80055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.541+0000 7f0a9a59c700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a80053350 0x7f0a80055810 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f0a84002f70 tx=0x7f0a84015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.541+0000 7f0a937fe700 1 -- 172.21.2.104:0/385869968 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0a8c0422e0 con 0x7f0a9c0affb0 2021-07-11T04:00:18.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 -- 172.21.2.104:0/385869968 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a9c0ac760 con 0x7f0a9c0affb0 2021-07-11T04:00:18.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0a937fe700 1 -- 172.21.2.104:0/385869968 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+10 (secure 0 0 0) 0x7f0a8c018070 con 0x7f0a9c0affb0 2021-07-11T04:00:18.630 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:18.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 -- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a80053350 msgr2=0x7f0a80055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a80053350 0x7f0a80055810 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f0a84002f70 tx=0x7f0a84015000).stop 2021-07-11T04:00:18.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 -- 172.21.2.104:0/385869968 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a9c0affb0 msgr2=0x7f0a9c149f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a9c0affb0 0x7f0a9c149f50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f0a8c002f70 tx=0x7f0a8c006ec0).stop 2021-07-11T04:00:18.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 -- 172.21.2.104:0/385869968 shutdown_connections 2021-07-11T04:00:18.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a9c0b1ec0 0x7f0a9c0aab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a80053350 0x7f0a80055810 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a9c0affb0 0x7f0a9c149f50 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 --2- 172.21.2.104:0/385869968 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a9c0a70c0 0x7f0a9c0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 -- 172.21.2.104:0/385869968 >> 172.21.2.104:0/385869968 conn(0x7f0a9c05f6b0 msgr2=0x7f0a9c146dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 -- 172.21.2.104:0/385869968 shutdown_connections 2021-07-11T04:00:18.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.625+0000 7f0aa0d42700 1 -- 172.21.2.104:0/385869968 wait complete. 2021-07-11T04:00:18.638 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:00:18.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 Processor -- start 2021-07-11T04:00:18.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 -- start start 2021-07-11T04:00:18.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9cdc060100 0x7f9cdc064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cdc0b38d0 0x7f9cdc065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cdc0604e0 con 0x7f9cdc144440 2021-07-11T04:00:18.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cdc0656f0 con 0x7f9cdc060100 2021-07-11T04:00:18.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc065c80 con 0x7f9cdc0b38d0 2021-07-11T04:00:18.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce08d4700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f9cdc144440 0x7f9cdc144970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44120/0 (socket says 172.21.2.104:44120) 2021-07-11T04:00:18.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce08d4700 1 -- 172.21.2.104:0/4024426206 learned_addr learned my addr 172.21.2.104:0/4024426206 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:18.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce10d5700 1 --2- 172.21.2.104:0/4024426206 >> v2:172.21.2.104:3300/0 conn(0x7f9cdc060100 0x7f9cdc064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ccbfff700 1 -- 172.21.2.104:0/4024426206 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279160070 0 0) 0x7f9cdc0604e0 con 0x7f9cdc144440 2021-07-11T04:00:18.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce10d5700 1 -- 172.21.2.104:0/4024426206 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cdc0b38d0 msgr2=0x7f9cdc065200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:18.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce10d5700 1 --2- 172.21.2.104:0/4024426206 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cdc0b38d0 0x7f9cdc065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce10d5700 1 -- 172.21.2.104:0/4024426206 >> v1:172.21.2.104:6789/0 conn(0x7f9cdc144440 legacy=0x7f9cdc144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce10d5700 1 -- 172.21.2.104:0/4024426206 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc0b23b0 con 0x7f9cdc060100 2021-07-11T04:00:18.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce10d5700 1 --2- 172.21.2.104:0/4024426206 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cdc060100 0x7f9cdc064d10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f9cd8002f70 tx=0x7f9cd8006df0).ready entity=mon.0 client_cookie=f81ef3fe8fd193b6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ccbfff700 1 -- 172.21.2.104:0/4024426206 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cd800d570 con 0x7f9cdc060100 2021-07-11T04:00:18.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ccbfff700 1 -- 172.21.2.104:0/4024426206 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd8000e90 con 0x7f9cdc060100 2021-07-11T04:00:18.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ccbfff700 1 -- 172.21.2.104:0/4024426206 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cd800d900 con 0x7f9cdc060100 2021-07-11T04:00:18.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 -- 172.21.2.104:0/4024426206 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cdc060100 msgr2=0x7f9cdc064d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 --2- 172.21.2.104:0/4024426206 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cdc060100 0x7f9cdc064d10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f9cd8002f70 tx=0x7f9cd8006df0).stop 2021-07-11T04:00:18.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 -- 172.21.2.104:0/4024426206 shutdown_connections 2021-07-11T04:00:18.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 --2- 172.21.2.104:0/4024426206 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cdc0b38d0 0x7f9cdc065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 --2- 172.21.2.104:0/4024426206 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cdc060100 0x7f9cdc064d10 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 -- 172.21.2.104:0/4024426206 >> 172.21.2.104:0/4024426206 conn(0x7f9cdc05cf60 msgr2=0x7f9cdc05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 -- 172.21.2.104:0/4024426206 shutdown_connections 2021-07-11T04:00:18.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 -- 172.21.2.104:0/4024426206 wait complete. 2021-07-11T04:00:18.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.689+0000 7f9ce3339700 1 Processor -- start 2021-07-11T04:00:18.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce3339700 1 -- start start 2021-07-11T04:00:18.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce3339700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cdc060100 0x7f9cdc12f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce3339700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cdc0b38d0 0x7f9cdc0b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce3339700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cdc0b0520 0x7f9cdc0b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce3339700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc0af7b0 con 0x7f9cdc0b0520 2021-07-11T04:00:18.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce3339700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc0b5b10 con 0x7f9cdc0b38d0 2021-07-11T04:00:18.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce3339700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc0b60b0 con 0x7f9cdc060100 2021-07-11T04:00:18.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce18d6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cdc0b0520 0x7f9cdc0b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce10d5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cdc060100 0x7f9cdc12f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce18d6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cdc0b0520 0x7f9cdc0b6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47358/0 (socket says 172.21.2.104:47358) 2021-07-11T04:00:18.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce18d6700 1 -- 172.21.2.104:0/2077693214 learned_addr learned my addr 172.21.2.104:0/2077693214 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:18.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce08d4700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cdc0b38d0 0x7f9cdc0b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce18d6700 1 -- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cdc060100 msgr2=0x7f9cdc12f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce18d6700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cdc060100 0x7f9cdc12f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce18d6700 1 -- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cdc0b38d0 msgr2=0x7f9cdc0b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce18d6700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cdc0b38d0 0x7f9cdc0b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce18d6700 1 -- 172.21.2.104:0/2077693214 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd800b180 con 0x7f9cdc0b0520 2021-07-11T04:00:18.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce10d5700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cdc060100 0x7f9cdc12f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:00:18.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce18d6700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cdc0b0520 0x7f9cdc0b6ad0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f9cdc0af5c0 tx=0x7f9cd40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9cca7fc700 1 -- 172.21.2.104:0/2077693214 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cd400f510 con 0x7f9cdc0b0520 2021-07-11T04:00:18.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9cca7fc700 1 -- 172.21.2.104:0/2077693214 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd400f670 con 0x7f9cdc0b0520 2021-07-11T04:00:18.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce3339700 1 -- 172.21.2.104:0/2077693214 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cdc0b6350 con 0x7f9cdc0b0520 2021-07-11T04:00:18.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9cca7fc700 1 -- 172.21.2.104:0/2077693214 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9cd400f7e0 con 0x7f9cdc0b0520 2021-07-11T04:00:18.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce3339700 1 -- 172.21.2.104:0/2077693214 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cdc12fab0 con 0x7f9cdc0b0520 2021-07-11T04:00:18.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce3339700 1 -- 172.21.2.104:0/2077693214 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cdc057350 con 0x7f9cdc0b0520 2021-07-11T04:00:18.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9cca7fc700 1 -- 172.21.2.104:0/2077693214 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9cd400f070 con 0x7f9cdc0b0520 2021-07-11T04:00:18.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9cca7fc700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9cbc053300 0x7f9cbc0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9cca7fc700 1 -- 172.21.2.104:0/2077693214 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9cd406b070 con 0x7f9cdc0b0520 2021-07-11T04:00:18.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce10d5700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9cbc053300 0x7f9cbc0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.693+0000 7f9ce10d5700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9cbc053300 0x7f9cbc0557c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f9cd801b000 tx=0x7f9cd8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.697+0000 7f9cca7fc700 1 -- 172.21.2.104:0/2077693214 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9cd40423b0 con 0x7f9cdc0b0520 2021-07-11T04:00:18.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 -- 172.21.2.104:0/2077693214 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9cdc0698e0 con 0x7f9cdc0b0520 2021-07-11T04:00:18.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9cca7fc700 1 -- 172.21.2.104:0/2077693214 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9cd40459d0 con 0x7f9cdc0b0520 2021-07-11T04:00:18.786 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:18.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 -- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9cbc053300 msgr2=0x7f9cbc0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9cbc053300 0x7f9cbc0557c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f9cd801b000 tx=0x7f9cd8015000).stop 2021-07-11T04:00:18.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 -- 172.21.2.104:0/2077693214 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cdc0b0520 msgr2=0x7f9cdc0b6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cdc0b0520 0x7f9cdc0b6ad0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f9cdc0af5c0 tx=0x7f9cd40091d0).stop 2021-07-11T04:00:18.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 -- 172.21.2.104:0/2077693214 shutdown_connections 2021-07-11T04:00:18.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9cbc053300 0x7f9cbc0557c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9cdc0b0520 0x7f9cdc0b6ad0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9cdc0b38d0 0x7f9cdc0b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 --2- 172.21.2.104:0/2077693214 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9cdc060100 0x7f9cdc12f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 -- 172.21.2.104:0/2077693214 >> 172.21.2.104:0/2077693214 conn(0x7f9cdc05cf60 msgr2=0x7f9cdc13ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 -- 172.21.2.104:0/2077693214 shutdown_connections 2021-07-11T04:00:18.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.781+0000 7f9ce3339700 1 -- 172.21.2.104:0/2077693214 wait complete. 2021-07-11T04:00:18.792 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:00:18.794 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:18.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 Processor -- start 2021-07-11T04:00:18.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 -- start start 2021-07-11T04:00:18.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f99680ad150 0x7f99680b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680ac370 con 0x7f99680a9da0 2021-07-11T04:00:18.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680aac40 con 0x7f99680aa860 2021-07-11T04:00:18.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99680abb50 con 0x7f99680ad150 2021-07-11T04:00:18.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f99677fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f99680a9da0 0x7f9968140340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44132/0 (socket says 172.21.2.104:44132) 2021-07-11T04:00:18.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f99677fe700 1 -- 172.21.2.104:0/3758466761 learned_addr learned my addr 172.21.2.104:0/3758466761 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:18.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 131567734 0 0) 0x7f99680ac370 con 0x7f99680a9da0 2021-07-11T04:00:18.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f994c003260 con 0x7f99680a9da0 2021-07-11T04:00:18.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1281314337 0 0) 0x7f994c003260 con 0x7f99680a9da0 2021-07-11T04:00:18.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 >> v2:172.21.2.105:3300/0 conn(0x7f99680ad150 msgr2=0x7f99680b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:18.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 --2- 172.21.2.104:0/3758466761 >> v2:172.21.2.105:3300/0 conn(0x7f99680ad150 0x7f99680b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 >> v1:172.21.2.105:6789/0 conn(0x7f99680aa860 legacy=0x7f99680aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:18.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99680b16b0 con 0x7f99680a9da0 2021-07-11T04:00:18.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f99600019e0 con 0x7f99680a9da0 2021-07-11T04:00:18.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 >> v1:172.21.2.104:6789/0 conn(0x7f99680a9da0 legacy=0x7f9968140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 --2- 172.21.2.104:0/3758466761 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f994c003820 0x7f994c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 --2- 172.21.2.104:0/3758466761 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f994c004500 0x7f994c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 --2- 172.21.2.104:0/3758466761 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f994c004ec0 0x7f994c00fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c010330 con 0x7f994c004ec0 2021-07-11T04:00:18.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c0104a0 con 0x7f994c003820 2021-07-11T04:00:18.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994c010660 con 0x7f994c004500 2021-07-11T04:00:18.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f99677fe700 1 --2- 172.21.2.104:0/3758466761 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f994c004ec0 0x7f994c00fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f99677fe700 1 -- 172.21.2.104:0/3758466761 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f994c004500 msgr2=0x7f994c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:18.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f99677fe700 1 --2- 172.21.2.104:0/3758466761 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f994c004500 0x7f994c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f99677fe700 1 -- 172.21.2.104:0/3758466761 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f994c003820 msgr2=0x7f994c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:18.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f99677fe700 1 --2- 172.21.2.104:0/3758466761 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f994c003820 0x7f994c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f99677fe700 1 -- 172.21.2.104:0/3758466761 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c003260 con 0x7f994c004ec0 2021-07-11T04:00:18.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f99677fe700 1 -- 172.21.2.104:0/3758466761 reap_dead start 2021-07-11T04:00:18.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f99677fe700 1 --2- 172.21.2.104:0/3758466761 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f994c004ec0 0x7f994c00fd60 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f99680b63e0 tx=0x7f99600020e0).ready entity=mon.0 client_cookie=7daada92c13f78f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996000cc60 con 0x7f994c004ec0 2021-07-11T04:00:18.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996000cdc0 con 0x7f994c004ec0 2021-07-11T04:00:18.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f9965ffb700 1 -- 172.21.2.104:0/3758466761 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9960005b80 con 0x7f994c004ec0 2021-07-11T04:00:18.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 -- 172.21.2.104:0/3758466761 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f994c004ec0 msgr2=0x7f994c00fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 --2- 172.21.2.104:0/3758466761 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f994c004ec0 0x7f994c00fd60 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f99680b63e0 tx=0x7f99600020e0).stop 2021-07-11T04:00:18.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 -- 172.21.2.104:0/3758466761 shutdown_connections 2021-07-11T04:00:18.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 --2- 172.21.2.104:0/3758466761 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f994c004ec0 0x7f994c00fd60 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 -- 172.21.2.104:0/3758466761 >> 172.21.2.104:0/3758466761 conn(0x7f99680b64d0 msgr2=0x7f99680b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 -- 172.21.2.104:0/3758466761 shutdown_connections 2021-07-11T04:00:18.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.841+0000 7f996d7e7700 1 -- 172.21.2.104:0/3758466761 wait complete. 2021-07-11T04:00:18.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f996d7e7700 1 Processor -- start 2021-07-11T04:00:18.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f996d7e7700 1 -- start start 2021-07-11T04:00:18.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f996d7e7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f994c004ec0 0x7f996813f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f996d7e7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f996813b9c0 0x7f996813be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f996d7e7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f996813c310 0x7f996814dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f996d7e7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996814d990 con 0x7f996813b9c0 2021-07-11T04:00:18.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f996d7e7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99680aee90 con 0x7f994c004ec0 2021-07-11T04:00:18.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f996d7e7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99680ae870 con 0x7f996813c310 2021-07-11T04:00:18.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f99667fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f996813b9c0 0x7f996813be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f99667fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f996813b9c0 0x7f996813be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47374/0 (socket says 172.21.2.104:47374) 2021-07-11T04:00:18.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f99667fc700 1 -- 172.21.2.104:0/1225457497 learned_addr learned my addr 172.21.2.104:0/1225457497 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:18.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f99667fc700 1 -- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f996813c310 msgr2=0x7f996814dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f99667fc700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f996813c310 0x7f996814dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f99667fc700 1 -- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f994c004ec0 msgr2=0x7f996813f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f99667fc700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f994c004ec0 0x7f996813f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f99667fc700 1 -- 172.21.2.104:0/1225457497 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99680ae5d0 con 0x7f996813b9c0 2021-07-11T04:00:18.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f99667fc700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f996813b9c0 0x7f996813be20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f99680aeb10 tx=0x7f9958006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f9957fff700 1 -- 172.21.2.104:0/1225457497 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9958013460 con 0x7f996813b9c0 2021-07-11T04:00:18.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f9957fff700 1 -- 172.21.2.104:0/1225457497 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99580135c0 con 0x7f996813b9c0 2021-07-11T04:00:18.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f996d7e7700 1 -- 172.21.2.104:0/1225457497 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99680aebf0 con 0x7f996813b9c0 2021-07-11T04:00:18.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f9957fff700 1 -- 172.21.2.104:0/1225457497 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9958013920 con 0x7f996813b9c0 2021-07-11T04:00:18.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f996d7e7700 1 -- 172.21.2.104:0/1225457497 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99680ad610 con 0x7f996813b9c0 2021-07-11T04:00:18.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f996d7e7700 1 -- 172.21.2.104:0/1225457497 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9968063880 con 0x7f996813b9c0 2021-07-11T04:00:18.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f9957fff700 1 -- 172.21.2.104:0/1225457497 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f995801a060 con 0x7f996813b9c0 2021-07-11T04:00:18.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f9957fff700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9944053300 0x7f99440557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:18.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f9957fff700 1 -- 172.21.2.104:0/1225457497 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f995806aec0 con 0x7f996813b9c0 2021-07-11T04:00:18.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.845+0000 7f9966ffd700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9944053300 0x7f99440557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:18.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.849+0000 7f9966ffd700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9944053300 0x7f99440557c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f995c000f70 tx=0x7f995c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:18.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.849+0000 7f9957fff700 1 -- 172.21.2.104:0/1225457497 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9958042200 con 0x7f996813b9c0 2021-07-11T04:00:18.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 -- 172.21.2.104:0/1225457497 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f996814dae0 con 0x7f996813b9c0 2021-07-11T04:00:18.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f9957fff700 1 -- 172.21.2.104:0/1225457497 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f9958045820 con 0x7f996813b9c0 2021-07-11T04:00:18.963 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:18.964 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/43570850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":43570850},{"type":"v1","addr":"172.21.2.105:6835","nonce":43570850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/3334136782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3334136782},{"type":"v1","addr":"172.21.2.104:6833","nonce":3334136782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:15.674524+0000","modified":"2021-07-11T04:00:16.678211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/4286166182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":4286166182},{"type":"v1","addr":"172.21.2.104:6835","nonce":4286166182}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-07-11T04:00:18.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 -- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9944053300 msgr2=0x7f99440557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9944053300 0x7f99440557c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f995c000f70 tx=0x7f995c0069b0).stop 2021-07-11T04:00:18.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 -- 172.21.2.104:0/1225457497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f996813b9c0 msgr2=0x7f996813be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:18.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f996813b9c0 0x7f996813be20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f99680aeb10 tx=0x7f9958006e40).stop 2021-07-11T04:00:18.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 -- 172.21.2.104:0/1225457497 shutdown_connections 2021-07-11T04:00:18.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f996813c310 0x7f996814dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9944053300 0x7f99440557c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f996813b9c0 0x7f996813be20 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:18.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 --2- 172.21.2.104:0/1225457497 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f994c004ec0 0x7f996813f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:18.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 -- 172.21.2.104:0/1225457497 >> 172.21.2.104:0/1225457497 conn(0x7f99680b64d0 msgr2=0x7f99680ab430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:18.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 -- 172.21.2.104:0/1225457497 shutdown_connections 2021-07-11T04:00:18.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:18.957+0000 7f996d7e7700 1 -- 172.21.2.104:0/1225457497 wait complete. 2021-07-11T04:00:18.968 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 39 2021-07-11T04:00:19.272 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:00:15.674524+0000', 'modified': '2021-07-11T04:00:16.678211+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6636}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6636': {'gid': 6636, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/4286166182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 4286166182}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 4286166182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:00:19.273 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:00:19.274 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-07-11T04:00:19.340 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":6636,"want_state":"up:active","state":"up:active","rank_uptime":3.657813762,"mdsmap_epoch":39,"osdmap_epoch":87,"osdmap_epoch_barrier":86,"uptime":7.877390653} 2021-07-11T04:00:19.341 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 6636, "mdsmap_epoch": 39, "osdmap_epoch": 87, "osdmap_epoch_barrier": 86, "rank_uptime": 3.657813762, "state": "up:active", "uptime": 7.877390653, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:00:19.341 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:00:19.342 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T04:00:19.342 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T04:00:19.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:19.343 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:19.356 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:00:19.388 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:19.389 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:19.445 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:19.447 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:19.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:19.448 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:00:19.461 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:00:19.462 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:19.471 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:00:19.485 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:19.642 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:19.643 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T04:00:19.511+0000 7f4683cd1080 -1 init, newargv = 0x56170f1df210 newargc=9 2021-07-11T04:00:19.644 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15312]: starting ceph client 2021-07-11T04:00:19.644 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15312]: starting fuse 2021-07-11T04:00:19.655 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:19.655 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:19.656 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:19.657 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:00:19.702 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T04:00:19.703 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:00:19.704 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:19.704 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T04:00:19.705 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T04:00:19.705 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T04:00:19.706 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T04:00:19.706 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T04:00:19.707 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:19.707 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T04:00:19.708 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T04:00:19.708 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T04:00:19.708 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:19.709 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T04:00:19.709 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T04:00:19.710 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T04:00:19.710 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T04:00:19.710 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:19.711 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T04:00:19.711 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:00:19.712 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:00:19.712 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T04:00:19.712 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:00:19.713 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:19.713 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T04:00:19.714 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T04:00:19.767 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15312.asok 2021-07-11T04:00:19.771 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15312.asok 2021-07-11T04:00:19.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:19.773 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15312.asok status 2021-07-11T04:00:19.823 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T04:00:19.823 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T04:00:19.823 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:00:19.824 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:00:19.824 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T04:00:19.824 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T04:00:19.824 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T04:00:19.824 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15312", 2021-07-11T04:00:19.825 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T04:00:19.825 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:19.825 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T04:00:19.825 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T04:00:19.825 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4972, 2021-07-11T04:00:19.826 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T04:00:19.826 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T04:00:19.826 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T04:00:19.826 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4972 2021-07-11T04:00:19.827 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:19.827 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:00:19.827 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:00:19.828 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:00:19.828 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2422503571 2021-07-11T04:00:19.828 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T04:00:19.828 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:19.828 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:00:19.829 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:00:19.829 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:00:19.829 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 2422503571 2021-07-11T04:00:19.829 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:19.830 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4972 172.21.2.119:0/2422503571", 2021-07-11T04:00:19.830 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/2422503571", 2021-07-11T04:00:19.830 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T04:00:19.830 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 39, 2021-07-11T04:00:19.830 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 87, 2021-07-11T04:00:19.831 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:00:19.831 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T04:00:19.831 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T04:00:19.831 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T04:00:19.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:19.833 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:19.834 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:19.840 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:00:19.840 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:00:19.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:19.841 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:19.850 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:19.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.901+0000 7fb28f48b700 1 Processor -- start 2021-07-11T04:00:19.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.901+0000 7fb28f48b700 1 -- start start 2021-07-11T04:00:19.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb2880a76b0 0x7fb288066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:19.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2880a7a90 con 0x7fb2880b1970 2021-07-11T04:00:19.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb288147620 con 0x7fb2880b0060 2021-07-11T04:00:19.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2881478c0 con 0x7fb2880a76b0 2021-07-11T04:00:19.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28ca26700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb2880b1970 0x7fb28805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44152/0 (socket says 172.21.2.104:44152) 2021-07-11T04:00:19.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28ca26700 1 -- 172.21.2.104:0/2840749037 learned_addr learned my addr 172.21.2.104:0/2840749037 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:19.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338479454 0 0) 0x7fb2880a7a90 con 0x7fb2880b1970 2021-07-11T04:00:19.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb270003260 con 0x7fb2880b1970 2021-07-11T04:00:19.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28d227700 1 --2- 172.21.2.104:0/2840749037 >> v2:172.21.2.105:3300/0 conn(0x7fb2880a76b0 0x7fb288066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:19.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3657558343 0 0) 0x7fb270003260 con 0x7fb2880b1970 2021-07-11T04:00:19.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 >> v2:172.21.2.105:3300/0 conn(0x7fb2880a76b0 msgr2=0x7fb288066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:19.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 --2- 172.21.2.104:0/2840749037 >> v2:172.21.2.105:3300/0 conn(0x7fb2880a76b0 0x7fb288066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:19.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 >> v1:172.21.2.105:6789/0 conn(0x7fb2880b0060 legacy=0x7fb288151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:19.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb288147b60 con 0x7fb2880b1970 2021-07-11T04:00:19.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fb27c001680 con 0x7fb2880b1970 2021-07-11T04:00:19.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 >> v1:172.21.2.104:6789/0 conn(0x7fb2880b1970 legacy=0x7fb28805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:19.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 --2- 172.21.2.104:0/2840749037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2700037d0 0x7fb270003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:19.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 --2- 172.21.2.104:0/2840749037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb270004500 0x7fb270004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:19.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 --2- 172.21.2.104:0/2840749037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb270004f90 0x7fb27000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:19.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2881478c0 con 0x7fb2700037d0 2021-07-11T04:00:19.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb27000be00 con 0x7fb270004500 2021-07-11T04:00:19.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27000bfc0 con 0x7fb270004f90 2021-07-11T04:00:19.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28d227700 1 --2- 172.21.2.104:0/2840749037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2700037d0 0x7fb270003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:19.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28ca26700 1 --2- 172.21.2.104:0/2840749037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb270004500 0x7fb270004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:19.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28d227700 1 -- 172.21.2.104:0/2840749037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb270004f90 msgr2=0x7fb27000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:19.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28d227700 1 --2- 172.21.2.104:0/2840749037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb270004f90 0x7fb27000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:19.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28d227700 1 -- 172.21.2.104:0/2840749037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb270004500 msgr2=0x7fb270004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:19.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28d227700 1 --2- 172.21.2.104:0/2840749037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb270004500 0x7fb270004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:19.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28d227700 1 -- 172.21.2.104:0/2840749037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb288147b60 con 0x7fb2700037d0 2021-07-11T04:00:19.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28da28700 1 -- 172.21.2.104:0/2840749037 reap_dead start 2021-07-11T04:00:19.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28ca26700 1 --2- 172.21.2.104:0/2840749037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb270004500 0x7fb270004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:19.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28d227700 1 --2- 172.21.2.104:0/2840749037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2700037d0 0x7fb270003bd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7fb28400a000 tx=0x7fb284008040).ready entity=mon.0 client_cookie=56bed281b8e2bf28 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:19.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb284013670 con 0x7fb2700037d0 2021-07-11T04:00:19.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2840137d0 con 0x7fb2700037d0 2021-07-11T04:00:19.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb27bfff700 1 -- 172.21.2.104:0/2840749037 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb284013b30 con 0x7fb2700037d0 2021-07-11T04:00:19.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 -- 172.21.2.104:0/2840749037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2700037d0 msgr2=0x7fb270003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:19.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 --2- 172.21.2.104:0/2840749037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2700037d0 0x7fb270003bd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7fb28400a000 tx=0x7fb284008040).stop 2021-07-11T04:00:19.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 -- 172.21.2.104:0/2840749037 shutdown_connections 2021-07-11T04:00:19.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 --2- 172.21.2.104:0/2840749037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2700037d0 0x7fb270003bd0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:19.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 -- 172.21.2.104:0/2840749037 >> 172.21.2.104:0/2840749037 conn(0x7fb2880b53e0 msgr2=0x7fb2880b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:19.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 -- 172.21.2.104:0/2840749037 shutdown_connections 2021-07-11T04:00:19.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 -- 172.21.2.104:0/2840749037 wait complete. 2021-07-11T04:00:19.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 Processor -- start 2021-07-11T04:00:19.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.905+0000 7fb28f48b700 1 -- start start 2021-07-11T04:00:19.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28f48b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb2700037d0 0x7fb2880a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:19.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28f48b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb2880adea0 0x7fb2880abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:19.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28f48b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2880ae920 0x7fb2880aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:19.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28f48b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2880ad590 con 0x7fb2880ae920 2021-07-11T04:00:19.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28f48b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2880ae3e0 con 0x7fb2880adea0 2021-07-11T04:00:19.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28f48b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2880ad7c0 con 0x7fb2700037d0 2021-07-11T04:00:19.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28da28700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2880ae920 0x7fb2880aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:19.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28da28700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2880ae920 0x7fb2880aeed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47396/0 (socket says 172.21.2.104:47396) 2021-07-11T04:00:19.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28da28700 1 -- 172.21.2.104:0/1812440631 learned_addr learned my addr 172.21.2.104:0/1812440631 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:19.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28d227700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb2700037d0 0x7fb2880a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:19.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28ca26700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb2880adea0 0x7fb2880abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:19.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28da28700 1 -- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb2700037d0 msgr2=0x7fb2880a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:19.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28da28700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb2700037d0 0x7fb2880a9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:19.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28da28700 1 -- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb2880adea0 msgr2=0x7fb2880abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:19.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28da28700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb2880adea0 0x7fb2880abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:19.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28da28700 1 -- 172.21.2.104:0/1812440631 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880ada60 con 0x7fb2880ae920 2021-07-11T04:00:19.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28da28700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2880ae920 0x7fb2880aeed0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fb27c001110 tx=0x7fb280007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:19.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb27a7fc700 1 -- 172.21.2.104:0/1812440631 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb28000ceb0 con 0x7fb2880ae920 2021-07-11T04:00:19.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb27a7fc700 1 -- 172.21.2.104:0/1812440631 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb280005a90 con 0x7fb2880ae920 2021-07-11T04:00:19.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb27a7fc700 1 -- 172.21.2.104:0/1812440631 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb280005e30 con 0x7fb2880ae920 2021-07-11T04:00:19.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28f48b700 1 -- 172.21.2.104:0/1812440631 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2880abfd0 con 0x7fb2880ae920 2021-07-11T04:00:19.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28f48b700 1 -- 172.21.2.104:0/1812440631 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2880af3c0 con 0x7fb2880ae920 2021-07-11T04:00:19.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28f48b700 1 -- 172.21.2.104:0/1812440631 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb288057350 con 0x7fb2880ae920 2021-07-11T04:00:19.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb27a7fc700 1 -- 172.21.2.104:0/1812440631 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb28001c070 con 0x7fb2880ae920 2021-07-11T04:00:19.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb27a7fc700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb268053330 0x7fb2680557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:19.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb27a7fc700 1 -- 172.21.2.104:0/1812440631 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb28006a750 con 0x7fb2880ae920 2021-07-11T04:00:19.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.909+0000 7fb28d227700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb268053330 0x7fb2680557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:19.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.913+0000 7fb28d227700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb268053330 0x7fb2680557f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fb284014000 tx=0x7fb284008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:19.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.913+0000 7fb27a7fc700 1 -- 172.21.2.104:0/1812440631 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb280041a90 con 0x7fb2880ae920 2021-07-11T04:00:19.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.993+0000 7fb28f48b700 1 -- 172.21.2.104:0/1812440631 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2880698e0 con 0x7fb2880ae920 2021-07-11T04:00:20.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.993+0000 7fb27a7fc700 1 -- 172.21.2.104:0/1812440631 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16041 (secure 0 0 0) 0x7fb2800450b0 con 0x7fb2880ae920 2021-07-11T04:00:20.000 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:20.001 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":87,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:16.679409+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:13.631492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:14.726743+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:20.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 -- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb268053330 msgr2=0x7fb2680557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb268053330 0x7fb2680557f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fb284014000 tx=0x7fb284008040).stop 2021-07-11T04:00:20.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 -- 172.21.2.104:0/1812440631 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2880ae920 msgr2=0x7fb2880aeed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2880ae920 0x7fb2880aeed0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fb27c001110 tx=0x7fb280007db0).stop 2021-07-11T04:00:20.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 -- 172.21.2.104:0/1812440631 shutdown_connections 2021-07-11T04:00:20.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb268053330 0x7fb2680557f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb2880ae920 0x7fb2880aeed0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb2880adea0 0x7fb2880abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 --2- 172.21.2.104:0/1812440631 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb2700037d0 0x7fb2880a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 -- 172.21.2.104:0/1812440631 >> 172.21.2.104:0/1812440631 conn(0x7fb2880b53e0 msgr2=0x7fb28805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:20.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 -- 172.21.2.104:0/1812440631 shutdown_connections 2021-07-11T04:00:20.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:19.997+0000 7fb28f48b700 1 -- 172.21.2.104:0/1812440631 wait complete. 2021-07-11T04:00:20.008 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:00:13.631492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:00:14.726743+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:00:20.009 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:00:20.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85ddd9f700 1 Processor -- start 2021-07-11T04:00:20.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85ddd9f700 1 -- start start 2021-07-11T04:00:20.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85ddd9f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f85d80b12d0 0x7f85d80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85ddd9f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85d80b22a0 0x7f85d8141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85ddd9f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d80a77d0 con 0x7f85d80b1ec0 2021-07-11T04:00:20.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85ddd9f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d80b1740 con 0x7f85d80b12d0 2021-07-11T04:00:20.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85ddd9f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d80b0650 con 0x7f85d80b22a0 2021-07-11T04:00:20.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d77fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f85d80b1ec0 0x7f85d80a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44168/0 (socket says 172.21.2.104:44168) 2021-07-11T04:00:20.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d77fe700 1 -- 172.21.2.104:0/68710808 learned_addr learned my addr 172.21.2.104:0/68710808 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:20.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201065779 0 0) 0x7f85d80a77d0 con 0x7f85d80b1ec0 2021-07-11T04:00:20.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85c4003260 con 0x7f85d80b1ec0 2021-07-11T04:00:20.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2851666976 0 0) 0x7f85c4003260 con 0x7f85d80b1ec0 2021-07-11T04:00:20.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85d80b22a0 msgr2=0x7f85d8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 --2- 172.21.2.104:0/68710808 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85d80b22a0 0x7f85d8141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 >> v2:172.21.2.105:3300/0 conn(0x7f85d80b12d0 msgr2=0x7f85d80affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 --2- 172.21.2.104:0/68710808 >> v2:172.21.2.105:3300/0 conn(0x7f85d80b12d0 0x7f85d80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d80b08f0 con 0x7f85d80b1ec0 2021-07-11T04:00:20.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f85c8001660 con 0x7f85d80b1ec0 2021-07-11T04:00:20.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 >> v1:172.21.2.104:6789/0 conn(0x7f85d80b1ec0 legacy=0x7f85d80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 --2- 172.21.2.104:0/68710808 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85c40037d0 0x7f85c4003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 --2- 172.21.2.104:0/68710808 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c4004500 0x7f85c4004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 --2- 172.21.2.104:0/68710808 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85d80b22a0 0x7f85c400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d80b1740 con 0x7f85c4004500 2021-07-11T04:00:20.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d80b0650 con 0x7f85c40037d0 2021-07-11T04:00:20.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c4005300 con 0x7f85d80b22a0 2021-07-11T04:00:20.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.057+0000 7f85d6ffd700 1 --2- 172.21.2.104:0/68710808 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c4004500 0x7f85c4004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d6ffd700 1 -- 172.21.2.104:0/68710808 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85d80b22a0 msgr2=0x7f85c400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d6ffd700 1 --2- 172.21.2.104:0/68710808 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85d80b22a0 0x7f85c400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d6ffd700 1 -- 172.21.2.104:0/68710808 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85c40037d0 msgr2=0x7f85c4003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d6ffd700 1 --2- 172.21.2.104:0/68710808 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85c40037d0 0x7f85c4003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d6ffd700 1 -- 172.21.2.104:0/68710808 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c4003260 con 0x7f85c4004500 2021-07-11T04:00:20.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d6ffd700 1 --2- 172.21.2.104:0/68710808 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c4004500 0x7f85c4004980 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f85cc002f70 tx=0x7f85cc006ca0).ready entity=mon.0 client_cookie=bdb19e43b9575b6d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85cc012410 con 0x7f85c4004500 2021-07-11T04:00:20.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85cc012570 con 0x7f85c4004500 2021-07-11T04:00:20.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d67fc700 1 -- 172.21.2.104:0/68710808 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85cc0128d0 con 0x7f85c4004500 2021-07-11T04:00:20.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- 172.21.2.104:0/68710808 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c4004500 msgr2=0x7f85c4004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 --2- 172.21.2.104:0/68710808 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c4004500 0x7f85c4004980 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f85cc002f70 tx=0x7f85cc006ca0).stop 2021-07-11T04:00:20.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d7fff700 1 -- 172.21.2.104:0/68710808 reap_dead start 2021-07-11T04:00:20.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- 172.21.2.104:0/68710808 shutdown_connections 2021-07-11T04:00:20.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- 172.21.2.104:0/68710808 >> 172.21.2.104:0/68710808 conn(0x7f85d805f6b0 msgr2=0x7f85d805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:20.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- 172.21.2.104:0/68710808 shutdown_connections 2021-07-11T04:00:20.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- 172.21.2.104:0/68710808 wait complete. 2021-07-11T04:00:20.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 Processor -- start 2021-07-11T04:00:20.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- start start 2021-07-11T04:00:20.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85d80afa20 0x7f85d815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85d8148af0 0x7f85d80ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85d80ae810 0x7f85d81460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d81459d0 con 0x7f85d80afa20 2021-07-11T04:00:20.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d8149070 con 0x7f85d8148af0 2021-07-11T04:00:20.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d8149310 con 0x7f85d80ae810 2021-07-11T04:00:20.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85d80afa20 0x7f85d815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85d80afa20 0x7f85d815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47414/0 (socket says 172.21.2.104:47414) 2021-07-11T04:00:20.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d77fe700 1 -- 172.21.2.104:0/2776660940 learned_addr learned my addr 172.21.2.104:0/2776660940 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:20.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d77fe700 1 -- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85d80ae810 msgr2=0x7f85d81460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d77fe700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85d80ae810 0x7f85d81460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d77fe700 1 -- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85d8148af0 msgr2=0x7f85d80ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d77fe700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85d8148af0 0x7f85d80ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d77fe700 1 -- 172.21.2.104:0/2776660940 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d8145c00 con 0x7f85d80afa20 2021-07-11T04:00:20.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d77fe700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85d80afa20 0x7f85d815aa80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f85c8001370 tx=0x7f85d80b0a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d4ff9700 1 -- 172.21.2.104:0/2776660940 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85c8007170 con 0x7f85d80afa20 2021-07-11T04:00:20.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d4ff9700 1 -- 172.21.2.104:0/2776660940 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c80072d0 con 0x7f85d80afa20 2021-07-11T04:00:20.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85d4ff9700 1 -- 172.21.2.104:0/2776660940 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85c80049e0 con 0x7f85d80afa20 2021-07-11T04:00:20.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- 172.21.2.104:0/2776660940 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d815af70 con 0x7f85d80afa20 2021-07-11T04:00:20.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- 172.21.2.104:0/2776660940 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d80afe00 con 0x7f85d80afa20 2021-07-11T04:00:20.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.061+0000 7f85ddd9f700 1 -- 172.21.2.104:0/2776660940 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d8145d90 con 0x7f85d80afa20 2021-07-11T04:00:20.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.065+0000 7f85d4ff9700 1 -- 172.21.2.104:0/2776660940 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f85c8010070 con 0x7f85d80afa20 2021-07-11T04:00:20.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.065+0000 7f85d4ff9700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85b8053380 0x7f85b8055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.065+0000 7f85d4ff9700 1 -- 172.21.2.104:0/2776660940 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f85c806b2f0 con 0x7f85d80afa20 2021-07-11T04:00:20.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.065+0000 7f85d6ffd700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85b8053380 0x7f85b8055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.065+0000 7f85d6ffd700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85b8053380 0x7f85b8055840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f85cc002f70 tx=0x7f85cc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.065+0000 7f85d4ff9700 1 -- 172.21.2.104:0/2776660940 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f85c8042630 con 0x7f85d80afa20 2021-07-11T04:00:20.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.149+0000 7f85ddd9f700 1 -- 172.21.2.104:0/2776660940 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f85d8057350 con 0x7f85d80afa20 2021-07-11T04:00:20.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.149+0000 7f85d4ff9700 1 -- 172.21.2.104:0/2776660940 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v87) v1 ==== 110+0+10 (secure 0 0 0) 0x7f85c8045c50 con 0x7f85d80afa20 2021-07-11T04:00:20.156 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:00:20.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.149+0000 7f85ddd9f700 1 -- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85b8053380 msgr2=0x7f85b8055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.149+0000 7f85ddd9f700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85b8053380 0x7f85b8055840 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f85cc002f70 tx=0x7f85cc015000).stop 2021-07-11T04:00:20.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.149+0000 7f85ddd9f700 1 -- 172.21.2.104:0/2776660940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85d80afa20 msgr2=0x7f85d815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.149+0000 7f85ddd9f700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85d80afa20 0x7f85d815aa80 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f85c8001370 tx=0x7f85d80b0a80).stop 2021-07-11T04:00:20.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.149+0000 7f85ddd9f700 1 -- 172.21.2.104:0/2776660940 shutdown_connections 2021-07-11T04:00:20.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.149+0000 7f85ddd9f700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85d80ae810 0x7f85d81460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.149+0000 7f85ddd9f700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85d8148af0 0x7f85d80ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.149+0000 7f85ddd9f700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85b8053380 0x7f85b8055840 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.149+0000 7f85ddd9f700 1 --2- 172.21.2.104:0/2776660940 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85d80afa20 0x7f85d815aa80 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.153+0000 7f85ddd9f700 1 -- 172.21.2.104:0/2776660940 >> 172.21.2.104:0/2776660940 conn(0x7f85d805f6b0 msgr2=0x7f85d81405e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:20.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.153+0000 7f85ddd9f700 1 -- 172.21.2.104:0/2776660940 shutdown_connections 2021-07-11T04:00:20.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.153+0000 7f85ddd9f700 1 -- 172.21.2.104:0/2776660940 wait complete. 2021-07-11T04:00:20.162 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:00:20.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.213+0000 7f14892c2700 1 Processor -- start 2021-07-11T04:00:20.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.213+0000 7f14892c2700 1 -- start start 2021-07-11T04:00:20.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.213+0000 7f14892c2700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f14840b1ec0 0x7f14840b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.213+0000 7f14892c2700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14840b0420 con 0x7f14840b22a0 2021-07-11T04:00:20.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.213+0000 7f14892c2700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14840a77d0 con 0x7f14840affb0 2021-07-11T04:00:20.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.213+0000 7f14892c2700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14840b0650 con 0x7f14840b1ec0 2021-07-11T04:00:20.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.213+0000 7f14837fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f14840b22a0 0x7f1484141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44188/0 (socket says 172.21.2.104:44188) 2021-07-11T04:00:20.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.213+0000 7f14837fe700 1 -- 172.21.2.104:0/2717604247 learned_addr learned my addr 172.21.2.104:0/2717604247 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:20.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.213+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278897484 0 0) 0x7f14840b0420 con 0x7f14840b22a0 2021-07-11T04:00:20.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.213+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1464003260 con 0x7f14840b22a0 2021-07-11T04:00:20.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1438631053 0 0) 0x7f1464003260 con 0x7f14840b22a0 2021-07-11T04:00:20.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 >> v2:172.21.2.105:3300/0 conn(0x7f14840b1ec0 msgr2=0x7f14840b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 --2- 172.21.2.104:0/2717604247 >> v2:172.21.2.105:3300/0 conn(0x7f14840b1ec0 0x7f14840b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 >> v1:172.21.2.105:6789/0 conn(0x7f14840affb0 legacy=0x7f14840a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14840b08f0 con 0x7f14840b22a0 2021-07-11T04:00:20.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f147c0019e0 con 0x7f14840b22a0 2021-07-11T04:00:20.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 >> v1:172.21.2.104:6789/0 conn(0x7f14840b22a0 legacy=0x7f1484141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 --2- 172.21.2.104:0/2717604247 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1464003820 0x7f1464003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 --2- 172.21.2.104:0/2717604247 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1464004500 0x7f1464004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 --2- 172.21.2.104:0/2717604247 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1464005050 0x7f146400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14840b0650 con 0x7f1464004500 2021-07-11T04:00:20.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146400bf40 con 0x7f1464003820 2021-07-11T04:00:20.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146400c100 con 0x7f1464005050 2021-07-11T04:00:20.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 --2- 172.21.2.104:0/2717604247 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1464004500 0x7f1464004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 -- 172.21.2.104:0/2717604247 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1464005050 msgr2=0x7f146400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 --2- 172.21.2.104:0/2717604247 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1464005050 0x7f146400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 -- 172.21.2.104:0/2717604247 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1464003820 msgr2=0x7f1464003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 --2- 172.21.2.104:0/2717604247 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1464003820 0x7f1464003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 -- 172.21.2.104:0/2717604247 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1464003260 con 0x7f1464004500 2021-07-11T04:00:20.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14837fe700 1 -- 172.21.2.104:0/2717604247 reap_dead start 2021-07-11T04:00:20.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 --2- 172.21.2.104:0/2717604247 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1464004500 0x7f1464004980 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f1474000f70 tx=0x7f14740079e0).ready entity=mon.0 client_cookie=77969835216ce13b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1474007cf0 con 0x7f1464004500 2021-07-11T04:00:20.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1474014410 con 0x7f1464004500 2021-07-11T04:00:20.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1481ffb700 1 -- 172.21.2.104:0/2717604247 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1474014770 con 0x7f1464004500 2021-07-11T04:00:20.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 -- 172.21.2.104:0/2717604247 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1464004500 msgr2=0x7f1464004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 --2- 172.21.2.104:0/2717604247 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1464004500 0x7f1464004980 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f1474000f70 tx=0x7f14740079e0).stop 2021-07-11T04:00:20.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 -- 172.21.2.104:0/2717604247 shutdown_connections 2021-07-11T04:00:20.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 --2- 172.21.2.104:0/2717604247 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1464004500 0x7f1464004980 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 -- 172.21.2.104:0/2717604247 >> 172.21.2.104:0/2717604247 conn(0x7f148405f6b0 msgr2=0x7f148405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:20.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 -- 172.21.2.104:0/2717604247 shutdown_connections 2021-07-11T04:00:20.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 -- 172.21.2.104:0/2717604247 wait complete. 2021-07-11T04:00:20.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 Processor -- start 2021-07-11T04:00:20.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 -- start start 2021-07-11T04:00:20.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1464004500 0x7f1484140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14841569b0 0x7f148415a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14840ab010 0x7f148415aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148414ba40 con 0x7f14841569b0 2021-07-11T04:00:20.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14840ab860 con 0x7f14840ab010 2021-07-11T04:00:20.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14840abac0 con 0x7f1464004500 2021-07-11T04:00:20.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14841569b0 0x7f148415a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14841569b0 0x7f148415a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47436/0 (socket says 172.21.2.104:47436) 2021-07-11T04:00:20.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 -- 172.21.2.104:0/256396083 learned_addr learned my addr 172.21.2.104:0/256396083 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:20.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 -- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1464004500 msgr2=0x7f1484140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f1482ffd700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1464004500 0x7f1484140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1464004500 0x7f1484140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 -- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14840ab010 msgr2=0x7f148415aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14840ab010 0x7f148415aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 -- 172.21.2.104:0/256396083 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148415a100 con 0x7f14841569b0 2021-07-11T04:00:20.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14827fc700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14841569b0 0x7f148415a5a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f1474000f70 tx=0x7f1474015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f147bfff700 1 -- 172.21.2.104:0/256396083 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f14740137c0 con 0x7f14841569b0 2021-07-11T04:00:20.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f147bfff700 1 -- 172.21.2.104:0/256396083 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1474013920 con 0x7f14841569b0 2021-07-11T04:00:20.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 -- 172.21.2.104:0/256396083 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148413c0d0 con 0x7f14841569b0 2021-07-11T04:00:20.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f147bfff700 1 -- 172.21.2.104:0/256396083 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1474013ca0 con 0x7f14841569b0 2021-07-11T04:00:20.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.217+0000 7f14892c2700 1 -- 172.21.2.104:0/256396083 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14840aadd0 con 0x7f14841569b0 2021-07-11T04:00:20.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.221+0000 7f14892c2700 1 -- 172.21.2.104:0/256396083 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148415a290 con 0x7f14841569b0 2021-07-11T04:00:20.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.221+0000 7f147bfff700 1 -- 172.21.2.104:0/256396083 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f147402a030 con 0x7f14841569b0 2021-07-11T04:00:20.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.221+0000 7f147bfff700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1468053330 0x7f14680557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.221+0000 7f147bfff700 1 -- 172.21.2.104:0/256396083 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f147401c040 con 0x7f14841569b0 2021-07-11T04:00:20.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.221+0000 7f1482ffd700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1468053330 0x7f14680557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.221+0000 7f1482ffd700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1468053330 0x7f14680557f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f146c007000 tx=0x7f146c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.221+0000 7f147bfff700 1 -- 172.21.2.104:0/256396083 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1474043d10 con 0x7f14841569b0 2021-07-11T04:00:20.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 -- 172.21.2.104:0/256396083 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1484057350 con 0x7f14841569b0 2021-07-11T04:00:20.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f147bfff700 1 -- 172.21.2.104:0/256396083 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v87) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1474047330 con 0x7f14841569b0 2021-07-11T04:00:20.309 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:20.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 -- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1468053330 msgr2=0x7f14680557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1468053330 0x7f14680557f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f146c007000 tx=0x7f146c008040).stop 2021-07-11T04:00:20.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 -- 172.21.2.104:0/256396083 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14841569b0 msgr2=0x7f148415a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14841569b0 0x7f148415a5a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f1474000f70 tx=0x7f1474015000).stop 2021-07-11T04:00:20.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 -- 172.21.2.104:0/256396083 shutdown_connections 2021-07-11T04:00:20.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f14840ab010 0x7f148415aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1468053330 0x7f14680557f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f14841569b0 0x7f148415a5a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 --2- 172.21.2.104:0/256396083 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1464004500 0x7f1484140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 -- 172.21.2.104:0/256396083 >> 172.21.2.104:0/256396083 conn(0x7f148405f6b0 msgr2=0x7f148413c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:20.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 -- 172.21.2.104:0/256396083 shutdown_connections 2021-07-11T04:00:20.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.305+0000 7f14892c2700 1 -- 172.21.2.104:0/256396083 wait complete. 2021-07-11T04:00:20.316 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:00:20.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 Processor -- start 2021-07-11T04:00:20.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 -- start start 2021-07-11T04:00:20.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70880b1ec0 0x7f70880b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70880a77d0 con 0x7f70880b22a0 2021-07-11T04:00:20.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70880b03f0 con 0x7f70880affb0 2021-07-11T04:00:20.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70880b0650 con 0x7f70880b1ec0 2021-07-11T04:00:20.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708e3b2700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f70880b22a0 0x7f7088141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44210/0 (socket says 172.21.2.104:44210) 2021-07-11T04:00:20.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708e3b2700 1 -- 172.21.2.104:0/4236063602 learned_addr learned my addr 172.21.2.104:0/4236063602 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:20.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794902053 0 0) 0x7f70880a77d0 con 0x7f70880b22a0 2021-07-11T04:00:20.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7070003260 con 0x7f70880b22a0 2021-07-11T04:00:20.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708dbb1700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70880b1ec0 0x7f70880b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3741375595 0 0) 0x7f7070003260 con 0x7f70880b22a0 2021-07-11T04:00:20.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70880b1ec0 msgr2=0x7f70880b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70880b1ec0 0x7f70880b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 >> v1:172.21.2.105:6789/0 conn(0x7f70880affb0 legacy=0x7f70880a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880b08f0 con 0x7f70880b22a0 2021-07-11T04:00:20.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f70800019e0 con 0x7f70880b22a0 2021-07-11T04:00:20.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708dbb1700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70880b1ec0 0x7f70880b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:20.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 >> v1:172.21.2.104:6789/0 conn(0x7f70880b22a0 legacy=0x7f7088141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7070003800 0x7f7070003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7070004500 0x7f7070004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70880b1ec0 0x7f707000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70880b0650 con 0x7f7070004500 2021-07-11T04:00:20.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7070005270 con 0x7f7070003800 2021-07-11T04:00:20.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707000bc40 con 0x7f70880b1ec0 2021-07-11T04:00:20.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708d3b0700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7070004500 0x7f7070004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708d3b0700 1 -- 172.21.2.104:0/4236063602 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70880b1ec0 msgr2=0x7f707000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708dbb1700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7070003800 0x7f7070003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708d3b0700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f70880b1ec0 0x7f707000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708d3b0700 1 -- 172.21.2.104:0/4236063602 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7070003800 msgr2=0x7f7070003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708d3b0700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7070003800 0x7f7070003c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708d3b0700 1 -- 172.21.2.104:0/4236063602 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070003260 con 0x7f7070004500 2021-07-11T04:00:20.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708d3b0700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7070004500 0x7f7070004980 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f7084001fa0 tx=0x7f7084007a00).ready entity=mon.0 client_cookie=29d07584c26573fd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7084007070 con 0x7f7070004500 2021-07-11T04:00:20.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70840071d0 con 0x7f7070004500 2021-07-11T04:00:20.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708cbaf700 1 -- 172.21.2.104:0/4236063602 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7084006510 con 0x7f7070004500 2021-07-11T04:00:20.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 -- 172.21.2.104:0/4236063602 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7070004500 msgr2=0x7f7070004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 --2- 172.21.2.104:0/4236063602 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7070004500 0x7f7070004980 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7f7084001fa0 tx=0x7f7084007a00).stop 2021-07-11T04:00:20.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708e3b2700 1 -- 172.21.2.104:0/4236063602 reap_dead start 2021-07-11T04:00:20.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 -- 172.21.2.104:0/4236063602 shutdown_connections 2021-07-11T04:00:20.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 -- 172.21.2.104:0/4236063602 >> 172.21.2.104:0/4236063602 conn(0x7f708805f6b0 msgr2=0x7f708805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:20.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 -- 172.21.2.104:0/4236063602 shutdown_connections 2021-07-11T04:00:20.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.365+0000 7f708fe15700 1 -- 172.21.2.104:0/4236063602 wait complete. 2021-07-11T04:00:20.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708fe15700 1 Processor -- start 2021-07-11T04:00:20.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708fe15700 1 -- start start 2021-07-11T04:00:20.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708fe15700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f708813c1f0 0x7f708813c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708fe15700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f708815a3d0 0x7f708815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708fe15700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f708815ad20 0x7f708813cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708fe15700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708814a9e0 con 0x7f708815ad20 2021-07-11T04:00:20.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708fe15700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7088149040 con 0x7f708815a3d0 2021-07-11T04:00:20.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708fe15700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708814abd0 con 0x7f708813c1f0 2021-07-11T04:00:20.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708e3b2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f708815ad20 0x7f708813cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708e3b2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f708815ad20 0x7f708813cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47454/0 (socket says 172.21.2.104:47454) 2021-07-11T04:00:20.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708e3b2700 1 -- 172.21.2.104:0/4035858572 learned_addr learned my addr 172.21.2.104:0/4035858572 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:20.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708d3b0700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f708815a3d0 0x7f708815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708e3b2700 1 -- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f708813c1f0 msgr2=0x7f708813c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708dbb1700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f708813c1f0 0x7f708813c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708e3b2700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f708813c1f0 0x7f708813c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708e3b2700 1 -- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f708815a3d0 msgr2=0x7f708815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708e3b2700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f708815a3d0 0x7f708815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708e3b2700 1 -- 172.21.2.104:0/4035858572 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708814ae70 con 0x7f708815ad20 2021-07-11T04:00:20.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708d3b0700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f708815a3d0 0x7f708815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:20.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708e3b2700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f708815ad20 0x7f708813cd50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f708813e680 tx=0x7f70880b2a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f7076ffd700 1 -- 172.21.2.104:0/4035858572 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7080004b70 con 0x7f708815ad20 2021-07-11T04:00:20.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f7076ffd700 1 -- 172.21.2.104:0/4035858572 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7080004cd0 con 0x7f708815ad20 2021-07-11T04:00:20.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708fe15700 1 -- 172.21.2.104:0/4035858572 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70880ac140 con 0x7f708815ad20 2021-07-11T04:00:20.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708fe15700 1 -- 172.21.2.104:0/4035858572 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7088148b30 con 0x7f708815ad20 2021-07-11T04:00:20.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f7076ffd700 1 -- 172.21.2.104:0/4035858572 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7080007220 con 0x7f708815ad20 2021-07-11T04:00:20.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f708fe15700 1 -- 172.21.2.104:0/4035858572 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7088057350 con 0x7f708815ad20 2021-07-11T04:00:20.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f7076ffd700 1 -- 172.21.2.104:0/4035858572 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f7080010070 con 0x7f708815ad20 2021-07-11T04:00:20.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f7076ffd700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7068053330 0x7f70680557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.369+0000 7f7076ffd700 1 -- 172.21.2.104:0/4035858572 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f708006b140 con 0x7f708815ad20 2021-07-11T04:00:20.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.373+0000 7f708dbb1700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7068053330 0x7f70680557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.373+0000 7f708dbb1700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7068053330 0x7f70680557f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f7078009000 tx=0x7f7078008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.373+0000 7f7076ffd700 1 -- 172.21.2.104:0/4035858572 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7080042370 con 0x7f708815ad20 2021-07-11T04:00:20.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.453+0000 7f708fe15700 1 -- 172.21.2.104:0/4035858572 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7088140c10 con 0x7f708815ad20 2021-07-11T04:00:20.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f7076ffd700 1 -- 172.21.2.104:0/4035858572 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7080045990 con 0x7f708815ad20 2021-07-11T04:00:20.460 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:20.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 -- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7068053330 msgr2=0x7f70680557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7068053330 0x7f70680557f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f7078009000 tx=0x7f7078008040).stop 2021-07-11T04:00:20.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 -- 172.21.2.104:0/4035858572 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f708815ad20 msgr2=0x7f708813cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f708815ad20 0x7f708813cd50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f708813e680 tx=0x7f70880b2a00).stop 2021-07-11T04:00:20.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 -- 172.21.2.104:0/4035858572 shutdown_connections 2021-07-11T04:00:20.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7068053330 0x7f70680557f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f708815ad20 0x7f708813cd50 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f708815a3d0 0x7f708815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 --2- 172.21.2.104:0/4035858572 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f708813c1f0 0x7f708813c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 -- 172.21.2.104:0/4035858572 >> 172.21.2.104:0/4035858572 conn(0x7f708805f6b0 msgr2=0x7f70880a8110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:20.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 -- 172.21.2.104:0/4035858572 shutdown_connections 2021-07-11T04:00:20.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.457+0000 7f708fe15700 1 -- 172.21.2.104:0/4035858572 wait complete. 2021-07-11T04:00:20.469 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:00:20.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 Processor -- start 2021-07-11T04:00:20.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 -- start start 2021-07-11T04:00:20.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0ac40a9ec0 0x7f0ac40aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0ac40603d0 0x7f0ac413ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac4061990 con 0x7f0ac413d1c0 2021-07-11T04:00:20.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac4061c30 con 0x7f0ac40603d0 2021-07-11T04:00:20.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac4060b90 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0ac9201700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0ac40603d0 0x7f0ac413ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0ac9201700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0ac40603d0 0x7f0ac413ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47462/0 (socket says 172.21.2.104:47462) 2021-07-11T04:00:20.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0ac9201700 1 -- 172.21.2.104:0/1827865375 learned_addr learned my addr 172.21.2.104:0/1827865375 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:20.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0ac9201700 1 -- 172.21.2.104:0/1827865375 >> v2:172.21.2.105:3300/0 conn(0x7f0ac40a9ec0 msgr2=0x7f0ac40aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0ac9201700 1 --2- 172.21.2.104:0/1827865375 >> v2:172.21.2.105:3300/0 conn(0x7f0ac40a9ec0 0x7f0ac40aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0ac9201700 1 -- 172.21.2.104:0/1827865375 >> v1:172.21.2.105:6789/0 conn(0x7f0ac413d1c0 legacy=0x7f0ac413f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0ac9201700 1 -- 172.21.2.104:0/1827865375 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac4060e30 con 0x7f0ac40603d0 2021-07-11T04:00:20.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0ac9201700 1 --2- 172.21.2.104:0/1827865375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ac40603d0 0x7f0ac413ccd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f0ac40aaaf0 tx=0x7f0ac0006e40).ready entity=mon.0 client_cookie=2aa8910c16f3c086 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0ac8a00700 1 -- 172.21.2.104:0/1827865375 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac000bca0 con 0x7f0ac40603d0 2021-07-11T04:00:20.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0ac8a00700 1 -- 172.21.2.104:0/1827865375 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac0000e90 con 0x7f0ac40603d0 2021-07-11T04:00:20.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0ac8a00700 1 -- 172.21.2.104:0/1827865375 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac00153f0 con 0x7f0ac40603d0 2021-07-11T04:00:20.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 -- 172.21.2.104:0/1827865375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ac40603d0 msgr2=0x7f0ac413ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 --2- 172.21.2.104:0/1827865375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ac40603d0 0x7f0ac413ccd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f0ac40aaaf0 tx=0x7f0ac0006e40).stop 2021-07-11T04:00:20.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 -- 172.21.2.104:0/1827865375 shutdown_connections 2021-07-11T04:00:20.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 --2- 172.21.2.104:0/1827865375 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ac40603d0 0x7f0ac413ccd0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 --2- 172.21.2.104:0/1827865375 >> v2:172.21.2.105:3300/0 conn(0x7f0ac40a9ec0 0x7f0ac40aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 -- 172.21.2.104:0/1827865375 >> 172.21.2.104:0/1827865375 conn(0x7f0ac40b2970 msgr2=0x7f0ac40b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:20.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 -- 172.21.2.104:0/1827865375 shutdown_connections 2021-07-11T04:00:20.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.517+0000 7f0acbc66700 1 -- 172.21.2.104:0/1827865375 wait complete. 2021-07-11T04:00:20.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0acbc66700 1 Processor -- start 2021-07-11T04:00:20.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0acbc66700 1 -- start start 2021-07-11T04:00:20.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0acbc66700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ac40603d0 0x7f0ac40aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0acbc66700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ac40a9ec0 0x7f0ac41370b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0acbc66700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ac413d1c0 0x7f0ac41375a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0acbc66700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac40ad140 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0acbc66700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac40b00b0 con 0x7f0ac413d1c0 2021-07-11T04:00:20.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0acbc66700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac4141d60 con 0x7f0ac40603d0 2021-07-11T04:00:20.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ac9201700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ac40a9ec0 0x7f0ac41370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ac9201700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ac40a9ec0 0x7f0ac41370b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47468/0 (socket says 172.21.2.104:47468) 2021-07-11T04:00:20.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ac9201700 1 -- 172.21.2.104:0/2731471223 learned_addr learned my addr 172.21.2.104:0/2731471223 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:20.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ac9201700 1 -- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ac40603d0 msgr2=0x7f0ac40aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ac9201700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ac40603d0 0x7f0ac40aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ac9201700 1 -- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ac413d1c0 msgr2=0x7f0ac41375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ac9201700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ac413d1c0 0x7f0ac41375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ac9201700 1 -- 172.21.2.104:0/2731471223 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac000d4d0 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ac9201700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ac40a9ec0 0x7f0ac41370b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7f0ac0006fc0 tx=0x7f0ac0018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ab6ffd700 1 -- 172.21.2.104:0/2731471223 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac0015a40 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ab6ffd700 1 -- 172.21.2.104:0/2731471223 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac0015ba0 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0acbc66700 1 -- 172.21.2.104:0/2731471223 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac40ad370 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0acbc66700 1 -- 172.21.2.104:0/2731471223 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac4140c80 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ab6ffd700 1 -- 172.21.2.104:0/2731471223 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ac001d410 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0acbc66700 1 -- 172.21.2.104:0/2731471223 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa0003410 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ab6ffd700 1 -- 172.21.2.104:0/2731471223 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0ac0024030 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ab6ffd700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0aa40531a0 0x7f0aa4055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ab6ffd700 1 -- 172.21.2.104:0/2731471223 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0ac000b040 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.521+0000 7f0ac9a02700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0aa40531a0 0x7f0aa4055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.525+0000 7f0ac9a02700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0aa40531a0 0x7f0aa4055660 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f0ab8002f70 tx=0x7f0ab8006ab0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.525+0000 7f0ab6ffd700 1 -- 172.21.2.104:0/2731471223 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0ac0043830 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 -- 172.21.2.104:0/2731471223 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0aa0003c40 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0ab6ffd700 1 -- 172.21.2.104:0/2731471223 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0ac00220a0 con 0x7f0ac40a9ec0 2021-07-11T04:00:20.611 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:20.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 -- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0aa40531a0 msgr2=0x7f0aa4055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0aa40531a0 0x7f0aa4055660 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f0ab8002f70 tx=0x7f0ab8006ab0).stop 2021-07-11T04:00:20.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 -- 172.21.2.104:0/2731471223 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ac40a9ec0 msgr2=0x7f0ac41370b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ac40a9ec0 0x7f0ac41370b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7f0ac0006fc0 tx=0x7f0ac0018000).stop 2021-07-11T04:00:20.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 -- 172.21.2.104:0/2731471223 shutdown_connections 2021-07-11T04:00:20.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ac413d1c0 0x7f0ac41375a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0aa40531a0 0x7f0aa4055660 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ac40a9ec0 0x7f0ac41370b0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 --2- 172.21.2.104:0/2731471223 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ac40603d0 0x7f0ac40aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 -- 172.21.2.104:0/2731471223 >> 172.21.2.104:0/2731471223 conn(0x7f0ac40b2970 msgr2=0x7f0ac405bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:20.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 -- 172.21.2.104:0/2731471223 shutdown_connections 2021-07-11T04:00:20.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.605+0000 7f0acbc66700 1 -- 172.21.2.104:0/2731471223 wait complete. 2021-07-11T04:00:20.624 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:00:20.625 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:20.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.681+0000 7f19843ae700 1 Processor -- start 2021-07-11T04:00:20.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.681+0000 7f19843ae700 1 -- start start 2021-07-11T04:00:20.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.681+0000 7f19843ae700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f197c0b2660 0x7f197c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.681+0000 7f19843ae700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f197c097100 0x7f197c13fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.681+0000 7f19843ae700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c0b2a40 con 0x7f197c096cc0 2021-07-11T04:00:20.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.681+0000 7f19843ae700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f197c097b80 con 0x7f197c097100 2021-07-11T04:00:20.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.681+0000 7f19843ae700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c097e20 con 0x7f197c0b2660 2021-07-11T04:00:20.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f1981949700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f197c096cc0 0x7f197c13f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44244/0 (socket says 172.21.2.104:44244) 2021-07-11T04:00:20.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f1981949700 1 -- 172.21.2.104:0/2540305055 learned_addr learned my addr 172.21.2.104:0/2540305055 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:20.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 --2- 172.21.2.104:0/2540305055 >> v2:172.21.2.104:3300/0 conn(0x7f197c097100 0x7f197c13fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 -- 172.21.2.104:0/2540305055 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f197c0b2660 msgr2=0x7f197c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:20.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 --2- 172.21.2.104:0/2540305055 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f197c0b2660 0x7f197c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 -- 172.21.2.104:0/2540305055 >> v1:172.21.2.104:6789/0 conn(0x7f197c096cc0 legacy=0x7f197c13f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 -- 172.21.2.104:0/2540305055 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0980c0 con 0x7f197c097100 2021-07-11T04:00:20.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 --2- 172.21.2.104:0/2540305055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f197c097100 0x7f197c13fc70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f19740032f0 tx=0x7f19740091d0).ready entity=mon.0 client_cookie=3975cda16447d36c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f1981148700 1 -- 172.21.2.104:0/2540305055 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1974009ea0 con 0x7f197c097100 2021-07-11T04:00:20.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f1981148700 1 -- 172.21.2.104:0/2540305055 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197400cba0 con 0x7f197c097100 2021-07-11T04:00:20.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f1981148700 1 -- 172.21.2.104:0/2540305055 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1974010410 con 0x7f197c097100 2021-07-11T04:00:20.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 -- 172.21.2.104:0/2540305055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f197c097100 msgr2=0x7f197c13fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 --2- 172.21.2.104:0/2540305055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f197c097100 0x7f197c13fc70 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f19740032f0 tx=0x7f19740091d0).stop 2021-07-11T04:00:20.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 -- 172.21.2.104:0/2540305055 shutdown_connections 2021-07-11T04:00:20.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 --2- 172.21.2.104:0/2540305055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f197c097100 0x7f197c13fc70 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 --2- 172.21.2.104:0/2540305055 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f197c0b2660 0x7f197c05f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 -- 172.21.2.104:0/2540305055 >> 172.21.2.104:0/2540305055 conn(0x7f197c062f10 msgr2=0x7f197c0632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:20.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 -- 172.21.2.104:0/2540305055 shutdown_connections 2021-07-11T04:00:20.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 -- 172.21.2.104:0/2540305055 wait complete. 2021-07-11T04:00:20.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 Processor -- start 2021-07-11T04:00:20.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 -- start start 2021-07-11T04:00:20.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f197c096cc0 0x7f197c0ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f197c097100 0x7f197c0ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f197c0b2660 0x7f197c143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c143570 con 0x7f197c0b2660 2021-07-11T04:00:20.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c143800 con 0x7f197c096cc0 2021-07-11T04:00:20.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c0aee60 con 0x7f197c097100 2021-07-11T04:00:20.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f197c0b2660 0x7f197c143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f197c0b2660 0x7f197c143080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47484/0 (socket says 172.21.2.104:47484) 2021-07-11T04:00:20.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 -- 172.21.2.104:0/1918534 learned_addr learned my addr 172.21.2.104:0/1918534 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:20.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 -- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f197c097100 msgr2=0x7f197c0ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f197c097100 0x7f197c0ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 -- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f197c096cc0 msgr2=0x7f197c0ae850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f197c096cc0 0x7f197c0ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 -- 172.21.2.104:0/1918534 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1974009470 con 0x7f197c0b2660 2021-07-11T04:00:20.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f198294b700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f197c0b2660 0x7f197c143080 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f19740032f0 tx=0x7f1974017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f196b7fe700 1 -- 172.21.2.104:0/1918534 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f197400cee0 con 0x7f197c0b2660 2021-07-11T04:00:20.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f196b7fe700 1 -- 172.21.2.104:0/1918534 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19740185f0 con 0x7f197c0b2660 2021-07-11T04:00:20.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 -- 172.21.2.104:0/1918534 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c0af100 con 0x7f197c0b2660 2021-07-11T04:00:20.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f19843ae700 1 -- 172.21.2.104:0/1918534 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c151bb0 con 0x7f197c0b2660 2021-07-11T04:00:20.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.685+0000 7f196b7fe700 1 -- 172.21.2.104:0/1918534 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1974018950 con 0x7f197c0b2660 2021-07-11T04:00:20.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.689+0000 7f19843ae700 1 -- 172.21.2.104:0/1918534 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c0af290 con 0x7f197c0b2660 2021-07-11T04:00:20.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.689+0000 7f196b7fe700 1 -- 172.21.2.104:0/1918534 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f197402b070 con 0x7f197c0b2660 2021-07-11T04:00:20.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.689+0000 7f196b7fe700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f195c053270 0x7f195c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:20.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.689+0000 7f196b7fe700 1 -- 172.21.2.104:0/1918534 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f197406e010 con 0x7f197c0b2660 2021-07-11T04:00:20.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.689+0000 7f198214a700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f195c053270 0x7f195c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:20.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.689+0000 7f198214a700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f195c053270 0x7f195c055730 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f1978002f70 tx=0x7f1978006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:20.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.689+0000 7f196b7fe700 1 -- 172.21.2.104:0/1918534 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1974045240 con 0x7f197c0b2660 2021-07-11T04:00:20.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 -- 172.21.2.104:0/1918534 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f197c057350 con 0x7f197c0b2660 2021-07-11T04:00:20.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f196b7fe700 1 -- 172.21.2.104:0/1918534 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f1974048860 con 0x7f197c0b2660 2021-07-11T04:00:20.803 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:20.803 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/43570850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":43570850},{"type":"v1","addr":"172.21.2.105:6835","nonce":43570850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/3334136782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3334136782},{"type":"v1","addr":"172.21.2.104:6833","nonce":3334136782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:15.674524+0000","modified":"2021-07-11T04:00:16.678211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/4286166182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":4286166182},{"type":"v1","addr":"172.21.2.104:6835","nonce":4286166182}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-07-11T04:00:20.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 -- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f195c053270 msgr2=0x7f195c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f195c053270 0x7f195c055730 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f1978002f70 tx=0x7f1978006a10).stop 2021-07-11T04:00:20.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 -- 172.21.2.104:0/1918534 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f197c0b2660 msgr2=0x7f197c143080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:20.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f197c0b2660 0x7f197c143080 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f19740032f0 tx=0x7f1974017000).stop 2021-07-11T04:00:20.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 -- 172.21.2.104:0/1918534 shutdown_connections 2021-07-11T04:00:20.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f195c053270 0x7f195c055730 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f197c0b2660 0x7f197c143080 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:20.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f197c097100 0x7f197c0ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 --2- 172.21.2.104:0/1918534 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f197c096cc0 0x7f197c0ae850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:20.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 -- 172.21.2.104:0/1918534 >> 172.21.2.104:0/1918534 conn(0x7f197c062f10 msgr2=0x7f197c0b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:20.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 -- 172.21.2.104:0/1918534 shutdown_connections 2021-07-11T04:00:20.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:20.797+0000 7f19843ae700 1 -- 172.21.2.104:0/1918534 wait complete. 2021-07-11T04:00:20.807 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 39 2021-07-11T04:00:21.111 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:00:15.674524+0000', 'modified': '2021-07-11T04:00:16.678211+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6636}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6636': {'gid': 6636, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/4286166182', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 4286166182}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 4286166182}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:00:21.112 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:00:21.113 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-07-11T04:00:21.181 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":6636,"want_state":"up:active","state":"up:active","rank_uptime":5.4997779820000003,"mdsmap_epoch":39,"osdmap_epoch":87,"osdmap_epoch_barrier":86,"uptime":9.7193561160000002} 2021-07-11T04:00:21.182 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 6636, "mdsmap_epoch": 39, "osdmap_epoch": 87, "osdmap_epoch_barrier": 86, "rank_uptime": 5.499777982, "state": "up:active", "uptime": 9.719356116, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:00:21.182 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:00:21.183 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-07-11T04:00:21.183 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba023.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-07-11T04:00:21.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:21.184 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:21.195 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:00:21.249 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:21.250 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:21.255 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:21.256 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:21.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:21.258 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:00:21.304 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:00:21.305 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:21.308 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:00:21.320 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:21.321 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:21.339 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:21.340 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:21.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:21.341 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:00:21.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:2021-07-11T04:00:21.348+0000 7f2182ac0080 -1 init, newargv = 0x5571958f9b80 newargc=9 2021-07-11T04:00:21.349 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14749]: starting ceph client 2021-07-11T04:00:21.363 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14749]: starting fuse 2021-07-11T04:00:22.454 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:00:22.468 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:22.468 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:22.519 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:22.520 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:22.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:22.521 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:00:22.527 INFO:teuthology.orchestra.run.gibba023.stdout:52 2021-07-11T04:00:22.527 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:00:22.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:22.529 DEBUG:teuthology.orchestra.run.gibba023:> sudo python3 -c ' 2021-07-11T04:00:22.529 DEBUG:teuthology.orchestra.run.gibba023:> import glob 2021-07-11T04:00:22.530 DEBUG:teuthology.orchestra.run.gibba023:> import re 2021-07-11T04:00:22.531 DEBUG:teuthology.orchestra.run.gibba023:> import os 2021-07-11T04:00:22.532 DEBUG:teuthology.orchestra.run.gibba023:> import subprocess 2021-07-11T04:00:22.533 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:00:22.533 DEBUG:teuthology.orchestra.run.gibba023:> def find_socket(client_name): 2021-07-11T04:00:22.534 DEBUG:teuthology.orchestra.run.gibba023:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-07-11T04:00:22.535 DEBUG:teuthology.orchestra.run.gibba023:> files = glob.glob(asok_path) 2021-07-11T04:00:22.536 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:00:22.537 DEBUG:teuthology.orchestra.run.gibba023:> # Given a non-glob path, it better be there 2021-07-11T04:00:22.537 DEBUG:teuthology.orchestra.run.gibba023:> if "*" not in asok_path: 2021-07-11T04:00:22.538 DEBUG:teuthology.orchestra.run.gibba023:> assert(len(files) == 1) 2021-07-11T04:00:22.539 DEBUG:teuthology.orchestra.run.gibba023:> return files[0] 2021-07-11T04:00:22.540 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:00:22.540 DEBUG:teuthology.orchestra.run.gibba023:> for f in files: 2021-07-11T04:00:22.541 DEBUG:teuthology.orchestra.run.gibba023:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:00:22.542 DEBUG:teuthology.orchestra.run.gibba023:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:00:22.542 DEBUG:teuthology.orchestra.run.gibba023:> return f 2021-07-11T04:00:22.543 DEBUG:teuthology.orchestra.run.gibba023:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:00:22.544 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:00:22.544 DEBUG:teuthology.orchestra.run.gibba023:> print(find_socket("client.1")) 2021-07-11T04:00:22.545 DEBUG:teuthology.orchestra.run.gibba023:> ' 2021-07-11T04:00:22.581 INFO:teuthology.orchestra.run.gibba023.stdout:/var/run/ceph/ceph-client.1.14749.asok 2021-07-11T04:00:22.583 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14749.asok 2021-07-11T04:00:22.584 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:22.584 DEBUG:teuthology.orchestra.run.gibba023:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14749.asok status 2021-07-11T04:00:22.688 INFO:teuthology.orchestra.run.gibba023.stdout:{ 2021-07-11T04:00:22.688 INFO:teuthology.orchestra.run.gibba023.stdout: "metadata": { 2021-07-11T04:00:22.689 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:00:22.689 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:00:22.689 INFO:teuthology.orchestra.run.gibba023.stdout: "entity_id": "1", 2021-07-11T04:00:22.689 INFO:teuthology.orchestra.run.gibba023.stdout: "hostname": "gibba023", 2021-07-11T04:00:22.690 INFO:teuthology.orchestra.run.gibba023.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-07-11T04:00:22.690 INFO:teuthology.orchestra.run.gibba023.stdout: "pid": "14749", 2021-07-11T04:00:22.690 INFO:teuthology.orchestra.run.gibba023.stdout: "root": "/" 2021-07-11T04:00:22.690 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:00:22.690 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_count": 0, 2021-07-11T04:00:22.691 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_pinned_count": 0, 2021-07-11T04:00:22.691 INFO:teuthology.orchestra.run.gibba023.stdout: "id": 4981, 2021-07-11T04:00:22.691 INFO:teuthology.orchestra.run.gibba023.stdout: "inst": { 2021-07-11T04:00:22.691 INFO:teuthology.orchestra.run.gibba023.stdout: "name": { 2021-07-11T04:00:22.692 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "client", 2021-07-11T04:00:22.692 INFO:teuthology.orchestra.run.gibba023.stdout: "num": 4981 2021-07-11T04:00:22.692 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:00:22.692 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:00:22.693 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:00:22.693 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:00:22.693 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 2285345153 2021-07-11T04:00:22.693 INFO:teuthology.orchestra.run.gibba023.stdout: } 2021-07-11T04:00:22.693 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:00:22.694 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:00:22.694 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:00:22.694 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:00:22.694 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 2285345153 2021-07-11T04:00:22.695 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:00:22.695 INFO:teuthology.orchestra.run.gibba023.stdout: "inst_str": "client.4981 172.21.2.123:0/2285345153", 2021-07-11T04:00:22.695 INFO:teuthology.orchestra.run.gibba023.stdout: "addr_str": "172.21.2.123:0/2285345153", 2021-07-11T04:00:22.695 INFO:teuthology.orchestra.run.gibba023.stdout: "inode_count": 1, 2021-07-11T04:00:22.695 INFO:teuthology.orchestra.run.gibba023.stdout: "mds_epoch": 39, 2021-07-11T04:00:22.696 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch": 87, 2021-07-11T04:00:22.696 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:00:22.696 INFO:teuthology.orchestra.run.gibba023.stdout: "blacklisted": false, 2021-07-11T04:00:22.696 INFO:teuthology.orchestra.run.gibba023.stdout: "fs_name": "cephfs" 2021-07-11T04:00:22.697 INFO:teuthology.orchestra.run.gibba023.stdout:} 2021-07-11T04:00:22.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:22.698 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:22.699 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:22.711 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:00:22.713 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:00:22.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:22.714 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:22.772 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:22.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5eced8e700 1 Processor -- start 2021-07-11T04:00:22.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5eced8e700 1 -- start start 2021-07-11T04:00:22.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5eced8e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5ec80b1970 0x7f5ec805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:22.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5eced8e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec80a7a90 con 0x7f5ec80a76b0 2021-07-11T04:00:22.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5eced8e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec8147620 con 0x7f5ec8067180 2021-07-11T04:00:22.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5eced8e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec81478c0 con 0x7f5ec80b1970 2021-07-11T04:00:22.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5eccb2a700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f5ec80a76b0 0x7f5ec8066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44256/0 (socket says 172.21.2.104:44256) 2021-07-11T04:00:22.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5eccb2a700 1 -- 172.21.2.104:0/857515553 learned_addr learned my addr 172.21.2.104:0/857515553 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:22.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384345594 0 0) 0x7f5ec80a7a90 con 0x7f5ec80a76b0 2021-07-11T04:00:22.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ea8003260 con 0x7f5ec80a76b0 2021-07-11T04:00:22.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ecd32b700 1 --2- 172.21.2.104:0/857515553 >> v2:172.21.2.105:3300/0 conn(0x7f5ec80b1970 0x7f5ec805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:22.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3305487945 0 0) 0x7f5ea8003260 con 0x7f5ec80a76b0 2021-07-11T04:00:22.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 >> v2:172.21.2.105:3300/0 conn(0x7f5ec80b1970 msgr2=0x7f5ec805fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:22.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 --2- 172.21.2.104:0/857515553 >> v2:172.21.2.105:3300/0 conn(0x7f5ec80b1970 0x7f5ec805fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:22.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 >> v1:172.21.2.105:6789/0 conn(0x7f5ec8067180 legacy=0x7f5ec805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:22.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec8147b60 con 0x7f5ec80a76b0 2021-07-11T04:00:22.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f5eb0001660 con 0x7f5ec80a76b0 2021-07-11T04:00:22.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 >> v1:172.21.2.104:6789/0 conn(0x7f5ec80a76b0 legacy=0x7f5ec8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:22.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 --2- 172.21.2.104:0/857515553 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ea8003820 0x7f5ea8003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:22.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 --2- 172.21.2.104:0/857515553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 0x7f5ea8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:22.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 --2- 172.21.2.104:0/857515553 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ea8004f90 0x7f5ea800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:22.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec81478c0 con 0x7f5ea8004500 2021-07-11T04:00:22.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea800be80 con 0x7f5ea8003820 2021-07-11T04:00:22.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea800c040 con 0x7f5ea8004f90 2021-07-11T04:00:22.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec7fff700 1 --2- 172.21.2.104:0/857515553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 0x7f5ea8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:22.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec7fff700 1 -- 172.21.2.104:0/857515553 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ea8004f90 msgr2=0x7f5ea800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:22.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec7fff700 1 --2- 172.21.2.104:0/857515553 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ea8004f90 0x7f5ea800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:22.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5eccb2a700 1 --2- 172.21.2.104:0/857515553 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ea8003820 0x7f5ea8003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:22.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec7fff700 1 -- 172.21.2.104:0/857515553 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ea8003820 msgr2=0x7f5ea8003c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:22.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec7fff700 1 --2- 172.21.2.104:0/857515553 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ea8003820 0x7f5ea8003c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:22.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ecd32b700 1 -- 172.21.2.104:0/857515553 reap_dead start 2021-07-11T04:00:22.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.826+0000 7f5ec7fff700 1 -- 172.21.2.104:0/857515553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea8003260 con 0x7f5ea8004500 2021-07-11T04:00:22.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5ec7fff700 1 --2- 172.21.2.104:0/857515553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 0x7f5ea8004980 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f5eb8001fa0 tx=0x7f5eb8007a00).ready entity=mon.0 client_cookie=a210d0087352edbd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:22.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eb8007d10 con 0x7f5ea8004500 2021-07-11T04:00:22.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb8007e70 con 0x7f5ea8004500 2021-07-11T04:00:22.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5ec77fe700 1 -- 172.21.2.104:0/857515553 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eb8007220 con 0x7f5ea8004500 2021-07-11T04:00:22.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- 172.21.2.104:0/857515553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 msgr2=0x7f5ea8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:22.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 --2- 172.21.2.104:0/857515553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 0x7f5ea8004980 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f5eb8001fa0 tx=0x7f5eb8007a00).stop 2021-07-11T04:00:22.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- 172.21.2.104:0/857515553 shutdown_connections 2021-07-11T04:00:22.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 --2- 172.21.2.104:0/857515553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 0x7f5ea8004980 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:22.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- 172.21.2.104:0/857515553 >> 172.21.2.104:0/857515553 conn(0x7f5ec80b53e0 msgr2=0x7f5ec80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:22.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- 172.21.2.104:0/857515553 shutdown_connections 2021-07-11T04:00:22.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- 172.21.2.104:0/857515553 wait complete. 2021-07-11T04:00:22.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 Processor -- start 2021-07-11T04:00:22.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- start start 2021-07-11T04:00:22.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 0x7f5ec8146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:22.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ec80aac10 0x7f5ec8157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:22.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ec80a9230 0x7f5ec8159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:22.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80ad030 con 0x7f5ea8004500 2021-07-11T04:00:22.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec8157620 con 0x7f5ec80a9230 2021-07-11T04:00:22.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80abec0 con 0x7f5ec80aac10 2021-07-11T04:00:22.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eccb2a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 0x7f5ec8146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:22.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eccb2a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 0x7f5ec8146790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47502/0 (socket says 172.21.2.104:47502) 2021-07-11T04:00:22.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eccb2a700 1 -- 172.21.2.104:0/2578282714 learned_addr learned my addr 172.21.2.104:0/2578282714 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:22.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eccb2a700 1 -- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ec80aac10 msgr2=0x7f5ec8157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:22.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eccb2a700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ec80aac10 0x7f5ec8157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:22.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eccb2a700 1 -- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ec80a9230 msgr2=0x7f5ec8159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:22.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eccb2a700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ec80a9230 0x7f5ec8159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:22.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eccb2a700 1 -- 172.21.2.104:0/2578282714 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec81569f0 con 0x7f5ea8004500 2021-07-11T04:00:22.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eccb2a700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 0x7f5ec8146790 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f5ec81577b0 tx=0x7f5eb0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:22.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5ec5ffb700 1 -- 172.21.2.104:0/2578282714 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eb0007170 con 0x7f5ea8004500 2021-07-11T04:00:22.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5ec5ffb700 1 -- 172.21.2.104:0/2578282714 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb00072d0 con 0x7f5ea8004500 2021-07-11T04:00:22.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- 172.21.2.104:0/2578282714 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec8156c90 con 0x7f5ea8004500 2021-07-11T04:00:22.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5ec5ffb700 1 -- 172.21.2.104:0/2578282714 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5eb0004920 con 0x7f5ea8004500 2021-07-11T04:00:22.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- 172.21.2.104:0/2578282714 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec81566e0 con 0x7f5ea8004500 2021-07-11T04:00:22.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5eced8e700 1 -- 172.21.2.104:0/2578282714 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec8156e20 con 0x7f5ea8004500 2021-07-11T04:00:22.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5ec5ffb700 1 -- 172.21.2.104:0/2578282714 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5eb0004a80 con 0x7f5ea8004500 2021-07-11T04:00:22.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5ec5ffb700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5eac0532d0 0x7f5eac055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:22.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.830+0000 7f5ec5ffb700 1 -- 172.21.2.104:0/2578282714 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5eb006b8d0 con 0x7f5ea8004500 2021-07-11T04:00:22.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.834+0000 7f5ec7fff700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5eac0532d0 0x7f5eac055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:22.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.834+0000 7f5ec7fff700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5eac0532d0 0x7f5eac055790 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f5eb8001fa0 tx=0x7f5eb8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:22.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.834+0000 7f5ec5ffb700 1 -- 172.21.2.104:0/2578282714 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5eb0042c10 con 0x7f5ea8004500 2021-07-11T04:00:22.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.942+0000 7f5eced8e700 1 -- 172.21.2.104:0/2578282714 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ec8057350 con 0x7f5ea8004500 2021-07-11T04:00:22.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.942+0000 7f5ec5ffb700 1 -- 172.21.2.104:0/2578282714 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2487 (secure 0 0 0) 0x7f5eb0046230 con 0x7f5ea8004500 2021-07-11T04:00:22.947 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:22.947 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/43570850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":43570850},{"type":"v1","addr":"172.21.2.105:6835","nonce":43570850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/3334136782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3334136782},{"type":"v1","addr":"172.21.2.104:6833","nonce":3334136782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:15.674524+0000","modified":"2021-07-11T04:00:16.678211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/4286166182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":4286166182},{"type":"v1","addr":"172.21.2.104:6835","nonce":4286166182}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-07-11T04:00:22.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.942+0000 7f5eced8e700 1 -- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5eac0532d0 msgr2=0x7f5eac055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:22.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.942+0000 7f5eced8e700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5eac0532d0 0x7f5eac055790 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f5eb8001fa0 tx=0x7f5eb8015000).stop 2021-07-11T04:00:22.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.942+0000 7f5eced8e700 1 -- 172.21.2.104:0/2578282714 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 msgr2=0x7f5ec8146790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:22.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.942+0000 7f5eced8e700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 0x7f5ec8146790 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f5ec81577b0 tx=0x7f5eb0009040).stop 2021-07-11T04:00:22.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.946+0000 7f5eced8e700 1 -- 172.21.2.104:0/2578282714 shutdown_connections 2021-07-11T04:00:22.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.946+0000 7f5eced8e700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ec80a9230 0x7f5ec8159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:22.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.946+0000 7f5eced8e700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ec80aac10 0x7f5ec8157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:22.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.946+0000 7f5eced8e700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5eac0532d0 0x7f5eac055790 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:22.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.946+0000 7f5eced8e700 1 --2- 172.21.2.104:0/2578282714 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea8004500 0x7f5ec8146790 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:22.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.946+0000 7f5eced8e700 1 -- 172.21.2.104:0/2578282714 >> 172.21.2.104:0/2578282714 conn(0x7f5ec80b53e0 msgr2=0x7f5ec805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:22.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.946+0000 7f5eced8e700 1 -- 172.21.2.104:0/2578282714 shutdown_connections 2021-07-11T04:00:22.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:22.946+0000 7f5eced8e700 1 -- 172.21.2.104:0/2578282714 wait complete. 2021-07-11T04:00:22.951 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 39 2021-07-11T04:00:22.963 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump inode 1 2021-07-11T04:00:23.020 INFO:teuthology.orchestra.run.gibba004.stdout:{"path":"/","ino":1,"rdev":0,"ctime":"2021-07-11T04:00:19.848841+0000","btime":"2021-07-11T04:00:15.678783+0000","mode":17407,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":4194304,"stripe_count":1,"object_size":4194304,"pool_id":16,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":0,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2021-07-11T04:00:15.678783+0000","atime":"0.000000","time_warp_seq":0,"change_attr":1,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2021-07-11T04:00:19.848841+0000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"2021-07-11T04:00:15.678783+0000"},"version":3,"file_data_version":0,"xattr_version":1,"backtrace_version":3,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":2,"is_frozen":false,"is_freezing":false,"pins":{"request":1,"lock":3,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1,"authpin":1},"nref":9,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":1,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"xlockdone","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":1,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{},"xattrlock":{},"snaplock":{"gather_set":[],"state":"sync","is_leased":false,"num_rdlocks":1,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"nestlock":{},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent"],"client_caps":[{"client_id":4972,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2},{"client_id":4981,"pending":"pLsXsFs","issued":"pLsXsFs","wanted":"-","last_sent":2}],"loner":-1,"want_loner":-1,"mds_caps_wanted":[]} 2021-07-11T04:00:23.021 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "2021-07-11T04:00:15.678783+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "0.000000", "auth_pins": 2, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 1, "state": "xlockdone", "xlock_by": {} }, "backtrace_version": 3, "btime": "2021-07-11T04:00:15.678783+0000", "change_attr": 1, "client_caps": [ { "client_id": 4972, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" }, { "client_id": 4981, "issued": "pLsXsFs", "last_sent": 2, "pending": "pLsXsFs", "wanted": "-" } ], "client_ranges": [], "ctime": "2021-07-11T04:00:19.848841+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": {}, "flocklock": {}, "gid": 0, "ino": 1, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 4194304, "pool_id": 16, "pool_ns": "", "stripe_count": 1, "stripe_unit": 4194304 }, "linklock": {}, "loner": -1, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 17407, "mtime": "2021-07-11T04:00:15.678783+0000", "nestlock": {}, "nlink": 1, "nref": 9, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/", "pins": { "authpin": 1, "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "lock": 3, "request": 1 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "2021-07-11T04:00:19.848841+0000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": { "gather_set": [], "is_leased": false, "num_rdlocks": 1, "num_wrlocks": 0, "num_xlocks": 0, "state": "sync", "xlock_by": {} }, "states": [ "auth", "dirty", "dirtyparent" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 0, "truncate_size": 18446744073709551615, "uid": 0, "version": 3, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 1, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": -1, "xattr_version": 1, "xattrlock": {}, "xattrs": [] } 2021-07-11T04:00:23.021 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:00:23.029 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:23.041 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:23.056 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:23.071 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:23.087 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:00:23.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:23.088 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:23.089 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:23.120 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:00:23.122 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:00:23.122 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T04:00:23.123 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:00:23.123 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:23.137 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15312]: fuse finished with error 0 and tester_r 0 2021-07-11T04:00:23.138 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:23.139 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:23.139 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:23.191 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T04:00:23.193 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:00:23.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:00:23.308 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:23.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:23.315 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:23.316 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:23.329 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:00:23.331 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:00:23.331 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba023.front.sepia.ceph.com... 2021-07-11T04:00:23.332 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:00:23.333 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:23.356 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14749]: fuse finished with error 0 and tester_r 0 2021-07-11T04:00:23.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:23.361 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:23.362 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:23.413 INFO:teuthology.orchestra.run.gibba023.stdout:ext2/ext3 2021-07-11T04:00:23.414 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:00:23.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:00:23.527 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:23.531 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:00:23.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.582+0000 7fc8a70b3700 1 Processor -- start 2021-07-11T04:00:23.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.582+0000 7fc8a70b3700 1 -- start start 2021-07-11T04:00:23.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.582+0000 7fc8a70b3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc8a00b1db0 0x7fc8a00b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:23.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.582+0000 7fc8a70b3700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a00b0350 con 0x7fc8a00a70c0 2021-07-11T04:00:23.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.582+0000 7fc8a70b3700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a00b05f0 con 0x7fc8a01483a0 2021-07-11T04:00:23.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.582+0000 7fc8a70b3700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a00a90c0 con 0x7fc8a00b1db0 2021-07-11T04:00:23.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.582+0000 7fc8a4e4f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc8a00b1db0 0x7fc8a00b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:23.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.582+0000 7fc8a5650700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc8a00a70c0 0x7fc8a0144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44276/0 (socket says 172.21.2.104:44276) 2021-07-11T04:00:23.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.582+0000 7fc8a5650700 1 -- 172.21.2.104:0/44609284 learned_addr learned my addr 172.21.2.104:0/44609284 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:23.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a4e4f700 1 -- 172.21.2.104:0/44609284 >> v1:172.21.2.105:6789/0 conn(0x7fc8a01483a0 legacy=0x7fc8a00b2190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:23.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a4e4f700 1 -- 172.21.2.104:0/44609284 >> v1:172.21.2.104:6789/0 conn(0x7fc8a00a70c0 legacy=0x7fc8a0144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:23.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a4e4f700 1 -- 172.21.2.104:0/44609284 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a00b0890 con 0x7fc8a00b1db0 2021-07-11T04:00:23.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a4e4f700 1 --2- 172.21.2.104:0/44609284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1db0 0x7fc8a00b0fd0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7fc8a0147730 tx=0x7fc890006df0).ready entity=mon.0 client_cookie=28aff2564bdebbe4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:23.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89f7fe700 1 -- 172.21.2.104:0/44609284 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc89000d5c0 con 0x7fc8a00b1db0 2021-07-11T04:00:23.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89f7fe700 1 -- 172.21.2.104:0/44609284 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc890000e90 con 0x7fc8a00b1db0 2021-07-11T04:00:23.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89f7fe700 1 -- 172.21.2.104:0/44609284 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc89000d910 con 0x7fc8a00b1db0 2021-07-11T04:00:23.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 -- 172.21.2.104:0/44609284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1db0 msgr2=0x7fc8a00b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:23.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 --2- 172.21.2.104:0/44609284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1db0 0x7fc8a00b0fd0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7fc8a0147730 tx=0x7fc890006df0).stop 2021-07-11T04:00:23.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 -- 172.21.2.104:0/44609284 shutdown_connections 2021-07-11T04:00:23.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 --2- 172.21.2.104:0/44609284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1db0 0x7fc8a00b0fd0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:23.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 -- 172.21.2.104:0/44609284 >> 172.21.2.104:0/44609284 conn(0x7fc8a005f6b0 msgr2=0x7fc8a005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:23.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 -- 172.21.2.104:0/44609284 shutdown_connections 2021-07-11T04:00:23.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 -- 172.21.2.104:0/44609284 wait complete. 2021-07-11T04:00:23.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 Processor -- start 2021-07-11T04:00:23.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 -- start start 2021-07-11T04:00:23.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc8a00a70c0 0x7fc8a00adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:23.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1db0 0x7fc8a0156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:23.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a01483a0 0x7fc8a0157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:23.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a0156b60 con 0x7fc8a00b1db0 2021-07-11T04:00:23.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a015a0d0 con 0x7fc8a00a70c0 2021-07-11T04:00:23.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a014b420 con 0x7fc8a01483a0 2021-07-11T04:00:23.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1db0 0x7fc8a0156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:23.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1db0 0x7fc8a0156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47514/0 (socket says 172.21.2.104:47514) 2021-07-11T04:00:23.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89ffff700 1 -- 172.21.2.104:0/1881362768 learned_addr learned my addr 172.21.2.104:0/1881362768 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:23.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89ffff700 1 -- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a01483a0 msgr2=0x7fc8a0157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:23.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89ffff700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a01483a0 0x7fc8a0157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:23.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89ffff700 1 -- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc8a00a70c0 msgr2=0x7fc8a00adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:23.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89ffff700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc8a00a70c0 0x7fc8a00adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:23.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89ffff700 1 -- 172.21.2.104:0/1881362768 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89000b180 con 0x7fc8a00b1db0 2021-07-11T04:00:23.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89ffff700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1db0 0x7fc8a0156f50 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7fc888002ef0 tx=0x7fc888006d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:23.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89dffb700 1 -- 172.21.2.104:0/1881362768 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc888009e00 con 0x7fc8a00b1db0 2021-07-11T04:00:23.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89dffb700 1 -- 172.21.2.104:0/1881362768 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc888002320 con 0x7fc8a00b1db0 2021-07-11T04:00:23.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc89dffb700 1 -- 172.21.2.104:0/1881362768 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc888002680 con 0x7fc8a00b1db0 2021-07-11T04:00:23.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 -- 172.21.2.104:0/1881362768 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a014b6c0 con 0x7fc8a00b1db0 2021-07-11T04:00:23.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.586+0000 7fc8a70b3700 1 -- 172.21.2.104:0/1881362768 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a00ae540 con 0x7fc8a00b1db0 2021-07-11T04:00:23.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.590+0000 7fc8a70b3700 1 -- 172.21.2.104:0/1881362768 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8a00a9c50 con 0x7fc8a00b1db0 2021-07-11T04:00:23.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.590+0000 7fc89dffb700 1 -- 172.21.2.104:0/1881362768 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc88800f030 con 0x7fc8a00b1db0 2021-07-11T04:00:23.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.590+0000 7fc89dffb700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053260 0x7fc884055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:23.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.590+0000 7fc89dffb700 1 -- 172.21.2.104:0/1881362768 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc88806b260 con 0x7fc8a00b1db0 2021-07-11T04:00:23.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.590+0000 7fc8a4e4f700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053260 0x7fc884055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:23.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.590+0000 7fc8a4e4f700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053260 0x7fc884055720 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fc8a0147730 tx=0x7fc890014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:23.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.590+0000 7fc89dffb700 1 -- 172.21.2.104:0/1881362768 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc888042490 con 0x7fc8a00b1db0 2021-07-11T04:00:23.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.670+0000 7fc8a70b3700 1 -- 172.21.2.104:0/1881362768 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc8a0057350 con 0x7fc8a00b1db0 2021-07-11T04:00:23.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc89dffb700 1 -- 172.21.2.104:0/1881362768 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v87) v1 ==== 81+0+33648 (secure 0 0 0) 0x7fc888045ab0 con 0x7fc8a00b1db0 2021-07-11T04:00:23.677 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:23.677 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:00:23.677 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 87, 2021-07-11T04:00:23.678 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:00:23.678 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:00:23.678 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:00:16.679409+0000", 2021-07-11T04:00:23.678 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:23.679 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:23.679 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:00:23.679 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:00:23.679 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:00:23.680 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:00:23.680 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:00:23.680 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:00:23.680 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:00:23.680 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:23.681 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:00:23.681 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:00:23.681 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:00:23.681 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:00:23.681 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:00:23.682 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 16, 2021-07-11T04:00:23.682 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:00:23.682 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:00:23.682 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:00:23.683 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:00:23.683 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:00:23.683 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.684 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:00:23.684 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:00:23.684 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:00:23.684 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:23.685 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:23.685 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:23.685 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:23.685 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:23.686 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:23.686 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:23.686 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:23.686 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:00:23.686 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:00:23.687 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:00:23.687 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:00:23.687 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:00:23.687 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:23.688 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:23.688 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:23.688 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:23.688 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:23.688 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:23.689 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:23.689 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.689 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:00:23.689 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:23.689 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:23.690 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:23.690 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:23.690 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:23.690 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:23.691 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:23.691 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:23.691 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:23.691 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:23.691 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:23.692 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:23.692 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:23.692 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:23.692 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:23.692 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:23.693 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:23.693 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:23.693 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:23.693 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:23.694 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:23.694 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:23.694 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:23.695 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:23.695 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:23.695 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:23.695 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.696 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:23.696 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:23.696 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:23.696 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:23.696 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:23.697 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:23.697 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:23.697 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:23.697 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:23.698 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:23.698 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:23.698 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:00:23.698 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:00:23.698 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.699 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:23.699 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:00:23.699 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.699 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.700 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.700 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:00:23.700 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:00:23.700 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:00:23.700 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:23.701 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:23.701 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:23.701 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:23.701 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:23.701 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:23.702 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:23.702 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:23.702 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:23.702 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:23.703 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:23.703 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:23.703 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:23.703 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:23.703 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:23.704 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:23.705 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:23.705 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:23.705 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:23.706 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:23.706 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.706 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:00:23.706 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:23.707 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:23.707 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:23.707 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:23.707 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:23.707 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:23.708 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:23.708 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:23.708 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:23.708 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:23.709 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:23.709 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:23.709 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:23.709 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:23.709 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:23.710 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:23.710 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:23.710 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:23.710 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:23.711 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:23.711 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:23.711 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:23.711 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:23.711 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:23.712 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:23.712 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:23.712 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.712 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:23.712 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:23.713 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:23.713 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:23.713 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:23.713 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:23.714 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:23.714 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:23.714 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:23.715 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:23.715 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:23.715 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:00:23.715 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:23.716 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:00:23.716 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.716 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.716 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.716 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 15, 2021-07-11T04:00:23.717 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T04:00:23.717 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:00:13.631492+0000", 2021-07-11T04:00:23.717 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:23.717 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:23.718 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:23.718 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:23.718 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:23.718 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:23.718 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:23.719 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:23.719 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:23.719 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:23.719 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:23.720 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:23.720 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:23.720 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:23.720 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:23.720 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:23.721 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:23.721 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:23.721 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:23.721 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:23.721 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.722 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "86", 2021-07-11T04:00:23.722 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:23.722 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:23.722 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:23.723 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:23.723 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:23.723 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:23.723 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:23.723 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:23.724 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:23.724 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:23.724 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:23.724 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:23.725 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:23.725 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:23.725 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:23.725 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:23.725 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:23.726 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:23.726 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:23.726 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:23.726 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:23.727 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:23.727 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:23.727 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:23.727 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:23.727 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:23.728 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.728 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:23.728 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:23.728 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:23.728 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:23.729 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:23.729 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:23.729 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:23.729 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:23.730 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:23.730 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:23.730 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:23.730 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:00:23.730 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T04:00:23.731 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T04:00:23.731 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T04:00:23.731 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.731 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:23.731 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:00:23.732 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T04:00:23.732 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.732 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.732 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.733 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.733 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 16, 2021-07-11T04:00:23.733 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T04:00:23.733 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:00:14.726743+0000", 2021-07-11T04:00:23.733 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:23.734 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:23.734 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:23.734 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:23.734 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:23.734 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:23.735 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:23.735 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:23.735 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:23.735 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:23.736 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:23.736 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:23.736 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:23.736 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:23.736 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:23.737 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:23.737 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:23.737 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:23.737 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:23.738 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:23.738 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.738 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "87", 2021-07-11T04:00:23.738 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:23.738 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:23.739 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:23.739 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:23.739 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:23.739 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:23.740 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:23.740 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:23.740 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:23.740 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:23.740 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:23.741 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:23.741 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:23.741 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:23.741 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:23.741 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:23.742 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:23.742 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:23.742 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:23.742 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:23.743 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:23.743 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:23.743 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:23.743 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:23.743 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:23.744 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:23.744 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.744 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:23.744 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:23.745 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:23.745 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:23.745 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:23.745 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:23.745 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:23.746 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:23.746 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:23.746 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:23.746 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:23.746 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:00:23.747 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:23.747 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:00:23.747 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T04:00:23.747 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.748 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.748 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.748 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:23.748 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:00:23.748 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.749 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:23.749 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:00:23.749 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:23.749 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:23.750 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:23.750 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:23.750 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:23.750 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:23.750 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:23.751 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:23.751 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:23.751 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:23.751 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:23.751 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.752 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.752 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.752 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:00:23.752 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:23.753 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.753 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.753 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.753 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:00:23.754 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:23.754 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.754 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.754 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.754 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:23.755 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.755 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.755 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.755 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:00:23.755 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:23.756 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.756 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.756 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.756 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:00:23.757 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:23.757 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.757 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.757 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.757 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:23.758 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.758 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.758 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.758 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:00:23.758 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:23.759 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.759 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.759 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.759 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:00:23.760 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:23.760 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.760 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.760 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.760 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:23.761 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.761 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.761 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.761 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:00:23.762 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:23.762 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.762 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.762 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.763 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:00:23.763 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:23.763 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.763 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.763 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.764 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:00:23.764 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:00:23.764 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:00:23.764 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:00:23.765 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:23.765 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:23.765 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:23.765 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.765 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.766 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.766 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:23.766 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:00:23.766 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:23.767 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:23.767 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:23.767 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:23.767 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:23.767 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:23.768 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:23.768 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 83, 2021-07-11T04:00:23.768 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:23.768 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:23.768 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:23.769 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.769 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.769 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.769 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:00:23.770 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:23.770 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.770 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.770 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.770 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:00:23.771 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:23.771 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.771 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.771 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.772 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:23.772 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.772 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.772 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.773 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:00:23.773 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:23.773 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.773 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.773 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.774 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:00:23.774 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:23.774 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.774 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.775 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.775 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:23.775 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.775 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.775 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.776 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:00:23.776 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:23.776 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.776 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.776 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.777 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:00:23.777 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:23.777 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.777 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.778 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.778 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:23.778 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.778 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.778 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.779 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:00:23.779 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:23.779 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.779 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.779 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.780 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:00:23.780 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:23.780 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.780 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.781 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.781 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:00:23.781 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:00:23.781 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:00:23.781 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:00:23.782 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:23.782 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:23.782 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:23.782 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.783 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.783 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.783 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:23.783 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:00:23.784 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:23.784 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:23.784 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:23.784 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:23.784 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:23.785 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:23.785 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:23.785 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:23.785 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:23.785 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:23.786 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:23.786 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.786 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.786 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.787 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:00:23.787 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:23.787 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.787 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.787 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.788 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:00:23.788 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:23.788 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.788 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.789 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.789 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:23.789 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.789 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.789 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.790 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:00:23.790 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:23.790 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.790 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.790 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.791 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:00:23.791 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:23.791 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.791 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.792 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.792 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:23.792 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.792 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.793 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.793 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:00:23.793 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:23.793 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.793 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.794 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.794 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:00:23.794 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:23.794 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.794 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.795 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.795 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:23.795 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.795 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.796 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.796 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:00:23.796 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:23.796 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.796 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.797 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.797 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:00:23.797 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:23.797 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.797 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.798 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.798 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:00:23.798 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:00:23.798 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:00:23.799 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:00:23.799 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:23.799 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:23.799 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:23.800 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.800 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.800 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.800 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:23.800 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:00:23.801 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:23.801 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:23.801 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:23.801 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:23.801 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:23.802 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:23.802 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:23.802 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:23.802 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:23.803 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:23.803 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:23.803 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.803 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.803 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.804 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:00:23.804 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:23.804 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.804 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.805 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.805 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:00:23.805 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:23.805 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.805 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.806 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.806 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:23.806 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.806 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.806 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.807 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:00:23.807 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:23.807 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.807 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.808 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.808 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:00:23.808 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:23.808 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.808 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.809 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.809 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:23.809 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.809 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.810 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.810 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:00:23.810 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:23.810 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.810 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.811 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.811 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:00:23.811 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:23.811 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.812 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.812 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.812 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:23.812 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.812 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.813 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.813 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:00:23.813 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:23.813 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.814 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.814 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.814 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:00:23.814 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:23.814 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.815 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.815 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.815 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:00:23.815 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:00:23.816 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:00:23.816 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:00:23.816 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:23.816 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:23.816 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:23.817 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.817 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.817 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.817 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:23.818 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:00:23.818 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:23.818 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:23.818 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:23.818 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:23.819 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:23.819 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:23.819 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:23.819 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:23.819 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:23.820 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:23.820 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:23.820 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.820 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.821 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.821 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:00:23.821 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:23.821 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.822 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.822 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.822 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:00:23.822 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:23.822 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.823 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.823 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.823 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:23.823 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.824 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.824 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.824 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:00:23.824 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:23.824 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.825 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.825 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.825 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:00:23.825 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:23.825 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.826 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.826 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.826 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:23.826 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.827 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.827 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.827 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:00:23.827 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:23.827 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.828 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.828 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.828 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:00:23.828 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:23.829 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.829 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.829 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.829 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:23.829 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.830 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.830 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.830 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:00:23.830 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:23.831 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.831 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.831 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.831 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:00:23.831 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:23.832 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.832 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.832 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.832 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:00:23.833 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:00:23.833 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:00:23.833 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:00:23.833 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:23.833 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:23.834 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:23.834 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.834 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.834 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.834 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:23.835 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:00:23.835 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:23.835 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:23.835 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:23.836 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:23.836 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:23.836 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:23.836 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:23.836 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:23.837 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:23.837 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:23.837 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:23.837 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.837 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.838 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.838 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:00:23.838 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:23.838 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.839 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.839 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.839 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:00:23.839 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:23.839 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.840 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.840 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.840 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:23.840 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.841 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.841 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.841 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:00:23.841 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:23.842 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.842 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.842 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.842 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:00:23.842 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:23.843 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.843 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.843 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.843 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:23.844 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.844 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.844 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.844 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:00:23.844 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:23.845 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.845 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.845 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.845 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:00:23.845 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:23.846 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.846 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.846 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.846 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:23.847 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.847 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.847 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.847 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:00:23.847 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:23.848 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.848 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.848 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.848 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:00:23.849 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:23.849 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.849 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.849 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.849 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:00:23.850 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:00:23.850 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:00:23.850 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:00:23.850 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:23.850 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:23.851 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:23.851 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.851 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.851 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.852 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:23.852 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:00:23.852 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:23.852 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:23.852 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:23.853 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:23.853 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:23.853 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:23.853 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:23.854 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:23.854 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:23.854 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:23.854 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:23.854 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.855 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.855 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.855 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:00:23.855 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:23.855 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.856 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.856 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.856 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:00:23.856 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:23.857 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.857 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.857 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.857 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:23.857 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.858 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.858 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.858 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:00:23.858 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:23.858 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.859 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.859 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.859 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:00:23.859 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:23.860 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.860 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.860 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.860 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:23.860 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.861 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.861 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.861 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:00:23.861 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:23.862 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.862 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.862 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.862 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:00:23.862 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:23.863 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.863 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.863 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.863 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:23.864 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.864 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.864 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.864 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:00:23.864 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:23.865 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.865 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.865 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.865 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:00:23.865 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:23.866 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.866 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.866 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.866 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:00:23.867 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:00:23.867 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:00:23.867 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:00:23.867 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:23.867 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:23.868 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:23.868 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.868 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.868 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.869 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:23.869 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:00:23.869 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:23.869 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:23.869 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:23.870 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:23.870 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:23.870 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:23.870 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:23.871 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 73, 2021-07-11T04:00:23.871 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:23.871 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:23.871 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:23.871 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.872 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.872 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.872 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:00:23.872 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:23.873 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.873 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.873 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.873 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:00:23.873 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:23.874 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.874 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.874 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.874 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:23.875 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.875 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.875 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.875 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:00:23.875 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:23.876 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.876 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.876 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.876 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:00:23.876 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:23.877 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.877 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.877 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.877 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:23.877 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.878 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.878 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.878 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:00:23.878 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:23.879 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.879 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.879 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.879 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:00:23.879 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:23.880 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.880 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.880 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.880 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:23.881 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:23.881 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.881 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:23.881 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:00:23.882 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:23.882 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.882 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.882 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:23.882 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:00:23.883 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:23.883 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.883 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.883 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.884 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:00:23.884 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:00:23.884 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:00:23.884 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:00:23.884 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:23.885 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:23.885 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:23.885 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:23.885 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.885 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:23.886 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:00:23.886 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.886 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:23.886 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:23.886 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:23.887 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:23.887 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:23.887 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:23.887 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:00:23.888 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:23.888 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.888 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.888 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:23.888 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:23.889 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:23.889 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:23.889 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:23.889 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:23.889 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:00:23.890 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:23.890 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.890 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.890 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:23.891 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:23.891 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:23.891 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:23.891 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:23.891 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:23.892 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:00:23.892 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:23.892 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.892 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.893 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:23.893 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:23.893 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:23.893 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:23.893 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:23.894 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:23.894 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:00:23.894 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:23.894 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.895 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.895 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:23.895 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:23.895 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:23.895 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:23.896 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:23.896 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:23.896 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:00:23.896 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:23.896 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.897 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.897 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:23.897 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:23.897 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:23.897 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:23.898 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:23.898 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:23.898 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:00:23.898 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:23.899 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.899 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.899 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:23.899 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:23.899 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:23.900 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:23.900 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:23.900 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:23.900 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:00:23.901 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:23.901 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.901 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:23.901 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:23.902 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:23.902 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:23.902 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:23.902 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:23.902 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:23.903 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:00:23.903 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:23.903 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.903 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:23.903 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:00:23.904 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:00:23.904 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:00:23.904 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:00:23.904 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": {}, 2021-07-11T04:00:23.905 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:00:23.905 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:00:23.905 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:00:23.905 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:00:23.905 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:00:23.906 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:00:23.906 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:00:23.906 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:00:23.906 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:23.906 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:23.907 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:00:23.907 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:00:23.907 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:00:23.907 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:00:23.907 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:00:23.908 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:00:23.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 -- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053260 msgr2=0x7fc884055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:23.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053260 0x7fc884055720 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fc8a0147730 tx=0x7fc890014000).stop 2021-07-11T04:00:23.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 -- 172.21.2.104:0/1881362768 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1db0 msgr2=0x7fc8a0156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:23.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1db0 0x7fc8a0156f50 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7fc888002ef0 tx=0x7fc888006d40).stop 2021-07-11T04:00:23.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 -- 172.21.2.104:0/1881362768 shutdown_connections 2021-07-11T04:00:23.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc8a01483a0 0x7fc8a0157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:23.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc884053260 0x7fc884055720 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:23.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc8a00b1db0 0x7fc8a0156f50 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:23.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 --2- 172.21.2.104:0/1881362768 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc8a00a70c0 0x7fc8a00adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:23.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 -- 172.21.2.104:0/1881362768 >> 172.21.2.104:0/1881362768 conn(0x7fc8a005f6b0 msgr2=0x7fc8a00b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:23.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 -- 172.21.2.104:0/1881362768 shutdown_connections 2021-07-11T04:00:23.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:23.674+0000 7fc8a70b3700 1 -- 172.21.2.104:0/1881362768 wait complete. 2021-07-11T04:00:23.913 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:24.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde989f700 1 Processor -- start 2021-07-11T04:00:24.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde989f700 1 -- start start 2021-07-11T04:00:24.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde989f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7efde4067180 0x7efde405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde989f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7efde40b1970 0x7efde405fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde989f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde40a7a90 con 0x7efde40a76b0 2021-07-11T04:00:24.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde989f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde4147620 con 0x7efde4067180 2021-07-11T04:00:24.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde989f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde41478c0 con 0x7efde40b1970 2021-07-11T04:00:24.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde27fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7efde4067180 0x7efde405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde27fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7efde4067180 0x7efde405f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47526/0 (socket says 172.21.2.104:47526) 2021-07-11T04:00:24.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde27fc700 1 -- 172.21.2.104:0/1002681672 learned_addr learned my addr 172.21.2.104:0/1002681672 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:24.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde27fc700 1 -- 172.21.2.104:0/1002681672 >> v2:172.21.2.105:3300/0 conn(0x7efde40b1970 msgr2=0x7efde405fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:24.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde27fc700 1 --2- 172.21.2.104:0/1002681672 >> v2:172.21.2.105:3300/0 conn(0x7efde40b1970 0x7efde405fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:24.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde27fc700 1 -- 172.21.2.104:0/1002681672 >> v1:172.21.2.105:6789/0 conn(0x7efde40a76b0 legacy=0x7efde4066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:24.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde27fc700 1 -- 172.21.2.104:0/1002681672 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde4147b60 con 0x7efde4067180 2021-07-11T04:00:24.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde27fc700 1 --2- 172.21.2.104:0/1002681672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efde4067180 0x7efde405f6b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7efde406a0e0 tx=0x7efdd4006eb0).ready entity=mon.0 client_cookie=c952aae87f5dbfb4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:24.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde1ffb700 1 -- 172.21.2.104:0/1002681672 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdd400d5c0 con 0x7efde4067180 2021-07-11T04:00:24.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde1ffb700 1 -- 172.21.2.104:0/1002681672 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd4000e90 con 0x7efde4067180 2021-07-11T04:00:24.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde1ffb700 1 -- 172.21.2.104:0/1002681672 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdd400d910 con 0x7efde4067180 2021-07-11T04:00:24.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde989f700 1 -- 172.21.2.104:0/1002681672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efde4067180 msgr2=0x7efde405f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.022+0000 7efde989f700 1 --2- 172.21.2.104:0/1002681672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efde4067180 0x7efde405f6b0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7efde406a0e0 tx=0x7efdd4006eb0).stop 2021-07-11T04:00:24.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 -- 172.21.2.104:0/1002681672 shutdown_connections 2021-07-11T04:00:24.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 --2- 172.21.2.104:0/1002681672 >> v2:172.21.2.105:3300/0 conn(0x7efde40b1970 0x7efde405fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:24.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 --2- 172.21.2.104:0/1002681672 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efde4067180 0x7efde405f6b0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:24.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 -- 172.21.2.104:0/1002681672 >> 172.21.2.104:0/1002681672 conn(0x7efde40b53e0 msgr2=0x7efde40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:24.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 -- 172.21.2.104:0/1002681672 shutdown_connections 2021-07-11T04:00:24.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 -- 172.21.2.104:0/1002681672 wait complete. 2021-07-11T04:00:24.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 Processor -- start 2021-07-11T04:00:24.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 -- start start 2021-07-11T04:00:24.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efde4067180 0x7efde40aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efde40a76b0 0x7efde40abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efde40b1970 0x7efde40ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde40ac160 con 0x7efde4067180 2021-07-11T04:00:24.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde40aee90 con 0x7efde40b1970 2021-07-11T04:00:24.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde40af830 con 0x7efde40a76b0 2021-07-11T04:00:24.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efde4067180 0x7efde40aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efde4067180 0x7efde40aa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47528/0 (socket says 172.21.2.104:47528) 2021-07-11T04:00:24.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde2ffd700 1 -- 172.21.2.104:0/64782553 learned_addr learned my addr 172.21.2.104:0/64782553 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:24.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde2ffd700 1 -- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efde40a76b0 msgr2=0x7efde40abac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde2ffd700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efde40a76b0 0x7efde40abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:24.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde2ffd700 1 -- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efde40b1970 msgr2=0x7efde40ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde2ffd700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efde40b1970 0x7efde40ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:24.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde2ffd700 1 -- 172.21.2.104:0/64782553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd400b1e0 con 0x7efde4067180 2021-07-11T04:00:24.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde2ffd700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efde4067180 0x7efde40aa3e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7efdcc000f70 tx=0x7efdcc007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:24.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efddbfff700 1 -- 172.21.2.104:0/64782553 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdcc00ce80 con 0x7efde4067180 2021-07-11T04:00:24.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efddbfff700 1 -- 172.21.2.104:0/64782553 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdcc013450 con 0x7efde4067180 2021-07-11T04:00:24.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efddbfff700 1 -- 172.21.2.104:0/64782553 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efdcc0137b0 con 0x7efde4067180 2021-07-11T04:00:24.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 -- 172.21.2.104:0/64782553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde40af0f0 con 0x7efde4067180 2021-07-11T04:00:24.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 -- 172.21.2.104:0/64782553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde40acf60 con 0x7efde4067180 2021-07-11T04:00:24.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efde989f700 1 -- 172.21.2.104:0/64782553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde40aad10 con 0x7efde4067180 2021-07-11T04:00:24.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.026+0000 7efddbfff700 1 -- 172.21.2.104:0/64782553 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7efdcc010070 con 0x7efde4067180 2021-07-11T04:00:24.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.030+0000 7efddbfff700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efdc80531f0 0x7efdc80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.030+0000 7efddbfff700 1 -- 172.21.2.104:0/64782553 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7efdcc06ad90 con 0x7efde4067180 2021-07-11T04:00:24.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.030+0000 7efde27fc700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efdc80531f0 0x7efdc80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.030+0000 7efde27fc700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efdc80531f0 0x7efdc80556b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7efde406a0e0 tx=0x7efdd4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:24.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.030+0000 7efddbfff700 1 -- 172.21.2.104:0/64782553 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7efde40aad10 con 0x7efde4067180 2021-07-11T04:00:24.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 -- 172.21.2.104:0/64782553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efde40aad10 con 0x7efde4067180 2021-07-11T04:00:24.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efddbfff700 1 -- 172.21.2.104:0/64782553 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+2487 (secure 0 0 0) 0x7efde40aad10 con 0x7efde4067180 2021-07-11T04:00:24.141 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:24.141 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":39,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/43570850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":43570850},{"type":"v1","addr":"172.21.2.105:6835","nonce":43570850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/3334136782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3334136782},{"type":"v1","addr":"172.21.2.104:6833","nonce":3334136782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:15.674524+0000","modified":"2021-07-11T04:00:16.678211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6636},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6636":{"gid":6636,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/4286166182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":4286166182},{"type":"v1","addr":"172.21.2.104:6835","nonce":4286166182}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-07-11T04:00:24.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 -- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efdc80531f0 msgr2=0x7efdc80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efdc80531f0 0x7efdc80556b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7efde406a0e0 tx=0x7efdd4015000).stop 2021-07-11T04:00:24.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 -- 172.21.2.104:0/64782553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efde4067180 msgr2=0x7efde40aa3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efde4067180 0x7efde40aa3e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7efdcc000f70 tx=0x7efdcc007c30).stop 2021-07-11T04:00:24.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 -- 172.21.2.104:0/64782553 shutdown_connections 2021-07-11T04:00:24.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efde40b1970 0x7efde40ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:24.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efde40a76b0 0x7efde40abac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:24.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efdc80531f0 0x7efdc80556b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:24.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 --2- 172.21.2.104:0/64782553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efde4067180 0x7efde40aa3e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:24.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 -- 172.21.2.104:0/64782553 >> 172.21.2.104:0/64782553 conn(0x7efde40b53e0 msgr2=0x7efde4149a60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:24.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 -- 172.21.2.104:0/64782553 shutdown_connections 2021-07-11T04:00:24.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.138+0000 7efde989f700 1 -- 172.21.2.104:0/64782553 wait complete. 2021-07-11T04:00:24.145 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 39 2021-07-11T04:00:24.149 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T04:00:24.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febcdd16700 1 Processor -- start 2021-07-11T04:00:24.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febcdd16700 1 -- start start 2021-07-11T04:00:24.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febcdd16700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7febc8141b40 0x7febc814cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febcdd16700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc80b31d0 con 0x7febc80b1170 2021-07-11T04:00:24.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febcdd16700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc80b7bf0 con 0x7febc80b7810 2021-07-11T04:00:24.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febcdd16700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc80675c0 con 0x7febc8141b40 2021-07-11T04:00:24.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febc6ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7febc80b1170 0x7febc8141430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44306/0 (socket says 172.21.2.104:44306) 2021-07-11T04:00:24.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febc6ffd700 1 -- 172.21.2.104:0/1928061209 learned_addr learned my addr 172.21.2.104:0/1928061209 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:24.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febc7fff700 1 --2- 172.21.2.104:0/1928061209 >> v2:172.21.2.104:3300/0 conn(0x7febc8141b40 0x7febc814cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febc7fff700 1 -- 172.21.2.104:0/1928061209 >> v1:172.21.2.105:6789/0 conn(0x7febc80b7810 legacy=0x7febc81445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:24.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febc7fff700 1 -- 172.21.2.104:0/1928061209 >> v1:172.21.2.104:6789/0 conn(0x7febc80b1170 legacy=0x7febc8141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febc7fff700 1 -- 172.21.2.104:0/1928061209 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80aae70 con 0x7febc8141b40 2021-07-11T04:00:24.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febc7fff700 1 --2- 172.21.2.104:0/1928061209 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febc8141b40 0x7febc814cb80 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7febc8060740 tx=0x7febc00091d0).ready entity=mon.0 client_cookie=c6d4b2e8510bcb80 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:24.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febc67fc700 1 -- 172.21.2.104:0/1928061209 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febc0009ea0 con 0x7febc8141b40 2021-07-11T04:00:24.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febc67fc700 1 -- 172.21.2.104:0/1928061209 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc000b860 con 0x7febc8141b40 2021-07-11T04:00:24.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.206+0000 7febc67fc700 1 -- 172.21.2.104:0/1928061209 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febc000b9d0 con 0x7febc8141b40 2021-07-11T04:00:24.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- 172.21.2.104:0/1928061209 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febc8141b40 msgr2=0x7febc814cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 --2- 172.21.2.104:0/1928061209 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febc8141b40 0x7febc814cb80 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7febc8060740 tx=0x7febc00091d0).stop 2021-07-11T04:00:24.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- 172.21.2.104:0/1928061209 shutdown_connections 2021-07-11T04:00:24.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 --2- 172.21.2.104:0/1928061209 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febc8141b40 0x7febc814cb80 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:24.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- 172.21.2.104:0/1928061209 >> 172.21.2.104:0/1928061209 conn(0x7febc805f6b0 msgr2=0x7febc805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:24.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- 172.21.2.104:0/1928061209 shutdown_connections 2021-07-11T04:00:24.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- 172.21.2.104:0/1928061209 wait complete. 2021-07-11T04:00:24.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 Processor -- start 2021-07-11T04:00:24.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- start start 2021-07-11T04:00:24.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7febc80b1170 0x7febc813f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febc80b7810 0x7febc80ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7febc8141b40 0x7febc80ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc8144240 con 0x7febc80b7810 2021-07-11T04:00:24.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc81447f0 con 0x7febc80b1170 2021-07-11T04:00:24.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc8144a90 con 0x7febc8141b40 2021-07-11T04:00:24.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febc80b7810 0x7febc80ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febc80b7810 0x7febc80ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47542/0 (socket says 172.21.2.104:47542) 2021-07-11T04:00:24.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc6ffd700 1 -- 172.21.2.104:0/210351060 learned_addr learned my addr 172.21.2.104:0/210351060 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:24.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc6ffd700 1 -- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7febc8141b40 msgr2=0x7febc80ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc6ffd700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7febc8141b40 0x7febc80ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:24.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc6ffd700 1 -- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7febc80b1170 msgr2=0x7febc813f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc6ffd700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7febc80b1170 0x7febc813f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:24.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc6ffd700 1 -- 172.21.2.104:0/210351060 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0009470 con 0x7febc80b7810 2021-07-11T04:00:24.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc6ffd700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febc80b7810 0x7febc80ace00 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7febc8067ca0 tx=0x7febb0007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:24.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc4ff9700 1 -- 172.21.2.104:0/210351060 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febb000ce30 con 0x7febc80b7810 2021-07-11T04:00:24.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc4ff9700 1 -- 172.21.2.104:0/210351060 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb00072e0 con 0x7febc80b7810 2021-07-11T04:00:24.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- 172.21.2.104:0/210351060 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc8144d30 con 0x7febc80b7810 2021-07-11T04:00:24.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- 172.21.2.104:0/210351060 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc80ad7e0 con 0x7febc80b7810 2021-07-11T04:00:24.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc4ff9700 1 -- 172.21.2.104:0/210351060 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7febb0005800 con 0x7febc80b7810 2021-07-11T04:00:24.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febcdd16700 1 -- 172.21.2.104:0/210351060 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc8144ec0 con 0x7febc80b7810 2021-07-11T04:00:24.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc4ff9700 1 -- 172.21.2.104:0/210351060 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7febb0010070 con 0x7febc80b7810 2021-07-11T04:00:24.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.210+0000 7febc4ff9700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febac0532d0 0x7febac055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.214+0000 7febc4ff9700 1 -- 172.21.2.104:0/210351060 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6069+0+0 (secure 0 0 0) 0x7febb006abc0 con 0x7febc80b7810 2021-07-11T04:00:24.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.214+0000 7febc77fe700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febac0532d0 0x7febac055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.214+0000 7febc77fe700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febac0532d0 0x7febac055790 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7febb8002f90 tx=0x7febb80069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:24.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.214+0000 7febc4ff9700 1 -- 172.21.2.104:0/210351060 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7febb0041f00 con 0x7febc80b7810 2021-07-11T04:00:24.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.322+0000 7febcdd16700 1 -- 172.21.2.104:0/210351060 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7febc8057350 con 0x7febc80b7810 2021-07-11T04:00:24.722 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T04:00:24.718+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T04:00:24.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.730+0000 7febc4ff9700 1 -- 172.21.2.104:0/210351060 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7febb0045520 con 0x7febc80b7810 2021-07-11T04:00:24.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 -- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febac0532d0 msgr2=0x7febac055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febac0532d0 0x7febac055790 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7febb8002f90 tx=0x7febb80069b0).stop 2021-07-11T04:00:24.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 -- 172.21.2.104:0/210351060 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febc80b7810 msgr2=0x7febc80ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febc80b7810 0x7febc80ace00 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7febc8067ca0 tx=0x7febb0007c40).stop 2021-07-11T04:00:24.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 -- 172.21.2.104:0/210351060 shutdown_connections 2021-07-11T04:00:24.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7febc8141b40 0x7febc80ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:24.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7febac0532d0 0x7febac055790 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:24.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7febc80b7810 0x7febc80ace00 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:24.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 --2- 172.21.2.104:0/210351060 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7febc80b1170 0x7febc813f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:24.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 -- 172.21.2.104:0/210351060 >> 172.21.2.104:0/210351060 conn(0x7febc805f6b0 msgr2=0x7febc8141000 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:24.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 -- 172.21.2.104:0/210351060 shutdown_connections 2021-07-11T04:00:24.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.734+0000 7febcdd16700 1 -- 172.21.2.104:0/210351060 wait complete. 2021-07-11T04:00:24.780 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T04:00:24.782 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:24.783 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T04:00:24.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.886+0000 7f96e3538700 1 Processor -- start 2021-07-11T04:00:24.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.886+0000 7f96e3538700 1 -- start start 2021-07-11T04:00:24.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.886+0000 7f96e3538700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96dc0a76b0 0x7f96dc066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.886+0000 7f96e3538700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f96dc0b1970 0x7f96dc066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.886+0000 7f96e3538700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96dc067450 con 0x7f96dc0b0060 2021-07-11T04:00:24.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.886+0000 7f96e3538700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96dc0a7a90 con 0x7f96dc0b1970 2021-07-11T04:00:24.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.886+0000 7f96e3538700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc147620 con 0x7f96dc0a76b0 2021-07-11T04:00:24.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e12d4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96dc0a76b0 0x7f96dc066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e0ad3700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f96dc0b1970 0x7f96dc066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e1ad5700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f96dc0b0060 0x7f96dc05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:43704/0 (socket says 172.21.2.104:43704) 2021-07-11T04:00:24.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e1ad5700 1 -- 172.21.2.104:0/1511342895 learned_addr learned my addr 172.21.2.104:0/1511342895 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:24.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e12d4700 1 -- 172.21.2.104:0/1511342895 >> v2:172.21.2.105:3300/0 conn(0x7f96dc0b1970 msgr2=0x7f96dc066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:24.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e12d4700 1 --2- 172.21.2.104:0/1511342895 >> v2:172.21.2.105:3300/0 conn(0x7f96dc0b1970 0x7f96dc066f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:24.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e12d4700 1 -- 172.21.2.104:0/1511342895 >> v1:172.21.2.105:6789/0 conn(0x7f96dc0b0060 legacy=0x7f96dc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e12d4700 1 -- 172.21.2.104:0/1511342895 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96dc1478c0 con 0x7f96dc0a76b0 2021-07-11T04:00:24.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e0ad3700 1 --2- 172.21.2.104:0/1511342895 >> v2:172.21.2.105:3300/0 conn(0x7f96dc0b1970 0x7f96dc066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:00:24.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e12d4700 1 --2- 172.21.2.104:0/1511342895 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96dc0a76b0 0x7f96dc066a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f96d0002f70 tx=0x7f96d0006df0).ready entity=mon.2 client_cookie=bff7a11b3508768d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:24.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96cffff700 1 -- 172.21.2.104:0/1511342895 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d000d570 con 0x7f96dc0a76b0 2021-07-11T04:00:24.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96cffff700 1 -- 172.21.2.104:0/1511342895 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d0000e90 con 0x7f96dc0a76b0 2021-07-11T04:00:24.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96cffff700 1 -- 172.21.2.104:0/1511342895 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d000d8c0 con 0x7f96dc0a76b0 2021-07-11T04:00:24.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e3538700 1 -- 172.21.2.104:0/1511342895 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96dc0a76b0 msgr2=0x7f96dc066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e3538700 1 --2- 172.21.2.104:0/1511342895 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96dc0a76b0 0x7f96dc066a70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f96d0002f70 tx=0x7f96d0006df0).stop 2021-07-11T04:00:24.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e3538700 1 -- 172.21.2.104:0/1511342895 shutdown_connections 2021-07-11T04:00:24.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e3538700 1 --2- 172.21.2.104:0/1511342895 >> v2:172.21.2.105:3300/0 conn(0x7f96dc0b1970 0x7f96dc066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:24.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e3538700 1 --2- 172.21.2.104:0/1511342895 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96dc0a76b0 0x7f96dc066a70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:24.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e3538700 1 -- 172.21.2.104:0/1511342895 >> 172.21.2.104:0/1511342895 conn(0x7f96dc0b53e0 msgr2=0x7f96dc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:24.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e3538700 1 -- 172.21.2.104:0/1511342895 shutdown_connections 2021-07-11T04:00:24.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.890+0000 7f96e3538700 1 -- 172.21.2.104:0/1511342895 wait complete. 2021-07-11T04:00:24.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e3538700 1 Processor -- start 2021-07-11T04:00:24.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e3538700 1 -- start start 2021-07-11T04:00:24.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e3538700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96dc0a76b0 0x7f96dc14d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e3538700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96dc0b0060 0x7f96dc0ab340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e3538700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f96dc0b1970 0x7f96dc0ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e3538700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc14e630 con 0x7f96dc0b0060 2021-07-11T04:00:24.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e3538700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0aaf10 con 0x7f96dc0b1970 2021-07-11T04:00:24.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e3538700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96dc0ac8a0 con 0x7f96dc0a76b0 2021-07-11T04:00:24.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e0ad3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96dc0b0060 0x7f96dc0ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e0ad3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96dc0b0060 0x7f96dc0ab340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47570/0 (socket says 172.21.2.104:47570) 2021-07-11T04:00:24.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e0ad3700 1 -- 172.21.2.104:0/2889702667 learned_addr learned my addr 172.21.2.104:0/2889702667 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:24.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e12d4700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96dc0a76b0 0x7f96dc14d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e1ad5700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f96dc0b1970 0x7f96dc0ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e0ad3700 1 -- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96dc0a76b0 msgr2=0x7f96dc14d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e0ad3700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96dc0a76b0 0x7f96dc14d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:24.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e0ad3700 1 -- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f96dc0b1970 msgr2=0x7f96dc0ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:24.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e0ad3700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f96dc0b1970 0x7f96dc0ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:24.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e0ad3700 1 -- 172.21.2.104:0/2889702667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d800b180 con 0x7f96dc0b0060 2021-07-11T04:00:24.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e1ad5700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f96dc0b1970 0x7f96dc0ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:24.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e0ad3700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96dc0b0060 0x7f96dc0ab340 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f96d800c000 tx=0x7f96d8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:24.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96ce7fc700 1 -- 172.21.2.104:0/2889702667 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d8010780 con 0x7f96dc0b0060 2021-07-11T04:00:24.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96ce7fc700 1 -- 172.21.2.104:0/2889702667 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d80108e0 con 0x7f96dc0b0060 2021-07-11T04:00:24.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e3538700 1 -- 172.21.2.104:0/2889702667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d000b180 con 0x7f96dc0b0060 2021-07-11T04:00:24.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96ce7fc700 1 -- 172.21.2.104:0/2889702667 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f96d8010c40 con 0x7f96dc0b0060 2021-07-11T04:00:24.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e3538700 1 -- 172.21.2.104:0/2889702667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96dc0ae7a0 con 0x7f96dc0b0060 2021-07-11T04:00:24.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96e3538700 1 -- 172.21.2.104:0/2889702667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96dc0acc50 con 0x7f96dc0b0060 2021-07-11T04:00:24.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96ce7fc700 1 -- 172.21.2.104:0/2889702667 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f96d801c460 con 0x7f96dc0b0060 2021-07-11T04:00:24.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96ce7fc700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96bc0531f0 0x7f96bc0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:24.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.894+0000 7f96ce7fc700 1 -- 172.21.2.104:0/2889702667 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f96d806bca0 con 0x7f96dc0b0060 2021-07-11T04:00:24.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.898+0000 7f96e12d4700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96bc0531f0 0x7f96bc0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:24.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.898+0000 7f96e12d4700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96bc0531f0 0x7f96bc0556b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f96d0019000 tx=0x7f96d0014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:24.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:24.898+0000 7f96ce7fc700 1 -- 172.21.2.104:0/2889702667 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f96d8042e20 con 0x7f96dc0b0060 2021-07-11T04:00:25.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.002+0000 7f96e3538700 1 -- 172.21.2.104:0/2889702667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96dc057350 con 0x7f96dc0b0060 2021-07-11T04:00:25.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96ce7fc700 1 -- 172.21.2.104:0/2889702667 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+2126 (secure 0 0 0) 0x7f96d8046440 con 0x7f96dc0b0060 2021-07-11T04:00:25.011 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:25.015 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":40,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/43570850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":43570850},{"type":"v1","addr":"172.21.2.105:6835","nonce":43570850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/3334136782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3334136782},{"type":"v1","addr":"172.21.2.104:6833","nonce":3334136782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:15.674524+0000","modified":"2021-07-11T04:00:24.722774+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2021-07-11T04:00:25.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 -- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96bc0531f0 msgr2=0x7f96bc0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:25.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96bc0531f0 0x7f96bc0556b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f96d0019000 tx=0x7f96d0014000).stop 2021-07-11T04:00:25.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 -- 172.21.2.104:0/2889702667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96dc0b0060 msgr2=0x7f96dc0ab340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:25.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96dc0b0060 0x7f96dc0ab340 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f96d800c000 tx=0x7f96d8006df0).stop 2021-07-11T04:00:25.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 -- 172.21.2.104:0/2889702667 shutdown_connections 2021-07-11T04:00:25.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f96dc0b1970 0x7f96dc0ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:25.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f96bc0531f0 0x7f96bc0556b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:25.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f96dc0b0060 0x7f96dc0ab340 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:25.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 --2- 172.21.2.104:0/2889702667 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f96dc0a76b0 0x7f96dc14d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:25.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 -- 172.21.2.104:0/2889702667 >> 172.21.2.104:0/2889702667 conn(0x7f96dc0b53e0 msgr2=0x7f96dc05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:25.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 -- 172.21.2.104:0/2889702667 shutdown_connections 2021-07-11T04:00:25.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.006+0000 7f96e3538700 1 -- 172.21.2.104:0/2889702667 wait complete. 2021-07-11T04:00:25.028 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 40 2021-07-11T04:00:25.029 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T04:00:25.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f34f4700 1 Processor -- start 2021-07-11T04:00:25.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f34f4700 1 -- start start 2021-07-11T04:00:25.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f34f4700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe3ec14ab80 0x7fe3ec05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f34f4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe3ec0aff10 0x7fe3ec14af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f34f4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3ec0ac9b0 0x7fe3ec0af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f34f4700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ec061360 con 0x7fe3ec0aff10 2021-07-11T04:00:25.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f34f4700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ec14b5e0 con 0x7fe3ec14ab80 2021-07-11T04:00:25.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f34f4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec14b880 con 0x7fe3ec0ac9b0 2021-07-11T04:00:25.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f0a8f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe3ec0aff10 0x7fe3ec14af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:25.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f0a8f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe3ec0aff10 0x7fe3ec14af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47578/0 (socket says 172.21.2.104:47578) 2021-07-11T04:00:25.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f0a8f700 1 -- 172.21.2.104:0/1273501304 learned_addr learned my addr 172.21.2.104:0/1273501304 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:25.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f0a8f700 1 -- 172.21.2.104:0/1273501304 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3ec0ac9b0 msgr2=0x7fe3ec0af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:25.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f0a8f700 1 --2- 172.21.2.104:0/1273501304 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3ec0ac9b0 0x7fe3ec0af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f0a8f700 1 -- 172.21.2.104:0/1273501304 >> v2:172.21.2.105:3300/0 conn(0x7fe3ec14ab80 msgr2=0x7fe3ec05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:25.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f0a8f700 1 --2- 172.21.2.104:0/1273501304 >> v2:172.21.2.105:3300/0 conn(0x7fe3ec14ab80 0x7fe3ec05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f0a8f700 1 -- 172.21.2.104:0/1273501304 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0aed20 con 0x7fe3ec0aff10 2021-07-11T04:00:25.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f0a8f700 1 --2- 172.21.2.104:0/1273501304 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3ec0aff10 0x7fe3ec14af60 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7fe3e0002f70 tx=0x7fe3e0006eb0).ready entity=mon.0 client_cookie=14dd5aefcd85706e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:25.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3dffff700 1 -- 172.21.2.104:0/1273501304 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3e000d570 con 0x7fe3ec0aff10 2021-07-11T04:00:25.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3dffff700 1 -- 172.21.2.104:0/1273501304 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e0000e90 con 0x7fe3ec0aff10 2021-07-11T04:00:25.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3dffff700 1 -- 172.21.2.104:0/1273501304 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3e000d900 con 0x7fe3ec0aff10 2021-07-11T04:00:25.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f34f4700 1 -- 172.21.2.104:0/1273501304 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3ec0aff10 msgr2=0x7fe3ec14af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:25.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.086+0000 7fe3f34f4700 1 --2- 172.21.2.104:0/1273501304 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3ec0aff10 0x7fe3ec14af60 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7fe3e0002f70 tx=0x7fe3e0006eb0).stop 2021-07-11T04:00:25.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 -- 172.21.2.104:0/1273501304 shutdown_connections 2021-07-11T04:00:25.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 --2- 172.21.2.104:0/1273501304 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3ec0ac9b0 0x7fe3ec0af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 --2- 172.21.2.104:0/1273501304 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3ec0aff10 0x7fe3ec14af60 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:25.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 --2- 172.21.2.104:0/1273501304 >> v2:172.21.2.105:3300/0 conn(0x7fe3ec14ab80 0x7fe3ec05f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 -- 172.21.2.104:0/1273501304 >> 172.21.2.104:0/1273501304 conn(0x7fe3ec0b5570 msgr2=0x7fe3ec0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:25.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 -- 172.21.2.104:0/1273501304 shutdown_connections 2021-07-11T04:00:25.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 -- 172.21.2.104:0/1273501304 wait complete. 2021-07-11T04:00:25.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 Processor -- start 2021-07-11T04:00:25.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 -- start start 2021-07-11T04:00:25.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe3ec0ac9b0 0x7fe3ec1424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3ec0aff10 0x7fe3ec066c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3ec14ab80 0x7fe3ec067180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec063580 con 0x7fe3ec0aff10 2021-07-11T04:00:25.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec144b60 con 0x7fe3ec0ac9b0 2021-07-11T04:00:25.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec144e00 con 0x7fe3ec14ab80 2021-07-11T04:00:25.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f0a8f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3ec0aff10 0x7fe3ec066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:25.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f0a8f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3ec0aff10 0x7fe3ec066c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47582/0 (socket says 172.21.2.104:47582) 2021-07-11T04:00:25.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f0a8f700 1 -- 172.21.2.104:0/3893509006 learned_addr learned my addr 172.21.2.104:0/3893509006 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:25.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f0a8f700 1 -- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3ec14ab80 msgr2=0x7fe3ec067180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:25.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f0a8f700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3ec14ab80 0x7fe3ec067180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f0a8f700 1 -- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe3ec0ac9b0 msgr2=0x7fe3ec1424d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:25.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f0a8f700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe3ec0ac9b0 0x7fe3ec1424d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f0a8f700 1 -- 172.21.2.104:0/3893509006 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e000b1e0 con 0x7fe3ec0aff10 2021-07-11T04:00:25.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f0a8f700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3ec0aff10 0x7fe3ec066c90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7fe3e0002f70 tx=0x7fe3e0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3de7fc700 1 -- 172.21.2.104:0/3893509006 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3e0021070 con 0x7fe3ec0aff10 2021-07-11T04:00:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3de7fc700 1 -- 172.21.2.104:0/3893509006 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e0000e90 con 0x7fe3ec0aff10 2021-07-11T04:00:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3de7fc700 1 -- 172.21.2.104:0/3893509006 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe3e0011bc0 con 0x7fe3ec0aff10 2021-07-11T04:00:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 -- 172.21.2.104:0/3893509006 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ec063770 con 0x7fe3ec0aff10 2021-07-11T04:00:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 -- 172.21.2.104:0/3893509006 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ec067670 con 0x7fe3ec0aff10 2021-07-11T04:00:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3f34f4700 1 -- 172.21.2.104:0/3893509006 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ec057350 con 0x7fe3ec0aff10 2021-07-11T04:00:25.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3de7fc700 1 -- 172.21.2.104:0/3893509006 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe3e000f070 con 0x7fe3ec0aff10 2021-07-11T04:00:25.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3de7fc700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe3cc053240 0x7fe3cc055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.090+0000 7fe3de7fc700 1 -- 172.21.2.104:0/3893509006 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fe3e006caa0 con 0x7fe3ec0aff10 2021-07-11T04:00:25.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.094+0000 7fe3f1290700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe3cc053240 0x7fe3cc055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:25.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.094+0000 7fe3f1290700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe3cc053240 0x7fe3cc055700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fe3e8002f70 tx=0x7fe3e8006990).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:25.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.094+0000 7fe3de7fc700 1 -- 172.21.2.104:0/3893509006 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe3e0043c80 con 0x7fe3ec0aff10 2021-07-11T04:00:25.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.202+0000 7fe3f34f4700 1 -- 172.21.2.104:0/3893509006 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe3ec044650 con 0x7fe3ec0aff10 2021-07-11T04:00:25.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3de7fc700 1 -- 172.21.2.104:0/3893509006 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe3e00472a0 con 0x7fe3ec0aff10 2021-07-11T04:00:25.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 -- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe3cc053240 msgr2=0x7fe3cc055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:25.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe3cc053240 0x7fe3cc055700 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7fe3e8002f70 tx=0x7fe3e8006990).stop 2021-07-11T04:00:25.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 -- 172.21.2.104:0/3893509006 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3ec0aff10 msgr2=0x7fe3ec066c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:25.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3ec0aff10 0x7fe3ec066c90 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7fe3e0002f70 tx=0x7fe3e0015000).stop 2021-07-11T04:00:25.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 -- 172.21.2.104:0/3893509006 shutdown_connections 2021-07-11T04:00:25.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3ec14ab80 0x7fe3ec067180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe3cc053240 0x7fe3cc055700 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:25.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe3ec0aff10 0x7fe3ec066c90 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:25.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 --2- 172.21.2.104:0/3893509006 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe3ec0ac9b0 0x7fe3ec1424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 -- 172.21.2.104:0/3893509006 >> 172.21.2.104:0/3893509006 conn(0x7fe3ec0b5570 msgr2=0x7fe3ec0acfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:25.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 -- 172.21.2.104:0/3893509006 shutdown_connections 2021-07-11T04:00:25.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.738+0000 7fe3f34f4700 1 -- 172.21.2.104:0/3893509006 wait complete. 2021-07-11T04:00:25.748 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T04:00:25.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.842+0000 7f99791cd700 1 Processor -- start 2021-07-11T04:00:25.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.842+0000 7f99791cd700 1 -- start start 2021-07-11T04:00:25.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.842+0000 7f99791cd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99741483a0 0x7f99740b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.842+0000 7f99791cd700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740b0350 con 0x7f99740b1db0 2021-07-11T04:00:25.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.842+0000 7f99791cd700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740b05f0 con 0x7f99740a70c0 2021-07-11T04:00:25.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.842+0000 7f99791cd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740a90c0 con 0x7f99741483a0 2021-07-11T04:00:25.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9972d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f99740b1db0 0x7f99740b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44360/0 (socket says 172.21.2.104:44360) 2021-07-11T04:00:25.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9972d9d700 1 -- 172.21.2.104:0/1900594688 learned_addr learned my addr 172.21.2.104:0/1900594688 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:25.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 682194640 0 0) 0x7f99740b0350 con 0x7f99740b1db0 2021-07-11T04:00:25.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9954003260 con 0x7f99740b1db0 2021-07-11T04:00:25.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997259c700 1 --2- 172.21.2.104:0/1900594688 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99741483a0 0x7f99740b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:25.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3653312019 0 0) 0x7f9954003260 con 0x7f99740b1db0 2021-07-11T04:00:25.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99741483a0 msgr2=0x7f99740b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:25.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 --2- 172.21.2.104:0/1900594688 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99741483a0 0x7f99740b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:25.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 >> v1:172.21.2.105:6789/0 conn(0x7f99740a70c0 legacy=0x7f9974144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:25.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740b0890 con 0x7f99740b1db0 2021-07-11T04:00:25.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f9964001660 con 0x7f99740b1db0 2021-07-11T04:00:25.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 >> v1:172.21.2.104:6789/0 conn(0x7f99740b1db0 legacy=0x7f99740b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:25.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 --2- 172.21.2.104:0/1900594688 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99741483a0 0x7f9954003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 --2- 172.21.2.104:0/1900594688 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9954003ef0 0x7f9954004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 --2- 172.21.2.104:0/1900594688 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9954004bf0 0x7f995400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540052c0 con 0x7f9954003ef0 2021-07-11T04:00:25.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995400bc40 con 0x7f9954004bf0 2021-07-11T04:00:25.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995400bd80 con 0x7f99741483a0 2021-07-11T04:00:25.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997259c700 1 --2- 172.21.2.104:0/1900594688 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9954003ef0 0x7f9954004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:25.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997259c700 1 -- 172.21.2.104:0/1900594688 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99741483a0 msgr2=0x7f9954003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:25.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997259c700 1 --2- 172.21.2.104:0/1900594688 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99741483a0 0x7f9954003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997259c700 1 -- 172.21.2.104:0/1900594688 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9954004bf0 msgr2=0x7f995400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:25.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997259c700 1 --2- 172.21.2.104:0/1900594688 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9954004bf0 0x7f995400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997259c700 1 -- 172.21.2.104:0/1900594688 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9954003260 con 0x7f9954003ef0 2021-07-11T04:00:25.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997259c700 1 --2- 172.21.2.104:0/1900594688 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9954003ef0 0x7f9954004500 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f995c007000 tx=0x7f995c008040).ready entity=mon.0 client_cookie=a2bfe8c3bf980724 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:25.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f995c012470 con 0x7f9954003ef0 2021-07-11T04:00:25.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c00c030 con 0x7f9954003ef0 2021-07-11T04:00:25.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f9971d9b700 1 -- 172.21.2.104:0/1900594688 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f995c0127c0 con 0x7f9954003ef0 2021-07-11T04:00:25.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 -- 172.21.2.104:0/1900594688 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9954003ef0 msgr2=0x7f9954004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:25.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 --2- 172.21.2.104:0/1900594688 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9954003ef0 0x7f9954004500 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f995c007000 tx=0x7f995c008040).stop 2021-07-11T04:00:25.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997359e700 1 -- 172.21.2.104:0/1900594688 reap_dead start 2021-07-11T04:00:25.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 -- 172.21.2.104:0/1900594688 shutdown_connections 2021-07-11T04:00:25.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 -- 172.21.2.104:0/1900594688 >> 172.21.2.104:0/1900594688 conn(0x7f997405f6b0 msgr2=0x7f997405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:25.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 -- 172.21.2.104:0/1900594688 shutdown_connections 2021-07-11T04:00:25.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 -- 172.21.2.104:0/1900594688 wait complete. 2021-07-11T04:00:25.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 Processor -- start 2021-07-11T04:00:25.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 -- start start 2021-07-11T04:00:25.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9974142520 0x7f997413f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f997413fcf0 0x7f9974148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9974149180 0x7f9974149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740adc70 con 0x7f9974149180 2021-07-11T04:00:25.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9974142900 con 0x7f9974142520 2021-07-11T04:00:25.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740aeaa0 con 0x7f997413fcf0 2021-07-11T04:00:25.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997359e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9974149180 0x7f9974149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:25.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997359e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9974149180 0x7f9974149600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47606/0 (socket says 172.21.2.104:47606) 2021-07-11T04:00:25.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997359e700 1 -- 172.21.2.104:0/546166981 learned_addr learned my addr 172.21.2.104:0/546166981 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:25.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997359e700 1 -- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f997413fcf0 msgr2=0x7f9974148c90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:25.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997359e700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f997413fcf0 0x7f9974148c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997359e700 1 -- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9974142520 msgr2=0x7f997413f800 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:25.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997359e700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9974142520 0x7f997413f800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:25.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997359e700 1 -- 172.21.2.104:0/546166981 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740ae4e0 con 0x7f9974149180 2021-07-11T04:00:25.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f997359e700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9974149180 0x7f9974149600 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f9964001110 tx=0x7f996c007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:25.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f996bfff700 1 -- 172.21.2.104:0/546166981 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996c00ceb0 con 0x7f9974149180 2021-07-11T04:00:25.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f996bfff700 1 -- 172.21.2.104:0/546166981 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c005a90 con 0x7f9974149180 2021-07-11T04:00:25.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 -- 172.21.2.104:0/546166981 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99740adea0 con 0x7f9974149180 2021-07-11T04:00:25.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f996bfff700 1 -- 172.21.2.104:0/546166981 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f996c005e30 con 0x7f9974149180 2021-07-11T04:00:25.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.846+0000 7f99791cd700 1 -- 172.21.2.104:0/546166981 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99740ae780 con 0x7f9974149180 2021-07-11T04:00:25.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.850+0000 7f99791cd700 1 -- 172.21.2.104:0/546166981 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99740ae220 con 0x7f9974149180 2021-07-11T04:00:25.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.850+0000 7f996bfff700 1 -- 172.21.2.104:0/546166981 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f996c01a060 con 0x7f9974149180 2021-07-11T04:00:25.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.850+0000 7f996bfff700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9958053330 0x7f99580557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:25.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.850+0000 7f996bfff700 1 -- 172.21.2.104:0/546166981 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f996c06ade0 con 0x7f9974149180 2021-07-11T04:00:25.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.850+0000 7f9972d9d700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9958053330 0x7f99580557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:25.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.850+0000 7f9972d9d700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9958053330 0x7f99580557f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f9974061440 tx=0x7f99740b0780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:25.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.850+0000 7f996bfff700 1 -- 172.21.2.104:0/546166981 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f996c041fc0 con 0x7f9974149180 2021-07-11T04:00:25.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:25.934+0000 7f99791cd700 1 -- 172.21.2.104:0/546166981 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9974057350 con 0x7f9974149180 2021-07-11T04:00:26.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.742+0000 7f996bfff700 1 -- 172.21.2.104:0/546166981 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 185+0+0 (secure 0 0 0) 0x7f996c018070 con 0x7f9974149180 2021-07-11T04:00:26.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 -- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9958053330 msgr2=0x7f99580557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:26.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9958053330 0x7f99580557f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f9974061440 tx=0x7f99740b0780).stop 2021-07-11T04:00:26.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 -- 172.21.2.104:0/546166981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9974149180 msgr2=0x7f9974149600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:26.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9974149180 0x7f9974149600 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f9964001110 tx=0x7f996c007dc0).stop 2021-07-11T04:00:26.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 -- 172.21.2.104:0/546166981 shutdown_connections 2021-07-11T04:00:26.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9958053330 0x7f99580557f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:26.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9974149180 0x7f9974149600 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:26.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f997413fcf0 0x7f9974148c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:26.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 --2- 172.21.2.104:0/546166981 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9974142520 0x7f997413f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:26.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 -- 172.21.2.104:0/546166981 >> 172.21.2.104:0/546166981 conn(0x7f997405f6b0 msgr2=0x7f99740af760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:26.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 -- 172.21.2.104:0/546166981 shutdown_connections 2021-07-11T04:00:26.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.746+0000 7f99791cd700 1 -- 172.21.2.104:0/546166981 wait complete. 2021-07-11T04:00:26.751 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T04:00:26.754 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T04:00:26.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 Processor -- start 2021-07-11T04:00:26.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- start start 2021-07-11T04:00:26.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3d080b6900 0x7f3d080acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:26.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3d080aa5d0 0x7f3d0813c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:26.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d080b6ce0 con 0x7f3d08060910 2021-07-11T04:00:26.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d080611e0 con 0x7f3d080b6900 2021-07-11T04:00:26.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d08061480 con 0x7f3d080aa5d0 2021-07-11T04:00:26.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d077fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f3d08060910 0x7f3d080a9ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44382/0 (socket says 172.21.2.104:44382) 2021-07-11T04:00:26.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d077fe700 1 -- 172.21.2.104:0/3697102713 learned_addr learned my addr 172.21.2.104:0/3697102713 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:26.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d07fff700 1 --2- 172.21.2.104:0/3697102713 >> v2:172.21.2.104:3300/0 conn(0x7f3d080b6900 0x7f3d080acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:26.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d07fff700 1 -- 172.21.2.104:0/3697102713 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3d080aa5d0 msgr2=0x7f3d0813c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:26.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d07fff700 1 --2- 172.21.2.104:0/3697102713 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3d080aa5d0 0x7f3d0813c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:26.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d07fff700 1 -- 172.21.2.104:0/3697102713 >> v1:172.21.2.104:6789/0 conn(0x7f3d08060910 legacy=0x7f3d080a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:26.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d07fff700 1 -- 172.21.2.104:0/3697102713 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d08061720 con 0x7f3d080b6900 2021-07-11T04:00:26.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d07fff700 1 --2- 172.21.2.104:0/3697102713 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3d080b6900 0x7f3d080acd50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f3cf0001110 tx=0x7f3cf8006e80).ready entity=mon.0 client_cookie=6bc4d4e55fd8b1a1 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:26.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d06ffd700 1 -- 172.21.2.104:0/3697102713 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cf8010810 con 0x7f3d080b6900 2021-07-11T04:00:26.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d06ffd700 1 -- 172.21.2.104:0/3697102713 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf8000e90 con 0x7f3d080b6900 2021-07-11T04:00:26.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d06ffd700 1 -- 172.21.2.104:0/3697102713 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3cf8010b60 con 0x7f3d080b6900 2021-07-11T04:00:26.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/3697102713 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3d080b6900 msgr2=0x7f3d080acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:26.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 --2- 172.21.2.104:0/3697102713 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3d080b6900 0x7f3d080acd50 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7f3cf0001110 tx=0x7f3cf8006e80).stop 2021-07-11T04:00:26.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/3697102713 shutdown_connections 2021-07-11T04:00:26.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 --2- 172.21.2.104:0/3697102713 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3d080aa5d0 0x7f3d0813c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:26.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 --2- 172.21.2.104:0/3697102713 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3d080b6900 0x7f3d080acd50 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:26.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/3697102713 >> 172.21.2.104:0/3697102713 conn(0x7f3d080b3480 msgr2=0x7f3d080b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:26.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/3697102713 shutdown_connections 2021-07-11T04:00:26.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/3697102713 wait complete. 2021-07-11T04:00:26.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 Processor -- start 2021-07-11T04:00:26.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- start start 2021-07-11T04:00:26.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3d08060910 0x7f3d08139a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:26.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3d080aa5d0 0x7f3d0813eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:26.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3d080b6900 0x7f3d0813eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:26.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0813d930 con 0x7f3d080b6900 2021-07-11T04:00:26.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0813db40 con 0x7f3d080aa5d0 2021-07-11T04:00:26.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0e6e9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0813bb80 con 0x7f3d08060910 2021-07-11T04:00:26.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0cc86700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3d080b6900 0x7f3d0813eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:26.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0cc86700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3d080b6900 0x7f3d0813eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47620/0 (socket says 172.21.2.104:47620) 2021-07-11T04:00:26.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0cc86700 1 -- 172.21.2.104:0/666614636 learned_addr learned my addr 172.21.2.104:0/666614636 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:26.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0cc86700 1 -- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3d08060910 msgr2=0x7f3d08139a60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:26.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0cc86700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3d08060910 0x7f3d08139a60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:26.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0cc86700 1 -- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3d080aa5d0 msgr2=0x7f3d0813eb00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:26.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0cc86700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3d080aa5d0 0x7f3d0813eb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:26.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.806+0000 7f3d0cc86700 1 -- 172.21.2.104:0/666614636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf800c310 con 0x7f3d080b6900 2021-07-11T04:00:26.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d0cc86700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3d080b6900 0x7f3d0813eff0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f3d000032f0 tx=0x7f3d0000b4c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:26.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d057fa700 1 -- 172.21.2.104:0/666614636 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d0000ee00 con 0x7f3d080b6900 2021-07-11T04:00:26.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d057fa700 1 -- 172.21.2.104:0/666614636 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d00017420 con 0x7f3d080b6900 2021-07-11T04:00:26.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/666614636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0813d690 con 0x7f3d080b6900 2021-07-11T04:00:26.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d057fa700 1 -- 172.21.2.104:0/666614636 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3d00017780 con 0x7f3d080b6900 2021-07-11T04:00:26.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/666614636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0813a7a0 con 0x7f3d080b6900 2021-07-11T04:00:26.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/666614636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d08057350 con 0x7f3d080b6900 2021-07-11T04:00:26.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d057fa700 1 -- 172.21.2.104:0/666614636 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3d000178e0 con 0x7f3d080b6900 2021-07-11T04:00:26.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d057fa700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cec053270 0x7f3cec055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:26.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d057fa700 1 -- 172.21.2.104:0/666614636 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f3d0006dec0 con 0x7f3d080b6900 2021-07-11T04:00:26.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d07fff700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cec053270 0x7f3cec055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:26.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d07fff700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cec053270 0x7f3cec055730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f3cf0001110 tx=0x7f3d0813f6e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:26.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.810+0000 7f3d057fa700 1 -- 172.21.2.104:0/666614636 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3d00045410 con 0x7f3d080b6900 2021-07-11T04:00:26.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:26.894+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/666614636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3d08044650 con 0x7f3d080b6900 2021-07-11T04:00:27.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d057fa700 1 -- 172.21.2.104:0/666614636 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 173+0+0 (secure 0 0 0) 0x7f3d00048a30 con 0x7f3d080b6900 2021-07-11T04:00:27.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cec053270 msgr2=0x7f3cec055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:27.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cec053270 0x7f3cec055730 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f3cf0001110 tx=0x7f3d0813f6e0).stop 2021-07-11T04:00:27.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/666614636 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3d080b6900 msgr2=0x7f3d0813eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:27.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3d080b6900 0x7f3d0813eff0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f3d000032f0 tx=0x7f3d0000b4c0).stop 2021-07-11T04:00:27.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/666614636 shutdown_connections 2021-07-11T04:00:27.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cec053270 0x7f3cec055730 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:27.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3d080b6900 0x7f3d0813eff0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:27.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3d080aa5d0 0x7f3d0813eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 --2- 172.21.2.104:0/666614636 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3d08060910 0x7f3d08139a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/666614636 >> 172.21.2.104:0/666614636 conn(0x7f3d080b3480 msgr2=0x7f3d08136310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:27.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/666614636 shutdown_connections 2021-07-11T04:00:27.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.758+0000 7f3d0e6e9700 1 -- 172.21.2.104:0/666614636 wait complete. 2021-07-11T04:00:27.765 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T04:00:27.768 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-07-11T04:00:27.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.818+0000 7f34fbaf7700 1 Processor -- start 2021-07-11T04:00:27.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.818+0000 7f34fbaf7700 1 -- start start 2021-07-11T04:00:27.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.818+0000 7f34fbaf7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f34f4060100 0x7f34f4064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.818+0000 7f34fbaf7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f34f4144440 0x7f34f4065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.818+0000 7f34fbaf7700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f40604e0 con 0x7f34f40b38d0 2021-07-11T04:00:27.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.818+0000 7f34fbaf7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f40656f0 con 0x7f34f4060100 2021-07-11T04:00:27.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.818+0000 7f34fbaf7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f4065c80 con 0x7f34f4144440 2021-07-11T04:00:27.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fa094700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f34f40b38d0 0x7f34f4144970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44394/0 (socket says 172.21.2.104:44394) 2021-07-11T04:00:27.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fa094700 1 -- 172.21.2.104:0/2768028314 learned_addr learned my addr 172.21.2.104:0/2768028314 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:27.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34f9893700 1 --2- 172.21.2.104:0/2768028314 >> v2:172.21.2.104:3300/0 conn(0x7f34f4060100 0x7f34f4064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:27.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34f9893700 1 -- 172.21.2.104:0/2768028314 >> v2:172.21.2.105:3300/0 conn(0x7f34f4144440 msgr2=0x7f34f4065200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:27.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34f9893700 1 --2- 172.21.2.104:0/2768028314 >> v2:172.21.2.105:3300/0 conn(0x7f34f4144440 0x7f34f4065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34f9893700 1 -- 172.21.2.104:0/2768028314 >> v1:172.21.2.104:6789/0 conn(0x7f34f40b38d0 legacy=0x7f34f4144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:27.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34f9893700 1 -- 172.21.2.104:0/2768028314 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f40b23b0 con 0x7f34f4060100 2021-07-11T04:00:27.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34f9893700 1 --2- 172.21.2.104:0/2768028314 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34f4060100 0x7f34f4064d10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f34e8002f70 tx=0x7f34e8006df0).ready entity=mon.0 client_cookie=2a3bb9dd45607f80 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:27.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34f8891700 1 -- 172.21.2.104:0/2768028314 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e800d5c0 con 0x7f34f4060100 2021-07-11T04:00:27.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34f8891700 1 -- 172.21.2.104:0/2768028314 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e8000e90 con 0x7f34f4060100 2021-07-11T04:00:27.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34f8891700 1 -- 172.21.2.104:0/2768028314 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34e800d910 con 0x7f34f4060100 2021-07-11T04:00:27.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- 172.21.2.104:0/2768028314 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34f4060100 msgr2=0x7f34f4064d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:27.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 --2- 172.21.2.104:0/2768028314 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34f4060100 0x7f34f4064d10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f34e8002f70 tx=0x7f34e8006df0).stop 2021-07-11T04:00:27.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- 172.21.2.104:0/2768028314 shutdown_connections 2021-07-11T04:00:27.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 --2- 172.21.2.104:0/2768028314 >> v2:172.21.2.105:3300/0 conn(0x7f34f4144440 0x7f34f4065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 --2- 172.21.2.104:0/2768028314 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34f4060100 0x7f34f4064d10 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:27.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- 172.21.2.104:0/2768028314 >> 172.21.2.104:0/2768028314 conn(0x7f34f405cf60 msgr2=0x7f34f405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:27.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- 172.21.2.104:0/2768028314 shutdown_connections 2021-07-11T04:00:27.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- 172.21.2.104:0/2768028314 wait complete. 2021-07-11T04:00:27.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 Processor -- start 2021-07-11T04:00:27.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- start start 2021-07-11T04:00:27.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34f4060100 0x7f34f40ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f34f40b38d0 0x7f34f40ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34f40621b0 0x7f34f40aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f4156920 con 0x7f34f40621b0 2021-07-11T04:00:27.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f4062640 con 0x7f34f40b38d0 2021-07-11T04:00:27.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f40ac2a0 con 0x7f34f4060100 2021-07-11T04:00:27.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fa094700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34f40621b0 0x7f34f40aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:27.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fa094700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34f40621b0 0x7f34f40aedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47636/0 (socket says 172.21.2.104:47636) 2021-07-11T04:00:27.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fa094700 1 -- 172.21.2.104:0/4159538242 learned_addr learned my addr 172.21.2.104:0/4159538242 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:27.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fa094700 1 -- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34f4060100 msgr2=0x7f34f40ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:27.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fa094700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34f4060100 0x7f34f40ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fa094700 1 -- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f34f40b38d0 msgr2=0x7f34f40ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:27.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fa094700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f34f40b38d0 0x7f34f40ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fa094700 1 -- 172.21.2.104:0/4159538242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e800b180 con 0x7f34f40621b0 2021-07-11T04:00:27.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fa094700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34f40621b0 0x7f34f40aedf0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f34f405d460 tx=0x7f34ec007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:27.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34e6ffd700 1 -- 172.21.2.104:0/4159538242 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34ec005ba0 con 0x7f34f40621b0 2021-07-11T04:00:27.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34e6ffd700 1 -- 172.21.2.104:0/4159538242 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec005d00 con 0x7f34f40621b0 2021-07-11T04:00:27.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- 172.21.2.104:0/4159538242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f40ac540 con 0x7f34f40621b0 2021-07-11T04:00:27.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34e6ffd700 1 -- 172.21.2.104:0/4159538242 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f34ec0072c0 con 0x7f34f40621b0 2021-07-11T04:00:27.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- 172.21.2.104:0/4159538242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f40af2e0 con 0x7f34f40621b0 2021-07-11T04:00:27.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.822+0000 7f34fbaf7700 1 -- 172.21.2.104:0/4159538242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f40ac0e0 con 0x7f34f40621b0 2021-07-11T04:00:27.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.826+0000 7f34e6ffd700 1 -- 172.21.2.104:0/4159538242 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f34ec01a060 con 0x7f34f40621b0 2021-07-11T04:00:27.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.826+0000 7f34e6ffd700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34d4053300 0x7f34d40557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.826+0000 7f34e6ffd700 1 -- 172.21.2.104:0/4159538242 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f34ec06acc0 con 0x7f34f40621b0 2021-07-11T04:00:27.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.826+0000 7f34f9893700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34d4053300 0x7f34d40557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:27.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.826+0000 7f34f9893700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34d4053300 0x7f34d40557c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f34e8002f70 tx=0x7f34e8014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:27.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.826+0000 7f34e6ffd700 1 -- 172.21.2.104:0/4159538242 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f34ec0422e0 con 0x7f34f40621b0 2021-07-11T04:00:27.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.910+0000 7f34fbaf7700 1 -- 172.21.2.104:0/4159538242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f34f40af8a0 con 0x7f34f40621b0 2021-07-11T04:00:27.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34e6ffd700 1 -- 172.21.2.104:0/4159538242 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f34ec018070 con 0x7f34f40621b0 2021-07-11T04:00:27.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 -- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34d4053300 msgr2=0x7f34d40557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:27.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34d4053300 0x7f34d40557c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f34e8002f70 tx=0x7f34e8014000).stop 2021-07-11T04:00:27.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 -- 172.21.2.104:0/4159538242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34f40621b0 msgr2=0x7f34f40aedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:27.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34f40621b0 0x7f34f40aedf0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f34f405d460 tx=0x7f34ec007f10).stop 2021-07-11T04:00:27.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 -- 172.21.2.104:0/4159538242 shutdown_connections 2021-07-11T04:00:27.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f34d4053300 0x7f34d40557c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:27.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f34f40621b0 0x7f34f40aedf0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:27.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f34f40b38d0 0x7f34f40ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 --2- 172.21.2.104:0/4159538242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f34f4060100 0x7f34f40ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 -- 172.21.2.104:0/4159538242 >> 172.21.2.104:0/4159538242 conn(0x7f34f405cf60 msgr2=0x7f34f4140ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:27.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 -- 172.21.2.104:0/4159538242 shutdown_connections 2021-07-11T04:00:27.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.918+0000 7f34fbaf7700 1 -- 172.21.2.104:0/4159538242 wait complete. 2021-07-11T04:00:27.928 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode' 2021-07-11T04:00:27.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.974+0000 7f7a7ae64700 1 Processor -- start 2021-07-11T04:00:27.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- start start 2021-07-11T04:00:27.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7a740b7810 0x7f7a741445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f7a740b1170 0x7f7a74144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a740b31d0 con 0x7f7a74064870 2021-07-11T04:00:27.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a740b7bf0 con 0x7f7a740b7810 2021-07-11T04:00:27.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a740675c0 con 0x7f7a740b1170 2021-07-11T04:00:27.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a79401700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f7a74064870 0x7f7a74064d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44410/0 (socket says 172.21.2.104:44410) 2021-07-11T04:00:27.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a79401700 1 -- 172.21.2.104:0/3440142679 learned_addr learned my addr 172.21.2.104:0/3440142679 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:27.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 --2- 172.21.2.104:0/3440142679 >> v2:172.21.2.104:3300/0 conn(0x7f7a740b7810 0x7f7a741445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:27.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a737fe700 1 -- 172.21.2.104:0/3440142679 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053580883 0 0) 0x7f7a740b31d0 con 0x7f7a74064870 2021-07-11T04:00:27.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 -- 172.21.2.104:0/3440142679 >> v2:172.21.2.105:3300/0 conn(0x7f7a740b1170 msgr2=0x7f7a74144a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:27.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 --2- 172.21.2.104:0/3440142679 >> v2:172.21.2.105:3300/0 conn(0x7f7a740b1170 0x7f7a74144a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 -- 172.21.2.104:0/3440142679 >> v1:172.21.2.104:6789/0 conn(0x7f7a74064870 legacy=0x7f7a74064d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:27.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 -- 172.21.2.104:0/3440142679 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a740aae70 con 0x7f7a740b7810 2021-07-11T04:00:27.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 --2- 172.21.2.104:0/3440142679 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7a740b7810 0x7f7a741445a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f7a64002f70 tx=0x7f7a64006df0).ready entity=mon.0 client_cookie=55cb036faf72acfa server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:27.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a737fe700 1 -- 172.21.2.104:0/3440142679 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a6400d5c0 con 0x7f7a740b7810 2021-07-11T04:00:27.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a737fe700 1 -- 172.21.2.104:0/3440142679 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a64000e90 con 0x7f7a740b7810 2021-07-11T04:00:27.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a737fe700 1 -- 172.21.2.104:0/3440142679 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a6400d910 con 0x7f7a740b7810 2021-07-11T04:00:27.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- 172.21.2.104:0/3440142679 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7a740b7810 msgr2=0x7f7a741445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:27.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 --2- 172.21.2.104:0/3440142679 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7a740b7810 0x7f7a741445a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f7a64002f70 tx=0x7f7a64006df0).stop 2021-07-11T04:00:27.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- 172.21.2.104:0/3440142679 shutdown_connections 2021-07-11T04:00:27.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 --2- 172.21.2.104:0/3440142679 >> v2:172.21.2.105:3300/0 conn(0x7f7a740b1170 0x7f7a74144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 --2- 172.21.2.104:0/3440142679 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7a740b7810 0x7f7a741445a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:27.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- 172.21.2.104:0/3440142679 >> 172.21.2.104:0/3440142679 conn(0x7f7a7405f6b0 msgr2=0x7f7a7405fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:27.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- 172.21.2.104:0/3440142679 shutdown_connections 2021-07-11T04:00:27.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- 172.21.2.104:0/3440142679 wait complete. 2021-07-11T04:00:27.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 Processor -- start 2021-07-11T04:00:27.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- start start 2021-07-11T04:00:27.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7a74064870 0x7f7a74061310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7a740b1170 0x7f7a74156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7a740b7810 0x7f7a7415a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a74138790 con 0x7f7a74064870 2021-07-11T04:00:27.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a74159b00 con 0x7f7a740b1170 2021-07-11T04:00:27.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a740a70c0 con 0x7f7a740b7810 2021-07-11T04:00:27.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7a74064870 0x7f7a74061310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:27.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7a74064870 0x7f7a74061310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47644/0 (socket says 172.21.2.104:47644) 2021-07-11T04:00:27.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 -- 172.21.2.104:0/2838981 learned_addr learned my addr 172.21.2.104:0/2838981 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:27.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 -- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7a740b7810 msgr2=0x7f7a7415a1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:27.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7a740b7810 0x7f7a7415a1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 -- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7a740b1170 msgr2=0x7f7a74156490 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:27.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7a740b1170 0x7f7a74156490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:27.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 -- 172.21.2.104:0/2838981 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6400b180 con 0x7f7a74064870 2021-07-11T04:00:27.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a78c00700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7a74064870 0x7f7a74061310 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f7a64002f70 tx=0x7f7a64014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:27.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a71ffb700 1 -- 172.21.2.104:0/2838981 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a6401f440 con 0x7f7a74064870 2021-07-11T04:00:27.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a71ffb700 1 -- 172.21.2.104:0/2838981 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6401f5a0 con 0x7f7a74064870 2021-07-11T04:00:27.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- 172.21.2.104:0/2838981 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a74159d30 con 0x7f7a74064870 2021-07-11T04:00:27.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a71ffb700 1 -- 172.21.2.104:0/2838981 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7a6401f900 con 0x7f7a74064870 2021-07-11T04:00:27.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.978+0000 7f7a7ae64700 1 -- 172.21.2.104:0/2838981 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a740a7360 con 0x7f7a74064870 2021-07-11T04:00:27.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.982+0000 7f7a7ae64700 1 -- 172.21.2.104:0/2838981 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a74057350 con 0x7f7a74064870 2021-07-11T04:00:27.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.982+0000 7f7a71ffb700 1 -- 172.21.2.104:0/2838981 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f7a64025020 con 0x7f7a74064870 2021-07-11T04:00:27.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.982+0000 7f7a71ffb700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7a58053160 0x7f7a58055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:27.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.982+0000 7f7a71ffb700 1 -- 172.21.2.104:0/2838981 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7a6406c580 con 0x7f7a74064870 2021-07-11T04:00:27.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.982+0000 7f7a73fff700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7a58053160 0x7f7a58055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:27.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.982+0000 7f7a73fff700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7a58053160 0x7f7a58055620 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f7a5c002fb0 tx=0x7f7a5c006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:27.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:27.982+0000 7f7a71ffb700 1 -- 172.21.2.104:0/2838981 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7a64043a90 con 0x7f7a74064870 2021-07-11T04:00:28.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.094+0000 7f7a7ae64700 1 -- 172.21.2.104:0/2838981 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]} v 0) v1 -- 0x7f7a740698e0 con 0x7f7a74064870 2021-07-11T04:00:28.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.766+0000 7f7a71ffb700 1 -- 172.21.2.104:0/2838981 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode"]}]=0 v296) v1 ==== 126+0+0 (secure 0 0 0) 0x7f7a640470b0 con 0x7f7a74064870 2021-07-11T04:00:28.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 -- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7a58053160 msgr2=0x7f7a58055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:28.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7a58053160 0x7f7a58055620 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f7a5c002fb0 tx=0x7f7a5c006930).stop 2021-07-11T04:00:28.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 -- 172.21.2.104:0/2838981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7a74064870 msgr2=0x7f7a74061310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:28.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7a74064870 0x7f7a74061310 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f7a64002f70 tx=0x7f7a64014000).stop 2021-07-11T04:00:28.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 -- 172.21.2.104:0/2838981 shutdown_connections 2021-07-11T04:00:28.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7a740b7810 0x7f7a7415a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:28.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7a740b1170 0x7f7a74156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:28.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7a58053160 0x7f7a58055620 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:28.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 --2- 172.21.2.104:0/2838981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7a74064870 0x7f7a74061310 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:28.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 -- 172.21.2.104:0/2838981 >> 172.21.2.104:0/2838981 conn(0x7f7a7405f6b0 msgr2=0x7f7a7405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:28.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 -- 172.21.2.104:0/2838981 shutdown_connections 2021-07-11T04:00:28.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.770+0000 7f7a7ae64700 1 -- 172.21.2.104:0/2838981 wait complete. 2021-07-11T04:00:28.781 INFO:tasks.cephfs_test_runner:test_dump_inode (tasks.cephfs.test_misc.TestMisc) ... ok 2021-07-11T04:00:28.781 INFO:tasks.cephfs_test_runner:Starting test: test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:00:28.782 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2021-07-11T04:00:28.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.830+0000 7fe647e67700 1 Processor -- start 2021-07-11T04:00:28.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.830+0000 7fe647e67700 1 -- start start 2021-07-11T04:00:28.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.830+0000 7fe647e67700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe6401483a0 0x7fe6400b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:28.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.830+0000 7fe647e67700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe6400a70c0 0x7fe640144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:28.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.830+0000 7fe647e67700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6400b0350 con 0x7fe6400b1db0 2021-07-11T04:00:28.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.830+0000 7fe647e67700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6400a90c0 con 0x7fe6401483a0 2021-07-11T04:00:28.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.830+0000 7fe647e67700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6400b05f0 con 0x7fe6400a70c0 2021-07-11T04:00:28.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.830+0000 7fe645402700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe6401483a0 0x7fe6400b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:28.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.830+0000 7fe645402700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe6401483a0 0x7fe6400b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47654/0 (socket says 172.21.2.104:47654) 2021-07-11T04:00:28.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.830+0000 7fe645402700 1 -- 172.21.2.104:0/2528257141 learned_addr learned my addr 172.21.2.104:0/2528257141 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:28.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe645402700 1 -- 172.21.2.104:0/2528257141 >> v2:172.21.2.105:3300/0 conn(0x7fe6400a70c0 msgr2=0x7fe640144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:28.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe645402700 1 --2- 172.21.2.104:0/2528257141 >> v2:172.21.2.105:3300/0 conn(0x7fe6400a70c0 0x7fe640144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:28.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe645402700 1 -- 172.21.2.104:0/2528257141 >> v1:172.21.2.105:6789/0 conn(0x7fe6400b1db0 legacy=0x7fe6400b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:28.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe645402700 1 -- 172.21.2.104:0/2528257141 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6400b0890 con 0x7fe6401483a0 2021-07-11T04:00:28.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe645402700 1 --2- 172.21.2.104:0/2528257141 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe6401483a0 0x7fe6400b0fd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7fe6400ace60 tx=0x7fe63c006eb0).ready entity=mon.0 client_cookie=cae461cd8416a0b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:28.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe644c01700 1 -- 172.21.2.104:0/2528257141 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe63c00d5c0 con 0x7fe6401483a0 2021-07-11T04:00:28.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe644c01700 1 -- 172.21.2.104:0/2528257141 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63c000e90 con 0x7fe6401483a0 2021-07-11T04:00:28.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe644c01700 1 -- 172.21.2.104:0/2528257141 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe63c00d950 con 0x7fe6401483a0 2021-07-11T04:00:28.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 -- 172.21.2.104:0/2528257141 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe6401483a0 msgr2=0x7fe6400b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:28.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 --2- 172.21.2.104:0/2528257141 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe6401483a0 0x7fe6400b0fd0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7fe6400ace60 tx=0x7fe63c006eb0).stop 2021-07-11T04:00:28.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 -- 172.21.2.104:0/2528257141 shutdown_connections 2021-07-11T04:00:28.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 --2- 172.21.2.104:0/2528257141 >> v2:172.21.2.105:3300/0 conn(0x7fe6400a70c0 0x7fe640144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:28.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 --2- 172.21.2.104:0/2528257141 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe6401483a0 0x7fe6400b0fd0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:28.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 -- 172.21.2.104:0/2528257141 >> 172.21.2.104:0/2528257141 conn(0x7fe64005f6b0 msgr2=0x7fe64005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:28.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 -- 172.21.2.104:0/2528257141 shutdown_connections 2021-07-11T04:00:28.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 -- 172.21.2.104:0/2528257141 wait complete. 2021-07-11T04:00:28.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 Processor -- start 2021-07-11T04:00:28.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 -- start start 2021-07-11T04:00:28.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6400a70c0 0x7fe640144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:28.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe6400b1db0 0x7fe640159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:28.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe6401483a0 0x7fe64015a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:28.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400acea0 con 0x7fe6401483a0 2021-07-11T04:00:28.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400afbc0 con 0x7fe6400b1db0 2021-07-11T04:00:28.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6401473c0 con 0x7fe6400a70c0 2021-07-11T04:00:28.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe646404700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe6401483a0 0x7fe64015a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:28.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe646404700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe6401483a0 0x7fe64015a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47660/0 (socket says 172.21.2.104:47660) 2021-07-11T04:00:28.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe646404700 1 -- 172.21.2.104:0/131308601 learned_addr learned my addr 172.21.2.104:0/131308601 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:28.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe645c03700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6400a70c0 0x7fe640144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:28.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe646404700 1 -- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6400a70c0 msgr2=0x7fe640144990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:28.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe646404700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6400a70c0 0x7fe640144990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:28.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe645402700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe6400b1db0 0x7fe640159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:28.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe646404700 1 -- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe6400b1db0 msgr2=0x7fe640159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:28.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe646404700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe6400b1db0 0x7fe640159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:28.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe646404700 1 -- 172.21.2.104:0/131308601 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c00b1e0 con 0x7fe6401483a0 2021-07-11T04:00:28.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe645c03700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6400a70c0 0x7fe640144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:28.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe645402700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe6400b1db0 0x7fe640159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:28.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe646404700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe6401483a0 0x7fe64015a420 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7fe6380032f0 tx=0x7fe638009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:28.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe632ffd700 1 -- 172.21.2.104:0/131308601 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe63800d580 con 0x7fe6401483a0 2021-07-11T04:00:28.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe632ffd700 1 -- 172.21.2.104:0/131308601 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63800d6e0 con 0x7fe6401483a0 2021-07-11T04:00:28.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 -- 172.21.2.104:0/131308601 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6400ad0d0 con 0x7fe6401483a0 2021-07-11T04:00:28.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe647e67700 1 -- 172.21.2.104:0/131308601 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe640146eb0 con 0x7fe6401483a0 2021-07-11T04:00:28.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.834+0000 7fe632ffd700 1 -- 172.21.2.104:0/131308601 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe63800da40 con 0x7fe6401483a0 2021-07-11T04:00:28.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.838+0000 7fe647e67700 1 -- 172.21.2.104:0/131308601 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6400ad260 con 0x7fe6401483a0 2021-07-11T04:00:28.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.838+0000 7fe632ffd700 1 -- 172.21.2.104:0/131308601 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe63800dba0 con 0x7fe6401483a0 2021-07-11T04:00:28.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.838+0000 7fe632ffd700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe620053380 0x7fe620055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:28.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.838+0000 7fe632ffd700 1 -- 172.21.2.104:0/131308601 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe63806b5f0 con 0x7fe6401483a0 2021-07-11T04:00:28.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.838+0000 7fe645c03700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe620053380 0x7fe620055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:28.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.838+0000 7fe645c03700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe620053380 0x7fe620055840 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fe634008000 tx=0x7fe634009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:28.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.838+0000 7fe632ffd700 1 -- 172.21.2.104:0/131308601 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe638042b00 con 0x7fe6401483a0 2021-07-11T04:00:28.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:28.950+0000 7fe647e67700 1 -- 172.21.2.104:0/131308601 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7fe640057350 con 0x7fe6401483a0 2021-07-11T04:00:29.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.770+0000 7fe632ffd700 1 -- 172.21.2.104:0/131308601 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v297) v1 ==== 141+0+0 (secure 0 0 0) 0x7fe638046120 con 0x7fe6401483a0 2021-07-11T04:00:29.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 -- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe620053380 msgr2=0x7fe620055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:29.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe620053380 0x7fe620055840 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7fe634008000 tx=0x7fe634009040).stop 2021-07-11T04:00:29.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 -- 172.21.2.104:0/131308601 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe6401483a0 msgr2=0x7fe64015a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:29.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe6401483a0 0x7fe64015a420 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7fe6380032f0 tx=0x7fe638009220).stop 2021-07-11T04:00:29.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 -- 172.21.2.104:0/131308601 shutdown_connections 2021-07-11T04:00:29.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe620053380 0x7fe620055840 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:29.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe6401483a0 0x7fe64015a420 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:29.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe6400b1db0 0x7fe640159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:29.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 --2- 172.21.2.104:0/131308601 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe6400a70c0 0x7fe640144990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:29.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 -- 172.21.2.104:0/131308601 >> 172.21.2.104:0/131308601 conn(0x7fe64005f6b0 msgr2=0x7fe6400a7550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:29.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 -- 172.21.2.104:0/131308601 shutdown_connections 2021-07-11T04:00:29.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.774+0000 7fe647e67700 1 -- 172.21.2.104:0/131308601 wait complete. 2021-07-11T04:00:29.783 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T04:00:29.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 Processor -- start 2021-07-11T04:00:29.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 -- start start 2021-07-11T04:00:29.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28d80b12d0 0x7f28d80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:29.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f28d80b22a0 0x7f28d8141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:29.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d80a77d0 con 0x7f28d80b1ec0 2021-07-11T04:00:29.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d80b1740 con 0x7f28d80b22a0 2021-07-11T04:00:29.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d80b0650 con 0x7f28d80b12d0 2021-07-11T04:00:29.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28d7fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f28d80b22a0 0x7f28d8141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:29.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28d77fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f28d80b1ec0 0x7f28d80a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44434/0 (socket says 172.21.2.104:44434) 2021-07-11T04:00:29.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28d77fe700 1 -- 172.21.2.104:0/1725138101 learned_addr learned my addr 172.21.2.104:0/1725138101 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:29.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28d7fff700 1 -- 172.21.2.104:0/1725138101 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28d80b12d0 msgr2=0x7f28d80affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:29.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28d7fff700 1 --2- 172.21.2.104:0/1725138101 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28d80b12d0 0x7f28d80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:29.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28d7fff700 1 -- 172.21.2.104:0/1725138101 >> v1:172.21.2.104:6789/0 conn(0x7f28d80b1ec0 legacy=0x7f28d80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:29.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28d7fff700 1 -- 172.21.2.104:0/1725138101 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d80b08f0 con 0x7f28d80b22a0 2021-07-11T04:00:29.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28d7fff700 1 --2- 172.21.2.104:0/1725138101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28d80b22a0 0x7f28d8141250 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f28d00032f0 tx=0x7f28d00091d0).ready entity=mon.0 client_cookie=9152dcb7e8e855cc server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:29.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28d67fc700 1 -- 172.21.2.104:0/1725138101 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28d0009ea0 con 0x7f28d80b22a0 2021-07-11T04:00:29.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28d67fc700 1 -- 172.21.2.104:0/1725138101 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d000b8a0 con 0x7f28d80b22a0 2021-07-11T04:00:29.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28d67fc700 1 -- 172.21.2.104:0/1725138101 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28d000ba10 con 0x7f28d80b22a0 2021-07-11T04:00:29.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 -- 172.21.2.104:0/1725138101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28d80b22a0 msgr2=0x7f28d8141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:29.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 --2- 172.21.2.104:0/1725138101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28d80b22a0 0x7f28d8141250 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f28d00032f0 tx=0x7f28d00091d0).stop 2021-07-11T04:00:29.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 -- 172.21.2.104:0/1725138101 shutdown_connections 2021-07-11T04:00:29.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 --2- 172.21.2.104:0/1725138101 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28d80b22a0 0x7f28d8141250 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:29.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 --2- 172.21.2.104:0/1725138101 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28d80b12d0 0x7f28d80affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:29.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 -- 172.21.2.104:0/1725138101 >> 172.21.2.104:0/1725138101 conn(0x7f28d805f6b0 msgr2=0x7f28d805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:29.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 -- 172.21.2.104:0/1725138101 shutdown_connections 2021-07-11T04:00:29.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 -- 172.21.2.104:0/1725138101 wait complete. 2021-07-11T04:00:29.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 Processor -- start 2021-07-11T04:00:29.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.834+0000 7f28ddd4c700 1 -- start start 2021-07-11T04:00:29.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28ddd4c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28d80b12d0 0x7f28d80ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:29.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28ddd4c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28d80b1ec0 0x7f28d8156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:29.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28ddd4c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28d80b22a0 0x7f28d8157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:29.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28ddd4c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d80b3420 con 0x7f28d80b1ec0 2021-07-11T04:00:29.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28ddd4c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d814b1f0 con 0x7f28d80b12d0 2021-07-11T04:00:29.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28ddd4c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d814ac80 con 0x7f28d80b22a0 2021-07-11T04:00:29.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28d80b1ec0 0x7f28d8156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:29.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28d80b1ec0 0x7f28d8156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47676/0 (socket says 172.21.2.104:47676) 2021-07-11T04:00:29.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d6ffd700 1 -- 172.21.2.104:0/270200969 learned_addr learned my addr 172.21.2.104:0/270200969 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:29.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d6ffd700 1 -- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28d80b22a0 msgr2=0x7f28d8157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:29.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d6ffd700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28d80b22a0 0x7f28d8157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:29.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d6ffd700 1 -- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28d80b12d0 msgr2=0x7f28d80ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:29.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d6ffd700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28d80b12d0 0x7f28d80ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:29.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d6ffd700 1 -- 172.21.2.104:0/270200969 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d0009470 con 0x7f28d80b1ec0 2021-07-11T04:00:29.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d6ffd700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28d80b1ec0 0x7f28d8156f80 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f28c8002f70 tx=0x7f28c8006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:29.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d4ff9700 1 -- 172.21.2.104:0/270200969 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c8013460 con 0x7f28d80b1ec0 2021-07-11T04:00:29.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d4ff9700 1 -- 172.21.2.104:0/270200969 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c80135c0 con 0x7f28d80b1ec0 2021-07-11T04:00:29.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d4ff9700 1 -- 172.21.2.104:0/270200969 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c8013920 con 0x7f28d80b1ec0 2021-07-11T04:00:29.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28ddd4c700 1 -- 172.21.2.104:0/270200969 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d814b440 con 0x7f28d80b1ec0 2021-07-11T04:00:29.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28ddd4c700 1 -- 172.21.2.104:0/270200969 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d8156ab0 con 0x7f28d80b1ec0 2021-07-11T04:00:29.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28ddd4c700 1 -- 172.21.2.104:0/270200969 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d8057350 con 0x7f28d80b1ec0 2021-07-11T04:00:29.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d4ff9700 1 -- 172.21.2.104:0/270200969 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f28c800f070 con 0x7f28d80b1ec0 2021-07-11T04:00:29.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d4ff9700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28bc053180 0x7f28bc055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:29.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d4ff9700 1 -- 172.21.2.104:0/270200969 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f28c806aac0 con 0x7f28d80b1ec0 2021-07-11T04:00:29.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.838+0000 7f28d77fe700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28bc053180 0x7f28bc055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:29.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.842+0000 7f28d77fe700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28bc053180 0x7f28bc055640 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f28d806a0f0 tx=0x7f28c0007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:29.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.842+0000 7f28d4ff9700 1 -- 172.21.2.104:0/270200969 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f28c80420e0 con 0x7f28d80b1ec0 2021-07-11T04:00:29.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.922+0000 7f28ddd4c700 1 -- 172.21.2.104:0/270200969 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f28d80ac760 con 0x7f28d80b1ec0 2021-07-11T04:00:29.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.930+0000 7f28d4ff9700 1 -- 172.21.2.104:0/270200969 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f28c8045700 con 0x7f28d80b1ec0 2021-07-11T04:00:29.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 -- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28bc053180 msgr2=0x7f28bc055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:29.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28bc053180 0x7f28bc055640 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f28d806a0f0 tx=0x7f28c0007800).stop 2021-07-11T04:00:29.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 -- 172.21.2.104:0/270200969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28d80b1ec0 msgr2=0x7f28d8156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:29.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28d80b1ec0 0x7f28d8156f80 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7f28c8002f70 tx=0x7f28c8006ec0).stop 2021-07-11T04:00:29.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 -- 172.21.2.104:0/270200969 shutdown_connections 2021-07-11T04:00:29.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28d80b22a0 0x7f28d8157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:29.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28bc053180 0x7f28bc055640 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:29.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28d80b1ec0 0x7f28d8156f80 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:29.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 --2- 172.21.2.104:0/270200969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28d80b12d0 0x7f28d80ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:29.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 -- 172.21.2.104:0/270200969 >> 172.21.2.104:0/270200969 conn(0x7f28d805f6b0 msgr2=0x7f28d80a70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:29.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 -- 172.21.2.104:0/270200969 shutdown_connections 2021-07-11T04:00:29.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:29.934+0000 7f28ddd4c700 1 -- 172.21.2.104:0/270200969 wait complete. 2021-07-11T04:00:29.943 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:00:29.944 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:29.952 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:29.962 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:29.977 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:29.991 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:00:29.991 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:29.992 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:29.992 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:29.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:00:29.997 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T04:00:29.997 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T04:00:29.998 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:29.998 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:29.999 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:30.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:00:30.037 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T04:00:30.038 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T04:00:30.038 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:00:30.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.090+0000 7f828486b700 1 Processor -- start 2021-07-11T04:00:30.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.090+0000 7f828486b700 1 -- start start 2021-07-11T04:00:30.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.090+0000 7f828486b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f82801483a0 0x7f82800b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.090+0000 7f828486b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f82800a70c0 0x7f8280144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.090+0000 7f828486b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82800b0350 con 0x7f82800b1db0 2021-07-11T04:00:30.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.090+0000 7f828486b700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82800a90c0 con 0x7f82801483a0 2021-07-11T04:00:30.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.090+0000 7f828486b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82800b05f0 con 0x7f82800a70c0 2021-07-11T04:00:30.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827dd9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f82801483a0 0x7f82800b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827dd9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f82801483a0 0x7f82800b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47682/0 (socket says 172.21.2.104:47682) 2021-07-11T04:00:30.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827dd9b700 1 -- 172.21.2.104:0/4076634337 learned_addr learned my addr 172.21.2.104:0/4076634337 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:30.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827dd9b700 1 -- 172.21.2.104:0/4076634337 >> v2:172.21.2.105:3300/0 conn(0x7f82800a70c0 msgr2=0x7f8280144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:30.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827dd9b700 1 --2- 172.21.2.104:0/4076634337 >> v2:172.21.2.105:3300/0 conn(0x7f82800a70c0 0x7f8280144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:30.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827dd9b700 1 -- 172.21.2.104:0/4076634337 >> v1:172.21.2.105:6789/0 conn(0x7f82800b1db0 legacy=0x7f82800b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:30.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827dd9b700 1 -- 172.21.2.104:0/4076634337 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82800b0890 con 0x7f82801483a0 2021-07-11T04:00:30.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827dd9b700 1 --2- 172.21.2.104:0/4076634337 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82801483a0 0x7f82800b0fd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f82800ace60 tx=0x7f8268006eb0).ready entity=mon.0 client_cookie=5e991812af941620 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827d59a700 1 -- 172.21.2.104:0/4076634337 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f826800d5c0 con 0x7f82801483a0 2021-07-11T04:00:30.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827d59a700 1 -- 172.21.2.104:0/4076634337 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8268000e90 con 0x7f82801483a0 2021-07-11T04:00:30.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827d59a700 1 -- 172.21.2.104:0/4076634337 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f826800d950 con 0x7f82801483a0 2021-07-11T04:00:30.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 -- 172.21.2.104:0/4076634337 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82801483a0 msgr2=0x7f82800b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 --2- 172.21.2.104:0/4076634337 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82801483a0 0x7f82800b0fd0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f82800ace60 tx=0x7f8268006eb0).stop 2021-07-11T04:00:30.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 -- 172.21.2.104:0/4076634337 shutdown_connections 2021-07-11T04:00:30.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 --2- 172.21.2.104:0/4076634337 >> v2:172.21.2.105:3300/0 conn(0x7f82800a70c0 0x7f8280144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:30.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 --2- 172.21.2.104:0/4076634337 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82801483a0 0x7f82800b0fd0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 -- 172.21.2.104:0/4076634337 >> 172.21.2.104:0/4076634337 conn(0x7f828005f6b0 msgr2=0x7f828005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:30.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 -- 172.21.2.104:0/4076634337 shutdown_connections 2021-07-11T04:00:30.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 -- 172.21.2.104:0/4076634337 wait complete. 2021-07-11T04:00:30.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 Processor -- start 2021-07-11T04:00:30.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 -- start start 2021-07-11T04:00:30.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82800a70c0 0x7f82800ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82800b1db0 0x7f828013f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82801483a0 0x7f828013fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828014a1e0 con 0x7f82801483a0 2021-07-11T04:00:30.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82800a9bd0 con 0x7f82800a70c0 2021-07-11T04:00:30.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82801402c0 con 0x7f82800b1db0 2021-07-11T04:00:30.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82801483a0 0x7f828013fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82801483a0 0x7f828013fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47686/0 (socket says 172.21.2.104:47686) 2021-07-11T04:00:30.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827ed9d700 1 -- 172.21.2.104:0/2462280553 learned_addr learned my addr 172.21.2.104:0/2462280553 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:30.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827ed9d700 1 -- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82800b1db0 msgr2=0x7f828013f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827ed9d700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82800b1db0 0x7f828013f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:30.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827ed9d700 1 -- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82800a70c0 msgr2=0x7f82800ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827ed9d700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82800a70c0 0x7f82800ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:30.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827ed9d700 1 -- 172.21.2.104:0/2462280553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826800b1e0 con 0x7f82801483a0 2021-07-11T04:00:30.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f827ed9d700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82801483a0 0x7f828013fdd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f82800a99e0 tx=0x7f8278009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f82777fe700 1 -- 172.21.2.104:0/2462280553 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f827800d580 con 0x7f82801483a0 2021-07-11T04:00:30.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f82777fe700 1 -- 172.21.2.104:0/2462280553 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f827800d6e0 con 0x7f82801483a0 2021-07-11T04:00:30.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 -- 172.21.2.104:0/2462280553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8280149c50 con 0x7f82801483a0 2021-07-11T04:00:30.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f82777fe700 1 -- 172.21.2.104:0/2462280553 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f827800da40 con 0x7f82801483a0 2021-07-11T04:00:30.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.094+0000 7f828486b700 1 -- 172.21.2.104:0/2462280553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82800ac7f0 con 0x7f82801483a0 2021-07-11T04:00:30.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.098+0000 7f828486b700 1 -- 172.21.2.104:0/2462280553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8280057350 con 0x7f82801483a0 2021-07-11T04:00:30.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.098+0000 7f82777fe700 1 -- 172.21.2.104:0/2462280553 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f827800dba0 con 0x7f82801483a0 2021-07-11T04:00:30.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.098+0000 7f82777fe700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82640531f0 0x7f82640556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.098+0000 7f82777fe700 1 -- 172.21.2.104:0/2462280553 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f827806b650 con 0x7f82801483a0 2021-07-11T04:00:30.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.098+0000 7f827e59c700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82640531f0 0x7f82640556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.098+0000 7f827e59c700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82640531f0 0x7f82640556b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f8270000f70 tx=0x7f82700077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.098+0000 7f82777fe700 1 -- 172.21.2.104:0/2462280553 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8278042b60 con 0x7f82801483a0 2021-07-11T04:00:30.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.178+0000 7f828486b700 1 -- 172.21.2.104:0/2462280553 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8280044650 con 0x7f82801483a0 2021-07-11T04:00:30.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f82777fe700 1 -- 172.21.2.104:0/2462280553 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v90) v1 ==== 81+0+28828 (secure 0 0 0) 0x7f8278046180 con 0x7f82801483a0 2021-07-11T04:00:30.185 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:30.185 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:00:30.185 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 90, 2021-07-11T04:00:30.185 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:00:30.186 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:00:30.186 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:00:27.749565+0000", 2021-07-11T04:00:30.186 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:30.186 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:30.187 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:00:30.187 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:00:30.187 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:00:30.187 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:00:30.187 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:00:30.188 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:00:30.188 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:00:30.188 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:30.188 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:00:30.189 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:00:30.189 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:00:30.189 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:00:30.189 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:00:30.189 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 16, 2021-07-11T04:00:30.190 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:00:30.190 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:00:30.190 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:00:30.190 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:00:30.191 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:00:30.191 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.191 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:00:30.191 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:00:30.191 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:00:30.192 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:30.192 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:30.192 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:30.192 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:30.193 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:30.193 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:30.193 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:30.193 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:30.193 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:00:30.194 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:00:30.194 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:00:30.194 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:00:30.194 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:00:30.195 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:30.195 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:30.195 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:30.195 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:30.195 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:30.196 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:30.196 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:30.196 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.196 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:00:30.196 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:30.197 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:30.197 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:30.197 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:30.197 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:30.198 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:30.198 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:30.198 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:30.198 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:30.198 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:30.199 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:30.199 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:30.199 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:30.199 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:30.200 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:30.200 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:30.200 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:30.200 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:30.200 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:30.201 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:30.201 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:30.201 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:30.201 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:30.201 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:30.202 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:30.202 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:30.202 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.202 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:30.203 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:30.203 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:30.203 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:30.203 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:30.203 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:30.204 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:30.204 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:30.204 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:30.204 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:30.205 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:30.205 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:00:30.205 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:00:30.205 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.205 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:30.206 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:00:30.206 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.206 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.206 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.206 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:00:30.207 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:00:30.207 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:00:30.207 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:30.207 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:30.208 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:30.208 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:30.208 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:30.208 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:30.208 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:30.209 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:30.209 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:30.209 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:30.209 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:30.210 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:30.210 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:30.210 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:30.211 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:30.211 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:30.211 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:30.211 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:30.211 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:30.212 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:30.212 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.212 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:00:30.212 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:30.212 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:30.213 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:30.213 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:30.213 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:30.213 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:30.214 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:30.214 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:30.214 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:30.214 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:30.214 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:30.215 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:30.215 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:30.215 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:30.215 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:30.216 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:30.216 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:30.216 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:30.216 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:30.216 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:30.217 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:30.217 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:30.217 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:30.217 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:30.217 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:30.218 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:30.218 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.218 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:30.218 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:30.219 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:30.219 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:30.219 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:30.219 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:30.219 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:30.220 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:30.220 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:30.220 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:30.220 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:30.221 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:00:30.221 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:30.221 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:00:30.221 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.221 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.222 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:30.222 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:00:30.222 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.222 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:30.223 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:00:30.223 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:30.223 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:30.223 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:30.223 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:30.224 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:30.224 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:30.224 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:30.224 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:30.224 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:30.225 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:30.225 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:30.225 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.225 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.226 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.226 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:00:30.226 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:30.226 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.226 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.227 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.227 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:00:30.227 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:30.227 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.228 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.228 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.228 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:30.228 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.229 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.229 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.229 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:00:30.229 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:30.230 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.230 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.230 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.230 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:00:30.231 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:30.231 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.231 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.231 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.231 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:30.232 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.232 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.232 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.232 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:00:30.232 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:30.233 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.233 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.233 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.233 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:00:30.234 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:30.234 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.234 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.234 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.234 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:30.235 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.235 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.235 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.235 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:00:30.236 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:30.236 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.236 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.236 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.236 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:00:30.237 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:30.237 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.237 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.237 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.237 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:00:30.238 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:00:30.238 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:00:30.238 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:00:30.238 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:30.239 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:30.239 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:30.239 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.239 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.239 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.240 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:30.240 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:00:30.240 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:30.240 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:30.241 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:30.241 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:30.241 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:30.241 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:30.241 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:30.242 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 83, 2021-07-11T04:00:30.242 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:30.242 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:30.242 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:30.243 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.243 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.243 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.243 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:00:30.243 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:30.244 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.244 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.244 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.244 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:00:30.245 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:30.245 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.245 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.245 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.245 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:30.246 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.246 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.246 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.246 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:00:30.247 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:30.247 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.247 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.247 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.247 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:00:30.248 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:30.248 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.248 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.248 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.248 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:30.249 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.249 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.249 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.249 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:00:30.250 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:30.250 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.250 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.250 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.250 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:00:30.251 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:30.251 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.251 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.251 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.252 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:30.252 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.252 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.252 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.252 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:00:30.253 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:30.253 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.253 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.253 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.253 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:00:30.254 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:30.254 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.254 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.254 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.255 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:00:30.255 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:00:30.255 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:00:30.255 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:00:30.255 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:30.256 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:30.256 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:30.256 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.256 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.256 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.257 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:30.257 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:00:30.257 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:30.258 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:30.258 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:30.258 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:30.258 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:30.258 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:30.259 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:30.259 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:30.259 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:30.259 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:30.259 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:30.260 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.260 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.260 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.260 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:00:30.261 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:30.261 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.261 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.261 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.261 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:00:30.262 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:30.262 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.262 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.262 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.263 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:30.263 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.263 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.263 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.263 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:00:30.264 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:30.264 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.264 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.264 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.264 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:00:30.265 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:30.265 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.265 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.265 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.266 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:30.266 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.266 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.266 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.266 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:00:30.267 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:30.267 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.267 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.267 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.267 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:00:30.268 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:30.268 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.268 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.268 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.269 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:30.269 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.269 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.269 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.270 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:00:30.270 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:30.270 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.270 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.270 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.271 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:00:30.271 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:30.271 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.271 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.272 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.272 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:00:30.272 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:00:30.272 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:00:30.272 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:00:30.273 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:30.273 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:30.273 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:30.273 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.273 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.274 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.274 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:30.274 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:00:30.274 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:30.275 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:30.275 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:30.275 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:30.275 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:30.275 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:30.276 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:30.276 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:30.276 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:30.276 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:30.276 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:30.277 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.277 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.277 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.277 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:00:30.278 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:30.278 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.278 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.278 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.278 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:00:30.279 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:30.279 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.279 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.279 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.279 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:30.280 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.280 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.280 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.280 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:00:30.281 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:30.281 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.281 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.281 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.281 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:00:30.282 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:30.282 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.282 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.282 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.282 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:30.283 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.283 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.283 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.283 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:00:30.284 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:30.284 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.284 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.284 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.284 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:00:30.285 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:30.285 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.285 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.285 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.285 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:30.286 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.286 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.286 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.286 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:00:30.287 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:30.287 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.287 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.287 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.287 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:00:30.288 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:30.288 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.288 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.288 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.288 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:00:30.289 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:00:30.289 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:00:30.289 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:00:30.289 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:30.290 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:30.290 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:30.290 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.290 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.290 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.291 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:30.291 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:00:30.291 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:30.291 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:30.291 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:30.292 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:30.292 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:30.292 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:30.292 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:30.293 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:30.293 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:30.293 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:30.293 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:30.293 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.294 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.294 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.294 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:00:30.294 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:30.294 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.295 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.295 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.295 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:00:30.295 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:30.296 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.296 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.296 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.296 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:30.297 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.297 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.297 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.297 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:00:30.297 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:30.298 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.298 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.298 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.298 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:00:30.299 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:30.299 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.299 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.299 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.299 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:30.300 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.300 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.300 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.300 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:00:30.300 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:30.301 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.301 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.301 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.301 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:00:30.302 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:30.302 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.302 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.302 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.302 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:30.303 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.303 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.303 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.303 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:00:30.303 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:30.304 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.304 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.304 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.304 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:00:30.305 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:30.305 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.305 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.305 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.305 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:00:30.306 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:00:30.306 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:00:30.306 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:00:30.306 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:30.307 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:30.307 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:30.307 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.307 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.307 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.308 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:30.308 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:00:30.308 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:30.308 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:30.308 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:30.309 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:30.309 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:30.309 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:30.309 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:30.310 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:30.310 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:30.310 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:30.310 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:30.310 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.311 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.311 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.311 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:00:30.311 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:30.311 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.312 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.312 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.312 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:00:30.312 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:30.313 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.313 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.313 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.313 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:30.313 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.314 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.314 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.314 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:00:30.314 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:30.315 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.315 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.315 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.315 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:00:30.315 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:30.316 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.316 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.316 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.316 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:30.316 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.317 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.317 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.317 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:00:30.317 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:30.318 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.318 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.318 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.318 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:00:30.318 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:30.319 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.319 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.319 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.319 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:30.319 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.320 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.320 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.320 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:00:30.321 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:30.321 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.321 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.321 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.322 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:00:30.322 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:30.322 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.322 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.323 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.323 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:00:30.323 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:00:30.323 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:00:30.324 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:00:30.324 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:30.324 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:30.324 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:30.324 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.325 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.325 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.325 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:30.325 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:00:30.325 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:30.326 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:30.326 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:30.326 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:30.326 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:30.327 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:30.327 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:30.327 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:30.327 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:30.327 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:30.328 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:30.328 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.328 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.328 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.328 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:00:30.329 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:30.329 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.329 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.329 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.330 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:00:30.330 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:30.330 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.330 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.331 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.331 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:30.331 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.331 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.331 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.332 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:00:30.332 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:30.332 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.332 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.333 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.333 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:00:30.333 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:30.333 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.333 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.334 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.334 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:30.334 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.334 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.338 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.339 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:00:30.339 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:30.339 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.339 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.339 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.340 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:00:30.340 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:30.340 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.340 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.341 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.341 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:30.341 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.341 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.341 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.342 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:00:30.342 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:30.342 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.342 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.343 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.343 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:00:30.343 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:30.343 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.343 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.344 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.344 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:00:30.344 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:00:30.344 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:00:30.344 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:00:30.345 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:30.345 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:30.345 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:30.345 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.346 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.346 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.346 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:30.346 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:00:30.346 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:30.347 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:30.347 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:30.347 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:30.347 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:30.347 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:30.348 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:30.348 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 73, 2021-07-11T04:00:30.348 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:30.348 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:30.349 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:30.349 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.349 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.349 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.349 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:00:30.350 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:30.350 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.350 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.350 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.350 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:00:30.351 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:30.351 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.351 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.351 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.352 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:30.352 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.352 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.352 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.352 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:00:30.353 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:30.353 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.353 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.353 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.353 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:00:30.354 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:30.354 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.354 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.354 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.355 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:30.355 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.355 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.355 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.355 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:00:30.356 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:30.356 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.356 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.356 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.357 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:00:30.357 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:30.357 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.357 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.358 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.358 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:30.358 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:30.358 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.359 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:30.359 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:00:30.359 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:30.359 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.359 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.360 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:30.360 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:00:30.361 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:30.361 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.361 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.361 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.362 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:00:30.362 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:00:30.362 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:00:30.362 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:00:30.362 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:30.363 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:30.363 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:30.363 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:30.363 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.364 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:30.364 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:00:30.364 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.364 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:30.364 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:30.365 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:30.365 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:30.365 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:30.365 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:30.365 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:00:30.366 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:30.366 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.366 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.366 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:30.367 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:30.367 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:30.367 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:30.367 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:30.367 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:30.368 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:00:30.368 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:30.368 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.368 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.369 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:30.369 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:30.369 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:30.369 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:30.369 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:30.370 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:30.370 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:00:30.370 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:30.370 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.370 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.371 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:30.371 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:30.371 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:30.371 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:30.372 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:30.372 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:30.372 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:00:30.372 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:30.372 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.373 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.373 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:30.373 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:30.373 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:30.373 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:30.374 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:30.374 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:30.374 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:00:30.374 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:30.375 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.375 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.375 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:30.375 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:30.375 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:30.376 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:30.376 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:30.376 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:30.376 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:00:30.377 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:30.377 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.377 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.377 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:30.377 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:30.378 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:30.378 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:30.378 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:30.378 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:30.378 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:00:30.379 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:30.379 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.379 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:30.379 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:30.380 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:30.380 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:30.380 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:30.380 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:30.380 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:30.381 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:00:30.381 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:30.381 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.381 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:30.381 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:00:30.382 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:00:30.382 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:00:30.382 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:00:30.382 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T04:00:30.383 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6835/4286166182": "2021-07-12T04:00:24.326723+0000", 2021-07-11T04:00:30.383 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6834/4286166182": "2021-07-12T04:00:24.326723+0000" 2021-07-11T04:00:30.383 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.383 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:00:30.383 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:00:30.384 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:00:30.384 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:00:30.384 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:00:30.384 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:00:30.384 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:00:30.385 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:00:30.385 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:30.385 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:30.385 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:00:30.386 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:00:30.386 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:00:30.386 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:00:30.386 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:00:30.386 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:00:30.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 -- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82640531f0 msgr2=0x7f82640556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82640531f0 0x7f82640556b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f8270000f70 tx=0x7f82700077f0).stop 2021-07-11T04:00:30.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 -- 172.21.2.104:0/2462280553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82801483a0 msgr2=0x7f828013fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82801483a0 0x7f828013fdd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f82800a99e0 tx=0x7f8278009220).stop 2021-07-11T04:00:30.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 -- 172.21.2.104:0/2462280553 shutdown_connections 2021-07-11T04:00:30.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82640531f0 0x7f82640556b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82801483a0 0x7f828013fdd0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82800b1db0 0x7f828013f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:30.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 --2- 172.21.2.104:0/2462280553 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82800a70c0 0x7f82800ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:30.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 -- 172.21.2.104:0/2462280553 >> 172.21.2.104:0/2462280553 conn(0x7f828005f6b0 msgr2=0x7f8280147730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:30.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 -- 172.21.2.104:0/2462280553 shutdown_connections 2021-07-11T04:00:30.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.182+0000 7f828486b700 1 -- 172.21.2.104:0/2462280553 wait complete. 2021-07-11T04:00:30.391 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:30.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.502+0000 7fa61f7df700 1 Processor -- start 2021-07-11T04:00:30.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.502+0000 7fa61f7df700 1 -- start start 2021-07-11T04:00:30.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.502+0000 7fa61f7df700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa6180a76b0 0x7fa618066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.502+0000 7fa61f7df700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa6180b0060 0x7fa61805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.502+0000 7fa61f7df700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6180a7a90 con 0x7fa6180b1970 2021-07-11T04:00:30.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.502+0000 7fa61f7df700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa618147620 con 0x7fa6180b0060 2021-07-11T04:00:30.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.502+0000 7fa61f7df700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6181478c0 con 0x7fa6180a76b0 2021-07-11T04:00:30.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61dd7c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa6180b0060 0x7fa61805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61dd7c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa6180b0060 0x7fa61805fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47696/0 (socket says 172.21.2.104:47696) 2021-07-11T04:00:30.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61dd7c700 1 -- 172.21.2.104:0/3714903380 learned_addr learned my addr 172.21.2.104:0/3714903380 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:30.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61dd7c700 1 -- 172.21.2.104:0/3714903380 >> v2:172.21.2.105:3300/0 conn(0x7fa6180a76b0 msgr2=0x7fa618066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:30.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61dd7c700 1 --2- 172.21.2.104:0/3714903380 >> v2:172.21.2.105:3300/0 conn(0x7fa6180a76b0 0x7fa618066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:30.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61dd7c700 1 -- 172.21.2.104:0/3714903380 >> v1:172.21.2.105:6789/0 conn(0x7fa6180b1970 legacy=0x7fa61805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:30.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61dd7c700 1 -- 172.21.2.104:0/3714903380 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa618147b60 con 0x7fa6180b0060 2021-07-11T04:00:30.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61dd7c700 1 --2- 172.21.2.104:0/3714903380 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6180b0060 0x7fa61805fdc0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7fa61806a0e0 tx=0x7fa610009290).ready entity=mon.0 client_cookie=7229c1265dea45fa server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa60bfff700 1 -- 172.21.2.104:0/3714903380 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa61000b830 con 0x7fa6180b0060 2021-07-11T04:00:30.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa60bfff700 1 -- 172.21.2.104:0/3714903380 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61000b990 con 0x7fa6180b0060 2021-07-11T04:00:30.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa60bfff700 1 -- 172.21.2.104:0/3714903380 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa61000bb00 con 0x7fa6180b0060 2021-07-11T04:00:30.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 -- 172.21.2.104:0/3714903380 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6180b0060 msgr2=0x7fa61805fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 --2- 172.21.2.104:0/3714903380 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6180b0060 0x7fa61805fdc0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7fa61806a0e0 tx=0x7fa610009290).stop 2021-07-11T04:00:30.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 -- 172.21.2.104:0/3714903380 shutdown_connections 2021-07-11T04:00:30.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 --2- 172.21.2.104:0/3714903380 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6180b0060 0x7fa61805fdc0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 --2- 172.21.2.104:0/3714903380 >> v2:172.21.2.105:3300/0 conn(0x7fa6180a76b0 0x7fa618066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:30.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 -- 172.21.2.104:0/3714903380 >> 172.21.2.104:0/3714903380 conn(0x7fa6180b53e0 msgr2=0x7fa6180b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:30.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 -- 172.21.2.104:0/3714903380 shutdown_connections 2021-07-11T04:00:30.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 -- 172.21.2.104:0/3714903380 wait complete. 2021-07-11T04:00:30.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 Processor -- start 2021-07-11T04:00:30.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 -- start start 2021-07-11T04:00:30.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6180a76b0 0x7fa6180ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6180b0060 0x7fa6180a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6180b1970 0x7fa6180a9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6180a9c10 con 0x7fa6180b0060 2021-07-11T04:00:30.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61814e890 con 0x7fa6180a76b0 2021-07-11T04:00:30.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61814eb30 con 0x7fa6180b1970 2021-07-11T04:00:30.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61cd7a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6180b0060 0x7fa6180a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61cd7a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6180b0060 0x7fa6180a9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47700/0 (socket says 172.21.2.104:47700) 2021-07-11T04:00:30.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61cd7a700 1 -- 172.21.2.104:0/2762435420 learned_addr learned my addr 172.21.2.104:0/2762435420 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:30.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61cd7a700 1 -- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6180b1970 msgr2=0x7fa6180a9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61cd7a700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6180b1970 0x7fa6180a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:30.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61cd7a700 1 -- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6180a76b0 msgr2=0x7fa6180ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61d57b700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6180a76b0 0x7fa6180ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61cd7a700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6180a76b0 0x7fa6180ae970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61cd7a700 1 -- 172.21.2.104:0/2762435420 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa610009530 con 0x7fa6180b0060 2021-07-11T04:00:30.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61d57b700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6180a76b0 0x7fa6180ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:30.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61cd7a700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6180b0060 0x7fa6180a9230 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7fa614002f10 tx=0x7fa614006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa60a7fc700 1 -- 172.21.2.104:0/2762435420 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa614009e00 con 0x7fa6180b0060 2021-07-11T04:00:30.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa60a7fc700 1 -- 172.21.2.104:0/2762435420 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61400f510 con 0x7fa6180b0060 2021-07-11T04:00:30.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 -- 172.21.2.104:0/2762435420 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61814dae0 con 0x7fa6180b0060 2021-07-11T04:00:30.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa60a7fc700 1 -- 172.21.2.104:0/2762435420 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa61400f680 con 0x7fa6180b0060 2021-07-11T04:00:30.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.506+0000 7fa61f7df700 1 -- 172.21.2.104:0/2762435420 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6180af460 con 0x7fa6180b0060 2021-07-11T04:00:30.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.510+0000 7fa61f7df700 1 -- 172.21.2.104:0/2762435420 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61814dc70 con 0x7fa6180b0060 2021-07-11T04:00:30.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.510+0000 7fa60a7fc700 1 -- 172.21.2.104:0/2762435420 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa61400f070 con 0x7fa6180b0060 2021-07-11T04:00:30.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.510+0000 7fa60a7fc700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa5f80531f0 0x7fa5f80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.510+0000 7fa60a7fc700 1 -- 172.21.2.104:0/2762435420 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa61406ab40 con 0x7fa6180b0060 2021-07-11T04:00:30.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.510+0000 7fa61d57b700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa5f80531f0 0x7fa5f80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.510+0000 7fa61d57b700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa5f80531f0 0x7fa5f80556b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fa60c007000 tx=0x7fa60c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.510+0000 7fa60a7fc700 1 -- 172.21.2.104:0/2762435420 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa614042050 con 0x7fa6180b0060 2021-07-11T04:00:30.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 -- 172.21.2.104:0/2762435420 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa618057350 con 0x7fa6180b0060 2021-07-11T04:00:30.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa60a7fc700 1 -- 172.21.2.104:0/2762435420 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+1556 (secure 0 0 0) 0x7fa614045670 con 0x7fa6180b0060 2021-07-11T04:00:30.623 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:30.624 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/43570850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":43570850},{"type":"v1","addr":"172.21.2.105:6835","nonce":43570850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/3334136782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3334136782},{"type":"v1","addr":"172.21.2.104:6833","nonce":3334136782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/3062926086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":3062926086},{"type":"v1","addr":"172.21.2.104:6835","nonce":3062926086}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":41}],"filesystems":[]} 2021-07-11T04:00:30.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 -- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa5f80531f0 msgr2=0x7fa5f80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa5f80531f0 0x7fa5f80556b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fa60c007000 tx=0x7fa60c008040).stop 2021-07-11T04:00:30.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 -- 172.21.2.104:0/2762435420 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6180b0060 msgr2=0x7fa6180a9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6180b0060 0x7fa6180a9230 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7fa614002f10 tx=0x7fa614006d60).stop 2021-07-11T04:00:30.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 -- 172.21.2.104:0/2762435420 shutdown_connections 2021-07-11T04:00:30.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa6180b1970 0x7fa6180a9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:30.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa5f80531f0 0x7fa5f80556b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa6180b0060 0x7fa6180a9230 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 --2- 172.21.2.104:0/2762435420 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa6180a76b0 0x7fa6180ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 -- 172.21.2.104:0/2762435420 >> 172.21.2.104:0/2762435420 conn(0x7fa6180b53e0 msgr2=0x7fa61814ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:30.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 -- 172.21.2.104:0/2762435420 shutdown_connections 2021-07-11T04:00:30.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.618+0000 7fa61f7df700 1 -- 172.21.2.104:0/2762435420 wait complete. 2021-07-11T04:00:30.630 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 41 2021-07-11T04:00:30.632 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:30.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.682+0000 7f0c84b22700 1 Processor -- start 2021-07-11T04:00:30.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.682+0000 7f0c84b22700 1 -- start start 2021-07-11T04:00:30.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.682+0000 7f0c84b22700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c80060d00 0x7f0c800aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.682+0000 7f0c84b22700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0c800afd80 0x7f0c80143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.682+0000 7f0c84b22700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c800671e0 con 0x7f0c80066e00 2021-07-11T04:00:30.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.682+0000 7f0c84b22700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c800b01b0 con 0x7f0c800afd80 2021-07-11T04:00:30.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.682+0000 7f0c84b22700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800a9450 con 0x7f0c80060d00 2021-07-11T04:00:30.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7ed9d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0c800afd80 0x7f0c80143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7dd9b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c80060d00 0x7f0c800aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7ed9d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0c800afd80 0x7f0c80143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:55270/0 (socket says 172.21.2.104:55270) 2021-07-11T04:00:30.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7ed9d700 1 -- 172.21.2.104:0/899360746 learned_addr learned my addr 172.21.2.104:0/899360746 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:30.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7dd9b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c80060d00 0x7f0c800aaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:50874/0 (socket says 172.21.2.104:50874) 2021-07-11T04:00:30.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7e59c700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f0c80066e00 0x7f0c800aa800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:43846/0 (socket says 172.21.2.104:43846) 2021-07-11T04:00:30.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7ed9d700 1 -- 172.21.2.104:0/899360746 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c80060d00 msgr2=0x7f0c800aaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:30.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7ed9d700 1 --2- 172.21.2.104:0/899360746 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c80060d00 0x7f0c800aaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7ed9d700 1 -- 172.21.2.104:0/899360746 >> v1:172.21.2.105:6789/0 conn(0x7f0c80066e00 legacy=0x7f0c800aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7ed9d700 1 -- 172.21.2.104:0/899360746 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800b03b0 con 0x7f0c800afd80 2021-07-11T04:00:30.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7ed9d700 1 --2- 172.21.2.104:0/899360746 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0c800afd80 0x7f0c80143460 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f0c70002f70 tx=0x7f0c70006eb0).ready entity=mon.1 client_cookie=8f23d73f3783d394 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7d59a700 1 -- 172.21.2.104:0/899360746 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7000d570 con 0x7f0c800afd80 2021-07-11T04:00:30.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7d59a700 1 -- 172.21.2.104:0/899360746 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c70000e90 con 0x7f0c800afd80 2021-07-11T04:00:30.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c7d59a700 1 -- 172.21.2.104:0/899360746 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c7000d900 con 0x7f0c800afd80 2021-07-11T04:00:30.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c84b22700 1 -- 172.21.2.104:0/899360746 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0c800afd80 msgr2=0x7f0c80143460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c84b22700 1 --2- 172.21.2.104:0/899360746 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0c800afd80 0x7f0c80143460 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f0c70002f70 tx=0x7f0c70006eb0).stop 2021-07-11T04:00:30.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c84b22700 1 -- 172.21.2.104:0/899360746 shutdown_connections 2021-07-11T04:00:30.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c84b22700 1 --2- 172.21.2.104:0/899360746 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0c800afd80 0x7f0c80143460 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c84b22700 1 --2- 172.21.2.104:0/899360746 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c80060d00 0x7f0c800aaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c84b22700 1 -- 172.21.2.104:0/899360746 >> 172.21.2.104:0/899360746 conn(0x7f0c800b53e0 msgr2=0x7f0c800b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:30.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c84b22700 1 -- 172.21.2.104:0/899360746 shutdown_connections 2021-07-11T04:00:30.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.686+0000 7f0c84b22700 1 -- 172.21.2.104:0/899360746 wait complete. 2021-07-11T04:00:30.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c84b22700 1 Processor -- start 2021-07-11T04:00:30.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c84b22700 1 -- start start 2021-07-11T04:00:30.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c84b22700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0c80060d00 0x7f0c800af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c84b22700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0c80066e00 0x7f0c800a7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c84b22700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c800afd80 0x7f0c800b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c84b22700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c800b4cb0 con 0x7f0c80060d00 2021-07-11T04:00:30.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c84b22700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80157010 con 0x7f0c80066e00 2021-07-11T04:00:30.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c84b22700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c801572b0 con 0x7f0c800afd80 2021-07-11T04:00:30.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0c80060d00 0x7f0c800af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0c80060d00 0x7f0c800af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47712/0 (socket says 172.21.2.104:47712) 2021-07-11T04:00:30.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7e59c700 1 -- 172.21.2.104:0/1341374761 learned_addr learned my addr 172.21.2.104:0/1341374761 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:30.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7e59c700 1 -- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c800afd80 msgr2=0x7f0c800b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7ed9d700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c800afd80 0x7f0c800b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7dd9b700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0c80066e00 0x7f0c800a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7e59c700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c800afd80 0x7f0c800b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7e59c700 1 -- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0c80066e00 msgr2=0x7f0c800a7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7e59c700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0c80066e00 0x7f0c800a7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7e59c700 1 -- 172.21.2.104:0/1341374761 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7000b1e0 con 0x7f0c80060d00 2021-07-11T04:00:30.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7dd9b700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0c80066e00 0x7f0c800a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:30.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7ed9d700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c800afd80 0x7f0c800b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:30.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7e59c700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0c80060d00 0x7f0c800af4d0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f0c8005f320 tx=0x7f0c68007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c777fe700 1 -- 172.21.2.104:0/1341374761 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c6800ce30 con 0x7f0c80060d00 2021-07-11T04:00:30.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c84b22700 1 -- 172.21.2.104:0/1341374761 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c800b4930 con 0x7f0c80060d00 2021-07-11T04:00:30.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c777fe700 1 -- 172.21.2.104:0/1341374761 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c680072e0 con 0x7f0c80060d00 2021-07-11T04:00:30.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c777fe700 1 -- 172.21.2.104:0/1341374761 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0c68004710 con 0x7f0c80060d00 2021-07-11T04:00:30.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c84b22700 1 -- 172.21.2.104:0/1341374761 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c80062bd0 con 0x7f0c80060d00 2021-07-11T04:00:30.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c84b22700 1 -- 172.21.2.104:0/1341374761 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c80057350 con 0x7f0c80060d00 2021-07-11T04:00:30.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c777fe700 1 -- 172.21.2.104:0/1341374761 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0c6801b070 con 0x7f0c80060d00 2021-07-11T04:00:30.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c777fe700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c640532b0 0x7f0c64055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c777fe700 1 -- 172.21.2.104:0/1341374761 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0c6806a760 con 0x7f0c80060d00 2021-07-11T04:00:30.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.690+0000 7f0c7dd9b700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c640532b0 0x7f0c64055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.694+0000 7f0c7dd9b700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c640532b0 0x7f0c64055770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f0c7800b000 tx=0x7f0c78008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.694+0000 7f0c777fe700 1 -- 172.21.2.104:0/1341374761 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0c68041c80 con 0x7f0c80060d00 2021-07-11T04:00:30.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 -- 172.21.2.104:0/1341374761 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c800626b0 con 0x7f0c80060d00 2021-07-11T04:00:30.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c777fe700 1 -- 172.21.2.104:0/1341374761 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+1556 (secure 0 0 0) 0x7f0c680452a0 con 0x7f0c80060d00 2021-07-11T04:00:30.805 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:30.806 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4912,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/43570850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":43570850},{"type":"v1","addr":"172.21.2.105:6835","nonce":43570850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":36},{"gid":6630,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/3334136782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":3334136782},{"type":"v1","addr":"172.21.2.104:6833","nonce":3334136782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":37},{"gid":6885,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/3062926086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":3062926086},{"type":"v1","addr":"172.21.2.104:6835","nonce":3062926086}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":41}],"filesystems":[]} 2021-07-11T04:00:30.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 -- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c640532b0 msgr2=0x7f0c64055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c640532b0 0x7f0c64055770 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f0c7800b000 tx=0x7f0c78008040).stop 2021-07-11T04:00:30.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 -- 172.21.2.104:0/1341374761 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0c80060d00 msgr2=0x7f0c800af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0c80060d00 0x7f0c800af4d0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7f0c8005f320 tx=0x7f0c68007bf0).stop 2021-07-11T04:00:30.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 -- 172.21.2.104:0/1341374761 shutdown_connections 2021-07-11T04:00:30.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0c800afd80 0x7f0c800b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0c80066e00 0x7f0c800a7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0c640532b0 0x7f0c64055770 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 --2- 172.21.2.104:0/1341374761 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0c80060d00 0x7f0c800af4d0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 -- 172.21.2.104:0/1341374761 >> 172.21.2.104:0/1341374761 conn(0x7f0c800b53e0 msgr2=0x7f0c800b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:30.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 -- 172.21.2.104:0/1341374761 shutdown_connections 2021-07-11T04:00:30.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.802+0000 7f0c84b22700 1 -- 172.21.2.104:0/1341374761 wait complete. 2021-07-11T04:00:30.810 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 41 2021-07-11T04:00:30.813 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T04:00:30.813 INFO:tasks.ceph.mds.a:Stopping old one... 2021-07-11T04:00:30.814 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T04:00:30.814 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:00:30.814 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T04:00:30.814 INFO:tasks.ceph.mds.c:Stopping old one... 2021-07-11T04:00:30.815 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T04:00:30.815 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:00:30.816 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T04:00:30.816 INFO:tasks.ceph.mds.b:Stopping old one... 2021-07-11T04:00:30.816 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T04:00:30.817 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:00:30.825 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T04:00:30.826 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T04:00:30.828 INFO:tasks.ceph.mds.b:Started 2021-07-11T04:00:30.829 DEBUG:teuthology.parallel:result is None 2021-07-11T04:00:30.861 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T04:00:30.862 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T04:00:30.863 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T04:00:30.863 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T04:00:30.866 INFO:tasks.ceph.mds.a:Started 2021-07-11T04:00:30.866 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T04:00:30.867 INFO:tasks.ceph.mds.c:Started 2021-07-11T04:00:30.867 DEBUG:teuthology.parallel:result is None 2021-07-11T04:00:30.868 DEBUG:teuthology.parallel:result is None 2021-07-11T04:00:30.868 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T04:00:30.899 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T04:00:30.900 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T04:00:30.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.918+0000 7f4628cd1700 1 Processor -- start 2021-07-11T04:00:30.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.918+0000 7f4628cd1700 1 -- start start 2021-07-11T04:00:30.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240aad80 0x7f4624067300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4624065960 con 0x7f46240ab160 2021-07-11T04:00:30.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4624064f10 con 0x7f46240b4d50 2021-07-11T04:00:30.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46240b3ce0 con 0x7f46240aad80 2021-07-11T04:00:30.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4622ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f46240ab160 0x7f4624064800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44520/0 (socket says 172.21.2.104:44520) 2021-07-11T04:00:30.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4622ffd700 1 -- 172.21.2.104:0/4223204042 learned_addr learned my addr 172.21.2.104:0/4223204042 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:30.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 --2- 172.21.2.104:0/4223204042 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240aad80 0x7f4624067300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46227fc700 1 -- 172.21.2.104:0/4223204042 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346099285 0 0) 0x7f4624065960 con 0x7f46240ab160 2021-07-11T04:00:30.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46227fc700 1 -- 172.21.2.104:0/4223204042 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4610003260 con 0x7f46240ab160 2021-07-11T04:00:30.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46227fc700 1 -- 172.21.2.104:0/4223204042 <== mon.1 v1:172.21.2.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338469611 0 0) 0x7f4624064f10 con 0x7f46240b4d50 2021-07-11T04:00:30.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46227fc700 1 -- 172.21.2.104:0/4223204042 --> v1:172.21.2.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4624065960 con 0x7f46240b4d50 2021-07-11T04:00:30.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 -- 172.21.2.104:0/4223204042 >> v1:172.21.2.105:6789/0 conn(0x7f46240b4d50 legacy=0x7f46240b7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 -- 172.21.2.104:0/4223204042 >> v1:172.21.2.104:6789/0 conn(0x7f46240ab160 legacy=0x7f4624064800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 -- 172.21.2.104:0/4223204042 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4624066e60 con 0x7f46240aad80 2021-07-11T04:00:30.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 --2- 172.21.2.104:0/4223204042 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240aad80 0x7f4624067300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f461c0032f0 tx=0x7f461c0091d0).ready entity=mon.2 client_cookie=45111230333f1f51 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46227fc700 1 -- 172.21.2.104:0/4223204042 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f461c00b640 con 0x7f46240aad80 2021-07-11T04:00:30.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46227fc700 1 -- 172.21.2.104:0/4223204042 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c00b7a0 con 0x7f46240aad80 2021-07-11T04:00:30.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46227fc700 1 -- 172.21.2.104:0/4223204042 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f461c00b910 con 0x7f46240aad80 2021-07-11T04:00:30.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- 172.21.2.104:0/4223204042 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240aad80 msgr2=0x7f4624067300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 --2- 172.21.2.104:0/4223204042 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240aad80 0x7f4624067300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f461c0032f0 tx=0x7f461c0091d0).stop 2021-07-11T04:00:30.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- 172.21.2.104:0/4223204042 shutdown_connections 2021-07-11T04:00:30.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 --2- 172.21.2.104:0/4223204042 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240aad80 0x7f4624067300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- 172.21.2.104:0/4223204042 >> 172.21.2.104:0/4223204042 conn(0x7f4624062be0 msgr2=0x7f462405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:30.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- 172.21.2.104:0/4223204042 shutdown_connections 2021-07-11T04:00:30.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- 172.21.2.104:0/4223204042 wait complete. 2021-07-11T04:00:30.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 Processor -- start 2021-07-11T04:00:30.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- start start 2021-07-11T04:00:30.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46240aad80 0x7f46240af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240ab160 0x7f46240b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46240b4d50 0x7f46240b7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46240b2000 con 0x7f46240aad80 2021-07-11T04:00:30.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46240b0770 con 0x7f46240b4d50 2021-07-11T04:00:30.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46240b1ac0 con 0x7f46240ab160 2021-07-11T04:00:30.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46240aad80 0x7f46240af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46240aad80 0x7f46240af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47754/0 (socket says 172.21.2.104:47754) 2021-07-11T04:00:30.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 -- 172.21.2.104:0/3353380294 learned_addr learned my addr 172.21.2.104:0/3353380294 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:30.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4622ffd700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240ab160 0x7f46240b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4623fff700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46240b4d50 0x7f46240b7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 -- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240ab160 msgr2=0x7f46240b7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240ab160 0x7f46240b7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 -- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46240b4d50 msgr2=0x7f46240b7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:30.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46240b4d50 0x7f46240b7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:30.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 -- 172.21.2.104:0/3353380294 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c009470 con 0x7f46240aad80 2021-07-11T04:00:30.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4622ffd700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240ab160 0x7f46240b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:00:30.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f46237fe700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46240aad80 0x7f46240af000 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f461c0032f0 tx=0x7f461c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4620ff9700 1 -- 172.21.2.104:0/3353380294 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f461c011e20 con 0x7f46240aad80 2021-07-11T04:00:30.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4620ff9700 1 -- 172.21.2.104:0/3353380294 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c021440 con 0x7f46240aad80 2021-07-11T04:00:30.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- 172.21.2.104:0/3353380294 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46240b1d60 con 0x7f46240aad80 2021-07-11T04:00:30.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4620ff9700 1 -- 172.21.2.104:0/3353380294 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f461c0217a0 con 0x7f46240aad80 2021-07-11T04:00:30.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.922+0000 7f4628cd1700 1 -- 172.21.2.104:0/3353380294 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46240b8100 con 0x7f46240aad80 2021-07-11T04:00:30.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.926+0000 7f4628cd1700 1 -- 172.21.2.104:0/3353380294 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46240af690 con 0x7f46240aad80 2021-07-11T04:00:30.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.926+0000 7f4620ff9700 1 -- 172.21.2.104:0/3353380294 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f461c027020 con 0x7f46240aad80 2021-07-11T04:00:30.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.926+0000 7f4620ff9700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46040531f0 0x7f46040556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:30.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.926+0000 7f4620ff9700 1 -- 172.21.2.104:0/3353380294 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f461c06c4c0 con 0x7f46240aad80 2021-07-11T04:00:30.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.926+0000 7f4622ffd700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46040531f0 0x7f46040556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:30.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.926+0000 7f4622ffd700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46040531f0 0x7f46040556b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f461800b000 tx=0x7f4618009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:30.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:30.926+0000 7f4620ff9700 1 -- 172.21.2.104:0/3353380294 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f461c0439d0 con 0x7f46240aad80 2021-07-11T04:00:31.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.018+0000 7f4628cd1700 1 -- 172.21.2.104:0/3353380294 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f4624057350 con 0x7f46240aad80 2021-07-11T04:00:31.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.942+0000 7f4620ff9700 1 -- 172.21.2.104:0/3353380294 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v93) v1 ==== 97+0+0 (secure 0 0 0) 0x7f461c046ff0 con 0x7f46240aad80 2021-07-11T04:00:31.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 -- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46040531f0 msgr2=0x7f46040556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:31.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46040531f0 0x7f46040556b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f461800b000 tx=0x7f4618009040).stop 2021-07-11T04:00:31.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 -- 172.21.2.104:0/3353380294 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46240aad80 msgr2=0x7f46240af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:31.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46240aad80 0x7f46240af000 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f461c0032f0 tx=0x7f461c016000).stop 2021-07-11T04:00:31.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 -- 172.21.2.104:0/3353380294 shutdown_connections 2021-07-11T04:00:31.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46240b4d50 0x7f46240b7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:31.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46240ab160 0x7f46240b7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:31.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46040531f0 0x7f46040556b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:31.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 --2- 172.21.2.104:0/3353380294 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46240aad80 0x7f46240af000 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:31.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 -- 172.21.2.104:0/3353380294 >> 172.21.2.104:0/3353380294 conn(0x7f4624062be0 msgr2=0x7f46241418c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:31.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 -- 172.21.2.104:0/3353380294 shutdown_connections 2021-07-11T04:00:31.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:31.946+0000 7f4628cd1700 1 -- 172.21.2.104:0/3353380294 wait complete. 2021-07-11T04:00:31.954 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T04:00:31.978 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T04:00:31.979 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T04:00:32.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2237ed3700 1 Processor -- start 2021-07-11T04:00:32.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2237ed3700 1 -- start start 2021-07-11T04:00:32.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2237ed3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f22300b1db0 0x7f22300b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2237ed3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22301483a0 0x7f22300b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2237ed3700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22300b0350 con 0x7f22300a70c0 2021-07-11T04:00:32.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2237ed3700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22300a90c0 con 0x7f22300b1db0 2021-07-11T04:00:32.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2237ed3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b05f0 con 0x7f22301483a0 2021-07-11T04:00:32.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2236470700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f22300a70c0 0x7f2230144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44534/0 (socket says 172.21.2.104:44534) 2021-07-11T04:00:32.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2236470700 1 -- 172.21.2.104:0/3762571495 learned_addr learned my addr 172.21.2.104:0/3762571495 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:32.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2235c6f700 1 --2- 172.21.2.104:0/3762571495 >> v2:172.21.2.104:3300/0 conn(0x7f22300b1db0 0x7f22300b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2235c6f700 1 -- 172.21.2.104:0/3762571495 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22301483a0 msgr2=0x7f22300b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2235c6f700 1 --2- 172.21.2.104:0/3762571495 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22301483a0 0x7f22300b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2235c6f700 1 -- 172.21.2.104:0/3762571495 >> v1:172.21.2.104:6789/0 conn(0x7f22300a70c0 legacy=0x7f2230144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2235c6f700 1 -- 172.21.2.104:0/3762571495 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300b0890 con 0x7f22300b1db0 2021-07-11T04:00:32.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2235c6f700 1 --2- 172.21.2.104:0/3762571495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22300b1db0 0x7f22300b0fd0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f222c002f70 tx=0x7f222c006df0).ready entity=mon.0 client_cookie=bee1ad03a8c7e025 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2234c6d700 1 -- 172.21.2.104:0/3762571495 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f222c00d570 con 0x7f22300b1db0 2021-07-11T04:00:32.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2234c6d700 1 -- 172.21.2.104:0/3762571495 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c000e90 con 0x7f22300b1db0 2021-07-11T04:00:32.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2234c6d700 1 -- 172.21.2.104:0/3762571495 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f222c00d8c0 con 0x7f22300b1db0 2021-07-11T04:00:32.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2237ed3700 1 -- 172.21.2.104:0/3762571495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22300b1db0 msgr2=0x7f22300b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.070+0000 7f2237ed3700 1 --2- 172.21.2.104:0/3762571495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22300b1db0 0x7f22300b0fd0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f222c002f70 tx=0x7f222c006df0).stop 2021-07-11T04:00:32.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 -- 172.21.2.104:0/3762571495 shutdown_connections 2021-07-11T04:00:32.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 --2- 172.21.2.104:0/3762571495 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22301483a0 0x7f22300b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 --2- 172.21.2.104:0/3762571495 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22300b1db0 0x7f22300b0fd0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 -- 172.21.2.104:0/3762571495 >> 172.21.2.104:0/3762571495 conn(0x7f223005f6b0 msgr2=0x7f223005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:32.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 -- 172.21.2.104:0/3762571495 shutdown_connections 2021-07-11T04:00:32.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 -- 172.21.2.104:0/3762571495 wait complete. 2021-07-11T04:00:32.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 Processor -- start 2021-07-11T04:00:32.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 -- start start 2021-07-11T04:00:32.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22300a70c0 0x7f2230140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22300b1db0 0x7f2230148f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22301483a0 0x7f2230149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2230149910 con 0x7f22300b1db0 2021-07-11T04:00:32.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2230148b00 con 0x7f22301483a0 2021-07-11T04:00:32.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223013f280 con 0x7f22300a70c0 2021-07-11T04:00:32.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f223546e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22300b1db0 0x7f2230148f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f223546e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22300b1db0 0x7f2230148f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47776/0 (socket says 172.21.2.104:47776) 2021-07-11T04:00:32.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f223546e700 1 -- 172.21.2.104:0/3064981057 learned_addr learned my addr 172.21.2.104:0/3064981057 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:32.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f223546e700 1 -- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22300a70c0 msgr2=0x7f2230140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f223546e700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22300a70c0 0x7f2230140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f223546e700 1 -- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22301483a0 msgr2=0x7f2230149420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f223546e700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22301483a0 0x7f2230149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f223546e700 1 -- 172.21.2.104:0/3064981057 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c00b180 con 0x7f22300b1db0 2021-07-11T04:00:32.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f223546e700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22300b1db0 0x7f2230148f30 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7f2224002f70 tx=0x7f2224006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2222ffd700 1 -- 172.21.2.104:0/3064981057 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2224013460 con 0x7f22300b1db0 2021-07-11T04:00:32.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2222ffd700 1 -- 172.21.2.104:0/3064981057 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22240135c0 con 0x7f22300b1db0 2021-07-11T04:00:32.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 -- 172.21.2.104:0/3064981057 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223013f520 con 0x7f22300b1db0 2021-07-11T04:00:32.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2222ffd700 1 -- 172.21.2.104:0/3064981057 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2224013920 con 0x7f22300b1db0 2021-07-11T04:00:32.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 -- 172.21.2.104:0/3064981057 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223014b7a0 con 0x7f22300b1db0 2021-07-11T04:00:32.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2237ed3700 1 -- 172.21.2.104:0/3064981057 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2230148c40 con 0x7f22300b1db0 2021-07-11T04:00:32.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2222ffd700 1 -- 172.21.2.104:0/3064981057 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f222401a060 con 0x7f22300b1db0 2021-07-11T04:00:32.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2222ffd700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2210053380 0x7f2210055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.074+0000 7f2222ffd700 1 -- 172.21.2.104:0/3064981057 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f222406ac10 con 0x7f22300b1db0 2021-07-11T04:00:32.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.078+0000 7f2235c6f700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2210053380 0x7f2210055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.078+0000 7f2235c6f700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2210053380 0x7f2210055840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f222c002f70 tx=0x7f222c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.078+0000 7f2222ffd700 1 -- 172.21.2.104:0/3064981057 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2224042170 con 0x7f22300b1db0 2021-07-11T04:00:32.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.186+0000 7f2237ed3700 1 -- 172.21.2.104:0/3064981057 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2230057350 con 0x7f22300b1db0 2021-07-11T04:00:32.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.186+0000 7f2222ffd700 1 -- 172.21.2.104:0/3064981057 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f2224018070 con 0x7f22300b1db0 2021-07-11T04:00:32.192 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:32.192 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:00:32.192 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T04:00:32.193 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.193 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T04:00:32.193 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T04:00:32.193 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.194 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:32.194 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.194 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.196 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.196 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.196 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.196 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.197 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T04:00:32.197 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T04:00:32.197 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.197 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:32.198 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.198 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.198 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.198 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.199 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.199 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.199 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T04:00:32.199 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T04:00:32.199 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.200 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:32.200 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.200 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.200 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.201 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.201 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.201 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.201 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T04:00:32.201 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T04:00:32.202 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.202 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.202 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.204 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.205 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.206 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.206 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.206 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T04:00:32.206 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T04:00:32.207 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.207 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.207 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.207 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.208 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.208 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.208 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.209 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T04:00:32.209 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T04:00:32.209 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.209 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.210 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.210 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.211 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.211 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.211 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.211 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T04:00:32.212 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T04:00:32.212 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.212 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.212 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.213 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.213 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.213 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.213 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.213 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T04:00:32.214 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T04:00:32.214 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.214 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.214 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.215 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.215 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.215 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.215 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.216 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T04:00:32.216 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T04:00:32.216 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.216 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.216 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.217 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.217 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.217 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.217 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.217 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T04:00:32.218 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T04:00:32.218 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.218 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.219 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.219 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.219 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.219 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.220 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.220 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T04:00:32.220 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T04:00:32.220 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.221 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.221 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.221 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.221 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.221 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.222 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.222 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T04:00:32.222 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T04:00:32.222 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.223 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:32.223 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:00:32.223 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:00:32.223 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.223 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.224 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.224 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T04:00:32.224 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T04:00:32.224 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.224 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:32.225 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:00:32.225 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:00:32.225 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.225 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.225 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.226 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T04:00:32.226 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T04:00:32.226 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.226 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:00:32.227 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:32.227 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:32.227 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.227 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.227 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.228 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.228 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T04:00:32.228 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T04:00:32.228 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.228 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:00:32.229 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:00:32.229 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.229 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.229 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.230 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.230 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T04:00:32.230 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T04:00:32.230 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:32.230 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:00:32.231 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:00:32.231 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:32.231 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.231 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.231 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:32.232 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:00:32.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 -- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2210053380 msgr2=0x7f2210055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2210053380 0x7f2210055840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f222c002f70 tx=0x7f222c015000).stop 2021-07-11T04:00:32.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 -- 172.21.2.104:0/3064981057 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22300b1db0 msgr2=0x7f2230148f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22300b1db0 0x7f2230148f30 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7f2224002f70 tx=0x7f2224006df0).stop 2021-07-11T04:00:32.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 -- 172.21.2.104:0/3064981057 shutdown_connections 2021-07-11T04:00:32.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22301483a0 0x7f2230149420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2210053380 0x7f2210055840 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22300b1db0 0x7f2230148f30 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 --2- 172.21.2.104:0/3064981057 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22300a70c0 0x7f2230140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 -- 172.21.2.104:0/3064981057 >> 172.21.2.104:0/3064981057 conn(0x7f223005f6b0 msgr2=0x7f2230147770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:32.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 -- 172.21.2.104:0/3064981057 shutdown_connections 2021-07-11T04:00:32.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.190+0000 7f2237ed3700 1 -- 172.21.2.104:0/3064981057 wait complete. 2021-07-11T04:00:32.236 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:32.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 Processor -- start 2021-07-11T04:00:32.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 -- start start 2021-07-11T04:00:32.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f612c0b1170 0x7f612c141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612c0b31d0 con 0x7f612c0b7810 2021-07-11T04:00:32.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612c0b7bf0 con 0x7f612c141920 2021-07-11T04:00:32.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0675c0 con 0x7f612c0b1170 2021-07-11T04:00:32.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f613102c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f612c0b7810 0x7f612c1445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44548/0 (socket says 172.21.2.104:44548) 2021-07-11T04:00:32.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f613102c700 1 -- 172.21.2.104:0/1325874833 learned_addr learned my addr 172.21.2.104:0/1325874833 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:32.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89823807 0 0) 0x7f612c0b31d0 con 0x7f612c0b7810 2021-07-11T04:00:32.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6114003260 con 0x7f612c0b7810 2021-07-11T04:00:32.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3482644634 0 0) 0x7f6114003260 con 0x7f612c0b7810 2021-07-11T04:00:32.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f612c0b1170 msgr2=0x7f612c141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 --2- 172.21.2.104:0/1325874833 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f612c0b1170 0x7f612c141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 >> v1:172.21.2.105:6789/0 conn(0x7f612c141920 legacy=0x7f612c14bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c0aae70 con 0x7f612c0b7810 2021-07-11T04:00:32.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f6128001660 con 0x7f612c0b7810 2021-07-11T04:00:32.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 >> v1:172.21.2.104:6789/0 conn(0x7f612c0b7810 legacy=0x7f612c1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 --2- 172.21.2.104:0/1325874833 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6114003800 0x7f6114003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 --2- 172.21.2.104:0/1325874833 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6114004500 0x7f6114004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 --2- 172.21.2.104:0/1325874833 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f612c0b1170 0x7f611400fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0675c0 con 0x7f6114004500 2021-07-11T04:00:32.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61140052e0 con 0x7f6114003800 2021-07-11T04:00:32.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6114010050 con 0x7f612c0b1170 2021-07-11T04:00:32.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f613082b700 1 --2- 172.21.2.104:0/1325874833 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6114004500 0x7f6114004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f613082b700 1 -- 172.21.2.104:0/1325874833 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f612c0b1170 msgr2=0x7f611400fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f613082b700 1 --2- 172.21.2.104:0/1325874833 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f612c0b1170 0x7f611400fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f613082b700 1 -- 172.21.2.104:0/1325874833 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6114003800 msgr2=0x7f6114003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f613082b700 1 --2- 172.21.2.104:0/1325874833 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6114003800 0x7f6114003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f613082b700 1 -- 172.21.2.104:0/1325874833 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6114003260 con 0x7f6114004500 2021-07-11T04:00:32.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f613082b700 1 --2- 172.21.2.104:0/1325874833 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6114004500 0x7f6114004980 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f6120002f70 tx=0x7f6120006ca0).ready entity=mon.0 client_cookie=9799148527c2a623 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6120012410 con 0x7f6114004500 2021-07-11T04:00:32.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6120012570 con 0x7f6114004500 2021-07-11T04:00:32.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f611ffff700 1 -- 172.21.2.104:0/1325874833 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61200128d0 con 0x7f6114004500 2021-07-11T04:00:32.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 -- 172.21.2.104:0/1325874833 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6114004500 msgr2=0x7f6114004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 --2- 172.21.2.104:0/1325874833 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6114004500 0x7f6114004980 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f6120002f70 tx=0x7f6120006ca0).stop 2021-07-11T04:00:32.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f613182d700 1 -- 172.21.2.104:0/1325874833 reap_dead start 2021-07-11T04:00:32.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 -- 172.21.2.104:0/1325874833 shutdown_connections 2021-07-11T04:00:32.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 -- 172.21.2.104:0/1325874833 >> 172.21.2.104:0/1325874833 conn(0x7f612c05f6b0 msgr2=0x7f612c05fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:32.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 -- 172.21.2.104:0/1325874833 shutdown_connections 2021-07-11T04:00:32.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 -- 172.21.2.104:0/1325874833 wait complete. 2021-07-11T04:00:32.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 Processor -- start 2021-07-11T04:00:32.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.290+0000 7f6133290700 1 -- start start 2021-07-11T04:00:32.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f6133290700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f612c0ab7b0 0x7f612c15a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f6133290700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f612c0a7dc0 0x7f612c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f6133290700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f612c142cd0 0x7f612c143140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f6133290700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0a7b40 con 0x7f612c142cd0 2021-07-11T04:00:32.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f6133290700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0abb90 con 0x7f612c0a7dc0 2021-07-11T04:00:32.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f6133290700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0b66d0 con 0x7f612c0ab7b0 2021-07-11T04:00:32.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613182d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f612c142cd0 0x7f612c143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613182d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f612c142cd0 0x7f612c143140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47796/0 (socket says 172.21.2.104:47796) 2021-07-11T04:00:32.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613182d700 1 -- 172.21.2.104:0/3867137802 learned_addr learned my addr 172.21.2.104:0/3867137802 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:32.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613082b700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f612c0a7dc0 0x7f612c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613102c700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f612c0ab7b0 0x7f612c15a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613182d700 1 -- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f612c0ab7b0 msgr2=0x7f612c15a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613182d700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f612c0ab7b0 0x7f612c15a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613182d700 1 -- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f612c0a7dc0 msgr2=0x7f612c15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613182d700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f612c0a7dc0 0x7f612c15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613182d700 1 -- 172.21.2.104:0/3867137802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c0b6970 con 0x7f612c142cd0 2021-07-11T04:00:32.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613182d700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f612c142cd0 0x7f612c143140 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f6128001370 tx=0x7f6124007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f611e7fc700 1 -- 172.21.2.104:0/3867137802 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6124004850 con 0x7f612c142cd0 2021-07-11T04:00:32.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f611e7fc700 1 -- 172.21.2.104:0/3867137802 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61240049b0 con 0x7f612c142cd0 2021-07-11T04:00:32.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f611e7fc700 1 -- 172.21.2.104:0/3867137802 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6124004d50 con 0x7f612c142cd0 2021-07-11T04:00:32.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f6133290700 1 -- 172.21.2.104:0/3867137802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612c159e10 con 0x7f612c142cd0 2021-07-11T04:00:32.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f6133290700 1 -- 172.21.2.104:0/3867137802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c0aa560 con 0x7f612c142cd0 2021-07-11T04:00:32.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f6133290700 1 -- 172.21.2.104:0/3867137802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612c159fa0 con 0x7f612c142cd0 2021-07-11T04:00:32.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f611e7fc700 1 -- 172.21.2.104:0/3867137802 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f612401c070 con 0x7f612c142cd0 2021-07-11T04:00:32.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f611e7fc700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f610c053260 0x7f610c055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f611e7fc700 1 -- 172.21.2.104:0/3867137802 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f612406a3b0 con 0x7f612c142cd0 2021-07-11T04:00:32.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.294+0000 7f613102c700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f610c053260 0x7f610c055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.298+0000 7f613102c700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f610c053260 0x7f610c055720 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f612800b000 tx=0x7f612c066a60).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.298+0000 7f611e7fc700 1 -- 172.21.2.104:0/3867137802 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6124041910 con 0x7f612c142cd0 2021-07-11T04:00:32.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.378+0000 7f6133290700 1 -- 172.21.2.104:0/3867137802 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f612c057350 con 0x7f612c142cd0 2021-07-11T04:00:32.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f611e7fc700 1 -- 172.21.2.104:0/3867137802 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+13182 (secure 0 0 0) 0x7f6124044f30 con 0x7f612c142cd0 2021-07-11T04:00:32.384 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:32.384 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":93,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:31.933940+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:32.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 -- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f610c053260 msgr2=0x7f610c055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f610c053260 0x7f610c055720 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f612800b000 tx=0x7f612c066a60).stop 2021-07-11T04:00:32.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 -- 172.21.2.104:0/3867137802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f612c142cd0 msgr2=0x7f612c143140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f612c142cd0 0x7f612c143140 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f6128001370 tx=0x7f6124007dc0).stop 2021-07-11T04:00:32.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 -- 172.21.2.104:0/3867137802 shutdown_connections 2021-07-11T04:00:32.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f610c053260 0x7f610c055720 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f612c142cd0 0x7f612c143140 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f612c0a7dc0 0x7f612c15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 --2- 172.21.2.104:0/3867137802 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f612c0ab7b0 0x7f612c15a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 -- 172.21.2.104:0/3867137802 >> 172.21.2.104:0/3867137802 conn(0x7f612c05f6b0 msgr2=0x7f612c05bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:32.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 -- 172.21.2.104:0/3867137802 shutdown_connections 2021-07-11T04:00:32.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.382+0000 7f6133290700 1 -- 172.21.2.104:0/3867137802 wait complete. 2021-07-11T04:00:32.392 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T04:00:32.392 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:00:32.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.442+0000 7fcd605fd700 1 Processor -- start 2021-07-11T04:00:32.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.442+0000 7fcd605fd700 1 -- start start 2021-07-11T04:00:32.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.442+0000 7fcd605fd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd5813c9c0 0x7fcd580b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.442+0000 7fcd605fd700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd580b5f40 con 0x7fcd5805d760 2021-07-11T04:00:32.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.442+0000 7fcd605fd700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd580ad420 con 0x7fcd580ad040 2021-07-11T04:00:32.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.442+0000 7fcd605fd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580aeb60 con 0x7fcd5813c9c0 2021-07-11T04:00:32.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.442+0000 7fcd5eb9a700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fcd5805d760 0x7fcd5813f070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44572/0 (socket says 172.21.2.104:44572) 2021-07-11T04:00:32.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.442+0000 7fcd5eb9a700 1 -- 172.21.2.104:0/3915031867 learned_addr learned my addr 172.21.2.104:0/3915031867 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:32.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1137412822 0 0) 0x7fcd580b5f40 con 0x7fcd5805d760 2021-07-11T04:00:32.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd40003260 con 0x7fcd5805d760 2021-07-11T04:00:32.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2167692462 0 0) 0x7fcd40003260 con 0x7fcd5805d760 2021-07-11T04:00:32.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd5813c9c0 msgr2=0x7fcd580b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 --2- 172.21.2.104:0/3915031867 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd5813c9c0 0x7fcd580b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 >> v1:172.21.2.105:6789/0 conn(0x7fcd580ad040 legacy=0x7fcd5813c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580aee00 con 0x7fcd5805d760 2021-07-11T04:00:32.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fcd500019e0 con 0x7fcd5805d760 2021-07-11T04:00:32.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 >> v1:172.21.2.104:6789/0 conn(0x7fcd5805d760 legacy=0x7fcd5813f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 --2- 172.21.2.104:0/3915031867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd40003800 0x7fcd40003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 --2- 172.21.2.104:0/3915031867 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd5813c9c0 0x7fcd40004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 --2- 172.21.2.104:0/3915031867 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd40004bf0 0x7fcd4000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd580aeb60 con 0x7fcd40003800 2021-07-11T04:00:32.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd40005250 con 0x7fcd40004bf0 2021-07-11T04:00:32.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000bc40 con 0x7fcd5813c9c0 2021-07-11T04:00:32.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5e399700 1 --2- 172.21.2.104:0/3915031867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd40003800 0x7fcd40003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5e399700 1 -- 172.21.2.104:0/3915031867 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd5813c9c0 msgr2=0x7fcd40004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5e399700 1 --2- 172.21.2.104:0/3915031867 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd5813c9c0 0x7fcd40004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5e399700 1 -- 172.21.2.104:0/3915031867 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd40004bf0 msgr2=0x7fcd4000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5e399700 1 --2- 172.21.2.104:0/3915031867 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd40004bf0 0x7fcd4000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5e399700 1 -- 172.21.2.104:0/3915031867 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd580aee00 con 0x7fcd40003800 2021-07-11T04:00:32.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5e399700 1 --2- 172.21.2.104:0/3915031867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd40003800 0x7fcd40003c00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fcd54000f70 tx=0x7fcd540079e0).ready entity=mon.0 client_cookie=9d99b53e362fccc5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd54007cf0 con 0x7fcd40003800 2021-07-11T04:00:32.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd54014410 con 0x7fcd40003800 2021-07-11T04:00:32.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5d397700 1 -- 172.21.2.104:0/3915031867 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd54014770 con 0x7fcd40003800 2021-07-11T04:00:32.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 -- 172.21.2.104:0/3915031867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd40003800 msgr2=0x7fcd40003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 --2- 172.21.2.104:0/3915031867 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd40003800 0x7fcd40003c00 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fcd54000f70 tx=0x7fcd540079e0).stop 2021-07-11T04:00:32.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5eb9a700 1 -- 172.21.2.104:0/3915031867 reap_dead start 2021-07-11T04:00:32.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 -- 172.21.2.104:0/3915031867 shutdown_connections 2021-07-11T04:00:32.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 -- 172.21.2.104:0/3915031867 >> 172.21.2.104:0/3915031867 conn(0x7fcd5805c490 msgr2=0x7fcd5805c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:32.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 -- 172.21.2.104:0/3915031867 shutdown_connections 2021-07-11T04:00:32.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 -- 172.21.2.104:0/3915031867 wait complete. 2021-07-11T04:00:32.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 Processor -- start 2021-07-11T04:00:32.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 -- start start 2021-07-11T04:00:32.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd58137800 0x7fcd5813d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd5815a7f0 0x7fcd581565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd5813d5d0 0x7fcd58156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5813fdf0 con 0x7fcd5815a7f0 2021-07-11T04:00:32.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58137be0 con 0x7fcd5813d5d0 2021-07-11T04:00:32.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58159b70 con 0x7fcd58137800 2021-07-11T04:00:32.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5db98700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd5815a7f0 0x7fcd581565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5db98700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd5815a7f0 0x7fcd581565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47816/0 (socket says 172.21.2.104:47816) 2021-07-11T04:00:32.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5db98700 1 -- 172.21.2.104:0/2054495320 learned_addr learned my addr 172.21.2.104:0/2054495320 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:32.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5db98700 1 -- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd58137800 msgr2=0x7fcd5813d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5db98700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd58137800 0x7fcd5813d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5db98700 1 -- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd5813d5d0 msgr2=0x7fcd58156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5db98700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd5813d5d0 0x7fcd58156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5db98700 1 -- 172.21.2.104:0/2054495320 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd58159e10 con 0x7fcd5815a7f0 2021-07-11T04:00:32.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd5db98700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd5815a7f0 0x7fcd581565a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7fcd48002f70 tx=0x7fcd48006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd477fe700 1 -- 172.21.2.104:0/2054495320 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd4800e470 con 0x7fcd5815a7f0 2021-07-11T04:00:32.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd477fe700 1 -- 172.21.2.104:0/2054495320 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd48000e90 con 0x7fcd5815a7f0 2021-07-11T04:00:32.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd477fe700 1 -- 172.21.2.104:0/2054495320 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd4800e7c0 con 0x7fcd5815a7f0 2021-07-11T04:00:32.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 -- 172.21.2.104:0/2054495320 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5815a0b0 con 0x7fcd5815a7f0 2021-07-11T04:00:32.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.446+0000 7fcd605fd700 1 -- 172.21.2.104:0/2054495320 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd58141a60 con 0x7fcd5815a7f0 2021-07-11T04:00:32.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.450+0000 7fcd605fd700 1 -- 172.21.2.104:0/2054495320 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd58057350 con 0x7fcd5815a7f0 2021-07-11T04:00:32.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.450+0000 7fcd477fe700 1 -- 172.21.2.104:0/2054495320 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcd4800f070 con 0x7fcd5815a7f0 2021-07-11T04:00:32.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.450+0000 7fcd477fe700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd38053210 0x7fcd380556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.450+0000 7fcd477fe700 1 -- 172.21.2.104:0/2054495320 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcd4806a500 con 0x7fcd5815a7f0 2021-07-11T04:00:32.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.450+0000 7fcd5e399700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd38053210 0x7fcd380556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.450+0000 7fcd5e399700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd38053210 0x7fcd380556d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fcd54000f70 tx=0x7fcd54015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.450+0000 7fcd477fe700 1 -- 172.21.2.104:0/2054495320 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcd48041b70 con 0x7fcd5815a7f0 2021-07-11T04:00:32.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.534+0000 7fcd605fd700 1 -- 172.21.2.104:0/2054495320 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcd580ac990 con 0x7fcd5815a7f0 2021-07-11T04:00:32.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.534+0000 7fcd477fe700 1 -- 172.21.2.104:0/2054495320 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v93) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcd48045190 con 0x7fcd5815a7f0 2021-07-11T04:00:32.538 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:00:32.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.534+0000 7fcd605fd700 1 -- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd38053210 msgr2=0x7fcd380556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.534+0000 7fcd605fd700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd38053210 0x7fcd380556d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fcd54000f70 tx=0x7fcd54015000).stop 2021-07-11T04:00:32.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.534+0000 7fcd605fd700 1 -- 172.21.2.104:0/2054495320 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd5815a7f0 msgr2=0x7fcd581565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.534+0000 7fcd605fd700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd5815a7f0 0x7fcd581565a0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7fcd48002f70 tx=0x7fcd48006e40).stop 2021-07-11T04:00:32.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.538+0000 7fcd605fd700 1 -- 172.21.2.104:0/2054495320 shutdown_connections 2021-07-11T04:00:32.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.538+0000 7fcd605fd700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd5813d5d0 0x7fcd58156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.538+0000 7fcd605fd700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd38053210 0x7fcd380556d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.538+0000 7fcd605fd700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd5815a7f0 0x7fcd581565a0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.538+0000 7fcd605fd700 1 --2- 172.21.2.104:0/2054495320 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd58137800 0x7fcd5813d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.538+0000 7fcd605fd700 1 -- 172.21.2.104:0/2054495320 >> 172.21.2.104:0/2054495320 conn(0x7fcd5805c490 msgr2=0x7fcd5814e080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:32.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.538+0000 7fcd605fd700 1 -- 172.21.2.104:0/2054495320 shutdown_connections 2021-07-11T04:00:32.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.538+0000 7fcd605fd700 1 -- 172.21.2.104:0/2054495320 wait complete. 2021-07-11T04:00:32.545 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:00:32.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.594+0000 7ff37bb64700 1 Processor -- start 2021-07-11T04:00:32.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.594+0000 7ff37bb64700 1 -- start start 2021-07-11T04:00:32.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.594+0000 7ff37bb64700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff374141960 0x7ff3740affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.594+0000 7ff37bb64700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3740a77d0 con 0x7ff3740b12d0 2021-07-11T04:00:32.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.594+0000 7ff37bb64700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3740b0650 con 0x7ff3740b1ec0 2021-07-11T04:00:32.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.594+0000 7ff37bb64700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3740b1740 con 0x7ff374141960 2021-07-11T04:00:32.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3790ff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff3740b12d0 0x7ff374141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44592/0 (socket says 172.21.2.104:44592) 2021-07-11T04:00:32.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3790ff700 1 -- 172.21.2.104:0/3608177297 learned_addr learned my addr 172.21.2.104:0/3608177297 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:32.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395865827 0 0) 0x7ff3740a77d0 con 0x7ff3740b12d0 2021-07-11T04:00:32.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff35c003260 con 0x7ff3740b12d0 2021-07-11T04:00:32.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1021834726 0 0) 0x7ff35c003260 con 0x7ff3740b12d0 2021-07-11T04:00:32.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 >> v2:172.21.2.105:3300/0 conn(0x7ff374141960 msgr2=0x7ff3740affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 --2- 172.21.2.104:0/3608177297 >> v2:172.21.2.105:3300/0 conn(0x7ff374141960 0x7ff3740affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 >> v1:172.21.2.105:6789/0 conn(0x7ff3740b1ec0 legacy=0x7ff3740a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3740b08f0 con 0x7ff3740b12d0 2021-07-11T04:00:32.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7ff370001660 con 0x7ff3740b12d0 2021-07-11T04:00:32.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 >> v1:172.21.2.104:6789/0 conn(0x7ff3740b12d0 legacy=0x7ff374141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 --2- 172.21.2.104:0/3608177297 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff35c003750 0x7ff35c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 --2- 172.21.2.104:0/3608177297 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff35c004500 0x7ff35c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 --2- 172.21.2.104:0/3608177297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff35c004f90 0x7ff35c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740b1740 con 0x7ff35c004f90 2021-07-11T04:00:32.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c00be80 con 0x7ff35c003750 2021-07-11T04:00:32.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c00c040 con 0x7ff35c004500 2021-07-11T04:00:32.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37a101700 1 --2- 172.21.2.104:0/3608177297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff35c004f90 0x7ff35c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37a101700 1 -- 172.21.2.104:0/3608177297 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff35c004500 msgr2=0x7ff35c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37a101700 1 --2- 172.21.2.104:0/3608177297 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff35c004500 0x7ff35c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37a101700 1 -- 172.21.2.104:0/3608177297 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff35c003750 msgr2=0x7ff35c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37a101700 1 --2- 172.21.2.104:0/3608177297 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff35c003750 0x7ff35c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37a101700 1 -- 172.21.2.104:0/3608177297 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c003260 con 0x7ff35c004f90 2021-07-11T04:00:32.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37a101700 1 -- 172.21.2.104:0/3608177297 reap_dead start 2021-07-11T04:00:32.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37a101700 1 --2- 172.21.2.104:0/3608177297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff35c004f90 0x7ff35c00b750 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7ff36c0032f0 tx=0x7ff36c006f90).ready entity=mon.0 client_cookie=ae5d5c8c2d5b249 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff36c00b460 con 0x7ff35c004f90 2021-07-11T04:00:32.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c00b5c0 con 0x7ff35c004f90 2021-07-11T04:00:32.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3788fe700 1 -- 172.21.2.104:0/3608177297 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff36c00b730 con 0x7ff35c004f90 2021-07-11T04:00:32.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 -- 172.21.2.104:0/3608177297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff35c004f90 msgr2=0x7ff35c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 --2- 172.21.2.104:0/3608177297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff35c004f90 0x7ff35c00b750 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7ff36c0032f0 tx=0x7ff36c006f90).stop 2021-07-11T04:00:32.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 -- 172.21.2.104:0/3608177297 shutdown_connections 2021-07-11T04:00:32.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 --2- 172.21.2.104:0/3608177297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff35c004f90 0x7ff35c00b750 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 -- 172.21.2.104:0/3608177297 >> 172.21.2.104:0/3608177297 conn(0x7ff37405f6b0 msgr2=0x7ff37405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:32.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 -- 172.21.2.104:0/3608177297 shutdown_connections 2021-07-11T04:00:32.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 -- 172.21.2.104:0/3608177297 wait complete. 2021-07-11T04:00:32.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 Processor -- start 2021-07-11T04:00:32.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 -- start start 2021-07-11T04:00:32.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff35c004f90 0x7ff37413b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3740ab170 0x7ff3740ab900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3740abdf0 0x7ff374149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37414b110 con 0x7ff3740ab170 2021-07-11T04:00:32.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740aa920 con 0x7ff35c004f90 2021-07-11T04:00:32.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff37bb64700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37414b340 con 0x7ff3740abdf0 2021-07-11T04:00:32.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3790ff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3740ab170 0x7ff3740ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3790ff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3740ab170 0x7ff3740ab900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47832/0 (socket says 172.21.2.104:47832) 2021-07-11T04:00:32.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.598+0000 7ff3790ff700 1 -- 172.21.2.104:0/2177694037 learned_addr learned my addr 172.21.2.104:0/2177694037 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:32.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff3790ff700 1 -- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3740abdf0 msgr2=0x7ff374149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff3790ff700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3740abdf0 0x7ff374149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff3790ff700 1 -- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff35c004f90 msgr2=0x7ff37413b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff3790ff700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff35c004f90 0x7ff37413b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff3790ff700 1 -- 172.21.2.104:0/2177694037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37414b5e0 con 0x7ff3740ab170 2021-07-11T04:00:32.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff3790ff700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3740ab170 0x7ff3740ab900 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7ff3740b2f50 tx=0x7ff3740b0a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff362ffd700 1 -- 172.21.2.104:0/2177694037 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff370007170 con 0x7ff3740ab170 2021-07-11T04:00:32.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff362ffd700 1 -- 172.21.2.104:0/2177694037 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3700072d0 con 0x7ff3740ab170 2021-07-11T04:00:32.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff362ffd700 1 -- 172.21.2.104:0/2177694037 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff370004920 con 0x7ff3740ab170 2021-07-11T04:00:32.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff37bb64700 1 -- 172.21.2.104:0/2177694037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3740a9620 con 0x7ff3740ab170 2021-07-11T04:00:32.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff37bb64700 1 -- 172.21.2.104:0/2177694037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3740aabc0 con 0x7ff3740ab170 2021-07-11T04:00:32.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff37bb64700 1 -- 172.21.2.104:0/2177694037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff37414b770 con 0x7ff3740ab170 2021-07-11T04:00:32.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff362ffd700 1 -- 172.21.2.104:0/2177694037 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff37001b070 con 0x7ff3740ab170 2021-07-11T04:00:32.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff362ffd700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff3540532b0 0x7ff354055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff362ffd700 1 -- 172.21.2.104:0/2177694037 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff37006a8d0 con 0x7ff3740ab170 2021-07-11T04:00:32.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff379900700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff3540532b0 0x7ff354055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.602+0000 7ff379900700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff3540532b0 0x7ff354055770 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7ff364000f70 tx=0x7ff3640069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.606+0000 7ff362ffd700 1 -- 172.21.2.104:0/2177694037 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff370041ee0 con 0x7ff3740ab170 2021-07-11T04:00:32.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.686+0000 7ff37bb64700 1 -- 172.21.2.104:0/2177694037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff374057350 con 0x7ff3740ab170 2021-07-11T04:00:32.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.686+0000 7ff362ffd700 1 -- 172.21.2.104:0/2177694037 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v93) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff370045500 con 0x7ff3740ab170 2021-07-11T04:00:32.691 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:32.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.686+0000 7ff37bb64700 1 -- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff3540532b0 msgr2=0x7ff354055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.686+0000 7ff37bb64700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff3540532b0 0x7ff354055770 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7ff364000f70 tx=0x7ff3640069b0).stop 2021-07-11T04:00:32.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.686+0000 7ff37bb64700 1 -- 172.21.2.104:0/2177694037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3740ab170 msgr2=0x7ff3740ab900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.686+0000 7ff37bb64700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3740ab170 0x7ff3740ab900 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7ff3740b2f50 tx=0x7ff3740b0a80).stop 2021-07-11T04:00:32.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.690+0000 7ff37bb64700 1 -- 172.21.2.104:0/2177694037 shutdown_connections 2021-07-11T04:00:32.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.690+0000 7ff37bb64700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff3740abdf0 0x7ff374149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.690+0000 7ff37bb64700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff3540532b0 0x7ff354055770 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.690+0000 7ff37bb64700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff3740ab170 0x7ff3740ab900 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.690+0000 7ff37bb64700 1 --2- 172.21.2.104:0/2177694037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff35c004f90 0x7ff37413b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.690+0000 7ff37bb64700 1 -- 172.21.2.104:0/2177694037 >> 172.21.2.104:0/2177694037 conn(0x7ff37405f6b0 msgr2=0x7ff37413c600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:32.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.690+0000 7ff37bb64700 1 -- 172.21.2.104:0/2177694037 shutdown_connections 2021-07-11T04:00:32.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.690+0000 7ff37bb64700 1 -- 172.21.2.104:0/2177694037 wait complete. 2021-07-11T04:00:32.698 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T04:00:32.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.746+0000 7f33b920d700 1 Processor -- start 2021-07-11T04:00:32.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.746+0000 7f33b920d700 1 -- start start 2021-07-11T04:00:32.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f33b40b04a0 0x7f33b40add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f33b40b0880 0x7f33b41405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b40b2af0 con 0x7f33b40aa120 2021-07-11T04:00:32.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b40b2770 con 0x7f33b40b04a0 2021-07-11T04:00:32.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b40b2d20 con 0x7f33b40b0880 2021-07-11T04:00:32.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b259c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f33b40aa120 0x7f33b40ae220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44610/0 (socket says 172.21.2.104:44610) 2021-07-11T04:00:32.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b259c700 1 -- 172.21.2.104:0/3375050591 learned_addr learned my addr 172.21.2.104:0/3375050591 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:32.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b2d9d700 1 --2- 172.21.2.104:0/3375050591 >> v2:172.21.2.104:3300/0 conn(0x7f33b40b04a0 0x7f33b40add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b2d9d700 1 -- 172.21.2.104:0/3375050591 >> v2:172.21.2.105:3300/0 conn(0x7f33b40b0880 msgr2=0x7f33b41405a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:32.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b2d9d700 1 --2- 172.21.2.104:0/3375050591 >> v2:172.21.2.105:3300/0 conn(0x7f33b40b0880 0x7f33b41405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b2d9d700 1 -- 172.21.2.104:0/3375050591 >> v1:172.21.2.104:6789/0 conn(0x7f33b40aa120 legacy=0x7f33b40ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b2d9d700 1 -- 172.21.2.104:0/3375050591 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b40a9b70 con 0x7f33b40b04a0 2021-07-11T04:00:32.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b2d9d700 1 --2- 172.21.2.104:0/3375050591 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b40b04a0 0x7f33b40add30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f339c001110 tx=0x7f33a4006df0).ready entity=mon.0 client_cookie=b3b3bd516bd83394 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b1d9b700 1 -- 172.21.2.104:0/3375050591 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a400d570 con 0x7f33b40b04a0 2021-07-11T04:00:32.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b1d9b700 1 -- 172.21.2.104:0/3375050591 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4000e90 con 0x7f33b40b04a0 2021-07-11T04:00:32.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b1d9b700 1 -- 172.21.2.104:0/3375050591 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f33a400d8c0 con 0x7f33b40b04a0 2021-07-11T04:00:32.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- 172.21.2.104:0/3375050591 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b40b04a0 msgr2=0x7f33b40add30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 --2- 172.21.2.104:0/3375050591 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b40b04a0 0x7f33b40add30 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f339c001110 tx=0x7f33a4006df0).stop 2021-07-11T04:00:32.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- 172.21.2.104:0/3375050591 shutdown_connections 2021-07-11T04:00:32.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 --2- 172.21.2.104:0/3375050591 >> v2:172.21.2.105:3300/0 conn(0x7f33b40b0880 0x7f33b41405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 --2- 172.21.2.104:0/3375050591 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b40b04a0 0x7f33b40add30 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- 172.21.2.104:0/3375050591 >> 172.21.2.104:0/3375050591 conn(0x7f33b40607c0 msgr2=0x7f33b4060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:32.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- 172.21.2.104:0/3375050591 shutdown_connections 2021-07-11T04:00:32.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- 172.21.2.104:0/3375050591 wait complete. 2021-07-11T04:00:32.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 Processor -- start 2021-07-11T04:00:32.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- start start 2021-07-11T04:00:32.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33b40aa120 0x7f33b4066de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b40b04a0 0x7f33b41356f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b40b0880 0x7f33b40641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4065640 con 0x7f33b40b04a0 2021-07-11T04:00:32.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4063960 con 0x7f33b40aa120 2021-07-11T04:00:32.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b406a270 con 0x7f33b40b0880 2021-07-11T04:00:32.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b40b04a0 0x7f33b41356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b40b04a0 0x7f33b41356f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47846/0 (socket says 172.21.2.104:47846) 2021-07-11T04:00:32.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b259c700 1 -- 172.21.2.104:0/4086688191 learned_addr learned my addr 172.21.2.104:0/4086688191 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:32.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b259c700 1 -- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b40b0880 msgr2=0x7f33b40641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b259c700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b40b0880 0x7f33b40641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b259c700 1 -- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33b40aa120 msgr2=0x7f33b4066de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b259c700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33b40aa120 0x7f33b4066de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b259c700 1 -- 172.21.2.104:0/4086688191 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a400b180 con 0x7f33b40b04a0 2021-07-11T04:00:32.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b259c700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b40b04a0 0x7f33b41356f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f33b4132900 tx=0x7f339c007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33abfff700 1 -- 172.21.2.104:0/4086688191 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f339c0072e0 con 0x7f33b40b04a0 2021-07-11T04:00:32.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33abfff700 1 -- 172.21.2.104:0/4086688191 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c007440 con 0x7f33b40b04a0 2021-07-11T04:00:32.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- 172.21.2.104:0/4086688191 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b405ef30 con 0x7f33b40b04a0 2021-07-11T04:00:32.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33abfff700 1 -- 172.21.2.104:0/4086688191 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f339c005800 con 0x7f33b40b04a0 2021-07-11T04:00:32.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.750+0000 7f33b920d700 1 -- 172.21.2.104:0/4086688191 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b40666a0 con 0x7f33b40b04a0 2021-07-11T04:00:32.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.754+0000 7f33b920d700 1 -- 172.21.2.104:0/4086688191 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b4057350 con 0x7f33b40b04a0 2021-07-11T04:00:32.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.754+0000 7f33abfff700 1 -- 172.21.2.104:0/4086688191 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f339c01a060 con 0x7f33b40b04a0 2021-07-11T04:00:32.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.754+0000 7f33abfff700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3398053330 0x7f33980557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.754+0000 7f33abfff700 1 -- 172.21.2.104:0/4086688191 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f339c06ac90 con 0x7f33b40b04a0 2021-07-11T04:00:32.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.754+0000 7f33b2d9d700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3398053330 0x7f33980557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.754+0000 7f33b2d9d700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3398053330 0x7f33980557f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f339c001110 tx=0x7f33a4014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.754+0000 7f33abfff700 1 -- 172.21.2.104:0/4086688191 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f339c0422a0 con 0x7f33b40b04a0 2021-07-11T04:00:32.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.838+0000 7f33b920d700 1 -- 172.21.2.104:0/4086688191 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f33b40ad630 con 0x7f33b40b04a0 2021-07-11T04:00:32.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.838+0000 7f33abfff700 1 -- 172.21.2.104:0/4086688191 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v93) v1 ==== 84+0+143 (secure 0 0 0) 0x7f339c018030 con 0x7f33b40b04a0 2021-07-11T04:00:32.867 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:32.868 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T04:00:32.873 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.873 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T04:00:32.875 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T04:00:32.875 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:32.877 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:32.877 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T04:00:32.879 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T04:00:32.879 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:32.880 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T04:00:32.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.838+0000 7f33b920d700 1 -- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3398053330 msgr2=0x7f33980557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.842+0000 7f33b920d700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3398053330 0x7f33980557f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f339c001110 tx=0x7f33a4014000).stop 2021-07-11T04:00:32.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.842+0000 7f33b920d700 1 -- 172.21.2.104:0/4086688191 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b40b04a0 msgr2=0x7f33b41356f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.842+0000 7f33b920d700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b40b04a0 0x7f33b41356f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f33b4132900 tx=0x7f339c007bd0).stop 2021-07-11T04:00:32.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.842+0000 7f33b920d700 1 -- 172.21.2.104:0/4086688191 shutdown_connections 2021-07-11T04:00:32.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.842+0000 7f33b920d700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33b40b0880 0x7f33b40641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.842+0000 7f33b920d700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3398053330 0x7f33980557f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.842+0000 7f33b920d700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33b40b04a0 0x7f33b41356f0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.842+0000 7f33b920d700 1 --2- 172.21.2.104:0/4086688191 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33b40aa120 0x7f33b4066de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:32.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.842+0000 7f33b920d700 1 -- 172.21.2.104:0/4086688191 >> 172.21.2.104:0/4086688191 conn(0x7f33b40607c0 msgr2=0x7f33b40b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:32.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.842+0000 7f33b920d700 1 -- 172.21.2.104:0/4086688191 shutdown_connections 2021-07-11T04:00:32.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.842+0000 7f33b920d700 1 -- 172.21.2.104:0/4086688191 wait complete. 2021-07-11T04:00:32.888 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T04:00:32.889 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T04:00:32.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6653689700 1 Processor -- start 2021-07-11T04:00:32.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6653689700 1 -- start start 2021-07-11T04:00:32.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6653689700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f664c0b12d0 0x7f664c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6653689700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f664c0b22a0 0x7f664c141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6653689700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664c0a77d0 con 0x7f664c0b1ec0 2021-07-11T04:00:32.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6653689700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664c0b1740 con 0x7f664c0b12d0 2021-07-11T04:00:32.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6653689700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0b0650 con 0x7f664c0b22a0 2021-07-11T04:00:32.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6650c24700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f664c0b12d0 0x7f664c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6651c26700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f664c0b22a0 0x7f664c141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6651425700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f664c0b1ec0 0x7f664c0a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:43990/0 (socket says 172.21.2.104:43990) 2021-07-11T04:00:32.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6650c24700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f664c0b12d0 0x7f664c0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:55416/0 (socket says 172.21.2.104:55416) 2021-07-11T04:00:32.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6651425700 1 -- 172.21.2.104:0/3835648038 learned_addr learned my addr 172.21.2.104:0/3835648038 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:32.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f663ffff700 1 -- 172.21.2.104:0/3835648038 <== mon.1 v1:172.21.2.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532729064 0 0) 0x7f664c0a77d0 con 0x7f664c0b1ec0 2021-07-11T04:00:32.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6651c26700 1 -- 172.21.2.104:0/3835648038 >> v2:172.21.2.105:3300/0 conn(0x7f664c0b12d0 msgr2=0x7f664c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:32.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6651c26700 1 --2- 172.21.2.104:0/3835648038 >> v2:172.21.2.105:3300/0 conn(0x7f664c0b12d0 0x7f664c0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6651c26700 1 -- 172.21.2.104:0/3835648038 >> v1:172.21.2.105:6789/0 conn(0x7f664c0b1ec0 legacy=0x7f664c0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6651c26700 1 -- 172.21.2.104:0/3835648038 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c0b08f0 con 0x7f664c0b22a0 2021-07-11T04:00:32.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6650c24700 1 --2- 172.21.2.104:0/3835648038 >> v2:172.21.2.105:3300/0 conn(0x7f664c0b12d0 0x7f664c0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:00:32.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6651c26700 1 --2- 172.21.2.104:0/3835648038 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f664c0b22a0 0x7f664c141250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f66440032f0 tx=0x7f66440091d0).ready entity=mon.2 client_cookie=9734c59936611184 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f663ffff700 1 -- 172.21.2.104:0/3835648038 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664400b6c0 con 0x7f664c0b22a0 2021-07-11T04:00:32.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f663ffff700 1 -- 172.21.2.104:0/3835648038 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664400b820 con 0x7f664c0b22a0 2021-07-11T04:00:32.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f663ffff700 1 -- 172.21.2.104:0/3835648038 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664400b990 con 0x7f664c0b22a0 2021-07-11T04:00:32.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6653689700 1 -- 172.21.2.104:0/3835648038 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f664c0b22a0 msgr2=0x7f664c141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.950+0000 7f6653689700 1 --2- 172.21.2.104:0/3835648038 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f664c0b22a0 0x7f664c141250 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f66440032f0 tx=0x7f66440091d0).stop 2021-07-11T04:00:32.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 -- 172.21.2.104:0/3835648038 shutdown_connections 2021-07-11T04:00:32.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 --2- 172.21.2.104:0/3835648038 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f664c0b22a0 0x7f664c141250 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 --2- 172.21.2.104:0/3835648038 >> v2:172.21.2.105:3300/0 conn(0x7f664c0b12d0 0x7f664c0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 -- 172.21.2.104:0/3835648038 >> 172.21.2.104:0/3835648038 conn(0x7f664c05f6b0 msgr2=0x7f664c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:32.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 -- 172.21.2.104:0/3835648038 shutdown_connections 2021-07-11T04:00:32.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 -- 172.21.2.104:0/3835648038 wait complete. 2021-07-11T04:00:32.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 Processor -- start 2021-07-11T04:00:32.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 -- start start 2021-07-11T04:00:32.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f664c0b12d0 0x7f664c0ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f664c0b1ec0 0x7f664c0af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f664c0b22a0 0x7f664c0afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c14aa90 con 0x7f664c0b1ec0 2021-07-11T04:00:32.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c14a830 con 0x7f664c0b22a0 2021-07-11T04:00:32.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c0af3b0 con 0x7f664c0b12d0 2021-07-11T04:00:32.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6650c24700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f664c0b1ec0 0x7f664c0af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6650c24700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f664c0b1ec0 0x7f664c0af850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47862/0 (socket says 172.21.2.104:47862) 2021-07-11T04:00:32.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6650c24700 1 -- 172.21.2.104:0/623843575 learned_addr learned my addr 172.21.2.104:0/623843575 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:32.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6651425700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f664c0b12d0 0x7f664c0ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6651c26700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f664c0b22a0 0x7f664c0afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6650c24700 1 -- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f664c0b12d0 msgr2=0x7f664c0ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6650c24700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f664c0b12d0 0x7f664c0ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6650c24700 1 -- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f664c0b22a0 msgr2=0x7f664c0afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:32.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6650c24700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f664c0b22a0 0x7f664c0afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:32.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6650c24700 1 -- 172.21.2.104:0/623843575 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664000b180 con 0x7f664c0b1ec0 2021-07-11T04:00:32.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6651c26700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f664c0b22a0 0x7f664c0afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:00:32.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6650c24700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f664c0b1ec0 0x7f664c0af850 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7f664000c000 tx=0x7f6640006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f663e7fc700 1 -- 172.21.2.104:0/623843575 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664000c7e0 con 0x7f664c0b1ec0 2021-07-11T04:00:32.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f663e7fc700 1 -- 172.21.2.104:0/623843575 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664000c940 con 0x7f664c0b1ec0 2021-07-11T04:00:32.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 -- 172.21.2.104:0/623843575 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6644009470 con 0x7f664c0b1ec0 2021-07-11T04:00:32.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f663e7fc700 1 -- 172.21.2.104:0/623843575 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f664000cca0 con 0x7f664c0b1ec0 2021-07-11T04:00:32.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 -- 172.21.2.104:0/623843575 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664c14bb40 con 0x7f664c0b1ec0 2021-07-11T04:00:32.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f6653689700 1 -- 172.21.2.104:0/623843575 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664c057350 con 0x7f664c0b1ec0 2021-07-11T04:00:32.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f663e7fc700 1 -- 172.21.2.104:0/623843575 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f664001c460 con 0x7f664c0b1ec0 2021-07-11T04:00:32.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f663e7fc700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f662c053180 0x7f662c055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:32.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.954+0000 7f663e7fc700 1 -- 172.21.2.104:0/623843575 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f664006b700 con 0x7f664c0b1ec0 2021-07-11T04:00:32.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.958+0000 7f6651425700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f662c053180 0x7f662c055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:32.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.958+0000 7f6651425700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f662c053180 0x7f662c055640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f664800d000 tx=0x7f6648007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:32.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:32.958+0000 7f663e7fc700 1 -- 172.21.2.104:0/623843575 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6640042d10 con 0x7f664c0b1ec0 2021-07-11T04:00:33.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.042+0000 7f6653689700 1 -- 172.21.2.104:0/623843575 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f664c0af570 con 0x7f664c0b1ec0 2021-07-11T04:00:33.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f663e7fc700 1 -- 172.21.2.104:0/623843575 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v94) v1 ==== 133+0+0 (secure 0 0 0) 0x7f664000ce00 con 0x7f664c0b1ec0 2021-07-11T04:00:33.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 -- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f662c053180 msgr2=0x7f662c055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:33.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f662c053180 0x7f662c055640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f664800d000 tx=0x7f6648007a00).stop 2021-07-11T04:00:33.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 -- 172.21.2.104:0/623843575 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f664c0b1ec0 msgr2=0x7f664c0af850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:33.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f664c0b1ec0 0x7f664c0af850 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7f664000c000 tx=0x7f6640006df0).stop 2021-07-11T04:00:33.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 -- 172.21.2.104:0/623843575 shutdown_connections 2021-07-11T04:00:33.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f664c0b22a0 0x7f664c0afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:33.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f662c053180 0x7f662c055640 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:33.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f664c0b1ec0 0x7f664c0af850 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:33.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 --2- 172.21.2.104:0/623843575 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f664c0b12d0 0x7f664c0ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:33.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 -- 172.21.2.104:0/623843575 >> 172.21.2.104:0/623843575 conn(0x7f664c05f6b0 msgr2=0x7f664c13b310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:33.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 -- 172.21.2.104:0/623843575 shutdown_connections 2021-07-11T04:00:33.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:33.950+0000 7f6653689700 1 -- 172.21.2.104:0/623843575 wait complete. 2021-07-11T04:00:33.957 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T04:00:33.963 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T04:00:34.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 Processor -- start 2021-07-11T04:00:34.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 -- start start 2021-07-11T04:00:34.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5ea00b1ec0 0x7f5ea00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5ea00affb0 0x7f5ea00b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ea00a70c0 0x7f5ea0141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea00b0420 con 0x7f5ea00b1ec0 2021-07-11T04:00:34.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea00b2790 con 0x7f5ea00affb0 2021-07-11T04:00:34.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00b0650 con 0x7f5ea00a70c0 2021-07-11T04:00:34.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9ed9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5ea00b1ec0 0x7f5ea00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:34.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9ed9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5ea00b1ec0 0x7f5ea00b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47866/0 (socket says 172.21.2.104:47866) 2021-07-11T04:00:34.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9ed9d700 1 -- 172.21.2.104:0/209356398 learned_addr learned my addr 172.21.2.104:0/209356398 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:34.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9ed9d700 1 -- 172.21.2.104:0/209356398 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ea00a70c0 msgr2=0x7f5ea0141250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:34.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9ed9d700 1 --2- 172.21.2.104:0/209356398 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ea00a70c0 0x7f5ea0141250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:34.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9ed9d700 1 -- 172.21.2.104:0/209356398 >> v2:172.21.2.105:3300/0 conn(0x7f5ea00affb0 msgr2=0x7f5ea00b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:34.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9ed9d700 1 --2- 172.21.2.104:0/209356398 >> v2:172.21.2.105:3300/0 conn(0x7f5ea00affb0 0x7f5ea00b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:34.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9ed9d700 1 -- 172.21.2.104:0/209356398 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea00b08f0 con 0x7f5ea00b1ec0 2021-07-11T04:00:34.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9ed9d700 1 --2- 172.21.2.104:0/209356398 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea00b1ec0 0x7f5ea00b12d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f5e90002f70 tx=0x7f5e90006eb0).ready entity=mon.0 client_cookie=6810d9e1b45f12ac server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:34.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9dd9b700 1 -- 172.21.2.104:0/209356398 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9000d570 con 0x7f5ea00b1ec0 2021-07-11T04:00:34.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9dd9b700 1 -- 172.21.2.104:0/209356398 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e90000e90 con 0x7f5ea00b1ec0 2021-07-11T04:00:34.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5e9dd9b700 1 -- 172.21.2.104:0/209356398 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e9000d900 con 0x7f5ea00b1ec0 2021-07-11T04:00:34.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 -- 172.21.2.104:0/209356398 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea00b1ec0 msgr2=0x7f5ea00b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:34.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 --2- 172.21.2.104:0/209356398 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea00b1ec0 0x7f5ea00b12d0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f5e90002f70 tx=0x7f5e90006eb0).stop 2021-07-11T04:00:34.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 -- 172.21.2.104:0/209356398 shutdown_connections 2021-07-11T04:00:34.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 --2- 172.21.2.104:0/209356398 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ea00a70c0 0x7f5ea0141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:34.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 --2- 172.21.2.104:0/209356398 >> v2:172.21.2.105:3300/0 conn(0x7f5ea00affb0 0x7f5ea00b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:34.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 --2- 172.21.2.104:0/209356398 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea00b1ec0 0x7f5ea00b12d0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:34.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 -- 172.21.2.104:0/209356398 >> 172.21.2.104:0/209356398 conn(0x7f5ea005f6b0 msgr2=0x7f5ea005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:34.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 -- 172.21.2.104:0/209356398 shutdown_connections 2021-07-11T04:00:34.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 -- 172.21.2.104:0/209356398 wait complete. 2021-07-11T04:00:34.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 Processor -- start 2021-07-11T04:00:34.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.018+0000 7f5ea504f700 1 -- start start 2021-07-11T04:00:34.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5ea504f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea00a70c0 0x7f5ea0140630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5ea504f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ea00affb0 0x7f5ea00ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5ea504f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ea00b1ec0 0x7f5ea00aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5ea504f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00a9c40 con 0x7f5ea00a70c0 2021-07-11T04:00:34.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5ea504f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00adf30 con 0x7f5ea00affb0 2021-07-11T04:00:34.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5ea504f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00ad360 con 0x7f5ea00b1ec0 2021-07-11T04:00:34.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e9ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea00a70c0 0x7f5ea0140630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:34.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e9ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea00a70c0 0x7f5ea0140630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47872/0 (socket says 172.21.2.104:47872) 2021-07-11T04:00:34.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e9ed9d700 1 -- 172.21.2.104:0/2165368726 learned_addr learned my addr 172.21.2.104:0/2165368726 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:34.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e9ed9d700 1 -- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ea00b1ec0 msgr2=0x7f5ea00aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:34.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e9ed9d700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ea00b1ec0 0x7f5ea00aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:34.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e9ed9d700 1 -- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ea00affb0 msgr2=0x7f5ea00ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:34.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e9ed9d700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ea00affb0 0x7f5ea00ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:34.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e9ed9d700 1 -- 172.21.2.104:0/2165368726 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9000b1e0 con 0x7f5ea00a70c0 2021-07-11T04:00:34.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e9ed9d700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea00a70c0 0x7f5ea0140630 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f5e90002f70 tx=0x7f5e90015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:34.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e97fff700 1 -- 172.21.2.104:0/2165368726 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e90021070 con 0x7f5ea00a70c0 2021-07-11T04:00:34.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e97fff700 1 -- 172.21.2.104:0/2165368726 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e90000e90 con 0x7f5ea00a70c0 2021-07-11T04:00:34.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e97fff700 1 -- 172.21.2.104:0/2165368726 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e90011bc0 con 0x7f5ea00a70c0 2021-07-11T04:00:34.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5ea504f700 1 -- 172.21.2.104:0/2165368726 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea00ad5c0 con 0x7f5ea00a70c0 2021-07-11T04:00:34.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5ea504f700 1 -- 172.21.2.104:0/2165368726 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea00ab6f0 con 0x7f5ea00a70c0 2021-07-11T04:00:34.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5ea504f700 1 -- 172.21.2.104:0/2165368726 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea0057350 con 0x7f5ea00a70c0 2021-07-11T04:00:34.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e97fff700 1 -- 172.21.2.104:0/2165368726 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5e9000f070 con 0x7f5ea00a70c0 2021-07-11T04:00:34.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e97fff700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e84053240 0x7f5e84055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e97fff700 1 -- 172.21.2.104:0/2165368726 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f5e9006d470 con 0x7f5ea00a70c0 2021-07-11T04:00:34.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e9e59c700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e84053240 0x7f5e84055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:34.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.022+0000 7f5e9e59c700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e84053240 0x7f5e84055700 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f5e88002f70 tx=0x7f5e88006a90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:34.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.026+0000 7f5e97fff700 1 -- 172.21.2.104:0/2165368726 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5e90044870 con 0x7f5ea00a70c0 2021-07-11T04:00:34.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.106+0000 7f5ea504f700 1 -- 172.21.2.104:0/2165368726 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f5ea00ac760 con 0x7f5ea00a70c0 2021-07-11T04:00:34.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5e97fff700 1 -- 172.21.2.104:0/2165368726 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v95) v1 ==== 125+0+0 (secure 0 0 0) 0x7f5e90047e90 con 0x7f5ea00a70c0 2021-07-11T04:00:34.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 -- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e84053240 msgr2=0x7f5e84055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:34.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e84053240 0x7f5e84055700 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f5e88002f70 tx=0x7f5e88006a90).stop 2021-07-11T04:00:34.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 -- 172.21.2.104:0/2165368726 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea00a70c0 msgr2=0x7f5ea0140630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:34.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea00a70c0 0x7f5ea0140630 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f5e90002f70 tx=0x7f5e90015000).stop 2021-07-11T04:00:34.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 -- 172.21.2.104:0/2165368726 shutdown_connections 2021-07-11T04:00:34.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5ea00b1ec0 0x7f5ea00aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:34.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5ea00affb0 0x7f5ea00ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:34.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5e84053240 0x7f5e84055700 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:34.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 --2- 172.21.2.104:0/2165368726 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5ea00a70c0 0x7f5ea0140630 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:34.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 -- 172.21.2.104:0/2165368726 >> 172.21.2.104:0/2165368726 conn(0x7f5ea005f6b0 msgr2=0x7f5ea0145e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:34.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 -- 172.21.2.104:0/2165368726 shutdown_connections 2021-07-11T04:00:34.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.882+0000 7f5ea504f700 1 -- 172.21.2.104:0/2165368726 wait complete. 2021-07-11T04:00:34.891 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T04:00:34.894 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T04:00:34.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 Processor -- start 2021-07-11T04:00:34.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 -- start start 2021-07-11T04:00:34.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcd200b3240 0x7fcd20066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd200b1840 0x7fcd20066b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcd200b0e70 0x7fcd20142890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd200b3620 con 0x7fcd200b0e70 2021-07-11T04:00:34.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd201436f0 con 0x7fcd200b3240 2021-07-11T04:00:34.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd20065ca0 con 0x7fcd200b1840 2021-07-11T04:00:34.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd24e4a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcd200b0e70 0x7fcd20142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:34.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd24e4a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcd200b0e70 0x7fcd20142890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47880/0 (socket says 172.21.2.104:47880) 2021-07-11T04:00:34.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd24e4a700 1 -- 172.21.2.104:0/3408984002 learned_addr learned my addr 172.21.2.104:0/3408984002 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:34.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd1ffff700 1 --2- 172.21.2.104:0/3408984002 >> v2:172.21.2.105:3300/0 conn(0x7fcd200b3240 0x7fcd20066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:34.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd1f7fe700 1 --2- 172.21.2.104:0/3408984002 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd200b1840 0x7fcd20066b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:34.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd24e4a700 1 -- 172.21.2.104:0/3408984002 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd200b1840 msgr2=0x7fcd20066b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:34.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd24e4a700 1 --2- 172.21.2.104:0/3408984002 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd200b1840 0x7fcd20066b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:34.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd24e4a700 1 -- 172.21.2.104:0/3408984002 >> v2:172.21.2.105:3300/0 conn(0x7fcd200b3240 msgr2=0x7fcd20066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:34.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd24e4a700 1 --2- 172.21.2.104:0/3408984002 >> v2:172.21.2.105:3300/0 conn(0x7fcd200b3240 0x7fcd20066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:34.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd24e4a700 1 -- 172.21.2.104:0/3408984002 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd20065f40 con 0x7fcd200b0e70 2021-07-11T04:00:34.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd1ffff700 1 --2- 172.21.2.104:0/3408984002 >> v2:172.21.2.105:3300/0 conn(0x7fcd200b3240 0x7fcd20066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:34.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd1f7fe700 1 --2- 172.21.2.104:0/3408984002 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd200b1840 0x7fcd20066b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:34.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd24e4a700 1 --2- 172.21.2.104:0/3408984002 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd200b0e70 0x7fcd20142890 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7fcd180032f0 tx=0x7fcd18009290).ready entity=mon.0 client_cookie=e3521641b721aebe server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:34.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd1effd700 1 -- 172.21.2.104:0/3408984002 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1800b870 con 0x7fcd200b0e70 2021-07-11T04:00:34.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd1effd700 1 -- 172.21.2.104:0/3408984002 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1800b9d0 con 0x7fcd200b0e70 2021-07-11T04:00:34.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd1effd700 1 -- 172.21.2.104:0/3408984002 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1800bb40 con 0x7fcd200b0e70 2021-07-11T04:00:34.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 -- 172.21.2.104:0/3408984002 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd200b0e70 msgr2=0x7fcd20142890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:34.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 --2- 172.21.2.104:0/3408984002 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd200b0e70 0x7fcd20142890 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7fcd180032f0 tx=0x7fcd18009290).stop 2021-07-11T04:00:34.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 -- 172.21.2.104:0/3408984002 shutdown_connections 2021-07-11T04:00:34.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 --2- 172.21.2.104:0/3408984002 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd200b0e70 0x7fcd20142890 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:34.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 --2- 172.21.2.104:0/3408984002 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd200b1840 0x7fcd20066b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:34.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 --2- 172.21.2.104:0/3408984002 >> v2:172.21.2.105:3300/0 conn(0x7fcd200b3240 0x7fcd20066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:34.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 -- 172.21.2.104:0/3408984002 >> 172.21.2.104:0/3408984002 conn(0x7fcd2005c490 msgr2=0x7fcd2005c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:34.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 -- 172.21.2.104:0/3408984002 shutdown_connections 2021-07-11T04:00:34.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 -- 172.21.2.104:0/3408984002 wait complete. 2021-07-11T04:00:34.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 Processor -- start 2021-07-11T04:00:34.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 -- start start 2021-07-11T04:00:34.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.946+0000 7fcd268ad700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd200b0e70 0x7fcd2009ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd268ad700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd200b1840 0x7fcd200a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd268ad700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd200b3240 0x7fcd200a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd268ad700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd200abbd0 con 0x7fcd200b1840 2021-07-11T04:00:34.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd268ad700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd200ae130 con 0x7fcd200b3240 2021-07-11T04:00:34.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd268ad700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd200acb50 con 0x7fcd200b0e70 2021-07-11T04:00:34.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd200b1840 0x7fcd200a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:34.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd200b1840 0x7fcd200a9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47890/0 (socket says 172.21.2.104:47890) 2021-07-11T04:00:34.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1f7fe700 1 -- 172.21.2.104:0/1883912782 learned_addr learned my addr 172.21.2.104:0/1883912782 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:34.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1ffff700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd200b0e70 0x7fcd2009ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:34.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd24e4a700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd200b3240 0x7fcd200a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:34.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1f7fe700 1 -- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd200b0e70 msgr2=0x7fcd2009ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:34.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1f7fe700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd200b0e70 0x7fcd2009ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:34.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1f7fe700 1 -- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd200b3240 msgr2=0x7fcd200a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:34.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1f7fe700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd200b3240 0x7fcd200a9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:34.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1f7fe700 1 -- 172.21.2.104:0/1883912782 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd18009530 con 0x7fcd200b1840 2021-07-11T04:00:34.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1f7fe700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd200b1840 0x7fcd200a9570 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7fcd14007000 tx=0x7fcd14008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:34.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1d7fa700 1 -- 172.21.2.104:0/1883912782 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd14010580 con 0x7fcd200b1840 2021-07-11T04:00:34.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd268ad700 1 -- 172.21.2.104:0/1883912782 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd200ad0b0 con 0x7fcd200b1840 2021-07-11T04:00:34.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1d7fa700 1 -- 172.21.2.104:0/1883912782 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd140106e0 con 0x7fcd200b1840 2021-07-11T04:00:34.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1d7fa700 1 -- 172.21.2.104:0/1883912782 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd14010a40 con 0x7fcd200b1840 2021-07-11T04:00:34.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd268ad700 1 -- 172.21.2.104:0/1883912782 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd200acdf0 con 0x7fcd200b1840 2021-07-11T04:00:34.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd268ad700 1 -- 172.21.2.104:0/1883912782 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd20057350 con 0x7fcd200b1840 2021-07-11T04:00:34.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1d7fa700 1 -- 172.21.2.104:0/1883912782 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcd14010ba0 con 0x7fcd200b1840 2021-07-11T04:00:34.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1d7fa700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd000531f0 0x7fcd000556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:34.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.950+0000 7fcd1d7fa700 1 -- 172.21.2.104:0/1883912782 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fcd1406b400 con 0x7fcd200b1840 2021-07-11T04:00:34.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.954+0000 7fcd1ffff700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd000531f0 0x7fcd000556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:34.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.954+0000 7fcd1ffff700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd000531f0 0x7fcd000556b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fcd1000b000 tx=0x7fcd10008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:34.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:34.954+0000 7fcd1d7fa700 1 -- 172.21.2.104:0/1883912782 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcd14042640 con 0x7fcd200b1840 2021-07-11T04:00:35.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.066+0000 7fcd268ad700 1 -- 172.21.2.104:0/1883912782 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fcd200ad5f0 con 0x7fcd200b1840 2021-07-11T04:00:35.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd1d7fa700 1 -- 172.21.2.104:0/1883912782 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 17 and data pool 18 v44) v1 ==== 189+0+0 (secure 0 0 0) 0x7fcd14045c60 con 0x7fcd200b1840 2021-07-11T04:00:35.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 -- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd000531f0 msgr2=0x7fcd000556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd000531f0 0x7fcd000556b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fcd1000b000 tx=0x7fcd10008040).stop 2021-07-11T04:00:35.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 -- 172.21.2.104:0/1883912782 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd200b1840 msgr2=0x7fcd200a9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd200b1840 0x7fcd200a9570 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7fcd14007000 tx=0x7fcd14008040).stop 2021-07-11T04:00:35.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 -- 172.21.2.104:0/1883912782 shutdown_connections 2021-07-11T04:00:35.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd200b3240 0x7fcd200a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcd000531f0 0x7fcd000556b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd200b1840 0x7fcd200a9570 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 --2- 172.21.2.104:0/1883912782 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd200b0e70 0x7fcd2009ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 -- 172.21.2.104:0/1883912782 >> 172.21.2.104:0/1883912782 conn(0x7fcd2005c490 msgr2=0x7fcd20064550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:35.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 -- 172.21.2.104:0/1883912782 shutdown_connections 2021-07-11T04:00:35.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.070+0000 7fcd268ad700 1 -- 172.21.2.104:0/1883912782 wait complete. 2021-07-11T04:00:35.078 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 17 and data pool 18 2021-07-11T04:00:35.083 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:35.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.130+0000 7f8dfd71e700 1 Processor -- start 2021-07-11T04:00:35.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.130+0000 7f8dfd71e700 1 -- start start 2021-07-11T04:00:35.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.130+0000 7f8dfd71e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8df8067180 0x7f8df805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.130+0000 7f8dfd71e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8df80b1970 0x7f8df805fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.130+0000 7f8dfd71e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df80a7a90 con 0x7f8df80a76b0 2021-07-11T04:00:35.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.130+0000 7f8dfd71e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df8147620 con 0x7f8df80b1970 2021-07-11T04:00:35.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.130+0000 7f8dfd71e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81478c0 con 0x7f8df8067180 2021-07-11T04:00:35.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.130+0000 7f8df77fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8df80b1970 0x7f8df805fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:35.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.130+0000 7f8df6ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8df80a76b0 0x7f8df8066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44670/0 (socket says 172.21.2.104:44670) 2021-07-11T04:00:35.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.130+0000 7f8df6ffd700 1 -- 172.21.2.104:0/966320974 learned_addr learned my addr 172.21.2.104:0/966320974 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:35.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 -- 172.21.2.104:0/966320974 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8df8067180 msgr2=0x7f8df805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:35.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 --2- 172.21.2.104:0/966320974 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8df8067180 0x7f8df805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 -- 172.21.2.104:0/966320974 >> v1:172.21.2.104:6789/0 conn(0x7f8df80a76b0 legacy=0x7f8df8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 -- 172.21.2.104:0/966320974 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df8147b60 con 0x7f8df80b1970 2021-07-11T04:00:35.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 --2- 172.21.2.104:0/966320974 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8df80b1970 0x7f8df805fba0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f8de0001110 tx=0x7f8df00091d0).ready entity=mon.0 client_cookie=768e6075208582ac server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:35.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df5ffb700 1 -- 172.21.2.104:0/966320974 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df0009ea0 con 0x7f8df80b1970 2021-07-11T04:00:35.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df5ffb700 1 -- 172.21.2.104:0/966320974 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df000b760 con 0x7f8df80b1970 2021-07-11T04:00:35.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df5ffb700 1 -- 172.21.2.104:0/966320974 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df000b8d0 con 0x7f8df80b1970 2021-07-11T04:00:35.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- 172.21.2.104:0/966320974 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8df80b1970 msgr2=0x7f8df805fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 --2- 172.21.2.104:0/966320974 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8df80b1970 0x7f8df805fba0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f8de0001110 tx=0x7f8df00091d0).stop 2021-07-11T04:00:35.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- 172.21.2.104:0/966320974 shutdown_connections 2021-07-11T04:00:35.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 --2- 172.21.2.104:0/966320974 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8df80b1970 0x7f8df805fba0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 --2- 172.21.2.104:0/966320974 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8df8067180 0x7f8df805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- 172.21.2.104:0/966320974 >> 172.21.2.104:0/966320974 conn(0x7f8df80b53e0 msgr2=0x7f8df80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:35.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- 172.21.2.104:0/966320974 shutdown_connections 2021-07-11T04:00:35.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- 172.21.2.104:0/966320974 wait complete. 2021-07-11T04:00:35.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 Processor -- start 2021-07-11T04:00:35.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- start start 2021-07-11T04:00:35.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8df8067180 0x7f8df814d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8df80a76b0 0x7f8df80aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8df80b1970 0x7f8df80ac6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80aed60 con 0x7f8df80b1970 2021-07-11T04:00:35.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80aaeb0 con 0x7f8df8067180 2021-07-11T04:00:35.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80aefb0 con 0x7f8df80a76b0 2021-07-11T04:00:35.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8df80b1970 0x7f8df80ac6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:35.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8df80b1970 0x7f8df80ac6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47912/0 (socket says 172.21.2.104:47912) 2021-07-11T04:00:35.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 -- 172.21.2.104:0/2464200994 learned_addr learned my addr 172.21.2.104:0/2464200994 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:35.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 -- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8df80a76b0 msgr2=0x7f8df80aa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8df80a76b0 0x7f8df80aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 -- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8df8067180 msgr2=0x7f8df814d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8df8067180 0x7f8df814d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 -- 172.21.2.104:0/2464200994 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df0009470 con 0x7f8df80b1970 2021-07-11T04:00:35.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8df77fe700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8df80b1970 0x7f8df80ac6e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f8de0001110 tx=0x7f8df0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:35.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8deffff700 1 -- 172.21.2.104:0/2464200994 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df0006ea0 con 0x7f8df80b1970 2021-07-11T04:00:35.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8deffff700 1 -- 172.21.2.104:0/2464200994 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df00032d0 con 0x7f8df80b1970 2021-07-11T04:00:35.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8deffff700 1 -- 172.21.2.104:0/2464200994 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8df0021630 con 0x7f8df80b1970 2021-07-11T04:00:35.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- 172.21.2.104:0/2464200994 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df80af250 con 0x7f8df80b1970 2021-07-11T04:00:35.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- 172.21.2.104:0/2464200994 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df80aa490 con 0x7f8df80b1970 2021-07-11T04:00:35.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8dfd71e700 1 -- 172.21.2.104:0/2464200994 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df80acf10 con 0x7f8df80b1970 2021-07-11T04:00:35.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.134+0000 7f8deffff700 1 -- 172.21.2.104:0/2464200994 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8df000f070 con 0x7f8df80b1970 2021-07-11T04:00:35.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.138+0000 7f8deffff700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ddc053180 0x7f8ddc055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.138+0000 7f8deffff700 1 -- 172.21.2.104:0/2464200994 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8df006c660 con 0x7f8df80b1970 2021-07-11T04:00:35.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.138+0000 7f8df6ffd700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ddc053180 0x7f8ddc055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:35.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.138+0000 7f8df6ffd700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ddc053180 0x7f8ddc055640 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f8de00011d0 tx=0x7f8de00077b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:35.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.138+0000 7f8deffff700 1 -- 172.21.2.104:0/2464200994 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8df0043980 con 0x7f8df80b1970 2021-07-11T04:00:35.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 -- 172.21.2.104:0/2464200994 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8df8057350 con 0x7f8df80b1970 2021-07-11T04:00:35.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8deffff700 1 -- 172.21.2.104:0/2464200994 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f8df0046f20 con 0x7f8df80b1970 2021-07-11T04:00:35.225 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:35.225 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":96,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:35.070310+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:33.058069+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:34.122127+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:35.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 -- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ddc053180 msgr2=0x7f8ddc055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ddc053180 0x7f8ddc055640 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f8de00011d0 tx=0x7f8de00077b0).stop 2021-07-11T04:00:35.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 -- 172.21.2.104:0/2464200994 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8df80b1970 msgr2=0x7f8df80ac6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8df80b1970 0x7f8df80ac6e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f8de0001110 tx=0x7f8df0016000).stop 2021-07-11T04:00:35.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 -- 172.21.2.104:0/2464200994 shutdown_connections 2021-07-11T04:00:35.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8ddc053180 0x7f8ddc055640 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8df80b1970 0x7f8df80ac6e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8df80a76b0 0x7f8df80aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 --2- 172.21.2.104:0/2464200994 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8df8067180 0x7f8df814d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 -- 172.21.2.104:0/2464200994 >> 172.21.2.104:0/2464200994 conn(0x7f8df80b53e0 msgr2=0x7f8df8149b20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:35.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 -- 172.21.2.104:0/2464200994 shutdown_connections 2021-07-11T04:00:35.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.222+0000 7f8dfd71e700 1 -- 172.21.2.104:0/2464200994 wait complete. 2021-07-11T04:00:35.235 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:35.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedd299e700 1 Processor -- start 2021-07-11T04:00:35.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedd299e700 1 -- start start 2021-07-11T04:00:35.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedd299e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fedcc067180 0x7fedcc05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedd299e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fedcc0b1970 0x7fedcc05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedd299e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedcc0a7a90 con 0x7fedcc0a76b0 2021-07-11T04:00:35.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedd299e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedcc147620 con 0x7fedcc067180 2021-07-11T04:00:35.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedd299e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1478c0 con 0x7fedcc0b1970 2021-07-11T04:00:35.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedcbfff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fedcc0a76b0 0x7fedcc066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44684/0 (socket says 172.21.2.104:44684) 2021-07-11T04:00:35.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedcbfff700 1 -- 172.21.2.104:0/2723521065 learned_addr learned my addr 172.21.2.104:0/2723521065 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:35.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005759035 0 0) 0x7fedcc0a7a90 con 0x7fedcc0a76b0 2021-07-11T04:00:35.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedac003260 con 0x7fedcc0a76b0 2021-07-11T04:00:35.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 479386952 0 0) 0x7fedac003260 con 0x7fedcc0a76b0 2021-07-11T04:00:35.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fedcc0b1970 msgr2=0x7fedcc05fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:35.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedcaffd700 1 --2- 172.21.2.104:0/2723521065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fedcc0b1970 0x7fedcc05fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.282+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 >> v2:172.21.2.105:3300/0 conn(0x7fedcc067180 msgr2=0x7fedcc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:35.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 --2- 172.21.2.104:0/2723521065 >> v2:172.21.2.105:3300/0 conn(0x7fedcc067180 0x7fedcc05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc147b60 con 0x7fedcc0a76b0 2021-07-11T04:00:35.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fedb4001660 con 0x7fedcc0a76b0 2021-07-11T04:00:35.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 >> v1:172.21.2.104:6789/0 conn(0x7fedcc0a76b0 legacy=0x7fedcc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 --2- 172.21.2.104:0/2723521065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedac003750 0x7fedac003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 --2- 172.21.2.104:0/2723521065 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fedac004500 0x7fedac004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 --2- 172.21.2.104:0/2723521065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fedcc0b1970 0x7fedac004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc147620 con 0x7fedac003750 2021-07-11T04:00:35.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc1478c0 con 0x7fedac004500 2021-07-11T04:00:35.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac00b990 con 0x7fedcc0b1970 2021-07-11T04:00:35.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcbfff700 1 --2- 172.21.2.104:0/2723521065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedac003750 0x7fedac003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:35.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcbfff700 1 -- 172.21.2.104:0/2723521065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fedcc0b1970 msgr2=0x7fedac004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:35.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcbfff700 1 --2- 172.21.2.104:0/2723521065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fedcc0b1970 0x7fedac004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcbfff700 1 -- 172.21.2.104:0/2723521065 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fedac004500 msgr2=0x7fedac004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:35.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcbfff700 1 --2- 172.21.2.104:0/2723521065 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fedac004500 0x7fedac004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcbfff700 1 -- 172.21.2.104:0/2723521065 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac003260 con 0x7fedac003750 2021-07-11T04:00:35.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcbfff700 1 --2- 172.21.2.104:0/2723521065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedac003750 0x7fedac003b70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fedb4001370 tx=0x7fedb4001e50).ready entity=mon.0 client_cookie=24cca80d5d27e3cd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:35.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedb40070d0 con 0x7fedac003750 2021-07-11T04:00:35.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb4007230 con 0x7fedac003750 2021-07-11T04:00:35.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcaffd700 1 -- 172.21.2.104:0/2723521065 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedb400c9c0 con 0x7fedac003750 2021-07-11T04:00:35.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 -- 172.21.2.104:0/2723521065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedac003750 msgr2=0x7fedac003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 --2- 172.21.2.104:0/2723521065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedac003750 0x7fedac003b70 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7fedb4001370 tx=0x7fedb4001e50).stop 2021-07-11T04:00:35.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd0f3b700 1 -- 172.21.2.104:0/2723521065 reap_dead start 2021-07-11T04:00:35.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 -- 172.21.2.104:0/2723521065 shutdown_connections 2021-07-11T04:00:35.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 -- 172.21.2.104:0/2723521065 >> 172.21.2.104:0/2723521065 conn(0x7fedcc0b53e0 msgr2=0x7fedcc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:35.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 -- 172.21.2.104:0/2723521065 shutdown_connections 2021-07-11T04:00:35.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 -- 172.21.2.104:0/2723521065 wait complete. 2021-07-11T04:00:35.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 Processor -- start 2021-07-11T04:00:35.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 -- start start 2021-07-11T04:00:35.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fedcc0b3860 0x7fedcc0ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fedcc0af400 0x7fedcc0ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedcc0abcc0 0x7fedcc144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0ae380 con 0x7fedcc0abcc0 2021-07-11T04:00:35.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0b3c40 con 0x7fedcc0af400 2021-07-11T04:00:35.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedcc0b44f0 con 0x7fedcc0b3860 2021-07-11T04:00:35.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd0f3b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedcc0abcc0 0x7fedcc144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:35.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd0f3b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedcc0abcc0 0x7fedcc144670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47930/0 (socket says 172.21.2.104:47930) 2021-07-11T04:00:35.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd0f3b700 1 -- 172.21.2.104:0/672322132 learned_addr learned my addr 172.21.2.104:0/672322132 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:35.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcbfff700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fedcc0b3860 0x7fedcc0ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:35.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedcb7fe700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fedcc0af400 0x7fedcc0ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:35.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd0f3b700 1 -- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fedcc0b3860 msgr2=0x7fedcc0ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd0f3b700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fedcc0b3860 0x7fedcc0ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd0f3b700 1 -- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fedcc0af400 msgr2=0x7fedcc0ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd0f3b700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fedcc0af400 0x7fedcc0ab7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd0f3b700 1 -- 172.21.2.104:0/672322132 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedcc0a9460 con 0x7fedcc0abcc0 2021-07-11T04:00:35.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd0f3b700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedcc0abcc0 0x7fedcc144670 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7fedb40018c0 tx=0x7fedc400a060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:35.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedc97fa700 1 -- 172.21.2.104:0/672322132 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedc4005a20 con 0x7fedcc0abcc0 2021-07-11T04:00:35.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedc97fa700 1 -- 172.21.2.104:0/672322132 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc4005b80 con 0x7fedcc0abcc0 2021-07-11T04:00:35.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedc97fa700 1 -- 172.21.2.104:0/672322132 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedc4007070 con 0x7fedcc0abcc0 2021-07-11T04:00:35.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 -- 172.21.2.104:0/672322132 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedcc0aecc0 con 0x7fedcc0abcc0 2021-07-11T04:00:35.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.286+0000 7fedd299e700 1 -- 172.21.2.104:0/672322132 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedcc0a9700 con 0x7fedcc0abcc0 2021-07-11T04:00:35.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.290+0000 7fedd299e700 1 -- 172.21.2.104:0/672322132 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedcc057350 con 0x7fedcc0abcc0 2021-07-11T04:00:35.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.290+0000 7fedc97fa700 1 -- 172.21.2.104:0/672322132 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fedc401d070 con 0x7fedcc0abcc0 2021-07-11T04:00:35.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.290+0000 7fedc97fa700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedb0053210 0x7fedb00556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.290+0000 7fedc97fa700 1 -- 172.21.2.104:0/672322132 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fedc406a6a0 con 0x7fedcc0abcc0 2021-07-11T04:00:35.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.290+0000 7fedcbfff700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedb0053210 0x7fedb00556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:35.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.290+0000 7fedcbfff700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedb0053210 0x7fedb00556d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fedb4018000 tx=0x7fedb4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:35.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.290+0000 7fedc97fa700 1 -- 172.21.2.104:0/672322132 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fedc40419e0 con 0x7fedcc0abcc0 2021-07-11T04:00:35.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 -- 172.21.2.104:0/672322132 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedcc0698e0 con 0x7fedcc0abcc0 2021-07-11T04:00:35.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedc97fa700 1 -- 172.21.2.104:0/672322132 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16067 (secure 0 0 0) 0x7fedc4045000 con 0x7fedcc0abcc0 2021-07-11T04:00:35.376 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:35.377 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":96,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:35.070310+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:33.058069+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:34.122127+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:35.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 -- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedb0053210 msgr2=0x7fedb00556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedb0053210 0x7fedb00556d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fedb4018000 tx=0x7fedb4015000).stop 2021-07-11T04:00:35.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 -- 172.21.2.104:0/672322132 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedcc0abcc0 msgr2=0x7fedcc144670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedcc0abcc0 0x7fedcc144670 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7fedb40018c0 tx=0x7fedc400a060).stop 2021-07-11T04:00:35.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 -- 172.21.2.104:0/672322132 shutdown_connections 2021-07-11T04:00:35.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedb0053210 0x7fedb00556d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fedcc0abcc0 0x7fedcc144670 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fedcc0af400 0x7fedcc0ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 --2- 172.21.2.104:0/672322132 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fedcc0b3860 0x7fedcc0ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 -- 172.21.2.104:0/672322132 >> 172.21.2.104:0/672322132 conn(0x7fedcc0b53e0 msgr2=0x7fedcc0a7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:35.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 -- 172.21.2.104:0/672322132 shutdown_connections 2021-07-11T04:00:35.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.374+0000 7fedd299e700 1 -- 172.21.2.104:0/672322132 wait complete. 2021-07-11T04:00:35.384 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T04:00:35.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88fe9b6700 1 Processor -- start 2021-07-11T04:00:35.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88fe9b6700 1 -- start start 2021-07-11T04:00:35.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88fe9b6700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f88f80b4180 0x7f88f8140940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88fe9b6700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f88f80b4690 0x7f88f80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88fe9b6700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f8143b60 con 0x7f88f8146950 2021-07-11T04:00:35.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88fe9b6700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f8147510 con 0x7f88f80b4180 2021-07-11T04:00:35.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88fe9b6700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f8143e00 con 0x7f88f80b4690 2021-07-11T04:00:35.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88f77fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f88f80b4180 0x7f88f8140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:35.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88f77fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f88f80b4180 0x7f88f8140940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47940/0 (socket says 172.21.2.104:47940) 2021-07-11T04:00:35.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88f77fe700 1 -- 172.21.2.104:0/503209009 learned_addr learned my addr 172.21.2.104:0/503209009 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:35.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88f77fe700 1 -- 172.21.2.104:0/503209009 >> v2:172.21.2.105:3300/0 conn(0x7f88f80b4690 msgr2=0x7f88f80b4ed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:35.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88f77fe700 1 --2- 172.21.2.104:0/503209009 >> v2:172.21.2.105:3300/0 conn(0x7f88f80b4690 0x7f88f80b4ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88f77fe700 1 -- 172.21.2.104:0/503209009 >> v1:172.21.2.105:6789/0 conn(0x7f88f8146950 legacy=0x7f88f8140230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:35.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88f77fe700 1 -- 172.21.2.104:0/503209009 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f80b0c90 con 0x7f88f80b4180 2021-07-11T04:00:35.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88f77fe700 1 --2- 172.21.2.104:0/503209009 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b4180 0x7f88f8140940 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f88f8060c50 tx=0x7f88e8006eb0).ready entity=mon.0 client_cookie=5e3f7961397761f5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:35.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88f6ffd700 1 -- 172.21.2.104:0/503209009 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e800d5c0 con 0x7f88f80b4180 2021-07-11T04:00:35.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88f6ffd700 1 -- 172.21.2.104:0/503209009 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e8000e90 con 0x7f88f80b4180 2021-07-11T04:00:35.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88f6ffd700 1 -- 172.21.2.104:0/503209009 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e800d910 con 0x7f88f80b4180 2021-07-11T04:00:35.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88fe9b6700 1 -- 172.21.2.104:0/503209009 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b4180 msgr2=0x7f88f8140940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.434+0000 7f88fe9b6700 1 --2- 172.21.2.104:0/503209009 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b4180 0x7f88f8140940 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f88f8060c50 tx=0x7f88e8006eb0).stop 2021-07-11T04:00:35.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 -- 172.21.2.104:0/503209009 shutdown_connections 2021-07-11T04:00:35.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 --2- 172.21.2.104:0/503209009 >> v2:172.21.2.105:3300/0 conn(0x7f88f80b4690 0x7f88f80b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 --2- 172.21.2.104:0/503209009 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b4180 0x7f88f8140940 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:35.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 -- 172.21.2.104:0/503209009 >> 172.21.2.104:0/503209009 conn(0x7f88f805c490 msgr2=0x7f88f805c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:35.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 -- 172.21.2.104:0/503209009 shutdown_connections 2021-07-11T04:00:35.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 -- 172.21.2.104:0/503209009 wait complete. 2021-07-11T04:00:35.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 Processor -- start 2021-07-11T04:00:35.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 -- start start 2021-07-11T04:00:35.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88f80b4180 0x7f88f805cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b4690 0x7f88f8137390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f8146950 0x7f88f80b7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f80b58d0 con 0x7f88f80b4690 2021-07-11T04:00:35.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f80b6010 con 0x7f88f80b4180 2021-07-11T04:00:35.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f80b62b0 con 0x7f88f8146950 2021-07-11T04:00:35.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b4690 0x7f88f8137390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:35.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b4690 0x7f88f8137390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47944/0 (socket says 172.21.2.104:47944) 2021-07-11T04:00:35.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f77fe700 1 -- 172.21.2.104:0/1334195202 learned_addr learned my addr 172.21.2.104:0/1334195202 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:35.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f77fe700 1 -- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f8146950 msgr2=0x7f88f80b7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f77fe700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f8146950 0x7f88f80b7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f77fe700 1 -- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88f80b4180 msgr2=0x7f88f805cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:35.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f77fe700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88f80b4180 0x7f88f805cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:35.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f77fe700 1 -- 172.21.2.104:0/1334195202 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e800b1e0 con 0x7f88f80b4690 2021-07-11T04:00:35.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f77fe700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b4690 0x7f88f8137390 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f88f8060c50 tx=0x7f88e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:35.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f57fa700 1 -- 172.21.2.104:0/1334195202 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e8021070 con 0x7f88f80b4690 2021-07-11T04:00:35.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f57fa700 1 -- 172.21.2.104:0/1334195202 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e8000e90 con 0x7f88f80b4690 2021-07-11T04:00:35.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 -- 172.21.2.104:0/1334195202 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f8141dc0 con 0x7f88f80b4690 2021-07-11T04:00:35.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f57fa700 1 -- 172.21.2.104:0/1334195202 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f88e8002f50 con 0x7f88f80b4690 2021-07-11T04:00:35.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 -- 172.21.2.104:0/1334195202 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f80b68f0 con 0x7f88f80b4690 2021-07-11T04:00:35.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88fe9b6700 1 -- 172.21.2.104:0/1334195202 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f80b5ad0 con 0x7f88f80b4690 2021-07-11T04:00:35.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f57fa700 1 -- 172.21.2.104:0/1334195202 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f88e800f070 con 0x7f88f80b4690 2021-07-11T04:00:35.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f57fa700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88dc053300 0x7f88dc0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:35.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f57fa700 1 -- 172.21.2.104:0/1334195202 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f88e806c7f0 con 0x7f88f80b4690 2021-07-11T04:00:35.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.438+0000 7f88f7fff700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88dc053300 0x7f88dc0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:35.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.442+0000 7f88f7fff700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88dc053300 0x7f88dc0557c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f88e0000ed0 tx=0x7f88e0006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:35.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.442+0000 7f88f57fa700 1 -- 172.21.2.104:0/1334195202 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f88e8043b30 con 0x7f88f80b4690 2021-07-11T04:00:35.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:35.550+0000 7f88fe9b6700 1 -- 172.21.2.104:0/1334195202 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f88f80b5e50 con 0x7f88f80b4690 2021-07-11T04:00:36.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.082+0000 7f88f57fa700 1 -- 172.21.2.104:0/1334195202 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7f88e8047150 con 0x7f88f80b4690 2021-07-11T04:00:36.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 -- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88dc053300 msgr2=0x7f88dc0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88dc053300 0x7f88dc0557c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f88e0000ed0 tx=0x7f88e0006930).stop 2021-07-11T04:00:36.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 -- 172.21.2.104:0/1334195202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b4690 msgr2=0x7f88f8137390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b4690 0x7f88f8137390 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f88f8060c50 tx=0x7f88e8015000).stop 2021-07-11T04:00:36.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 -- 172.21.2.104:0/1334195202 shutdown_connections 2021-07-11T04:00:36.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f88f8146950 0x7f88f80b7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f88dc053300 0x7f88dc0557c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f88f80b4690 0x7f88f8137390 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 --2- 172.21.2.104:0/1334195202 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f88f80b4180 0x7f88f805cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 -- 172.21.2.104:0/1334195202 >> 172.21.2.104:0/1334195202 conn(0x7f88f805c490 msgr2=0x7f88f813ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:36.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 -- 172.21.2.104:0/1334195202 shutdown_connections 2021-07-11T04:00:36.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.086+0000 7f88fe9b6700 1 -- 172.21.2.104:0/1334195202 wait complete. 2021-07-11T04:00:36.095 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:36.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 Processor -- start 2021-07-11T04:00:36.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 -- start start 2021-07-11T04:00:36.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbabc0a76b0 0x7fbabc066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbabc0b1970 0x7fbabc066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbabc067450 con 0x7fbabc0b0060 2021-07-11T04:00:36.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbabc0a7a90 con 0x7fbabc0b1970 2021-07-11T04:00:36.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc147620 con 0x7fbabc0a76b0 2021-07-11T04:00:36.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac1fd8700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fbabc0b0060 0x7fbabc05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44722/0 (socket says 172.21.2.104:44722) 2021-07-11T04:00:36.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac1fd8700 1 -- 172.21.2.104:0/2899809516 learned_addr learned my addr 172.21.2.104:0/2899809516 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:36.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac0fd6700 1 --2- 172.21.2.104:0/2899809516 >> v2:172.21.2.104:3300/0 conn(0x7fbabc0b1970 0x7fbabc066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac0fd6700 1 -- 172.21.2.104:0/2899809516 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbabc0a76b0 msgr2=0x7fbabc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:36.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac0fd6700 1 --2- 172.21.2.104:0/2899809516 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbabc0a76b0 0x7fbabc066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac0fd6700 1 -- 172.21.2.104:0/2899809516 >> v1:172.21.2.104:6789/0 conn(0x7fbabc0b0060 legacy=0x7fbabc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac0fd6700 1 -- 172.21.2.104:0/2899809516 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbabc1478c0 con 0x7fbabc0b1970 2021-07-11T04:00:36.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac0fd6700 1 --2- 172.21.2.104:0/2899809516 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbabc0b1970 0x7fbabc066f60 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7fbaac002f70 tx=0x7fbaac006df0).ready entity=mon.0 client_cookie=f40391475c63619a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbaabfff700 1 -- 172.21.2.104:0/2899809516 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaac00d570 con 0x7fbabc0b1970 2021-07-11T04:00:36.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbaabfff700 1 -- 172.21.2.104:0/2899809516 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaac000e90 con 0x7fbabc0b1970 2021-07-11T04:00:36.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbaabfff700 1 -- 172.21.2.104:0/2899809516 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbaac00d8c0 con 0x7fbabc0b1970 2021-07-11T04:00:36.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 -- 172.21.2.104:0/2899809516 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbabc0b1970 msgr2=0x7fbabc066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 --2- 172.21.2.104:0/2899809516 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbabc0b1970 0x7fbabc066f60 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7fbaac002f70 tx=0x7fbaac006df0).stop 2021-07-11T04:00:36.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 -- 172.21.2.104:0/2899809516 shutdown_connections 2021-07-11T04:00:36.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 --2- 172.21.2.104:0/2899809516 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbabc0b1970 0x7fbabc066f60 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 --2- 172.21.2.104:0/2899809516 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbabc0a76b0 0x7fbabc066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 -- 172.21.2.104:0/2899809516 >> 172.21.2.104:0/2899809516 conn(0x7fbabc0b53e0 msgr2=0x7fbabc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:36.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 -- 172.21.2.104:0/2899809516 shutdown_connections 2021-07-11T04:00:36.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.142+0000 7fbac3a3b700 1 -- 172.21.2.104:0/2899809516 wait complete. 2021-07-11T04:00:36.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac3a3b700 1 Processor -- start 2021-07-11T04:00:36.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac3a3b700 1 -- start start 2021-07-11T04:00:36.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac3a3b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbabc0a76b0 0x7fbabc0abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac3a3b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbabc0b0060 0x7fbabc0ae630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac3a3b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbabc0b1970 0x7fbabc0acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac3a3b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc0ab640 con 0x7fbabc0a76b0 2021-07-11T04:00:36.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac3a3b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc0ab8e0 con 0x7fbabc0b1970 2021-07-11T04:00:36.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac3a3b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbabc0ac520 con 0x7fbabc0b0060 2021-07-11T04:00:36.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac17d7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbabc0a76b0 0x7fbabc0abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac17d7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbabc0a76b0 0x7fbabc0abf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47958/0 (socket says 172.21.2.104:47958) 2021-07-11T04:00:36.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac17d7700 1 -- 172.21.2.104:0/183288041 learned_addr learned my addr 172.21.2.104:0/183288041 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:36.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac17d7700 1 -- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbabc0b0060 msgr2=0x7fbabc0ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac17d7700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbabc0b0060 0x7fbabc0ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac17d7700 1 -- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbabc0b1970 msgr2=0x7fbabc0acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac17d7700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbabc0b1970 0x7fbabc0acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac17d7700 1 -- 172.21.2.104:0/183288041 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaac00b180 con 0x7fbabc0a76b0 2021-07-11T04:00:36.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac17d7700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbabc0a76b0 0x7fbabc0abf30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7fbab8002f70 tx=0x7fbab8006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbaaa7fc700 1 -- 172.21.2.104:0/183288041 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbab8013460 con 0x7fbabc0a76b0 2021-07-11T04:00:36.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbaaa7fc700 1 -- 172.21.2.104:0/183288041 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbab80135c0 con 0x7fbabc0a76b0 2021-07-11T04:00:36.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac3a3b700 1 -- 172.21.2.104:0/183288041 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbabc0ac7c0 con 0x7fbabc0a76b0 2021-07-11T04:00:36.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbaaa7fc700 1 -- 172.21.2.104:0/183288041 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbab8013920 con 0x7fbabc0a76b0 2021-07-11T04:00:36.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac3a3b700 1 -- 172.21.2.104:0/183288041 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbabc0ab2c0 con 0x7fbabc0a76b0 2021-07-11T04:00:36.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbac3a3b700 1 -- 172.21.2.104:0/183288041 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbabc0ac950 con 0x7fbabc0a76b0 2021-07-11T04:00:36.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbaaa7fc700 1 -- 172.21.2.104:0/183288041 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbab801a060 con 0x7fbabc0a76b0 2021-07-11T04:00:36.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbaaa7fc700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba9c053260 0x7fba9c055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.146+0000 7fbaaa7fc700 1 -- 172.21.2.104:0/183288041 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbab806af90 con 0x7fbabc0a76b0 2021-07-11T04:00:36.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.150+0000 7fbac0fd6700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba9c053260 0x7fba9c055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.150+0000 7fbac0fd6700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba9c053260 0x7fba9c055720 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fbaac002f70 tx=0x7fbaac015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.150+0000 7fbaaa7fc700 1 -- 172.21.2.104:0/183288041 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbab80422d0 con 0x7fbabc0a76b0 2021-07-11T04:00:36.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.262+0000 7fbac3a3b700 1 -- 172.21.2.104:0/183288041 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbabc057350 con 0x7fbabc0a76b0 2021-07-11T04:00:36.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.262+0000 7fbaaa7fc700 1 -- 172.21.2.104:0/183288041 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2481 (secure 0 0 0) 0x7fbab8018070 con 0x7fbabc0a76b0 2021-07-11T04:00:36.267 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:36.268 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/873566337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":873566337},{"type":"v1","addr":"172.21.2.105:6835","nonce":873566337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/930744033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":930744033},{"type":"v1","addr":"172.21.2.104:6833","nonce":930744033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:35.070420+0000","modified":"2021-07-11T04:00:36.075051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/18122340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":18122340},{"type":"v1","addr":"172.21.2.104:6835","nonce":18122340}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-07-11T04:00:36.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 -- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba9c053260 msgr2=0x7fba9c055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba9c053260 0x7fba9c055720 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fbaac002f70 tx=0x7fbaac015000).stop 2021-07-11T04:00:36.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 -- 172.21.2.104:0/183288041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbabc0a76b0 msgr2=0x7fbabc0abf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbabc0a76b0 0x7fbabc0abf30 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7fbab8002f70 tx=0x7fbab8006ec0).stop 2021-07-11T04:00:36.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 -- 172.21.2.104:0/183288041 shutdown_connections 2021-07-11T04:00:36.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbabc0b1970 0x7fbabc0acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbabc0b0060 0x7fbabc0ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba9c053260 0x7fba9c055720 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 --2- 172.21.2.104:0/183288041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbabc0a76b0 0x7fbabc0abf30 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 -- 172.21.2.104:0/183288041 >> 172.21.2.104:0/183288041 conn(0x7fbabc0b53e0 msgr2=0x7fbabc1497c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:36.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 -- 172.21.2.104:0/183288041 shutdown_connections 2021-07-11T04:00:36.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.266+0000 7fbac3a3b700 1 -- 172.21.2.104:0/183288041 wait complete. 2021-07-11T04:00:36.272 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 45 2021-07-11T04:00:36.275 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:36.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f220aac8700 1 Processor -- start 2021-07-11T04:00:36.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f220aac8700 1 -- start start 2021-07-11T04:00:36.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f220aac8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f22040a76b0 0x7f2204066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f220aac8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f22040b1970 0x7f2204066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f220aac8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22040b0060 0x7f220405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f220aac8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2204067450 con 0x7f22040b1970 2021-07-11T04:00:36.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f220aac8700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22040a7a90 con 0x7f22040a76b0 2021-07-11T04:00:36.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f220aac8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2204147620 con 0x7f22040b0060 2021-07-11T04:00:36.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f2203fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f22040b1970 0x7f2204066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f2203fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f22040b1970 0x7f2204066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47968/0 (socket says 172.21.2.104:47968) 2021-07-11T04:00:36.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f2203fff700 1 -- 172.21.2.104:0/3958665764 learned_addr learned my addr 172.21.2.104:0/3958665764 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:36.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f2203fff700 1 -- 172.21.2.104:0/3958665764 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22040b0060 msgr2=0x7f220405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:36.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f2203fff700 1 --2- 172.21.2.104:0/3958665764 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22040b0060 0x7f220405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f2203fff700 1 -- 172.21.2.104:0/3958665764 >> v2:172.21.2.105:3300/0 conn(0x7f22040a76b0 msgr2=0x7f2204066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:36.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f2203fff700 1 --2- 172.21.2.104:0/3958665764 >> v2:172.21.2.105:3300/0 conn(0x7f22040a76b0 0x7f2204066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.322+0000 7f2203fff700 1 -- 172.21.2.104:0/3958665764 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22041478c0 con 0x7f22040b1970 2021-07-11T04:00:36.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2203fff700 1 --2- 172.21.2.104:0/3958665764 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22040b1970 0x7f2204066f60 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f21ec002f70 tx=0x7f21ec006eb0).ready entity=mon.0 client_cookie=3cadd0c47c13a7d7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f22037fe700 1 -- 172.21.2.104:0/3958665764 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21ec00d570 con 0x7f22040b1970 2021-07-11T04:00:36.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f22037fe700 1 -- 172.21.2.104:0/3958665764 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ec000e90 con 0x7f22040b1970 2021-07-11T04:00:36.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f22037fe700 1 -- 172.21.2.104:0/3958665764 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21ec00d900 con 0x7f22040b1970 2021-07-11T04:00:36.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- 172.21.2.104:0/3958665764 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22040b1970 msgr2=0x7f2204066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 --2- 172.21.2.104:0/3958665764 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22040b1970 0x7f2204066f60 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f21ec002f70 tx=0x7f21ec006eb0).stop 2021-07-11T04:00:36.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- 172.21.2.104:0/3958665764 shutdown_connections 2021-07-11T04:00:36.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 --2- 172.21.2.104:0/3958665764 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22040b0060 0x7f220405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 --2- 172.21.2.104:0/3958665764 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22040b1970 0x7f2204066f60 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 --2- 172.21.2.104:0/3958665764 >> v2:172.21.2.105:3300/0 conn(0x7f22040a76b0 0x7f2204066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- 172.21.2.104:0/3958665764 >> 172.21.2.104:0/3958665764 conn(0x7f22040b53e0 msgr2=0x7f22040b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:36.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- 172.21.2.104:0/3958665764 shutdown_connections 2021-07-11T04:00:36.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- 172.21.2.104:0/3958665764 wait complete. 2021-07-11T04:00:36.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 Processor -- start 2021-07-11T04:00:36.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- start start 2021-07-11T04:00:36.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22040a76b0 0x7f22040abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22040b0060 0x7f2204144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22040b1970 0x7f2204144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22040a9ba0 con 0x7f22040b1970 2021-07-11T04:00:36.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22040a9910 con 0x7f22040b0060 2021-07-11T04:00:36.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22040af310 con 0x7f22040a76b0 2021-07-11T04:00:36.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2209065700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22040b1970 0x7f2204144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2209065700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22040b1970 0x7f2204144af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47974/0 (socket says 172.21.2.104:47974) 2021-07-11T04:00:36.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2209065700 1 -- 172.21.2.104:0/101460187 learned_addr learned my addr 172.21.2.104:0/101460187 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:36.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2209065700 1 -- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22040a76b0 msgr2=0x7f22040abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2209065700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22040a76b0 0x7f22040abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2209065700 1 -- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22040b0060 msgr2=0x7f2204144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2209065700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22040b0060 0x7f2204144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2209065700 1 -- 172.21.2.104:0/101460187 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ec00b1e0 con 0x7f22040b1970 2021-07-11T04:00:36.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2209065700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22040b1970 0x7f2204144af0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f22040aecf0 tx=0x7f21fc0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2201ffb700 1 -- 172.21.2.104:0/101460187 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21fc005c30 con 0x7f22040b1970 2021-07-11T04:00:36.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2201ffb700 1 -- 172.21.2.104:0/101460187 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21fc005d90 con 0x7f22040b1970 2021-07-11T04:00:36.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2201ffb700 1 -- 172.21.2.104:0/101460187 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f21fc006220 con 0x7f22040b1970 2021-07-11T04:00:36.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- 172.21.2.104:0/101460187 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22040aedd0 con 0x7f22040b1970 2021-07-11T04:00:36.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- 172.21.2.104:0/101460187 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22040ab870 con 0x7f22040b1970 2021-07-11T04:00:36.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f220aac8700 1 -- 172.21.2.104:0/101460187 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2204057350 con 0x7f22040b1970 2021-07-11T04:00:36.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.326+0000 7f2201ffb700 1 -- 172.21.2.104:0/101460187 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f21fc00f030 con 0x7f22040b1970 2021-07-11T04:00:36.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.330+0000 7f2201ffb700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21e80531f0 0x7f21e80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.330+0000 7f2201ffb700 1 -- 172.21.2.104:0/101460187 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f21fc06b600 con 0x7f22040b1970 2021-07-11T04:00:36.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.330+0000 7f2208864700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21e80531f0 0x7f21e80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.330+0000 7f2208864700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21e80531f0 0x7f21e80556b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f21f4002f70 tx=0x7f21f4006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.330+0000 7f2201ffb700 1 -- 172.21.2.104:0/101460187 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f21fc042940 con 0x7f22040b1970 2021-07-11T04:00:36.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 -- 172.21.2.104:0/101460187 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22040698e0 con 0x7f22040b1970 2021-07-11T04:00:36.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f2201ffb700 1 -- 172.21.2.104:0/101460187 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16054 (secure 0 0 0) 0x7f21fc045f60 con 0x7f22040b1970 2021-07-11T04:00:36.417 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:36.417 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":97,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:36.073518+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:33.058069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:34.122127+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:36.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 -- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21e80531f0 msgr2=0x7f21e80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21e80531f0 0x7f21e80556b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f21f4002f70 tx=0x7f21f4006a80).stop 2021-07-11T04:00:36.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 -- 172.21.2.104:0/101460187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22040b1970 msgr2=0x7f2204144af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22040b1970 0x7f2204144af0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f22040aecf0 tx=0x7f21fc0091a0).stop 2021-07-11T04:00:36.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 -- 172.21.2.104:0/101460187 shutdown_connections 2021-07-11T04:00:36.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f21e80531f0 0x7f21e80556b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22040b1970 0x7f2204144af0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22040b0060 0x7f2204144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 --2- 172.21.2.104:0/101460187 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22040a76b0 0x7f22040abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 -- 172.21.2.104:0/101460187 >> 172.21.2.104:0/101460187 conn(0x7f22040b53e0 msgr2=0x7f22041499e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:36.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 -- 172.21.2.104:0/101460187 shutdown_connections 2021-07-11T04:00:36.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.414+0000 7f220aac8700 1 -- 172.21.2.104:0/101460187 wait complete. 2021-07-11T04:00:36.424 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:00:36.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 Processor -- start 2021-07-11T04:00:36.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 -- start start 2021-07-11T04:00:36.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcc1c0b3240 0x7fcc1c1470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc1c066400 0x7fcc1c143370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1c0b3620 con 0x7fcc1c064ab0 2021-07-11T04:00:36.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1c147590 con 0x7fcc1c0b3240 2021-07-11T04:00:36.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c065030 con 0x7fcc1c066400 2021-07-11T04:00:36.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc1b7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fcc1c064ab0 0x7fcc1c065cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44748/0 (socket says 172.21.2.104:44748) 2021-07-11T04:00:36.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc1b7fe700 1 -- 172.21.2.104:0/3948595685 learned_addr learned my addr 172.21.2.104:0/3948595685 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:36.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc1bfff700 1 --2- 172.21.2.104:0/3948595685 >> v2:172.21.2.104:3300/0 conn(0x7fcc1c0b3240 0x7fcc1c1470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc1bfff700 1 -- 172.21.2.104:0/3948595685 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc1c066400 msgr2=0x7fcc1c143370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:36.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc1bfff700 1 --2- 172.21.2.104:0/3948595685 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc1c066400 0x7fcc1c143370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc1bfff700 1 -- 172.21.2.104:0/3948595685 >> v1:172.21.2.104:6789/0 conn(0x7fcc1c064ab0 legacy=0x7fcc1c065cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc1bfff700 1 -- 172.21.2.104:0/3948595685 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1c0652d0 con 0x7fcc1c0b3240 2021-07-11T04:00:36.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc1bfff700 1 --2- 172.21.2.104:0/3948595685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc1c0b3240 0x7fcc1c1470a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fcc0c002f70 tx=0x7fcc0c006df0).ready entity=mon.0 client_cookie=778a61bfde07445d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc1affd700 1 -- 172.21.2.104:0/3948595685 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc0c00d570 con 0x7fcc1c0b3240 2021-07-11T04:00:36.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc1affd700 1 -- 172.21.2.104:0/3948595685 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0c000e90 con 0x7fcc1c0b3240 2021-07-11T04:00:36.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc1affd700 1 -- 172.21.2.104:0/3948595685 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc0c00d8c0 con 0x7fcc1c0b3240 2021-07-11T04:00:36.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 -- 172.21.2.104:0/3948595685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc1c0b3240 msgr2=0x7fcc1c1470a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 --2- 172.21.2.104:0/3948595685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc1c0b3240 0x7fcc1c1470a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fcc0c002f70 tx=0x7fcc0c006df0).stop 2021-07-11T04:00:36.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 -- 172.21.2.104:0/3948595685 shutdown_connections 2021-07-11T04:00:36.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 --2- 172.21.2.104:0/3948595685 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc1c066400 0x7fcc1c143370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 --2- 172.21.2.104:0/3948595685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc1c0b3240 0x7fcc1c1470a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 -- 172.21.2.104:0/3948595685 >> 172.21.2.104:0/3948595685 conn(0x7fcc1c05c490 msgr2=0x7fcc1c05c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:36.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 -- 172.21.2.104:0/3948595685 shutdown_connections 2021-07-11T04:00:36.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 -- 172.21.2.104:0/3948595685 wait complete. 2021-07-11T04:00:36.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 Processor -- start 2021-07-11T04:00:36.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.478+0000 7fcc228f5700 1 -- start start 2021-07-11T04:00:36.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc228f5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc1c064ab0 0x7fcc1c0ac790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc228f5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc1c066400 0x7fcc1c0ab5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc228f5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc1c0b3240 0x7fcc1c0abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc228f5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c144230 con 0x7fcc1c0b3240 2021-07-11T04:00:36.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc228f5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c0afc10 con 0x7fcc1c066400 2021-07-11T04:00:36.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc228f5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1c09e8b0 con 0x7fcc1c064ab0 2021-07-11T04:00:36.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc20e92700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc1c0b3240 0x7fcc1c0abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc20e92700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc1c0b3240 0x7fcc1c0abae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47986/0 (socket says 172.21.2.104:47986) 2021-07-11T04:00:36.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc20e92700 1 -- 172.21.2.104:0/1058936384 learned_addr learned my addr 172.21.2.104:0/1058936384 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:36.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc20e92700 1 -- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc1c064ab0 msgr2=0x7fcc1c0ac790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc20e92700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc1c064ab0 0x7fcc1c0ac790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc20e92700 1 -- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc1c066400 msgr2=0x7fcc1c0ab5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc20e92700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc1c066400 0x7fcc1c0ab5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc20e92700 1 -- 172.21.2.104:0/1058936384 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c00b180 con 0x7fcc1c0b3240 2021-07-11T04:00:36.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc20e92700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc1c0b3240 0x7fcc1c0abae0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7fcc140032f0 tx=0x7fcc14009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc197fa700 1 -- 172.21.2.104:0/1058936384 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc1400d580 con 0x7fcc1c0b3240 2021-07-11T04:00:36.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc197fa700 1 -- 172.21.2.104:0/1058936384 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1400d6e0 con 0x7fcc1c0b3240 2021-07-11T04:00:36.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc228f5700 1 -- 172.21.2.104:0/1058936384 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1c09eae0 con 0x7fcc1c0b3240 2021-07-11T04:00:36.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc197fa700 1 -- 172.21.2.104:0/1058936384 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc1400da40 con 0x7fcc1c0b3240 2021-07-11T04:00:36.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc228f5700 1 -- 172.21.2.104:0/1058936384 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1c157720 con 0x7fcc1c0b3240 2021-07-11T04:00:36.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc228f5700 1 -- 172.21.2.104:0/1058936384 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1c057350 con 0x7fcc1c0b3240 2021-07-11T04:00:36.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc197fa700 1 -- 172.21.2.104:0/1058936384 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcc1400dba0 con 0x7fcc1c0b3240 2021-07-11T04:00:36.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc197fa700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc00053300 0x7fcc000557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc197fa700 1 -- 172.21.2.104:0/1058936384 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcc1406b850 con 0x7fcc1c0b3240 2021-07-11T04:00:36.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.482+0000 7fcc1bfff700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc00053300 0x7fcc000557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.486+0000 7fcc1bfff700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc00053300 0x7fcc000557c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fcc0c002f70 tx=0x7fcc0c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.486+0000 7fcc197fa700 1 -- 172.21.2.104:0/1058936384 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcc14042b90 con 0x7fcc1c0b3240 2021-07-11T04:00:36.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.594+0000 7fcc228f5700 1 -- 172.21.2.104:0/1058936384 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fcc1c0616d0 con 0x7fcc1c0b3240 2021-07-11T04:00:36.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.598+0000 7fcc197fa700 1 -- 172.21.2.104:0/1058936384 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7fcc140461b0 con 0x7fcc1c0b3240 2021-07-11T04:00:36.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.598+0000 7fcc228f5700 1 -- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc00053300 msgr2=0x7fcc000557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.598+0000 7fcc228f5700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc00053300 0x7fcc000557c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fcc0c002f70 tx=0x7fcc0c015000).stop 2021-07-11T04:00:36.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.598+0000 7fcc228f5700 1 -- 172.21.2.104:0/1058936384 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc1c0b3240 msgr2=0x7fcc1c0abae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.598+0000 7fcc228f5700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc1c0b3240 0x7fcc1c0abae0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7fcc140032f0 tx=0x7fcc14009220).stop 2021-07-11T04:00:36.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.598+0000 7fcc228f5700 1 -- 172.21.2.104:0/1058936384 shutdown_connections 2021-07-11T04:00:36.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.598+0000 7fcc228f5700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc00053300 0x7fcc000557c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.598+0000 7fcc228f5700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc1c0b3240 0x7fcc1c0abae0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.598+0000 7fcc228f5700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc1c066400 0x7fcc1c0ab5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.598+0000 7fcc228f5700 1 --2- 172.21.2.104:0/1058936384 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc1c064ab0 0x7fcc1c0ac790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.598+0000 7fcc228f5700 1 -- 172.21.2.104:0/1058936384 >> 172.21.2.104:0/1058936384 conn(0x7fcc1c05c490 msgr2=0x7fcc1c0b1810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:36.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.602+0000 7fcc228f5700 1 -- 172.21.2.104:0/1058936384 shutdown_connections 2021-07-11T04:00:36.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.602+0000 7fcc228f5700 1 -- 172.21.2.104:0/1058936384 wait complete. 2021-07-11T04:00:36.606 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T04:00:36.609 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:00:36.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 Processor -- start 2021-07-11T04:00:36.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 -- start start 2021-07-11T04:00:36.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f78ec067180 0x7f78ec05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f78ec0b1970 0x7f78ec05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ec0a7a90 con 0x7f78ec0a76b0 2021-07-11T04:00:36.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ec147620 con 0x7f78ec067180 2021-07-11T04:00:36.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ec1478c0 con 0x7f78ec0b1970 2021-07-11T04:00:36.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f0dcb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f78ec067180 0x7f78ec05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f0dcb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f78ec067180 0x7f78ec05f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47994/0 (socket says 172.21.2.104:47994) 2021-07-11T04:00:36.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f0dcb700 1 -- 172.21.2.104:0/3101318464 learned_addr learned my addr 172.21.2.104:0/3101318464 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:36.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f0dcb700 1 -- 172.21.2.104:0/3101318464 >> v2:172.21.2.105:3300/0 conn(0x7f78ec0b1970 msgr2=0x7f78ec05fba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:36.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f0dcb700 1 --2- 172.21.2.104:0/3101318464 >> v2:172.21.2.105:3300/0 conn(0x7f78ec0b1970 0x7f78ec05fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f0dcb700 1 -- 172.21.2.104:0/3101318464 >> v1:172.21.2.105:6789/0 conn(0x7f78ec0a76b0 legacy=0x7f78ec066a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:36.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f0dcb700 1 -- 172.21.2.104:0/3101318464 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ec147b60 con 0x7f78ec067180 2021-07-11T04:00:36.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f0dcb700 1 --2- 172.21.2.104:0/3101318464 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f78ec067180 0x7f78ec05f6b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f78ec06a0e0 tx=0x7f78e8006eb0).ready entity=mon.0 client_cookie=f316e7fdfd59d4e8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78dffff700 1 -- 172.21.2.104:0/3101318464 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78e800d5c0 con 0x7f78ec067180 2021-07-11T04:00:36.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78dffff700 1 -- 172.21.2.104:0/3101318464 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e8000e90 con 0x7f78ec067180 2021-07-11T04:00:36.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78dffff700 1 -- 172.21.2.104:0/3101318464 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78e800d910 con 0x7f78ec067180 2021-07-11T04:00:36.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 -- 172.21.2.104:0/3101318464 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f78ec067180 msgr2=0x7f78ec05f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 --2- 172.21.2.104:0/3101318464 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f78ec067180 0x7f78ec05f6b0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f78ec06a0e0 tx=0x7f78e8006eb0).stop 2021-07-11T04:00:36.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 -- 172.21.2.104:0/3101318464 shutdown_connections 2021-07-11T04:00:36.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 --2- 172.21.2.104:0/3101318464 >> v2:172.21.2.105:3300/0 conn(0x7f78ec0b1970 0x7f78ec05fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 --2- 172.21.2.104:0/3101318464 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f78ec067180 0x7f78ec05f6b0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 -- 172.21.2.104:0/3101318464 >> 172.21.2.104:0/3101318464 conn(0x7f78ec0b53e0 msgr2=0x7f78ec0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:36.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 -- 172.21.2.104:0/3101318464 shutdown_connections 2021-07-11T04:00:36.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.662+0000 7f78f3830700 1 -- 172.21.2.104:0/3101318464 wait complete. 2021-07-11T04:00:36.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f3830700 1 Processor -- start 2021-07-11T04:00:36.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f3830700 1 -- start start 2021-07-11T04:00:36.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f3830700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f78ec067180 0x7f78ec0aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f3830700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f78ec0a76b0 0x7f78ec15a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f3830700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f78ec0b1970 0x7f78ec15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f3830700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec0a9600 con 0x7f78ec067180 2021-07-11T04:00:36.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f3830700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec0a9830 con 0x7f78ec0b1970 2021-07-11T04:00:36.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f3830700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ec0a9ad0 con 0x7f78ec0a76b0 2021-07-11T04:00:36.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f15cc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f78ec067180 0x7f78ec0aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f15cc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f78ec067180 0x7f78ec0aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:47998/0 (socket says 172.21.2.104:47998) 2021-07-11T04:00:36.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f15cc700 1 -- 172.21.2.104:0/3669866832 learned_addr learned my addr 172.21.2.104:0/3669866832 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:36.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f15cc700 1 -- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f78ec0a76b0 msgr2=0x7f78ec15a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f15cc700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f78ec0a76b0 0x7f78ec15a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f15cc700 1 -- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f78ec0b1970 msgr2=0x7f78ec15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f15cc700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f78ec0b1970 0x7f78ec15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f15cc700 1 -- 172.21.2.104:0/3669866832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e800b1e0 con 0x7f78ec067180 2021-07-11T04:00:36.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f15cc700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f78ec067180 0x7f78ec0aea20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f78e0002fb0 tx=0x7f78e0006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78de7fc700 1 -- 172.21.2.104:0/3669866832 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78e0013460 con 0x7f78ec067180 2021-07-11T04:00:36.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78de7fc700 1 -- 172.21.2.104:0/3669866832 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e00135c0 con 0x7f78ec067180 2021-07-11T04:00:36.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78de7fc700 1 -- 172.21.2.104:0/3669866832 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78e0013920 con 0x7f78ec067180 2021-07-11T04:00:36.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f3830700 1 -- 172.21.2.104:0/3669866832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ec0b3e50 con 0x7f78ec067180 2021-07-11T04:00:36.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f3830700 1 -- 172.21.2.104:0/3669866832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ec0aef10 con 0x7f78ec067180 2021-07-11T04:00:36.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f3830700 1 -- 172.21.2.104:0/3669866832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ec0b3c90 con 0x7f78ec067180 2021-07-11T04:00:36.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78de7fc700 1 -- 172.21.2.104:0/3669866832 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f78e000f070 con 0x7f78ec067180 2021-07-11T04:00:36.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78de7fc700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78cc0531f0 0x7f78cc0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78de7fc700 1 -- 172.21.2.104:0/3669866832 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f78e006acc0 con 0x7f78ec067180 2021-07-11T04:00:36.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f0dcb700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78cc0531f0 0x7f78cc0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.666+0000 7f78f0dcb700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78cc0531f0 0x7f78cc0556b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f78ec06a0e0 tx=0x7f78e8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.670+0000 7f78de7fc700 1 -- 172.21.2.104:0/3669866832 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f78e0042000 con 0x7f78ec067180 2021-07-11T04:00:36.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.778+0000 7f78f3830700 1 -- 172.21.2.104:0/3669866832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f78ec147460 con 0x7f78ec067180 2021-07-11T04:00:36.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78de7fc700 1 -- 172.21.2.104:0/3669866832 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7f78e0045620 con 0x7f78ec067180 2021-07-11T04:00:36.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 -- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78cc0531f0 msgr2=0x7f78cc0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78cc0531f0 0x7f78cc0556b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f78ec06a0e0 tx=0x7f78e8015000).stop 2021-07-11T04:00:36.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 -- 172.21.2.104:0/3669866832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f78ec067180 msgr2=0x7f78ec0aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f78ec067180 0x7f78ec0aea20 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f78e0002fb0 tx=0x7f78e0006e10).stop 2021-07-11T04:00:36.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 -- 172.21.2.104:0/3669866832 shutdown_connections 2021-07-11T04:00:36.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f78ec0b1970 0x7f78ec15a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f78ec0a76b0 0x7f78ec15a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:36.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78cc0531f0 0x7f78cc0556b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 --2- 172.21.2.104:0/3669866832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f78ec067180 0x7f78ec0aea20 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 -- 172.21.2.104:0/3669866832 >> 172.21.2.104:0/3669866832 conn(0x7f78ec0b53e0 msgr2=0x7f78ec0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:36.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 -- 172.21.2.104:0/3669866832 shutdown_connections 2021-07-11T04:00:36.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.786+0000 7f78f3830700 1 -- 172.21.2.104:0/3669866832 wait complete. 2021-07-11T04:00:36.792 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T04:00:36.795 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:36.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.842+0000 7f5e1ae15700 1 Processor -- start 2021-07-11T04:00:36.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.842+0000 7f5e1ae15700 1 -- start start 2021-07-11T04:00:36.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.842+0000 7f5e1ae15700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e140aa860 0x7f5e140b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.842+0000 7f5e1ae15700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5e140ad150 0x7f5e14140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.842+0000 7f5e1ae15700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e140ac370 con 0x7f5e140a9da0 2021-07-11T04:00:36.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.842+0000 7f5e1ae15700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e140abb50 con 0x7f5e140ad150 2021-07-11T04:00:36.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.842+0000 7f5e1ae15700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e140aac40 con 0x7f5e140aa860 2021-07-11T04:00:36.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e18bb1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e140aa860 0x7f5e140b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e13fff700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f5e140a9da0 0x7f5e140aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:44148/0 (socket says 172.21.2.104:44148) 2021-07-11T04:00:36.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e193b2700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5e140ad150 0x7f5e14140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e13fff700 1 -- 172.21.2.104:0/1493331241 learned_addr learned my addr 172.21.2.104:0/1493331241 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:36.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e18bb1700 1 -- 172.21.2.104:0/1493331241 >> v2:172.21.2.105:3300/0 conn(0x7f5e140ad150 msgr2=0x7f5e14140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:36.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e18bb1700 1 --2- 172.21.2.104:0/1493331241 >> v2:172.21.2.105:3300/0 conn(0x7f5e140ad150 0x7f5e14140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e18bb1700 1 -- 172.21.2.104:0/1493331241 >> v1:172.21.2.105:6789/0 conn(0x7f5e140a9da0 legacy=0x7f5e140aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e18bb1700 1 -- 172.21.2.104:0/1493331241 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e140b16b0 con 0x7f5e140aa860 2021-07-11T04:00:36.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e18bb1700 1 --2- 172.21.2.104:0/1493331241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e140aa860 0x7f5e140b11c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5e04002f70 tx=0x7f5e04006df0).ready entity=mon.2 client_cookie=c3f275e4b66d5b08 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e137fe700 1 -- 172.21.2.104:0/1493331241 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e0400d570 con 0x7f5e140aa860 2021-07-11T04:00:36.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e137fe700 1 -- 172.21.2.104:0/1493331241 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e04000e90 con 0x7f5e140aa860 2021-07-11T04:00:36.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e137fe700 1 -- 172.21.2.104:0/1493331241 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e0400d8c0 con 0x7f5e140aa860 2021-07-11T04:00:36.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1493331241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e140aa860 msgr2=0x7f5e140b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 --2- 172.21.2.104:0/1493331241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e140aa860 0x7f5e140b11c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5e04002f70 tx=0x7f5e04006df0).stop 2021-07-11T04:00:36.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1493331241 shutdown_connections 2021-07-11T04:00:36.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 --2- 172.21.2.104:0/1493331241 >> v2:172.21.2.105:3300/0 conn(0x7f5e140ad150 0x7f5e14140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 --2- 172.21.2.104:0/1493331241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e140aa860 0x7f5e140b11c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1493331241 >> 172.21.2.104:0/1493331241 conn(0x7f5e140b64d0 msgr2=0x7f5e140b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:36.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1493331241 shutdown_connections 2021-07-11T04:00:36.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1493331241 wait complete. 2021-07-11T04:00:36.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 Processor -- start 2021-07-11T04:00:36.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 -- start start 2021-07-11T04:00:36.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e140a9da0 0x7f5e140a8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.846+0000 7f5e1ae15700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e140aa860 0x7f5e1405f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e1ae15700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e140ad150 0x7f5e140ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e1ae15700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e140afd20 con 0x7f5e140a9da0 2021-07-11T04:00:36.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e1ae15700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e141460d0 con 0x7f5e140ad150 2021-07-11T04:00:36.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e1ae15700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e14146370 con 0x7f5e140aa860 2021-07-11T04:00:36.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e18bb1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e140a9da0 0x7f5e140a8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e18bb1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e140a9da0 0x7f5e140a8ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48014/0 (socket says 172.21.2.104:48014) 2021-07-11T04:00:36.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e18bb1700 1 -- 172.21.2.104:0/1978346343 learned_addr learned my addr 172.21.2.104:0/1978346343 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:36.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e193b2700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e140ad150 0x7f5e140ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e13fff700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e140aa860 0x7f5e1405f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e18bb1700 1 -- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e140aa860 msgr2=0x7f5e1405f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e18bb1700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e140aa860 0x7f5e1405f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e18bb1700 1 -- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e140ad150 msgr2=0x7f5e140ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e18bb1700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e140ad150 0x7f5e140ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e18bb1700 1 -- 172.21.2.104:0/1978346343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0400b180 con 0x7f5e140a9da0 2021-07-11T04:00:36.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e18bb1700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e140a9da0 0x7f5e140a8ff0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f5e04002f70 tx=0x7f5e04014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e11ffb700 1 -- 172.21.2.104:0/1978346343 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e04012a30 con 0x7f5e140a9da0 2021-07-11T04:00:36.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e11ffb700 1 -- 172.21.2.104:0/1978346343 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e04012b90 con 0x7f5e140a9da0 2021-07-11T04:00:36.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e11ffb700 1 -- 172.21.2.104:0/1978346343 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5e0401e430 con 0x7f5e140a9da0 2021-07-11T04:00:36.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1978346343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e140a88b0 con 0x7f5e140a9da0 2021-07-11T04:00:36.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1978346343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e140aec80 con 0x7f5e140a9da0 2021-07-11T04:00:36.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1978346343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e140aef30 con 0x7f5e140a9da0 2021-07-11T04:00:36.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e11ffb700 1 -- 172.21.2.104:0/1978346343 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5e0400f030 con 0x7f5e140a9da0 2021-07-11T04:00:36.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e11ffb700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5df80532b0 0x7f5df8055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:36.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e11ffb700 1 -- 172.21.2.104:0/1978346343 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5e0406c360 con 0x7f5e140a9da0 2021-07-11T04:00:36.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e13fff700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5df80532b0 0x7f5df8055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:36.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.850+0000 7f5e13fff700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5df80532b0 0x7f5df8055770 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f5dfc00b000 tx=0x7f5dfc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:36.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.854+0000 7f5e11ffb700 1 -- 172.21.2.104:0/1978346343 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5e040436a0 con 0x7f5e140a9da0 2021-07-11T04:00:36.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1978346343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e14057350 con 0x7f5e140a9da0 2021-07-11T04:00:36.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e11ffb700 1 -- 172.21.2.104:0/1978346343 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2481 (secure 0 0 0) 0x7f5e04046cc0 con 0x7f5e140a9da0 2021-07-11T04:00:36.968 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:36.968 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/873566337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":873566337},{"type":"v1","addr":"172.21.2.105:6835","nonce":873566337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/930744033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":930744033},{"type":"v1","addr":"172.21.2.104:6833","nonce":930744033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:35.070420+0000","modified":"2021-07-11T04:00:36.075051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/18122340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":18122340},{"type":"v1","addr":"172.21.2.104:6835","nonce":18122340}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-07-11T04:00:36.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5df80532b0 msgr2=0x7f5df8055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5df80532b0 0x7f5df8055770 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f5dfc00b000 tx=0x7f5dfc009040).stop 2021-07-11T04:00:36.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1978346343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e140a9da0 msgr2=0x7f5e140a8ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:36.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e140a9da0 0x7f5e140a8ff0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f5e04002f70 tx=0x7f5e04014000).stop 2021-07-11T04:00:36.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1978346343 shutdown_connections 2021-07-11T04:00:36.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5e140ad150 0x7f5e140ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5e140aa860 0x7f5e1405f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5df80532b0 0x7f5df8055770 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 --2- 172.21.2.104:0/1978346343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5e140a9da0 0x7f5e140a8ff0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:36.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1978346343 >> 172.21.2.104:0/1978346343 conn(0x7f5e140b64d0 msgr2=0x7f5e140607c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:36.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1978346343 shutdown_connections 2021-07-11T04:00:36.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:36.966+0000 7f5e1ae15700 1 -- 172.21.2.104:0/1978346343 wait complete. 2021-07-11T04:00:36.973 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 45 2021-07-11T04:00:37.277 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:00:35.070420+0000', 'modified': '2021-07-11T04:00:36.075051+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6972}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6972': {'gid': 6972, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/18122340', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 18122340}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 18122340}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:00:37.278 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:00:37.279 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:00:37.341 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":6972,"want_state":"up:active","state":"up:active","rank_uptime":2.26534046,"mdsmap_epoch":45,"osdmap_epoch":98,"osdmap_epoch_barrier":97,"uptime":6.4357218759999997} 2021-07-11T04:00:37.342 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 6972, "mdsmap_epoch": 45, "osdmap_epoch": 98, "osdmap_epoch_barrier": 97, "rank_uptime": 2.26534046, "state": "up:active", "uptime": 6.435721876, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:00:37.343 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:37.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 Processor -- start 2021-07-11T04:00:37.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 -- start start 2021-07-11T04:00:37.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb300067180 0x7fb30005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb3000b1970 0x7fb30005fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3000a7a90 con 0x7fb3000a76b0 2021-07-11T04:00:37.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb300147620 con 0x7fb3000b1970 2021-07-11T04:00:37.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3001478c0 con 0x7fb300067180 2021-07-11T04:00:37.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb2fed9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb3000b1970 0x7fb30005fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb2fed9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb3000b1970 0x7fb30005fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48024/0 (socket says 172.21.2.104:48024) 2021-07-11T04:00:37.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb2fed9d700 1 -- 172.21.2.104:0/2218051976 learned_addr learned my addr 172.21.2.104:0/2218051976 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:37.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb2fed9d700 1 -- 172.21.2.104:0/2218051976 >> v2:172.21.2.105:3300/0 conn(0x7fb300067180 msgr2=0x7fb30005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:37.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb2fed9d700 1 --2- 172.21.2.104:0/2218051976 >> v2:172.21.2.105:3300/0 conn(0x7fb300067180 0x7fb30005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb2fed9d700 1 -- 172.21.2.104:0/2218051976 >> v1:172.21.2.105:6789/0 conn(0x7fb3000a76b0 legacy=0x7fb300066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:37.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb2fed9d700 1 -- 172.21.2.104:0/2218051976 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb300147b60 con 0x7fb3000b1970 2021-07-11T04:00:37.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb2fed9d700 1 --2- 172.21.2.104:0/2218051976 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3000b1970 0x7fb30005fba0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7fb30006a0e0 tx=0x7fb2f8009290).ready entity=mon.0 client_cookie=2a13168a6e0e11fe server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb2fd59a700 1 -- 172.21.2.104:0/2218051976 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f800b830 con 0x7fb3000b1970 2021-07-11T04:00:37.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb2fd59a700 1 -- 172.21.2.104:0/2218051976 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f800b990 con 0x7fb3000b1970 2021-07-11T04:00:37.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb2fd59a700 1 -- 172.21.2.104:0/2218051976 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f800bb00 con 0x7fb3000b1970 2021-07-11T04:00:37.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 -- 172.21.2.104:0/2218051976 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3000b1970 msgr2=0x7fb30005fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 --2- 172.21.2.104:0/2218051976 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3000b1970 0x7fb30005fba0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7fb30006a0e0 tx=0x7fb2f8009290).stop 2021-07-11T04:00:37.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 -- 172.21.2.104:0/2218051976 shutdown_connections 2021-07-11T04:00:37.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 --2- 172.21.2.104:0/2218051976 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3000b1970 0x7fb30005fba0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:37.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 --2- 172.21.2.104:0/2218051976 >> v2:172.21.2.105:3300/0 conn(0x7fb300067180 0x7fb30005f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 -- 172.21.2.104:0/2218051976 >> 172.21.2.104:0/2218051976 conn(0x7fb3000b53e0 msgr2=0x7fb3000b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:37.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 -- 172.21.2.104:0/2218051976 shutdown_connections 2021-07-11T04:00:37.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.390+0000 7fb304af5700 1 -- 172.21.2.104:0/2218051976 wait complete. 2021-07-11T04:00:37.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb304af5700 1 Processor -- start 2021-07-11T04:00:37.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb304af5700 1 -- start start 2021-07-11T04:00:37.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb304af5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb300067180 0x7fb3000ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb304af5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3000a76b0 0x7fb300149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb304af5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3000b1970 0x7fb30015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb304af5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000b4e60 con 0x7fb3000a76b0 2021-07-11T04:00:37.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb304af5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb300159c60 con 0x7fb300067180 2021-07-11T04:00:37.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb304af5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000aa2e0 con 0x7fb3000b1970 2021-07-11T04:00:37.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2fdd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3000a76b0 0x7fb300149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2fdd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3000a76b0 0x7fb300149ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48028/0 (socket says 172.21.2.104:48028) 2021-07-11T04:00:37.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2fdd9b700 1 -- 172.21.2.104:0/1500263240 learned_addr learned my addr 172.21.2.104:0/1500263240 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:37.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2fdd9b700 1 -- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3000b1970 msgr2=0x7fb30015a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2fdd9b700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3000b1970 0x7fb30015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2fdd9b700 1 -- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb300067180 msgr2=0x7fb3000ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2fdd9b700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb300067180 0x7fb3000ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2fdd9b700 1 -- 172.21.2.104:0/1500263240 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f8009530 con 0x7fb3000a76b0 2021-07-11T04:00:37.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2fdd9b700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3000a76b0 0x7fb300149ed0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fb3000aaaf0 tx=0x7fb2f0006ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2f77fe700 1 -- 172.21.2.104:0/1500263240 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f000e470 con 0x7fb3000a76b0 2021-07-11T04:00:37.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2f77fe700 1 -- 172.21.2.104:0/1500263240 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f000e5d0 con 0x7fb3000a76b0 2021-07-11T04:00:37.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb304af5700 1 -- 172.21.2.104:0/1500263240 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3000aa580 con 0x7fb3000a76b0 2021-07-11T04:00:37.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2f77fe700 1 -- 172.21.2.104:0/1500263240 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb2f000e930 con 0x7fb3000a76b0 2021-07-11T04:00:37.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb304af5700 1 -- 172.21.2.104:0/1500263240 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb300156690 con 0x7fb3000a76b0 2021-07-11T04:00:37.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb304af5700 1 -- 172.21.2.104:0/1500263240 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3000aa050 con 0x7fb3000a76b0 2021-07-11T04:00:37.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2f77fe700 1 -- 172.21.2.104:0/1500263240 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb2f0018030 con 0x7fb3000a76b0 2021-07-11T04:00:37.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2f77fe700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2e40532b0 0x7fb2e4055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2f77fe700 1 -- 172.21.2.104:0/1500263240 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb2f006a940 con 0x7fb3000a76b0 2021-07-11T04:00:37.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.394+0000 7fb2fe59c700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2e40532b0 0x7fb2e4055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.398+0000 7fb2fe59c700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2e40532b0 0x7fb2e4055770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fb2e8000f70 tx=0x7fb2e80077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.398+0000 7fb2f77fe700 1 -- 172.21.2.104:0/1500263240 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb2f0041d30 con 0x7fb3000a76b0 2021-07-11T04:00:37.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.478+0000 7fb304af5700 1 -- 172.21.2.104:0/1500263240 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb300057350 con 0x7fb3000a76b0 2021-07-11T04:00:37.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb2f77fe700 1 -- 172.21.2.104:0/1500263240 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16041 (secure 0 0 0) 0x7fb2f0023ec0 con 0x7fb3000a76b0 2021-07-11T04:00:37.484 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:37.484 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":98,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:37.089468+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:33.058069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:34.122127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:37.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 -- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2e40532b0 msgr2=0x7fb2e4055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2e40532b0 0x7fb2e4055770 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fb2e8000f70 tx=0x7fb2e80077f0).stop 2021-07-11T04:00:37.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 -- 172.21.2.104:0/1500263240 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3000a76b0 msgr2=0x7fb300149ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3000a76b0 0x7fb300149ed0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fb3000aaaf0 tx=0x7fb2f0006ea0).stop 2021-07-11T04:00:37.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 -- 172.21.2.104:0/1500263240 shutdown_connections 2021-07-11T04:00:37.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb3000b1970 0x7fb30015a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb2e40532b0 0x7fb2e4055770 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:37.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb3000a76b0 0x7fb300149ed0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:37.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 --2- 172.21.2.104:0/1500263240 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb300067180 0x7fb3000ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 -- 172.21.2.104:0/1500263240 >> 172.21.2.104:0/1500263240 conn(0x7fb3000b53e0 msgr2=0x7fb300144740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:37.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 -- 172.21.2.104:0/1500263240 shutdown_connections 2021-07-11T04:00:37.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.482+0000 7fb304af5700 1 -- 172.21.2.104:0/1500263240 wait complete. 2021-07-11T04:00:37.492 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:00:33.058069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:00:34.122127+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:00:37.493 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:00:37.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.542+0000 7fc5b5d34700 1 Processor -- start 2021-07-11T04:00:37.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.542+0000 7fc5b5d34700 1 -- start start 2021-07-11T04:00:37.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.542+0000 7fc5b5d34700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc5b00b05b0 0x7fc5b00b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.542+0000 7fc5b5d34700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc5b00a9c10 0x7fc5b00b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.542+0000 7fc5b5d34700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b0143920 con 0x7fc5b00b3710 2021-07-11T04:00:37.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.542+0000 7fc5b5d34700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b0143bc0 con 0x7fc5b00a9c10 2021-07-11T04:00:37.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.542+0000 7fc5b5d34700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b0143e60 con 0x7fc5b00b05b0 2021-07-11T04:00:37.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc5b00a9c10 0x7fc5b00b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc5b00a9c10 0x7fc5b00b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48036/0 (socket says 172.21.2.104:48036) 2021-07-11T04:00:37.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 -- 172.21.2.104:0/695394529 learned_addr learned my addr 172.21.2.104:0/695394529 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:37.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 -- 172.21.2.104:0/695394529 >> v2:172.21.2.105:3300/0 conn(0x7fc5b00b05b0 msgr2=0x7fc5b00b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:37.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 --2- 172.21.2.104:0/695394529 >> v2:172.21.2.105:3300/0 conn(0x7fc5b00b05b0 0x7fc5b00b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 -- 172.21.2.104:0/695394529 >> v1:172.21.2.105:6789/0 conn(0x7fc5b00b3710 legacy=0x7fc5b013d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:37.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 -- 172.21.2.104:0/695394529 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b0144100 con 0x7fc5b00a9c10 2021-07-11T04:00:37.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 --2- 172.21.2.104:0/695394529 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b00a9c10 0x7fc5b00b10b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fc5b006aa50 tx=0x7fc5a800b530).ready entity=mon.0 client_cookie=c5e44df587e684f1 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5ae7fc700 1 -- 172.21.2.104:0/695394529 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5a800be70 con 0x7fc5b00a9c10 2021-07-11T04:00:37.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5ae7fc700 1 -- 172.21.2.104:0/695394529 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a8010470 con 0x7fc5b00a9c10 2021-07-11T04:00:37.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5ae7fc700 1 -- 172.21.2.104:0/695394529 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5a80107d0 con 0x7fc5b00a9c10 2021-07-11T04:00:37.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- 172.21.2.104:0/695394529 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b00a9c10 msgr2=0x7fc5b00b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 --2- 172.21.2.104:0/695394529 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b00a9c10 0x7fc5b00b10b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fc5b006aa50 tx=0x7fc5a800b530).stop 2021-07-11T04:00:37.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- 172.21.2.104:0/695394529 shutdown_connections 2021-07-11T04:00:37.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 --2- 172.21.2.104:0/695394529 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b00a9c10 0x7fc5b00b10b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:37.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 --2- 172.21.2.104:0/695394529 >> v2:172.21.2.105:3300/0 conn(0x7fc5b00b05b0 0x7fc5b00b0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- 172.21.2.104:0/695394529 >> 172.21.2.104:0/695394529 conn(0x7fc5b00607c0 msgr2=0x7fc5b0060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:37.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- 172.21.2.104:0/695394529 shutdown_connections 2021-07-11T04:00:37.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- 172.21.2.104:0/695394529 wait complete. 2021-07-11T04:00:37.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 Processor -- start 2021-07-11T04:00:37.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- start start 2021-07-11T04:00:37.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5b00a9c10 0x7fc5b013ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5b00b05b0 0x7fc5b014e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b00b3710 0x7fc5b014ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b014e0e0 con 0x7fc5b00b3710 2021-07-11T04:00:37.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b014ef20 con 0x7fc5b00a9c10 2021-07-11T04:00:37.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b013e350 con 0x7fc5b00b05b0 2021-07-11T04:00:37.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b00b3710 0x7fc5b014ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b00b3710 0x7fc5b014ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48044/0 (socket says 172.21.2.104:48044) 2021-07-11T04:00:37.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 -- 172.21.2.104:0/3444727943 learned_addr learned my addr 172.21.2.104:0/3444727943 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:37.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 -- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5b00b05b0 msgr2=0x7fc5b014e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5b00b05b0 0x7fc5b014e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 -- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5b00a9c10 msgr2=0x7fc5b013ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5b00a9c10 0x7fc5b013ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 -- 172.21.2.104:0/3444727943 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a800b7d0 con 0x7fc5b00b3710 2021-07-11T04:00:37.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5affff700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b00b3710 0x7fc5b014ea30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7fc5b006aa50 tx=0x7fc5b0143ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5acff9700 1 -- 172.21.2.104:0/3444727943 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5a80032d0 con 0x7fc5b00b3710 2021-07-11T04:00:37.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5acff9700 1 -- 172.21.2.104:0/3444727943 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a8018610 con 0x7fc5b00b3710 2021-07-11T04:00:37.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- 172.21.2.104:0/3444727943 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b013e5f0 con 0x7fc5b00b3710 2021-07-11T04:00:37.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5acff9700 1 -- 172.21.2.104:0/3444727943 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5a8018970 con 0x7fc5b00b3710 2021-07-11T04:00:37.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- 172.21.2.104:0/3444727943 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b013f940 con 0x7fc5b00b3710 2021-07-11T04:00:37.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5b5d34700 1 -- 172.21.2.104:0/3444727943 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b014e2a0 con 0x7fc5b00b3710 2021-07-11T04:00:37.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.546+0000 7fc5acff9700 1 -- 172.21.2.104:0/3444727943 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc5a802b070 con 0x7fc5b00b3710 2021-07-11T04:00:37.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.550+0000 7fc5acff9700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053300 0x7fc5940557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.550+0000 7fc5acff9700 1 -- 172.21.2.104:0/3444727943 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc5a806f150 con 0x7fc5b00b3710 2021-07-11T04:00:37.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.550+0000 7fc5af7fe700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053300 0x7fc5940557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.550+0000 7fc5af7fe700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053300 0x7fc5940557c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fc5a0002f70 tx=0x7fc5a0006ab0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.550+0000 7fc5acff9700 1 -- 172.21.2.104:0/3444727943 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc5b014e2a0 con 0x7fc5b00b3710 2021-07-11T04:00:37.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 -- 172.21.2.104:0/3444727943 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc5b014e2a0 con 0x7fc5b00b3710 2021-07-11T04:00:37.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5acff9700 1 -- 172.21.2.104:0/3444727943 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v98) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc5b014e2a0 con 0x7fc5b00b3710 2021-07-11T04:00:37.637 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:00:37.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 -- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053300 msgr2=0x7fc5940557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053300 0x7fc5940557c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fc5a0002f70 tx=0x7fc5a0006ab0).stop 2021-07-11T04:00:37.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 -- 172.21.2.104:0/3444727943 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b00b3710 msgr2=0x7fc5b014ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b00b3710 0x7fc5b014ea30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7fc5b006aa50 tx=0x7fc5b0143ab0).stop 2021-07-11T04:00:37.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 -- 172.21.2.104:0/3444727943 shutdown_connections 2021-07-11T04:00:37.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053300 0x7fc5940557c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:37.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b00b3710 0x7fc5b014ea30 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:37.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5b00b05b0 0x7fc5b014e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 --2- 172.21.2.104:0/3444727943 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5b00a9c10 0x7fc5b013ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 -- 172.21.2.104:0/3444727943 >> 172.21.2.104:0/3444727943 conn(0x7fc5b00607c0 msgr2=0x7fc5b00b0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:37.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 -- 172.21.2.104:0/3444727943 shutdown_connections 2021-07-11T04:00:37.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.634+0000 7fc5b5d34700 1 -- 172.21.2.104:0/3444727943 wait complete. 2021-07-11T04:00:37.644 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:00:37.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff26945700 1 Processor -- start 2021-07-11T04:00:37.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff26945700 1 -- start start 2021-07-11T04:00:37.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff26945700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7eff20060100 0x7eff20064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff26945700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7eff200b38d0 0x7eff20065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff26945700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff200604e0 con 0x7eff20144440 2021-07-11T04:00:37.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff26945700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff200656f0 con 0x7eff200b38d0 2021-07-11T04:00:37.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff26945700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff20065c80 con 0x7eff20060100 2021-07-11T04:00:37.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff24ee2700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7eff200b38d0 0x7eff20065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff24ee2700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7eff200b38d0 0x7eff20065200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48052/0 (socket says 172.21.2.104:48052) 2021-07-11T04:00:37.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff24ee2700 1 -- 172.21.2.104:0/4032578658 learned_addr learned my addr 172.21.2.104:0/4032578658 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:37.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff24ee2700 1 -- 172.21.2.104:0/4032578658 >> v2:172.21.2.105:3300/0 conn(0x7eff20060100 msgr2=0x7eff20064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:37.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff24ee2700 1 --2- 172.21.2.104:0/4032578658 >> v2:172.21.2.105:3300/0 conn(0x7eff20060100 0x7eff20064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff24ee2700 1 -- 172.21.2.104:0/4032578658 >> v1:172.21.2.105:6789/0 conn(0x7eff20144440 legacy=0x7eff20144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:37.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff24ee2700 1 -- 172.21.2.104:0/4032578658 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff200b23b0 con 0x7eff200b38d0 2021-07-11T04:00:37.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.694+0000 7eff24ee2700 1 --2- 172.21.2.104:0/4032578658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff200b38d0 0x7eff20065200 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7eff2005d460 tx=0x7eff18009290).ready entity=mon.0 client_cookie=49197764a9764221 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff1effd700 1 -- 172.21.2.104:0/4032578658 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff1800b830 con 0x7eff200b38d0 2021-07-11T04:00:37.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff1effd700 1 -- 172.21.2.104:0/4032578658 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1800b990 con 0x7eff200b38d0 2021-07-11T04:00:37.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff1effd700 1 -- 172.21.2.104:0/4032578658 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff1800bb40 con 0x7eff200b38d0 2021-07-11T04:00:37.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 -- 172.21.2.104:0/4032578658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff200b38d0 msgr2=0x7eff20065200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 --2- 172.21.2.104:0/4032578658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff200b38d0 0x7eff20065200 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7eff2005d460 tx=0x7eff18009290).stop 2021-07-11T04:00:37.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 -- 172.21.2.104:0/4032578658 shutdown_connections 2021-07-11T04:00:37.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 --2- 172.21.2.104:0/4032578658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff200b38d0 0x7eff20065200 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:37.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 --2- 172.21.2.104:0/4032578658 >> v2:172.21.2.105:3300/0 conn(0x7eff20060100 0x7eff20064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 -- 172.21.2.104:0/4032578658 >> 172.21.2.104:0/4032578658 conn(0x7eff2005cf60 msgr2=0x7eff2005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:37.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 -- 172.21.2.104:0/4032578658 shutdown_connections 2021-07-11T04:00:37.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 -- 172.21.2.104:0/4032578658 wait complete. 2021-07-11T04:00:37.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 Processor -- start 2021-07-11T04:00:37.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 -- start start 2021-07-11T04:00:37.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff20060100 0x7eff201414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff200b38d0 0x7eff2013db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff2013e020 0x7eff200b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2013bb40 con 0x7eff2013e020 2021-07-11T04:00:37.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff200af440 con 0x7eff20060100 2021-07-11T04:00:37.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff200af630 con 0x7eff200b38d0 2021-07-11T04:00:37.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff24ee2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff2013e020 0x7eff200b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff24ee2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff2013e020 0x7eff200b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48058/0 (socket says 172.21.2.104:48058) 2021-07-11T04:00:37.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff24ee2700 1 -- 172.21.2.104:0/2335690922 learned_addr learned my addr 172.21.2.104:0/2335690922 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:37.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff24ee2700 1 -- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff200b38d0 msgr2=0x7eff2013db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff24ee2700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff200b38d0 0x7eff2013db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff24ee2700 1 -- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff20060100 msgr2=0x7eff201414c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff24ee2700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff20060100 0x7eff201414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff24ee2700 1 -- 172.21.2.104:0/2335690922 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff18009530 con 0x7eff2013e020 2021-07-11T04:00:37.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff24ee2700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff2013e020 0x7eff200b6170 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7eff2005d460 tx=0x7eff18016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff1d7fa700 1 -- 172.21.2.104:0/2335690922 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff18011cc0 con 0x7eff2013e020 2021-07-11T04:00:37.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff1d7fa700 1 -- 172.21.2.104:0/2335690922 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff18011e20 con 0x7eff2013e020 2021-07-11T04:00:37.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 -- 172.21.2.104:0/2335690922 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff200af950 con 0x7eff2013e020 2021-07-11T04:00:37.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff1d7fa700 1 -- 172.21.2.104:0/2335690922 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff1801f610 con 0x7eff2013e020 2021-07-11T04:00:37.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.698+0000 7eff26945700 1 -- 172.21.2.104:0/2335690922 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff200ae300 con 0x7eff2013e020 2021-07-11T04:00:37.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.702+0000 7eff26945700 1 -- 172.21.2.104:0/2335690922 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff200ae040 con 0x7eff2013e020 2021-07-11T04:00:37.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.702+0000 7eff1d7fa700 1 -- 172.21.2.104:0/2335690922 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7eff1800f070 con 0x7eff2013e020 2021-07-11T04:00:37.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.702+0000 7eff1d7fa700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff040532b0 0x7eff04055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.702+0000 7eff1d7fa700 1 -- 172.21.2.104:0/2335690922 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7eff1806bb50 con 0x7eff2013e020 2021-07-11T04:00:37.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.702+0000 7eff1ffff700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff040532b0 0x7eff04055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.702+0000 7eff1ffff700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff040532b0 0x7eff04055770 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7eff08002f70 tx=0x7eff08006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.702+0000 7eff1d7fa700 1 -- 172.21.2.104:0/2335690922 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7eff200ae040 con 0x7eff2013e020 2021-07-11T04:00:37.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.782+0000 7eff26945700 1 -- 172.21.2.104:0/2335690922 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7eff200ae040 con 0x7eff2013e020 2021-07-11T04:00:37.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.782+0000 7eff1d7fa700 1 -- 172.21.2.104:0/2335690922 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v98) v1 ==== 92+0+10 (secure 0 0 0) 0x7eff200ae040 con 0x7eff2013e020 2021-07-11T04:00:37.787 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:37.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 -- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff040532b0 msgr2=0x7eff04055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff040532b0 0x7eff04055770 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7eff08002f70 tx=0x7eff08006a80).stop 2021-07-11T04:00:37.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 -- 172.21.2.104:0/2335690922 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff2013e020 msgr2=0x7eff200b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff2013e020 0x7eff200b6170 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7eff2005d460 tx=0x7eff18016000).stop 2021-07-11T04:00:37.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 -- 172.21.2.104:0/2335690922 shutdown_connections 2021-07-11T04:00:37.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff040532b0 0x7eff04055770 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:37.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff2013e020 0x7eff200b6170 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:37.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff200b38d0 0x7eff2013db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 --2- 172.21.2.104:0/2335690922 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff20060100 0x7eff201414c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 -- 172.21.2.104:0/2335690922 >> 172.21.2.104:0/2335690922 conn(0x7eff2005cf60 msgr2=0x7eff20066910 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:37.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 -- 172.21.2.104:0/2335690922 shutdown_connections 2021-07-11T04:00:37.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.786+0000 7eff26945700 1 -- 172.21.2.104:0/2335690922 wait complete. 2021-07-11T04:00:37.794 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:00:37.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.842+0000 7faca1363700 1 Processor -- start 2021-07-11T04:00:37.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.842+0000 7faca1363700 1 -- start start 2021-07-11T04:00:37.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.842+0000 7faca1363700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac9c0b12d0 0x7fac9c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.842+0000 7faca1363700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c0a77d0 con 0x7fac9c0b1ec0 2021-07-11T04:00:37.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.842+0000 7faca1363700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c0b0650 con 0x7fac9c0b22a0 2021-07-11T04:00:37.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.842+0000 7faca1363700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0b1740 con 0x7fac9c0b12d0 2021-07-11T04:00:37.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.842+0000 7fac9affd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fac9c0b1ec0 0x7fac9c0a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44830/0 (socket says 172.21.2.104:44830) 2021-07-11T04:00:37.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.842+0000 7fac9affd700 1 -- 172.21.2.104:0/3569677597 learned_addr learned my addr 172.21.2.104:0/3569677597 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:37.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2546082485 0 0) 0x7fac9c0a77d0 con 0x7fac9c0b1ec0 2021-07-11T04:00:37.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac80003260 con 0x7fac9c0b1ec0 2021-07-11T04:00:37.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2936501350 0 0) 0x7fac80003260 con 0x7fac9c0b1ec0 2021-07-11T04:00:37.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac9c0b12d0 msgr2=0x7fac9c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:37.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 --2- 172.21.2.104:0/3569677597 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac9c0b12d0 0x7fac9c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 >> v1:172.21.2.105:6789/0 conn(0x7fac9c0b22a0 legacy=0x7fac9c141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:37.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c0b08f0 con 0x7fac9c0b1ec0 2021-07-11T04:00:37.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fac8c001660 con 0x7fac9c0b1ec0 2021-07-11T04:00:37.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 >> v1:172.21.2.104:6789/0 conn(0x7fac9c0b1ec0 legacy=0x7fac9c0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 --2- 172.21.2.104:0/3569677597 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac9c0b12d0 0x7fac80003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 --2- 172.21.2.104:0/3569677597 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac80003ef0 0x7fac80004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 --2- 172.21.2.104:0/3569677597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac80004bf0 0x7fac8000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0b1740 con 0x7fac80004bf0 2021-07-11T04:00:37.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac800052f0 con 0x7fac80003ef0 2021-07-11T04:00:37.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8000bc40 con 0x7fac9c0b12d0 2021-07-11T04:00:37.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 --2- 172.21.2.104:0/3569677597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac80004bf0 0x7fac8000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 -- 172.21.2.104:0/3569677597 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac9c0b12d0 msgr2=0x7fac80003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:37.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 --2- 172.21.2.104:0/3569677597 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac9c0b12d0 0x7fac80003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 -- 172.21.2.104:0/3569677597 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac80003ef0 msgr2=0x7fac80004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:37.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 --2- 172.21.2.104:0/3569677597 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac80003ef0 0x7fac80004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 -- 172.21.2.104:0/3569677597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac80003260 con 0x7fac80004bf0 2021-07-11T04:00:37.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 --2- 172.21.2.104:0/3569677597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac80004bf0 0x7fac8000b750 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7fac940012f0 tx=0x7fac94007d60).ready entity=mon.0 client_cookie=8bc78ff1865a8b3b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac940014c0 con 0x7fac80004bf0 2021-07-11T04:00:37.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac94006870 con 0x7fac80004bf0 2021-07-11T04:00:37.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac99ffb700 1 -- 172.21.2.104:0/3569677597 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac94006bd0 con 0x7fac80004bf0 2021-07-11T04:00:37.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 -- 172.21.2.104:0/3569677597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac80004bf0 msgr2=0x7fac8000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 --2- 172.21.2.104:0/3569677597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac80004bf0 0x7fac8000b750 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7fac940012f0 tx=0x7fac94007d60).stop 2021-07-11T04:00:37.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 -- 172.21.2.104:0/3569677597 reap_dead start 2021-07-11T04:00:37.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 -- 172.21.2.104:0/3569677597 shutdown_connections 2021-07-11T04:00:37.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 -- 172.21.2.104:0/3569677597 >> 172.21.2.104:0/3569677597 conn(0x7fac9c05f6b0 msgr2=0x7fac9c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:37.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 -- 172.21.2.104:0/3569677597 shutdown_connections 2021-07-11T04:00:37.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 -- 172.21.2.104:0/3569677597 wait complete. 2021-07-11T04:00:37.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 Processor -- start 2021-07-11T04:00:37.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 -- start start 2021-07-11T04:00:37.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac9c138140 0x7fac9c149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac9c140890 0x7fac9c149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac9c14b700 0x7fac9c148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0abc50 con 0x7fac9c14b700 2021-07-11T04:00:37.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0ab040 con 0x7fac9c138140 2021-07-11T04:00:37.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c14afc0 con 0x7fac9c140890 2021-07-11T04:00:37.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac9c14b700 0x7fac9c148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac9c14b700 0x7fac9c148cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48076/0 (socket says 172.21.2.104:48076) 2021-07-11T04:00:37.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 -- 172.21.2.104:0/1650844552 learned_addr learned my addr 172.21.2.104:0/1650844552 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:37.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 -- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac9c140890 msgr2=0x7fac9c149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac9c140890 0x7fac9c149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 -- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac9c138140 msgr2=0x7fac9c149a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac9c138140 0x7fac9c149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 -- 172.21.2.104:0/1650844552 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c14b260 con 0x7fac9c14b700 2021-07-11T04:00:37.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac9b7fe700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac9c14b700 0x7fac9c148cb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7fac9c062300 tx=0x7fac94016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac8bfff700 1 -- 172.21.2.104:0/1650844552 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac94006c70 con 0x7fac9c14b700 2021-07-11T04:00:37.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac8bfff700 1 -- 172.21.2.104:0/1650844552 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac94006dd0 con 0x7fac9c14b700 2021-07-11T04:00:37.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 -- 172.21.2.104:0/1650844552 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c138cb0 con 0x7fac9c14b700 2021-07-11T04:00:37.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7fac8bfff700 1 -- 172.21.2.104:0/1650844552 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac94007220 con 0x7fac9c14b700 2021-07-11T04:00:37.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.846+0000 7faca1363700 1 -- 172.21.2.104:0/1650844552 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c1386c0 con 0x7fac9c14b700 2021-07-11T04:00:37.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.850+0000 7faca1363700 1 -- 172.21.2.104:0/1650844552 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9c14b3f0 con 0x7fac9c14b700 2021-07-11T04:00:37.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.850+0000 7fac8bfff700 1 -- 172.21.2.104:0/1650844552 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fac94010070 con 0x7fac9c14b700 2021-07-11T04:00:37.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.850+0000 7fac8bfff700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac78053260 0x7fac78055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:37.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.850+0000 7fac8bfff700 1 -- 172.21.2.104:0/1650844552 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fac9406c5b0 con 0x7fac9c14b700 2021-07-11T04:00:37.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.850+0000 7fac9affd700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac78053260 0x7fac78055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:37.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.850+0000 7fac9affd700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac78053260 0x7fac78055720 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fac9c06a190 tx=0x7fac9c0b07e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:37.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.850+0000 7fac8bfff700 1 -- 172.21.2.104:0/1650844552 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fac940437e0 con 0x7fac9c14b700 2021-07-11T04:00:37.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 -- 172.21.2.104:0/1650844552 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fac9c057350 con 0x7fac9c14b700 2021-07-11T04:00:37.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7fac8bfff700 1 -- 172.21.2.104:0/1650844552 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+10 (secure 0 0 0) 0x7fac94011370 con 0x7fac9c14b700 2021-07-11T04:00:37.939 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:37.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 -- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac78053260 msgr2=0x7fac78055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac78053260 0x7fac78055720 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fac9c06a190 tx=0x7fac9c0b07e0).stop 2021-07-11T04:00:37.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 -- 172.21.2.104:0/1650844552 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac9c14b700 msgr2=0x7fac9c148cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:37.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac9c14b700 0x7fac9c148cb0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7fac9c062300 tx=0x7fac94016000).stop 2021-07-11T04:00:37.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 -- 172.21.2.104:0/1650844552 shutdown_connections 2021-07-11T04:00:37.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac78053260 0x7fac78055720 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:37.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac9c14b700 0x7fac9c148cb0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:37.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac9c140890 0x7fac9c149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 --2- 172.21.2.104:0/1650844552 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac9c138140 0x7fac9c149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:37.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 -- 172.21.2.104:0/1650844552 >> 172.21.2.104:0/1650844552 conn(0x7fac9c05f6b0 msgr2=0x7fac9c13e370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:37.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.934+0000 7faca1363700 1 -- 172.21.2.104:0/1650844552 shutdown_connections 2021-07-11T04:00:37.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.938+0000 7faca1363700 1 -- 172.21.2.104:0/1650844552 wait complete. 2021-07-11T04:00:37.947 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:00:38.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 Processor -- start 2021-07-11T04:00:38.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 -- start start 2021-07-11T04:00:38.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33900b1ec0 0x7f33900b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f33900affb0 0x7f33900b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f33900a70c0 0x7f3390141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33900b2790 con 0x7f33900affb0 2021-07-11T04:00:38.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33900b03f0 con 0x7f33900a70c0 2021-07-11T04:00:38.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33900b0650 con 0x7f33900b1ec0 2021-07-11T04:00:38.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338e59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f33900affb0 0x7f33900b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:38.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338e59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f33900affb0 0x7f33900b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48082/0 (socket says 172.21.2.104:48082) 2021-07-11T04:00:38.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338e59c700 1 -- 172.21.2.104:0/557040749 learned_addr learned my addr 172.21.2.104:0/557040749 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:38.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338e59c700 1 -- 172.21.2.104:0/557040749 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33900b1ec0 msgr2=0x7f33900b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:38.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338e59c700 1 --2- 172.21.2.104:0/557040749 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33900b1ec0 0x7f33900b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338e59c700 1 -- 172.21.2.104:0/557040749 >> v2:172.21.2.105:3300/0 conn(0x7f33900a70c0 msgr2=0x7f3390141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:38.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338e59c700 1 --2- 172.21.2.104:0/557040749 >> v2:172.21.2.105:3300/0 conn(0x7f33900a70c0 0x7f3390141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338e59c700 1 -- 172.21.2.104:0/557040749 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33900b08f0 con 0x7f33900affb0 2021-07-11T04:00:38.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338e59c700 1 --2- 172.21.2.104:0/557040749 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33900affb0 0x7f33900b22a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f3380002f70 tx=0x7f3380006eb0).ready entity=mon.0 client_cookie=cba143e0e25d2367 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:38.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338dd9b700 1 -- 172.21.2.104:0/557040749 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338000d570 con 0x7f33900affb0 2021-07-11T04:00:38.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338dd9b700 1 -- 172.21.2.104:0/557040749 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3380000e90 con 0x7f33900affb0 2021-07-11T04:00:38.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f338dd9b700 1 -- 172.21.2.104:0/557040749 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338000d900 con 0x7f33900affb0 2021-07-11T04:00:38.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 -- 172.21.2.104:0/557040749 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33900affb0 msgr2=0x7f33900b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:38.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 --2- 172.21.2.104:0/557040749 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33900affb0 0x7f33900b22a0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f3380002f70 tx=0x7f3380006eb0).stop 2021-07-11T04:00:38.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 -- 172.21.2.104:0/557040749 shutdown_connections 2021-07-11T04:00:38.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 --2- 172.21.2.104:0/557040749 >> v2:172.21.2.105:3300/0 conn(0x7f33900a70c0 0x7f3390141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 --2- 172.21.2.104:0/557040749 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33900affb0 0x7f33900b22a0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:38.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 --2- 172.21.2.104:0/557040749 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33900b1ec0 0x7f33900b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 -- 172.21.2.104:0/557040749 >> 172.21.2.104:0/557040749 conn(0x7f339005f6b0 msgr2=0x7f339005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:38.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 -- 172.21.2.104:0/557040749 shutdown_connections 2021-07-11T04:00:38.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:37.998+0000 7f33951b0700 1 -- 172.21.2.104:0/557040749 wait complete. 2021-07-11T04:00:38.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f33951b0700 1 Processor -- start 2021-07-11T04:00:38.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f33951b0700 1 -- start start 2021-07-11T04:00:38.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f33951b0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33900a70c0 0x7f33900adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f33951b0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33900affb0 0x7f33900aa9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f33951b0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33900b1ec0 0x7f33900b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f33951b0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33900a9960 con 0x7f33900affb0 2021-07-11T04:00:38.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f33951b0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33900ac2e0 con 0x7f33900b1ec0 2021-07-11T04:00:38.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f33951b0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33900ab4c0 con 0x7f33900a70c0 2021-07-11T04:00:38.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f338e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33900affb0 0x7f33900aa9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:38.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f338e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33900affb0 0x7f33900aa9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48090/0 (socket says 172.21.2.104:48090) 2021-07-11T04:00:38.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f338e59c700 1 -- 172.21.2.104:0/1662436113 learned_addr learned my addr 172.21.2.104:0/1662436113 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:38.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f338e59c700 1 -- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33900a70c0 msgr2=0x7f33900adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:38.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f338e59c700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33900a70c0 0x7f33900adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f338e59c700 1 -- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33900b1ec0 msgr2=0x7f33900b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:38.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f338e59c700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33900b1ec0 0x7f33900b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f338e59c700 1 -- 172.21.2.104:0/1662436113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338000b1e0 con 0x7f33900affb0 2021-07-11T04:00:38.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f338e59c700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33900affb0 0x7f33900aa9f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f3380002f70 tx=0x7f3380015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:38.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f3387fff700 1 -- 172.21.2.104:0/1662436113 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3380006be0 con 0x7f33900affb0 2021-07-11T04:00:38.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f3387fff700 1 -- 172.21.2.104:0/1662436113 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338000de20 con 0x7f33900affb0 2021-07-11T04:00:38.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f3387fff700 1 -- 172.21.2.104:0/1662436113 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3380011e90 con 0x7f33900affb0 2021-07-11T04:00:38.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f33951b0700 1 -- 172.21.2.104:0/1662436113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33900ab1c0 con 0x7f33900affb0 2021-07-11T04:00:38.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f33951b0700 1 -- 172.21.2.104:0/1662436113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33900ad310 con 0x7f33900affb0 2021-07-11T04:00:38.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f33951b0700 1 -- 172.21.2.104:0/1662436113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33900ab680 con 0x7f33900affb0 2021-07-11T04:00:38.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f3387fff700 1 -- 172.21.2.104:0/1662436113 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f338000f070 con 0x7f33900affb0 2021-07-11T04:00:38.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f3387fff700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3374053240 0x7f3374055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f3387fff700 1 -- 172.21.2.104:0/1662436113 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f338006cd00 con 0x7f33900affb0 2021-07-11T04:00:38.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.002+0000 7f338ed9d700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3374053240 0x7f3374055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:38.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.006+0000 7f338ed9d700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3374053240 0x7f3374055700 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f3378002f70 tx=0x7f3378006990).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:38.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.006+0000 7f3387fff700 1 -- 172.21.2.104:0/1662436113 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3380044040 con 0x7f33900affb0 2021-07-11T04:00:38.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 -- 172.21.2.104:0/1662436113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f33900ad930 con 0x7f33900affb0 2021-07-11T04:00:38.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f3387fff700 1 -- 172.21.2.104:0/1662436113 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3380047660 con 0x7f33900affb0 2021-07-11T04:00:38.090 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:38.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 -- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3374053240 msgr2=0x7f3374055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:38.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3374053240 0x7f3374055700 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f3378002f70 tx=0x7f3378006990).stop 2021-07-11T04:00:38.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 -- 172.21.2.104:0/1662436113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33900affb0 msgr2=0x7f33900aa9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:38.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33900affb0 0x7f33900aa9f0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f3380002f70 tx=0x7f3380015000).stop 2021-07-11T04:00:38.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 -- 172.21.2.104:0/1662436113 shutdown_connections 2021-07-11T04:00:38.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33900b1ec0 0x7f33900b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3374053240 0x7f3374055700 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:38.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33900affb0 0x7f33900aa9f0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:38.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 --2- 172.21.2.104:0/1662436113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33900a70c0 0x7f33900adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 -- 172.21.2.104:0/1662436113 >> 172.21.2.104:0/1662436113 conn(0x7f339005f6b0 msgr2=0x7f339014e220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:38.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 -- 172.21.2.104:0/1662436113 shutdown_connections 2021-07-11T04:00:38.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.086+0000 7f33951b0700 1 -- 172.21.2.104:0/1662436113 wait complete. 2021-07-11T04:00:38.097 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:00:38.098 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:38.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61edaaf700 1 Processor -- start 2021-07-11T04:00:38.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61edaaf700 1 -- start start 2021-07-11T04:00:38.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61edaaf700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f61e80a76b0 0x7f61e8066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61edaaf700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e80a7a90 con 0x7f61e80b0060 2021-07-11T04:00:38.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61edaaf700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e8147620 con 0x7f61e80b1970 2021-07-11T04:00:38.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61edaaf700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e81478c0 con 0x7f61e80a76b0 2021-07-11T04:00:38.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e7fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f61e80b0060 0x7f61e8151f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44866/0 (socket says 172.21.2.104:44866) 2021-07-11T04:00:38.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e7fff700 1 -- 172.21.2.104:0/1271125612 learned_addr learned my addr 172.21.2.104:0/1271125612 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:38.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093002524 0 0) 0x7f61e80a7a90 con 0x7f61e80b0060 2021-07-11T04:00:38.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c8003260 con 0x7f61e80b0060 2021-07-11T04:00:38.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e77fe700 1 --2- 172.21.2.104:0/1271125612 >> v2:172.21.2.105:3300/0 conn(0x7f61e80a76b0 0x7f61e8066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:38.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1780162389 0 0) 0x7f61c8003260 con 0x7f61e80b0060 2021-07-11T04:00:38.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 >> v2:172.21.2.105:3300/0 conn(0x7f61e80a76b0 msgr2=0x7f61e8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:38.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 --2- 172.21.2.104:0/1271125612 >> v2:172.21.2.105:3300/0 conn(0x7f61e80a76b0 0x7f61e8066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:38.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 >> v1:172.21.2.105:6789/0 conn(0x7f61e80b1970 legacy=0x7f61e805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:38.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e8147b60 con 0x7f61e80b0060 2021-07-11T04:00:38.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f61e0001a00 con 0x7f61e80b0060 2021-07-11T04:00:38.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 >> v1:172.21.2.104:6789/0 conn(0x7f61e80b0060 legacy=0x7f61e8151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:38.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 --2- 172.21.2.104:0/1271125612 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61c8003820 0x7f61c8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 --2- 172.21.2.104:0/1271125612 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61c8004500 0x7f61c8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 --2- 172.21.2.104:0/1271125612 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61c8004f70 0x7f61c800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e81478c0 con 0x7f61c8004500 2021-07-11T04:00:38.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c800be50 con 0x7f61c8003820 2021-07-11T04:00:38.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c800c010 con 0x7f61c8004f70 2021-07-11T04:00:38.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e6ffd700 1 --2- 172.21.2.104:0/1271125612 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61c8004500 0x7f61c8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:38.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e6ffd700 1 -- 172.21.2.104:0/1271125612 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61c8004f70 msgr2=0x7f61c800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:38.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e6ffd700 1 --2- 172.21.2.104:0/1271125612 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61c8004f70 0x7f61c800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e6ffd700 1 -- 172.21.2.104:0/1271125612 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61c8003820 msgr2=0x7f61c8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:38.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e6ffd700 1 --2- 172.21.2.104:0/1271125612 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61c8003820 0x7f61c8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e6ffd700 1 -- 172.21.2.104:0/1271125612 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c8003260 con 0x7f61c8004500 2021-07-11T04:00:38.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e7fff700 1 -- 172.21.2.104:0/1271125612 reap_dead start 2021-07-11T04:00:38.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e6ffd700 1 --2- 172.21.2.104:0/1271125612 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61c8004500 0x7f61c8004980 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f61d0001fa0 tx=0x7f61d0007a00).ready entity=mon.0 client_cookie=bba83f75571e4625 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:38.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61d0007030 con 0x7f61c8004500 2021-07-11T04:00:38.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d0002e90 con 0x7f61c8004500 2021-07-11T04:00:38.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61e67fc700 1 -- 172.21.2.104:0/1271125612 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61d0007380 con 0x7f61c8004500 2021-07-11T04:00:38.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61edaaf700 1 -- 172.21.2.104:0/1271125612 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61c8004500 msgr2=0x7f61c8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:38.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.146+0000 7f61edaaf700 1 --2- 172.21.2.104:0/1271125612 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61c8004500 0x7f61c8004980 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f61d0001fa0 tx=0x7f61d0007a00).stop 2021-07-11T04:00:38.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 -- 172.21.2.104:0/1271125612 shutdown_connections 2021-07-11T04:00:38.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 --2- 172.21.2.104:0/1271125612 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61c8004500 0x7f61c8004980 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:38.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 -- 172.21.2.104:0/1271125612 >> 172.21.2.104:0/1271125612 conn(0x7f61e80b53e0 msgr2=0x7f61e80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:38.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 -- 172.21.2.104:0/1271125612 shutdown_connections 2021-07-11T04:00:38.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 -- 172.21.2.104:0/1271125612 wait complete. 2021-07-11T04:00:38.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 Processor -- start 2021-07-11T04:00:38.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 -- start start 2021-07-11T04:00:38.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61c8004500 0x7f61e80abf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e80a9230 0x7f61e8146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61e8066740 0x7f61e81442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80ae8c0 con 0x7f61e80a9230 2021-07-11T04:00:38.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80abd20 con 0x7f61c8004500 2021-07-11T04:00:38.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e80a9880 con 0x7f61e8066740 2021-07-11T04:00:38.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e80a9230 0x7f61e8146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:38.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e80a9230 0x7f61e8146790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48106/0 (socket says 172.21.2.104:48106) 2021-07-11T04:00:38.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e6ffd700 1 -- 172.21.2.104:0/106441262 learned_addr learned my addr 172.21.2.104:0/106441262 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:38.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e6ffd700 1 -- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61e8066740 msgr2=0x7f61e81442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:38.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e6ffd700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61e8066740 0x7f61e81442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e6ffd700 1 -- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61c8004500 msgr2=0x7f61e80abf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:38.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e6ffd700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61c8004500 0x7f61e80abf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e6ffd700 1 -- 172.21.2.104:0/106441262 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e80a9b20 con 0x7f61e80a9230 2021-07-11T04:00:38.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e6ffd700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e80a9230 0x7f61e8146790 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f61d0001fa0 tx=0x7f61d0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:38.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e4ff9700 1 -- 172.21.2.104:0/106441262 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61d0007430 con 0x7f61e80a9230 2021-07-11T04:00:38.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e4ff9700 1 -- 172.21.2.104:0/106441262 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61d0002e90 con 0x7f61e80a9230 2021-07-11T04:00:38.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e4ff9700 1 -- 172.21.2.104:0/106441262 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f61d0006700 con 0x7f61e80a9230 2021-07-11T04:00:38.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 -- 172.21.2.104:0/106441262 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e80ad960 con 0x7f61e80a9230 2021-07-11T04:00:38.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 -- 172.21.2.104:0/106441262 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e80ade00 con 0x7f61e80a9230 2021-07-11T04:00:38.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61edaaf700 1 -- 172.21.2.104:0/106441262 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e8066ee0 con 0x7f61e80a9230 2021-07-11T04:00:38.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e4ff9700 1 -- 172.21.2.104:0/106441262 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f61d0006860 con 0x7f61e80a9230 2021-07-11T04:00:38.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e4ff9700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61cc053330 0x7f61cc0557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:38.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.150+0000 7f61e4ff9700 1 -- 172.21.2.104:0/106441262 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f61d006c230 con 0x7f61e80a9230 2021-07-11T04:00:38.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.154+0000 7f61e77fe700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61cc053330 0x7f61cc0557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:38.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.154+0000 7f61e77fe700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61cc053330 0x7f61cc0557f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f61d800a000 tx=0x7f61d8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:38.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.154+0000 7f61e4ff9700 1 -- 172.21.2.104:0/106441262 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f61d0043460 con 0x7f61e80a9230 2021-07-11T04:00:38.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 -- 172.21.2.104:0/106441262 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61e80adaf0 con 0x7f61e80a9230 2021-07-11T04:00:38.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61e4ff9700 1 -- 172.21.2.104:0/106441262 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2481 (secure 0 0 0) 0x7f61d0025020 con 0x7f61e80a9230 2021-07-11T04:00:38.265 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:38.266 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/873566337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":873566337},{"type":"v1","addr":"172.21.2.105:6835","nonce":873566337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/930744033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":930744033},{"type":"v1","addr":"172.21.2.104:6833","nonce":930744033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:35.070420+0000","modified":"2021-07-11T04:00:36.075051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/18122340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":18122340},{"type":"v1","addr":"172.21.2.104:6835","nonce":18122340}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-07-11T04:00:38.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 -- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61cc053330 msgr2=0x7f61cc0557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:38.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61cc053330 0x7f61cc0557f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f61d800a000 tx=0x7f61d8008040).stop 2021-07-11T04:00:38.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 -- 172.21.2.104:0/106441262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e80a9230 msgr2=0x7f61e8146790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:38.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e80a9230 0x7f61e8146790 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f61d0001fa0 tx=0x7f61d0015000).stop 2021-07-11T04:00:38.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 -- 172.21.2.104:0/106441262 shutdown_connections 2021-07-11T04:00:38.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f61e8066740 0x7f61e81442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f61cc053330 0x7f61cc0557f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:38.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f61e80a9230 0x7f61e8146790 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:38.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 --2- 172.21.2.104:0/106441262 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f61c8004500 0x7f61e80abf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:38.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 -- 172.21.2.104:0/106441262 >> 172.21.2.104:0/106441262 conn(0x7f61e80b53e0 msgr2=0x7f61e814a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:38.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 -- 172.21.2.104:0/106441262 shutdown_connections 2021-07-11T04:00:38.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:38.262+0000 7f61edaaf700 1 -- 172.21.2.104:0/106441262 wait complete. 2021-07-11T04:00:38.269 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 45 2021-07-11T04:00:38.576 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:00:35.070420+0000', 'modified': '2021-07-11T04:00:36.075051+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6972}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6972': {'gid': 6972, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/18122340', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 18122340}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 18122340}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:00:38.576 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:00:38.577 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:00:38.642 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":6972,"want_state":"up:active","state":"up:active","rank_uptime":3.5658513040000002,"mdsmap_epoch":45,"osdmap_epoch":98,"osdmap_epoch_barrier":97,"uptime":7.7362337710000002} 2021-07-11T04:00:38.642 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 6972, "mdsmap_epoch": 45, "osdmap_epoch": 98, "osdmap_epoch_barrier": 97, "rank_uptime": 3.565851304, "state": "up:active", "uptime": 7.736233771, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:00:38.643 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:00:38.643 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T04:00:38.644 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T04:00:38.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:38.645 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:38.667 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:00:38.682 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:38.683 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:38.690 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:38.692 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:38.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:38.693 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:00:38.741 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:00:38.741 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:38.744 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:00:38.754 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:38.754 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:38.759 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:38.760 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:38.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:38.762 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:00:38.783 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T04:00:38.778+0000 7efcde713080 -1 init, newargv = 0x557128c67be0 newargc=9ceph-fuse[15393]: starting ceph client 2021-07-11T04:00:38.783 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr: 2021-07-11T04:00:38.797 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15393]: starting fuse 2021-07-11T04:00:39.888 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:00:39.901 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:39.901 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:39.951 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:39.951 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:39.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:39.953 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:00:39.963 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T04:00:39.963 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:00:39.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:39.964 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T04:00:39.965 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T04:00:39.965 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T04:00:39.966 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T04:00:39.966 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T04:00:39.967 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:39.968 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T04:00:39.968 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T04:00:39.969 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T04:00:39.969 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:39.970 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T04:00:39.970 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T04:00:39.971 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T04:00:39.971 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T04:00:39.972 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:39.972 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T04:00:39.973 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:00:39.986 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:00:39.987 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T04:00:39.987 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:00:39.987 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:39.988 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T04:00:39.988 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T04:00:40.031 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15393.asok 2021-07-11T04:00:40.033 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15393.asok 2021-07-11T04:00:40.034 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:40.034 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15393.asok status 2021-07-11T04:00:40.083 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T04:00:40.084 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T04:00:40.084 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:00:40.084 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:00:40.085 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T04:00:40.085 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T04:00:40.085 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T04:00:40.085 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15393", 2021-07-11T04:00:40.086 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T04:00:40.086 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:40.086 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T04:00:40.086 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T04:00:40.086 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4802, 2021-07-11T04:00:40.087 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T04:00:40.087 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T04:00:40.087 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T04:00:40.087 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4802 2021-07-11T04:00:40.088 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:40.088 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:00:40.088 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:00:40.088 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:00:40.089 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 400102582 2021-07-11T04:00:40.089 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T04:00:40.089 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:40.089 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:00:40.090 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:00:40.090 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:00:40.090 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 400102582 2021-07-11T04:00:40.090 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:40.091 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4802 172.21.2.119:0/400102582", 2021-07-11T04:00:40.091 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/400102582", 2021-07-11T04:00:40.092 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T04:00:40.092 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 45, 2021-07-11T04:00:40.092 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 98, 2021-07-11T04:00:40.092 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:00:40.092 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T04:00:40.093 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T04:00:40.093 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T04:00:40.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:40.095 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:40.095 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:40.101 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:00:40.102 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:00:40.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:40.103 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:40.153 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:40.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c6dffc700 1 Processor -- start 2021-07-11T04:00:40.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c6dffc700 1 -- start start 2021-07-11T04:00:40.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c6dffc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4c68067180 0x7f4c6805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c6dffc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c680b1970 0x7f4c6805fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c6dffc700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c680a7a90 con 0x7f4c680a76b0 2021-07-11T04:00:40.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c6dffc700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c68147620 con 0x7f4c68067180 2021-07-11T04:00:40.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c6dffc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c681478c0 con 0x7f4c680b1970 2021-07-11T04:00:40.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c677fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4c680a76b0 0x7f4c68066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44882/0 (socket says 172.21.2.104:44882) 2021-07-11T04:00:40.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c677fe700 1 -- 172.21.2.104:0/37218311 learned_addr learned my addr 172.21.2.104:0/37218311 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:40.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c66ffd700 1 --2- 172.21.2.104:0/37218311 >> v2:172.21.2.104:3300/0 conn(0x7f4c68067180 0x7f4c6805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c66ffd700 1 -- 172.21.2.104:0/37218311 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c680b1970 msgr2=0x7f4c6805fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:40.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c66ffd700 1 --2- 172.21.2.104:0/37218311 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c680b1970 0x7f4c6805fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c66ffd700 1 -- 172.21.2.104:0/37218311 >> v1:172.21.2.104:6789/0 conn(0x7f4c680a76b0 legacy=0x7f4c68066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.206+0000 7f4c66ffd700 1 -- 172.21.2.104:0/37218311 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c68147b60 con 0x7f4c68067180 2021-07-11T04:00:40.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c66ffd700 1 --2- 172.21.2.104:0/37218311 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c68067180 0x7f4c6805f6b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f4c50001110 tx=0x7f4c58006df0).ready entity=mon.0 client_cookie=ffc1611ee29cc870 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c667fc700 1 -- 172.21.2.104:0/37218311 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c5800d570 con 0x7f4c68067180 2021-07-11T04:00:40.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c667fc700 1 -- 172.21.2.104:0/37218311 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c58000e90 con 0x7f4c68067180 2021-07-11T04:00:40.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c667fc700 1 -- 172.21.2.104:0/37218311 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c5800d8c0 con 0x7f4c68067180 2021-07-11T04:00:40.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- 172.21.2.104:0/37218311 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c68067180 msgr2=0x7f4c6805f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 --2- 172.21.2.104:0/37218311 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c68067180 0x7f4c6805f6b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f4c50001110 tx=0x7f4c58006df0).stop 2021-07-11T04:00:40.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- 172.21.2.104:0/37218311 shutdown_connections 2021-07-11T04:00:40.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 --2- 172.21.2.104:0/37218311 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c68067180 0x7f4c6805f6b0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 --2- 172.21.2.104:0/37218311 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c680b1970 0x7f4c6805fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- 172.21.2.104:0/37218311 >> 172.21.2.104:0/37218311 conn(0x7f4c680b53e0 msgr2=0x7f4c680b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:40.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- 172.21.2.104:0/37218311 shutdown_connections 2021-07-11T04:00:40.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- 172.21.2.104:0/37218311 wait complete. 2021-07-11T04:00:40.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 Processor -- start 2021-07-11T04:00:40.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- start start 2021-07-11T04:00:40.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c68067180 0x7f4c68146790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c680a76b0 0x7f4c6815a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c680b1970 0x7f4c6815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680ac8a0 con 0x7f4c680b1970 2021-07-11T04:00:40.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c680acb40 con 0x7f4c68067180 2021-07-11T04:00:40.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68159ea0 con 0x7f4c680a76b0 2021-07-11T04:00:40.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c67fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c680b1970 0x7f4c6815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c67fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c680b1970 0x7f4c6815a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48124/0 (socket says 172.21.2.104:48124) 2021-07-11T04:00:40.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c67fff700 1 -- 172.21.2.104:0/2783644410 learned_addr learned my addr 172.21.2.104:0/2783644410 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:40.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c67fff700 1 -- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c680a76b0 msgr2=0x7f4c6815a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c67fff700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c680a76b0 0x7f4c6815a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c67fff700 1 -- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c68067180 msgr2=0x7f4c68146790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c67fff700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c68067180 0x7f4c68146790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c67fff700 1 -- 172.21.2.104:0/2783644410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5800b180 con 0x7f4c680b1970 2021-07-11T04:00:40.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c67fff700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c680b1970 0x7f4c6815a830 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f4c600032f0 tx=0x7f4c600091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c64ff9700 1 -- 172.21.2.104:0/2783644410 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c6000f510 con 0x7f4c680b1970 2021-07-11T04:00:40.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c64ff9700 1 -- 172.21.2.104:0/2783644410 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6000f670 con 0x7f4c680b1970 2021-07-11T04:00:40.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- 172.21.2.104:0/2783644410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c680aeee0 con 0x7f4c680b1970 2021-07-11T04:00:40.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- 172.21.2.104:0/2783644410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c680acf30 con 0x7f4c680b1970 2021-07-11T04:00:40.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c64ff9700 1 -- 172.21.2.104:0/2783644410 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c6000f7e0 con 0x7f4c680b1970 2021-07-11T04:00:40.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.210+0000 7f4c6dffc700 1 -- 172.21.2.104:0/2783644410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6815a060 con 0x7f4c680b1970 2021-07-11T04:00:40.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.214+0000 7f4c64ff9700 1 -- 172.21.2.104:0/2783644410 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4c6000f070 con 0x7f4c680b1970 2021-07-11T04:00:40.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.214+0000 7f4c64ff9700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c4c053160 0x7f4c4c055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.214+0000 7f4c64ff9700 1 -- 172.21.2.104:0/2783644410 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4c6006b0f0 con 0x7f4c680b1970 2021-07-11T04:00:40.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.214+0000 7f4c677fe700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c4c053160 0x7f4c4c055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.214+0000 7f4c677fe700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c4c053160 0x7f4c4c055620 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f4c500011d0 tx=0x7f4c50007790).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.214+0000 7f4c64ff9700 1 -- 172.21.2.104:0/2783644410 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4c60042430 con 0x7f4c680b1970 2021-07-11T04:00:40.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.294+0000 7f4c6dffc700 1 -- 172.21.2.104:0/2783644410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c680acd40 con 0x7f4c680b1970 2021-07-11T04:00:40.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.294+0000 7f4c64ff9700 1 -- 172.21.2.104:0/2783644410 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f4c60045a50 con 0x7f4c680b1970 2021-07-11T04:00:40.300 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:40.300 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":98,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:37.089468+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:33.058069+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:34.122127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:40.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 -- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c4c053160 msgr2=0x7f4c4c055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c4c053160 0x7f4c4c055620 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f4c500011d0 tx=0x7f4c50007790).stop 2021-07-11T04:00:40.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 -- 172.21.2.104:0/2783644410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c680b1970 msgr2=0x7f4c6815a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c680b1970 0x7f4c6815a830 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f4c600032f0 tx=0x7f4c600091a0).stop 2021-07-11T04:00:40.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 -- 172.21.2.104:0/2783644410 shutdown_connections 2021-07-11T04:00:40.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c4c053160 0x7f4c4c055620 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c680b1970 0x7f4c6815a830 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c680a76b0 0x7f4c6815a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 --2- 172.21.2.104:0/2783644410 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c68067180 0x7f4c68146790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 -- 172.21.2.104:0/2783644410 >> 172.21.2.104:0/2783644410 conn(0x7f4c680b53e0 msgr2=0x7f4c68149980 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:40.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 -- 172.21.2.104:0/2783644410 shutdown_connections 2021-07-11T04:00:40.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.298+0000 7f4c6dffc700 1 -- 172.21.2.104:0/2783644410 wait complete. 2021-07-11T04:00:40.307 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:00:33.058069+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:00:34.122127+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:00:40.308 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:00:40.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8673796700 1 Processor -- start 2021-07-11T04:00:40.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8673796700 1 -- start start 2021-07-11T04:00:40.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8673796700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f866c0b1ec0 0x7f866c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8673796700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f866c0b22a0 0x7f866c141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8673796700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c0a77d0 con 0x7f866c0affb0 2021-07-11T04:00:40.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8673796700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866c0b03f0 con 0x7f866c0b22a0 2021-07-11T04:00:40.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8673796700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0b0650 con 0x7f866c0b1ec0 2021-07-11T04:00:40.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8671532700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f866c0b1ec0 0x7f866c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8671532700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f866c0b1ec0 0x7f866c0b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:51292/0 (socket says 172.21.2.104:51292) 2021-07-11T04:00:40.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8671532700 1 -- 172.21.2.104:0/3297221460 learned_addr learned my addr 172.21.2.104:0/3297221460 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:40.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8671d33700 1 --2- 172.21.2.104:0/3297221460 >> v2:172.21.2.105:3300/0 conn(0x7f866c0b22a0 0x7f866c141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.358+0000 7f8670d31700 1 --1- 172.21.2.104:0/3297221460 >> v1:172.21.2.105:6789/0 conn(0x7f866c0affb0 0x7f866c0a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:44268/0 (socket says 172.21.2.104:44268) 2021-07-11T04:00:40.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8671532700 1 -- 172.21.2.104:0/3297221460 >> v2:172.21.2.105:3300/0 conn(0x7f866c0b22a0 msgr2=0x7f866c141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:40.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8671532700 1 --2- 172.21.2.104:0/3297221460 >> v2:172.21.2.105:3300/0 conn(0x7f866c0b22a0 0x7f866c141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8671532700 1 -- 172.21.2.104:0/3297221460 >> v1:172.21.2.105:6789/0 conn(0x7f866c0affb0 legacy=0x7f866c0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8671532700 1 -- 172.21.2.104:0/3297221460 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0b08f0 con 0x7f866c0b1ec0 2021-07-11T04:00:40.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8671532700 1 --2- 172.21.2.104:0/3297221460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f866c0b1ec0 0x7f866c0b12d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f8668002f70 tx=0x7f8668006eb0).ready entity=mon.2 client_cookie=6c742495c84ccb34 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f865ffff700 1 -- 172.21.2.104:0/3297221460 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f866800d570 con 0x7f866c0b1ec0 2021-07-11T04:00:40.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f865ffff700 1 -- 172.21.2.104:0/3297221460 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8668000e90 con 0x7f866c0b1ec0 2021-07-11T04:00:40.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f865ffff700 1 -- 172.21.2.104:0/3297221460 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f866800d900 con 0x7f866c0b1ec0 2021-07-11T04:00:40.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 -- 172.21.2.104:0/3297221460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f866c0b1ec0 msgr2=0x7f866c0b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 --2- 172.21.2.104:0/3297221460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f866c0b1ec0 0x7f866c0b12d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f8668002f70 tx=0x7f8668006eb0).stop 2021-07-11T04:00:40.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 -- 172.21.2.104:0/3297221460 shutdown_connections 2021-07-11T04:00:40.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 --2- 172.21.2.104:0/3297221460 >> v2:172.21.2.105:3300/0 conn(0x7f866c0b22a0 0x7f866c141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 --2- 172.21.2.104:0/3297221460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f866c0b1ec0 0x7f866c0b12d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 -- 172.21.2.104:0/3297221460 >> 172.21.2.104:0/3297221460 conn(0x7f866c05f6b0 msgr2=0x7f866c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:40.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 -- 172.21.2.104:0/3297221460 shutdown_connections 2021-07-11T04:00:40.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 -- 172.21.2.104:0/3297221460 wait complete. 2021-07-11T04:00:40.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 Processor -- start 2021-07-11T04:00:40.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 -- start start 2021-07-11T04:00:40.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f866c0affb0 0x7f866c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f866c0b1ec0 0x7f866c1569e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f866c0b22a0 0x7f866c0aba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c14b520 con 0x7f866c0b1ec0 2021-07-11T04:00:40.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c14a1e0 con 0x7f866c0b22a0 2021-07-11T04:00:40.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8673796700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0b6120 con 0x7f866c0affb0 2021-07-11T04:00:40.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8670d31700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f866c0b1ec0 0x7f866c1569e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8670d31700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f866c0b1ec0 0x7f866c1569e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48136/0 (socket says 172.21.2.104:48136) 2021-07-11T04:00:40.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8670d31700 1 -- 172.21.2.104:0/735385609 learned_addr learned my addr 172.21.2.104:0/735385609 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:40.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8670d31700 1 -- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f866c0affb0 msgr2=0x7f866c15aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8670d31700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f866c0affb0 0x7f866c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8670d31700 1 -- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f866c0b22a0 msgr2=0x7f866c0aba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8670d31700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f866c0b22a0 0x7f866c0aba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.362+0000 7f8670d31700 1 -- 172.21.2.104:0/735385609 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866800b1e0 con 0x7f866c0b1ec0 2021-07-11T04:00:40.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f8670d31700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f866c0b1ec0 0x7f866c1569e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f866c06a0f0 tx=0x7f8660007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f865e7fc700 1 -- 172.21.2.104:0/735385609 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f866000ce30 con 0x7f866c0b1ec0 2021-07-11T04:00:40.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f865e7fc700 1 -- 172.21.2.104:0/735385609 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86600072e0 con 0x7f866c0b1ec0 2021-07-11T04:00:40.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f865e7fc700 1 -- 172.21.2.104:0/735385609 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8660007450 con 0x7f866c0b1ec0 2021-07-11T04:00:40.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f8673796700 1 -- 172.21.2.104:0/735385609 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c15a050 con 0x7f866c0b1ec0 2021-07-11T04:00:40.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f8673796700 1 -- 172.21.2.104:0/735385609 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c0b5e10 con 0x7f866c0b1ec0 2021-07-11T04:00:40.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f8673796700 1 -- 172.21.2.104:0/735385609 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c057350 con 0x7f866c0b1ec0 2021-07-11T04:00:40.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f865e7fc700 1 -- 172.21.2.104:0/735385609 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f866001a070 con 0x7f866c0b1ec0 2021-07-11T04:00:40.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f865e7fc700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f864c053270 0x7f864c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f865e7fc700 1 -- 172.21.2.104:0/735385609 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f866006a7f0 con 0x7f866c0b1ec0 2021-07-11T04:00:40.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f8671532700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f864c053270 0x7f864c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f8671532700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f864c053270 0x7f864c055730 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f8668002f70 tx=0x7f8668015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.366+0000 7f865e7fc700 1 -- 172.21.2.104:0/735385609 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8660041ad0 con 0x7f866c0b1ec0 2021-07-11T04:00:40.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 -- 172.21.2.104:0/735385609 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f866c0ac760 con 0x7f866c0b1ec0 2021-07-11T04:00:40.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f865e7fc700 1 -- 172.21.2.104:0/735385609 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v98) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8660018020 con 0x7f866c0b1ec0 2021-07-11T04:00:40.453 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:00:40.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 -- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f864c053270 msgr2=0x7f864c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f864c053270 0x7f864c055730 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f8668002f70 tx=0x7f8668015000).stop 2021-07-11T04:00:40.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 -- 172.21.2.104:0/735385609 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f866c0b1ec0 msgr2=0x7f866c1569e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f866c0b1ec0 0x7f866c1569e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f866c06a0f0 tx=0x7f8660007c40).stop 2021-07-11T04:00:40.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 -- 172.21.2.104:0/735385609 shutdown_connections 2021-07-11T04:00:40.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f866c0b22a0 0x7f866c0aba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f864c053270 0x7f864c055730 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f866c0b1ec0 0x7f866c1569e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 --2- 172.21.2.104:0/735385609 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f866c0affb0 0x7f866c15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 -- 172.21.2.104:0/735385609 >> 172.21.2.104:0/735385609 conn(0x7f866c05f6b0 msgr2=0x7f866c13d900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:40.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 -- 172.21.2.104:0/735385609 shutdown_connections 2021-07-11T04:00:40.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.450+0000 7f8673796700 1 -- 172.21.2.104:0/735385609 wait complete. 2021-07-11T04:00:40.462 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:00:40.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.510+0000 7f404920a700 1 Processor -- start 2021-07-11T04:00:40.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.510+0000 7f404920a700 1 -- start start 2021-07-11T04:00:40.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f40440ac260 0x7f40440b1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40440b06c0 0x7f40440aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40440b1c60 con 0x7f40440a9960 2021-07-11T04:00:40.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40440a9d40 con 0x7f40440b06c0 2021-07-11T04:00:40.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40440b2940 con 0x7f40440ac260 2021-07-11T04:00:40.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40440b06c0 0x7f40440aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40440b06c0 0x7f40440aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48142/0 (socket says 172.21.2.104:48142) 2021-07-11T04:00:40.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 -- 172.21.2.104:0/2232120147 learned_addr learned my addr 172.21.2.104:0/2232120147 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:40.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 -- 172.21.2.104:0/2232120147 >> v2:172.21.2.105:3300/0 conn(0x7f40440ac260 msgr2=0x7f40440b1770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:40.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 --2- 172.21.2.104:0/2232120147 >> v2:172.21.2.105:3300/0 conn(0x7f40440ac260 0x7f40440b1770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 -- 172.21.2.104:0/2232120147 >> v1:172.21.2.105:6789/0 conn(0x7f40440a9960 legacy=0x7f40440aec60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:40.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 -- 172.21.2.104:0/2232120147 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40440b00c0 con 0x7f40440b06c0 2021-07-11T04:00:40.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 --2- 172.21.2.104:0/2232120147 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40440b06c0 0x7f40440aafa0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f40440ac830 tx=0x7f403c009290).ready entity=mon.0 client_cookie=3fef4323ea859949 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f4041d9b700 1 -- 172.21.2.104:0/2232120147 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403c00b830 con 0x7f40440b06c0 2021-07-11T04:00:40.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f4041d9b700 1 -- 172.21.2.104:0/2232120147 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c00b990 con 0x7f40440b06c0 2021-07-11T04:00:40.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f4041d9b700 1 -- 172.21.2.104:0/2232120147 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403c00bb40 con 0x7f40440b06c0 2021-07-11T04:00:40.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- 172.21.2.104:0/2232120147 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40440b06c0 msgr2=0x7f40440aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 --2- 172.21.2.104:0/2232120147 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40440b06c0 0x7f40440aafa0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f40440ac830 tx=0x7f403c009290).stop 2021-07-11T04:00:40.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- 172.21.2.104:0/2232120147 shutdown_connections 2021-07-11T04:00:40.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 --2- 172.21.2.104:0/2232120147 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40440b06c0 0x7f40440aafa0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 --2- 172.21.2.104:0/2232120147 >> v2:172.21.2.105:3300/0 conn(0x7f40440ac260 0x7f40440b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- 172.21.2.104:0/2232120147 >> 172.21.2.104:0/2232120147 conn(0x7f404405f6b0 msgr2=0x7f404405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:40.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- 172.21.2.104:0/2232120147 shutdown_connections 2021-07-11T04:00:40.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- 172.21.2.104:0/2232120147 wait complete. 2021-07-11T04:00:40.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 Processor -- start 2021-07-11T04:00:40.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- start start 2021-07-11T04:00:40.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40440a9960 0x7f4044064480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40440ac260 0x7f4044066ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40440b06c0 0x7f40440ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4044066fa0 con 0x7f40440b06c0 2021-07-11T04:00:40.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40440671d0 con 0x7f40440a9960 2021-07-11T04:00:40.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4044065110 con 0x7f40440ac260 2021-07-11T04:00:40.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40440b06c0 0x7f40440ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40440b06c0 0x7f40440ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48150/0 (socket says 172.21.2.104:48150) 2021-07-11T04:00:40.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 -- 172.21.2.104:0/3355921403 learned_addr learned my addr 172.21.2.104:0/3355921403 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:40.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 -- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40440ac260 msgr2=0x7f4044066ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40440ac260 0x7f4044066ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 -- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40440a9960 msgr2=0x7f4044064480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40440a9960 0x7f4044064480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 -- 172.21.2.104:0/3355921403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c009530 con 0x7f40440b06c0 2021-07-11T04:00:40.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404359e700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40440b06c0 0x7f40440ae5d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f40440ac830 tx=0x7f403c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f403bfff700 1 -- 172.21.2.104:0/3355921403 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403c011cc0 con 0x7f40440b06c0 2021-07-11T04:00:40.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f403bfff700 1 -- 172.21.2.104:0/3355921403 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f403c011e20 con 0x7f40440b06c0 2021-07-11T04:00:40.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- 172.21.2.104:0/3355921403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40440653b0 con 0x7f40440b06c0 2021-07-11T04:00:40.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f403bfff700 1 -- 172.21.2.104:0/3355921403 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f403c01f610 con 0x7f40440b06c0 2021-07-11T04:00:40.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.514+0000 7f404920a700 1 -- 172.21.2.104:0/3355921403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4044063310 con 0x7f40440b06c0 2021-07-11T04:00:40.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.518+0000 7f404920a700 1 -- 172.21.2.104:0/3355921403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4044057350 con 0x7f40440b06c0 2021-07-11T04:00:40.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.518+0000 7f403bfff700 1 -- 172.21.2.104:0/3355921403 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f403c026030 con 0x7f40440b06c0 2021-07-11T04:00:40.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.518+0000 7f403bfff700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40280532b0 0x7f4028055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.518+0000 7f403bfff700 1 -- 172.21.2.104:0/3355921403 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f403c06bd50 con 0x7f40440b06c0 2021-07-11T04:00:40.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.518+0000 7f4042d9d700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40280532b0 0x7f4028055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.518+0000 7f4042d9d700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40280532b0 0x7f4028055770 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f4034002fb0 tx=0x7f40340069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.518+0000 7f403bfff700 1 -- 172.21.2.104:0/3355921403 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f403c043090 con 0x7f40440b06c0 2021-07-11T04:00:40.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 -- 172.21.2.104:0/3355921403 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f40440b1c60 con 0x7f40440b06c0 2021-07-11T04:00:40.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f403bfff700 1 -- 172.21.2.104:0/3355921403 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v98) v1 ==== 92+0+10 (secure 0 0 0) 0x7f40440b1c60 con 0x7f40440b06c0 2021-07-11T04:00:40.605 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:40.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 -- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40280532b0 msgr2=0x7f4028055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40280532b0 0x7f4028055770 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f4034002fb0 tx=0x7f40340069b0).stop 2021-07-11T04:00:40.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 -- 172.21.2.104:0/3355921403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40440b06c0 msgr2=0x7f40440ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40440b06c0 0x7f40440ae5d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f40440ac830 tx=0x7f403c016000).stop 2021-07-11T04:00:40.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 -- 172.21.2.104:0/3355921403 shutdown_connections 2021-07-11T04:00:40.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40280532b0 0x7f4028055770 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40440b06c0 0x7f40440ae5d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40440ac260 0x7f4044066ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 --2- 172.21.2.104:0/3355921403 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40440a9960 0x7f4044064480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 -- 172.21.2.104:0/3355921403 >> 172.21.2.104:0/3355921403 conn(0x7f404405f6b0 msgr2=0x7f4044142740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:40.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 -- 172.21.2.104:0/3355921403 shutdown_connections 2021-07-11T04:00:40.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.602+0000 7f404920a700 1 -- 172.21.2.104:0/3355921403 wait complete. 2021-07-11T04:00:40.612 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:00:40.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 Processor -- start 2021-07-11T04:00:40.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 -- start start 2021-07-11T04:00:40.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff4a80b1ec0 0x7ff4a80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a80b0420 con 0x7ff4a80b22a0 2021-07-11T04:00:40.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a80a77d0 con 0x7ff4a80affb0 2021-07-11T04:00:40.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a80b0650 con 0x7ff4a80b1ec0 2021-07-11T04:00:40.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4a77fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff4a80b22a0 0x7ff4a8141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44928/0 (socket says 172.21.2.104:44928) 2021-07-11T04:00:40.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4a77fe700 1 -- 172.21.2.104:0/3233480301 learned_addr learned my addr 172.21.2.104:0/3233480301 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:40.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4a6ffd700 1 --2- 172.21.2.104:0/3233480301 >> v2:172.21.2.104:3300/0 conn(0x7ff4a80b1ec0 0x7ff4a80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4a6ffd700 1 -- 172.21.2.104:0/3233480301 >> v1:172.21.2.105:6789/0 conn(0x7ff4a80affb0 legacy=0x7ff4a80a70c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:40.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4a6ffd700 1 -- 172.21.2.104:0/3233480301 >> v1:172.21.2.104:6789/0 conn(0x7ff4a80b22a0 legacy=0x7ff4a8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4a6ffd700 1 -- 172.21.2.104:0/3233480301 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a80b08f0 con 0x7ff4a80b1ec0 2021-07-11T04:00:40.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4a6ffd700 1 --2- 172.21.2.104:0/3233480301 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4a80b1ec0 0x7ff4a80b12d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7ff4a80b2f50 tx=0x7ff498006df0).ready entity=mon.0 client_cookie=4f299e49ae08b58b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4a5ffb700 1 -- 172.21.2.104:0/3233480301 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff49800d5c0 con 0x7ff4a80b1ec0 2021-07-11T04:00:40.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4a5ffb700 1 -- 172.21.2.104:0/3233480301 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff498000e90 con 0x7ff4a80b1ec0 2021-07-11T04:00:40.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4a5ffb700 1 -- 172.21.2.104:0/3233480301 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff49800d910 con 0x7ff4a80b1ec0 2021-07-11T04:00:40.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/3233480301 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4a80b1ec0 msgr2=0x7ff4a80b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 --2- 172.21.2.104:0/3233480301 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4a80b1ec0 0x7ff4a80b12d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7ff4a80b2f50 tx=0x7ff498006df0).stop 2021-07-11T04:00:40.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/3233480301 shutdown_connections 2021-07-11T04:00:40.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 --2- 172.21.2.104:0/3233480301 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4a80b1ec0 0x7ff4a80b12d0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/3233480301 >> 172.21.2.104:0/3233480301 conn(0x7ff4a805f6b0 msgr2=0x7ff4a805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:40.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/3233480301 shutdown_connections 2021-07-11T04:00:40.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.662+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/3233480301 wait complete. 2021-07-11T04:00:40.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4ad3bb700 1 Processor -- start 2021-07-11T04:00:40.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4ad3bb700 1 -- start start 2021-07-11T04:00:40.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4ad3bb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4a80affb0 0x7ff4a80b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4ad3bb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff4a80b1ec0 0x7ff4a80ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4ad3bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4a80b22a0 0x7ff4a80ad310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4ad3bb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80aba30 con 0x7ff4a80affb0 2021-07-11T04:00:40.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4ad3bb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80b4590 con 0x7ff4a80b1ec0 2021-07-11T04:00:40.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4ad3bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80b4830 con 0x7ff4a80b22a0 2021-07-11T04:00:40.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4a6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4a80affb0 0x7ff4a80b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4a6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4a80affb0 0x7ff4a80b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48162/0 (socket says 172.21.2.104:48162) 2021-07-11T04:00:40.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4a6ffd700 1 -- 172.21.2.104:0/2822476828 learned_addr learned my addr 172.21.2.104:0/2822476828 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:40.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4a6ffd700 1 -- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4a80b22a0 msgr2=0x7ff4a80ad310 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:40.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4a6ffd700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4a80b22a0 0x7ff4a80ad310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4a6ffd700 1 -- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff4a80b1ec0 msgr2=0x7ff4a80ab100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:40.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4a6ffd700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff4a80b1ec0 0x7ff4a80ab100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4a6ffd700 1 -- 172.21.2.104:0/2822476828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49800b180 con 0x7ff4a80affb0 2021-07-11T04:00:40.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4a6ffd700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4a80affb0 0x7ff4a80b3550 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7ff4a80b2f50 tx=0x7ff498012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff49ffff700 1 -- 172.21.2.104:0/2822476828 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff498002f50 con 0x7ff4a80affb0 2021-07-11T04:00:40.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff49ffff700 1 -- 172.21.2.104:0/2822476828 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff498017030 con 0x7ff4a80affb0 2021-07-11T04:00:40.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/2822476828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a80aa9c0 con 0x7ff4a80affb0 2021-07-11T04:00:40.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff49ffff700 1 -- 172.21.2.104:0/2822476828 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff49801d470 con 0x7ff4a80affb0 2021-07-11T04:00:40.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/2822476828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a8140c30 con 0x7ff4a80affb0 2021-07-11T04:00:40.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/2822476828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a80b49f0 con 0x7ff4a80affb0 2021-07-11T04:00:40.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff49ffff700 1 -- 172.21.2.104:0/2822476828 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff498024030 con 0x7ff4a80affb0 2021-07-11T04:00:40.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff49ffff700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff48c0531f0 0x7ff48c0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff49ffff700 1 -- 172.21.2.104:0/2822476828 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff49806c2f0 con 0x7ff4a80affb0 2021-07-11T04:00:40.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.666+0000 7ff4a67fc700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff48c0531f0 0x7ff48c0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.670+0000 7ff4a67fc700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff48c0531f0 0x7ff48c0556b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7ff490002fb0 tx=0x7ff490006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.670+0000 7ff49ffff700 1 -- 172.21.2.104:0/2822476828 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff498043520 con 0x7ff4a80affb0 2021-07-11T04:00:40.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/2822476828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4a8057350 con 0x7ff4a80affb0 2021-07-11T04:00:40.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff49ffff700 1 -- 172.21.2.104:0/2822476828 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff4980220a0 con 0x7ff4a80affb0 2021-07-11T04:00:40.753 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:40.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff48c0531f0 msgr2=0x7ff48c0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff48c0531f0 0x7ff48c0556b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7ff490002fb0 tx=0x7ff490006930).stop 2021-07-11T04:00:40.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/2822476828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4a80affb0 msgr2=0x7ff4a80b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4a80affb0 0x7ff4a80b3550 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7ff4a80b2f50 tx=0x7ff498012000).stop 2021-07-11T04:00:40.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/2822476828 shutdown_connections 2021-07-11T04:00:40.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4a80b22a0 0x7ff4a80ad310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff4a80b1ec0 0x7ff4a80ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff48c0531f0 0x7ff48c0556b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 --2- 172.21.2.104:0/2822476828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4a80affb0 0x7ff4a80b3550 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/2822476828 >> 172.21.2.104:0/2822476828 conn(0x7ff4a805f6b0 msgr2=0x7ff4a805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:40.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/2822476828 shutdown_connections 2021-07-11T04:00:40.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.750+0000 7ff4ad3bb700 1 -- 172.21.2.104:0/2822476828 wait complete. 2021-07-11T04:00:40.760 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:00:40.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422ffb0700 1 Processor -- start 2021-07-11T04:00:40.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422ffb0700 1 -- start start 2021-07-11T04:00:40.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422ffb0700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f42280a9ec0 0x7f42280aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422ffb0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42280603d0 0x7f422813ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422ffb0700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4228061990 con 0x7f422813d1c0 2021-07-11T04:00:40.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422ffb0700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4228061c30 con 0x7f42280a9ec0 2021-07-11T04:00:40.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422ffb0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4228060b90 con 0x7f42280603d0 2021-07-11T04:00:40.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422e54d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f422813d1c0 0x7f422813f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44942/0 (socket says 172.21.2.104:44942) 2021-07-11T04:00:40.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422e54d700 1 -- 172.21.2.104:0/4262504258 learned_addr learned my addr 172.21.2.104:0/4262504258 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:40.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724365370 0 0) 0x7f4228061990 con 0x7f422813d1c0 2021-07-11T04:00:40.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4210003260 con 0x7f422813d1c0 2021-07-11T04:00:40.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422dd4c700 1 --2- 172.21.2.104:0/4262504258 >> v2:172.21.2.105:3300/0 conn(0x7f42280a9ec0 0x7f42280aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422d54b700 1 --2- 172.21.2.104:0/4262504258 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42280603d0 0x7f422813ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2896546098 0 0) 0x7f4210003260 con 0x7f422813d1c0 2021-07-11T04:00:40.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42280603d0 msgr2=0x7f422813ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:40.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 --2- 172.21.2.104:0/4262504258 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42280603d0 0x7f422813ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 >> v2:172.21.2.105:3300/0 conn(0x7f42280a9ec0 msgr2=0x7f42280aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:40.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 --2- 172.21.2.104:0/4262504258 >> v2:172.21.2.105:3300/0 conn(0x7f42280a9ec0 0x7f42280aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4228060e30 con 0x7f422813d1c0 2021-07-11T04:00:40.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f42200019e0 con 0x7f422813d1c0 2021-07-11T04:00:40.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422dd4c700 1 --2- 172.21.2.104:0/4262504258 >> v2:172.21.2.105:3300/0 conn(0x7f42280a9ec0 0x7f42280aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:40.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 >> v1:172.21.2.104:6789/0 conn(0x7f422813d1c0 legacy=0x7f422813f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 --2- 172.21.2.104:0/4262504258 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f42100037d0 0x7f4210003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 --2- 172.21.2.104:0/4262504258 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42280603d0 0x7f4210004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 --2- 172.21.2.104:0/4262504258 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4210004b10 0x7f4210004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4228061c30 con 0x7f4210004b10 2021-07-11T04:00:40.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4228060b90 con 0x7f42100037d0 2021-07-11T04:00:40.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421000ca70 con 0x7f42280603d0 2021-07-11T04:00:40.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422e54d700 1 --2- 172.21.2.104:0/4262504258 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4210004b10 0x7f4210004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422e54d700 1 -- 172.21.2.104:0/4262504258 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42280603d0 msgr2=0x7f4210004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:40.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422e54d700 1 --2- 172.21.2.104:0/4262504258 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42280603d0 0x7f4210004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422e54d700 1 -- 172.21.2.104:0/4262504258 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f42100037d0 msgr2=0x7f4210003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:40.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422e54d700 1 --2- 172.21.2.104:0/4262504258 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f42100037d0 0x7f4210003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.810+0000 7f422e54d700 1 -- 172.21.2.104:0/4262504258 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210003260 con 0x7f4210004b10 2021-07-11T04:00:40.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422e54d700 1 --2- 172.21.2.104:0/4262504258 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4210004b10 0x7f4210004f90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f422805f870 tx=0x7f4228060fc0).ready entity=mon.0 client_cookie=e7feb0d66420a19c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f422000cb70 con 0x7f4210004b10 2021-07-11T04:00:40.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422000ccd0 con 0x7f4210004b10 2021-07-11T04:00:40.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422cd4a700 1 -- 172.21.2.104:0/4262504258 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4220005b80 con 0x7f4210004b10 2021-07-11T04:00:40.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- 172.21.2.104:0/4262504258 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4210004b10 msgr2=0x7f4210004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 --2- 172.21.2.104:0/4262504258 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4210004b10 0x7f4210004f90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f422805f870 tx=0x7f4228060fc0).stop 2021-07-11T04:00:40.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422e54d700 1 -- 172.21.2.104:0/4262504258 reap_dead start 2021-07-11T04:00:40.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- 172.21.2.104:0/4262504258 shutdown_connections 2021-07-11T04:00:40.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- 172.21.2.104:0/4262504258 >> 172.21.2.104:0/4262504258 conn(0x7f42280b2970 msgr2=0x7f42280b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:40.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- 172.21.2.104:0/4262504258 shutdown_connections 2021-07-11T04:00:40.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- 172.21.2.104:0/4262504258 wait complete. 2021-07-11T04:00:40.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 Processor -- start 2021-07-11T04:00:40.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- start start 2021-07-11T04:00:40.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f422813dd10 0x7f42280a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42281371a0 0x7f4228137620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f422813f4a0 0x7f422813f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f422813e0f0 con 0x7f422813f4a0 2021-07-11T04:00:40.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42280a73e0 con 0x7f422813dd10 2021-07-11T04:00:40.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4228140540 con 0x7f42281371a0 2021-07-11T04:00:40.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422e54d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f422813f4a0 0x7f422813f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422e54d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f422813f4a0 0x7f422813f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48186/0 (socket says 172.21.2.104:48186) 2021-07-11T04:00:40.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422e54d700 1 -- 172.21.2.104:0/2435473838 learned_addr learned my addr 172.21.2.104:0/2435473838 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:40.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422dd4c700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f422813dd10 0x7f42280a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422e54d700 1 -- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42281371a0 msgr2=0x7f4228137620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422e54d700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42281371a0 0x7f4228137620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422e54d700 1 -- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f422813dd10 msgr2=0x7f42280a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422e54d700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f422813dd10 0x7f42280a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422e54d700 1 -- 172.21.2.104:0/2435473838 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42280a9600 con 0x7f422813f4a0 2021-07-11T04:00:40.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422dd4c700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f422813dd10 0x7f42280a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:40.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422e54d700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f422813f4a0 0x7f422813f920 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7f42280abbd0 tx=0x7f4220013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f4216ffd700 1 -- 172.21.2.104:0/2435473838 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f422001b070 con 0x7f422813f4a0 2021-07-11T04:00:40.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f4216ffd700 1 -- 172.21.2.104:0/2435473838 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42200070d0 con 0x7f422813f4a0 2021-07-11T04:00:40.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- 172.21.2.104:0/2435473838 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42280a7b60 con 0x7f422813f4a0 2021-07-11T04:00:40.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f4216ffd700 1 -- 172.21.2.104:0/2435473838 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4220007430 con 0x7f422813f4a0 2021-07-11T04:00:40.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- 172.21.2.104:0/2435473838 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42280a8000 con 0x7f422813f4a0 2021-07-11T04:00:40.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.814+0000 7f422ffb0700 1 -- 172.21.2.104:0/2435473838 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42280a7d20 con 0x7f422813f4a0 2021-07-11T04:00:40.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.818+0000 7f4216ffd700 1 -- 172.21.2.104:0/2435473838 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4220010070 con 0x7f422813f4a0 2021-07-11T04:00:40.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.818+0000 7f4216ffd700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4208053330 0x7f42080557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.818+0000 7f4216ffd700 1 -- 172.21.2.104:0/2435473838 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f422006ae00 con 0x7f422813f4a0 2021-07-11T04:00:40.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.818+0000 7f422dd4c700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4208053330 0x7f42080557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.818+0000 7f422dd4c700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4208053330 0x7f42080557f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f4218008000 tx=0x7f4218008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.818+0000 7f4216ffd700 1 -- 172.21.2.104:0/2435473838 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4220042140 con 0x7f422813f4a0 2021-07-11T04:00:40.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 -- 172.21.2.104:0/2435473838 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42280a9790 con 0x7f422813f4a0 2021-07-11T04:00:40.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f4216ffd700 1 -- 172.21.2.104:0/2435473838 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4220045760 con 0x7f422813f4a0 2021-07-11T04:00:40.908 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:40.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 -- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4208053330 msgr2=0x7f42080557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4208053330 0x7f42080557f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f4218008000 tx=0x7f4218008040).stop 2021-07-11T04:00:40.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 -- 172.21.2.104:0/2435473838 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f422813f4a0 msgr2=0x7f422813f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f422813f4a0 0x7f422813f920 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7f42280abbd0 tx=0x7f4220013000).stop 2021-07-11T04:00:40.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 -- 172.21.2.104:0/2435473838 shutdown_connections 2021-07-11T04:00:40.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4208053330 0x7f42080557f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f422813f4a0 0x7f422813f920 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42281371a0 0x7f4228137620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 --2- 172.21.2.104:0/2435473838 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f422813dd10 0x7f42280a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 -- 172.21.2.104:0/2435473838 >> 172.21.2.104:0/2435473838 conn(0x7f42280b2970 msgr2=0x7f422805c490 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:40.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 -- 172.21.2.104:0/2435473838 shutdown_connections 2021-07-11T04:00:40.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.902+0000 7f422ffb0700 1 -- 172.21.2.104:0/2435473838 wait complete. 2021-07-11T04:00:40.914 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:00:40.915 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:40.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 Processor -- start 2021-07-11T04:00:40.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 -- start start 2021-07-11T04:00:40.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc680a76b0 0x7fcc68066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680a7a90 con 0x7fcc680b1970 2021-07-11T04:00:40.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc68147620 con 0x7fcc680b0060 2021-07-11T04:00:40.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc681478c0 con 0x7fcc680a76b0 2021-07-11T04:00:40.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc677fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fcc680b1970 0x7fcc6805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:44960/0 (socket says 172.21.2.104:44960) 2021-07-11T04:00:40.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc677fe700 1 -- 172.21.2.104:0/3336050640 learned_addr learned my addr 172.21.2.104:0/3336050640 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:40.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102508046 0 0) 0x7fcc680a7a90 con 0x7fcc680b1970 2021-07-11T04:00:40.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc48003260 con 0x7fcc680b1970 2021-07-11T04:00:40.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc67fff700 1 --2- 172.21.2.104:0/3336050640 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc680a76b0 0x7fcc68066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2036961250 0 0) 0x7fcc48003260 con 0x7fcc680b1970 2021-07-11T04:00:40.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc680a76b0 msgr2=0x7fcc68066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:40.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 --2- 172.21.2.104:0/3336050640 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc680a76b0 0x7fcc68066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:40.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 >> v1:172.21.2.105:6789/0 conn(0x7fcc680b0060 legacy=0x7fcc68151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:40.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68147b60 con 0x7fcc680b1970 2021-07-11T04:00:40.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fcc50001660 con 0x7fcc680b1970 2021-07-11T04:00:40.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 >> v1:172.21.2.104:6789/0 conn(0x7fcc680b1970 legacy=0x7fcc6805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 --2- 172.21.2.104:0/3336050640 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc680a76b0 0x7fcc48003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 --2- 172.21.2.104:0/3336050640 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc48003e10 0x7fcc48004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 --2- 172.21.2.104:0/3336050640 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc48004b10 0x7fcc48004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc681478c0 con 0x7fcc48003e10 2021-07-11T04:00:40.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4800b960 con 0x7fcc48004b10 2021-07-11T04:00:40.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc4800bb20 con 0x7fcc680a76b0 2021-07-11T04:00:40.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc677fe700 1 --2- 172.21.2.104:0/3336050640 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc48003e10 0x7fcc48004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc677fe700 1 -- 172.21.2.104:0/3336050640 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc680a76b0 msgr2=0x7fcc48003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:40.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc677fe700 1 --2- 172.21.2.104:0/3336050640 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc680a76b0 0x7fcc48003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc677fe700 1 -- 172.21.2.104:0/3336050640 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc48004b10 msgr2=0x7fcc48004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:40.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc677fe700 1 --2- 172.21.2.104:0/3336050640 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc48004b10 0x7fcc48004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc677fe700 1 -- 172.21.2.104:0/3336050640 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc48003260 con 0x7fcc48003e10 2021-07-11T04:00:40.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc677fe700 1 --2- 172.21.2.104:0/3336050640 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc48003e10 0x7fcc48004500 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7fcc68148420 tx=0x7fcc50001d60).ready entity=mon.0 client_cookie=9ca903cc45cf2ada server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc500070d0 con 0x7fcc48003e10 2021-07-11T04:00:40.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc50007230 con 0x7fcc48003e10 2021-07-11T04:00:40.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc66ffd700 1 -- 172.21.2.104:0/3336050640 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc5000c8f0 con 0x7fcc48003e10 2021-07-11T04:00:40.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/3336050640 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc48003e10 msgr2=0x7fcc48004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 --2- 172.21.2.104:0/3336050640 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc48003e10 0x7fcc48004500 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7fcc68148420 tx=0x7fcc50001d60).stop 2021-07-11T04:00:40.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6c941700 1 -- 172.21.2.104:0/3336050640 reap_dead start 2021-07-11T04:00:40.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/3336050640 shutdown_connections 2021-07-11T04:00:40.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/3336050640 >> 172.21.2.104:0/3336050640 conn(0x7fcc680b53e0 msgr2=0x7fcc680b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:40.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/3336050640 shutdown_connections 2021-07-11T04:00:40.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/3336050640 wait complete. 2021-07-11T04:00:40.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 Processor -- start 2021-07-11T04:00:40.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.966+0000 7fcc6e3a4700 1 -- start start 2021-07-11T04:00:40.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc6e3a4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc68062910 0x7fcc6815a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc6e3a4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc68066110 0x7fcc68066590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc6e3a4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc680a9af0 0x7fcc680a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc6e3a4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6815ac20 con 0x7fcc68062910 2021-07-11T04:00:40.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc6e3a4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc68062cf0 con 0x7fcc68066110 2021-07-11T04:00:40.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc6e3a4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680ade10 con 0x7fcc680a9af0 2021-07-11T04:00:40.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc67fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc68062910 0x7fcc6815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc67fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc68062910 0x7fcc6815a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48204/0 (socket says 172.21.2.104:48204) 2021-07-11T04:00:40.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc67fff700 1 -- 172.21.2.104:0/2544631037 learned_addr learned my addr 172.21.2.104:0/2544631037 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:40.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc67fff700 1 -- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc680a9af0 msgr2=0x7fcc680a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc67fff700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc680a9af0 0x7fcc680a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc67fff700 1 -- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc68066110 msgr2=0x7fcc68066590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:40.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc67fff700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc68066110 0x7fcc68066590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:40.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc67fff700 1 -- 172.21.2.104:0/2544631037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680ae7d0 con 0x7fcc68062910 2021-07-11T04:00:40.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc67fff700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc68062910 0x7fcc6815a730 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7fcc5800a000 tx=0x7fcc58008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc657fa700 1 -- 172.21.2.104:0/2544631037 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc58016070 con 0x7fcc68062910 2021-07-11T04:00:40.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc657fa700 1 -- 172.21.2.104:0/2544631037 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc58000e90 con 0x7fcc68062910 2021-07-11T04:00:40.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc657fa700 1 -- 172.21.2.104:0/2544631037 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcc58011970 con 0x7fcc68062910 2021-07-11T04:00:40.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/2544631037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc680aea70 con 0x7fcc68062910 2021-07-11T04:00:40.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/2544631037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc680af4f0 con 0x7fcc68062910 2021-07-11T04:00:40.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/2544631037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc680af090 con 0x7fcc68062910 2021-07-11T04:00:40.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc657fa700 1 -- 172.21.2.104:0/2544631037 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcc58011ad0 con 0x7fcc68062910 2021-07-11T04:00:40.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc657fa700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc4c0531a0 0x7fcc4c055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:40.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc657fa700 1 -- 172.21.2.104:0/2544631037 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcc5806b600 con 0x7fcc68062910 2021-07-11T04:00:40.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc677fe700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc4c0531a0 0x7fcc4c055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:40.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.970+0000 7fcc677fe700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc4c0531a0 0x7fcc4c055660 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fcc68148420 tx=0x7fcc50015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:40.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:40.974+0000 7fcc657fa700 1 -- 172.21.2.104:0/2544631037 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcc58042830 con 0x7fcc68062910 2021-07-11T04:00:41.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/2544631037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc680a9960 con 0x7fcc68062910 2021-07-11T04:00:41.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc657fa700 1 -- 172.21.2.104:0/2544631037 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2481 (secure 0 0 0) 0x7fcc58045e50 con 0x7fcc68062910 2021-07-11T04:00:41.086 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:41.086 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/873566337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":873566337},{"type":"v1","addr":"172.21.2.105:6835","nonce":873566337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/930744033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":930744033},{"type":"v1","addr":"172.21.2.104:6833","nonce":930744033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:35.070420+0000","modified":"2021-07-11T04:00:36.075051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/18122340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":18122340},{"type":"v1","addr":"172.21.2.104:6835","nonce":18122340}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-07-11T04:00:41.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc4c0531a0 msgr2=0x7fcc4c055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:41.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc4c0531a0 0x7fcc4c055660 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fcc68148420 tx=0x7fcc50015000).stop 2021-07-11T04:00:41.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/2544631037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc68062910 msgr2=0x7fcc6815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:41.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc68062910 0x7fcc6815a730 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7fcc5800a000 tx=0x7fcc58008040).stop 2021-07-11T04:00:41.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/2544631037 shutdown_connections 2021-07-11T04:00:41.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcc680a9af0 0x7fcc680a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:41.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcc68066110 0x7fcc68066590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:41.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcc4c0531a0 0x7fcc4c055660 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:41.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 --2- 172.21.2.104:0/2544631037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcc68062910 0x7fcc6815a730 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:41.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/2544631037 >> 172.21.2.104:0/2544631037 conn(0x7fcc680b53e0 msgr2=0x7fcc68149e90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:41.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/2544631037 shutdown_connections 2021-07-11T04:00:41.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:41.082+0000 7fcc6e3a4700 1 -- 172.21.2.104:0/2544631037 wait complete. 2021-07-11T04:00:41.090 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 45 2021-07-11T04:00:41.395 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 45, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:00:35.070420+0000', 'modified': '2021-07-11T04:00:36.075051+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6972}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6972': {'gid': 6972, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/18122340', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 18122340}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 18122340}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:00:41.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:00:41.396 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:00:41.495 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":6972,"want_state":"up:active","state":"up:active","rank_uptime":6.4196362669999996,"mdsmap_epoch":45,"osdmap_epoch":98,"osdmap_epoch_barrier":97,"uptime":10.590041880999999} 2021-07-11T04:00:41.496 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 6972, "mdsmap_epoch": 45, "osdmap_epoch": 98, "osdmap_epoch_barrier": 97, "rank_uptime": 6.419636267, "state": "up:active", "uptime": 10.590041881, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:00:41.496 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:00:41.497 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-07-11T04:00:41.497 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba023.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-07-11T04:00:41.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:41.498 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:41.512 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:00:41.568 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:41.568 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:41.574 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:41.575 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:41.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:41.577 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:00:41.581 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:00:41.582 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:41.626 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:00:41.635 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:41.635 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:41.640 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:41.641 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:41.642 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:41.642 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:00:41.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:2021-07-11T04:00:41.661+0000 7f3d1f736080 -1 init, newargv = 0x562998595bd0 newargc=9 2021-07-11T04:00:41.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14834]: starting ceph client 2021-07-11T04:00:41.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14834]: starting fuse 2021-07-11T04:00:42.769 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:00:42.782 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:42.783 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:42.790 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:42.791 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:42.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:42.792 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:00:42.837 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:00:42.840 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:00:42.841 INFO:teuthology.orchestra.run.gibba023.stdout:52 2021-07-11T04:00:42.842 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:00:42.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:42.843 DEBUG:teuthology.orchestra.run.gibba023:> sudo python3 -c ' 2021-07-11T04:00:42.843 DEBUG:teuthology.orchestra.run.gibba023:> import glob 2021-07-11T04:00:42.844 DEBUG:teuthology.orchestra.run.gibba023:> import re 2021-07-11T04:00:42.844 DEBUG:teuthology.orchestra.run.gibba023:> import os 2021-07-11T04:00:42.845 DEBUG:teuthology.orchestra.run.gibba023:> import subprocess 2021-07-11T04:00:42.845 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:00:42.845 DEBUG:teuthology.orchestra.run.gibba023:> def find_socket(client_name): 2021-07-11T04:00:42.846 DEBUG:teuthology.orchestra.run.gibba023:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-07-11T04:00:42.846 DEBUG:teuthology.orchestra.run.gibba023:> files = glob.glob(asok_path) 2021-07-11T04:00:42.847 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:00:42.847 DEBUG:teuthology.orchestra.run.gibba023:> # Given a non-glob path, it better be there 2021-07-11T04:00:42.847 DEBUG:teuthology.orchestra.run.gibba023:> if "*" not in asok_path: 2021-07-11T04:00:42.848 DEBUG:teuthology.orchestra.run.gibba023:> assert(len(files) == 1) 2021-07-11T04:00:42.848 DEBUG:teuthology.orchestra.run.gibba023:> return files[0] 2021-07-11T04:00:42.849 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:00:42.849 DEBUG:teuthology.orchestra.run.gibba023:> for f in files: 2021-07-11T04:00:42.849 DEBUG:teuthology.orchestra.run.gibba023:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:00:42.850 DEBUG:teuthology.orchestra.run.gibba023:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:00:42.851 DEBUG:teuthology.orchestra.run.gibba023:> return f 2021-07-11T04:00:42.851 DEBUG:teuthology.orchestra.run.gibba023:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:00:42.851 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:00:42.852 DEBUG:teuthology.orchestra.run.gibba023:> print(find_socket("client.1")) 2021-07-11T04:00:42.852 DEBUG:teuthology.orchestra.run.gibba023:> ' 2021-07-11T04:00:42.856 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:00:42.859 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:00:42.880 INFO:teuthology.orchestra.run.gibba023.stdout:/var/run/ceph/ceph-client.1.14834.asok 2021-07-11T04:00:42.882 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14834.asok 2021-07-11T04:00:42.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:42.883 DEBUG:teuthology.orchestra.run.gibba023:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14834.asok status 2021-07-11T04:00:43.005 INFO:teuthology.orchestra.run.gibba023.stdout:{ 2021-07-11T04:00:43.006 INFO:teuthology.orchestra.run.gibba023.stdout: "metadata": { 2021-07-11T04:00:43.006 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:00:43.006 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:00:43.007 INFO:teuthology.orchestra.run.gibba023.stdout: "entity_id": "1", 2021-07-11T04:00:43.007 INFO:teuthology.orchestra.run.gibba023.stdout: "hostname": "gibba023", 2021-07-11T04:00:43.007 INFO:teuthology.orchestra.run.gibba023.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-07-11T04:00:43.007 INFO:teuthology.orchestra.run.gibba023.stdout: "pid": "14834", 2021-07-11T04:00:43.007 INFO:teuthology.orchestra.run.gibba023.stdout: "root": "/" 2021-07-11T04:00:43.008 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:00:43.008 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_count": 0, 2021-07-11T04:00:43.008 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_pinned_count": 0, 2021-07-11T04:00:43.009 INFO:teuthology.orchestra.run.gibba023.stdout: "id": 5062, 2021-07-11T04:00:43.009 INFO:teuthology.orchestra.run.gibba023.stdout: "inst": { 2021-07-11T04:00:43.009 INFO:teuthology.orchestra.run.gibba023.stdout: "name": { 2021-07-11T04:00:43.009 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "client", 2021-07-11T04:00:43.010 INFO:teuthology.orchestra.run.gibba023.stdout: "num": 5062 2021-07-11T04:00:43.010 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:00:43.010 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:00:43.010 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:00:43.011 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:00:43.011 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 838745187 2021-07-11T04:00:43.011 INFO:teuthology.orchestra.run.gibba023.stdout: } 2021-07-11T04:00:43.011 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:00:43.011 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:00:43.012 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:00:43.012 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:00:43.012 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 838745187 2021-07-11T04:00:43.012 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:00:43.012 INFO:teuthology.orchestra.run.gibba023.stdout: "inst_str": "client.5062 172.21.2.123:0/838745187", 2021-07-11T04:00:43.013 INFO:teuthology.orchestra.run.gibba023.stdout: "addr_str": "172.21.2.123:0/838745187", 2021-07-11T04:00:43.013 INFO:teuthology.orchestra.run.gibba023.stdout: "inode_count": 1, 2021-07-11T04:00:43.013 INFO:teuthology.orchestra.run.gibba023.stdout: "mds_epoch": 45, 2021-07-11T04:00:43.013 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch": 98, 2021-07-11T04:00:43.014 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:00:43.014 INFO:teuthology.orchestra.run.gibba023.stdout: "blacklisted": false, 2021-07-11T04:00:43.014 INFO:teuthology.orchestra.run.gibba023.stdout: "fs_name": "cephfs" 2021-07-11T04:00:43.015 INFO:teuthology.orchestra.run.gibba023.stdout:} 2021-07-11T04:00:43.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:43.018 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:43.019 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:43.027 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:00:43.028 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:00:43.029 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:43.030 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:43.077 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-07-11T04:00:43.078 DEBUG:teuthology.orchestra.run.gibba019:> cd /home/ubuntu/cephtest/mnt.0 && sudo mkdir -p foo 2021-07-11T04:00:43.125 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-07-11T04:00:43.125 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T04:00:43.126 DEBUG:teuthology.orchestra.run.gibba019:> import stat 2021-07-11T04:00:43.126 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:43.127 DEBUG:teuthology.orchestra.run.gibba019:> print(os.stat("/home/ubuntu/cephtest/mnt.0/foo").st_ino) 2021-07-11T04:00:43.127 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T04:00:43.167 INFO:teuthology.orchestra.run.gibba019.stdout:1099511627776 2021-07-11T04:00:43.376 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:43.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cf476700 1 Processor -- start 2021-07-11T04:00:43.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cf476700 1 -- start start 2021-07-11T04:00:43.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cf476700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f85c80b54a0 0x7f85c80af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:43.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cf476700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f85c81475a0 0x7f85c80acc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:43.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cf476700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c81479f0 con 0x7f85c80b0610 2021-07-11T04:00:43.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cf476700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c81452e0 con 0x7f85c80b54a0 2021-07-11T04:00:43.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cf476700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80adff0 con 0x7f85c81475a0 2021-07-11T04:00:43.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85ce474700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f85c80b54a0 0x7f85c80af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:43.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85ce474700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f85c80b54a0 0x7f85c80af550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48210/0 (socket says 172.21.2.104:48210) 2021-07-11T04:00:43.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85ce474700 1 -- 172.21.2.104:0/2268142583 learned_addr learned my addr 172.21.2.104:0/2268142583 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:43.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85ce474700 1 -- 172.21.2.104:0/2268142583 >> v2:172.21.2.105:3300/0 conn(0x7f85c81475a0 msgr2=0x7f85c80acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:43.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85ce474700 1 --2- 172.21.2.104:0/2268142583 >> v2:172.21.2.105:3300/0 conn(0x7f85c81475a0 0x7f85c80acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:43.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85ce474700 1 -- 172.21.2.104:0/2268142583 >> v1:172.21.2.105:6789/0 conn(0x7f85c80b0610 legacy=0x7f85c80675a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:43.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85ce474700 1 -- 172.21.2.104:0/2268142583 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80b5880 con 0x7f85c80b54a0 2021-07-11T04:00:43.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85ce474700 1 --2- 172.21.2.104:0/2268142583 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c80b54a0 0x7f85c80af550 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f85c8069d90 tx=0x7f85bc006eb0).ready entity=mon.0 client_cookie=b68be469da550d14 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:43.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cd472700 1 -- 172.21.2.104:0/2268142583 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85bc00d5c0 con 0x7f85c80b54a0 2021-07-11T04:00:43.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cd472700 1 -- 172.21.2.104:0/2268142583 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc000e90 con 0x7f85c80b54a0 2021-07-11T04:00:43.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cd472700 1 -- 172.21.2.104:0/2268142583 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85bc00d910 con 0x7f85c80b54a0 2021-07-11T04:00:43.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cf476700 1 -- 172.21.2.104:0/2268142583 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c80b54a0 msgr2=0x7f85c80af550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:43.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.427+0000 7f85cf476700 1 --2- 172.21.2.104:0/2268142583 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c80b54a0 0x7f85c80af550 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f85c8069d90 tx=0x7f85bc006eb0).stop 2021-07-11T04:00:43.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 -- 172.21.2.104:0/2268142583 shutdown_connections 2021-07-11T04:00:43.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 --2- 172.21.2.104:0/2268142583 >> v2:172.21.2.105:3300/0 conn(0x7f85c81475a0 0x7f85c80acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:43.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 --2- 172.21.2.104:0/2268142583 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c80b54a0 0x7f85c80af550 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:43.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 -- 172.21.2.104:0/2268142583 >> 172.21.2.104:0/2268142583 conn(0x7f85c8062560 msgr2=0x7f85c8062940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:43.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 -- 172.21.2.104:0/2268142583 shutdown_connections 2021-07-11T04:00:43.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 -- 172.21.2.104:0/2268142583 wait complete. 2021-07-11T04:00:43.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 Processor -- start 2021-07-11T04:00:43.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 -- start start 2021-07-11T04:00:43.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85c80b0610 0x7f85c8142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:43.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85c80b54a0 0x7f85c815a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:43.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c81475a0 0x7f85c813fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:43.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c815a2e0 con 0x7f85c81475a0 2021-07-11T04:00:43.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8145760 con 0x7f85c80b54a0 2021-07-11T04:00:43.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81454c0 con 0x7f85c80b0610 2021-07-11T04:00:43.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cec75700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c81475a0 0x7f85c813fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:43.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cec75700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c81475a0 0x7f85c813fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48218/0 (socket says 172.21.2.104:48218) 2021-07-11T04:00:43.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cec75700 1 -- 172.21.2.104:0/2010640113 learned_addr learned my addr 172.21.2.104:0/2010640113 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:43.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cec75700 1 -- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85c80b0610 msgr2=0x7f85c8142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:43.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cec75700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85c80b0610 0x7f85c8142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:43.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cec75700 1 -- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85c80b54a0 msgr2=0x7f85c815a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:43.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cec75700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85c80b54a0 0x7f85c815a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:43.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cec75700 1 -- 172.21.2.104:0/2010640113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85bc00b1e0 con 0x7f85c81475a0 2021-07-11T04:00:43.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cec75700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c81475a0 0x7f85c813fd00 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f85c00032f0 tx=0x7f85c00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:43.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85bb7fe700 1 -- 172.21.2.104:0/2010640113 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85c000f760 con 0x7f85c81475a0 2021-07-11T04:00:43.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85bb7fe700 1 -- 172.21.2.104:0/2010640113 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c000f8c0 con 0x7f85c81475a0 2021-07-11T04:00:43.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 -- 172.21.2.104:0/2010640113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c815a540 con 0x7f85c81475a0 2021-07-11T04:00:43.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 -- 172.21.2.104:0/2010640113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c80b2e50 con 0x7f85c81475a0 2021-07-11T04:00:43.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85bb7fe700 1 -- 172.21.2.104:0/2010640113 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f85c000fc20 con 0x7f85c81475a0 2021-07-11T04:00:43.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85cf476700 1 -- 172.21.2.104:0/2010640113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c815a6d0 con 0x7f85c81475a0 2021-07-11T04:00:43.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.431+0000 7f85bb7fe700 1 -- 172.21.2.104:0/2010640113 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f85c001a460 con 0x7f85c81475a0 2021-07-11T04:00:43.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.435+0000 7f85bb7fe700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85a8053260 0x7f85a8055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:43.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.435+0000 7f85bb7fe700 1 -- 172.21.2.104:0/2010640113 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f85c006b930 con 0x7f85c81475a0 2021-07-11T04:00:43.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.435+0000 7f85ce474700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85a8053260 0x7f85a8055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:43.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.435+0000 7f85ce474700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85a8053260 0x7f85a8055720 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f85c8069d90 tx=0x7f85bc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:43.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.435+0000 7f85bb7fe700 1 -- 172.21.2.104:0/2010640113 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f85c0042b00 con 0x7f85c81475a0 2021-07-11T04:00:43.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 -- 172.21.2.104:0/2010640113 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85c8057350 con 0x7f85c81475a0 2021-07-11T04:00:43.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85bb7fe700 1 -- 172.21.2.104:0/2010640113 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2481 (secure 0 0 0) 0x7f85c0046120 con 0x7f85c81475a0 2021-07-11T04:00:43.545 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:43.545 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/873566337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":873566337},{"type":"v1","addr":"172.21.2.105:6835","nonce":873566337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/930744033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":930744033},{"type":"v1","addr":"172.21.2.104:6833","nonce":930744033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:35.070420+0000","modified":"2021-07-11T04:00:36.075051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/18122340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":18122340},{"type":"v1","addr":"172.21.2.104:6835","nonce":18122340}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-07-11T04:00:43.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 -- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85a8053260 msgr2=0x7f85a8055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:43.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85a8053260 0x7f85a8055720 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f85c8069d90 tx=0x7f85bc015000).stop 2021-07-11T04:00:43.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 -- 172.21.2.104:0/2010640113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c81475a0 msgr2=0x7f85c813fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:43.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c81475a0 0x7f85c813fd00 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f85c00032f0 tx=0x7f85c00091d0).stop 2021-07-11T04:00:43.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 -- 172.21.2.104:0/2010640113 shutdown_connections 2021-07-11T04:00:43.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f85a8053260 0x7f85a8055720 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:43.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f85c81475a0 0x7f85c813fd00 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:43.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f85c80b54a0 0x7f85c815a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:43.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 --2- 172.21.2.104:0/2010640113 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85c80b0610 0x7f85c8142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:43.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 -- 172.21.2.104:0/2010640113 >> 172.21.2.104:0/2010640113 conn(0x7f85c8062560 msgr2=0x7f85c8147bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:43.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 -- 172.21.2.104:0/2010640113 shutdown_connections 2021-07-11T04:00:43.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:43.543+0000 7f85cf476700 1 -- 172.21.2.104:0/2010640113 wait complete. 2021-07-11T04:00:43.549 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 45 2021-07-11T04:00:43.552 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json dump inode 0x10000000000 2021-07-11T04:00:43.609 INFO:teuthology.orchestra.run.gibba004.stdout:{"path":"/foo","ino":1099511627776,"rdev":0,"ctime":"2021-07-11T04:00:43.109799+0000","btime":"2021-07-11T04:00:43.109799+0000","mode":16877,"uid":0,"gid":0,"nlink":1,"dir_layout":{"dir_hash":2,"unused1":0,"unused2":0,"unused3":0},"layout":{"stripe_unit":0,"stripe_count":0,"object_size":0,"pool_id":-1,"pool_ns":""},"old_pools":[],"size":0,"truncate_seq":1,"truncate_size":18446744073709551615,"truncate_from":0,"truncate_pending":0,"mtime":"2021-07-11T04:00:43.109799+0000","atime":"2021-07-11T04:00:43.109799+0000","time_warp_seq":0,"change_attr":0,"export_pin":-1,"export_ephemeral_random_pin":0,"export_ephemeral_distributed_pin":false,"client_ranges":[],"dirstat":{"version":0,"mtime":"0.000000","num_files":0,"num_subdirs":0,"change_attr":0},"rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"accounted_rstat":{"version":0,"rbytes":0,"rfiles":0,"rsubdirs":1,"rsnaps":0,"rctime":"0.000000"},"version":2,"file_data_version":0,"xattr_version":1,"backtrace_version":2,"stray_prior_path":"","max_size_ever":0,"quota":{"max_bytes":0,"max_files":0},"last_scrub_stamp":"0.000000","last_scrub_version":0,"symlink":"","xattrs":[],"dirfragtree":{"splits":[]},"old_inodes":[],"oldest_snap":18446744073709551614,"damage_flags":0,"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"request":0,"dirfrag":1,"caps":1,"dirtyparent":1,"dirty":1},"nref":4,"versionlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"authlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"linklock":{},"dirfragtreelock":{},"filelock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"xattrlock":{"gather_set":[],"state":"excl","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"snaplock":{},"nestlock":{"gather_set":[],"state":"lock","is_leased":false,"num_rdlocks":0,"num_wrlocks":0,"num_xlocks":0,"xlock_by":{}},"flocklock":{},"policylock":{},"states":["auth","dirty","dirtyparent","dirtypool"],"client_caps":[{"client_id":4802,"pending":"pAsxLsXsxFsx","issued":"pAsxLsXsxFsx","wanted":"-","last_sent":1}],"loner":4802,"want_loner":4802,"mds_caps_wanted":[]} 2021-07-11T04:00:43.610 DEBUG:tasks.cephfs.filesystem:_json_asok output { "accounted_rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "atime": "2021-07-11T04:00:43.109799+0000", "auth_pins": 0, "auth_state": { "replicas": {} }, "authlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "backtrace_version": 2, "btime": "2021-07-11T04:00:43.109799+0000", "change_attr": 0, "client_caps": [ { "client_id": 4802, "issued": "pAsxLsXsxFsx", "last_sent": 1, "pending": "pAsxLsXsxFsx", "wanted": "-" } ], "client_ranges": [], "ctime": "2021-07-11T04:00:43.109799+0000", "damage_flags": 0, "dir_layout": { "dir_hash": 2, "unused1": 0, "unused2": 0, "unused3": 0 }, "dirfragtree": { "splits": [] }, "dirfragtreelock": {}, "dirstat": { "change_attr": 0, "mtime": "0.000000", "num_files": 0, "num_subdirs": 0, "version": 0 }, "export_ephemeral_distributed_pin": false, "export_ephemeral_random_pin": 0, "export_pin": -1, "file_data_version": 0, "filelock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "flocklock": {}, "gid": 0, "ino": 1099511627776, "is_auth": true, "is_freezing": false, "is_frozen": false, "last_scrub_stamp": "0.000000", "last_scrub_version": 0, "layout": { "object_size": 0, "pool_id": -1, "pool_ns": "", "stripe_count": 0, "stripe_unit": 0 }, "linklock": {}, "loner": 4802, "max_size_ever": 0, "mds_caps_wanted": [], "mode": 16877, "mtime": "2021-07-11T04:00:43.109799+0000", "nestlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "nlink": 1, "nref": 4, "old_inodes": [], "old_pools": [], "oldest_snap": 18446744073709551614, "path": "/foo", "pins": { "caps": 1, "dirfrag": 1, "dirty": 1, "dirtyparent": 1, "request": 0 }, "policylock": {}, "quota": { "max_bytes": 0, "max_files": 0 }, "rdev": 0, "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "rstat": { "rbytes": 0, "rctime": "0.000000", "rfiles": 0, "rsnaps": 0, "rsubdirs": 1, "version": 0 }, "size": 0, "snaplock": {}, "states": [ "auth", "dirty", "dirtyparent", "dirtypool" ], "stray_prior_path": "", "symlink": "", "time_warp_seq": 0, "truncate_from": 0, "truncate_pending": 0, "truncate_seq": 1, "truncate_size": 18446744073709551615, "uid": 0, "version": 2, "versionlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "lock", "xlock_by": {} }, "want_loner": 4802, "xattr_version": 1, "xattrlock": { "gather_set": [], "is_leased": false, "num_rdlocks": 0, "num_wrlocks": 0, "num_xlocks": 0, "state": "excl", "xlock_by": {} }, "xattrs": [] } 2021-07-11T04:00:43.611 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:00:43.611 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:43.619 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:43.633 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:43.647 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:43.661 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:00:43.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:43.663 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:43.663 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:43.692 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:00:43.694 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:00:43.694 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T04:00:43.695 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:00:43.696 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:43.705 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15393]: fuse finished with error 0 and tester_r 0 2021-07-11T04:00:43.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:43.707 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:43.708 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:43.713 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T04:00:43.714 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:00:43.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:00:43.794 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:43.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:43.798 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:43.798 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:43.809 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:00:43.811 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:00:43.811 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba023.front.sepia.ceph.com... 2021-07-11T04:00:43.811 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:00:43.812 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:43.865 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14834]: fuse finished with error 0 and tester_r 0 2021-07-11T04:00:43.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:43.867 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:43.867 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:43.871 INFO:teuthology.orchestra.run.gibba023.stdout:ext2/ext3 2021-07-11T04:00:43.871 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:00:43.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:00:43.982 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:44.028 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:00:44.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 Processor -- start 2021-07-11T04:00:44.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 -- start start 2021-07-11T04:00:44.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f203c0b2580 0x7f203c14ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f203c0a8ac0 0x7f203c14b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f203c0a70c0 con 0x7f203c0b0010 2021-07-11T04:00:44.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c0a72f0 con 0x7f203c0b2580 2021-07-11T04:00:44.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f203c0a7590 con 0x7f203c0a8ac0 2021-07-11T04:00:44.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f203bfff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f203c0b2580 0x7f203c14ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:44.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f203bfff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f203c0b2580 0x7f203c14ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48222/0 (socket says 172.21.2.104:48222) 2021-07-11T04:00:44.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f203bfff700 1 -- 172.21.2.104:0/2237072511 learned_addr learned my addr 172.21.2.104:0/2237072511 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:44.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f203bfff700 1 -- 172.21.2.104:0/2237072511 >> v2:172.21.2.105:3300/0 conn(0x7f203c0a8ac0 msgr2=0x7f203c14b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:44.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f203bfff700 1 --2- 172.21.2.104:0/2237072511 >> v2:172.21.2.105:3300/0 conn(0x7f203c0a8ac0 0x7f203c14b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f203bfff700 1 -- 172.21.2.104:0/2237072511 >> v1:172.21.2.105:6789/0 conn(0x7f203c0b0010 legacy=0x7f203c0a83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:44.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f203bfff700 1 -- 172.21.2.104:0/2237072511 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0a7830 con 0x7f203c0b2580 2021-07-11T04:00:44.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f203bfff700 1 --2- 172.21.2.104:0/2237072511 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f203c0b2580 0x7f203c14ae40 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f203c06b7c0 tx=0x7f2024006eb0).ready entity=mon.0 client_cookie=d9597dcc9f41b3b2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:44.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f203affd700 1 -- 172.21.2.104:0/2237072511 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f202400d5c0 con 0x7f203c0b2580 2021-07-11T04:00:44.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f203affd700 1 -- 172.21.2.104:0/2237072511 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2024000e90 con 0x7f203c0b2580 2021-07-11T04:00:44.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f203affd700 1 -- 172.21.2.104:0/2237072511 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f202400d910 con 0x7f203c0b2580 2021-07-11T04:00:44.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 -- 172.21.2.104:0/2237072511 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f203c0b2580 msgr2=0x7f203c14ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 --2- 172.21.2.104:0/2237072511 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f203c0b2580 0x7f203c14ae40 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f203c06b7c0 tx=0x7f2024006eb0).stop 2021-07-11T04:00:44.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 -- 172.21.2.104:0/2237072511 shutdown_connections 2021-07-11T04:00:44.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 --2- 172.21.2.104:0/2237072511 >> v2:172.21.2.105:3300/0 conn(0x7f203c0a8ac0 0x7f203c14b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 --2- 172.21.2.104:0/2237072511 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f203c0b2580 0x7f203c14ae40 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:44.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 -- 172.21.2.104:0/2237072511 >> 172.21.2.104:0/2237072511 conn(0x7f203c060180 msgr2=0x7f203c0b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:44.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 -- 172.21.2.104:0/2237072511 shutdown_connections 2021-07-11T04:00:44.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.083+0000 7f2041067700 1 -- 172.21.2.104:0/2237072511 wait complete. 2021-07-11T04:00:44.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f2041067700 1 Processor -- start 2021-07-11T04:00:44.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f2041067700 1 -- start start 2021-07-11T04:00:44.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f2041067700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f203c0a8ac0 0x7f203c1598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f2041067700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f203c0b0010 0x7f203c143f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f2041067700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f203c0b2580 0x7f203c156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f2041067700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0abab0 con 0x7f203c0b0010 2021-07-11T04:00:44.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f2041067700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c0abd50 con 0x7f203c0a8ac0 2021-07-11T04:00:44.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f2041067700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f203c156410 con 0x7f203c0b2580 2021-07-11T04:00:44.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f203b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f203c0b0010 0x7f203c143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:44.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f203b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f203c0b0010 0x7f203c143f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48230/0 (socket says 172.21.2.104:48230) 2021-07-11T04:00:44.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f203b7fe700 1 -- 172.21.2.104:0/2536158963 learned_addr learned my addr 172.21.2.104:0/2536158963 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:44.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f203b7fe700 1 -- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f203c0b2580 msgr2=0x7f203c156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f203b7fe700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f203c0b2580 0x7f203c156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f203b7fe700 1 -- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f203c0a8ac0 msgr2=0x7f203c1598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f203b7fe700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f203c0a8ac0 0x7f203c1598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f203b7fe700 1 -- 172.21.2.104:0/2536158963 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202400b1e0 con 0x7f203c0b0010 2021-07-11T04:00:44.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f203b7fe700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f203c0b0010 0x7f203c143f80 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f202c000f70 tx=0x7f202c007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:44.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f20397fa700 1 -- 172.21.2.104:0/2536158963 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f202c013450 con 0x7f203c0b0010 2021-07-11T04:00:44.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f20397fa700 1 -- 172.21.2.104:0/2536158963 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202c0135b0 con 0x7f203c0b0010 2021-07-11T04:00:44.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f2041067700 1 -- 172.21.2.104:0/2536158963 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c1566b0 con 0x7f203c0b0010 2021-07-11T04:00:44.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f2041067700 1 -- 172.21.2.104:0/2536158963 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c0aa330 con 0x7f203c0b0010 2021-07-11T04:00:44.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f20397fa700 1 -- 172.21.2.104:0/2536158963 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f202c013910 con 0x7f203c0b0010 2021-07-11T04:00:44.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f2041067700 1 -- 172.21.2.104:0/2536158963 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f203c1571e0 con 0x7f203c0b0010 2021-07-11T04:00:44.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f20397fa700 1 -- 172.21.2.104:0/2536158963 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f202c01a060 con 0x7f203c0b0010 2021-07-11T04:00:44.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f20397fa700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2020053350 0x7f2020055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f20397fa700 1 -- 172.21.2.104:0/2536158963 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f202c06aef0 con 0x7f203c0b0010 2021-07-11T04:00:44.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.087+0000 7f203bfff700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2020053350 0x7f2020055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:44.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.091+0000 7f203bfff700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2020053350 0x7f2020055810 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f203c06b7c0 tx=0x7f2024015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:44.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.091+0000 7f20397fa700 1 -- 172.21.2.104:0/2536158963 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f202c042120 con 0x7f203c0b0010 2021-07-11T04:00:44.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.171+0000 7f2041067700 1 -- 172.21.2.104:0/2536158963 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f203c156840 con 0x7f203c0b0010 2021-07-11T04:00:44.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.171+0000 7f20397fa700 1 -- 172.21.2.104:0/2536158963 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v98) v1 ==== 81+0+33648 (secure 0 0 0) 0x7f202c018070 con 0x7f203c0b0010 2021-07-11T04:00:44.174 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:44.174 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:00:44.175 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 98, 2021-07-11T04:00:44.175 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:00:44.175 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:00:44.175 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:00:37.089468+0000", 2021-07-11T04:00:44.176 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:44.176 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:44.176 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:00:44.176 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:00:44.176 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:00:44.177 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:00:44.177 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:00:44.177 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:00:44.177 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:00:44.178 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:44.178 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:00:44.178 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:00:44.178 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:00:44.178 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:00:44.179 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:00:44.179 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 18, 2021-07-11T04:00:44.179 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:00:44.179 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:00:44.180 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:00:44.180 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:00:44.180 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:00:44.180 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.180 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:00:44.181 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:00:44.181 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:00:44.181 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:44.181 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:44.182 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:44.182 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:44.182 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:44.182 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:44.182 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:44.183 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:44.183 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:00:44.183 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:00:44.183 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:00:44.184 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:00:44.184 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:00:44.184 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:44.184 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:44.184 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:44.185 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:44.186 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:44.186 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:44.186 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:44.186 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.187 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:00:44.187 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:44.187 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:44.187 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:44.187 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:44.188 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:44.188 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:44.189 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:44.189 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:44.189 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:44.189 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:44.190 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:44.190 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:44.190 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:44.190 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:44.190 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:44.191 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:44.191 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:44.191 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:44.191 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:44.192 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:44.192 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:44.192 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:44.192 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:44.192 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:44.193 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:44.193 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:44.193 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.193 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:44.194 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:44.194 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:44.194 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:44.195 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:44.195 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:44.195 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:44.195 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:44.196 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:44.196 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:44.196 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:44.196 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:00:44.196 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:00:44.197 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.197 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:44.197 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:00:44.197 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.197 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.198 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.198 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:00:44.198 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:00:44.198 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:00:44.199 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:44.199 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:44.200 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:44.200 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:44.200 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:44.200 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:44.200 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:44.201 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:44.201 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:44.201 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:44.201 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:44.202 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:44.203 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:44.203 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:44.203 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:44.203 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:44.203 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:44.204 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:44.204 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:44.204 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:44.204 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.205 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:00:44.205 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:44.205 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:44.205 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:44.206 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:44.206 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:44.206 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:44.206 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:44.206 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:44.207 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:44.207 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:44.207 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:44.207 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:44.208 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:44.208 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:44.208 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:44.208 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:44.209 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:44.209 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:44.209 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:44.209 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:44.210 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:44.210 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:44.210 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:44.210 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:44.210 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:44.211 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:44.211 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.211 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:44.211 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:44.212 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:44.212 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:44.212 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:44.213 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:44.213 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:44.213 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:44.213 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:44.213 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:44.214 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:44.214 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:00:44.214 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:44.214 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:00:44.215 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.215 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.215 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.215 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 17, 2021-07-11T04:00:44.216 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T04:00:44.216 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:00:33.058069+0000", 2021-07-11T04:00:44.216 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:44.216 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:44.217 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:44.217 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:44.217 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:44.217 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:44.217 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:44.218 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:44.218 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:44.218 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:44.218 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:44.219 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:44.219 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:44.219 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:44.219 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:44.219 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:44.220 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:44.220 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:44.220 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:44.220 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:44.221 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.221 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "97", 2021-07-11T04:00:44.221 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:44.221 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:44.221 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:44.222 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:44.222 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:44.222 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:44.222 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:44.223 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:44.223 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:44.223 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:44.223 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:44.223 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:44.224 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:44.224 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:44.224 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:44.224 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:44.225 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:44.225 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:44.225 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:44.225 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:44.226 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:44.226 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:44.226 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:44.226 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:44.226 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:44.227 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:44.227 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.227 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:44.227 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:44.228 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:44.228 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:44.228 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:44.228 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:44.229 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:44.229 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:44.229 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:44.229 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:44.230 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:44.230 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:00:44.230 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T04:00:44.230 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T04:00:44.230 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T04:00:44.231 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.231 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:44.231 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:00:44.231 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T04:00:44.232 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.232 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.232 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.232 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.233 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 18, 2021-07-11T04:00:44.233 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T04:00:44.233 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:00:34.122127+0000", 2021-07-11T04:00:44.233 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:44.234 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:44.234 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:44.234 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:44.234 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:44.235 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:44.235 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:44.235 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:44.235 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:44.235 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:44.236 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:44.236 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:44.236 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:44.236 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:44.237 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:44.237 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:44.237 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:44.237 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:44.237 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:44.238 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:44.238 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.238 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "98", 2021-07-11T04:00:44.238 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:44.239 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:44.239 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:44.239 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:44.239 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:44.240 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:44.240 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:44.240 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:44.240 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:44.241 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:44.241 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:44.241 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:44.241 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:44.242 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:44.242 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:44.242 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:44.242 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:44.243 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:44.243 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:44.243 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:44.243 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:44.243 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:44.244 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:44.244 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:44.244 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:44.244 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:44.245 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.245 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:44.245 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:44.245 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:44.245 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:44.246 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:44.246 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:44.246 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:44.248 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:44.249 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:44.249 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:44.249 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:44.250 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:00:44.250 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:44.250 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:00:44.250 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T04:00:44.250 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.251 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.251 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.251 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:44.251 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:00:44.252 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.252 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:44.252 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:00:44.252 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:44.253 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:44.253 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:44.253 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:44.253 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:44.253 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:44.254 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:44.254 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 95, 2021-07-11T04:00:44.254 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:44.254 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:44.255 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:44.255 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.255 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.255 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.256 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:00:44.256 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:44.256 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.256 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.256 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.257 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:00:44.257 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:44.257 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.257 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.258 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.258 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:44.258 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.258 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.259 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.259 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:00:44.259 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:44.259 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.260 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.260 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.260 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:00:44.260 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:44.260 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.261 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.261 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.261 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:44.261 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.262 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.262 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.262 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:00:44.262 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:44.262 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.263 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.263 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.263 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:00:44.263 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:44.264 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.264 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.264 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.264 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:44.264 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.265 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.265 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.265 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:00:44.265 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:44.266 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.266 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.266 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.266 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:00:44.267 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:44.267 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.267 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.267 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.267 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:00:44.268 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:00:44.268 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:00:44.268 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:00:44.268 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:44.269 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:44.269 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:44.269 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.269 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.270 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.270 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:44.270 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:00:44.270 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:44.271 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:44.271 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:44.271 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:44.271 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:44.271 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:44.272 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:44.272 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 95, 2021-07-11T04:00:44.272 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:44.272 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:44.273 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:44.273 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.273 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.273 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.274 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:00:44.274 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:44.274 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.274 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.275 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.275 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:00:44.275 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:44.275 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.275 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.276 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.276 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:44.276 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.276 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.276 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.277 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:00:44.277 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:44.277 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.277 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.278 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.278 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:00:44.278 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:44.278 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.279 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.279 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.279 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:44.279 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.279 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.280 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.280 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:00:44.280 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:44.280 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.281 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.281 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.281 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:00:44.281 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:44.281 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.282 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.282 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.282 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:44.282 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.283 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.283 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.283 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:00:44.283 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:44.283 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.284 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.284 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.284 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:00:44.284 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:44.285 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.285 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.285 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.285 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:00:44.285 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:00:44.286 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:00:44.286 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:00:44.286 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:44.287 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:44.287 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:44.287 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.287 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.288 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.288 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:44.288 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:00:44.288 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:44.288 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:44.289 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:44.289 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:44.289 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:44.289 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:44.290 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:44.290 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 96, 2021-07-11T04:00:44.290 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:44.291 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:44.291 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:44.291 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.292 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.292 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.292 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:00:44.292 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:44.293 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.293 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.293 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.293 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:00:44.293 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:44.294 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.294 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.294 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.295 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:44.295 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.295 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.295 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.295 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:00:44.296 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:44.296 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.296 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.296 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.297 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:00:44.297 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:44.297 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.297 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.297 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.298 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:44.298 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.298 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.298 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.299 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:00:44.299 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:44.299 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.299 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.299 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.300 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:00:44.300 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:44.300 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.300 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.301 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.301 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:44.301 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.301 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.302 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.302 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:00:44.302 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:44.302 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.303 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.303 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.303 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:00:44.303 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:44.303 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.304 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.304 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.304 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:00:44.304 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:00:44.304 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:00:44.305 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:00:44.305 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:44.305 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:44.305 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:44.306 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.306 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.306 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.306 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:44.306 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:00:44.307 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:44.307 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:44.307 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:44.307 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:44.308 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:44.308 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:44.308 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:44.309 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 96, 2021-07-11T04:00:44.309 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:44.309 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:44.309 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:44.310 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.310 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.310 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.310 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:00:44.310 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:44.311 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.311 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.311 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.311 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:00:44.312 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:44.312 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.312 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.312 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.312 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:44.313 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.313 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.313 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.313 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:00:44.314 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:44.314 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.314 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.314 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.314 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:00:44.315 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:44.315 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.315 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.315 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.316 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:44.316 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.316 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.316 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.316 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:00:44.317 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:44.317 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.317 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.317 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.318 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:00:44.318 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:44.318 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.318 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.319 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.319 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:44.319 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.319 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.319 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.320 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:00:44.320 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:44.320 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.320 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.321 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.321 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:00:44.321 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:44.321 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.321 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.322 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.322 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:00:44.322 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:00:44.322 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:00:44.323 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:00:44.323 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:44.323 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:44.323 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:44.323 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.324 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.324 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.324 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:44.324 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:00:44.325 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:44.325 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:44.325 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:44.325 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:44.326 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:44.326 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:44.326 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:44.326 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 94, 2021-07-11T04:00:44.327 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:44.327 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:44.327 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:44.327 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.328 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.328 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.328 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:00:44.328 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:44.328 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.329 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.329 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.329 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:00:44.329 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:44.330 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.330 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.330 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.330 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:44.330 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.331 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.331 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.331 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:00:44.332 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:44.332 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.332 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.332 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.332 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:00:44.333 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:44.333 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.333 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.333 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.334 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:44.334 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.334 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.334 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.335 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:00:44.335 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:44.335 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.335 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.335 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.336 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:00:44.336 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:44.336 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.336 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.336 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.337 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:44.337 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.337 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.337 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.338 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:00:44.338 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:44.338 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.338 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.338 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.339 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:00:44.339 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:44.339 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.339 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.340 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.340 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:00:44.340 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:00:44.340 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:00:44.340 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:00:44.341 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:44.341 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:44.341 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:44.341 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.342 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.342 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.342 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:44.342 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:00:44.342 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:44.343 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:44.343 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:44.343 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:44.343 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:44.344 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:44.344 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:44.344 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:44.344 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:44.344 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:44.345 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:44.345 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.345 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.345 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.346 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:00:44.346 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:44.346 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.346 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.347 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.347 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:00:44.347 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:44.347 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.348 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.348 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.348 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:44.348 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.349 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.349 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.349 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:00:44.349 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:44.349 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.350 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.350 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.350 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:00:44.350 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:44.351 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.351 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.351 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.351 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:44.351 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.352 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.352 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.352 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:00:44.352 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:44.353 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.353 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.353 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.353 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:00:44.354 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:44.354 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.354 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.354 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.355 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:44.355 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.355 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.355 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.355 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:00:44.356 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:44.356 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.356 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.356 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.357 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:00:44.357 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:44.357 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.357 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.358 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.358 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:00:44.358 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:00:44.358 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:00:44.358 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:00:44.359 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:44.359 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:44.359 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:44.359 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.360 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.360 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.360 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:44.360 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:00:44.360 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:44.361 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:44.361 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:44.361 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:44.361 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:44.362 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:44.362 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:44.362 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 94, 2021-07-11T04:00:44.362 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:44.362 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:44.363 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:44.363 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.363 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.363 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.363 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:00:44.364 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:44.364 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.364 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.364 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.365 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:00:44.365 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:44.365 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.365 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.366 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.366 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:44.366 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.366 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.366 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.367 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:00:44.367 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:44.367 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.367 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.368 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.368 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:00:44.368 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:44.368 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.369 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.369 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.369 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:44.369 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.369 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.370 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.370 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:00:44.370 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:44.370 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.371 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.371 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.371 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:00:44.371 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:44.371 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.372 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.372 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.372 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:44.372 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.373 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.373 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.373 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:00:44.373 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:44.374 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.374 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.374 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.374 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:00:44.375 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:44.375 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.375 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.375 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.376 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:00:44.376 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:00:44.376 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:00:44.376 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:00:44.376 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:44.377 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:44.377 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:44.377 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.377 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.378 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.378 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:44.379 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:00:44.380 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:44.380 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:44.380 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:44.380 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:44.381 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:44.381 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:44.381 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:44.381 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 95, 2021-07-11T04:00:44.382 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:44.382 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:44.382 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:44.382 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.383 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.383 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.383 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:00:44.383 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:44.384 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.384 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.384 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.384 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:00:44.384 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:44.385 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.385 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.385 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.385 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:44.386 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.386 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.386 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.386 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:00:44.386 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:44.387 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.387 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.387 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.387 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:00:44.388 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:44.388 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.388 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.388 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.389 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:44.389 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.389 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.389 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.389 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:00:44.390 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:44.390 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.390 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.390 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.391 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:00:44.391 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:44.391 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.391 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.392 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.392 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:44.392 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:44.392 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.392 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:44.393 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:00:44.393 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:44.393 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.393 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.394 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:44.394 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:00:44.394 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:44.394 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.394 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.395 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.395 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:00:44.395 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:00:44.395 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:00:44.396 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:00:44.396 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:44.396 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:44.396 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:44.396 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:44.397 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.397 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:44.397 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:00:44.397 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.398 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:44.398 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:44.398 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:44.398 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:44.399 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:44.399 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:44.399 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:00:44.399 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:44.399 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.400 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.400 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:44.400 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:44.400 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:44.401 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:44.401 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:44.401 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:44.401 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:00:44.402 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:44.402 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.402 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.402 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:44.402 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:44.403 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:44.403 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:44.403 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:44.403 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:44.404 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:00:44.404 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:44.404 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.404 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.404 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:44.405 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:44.405 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:44.405 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:44.405 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:44.405 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:44.406 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:00:44.406 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:44.406 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.406 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.407 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:44.407 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:44.407 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:44.407 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:44.407 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:44.408 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:44.408 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:00:44.408 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:44.408 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.409 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.409 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:44.409 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:44.409 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:44.410 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:44.410 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:44.410 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:44.410 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:00:44.410 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:44.411 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.411 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.411 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:44.411 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:44.412 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:44.412 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:44.412 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:44.412 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:44.412 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:00:44.413 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:44.413 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.413 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:44.413 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:44.414 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:44.414 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:44.414 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:44.414 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:44.414 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:44.415 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:00:44.415 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:44.415 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.415 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:44.416 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:00:44.416 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:00:44.416 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:00:44.416 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:00:44.416 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": {}, 2021-07-11T04:00:44.417 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:00:44.417 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:00:44.417 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:00:44.417 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:00:44.418 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:00:44.418 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:00:44.418 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:00:44.418 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:00:44.418 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:44.419 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:44.419 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:00:44.419 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:00:44.419 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:00:44.420 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:00:44.420 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:00:44.420 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:00:44.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.171+0000 7f2041067700 1 -- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2020053350 msgr2=0x7f2020055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.171+0000 7f2041067700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2020053350 0x7f2020055810 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f203c06b7c0 tx=0x7f2024015000).stop 2021-07-11T04:00:44.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.171+0000 7f2041067700 1 -- 172.21.2.104:0/2536158963 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f203c0b0010 msgr2=0x7f203c143f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.171+0000 7f2041067700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f203c0b0010 0x7f203c143f80 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f202c000f70 tx=0x7f202c007bd0).stop 2021-07-11T04:00:44.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.175+0000 7f2041067700 1 -- 172.21.2.104:0/2536158963 shutdown_connections 2021-07-11T04:00:44.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.175+0000 7f2041067700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f203c0b2580 0x7f203c156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.175+0000 7f2041067700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2020053350 0x7f2020055810 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:44.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.175+0000 7f2041067700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f203c0b0010 0x7f203c143f80 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:44.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.175+0000 7f2041067700 1 --2- 172.21.2.104:0/2536158963 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f203c0a8ac0 0x7f203c1598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.175+0000 7f2041067700 1 -- 172.21.2.104:0/2536158963 >> 172.21.2.104:0/2536158963 conn(0x7f203c060180 msgr2=0x7f203c145ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:44.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.175+0000 7f2041067700 1 -- 172.21.2.104:0/2536158963 shutdown_connections 2021-07-11T04:00:44.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.175+0000 7f2041067700 1 -- 172.21.2.104:0/2536158963 wait complete. 2021-07-11T04:00:44.425 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:44.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa8105cc700 1 Processor -- start 2021-07-11T04:00:44.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa8105cc700 1 -- start start 2021-07-11T04:00:44.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa8105cc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa8080a76b0 0x7fa808066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa8105cc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa8080b0060 0x7fa80805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa8105cc700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8080a7a90 con 0x7fa8080b1970 2021-07-11T04:00:44.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa8105cc700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa808147620 con 0x7fa8080a76b0 2021-07-11T04:00:44.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa8105cc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8081478c0 con 0x7fa8080b0060 2021-07-11T04:00:44.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa80e368700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa8080a76b0 0x7fa808066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:44.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa80e368700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa8080a76b0 0x7fa808066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48236/0 (socket says 172.21.2.104:48236) 2021-07-11T04:00:44.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa80e368700 1 -- 172.21.2.104:0/3653819725 learned_addr learned my addr 172.21.2.104:0/3653819725 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:44.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa80e368700 1 -- 172.21.2.104:0/3653819725 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa8080b0060 msgr2=0x7fa80805fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:44.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa80e368700 1 --2- 172.21.2.104:0/3653819725 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa8080b0060 0x7fa80805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa80e368700 1 -- 172.21.2.104:0/3653819725 >> v1:172.21.2.105:6789/0 conn(0x7fa8080b1970 legacy=0x7fa80805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:44.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa80e368700 1 -- 172.21.2.104:0/3653819725 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa808147b60 con 0x7fa8080a76b0 2021-07-11T04:00:44.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa80e368700 1 --2- 172.21.2.104:0/3653819725 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa8080a76b0 0x7fa808066a70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fa80806a0e0 tx=0x7fa7fc006eb0).ready entity=mon.0 client_cookie=9d9b4a3105dcf25f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:44.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa80d366700 1 -- 172.21.2.104:0/3653819725 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7fc00d570 con 0x7fa8080a76b0 2021-07-11T04:00:44.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa80d366700 1 -- 172.21.2.104:0/3653819725 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7fc000e90 con 0x7fa8080a76b0 2021-07-11T04:00:44.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa80d366700 1 -- 172.21.2.104:0/3653819725 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7fc00d8c0 con 0x7fa8080a76b0 2021-07-11T04:00:44.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa8105cc700 1 -- 172.21.2.104:0/3653819725 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa8080a76b0 msgr2=0x7fa808066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.491+0000 7fa8105cc700 1 --2- 172.21.2.104:0/3653819725 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa8080a76b0 0x7fa808066a70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fa80806a0e0 tx=0x7fa7fc006eb0).stop 2021-07-11T04:00:44.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 -- 172.21.2.104:0/3653819725 shutdown_connections 2021-07-11T04:00:44.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 --2- 172.21.2.104:0/3653819725 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa8080b0060 0x7fa80805fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 --2- 172.21.2.104:0/3653819725 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa8080a76b0 0x7fa808066a70 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:44.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 -- 172.21.2.104:0/3653819725 >> 172.21.2.104:0/3653819725 conn(0x7fa8080b53e0 msgr2=0x7fa8080b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:44.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 -- 172.21.2.104:0/3653819725 shutdown_connections 2021-07-11T04:00:44.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 -- 172.21.2.104:0/3653819725 wait complete. 2021-07-11T04:00:44.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 Processor -- start 2021-07-11T04:00:44.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 -- start start 2021-07-11T04:00:44.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa8080a76b0 0x7fa8080aa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa8080b0060 0x7fa8080ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa8080b1970 0x7fa8080ab280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8080a9f70 con 0x7fa8080b1970 2021-07-11T04:00:44.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8080af6a0 con 0x7fa8080a76b0 2021-07-11T04:00:44.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8080ac6a0 con 0x7fa8080b0060 2021-07-11T04:00:44.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa80eb69700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa8080b1970 0x7fa8080ab280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:44.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa80eb69700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa8080b1970 0x7fa8080ab280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48244/0 (socket says 172.21.2.104:48244) 2021-07-11T04:00:44.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa80eb69700 1 -- 172.21.2.104:0/4236380068 learned_addr learned my addr 172.21.2.104:0/4236380068 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:44.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa80eb69700 1 -- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa8080b0060 msgr2=0x7fa8080ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa80eb69700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa8080b0060 0x7fa8080ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa80eb69700 1 -- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa8080a76b0 msgr2=0x7fa8080aa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa80eb69700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa8080a76b0 0x7fa8080aa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa80eb69700 1 -- 172.21.2.104:0/4236380068 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc00b1e0 con 0x7fa8080b1970 2021-07-11T04:00:44.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa80eb69700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa8080b1970 0x7fa8080ab280 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7fa8000032f0 tx=0x7fa800009250).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:44.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa7fb7fe700 1 -- 172.21.2.104:0/4236380068 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa80000f7e0 con 0x7fa8080b1970 2021-07-11T04:00:44.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa7fb7fe700 1 -- 172.21.2.104:0/4236380068 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80000f940 con 0x7fa8080b1970 2021-07-11T04:00:44.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 -- 172.21.2.104:0/4236380068 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8080ac940 con 0x7fa8080b1970 2021-07-11T04:00:44.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa7fb7fe700 1 -- 172.21.2.104:0/4236380068 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa80000fca0 con 0x7fa8080b1970 2021-07-11T04:00:44.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 -- 172.21.2.104:0/4236380068 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8080aaca0 con 0x7fa8080b1970 2021-07-11T04:00:44.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa8105cc700 1 -- 172.21.2.104:0/4236380068 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8080ae980 con 0x7fa8080b1970 2021-07-11T04:00:44.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.495+0000 7fa7fb7fe700 1 -- 172.21.2.104:0/4236380068 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa800010480 con 0x7fa8080b1970 2021-07-11T04:00:44.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.499+0000 7fa7fb7fe700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7e8053300 0x7fa7e80557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.499+0000 7fa7fb7fe700 1 -- 172.21.2.104:0/4236380068 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa80006bb00 con 0x7fa8080b1970 2021-07-11T04:00:44.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.499+0000 7fa80e368700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7e8053300 0x7fa7e80557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:44.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.499+0000 7fa80e368700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7e8053300 0x7fa7e80557c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fa80806a0e0 tx=0x7fa7fc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:44.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.499+0000 7fa7fb7fe700 1 -- 172.21.2.104:0/4236380068 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa800042de0 con 0x7fa8080b1970 2021-07-11T04:00:44.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa8105cc700 1 -- 172.21.2.104:0/4236380068 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8080aa8d0 con 0x7fa8080b1970 2021-07-11T04:00:44.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa7fb7fe700 1 -- 172.21.2.104:0/4236380068 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+2481 (secure 0 0 0) 0x7fa800046400 con 0x7fa8080b1970 2021-07-11T04:00:44.610 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:44.610 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":45,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/873566337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":873566337},{"type":"v1","addr":"172.21.2.105:6835","nonce":873566337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/930744033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":930744033},{"type":"v1","addr":"172.21.2.104:6833","nonce":930744033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":45,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:35.070420+0000","modified":"2021-07-11T04:00:36.075051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6972},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6972":{"gid":6972,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/18122340","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":18122340},{"type":"v1","addr":"172.21.2.104:6835","nonce":18122340}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-07-11T04:00:44.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa8105cc700 1 -- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7e8053300 msgr2=0x7fa7e80557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa8105cc700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7e8053300 0x7fa7e80557c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fa80806a0e0 tx=0x7fa7fc015000).stop 2021-07-11T04:00:44.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa8105cc700 1 -- 172.21.2.104:0/4236380068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa8080b1970 msgr2=0x7fa8080ab280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa8105cc700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa8080b1970 0x7fa8080ab280 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7fa8000032f0 tx=0x7fa800009250).stop 2021-07-11T04:00:44.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa8105cc700 1 -- 172.21.2.104:0/4236380068 shutdown_connections 2021-07-11T04:00:44.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa8105cc700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa7e8053300 0x7fa7e80557c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:44.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa8105cc700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa8080b1970 0x7fa8080ab280 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:44.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa8105cc700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa8080b0060 0x7fa8080ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa8105cc700 1 --2- 172.21.2.104:0/4236380068 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa8080a76b0 0x7fa8080aa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.607+0000 7fa8105cc700 1 -- 172.21.2.104:0/4236380068 >> 172.21.2.104:0/4236380068 conn(0x7fa8080b53e0 msgr2=0x7fa80805f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:44.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.611+0000 7fa8105cc700 1 -- 172.21.2.104:0/4236380068 shutdown_connections 2021-07-11T04:00:44.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.611+0000 7fa8105cc700 1 -- 172.21.2.104:0/4236380068 wait complete. 2021-07-11T04:00:44.614 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 45 2021-07-11T04:00:44.617 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T04:00:44.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 Processor -- start 2021-07-11T04:00:44.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- start start 2021-07-11T04:00:44.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9070067180 0x7f907005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90700b1970 0x7f907005fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90700a7a90 con 0x7f90700a76b0 2021-07-11T04:00:44.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9070147620 con 0x7f9070067180 2021-07-11T04:00:44.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90701478c0 con 0x7f90700b1970 2021-07-11T04:00:44.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f906dd9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9070067180 0x7f907005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:44.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f906dd9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9070067180 0x7f907005f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48252/0 (socket says 172.21.2.104:48252) 2021-07-11T04:00:44.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f906dd9b700 1 -- 172.21.2.104:0/2503564041 learned_addr learned my addr 172.21.2.104:0/2503564041 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:44.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f906dd9b700 1 -- 172.21.2.104:0/2503564041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90700b1970 msgr2=0x7f907005fba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:44.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f906dd9b700 1 --2- 172.21.2.104:0/2503564041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90700b1970 0x7f907005fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f906dd9b700 1 -- 172.21.2.104:0/2503564041 >> v1:172.21.2.105:6789/0 conn(0x7f90700a76b0 legacy=0x7f9070066a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:44.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f906dd9b700 1 -- 172.21.2.104:0/2503564041 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9070147b60 con 0x7f9070067180 2021-07-11T04:00:44.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f906dd9b700 1 --2- 172.21.2.104:0/2503564041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9070067180 0x7f907005f6b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f907006a0e0 tx=0x7f9064006eb0).ready entity=mon.0 client_cookie=48cb43050e0ec31 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:44.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f906d59a700 1 -- 172.21.2.104:0/2503564041 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f906400d570 con 0x7f9070067180 2021-07-11T04:00:44.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f906d59a700 1 -- 172.21.2.104:0/2503564041 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9064000e90 con 0x7f9070067180 2021-07-11T04:00:44.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f906d59a700 1 -- 172.21.2.104:0/2503564041 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f906400d8c0 con 0x7f9070067180 2021-07-11T04:00:44.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- 172.21.2.104:0/2503564041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9070067180 msgr2=0x7f907005f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 --2- 172.21.2.104:0/2503564041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9070067180 0x7f907005f6b0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f907006a0e0 tx=0x7f9064006eb0).stop 2021-07-11T04:00:44.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- 172.21.2.104:0/2503564041 shutdown_connections 2021-07-11T04:00:44.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 --2- 172.21.2.104:0/2503564041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90700b1970 0x7f907005fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 --2- 172.21.2.104:0/2503564041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9070067180 0x7f907005f6b0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:44.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- 172.21.2.104:0/2503564041 >> 172.21.2.104:0/2503564041 conn(0x7f90700b53e0 msgr2=0x7f90700b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:44.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- 172.21.2.104:0/2503564041 shutdown_connections 2021-07-11T04:00:44.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- 172.21.2.104:0/2503564041 wait complete. 2021-07-11T04:00:44.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 Processor -- start 2021-07-11T04:00:44.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- start start 2021-07-11T04:00:44.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9070067180 0x7f90700a9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90700a76b0 0x7f90700ae020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90700b1970 0x7f90700ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90700aa240 con 0x7f90700b1970 2021-07-11T04:00:44.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90701574f0 con 0x7f90700a76b0 2021-07-11T04:00:44.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.671+0000 7f9074877700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90700a9b20 con 0x7f9070067180 2021-07-11T04:00:44.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f906ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90700b1970 0x7f90700ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:44.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f906ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90700b1970 0x7f90700ad4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48256/0 (socket says 172.21.2.104:48256) 2021-07-11T04:00:44.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f906ed9d700 1 -- 172.21.2.104:0/3321012859 learned_addr learned my addr 172.21.2.104:0/3321012859 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:44.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f906ed9d700 1 -- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9070067180 msgr2=0x7f90700a9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f906ed9d700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9070067180 0x7f90700a9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f906ed9d700 1 -- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90700a76b0 msgr2=0x7f90700ae020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:44.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f906ed9d700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90700a76b0 0x7f90700ae020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:44.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f906ed9d700 1 -- 172.21.2.104:0/3321012859 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906400b1e0 con 0x7f90700b1970 2021-07-11T04:00:44.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f906ed9d700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90700b1970 0x7f90700ad4c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f90700ad270 tx=0x7f90680090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:44.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f905f7fe700 1 -- 172.21.2.104:0/3321012859 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90680014e0 con 0x7f90700b1970 2021-07-11T04:00:44.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f905f7fe700 1 -- 172.21.2.104:0/3321012859 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9068001640 con 0x7f90700b1970 2021-07-11T04:00:44.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f905f7fe700 1 -- 172.21.2.104:0/3321012859 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90680019a0 con 0x7f90700b1970 2021-07-11T04:00:44.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f9074877700 1 -- 172.21.2.104:0/3321012859 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90700a9fa0 con 0x7f90700b1970 2021-07-11T04:00:44.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f9074877700 1 -- 172.21.2.104:0/3321012859 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90700ae510 con 0x7f90700b1970 2021-07-11T04:00:44.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f9074877700 1 -- 172.21.2.104:0/3321012859 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9070157320 con 0x7f90700b1970 2021-07-11T04:00:44.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f905f7fe700 1 -- 172.21.2.104:0/3321012859 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9068018030 con 0x7f90700b1970 2021-07-11T04:00:44.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f905f7fe700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f904c053270 0x7f904c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:44.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f905f7fe700 1 -- 172.21.2.104:0/3321012859 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f906806ade0 con 0x7f90700b1970 2021-07-11T04:00:44.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f906e59c700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f904c053270 0x7f904c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:44.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.675+0000 7f906e59c700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f904c053270 0x7f904c055730 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f9060002fb0 tx=0x7f90600069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:44.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.679+0000 7f905f7fe700 1 -- 172.21.2.104:0/3321012859 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f90680421d0 con 0x7f90700b1970 2021-07-11T04:00:44.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:44.783+0000 7f9074877700 1 -- 172.21.2.104:0/3321012859 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9070057350 con 0x7f90700b1970 2021-07-11T04:00:45.356 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T04:00:45.355+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T04:00:45.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.367+0000 7f905f7fe700 1 -- 172.21.2.104:0/3321012859 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7f90680457f0 con 0x7f90700b1970 2021-07-11T04:00:45.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.367+0000 7f9074877700 1 -- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f904c053270 msgr2=0x7f904c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:45.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.367+0000 7f9074877700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f904c053270 0x7f904c055730 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f9060002fb0 tx=0x7f90600069d0).stop 2021-07-11T04:00:45.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.367+0000 7f9074877700 1 -- 172.21.2.104:0/3321012859 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90700b1970 msgr2=0x7f90700ad4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:45.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.367+0000 7f9074877700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90700b1970 0x7f90700ad4c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f90700ad270 tx=0x7f90680090f0).stop 2021-07-11T04:00:45.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.371+0000 7f9074877700 1 -- 172.21.2.104:0/3321012859 shutdown_connections 2021-07-11T04:00:45.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.371+0000 7f9074877700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f904c053270 0x7f904c055730 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:45.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.371+0000 7f9074877700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90700b1970 0x7f90700ad4c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:45.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.371+0000 7f9074877700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90700a76b0 0x7f90700ae020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.371+0000 7f9074877700 1 --2- 172.21.2.104:0/3321012859 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9070067180 0x7f90700a9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.371+0000 7f9074877700 1 -- 172.21.2.104:0/3321012859 >> 172.21.2.104:0/3321012859 conn(0x7f90700b53e0 msgr2=0x7f9070149dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:45.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.371+0000 7f9074877700 1 -- 172.21.2.104:0/3321012859 shutdown_connections 2021-07-11T04:00:45.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.371+0000 7f9074877700 1 -- 172.21.2.104:0/3321012859 wait complete. 2021-07-11T04:00:45.374 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T04:00:45.377 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:45.382 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T04:00:45.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75fbf75700 1 Processor -- start 2021-07-11T04:00:45.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75fbf75700 1 -- start start 2021-07-11T04:00:45.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75fbf75700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f75f40b7810 0x7f75f41445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75fbf75700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f75f40b1170 0x7f75f4144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75fbf75700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f40b31d0 con 0x7f75f4064870 2021-07-11T04:00:45.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75fbf75700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f40b7bf0 con 0x7f75f40b1170 2021-07-11T04:00:45.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75fbf75700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f40675c0 con 0x7f75f40b7810 2021-07-11T04:00:45.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75f9510700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f75f40b1170 0x7f75f4144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:45.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75f9510700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f75f40b1170 0x7f75f4144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48282/0 (socket says 172.21.2.104:48282) 2021-07-11T04:00:45.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75f9510700 1 -- 172.21.2.104:0/1576429185 learned_addr learned my addr 172.21.2.104:0/1576429185 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:45.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75f9510700 1 -- 172.21.2.104:0/1576429185 >> v2:172.21.2.105:3300/0 conn(0x7f75f40b7810 msgr2=0x7f75f41445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:45.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75f9510700 1 --2- 172.21.2.104:0/1576429185 >> v2:172.21.2.105:3300/0 conn(0x7f75f40b7810 0x7f75f41445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75f9510700 1 -- 172.21.2.104:0/1576429185 >> v1:172.21.2.105:6789/0 conn(0x7f75f4064870 legacy=0x7f75f4064d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:45.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75f9510700 1 -- 172.21.2.104:0/1576429185 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f40aae70 con 0x7f75f40b1170 2021-07-11T04:00:45.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.427+0000 7f75f9510700 1 --2- 172.21.2.104:0/1576429185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75f40b1170 0x7f75f4144a90 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f75f4067ca0 tx=0x7f75e8006eb0).ready entity=mon.0 client_cookie=ff4a1ec0392d3d25 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:45.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75f8d0f700 1 -- 172.21.2.104:0/1576429185 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75e800d5c0 con 0x7f75f40b1170 2021-07-11T04:00:45.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75f8d0f700 1 -- 172.21.2.104:0/1576429185 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e8000e90 con 0x7f75f40b1170 2021-07-11T04:00:45.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75f8d0f700 1 -- 172.21.2.104:0/1576429185 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75e800d910 con 0x7f75f40b1170 2021-07-11T04:00:45.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- 172.21.2.104:0/1576429185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75f40b1170 msgr2=0x7f75f4144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:45.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 --2- 172.21.2.104:0/1576429185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75f40b1170 0x7f75f4144a90 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f75f4067ca0 tx=0x7f75e8006eb0).stop 2021-07-11T04:00:45.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- 172.21.2.104:0/1576429185 shutdown_connections 2021-07-11T04:00:45.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 --2- 172.21.2.104:0/1576429185 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75f40b1170 0x7f75f4144a90 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:45.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 --2- 172.21.2.104:0/1576429185 >> v2:172.21.2.105:3300/0 conn(0x7f75f40b7810 0x7f75f41445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- 172.21.2.104:0/1576429185 >> 172.21.2.104:0/1576429185 conn(0x7f75f405f6b0 msgr2=0x7f75f405fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:45.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- 172.21.2.104:0/1576429185 shutdown_connections 2021-07-11T04:00:45.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- 172.21.2.104:0/1576429185 wait complete. 2021-07-11T04:00:45.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 Processor -- start 2021-07-11T04:00:45.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- start start 2021-07-11T04:00:45.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75f4064870 0x7f75f40b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75f40b1170 0x7f75f415a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75f40b7810 0x7f75f415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40af500 con 0x7f75f40b7810 2021-07-11T04:00:45.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40b05a0 con 0x7f75f4064870 2021-07-11T04:00:45.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f40aa930 con 0x7f75f40b1170 2021-07-11T04:00:45.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fa512700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75f40b7810 0x7f75f415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:45.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fa512700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75f40b7810 0x7f75f415a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48288/0 (socket says 172.21.2.104:48288) 2021-07-11T04:00:45.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fa512700 1 -- 172.21.2.104:0/2110466779 learned_addr learned my addr 172.21.2.104:0/2110466779 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:45.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fa512700 1 -- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75f40b1170 msgr2=0x7f75f415a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:45.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fa512700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75f40b1170 0x7f75f415a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fa512700 1 -- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75f4064870 msgr2=0x7f75f40b5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:45.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fa512700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75f4064870 0x7f75f40b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fa512700 1 -- 172.21.2.104:0/2110466779 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e800b1e0 con 0x7f75f40b7810 2021-07-11T04:00:45.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fa512700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75f40b7810 0x7f75f415a7d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f75ec003310 tx=0x7f75ec0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:45.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75e6ffd700 1 -- 172.21.2.104:0/2110466779 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75ec001500 con 0x7f75f40b7810 2021-07-11T04:00:45.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75e6ffd700 1 -- 172.21.2.104:0/2110466779 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec001660 con 0x7f75f40b7810 2021-07-11T04:00:45.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- 172.21.2.104:0/2110466779 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f40b0060 con 0x7f75f40b7810 2021-07-11T04:00:45.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75e6ffd700 1 -- 172.21.2.104:0/2110466779 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75ec0019c0 con 0x7f75f40b7810 2021-07-11T04:00:45.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- 172.21.2.104:0/2110466779 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f4061cc0 con 0x7f75f40b7810 2021-07-11T04:00:45.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.431+0000 7f75fbf75700 1 -- 172.21.2.104:0/2110466779 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f4057350 con 0x7f75f40b7810 2021-07-11T04:00:45.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.435+0000 7f75e6ffd700 1 -- 172.21.2.104:0/2110466779 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f75ec018030 con 0x7f75f40b7810 2021-07-11T04:00:45.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.435+0000 7f75e6ffd700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75d40532d0 0x7f75d4055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.435+0000 7f75e6ffd700 1 -- 172.21.2.104:0/2110466779 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f75ec06b090 con 0x7f75f40b7810 2021-07-11T04:00:45.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.435+0000 7f75f9d11700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75d40532d0 0x7f75d4055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:45.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.435+0000 7f75f9d11700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75d40532d0 0x7f75d4055790 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f75f0002f70 tx=0x7f75f0006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:45.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.435+0000 7f75e6ffd700 1 -- 172.21.2.104:0/2110466779 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f75ec042320 con 0x7f75f40b7810 2021-07-11T04:00:45.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 -- 172.21.2.104:0/2110466779 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75f40b63f0 con 0x7f75f40b7810 2021-07-11T04:00:45.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75e6ffd700 1 -- 172.21.2.104:0/2110466779 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2126 (secure 0 0 0) 0x7f75ec045940 con 0x7f75f40b7810 2021-07-11T04:00:45.724 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:45.724 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":46,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/873566337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":873566337},{"type":"v1","addr":"172.21.2.105:6835","nonce":873566337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/930744033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":930744033},{"type":"v1","addr":"172.21.2.104:6833","nonce":930744033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:35.070420+0000","modified":"2021-07-11T04:00:45.358455+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2021-07-11T04:00:45.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 -- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75d40532d0 msgr2=0x7f75d4055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:45.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75d40532d0 0x7f75d4055790 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f75f0002f70 tx=0x7f75f0006a10).stop 2021-07-11T04:00:45.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 -- 172.21.2.104:0/2110466779 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75f40b7810 msgr2=0x7f75f415a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:45.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75f40b7810 0x7f75f415a7d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7f75ec003310 tx=0x7f75ec0091a0).stop 2021-07-11T04:00:45.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 -- 172.21.2.104:0/2110466779 shutdown_connections 2021-07-11T04:00:45.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75d40532d0 0x7f75d4055790 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:45.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75f40b7810 0x7f75f415a7d0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:45.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75f40b1170 0x7f75f415a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 --2- 172.21.2.104:0/2110466779 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75f4064870 0x7f75f40b5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 -- 172.21.2.104:0/2110466779 >> 172.21.2.104:0/2110466779 conn(0x7f75f405f6b0 msgr2=0x7f75f405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:45.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 -- 172.21.2.104:0/2110466779 shutdown_connections 2021-07-11T04:00:45.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.547+0000 7f75fbf75700 1 -- 172.21.2.104:0/2110466779 wait complete. 2021-07-11T04:00:45.731 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 46 2021-07-11T04:00:45.732 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T04:00:45.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9efac11700 1 Processor -- start 2021-07-11T04:00:45.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9efac11700 1 -- start start 2021-07-11T04:00:45.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9efac11700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9ef414ab80 0x7f9ef405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9efac11700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9ef40ac9b0 0x7f9ef414af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9efac11700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ef4061360 con 0x7f9ef40aff10 2021-07-11T04:00:45.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9efac11700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef414b5e0 con 0x7f9ef414ab80 2021-07-11T04:00:45.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9efac11700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ef414b880 con 0x7f9ef40ac9b0 2021-07-11T04:00:45.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef3fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f9ef40aff10 0x7f9ef40af1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45064/0 (socket says 172.21.2.104:45064) 2021-07-11T04:00:45.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef3fff700 1 -- 172.21.2.104:0/1977589600 learned_addr learned my addr 172.21.2.104:0/1977589600 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:45.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef89ad700 1 --2- 172.21.2.104:0/1977589600 >> v2:172.21.2.104:3300/0 conn(0x7f9ef414ab80 0x7f9ef405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:45.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef37fe700 1 -- 172.21.2.104:0/1977589600 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3631523327 0 0) 0x7f9ef4061360 con 0x7f9ef40aff10 2021-07-11T04:00:45.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef37fe700 1 -- 172.21.2.104:0/1977589600 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed4003260 con 0x7f9ef40aff10 2021-07-11T04:00:45.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef89ad700 1 -- 172.21.2.104:0/1977589600 >> v2:172.21.2.105:3300/0 conn(0x7f9ef40ac9b0 msgr2=0x7f9ef414af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:45.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef89ad700 1 --2- 172.21.2.104:0/1977589600 >> v2:172.21.2.105:3300/0 conn(0x7f9ef40ac9b0 0x7f9ef414af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef89ad700 1 -- 172.21.2.104:0/1977589600 >> v1:172.21.2.104:6789/0 conn(0x7f9ef40aff10 legacy=0x7f9ef40af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:45.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef89ad700 1 -- 172.21.2.104:0/1977589600 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef40aed20 con 0x7f9ef414ab80 2021-07-11T04:00:45.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef89ad700 1 --2- 172.21.2.104:0/1977589600 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ef414ab80 0x7f9ef405f6b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f9ee4002f70 tx=0x7f9ee4006df0).ready entity=mon.0 client_cookie=babc9246795f289d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:45.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef37fe700 1 -- 172.21.2.104:0/1977589600 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ee400d570 con 0x7f9ef414ab80 2021-07-11T04:00:45.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef37fe700 1 -- 172.21.2.104:0/1977589600 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ee4000e90 con 0x7f9ef414ab80 2021-07-11T04:00:45.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9ef37fe700 1 -- 172.21.2.104:0/1977589600 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ee400d8c0 con 0x7f9ef414ab80 2021-07-11T04:00:45.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9efac11700 1 -- 172.21.2.104:0/1977589600 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ef414ab80 msgr2=0x7f9ef405f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:45.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.835+0000 7f9efac11700 1 --2- 172.21.2.104:0/1977589600 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ef414ab80 0x7f9ef405f6b0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f9ee4002f70 tx=0x7f9ee4006df0).stop 2021-07-11T04:00:45.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 -- 172.21.2.104:0/1977589600 shutdown_connections 2021-07-11T04:00:45.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 --2- 172.21.2.104:0/1977589600 >> v2:172.21.2.105:3300/0 conn(0x7f9ef40ac9b0 0x7f9ef414af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 --2- 172.21.2.104:0/1977589600 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ef414ab80 0x7f9ef405f6b0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:45.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 -- 172.21.2.104:0/1977589600 >> 172.21.2.104:0/1977589600 conn(0x7f9ef40b5570 msgr2=0x7f9ef40b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:45.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 -- 172.21.2.104:0/1977589600 shutdown_connections 2021-07-11T04:00:45.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 -- 172.21.2.104:0/1977589600 wait complete. 2021-07-11T04:00:45.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 Processor -- start 2021-07-11T04:00:45.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 -- start start 2021-07-11T04:00:45.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ef40ac9b0 0x7f9ef4159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ef40aff10 0x7f9ef4063580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ef414ab80 0x7f9ef4141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef4146930 con 0x7f9ef414ab80 2021-07-11T04:00:45.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef4143570 con 0x7f9ef40ac9b0 2021-07-11T04:00:45.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ef415a640 con 0x7f9ef40aff10 2021-07-11T04:00:45.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef91ae700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ef414ab80 0x7f9ef4141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:45.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef91ae700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ef414ab80 0x7f9ef4141f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48304/0 (socket says 172.21.2.104:48304) 2021-07-11T04:00:45.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef91ae700 1 -- 172.21.2.104:0/1005656815 learned_addr learned my addr 172.21.2.104:0/1005656815 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:45.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef91ae700 1 -- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ef40aff10 msgr2=0x7f9ef4063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:45.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef91ae700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ef40aff10 0x7f9ef4063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef91ae700 1 -- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ef40ac9b0 msgr2=0x7f9ef4159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:45.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef91ae700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ef40ac9b0 0x7f9ef4159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:45.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef91ae700 1 -- 172.21.2.104:0/1005656815 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee400b180 con 0x7f9ef414ab80 2021-07-11T04:00:45.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef91ae700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ef414ab80 0x7f9ef4141f10 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f9eec0032f0 tx=0x7f9eec009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:45.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef1ffb700 1 -- 172.21.2.104:0/1005656815 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eec00d580 con 0x7f9ef414ab80 2021-07-11T04:00:45.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef1ffb700 1 -- 172.21.2.104:0/1005656815 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eec00d6e0 con 0x7f9ef414ab80 2021-07-11T04:00:45.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 -- 172.21.2.104:0/1005656815 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef41437a0 con 0x7f9ef414ab80 2021-07-11T04:00:45.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef1ffb700 1 -- 172.21.2.104:0/1005656815 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9eec00da40 con 0x7f9ef414ab80 2021-07-11T04:00:45.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 -- 172.21.2.104:0/1005656815 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ef4147010 con 0x7f9ef414ab80 2021-07-11T04:00:45.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9efac11700 1 -- 172.21.2.104:0/1005656815 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ef415a780 con 0x7f9ef414ab80 2021-07-11T04:00:45.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef1ffb700 1 -- 172.21.2.104:0/1005656815 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9eec00dba0 con 0x7f9ef414ab80 2021-07-11T04:00:45.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef1ffb700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ed80531f0 0x7f9ed80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:45.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.839+0000 7f9ef1ffb700 1 -- 172.21.2.104:0/1005656815 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f9eec06b760 con 0x7f9ef414ab80 2021-07-11T04:00:45.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.843+0000 7f9ef89ad700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ed80531f0 0x7f9ed80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:45.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.843+0000 7f9ef89ad700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ed80531f0 0x7f9ed80556b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f9ee4002f70 tx=0x7f9ee4014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:45.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.843+0000 7f9ef1ffb700 1 -- 172.21.2.104:0/1005656815 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9eec042a50 con 0x7f9ef414ab80 2021-07-11T04:00:45.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:45.951+0000 7f9efac11700 1 -- 172.21.2.104:0/1005656815 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9ef4057350 con 0x7f9ef414ab80 2021-07-11T04:00:46.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.379+0000 7f9ef1ffb700 1 -- 172.21.2.104:0/1005656815 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9eec046070 con 0x7f9ef414ab80 2021-07-11T04:00:46.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 -- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ed80531f0 msgr2=0x7f9ed80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:46.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ed80531f0 0x7f9ed80556b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f9ee4002f70 tx=0x7f9ee4014000).stop 2021-07-11T04:00:46.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 -- 172.21.2.104:0/1005656815 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ef414ab80 msgr2=0x7f9ef4141f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:46.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ef414ab80 0x7f9ef4141f10 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f9eec0032f0 tx=0x7f9eec009220).stop 2021-07-11T04:00:46.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 -- 172.21.2.104:0/1005656815 shutdown_connections 2021-07-11T04:00:46.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9ed80531f0 0x7f9ed80556b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:46.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ef414ab80 0x7f9ef4141f10 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:46.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ef40aff10 0x7f9ef4063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:46.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 --2- 172.21.2.104:0/1005656815 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ef40ac9b0 0x7f9ef4159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:46.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 -- 172.21.2.104:0/1005656815 >> 172.21.2.104:0/1005656815 conn(0x7f9ef40b5570 msgr2=0x7f9ef4060cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:46.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 -- 172.21.2.104:0/1005656815 shutdown_connections 2021-07-11T04:00:46.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.383+0000 7f9efac11700 1 -- 172.21.2.104:0/1005656815 wait complete. 2021-07-11T04:00:46.396 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T04:00:46.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.451+0000 7f5775caf700 1 Processor -- start 2021-07-11T04:00:46.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.451+0000 7f5775caf700 1 -- start start 2021-07-11T04:00:46.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.451+0000 7f5775caf700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700b1db0 0x7f57700b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:46.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.451+0000 7f5775caf700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57700b0350 con 0x7f57700a70c0 2021-07-11T04:00:46.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.451+0000 7f5775caf700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57700b05f0 con 0x7f57701483a0 2021-07-11T04:00:46.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.451+0000 7f5775caf700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57700a90c0 con 0x7f57700b1db0 2021-07-11T04:00:46.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.451+0000 7f576ffff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f57700a70c0 0x7f5770144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45082/0 (socket says 172.21.2.104:45082) 2021-07-11T04:00:46.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.451+0000 7f576ffff700 1 -- 172.21.2.104:0/276220262 learned_addr learned my addr 172.21.2.104:0/276220262 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:46.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.451+0000 7f576f7fe700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700b1db0 0x7f57700b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:46.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.451+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622093328 0 0) 0x7f57700b0350 con 0x7f57700a70c0 2021-07-11T04:00:46.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.451+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5750003260 con 0x7f57700a70c0 2021-07-11T04:00:46.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2519676034 0 0) 0x7f5750003260 con 0x7f57700a70c0 2021-07-11T04:00:46.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700b1db0 msgr2=0x7f57700b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:46.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700b1db0 0x7f57700b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:46.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 >> v1:172.21.2.105:6789/0 conn(0x7f57701483a0 legacy=0x7f57700b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:46.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700b0890 con 0x7f57700a70c0 2021-07-11T04:00:46.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f57680019e0 con 0x7f57700a70c0 2021-07-11T04:00:46.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 >> v1:172.21.2.104:6789/0 conn(0x7f57700a70c0 legacy=0x7f5770144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:46.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5750003780 0x7f5750003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:46.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5750004500 0x7f5750004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:46.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700b1db0 0x7f5750004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:46.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575000b750 con 0x7f5750004500 2021-07-11T04:00:46.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575000b8c0 con 0x7f5750003780 2021-07-11T04:00:46.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f575000ba80 con 0x7f57700b1db0 2021-07-11T04:00:46.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576effd700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5750004500 0x7f5750004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:46.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576f7fe700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5750003780 0x7f5750003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:46.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576ffff700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700b1db0 0x7f5750004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:46.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576effd700 1 -- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700b1db0 msgr2=0x7f5750004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:46.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576effd700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700b1db0 0x7f5750004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:46.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576effd700 1 -- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5750003780 msgr2=0x7f5750003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:46.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576effd700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5750003780 0x7f5750003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:46.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576effd700 1 -- 172.21.2.104:0/276220262 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5750003260 con 0x7f5750004500 2021-07-11T04:00:46.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576ffff700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700b1db0 0x7f5750004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:46.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576effd700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5750004500 0x7f5750004980 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f5770061440 tx=0x7f5760007a00).ready entity=mon.0 client_cookie=5f59850fdef90f07 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:46.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5760007d10 con 0x7f5750004500 2021-07-11T04:00:46.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5760007030 con 0x7f5750004500 2021-07-11T04:00:46.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576e7fc700 1 -- 172.21.2.104:0/276220262 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5760007390 con 0x7f5750004500 2021-07-11T04:00:46.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- 172.21.2.104:0/276220262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5750004500 msgr2=0x7f5750004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:46.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 --2- 172.21.2.104:0/276220262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5750004500 0x7f5750004980 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f5770061440 tx=0x7f5760007a00).stop 2021-07-11T04:00:46.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576ffff700 1 -- 172.21.2.104:0/276220262 reap_dead start 2021-07-11T04:00:46.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- 172.21.2.104:0/276220262 shutdown_connections 2021-07-11T04:00:46.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- 172.21.2.104:0/276220262 >> 172.21.2.104:0/276220262 conn(0x7f577005f6b0 msgr2=0x7f577005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:46.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- 172.21.2.104:0/276220262 shutdown_connections 2021-07-11T04:00:46.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- 172.21.2.104:0/276220262 wait complete. 2021-07-11T04:00:46.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 Processor -- start 2021-07-11T04:00:46.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- start start 2021-07-11T04:00:46.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5770141ee0 0x7f577013d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:46.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5770146fe0 0x7f577013dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:46.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700acea0 0x7f5770159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:46.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5770156620 con 0x7f5770141ee0 2021-07-11T04:00:46.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57701474b0 con 0x7f5770146fe0 2021-07-11T04:00:46.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5770146260 con 0x7f57700acea0 2021-07-11T04:00:46.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5770141ee0 0x7f577013d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:46.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5770141ee0 0x7f577013d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48322/0 (socket says 172.21.2.104:48322) 2021-07-11T04:00:46.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576f7fe700 1 -- 172.21.2.104:0/520630160 learned_addr learned my addr 172.21.2.104:0/520630160 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:46.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576f7fe700 1 -- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700acea0 msgr2=0x7f5770159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:46.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576f7fe700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700acea0 0x7f5770159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:46.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576f7fe700 1 -- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5770146fe0 msgr2=0x7f577013dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:46.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576f7fe700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5770146fe0 0x7f577013dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:46.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576f7fe700 1 -- 172.21.2.104:0/520630160 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57701422c0 con 0x7f5770141ee0 2021-07-11T04:00:46.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576f7fe700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5770141ee0 0x7f577013d870 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f575800b000 tx=0x7f5758008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:46.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576cff9700 1 -- 172.21.2.104:0/520630160 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f575800e6c0 con 0x7f5770141ee0 2021-07-11T04:00:46.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576cff9700 1 -- 172.21.2.104:0/520630160 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575800e820 con 0x7f5770141ee0 2021-07-11T04:00:46.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- 172.21.2.104:0/520630160 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5770145d20 con 0x7f5770141ee0 2021-07-11T04:00:46.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f576cff9700 1 -- 172.21.2.104:0/520630160 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f575800eb80 con 0x7f5770141ee0 2021-07-11T04:00:46.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- 172.21.2.104:0/520630160 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577013e250 con 0x7f5770141ee0 2021-07-11T04:00:46.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.455+0000 7f5775caf700 1 -- 172.21.2.104:0/520630160 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5770159bf0 con 0x7f5770141ee0 2021-07-11T04:00:46.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.459+0000 7f576cff9700 1 -- 172.21.2.104:0/520630160 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5758024070 con 0x7f5770141ee0 2021-07-11T04:00:46.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.459+0000 7f576cff9700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5754053270 0x7f5754055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:46.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.459+0000 7f576cff9700 1 -- 172.21.2.104:0/520630160 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f575806c010 con 0x7f5770141ee0 2021-07-11T04:00:46.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.459+0000 7f576effd700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5754053270 0x7f5754055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:46.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.459+0000 7f576effd700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5754053270 0x7f5754055730 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f5770061440 tx=0x7f5760013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:46.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.459+0000 7f576cff9700 1 -- 172.21.2.104:0/520630160 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5758043190 con 0x7f5770141ee0 2021-07-11T04:00:46.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:46.543+0000 7f5775caf700 1 -- 172.21.2.104:0/520630160 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5770057350 con 0x7f5770141ee0 2021-07-11T04:00:47.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f576cff9700 1 -- 172.21.2.104:0/520630160 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) v1 ==== 185+0+0 (secure 0 0 0) 0x7f57580467b0 con 0x7f5770141ee0 2021-07-11T04:00:47.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 -- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5754053270 msgr2=0x7f5754055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:47.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5754053270 0x7f5754055730 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f5770061440 tx=0x7f5760013000).stop 2021-07-11T04:00:47.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 -- 172.21.2.104:0/520630160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5770141ee0 msgr2=0x7f577013d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:47.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5770141ee0 0x7f577013d870 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f575800b000 tx=0x7f5758008040).stop 2021-07-11T04:00:47.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 -- 172.21.2.104:0/520630160 shutdown_connections 2021-07-11T04:00:47.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57700acea0 0x7f5770159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:47.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5770146fe0 0x7f577013dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:47.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5754053270 0x7f5754055730 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:47.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 --2- 172.21.2.104:0/520630160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5770141ee0 0x7f577013d870 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:47.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 -- 172.21.2.104:0/520630160 >> 172.21.2.104:0/520630160 conn(0x7f577005f6b0 msgr2=0x7f57700b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:47.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 -- 172.21.2.104:0/520630160 shutdown_connections 2021-07-11T04:00:47.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.391+0000 7f5775caf700 1 -- 172.21.2.104:0/520630160 wait complete. 2021-07-11T04:00:47.396 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T04:00:47.400 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T04:00:47.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.451+0000 7f81b1bef700 1 Processor -- start 2021-07-11T04:00:47.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.451+0000 7f81b1bef700 1 -- start start 2021-07-11T04:00:47.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.451+0000 7f81b1bef700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81ac0b1db0 0x7f81ac0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:47.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.451+0000 7f81b1bef700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f81ac0a70c0 0x7f81ac144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:47.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.451+0000 7f81b1bef700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81ac0b0350 con 0x7f81ac1483a0 2021-07-11T04:00:47.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.451+0000 7f81b1bef700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac0a90c0 con 0x7f81ac0a70c0 2021-07-11T04:00:47.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.451+0000 7f81b1bef700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0b05f0 con 0x7f81ac0b1db0 2021-07-11T04:00:47.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.451+0000 7f81abfff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f81ac0a70c0 0x7f81ac144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:47.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.451+0000 7f81abfff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f81ac0a70c0 0x7f81ac144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48334/0 (socket says 172.21.2.104:48334) 2021-07-11T04:00:47.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.451+0000 7f81abfff700 1 -- 172.21.2.104:0/2624273160 learned_addr learned my addr 172.21.2.104:0/2624273160 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:47.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81ab7fe700 1 --2- 172.21.2.104:0/2624273160 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81ac0b1db0 0x7f81ac0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:47.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 -- 172.21.2.104:0/2624273160 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81ac0b1db0 msgr2=0x7f81ac0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:47.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 --2- 172.21.2.104:0/2624273160 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81ac0b1db0 0x7f81ac0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:47.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 -- 172.21.2.104:0/2624273160 >> v1:172.21.2.105:6789/0 conn(0x7f81ac1483a0 legacy=0x7f81ac0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:47.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 -- 172.21.2.104:0/2624273160 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac0b0890 con 0x7f81ac0a70c0 2021-07-11T04:00:47.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81ab7fe700 1 --2- 172.21.2.104:0/2624273160 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81ac0b1db0 0x7f81ac0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:47.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 --2- 172.21.2.104:0/2624273160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81ac0a70c0 0x7f81ac144720 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f81ac0613a0 tx=0x7f81a4009290).ready entity=mon.0 client_cookie=ec2c1c175723e9af server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:47.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81aa7fc700 1 -- 172.21.2.104:0/2624273160 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81a400b860 con 0x7f81ac0a70c0 2021-07-11T04:00:47.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81aa7fc700 1 -- 172.21.2.104:0/2624273160 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a400b9c0 con 0x7f81ac0a70c0 2021-07-11T04:00:47.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81aa7fc700 1 -- 172.21.2.104:0/2624273160 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81a400bb70 con 0x7f81ac0a70c0 2021-07-11T04:00:47.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- 172.21.2.104:0/2624273160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81ac0a70c0 msgr2=0x7f81ac144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:47.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 --2- 172.21.2.104:0/2624273160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81ac0a70c0 0x7f81ac144720 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f81ac0613a0 tx=0x7f81a4009290).stop 2021-07-11T04:00:47.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- 172.21.2.104:0/2624273160 shutdown_connections 2021-07-11T04:00:47.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 --2- 172.21.2.104:0/2624273160 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81ac0a70c0 0x7f81ac144720 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:47.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 --2- 172.21.2.104:0/2624273160 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81ac0b1db0 0x7f81ac0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:47.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- 172.21.2.104:0/2624273160 >> 172.21.2.104:0/2624273160 conn(0x7f81ac05f6b0 msgr2=0x7f81ac05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:47.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- 172.21.2.104:0/2624273160 shutdown_connections 2021-07-11T04:00:47.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- 172.21.2.104:0/2624273160 wait complete. 2021-07-11T04:00:47.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 Processor -- start 2021-07-11T04:00:47.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- start start 2021-07-11T04:00:47.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81ac0a70c0 0x7f81ac0627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:47.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81ac0b1db0 0x7f81ac0b3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:47.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81ac1483a0 0x7f81ac0abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:47.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac066db0 con 0x7f81ac1483a0 2021-07-11T04:00:47.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0ab410 con 0x7f81ac0b1db0 2021-07-11T04:00:47.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac0ab6b0 con 0x7f81ac0a70c0 2021-07-11T04:00:47.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81ac1483a0 0x7f81ac0abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:47.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81ac1483a0 0x7f81ac0abfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48336/0 (socket says 172.21.2.104:48336) 2021-07-11T04:00:47.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 -- 172.21.2.104:0/2744311109 learned_addr learned my addr 172.21.2.104:0/2744311109 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:47.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 -- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81ac0a70c0 msgr2=0x7f81ac0627c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:47.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81ac0a70c0 0x7f81ac0627c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:47.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 -- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81ac0b1db0 msgr2=0x7f81ac0b3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:47.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81ac0b1db0 0x7f81ac0b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:47.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 -- 172.21.2.104:0/2744311109 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a4009530 con 0x7f81ac1483a0 2021-07-11T04:00:47.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81abfff700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81ac1483a0 0x7f81ac0abfc0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7f81ac0613a0 tx=0x7f81a4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:47.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81a8ff9700 1 -- 172.21.2.104:0/2744311109 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81a4011cf0 con 0x7f81ac1483a0 2021-07-11T04:00:47.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81a8ff9700 1 -- 172.21.2.104:0/2744311109 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4011e50 con 0x7f81ac1483a0 2021-07-11T04:00:47.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81a8ff9700 1 -- 172.21.2.104:0/2744311109 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81a4020620 con 0x7f81ac1483a0 2021-07-11T04:00:47.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- 172.21.2.104:0/2744311109 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81ac0ab950 con 0x7f81ac1483a0 2021-07-11T04:00:47.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- 172.21.2.104:0/2744311109 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac065630 con 0x7f81ac1483a0 2021-07-11T04:00:47.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.455+0000 7f81b1bef700 1 -- 172.21.2.104:0/2744311109 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac066f70 con 0x7f81ac1483a0 2021-07-11T04:00:47.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.459+0000 7f81a8ff9700 1 -- 172.21.2.104:0/2744311109 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f81a4026020 con 0x7f81ac1483a0 2021-07-11T04:00:47.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.459+0000 7f81a8ff9700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8190053160 0x7f8190055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:47.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.459+0000 7f81a8ff9700 1 -- 172.21.2.104:0/2744311109 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f81a406c240 con 0x7f81ac1483a0 2021-07-11T04:00:47.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.459+0000 7f81ab7fe700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8190053160 0x7f8190055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:47.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.459+0000 7f81ab7fe700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8190053160 0x7f8190055620 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f819c007000 tx=0x7f819c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:47.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.459+0000 7f81a8ff9700 1 -- 172.21.2.104:0/2744311109 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f81ac066f70 con 0x7f81ac1483a0 2021-07-11T04:00:47.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:47.547+0000 7f81b1bef700 1 -- 172.21.2.104:0/2744311109 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f81ac057350 con 0x7f81ac1483a0 2021-07-11T04:00:48.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.391+0000 7f81a8ff9700 1 -- 172.21.2.104:0/2744311109 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) v1 ==== 173+0+0 (secure 0 0 0) 0x7f81a4015020 con 0x7f81ac1483a0 2021-07-11T04:00:48.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 -- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8190053160 msgr2=0x7f8190055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:48.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8190053160 0x7f8190055620 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f819c007000 tx=0x7f819c008040).stop 2021-07-11T04:00:48.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 -- 172.21.2.104:0/2744311109 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81ac1483a0 msgr2=0x7f81ac0abfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:48.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81ac1483a0 0x7f81ac0abfc0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7f81ac0613a0 tx=0x7f81a4016000).stop 2021-07-11T04:00:48.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 -- 172.21.2.104:0/2744311109 shutdown_connections 2021-07-11T04:00:48.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8190053160 0x7f8190055620 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:48.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81ac1483a0 0x7f81ac0abfc0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:48.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81ac0b1db0 0x7f81ac0b3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 --2- 172.21.2.104:0/2744311109 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81ac0a70c0 0x7f81ac0627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 -- 172.21.2.104:0/2744311109 >> 172.21.2.104:0/2744311109 conn(0x7f81ac05f6b0 msgr2=0x7f81ac146e20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:48.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 -- 172.21.2.104:0/2744311109 shutdown_connections 2021-07-11T04:00:48.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.395+0000 7f81b1bef700 1 -- 172.21.2.104:0/2744311109 wait complete. 2021-07-11T04:00:48.399 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T04:00:48.402 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-07-11T04:00:48.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.451+0000 7f73d8054700 1 Processor -- start 2021-07-11T04:00:48.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.451+0000 7f73d8054700 1 -- start start 2021-07-11T04:00:48.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.451+0000 7f73d8054700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f73d00b1ec0 0x7f73d00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.451+0000 7f73d8054700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f73d00b22a0 0x7f73d0141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.451+0000 7f73d8054700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d00b0420 con 0x7f73d00affb0 2021-07-11T04:00:48.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.451+0000 7f73d8054700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d00a77d0 con 0x7f73d00b1ec0 2021-07-11T04:00:48.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.451+0000 7f73d8054700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d00b0650 con 0x7f73d00b22a0 2021-07-11T04:00:48.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f73d00b1ec0 0x7f73d00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:48.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f73d00b1ec0 0x7f73d00b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48344/0 (socket says 172.21.2.104:48344) 2021-07-11T04:00:48.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 -- 172.21.2.104:0/83096164 learned_addr learned my addr 172.21.2.104:0/83096164 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:48.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 -- 172.21.2.104:0/83096164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f73d00b22a0 msgr2=0x7f73d0141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:48.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 --2- 172.21.2.104:0/83096164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f73d00b22a0 0x7f73d0141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 -- 172.21.2.104:0/83096164 >> v1:172.21.2.105:6789/0 conn(0x7f73d00affb0 legacy=0x7f73d00a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:48.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 -- 172.21.2.104:0/83096164 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d00b08f0 con 0x7f73d00b1ec0 2021-07-11T04:00:48.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 --2- 172.21.2.104:0/83096164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f73d00b1ec0 0x7f73d00b12d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7f73d00b2f50 tx=0x7f73cc006eb0).ready entity=mon.0 client_cookie=db9ae834bf9db012 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:48.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d4dee700 1 -- 172.21.2.104:0/83096164 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73cc00d570 con 0x7f73d00b1ec0 2021-07-11T04:00:48.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d4dee700 1 -- 172.21.2.104:0/83096164 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc000e90 con 0x7f73d00b1ec0 2021-07-11T04:00:48.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d4dee700 1 -- 172.21.2.104:0/83096164 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73cc00d8c0 con 0x7f73d00b1ec0 2021-07-11T04:00:48.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 -- 172.21.2.104:0/83096164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f73d00b1ec0 msgr2=0x7f73d00b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:48.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 --2- 172.21.2.104:0/83096164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f73d00b1ec0 0x7f73d00b12d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7f73d00b2f50 tx=0x7f73cc006eb0).stop 2021-07-11T04:00:48.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 -- 172.21.2.104:0/83096164 shutdown_connections 2021-07-11T04:00:48.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 --2- 172.21.2.104:0/83096164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f73d00b22a0 0x7f73d0141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 --2- 172.21.2.104:0/83096164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f73d00b1ec0 0x7f73d00b12d0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:48.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 -- 172.21.2.104:0/83096164 >> 172.21.2.104:0/83096164 conn(0x7f73d005f6b0 msgr2=0x7f73d005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:48.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 -- 172.21.2.104:0/83096164 shutdown_connections 2021-07-11T04:00:48.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 -- 172.21.2.104:0/83096164 wait complete. 2021-07-11T04:00:48.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 Processor -- start 2021-07-11T04:00:48.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 -- start start 2021-07-11T04:00:48.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f73d00affb0 0x7f73d00ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f73d00b1ec0 0x7f73d01574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f73d00b22a0 0x7f73d00b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0140080 con 0x7f73d00affb0 2021-07-11T04:00:48.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d0157990 con 0x7f73d00b1ec0 2021-07-11T04:00:48.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d013f8a0 con 0x7f73d00b22a0 2021-07-11T04:00:48.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f73d00affb0 0x7f73d00ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:48.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f73d00affb0 0x7f73d00ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48350/0 (socket says 172.21.2.104:48350) 2021-07-11T04:00:48.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 -- 172.21.2.104:0/2612066192 learned_addr learned my addr 172.21.2.104:0/2612066192 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:48.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 -- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f73d00b22a0 msgr2=0x7f73d00b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:48.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f73d00b22a0 0x7f73d00b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 -- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f73d00b1ec0 msgr2=0x7f73d01574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:48.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f73d00b1ec0 0x7f73d01574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 -- 172.21.2.104:0/2612066192 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc00b1e0 con 0x7f73d00affb0 2021-07-11T04:00:48.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d5df0700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f73d00affb0 0x7f73d00ad860 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7f73d00b2f50 tx=0x7f73cc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:48.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73c2ffd700 1 -- 172.21.2.104:0/2612066192 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73cc021070 con 0x7f73d00affb0 2021-07-11T04:00:48.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73c2ffd700 1 -- 172.21.2.104:0/2612066192 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc000e90 con 0x7f73d00affb0 2021-07-11T04:00:48.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73c2ffd700 1 -- 172.21.2.104:0/2612066192 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f73cc002f50 con 0x7f73d00affb0 2021-07-11T04:00:48.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 -- 172.21.2.104:0/2612066192 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d013fb40 con 0x7f73d00affb0 2021-07-11T04:00:48.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.455+0000 7f73d8054700 1 -- 172.21.2.104:0/2612066192 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d00b4b60 con 0x7f73d00affb0 2021-07-11T04:00:48.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.459+0000 7f73d8054700 1 -- 172.21.2.104:0/2612066192 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d00b3260 con 0x7f73d00affb0 2021-07-11T04:00:48.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.459+0000 7f73c2ffd700 1 -- 172.21.2.104:0/2612066192 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f73cc00f070 con 0x7f73d00affb0 2021-07-11T04:00:48.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.459+0000 7f73c2ffd700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73b0053380 0x7f73b0055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.459+0000 7f73c2ffd700 1 -- 172.21.2.104:0/2612066192 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f73cc06c790 con 0x7f73d00affb0 2021-07-11T04:00:48.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.459+0000 7f73d55ef700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73b0053380 0x7f73b0055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:48.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.459+0000 7f73d55ef700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73b0053380 0x7f73b0055840 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f73c4002f90 tx=0x7f73c40069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:48.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.459+0000 7f73c2ffd700 1 -- 172.21.2.104:0/2612066192 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f73d00b3260 con 0x7f73d00affb0 2021-07-11T04:00:48.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.543+0000 7f73d8054700 1 -- 172.21.2.104:0/2612066192 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f73d00b3210 con 0x7f73d00affb0 2021-07-11T04:00:48.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73c2ffd700 1 -- 172.21.2.104:0/2612066192 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f73cc043c40 con 0x7f73d00affb0 2021-07-11T04:00:48.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 -- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73b0053380 msgr2=0x7f73b0055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:48.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73b0053380 0x7f73b0055840 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f73c4002f90 tx=0x7f73c40069b0).stop 2021-07-11T04:00:48.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 -- 172.21.2.104:0/2612066192 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f73d00affb0 msgr2=0x7f73d00ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:48.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f73d00affb0 0x7f73d00ad860 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7f73d00b2f50 tx=0x7f73cc015000).stop 2021-07-11T04:00:48.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 -- 172.21.2.104:0/2612066192 shutdown_connections 2021-07-11T04:00:48.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f73d00b22a0 0x7f73d00b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f73d00b1ec0 0x7f73d01574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73b0053380 0x7f73b0055840 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:48.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 --2- 172.21.2.104:0/2612066192 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f73d00affb0 0x7f73d00ad860 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:48.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 -- 172.21.2.104:0/2612066192 >> 172.21.2.104:0/2612066192 conn(0x7f73d005f6b0 msgr2=0x7f73d005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:48.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 -- 172.21.2.104:0/2612066192 shutdown_connections 2021-07-11T04:00:48.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.551+0000 7f73d8054700 1 -- 172.21.2.104:0/2612066192 wait complete. 2021-07-11T04:00:48.559 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical' 2021-07-11T04:00:48.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.607+0000 7f5a000a5700 1 Processor -- start 2021-07-11T04:00:48.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- start start 2021-07-11T04:00:48.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f59f80b1db0 0x7f59f80b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59f81483a0 0x7f59f80b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f80b0350 con 0x7f59f80a70c0 2021-07-11T04:00:48.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f80a90c0 con 0x7f59f80b1db0 2021-07-11T04:00:48.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f80b05f0 con 0x7f59f81483a0 2021-07-11T04:00:48.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fde41700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f59f80b1db0 0x7f59f80b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:48.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fde41700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f59f80b1db0 0x7f59f80b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48358/0 (socket says 172.21.2.104:48358) 2021-07-11T04:00:48.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fde41700 1 -- 172.21.2.104:0/276579910 learned_addr learned my addr 172.21.2.104:0/276579910 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:48.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fde41700 1 -- 172.21.2.104:0/276579910 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59f81483a0 msgr2=0x7f59f80b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:48.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fde41700 1 --2- 172.21.2.104:0/276579910 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59f81483a0 0x7f59f80b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fde41700 1 -- 172.21.2.104:0/276579910 >> v1:172.21.2.105:6789/0 conn(0x7f59f80a70c0 legacy=0x7f59f8144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:48.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fde41700 1 -- 172.21.2.104:0/276579910 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f80b0890 con 0x7f59f80b1db0 2021-07-11T04:00:48.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fde41700 1 --2- 172.21.2.104:0/276579910 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59f80b1db0 0x7f59f80b0fd0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f59f80613a0 tx=0x7f59f4006eb0).ready entity=mon.0 client_cookie=1930c039ba759a42 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:48.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fce3f700 1 -- 172.21.2.104:0/276579910 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59f400d570 con 0x7f59f80b1db0 2021-07-11T04:00:48.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fce3f700 1 -- 172.21.2.104:0/276579910 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f4000e90 con 0x7f59f80b1db0 2021-07-11T04:00:48.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fce3f700 1 -- 172.21.2.104:0/276579910 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59f400d8c0 con 0x7f59f80b1db0 2021-07-11T04:00:48.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- 172.21.2.104:0/276579910 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59f80b1db0 msgr2=0x7f59f80b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:48.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 --2- 172.21.2.104:0/276579910 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59f80b1db0 0x7f59f80b0fd0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f59f80613a0 tx=0x7f59f4006eb0).stop 2021-07-11T04:00:48.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- 172.21.2.104:0/276579910 shutdown_connections 2021-07-11T04:00:48.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 --2- 172.21.2.104:0/276579910 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59f81483a0 0x7f59f80b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 --2- 172.21.2.104:0/276579910 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59f80b1db0 0x7f59f80b0fd0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:48.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- 172.21.2.104:0/276579910 >> 172.21.2.104:0/276579910 conn(0x7f59f805f6b0 msgr2=0x7f59f805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:48.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- 172.21.2.104:0/276579910 shutdown_connections 2021-07-11T04:00:48.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- 172.21.2.104:0/276579910 wait complete. 2021-07-11T04:00:48.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 Processor -- start 2021-07-11T04:00:48.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- start start 2021-07-11T04:00:48.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59f80a70c0 0x7f59f80aac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59f80b1db0 0x7f59f81568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f59f81483a0 0x7f59f8156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f80ac720 con 0x7f59f80b1db0 2021-07-11T04:00:48.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f8149e40 con 0x7f59f81483a0 2021-07-11T04:00:48.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f814a0e0 con 0x7f59f80a70c0 2021-07-11T04:00:48.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fd640700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59f80b1db0 0x7f59f81568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:48.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fd640700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59f80b1db0 0x7f59f81568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48368/0 (socket says 172.21.2.104:48368) 2021-07-11T04:00:48.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fd640700 1 -- 172.21.2.104:0/3118511494 learned_addr learned my addr 172.21.2.104:0/3118511494 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:48.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fd640700 1 -- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59f80a70c0 msgr2=0x7f59f80aac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:48.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fd640700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59f80a70c0 0x7f59f80aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fd640700 1 -- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f59f81483a0 msgr2=0x7f59f8156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:48.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fd640700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f59f81483a0 0x7f59f8156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:48.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fd640700 1 -- 172.21.2.104:0/3118511494 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f400b1e0 con 0x7f59f80b1db0 2021-07-11T04:00:48.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59fd640700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59f80b1db0 0x7f59f81568c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f59e8002f70 tx=0x7f59e8006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:48.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59eeffd700 1 -- 172.21.2.104:0/3118511494 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59e800e470 con 0x7f59f80b1db0 2021-07-11T04:00:48.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59eeffd700 1 -- 172.21.2.104:0/3118511494 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e800e5d0 con 0x7f59f80b1db0 2021-07-11T04:00:48.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f59eeffd700 1 -- 172.21.2.104:0/3118511494 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f59e800e930 con 0x7f59f80b1db0 2021-07-11T04:00:48.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- 172.21.2.104:0/3118511494 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f814a380 con 0x7f59f80b1db0 2021-07-11T04:00:48.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.611+0000 7f5a000a5700 1 -- 172.21.2.104:0/3118511494 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f80ac210 con 0x7f59f80b1db0 2021-07-11T04:00:48.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.615+0000 7f5a000a5700 1 -- 172.21.2.104:0/3118511494 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f814a510 con 0x7f59f80b1db0 2021-07-11T04:00:48.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.615+0000 7f59eeffd700 1 -- 172.21.2.104:0/3118511494 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f59e8018030 con 0x7f59f80b1db0 2021-07-11T04:00:48.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.615+0000 7f59eeffd700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f59e0053270 0x7f59e0055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:48.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.615+0000 7f59eeffd700 1 -- 172.21.2.104:0/3118511494 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f59e806a6c0 con 0x7f59f80b1db0 2021-07-11T04:00:48.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.615+0000 7f59fde41700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f59e0053270 0x7f59e0055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:48.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.615+0000 7f59fde41700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f59e0053270 0x7f59e0055730 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f59f80613a0 tx=0x7f59f4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:48.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.615+0000 7f59eeffd700 1 -- 172.21.2.104:0/3118511494 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f59e8041d30 con 0x7f59f80b1db0 2021-07-11T04:00:48.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:48.727+0000 7f5a000a5700 1 -- 172.21.2.104:0/3118511494 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]} v 0) v1 -- 0x7f59f8057350 con 0x7f59f80b1db0 2021-07-11T04:00:49.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f59eeffd700 1 -- 172.21.2.104:0/3118511494 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_dump_inode_hexademical"]}]=0 v315) v1 ==== 138+0+0 (secure 0 0 0) 0x7f59e8045350 con 0x7f59f80b1db0 2021-07-11T04:00:49.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 -- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f59e0053270 msgr2=0x7f59e0055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:49.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f59e0053270 0x7f59e0055730 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f59f80613a0 tx=0x7f59f4015000).stop 2021-07-11T04:00:49.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 -- 172.21.2.104:0/3118511494 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59f80b1db0 msgr2=0x7f59f81568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:49.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59f80b1db0 0x7f59f81568c0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f59e8002f70 tx=0x7f59e8006e20).stop 2021-07-11T04:00:49.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 -- 172.21.2.104:0/3118511494 shutdown_connections 2021-07-11T04:00:49.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f59f81483a0 0x7f59f8156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:49.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f59e0053270 0x7f59e0055730 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:49.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59f80b1db0 0x7f59f81568c0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:49.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 --2- 172.21.2.104:0/3118511494 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59f80a70c0 0x7f59f80aac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:49.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 -- 172.21.2.104:0/3118511494 >> 172.21.2.104:0/3118511494 conn(0x7f59f805f6b0 msgr2=0x7f59f80a8720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:49.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 -- 172.21.2.104:0/3118511494 shutdown_connections 2021-07-11T04:00:49.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.403+0000 7f5a000a5700 1 -- 172.21.2.104:0/3118511494 wait complete. 2021-07-11T04:00:49.411 INFO:tasks.cephfs_test_runner:test_dump_inode_hexademical (tasks.cephfs.test_misc.TestMisc) ... ok 2021-07-11T04:00:49.411 INFO:tasks.cephfs_test_runner:Starting test: test_filtered_df (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:00:49.412 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2021-07-11T04:00:49.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 Processor -- start 2021-07-11T04:00:49.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 -- start start 2021-07-11T04:00:49.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbc240b0880 0x7fbc2414ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:49.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240b2af0 con 0x7fbc240aa120 2021-07-11T04:00:49.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc240b2770 con 0x7fbc240b04a0 2021-07-11T04:00:49.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc240b2d20 con 0x7fbc240b0880 2021-07-11T04:00:49.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc29540700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fbc240aa120 0x7fbc240ae440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45140/0 (socket says 172.21.2.104:45140) 2021-07-11T04:00:49.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc29540700 1 -- 172.21.2.104:0/4069547123 learned_addr learned my addr 172.21.2.104:0/4069547123 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:49.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2a542700 1 --2- 172.21.2.104:0/4069547123 >> v2:172.21.2.104:3300/0 conn(0x7fbc240b0880 0x7fbc2414ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:49.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2a542700 1 -- 172.21.2.104:0/4069547123 >> v1:172.21.2.105:6789/0 conn(0x7fbc240b04a0 legacy=0x7fbc240add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:49.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2a542700 1 -- 172.21.2.104:0/4069547123 >> v1:172.21.2.104:6789/0 conn(0x7fbc240aa120 legacy=0x7fbc240ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:49.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2a542700 1 -- 172.21.2.104:0/4069547123 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc240a9b70 con 0x7fbc240b0880 2021-07-11T04:00:49.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2a542700 1 --2- 172.21.2.104:0/4069547123 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc240b0880 0x7fbc2414ac60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7fbc240ad3d0 tx=0x7fbc1c0091d0).ready entity=mon.0 client_cookie=a4bb8b369675d76b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:49.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc28d3f700 1 -- 172.21.2.104:0/4069547123 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc1c00b820 con 0x7fbc240b0880 2021-07-11T04:00:49.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc28d3f700 1 -- 172.21.2.104:0/4069547123 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c00b980 con 0x7fbc240b0880 2021-07-11T04:00:49.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc28d3f700 1 -- 172.21.2.104:0/4069547123 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc1c00baf0 con 0x7fbc240b0880 2021-07-11T04:00:49.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/4069547123 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc240b0880 msgr2=0x7fbc2414ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:49.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 --2- 172.21.2.104:0/4069547123 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc240b0880 0x7fbc2414ac60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7fbc240ad3d0 tx=0x7fbc1c0091d0).stop 2021-07-11T04:00:49.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/4069547123 shutdown_connections 2021-07-11T04:00:49.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 --2- 172.21.2.104:0/4069547123 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc240b0880 0x7fbc2414ac60 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:49.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/4069547123 >> 172.21.2.104:0/4069547123 conn(0x7fbc240607c0 msgr2=0x7fbc24060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:49.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/4069547123 shutdown_connections 2021-07-11T04:00:49.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/4069547123 wait complete. 2021-07-11T04:00:49.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 Processor -- start 2021-07-11T04:00:49.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.463+0000 7fbc2bfa5700 1 -- start start 2021-07-11T04:00:49.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2bfa5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc240aa120 0x7fbc24064b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:49.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2bfa5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc240b04a0 0x7fbc24134b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:49.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2bfa5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc240b0880 0x7fbc240674a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:49.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2bfa5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240aaac0 con 0x7fbc240b0880 2021-07-11T04:00:49.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2bfa5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240ab5d0 con 0x7fbc240b04a0 2021-07-11T04:00:49.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2bfa5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc240ab870 con 0x7fbc240aa120 2021-07-11T04:00:49.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2a542700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc240b0880 0x7fbc240674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:49.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2a542700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc240b0880 0x7fbc240674a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48382/0 (socket says 172.21.2.104:48382) 2021-07-11T04:00:49.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2a542700 1 -- 172.21.2.104:0/3759442806 learned_addr learned my addr 172.21.2.104:0/3759442806 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:49.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc29d41700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc240aa120 0x7fbc24064b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:49.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc29540700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc240b04a0 0x7fbc24134b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:49.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2a542700 1 -- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc240aa120 msgr2=0x7fbc24064b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:49.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2a542700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc240aa120 0x7fbc24064b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:49.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2a542700 1 -- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc240b04a0 msgr2=0x7fbc24134b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:49.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2a542700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc240b04a0 0x7fbc24134b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:49.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2a542700 1 -- 172.21.2.104:0/3759442806 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c009470 con 0x7fbc240b0880 2021-07-11T04:00:49.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2a542700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc240b0880 0x7fbc240674a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7fbc240ad3d0 tx=0x7fbc1c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:49.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc16ffd700 1 -- 172.21.2.104:0/3759442806 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc1c011d70 con 0x7fbc240b0880 2021-07-11T04:00:49.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc16ffd700 1 -- 172.21.2.104:0/3759442806 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c021440 con 0x7fbc240b0880 2021-07-11T04:00:49.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/3759442806 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc240abb10 con 0x7fbc240b0880 2021-07-11T04:00:49.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc16ffd700 1 -- 172.21.2.104:0/3759442806 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbc1c0217a0 con 0x7fbc240b0880 2021-07-11T04:00:49.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/3759442806 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc24067990 con 0x7fbc240b0880 2021-07-11T04:00:49.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/3759442806 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc24132fe0 con 0x7fbc240b0880 2021-07-11T04:00:49.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc16ffd700 1 -- 172.21.2.104:0/3759442806 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbc1c027020 con 0x7fbc240b0880 2021-07-11T04:00:49.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc16ffd700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc04053350 0x7fbc04055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:49.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc16ffd700 1 -- 172.21.2.104:0/3759442806 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbc1c06c580 con 0x7fbc240b0880 2021-07-11T04:00:49.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc29d41700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc04053350 0x7fbc04055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:49.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.467+0000 7fbc29d41700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc04053350 0x7fbc04055810 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fbc2000d000 tx=0x7fbc20007b70).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:49.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.471+0000 7fbc16ffd700 1 -- 172.21.2.104:0/3759442806 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbc24132fe0 con 0x7fbc240b0880 2021-07-11T04:00:49.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:49.583+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/3759442806 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7fbc24132fe0 con 0x7fbc240b0880 2021-07-11T04:00:50.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc16ffd700 1 -- 172.21.2.104:0/3759442806 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v316) v1 ==== 130+0+0 (secure 0 0 0) 0x7fbc24132fe0 con 0x7fbc240b0880 2021-07-11T04:00:50.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc04053350 msgr2=0x7fbc04055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:50.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc04053350 0x7fbc04055810 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fbc2000d000 tx=0x7fbc20007b70).stop 2021-07-11T04:00:50.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/3759442806 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc240b0880 msgr2=0x7fbc240674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:50.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc240b0880 0x7fbc240674a0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7fbc240ad3d0 tx=0x7fbc1c016000).stop 2021-07-11T04:00:50.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/3759442806 shutdown_connections 2021-07-11T04:00:50.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbc04053350 0x7fbc04055810 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:50.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbc240b0880 0x7fbc240674a0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:50.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbc240b04a0 0x7fbc24134b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:50.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 --2- 172.21.2.104:0/3759442806 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbc240aa120 0x7fbc24064b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:50.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/3759442806 >> 172.21.2.104:0/3759442806 conn(0x7fbc240607c0 msgr2=0x7fbc2414c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:50.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/3759442806 shutdown_connections 2021-07-11T04:00:50.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.407+0000 7fbc2bfa5700 1 -- 172.21.2.104:0/3759442806 wait complete. 2021-07-11T04:00:50.417 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T04:00:50.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.467+0000 7fc1fffec700 1 Processor -- start 2021-07-11T04:00:50.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.467+0000 7fc1fffec700 1 -- start start 2021-07-11T04:00:50.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.467+0000 7fc1fffec700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f80b12d0 0x7fc1f80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.467+0000 7fc1fffec700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc1f80b22a0 0x7fc1f8141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.467+0000 7fc1fffec700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f80a77d0 con 0x7fc1f80b1ec0 2021-07-11T04:00:50.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.467+0000 7fc1fffec700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f80b1740 con 0x7fc1f80b22a0 2021-07-11T04:00:50.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.467+0000 7fc1fffec700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80b0650 con 0x7fc1f80b12d0 2021-07-11T04:00:50.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.467+0000 7fc1fdd88700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc1f80b1ec0 0x7fc1f80a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45154/0 (socket says 172.21.2.104:45154) 2021-07-11T04:00:50.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.467+0000 7fc1fdd88700 1 -- 172.21.2.104:0/1099588182 learned_addr learned my addr 172.21.2.104:0/1099588182 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:50.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140573671 0 0) 0x7fc1f80a77d0 con 0x7fc1f80b1ec0 2021-07-11T04:00:50.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e0003260 con 0x7fc1f80b1ec0 2021-07-11T04:00:50.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fd587700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f80b12d0 0x7fc1f80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:50.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fe589700 1 --2- 172.21.2.104:0/1099588182 >> v2:172.21.2.105:3300/0 conn(0x7fc1f80b22a0 0x7fc1f8141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:50.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 660964327 0 0) 0x7fc1e0003260 con 0x7fc1f80b1ec0 2021-07-11T04:00:50.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f80b12d0 msgr2=0x7fc1f80affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:50.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f80b12d0 0x7fc1f80affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:50.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 >> v2:172.21.2.105:3300/0 conn(0x7fc1f80b22a0 msgr2=0x7fc1f8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:50.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 --2- 172.21.2.104:0/1099588182 >> v2:172.21.2.105:3300/0 conn(0x7fc1f80b22a0 0x7fc1f8141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:50.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f80b08f0 con 0x7fc1f80b1ec0 2021-07-11T04:00:50.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fc1f4001660 con 0x7fc1f80b1ec0 2021-07-11T04:00:50.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 >> v1:172.21.2.104:6789/0 conn(0x7fc1f80b1ec0 legacy=0x7fc1f80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:50.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1e00037d0 0x7fc1e0003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f80b12d0 0x7fc1e0004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00049f0 0x7fc1e0004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80b1740 con 0x7fc1e00049f0 2021-07-11T04:00:50.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f80b0650 con 0x7fc1e00037d0 2021-07-11T04:00:50.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e000b8b0 con 0x7fc1f80b12d0 2021-07-11T04:00:50.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fe589700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00049f0 0x7fc1e0004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:50.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fd587700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f80b12d0 0x7fc1e0004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:50.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fe589700 1 -- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f80b12d0 msgr2=0x7fc1e0004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:50.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fe589700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f80b12d0 0x7fc1e0004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:50.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fe589700 1 -- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1e00037d0 msgr2=0x7fc1e0003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:50.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fdd88700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1e00037d0 0x7fc1e0003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:50.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fe589700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1e00037d0 0x7fc1e0003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:50.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fe589700 1 -- 172.21.2.104:0/1099588182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e0003260 con 0x7fc1e00049f0 2021-07-11T04:00:50.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fdd88700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1e00037d0 0x7fc1e0003bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:50.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fe589700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00049f0 0x7fc1e0004e70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7fc1f000a000 tx=0x7fc1f0008040).ready entity=mon.0 client_cookie=c6ef08fed2b6319c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:50.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1f000c070 con 0x7fc1e00049f0 2021-07-11T04:00:50.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f000e5f0 con 0x7fc1e00049f0 2021-07-11T04:00:50.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fcd86700 1 -- 172.21.2.104:0/1099588182 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1f000e950 con 0x7fc1e00049f0 2021-07-11T04:00:50.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 -- 172.21.2.104:0/1099588182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00049f0 msgr2=0x7fc1e0004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:50.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 --2- 172.21.2.104:0/1099588182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1e00049f0 0x7fc1e0004e70 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7fc1f000a000 tx=0x7fc1f0008040).stop 2021-07-11T04:00:50.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fe589700 1 -- 172.21.2.104:0/1099588182 reap_dead start 2021-07-11T04:00:50.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 -- 172.21.2.104:0/1099588182 shutdown_connections 2021-07-11T04:00:50.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 -- 172.21.2.104:0/1099588182 >> 172.21.2.104:0/1099588182 conn(0x7fc1f805f6b0 msgr2=0x7fc1f805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:50.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 -- 172.21.2.104:0/1099588182 shutdown_connections 2021-07-11T04:00:50.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 -- 172.21.2.104:0/1099588182 wait complete. 2021-07-11T04:00:50.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 Processor -- start 2021-07-11T04:00:50.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 -- start start 2021-07-11T04:00:50.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f815a550 0x7fc1f815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1f8145270 0x7fc1f80ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1f81353e0 0x7fc1f8135890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f813e500 con 0x7fc1f8145270 2021-07-11T04:00:50.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f815a930 con 0x7fc1f81353e0 2021-07-11T04:00:50.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fffec700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f813e010 con 0x7fc1f815a550 2021-07-11T04:00:50.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fd587700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1f8145270 0x7fc1f80ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:50.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fd587700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1f8145270 0x7fc1f80ab7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48400/0 (socket says 172.21.2.104:48400) 2021-07-11T04:00:50.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fd587700 1 -- 172.21.2.104:0/1688685608 learned_addr learned my addr 172.21.2.104:0/1688685608 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:50.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fd587700 1 -- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f815a550 msgr2=0x7fc1f815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:50.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fd587700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f815a550 0x7fc1f815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:50.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fd587700 1 -- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1f81353e0 msgr2=0x7fc1f8135890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:50.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fd587700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1f81353e0 0x7fc1f8135890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:50.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fd587700 1 -- 172.21.2.104:0/1688685608 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f813da40 con 0x7fc1f8145270 2021-07-11T04:00:50.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.471+0000 7fc1fd587700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1f8145270 0x7fc1f80ab7e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7fc1ec0030b0 tx=0x7fc1ec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:50.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.475+0000 7fc1eaffd700 1 -- 172.21.2.104:0/1688685608 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1ec013800 con 0x7fc1f8145270 2021-07-11T04:00:50.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.475+0000 7fc1eaffd700 1 -- 172.21.2.104:0/1688685608 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ec000e90 con 0x7fc1f8145270 2021-07-11T04:00:50.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.475+0000 7fc1fffec700 1 -- 172.21.2.104:0/1688685608 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f80a95a0 con 0x7fc1f8145270 2021-07-11T04:00:50.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.475+0000 7fc1fffec700 1 -- 172.21.2.104:0/1688685608 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f813e1f0 con 0x7fc1f8145270 2021-07-11T04:00:50.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.475+0000 7fc1eaffd700 1 -- 172.21.2.104:0/1688685608 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc1ec013b50 con 0x7fc1f8145270 2021-07-11T04:00:50.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.475+0000 7fc1fffec700 1 -- 172.21.2.104:0/1688685608 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f80a9730 con 0x7fc1f8145270 2021-07-11T04:00:50.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.475+0000 7fc1eaffd700 1 -- 172.21.2.104:0/1688685608 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc1ec013cb0 con 0x7fc1f8145270 2021-07-11T04:00:50.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.475+0000 7fc1eaffd700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1d8053260 0x7fc1d8055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.475+0000 7fc1eaffd700 1 -- 172.21.2.104:0/1688685608 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc1ec06b330 con 0x7fc1f8145270 2021-07-11T04:00:50.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.475+0000 7fc1fdd88700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1d8053260 0x7fc1d8055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:50.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.475+0000 7fc1fdd88700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1d8053260 0x7fc1d8055720 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fc1f4008000 tx=0x7fc1f4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:50.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.479+0000 7fc1eaffd700 1 -- 172.21.2.104:0/1688685608 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc1ec042950 con 0x7fc1f8145270 2021-07-11T04:00:50.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.559+0000 7fc1fffec700 1 -- 172.21.2.104:0/1688685608 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc1f8057350 con 0x7fc1f8145270 2021-07-11T04:00:50.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1eaffd700 1 -- 172.21.2.104:0/1688685608 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc1ec017020 con 0x7fc1f8145270 2021-07-11T04:00:50.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 -- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1d8053260 msgr2=0x7fc1d8055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:50.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1d8053260 0x7fc1d8055720 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fc1f4008000 tx=0x7fc1f4009040).stop 2021-07-11T04:00:50.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 -- 172.21.2.104:0/1688685608 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1f8145270 msgr2=0x7fc1f80ab7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:50.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1f8145270 0x7fc1f80ab7e0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7fc1ec0030b0 tx=0x7fc1ec008040).stop 2021-07-11T04:00:50.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 -- 172.21.2.104:0/1688685608 shutdown_connections 2021-07-11T04:00:50.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc1f81353e0 0x7fc1f8135890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:50.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc1d8053260 0x7fc1d8055720 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:50.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc1f8145270 0x7fc1f80ab7e0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:50.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 --2- 172.21.2.104:0/1688685608 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc1f815a550 0x7fc1f815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:50.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 -- 172.21.2.104:0/1688685608 >> 172.21.2.104:0/1688685608 conn(0x7fc1f805f6b0 msgr2=0x7fc1f8136b30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:50.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 -- 172.21.2.104:0/1688685608 shutdown_connections 2021-07-11T04:00:50.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.567+0000 7fc1fffec700 1 -- 172.21.2.104:0/1688685608 wait complete. 2021-07-11T04:00:50.575 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:00:50.576 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:50.584 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:50.593 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:50.606 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:00:50.623 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:00:50.623 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:50.624 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:50.624 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:50.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:00:50.630 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T04:00:50.630 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T04:00:50.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:50.631 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:50.632 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:00:50.641 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T04:00:50.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:00:50.642 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T04:00:50.643 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:00:50.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 Processor -- start 2021-07-11T04:00:50.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 -- start start 2021-07-11T04:00:50.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7efd240b1db0 0x7efd240b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7efd240a70c0 0x7efd24144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd240b0350 con 0x7efd241483a0 2021-07-11T04:00:50.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd240a90c0 con 0x7efd240a70c0 2021-07-11T04:00:50.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd240b05f0 con 0x7efd240b1db0 2021-07-11T04:00:50.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd23fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7efd240a70c0 0x7efd24144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:50.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd23fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7efd240a70c0 0x7efd24144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48406/0 (socket says 172.21.2.104:48406) 2021-07-11T04:00:50.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd23fff700 1 -- 172.21.2.104:0/1483604905 learned_addr learned my addr 172.21.2.104:0/1483604905 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:50.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd23fff700 1 -- 172.21.2.104:0/1483604905 >> v2:172.21.2.105:3300/0 conn(0x7efd240b1db0 msgr2=0x7efd240b0fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:50.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd23fff700 1 --2- 172.21.2.104:0/1483604905 >> v2:172.21.2.105:3300/0 conn(0x7efd240b1db0 0x7efd240b0fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:50.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd23fff700 1 -- 172.21.2.104:0/1483604905 >> v1:172.21.2.105:6789/0 conn(0x7efd241483a0 legacy=0x7efd240b2190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:50.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd23fff700 1 -- 172.21.2.104:0/1483604905 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd240b0890 con 0x7efd240a70c0 2021-07-11T04:00:50.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd23fff700 1 --2- 172.21.2.104:0/1483604905 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efd240a70c0 0x7efd24144720 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7efd24147730 tx=0x7efd1c009290).ready entity=mon.0 client_cookie=c205a137f723580c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:50.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd227fc700 1 -- 172.21.2.104:0/1483604905 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd1c00b820 con 0x7efd240a70c0 2021-07-11T04:00:50.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd227fc700 1 -- 172.21.2.104:0/1483604905 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1c00b980 con 0x7efd240a70c0 2021-07-11T04:00:50.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd227fc700 1 -- 172.21.2.104:0/1483604905 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd1c00bb30 con 0x7efd240a70c0 2021-07-11T04:00:50.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 -- 172.21.2.104:0/1483604905 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efd240a70c0 msgr2=0x7efd24144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:50.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 --2- 172.21.2.104:0/1483604905 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efd240a70c0 0x7efd24144720 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7efd24147730 tx=0x7efd1c009290).stop 2021-07-11T04:00:50.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 -- 172.21.2.104:0/1483604905 shutdown_connections 2021-07-11T04:00:50.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 --2- 172.21.2.104:0/1483604905 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efd240a70c0 0x7efd24144720 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:50.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 --2- 172.21.2.104:0/1483604905 >> v2:172.21.2.105:3300/0 conn(0x7efd240b1db0 0x7efd240b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:50.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 -- 172.21.2.104:0/1483604905 >> 172.21.2.104:0/1483604905 conn(0x7efd2405f6b0 msgr2=0x7efd2405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:50.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 -- 172.21.2.104:0/1483604905 shutdown_connections 2021-07-11T04:00:50.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 -- 172.21.2.104:0/1483604905 wait complete. 2021-07-11T04:00:50.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 Processor -- start 2021-07-11T04:00:50.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.699+0000 7efd2a1bf700 1 -- start start 2021-07-11T04:00:50.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd2a1bf700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efd240a70c0 0x7efd240ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd2a1bf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efd240b1db0 0x7efd24145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd2a1bf700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efd241483a0 0x7efd241461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd2a1bf700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd240af280 con 0x7efd240b1db0 2021-07-11T04:00:50.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd2a1bf700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd240af520 con 0x7efd240a70c0 2021-07-11T04:00:50.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd2a1bf700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd240af7c0 con 0x7efd241483a0 2021-07-11T04:00:50.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd22ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efd240b1db0 0x7efd24145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:50.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd22ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efd240b1db0 0x7efd24145cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48412/0 (socket says 172.21.2.104:48412) 2021-07-11T04:00:50.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd22ffd700 1 -- 172.21.2.104:0/4168261729 learned_addr learned my addr 172.21.2.104:0/4168261729 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:50.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd22ffd700 1 -- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efd241483a0 msgr2=0x7efd241461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:50.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd22ffd700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efd241483a0 0x7efd241461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:50.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd22ffd700 1 -- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efd240a70c0 msgr2=0x7efd240ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:50.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd22ffd700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efd240a70c0 0x7efd240ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:50.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd22ffd700 1 -- 172.21.2.104:0/4168261729 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c009530 con 0x7efd240b1db0 2021-07-11T04:00:50.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd22ffd700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efd240b1db0 0x7efd24145cc0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7efd0c000ed0 tx=0x7efd0c006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:50.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd20ff9700 1 -- 172.21.2.104:0/4168261729 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd0c009e00 con 0x7efd240b1db0 2021-07-11T04:00:50.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd20ff9700 1 -- 172.21.2.104:0/4168261729 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0c012410 con 0x7efd240b1db0 2021-07-11T04:00:50.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd20ff9700 1 -- 172.21.2.104:0/4168261729 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efd0c012770 con 0x7efd240b1db0 2021-07-11T04:00:50.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd2a1bf700 1 -- 172.21.2.104:0/4168261729 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd240afa60 con 0x7efd240b1db0 2021-07-11T04:00:50.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd2a1bf700 1 -- 172.21.2.104:0/4168261729 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd240ad8d0 con 0x7efd240b1db0 2021-07-11T04:00:50.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd2a1bf700 1 -- 172.21.2.104:0/4168261729 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd24057350 con 0x7efd240b1db0 2021-07-11T04:00:50.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd20ff9700 1 -- 172.21.2.104:0/4168261729 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7efd0c00f030 con 0x7efd240b1db0 2021-07-11T04:00:50.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd20ff9700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efd08053160 0x7efd08055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:50.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd20ff9700 1 -- 172.21.2.104:0/4168261729 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efd0c06adf0 con 0x7efd240b1db0 2021-07-11T04:00:50.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.703+0000 7efd237fe700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efd08053160 0x7efd08055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:50.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.707+0000 7efd237fe700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efd08053160 0x7efd08055620 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7efd14002fb0 tx=0x7efd140069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:50.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.707+0000 7efd20ff9700 1 -- 172.21.2.104:0/4168261729 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7efd0c042410 con 0x7efd240b1db0 2021-07-11T04:00:50.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.787+0000 7efd2a1bf700 1 -- 172.21.2.104:0/4168261729 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7efd241488e0 con 0x7efd240b1db0 2021-07-11T04:00:50.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.787+0000 7efd20ff9700 1 -- 172.21.2.104:0/4168261729 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v101) v1 ==== 81+0+28825 (secure 0 0 0) 0x7efd0c045a30 con 0x7efd240b1db0 2021-07-11T04:00:50.791 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:50.791 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:00:50.791 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 101, 2021-07-11T04:00:50.791 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:00:50.792 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:00:50.792 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:00:48.392166+0000", 2021-07-11T04:00:50.792 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:50.792 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:00:50.793 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:00:50.793 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:00:50.793 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:00:50.793 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:00:50.793 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:00:50.794 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:00:50.794 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:00:50.794 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:50.794 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:00:50.795 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:00:50.795 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:00:50.795 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:00:50.795 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:00:50.795 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 18, 2021-07-11T04:00:50.796 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:00:50.796 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:00:50.797 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:00:50.797 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:00:50.797 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:00:50.798 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.798 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:00:50.798 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:00:50.798 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:00:50.799 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:50.799 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:50.799 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:50.799 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:50.799 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:50.800 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:50.800 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:50.800 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:50.800 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:00:50.801 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:00:50.801 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:00:50.801 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:00:50.801 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:00:50.801 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:50.802 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:50.802 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:50.802 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:50.802 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:50.803 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:50.803 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:50.803 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.803 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:00:50.803 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:50.804 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:50.804 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:50.804 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:50.804 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:50.804 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:50.805 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:50.805 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:50.805 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:50.805 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:50.806 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:50.806 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:50.806 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:50.809 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:50.810 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:50.810 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:50.810 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:50.810 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:50.811 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:50.811 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:50.811 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:50.811 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:50.811 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:50.812 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:50.812 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:50.812 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:50.813 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.813 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:50.813 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:50.813 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:50.814 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:50.814 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:50.814 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:50.815 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:50.815 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:50.815 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:50.815 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:50.816 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:50.816 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:00:50.816 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:00:50.816 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.816 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:50.817 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:00:50.817 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.817 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.817 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.817 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:00:50.818 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:00:50.818 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:00:50.818 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:00:50.818 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:00:50.819 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:00:50.819 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:00:50.819 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:00:50.819 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:00:50.819 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:00:50.820 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:00:50.820 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:00:50.820 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:00:50.820 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:00:50.821 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:00:50.821 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:00:50.821 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:00:50.821 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:00:50.821 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:00:50.822 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:00:50.822 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:00:50.822 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:00:50.822 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:00:50.823 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.823 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:00:50.823 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:00:50.824 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:00:50.824 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:00:50.824 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:00:50.825 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:00:50.825 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:00:50.825 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:00:50.825 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:00:50.826 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:00:50.826 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:00:50.826 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:00:50.826 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:00:50.827 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:00:50.827 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:00:50.827 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:00:50.827 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:00:50.827 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:00:50.828 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:00:50.828 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:00:50.828 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:00:50.828 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:00:50.829 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:00:50.829 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:00:50.829 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:00:50.829 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:00:50.829 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:00:50.830 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.830 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:00:50.830 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:00:50.831 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:00:50.831 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:00:50.832 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:00:50.832 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:00:50.832 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:00:50.832 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:00:50.833 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:00:50.833 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:00:50.833 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:00:50.834 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:00:50.834 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:00:50.834 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:00:50.834 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.834 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.835 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:50.836 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:00:50.836 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.836 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:50.836 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:00:50.836 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:50.837 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:50.837 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:50.837 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:50.838 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:50.838 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:50.838 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:50.838 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 95, 2021-07-11T04:00:50.839 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:50.839 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:50.839 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:50.839 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.840 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.840 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.840 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:00:50.840 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:50.840 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.841 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.841 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.841 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:00:50.841 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:50.843 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.843 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.843 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.844 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:50.844 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.844 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.844 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.845 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:00:50.845 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:50.845 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.846 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.846 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.846 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:00:50.846 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:50.847 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.847 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.847 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.847 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:50.848 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.848 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.848 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.848 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:00:50.848 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:50.849 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.849 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.849 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.849 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:00:50.849 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:50.850 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.850 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.850 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.850 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:50.851 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.851 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.851 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.851 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:00:50.851 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:50.852 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.852 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.852 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.852 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:00:50.852 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:00:50.853 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.853 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.853 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.853 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:00:50.854 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:00:50.854 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:00:50.854 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:00:50.855 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:50.855 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:50.855 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:50.855 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.857 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.857 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.857 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:50.858 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:00:50.858 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:50.858 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:50.858 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:50.858 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:50.859 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:50.859 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:50.859 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:50.859 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 95, 2021-07-11T04:00:50.860 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:50.860 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:50.860 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:50.860 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.860 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.861 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.861 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:00:50.861 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:50.861 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.861 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.862 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.862 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:00:50.862 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:50.862 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.863 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.863 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.863 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:50.863 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.863 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.864 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.864 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:00:50.865 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:50.865 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.865 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.865 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.865 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:00:50.866 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:50.866 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.866 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.866 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.867 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:50.868 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.869 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.869 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.869 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:00:50.869 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:50.869 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.870 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.870 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.870 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:00:50.870 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:50.871 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.871 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.871 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.871 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:50.871 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.872 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.872 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.872 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:00:50.872 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:50.873 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.873 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.873 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.873 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:00:50.873 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:00:50.874 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.874 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.874 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.875 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:00:50.875 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:00:50.875 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:00:50.875 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:00:50.875 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:50.876 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:50.876 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:50.876 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.876 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.877 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.877 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:50.877 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:00:50.877 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:50.877 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:50.878 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:50.878 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:50.878 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:50.878 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:50.879 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:50.879 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 96, 2021-07-11T04:00:50.879 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:50.879 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:50.879 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:50.880 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.880 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.880 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.880 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:00:50.880 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:50.881 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.881 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.881 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.881 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:00:50.882 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:50.882 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.882 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.882 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.882 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:50.883 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.883 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.883 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.883 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:00:50.884 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:50.884 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.884 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.884 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.884 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:00:50.885 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:50.885 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.885 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.886 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.886 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:50.886 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.886 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.887 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.887 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:00:50.887 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:50.887 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.887 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.888 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.888 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:00:50.888 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:50.888 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.889 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.889 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.889 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:50.889 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.889 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.890 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.890 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:00:50.890 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:50.890 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.890 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.891 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.891 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:00:50.891 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:00:50.891 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.892 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.892 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.892 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:00:50.892 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:00:50.892 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:00:50.893 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:00:50.893 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:50.893 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:50.893 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:50.893 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.894 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.894 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.894 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:50.894 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:00:50.895 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:50.895 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:50.895 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:50.895 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:50.896 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:50.896 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:50.896 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:50.896 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 96, 2021-07-11T04:00:50.897 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:50.897 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:50.897 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:50.897 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.897 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.898 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.898 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:00:50.898 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:50.898 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.899 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.899 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.899 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:00:50.899 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:50.899 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.900 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.900 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.900 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:50.900 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.900 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.901 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.901 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:00:50.901 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:50.901 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.902 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.902 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.902 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:00:50.902 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:50.902 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.903 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.903 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.903 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:50.903 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.904 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.904 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.904 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:00:50.904 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:50.904 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.905 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.905 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.905 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:00:50.906 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:50.906 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.906 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.906 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.907 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:50.907 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.907 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.907 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.907 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:00:50.908 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:50.908 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.908 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.908 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.908 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:00:50.909 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:00:50.909 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.909 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.909 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.910 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:00:50.910 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:00:50.910 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:00:50.910 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:00:50.910 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:50.911 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:50.911 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:50.911 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.911 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.911 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.912 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:50.912 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:00:50.912 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:50.912 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:50.913 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:50.913 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:50.913 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:50.913 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:50.913 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:50.914 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 94, 2021-07-11T04:00:50.914 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:50.914 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:50.914 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:50.914 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.915 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.915 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.915 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:00:50.915 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:50.916 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.916 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.916 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.916 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:00:50.917 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:50.917 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.917 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.917 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.918 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:50.918 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.918 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.918 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.918 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:00:50.919 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:50.919 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.919 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.919 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.920 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:00:50.920 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:50.920 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.920 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.920 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.921 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:50.921 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.921 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.921 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.921 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:00:50.922 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:50.922 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.922 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.922 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.923 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:00:50.923 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:50.923 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.923 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.923 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.924 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:50.924 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.924 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.924 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.924 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:00:50.925 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:50.925 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.925 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.925 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.926 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:00:50.926 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:00:50.926 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.926 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.927 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.927 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:00:50.927 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:00:50.927 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:00:50.928 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:00:50.928 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:50.928 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:50.928 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:50.929 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.929 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.929 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.929 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:50.929 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:00:50.930 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:50.930 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:50.930 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:50.930 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:50.931 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:50.931 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:50.931 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:50.931 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 84, 2021-07-11T04:00:50.931 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:50.932 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:50.932 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:50.932 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.932 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.933 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.933 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:00:50.933 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:50.933 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.933 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.934 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.934 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:00:50.934 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:50.935 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.935 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.935 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.936 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:50.936 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.937 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.937 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.937 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:00:50.937 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:50.937 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.938 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.938 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.938 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:00:50.938 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:50.939 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.939 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.939 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.939 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:50.939 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.940 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.940 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.940 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:00:50.940 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:50.941 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.941 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.941 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.941 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:00:50.941 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:50.942 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.942 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.942 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.942 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:50.943 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.943 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.943 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.943 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:00:50.943 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:50.944 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.944 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.944 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.944 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:00:50.944 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:00:50.945 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.945 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.945 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.945 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:00:50.946 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:00:50.946 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:00:50.946 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:00:50.946 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:50.947 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:50.947 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:50.947 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.947 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.948 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.948 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:50.948 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:00:50.948 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:50.948 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:50.949 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:50.949 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:50.949 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:50.949 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:50.950 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:50.950 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 94, 2021-07-11T04:00:50.950 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:50.950 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:50.950 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:50.951 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.951 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.951 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.951 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:00:50.951 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:50.952 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.952 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.952 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.952 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:00:50.953 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:50.953 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.953 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.953 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.953 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:50.954 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.954 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.954 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.954 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:00:50.955 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:50.955 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.955 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.955 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.955 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:00:50.956 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:50.956 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.956 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.956 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.956 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:50.957 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.957 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.957 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.958 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:00:50.958 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:50.958 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.958 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.959 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.959 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:00:50.959 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:50.959 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.959 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.960 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.960 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:50.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.960 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.961 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.961 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:00:50.961 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:50.961 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.961 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.962 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.962 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:00:50.962 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:00:50.962 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.962 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.963 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.963 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:00:50.963 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:00:50.963 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:00:50.964 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:00:50.964 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:50.964 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:50.964 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:50.964 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.965 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.965 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.965 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:50.965 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:00:50.965 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:00:50.966 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:00:50.966 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:00:50.966 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:00:50.966 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:00:50.966 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:00:50.967 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:00:50.967 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 95, 2021-07-11T04:00:50.967 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:00:50.968 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:00:50.968 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:00:50.968 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.968 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.969 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.969 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:00:50.969 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:50.969 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.969 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.970 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.970 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:00:50.970 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:50.970 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.971 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.971 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.971 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:00:50.971 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.971 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.972 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.972 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:00:50.972 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:50.972 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.972 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.973 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.973 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:00:50.973 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:50.973 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.974 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.974 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.974 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:00:50.974 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.974 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.975 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.975 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:00:50.975 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:50.975 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.975 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.976 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.976 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:00:50.976 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:50.976 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.977 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.977 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.977 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:00:50.977 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:00:50.978 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.978 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:00:50.978 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:00:50.978 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:50.979 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.979 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.979 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:00:50.979 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:00:50.980 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:00:50.980 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.980 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.980 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.980 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:00:50.981 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:00:50.981 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:00:50.981 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:00:50.981 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:00:50.982 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:00:50.982 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:00:50.982 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:50.982 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:50.982 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:50.983 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:00:50.983 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.983 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:00:50.983 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:50.983 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:50.984 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:50.984 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:50.984 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:50.984 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:00:50.985 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:50.985 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.985 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.985 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:00:50.985 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:50.986 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:50.986 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:50.986 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:50.986 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:50.987 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:00:50.987 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:50.987 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.987 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.987 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:00:50.988 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:50.988 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:50.989 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:50.989 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:50.989 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:50.989 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:00:50.990 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:50.990 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.990 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.990 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:00:50.990 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:50.991 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:50.991 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:50.991 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:50.991 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:50.992 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:00:50.992 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:50.992 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.992 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:50.992 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:00:50.993 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:50.993 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:50.993 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:50.993 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:50.993 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:50.994 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:00:50.999 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:50.999 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:50.999 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:51.000 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:00:51.000 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:51.000 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:51.000 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:51.001 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:51.001 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:51.001 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:00:51.001 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:51.001 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:51.002 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:51.002 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:00:51.002 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:51.002 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:51.002 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:51.003 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:51.003 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:51.003 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:00:51.003 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:51.004 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:51.004 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:51.004 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:00:51.004 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:00:51.004 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:00:51.005 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:00:51.005 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:00:51.005 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:00:51.005 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:00:51.005 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:00:51.006 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:51.006 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:00:51.006 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:00:51.006 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:00:51.007 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:00:51.007 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:00:51.007 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T04:00:51.007 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6835/18122340": "2021-07-12T04:00:44.788999+0000", 2021-07-11T04:00:51.007 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6834/18122340": "2021-07-12T04:00:44.788999+0000" 2021-07-11T04:00:51.008 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:51.008 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:00:51.008 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:00:51.008 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:00:51.009 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:00:51.009 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:00:51.009 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:00:51.010 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:00:51.010 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:00:51.010 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:51.010 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:51.010 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:00:51.011 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:00:51.011 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:00:51.011 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:00:51.011 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:00:51.012 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:00:51.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 -- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efd08053160 msgr2=0x7efd08055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efd08053160 0x7efd08055620 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7efd14002fb0 tx=0x7efd140069d0).stop 2021-07-11T04:00:51.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 -- 172.21.2.104:0/4168261729 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efd240b1db0 msgr2=0x7efd24145cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efd240b1db0 0x7efd24145cc0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7efd0c000ed0 tx=0x7efd0c006d60).stop 2021-07-11T04:00:51.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 -- 172.21.2.104:0/4168261729 shutdown_connections 2021-07-11T04:00:51.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efd241483a0 0x7efd241461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:51.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efd08053160 0x7efd08055620 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efd240b1db0 0x7efd24145cc0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 --2- 172.21.2.104:0/4168261729 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efd240a70c0 0x7efd240ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:51.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 -- 172.21.2.104:0/4168261729 >> 172.21.2.104:0/4168261729 conn(0x7efd2405f6b0 msgr2=0x7efd240b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:51.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 -- 172.21.2.104:0/4168261729 shutdown_connections 2021-07-11T04:00:51.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:50.791+0000 7efd2a1bf700 1 -- 172.21.2.104:0/4168261729 wait complete. 2021-07-11T04:00:51.016 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:51.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 Processor -- start 2021-07-11T04:00:51.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 -- start start 2021-07-11T04:00:51.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7effe0060280 0x7effe00668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7effe00b19a0 0x7effe0066da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe00abea0 con 0x7effe00a97a0 2021-07-11T04:00:51.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe00ab310 con 0x7effe00b19a0 2021-07-11T04:00:51.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe0140300 con 0x7effe0060280 2021-07-11T04:00:51.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe54f8700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7effe00a97a0 0x7effe00a9be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45190/0 (socket says 172.21.2.104:45190) 2021-07-11T04:00:51.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe54f8700 1 -- 172.21.2.104:0/1407971216 learned_addr learned my addr 172.21.2.104:0/1407971216 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:51.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effdffff700 1 --2- 172.21.2.104:0/1407971216 >> v2:172.21.2.104:3300/0 conn(0x7effe00b19a0 0x7effe0066da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effdffff700 1 -- 172.21.2.104:0/1407971216 >> v2:172.21.2.105:3300/0 conn(0x7effe0060280 msgr2=0x7effe00668b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:51.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effdffff700 1 --2- 172.21.2.104:0/1407971216 >> v2:172.21.2.105:3300/0 conn(0x7effe0060280 0x7effe00668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:51.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effdffff700 1 -- 172.21.2.104:0/1407971216 >> v1:172.21.2.104:6789/0 conn(0x7effe00a97a0 legacy=0x7effe00a9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effdffff700 1 -- 172.21.2.104:0/1407971216 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe01405a0 con 0x7effe00b19a0 2021-07-11T04:00:51.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effdffff700 1 --2- 172.21.2.104:0/1407971216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effe00b19a0 0x7effe0066da0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7effc8002f70 tx=0x7effc8007f00).ready entity=mon.0 client_cookie=27c0cd02a6a4364c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:51.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effdf7fe700 1 -- 172.21.2.104:0/1407971216 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc8010510 con 0x7effe00b19a0 2021-07-11T04:00:51.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effdf7fe700 1 -- 172.21.2.104:0/1407971216 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc8000e90 con 0x7effe00b19a0 2021-07-11T04:00:51.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effdf7fe700 1 -- 172.21.2.104:0/1407971216 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc8010670 con 0x7effe00b19a0 2021-07-11T04:00:51.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 -- 172.21.2.104:0/1407971216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effe00b19a0 msgr2=0x7effe0066da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 --2- 172.21.2.104:0/1407971216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effe00b19a0 0x7effe0066da0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7effc8002f70 tx=0x7effc8007f00).stop 2021-07-11T04:00:51.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 -- 172.21.2.104:0/1407971216 shutdown_connections 2021-07-11T04:00:51.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 --2- 172.21.2.104:0/1407971216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effe00b19a0 0x7effe0066da0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 --2- 172.21.2.104:0/1407971216 >> v2:172.21.2.105:3300/0 conn(0x7effe0060280 0x7effe00668b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:51.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 -- 172.21.2.104:0/1407971216 >> 172.21.2.104:0/1407971216 conn(0x7effe00b53e0 msgr2=0x7effe00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:51.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 -- 172.21.2.104:0/1407971216 shutdown_connections 2021-07-11T04:00:51.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 -- 172.21.2.104:0/1407971216 wait complete. 2021-07-11T04:00:51.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 Processor -- start 2021-07-11T04:00:51.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.079+0000 7effe6f5b700 1 -- start start 2021-07-11T04:00:51.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effe6f5b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7effe0060280 0x7effe00b46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effe6f5b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effe00a97a0 0x7effe00ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effe6f5b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effe00b19a0 0x7effe00aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effe6f5b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe00641d0 con 0x7effe00a97a0 2021-07-11T04:00:51.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effe6f5b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe0064470 con 0x7effe0060280 2021-07-11T04:00:51.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effe6f5b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe0064710 con 0x7effe00b19a0 2021-07-11T04:00:51.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effdffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effe00a97a0 0x7effe00ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effdffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effe00a97a0 0x7effe00ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48428/0 (socket says 172.21.2.104:48428) 2021-07-11T04:00:51.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effdffff700 1 -- 172.21.2.104:0/1708971955 learned_addr learned my addr 172.21.2.104:0/1708971955 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:51.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effdffff700 1 -- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effe00b19a0 msgr2=0x7effe00aed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effdffff700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effe00b19a0 0x7effe00aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:51.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effdffff700 1 -- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7effe0060280 msgr2=0x7effe00b46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effdffff700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7effe0060280 0x7effe00b46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:51.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effdffff700 1 -- 172.21.2.104:0/1708971955 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc800c200 con 0x7effe00a97a0 2021-07-11T04:00:51.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effdffff700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effe00a97a0 0x7effe00ae890 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7effc8002f70 tx=0x7effc8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:51.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effddffb700 1 -- 172.21.2.104:0/1708971955 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc8010510 con 0x7effe00a97a0 2021-07-11T04:00:51.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effddffb700 1 -- 172.21.2.104:0/1708971955 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effc8000e90 con 0x7effe00a97a0 2021-07-11T04:00:51.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effe6f5b700 1 -- 172.21.2.104:0/1708971955 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe00649b0 con 0x7effe00a97a0 2021-07-11T04:00:51.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effddffb700 1 -- 172.21.2.104:0/1708971955 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7effc8010860 con 0x7effe00a97a0 2021-07-11T04:00:51.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effe6f5b700 1 -- 172.21.2.104:0/1708971955 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe00ae520 con 0x7effe00a97a0 2021-07-11T04:00:51.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effe6f5b700 1 -- 172.21.2.104:0/1708971955 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe0064b40 con 0x7effe00a97a0 2021-07-11T04:00:51.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effddffb700 1 -- 172.21.2.104:0/1708971955 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7effc8010070 con 0x7effe00a97a0 2021-07-11T04:00:51.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effddffb700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effc4053350 0x7effc4055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effddffb700 1 -- 172.21.2.104:0/1708971955 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5335+0+0 (secure 0 0 0) 0x7effc806b5f0 con 0x7effe00a97a0 2021-07-11T04:00:51.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effe4cf7700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effc4053350 0x7effc4055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.083+0000 7effe4cf7700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effc4053350 0x7effc4055810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7effd0002f70 tx=0x7effd0006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:51.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.087+0000 7effddffb700 1 -- 172.21.2.104:0/1708971955 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7effc8042cd0 con 0x7effe00a97a0 2021-07-11T04:00:51.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 -- 172.21.2.104:0/1708971955 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effe0057350 con 0x7effe00a97a0 2021-07-11T04:00:51.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effddffb700 1 -- 172.21.2.104:0/1708971955 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+1553 (secure 0 0 0) 0x7effc8016070 con 0x7effe00a97a0 2021-07-11T04:00:51.193 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:51.193 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/873566337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":873566337},{"type":"v1","addr":"172.21.2.105:6835","nonce":873566337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/930744033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":930744033},{"type":"v1","addr":"172.21.2.104:6833","nonce":930744033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7203,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/232384071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":232384071},{"type":"v1","addr":"172.21.2.104:6835","nonce":232384071}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[]} 2021-07-11T04:00:51.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 -- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effc4053350 msgr2=0x7effc4055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effc4053350 0x7effc4055810 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7effd0002f70 tx=0x7effd0006a80).stop 2021-07-11T04:00:51.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 -- 172.21.2.104:0/1708971955 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effe00a97a0 msgr2=0x7effe00ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effe00a97a0 0x7effe00ae890 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7effc8002f70 tx=0x7effc8017000).stop 2021-07-11T04:00:51.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 -- 172.21.2.104:0/1708971955 shutdown_connections 2021-07-11T04:00:51.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7effe00b19a0 0x7effe00aed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:51.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7effc4053350 0x7effc4055810 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7effe00a97a0 0x7effe00ae890 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 --2- 172.21.2.104:0/1708971955 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7effe0060280 0x7effe00b46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:51.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 -- 172.21.2.104:0/1708971955 >> 172.21.2.104:0/1708971955 conn(0x7effe00b53e0 msgr2=0x7effe00a9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:51.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 -- 172.21.2.104:0/1708971955 shutdown_connections 2021-07-11T04:00:51.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.191+0000 7effe6f5b700 1 -- 172.21.2.104:0/1708971955 wait complete. 2021-07-11T04:00:51.197 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 47 2021-07-11T04:00:51.200 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:51.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.275+0000 7fb71a422700 1 Processor -- start 2021-07-11T04:00:51.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.275+0000 7fb71a422700 1 -- start start 2021-07-11T04:00:51.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.275+0000 7fb71a422700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7140a76b0 0x7fb714066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.275+0000 7fb71a422700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb7140b0060 0x7fb71405fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.275+0000 7fb71a422700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7140a7a90 con 0x7fb7140b1970 2021-07-11T04:00:51.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.275+0000 7fb71a422700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb714147620 con 0x7fb7140b0060 2021-07-11T04:00:51.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.275+0000 7fb71a422700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7141478c0 con 0x7fb7140a76b0 2021-07-11T04:00:51.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb713fff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7140a76b0 0x7fb714066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb7137fe700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fb7140b1970 0x7fb71405f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:44576/0 (socket says 172.21.2.104:44576) 2021-07-11T04:00:51.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb7137fe700 1 -- 172.21.2.104:0/987806600 learned_addr learned my addr 172.21.2.104:0/987806600 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:51.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb7189bf700 1 --2- 172.21.2.104:0/987806600 >> v2:172.21.2.105:3300/0 conn(0x7fb7140b0060 0x7fb71405fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb7189bf700 1 -- 172.21.2.104:0/987806600 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7140a76b0 msgr2=0x7fb714066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:51.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb7189bf700 1 --2- 172.21.2.104:0/987806600 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7140a76b0 0x7fb714066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb7189bf700 1 -- 172.21.2.104:0/987806600 >> v1:172.21.2.105:6789/0 conn(0x7fb7140b1970 legacy=0x7fb71405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb7189bf700 1 -- 172.21.2.104:0/987806600 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb714147b60 con 0x7fb7140b0060 2021-07-11T04:00:51.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb7189bf700 1 --2- 172.21.2.104:0/987806600 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7140b0060 0x7fb71405fdc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fb70c0032f0 tx=0x7fb70c0091d0).ready entity=mon.1 client_cookie=aae2685c40dbfb11 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:51.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb712ffd700 1 -- 172.21.2.104:0/987806600 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb70c00b6c0 con 0x7fb7140b0060 2021-07-11T04:00:51.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb712ffd700 1 -- 172.21.2.104:0/987806600 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70c00b820 con 0x7fb7140b0060 2021-07-11T04:00:51.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb712ffd700 1 -- 172.21.2.104:0/987806600 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb70c00b990 con 0x7fb7140b0060 2021-07-11T04:00:51.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb71a422700 1 -- 172.21.2.104:0/987806600 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7140b0060 msgr2=0x7fb71405fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb71a422700 1 --2- 172.21.2.104:0/987806600 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7140b0060 0x7fb71405fdc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fb70c0032f0 tx=0x7fb70c0091d0).stop 2021-07-11T04:00:51.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb71a422700 1 -- 172.21.2.104:0/987806600 shutdown_connections 2021-07-11T04:00:51.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb71a422700 1 --2- 172.21.2.104:0/987806600 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7140b0060 0x7fb71405fdc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb71a422700 1 --2- 172.21.2.104:0/987806600 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7140a76b0 0x7fb714066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb71a422700 1 -- 172.21.2.104:0/987806600 >> 172.21.2.104:0/987806600 conn(0x7fb7140b53e0 msgr2=0x7fb7140b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:51.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb71a422700 1 -- 172.21.2.104:0/987806600 shutdown_connections 2021-07-11T04:00:51.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.279+0000 7fb71a422700 1 -- 172.21.2.104:0/987806600 wait complete. 2021-07-11T04:00:51.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb71a422700 1 Processor -- start 2021-07-11T04:00:51.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb71a422700 1 -- start start 2021-07-11T04:00:51.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb71a422700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7140a76b0 0x7fb7140ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb71a422700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7140b0060 0x7fb7140ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb71a422700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7140b1970 0x7fb71414cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb71a422700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7141575e0 con 0x7fb7140a76b0 2021-07-11T04:00:51.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb71a422700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb714156960 con 0x7fb7140b1970 2021-07-11T04:00:51.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb71a422700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb714156c00 con 0x7fb7140b0060 2021-07-11T04:00:51.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb713fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7140a76b0 0x7fb7140ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb713fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7140a76b0 0x7fb7140ac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48442/0 (socket says 172.21.2.104:48442) 2021-07-11T04:00:51.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb713fff700 1 -- 172.21.2.104:0/937400498 learned_addr learned my addr 172.21.2.104:0/937400498 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:51.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb7189bf700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7140b1970 0x7fb71414cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb7137fe700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7140b0060 0x7fb7140ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb713fff700 1 -- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7140b0060 msgr2=0x7fb7140ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb713fff700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7140b0060 0x7fb7140ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb713fff700 1 -- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7140b1970 msgr2=0x7fb71414cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb713fff700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7140b1970 0x7fb71414cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb713fff700 1 -- 172.21.2.104:0/937400498 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c009470 con 0x7fb7140a76b0 2021-07-11T04:00:51.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb7137fe700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7140b0060 0x7fb7140ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:51.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb713fff700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7140a76b0 0x7fb7140ac5d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7fb70400c000 tx=0x7fb704006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:51.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb7117fa700 1 -- 172.21.2.104:0/937400498 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb704010780 con 0x7fb7140a76b0 2021-07-11T04:00:51.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb7117fa700 1 -- 172.21.2.104:0/937400498 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7040108e0 con 0x7fb7140a76b0 2021-07-11T04:00:51.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb7117fa700 1 -- 172.21.2.104:0/937400498 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb704010c40 con 0x7fb7140a76b0 2021-07-11T04:00:51.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb71a422700 1 -- 172.21.2.104:0/937400498 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb714156ea0 con 0x7fb7140a76b0 2021-07-11T04:00:51.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb71a422700 1 -- 172.21.2.104:0/937400498 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb714156650 con 0x7fb7140a76b0 2021-07-11T04:00:51.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb71a422700 1 -- 172.21.2.104:0/937400498 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb714057350 con 0x7fb7140a76b0 2021-07-11T04:00:51.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb7117fa700 1 -- 172.21.2.104:0/937400498 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb70401c460 con 0x7fb7140a76b0 2021-07-11T04:00:51.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb7117fa700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6f80532b0 0x7fb6f8055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb7117fa700 1 -- 172.21.2.104:0/937400498 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb70406b690 con 0x7fb7140a76b0 2021-07-11T04:00:51.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.283+0000 7fb7137fe700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6f80532b0 0x7fb6f8055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.287+0000 7fb7137fe700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6f80532b0 0x7fb6f8055770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fb6fc008000 tx=0x7fb6fc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:51.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.287+0000 7fb7117fa700 1 -- 172.21.2.104:0/937400498 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb704042c50 con 0x7fb7140a76b0 2021-07-11T04:00:51.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 -- 172.21.2.104:0/937400498 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb714157340 con 0x7fb7140a76b0 2021-07-11T04:00:51.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb7117fa700 1 -- 172.21.2.104:0/937400498 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+1553 (secure 0 0 0) 0x7fb704046270 con 0x7fb7140a76b0 2021-07-11T04:00:51.394 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:51.395 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5008,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/873566337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":873566337},{"type":"v1","addr":"172.21.2.105:6835","nonce":873566337}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":43},{"gid":6969,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/930744033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":930744033},{"type":"v1","addr":"172.21.2.104:6833","nonce":930744033}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":42},{"gid":7203,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/232384071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":232384071},{"type":"v1","addr":"172.21.2.104:6835","nonce":232384071}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":47}],"filesystems":[]} 2021-07-11T04:00:51.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 -- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6f80532b0 msgr2=0x7fb6f8055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6f80532b0 0x7fb6f8055770 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fb6fc008000 tx=0x7fb6fc009040).stop 2021-07-11T04:00:51.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 -- 172.21.2.104:0/937400498 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7140a76b0 msgr2=0x7fb7140ac5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7140a76b0 0x7fb7140ac5d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7fb70400c000 tx=0x7fb704006df0).stop 2021-07-11T04:00:51.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 -- 172.21.2.104:0/937400498 shutdown_connections 2021-07-11T04:00:51.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7140b1970 0x7fb71414cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7140b0060 0x7fb7140ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6f80532b0 0x7fb6f8055770 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 --2- 172.21.2.104:0/937400498 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7140a76b0 0x7fb7140ac5d0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 -- 172.21.2.104:0/937400498 >> 172.21.2.104:0/937400498 conn(0x7fb7140b53e0 msgr2=0x7fb714146400 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:51.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 -- 172.21.2.104:0/937400498 shutdown_connections 2021-07-11T04:00:51.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.391+0000 7fb71a422700 1 -- 172.21.2.104:0/937400498 wait complete. 2021-07-11T04:00:51.402 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 47 2021-07-11T04:00:51.405 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T04:00:51.406 INFO:tasks.ceph.mds.a:Stopping old one... 2021-07-11T04:00:51.407 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T04:00:51.407 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:00:51.408 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T04:00:51.409 INFO:tasks.ceph.mds.c:Stopping old one... 2021-07-11T04:00:51.409 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T04:00:51.409 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:00:51.409 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T04:00:51.410 INFO:tasks.ceph.mds.b:Stopping old one... 2021-07-11T04:00:51.410 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T04:00:51.410 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:00:51.421 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T04:00:51.422 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T04:00:51.436 INFO:tasks.ceph.mds.b:Started 2021-07-11T04:00:51.437 DEBUG:teuthology.parallel:result is None 2021-07-11T04:00:51.470 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T04:00:51.474 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T04:00:51.474 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T04:00:51.475 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T04:00:51.475 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T04:00:51.495 INFO:tasks.ceph.mds.a:Started 2021-07-11T04:00:51.496 DEBUG:teuthology.parallel:result is None 2021-07-11T04:00:51.501 INFO:tasks.ceph.mds.c:Started 2021-07-11T04:00:51.502 DEBUG:teuthology.parallel:result is None 2021-07-11T04:00:51.503 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T04:00:51.534 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T04:00:51.535 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T04:00:51.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 Processor -- start 2021-07-11T04:00:51.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 -- start start 2021-07-11T04:00:51.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5c180b1970 0x7f5c1805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c180a7a90 con 0x7f5c18067180 2021-07-11T04:00:51.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c18147620 con 0x7f5c180a76b0 2021-07-11T04:00:51.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c181478c0 con 0x7f5c180b1970 2021-07-11T04:00:51.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c167fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f5c18067180 0x7f5c1805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45238/0 (socket says 172.21.2.104:45238) 2021-07-11T04:00:51.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c167fc700 1 -- 172.21.2.104:0/2016883259 learned_addr learned my addr 172.21.2.104:0/2016883259 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:51.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c177fe700 1 --2- 172.21.2.104:0/2016883259 >> v2:172.21.2.104:3300/0 conn(0x7f5c180b1970 0x7f5c1805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c177fe700 1 -- 172.21.2.104:0/2016883259 >> v1:172.21.2.105:6789/0 conn(0x7f5c180a76b0 legacy=0x7f5c18066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:51.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c177fe700 1 -- 172.21.2.104:0/2016883259 >> v1:172.21.2.104:6789/0 conn(0x7f5c18067180 legacy=0x7f5c1805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c177fe700 1 -- 172.21.2.104:0/2016883259 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c18147b60 con 0x7f5c180b1970 2021-07-11T04:00:51.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c177fe700 1 --2- 172.21.2.104:0/2016883259 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c180b1970 0x7f5c1805fdc0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f5c1806a180 tx=0x7f5c100091d0).ready entity=mon.0 client_cookie=e202efa4557deb5f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:51.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c15ffb700 1 -- 172.21.2.104:0/2016883259 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c10009ea0 con 0x7f5c180b1970 2021-07-11T04:00:51.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c15ffb700 1 -- 172.21.2.104:0/2016883259 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1000b830 con 0x7f5c180b1970 2021-07-11T04:00:51.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c15ffb700 1 -- 172.21.2.104:0/2016883259 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c1000b9e0 con 0x7f5c180b1970 2021-07-11T04:00:51.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/2016883259 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c180b1970 msgr2=0x7f5c1805fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:51.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 --2- 172.21.2.104:0/2016883259 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c180b1970 0x7f5c1805fdc0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f5c1806a180 tx=0x7f5c100091d0).stop 2021-07-11T04:00:51.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/2016883259 shutdown_connections 2021-07-11T04:00:51.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 --2- 172.21.2.104:0/2016883259 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c180b1970 0x7f5c1805fdc0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:51.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/2016883259 >> 172.21.2.104:0/2016883259 conn(0x7f5c180b53e0 msgr2=0x7f5c180b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:51.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/2016883259 shutdown_connections 2021-07-11T04:00:51.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.571+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/2016883259 wait complete. 2021-07-11T04:00:51.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c1d7e4700 1 Processor -- start 2021-07-11T04:00:51.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c1d7e4700 1 -- start start 2021-07-11T04:00:51.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c1d7e4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c18067180 0x7f5c180abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c1d7e4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c180a76b0 0x7f5c180aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c1d7e4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c180b1970 0x7f5c180ab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c1d7e4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c180ab7e0 con 0x7f5c18067180 2021-07-11T04:00:51.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c1d7e4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c180ae0e0 con 0x7f5c180b1970 2021-07-11T04:00:51.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c1d7e4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c180a93b0 con 0x7f5c180a76b0 2021-07-11T04:00:51.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c16ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c18067180 0x7f5c180abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c16ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c18067180 0x7f5c180abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48476/0 (socket says 172.21.2.104:48476) 2021-07-11T04:00:51.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c16ffd700 1 -- 172.21.2.104:0/933736862 learned_addr learned my addr 172.21.2.104:0/933736862 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:51.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c16ffd700 1 -- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c180a76b0 msgr2=0x7f5c180aae00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:51.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c16ffd700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c180a76b0 0x7f5c180aae00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:51.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c16ffd700 1 -- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c180b1970 msgr2=0x7f5c180ab2f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:51.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c16ffd700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c180b1970 0x7f5c180ab2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:51.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c16ffd700 1 -- 172.21.2.104:0/933736862 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c10009470 con 0x7f5c18067180 2021-07-11T04:00:51.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c16ffd700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c18067180 0x7f5c180abe40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7f5c08002f90 tx=0x7f5c08006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:51.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c0ffff700 1 -- 172.21.2.104:0/933736862 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c08013460 con 0x7f5c18067180 2021-07-11T04:00:51.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c0ffff700 1 -- 172.21.2.104:0/933736862 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c080135c0 con 0x7f5c18067180 2021-07-11T04:00:51.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c0ffff700 1 -- 172.21.2.104:0/933736862 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c08013920 con 0x7f5c18067180 2021-07-11T04:00:51.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/933736862 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c180a9650 con 0x7f5c18067180 2021-07-11T04:00:51.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/933736862 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c180aebb0 con 0x7f5c18067180 2021-07-11T04:00:51.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/933736862 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c180a97e0 con 0x7f5c18067180 2021-07-11T04:00:51.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c0ffff700 1 -- 172.21.2.104:0/933736862 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5c0801a060 con 0x7f5c18067180 2021-07-11T04:00:51.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c0ffff700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bfc0531a0 0x7f5bfc055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:51.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c0ffff700 1 -- 172.21.2.104:0/933736862 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5c0806ac30 con 0x7f5c18067180 2021-07-11T04:00:51.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c167fc700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bfc0531a0 0x7f5bfc055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:51.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.575+0000 7f5c167fc700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bfc0531a0 0x7f5bfc055660 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f5c1806a0e0 tx=0x7f5c00007790).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:51.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.579+0000 7f5c0ffff700 1 -- 172.21.2.104:0/933736862 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5c08042140 con 0x7f5c18067180 2021-07-11T04:00:51.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:51.671+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/933736862 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7f5c18057350 con 0x7f5c18067180 2021-07-11T04:00:52.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.559+0000 7f5c0ffff700 1 -- 172.21.2.104:0/933736862 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v104) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5c08018070 con 0x7f5c18067180 2021-07-11T04:00:52.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bfc0531a0 msgr2=0x7f5bfc055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bfc0531a0 0x7f5bfc055660 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f5c1806a0e0 tx=0x7f5c00007790).stop 2021-07-11T04:00:52.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/933736862 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c18067180 msgr2=0x7f5c180abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c18067180 0x7f5c180abe40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7f5c08002f90 tx=0x7f5c08006df0).stop 2021-07-11T04:00:52.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/933736862 shutdown_connections 2021-07-11T04:00:52.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c180b1970 0x7f5c180ab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:52.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c180a76b0 0x7f5c180aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:52.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bfc0531a0 0x7f5bfc055660 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:52.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 --2- 172.21.2.104:0/933736862 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c18067180 0x7f5c180abe40 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:52.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/933736862 >> 172.21.2.104:0/933736862 conn(0x7f5c180b53e0 msgr2=0x7f5c1805f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:52.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/933736862 shutdown_connections 2021-07-11T04:00:52.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.563+0000 7f5c1d7e4700 1 -- 172.21.2.104:0/933736862 wait complete. 2021-07-11T04:00:52.567 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T04:00:52.570 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T04:00:52.571 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T04:00:52.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8aedc56700 1 Processor -- start 2021-07-11T04:00:52.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8aedc56700 1 -- start start 2021-07-11T04:00:52.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8aedc56700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8ae80b1db0 0x7f8ae80b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:52.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8aedc56700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae80b0350 con 0x7f8ae80a70c0 2021-07-11T04:00:52.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8aedc56700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae80b05f0 con 0x7f8ae81483a0 2021-07-11T04:00:52.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8aedc56700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae80a90c0 con 0x7f8ae80b1db0 2021-07-11T04:00:52.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8ae7fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8ae80a70c0 0x7f8ae8144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45260/0 (socket says 172.21.2.104:45260) 2021-07-11T04:00:52.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8ae77fe700 1 --2- 172.21.2.104:0/3869793748 >> v2:172.21.2.104:3300/0 conn(0x7f8ae80b1db0 0x7f8ae80b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:52.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8ae7fff700 1 -- 172.21.2.104:0/3869793748 learned_addr learned my addr 172.21.2.104:0/3869793748 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:52.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8ae77fe700 1 -- 172.21.2.104:0/3869793748 >> v1:172.21.2.105:6789/0 conn(0x7f8ae81483a0 legacy=0x7f8ae80b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:52.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8ae77fe700 1 -- 172.21.2.104:0/3869793748 >> v1:172.21.2.104:6789/0 conn(0x7f8ae80a70c0 legacy=0x7f8ae8144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8ae77fe700 1 -- 172.21.2.104:0/3869793748 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80b0890 con 0x7f8ae80b1db0 2021-07-11T04:00:52.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8ae77fe700 1 --2- 172.21.2.104:0/3869793748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ae80b1db0 0x7f8ae80b0fd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f8ae0001490 tx=0x7f8ad0006df0).ready entity=mon.0 client_cookie=9e5dfc7c5f611f7a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:52.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8ae67fc700 1 -- 172.21.2.104:0/3869793748 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ad000d570 con 0x7f8ae80b1db0 2021-07-11T04:00:52.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8ae67fc700 1 -- 172.21.2.104:0/3869793748 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad0000e90 con 0x7f8ae80b1db0 2021-07-11T04:00:52.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8ae67fc700 1 -- 172.21.2.104:0/3869793748 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ad000d8c0 con 0x7f8ae80b1db0 2021-07-11T04:00:52.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8aedc56700 1 -- 172.21.2.104:0/3869793748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ae80b1db0 msgr2=0x7f8ae80b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.699+0000 7f8aedc56700 1 --2- 172.21.2.104:0/3869793748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ae80b1db0 0x7f8ae80b0fd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f8ae0001490 tx=0x7f8ad0006df0).stop 2021-07-11T04:00:52.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 -- 172.21.2.104:0/3869793748 shutdown_connections 2021-07-11T04:00:52.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 --2- 172.21.2.104:0/3869793748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ae80b1db0 0x7f8ae80b0fd0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:52.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 -- 172.21.2.104:0/3869793748 >> 172.21.2.104:0/3869793748 conn(0x7f8ae805f6b0 msgr2=0x7f8ae805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:52.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 -- 172.21.2.104:0/3869793748 shutdown_connections 2021-07-11T04:00:52.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 -- 172.21.2.104:0/3869793748 wait complete. 2021-07-11T04:00:52.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 Processor -- start 2021-07-11T04:00:52.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 -- start start 2021-07-11T04:00:52.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ae80a70c0 0x7f8ae81400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:52.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ae80b1db0 0x7f8ae8145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:52.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ae81483a0 0x7f8ae8145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:52.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8156870 con 0x7f8ae81483a0 2021-07-11T04:00:52.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8146150 con 0x7f8ae80a70c0 2021-07-11T04:00:52.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8156a60 con 0x7f8ae80b1db0 2021-07-11T04:00:52.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ae81483a0 0x7f8ae8145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:52.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ae81483a0 0x7f8ae8145c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48498/0 (socket says 172.21.2.104:48498) 2021-07-11T04:00:52.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae7fff700 1 -- 172.21.2.104:0/179609744 learned_addr learned my addr 172.21.2.104:0/179609744 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:52.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae7fff700 1 -- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ae80b1db0 msgr2=0x7f8ae8145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae7fff700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ae80b1db0 0x7f8ae8145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:52.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae7fff700 1 -- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ae80a70c0 msgr2=0x7f8ae81400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae7fff700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ae80a70c0 0x7f8ae81400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:52.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae7fff700 1 -- 172.21.2.104:0/179609744 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad000b180 con 0x7f8ae81483a0 2021-07-11T04:00:52.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae7fff700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ae81483a0 0x7f8ae8145c60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f8ae0000f40 tx=0x7f8ae0007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:52.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae4ff9700 1 -- 172.21.2.104:0/179609744 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ae0005ba0 con 0x7f8ae81483a0 2021-07-11T04:00:52.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae4ff9700 1 -- 172.21.2.104:0/179609744 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae0005d00 con 0x7f8ae81483a0 2021-07-11T04:00:52.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 -- 172.21.2.104:0/179609744 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae8156d00 con 0x7f8ae81483a0 2021-07-11T04:00:52.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae4ff9700 1 -- 172.21.2.104:0/179609744 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ae0007300 con 0x7f8ae81483a0 2021-07-11T04:00:52.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 -- 172.21.2.104:0/179609744 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae81405b0 con 0x7f8ae81483a0 2021-07-11T04:00:52.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8aedc56700 1 -- 172.21.2.104:0/179609744 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae8057350 con 0x7f8ae81483a0 2021-07-11T04:00:52.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.703+0000 7f8ae4ff9700 1 -- 172.21.2.104:0/179609744 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8ae0010030 con 0x7f8ae81483a0 2021-07-11T04:00:52.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.707+0000 7f8ae4ff9700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8acc053300 0x7f8acc0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:52.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.707+0000 7f8ae4ff9700 1 -- 172.21.2.104:0/179609744 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8ae006b3b0 con 0x7f8ae81483a0 2021-07-11T04:00:52.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.707+0000 7f8ae77fe700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8acc053300 0x7f8acc0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:52.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.707+0000 7f8ae77fe700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8acc053300 0x7f8acc0557c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f8ae0001490 tx=0x7f8ad0014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:52.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.707+0000 7f8ae4ff9700 1 -- 172.21.2.104:0/179609744 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8ae0042980 con 0x7f8ae81483a0 2021-07-11T04:00:52.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.815+0000 7f8aedc56700 1 -- 172.21.2.104:0/179609744 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8ae8044650 con 0x7f8ae81483a0 2021-07-11T04:00:52.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.815+0000 7f8ae4ff9700 1 -- 172.21.2.104:0/179609744 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v26) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f8ae0045fa0 con 0x7f8ae81483a0 2021-07-11T04:00:52.819 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:52.820 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:00:52.820 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T04:00:52.820 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.821 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T04:00:52.821 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T04:00:52.822 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.822 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:52.823 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.823 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.823 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.824 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.824 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.825 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.826 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T04:00:52.826 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T04:00:52.827 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.827 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:52.827 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.828 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.828 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.829 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.830 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.830 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.831 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T04:00:52.831 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T04:00:52.831 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.832 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:52.832 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.833 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.833 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.834 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.834 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.834 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.835 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T04:00:52.836 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T04:00:52.836 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.837 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.837 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.838 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.838 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.838 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.839 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.839 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T04:00:52.840 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T04:00:52.840 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.841 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.841 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.842 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.842 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.843 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.843 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.843 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T04:00:52.844 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T04:00:52.844 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.845 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.845 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.846 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.846 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.847 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.847 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.848 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T04:00:52.848 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T04:00:52.849 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.849 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.849 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.850 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.851 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.851 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.852 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.852 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T04:00:52.853 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T04:00:52.853 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.853 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.854 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.854 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.855 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.855 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.856 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.857 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T04:00:52.857 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T04:00:52.857 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.858 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.858 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.859 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.859 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.860 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.861 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.862 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T04:00:52.862 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T04:00:52.863 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.863 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.863 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.864 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.864 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.865 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.865 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.866 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T04:00:52.866 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T04:00:52.867 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.868 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.868 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.868 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.869 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.869 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.870 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.870 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T04:00:52.871 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T04:00:52.871 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.872 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:52.872 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:00:52.873 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:00:52.873 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.874 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.874 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.874 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T04:00:52.875 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T04:00:52.875 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.875 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:00:52.875 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:00:52.876 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:00:52.876 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.876 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.876 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.877 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T04:00:52.877 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T04:00:52.878 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.878 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:00:52.878 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:00:52.879 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:00:52.879 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.879 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.879 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.879 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.880 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T04:00:52.880 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T04:00:52.880 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.880 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:00:52.881 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:00:52.881 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.881 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.881 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:52.882 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:52.882 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T04:00:52.883 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T04:00:52.883 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:00:52.883 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:00:52.883 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:00:52.883 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:00:52.884 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.884 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:52.884 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:00:52.884 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:00:52.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.815+0000 7f8aedc56700 1 -- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8acc053300 msgr2=0x7f8acc0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.815+0000 7f8aedc56700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8acc053300 0x7f8acc0557c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f8ae0001490 tx=0x7f8ad0014000).stop 2021-07-11T04:00:52.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.815+0000 7f8aedc56700 1 -- 172.21.2.104:0/179609744 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ae81483a0 msgr2=0x7f8ae8145c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.815+0000 7f8aedc56700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ae81483a0 0x7f8ae8145c60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f8ae0000f40 tx=0x7f8ae0007ef0).stop 2021-07-11T04:00:52.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.819+0000 7f8aedc56700 1 -- 172.21.2.104:0/179609744 shutdown_connections 2021-07-11T04:00:52.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.819+0000 7f8aedc56700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8acc053300 0x7f8acc0557c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:52.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.819+0000 7f8aedc56700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8ae81483a0 0x7f8ae8145c60 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:52.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.819+0000 7f8aedc56700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8ae80b1db0 0x7f8ae8145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:52.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.819+0000 7f8aedc56700 1 --2- 172.21.2.104:0/179609744 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8ae80a70c0 0x7f8ae81400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:52.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.819+0000 7f8aedc56700 1 -- 172.21.2.104:0/179609744 >> 172.21.2.104:0/179609744 conn(0x7f8ae805f6b0 msgr2=0x7f8ae80b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:52.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.819+0000 7f8aedc56700 1 -- 172.21.2.104:0/179609744 shutdown_connections 2021-07-11T04:00:52.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.819+0000 7f8aedc56700 1 -- 172.21.2.104:0/179609744 wait complete. 2021-07-11T04:00:52.890 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:52.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd17bc88700 1 Processor -- start 2021-07-11T04:00:52.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd17bc88700 1 -- start start 2021-07-11T04:00:52.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd17bc88700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd1740a76b0 0x7fd174066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:52.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd17bc88700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1740a7a90 con 0x7fd1740b0060 2021-07-11T04:00:52.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd17bc88700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd174147620 con 0x7fd1740b1970 2021-07-11T04:00:52.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd17bc88700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1741478c0 con 0x7fd1740a76b0 2021-07-11T04:00:52.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd17a225700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd1740b0060 0x7fd174151f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45270/0 (socket says 172.21.2.104:45270) 2021-07-11T04:00:52.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd17a225700 1 -- 172.21.2.104:0/1549229055 learned_addr learned my addr 172.21.2.104:0/1549229055 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:52.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd179a24700 1 --2- 172.21.2.104:0/1549229055 >> v2:172.21.2.104:3300/0 conn(0x7fd1740a76b0 0x7fd174066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd179a24700 1 -- 172.21.2.104:0/1549229055 >> v1:172.21.2.105:6789/0 conn(0x7fd1740b1970 legacy=0x7fd17405f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd179a24700 1 -- 172.21.2.104:0/1549229055 >> v1:172.21.2.104:6789/0 conn(0x7fd1740b0060 legacy=0x7fd174151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd179a24700 1 -- 172.21.2.104:0/1549229055 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd174147b60 con 0x7fd1740a76b0 2021-07-11T04:00:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd179a24700 1 --2- 172.21.2.104:0/1549229055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1740a76b0 0x7fd174066a70 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7fd17406a180 tx=0x7fd170006df0).ready entity=mon.0 client_cookie=173865fc927e09f5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd178a22700 1 -- 172.21.2.104:0/1549229055 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd17000d5c0 con 0x7fd1740a76b0 2021-07-11T04:00:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd178a22700 1 -- 172.21.2.104:0/1549229055 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd170000e90 con 0x7fd1740a76b0 2021-07-11T04:00:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd178a22700 1 -- 172.21.2.104:0/1549229055 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd17000d910 con 0x7fd1740a76b0 2021-07-11T04:00:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd17bc88700 1 -- 172.21.2.104:0/1549229055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1740a76b0 msgr2=0x7fd174066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.943+0000 7fd17bc88700 1 --2- 172.21.2.104:0/1549229055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1740a76b0 0x7fd174066a70 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7fd17406a180 tx=0x7fd170006df0).stop 2021-07-11T04:00:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 -- 172.21.2.104:0/1549229055 shutdown_connections 2021-07-11T04:00:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 --2- 172.21.2.104:0/1549229055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1740a76b0 0x7fd174066a70 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 -- 172.21.2.104:0/1549229055 >> 172.21.2.104:0/1549229055 conn(0x7fd1740b53e0 msgr2=0x7fd1740b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:52.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 -- 172.21.2.104:0/1549229055 shutdown_connections 2021-07-11T04:00:52.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 -- 172.21.2.104:0/1549229055 wait complete. 2021-07-11T04:00:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 Processor -- start 2021-07-11T04:00:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 -- start start 2021-07-11T04:00:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd1740a76b0 0x7fd174144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1740b0060 0x7fd174144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1740b1970 0x7fd174159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740af050 con 0x7fd1740b0060 2021-07-11T04:00:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740ad980 con 0x7fd1740a76b0 2021-07-11T04:00:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1740ad350 con 0x7fd1740b1970 2021-07-11T04:00:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd179223700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1740b0060 0x7fd174144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd179223700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1740b0060 0x7fd174144af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48512/0 (socket says 172.21.2.104:48512) 2021-07-11T04:00:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd179223700 1 -- 172.21.2.104:0/1190591013 learned_addr learned my addr 172.21.2.104:0/1190591013 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd179223700 1 -- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1740b1970 msgr2=0x7fd174159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd179223700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1740b1970 0x7fd174159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd179223700 1 -- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd1740a76b0 msgr2=0x7fd174144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd179223700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd1740a76b0 0x7fd174144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:52.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd179223700 1 -- 172.21.2.104:0/1190591013 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17000b180 con 0x7fd1740b0060 2021-07-11T04:00:52.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd179223700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1740b0060 0x7fd174144af0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fd168002ef0 tx=0x7fd168006d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:52.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd166ffd700 1 -- 172.21.2.104:0/1190591013 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd168009e00 con 0x7fd1740b0060 2021-07-11T04:00:52.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd166ffd700 1 -- 172.21.2.104:0/1190591013 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd168002320 con 0x7fd1740b0060 2021-07-11T04:00:52.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 -- 172.21.2.104:0/1190591013 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd174144fe0 con 0x7fd1740b0060 2021-07-11T04:00:52.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd166ffd700 1 -- 172.21.2.104:0/1190591013 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd168002680 con 0x7fd1740b0060 2021-07-11T04:00:52.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 -- 172.21.2.104:0/1190591013 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1740a93d0 con 0x7fd1740b0060 2021-07-11T04:00:52.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd17bc88700 1 -- 172.21.2.104:0/1190591013 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1740af1c0 con 0x7fd1740b0060 2021-07-11T04:00:52.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.947+0000 7fd166ffd700 1 -- 172.21.2.104:0/1190591013 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd16800f030 con 0x7fd1740b0060 2021-07-11T04:00:52.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.951+0000 7fd166ffd700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd154053330 0x7fd1540557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:52.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.951+0000 7fd166ffd700 1 -- 172.21.2.104:0/1190591013 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd16806aea0 con 0x7fd1740b0060 2021-07-11T04:00:52.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.951+0000 7fd179a24700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd154053330 0x7fd1540557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:52.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.951+0000 7fd179a24700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd154053330 0x7fd1540557f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fd17406a180 tx=0x7fd170012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:52.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:52.951+0000 7fd166ffd700 1 -- 172.21.2.104:0/1190591013 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd1680424b0 con 0x7fd1740b0060 2021-07-11T04:00:53.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 -- 172.21.2.104:0/1190591013 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd174057350 con 0x7fd1740b0060 2021-07-11T04:00:53.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd166ffd700 1 -- 172.21.2.104:0/1190591013 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+13183 (secure 0 0 0) 0x7fd168045ad0 con 0x7fd1740b0060 2021-07-11T04:00:53.036 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:53.036 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":104,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:52.553445+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":95,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:53.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 -- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd154053330 msgr2=0x7fd1540557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd154053330 0x7fd1540557f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7fd17406a180 tx=0x7fd170012000).stop 2021-07-11T04:00:53.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 -- 172.21.2.104:0/1190591013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1740b0060 msgr2=0x7fd174144af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1740b0060 0x7fd174144af0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7fd168002ef0 tx=0x7fd168006d40).stop 2021-07-11T04:00:53.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 -- 172.21.2.104:0/1190591013 shutdown_connections 2021-07-11T04:00:53.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1740b1970 0x7fd174159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd154053330 0x7fd1540557f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1740b0060 0x7fd174144af0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 --2- 172.21.2.104:0/1190591013 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd1740a76b0 0x7fd174144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 -- 172.21.2.104:0/1190591013 >> 172.21.2.104:0/1190591013 conn(0x7fd1740b53e0 msgr2=0x7fd174066820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:53.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 -- 172.21.2.104:0/1190591013 shutdown_connections 2021-07-11T04:00:53.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.035+0000 7fd17bc88700 1 -- 172.21.2.104:0/1190591013 wait complete. 2021-07-11T04:00:53.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T04:00:53.045 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:00:53.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.095+0000 7f6705461700 1 Processor -- start 2021-07-11T04:00:53.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.095+0000 7f6705461700 1 -- start start 2021-07-11T04:00:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.095+0000 7f6705461700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f67000b12d0 0x7f67000affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.095+0000 7f6705461700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67000b22a0 0x7f6700141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.095+0000 7f6705461700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67000a77d0 con 0x7f67000b1ec0 2021-07-11T04:00:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.095+0000 7f6705461700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67000b1740 con 0x7f67000b12d0 2021-07-11T04:00:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.095+0000 7f6705461700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67000b0650 con 0x7f67000b22a0 2021-07-11T04:00:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.095+0000 7f66fe7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f67000b12d0 0x7f67000affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.095+0000 7f66fe7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f67000b12d0 0x7f67000affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48518/0 (socket says 172.21.2.104:48518) 2021-07-11T04:00:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.095+0000 7f66fe7fc700 1 -- 172.21.2.104:0/2686194200 learned_addr learned my addr 172.21.2.104:0/2686194200 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66fe7fc700 1 -- 172.21.2.104:0/2686194200 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67000b22a0 msgr2=0x7f6700141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66fe7fc700 1 --2- 172.21.2.104:0/2686194200 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67000b22a0 0x7f6700141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66fe7fc700 1 -- 172.21.2.104:0/2686194200 >> v1:172.21.2.105:6789/0 conn(0x7f67000b1ec0 legacy=0x7f67000a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66fe7fc700 1 -- 172.21.2.104:0/2686194200 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67000b08f0 con 0x7f67000b12d0 2021-07-11T04:00:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66fe7fc700 1 --2- 172.21.2.104:0/2686194200 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67000b12d0 0x7f67000affb0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f67000b2f50 tx=0x7f66e8006eb0).ready entity=mon.0 client_cookie=e9a3ebc26b0a6f84 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66fdffb700 1 -- 172.21.2.104:0/2686194200 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66e800d570 con 0x7f67000b12d0 2021-07-11T04:00:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66fdffb700 1 -- 172.21.2.104:0/2686194200 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e8000e90 con 0x7f67000b12d0 2021-07-11T04:00:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66fdffb700 1 -- 172.21.2.104:0/2686194200 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66e800d8c0 con 0x7f67000b12d0 2021-07-11T04:00:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 -- 172.21.2.104:0/2686194200 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67000b12d0 msgr2=0x7f67000affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 --2- 172.21.2.104:0/2686194200 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67000b12d0 0x7f67000affb0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f67000b2f50 tx=0x7f66e8006eb0).stop 2021-07-11T04:00:53.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 -- 172.21.2.104:0/2686194200 shutdown_connections 2021-07-11T04:00:53.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 --2- 172.21.2.104:0/2686194200 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67000b22a0 0x7f6700141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 --2- 172.21.2.104:0/2686194200 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67000b12d0 0x7f67000affb0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 -- 172.21.2.104:0/2686194200 >> 172.21.2.104:0/2686194200 conn(0x7f670005f6b0 msgr2=0x7f670005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:53.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 -- 172.21.2.104:0/2686194200 shutdown_connections 2021-07-11T04:00:53.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 -- 172.21.2.104:0/2686194200 wait complete. 2021-07-11T04:00:53.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 Processor -- start 2021-07-11T04:00:53.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 -- start start 2021-07-11T04:00:53.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67000b12d0 0x7f670013f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67000b1ec0 0x7f67001495b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67000b22a0 0x7f670013c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000ab930 con 0x7f67000b12d0 2021-07-11T04:00:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000ac0b0 con 0x7f67000b22a0 2021-07-11T04:00:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67001481e0 con 0x7f67000b1ec0 2021-07-11T04:00:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66feffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67000b12d0 0x7f670013f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66feffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67000b12d0 0x7f670013f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48522/0 (socket says 172.21.2.104:48522) 2021-07-11T04:00:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66feffd700 1 -- 172.21.2.104:0/3560036072 learned_addr learned my addr 172.21.2.104:0/3560036072 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66feffd700 1 -- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67000b1ec0 msgr2=0x7f67001495b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66feffd700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67000b1ec0 0x7f67001495b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66feffd700 1 -- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67000b22a0 msgr2=0x7f670013c7b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66feffd700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67000b22a0 0x7f670013c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66feffd700 1 -- 172.21.2.104:0/3560036072 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e800b1e0 con 0x7f67000b12d0 2021-07-11T04:00:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66feffd700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67000b12d0 0x7f670013f040 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f66f0000f70 tx=0x7f66f0007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66f7fff700 1 -- 172.21.2.104:0/3560036072 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66f000ce80 con 0x7f67000b12d0 2021-07-11T04:00:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66f7fff700 1 -- 172.21.2.104:0/3560036072 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0013450 con 0x7f67000b12d0 2021-07-11T04:00:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 -- 172.21.2.104:0/3560036072 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6700148480 con 0x7f67000b12d0 2021-07-11T04:00:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f66f7fff700 1 -- 172.21.2.104:0/3560036072 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66f00137b0 con 0x7f67000b12d0 2021-07-11T04:00:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.099+0000 7f6705461700 1 -- 172.21.2.104:0/3560036072 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67000627c0 con 0x7f67000b12d0 2021-07-11T04:00:53.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.103+0000 7f6705461700 1 -- 172.21.2.104:0/3560036072 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67000ac220 con 0x7f67000b12d0 2021-07-11T04:00:53.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.103+0000 7f66f7fff700 1 -- 172.21.2.104:0/3560036072 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f66f001a060 con 0x7f67000b12d0 2021-07-11T04:00:53.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.103+0000 7f66f7fff700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f66e4053270 0x7f66e4055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.103+0000 7f66f7fff700 1 -- 172.21.2.104:0/3560036072 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f66f006ab20 con 0x7f67000b12d0 2021-07-11T04:00:53.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.103+0000 7f66fe7fc700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f66e4053270 0x7f66e4055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.103+0000 7f66fe7fc700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f66e4053270 0x7f66e4055730 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f67000b2f50 tx=0x7f66e8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.103+0000 7f66f7fff700 1 -- 172.21.2.104:0/3560036072 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f67000ac220 con 0x7f67000b12d0 2021-07-11T04:00:53.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.183+0000 7f6705461700 1 -- 172.21.2.104:0/3560036072 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f67000ac220 con 0x7f67000b12d0 2021-07-11T04:00:53.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.183+0000 7f66f7fff700 1 -- 172.21.2.104:0/3560036072 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v104) v1 ==== 110+0+10 (secure 0 0 0) 0x7f67000ac220 con 0x7f67000b12d0 2021-07-11T04:00:53.186 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:00:53.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 -- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f66e4053270 msgr2=0x7f66e4055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f66e4053270 0x7f66e4055730 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f67000b2f50 tx=0x7f66e8015000).stop 2021-07-11T04:00:53.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 -- 172.21.2.104:0/3560036072 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67000b12d0 msgr2=0x7f670013f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67000b12d0 0x7f670013f040 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f66f0000f70 tx=0x7f66f0007c30).stop 2021-07-11T04:00:53.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 -- 172.21.2.104:0/3560036072 shutdown_connections 2021-07-11T04:00:53.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67000b22a0 0x7f670013c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67000b1ec0 0x7f67001495b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f66e4053270 0x7f66e4055730 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 --2- 172.21.2.104:0/3560036072 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f67000b12d0 0x7f670013f040 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 -- 172.21.2.104:0/3560036072 >> 172.21.2.104:0/3560036072 conn(0x7f670005f6b0 msgr2=0x7f67000a8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:53.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 -- 172.21.2.104:0/3560036072 shutdown_connections 2021-07-11T04:00:53.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.187+0000 7f6705461700 1 -- 172.21.2.104:0/3560036072 wait complete. 2021-07-11T04:00:53.193 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:00:53.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.243+0000 7f60d2bbc700 1 Processor -- start 2021-07-11T04:00:53.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.243+0000 7f60d2bbc700 1 -- start start 2021-07-11T04:00:53.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.243+0000 7f60d2bbc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60cc0b1ec0 0x7f60cc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.243+0000 7f60d2bbc700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f60cc0affb0 0x7f60cc0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.243+0000 7f60d2bbc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f60cc0a70c0 0x7f60cc141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.243+0000 7f60d2bbc700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60cc0b2790 con 0x7f60cc0a70c0 2021-07-11T04:00:53.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.243+0000 7f60d2bbc700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60cc0b03f0 con 0x7f60cc0affb0 2021-07-11T04:00:53.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.243+0000 7f60d2bbc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc0b0650 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f60cc0a70c0 0x7f60cc141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f60cc0a70c0 0x7f60cc141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48530/0 (socket says 172.21.2.104:48530) 2021-07-11T04:00:53.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 -- 172.21.2.104:0/876365502 learned_addr learned my addr 172.21.2.104:0/876365502 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:53.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 -- 172.21.2.104:0/876365502 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60cc0b1ec0 msgr2=0x7f60cc0b12d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:53.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 --2- 172.21.2.104:0/876365502 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60cc0b1ec0 0x7f60cc0b12d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 -- 172.21.2.104:0/876365502 >> v2:172.21.2.105:3300/0 conn(0x7f60cc0affb0 msgr2=0x7f60cc0b22a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:53.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 --2- 172.21.2.104:0/876365502 >> v2:172.21.2.105:3300/0 conn(0x7f60cc0affb0 0x7f60cc0b22a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 -- 172.21.2.104:0/876365502 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc0b08f0 con 0x7f60cc0a70c0 2021-07-11T04:00:53.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 --2- 172.21.2.104:0/876365502 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60cc0a70c0 0x7f60cc141250 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f60c40032f0 tx=0x7f60c4009290).ready entity=mon.0 client_cookie=aa95095c12e184c2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60cb7fe700 1 -- 172.21.2.104:0/876365502 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c400b870 con 0x7f60cc0a70c0 2021-07-11T04:00:53.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60cb7fe700 1 -- 172.21.2.104:0/876365502 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c400b9d0 con 0x7f60cc0a70c0 2021-07-11T04:00:53.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60cb7fe700 1 -- 172.21.2.104:0/876365502 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c400bb80 con 0x7f60cc0a70c0 2021-07-11T04:00:53.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 -- 172.21.2.104:0/876365502 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60cc0a70c0 msgr2=0x7f60cc141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 --2- 172.21.2.104:0/876365502 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60cc0a70c0 0x7f60cc141250 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f60c40032f0 tx=0x7f60c4009290).stop 2021-07-11T04:00:53.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 -- 172.21.2.104:0/876365502 shutdown_connections 2021-07-11T04:00:53.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 --2- 172.21.2.104:0/876365502 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60cc0a70c0 0x7f60cc141250 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 --2- 172.21.2.104:0/876365502 >> v2:172.21.2.105:3300/0 conn(0x7f60cc0affb0 0x7f60cc0b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 --2- 172.21.2.104:0/876365502 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60cc0b1ec0 0x7f60cc0b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 -- 172.21.2.104:0/876365502 >> 172.21.2.104:0/876365502 conn(0x7f60cc05f6b0 msgr2=0x7f60cc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 -- 172.21.2.104:0/876365502 shutdown_connections 2021-07-11T04:00:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 -- 172.21.2.104:0/876365502 wait complete. 2021-07-11T04:00:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 Processor -- start 2021-07-11T04:00:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 -- start start 2021-07-11T04:00:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60cc0a70c0 0x7f60cc15aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f60cc0affb0 0x7f60cc1401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60cc0b1ec0 0x7f60cc156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc145420 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc15a0b0 con 0x7f60cc0affb0 2021-07-11T04:00:53.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60cc15a350 con 0x7f60cc0a70c0 2021-07-11T04:00:53.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60cc0b1ec0 0x7f60cc156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60cc0b1ec0 0x7f60cc156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48540/0 (socket says 172.21.2.104:48540) 2021-07-11T04:00:53.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 -- 172.21.2.104:0/3600155078 learned_addr learned my addr 172.21.2.104:0/3600155078 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:53.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60cbfff700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f60cc0affb0 0x7f60cc1401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d0958700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60cc0a70c0 0x7f60cc15aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 -- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60cc0a70c0 msgr2=0x7f60cc15aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60cc0a70c0 0x7f60cc15aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 -- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f60cc0affb0 msgr2=0x7f60cc1401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f60cc0affb0 0x7f60cc1401d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 -- 172.21.2.104:0/3600155078 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c4009530 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d1159700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60cc0b1ec0 0x7f60cc156a60 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f60c40032f0 tx=0x7f60c4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60c9ffb700 1 -- 172.21.2.104:0/3600155078 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c4011c80 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60c9ffb700 1 -- 172.21.2.104:0/3600155078 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60c4011de0 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 -- 172.21.2.104:0/3600155078 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60cc15a5f0 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60c9ffb700 1 -- 172.21.2.104:0/3600155078 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f60c4020660 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.247+0000 7f60d2bbc700 1 -- 172.21.2.104:0/3600155078 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60cc0aabe0 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.251+0000 7f60d2bbc700 1 -- 172.21.2.104:0/3600155078 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60cc15a780 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.251+0000 7f60c9ffb700 1 -- 172.21.2.104:0/3600155078 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f60c400f070 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.251+0000 7f60c9ffb700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f60b0053160 0x7f60b0055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.251+0000 7f60c9ffb700 1 -- 172.21.2.104:0/3600155078 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f60c406bf00 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.251+0000 7f60d0958700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f60b0053160 0x7f60b0055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.251+0000 7f60d0958700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f60b0053160 0x7f60b0055620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f60b4007000 tx=0x7f60b4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.251+0000 7f60c9ffb700 1 -- 172.21.2.104:0/3600155078 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f60c4069cf0 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.335+0000 7f60d2bbc700 1 -- 172.21.2.104:0/3600155078 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f60cc057350 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.335+0000 7f60c9ffb700 1 -- 172.21.2.104:0/3600155078 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v104) v1 ==== 92+0+10 (secure 0 0 0) 0x7f60c4015030 con 0x7f60cc0b1ec0 2021-07-11T04:00:53.339 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:53.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 -- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f60b0053160 msgr2=0x7f60b0055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f60b0053160 0x7f60b0055620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f60b4007000 tx=0x7f60b4008040).stop 2021-07-11T04:00:53.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 -- 172.21.2.104:0/3600155078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60cc0b1ec0 msgr2=0x7f60cc156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60cc0b1ec0 0x7f60cc156a60 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f60c40032f0 tx=0x7f60c4016000).stop 2021-07-11T04:00:53.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 -- 172.21.2.104:0/3600155078 shutdown_connections 2021-07-11T04:00:53.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f60b0053160 0x7f60b0055620 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f60cc0b1ec0 0x7f60cc156a60 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f60cc0affb0 0x7f60cc1401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 --2- 172.21.2.104:0/3600155078 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f60cc0a70c0 0x7f60cc15aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 -- 172.21.2.104:0/3600155078 >> 172.21.2.104:0/3600155078 conn(0x7f60cc05f6b0 msgr2=0x7f60cc13e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:53.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 -- 172.21.2.104:0/3600155078 shutdown_connections 2021-07-11T04:00:53.342 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.339+0000 7f60d2bbc700 1 -- 172.21.2.104:0/3600155078 wait complete. 2021-07-11T04:00:53.347 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T04:00:53.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 Processor -- start 2021-07-11T04:00:53.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- start start 2021-07-11T04:00:53.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95380aa120 0x7f95380ae440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f95380b0880 0x7f95381405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95380b2af0 con 0x7f95380b04a0 2021-07-11T04:00:53.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95380b2770 con 0x7f95380b0880 2021-07-11T04:00:53.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95380b2d20 con 0x7f95380aa120 2021-07-11T04:00:53.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f9536d9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f95380b0880 0x7f95381405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f9536d9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f95380b0880 0x7f95381405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48548/0 (socket says 172.21.2.104:48548) 2021-07-11T04:00:53.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f9536d9d700 1 -- 172.21.2.104:0/2144057337 learned_addr learned my addr 172.21.2.104:0/2144057337 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:53.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f9536d9d700 1 -- 172.21.2.104:0/2144057337 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95380aa120 msgr2=0x7f95380ae440 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:53.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f9536d9d700 1 --2- 172.21.2.104:0/2144057337 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95380aa120 0x7f95380ae440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f9536d9d700 1 -- 172.21.2.104:0/2144057337 >> v1:172.21.2.105:6789/0 conn(0x7f95380b04a0 legacy=0x7f95380add30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:53.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f9536d9d700 1 -- 172.21.2.104:0/2144057337 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95380a9b70 con 0x7f95380b0880 2021-07-11T04:00:53.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f9536d9d700 1 --2- 172.21.2.104:0/2144057337 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95380b0880 0x7f95381405a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f95380ad3d0 tx=0x7f953000b530).ready entity=mon.0 client_cookie=5100edb463e96d5d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953559a700 1 -- 172.21.2.104:0/2144057337 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f953000be70 con 0x7f95380b0880 2021-07-11T04:00:53.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953559a700 1 -- 172.21.2.104:0/2144057337 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9530010470 con 0x7f95380b0880 2021-07-11T04:00:53.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953559a700 1 -- 172.21.2.104:0/2144057337 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9530010810 con 0x7f95380b0880 2021-07-11T04:00:53.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- 172.21.2.104:0/2144057337 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95380b0880 msgr2=0x7f95381405a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 --2- 172.21.2.104:0/2144057337 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95380b0880 0x7f95381405a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f95380ad3d0 tx=0x7f953000b530).stop 2021-07-11T04:00:53.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- 172.21.2.104:0/2144057337 shutdown_connections 2021-07-11T04:00:53.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 --2- 172.21.2.104:0/2144057337 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95380b0880 0x7f95381405a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 --2- 172.21.2.104:0/2144057337 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95380aa120 0x7f95380ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- 172.21.2.104:0/2144057337 >> 172.21.2.104:0/2144057337 conn(0x7f95380607c0 msgr2=0x7f9538060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:53.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- 172.21.2.104:0/2144057337 shutdown_connections 2021-07-11T04:00:53.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- 172.21.2.104:0/2144057337 wait complete. 2021-07-11T04:00:53.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 Processor -- start 2021-07-11T04:00:53.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- start start 2021-07-11T04:00:53.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95380aa120 0x7f9538140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95380b04a0 0x7f9538066f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95380b0880 0x7f95380673f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9538066110 con 0x7f95380aa120 2021-07-11T04:00:53.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9538064c70 con 0x7f95380b04a0 2021-07-11T04:00:53.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953cd69700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9538065380 con 0x7f95380b0880 2021-07-11T04:00:53.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95380aa120 0x7f9538140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95380aa120 0x7f9538140040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48554/0 (socket says 172.21.2.104:48554) 2021-07-11T04:00:53.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953659c700 1 -- 172.21.2.104:0/3520311281 learned_addr learned my addr 172.21.2.104:0/3520311281 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:53.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953659c700 1 -- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95380b0880 msgr2=0x7f95380673f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:53.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953659c700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95380b0880 0x7f95380673f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953659c700 1 -- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95380b04a0 msgr2=0x7f9538066f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953659c700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95380b04a0 0x7f9538066f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953659c700 1 -- 172.21.2.104:0/3520311281 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953000b7d0 con 0x7f95380aa120 2021-07-11T04:00:53.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f953659c700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95380aa120 0x7f9538140040 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f9520002fb0 tx=0x7f9520006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f952f7fe700 1 -- 172.21.2.104:0/3520311281 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9520013460 con 0x7f95380aa120 2021-07-11T04:00:53.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f952f7fe700 1 -- 172.21.2.104:0/3520311281 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95200135c0 con 0x7f95380aa120 2021-07-11T04:00:53.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.399+0000 7f952f7fe700 1 -- 172.21.2.104:0/3520311281 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9520013920 con 0x7f95380aa120 2021-07-11T04:00:53.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.403+0000 7f953cd69700 1 -- 172.21.2.104:0/3520311281 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9538065620 con 0x7f95380aa120 2021-07-11T04:00:53.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.403+0000 7f953cd69700 1 -- 172.21.2.104:0/3520311281 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9538143590 con 0x7f95380aa120 2021-07-11T04:00:53.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.403+0000 7f953cd69700 1 -- 172.21.2.104:0/3520311281 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95381516b0 con 0x7f95380aa120 2021-07-11T04:00:53.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.403+0000 7f952f7fe700 1 -- 172.21.2.104:0/3520311281 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f952000f070 con 0x7f95380aa120 2021-07-11T04:00:53.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.403+0000 7f952f7fe700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f951c053300 0x7f951c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.403+0000 7f952f7fe700 1 -- 172.21.2.104:0/3520311281 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f952006a8e0 con 0x7f95380aa120 2021-07-11T04:00:53.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.403+0000 7f9535d9b700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f951c053300 0x7f951c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.403+0000 7f9535d9b700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f951c053300 0x7f951c0557c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f9528002fb0 tx=0x7f95280069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.403+0000 7f952f7fe700 1 -- 172.21.2.104:0/3520311281 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9520041f50 con 0x7f95380aa120 2021-07-11T04:00:53.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.487+0000 7f953cd69700 1 -- 172.21.2.104:0/3520311281 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9538066ca0 con 0x7f95380aa120 2021-07-11T04:00:53.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.487+0000 7f952f7fe700 1 -- 172.21.2.104:0/3520311281 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v104) v1 ==== 84+0+143 (secure 0 0 0) 0x7f9520018030 con 0x7f95380aa120 2021-07-11T04:00:53.490 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:53.490 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T04:00:53.491 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:53.491 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T04:00:53.491 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T04:00:53.491 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:00:53.491 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:00:53.492 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T04:00:53.492 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T04:00:53.492 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:00:53.492 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T04:00:53.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 -- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f951c053300 msgr2=0x7f951c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f951c053300 0x7f951c0557c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f9528002fb0 tx=0x7f95280069b0).stop 2021-07-11T04:00:53.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 -- 172.21.2.104:0/3520311281 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95380aa120 msgr2=0x7f9538140040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95380aa120 0x7f9538140040 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f9520002fb0 tx=0x7f9520006e10).stop 2021-07-11T04:00:53.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 -- 172.21.2.104:0/3520311281 shutdown_connections 2021-07-11T04:00:53.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95380b0880 0x7f95380673f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95380b04a0 0x7f9538066f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f951c053300 0x7f951c0557c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 --2- 172.21.2.104:0/3520311281 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95380aa120 0x7f9538140040 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 -- 172.21.2.104:0/3520311281 >> 172.21.2.104:0/3520311281 conn(0x7f95380607c0 msgr2=0x7f95380b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:53.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 -- 172.21.2.104:0/3520311281 shutdown_connections 2021-07-11T04:00:53.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.491+0000 7f953cd69700 1 -- 172.21.2.104:0/3520311281 wait complete. 2021-07-11T04:00:53.498 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T04:00:53.498 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T04:00:53.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 Processor -- start 2021-07-11T04:00:53.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 -- start start 2021-07-11T04:00:53.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940603d0 0x7fea940aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fea9413f790 0x7fea9413fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea94061990 con 0x7fea940a9ec0 2021-07-11T04:00:53.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea94140110 con 0x7fea9413f790 2021-07-11T04:00:53.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea94061c30 con 0x7fea940603d0 2021-07-11T04:00:53.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea93fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fea940a9ec0 0x7fea9413ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45326/0 (socket says 172.21.2.104:45326) 2021-07-11T04:00:53.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea93fff700 1 -- 172.21.2.104:0/534897706 learned_addr learned my addr 172.21.2.104:0/534897706 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:53.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868688733 0 0) 0x7fea94061990 con 0x7fea940a9ec0 2021-07-11T04:00:53.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea74003260 con 0x7fea940a9ec0 2021-07-11T04:00:53.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9886d700 1 --2- 172.21.2.104:0/534897706 >> v2:172.21.2.105:3300/0 conn(0x7fea9413f790 0x7fea9413fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1676189001 0 0) 0x7fea74003260 con 0x7fea940a9ec0 2021-07-11T04:00:53.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940603d0 msgr2=0x7fea940aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:53.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 --2- 172.21.2.104:0/534897706 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940603d0 0x7fea940aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 >> v2:172.21.2.105:3300/0 conn(0x7fea9413f790 msgr2=0x7fea9413fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:53.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 --2- 172.21.2.104:0/534897706 >> v2:172.21.2.105:3300/0 conn(0x7fea9413f790 0x7fea9413fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94060b90 con 0x7fea940a9ec0 2021-07-11T04:00:53.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fea7c001660 con 0x7fea940a9ec0 2021-07-11T04:00:53.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 >> v1:172.21.2.104:6789/0 conn(0x7fea940a9ec0 legacy=0x7fea9413ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 --2- 172.21.2.104:0/534897706 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940603d0 0x7fea740037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 --2- 172.21.2.104:0/534897706 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea74003e90 0x7fea74004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 --2- 172.21.2.104:0/534897706 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea74004b10 0x7fea74004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea94140110 con 0x7fea74003e90 2021-07-11T04:00:53.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea94061c30 con 0x7fea74004b10 2021-07-11T04:00:53.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7400ba60 con 0x7fea940603d0 2021-07-11T04:00:53.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea937fe700 1 --2- 172.21.2.104:0/534897706 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea74003e90 0x7fea74004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea93fff700 1 --2- 172.21.2.104:0/534897706 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940603d0 0x7fea740037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9886d700 1 --2- 172.21.2.104:0/534897706 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea74004b10 0x7fea74004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea937fe700 1 -- 172.21.2.104:0/534897706 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940603d0 msgr2=0x7fea740037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:53.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea937fe700 1 --2- 172.21.2.104:0/534897706 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940603d0 0x7fea740037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea937fe700 1 -- 172.21.2.104:0/534897706 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea74004b10 msgr2=0x7fea74004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:53.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea937fe700 1 --2- 172.21.2.104:0/534897706 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea74004b10 0x7fea74004f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:53.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea937fe700 1 -- 172.21.2.104:0/534897706 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea94060b90 con 0x7fea74003e90 2021-07-11T04:00:53.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea937fe700 1 --2- 172.21.2.104:0/534897706 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea74003e90 0x7fea74004500 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7fea84002f70 tx=0x7fea84006ca0).ready entity=mon.0 client_cookie=9d210edc562b8df5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea84013410 con 0x7fea74003e90 2021-07-11T04:00:53.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea84013570 con 0x7fea74003e90 2021-07-11T04:00:53.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea92ffd700 1 -- 172.21.2.104:0/534897706 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea840138d0 con 0x7fea74003e90 2021-07-11T04:00:53.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 -- 172.21.2.104:0/534897706 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea74003e90 msgr2=0x7fea74004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 --2- 172.21.2.104:0/534897706 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea74003e90 0x7fea74004500 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7fea84002f70 tx=0x7fea84006ca0).stop 2021-07-11T04:00:53.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9886d700 1 -- 172.21.2.104:0/534897706 reap_dead start 2021-07-11T04:00:53.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 -- 172.21.2.104:0/534897706 shutdown_connections 2021-07-11T04:00:53.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 -- 172.21.2.104:0/534897706 >> 172.21.2.104:0/534897706 conn(0x7fea940b2970 msgr2=0x7fea940b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:53.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 -- 172.21.2.104:0/534897706 shutdown_connections 2021-07-11T04:00:53.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.551+0000 7fea9a2d0700 1 -- 172.21.2.104:0/534897706 wait complete. 2021-07-11T04:00:53.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea9a2d0700 1 Processor -- start 2021-07-11T04:00:53.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea9a2d0700 1 -- start start 2021-07-11T04:00:53.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea9a2d0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940b5020 0x7fea94156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea9a2d0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940a95a0 0x7fea94157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea9a2d0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea9415ab70 0x7fea940a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea9a2d0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea940ae1c0 con 0x7fea940b5020 2021-07-11T04:00:53.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea9a2d0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea940b5400 con 0x7fea9415ab70 2021-07-11T04:00:53.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea9a2d0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea940ad7e0 con 0x7fea940a95a0 2021-07-11T04:00:53.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea93fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940b5020 0x7fea94156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea93fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940b5020 0x7fea94156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48570/0 (socket says 172.21.2.104:48570) 2021-07-11T04:00:53.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea93fff700 1 -- 172.21.2.104:0/2516540778 learned_addr learned my addr 172.21.2.104:0/2516540778 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:53.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea93fff700 1 -- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940a95a0 msgr2=0x7fea94157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea93fff700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940a95a0 0x7fea94157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea93fff700 1 -- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea9415ab70 msgr2=0x7fea940a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:53.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea93fff700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea9415ab70 0x7fea940a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:53.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea93fff700 1 -- 172.21.2.104:0/2516540778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea940ada80 con 0x7fea940b5020 2021-07-11T04:00:53.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea93fff700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940b5020 0x7fea94156b90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7fea7c00b000 tx=0x7fea7c001430).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea917fa700 1 -- 172.21.2.104:0/2516540778 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea7c010710 con 0x7fea940b5020 2021-07-11T04:00:53.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea917fa700 1 -- 172.21.2.104:0/2516540778 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7c010870 con 0x7fea940b5020 2021-07-11T04:00:53.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea917fa700 1 -- 172.21.2.104:0/2516540778 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fea7c010bd0 con 0x7fea940b5020 2021-07-11T04:00:53.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea9a2d0700 1 -- 172.21.2.104:0/2516540778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea940add20 con 0x7fea940b5020 2021-07-11T04:00:53.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea9a2d0700 1 -- 172.21.2.104:0/2516540778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea94157570 con 0x7fea940b5020 2021-07-11T04:00:53.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea9a2d0700 1 -- 172.21.2.104:0/2516540778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea94057350 con 0x7fea940b5020 2021-07-11T04:00:53.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea917fa700 1 -- 172.21.2.104:0/2516540778 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fea7c010d30 con 0x7fea940b5020 2021-07-11T04:00:53.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea917fa700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea78053210 0x7fea780556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:53.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea917fa700 1 -- 172.21.2.104:0/2516540778 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fea7c06b470 con 0x7fea940b5020 2021-07-11T04:00:53.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.555+0000 7fea937fe700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea78053210 0x7fea780556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:53.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.559+0000 7fea937fe700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea78053210 0x7fea780556d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7fea84002f70 tx=0x7fea84014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:53.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.559+0000 7fea917fa700 1 -- 172.21.2.104:0/2516540778 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fea7c042ae0 con 0x7fea940b5020 2021-07-11T04:00:53.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:53.639+0000 7fea9a2d0700 1 -- 172.21.2.104:0/2516540778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fea940adeb0 con 0x7fea940b5020 2021-07-11T04:00:54.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.571+0000 7fea917fa700 1 -- 172.21.2.104:0/2516540778 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v105) v1 ==== 133+0+0 (secure 0 0 0) 0x7fea7c046100 con 0x7fea940b5020 2021-07-11T04:00:54.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 -- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea78053210 msgr2=0x7fea780556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:54.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea78053210 0x7fea780556d0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7fea84002f70 tx=0x7fea84014000).stop 2021-07-11T04:00:54.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 -- 172.21.2.104:0/2516540778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940b5020 msgr2=0x7fea94156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:54.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940b5020 0x7fea94156b90 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7fea7c00b000 tx=0x7fea7c001430).stop 2021-07-11T04:00:54.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 -- 172.21.2.104:0/2516540778 shutdown_connections 2021-07-11T04:00:54.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fea9415ab70 0x7fea940a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:54.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fea940a95a0 0x7fea94157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:54.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fea78053210 0x7fea780556d0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:54.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 --2- 172.21.2.104:0/2516540778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fea940b5020 0x7fea94156b90 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:54.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 -- 172.21.2.104:0/2516540778 >> 172.21.2.104:0/2516540778 conn(0x7fea940b2970 msgr2=0x7fea940b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:54.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 -- 172.21.2.104:0/2516540778 shutdown_connections 2021-07-11T04:00:54.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.575+0000 7fea9a2d0700 1 -- 172.21.2.104:0/2516540778 wait complete. 2021-07-11T04:00:54.579 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T04:00:54.582 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T04:00:54.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.643+0000 7fd00d6ee700 1 Processor -- start 2021-07-11T04:00:54.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.643+0000 7fd00d6ee700 1 -- start start 2021-07-11T04:00:54.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.643+0000 7fd00d6ee700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd0080b12d0 0x7fd0080affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:54.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.643+0000 7fd00d6ee700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd0080b22a0 0x7fd008141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:54.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.643+0000 7fd00d6ee700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0080a77d0 con 0x7fd0080b1ec0 2021-07-11T04:00:54.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.643+0000 7fd00d6ee700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0080b1740 con 0x7fd0080b12d0 2021-07-11T04:00:54.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.643+0000 7fd00d6ee700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0080b0650 con 0x7fd0080b22a0 2021-07-11T04:00:54.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd0067fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd0080b12d0 0x7fd0080affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:54.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd0067fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd0080b12d0 0x7fd0080affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48582/0 (socket says 172.21.2.104:48582) 2021-07-11T04:00:54.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd0067fc700 1 -- 172.21.2.104:0/213384635 learned_addr learned my addr 172.21.2.104:0/213384635 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:54.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd0067fc700 1 -- 172.21.2.104:0/213384635 >> v2:172.21.2.105:3300/0 conn(0x7fd0080b22a0 msgr2=0x7fd008141250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:54.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd0067fc700 1 --2- 172.21.2.104:0/213384635 >> v2:172.21.2.105:3300/0 conn(0x7fd0080b22a0 0x7fd008141250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:54.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd0067fc700 1 -- 172.21.2.104:0/213384635 >> v1:172.21.2.104:6789/0 conn(0x7fd0080b1ec0 legacy=0x7fd0080a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:54.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd0067fc700 1 -- 172.21.2.104:0/213384635 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0080b08f0 con 0x7fd0080b12d0 2021-07-11T04:00:54.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd0067fc700 1 --2- 172.21.2.104:0/213384635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0080b12d0 0x7fd0080affb0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7fcff8002f70 tx=0x7fcff8006eb0).ready entity=mon.0 client_cookie=3d6d94efd4e49faf server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:54.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd005ffb700 1 -- 172.21.2.104:0/213384635 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcff800d610 con 0x7fd0080b12d0 2021-07-11T04:00:54.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd005ffb700 1 -- 172.21.2.104:0/213384635 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff8000e90 con 0x7fd0080b12d0 2021-07-11T04:00:54.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd005ffb700 1 -- 172.21.2.104:0/213384635 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcff800d9a0 con 0x7fd0080b12d0 2021-07-11T04:00:54.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- 172.21.2.104:0/213384635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0080b12d0 msgr2=0x7fd0080affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:54.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 --2- 172.21.2.104:0/213384635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0080b12d0 0x7fd0080affb0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7fcff8002f70 tx=0x7fcff8006eb0).stop 2021-07-11T04:00:54.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- 172.21.2.104:0/213384635 shutdown_connections 2021-07-11T04:00:54.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 --2- 172.21.2.104:0/213384635 >> v2:172.21.2.105:3300/0 conn(0x7fd0080b22a0 0x7fd008141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:54.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 --2- 172.21.2.104:0/213384635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0080b12d0 0x7fd0080affb0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:54.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- 172.21.2.104:0/213384635 >> 172.21.2.104:0/213384635 conn(0x7fd00805f6b0 msgr2=0x7fd00805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:54.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- 172.21.2.104:0/213384635 shutdown_connections 2021-07-11T04:00:54.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- 172.21.2.104:0/213384635 wait complete. 2021-07-11T04:00:54.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 Processor -- start 2021-07-11T04:00:54.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- start start 2021-07-11T04:00:54.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0080b12d0 0x7fd0080aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:54.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0080b1ec0 0x7fd00814a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:54.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0080b22a0 0x7fd00814b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:54.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0080aa9e0 con 0x7fd0080b12d0 2021-07-11T04:00:54.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd008062940 con 0x7fd0080b1ec0 2021-07-11T04:00:54.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd008149a40 con 0x7fd0080b22a0 2021-07-11T04:00:54.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd006ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0080b12d0 0x7fd0080aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:54.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd006ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0080b12d0 0x7fd0080aa4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48586/0 (socket says 172.21.2.104:48586) 2021-07-11T04:00:54.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd006ffd700 1 -- 172.21.2.104:0/3369371823 learned_addr learned my addr 172.21.2.104:0/3369371823 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:54.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd006ffd700 1 -- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0080b22a0 msgr2=0x7fd00814b290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:54.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd006ffd700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0080b22a0 0x7fd00814b290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:54.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd006ffd700 1 -- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0080b1ec0 msgr2=0x7fd00814a1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:54.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd006ffd700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0080b1ec0 0x7fd00814a1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:54.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd006ffd700 1 -- 172.21.2.104:0/3369371823 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff800b1e0 con 0x7fd0080b12d0 2021-07-11T04:00:54.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd006ffd700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0080b12d0 0x7fd0080aa4f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7fd0080ad2d0 tx=0x7fcff0007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:54.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fcffffff700 1 -- 172.21.2.104:0/3369371823 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcff000ce30 con 0x7fd0080b12d0 2021-07-11T04:00:54.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fcffffff700 1 -- 172.21.2.104:0/3369371823 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff000f470 con 0x7fd0080b12d0 2021-07-11T04:00:54.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fcffffff700 1 -- 172.21.2.104:0/3369371823 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcff000f7d0 con 0x7fd0080b12d0 2021-07-11T04:00:54.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- 172.21.2.104:0/3369371823 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00814a6c0 con 0x7fd0080b12d0 2021-07-11T04:00:54.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- 172.21.2.104:0/3369371823 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00814b780 con 0x7fd0080b12d0 2021-07-11T04:00:54.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.647+0000 7fd00d6ee700 1 -- 172.21.2.104:0/3369371823 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00814ba30 con 0x7fd0080b12d0 2021-07-11T04:00:54.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.651+0000 7fcffffff700 1 -- 172.21.2.104:0/3369371823 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcff0019030 con 0x7fd0080b12d0 2021-07-11T04:00:54.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.651+0000 7fcffffff700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcfec053330 0x7fcfec0557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:54.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.651+0000 7fcffffff700 1 -- 172.21.2.104:0/3369371823 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5603+0+0 (secure 0 0 0) 0x7fcff006acf0 con 0x7fd0080b12d0 2021-07-11T04:00:54.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.651+0000 7fd0067fc700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcfec053330 0x7fcfec0557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:54.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.651+0000 7fd0067fc700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcfec053330 0x7fcfec0557f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fcff8002f70 tx=0x7fcff8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:54.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.651+0000 7fcffffff700 1 -- 172.21.2.104:0/3369371823 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcff00420f0 con 0x7fd0080b12d0 2021-07-11T04:00:54.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:54.735+0000 7fd00d6ee700 1 -- 172.21.2.104:0/3369371823 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fd0081486c0 con 0x7fd0080b12d0 2021-07-11T04:00:55.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fcffffff700 1 -- 172.21.2.104:0/3369371823 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v106) v1 ==== 125+0+0 (secure 0 0 0) 0x7fcff0045710 con 0x7fd0080b12d0 2021-07-11T04:00:55.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 -- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcfec053330 msgr2=0x7fcfec0557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcfec053330 0x7fcfec0557f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fcff8002f70 tx=0x7fcff8015000).stop 2021-07-11T04:00:55.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 -- 172.21.2.104:0/3369371823 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0080b12d0 msgr2=0x7fd0080aa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0080b12d0 0x7fd0080aa4f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7fd0080ad2d0 tx=0x7fcff0007c50).stop 2021-07-11T04:00:55.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 -- 172.21.2.104:0/3369371823 shutdown_connections 2021-07-11T04:00:55.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd0080b22a0 0x7fd00814b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:55.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd0080b1ec0 0x7fd00814a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:55.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcfec053330 0x7fcfec0557f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 --2- 172.21.2.104:0/3369371823 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd0080b12d0 0x7fd0080aa4f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 -- 172.21.2.104:0/3369371823 >> 172.21.2.104:0/3369371823 conn(0x7fd00805f6b0 msgr2=0x7fd0080af540 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:55.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 -- 172.21.2.104:0/3369371823 shutdown_connections 2021-07-11T04:00:55.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.479+0000 7fd00d6ee700 1 -- 172.21.2.104:0/3369371823 wait complete. 2021-07-11T04:00:55.485 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T04:00:55.490 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T04:00:55.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.543+0000 7f029e616700 1 Processor -- start 2021-07-11T04:00:55.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.543+0000 7f029e616700 1 -- start start 2021-07-11T04:00:55.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.543+0000 7f029e616700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0298067180 0x7f029805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.543+0000 7f029e616700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f02980b1970 0x7f029805fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.543+0000 7f029e616700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02980a7a90 con 0x7f02980a76b0 2021-07-11T04:00:55.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.543+0000 7f029e616700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0298147620 con 0x7f02980b1970 2021-07-11T04:00:55.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.543+0000 7f029e616700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02981478c0 con 0x7f0298067180 2021-07-11T04:00:55.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.543+0000 7f02977fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0298067180 0x7f029805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029cbb3700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f02980b1970 0x7f029805fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f02977fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0298067180 0x7f029805f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:51760/0 (socket says 172.21.2.104:51760) 2021-07-11T04:00:55.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f02977fe700 1 -- 172.21.2.104:0/344586545 learned_addr learned my addr 172.21.2.104:0/344586545 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:55.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f0297fff700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f02980a76b0 0x7f0298066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:44730/0 (socket says 172.21.2.104:44730) 2021-07-11T04:00:55.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029cbb3700 1 -- 172.21.2.104:0/344586545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0298067180 msgr2=0x7f029805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:55.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029cbb3700 1 --2- 172.21.2.104:0/344586545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0298067180 0x7f029805f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029cbb3700 1 -- 172.21.2.104:0/344586545 >> v1:172.21.2.105:6789/0 conn(0x7f02980a76b0 legacy=0x7f0298066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029cbb3700 1 -- 172.21.2.104:0/344586545 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0298147b60 con 0x7f02980b1970 2021-07-11T04:00:55.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029cbb3700 1 --2- 172.21.2.104:0/344586545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02980b1970 0x7f029805fba0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f02900032f0 tx=0x7f02900091d0).ready entity=mon.1 client_cookie=a094115bafb188da server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:55.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f0296ffd700 1 -- 172.21.2.104:0/344586545 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f029000b6c0 con 0x7f02980b1970 2021-07-11T04:00:55.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f0296ffd700 1 -- 172.21.2.104:0/344586545 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029000b820 con 0x7f02980b1970 2021-07-11T04:00:55.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f0296ffd700 1 -- 172.21.2.104:0/344586545 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f029000b990 con 0x7f02980b1970 2021-07-11T04:00:55.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029e616700 1 -- 172.21.2.104:0/344586545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02980b1970 msgr2=0x7f029805fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029e616700 1 --2- 172.21.2.104:0/344586545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02980b1970 0x7f029805fba0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f02900032f0 tx=0x7f02900091d0).stop 2021-07-11T04:00:55.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029e616700 1 -- 172.21.2.104:0/344586545 shutdown_connections 2021-07-11T04:00:55.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029e616700 1 --2- 172.21.2.104:0/344586545 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02980b1970 0x7f029805fba0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029e616700 1 --2- 172.21.2.104:0/344586545 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0298067180 0x7f029805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029e616700 1 -- 172.21.2.104:0/344586545 >> 172.21.2.104:0/344586545 conn(0x7f02980b53e0 msgr2=0x7f02980b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:55.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029e616700 1 -- 172.21.2.104:0/344586545 shutdown_connections 2021-07-11T04:00:55.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029e616700 1 -- 172.21.2.104:0/344586545 wait complete. 2021-07-11T04:00:55.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029e616700 1 Processor -- start 2021-07-11T04:00:55.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.547+0000 7f029e616700 1 -- start start 2021-07-11T04:00:55.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f029e616700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0298067180 0x7f02980b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f029e616700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02980a76b0 0x7f02980ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f029e616700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02980b1970 0x7f02980ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f029e616700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02980aaae0 con 0x7f0298067180 2021-07-11T04:00:55.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f029e616700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02980b5170 con 0x7f02980a76b0 2021-07-11T04:00:55.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f029e616700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02980ac520 con 0x7f02980b1970 2021-07-11T04:00:55.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f0297fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0298067180 0x7f02980b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f0297fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0298067180 0x7f02980b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48598/0 (socket says 172.21.2.104:48598) 2021-07-11T04:00:55.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f0297fff700 1 -- 172.21.2.104:0/3871365625 learned_addr learned my addr 172.21.2.104:0/3871365625 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:55.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f02977fe700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02980a76b0 0x7f02980ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f029cbb3700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02980b1970 0x7f02980ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f0297fff700 1 -- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02980b1970 msgr2=0x7f02980ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f0297fff700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02980b1970 0x7f02980ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f0297fff700 1 -- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02980a76b0 msgr2=0x7f02980ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f0297fff700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02980a76b0 0x7f02980ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f0297fff700 1 -- 172.21.2.104:0/3871365625 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0290009470 con 0x7f0298067180 2021-07-11T04:00:55.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f0297fff700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0298067180 0x7f02980b38d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7f0298148420 tx=0x7f0280007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:55.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f02957fa700 1 -- 172.21.2.104:0/3871365625 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f028000ce30 con 0x7f0298067180 2021-07-11T04:00:55.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f02957fa700 1 -- 172.21.2.104:0/3871365625 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0280007170 con 0x7f0298067180 2021-07-11T04:00:55.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f02957fa700 1 -- 172.21.2.104:0/3871365625 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0280004730 con 0x7f0298067180 2021-07-11T04:00:55.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f029e616700 1 -- 172.21.2.104:0/3871365625 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02980aad10 con 0x7f0298067180 2021-07-11T04:00:55.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f029e616700 1 -- 172.21.2.104:0/3871365625 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02980ac7c0 con 0x7f0298067180 2021-07-11T04:00:55.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f029e616700 1 -- 172.21.2.104:0/3871365625 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02980aeb10 con 0x7f0298067180 2021-07-11T04:00:55.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f02957fa700 1 -- 172.21.2.104:0/3871365625 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0280004890 con 0x7f0298067180 2021-07-11T04:00:55.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f02957fa700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f027c0532b0 0x7f027c055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f02957fa700 1 -- 172.21.2.104:0/3871365625 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f028006ae10 con 0x7f0298067180 2021-07-11T04:00:55.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.551+0000 7f02977fe700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f027c0532b0 0x7f027c055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.555+0000 7f02977fe700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f027c0532b0 0x7f027c055770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f0288008000 tx=0x7f0288008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:55.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.555+0000 7f02957fa700 1 -- 172.21.2.104:0/3871365625 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f02800420b0 con 0x7f0298067180 2021-07-11T04:00:55.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.663+0000 7f029e616700 1 -- 172.21.2.104:0/3871365625 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f0298057350 con 0x7f0298067180 2021-07-11T04:00:55.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f02957fa700 1 -- 172.21.2.104:0/3871365625 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 19 and data pool 20 v49) v1 ==== 189+0+0 (secure 0 0 0) 0x7f02800456d0 con 0x7f0298067180 2021-07-11T04:00:55.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 -- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f027c0532b0 msgr2=0x7f027c055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f027c0532b0 0x7f027c055770 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f0288008000 tx=0x7f0288008040).stop 2021-07-11T04:00:55.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 -- 172.21.2.104:0/3871365625 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0298067180 msgr2=0x7f02980b38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0298067180 0x7f02980b38d0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7f0298148420 tx=0x7f0280007bd0).stop 2021-07-11T04:00:55.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 -- 172.21.2.104:0/3871365625 shutdown_connections 2021-07-11T04:00:55.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02980b1970 0x7f02980ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02980a76b0 0x7f02980ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f027c0532b0 0x7f027c055770 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 --2- 172.21.2.104:0/3871365625 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0298067180 0x7f02980b38d0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 -- 172.21.2.104:0/3871365625 >> 172.21.2.104:0/3871365625 conn(0x7f02980b53e0 msgr2=0x7f0298060470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:55.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 -- 172.21.2.104:0/3871365625 shutdown_connections 2021-07-11T04:00:55.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.671+0000 7f029e616700 1 -- 172.21.2.104:0/3871365625 wait complete. 2021-07-11T04:00:55.680 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 19 and data pool 20 2021-07-11T04:00:55.681 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:55.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac83a0d700 1 Processor -- start 2021-07-11T04:00:55.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac83a0d700 1 -- start start 2021-07-11T04:00:55.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac83a0d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c067180 0x7fac7c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac83a0d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fac7c0b1970 0x7fac7c05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac83a0d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac7c0a7a90 con 0x7fac7c0a76b0 2021-07-11T04:00:55.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac83a0d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac7c147620 con 0x7fac7c0b1970 2021-07-11T04:00:55.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac83a0d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c1478c0 con 0x7fac7c067180 2021-07-11T04:00:55.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac817a9700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fac7c0a76b0 0x7fac7c066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45386/0 (socket says 172.21.2.104:45386) 2021-07-11T04:00:55.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac817a9700 1 -- 172.21.2.104:0/4200527913 learned_addr learned my addr 172.21.2.104:0/4200527913 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:55.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac81faa700 1 --2- 172.21.2.104:0/4200527913 >> v2:172.21.2.105:3300/0 conn(0x7fac7c0b1970 0x7fac7c05fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376369246 0 0) 0x7fac7c0a7a90 con 0x7fac7c0a76b0 2021-07-11T04:00:55.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac64003260 con 0x7fac7c0a76b0 2021-07-11T04:00:55.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac80fa8700 1 --2- 172.21.2.104:0/4200527913 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c067180 0x7fac7c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681299836 0 0) 0x7fac64003260 con 0x7fac7c0a76b0 2021-07-11T04:00:55.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c067180 msgr2=0x7fac7c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:55.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 --2- 172.21.2.104:0/4200527913 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c067180 0x7fac7c05f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 >> v2:172.21.2.105:3300/0 conn(0x7fac7c0b1970 msgr2=0x7fac7c05fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:55.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 --2- 172.21.2.104:0/4200527913 >> v2:172.21.2.105:3300/0 conn(0x7fac7c0b1970 0x7fac7c05fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c147b60 con 0x7fac7c0a76b0 2021-07-11T04:00:55.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fac78001680 con 0x7fac7c0a76b0 2021-07-11T04:00:55.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 >> v1:172.21.2.104:6789/0 conn(0x7fac7c0a76b0 legacy=0x7fac7c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 --2- 172.21.2.104:0/4200527913 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c067180 0x7fac64003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 --2- 172.21.2.104:0/4200527913 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac64003c80 0x7fac64004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 --2- 172.21.2.104:0/4200527913 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac640049f0 0x7fac64004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c147620 con 0x7fac64003c80 2021-07-11T04:00:55.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c1478c0 con 0x7fac640049f0 2021-07-11T04:00:55.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6400b750 con 0x7fac7c067180 2021-07-11T04:00:55.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac80fa8700 1 --2- 172.21.2.104:0/4200527913 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac64003c80 0x7fac64004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac80fa8700 1 -- 172.21.2.104:0/4200527913 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c067180 msgr2=0x7fac64003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:55.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac817a9700 1 --2- 172.21.2.104:0/4200527913 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c067180 0x7fac64003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac80fa8700 1 --2- 172.21.2.104:0/4200527913 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c067180 0x7fac64003750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac80fa8700 1 -- 172.21.2.104:0/4200527913 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac640049f0 msgr2=0x7fac64004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:55.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac80fa8700 1 --2- 172.21.2.104:0/4200527913 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac640049f0 0x7fac64004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:55.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac80fa8700 1 -- 172.21.2.104:0/4200527913 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c147b60 con 0x7fac64003c80 2021-07-11T04:00:55.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac80fa8700 1 --2- 172.21.2.104:0/4200527913 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac64003c80 0x7fac64004500 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fac640054d0 tx=0x7fac6c006df0).ready entity=mon.0 client_cookie=21a36d645a3870b5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:55.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac6c00e830 con 0x7fac64003c80 2021-07-11T04:00:55.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c00be90 con 0x7fac64003c80 2021-07-11T04:00:55.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac6bfff700 1 -- 172.21.2.104:0/4200527913 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac6c00eb80 con 0x7fac64003c80 2021-07-11T04:00:55.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac83a0d700 1 -- 172.21.2.104:0/4200527913 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac64003c80 msgr2=0x7fac64004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac83a0d700 1 --2- 172.21.2.104:0/4200527913 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac64003c80 0x7fac64004500 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fac640054d0 tx=0x7fac6c006df0).stop 2021-07-11T04:00:55.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.731+0000 7fac81faa700 1 -- 172.21.2.104:0/4200527913 reap_dead start 2021-07-11T04:00:55.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 -- 172.21.2.104:0/4200527913 shutdown_connections 2021-07-11T04:00:55.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 -- 172.21.2.104:0/4200527913 >> 172.21.2.104:0/4200527913 conn(0x7fac7c0b53e0 msgr2=0x7fac7c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:55.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 -- 172.21.2.104:0/4200527913 shutdown_connections 2021-07-11T04:00:55.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 -- 172.21.2.104:0/4200527913 wait complete. 2021-07-11T04:00:55.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 Processor -- start 2021-07-11T04:00:55.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 -- start start 2021-07-11T04:00:55.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c0ad7e0 0x7fac7c0ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac7c0ab060 0x7fac7c0aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac7c0adbc0 0x7fac7c0ab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c0ac330 con 0x7fac7c0ab060 2021-07-11T04:00:55.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c156a50 con 0x7fac7c0adbc0 2021-07-11T04:00:55.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7c0ae440 con 0x7fac7c0ad7e0 2021-07-11T04:00:55.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac80fa8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac7c0ab060 0x7fac7c0aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac80fa8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac7c0ab060 0x7fac7c0aedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48630/0 (socket says 172.21.2.104:48630) 2021-07-11T04:00:55.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac80fa8700 1 -- 172.21.2.104:0/2474639986 learned_addr learned my addr 172.21.2.104:0/2474639986 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:55.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac80fa8700 1 -- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c0ad7e0 msgr2=0x7fac7c0ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac80fa8700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c0ad7e0 0x7fac7c0ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:55.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac80fa8700 1 -- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac7c0adbc0 msgr2=0x7fac7c0ab6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac80fa8700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac7c0adbc0 0x7fac7c0ab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:55.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac80fa8700 1 -- 172.21.2.104:0/2474639986 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c156780 con 0x7fac7c0ab060 2021-07-11T04:00:55.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac80fa8700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac7c0ab060 0x7fac7c0aedd0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7fac640054d0 tx=0x7fac6c019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:55.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac6a7fc700 1 -- 172.21.2.104:0/2474639986 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac6c012e80 con 0x7fac7c0ab060 2021-07-11T04:00:55.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac6a7fc700 1 -- 172.21.2.104:0/2474639986 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6c000e90 con 0x7fac7c0ab060 2021-07-11T04:00:55.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 -- 172.21.2.104:0/2474639986 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7c0ae140 con 0x7fac7c0ab060 2021-07-11T04:00:55.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac6a7fc700 1 -- 172.21.2.104:0/2474639986 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac6c01d600 con 0x7fac7c0ab060 2021-07-11T04:00:55.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 -- 172.21.2.104:0/2474639986 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7c0a9750 con 0x7fac7c0ab060 2021-07-11T04:00:55.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac83a0d700 1 -- 172.21.2.104:0/2474639986 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7c0ae600 con 0x7fac7c0ab060 2021-07-11T04:00:55.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac6a7fc700 1 -- 172.21.2.104:0/2474639986 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fac6c02d070 con 0x7fac7c0ab060 2021-07-11T04:00:55.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.735+0000 7fac6a7fc700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac5c053240 0x7fac5c055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.739+0000 7fac6a7fc700 1 -- 172.21.2.104:0/2474639986 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fac6c06cef0 con 0x7fac7c0ab060 2021-07-11T04:00:55.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.739+0000 7fac817a9700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac5c053240 0x7fac5c055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.739+0000 7fac817a9700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac5c053240 0x7fac5c055700 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fac78008000 tx=0x7fac78009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:55.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.739+0000 7fac6a7fc700 1 -- 172.21.2.104:0/2474639986 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fac6c0440c0 con 0x7fac7c0ab060 2021-07-11T04:00:55.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 -- 172.21.2.104:0/2474639986 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac7c057350 con 0x7fac7c0ab060 2021-07-11T04:00:55.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac6a7fc700 1 -- 172.21.2.104:0/2474639986 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16065 (secure 0 0 0) 0x7fac6c00b3c0 con 0x7fac7c0ab060 2021-07-11T04:00:55.825 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:55.826 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":107,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:55.667859+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:53.655121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:54.748677+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:55.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 -- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac5c053240 msgr2=0x7fac5c055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac5c053240 0x7fac5c055700 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7fac78008000 tx=0x7fac78009040).stop 2021-07-11T04:00:55.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 -- 172.21.2.104:0/2474639986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac7c0ab060 msgr2=0x7fac7c0aedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac7c0ab060 0x7fac7c0aedd0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7fac640054d0 tx=0x7fac6c019000).stop 2021-07-11T04:00:55.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 -- 172.21.2.104:0/2474639986 shutdown_connections 2021-07-11T04:00:55.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fac7c0adbc0 0x7fac7c0ab6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:55.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac5c053240 0x7fac5c055700 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fac7c0ab060 0x7fac7c0aedd0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 --2- 172.21.2.104:0/2474639986 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fac7c0ad7e0 0x7fac7c0ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:55.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 -- 172.21.2.104:0/2474639986 >> 172.21.2.104:0/2474639986 conn(0x7fac7c0b53e0 msgr2=0x7fac7c144740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:55.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 -- 172.21.2.104:0/2474639986 shutdown_connections 2021-07-11T04:00:55.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.823+0000 7fac83a0d700 1 -- 172.21.2.104:0/2474639986 wait complete. 2021-07-11T04:00:55.833 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:55.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 Processor -- start 2021-07-11T04:00:55.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 -- start start 2021-07-11T04:00:55.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f13c8060d00 0x7f13c80aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80671e0 con 0x7f13c8066e00 2021-07-11T04:00:55.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c80a9450 con 0x7f13c80afd80 2021-07-11T04:00:55.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c80b0190 con 0x7f13c8060d00 2021-07-11T04:00:55.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cd2e7700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f13c8066e00 0x7f13c80aa800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45406/0 (socket says 172.21.2.104:45406) 2021-07-11T04:00:55.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cd2e7700 1 -- 172.21.2.104:0/2201224539 learned_addr learned my addr 172.21.2.104:0/2201224539 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:55.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13ccae6700 1 --2- 172.21.2.104:0/2201224539 >> v2:172.21.2.104:3300/0 conn(0x7f13c8060d00 0x7f13c80aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13ccae6700 1 -- 172.21.2.104:0/2201224539 >> v1:172.21.2.105:6789/0 conn(0x7f13c80afd80 legacy=0x7f13c8143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:55.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13ccae6700 1 -- 172.21.2.104:0/2201224539 >> v1:172.21.2.104:6789/0 conn(0x7f13c8066e00 legacy=0x7f13c80aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13ccae6700 1 -- 172.21.2.104:0/2201224539 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c80b03b0 con 0x7f13c8060d00 2021-07-11T04:00:55.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13ccae6700 1 --2- 172.21.2.104:0/2201224539 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13c8060d00 0x7f13c80aaf10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f13c805f320 tx=0x7f13c4006df0).ready entity=mon.0 client_cookie=a187af7f20096a97 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:55.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13bbfff700 1 -- 172.21.2.104:0/2201224539 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13c400d5c0 con 0x7f13c8060d00 2021-07-11T04:00:55.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13bbfff700 1 -- 172.21.2.104:0/2201224539 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c4000e90 con 0x7f13c8060d00 2021-07-11T04:00:55.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13bbfff700 1 -- 172.21.2.104:0/2201224539 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13c400d910 con 0x7f13c8060d00 2021-07-11T04:00:55.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 -- 172.21.2.104:0/2201224539 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13c8060d00 msgr2=0x7f13c80aaf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 --2- 172.21.2.104:0/2201224539 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13c8060d00 0x7f13c80aaf10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f13c805f320 tx=0x7f13c4006df0).stop 2021-07-11T04:00:55.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 -- 172.21.2.104:0/2201224539 shutdown_connections 2021-07-11T04:00:55.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 --2- 172.21.2.104:0/2201224539 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13c8060d00 0x7f13c80aaf10 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 -- 172.21.2.104:0/2201224539 >> 172.21.2.104:0/2201224539 conn(0x7f13c80b53e0 msgr2=0x7f13c80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:55.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 -- 172.21.2.104:0/2201224539 shutdown_connections 2021-07-11T04:00:55.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.887+0000 7f13cf54b700 1 -- 172.21.2.104:0/2201224539 wait complete. 2021-07-11T04:00:55.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cf54b700 1 Processor -- start 2021-07-11T04:00:55.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cf54b700 1 -- start start 2021-07-11T04:00:55.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cf54b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13c8060d00 0x7f13c80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cf54b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13c8066e00 0x7f13c814a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cf54b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13c80afd80 0x7f13c8152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cf54b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c814aa00 con 0x7f13c8060d00 2021-07-11T04:00:55.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cf54b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c814a770 con 0x7f13c80afd80 2021-07-11T04:00:55.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cf54b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c813dd60 con 0x7f13c8066e00 2021-07-11T04:00:55.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cd2e7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13c8060d00 0x7f13c80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cd2e7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13c8060d00 0x7f13c80b4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48646/0 (socket says 172.21.2.104:48646) 2021-07-11T04:00:55.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cd2e7700 1 -- 172.21.2.104:0/2661233415 learned_addr learned my addr 172.21.2.104:0/2661233415 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:55.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cd2e7700 1 -- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13c8066e00 msgr2=0x7f13c814a280 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:55.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cd2e7700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13c8066e00 0x7f13c814a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:55.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cd2e7700 1 -- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13c80afd80 msgr2=0x7f13c8152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cd2e7700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13c80afd80 0x7f13c8152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:55.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cd2e7700 1 -- 172.21.2.104:0/2661233415 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c400b180 con 0x7f13c8060d00 2021-07-11T04:00:55.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cd2e7700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13c8060d00 0x7f13c80b4d70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f13c80623b0 tx=0x7f13bc007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:55.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13ba7fc700 1 -- 172.21.2.104:0/2661233415 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13bc00ce30 con 0x7f13c8060d00 2021-07-11T04:00:55.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13ba7fc700 1 -- 172.21.2.104:0/2661233415 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc0072e0 con 0x7f13c8060d00 2021-07-11T04:00:55.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cf54b700 1 -- 172.21.2.104:0/2661233415 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c813e000 con 0x7f13c8060d00 2021-07-11T04:00:55.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cf54b700 1 -- 172.21.2.104:0/2661233415 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c813e740 con 0x7f13c8060d00 2021-07-11T04:00:55.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13ba7fc700 1 -- 172.21.2.104:0/2661233415 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f13bc0057c0 con 0x7f13c8060d00 2021-07-11T04:00:55.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13cf54b700 1 -- 172.21.2.104:0/2661233415 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c813e4a0 con 0x7f13c8060d00 2021-07-11T04:00:55.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13ba7fc700 1 -- 172.21.2.104:0/2661233415 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f13bc010070 con 0x7f13c8060d00 2021-07-11T04:00:55.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13ba7fc700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13a8053350 0x7f13a8055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:55.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13ba7fc700 1 -- 172.21.2.104:0/2661233415 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f13bc06abf0 con 0x7f13c8060d00 2021-07-11T04:00:55.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.891+0000 7f13ccae6700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13a8053350 0x7f13a8055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:55.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.895+0000 7f13ccae6700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13a8053350 0x7f13a8055810 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f13c805f320 tx=0x7f13c4012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:55.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.895+0000 7f13ba7fc700 1 -- 172.21.2.104:0/2661233415 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f13c813e4a0 con 0x7f13c8060d00 2021-07-11T04:00:55.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 -- 172.21.2.104:0/2661233415 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13c813e4a0 con 0x7f13c8060d00 2021-07-11T04:00:55.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13ba7fc700 1 -- 172.21.2.104:0/2661233415 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f13c813e4a0 con 0x7f13c8060d00 2021-07-11T04:00:55.977 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:55.978 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":107,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:55.667859+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:53.655121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:54.748677+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:55.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 -- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13a8053350 msgr2=0x7f13a8055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13a8053350 0x7f13a8055810 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f13c805f320 tx=0x7f13c4012000).stop 2021-07-11T04:00:55.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 -- 172.21.2.104:0/2661233415 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13c8060d00 msgr2=0x7f13c80b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:55.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13c8060d00 0x7f13c80b4d70 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f13c80623b0 tx=0x7f13bc007c30).stop 2021-07-11T04:00:55.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 -- 172.21.2.104:0/2661233415 shutdown_connections 2021-07-11T04:00:55.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13c80afd80 0x7f13c8152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:55.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13c8066e00 0x7f13c814a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:55.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f13a8053350 0x7f13a8055810 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 --2- 172.21.2.104:0/2661233415 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13c8060d00 0x7f13c80b4d70 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:55.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 -- 172.21.2.104:0/2661233415 >> 172.21.2.104:0/2661233415 conn(0x7f13c80b53e0 msgr2=0x7f13c8141880 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:55.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 -- 172.21.2.104:0/2661233415 shutdown_connections 2021-07-11T04:00:55.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:55.975+0000 7f13cf54b700 1 -- 172.21.2.104:0/2661233415 wait complete. 2021-07-11T04:00:55.985 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T04:00:56.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 Processor -- start 2021-07-11T04:00:56.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- start start 2021-07-11T04:00:56.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fab700b05b0 0x7fab700b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fab700b3710 0x7fab7013d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab7013da40 0x7fab700b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab70143920 con 0x7fab700b05b0 2021-07-11T04:00:56.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab70143bc0 con 0x7fab700b3710 2021-07-11T04:00:56.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab70143e60 con 0x7fab7013da40 2021-07-11T04:00:56.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7646f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fab700b05b0 0x7fab700b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:56.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7646f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fab700b05b0 0x7fab700b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48652/0 (socket says 172.21.2.104:48652) 2021-07-11T04:00:56.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7646f700 1 -- 172.21.2.104:0/862441175 learned_addr learned my addr 172.21.2.104:0/862441175 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:56.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7646f700 1 -- 172.21.2.104:0/862441175 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab7013da40 msgr2=0x7fab700b10b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:56.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7646f700 1 --2- 172.21.2.104:0/862441175 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab7013da40 0x7fab700b10b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7646f700 1 -- 172.21.2.104:0/862441175 >> v2:172.21.2.105:3300/0 conn(0x7fab700b3710 msgr2=0x7fab7013d550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:56.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7646f700 1 --2- 172.21.2.104:0/862441175 >> v2:172.21.2.105:3300/0 conn(0x7fab700b3710 0x7fab7013d550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7646f700 1 -- 172.21.2.104:0/862441175 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab70144100 con 0x7fab700b05b0 2021-07-11T04:00:56.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7646f700 1 --2- 172.21.2.104:0/862441175 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab700b05b0 0x7fab700b0990 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7fab60002f70 tx=0x7fab60006eb0).ready entity=mon.0 client_cookie=ab958a1e9eea4582 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:56.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7546d700 1 -- 172.21.2.104:0/862441175 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab60012a50 con 0x7fab700b05b0 2021-07-11T04:00:56.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7546d700 1 -- 172.21.2.104:0/862441175 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab60000e90 con 0x7fab700b05b0 2021-07-11T04:00:56.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab7546d700 1 -- 172.21.2.104:0/862441175 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab60012de0 con 0x7fab700b05b0 2021-07-11T04:00:56.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- 172.21.2.104:0/862441175 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab700b05b0 msgr2=0x7fab700b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 --2- 172.21.2.104:0/862441175 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab700b05b0 0x7fab700b0990 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7fab60002f70 tx=0x7fab60006eb0).stop 2021-07-11T04:00:56.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- 172.21.2.104:0/862441175 shutdown_connections 2021-07-11T04:00:56.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 --2- 172.21.2.104:0/862441175 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab7013da40 0x7fab700b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 --2- 172.21.2.104:0/862441175 >> v2:172.21.2.105:3300/0 conn(0x7fab700b3710 0x7fab7013d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 --2- 172.21.2.104:0/862441175 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab700b05b0 0x7fab700b0990 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:56.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- 172.21.2.104:0/862441175 >> 172.21.2.104:0/862441175 conn(0x7fab700607c0 msgr2=0x7fab70060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:56.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- 172.21.2.104:0/862441175 shutdown_connections 2021-07-11T04:00:56.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- 172.21.2.104:0/862441175 wait complete. 2021-07-11T04:00:56.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 Processor -- start 2021-07-11T04:00:56.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- start start 2021-07-11T04:00:56.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab700b05b0 0x7fab700654c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab700b3710 0x7fab70139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab7013da40 0x7fab7014e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab70067700 con 0x7fab700b3710 2021-07-11T04:00:56.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab70137890 con 0x7fab7013da40 2021-07-11T04:00:56.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.039+0000 7fab786d3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab70063a50 con 0x7fab700b05b0 2021-07-11T04:00:56.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab75c6e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab700b3710 0x7fab70139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:56.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab75c6e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab700b3710 0x7fab70139560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48660/0 (socket says 172.21.2.104:48660) 2021-07-11T04:00:56.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab75c6e700 1 -- 172.21.2.104:0/760391353 learned_addr learned my addr 172.21.2.104:0/760391353 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:56.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab75c6e700 1 -- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab700b05b0 msgr2=0x7fab700654c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab75c6e700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab700b05b0 0x7fab700654c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab75c6e700 1 -- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab7013da40 msgr2=0x7fab7014e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab75c6e700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab7013da40 0x7fab7014e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab75c6e700 1 -- 172.21.2.104:0/760391353 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6000b1e0 con 0x7fab700b3710 2021-07-11T04:00:56.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab75c6e700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab700b3710 0x7fab70139560 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fab6c002fb0 tx=0x7fab6c006e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:56.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab5f7fe700 1 -- 172.21.2.104:0/760391353 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab6c013460 con 0x7fab700b3710 2021-07-11T04:00:56.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab5f7fe700 1 -- 172.21.2.104:0/760391353 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6c0135c0 con 0x7fab700b3710 2021-07-11T04:00:56.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab786d3700 1 -- 172.21.2.104:0/760391353 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab70063cf0 con 0x7fab700b3710 2021-07-11T04:00:56.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab5f7fe700 1 -- 172.21.2.104:0/760391353 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab6c013920 con 0x7fab700b3710 2021-07-11T04:00:56.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab786d3700 1 -- 172.21.2.104:0/760391353 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab70067b10 con 0x7fab700b3710 2021-07-11T04:00:56.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab786d3700 1 -- 172.21.2.104:0/760391353 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab70137a00 con 0x7fab700b3710 2021-07-11T04:00:56.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab5f7fe700 1 -- 172.21.2.104:0/760391353 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fab6c00f070 con 0x7fab700b3710 2021-07-11T04:00:56.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab5f7fe700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab50053300 0x7fab500557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab5f7fe700 1 -- 172.21.2.104:0/760391353 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fab6c06ace0 con 0x7fab700b3710 2021-07-11T04:00:56.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab7646f700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab50053300 0x7fab500557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:56.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.043+0000 7fab7646f700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab50053300 0x7fab500557c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fab60002f70 tx=0x7fab60019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:56.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.047+0000 7fab5f7fe700 1 -- 172.21.2.104:0/760391353 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fab6c041f10 con 0x7fab700b3710 2021-07-11T04:00:56.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.151+0000 7fab786d3700 1 -- 172.21.2.104:0/760391353 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fab70057350 con 0x7fab700b3710 2021-07-11T04:00:56.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab5f7fe700 1 -- 172.21.2.104:0/760391353 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v50) v1 ==== 118+0+0 (secure 0 0 0) 0x7fab6c018030 con 0x7fab700b3710 2021-07-11T04:00:56.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 -- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab50053300 msgr2=0x7fab500557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab50053300 0x7fab500557c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7fab60002f70 tx=0x7fab60019000).stop 2021-07-11T04:00:56.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 -- 172.21.2.104:0/760391353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab700b3710 msgr2=0x7fab70139560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab700b3710 0x7fab70139560 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7fab6c002fb0 tx=0x7fab6c006e00).stop 2021-07-11T04:00:56.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 -- 172.21.2.104:0/760391353 shutdown_connections 2021-07-11T04:00:56.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab7013da40 0x7fab7014e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab50053300 0x7fab500557c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:56.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab700b3710 0x7fab70139560 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:56.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 --2- 172.21.2.104:0/760391353 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab700b05b0 0x7fab700654c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 -- 172.21.2.104:0/760391353 >> 172.21.2.104:0/760391353 conn(0x7fab700607c0 msgr2=0x7fab700b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:56.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 -- 172.21.2.104:0/760391353 shutdown_connections 2021-07-11T04:00:56.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.679+0000 7fab786d3700 1 -- 172.21.2.104:0/760391353 wait complete. 2021-07-11T04:00:56.688 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:56.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7b6d75700 1 Processor -- start 2021-07-11T04:00:56.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7b6d75700 1 -- start start 2021-07-11T04:00:56.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7b6d75700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7b00aa860 0x7fa7b00b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7b6d75700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa7b00a9da0 0x7fa7b00aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7b6d75700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b00ac370 con 0x7fa7b00acf30 2021-07-11T04:00:56.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7b6d75700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b00abb50 con 0x7fa7b00a9da0 2021-07-11T04:00:56.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7b6d75700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b00aac40 con 0x7fa7b00aa860 2021-07-11T04:00:56.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7affff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa7b00a9da0 0x7fa7b00aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:56.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7affff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa7b00a9da0 0x7fa7b00aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48670/0 (socket says 172.21.2.104:48670) 2021-07-11T04:00:56.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7affff700 1 -- 172.21.2.104:0/1016941809 learned_addr learned my addr 172.21.2.104:0/1016941809 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:56.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7affff700 1 -- 172.21.2.104:0/1016941809 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7b00aa860 msgr2=0x7fa7b00b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:56.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7affff700 1 --2- 172.21.2.104:0/1016941809 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7b00aa860 0x7fa7b00b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7affff700 1 -- 172.21.2.104:0/1016941809 >> v1:172.21.2.105:6789/0 conn(0x7fa7b00acf30 legacy=0x7fa7b01434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:56.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7affff700 1 -- 172.21.2.104:0/1016941809 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b00b16b0 con 0x7fa7b00a9da0 2021-07-11T04:00:56.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.747+0000 7fa7affff700 1 --2- 172.21.2.104:0/1016941809 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7b00a9da0 0x7fa7b00aca40 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7fa7b00ac920 tx=0x7fa798006f40).ready entity=mon.0 client_cookie=4b112ee77dfcae8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:56.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7af7fe700 1 -- 172.21.2.104:0/1016941809 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa79800dc30 con 0x7fa7b00a9da0 2021-07-11T04:00:56.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7af7fe700 1 -- 172.21.2.104:0/1016941809 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa798000e90 con 0x7fa7b00a9da0 2021-07-11T04:00:56.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7af7fe700 1 -- 172.21.2.104:0/1016941809 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa79800be90 con 0x7fa7b00a9da0 2021-07-11T04:00:56.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- 172.21.2.104:0/1016941809 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7b00a9da0 msgr2=0x7fa7b00aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 --2- 172.21.2.104:0/1016941809 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7b00a9da0 0x7fa7b00aca40 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7fa7b00ac920 tx=0x7fa798006f40).stop 2021-07-11T04:00:56.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- 172.21.2.104:0/1016941809 shutdown_connections 2021-07-11T04:00:56.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 --2- 172.21.2.104:0/1016941809 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7b00a9da0 0x7fa7b00aca40 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:56.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 --2- 172.21.2.104:0/1016941809 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7b00aa860 0x7fa7b00b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- 172.21.2.104:0/1016941809 >> 172.21.2.104:0/1016941809 conn(0x7fa7b00b64d0 msgr2=0x7fa7b00b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:56.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- 172.21.2.104:0/1016941809 shutdown_connections 2021-07-11T04:00:56.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- 172.21.2.104:0/1016941809 wait complete. 2021-07-11T04:00:56.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 Processor -- start 2021-07-11T04:00:56.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- start start 2021-07-11T04:00:56.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7b00a9da0 0x7fa7b0064f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7b00aa860 0x7fa7b00afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa7b00acf30 0x7fa7b0063bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b0063780 con 0x7fa7b00a9da0 2021-07-11T04:00:56.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b015a6e0 con 0x7fa7b00acf30 2021-07-11T04:00:56.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b015a980 con 0x7fa7b00aa860 2021-07-11T04:00:56.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b4b11700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7b00a9da0 0x7fa7b0064f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:56.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b4b11700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7b00a9da0 0x7fa7b0064f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48674/0 (socket says 172.21.2.104:48674) 2021-07-11T04:00:56.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b4b11700 1 -- 172.21.2.104:0/2048286410 learned_addr learned my addr 172.21.2.104:0/2048286410 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:56.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b4b11700 1 -- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7b00aa860 msgr2=0x7fa7b00afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b4b11700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7b00aa860 0x7fa7b00afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b4b11700 1 -- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa7b00acf30 msgr2=0x7fa7b0063bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b4b11700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa7b00acf30 0x7fa7b0063bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b4b11700 1 -- 172.21.2.104:0/2048286410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79800b460 con 0x7fa7b00a9da0 2021-07-11T04:00:56.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b4b11700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7b00a9da0 0x7fa7b0064f60 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7fa7a0002f70 tx=0x7fa7a0006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:56.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7adffb700 1 -- 172.21.2.104:0/2048286410 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7a000f510 con 0x7fa7b00a9da0 2021-07-11T04:00:56.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7adffb700 1 -- 172.21.2.104:0/2048286410 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a000f670 con 0x7fa7b00a9da0 2021-07-11T04:00:56.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7adffb700 1 -- 172.21.2.104:0/2048286410 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa7a000f7e0 con 0x7fa7b00a9da0 2021-07-11T04:00:56.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- 172.21.2.104:0/2048286410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b0065b40 con 0x7fa7b00a9da0 2021-07-11T04:00:56.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- 172.21.2.104:0/2048286410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b0141390 con 0x7fa7b00a9da0 2021-07-11T04:00:56.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7b6d75700 1 -- 172.21.2.104:0/2048286410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b00638f0 con 0x7fa7b00a9da0 2021-07-11T04:00:56.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7adffb700 1 -- 172.21.2.104:0/2048286410 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa7a000f070 con 0x7fa7b00a9da0 2021-07-11T04:00:56.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.751+0000 7fa7adffb700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa794053240 0x7fa794055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.755+0000 7fa7adffb700 1 -- 172.21.2.104:0/2048286410 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa7a006b0c0 con 0x7fa7b00a9da0 2021-07-11T04:00:56.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.755+0000 7fa7affff700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa794053240 0x7fa794055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:56.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.755+0000 7fa7affff700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa794053240 0x7fa794055700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fa7b00ac920 tx=0x7fa7b00ac290).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:56.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.755+0000 7fa7adffb700 1 -- 172.21.2.104:0/2048286410 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa7b00638f0 con 0x7fa7b00a9da0 2021-07-11T04:00:56.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 -- 172.21.2.104:0/2048286410 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7b015a520 con 0x7fa7b00a9da0 2021-07-11T04:00:56.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7adffb700 1 -- 172.21.2.104:0/2048286410 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2484 (secure 0 0 0) 0x7fa7a00422f0 con 0x7fa7b00a9da0 2021-07-11T04:00:56.866 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:56.866 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":50,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/3293345379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3293345379},{"type":"v1","addr":"172.21.2.105:6835","nonce":3293345379}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/57099992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":57099992},{"type":"v1","addr":"172.21.2.104:6833","nonce":57099992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:55.667970+0000","modified":"2021-07-11T04:00:56.671726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7287":{"gid":7287,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/871079070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":871079070},{"type":"v1","addr":"172.21.2.104:6835","nonce":871079070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-07-11T04:00:56.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 -- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa794053240 msgr2=0x7fa794055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa794053240 0x7fa794055700 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fa7b00ac920 tx=0x7fa7b00ac290).stop 2021-07-11T04:00:56.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 -- 172.21.2.104:0/2048286410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7b00a9da0 msgr2=0x7fa7b0064f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7b00a9da0 0x7fa7b0064f60 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7fa7a0002f70 tx=0x7fa7a0006ef0).stop 2021-07-11T04:00:56.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 -- 172.21.2.104:0/2048286410 shutdown_connections 2021-07-11T04:00:56.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa7b00acf30 0x7fa7b0063bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7b00aa860 0x7fa7b00afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa794053240 0x7fa794055700 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:56.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 --2- 172.21.2.104:0/2048286410 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7b00a9da0 0x7fa7b0064f60 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:56.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 -- 172.21.2.104:0/2048286410 >> 172.21.2.104:0/2048286410 conn(0x7fa7b00b64d0 msgr2=0x7fa7b014df10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:56.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 -- 172.21.2.104:0/2048286410 shutdown_connections 2021-07-11T04:00:56.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.863+0000 7fa7b6d75700 1 -- 172.21.2.104:0/2048286410 wait complete. 2021-07-11T04:00:56.870 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 50 2021-07-11T04:00:56.873 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:56.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 Processor -- start 2021-07-11T04:00:56.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 -- start start 2021-07-11T04:00:56.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f16e80b2660 0x7f16e805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f16e8097130 0x7f16e813fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16e80b2a40 con 0x7f16e8096cc0 2021-07-11T04:00:56.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e8097b80 con 0x7f16e8097130 2021-07-11T04:00:56.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16e8097e20 con 0x7f16e80b2660 2021-07-11T04:00:56.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16ed3ad700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f16e8097130 0x7f16e813fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:56.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16ed3ad700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f16e8097130 0x7f16e813fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48680/0 (socket says 172.21.2.104:48680) 2021-07-11T04:00:56.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16ed3ad700 1 -- 172.21.2.104:0/1125571764 learned_addr learned my addr 172.21.2.104:0/1125571764 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:56.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16ed3ad700 1 -- 172.21.2.104:0/1125571764 >> v2:172.21.2.105:3300/0 conn(0x7f16e80b2660 msgr2=0x7f16e805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:56.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16ed3ad700 1 --2- 172.21.2.104:0/1125571764 >> v2:172.21.2.105:3300/0 conn(0x7f16e80b2660 0x7f16e805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16ed3ad700 1 -- 172.21.2.104:0/1125571764 >> v1:172.21.2.105:6789/0 conn(0x7f16e8096cc0 legacy=0x7f16e813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:56.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16ed3ad700 1 -- 172.21.2.104:0/1125571764 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e80980c0 con 0x7f16e8097130 2021-07-11T04:00:56.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16ed3ad700 1 --2- 172.21.2.104:0/1125571764 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16e8097130 0x7f16e813fc70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f16e80986e0 tx=0x7f16e000b530).ready entity=mon.0 client_cookie=a9611a74269dbc76 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:56.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16e77fe700 1 -- 172.21.2.104:0/1125571764 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16e00155e0 con 0x7f16e8097130 2021-07-11T04:00:56.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16e77fe700 1 -- 172.21.2.104:0/1125571764 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e0015740 con 0x7f16e8097130 2021-07-11T04:00:56.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16e77fe700 1 -- 172.21.2.104:0/1125571764 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16e0015aa0 con 0x7f16e8097130 2021-07-11T04:00:56.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 -- 172.21.2.104:0/1125571764 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16e8097130 msgr2=0x7f16e813fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 --2- 172.21.2.104:0/1125571764 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16e8097130 0x7f16e813fc70 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f16e80986e0 tx=0x7f16e000b530).stop 2021-07-11T04:00:56.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 -- 172.21.2.104:0/1125571764 shutdown_connections 2021-07-11T04:00:56.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 --2- 172.21.2.104:0/1125571764 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16e8097130 0x7f16e813fc70 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:56.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 --2- 172.21.2.104:0/1125571764 >> v2:172.21.2.105:3300/0 conn(0x7f16e80b2660 0x7f16e805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 -- 172.21.2.104:0/1125571764 >> 172.21.2.104:0/1125571764 conn(0x7f16e8062f10 msgr2=0x7f16e80632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:56.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 -- 172.21.2.104:0/1125571764 shutdown_connections 2021-07-11T04:00:56.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 -- 172.21.2.104:0/1125571764 wait complete. 2021-07-11T04:00:56.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 Processor -- start 2021-07-11T04:00:56.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.927+0000 7f16eee10700 1 -- start start 2021-07-11T04:00:56.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16eee10700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16e8096cc0 0x7f16e8063f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16eee10700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16e8097130 0x7f16e8152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16eee10700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f16e80b2660 0x7f16e8152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16eee10700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e8153170 con 0x7f16e8097130 2021-07-11T04:00:56.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16eee10700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16e809b360 con 0x7f16e80b2660 2021-07-11T04:00:56.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16eee10700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16e80aa4c0 con 0x7f16e8096cc0 2021-07-11T04:00:56.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16e8097130 0x7f16e8152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:56.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16e8097130 0x7f16e8152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48686/0 (socket says 172.21.2.104:48686) 2021-07-11T04:00:56.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e7fff700 1 -- 172.21.2.104:0/1840865874 learned_addr learned my addr 172.21.2.104:0/1840865874 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:56.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e7fff700 1 -- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16e8096cc0 msgr2=0x7f16e8063f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e7fff700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16e8096cc0 0x7f16e8063f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e7fff700 1 -- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f16e80b2660 msgr2=0x7f16e8152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:56.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e7fff700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f16e80b2660 0x7f16e8152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:56.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e7fff700 1 -- 172.21.2.104:0/1840865874 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e000b7d0 con 0x7f16e8097130 2021-07-11T04:00:56.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e7fff700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16e8097130 0x7f16e8152790 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f16e809b860 tx=0x7f16d0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:56.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e5ffb700 1 -- 172.21.2.104:0/1840865874 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16d0013460 con 0x7f16e8097130 2021-07-11T04:00:56.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e5ffb700 1 -- 172.21.2.104:0/1840865874 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d00135c0 con 0x7f16e8097130 2021-07-11T04:00:56.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e5ffb700 1 -- 172.21.2.104:0/1840865874 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16d0013920 con 0x7f16e8097130 2021-07-11T04:00:56.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16eee10700 1 -- 172.21.2.104:0/1840865874 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e809b590 con 0x7f16e8097130 2021-07-11T04:00:56.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16eee10700 1 -- 172.21.2.104:0/1840865874 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e80aa990 con 0x7f16e8097130 2021-07-11T04:00:56.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16eee10700 1 -- 172.21.2.104:0/1840865874 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16e80acb00 con 0x7f16e8097130 2021-07-11T04:00:56.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e5ffb700 1 -- 172.21.2.104:0/1840865874 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f16d001a060 con 0x7f16e8097130 2021-07-11T04:00:56.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e5ffb700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16cc053300 0x7f16cc0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:56.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16e5ffb700 1 -- 172.21.2.104:0/1840865874 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f16d006aeb0 con 0x7f16e8097130 2021-07-11T04:00:56.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16ecbac700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16cc053300 0x7f16cc0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:56.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.931+0000 7f16ecbac700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16cc053300 0x7f16cc0557c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f16d8002f70 tx=0x7f16d8006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:56.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:56.935+0000 7f16e5ffb700 1 -- 172.21.2.104:0/1840865874 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f16d00420e0 con 0x7f16e8097130 2021-07-11T04:00:57.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 -- 172.21.2.104:0/1840865874 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16e8057350 con 0x7f16e8097130 2021-07-11T04:00:57.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16e5ffb700 1 -- 172.21.2.104:0/1840865874 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f16d0018070 con 0x7f16e8097130 2021-07-11T04:00:57.018 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:57.019 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":108,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:56.672922+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:53.655121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:54.748677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:57.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 -- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16cc053300 msgr2=0x7f16cc0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16cc053300 0x7f16cc0557c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f16d8002f70 tx=0x7f16d8006a10).stop 2021-07-11T04:00:57.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 -- 172.21.2.104:0/1840865874 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16e8097130 msgr2=0x7f16e8152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16e8097130 0x7f16e8152790 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f16e809b860 tx=0x7f16d0006df0).stop 2021-07-11T04:00:57.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 -- 172.21.2.104:0/1840865874 shutdown_connections 2021-07-11T04:00:57.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f16e80b2660 0x7f16e8152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:57.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16cc053300 0x7f16cc0557c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16e8097130 0x7f16e8152790 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 --2- 172.21.2.104:0/1840865874 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16e8096cc0 0x7f16e8063f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:57.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 -- 172.21.2.104:0/1840865874 >> 172.21.2.104:0/1840865874 conn(0x7f16e8062f10 msgr2=0x7f16e80b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:57.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.015+0000 7f16eee10700 1 -- 172.21.2.104:0/1840865874 shutdown_connections 2021-07-11T04:00:57.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.019+0000 7f16eee10700 1 -- 172.21.2.104:0/1840865874 wait complete. 2021-07-11T04:00:57.025 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:00:57.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 Processor -- start 2021-07-11T04:00:57.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 -- start start 2021-07-11T04:00:57.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe5ac0a76b0 0x7fe5ac066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe5ac0b0060 0x7fe5ac05fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ac0a7a90 con 0x7fe5ac0b1970 2021-07-11T04:00:57.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac147620 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ac1478c0 con 0x7fe5ac0b0060 2021-07-11T04:00:57.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b1ae4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe5ac0a76b0 0x7fe5ac066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b1ae4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe5ac0a76b0 0x7fe5ac066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48692/0 (socket says 172.21.2.104:48692) 2021-07-11T04:00:57.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b1ae4700 1 -- 172.21.2.104:0/1737045818 learned_addr learned my addr 172.21.2.104:0/1737045818 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:57.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b1ae4700 1 -- 172.21.2.104:0/1737045818 >> v2:172.21.2.105:3300/0 conn(0x7fe5ac0b0060 msgr2=0x7fe5ac05fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:57.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b1ae4700 1 --2- 172.21.2.104:0/1737045818 >> v2:172.21.2.105:3300/0 conn(0x7fe5ac0b0060 0x7fe5ac05fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:57.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b1ae4700 1 -- 172.21.2.104:0/1737045818 >> v1:172.21.2.105:6789/0 conn(0x7fe5ac0b1970 legacy=0x7fe5ac05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:57.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b1ae4700 1 -- 172.21.2.104:0/1737045818 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ac147b60 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b1ae4700 1 --2- 172.21.2.104:0/1737045818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5ac0a76b0 0x7fe5ac066a70 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fe5ac06a0e0 tx=0x7fe5a8006eb0).ready entity=mon.0 client_cookie=598c5b2d353c061b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:57.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b0ae2700 1 -- 172.21.2.104:0/1737045818 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a800d5c0 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b0ae2700 1 -- 172.21.2.104:0/1737045818 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a8000e90 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b0ae2700 1 -- 172.21.2.104:0/1737045818 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a800d910 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 -- 172.21.2.104:0/1737045818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5ac0a76b0 msgr2=0x7fe5ac066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 --2- 172.21.2.104:0/1737045818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5ac0a76b0 0x7fe5ac066a70 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fe5ac06a0e0 tx=0x7fe5a8006eb0).stop 2021-07-11T04:00:57.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 -- 172.21.2.104:0/1737045818 shutdown_connections 2021-07-11T04:00:57.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 --2- 172.21.2.104:0/1737045818 >> v2:172.21.2.105:3300/0 conn(0x7fe5ac0b0060 0x7fe5ac05fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:57.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 --2- 172.21.2.104:0/1737045818 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5ac0a76b0 0x7fe5ac066a70 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 -- 172.21.2.104:0/1737045818 >> 172.21.2.104:0/1737045818 conn(0x7fe5ac0b53e0 msgr2=0x7fe5ac0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:57.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 -- 172.21.2.104:0/1737045818 shutdown_connections 2021-07-11T04:00:57.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.079+0000 7fe5b3d48700 1 -- 172.21.2.104:0/1737045818 wait complete. 2021-07-11T04:00:57.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b3d48700 1 Processor -- start 2021-07-11T04:00:57.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b3d48700 1 -- start start 2021-07-11T04:00:57.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b3d48700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5ac0a76b0 0x7fe5ac0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b3d48700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5ac0b0060 0x7fe5ac1570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b3d48700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5ac0b1970 0x7fe5ac0b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b3d48700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0a9da0 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b3d48700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac0aedb0 con 0x7fe5ac0b1970 2021-07-11T04:00:57.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b3d48700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ac157590 con 0x7fe5ac0b0060 2021-07-11T04:00:57.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b1ae4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5ac0a76b0 0x7fe5ac0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b1ae4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5ac0a76b0 0x7fe5ac0ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48698/0 (socket says 172.21.2.104:48698) 2021-07-11T04:00:57.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b1ae4700 1 -- 172.21.2.104:0/856739768 learned_addr learned my addr 172.21.2.104:0/856739768 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:57.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b1ae4700 1 -- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5ac0b0060 msgr2=0x7fe5ac1570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b22e5700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5ac0b1970 0x7fe5ac0b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b12e3700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5ac0b0060 0x7fe5ac1570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b1ae4700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5ac0b0060 0x7fe5ac1570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b1ae4700 1 -- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5ac0b1970 msgr2=0x7fe5ac0b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b1ae4700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5ac0b1970 0x7fe5ac0b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b1ae4700 1 -- 172.21.2.104:0/856739768 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a800b1e0 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b22e5700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5ac0b1970 0x7fe5ac0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:57.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b12e3700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5ac0b0060 0x7fe5ac1570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:57.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b1ae4700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5ac0a76b0 0x7fe5ac0ae8c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fe5ac157720 tx=0x7fe5a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:57.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe59effd700 1 -- 172.21.2.104:0/856739768 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a8021070 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe59effd700 1 -- 172.21.2.104:0/856739768 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a8000e90 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b3d48700 1 -- 172.21.2.104:0/856739768 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ac0b4f50 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe59effd700 1 -- 172.21.2.104:0/856739768 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe5a8011ca0 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b3d48700 1 -- 172.21.2.104:0/856739768 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ac0aefe0 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b3d48700 1 -- 172.21.2.104:0/856739768 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ac057350 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe59effd700 1 -- 172.21.2.104:0/856739768 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe5a800f070 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe59effd700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe58c0532d0 0x7fe58c055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe59effd700 1 -- 172.21.2.104:0/856739768 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe5a806c970 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.083+0000 7fe5b12e3700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe58c0532d0 0x7fe58c055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.087+0000 7fe5b12e3700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe58c0532d0 0x7fe58c055790 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fe5a0007000 tx=0x7fe5a0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:57.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.087+0000 7fe59effd700 1 -- 172.21.2.104:0/856739768 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe5a8043cb0 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.195+0000 7fe5b3d48700 1 -- 172.21.2.104:0/856739768 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fe5ac0af1e0 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe59effd700 1 -- 172.21.2.104:0/856739768 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v27) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe5a80472d0 con 0x7fe5ac0a76b0 2021-07-11T04:00:57.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 -- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe58c0532d0 msgr2=0x7fe58c055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe58c0532d0 0x7fe58c055790 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fe5a0007000 tx=0x7fe5a0008040).stop 2021-07-11T04:00:57.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 -- 172.21.2.104:0/856739768 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5ac0a76b0 msgr2=0x7fe5ac0ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5ac0a76b0 0x7fe5ac0ae8c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fe5ac157720 tx=0x7fe5a8015000).stop 2021-07-11T04:00:57.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 -- 172.21.2.104:0/856739768 shutdown_connections 2021-07-11T04:00:57.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe5ac0b1970 0x7fe5ac0b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5ac0b0060 0x7fe5ac1570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe58c0532d0 0x7fe58c055790 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 --2- 172.21.2.104:0/856739768 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe5ac0a76b0 0x7fe5ac0ae8c0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 -- 172.21.2.104:0/856739768 >> 172.21.2.104:0/856739768 conn(0x7fe5ac0b53e0 msgr2=0x7fe5ac14a000 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:57.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 -- 172.21.2.104:0/856739768 shutdown_connections 2021-07-11T04:00:57.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.203+0000 7fe5b3d48700 1 -- 172.21.2.104:0/856739768 wait complete. 2021-07-11T04:00:57.207 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T04:00:57.210 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:00:57.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.263+0000 7fb99029a700 1 Processor -- start 2021-07-11T04:00:57.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.263+0000 7fb99029a700 1 -- start start 2021-07-11T04:00:57.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.263+0000 7fb99029a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb98805f6b0 0x7fb98813fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.263+0000 7fb99029a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9880b2a40 con 0x7fb9880973d0 2021-07-11T04:00:57.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.263+0000 7fb99029a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb988097b80 con 0x7fb9880b2660 2021-07-11T04:00:57.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.263+0000 7fb99029a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb988097e20 con 0x7fb98805f6b0 2021-07-11T04:00:57.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.263+0000 7fb98d835700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb9880973d0 0x7fb98813f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45478/0 (socket says 172.21.2.104:45478) 2021-07-11T04:00:57.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.263+0000 7fb98d835700 1 -- 172.21.2.104:0/3495158017 learned_addr learned my addr 172.21.2.104:0/3495158017 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:57.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.263+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588106494 0 0) 0x7fb9880b2a40 con 0x7fb9880973d0 2021-07-11T04:00:57.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb970004370 con 0x7fb9880973d0 2021-07-11T04:00:57.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98e837700 1 --2- 172.21.2.104:0/3495158017 >> v2:172.21.2.105:3300/0 conn(0x7fb98805f6b0 0x7fb98813fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 991342472 0 0) 0x7fb970004370 con 0x7fb9880973d0 2021-07-11T04:00:57.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 >> v2:172.21.2.105:3300/0 conn(0x7fb98805f6b0 msgr2=0x7fb98813fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:57.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 --2- 172.21.2.104:0/3495158017 >> v2:172.21.2.105:3300/0 conn(0x7fb98805f6b0 0x7fb98813fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 >> v1:172.21.2.105:6789/0 conn(0x7fb9880b2660 legacy=0x7fb988096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9880980c0 con 0x7fb9880973d0 2021-07-11T04:00:57.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fb97c001680 con 0x7fb9880973d0 2021-07-11T04:00:57.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 >> v1:172.21.2.104:6789/0 conn(0x7fb9880973d0 legacy=0x7fb98813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 --2- 172.21.2.104:0/3495158017 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb970004860 0x7fb970004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 --2- 172.21.2.104:0/3495158017 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb970005610 0x7fb970005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 --2- 172.21.2.104:0/3495158017 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb970005fd0 0x7fb970003180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb988097e20 con 0x7fb970004860 2021-07-11T04:00:57.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9700037d0 con 0x7fb970005fd0 2021-07-11T04:00:57.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb970003990 con 0x7fb970005610 2021-07-11T04:00:57.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98e036700 1 --2- 172.21.2.104:0/3495158017 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb970004860 0x7fb970004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98e837700 1 --2- 172.21.2.104:0/3495158017 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb970005fd0 0x7fb970003180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d835700 1 --2- 172.21.2.104:0/3495158017 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb970005610 0x7fb970005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98e036700 1 -- 172.21.2.104:0/3495158017 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb970005610 msgr2=0x7fb970005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:57.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98e036700 1 --2- 172.21.2.104:0/3495158017 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb970005610 0x7fb970005a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98e036700 1 -- 172.21.2.104:0/3495158017 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb970005fd0 msgr2=0x7fb970003180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:57.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98e036700 1 --2- 172.21.2.104:0/3495158017 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb970005fd0 0x7fb970003180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98e837700 1 -- 172.21.2.104:0/3495158017 reap_dead start 2021-07-11T04:00:57.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98e036700 1 -- 172.21.2.104:0/3495158017 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb970004370 con 0x7fb970004860 2021-07-11T04:00:57.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98e036700 1 --2- 172.21.2.104:0/3495158017 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb970004860 0x7fb970004c80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7fb9880b33f0 tx=0x7fb984007a00).ready entity=mon.0 client_cookie=dea71c3edfc5dc16 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:57.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb984007d10 con 0x7fb970004860 2021-07-11T04:00:57.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb984007030 con 0x7fb970004860 2021-07-11T04:00:57.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d034700 1 -- 172.21.2.104:0/3495158017 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb984007390 con 0x7fb970004860 2021-07-11T04:00:57.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 -- 172.21.2.104:0/3495158017 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb970004860 msgr2=0x7fb970004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 --2- 172.21.2.104:0/3495158017 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb970004860 0x7fb970004c80 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7fb9880b33f0 tx=0x7fb984007a00).stop 2021-07-11T04:00:57.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 -- 172.21.2.104:0/3495158017 shutdown_connections 2021-07-11T04:00:57.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 --2- 172.21.2.104:0/3495158017 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb970004860 0x7fb970004c80 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 -- 172.21.2.104:0/3495158017 >> 172.21.2.104:0/3495158017 conn(0x7fb988062f10 msgr2=0x7fb9880632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:57.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 -- 172.21.2.104:0/3495158017 shutdown_connections 2021-07-11T04:00:57.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 -- 172.21.2.104:0/3495158017 wait complete. 2021-07-11T04:00:57.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 Processor -- start 2021-07-11T04:00:57.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 -- start start 2021-07-11T04:00:57.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb970004860 0x7fb988149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb98809a150 0x7fb98809a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb98809aaa0 0x7fb98813bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb988139c80 con 0x7fb98809a150 2021-07-11T04:00:57.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb988098c10 con 0x7fb98809aaa0 2021-07-11T04:00:57.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb99029a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb988098e40 con 0x7fb970004860 2021-07-11T04:00:57.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d835700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb98809a150 0x7fb98809a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d835700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb98809a150 0x7fb98809a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48718/0 (socket says 172.21.2.104:48718) 2021-07-11T04:00:57.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d835700 1 -- 172.21.2.104:0/1521305799 learned_addr learned my addr 172.21.2.104:0/1521305799 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:57.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d835700 1 -- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb970004860 msgr2=0x7fb988149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d835700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb970004860 0x7fb988149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:57.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d835700 1 -- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb98809aaa0 msgr2=0x7fb98813bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d835700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb98809aaa0 0x7fb98813bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:57.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d835700 1 -- 172.21.2.104:0/1521305799 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9880b54d0 con 0x7fb98809a150 2021-07-11T04:00:57.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb98d835700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb98809a150 0x7fb98809a5b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7fb9880b51f0 tx=0x7fb9880b1b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:57.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.267+0000 7fb97b7fe700 1 -- 172.21.2.104:0/1521305799 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb97c010710 con 0x7fb98809a150 2021-07-11T04:00:57.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.271+0000 7fb97b7fe700 1 -- 172.21.2.104:0/1521305799 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c010870 con 0x7fb98809a150 2021-07-11T04:00:57.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.271+0000 7fb99029a700 1 -- 172.21.2.104:0/1521305799 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9880b5770 con 0x7fb98809a150 2021-07-11T04:00:57.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.271+0000 7fb97b7fe700 1 -- 172.21.2.104:0/1521305799 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb97c010bd0 con 0x7fb98809a150 2021-07-11T04:00:57.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.271+0000 7fb99029a700 1 -- 172.21.2.104:0/1521305799 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb988099b70 con 0x7fb98809a150 2021-07-11T04:00:57.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.271+0000 7fb99029a700 1 -- 172.21.2.104:0/1521305799 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb988099e40 con 0x7fb98809a150 2021-07-11T04:00:57.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.271+0000 7fb97b7fe700 1 -- 172.21.2.104:0/1521305799 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb97c01a020 con 0x7fb98809a150 2021-07-11T04:00:57.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.271+0000 7fb97b7fe700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb968053300 0x7fb9680557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.271+0000 7fb97b7fe700 1 -- 172.21.2.104:0/1521305799 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb97c06b240 con 0x7fb98809a150 2021-07-11T04:00:57.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.271+0000 7fb98e036700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb968053300 0x7fb9680557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.271+0000 7fb98e036700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb968053300 0x7fb9680557c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7fb9880b33f0 tx=0x7fb984015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:57.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.275+0000 7fb97b7fe700 1 -- 172.21.2.104:0/1521305799 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb97c042520 con 0x7fb98809a150 2021-07-11T04:00:57.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.383+0000 7fb99029a700 1 -- 172.21.2.104:0/1521305799 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fb988057350 con 0x7fb98809a150 2021-07-11T04:00:57.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb97b7fe700 1 -- 172.21.2.104:0/1521305799 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v28) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb97c045b40 con 0x7fb98809a150 2021-07-11T04:00:57.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 -- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb968053300 msgr2=0x7fb9680557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb968053300 0x7fb9680557c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7fb9880b33f0 tx=0x7fb984015000).stop 2021-07-11T04:00:57.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 -- 172.21.2.104:0/1521305799 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb98809a150 msgr2=0x7fb98809a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb98809a150 0x7fb98809a5b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7fb9880b51f0 tx=0x7fb9880b1b90).stop 2021-07-11T04:00:57.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 -- 172.21.2.104:0/1521305799 shutdown_connections 2021-07-11T04:00:57.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb98809aaa0 0x7fb98813bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:57.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb968053300 0x7fb9680557c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb98809a150 0x7fb98809a5b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 --2- 172.21.2.104:0/1521305799 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb970004860 0x7fb988149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:57.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 -- 172.21.2.104:0/1521305799 >> 172.21.2.104:0/1521305799 conn(0x7fb988062f10 msgr2=0x7fb98813f560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:57.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 -- 172.21.2.104:0/1521305799 shutdown_connections 2021-07-11T04:00:57.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.391+0000 7fb99029a700 1 -- 172.21.2.104:0/1521305799 wait complete. 2021-07-11T04:00:57.396 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T04:00:57.399 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:57.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23cfa42700 1 Processor -- start 2021-07-11T04:00:57.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23cfa42700 1 -- start start 2021-07-11T04:00:57.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23cfa42700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f23c8067180 0x7f23c805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23cfa42700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c80a7a90 con 0x7f23c80a76b0 2021-07-11T04:00:57.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23cfa42700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c8147620 con 0x7f23c80b1970 2021-07-11T04:00:57.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23cfa42700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c81478c0 con 0x7f23c8067180 2021-07-11T04:00:57.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23cd7de700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f23c80a76b0 0x7f23c8066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45494/0 (socket says 172.21.2.104:45494) 2021-07-11T04:00:57.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23cd7de700 1 -- 172.21.2.104:0/564262984 learned_addr learned my addr 172.21.2.104:0/564262984 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:57.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23ccfdd700 1 --2- 172.21.2.104:0/564262984 >> v2:172.21.2.104:3300/0 conn(0x7f23c8067180 0x7f23c805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23ccfdd700 1 -- 172.21.2.104:0/564262984 >> v1:172.21.2.105:6789/0 conn(0x7f23c80b1970 legacy=0x7f23c805fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:57.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23ccfdd700 1 -- 172.21.2.104:0/564262984 >> v1:172.21.2.104:6789/0 conn(0x7f23c80a76b0 legacy=0x7f23c8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23ccfdd700 1 -- 172.21.2.104:0/564262984 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c8147b60 con 0x7f23c8067180 2021-07-11T04:00:57.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23ccfdd700 1 --2- 172.21.2.104:0/564262984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23c8067180 0x7f23c805f6b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f23c806a180 tx=0x7f23bc006df0).ready entity=mon.0 client_cookie=8a34be2df092dc9b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:57.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23bbfff700 1 -- 172.21.2.104:0/564262984 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23bc00d5c0 con 0x7f23c8067180 2021-07-11T04:00:57.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23bbfff700 1 -- 172.21.2.104:0/564262984 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc000e90 con 0x7f23c8067180 2021-07-11T04:00:57.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23bbfff700 1 -- 172.21.2.104:0/564262984 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23bc00d910 con 0x7f23c8067180 2021-07-11T04:00:57.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23cfa42700 1 -- 172.21.2.104:0/564262984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23c8067180 msgr2=0x7f23c805f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.447+0000 7f23cfa42700 1 --2- 172.21.2.104:0/564262984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23c8067180 0x7f23c805f6b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f23c806a180 tx=0x7f23bc006df0).stop 2021-07-11T04:00:57.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 -- 172.21.2.104:0/564262984 shutdown_connections 2021-07-11T04:00:57.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 --2- 172.21.2.104:0/564262984 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23c8067180 0x7f23c805f6b0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 -- 172.21.2.104:0/564262984 >> 172.21.2.104:0/564262984 conn(0x7f23c80b53e0 msgr2=0x7f23c80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:57.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 -- 172.21.2.104:0/564262984 shutdown_connections 2021-07-11T04:00:57.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 -- 172.21.2.104:0/564262984 wait complete. 2021-07-11T04:00:57.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 Processor -- start 2021-07-11T04:00:57.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 -- start start 2021-07-11T04:00:57.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23c8067180 0x7f23c80b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f23c80a76b0 0x7f23c80a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23c80b1970 0x7f23c8156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c80aad90 con 0x7f23c80b1970 2021-07-11T04:00:57.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c80aafc0 con 0x7f23c80a76b0 2021-07-11T04:00:57.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c80ab260 con 0x7f23c8067180 2021-07-11T04:00:57.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cdfdf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23c80b1970 0x7f23c8156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cdfdf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23c80b1970 0x7f23c8156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48736/0 (socket says 172.21.2.104:48736) 2021-07-11T04:00:57.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cdfdf700 1 -- 172.21.2.104:0/1157093456 learned_addr learned my addr 172.21.2.104:0/1157093456 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:57.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cd7de700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23c8067180 0x7f23c80b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23ccfdd700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f23c80a76b0 0x7f23c80a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cdfdf700 1 -- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23c8067180 msgr2=0x7f23c80b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cdfdf700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23c8067180 0x7f23c80b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cdfdf700 1 -- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f23c80a76b0 msgr2=0x7f23c80a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cdfdf700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f23c80a76b0 0x7f23c80a9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cdfdf700 1 -- 172.21.2.104:0/1157093456 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc00b180 con 0x7f23c80b1970 2021-07-11T04:00:57.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cdfdf700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23c80b1970 0x7f23c8156960 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f23c00032b0 tx=0x7f23c00090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:57.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23ba7fc700 1 -- 172.21.2.104:0/1157093456 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23c000d4a0 con 0x7f23c80b1970 2021-07-11T04:00:57.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23ba7fc700 1 -- 172.21.2.104:0/1157093456 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c000d600 con 0x7f23c80b1970 2021-07-11T04:00:57.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 -- 172.21.2.104:0/1157093456 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c80ab500 con 0x7f23c80b1970 2021-07-11T04:00:57.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 -- 172.21.2.104:0/1157093456 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c80a9720 con 0x7f23c80b1970 2021-07-11T04:00:57.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23ba7fc700 1 -- 172.21.2.104:0/1157093456 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23c000d960 con 0x7f23c80b1970 2021-07-11T04:00:57.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.451+0000 7f23cfa42700 1 -- 172.21.2.104:0/1157093456 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a4003410 con 0x7f23c80b1970 2021-07-11T04:00:57.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.455+0000 7f23ba7fc700 1 -- 172.21.2.104:0/1157093456 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f23c0018030 con 0x7f23c80b1970 2021-07-11T04:00:57.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.455+0000 7f23ba7fc700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23a80531f0 0x7f23a80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.455+0000 7f23ba7fc700 1 -- 172.21.2.104:0/1157093456 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f23c006ae40 con 0x7f23c80b1970 2021-07-11T04:00:57.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.455+0000 7f23cd7de700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23a80531f0 0x7f23a80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.455+0000 7f23cd7de700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23a80531f0 0x7f23a80556b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f23c400b000 tx=0x7f23c4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:57.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.455+0000 7f23ba7fc700 1 -- 172.21.2.104:0/1157093456 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f23c00420c0 con 0x7f23c80b1970 2021-07-11T04:00:57.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 -- 172.21.2.104:0/1157093456 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23a4003c40 con 0x7f23c80b1970 2021-07-11T04:00:57.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23ba7fc700 1 -- 172.21.2.104:0/1157093456 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2484 (secure 0 0 0) 0x7f23c00456e0 con 0x7f23c80b1970 2021-07-11T04:00:57.566 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:57.567 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":50,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/3293345379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3293345379},{"type":"v1","addr":"172.21.2.105:6835","nonce":3293345379}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/57099992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":57099992},{"type":"v1","addr":"172.21.2.104:6833","nonce":57099992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:55.667970+0000","modified":"2021-07-11T04:00:56.671726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7287":{"gid":7287,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/871079070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":871079070},{"type":"v1","addr":"172.21.2.104:6835","nonce":871079070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-07-11T04:00:57.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 -- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23a80531f0 msgr2=0x7f23a80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23a80531f0 0x7f23a80556b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f23c400b000 tx=0x7f23c4009040).stop 2021-07-11T04:00:57.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 -- 172.21.2.104:0/1157093456 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23c80b1970 msgr2=0x7f23c8156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23c80b1970 0x7f23c8156960 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f23c00032b0 tx=0x7f23c00090f0).stop 2021-07-11T04:00:57.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 -- 172.21.2.104:0/1157093456 shutdown_connections 2021-07-11T04:00:57.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23a80531f0 0x7f23a80556b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23c80b1970 0x7f23c8156960 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f23c80a76b0 0x7f23c80a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 --2- 172.21.2.104:0/1157093456 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23c8067180 0x7f23c80b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 -- 172.21.2.104:0/1157093456 >> 172.21.2.104:0/1157093456 conn(0x7f23c80b53e0 msgr2=0x7f23c805fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:57.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 -- 172.21.2.104:0/1157093456 shutdown_connections 2021-07-11T04:00:57.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.563+0000 7f23cfa42700 1 -- 172.21.2.104:0/1157093456 wait complete. 2021-07-11T04:00:57.571 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 50 2021-07-11T04:00:57.875 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:00:55.667970+0000', 'modified': '2021-07-11T04:00:56.671726+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7287': {'gid': 7287, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/871079070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 871079070}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 871079070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:00:57.876 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:00:57.877 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:00:57.941 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":7287,"want_state":"up:active","state":"up:active","rank_uptime":2.2673143599999999,"mdsmap_epoch":50,"osdmap_epoch":108,"osdmap_epoch_barrier":107,"uptime":6.4000352859999996} 2021-07-11T04:00:57.942 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 7287, "mdsmap_epoch": 50, "osdmap_epoch": 108, "osdmap_epoch_barrier": 107, "rank_uptime": 2.26731436, "state": "up:active", "uptime": 6.400035286, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:00:57.942 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:57.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f3c5a700 1 Processor -- start 2021-07-11T04:00:57.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f3c5a700 1 -- start start 2021-07-11T04:00:57.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f3c5a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f93ec067180 0x7f93ec05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f3c5a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0b1970 0x7f93ec05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:57.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f3c5a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93ec0a7a90 con 0x7f93ec0a76b0 2021-07-11T04:00:57.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f3c5a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93ec147620 con 0x7f93ec067180 2021-07-11T04:00:57.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f3c5a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec1478c0 con 0x7f93ec0b1970 2021-07-11T04:00:57.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f11f5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f93ec067180 0x7f93ec05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f11f5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f93ec067180 0x7f93ec05f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48742/0 (socket says 172.21.2.104:48742) 2021-07-11T04:00:57.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f11f5700 1 -- 172.21.2.104:0/1820268915 learned_addr learned my addr 172.21.2.104:0/1820268915 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:57.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f21f7700 1 --2- 172.21.2.104:0/1820268915 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0b1970 0x7f93ec05fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:57.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f11f5700 1 -- 172.21.2.104:0/1820268915 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0b1970 msgr2=0x7f93ec05fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:57.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f11f5700 1 --2- 172.21.2.104:0/1820268915 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0b1970 0x7f93ec05fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:57.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f11f5700 1 -- 172.21.2.104:0/1820268915 >> v1:172.21.2.105:6789/0 conn(0x7f93ec0a76b0 legacy=0x7f93ec066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:57.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f11f5700 1 -- 172.21.2.104:0/1820268915 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93ec147b60 con 0x7f93ec067180 2021-07-11T04:00:57.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f21f7700 1 --2- 172.21.2.104:0/1820268915 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0b1970 0x7f93ec05fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:57.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f11f5700 1 --2- 172.21.2.104:0/1820268915 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93ec067180 0x7f93ec05f6b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f93ec06a0e0 tx=0x7f93e0006eb0).ready entity=mon.0 client_cookie=94338e74b8cf7062 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:57.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f09f4700 1 -- 172.21.2.104:0/1820268915 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e000d570 con 0x7f93ec067180 2021-07-11T04:00:57.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f09f4700 1 -- 172.21.2.104:0/1820268915 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e0000e90 con 0x7f93ec067180 2021-07-11T04:00:57.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f09f4700 1 -- 172.21.2.104:0/1820268915 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e000d8c0 con 0x7f93ec067180 2021-07-11T04:00:57.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1820268915 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93ec067180 msgr2=0x7f93ec05f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:57.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.991+0000 7f93f3c5a700 1 --2- 172.21.2.104:0/1820268915 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93ec067180 0x7f93ec05f6b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f93ec06a0e0 tx=0x7f93e0006eb0).stop 2021-07-11T04:00:58.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1820268915 shutdown_connections 2021-07-11T04:00:58.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 --2- 172.21.2.104:0/1820268915 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0b1970 0x7f93ec05fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 --2- 172.21.2.104:0/1820268915 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93ec067180 0x7f93ec05f6b0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1820268915 >> 172.21.2.104:0/1820268915 conn(0x7f93ec0b53e0 msgr2=0x7f93ec0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1820268915 shutdown_connections 2021-07-11T04:00:58.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1820268915 wait complete. 2021-07-11T04:00:58.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 Processor -- start 2021-07-11T04:00:58.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 -- start start 2021-07-11T04:00:58.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93ec067180 0x7f93ec0abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0a76b0 0x7f93ec0aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93ec0b1970 0x7f93ec0ab390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec146330 con 0x7f93ec067180 2021-07-11T04:00:58.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec0ab880 con 0x7f93ec0b1970 2021-07-11T04:00:58.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93ec146090 con 0x7f93ec0a76b0 2021-07-11T04:00:58.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f19f6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93ec067180 0x7f93ec0abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f19f6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93ec067180 0x7f93ec0abee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48746/0 (socket says 172.21.2.104:48746) 2021-07-11T04:00:58.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f19f6700 1 -- 172.21.2.104:0/1050050586 learned_addr learned my addr 172.21.2.104:0/1050050586 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:58.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f11f5700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0a76b0 0x7f93ec0aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f21f7700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93ec0b1970 0x7f93ec0ab390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f19f6700 1 -- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0a76b0 msgr2=0x7f93ec0aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f19f6700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0a76b0 0x7f93ec0aaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f19f6700 1 -- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93ec0b1970 msgr2=0x7f93ec0ab390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f19f6700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93ec0b1970 0x7f93ec0ab390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f19f6700 1 -- 172.21.2.104:0/1050050586 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e000b1e0 con 0x7f93ec067180 2021-07-11T04:00:58.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f11f5700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0a76b0 0x7f93ec0aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:00:58.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f19f6700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93ec067180 0x7f93ec0abee0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f93e8001fa0 tx=0x7f93e8007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93deffd700 1 -- 172.21.2.104:0/1050050586 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e800ce80 con 0x7f93ec067180 2021-07-11T04:00:58.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93deffd700 1 -- 172.21.2.104:0/1050050586 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e8007240 con 0x7f93ec067180 2021-07-11T04:00:58.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1050050586 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93ec0ae4c0 con 0x7f93ec067180 2021-07-11T04:00:58.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1050050586 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93ec0b4770 con 0x7f93ec067180 2021-07-11T04:00:58.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93deffd700 1 -- 172.21.2.104:0/1050050586 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f93e8006510 con 0x7f93ec067180 2021-07-11T04:00:58.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1050050586 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93ec057350 con 0x7f93ec067180 2021-07-11T04:00:58.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93deffd700 1 -- 172.21.2.104:0/1050050586 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f93e8010030 con 0x7f93ec067180 2021-07-11T04:00:58.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.995+0000 7f93deffd700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93cc0532b0 0x7f93cc055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.999+0000 7f93deffd700 1 -- 172.21.2.104:0/1050050586 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f93e806b180 con 0x7f93ec067180 2021-07-11T04:00:58.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.999+0000 7f93f11f5700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93cc0532b0 0x7f93cc055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.999+0000 7f93f11f5700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93cc0532b0 0x7f93cc055770 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f93e0016000 tx=0x7f93e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:57.999+0000 7f93deffd700 1 -- 172.21.2.104:0/1050050586 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f93e80424c0 con 0x7f93ec067180 2021-07-11T04:00:58.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.079+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1050050586 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93ec0ae960 con 0x7f93ec067180 2021-07-11T04:00:58.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.079+0000 7f93deffd700 1 -- 172.21.2.104:0/1050050586 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f93e8045ae0 con 0x7f93ec067180 2021-07-11T04:00:58.083 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:58.083 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":108,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:56.672922+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:53.655121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:54.748677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:58.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93cc0532b0 msgr2=0x7f93cc055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93cc0532b0 0x7f93cc055770 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f93e0016000 tx=0x7f93e0015000).stop 2021-07-11T04:00:58.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1050050586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93ec067180 msgr2=0x7f93ec0abee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93ec067180 0x7f93ec0abee0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f93e8001fa0 tx=0x7f93e8007c30).stop 2021-07-11T04:00:58.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1050050586 shutdown_connections 2021-07-11T04:00:58.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f93ec0b1970 0x7f93ec0ab390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f93ec0a76b0 0x7f93ec0aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f93cc0532b0 0x7f93cc055770 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 --2- 172.21.2.104:0/1050050586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f93ec067180 0x7f93ec0abee0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1050050586 >> 172.21.2.104:0/1050050586 conn(0x7f93ec0b53e0 msgr2=0x7f93ec05fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1050050586 shutdown_connections 2021-07-11T04:00:58.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.083+0000 7f93f3c5a700 1 -- 172.21.2.104:0/1050050586 wait complete. 2021-07-11T04:00:58.090 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:00:53.655121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:00:54.748677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:00:58.091 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:00:58.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 Processor -- start 2021-07-11T04:00:58.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 -- start start 2021-07-11T04:00:58.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31b40b1ec0 0x7f31b40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f31b40b22a0 0x7f31b4141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b40a77d0 con 0x7f31b40affb0 2021-07-11T04:00:58.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b40b03f0 con 0x7f31b40b22a0 2021-07-11T04:00:58.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40b0650 con 0x7f31b40b1ec0 2021-07-11T04:00:58.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b37fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f31b40b22a0 0x7f31b4141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b37fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f31b40b22a0 0x7f31b4141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48758/0 (socket says 172.21.2.104:48758) 2021-07-11T04:00:58.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b37fe700 1 -- 172.21.2.104:0/3964101200 learned_addr learned my addr 172.21.2.104:0/3964101200 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:58.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b37fe700 1 -- 172.21.2.104:0/3964101200 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31b40b1ec0 msgr2=0x7f31b40b12d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:58.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b37fe700 1 --2- 172.21.2.104:0/3964101200 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31b40b1ec0 0x7f31b40b12d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b37fe700 1 -- 172.21.2.104:0/3964101200 >> v1:172.21.2.105:6789/0 conn(0x7f31b40affb0 legacy=0x7f31b40a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:58.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b37fe700 1 -- 172.21.2.104:0/3964101200 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b40b08f0 con 0x7f31b40b22a0 2021-07-11T04:00:58.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b37fe700 1 --2- 172.21.2.104:0/3964101200 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31b40b22a0 0x7f31b4141250 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f31b406a0f0 tx=0x7f31ac009290).ready entity=mon.0 client_cookie=1b7485378fda465b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b1ffb700 1 -- 172.21.2.104:0/3964101200 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31ac00b860 con 0x7f31b40b22a0 2021-07-11T04:00:58.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b1ffb700 1 -- 172.21.2.104:0/3964101200 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ac00b9c0 con 0x7f31b40b22a0 2021-07-11T04:00:58.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b1ffb700 1 -- 172.21.2.104:0/3964101200 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31ac00bb70 con 0x7f31b40b22a0 2021-07-11T04:00:58.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 -- 172.21.2.104:0/3964101200 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31b40b22a0 msgr2=0x7f31b4141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 --2- 172.21.2.104:0/3964101200 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31b40b22a0 0x7f31b4141250 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f31b406a0f0 tx=0x7f31ac009290).stop 2021-07-11T04:00:58.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 -- 172.21.2.104:0/3964101200 shutdown_connections 2021-07-11T04:00:58.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 --2- 172.21.2.104:0/3964101200 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31b40b22a0 0x7f31b4141250 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 --2- 172.21.2.104:0/3964101200 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31b40b1ec0 0x7f31b40b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 -- 172.21.2.104:0/3964101200 >> 172.21.2.104:0/3964101200 conn(0x7f31b405f6b0 msgr2=0x7f31b405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 -- 172.21.2.104:0/3964101200 shutdown_connections 2021-07-11T04:00:58.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.143+0000 7f31b968c700 1 -- 172.21.2.104:0/3964101200 wait complete. 2021-07-11T04:00:58.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b968c700 1 Processor -- start 2021-07-11T04:00:58.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b968c700 1 -- start start 2021-07-11T04:00:58.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b968c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31b40affb0 0x7f31b4140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b968c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31b40b1ec0 0x7f31b415aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b968c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31b40b22a0 0x7f31b4157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b968c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40adee0 con 0x7f31b40affb0 2021-07-11T04:00:58.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b968c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40ad9a0 con 0x7f31b40b1ec0 2021-07-11T04:00:58.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b968c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40adc40 con 0x7f31b40b22a0 2021-07-11T04:00:58.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31b40affb0 0x7f31b4140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31b40affb0 0x7f31b4140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48760/0 (socket says 172.21.2.104:48760) 2021-07-11T04:00:58.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b2ffd700 1 -- 172.21.2.104:0/3324225643 learned_addr learned my addr 172.21.2.104:0/3324225643 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:58.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b2ffd700 1 -- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31b40b22a0 msgr2=0x7f31b4157140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:58.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b2ffd700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31b40b22a0 0x7f31b4157140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b2ffd700 1 -- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31b40b1ec0 msgr2=0x7f31b415aa40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:00:58.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b2ffd700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31b40b1ec0 0x7f31b415aa40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b2ffd700 1 -- 172.21.2.104:0/3324225643 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac009530 con 0x7f31b40affb0 2021-07-11T04:00:58.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b2ffd700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31b40affb0 0x7f31b4140530 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f31a8002fb0 tx=0x7f31a8006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31a3fff700 1 -- 172.21.2.104:0/3324225643 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a8013460 con 0x7f31b40affb0 2021-07-11T04:00:58.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31a3fff700 1 -- 172.21.2.104:0/3324225643 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a80135c0 con 0x7f31b40affb0 2021-07-11T04:00:58.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31a3fff700 1 -- 172.21.2.104:0/3324225643 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f31a8013920 con 0x7f31b40affb0 2021-07-11T04:00:58.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b968c700 1 -- 172.21.2.104:0/3324225643 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b4156ca0 con 0x7f31b40affb0 2021-07-11T04:00:58.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b968c700 1 -- 172.21.2.104:0/3324225643 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b40ae560 con 0x7f31b40affb0 2021-07-11T04:00:58.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b968c700 1 -- 172.21.2.104:0/3324225643 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b4156e30 con 0x7f31b40affb0 2021-07-11T04:00:58.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31a3fff700 1 -- 172.21.2.104:0/3324225643 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f31a800f070 con 0x7f31b40affb0 2021-07-11T04:00:58.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31a3fff700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3190053270 0x7f3190055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31a3fff700 1 -- 172.21.2.104:0/3324225643 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f31a806ab90 con 0x7f31b40affb0 2021-07-11T04:00:58.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.147+0000 7f31b27fc700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3190053270 0x7f3190055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.151+0000 7f31b27fc700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3190053270 0x7f3190055730 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f31a4002f90 tx=0x7f31a4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.151+0000 7f31a3fff700 1 -- 172.21.2.104:0/3324225643 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f31a8041ed0 con 0x7f31b40affb0 2021-07-11T04:00:58.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.231+0000 7f31b968c700 1 -- 172.21.2.104:0/3324225643 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f31b4057350 con 0x7f31b40affb0 2021-07-11T04:00:58.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.231+0000 7f31a3fff700 1 -- 172.21.2.104:0/3324225643 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v108) v1 ==== 110+0+10 (secure 0 0 0) 0x7f31a8018030 con 0x7f31b40affb0 2021-07-11T04:00:58.234 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:00:58.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 -- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3190053270 msgr2=0x7f3190055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3190053270 0x7f3190055730 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f31a4002f90 tx=0x7f31a4006930).stop 2021-07-11T04:00:58.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 -- 172.21.2.104:0/3324225643 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31b40affb0 msgr2=0x7f31b4140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31b40affb0 0x7f31b4140530 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f31a8002fb0 tx=0x7f31a8006e10).stop 2021-07-11T04:00:58.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 -- 172.21.2.104:0/3324225643 shutdown_connections 2021-07-11T04:00:58.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f31b40b22a0 0x7f31b4157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f31b40b1ec0 0x7f31b415aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3190053270 0x7f3190055730 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 --2- 172.21.2.104:0/3324225643 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f31b40affb0 0x7f31b4140530 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 -- 172.21.2.104:0/3324225643 >> 172.21.2.104:0/3324225643 conn(0x7f31b405f6b0 msgr2=0x7f31b41477a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 -- 172.21.2.104:0/3324225643 shutdown_connections 2021-07-11T04:00:58.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.235+0000 7f31b968c700 1 -- 172.21.2.104:0/3324225643 wait complete. 2021-07-11T04:00:58.242 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:00:58.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 Processor -- start 2021-07-11T04:00:58.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- start start 2021-07-11T04:00:58.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f06f80b1ec0 0x7f06f80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f80b0420 con 0x7f06f80affb0 2021-07-11T04:00:58.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06f80a77d0 con 0x7f06f80b22a0 2021-07-11T04:00:58.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06f80b0650 con 0x7f06f80b1ec0 2021-07-11T04:00:58.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f6ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f06f80b1ec0 0x7f06f80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f67fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f06f80affb0 0x7f06f80a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45540/0 (socket says 172.21.2.104:45540) 2021-07-11T04:00:58.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f67fc700 1 -- 172.21.2.104:0/720587904 learned_addr learned my addr 172.21.2.104:0/720587904 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:58.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f6ffd700 1 -- 172.21.2.104:0/720587904 >> v1:172.21.2.105:6789/0 conn(0x7f06f80b22a0 legacy=0x7f06f8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:58.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f6ffd700 1 -- 172.21.2.104:0/720587904 >> v1:172.21.2.104:6789/0 conn(0x7f06f80affb0 legacy=0x7f06f80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f5ffb700 1 -- 172.21.2.104:0/720587904 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672626228 0 0) 0x7f06f80b0420 con 0x7f06f80affb0 2021-07-11T04:00:58.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f6ffd700 1 -- 172.21.2.104:0/720587904 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f80b08f0 con 0x7f06f80b1ec0 2021-07-11T04:00:58.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f6ffd700 1 --2- 172.21.2.104:0/720587904 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06f80b1ec0 0x7f06f80b12d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f06f80b2f50 tx=0x7f06e8006df0).ready entity=mon.0 client_cookie=f742c0da21ff1cc7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f5ffb700 1 -- 172.21.2.104:0/720587904 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06e800d5c0 con 0x7f06f80b1ec0 2021-07-11T04:00:58.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f5ffb700 1 -- 172.21.2.104:0/720587904 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e8000e90 con 0x7f06f80b1ec0 2021-07-11T04:00:58.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f5ffb700 1 -- 172.21.2.104:0/720587904 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06e800d910 con 0x7f06f80b1ec0 2021-07-11T04:00:58.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- 172.21.2.104:0/720587904 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06f80b1ec0 msgr2=0x7f06f80b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 --2- 172.21.2.104:0/720587904 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06f80b1ec0 0x7f06f80b12d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f06f80b2f50 tx=0x7f06e8006df0).stop 2021-07-11T04:00:58.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- 172.21.2.104:0/720587904 shutdown_connections 2021-07-11T04:00:58.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 --2- 172.21.2.104:0/720587904 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06f80b1ec0 0x7f06f80b12d0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- 172.21.2.104:0/720587904 >> 172.21.2.104:0/720587904 conn(0x7f06f805f6b0 msgr2=0x7f06f805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- 172.21.2.104:0/720587904 shutdown_connections 2021-07-11T04:00:58.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- 172.21.2.104:0/720587904 wait complete. 2021-07-11T04:00:58.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 Processor -- start 2021-07-11T04:00:58.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- start start 2021-07-11T04:00:58.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f06f80affb0 0x7f06f815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f06f80b1ec0 0x7f06f81405b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06f80b22a0 0x7f06f8156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f815a1a0 con 0x7f06f80b22a0 2021-07-11T04:00:58.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f815a440 con 0x7f06f80affb0 2021-07-11T04:00:58.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06fd6bf700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f815a6e0 con 0x7f06f80b1ec0 2021-07-11T04:00:58.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06f80b22a0 0x7f06f8156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06f80b22a0 0x7f06f8156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48776/0 (socket says 172.21.2.104:48776) 2021-07-11T04:00:58.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.295+0000 7f06f77fe700 1 -- 172.21.2.104:0/1226891290 learned_addr learned my addr 172.21.2.104:0/1226891290 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:58.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06f77fe700 1 -- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f06f80b1ec0 msgr2=0x7f06f81405b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06f77fe700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f06f80b1ec0 0x7f06f81405b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06f77fe700 1 -- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f06f80affb0 msgr2=0x7f06f815ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06f77fe700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f06f80affb0 0x7f06f815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06f77fe700 1 -- 172.21.2.104:0/1226891290 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e800b180 con 0x7f06f80b22a0 2021-07-11T04:00:58.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06f77fe700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06f80b22a0 0x7f06f8156ad0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f06f0003290 tx=0x7f06f00090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06effff700 1 -- 172.21.2.104:0/1226891290 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06f000f510 con 0x7f06f80b22a0 2021-07-11T04:00:58.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06effff700 1 -- 172.21.2.104:0/1226891290 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f000f670 con 0x7f06f80b22a0 2021-07-11T04:00:58.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06fd6bf700 1 -- 172.21.2.104:0/1226891290 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06f80b4470 con 0x7f06f80b22a0 2021-07-11T04:00:58.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06fd6bf700 1 -- 172.21.2.104:0/1226891290 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f80ad180 con 0x7f06f80b22a0 2021-07-11T04:00:58.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06effff700 1 -- 172.21.2.104:0/1226891290 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06f000f7e0 con 0x7f06f80b22a0 2021-07-11T04:00:58.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06fd6bf700 1 -- 172.21.2.104:0/1226891290 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f815a8a0 con 0x7f06f80b22a0 2021-07-11T04:00:58.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06effff700 1 -- 172.21.2.104:0/1226891290 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f06f000f070 con 0x7f06f80b22a0 2021-07-11T04:00:58.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06effff700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f06dc053240 0x7f06dc055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06effff700 1 -- 172.21.2.104:0/1226891290 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f06f006b0c0 con 0x7f06f80b22a0 2021-07-11T04:00:58.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06f6ffd700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f06dc053240 0x7f06dc055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.299+0000 7f06f6ffd700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f06dc053240 0x7f06dc055700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f06f80b2f50 tx=0x7f06e8012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.303+0000 7f06effff700 1 -- 172.21.2.104:0/1226891290 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f06f00422f0 con 0x7f06f80b22a0 2021-07-11T04:00:58.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 -- 172.21.2.104:0/1226891290 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f06f8057350 con 0x7f06f80b22a0 2021-07-11T04:00:58.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06effff700 1 -- 172.21.2.104:0/1226891290 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7f06f0045910 con 0x7f06f80b22a0 2021-07-11T04:00:58.385 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:58.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 -- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f06dc053240 msgr2=0x7f06dc055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f06dc053240 0x7f06dc055700 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f06f80b2f50 tx=0x7f06e8012000).stop 2021-07-11T04:00:58.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 -- 172.21.2.104:0/1226891290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06f80b22a0 msgr2=0x7f06f8156ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06f80b22a0 0x7f06f8156ad0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f06f0003290 tx=0x7f06f00090f0).stop 2021-07-11T04:00:58.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 -- 172.21.2.104:0/1226891290 shutdown_connections 2021-07-11T04:00:58.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f06dc053240 0x7f06dc055700 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f06f80b22a0 0x7f06f8156ad0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f06f80b1ec0 0x7f06f81405b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 --2- 172.21.2.104:0/1226891290 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f06f80affb0 0x7f06f815ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 -- 172.21.2.104:0/1226891290 >> 172.21.2.104:0/1226891290 conn(0x7f06f805f6b0 msgr2=0x7f06f813e380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 -- 172.21.2.104:0/1226891290 shutdown_connections 2021-07-11T04:00:58.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.383+0000 7f06fd6bf700 1 -- 172.21.2.104:0/1226891290 wait complete. 2021-07-11T04:00:58.392 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:00:58.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4698602700 1 Processor -- start 2021-07-11T04:00:58.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4698602700 1 -- start start 2021-07-11T04:00:58.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4698602700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46900603d0 0x7f46900aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4698602700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f469013f790 0x7f469013fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4698602700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4690061990 con 0x7f46900a9ec0 2021-07-11T04:00:58.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4698602700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4690140110 con 0x7f469013f790 2021-07-11T04:00:58.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4698602700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4690061c30 con 0x7f46900603d0 2021-07-11T04:00:58.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4696b9f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f469013f790 0x7f469013fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4696b9f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f469013f790 0x7f469013fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48786/0 (socket says 172.21.2.104:48786) 2021-07-11T04:00:58.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4696b9f700 1 -- 172.21.2.104:0/1256114570 learned_addr learned my addr 172.21.2.104:0/1256114570 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:58.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4696b9f700 1 -- 172.21.2.104:0/1256114570 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46900603d0 msgr2=0x7f46900aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:58.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4696b9f700 1 --2- 172.21.2.104:0/1256114570 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46900603d0 0x7f46900aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4696b9f700 1 -- 172.21.2.104:0/1256114570 >> v1:172.21.2.105:6789/0 conn(0x7f46900a9ec0 legacy=0x7f469013ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:58.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4696b9f700 1 -- 172.21.2.104:0/1256114570 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4690060b90 con 0x7f469013f790 2021-07-11T04:00:58.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4696b9f700 1 --2- 172.21.2.104:0/1256114570 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f469013f790 0x7f469013fc20 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f469005fd90 tx=0x7f4688009290).ready entity=mon.0 client_cookie=2c777861fd23a33d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f469539c700 1 -- 172.21.2.104:0/1256114570 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f468800ba50 con 0x7f469013f790 2021-07-11T04:00:58.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f469539c700 1 -- 172.21.2.104:0/1256114570 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468800bbb0 con 0x7f469013f790 2021-07-11T04:00:58.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f469539c700 1 -- 172.21.2.104:0/1256114570 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f468800f470 con 0x7f469013f790 2021-07-11T04:00:58.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4698602700 1 -- 172.21.2.104:0/1256114570 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f469013f790 msgr2=0x7f469013fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.443+0000 7f4698602700 1 --2- 172.21.2.104:0/1256114570 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f469013f790 0x7f469013fc20 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f469005fd90 tx=0x7f4688009290).stop 2021-07-11T04:00:58.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 -- 172.21.2.104:0/1256114570 shutdown_connections 2021-07-11T04:00:58.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 --2- 172.21.2.104:0/1256114570 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f469013f790 0x7f469013fc20 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 --2- 172.21.2.104:0/1256114570 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46900603d0 0x7f46900aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 -- 172.21.2.104:0/1256114570 >> 172.21.2.104:0/1256114570 conn(0x7f46900b2970 msgr2=0x7f46900b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 -- 172.21.2.104:0/1256114570 shutdown_connections 2021-07-11T04:00:58.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 -- 172.21.2.104:0/1256114570 wait complete. 2021-07-11T04:00:58.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 Processor -- start 2021-07-11T04:00:58.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 -- start start 2021-07-11T04:00:58.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46900603d0 0x7f46900ada90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46900a9ec0 0x7f4690142e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f469013f790 0x7f469015a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46900aeae0 con 0x7f469013f790 2021-07-11T04:00:58.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46900ae840 con 0x7f46900a9ec0 2021-07-11T04:00:58.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46900b0000 con 0x7f46900603d0 2021-07-11T04:00:58.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4696b9f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f469013f790 0x7f469015a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4696b9f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f469013f790 0x7f469015a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48792/0 (socket says 172.21.2.104:48792) 2021-07-11T04:00:58.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4696b9f700 1 -- 172.21.2.104:0/3914159168 learned_addr learned my addr 172.21.2.104:0/3914159168 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:58.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4696b9f700 1 -- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46900603d0 msgr2=0x7f46900ada90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4696b9f700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46900603d0 0x7f46900ada90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4696b9f700 1 -- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46900a9ec0 msgr2=0x7f4690142e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4696b9f700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46900a9ec0 0x7f4690142e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4696b9f700 1 -- 172.21.2.104:0/3914159168 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4688009530 con 0x7f469013f790 2021-07-11T04:00:58.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4696b9f700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f469013f790 0x7f469015a150 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f469005fd90 tx=0x7f4688018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f46837fe700 1 -- 172.21.2.104:0/3914159168 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4688017e90 con 0x7f469013f790 2021-07-11T04:00:58.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f46837fe700 1 -- 172.21.2.104:0/3914159168 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46880032f0 con 0x7f469013f790 2021-07-11T04:00:58.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 -- 172.21.2.104:0/3914159168 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46900b02a0 con 0x7f469013f790 2021-07-11T04:00:58.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f46837fe700 1 -- 172.21.2.104:0/3914159168 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46880205f0 con 0x7f469013f790 2021-07-11T04:00:58.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 -- 172.21.2.104:0/3914159168 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4690143360 con 0x7f469013f790 2021-07-11T04:00:58.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f4698602700 1 -- 172.21.2.104:0/3914159168 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46900b0430 con 0x7f469013f790 2021-07-11T04:00:58.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f46837fe700 1 -- 172.21.2.104:0/3914159168 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f468801f3a0 con 0x7f469013f790 2021-07-11T04:00:58.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f46837fe700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46700532b0 0x7f4670055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.447+0000 7f46837fe700 1 -- 172.21.2.104:0/3914159168 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f468806c530 con 0x7f469013f790 2021-07-11T04:00:58.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.451+0000 7f469639e700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46700532b0 0x7f4670055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.451+0000 7f469639e700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46700532b0 0x7f4670055770 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f4684000f70 tx=0x7f46840077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.451+0000 7f46837fe700 1 -- 172.21.2.104:0/3914159168 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4688043810 con 0x7f469013f790 2021-07-11T04:00:58.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 -- 172.21.2.104:0/3914159168 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4690057350 con 0x7f469013f790 2021-07-11T04:00:58.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f46837fe700 1 -- 172.21.2.104:0/3914159168 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4688046e30 con 0x7f469013f790 2021-07-11T04:00:58.536 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:58.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 -- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46700532b0 msgr2=0x7f4670055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46700532b0 0x7f4670055770 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f4684000f70 tx=0x7f46840077f0).stop 2021-07-11T04:00:58.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 -- 172.21.2.104:0/3914159168 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f469013f790 msgr2=0x7f469015a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f469013f790 0x7f469015a150 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f469005fd90 tx=0x7f4688018000).stop 2021-07-11T04:00:58.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 -- 172.21.2.104:0/3914159168 shutdown_connections 2021-07-11T04:00:58.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46700532b0 0x7f4670055770 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f469013f790 0x7f469015a150 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46900a9ec0 0x7f4690142e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 --2- 172.21.2.104:0/3914159168 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46900603d0 0x7f46900ada90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 -- 172.21.2.104:0/3914159168 >> 172.21.2.104:0/3914159168 conn(0x7f46900b2970 msgr2=0x7f46900b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 -- 172.21.2.104:0/3914159168 shutdown_connections 2021-07-11T04:00:58.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.535+0000 7f4698602700 1 -- 172.21.2.104:0/3914159168 wait complete. 2021-07-11T04:00:58.543 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:00:58.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f666f0b8700 1 Processor -- start 2021-07-11T04:00:58.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f666f0b8700 1 -- start start 2021-07-11T04:00:58.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f666f0b8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f66680b12d0 0x7f66680affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f666f0b8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f66680b22a0 0x7f6668141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f666f0b8700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66680a77d0 con 0x7f66680b1ec0 2021-07-11T04:00:58.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f666f0b8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66680b1740 con 0x7f66680b12d0 2021-07-11T04:00:58.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f666f0b8700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66680b0650 con 0x7f66680b22a0 2021-07-11T04:00:58.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f666ce54700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f66680b1ec0 0x7f66680a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45564/0 (socket says 172.21.2.104:45564) 2021-07-11T04:00:58.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f666ce54700 1 -- 172.21.2.104:0/1806617561 learned_addr learned my addr 172.21.2.104:0/1806617561 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:58.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f6667fff700 1 --2- 172.21.2.104:0/1806617561 >> v2:172.21.2.104:3300/0 conn(0x7f66680b12d0 0x7f66680affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f6667fff700 1 -- 172.21.2.104:0/1806617561 >> v2:172.21.2.105:3300/0 conn(0x7f66680b22a0 msgr2=0x7f6668141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:58.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f6667fff700 1 --2- 172.21.2.104:0/1806617561 >> v2:172.21.2.105:3300/0 conn(0x7f66680b22a0 0x7f6668141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f6667fff700 1 -- 172.21.2.104:0/1806617561 >> v1:172.21.2.104:6789/0 conn(0x7f66680b1ec0 legacy=0x7f66680a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f6667fff700 1 -- 172.21.2.104:0/1806617561 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66680b08f0 con 0x7f66680b12d0 2021-07-11T04:00:58.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f6667fff700 1 --2- 172.21.2.104:0/1806617561 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66680b12d0 0x7f66680affb0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f6650001110 tx=0x7f6658006df0).ready entity=mon.0 client_cookie=e0e078985e9c5cbd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f66677fe700 1 -- 172.21.2.104:0/1806617561 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665800d570 con 0x7f66680b12d0 2021-07-11T04:00:58.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f66677fe700 1 -- 172.21.2.104:0/1806617561 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6658000e90 con 0x7f66680b12d0 2021-07-11T04:00:58.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f66677fe700 1 -- 172.21.2.104:0/1806617561 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f665800d8c0 con 0x7f66680b12d0 2021-07-11T04:00:58.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f666f0b8700 1 -- 172.21.2.104:0/1806617561 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66680b12d0 msgr2=0x7f66680affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.595+0000 7f666f0b8700 1 --2- 172.21.2.104:0/1806617561 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66680b12d0 0x7f66680affb0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f6650001110 tx=0x7f6658006df0).stop 2021-07-11T04:00:58.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 -- 172.21.2.104:0/1806617561 shutdown_connections 2021-07-11T04:00:58.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 --2- 172.21.2.104:0/1806617561 >> v2:172.21.2.105:3300/0 conn(0x7f66680b22a0 0x7f6668141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 --2- 172.21.2.104:0/1806617561 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66680b12d0 0x7f66680affb0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 -- 172.21.2.104:0/1806617561 >> 172.21.2.104:0/1806617561 conn(0x7f666805f6b0 msgr2=0x7f666805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 -- 172.21.2.104:0/1806617561 shutdown_connections 2021-07-11T04:00:58.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 -- 172.21.2.104:0/1806617561 wait complete. 2021-07-11T04:00:58.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 Processor -- start 2021-07-11T04:00:58.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 -- start start 2021-07-11T04:00:58.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66680b12d0 0x7f6668148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66680b1ec0 0x7f6668064120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66680b22a0 0x7f6668149680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666814bb70 con 0x7f66680b12d0 2021-07-11T04:00:58.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6668064610 con 0x7f66680b1ec0 2021-07-11T04:00:58.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6668064840 con 0x7f66680b22a0 2021-07-11T04:00:58.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666ce54700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66680b12d0 0x7f6668148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666ce54700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66680b12d0 0x7f6668148320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48804/0 (socket says 172.21.2.104:48804) 2021-07-11T04:00:58.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666ce54700 1 -- 172.21.2.104:0/4227851214 learned_addr learned my addr 172.21.2.104:0/4227851214 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:58.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666ce54700 1 -- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66680b22a0 msgr2=0x7f6668149680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666ce54700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66680b22a0 0x7f6668149680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666ce54700 1 -- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66680b1ec0 msgr2=0x7f6668064120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666ce54700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66680b1ec0 0x7f6668064120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666ce54700 1 -- 172.21.2.104:0/4227851214 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665800b180 con 0x7f66680b12d0 2021-07-11T04:00:58.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666ce54700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66680b12d0 0x7f6668148320 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f66500011d0 tx=0x7f6650007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f6665ffb700 1 -- 172.21.2.104:0/4227851214 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66500072e0 con 0x7f66680b12d0 2021-07-11T04:00:58.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f6665ffb700 1 -- 172.21.2.104:0/4227851214 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6650007440 con 0x7f66680b12d0 2021-07-11T04:00:58.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f6665ffb700 1 -- 172.21.2.104:0/4227851214 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6650005800 con 0x7f66680b12d0 2021-07-11T04:00:58.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 -- 172.21.2.104:0/4227851214 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666814b630 con 0x7f66680b12d0 2021-07-11T04:00:58.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 -- 172.21.2.104:0/4227851214 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6668148b30 con 0x7f66680b12d0 2021-07-11T04:00:58.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f666f0b8700 1 -- 172.21.2.104:0/4227851214 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6668057350 con 0x7f66680b12d0 2021-07-11T04:00:58.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f6665ffb700 1 -- 172.21.2.104:0/4227851214 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6650010070 con 0x7f66680b12d0 2021-07-11T04:00:58.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f6665ffb700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f664c0531f0 0x7f664c0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.599+0000 7f6665ffb700 1 -- 172.21.2.104:0/4227851214 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f665006ada0 con 0x7f66680b12d0 2021-07-11T04:00:58.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.603+0000 7f6667fff700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f664c0531f0 0x7f664c0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.603+0000 7f6667fff700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f664c0531f0 0x7f664c0556b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f6650001110 tx=0x7f6658014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.603+0000 7f6665ffb700 1 -- 172.21.2.104:0/4227851214 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6650041fd0 con 0x7f66680b12d0 2021-07-11T04:00:58.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.683+0000 7f666f0b8700 1 -- 172.21.2.104:0/4227851214 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f66680ac760 con 0x7f66680b12d0 2021-07-11T04:00:58.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f6665ffb700 1 -- 172.21.2.104:0/4227851214 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6650018020 con 0x7f66680b12d0 2021-07-11T04:00:58.687 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:00:58.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 -- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f664c0531f0 msgr2=0x7f664c0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f664c0531f0 0x7f664c0556b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f6650001110 tx=0x7f6658014000).stop 2021-07-11T04:00:58.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 -- 172.21.2.104:0/4227851214 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66680b12d0 msgr2=0x7f6668148320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66680b12d0 0x7f6668148320 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f66500011d0 tx=0x7f6650007bc0).stop 2021-07-11T04:00:58.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 -- 172.21.2.104:0/4227851214 shutdown_connections 2021-07-11T04:00:58.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66680b22a0 0x7f6668149680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66680b1ec0 0x7f6668064120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:58.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f664c0531f0 0x7f664c0556b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 --2- 172.21.2.104:0/4227851214 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66680b12d0 0x7f6668148320 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 -- 172.21.2.104:0/4227851214 >> 172.21.2.104:0/4227851214 conn(0x7f666805f6b0 msgr2=0x7f66680af630 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 -- 172.21.2.104:0/4227851214 shutdown_connections 2021-07-11T04:00:58.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.687+0000 7f666f0b8700 1 -- 172.21.2.104:0/4227851214 wait complete. 2021-07-11T04:00:58.695 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:00:58.695 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:00:58.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.743+0000 7f9d7f4cf700 1 Processor -- start 2021-07-11T04:00:58.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.743+0000 7f9d7f4cf700 1 -- start start 2021-07-11T04:00:58.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9d780a76b0 0x7f9d78066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d780a7a90 con 0x7f9d780b0060 2021-07-11T04:00:58.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d78147620 con 0x7f9d780b1970 2021-07-11T04:00:58.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d781478c0 con 0x7f9d780a76b0 2021-07-11T04:00:58.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7da6c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f9d780b0060 0x7f9d78151f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45582/0 (socket says 172.21.2.104:45582) 2021-07-11T04:00:58.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7da6c700 1 -- 172.21.2.104:0/1900606345 learned_addr learned my addr 172.21.2.104:0/1900606345 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:58.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7d26b700 1 --2- 172.21.2.104:0/1900606345 >> v2:172.21.2.104:3300/0 conn(0x7f9d780a76b0 0x7f9d78066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7d26b700 1 -- 172.21.2.104:0/1900606345 >> v1:172.21.2.105:6789/0 conn(0x7f9d780b1970 legacy=0x7f9d7805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:58.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7d26b700 1 -- 172.21.2.104:0/1900606345 >> v1:172.21.2.104:6789/0 conn(0x7f9d780b0060 legacy=0x7f9d78151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7d26b700 1 -- 172.21.2.104:0/1900606345 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d78147b60 con 0x7f9d780a76b0 2021-07-11T04:00:58.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7d26b700 1 --2- 172.21.2.104:0/1900606345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d780a76b0 0x7f9d78066a70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f9d70001490 tx=0x7f9d74006df0).ready entity=mon.0 client_cookie=9b0f1c888dfa838d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d6bfff700 1 -- 172.21.2.104:0/1900606345 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d7400d4a0 con 0x7f9d780a76b0 2021-07-11T04:00:58.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d6bfff700 1 -- 172.21.2.104:0/1900606345 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d74000e90 con 0x7f9d780a76b0 2021-07-11T04:00:58.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d6bfff700 1 -- 172.21.2.104:0/1900606345 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d7400d7f0 con 0x7f9d780a76b0 2021-07-11T04:00:58.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/1900606345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d780a76b0 msgr2=0x7f9d78066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 --2- 172.21.2.104:0/1900606345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d780a76b0 0x7f9d78066a70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f9d70001490 tx=0x7f9d74006df0).stop 2021-07-11T04:00:58.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/1900606345 shutdown_connections 2021-07-11T04:00:58.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 --2- 172.21.2.104:0/1900606345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d780a76b0 0x7f9d78066a70 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/1900606345 >> 172.21.2.104:0/1900606345 conn(0x7f9d780b53e0 msgr2=0x7f9d780b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/1900606345 shutdown_connections 2021-07-11T04:00:58.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/1900606345 wait complete. 2021-07-11T04:00:58.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 Processor -- start 2021-07-11T04:00:58.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- start start 2021-07-11T04:00:58.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9d780a76b0 0x7f9d78066910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d780b0060 0x7f9d780ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9d780b1970 0x7f9d780aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780a96d0 con 0x7f9d780b0060 2021-07-11T04:00:58.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780a9970 con 0x7f9d780b1970 2021-07-11T04:00:58.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7f4cf700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d780ae740 con 0x7f9d780a76b0 2021-07-11T04:00:58.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7ca6a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d780b0060 0x7f9d780ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7ca6a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d780b0060 0x7f9d780ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48818/0 (socket says 172.21.2.104:48818) 2021-07-11T04:00:58.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7ca6a700 1 -- 172.21.2.104:0/4254708385 learned_addr learned my addr 172.21.2.104:0/4254708385 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:58.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7d26b700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9d780a76b0 0x7f9d78066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7da6c700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9d780b1970 0x7f9d780aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7ca6a700 1 -- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9d780a76b0 msgr2=0x7f9d78066910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7ca6a700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9d780a76b0 0x7f9d78066910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7ca6a700 1 -- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9d780b1970 msgr2=0x7f9d780aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7ca6a700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9d780b1970 0x7f9d780aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.747+0000 7f9d7ca6a700 1 -- 172.21.2.104:0/4254708385 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7400b180 con 0x7f9d780b0060 2021-07-11T04:00:58.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d7ca6a700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d780b0060 0x7f9d780ada00 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f9d6c002f10 tx=0x7f9d6c006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d6a7fc700 1 -- 172.21.2.104:0/4254708385 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d6c009e00 con 0x7f9d780b0060 2021-07-11T04:00:58.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d6a7fc700 1 -- 172.21.2.104:0/4254708385 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6c00f510 con 0x7f9d780b0060 2021-07-11T04:00:58.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d6a7fc700 1 -- 172.21.2.104:0/4254708385 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9d6c00f680 con 0x7f9d780b0060 2021-07-11T04:00:58.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/4254708385 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d780ae9e0 con 0x7f9d780b0060 2021-07-11T04:00:58.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/4254708385 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d780a9230 con 0x7f9d780b0060 2021-07-11T04:00:58.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/4254708385 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d780a9540 con 0x7f9d780b0060 2021-07-11T04:00:58.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d6a7fc700 1 -- 172.21.2.104:0/4254708385 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9d6c00f070 con 0x7f9d780b0060 2021-07-11T04:00:58.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d6a7fc700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d600531f0 0x7f9d600556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:58.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d6a7fc700 1 -- 172.21.2.104:0/4254708385 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9d6c06acd0 con 0x7f9d780b0060 2021-07-11T04:00:58.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d7d26b700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d600531f0 0x7f9d600556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:58.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d7d26b700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d600531f0 0x7f9d600556b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f9d74018000 tx=0x7f9d74014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:58.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.751+0000 7f9d6a7fc700 1 -- 172.21.2.104:0/4254708385 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9d6c042010 con 0x7f9d780b0060 2021-07-11T04:00:58.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/4254708385 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d78057350 con 0x7f9d780b0060 2021-07-11T04:00:58.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d6a7fc700 1 -- 172.21.2.104:0/4254708385 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2484 (secure 0 0 0) 0x7f9d6c045630 con 0x7f9d780b0060 2021-07-11T04:00:58.864 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:58.864 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":50,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/3293345379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3293345379},{"type":"v1","addr":"172.21.2.105:6835","nonce":3293345379}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/57099992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":57099992},{"type":"v1","addr":"172.21.2.104:6833","nonce":57099992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:55.667970+0000","modified":"2021-07-11T04:00:56.671726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7287":{"gid":7287,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/871079070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":871079070},{"type":"v1","addr":"172.21.2.104:6835","nonce":871079070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-07-11T04:00:58.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d600531f0 msgr2=0x7f9d600556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d600531f0 0x7f9d600556b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f9d74018000 tx=0x7f9d74014000).stop 2021-07-11T04:00:58.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/4254708385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d780b0060 msgr2=0x7f9d780ada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:58.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d780b0060 0x7f9d780ada00 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f9d6c002f10 tx=0x7f9d6c006d60).stop 2021-07-11T04:00:58.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/4254708385 shutdown_connections 2021-07-11T04:00:58.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9d780b1970 0x7f9d780aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9d600531f0 0x7f9d600556b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9d780b0060 0x7f9d780ada00 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 --2- 172.21.2.104:0/4254708385 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9d780a76b0 0x7f9d78066910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:58.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/4254708385 >> 172.21.2.104:0/4254708385 conn(0x7f9d780b53e0 msgr2=0x7f9d78146ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:58.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/4254708385 shutdown_connections 2021-07-11T04:00:58.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:58.863+0000 7f9d7f4cf700 1 -- 172.21.2.104:0/4254708385 wait complete. 2021-07-11T04:00:58.867 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 50 2021-07-11T04:00:59.173 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:00:55.667970+0000', 'modified': '2021-07-11T04:00:56.671726+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7287': {'gid': 7287, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/871079070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 871079070}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 871079070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:00:59.174 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:00:59.175 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:00:59.239 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":7287,"want_state":"up:active","state":"up:active","rank_uptime":3.5651806019999999,"mdsmap_epoch":50,"osdmap_epoch":108,"osdmap_epoch_barrier":107,"uptime":7.6979020059999996} 2021-07-11T04:00:59.239 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 7287, "mdsmap_epoch": 50, "osdmap_epoch": 108, "osdmap_epoch_barrier": 107, "rank_uptime": 3.565180602, "state": "up:active", "uptime": 7.697902006, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:00:59.240 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:00:59.240 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T04:00:59.240 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T04:00:59.241 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:59.241 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:59.252 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:00:59.263 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:59.264 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:59.316 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:59.316 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:59.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:59.318 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:00:59.323 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:00:59.324 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:59.326 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:00:59.336 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:00:59.336 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:00:59.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T04:00:59.362+0000 7ff9536f1080 -1 init, newargv = 0x5628ca26cb80 newargc=9 2021-07-11T04:00:59.365 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15483]: starting ceph client 2021-07-11T04:00:59.370 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:00:59.370 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:00:59.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:59.373 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:00:59.375 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15483]: starting fuse 2021-07-11T04:00:59.379 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T04:00:59.379 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:00:59.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:59.380 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T04:00:59.380 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T04:00:59.381 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T04:00:59.382 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T04:00:59.382 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T04:00:59.383 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:59.383 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T04:00:59.383 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T04:00:59.384 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T04:00:59.384 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:59.385 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T04:00:59.385 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T04:00:59.385 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T04:00:59.386 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T04:00:59.386 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:59.387 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T04:00:59.387 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:00:59.387 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:00:59.388 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T04:00:59.388 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:00:59.389 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:00:59.389 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T04:00:59.389 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T04:00:59.444 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15483.asok 2021-07-11T04:00:59.447 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15483.asok 2021-07-11T04:00:59.447 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:59.448 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15483.asok status 2021-07-11T04:00:59.501 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T04:00:59.501 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T04:00:59.502 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:00:59.502 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:00:59.502 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T04:00:59.502 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T04:00:59.503 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T04:00:59.503 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15483", 2021-07-11T04:00:59.503 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T04:00:59.503 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:59.503 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T04:00:59.504 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T04:00:59.504 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4859, 2021-07-11T04:00:59.504 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T04:00:59.504 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T04:00:59.504 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T04:00:59.505 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4859 2021-07-11T04:00:59.505 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:59.505 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:00:59.505 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:00:59.506 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:00:59.506 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3290234036 2021-07-11T04:00:59.506 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T04:00:59.506 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:59.506 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:00:59.507 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:00:59.507 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:00:59.507 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3290234036 2021-07-11T04:00:59.507 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:00:59.508 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4859 172.21.2.119:0/3290234036", 2021-07-11T04:00:59.508 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/3290234036", 2021-07-11T04:00:59.508 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T04:00:59.508 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 50, 2021-07-11T04:00:59.508 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 108, 2021-07-11T04:00:59.509 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:00:59.509 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T04:00:59.509 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T04:00:59.509 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T04:00:59.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:59.511 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:00:59.511 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:59.517 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:00:59.517 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:00:59.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:00:59.518 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:00:59.569 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:00:59.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.639+0000 7f266ee40700 1 Processor -- start 2021-07-11T04:00:59.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.639+0000 7f266ee40700 1 -- start start 2021-07-11T04:00:59.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.639+0000 7f266ee40700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f26680a76b0 0x7f2668066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.639+0000 7f266ee40700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f26680b1970 0x7f2668066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.639+0000 7f266ee40700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2668067450 con 0x7f26680b0060 2021-07-11T04:00:59.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.639+0000 7f266ee40700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26680a7a90 con 0x7f26680b1970 2021-07-11T04:00:59.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.639+0000 7f266ee40700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2668147620 con 0x7f26680a76b0 2021-07-11T04:00:59.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f2667fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f26680b1970 0x7f2668066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:59.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f2667fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f26680b1970 0x7f2668066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48826/0 (socket says 172.21.2.104:48826) 2021-07-11T04:00:59.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f2667fff700 1 -- 172.21.2.104:0/2885959868 learned_addr learned my addr 172.21.2.104:0/2885959868 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:59.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f2667fff700 1 -- 172.21.2.104:0/2885959868 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f26680a76b0 msgr2=0x7f2668066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:59.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f2667fff700 1 --2- 172.21.2.104:0/2885959868 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f26680a76b0 0x7f2668066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f2667fff700 1 -- 172.21.2.104:0/2885959868 >> v1:172.21.2.105:6789/0 conn(0x7f26680b0060 legacy=0x7f266805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:59.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f2667fff700 1 -- 172.21.2.104:0/2885959868 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26681478c0 con 0x7f26680b1970 2021-07-11T04:00:59.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f2667fff700 1 --2- 172.21.2.104:0/2885959868 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f26680b1970 0x7f2668066f60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f26680b6460 tx=0x7f265c006eb0).ready entity=mon.0 client_cookie=257518ce3f8b3f44 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:59.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f26677fe700 1 -- 172.21.2.104:0/2885959868 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f265c00d570 con 0x7f26680b1970 2021-07-11T04:00:59.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f26677fe700 1 -- 172.21.2.104:0/2885959868 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265c000e90 con 0x7f26680b1970 2021-07-11T04:00:59.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f26677fe700 1 -- 172.21.2.104:0/2885959868 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f265c00d8c0 con 0x7f26680b1970 2021-07-11T04:00:59.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 -- 172.21.2.104:0/2885959868 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f26680b1970 msgr2=0x7f2668066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 --2- 172.21.2.104:0/2885959868 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f26680b1970 0x7f2668066f60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f26680b6460 tx=0x7f265c006eb0).stop 2021-07-11T04:00:59.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 -- 172.21.2.104:0/2885959868 shutdown_connections 2021-07-11T04:00:59.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 --2- 172.21.2.104:0/2885959868 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f26680b1970 0x7f2668066f60 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:59.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 --2- 172.21.2.104:0/2885959868 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f26680a76b0 0x7f2668066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 -- 172.21.2.104:0/2885959868 >> 172.21.2.104:0/2885959868 conn(0x7f26680b53e0 msgr2=0x7f26680b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:59.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 -- 172.21.2.104:0/2885959868 shutdown_connections 2021-07-11T04:00:59.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 -- 172.21.2.104:0/2885959868 wait complete. 2021-07-11T04:00:59.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 Processor -- start 2021-07-11T04:00:59.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 -- start start 2021-07-11T04:00:59.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f26680a76b0 0x7f26680b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f26680b0060 0x7f266815a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f26680b1970 0x7f2668062770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2668159c10 con 0x7f26680a76b0 2021-07-11T04:00:59.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26680b4cc0 con 0x7f26680b0060 2021-07-11T04:00:59.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26680b4f50 con 0x7f26680b1970 2021-07-11T04:00:59.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266cbdc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f26680a76b0 0x7f26680b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:59.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266cbdc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f26680a76b0 0x7f26680b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48828/0 (socket says 172.21.2.104:48828) 2021-07-11T04:00:59.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266cbdc700 1 -- 172.21.2.104:0/3367245199 learned_addr learned my addr 172.21.2.104:0/3367245199 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:59.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266cbdc700 1 -- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f26680b1970 msgr2=0x7f2668062770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266cbdc700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f26680b1970 0x7f2668062770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266cbdc700 1 -- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f26680b0060 msgr2=0x7f266815a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266cbdc700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f26680b0060 0x7f266815a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266cbdc700 1 -- 172.21.2.104:0/3367245199 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265c00b1e0 con 0x7f26680a76b0 2021-07-11T04:00:59.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266cbdc700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f26680a76b0 0x7f26680b47d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f2658002f70 tx=0x7f2658006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:59.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f2665ffb700 1 -- 172.21.2.104:0/3367245199 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2658013460 con 0x7f26680a76b0 2021-07-11T04:00:59.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f2665ffb700 1 -- 172.21.2.104:0/3367245199 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26580135c0 con 0x7f26680a76b0 2021-07-11T04:00:59.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 -- 172.21.2.104:0/3367245199 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2668159e40 con 0x7f26680a76b0 2021-07-11T04:00:59.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f2665ffb700 1 -- 172.21.2.104:0/3367245199 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2658013920 con 0x7f26680a76b0 2021-07-11T04:00:59.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.643+0000 7f266ee40700 1 -- 172.21.2.104:0/3367245199 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266815aa70 con 0x7f26680a76b0 2021-07-11T04:00:59.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.647+0000 7f266ee40700 1 -- 172.21.2.104:0/3367245199 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2668057350 con 0x7f26680a76b0 2021-07-11T04:00:59.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.647+0000 7f2665ffb700 1 -- 172.21.2.104:0/3367245199 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f265801a060 con 0x7f26680a76b0 2021-07-11T04:00:59.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.647+0000 7f2665ffb700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2648053160 0x7f2648055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.647+0000 7f2665ffb700 1 -- 172.21.2.104:0/3367245199 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f265806b050 con 0x7f26680a76b0 2021-07-11T04:00:59.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.647+0000 7f2667fff700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2648053160 0x7f2648055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:59.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.647+0000 7f2667fff700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2648053160 0x7f2648055620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f26680b6460 tx=0x7f265c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:59.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.647+0000 7f2665ffb700 1 -- 172.21.2.104:0/3367245199 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2658068cf0 con 0x7f26680a76b0 2021-07-11T04:00:59.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 -- 172.21.2.104:0/3367245199 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26680698e0 con 0x7f26680a76b0 2021-07-11T04:00:59.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f2665ffb700 1 -- 172.21.2.104:0/3367245199 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f2658047070 con 0x7f26680a76b0 2021-07-11T04:00:59.734 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:00:59.735 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":108,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:00:56.672922+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:00:53.655121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2021-07-11T04:00:54.748677+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:00:59.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 -- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2648053160 msgr2=0x7f2648055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2648053160 0x7f2648055620 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f26680b6460 tx=0x7f265c015000).stop 2021-07-11T04:00:59.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 -- 172.21.2.104:0/3367245199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f26680a76b0 msgr2=0x7f26680b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f26680a76b0 0x7f26680b47d0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f2658002f70 tx=0x7f2658006ec0).stop 2021-07-11T04:00:59.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 -- 172.21.2.104:0/3367245199 shutdown_connections 2021-07-11T04:00:59.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f26680b1970 0x7f2668062770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f26680b0060 0x7f266815a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2648053160 0x7f2648055620 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:59.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 --2- 172.21.2.104:0/3367245199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f26680a76b0 0x7f26680b47d0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:59.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 -- 172.21.2.104:0/3367245199 >> 172.21.2.104:0/3367245199 conn(0x7f26680b53e0 msgr2=0x7f2668149570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:59.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 -- 172.21.2.104:0/3367245199 shutdown_connections 2021-07-11T04:00:59.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.731+0000 7f266ee40700 1 -- 172.21.2.104:0/3367245199 wait complete. 2021-07-11T04:00:59.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:00:53.655121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:00:54.748677+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '108', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:00:59.746 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:00:59.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 Processor -- start 2021-07-11T04:00:59.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 -- start start 2021-07-11T04:00:59.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f23cc0b1ec0 0x7f23cc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23cc0affb0 0x7f23cc0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23cc0b0420 con 0x7f23cc0a70c0 2021-07-11T04:00:59.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23cc0b2790 con 0x7f23cc0b1ec0 2021-07-11T04:00:59.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc0b0650 con 0x7f23cc0affb0 2021-07-11T04:00:59.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d2557700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f23cc0b1ec0 0x7f23cc0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:59.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d2557700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f23cc0b1ec0 0x7f23cc0b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48836/0 (socket says 172.21.2.104:48836) 2021-07-11T04:00:59.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d2557700 1 -- 172.21.2.104:0/3785254333 learned_addr learned my addr 172.21.2.104:0/3785254333 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:59.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d2557700 1 -- 172.21.2.104:0/3785254333 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23cc0affb0 msgr2=0x7f23cc0b22a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:59.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d2557700 1 --2- 172.21.2.104:0/3785254333 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23cc0affb0 0x7f23cc0b22a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d2557700 1 -- 172.21.2.104:0/3785254333 >> v1:172.21.2.105:6789/0 conn(0x7f23cc0a70c0 legacy=0x7f23cc141250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:00:59.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d2557700 1 -- 172.21.2.104:0/3785254333 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23cc0b08f0 con 0x7f23cc0b1ec0 2021-07-11T04:00:59.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d2557700 1 --2- 172.21.2.104:0/3785254333 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23cc0b1ec0 0x7f23cc0b12d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f23cc06a0f0 tx=0x7f23c0006eb0).ready entity=mon.0 client_cookie=db8c2d15b19abc0d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:59.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d1555700 1 -- 172.21.2.104:0/3785254333 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23c000d570 con 0x7f23cc0b1ec0 2021-07-11T04:00:59.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d1555700 1 -- 172.21.2.104:0/3785254333 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0000e90 con 0x7f23cc0b1ec0 2021-07-11T04:00:59.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d1555700 1 -- 172.21.2.104:0/3785254333 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23c000d8c0 con 0x7f23cc0b1ec0 2021-07-11T04:00:59.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 -- 172.21.2.104:0/3785254333 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23cc0b1ec0 msgr2=0x7f23cc0b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 --2- 172.21.2.104:0/3785254333 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23cc0b1ec0 0x7f23cc0b12d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f23cc06a0f0 tx=0x7f23c0006eb0).stop 2021-07-11T04:00:59.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 -- 172.21.2.104:0/3785254333 shutdown_connections 2021-07-11T04:00:59.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 --2- 172.21.2.104:0/3785254333 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23cc0affb0 0x7f23cc0b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 --2- 172.21.2.104:0/3785254333 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23cc0b1ec0 0x7f23cc0b12d0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:59.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 -- 172.21.2.104:0/3785254333 >> 172.21.2.104:0/3785254333 conn(0x7f23cc05f6b0 msgr2=0x7f23cc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:59.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 -- 172.21.2.104:0/3785254333 shutdown_connections 2021-07-11T04:00:59.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 -- 172.21.2.104:0/3785254333 wait complete. 2021-07-11T04:00:59.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.799+0000 7f23d47bb700 1 Processor -- start 2021-07-11T04:00:59.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d47bb700 1 -- start start 2021-07-11T04:00:59.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d47bb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23cc0a70c0 0x7f23cc1400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d47bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23cc0affb0 0x7f23cc13b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d47bb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f23cc0b1ec0 0x7f23cc13bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d47bb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc15b0c0 con 0x7f23cc0a70c0 2021-07-11T04:00:59.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d47bb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc15abb0 con 0x7f23cc0b1ec0 2021-07-11T04:00:59.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d47bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc15ae20 con 0x7f23cc0affb0 2021-07-11T04:00:59.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d2557700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23cc0a70c0 0x7f23cc1400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:59.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d2557700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23cc0a70c0 0x7f23cc1400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48842/0 (socket says 172.21.2.104:48842) 2021-07-11T04:00:59.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d2557700 1 -- 172.21.2.104:0/2932174074 learned_addr learned my addr 172.21.2.104:0/2932174074 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:59.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d2557700 1 -- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23cc0affb0 msgr2=0x7f23cc13b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d2557700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23cc0affb0 0x7f23cc13b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d2557700 1 -- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f23cc0b1ec0 msgr2=0x7f23cc13bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d2557700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f23cc0b1ec0 0x7f23cc13bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d2557700 1 -- 172.21.2.104:0/2932174074 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c000b1e0 con 0x7f23cc0a70c0 2021-07-11T04:00:59.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d2557700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23cc0a70c0 0x7f23cc1400a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f23cc06a0f0 tx=0x7f23c0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:59.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23bf7fe700 1 -- 172.21.2.104:0/2932174074 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23c00118a0 con 0x7f23cc0a70c0 2021-07-11T04:00:59.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23bf7fe700 1 -- 172.21.2.104:0/2932174074 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c0011a00 con 0x7f23cc0a70c0 2021-07-11T04:00:59.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d47bb700 1 -- 172.21.2.104:0/2932174074 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc1422f0 con 0x7f23cc0a70c0 2021-07-11T04:00:59.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d47bb700 1 -- 172.21.2.104:0/2932174074 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc142a30 con 0x7f23cc0a70c0 2021-07-11T04:00:59.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23bf7fe700 1 -- 172.21.2.104:0/2932174074 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f23c0011d60 con 0x7f23cc0a70c0 2021-07-11T04:00:59.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d47bb700 1 -- 172.21.2.104:0/2932174074 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc057350 con 0x7f23cc0a70c0 2021-07-11T04:00:59.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23bf7fe700 1 -- 172.21.2.104:0/2932174074 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f23c000f070 con 0x7f23cc0a70c0 2021-07-11T04:00:59.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23bf7fe700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23ac053380 0x7f23ac055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23bf7fe700 1 -- 172.21.2.104:0/2932174074 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f23c006ce80 con 0x7f23cc0a70c0 2021-07-11T04:00:59.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.803+0000 7f23d1d56700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23ac053380 0x7f23ac055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:59.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.807+0000 7f23d1d56700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23ac053380 0x7f23ac055840 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f23c8002fb0 tx=0x7f23c80069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:59.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.807+0000 7f23bf7fe700 1 -- 172.21.2.104:0/2932174074 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f23c00440b0 con 0x7f23cc0a70c0 2021-07-11T04:00:59.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.887+0000 7f23d47bb700 1 -- 172.21.2.104:0/2932174074 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f23cc0b6430 con 0x7f23cc0a70c0 2021-07-11T04:00:59.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.887+0000 7f23bf7fe700 1 -- 172.21.2.104:0/2932174074 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v108) v1 ==== 110+0+10 (secure 0 0 0) 0x7f23c00476d0 con 0x7f23cc0a70c0 2021-07-11T04:00:59.891 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:00:59.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.887+0000 7f23d47bb700 1 -- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23ac053380 msgr2=0x7f23ac055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.887+0000 7f23d47bb700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23ac053380 0x7f23ac055840 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f23c8002fb0 tx=0x7f23c80069d0).stop 2021-07-11T04:00:59.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.887+0000 7f23d47bb700 1 -- 172.21.2.104:0/2932174074 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23cc0a70c0 msgr2=0x7f23cc1400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.887+0000 7f23d47bb700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23cc0a70c0 0x7f23cc1400a0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f23cc06a0f0 tx=0x7f23c0015000).stop 2021-07-11T04:00:59.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.891+0000 7f23d47bb700 1 -- 172.21.2.104:0/2932174074 shutdown_connections 2021-07-11T04:00:59.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.891+0000 7f23d47bb700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f23cc0b1ec0 0x7f23cc13bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.891+0000 7f23d47bb700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f23cc0affb0 0x7f23cc13b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.891+0000 7f23d47bb700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f23ac053380 0x7f23ac055840 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:59.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.891+0000 7f23d47bb700 1 --2- 172.21.2.104:0/2932174074 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f23cc0a70c0 0x7f23cc1400a0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:59.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.891+0000 7f23d47bb700 1 -- 172.21.2.104:0/2932174074 >> 172.21.2.104:0/2932174074 conn(0x7f23cc05f6b0 msgr2=0x7f23cc1450c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:59.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.891+0000 7f23d47bb700 1 -- 172.21.2.104:0/2932174074 shutdown_connections 2021-07-11T04:00:59.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.891+0000 7f23d47bb700 1 -- 172.21.2.104:0/2932174074 wait complete. 2021-07-11T04:00:59.899 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:00:59.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d50bd2700 1 Processor -- start 2021-07-11T04:00:59.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d50bd2700 1 -- start start 2021-07-11T04:00:59.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d50bd2700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8d4c141960 0x7f8d4c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d50bd2700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4c0a77d0 con 0x7f8d4c0b1ec0 2021-07-11T04:00:59.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d50bd2700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4c0b0650 con 0x7f8d4c0b12d0 2021-07-11T04:00:59.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d50bd2700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d4c0b1740 con 0x7f8d4c141960 2021-07-11T04:00:59.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4a59c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8d4c0b1ec0 0x7f8d4c0a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45618/0 (socket says 172.21.2.104:45618) 2021-07-11T04:00:59.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4a59c700 1 -- 172.21.2.104:0/3675963873 learned_addr learned my addr 172.21.2.104:0/3675963873 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:00:59.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680826389 0 0) 0x7f8d4c0a77d0 con 0x7f8d4c0b1ec0 2021-07-11T04:00:59.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4ad9d700 1 --2- 172.21.2.104:0/3675963873 >> v2:172.21.2.105:3300/0 conn(0x7f8d4c141960 0x7f8d4c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:59.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d2c003260 con 0x7f8d4c0b1ec0 2021-07-11T04:00:59.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1042877631 0 0) 0x7f8d2c003260 con 0x7f8d4c0b1ec0 2021-07-11T04:00:59.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 >> v2:172.21.2.105:3300/0 conn(0x7f8d4c141960 msgr2=0x7f8d4c0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:59.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 --2- 172.21.2.104:0/3675963873 >> v2:172.21.2.105:3300/0 conn(0x7f8d4c141960 0x7f8d4c0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:59.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 >> v1:172.21.2.105:6789/0 conn(0x7f8d4c0b12d0 legacy=0x7f8d4c141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:59.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c0b08f0 con 0x7f8d4c0b1ec0 2021-07-11T04:00:59.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f8d34001660 con 0x7f8d4c0b1ec0 2021-07-11T04:00:59.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 >> v1:172.21.2.104:6789/0 conn(0x7f8d4c0b1ec0 legacy=0x7f8d4c0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 --2- 172.21.2.104:0/3675963873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d2c003820 0x7f8d2c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 --2- 172.21.2.104:0/3675963873 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d2c004500 0x7f8d2c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 --2- 172.21.2.104:0/3675963873 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d2c004f70 0x7f8d2c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c0b1740 con 0x7f8d2c003820 2021-07-11T04:00:59.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c00be50 con 0x7f8d2c004f70 2021-07-11T04:00:59.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c00c010 con 0x7f8d2c004500 2021-07-11T04:00:59.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.951+0000 7f8d4a59c700 1 --2- 172.21.2.104:0/3675963873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d2c003820 0x7f8d2c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:59.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4a59c700 1 -- 172.21.2.104:0/3675963873 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d2c004500 msgr2=0x7f8d2c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:59.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4a59c700 1 --2- 172.21.2.104:0/3675963873 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d2c004500 0x7f8d2c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4a59c700 1 -- 172.21.2.104:0/3675963873 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d2c004f70 msgr2=0x7f8d2c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:00:59.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4a59c700 1 --2- 172.21.2.104:0/3675963873 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d2c004f70 0x7f8d2c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4a59c700 1 -- 172.21.2.104:0/3675963873 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c003260 con 0x7f8d2c003820 2021-07-11T04:00:59.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4ad9d700 1 -- 172.21.2.104:0/3675963873 reap_dead start 2021-07-11T04:00:59.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4a59c700 1 --2- 172.21.2.104:0/3675963873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d2c003820 0x7f8d2c003c20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f8d4c0b2f50 tx=0x7f8d4c0b0a80).ready entity=mon.0 client_cookie=3aed8e9918a4849e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:59.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d340070d0 con 0x7f8d2c003820 2021-07-11T04:00:59.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d34007230 con 0x7f8d2c003820 2021-07-11T04:00:59.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4959a700 1 -- 172.21.2.104:0/3675963873 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d3400c8f0 con 0x7f8d2c003820 2021-07-11T04:00:59.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3675963873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d2c003820 msgr2=0x7f8d2c003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 --2- 172.21.2.104:0/3675963873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d2c003820 0x7f8d2c003c20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f8d4c0b2f50 tx=0x7f8d4c0b0a80).stop 2021-07-11T04:00:59.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3675963873 shutdown_connections 2021-07-11T04:00:59.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 --2- 172.21.2.104:0/3675963873 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d2c003820 0x7f8d2c003c20 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:00:59.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3675963873 >> 172.21.2.104:0/3675963873 conn(0x7f8d4c05f6b0 msgr2=0x7f8d4c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:00:59.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3675963873 shutdown_connections 2021-07-11T04:00:59.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3675963873 wait complete. 2021-07-11T04:00:59.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 Processor -- start 2021-07-11T04:00:59.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- start start 2021-07-11T04:00:59.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d2c003820 0x7f8d4c13fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d4c0ab080 0x7f8d4c149ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d4c0a9420 0x7f8d4c0a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c14b4b0 con 0x7f8d4c0a9420 2021-07-11T04:00:59.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c0abd20 con 0x7f8d4c0ab080 2021-07-11T04:00:59.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c14b6e0 con 0x7f8d2c003820 2021-07-11T04:00:59.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d4c0a9420 0x7f8d4c0a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:59.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d4c0a9420 0x7f8d4c0a98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48860/0 (socket says 172.21.2.104:48860) 2021-07-11T04:00:59.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4ad9d700 1 -- 172.21.2.104:0/3319541685 learned_addr learned my addr 172.21.2.104:0/3319541685 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:00:59.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4ad9d700 1 -- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d2c003820 msgr2=0x7f8d4c13fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4ad9d700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d2c003820 0x7f8d4c13fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4ad9d700 1 -- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d4c0ab080 msgr2=0x7f8d4c149ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:00:59.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4ad9d700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d4c0ab080 0x7f8d4c149ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:00:59.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4ad9d700 1 -- 172.21.2.104:0/3319541685 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c14b980 con 0x7f8d4c0a9420 2021-07-11T04:00:59.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d4ad9d700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d4c0a9420 0x7f8d4c0a98a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f8d34001110 tx=0x7f8d44008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:59.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d437fe700 1 -- 172.21.2.104:0/3319541685 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d4400c070 con 0x7f8d4c0a9420 2021-07-11T04:00:59.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d437fe700 1 -- 172.21.2.104:0/3319541685 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d440135e0 con 0x7f8d4c0a9420 2021-07-11T04:00:59.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3319541685 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4c0ab500 con 0x7f8d4c0a9420 2021-07-11T04:00:59.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d437fe700 1 -- 172.21.2.104:0/3319541685 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d44013940 con 0x7f8d4c0a9420 2021-07-11T04:00:59.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3319541685 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c0ac190 con 0x7f8d4c0a9420 2021-07-11T04:00:59.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3319541685 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d4c0abe90 con 0x7f8d4c0a9420 2021-07-11T04:00:59.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.955+0000 7f8d437fe700 1 -- 172.21.2.104:0/3319541685 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8d4401b030 con 0x7f8d4c0a9420 2021-07-11T04:00:59.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.959+0000 7f8d437fe700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d30053210 0x7f8d300556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:00:59.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.959+0000 7f8d437fe700 1 -- 172.21.2.104:0/3319541685 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8d4406a740 con 0x7f8d4c0a9420 2021-07-11T04:00:59.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.959+0000 7f8d4a59c700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d30053210 0x7f8d300556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:00:59.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.959+0000 7f8d4a59c700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d30053210 0x7f8d300556d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f8d4c0b2f50 tx=0x7f8d34013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:00:59.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:00:59.959+0000 7f8d437fe700 1 -- 172.21.2.104:0/3319541685 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8d4c0abe90 con 0x7f8d4c0a9420 2021-07-11T04:01:00.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3319541685 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8d4c0abe90 con 0x7f8d4c0a9420 2021-07-11T04:01:00.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d437fe700 1 -- 172.21.2.104:0/3319541685 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v108) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8d4c0abe90 con 0x7f8d4c0a9420 2021-07-11T04:01:00.044 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:00.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d30053210 msgr2=0x7f8d300556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d30053210 0x7f8d300556d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f8d4c0b2f50 tx=0x7f8d34013000).stop 2021-07-11T04:01:00.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3319541685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d4c0a9420 msgr2=0x7f8d4c0a98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d4c0a9420 0x7f8d4c0a98a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f8d34001110 tx=0x7f8d44008040).stop 2021-07-11T04:01:00.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3319541685 shutdown_connections 2021-07-11T04:01:00.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d30053210 0x7f8d300556d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d4c0a9420 0x7f8d4c0a98a0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d4c0ab080 0x7f8d4c149ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 --2- 172.21.2.104:0/3319541685 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d2c003820 0x7f8d4c13fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3319541685 >> 172.21.2.104:0/3319541685 conn(0x7f8d4c05f6b0 msgr2=0x7f8d4c0ad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:00.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3319541685 shutdown_connections 2021-07-11T04:01:00.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.043+0000 7f8d50bd2700 1 -- 172.21.2.104:0/3319541685 wait complete. 2021-07-11T04:01:00.051 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:01:00.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116cf5f700 1 Processor -- start 2021-07-11T04:01:00.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116cf5f700 1 -- start start 2021-07-11T04:01:00.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116cf5f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168141960 0x7f11680affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116cf5f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11680a77d0 con 0x7f11680b12d0 2021-07-11T04:01:00.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116cf5f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11680b0650 con 0x7f11680b1ec0 2021-07-11T04:01:00.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116cf5f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11680b1740 con 0x7f1168141960 2021-07-11T04:01:00.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f1165d9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f11680b12d0 0x7f1168141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45638/0 (socket says 172.21.2.104:45638) 2021-07-11T04:01:00.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f1165d9b700 1 -- 172.21.2.104:0/617763969 learned_addr learned my addr 172.21.2.104:0/617763969 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:00.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681910659 0 0) 0x7f11680a77d0 con 0x7f11680b12d0 2021-07-11T04:01:00.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1148003260 con 0x7f11680b12d0 2021-07-11T04:01:00.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f1166d9d700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168141960 0x7f11680affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2454187344 0 0) 0x7f1148003260 con 0x7f11680b12d0 2021-07-11T04:01:00.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168141960 msgr2=0x7f11680affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:00.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168141960 0x7f11680affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 >> v1:172.21.2.105:6789/0 conn(0x7f11680b1ec0 legacy=0x7f11680a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:00.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11680b08f0 con 0x7f11680b12d0 2021-07-11T04:01:00.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f1158001660 con 0x7f11680b12d0 2021-07-11T04:01:00.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 >> v1:172.21.2.104:6789/0 conn(0x7f11680b12d0 legacy=0x7f1168141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1148003780 0x7f1148003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1148004500 0x7f1148004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168141960 0x7f1148004e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11680b1740 con 0x7f1148003780 2021-07-11T04:01:00.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114800b800 con 0x7f1148004500 2021-07-11T04:01:00.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114800b990 con 0x7f1168141960 2021-07-11T04:01:00.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116659c700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1148003780 0x7f1148003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f1165d9b700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1148004500 0x7f1148004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f1166d9d700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168141960 0x7f1148004e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116659c700 1 -- 172.21.2.104:0/617763969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168141960 msgr2=0x7f1148004e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:00.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116659c700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168141960 0x7f1148004e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116659c700 1 -- 172.21.2.104:0/617763969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1148004500 msgr2=0x7f1148004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:00.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116659c700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1148004500 0x7f1148004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.115 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116659c700 1 -- 172.21.2.104:0/617763969 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11680b08f0 con 0x7f1148003780 2021-07-11T04:01:00.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116659c700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1148003780 0x7f1148003b80 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7f11480054d0 tx=0x7f1150007a00).ready entity=mon.0 client_cookie=9a2bf2088d90b9a4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:00.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1150007070 con 0x7f1148003780 2021-07-11T04:01:00.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11500071d0 con 0x7f1148003780 2021-07-11T04:01:00.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116559a700 1 -- 172.21.2.104:0/617763969 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1150006510 con 0x7f1148003780 2021-07-11T04:01:00.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116cf5f700 1 -- 172.21.2.104:0/617763969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1148003780 msgr2=0x7f1148003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f116cf5f700 1 --2- 172.21.2.104:0/617763969 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1148003780 0x7f1148003b80 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7f11480054d0 tx=0x7f1150007a00).stop 2021-07-11T04:01:00.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.103+0000 7f1166d9d700 1 -- 172.21.2.104:0/617763969 reap_dead start 2021-07-11T04:01:00.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 -- 172.21.2.104:0/617763969 shutdown_connections 2021-07-11T04:01:00.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 -- 172.21.2.104:0/617763969 >> 172.21.2.104:0/617763969 conn(0x7f116805f6b0 msgr2=0x7f116805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:00.117 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 -- 172.21.2.104:0/617763969 shutdown_connections 2021-07-11T04:01:00.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 -- 172.21.2.104:0/617763969 wait complete. 2021-07-11T04:01:00.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 Processor -- start 2021-07-11T04:01:00.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 -- start start 2021-07-11T04:01:00.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f116814a610 0x7f116815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.118 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f116814adf0 0x7f116815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168148b30 0x7f1168148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11680ae7b0 con 0x7f116814adf0 2021-07-11T04:01:00.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116814a9f0 con 0x7f116814a610 2021-07-11T04:01:00.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116815a130 con 0x7f1168148b30 2021-07-11T04:01:00.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f1165d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f116814adf0 0x7f116815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f1165d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f116814adf0 0x7f116815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48882/0 (socket says 172.21.2.104:48882) 2021-07-11T04:01:00.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f1165d9b700 1 -- 172.21.2.104:0/2225393549 learned_addr learned my addr 172.21.2.104:0/2225393549 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:00.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f1165d9b700 1 -- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168148b30 msgr2=0x7f1168148fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f1165d9b700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168148b30 0x7f1168148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f1165d9b700 1 -- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f116814a610 msgr2=0x7f116815a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f1165d9b700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f116814a610 0x7f116815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f1165d9b700 1 -- 172.21.2.104:0/2225393549 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116815af70 con 0x7f116814adf0 2021-07-11T04:01:00.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f1165d9b700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f116814adf0 0x7f116815aa80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f115800b000 tx=0x7f11580014c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:00.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f115f7fe700 1 -- 172.21.2.104:0/2225393549 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1158010710 con 0x7f116814adf0 2021-07-11T04:01:00.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f115f7fe700 1 -- 172.21.2.104:0/2225393549 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1158010870 con 0x7f116814adf0 2021-07-11T04:01:00.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 -- 172.21.2.104:0/2225393549 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11680ac2a0 con 0x7f116814adf0 2021-07-11T04:01:00.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f115f7fe700 1 -- 172.21.2.104:0/2225393549 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1158010bd0 con 0x7f116814adf0 2021-07-11T04:01:00.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 -- 172.21.2.104:0/2225393549 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11680aec80 con 0x7f116814adf0 2021-07-11T04:01:00.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f116cf5f700 1 -- 172.21.2.104:0/2225393549 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116815a2a0 con 0x7f116814adf0 2021-07-11T04:01:00.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f115f7fe700 1 -- 172.21.2.104:0/2225393549 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1158010d30 con 0x7f116814adf0 2021-07-11T04:01:00.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f115f7fe700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f114c053180 0x7f114c055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.107+0000 7f115f7fe700 1 -- 172.21.2.104:0/2225393549 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f115806b6e0 con 0x7f116814adf0 2021-07-11T04:01:00.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.111+0000 7f116659c700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f114c053180 0x7f114c055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.111+0000 7f116659c700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f114c053180 0x7f114c055640 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f11480054d0 tx=0x7f1150015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:00.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.111+0000 7f115f7fe700 1 -- 172.21.2.104:0/2225393549 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1158042910 con 0x7f116814adf0 2021-07-11T04:01:00.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.191+0000 7f116cf5f700 1 -- 172.21.2.104:0/2225393549 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1168057350 con 0x7f116814adf0 2021-07-11T04:01:00.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.191+0000 7f115f7fe700 1 -- 172.21.2.104:0/2225393549 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1158045f30 con 0x7f116814adf0 2021-07-11T04:01:00.195 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:00.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 -- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f114c053180 msgr2=0x7f114c055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f114c053180 0x7f114c055640 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f11480054d0 tx=0x7f1150015000).stop 2021-07-11T04:01:00.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 -- 172.21.2.104:0/2225393549 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f116814adf0 msgr2=0x7f116815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f116814adf0 0x7f116815aa80 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f115800b000 tx=0x7f11580014c0).stop 2021-07-11T04:01:00.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 -- 172.21.2.104:0/2225393549 shutdown_connections 2021-07-11T04:01:00.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1168148b30 0x7f1168148fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f114c053180 0x7f114c055640 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f116814adf0 0x7f116815aa80 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 --2- 172.21.2.104:0/2225393549 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f116814a610 0x7f116815a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 -- 172.21.2.104:0/2225393549 >> 172.21.2.104:0/2225393549 conn(0x7f116805f6b0 msgr2=0x7f116813a640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:00.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 -- 172.21.2.104:0/2225393549 shutdown_connections 2021-07-11T04:01:00.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.195+0000 7f116cf5f700 1 -- 172.21.2.104:0/2225393549 wait complete. 2021-07-11T04:01:00.201 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:01:00.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.251+0000 7f4a37f3f700 1 Processor -- start 2021-07-11T04:01:00.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.251+0000 7f4a37f3f700 1 -- start start 2021-07-11T04:01:00.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4a3005ff60 0x7f4a30140440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a30064910 con 0x7f4a300611d0 2021-07-11T04:01:00.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a30064bb0 con 0x7f4a300b2710 2021-07-11T04:01:00.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a30064e50 con 0x7f4a3005ff60 2021-07-11T04:01:00.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a364dc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4a300611d0 0x7f4a30143820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45656/0 (socket says 172.21.2.104:45656) 2021-07-11T04:01:00.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a364dc700 1 -- 172.21.2.104:0/1970921250 learned_addr learned my addr 172.21.2.104:0/1970921250 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:00.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1520068595 0 0) 0x7f4a30064910 con 0x7f4a300611d0 2021-07-11T04:01:00.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a18003260 con 0x7f4a300611d0 2021-07-11T04:01:00.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a35cdb700 1 --2- 172.21.2.104:0/1970921250 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4a3005ff60 0x7f4a30140440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3510156374 0 0) 0x7f4a18003260 con 0x7f4a300611d0 2021-07-11T04:01:00.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4a3005ff60 msgr2=0x7f4a30140440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:00.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 --2- 172.21.2.104:0/1970921250 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4a3005ff60 0x7f4a30140440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 >> v1:172.21.2.105:6789/0 conn(0x7f4a300b2710 legacy=0x7f4a30143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:00.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a300650f0 con 0x7f4a300611d0 2021-07-11T04:01:00.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f4a280019e0 con 0x7f4a300611d0 2021-07-11T04:01:00.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 >> v1:172.21.2.104:6789/0 conn(0x7f4a300611d0 legacy=0x7f4a30143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 --2- 172.21.2.104:0/1970921250 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4a18003800 0x7f4a18003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 --2- 172.21.2.104:0/1970921250 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a18004500 0x7f4a18004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 --2- 172.21.2.104:0/1970921250 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4a3005ff60 0x7f4a18004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a30064e50 con 0x7f4a18004500 2021-07-11T04:01:00.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1800b910 con 0x7f4a18003800 2021-07-11T04:01:00.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1800bad0 con 0x7f4a3005ff60 2021-07-11T04:01:00.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a354da700 1 --2- 172.21.2.104:0/1970921250 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a18004500 0x7f4a18004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a354da700 1 -- 172.21.2.104:0/1970921250 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4a3005ff60 msgr2=0x7f4a18004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:00.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a354da700 1 --2- 172.21.2.104:0/1970921250 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4a3005ff60 0x7f4a18004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a354da700 1 -- 172.21.2.104:0/1970921250 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4a18003800 msgr2=0x7f4a18003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:00.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a354da700 1 --2- 172.21.2.104:0/1970921250 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4a18003800 0x7f4a18003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a354da700 1 -- 172.21.2.104:0/1970921250 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a18003260 con 0x7f4a18004500 2021-07-11T04:01:00.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a354da700 1 --2- 172.21.2.104:0/1970921250 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a18004500 0x7f4a18004980 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f4a2c001fa0 tx=0x7f4a2c007a00).ready entity=mon.0 client_cookie=b049cbb7389649ea server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:00.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a2c007070 con 0x7f4a18004500 2021-07-11T04:01:00.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2c0071d0 con 0x7f4a18004500 2021-07-11T04:01:00.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a34cd9700 1 -- 172.21.2.104:0/1970921250 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a2c006510 con 0x7f4a18004500 2021-07-11T04:01:00.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1970921250 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a18004500 msgr2=0x7f4a18004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 --2- 172.21.2.104:0/1970921250 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a18004500 0x7f4a18004980 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f4a2c001fa0 tx=0x7f4a2c007a00).stop 2021-07-11T04:01:00.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a364dc700 1 -- 172.21.2.104:0/1970921250 reap_dead start 2021-07-11T04:01:00.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1970921250 shutdown_connections 2021-07-11T04:01:00.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1970921250 >> 172.21.2.104:0/1970921250 conn(0x7f4a3005cf60 msgr2=0x7f4a3005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:00.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1970921250 shutdown_connections 2021-07-11T04:01:00.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1970921250 wait complete. 2021-07-11T04:01:00.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 Processor -- start 2021-07-11T04:01:00.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- start start 2021-07-11T04:01:00.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a30066cf0 0x7f4a30141090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4a30142a60 0x7f4a30141580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4a30143760 0x7f4a3013c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a300670d0 con 0x7f4a30066cf0 2021-07-11T04:01:00.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a300ac270 con 0x7f4a30142a60 2021-07-11T04:01:00.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a37f3f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a30143f60 con 0x7f4a30143760 2021-07-11T04:01:00.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a35cdb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a30066cf0 0x7f4a30141090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a35cdb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a30066cf0 0x7f4a30141090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48902/0 (socket says 172.21.2.104:48902) 2021-07-11T04:01:00.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a35cdb700 1 -- 172.21.2.104:0/1751550135 learned_addr learned my addr 172.21.2.104:0/1751550135 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:00.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a35cdb700 1 -- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4a30143760 msgr2=0x7f4a3013c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a35cdb700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4a30143760 0x7f4a3013c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a35cdb700 1 -- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4a30142a60 msgr2=0x7f4a30141580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a35cdb700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4a30142a60 0x7f4a30141580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a35cdb700 1 -- 172.21.2.104:0/1751550135 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a300ad930 con 0x7f4a30066cf0 2021-07-11T04:01:00.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.255+0000 7f4a35cdb700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a30066cf0 0x7f4a30141090 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f4a24002f30 tx=0x7f4a24008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:00.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a22ffd700 1 -- 172.21.2.104:0/1751550135 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a24016070 con 0x7f4a30066cf0 2021-07-11T04:01:00.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a22ffd700 1 -- 172.21.2.104:0/1751550135 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a24000e90 con 0x7f4a30066cf0 2021-07-11T04:01:00.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1751550135 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a300adbd0 con 0x7f4a30066cf0 2021-07-11T04:01:00.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a22ffd700 1 -- 172.21.2.104:0/1751550135 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4a24011970 con 0x7f4a30066cf0 2021-07-11T04:01:00.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1751550135 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a30143c50 con 0x7f4a30066cf0 2021-07-11T04:01:00.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1751550135 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a30057350 con 0x7f4a30066cf0 2021-07-11T04:01:00.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a22ffd700 1 -- 172.21.2.104:0/1751550135 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4a24011ad0 con 0x7f4a30066cf0 2021-07-11T04:01:00.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a22ffd700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4a10053210 0x7f4a100556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a22ffd700 1 -- 172.21.2.104:0/1751550135 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4a2406b270 con 0x7f4a30066cf0 2021-07-11T04:01:00.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a354da700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4a10053210 0x7f4a100556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a354da700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4a10053210 0x7f4a100556d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f4a2c001fa0 tx=0x7f4a2c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:00.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.259+0000 7f4a22ffd700 1 -- 172.21.2.104:0/1751550135 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4a240425b0 con 0x7f4a30066cf0 2021-07-11T04:01:00.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1751550135 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a300698e0 con 0x7f4a30066cf0 2021-07-11T04:01:00.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a22ffd700 1 -- 172.21.2.104:0/1751550135 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4a24045bd0 con 0x7f4a30066cf0 2021-07-11T04:01:00.345 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:00.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4a10053210 msgr2=0x7f4a100556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4a10053210 0x7f4a100556d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f4a2c001fa0 tx=0x7f4a2c015000).stop 2021-07-11T04:01:00.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1751550135 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a30066cf0 msgr2=0x7f4a30141090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a30066cf0 0x7f4a30141090 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f4a24002f30 tx=0x7f4a24008040).stop 2021-07-11T04:01:00.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1751550135 shutdown_connections 2021-07-11T04:01:00.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4a30143760 0x7f4a3013c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4a30142a60 0x7f4a30141580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4a10053210 0x7f4a100556d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 --2- 172.21.2.104:0/1751550135 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4a30066cf0 0x7f4a30141090 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1751550135 >> 172.21.2.104:0/1751550135 conn(0x7f4a3005cf60 msgr2=0x7f4a300b51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:00.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1751550135 shutdown_connections 2021-07-11T04:01:00.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.343+0000 7f4a37f3f700 1 -- 172.21.2.104:0/1751550135 wait complete. 2021-07-11T04:01:00.352 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:01:00.353 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:00.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e1059d700 1 Processor -- start 2021-07-11T04:01:00.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e1059d700 1 -- start start 2021-07-11T04:01:00.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e1059d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4e080b0330 0x7f4e0814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e1059d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e08067840 con 0x7f4e080b0960 2021-07-11T04:01:00.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e1059d700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e080a9e20 con 0x7f4e08067460 2021-07-11T04:01:00.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e1059d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e080aa0c0 con 0x7f4e080b0330 2021-07-11T04:01:00.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e0db38700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4e080b0960 0x7f4e080abef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45676/0 (socket says 172.21.2.104:45676) 2021-07-11T04:01:00.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e0db38700 1 -- 172.21.2.104:0/3362238531 learned_addr learned my addr 172.21.2.104:0/3362238531 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:00.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e0eb3a700 1 --2- 172.21.2.104:0/3362238531 >> v2:172.21.2.104:3300/0 conn(0x7f4e080b0330 0x7f4e0814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e0d337700 1 -- 172.21.2.104:0/3362238531 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3553827516 0 0) 0x7f4e08067840 con 0x7f4e080b0960 2021-07-11T04:01:00.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e0d337700 1 -- 172.21.2.104:0/3362238531 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4df0003260 con 0x7f4e080b0960 2021-07-11T04:01:00.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e0eb3a700 1 -- 172.21.2.104:0/3362238531 >> v1:172.21.2.105:6789/0 conn(0x7f4e08067460 legacy=0x7f4e080ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:00.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e0eb3a700 1 -- 172.21.2.104:0/3362238531 >> v1:172.21.2.104:6789/0 conn(0x7f4e080b0960 legacy=0x7f4e080abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e0eb3a700 1 -- 172.21.2.104:0/3362238531 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e080ab340 con 0x7f4e080b0330 2021-07-11T04:01:00.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.407+0000 7f4e0eb3a700 1 --2- 172.21.2.104:0/3362238531 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4e080b0330 0x7f4e0814dd50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7f4e08062740 tx=0x7f4e04006df0).ready entity=mon.0 client_cookie=ec504f7267e25322 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:00.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0d337700 1 -- 172.21.2.104:0/3362238531 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e0400d5c0 con 0x7f4e080b0330 2021-07-11T04:01:00.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0d337700 1 -- 172.21.2.104:0/3362238531 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e04000e90 con 0x7f4e080b0330 2021-07-11T04:01:00.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0d337700 1 -- 172.21.2.104:0/3362238531 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4e0400d910 con 0x7f4e080b0330 2021-07-11T04:01:00.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- 172.21.2.104:0/3362238531 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4e080b0330 msgr2=0x7f4e0814dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 --2- 172.21.2.104:0/3362238531 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4e080b0330 0x7f4e0814dd50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7f4e08062740 tx=0x7f4e04006df0).stop 2021-07-11T04:01:00.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- 172.21.2.104:0/3362238531 shutdown_connections 2021-07-11T04:01:00.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 --2- 172.21.2.104:0/3362238531 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4e080b0330 0x7f4e0814dd50 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- 172.21.2.104:0/3362238531 >> 172.21.2.104:0/3362238531 conn(0x7f4e08061070 msgr2=0x7f4e0805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:00.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- 172.21.2.104:0/3362238531 shutdown_connections 2021-07-11T04:01:00.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- 172.21.2.104:0/3362238531 wait complete. 2021-07-11T04:01:00.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 Processor -- start 2021-07-11T04:01:00.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- start start 2021-07-11T04:01:00.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4e08067460 0x7f4e080b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4e080b0330 0x7f4e080a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4e080b0960 0x7f4e080a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e080a43b0 con 0x7f4e080b0330 2021-07-11T04:01:00.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081448d0 con 0x7f4e08067460 2021-07-11T04:01:00.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08144390 con 0x7f4e080b0960 2021-07-11T04:01:00.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0db38700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4e080b0330 0x7f4e080a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0db38700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4e080b0330 0x7f4e080a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48912/0 (socket says 172.21.2.104:48912) 2021-07-11T04:01:00.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0db38700 1 -- 172.21.2.104:0/1004952855 learned_addr learned my addr 172.21.2.104:0/1004952855 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:00.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0db38700 1 -- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4e080b0960 msgr2=0x7f4e080a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0db38700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4e080b0960 0x7f4e080a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0db38700 1 -- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4e08067460 msgr2=0x7f4e080b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0db38700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4e08067460 0x7f4e080b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0db38700 1 -- 172.21.2.104:0/1004952855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0400b180 con 0x7f4e080b0330 2021-07-11T04:01:00.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e0db38700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4e080b0330 0x7f4e080a1840 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f4df8001110 tx=0x7f4df8007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:00.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4df77fe700 1 -- 172.21.2.104:0/1004952855 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4df8007e10 con 0x7f4e080b0330 2021-07-11T04:01:00.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4df77fe700 1 -- 172.21.2.104:0/1004952855 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df80072e0 con 0x7f4e080b0330 2021-07-11T04:01:00.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4df77fe700 1 -- 172.21.2.104:0/1004952855 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4df8005800 con 0x7f4e080b0330 2021-07-11T04:01:00.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- 172.21.2.104:0/1004952855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e08144630 con 0x7f4e080b0330 2021-07-11T04:01:00.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- 172.21.2.104:0/1004952855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e080a0fe0 con 0x7f4e080b0330 2021-07-11T04:01:00.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4e1059d700 1 -- 172.21.2.104:0/1004952855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e08057350 con 0x7f4e080b0330 2021-07-11T04:01:00.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.411+0000 7f4df77fe700 1 -- 172.21.2.104:0/1004952855 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4df8010070 con 0x7f4e080b0330 2021-07-11T04:01:00.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.415+0000 7f4df77fe700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4de8053270 0x7f4de8055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:00.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.415+0000 7f4df77fe700 1 -- 172.21.2.104:0/1004952855 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4df806ac70 con 0x7f4e080b0330 2021-07-11T04:01:00.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.415+0000 7f4e0e339700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4de8053270 0x7f4de8055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:00.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.415+0000 7f4e0e339700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4de8053270 0x7f4de8055730 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f4e00003310 tx=0x7f4e00006d30).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:00.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.415+0000 7f4df77fe700 1 -- 172.21.2.104:0/1004952855 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4df8041ea0 con 0x7f4e080b0330 2021-07-11T04:01:00.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 -- 172.21.2.104:0/1004952855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e080a97a0 con 0x7f4e080b0330 2021-07-11T04:01:00.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4df77fe700 1 -- 172.21.2.104:0/1004952855 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2484 (secure 0 0 0) 0x7f4df80189d0 con 0x7f4e080b0330 2021-07-11T04:01:00.524 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:00.524 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":50,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/3293345379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3293345379},{"type":"v1","addr":"172.21.2.105:6835","nonce":3293345379}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/57099992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":57099992},{"type":"v1","addr":"172.21.2.104:6833","nonce":57099992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:55.667970+0000","modified":"2021-07-11T04:00:56.671726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7287":{"gid":7287,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/871079070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":871079070},{"type":"v1","addr":"172.21.2.104:6835","nonce":871079070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-07-11T04:01:00.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 -- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4de8053270 msgr2=0x7f4de8055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4de8053270 0x7f4de8055730 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f4e00003310 tx=0x7f4e00006d30).stop 2021-07-11T04:01:00.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 -- 172.21.2.104:0/1004952855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4e080b0330 msgr2=0x7f4e080a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:00.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4e080b0330 0x7f4e080a1840 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f4df8001110 tx=0x7f4df8007ad0).stop 2021-07-11T04:01:00.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 -- 172.21.2.104:0/1004952855 shutdown_connections 2021-07-11T04:01:00.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4e080b0960 0x7f4e080a1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4de8053270 0x7f4de8055730 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4e080b0330 0x7f4e080a1840 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:00.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 --2- 172.21.2.104:0/1004952855 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4e08067460 0x7f4e080b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:00.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 -- 172.21.2.104:0/1004952855 >> 172.21.2.104:0/1004952855 conn(0x7f4e08061070 msgr2=0x7f4e080b6160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:00.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 -- 172.21.2.104:0/1004952855 shutdown_connections 2021-07-11T04:01:00.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:00.523+0000 7f4e1059d700 1 -- 172.21.2.104:0/1004952855 wait complete. 2021-07-11T04:01:00.528 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 50 2021-07-11T04:01:00.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:00:55.667970+0000', 'modified': '2021-07-11T04:00:56.671726+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7287': {'gid': 7287, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/871079070', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 871079070}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 871079070}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:01:00.834 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:01:00.835 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:01:00.900 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":7287,"want_state":"up:active","state":"up:active","rank_uptime":5.226905565,"mdsmap_epoch":50,"osdmap_epoch":108,"osdmap_epoch_barrier":107,"uptime":9.3596270700000002} 2021-07-11T04:01:00.901 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 7287, "mdsmap_epoch": 50, "osdmap_epoch": 108, "osdmap_epoch_barrier": 107, "rank_uptime": 5.226905565, "state": "up:active", "uptime": 9.35962707, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:01:00.902 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:01:00.902 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-07-11T04:01:00.902 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba023.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-07-11T04:01:00.903 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:00.903 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:00.914 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:01:00.927 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:00.927 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:00.934 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:00.935 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:00.936 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:00.936 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:01:00.980 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:01:00.981 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:00.984 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:01:00.994 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:00.994 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:01.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14917]: starting ceph client 2021-07-11T04:01:01.024 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:2021-07-11T04:01:01.021+0000 7f53e43df080 -1 init, newargv = 0x5592538e1420 newargc=9 2021-07-11T04:01:01.029 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:01.030 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:01.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:01.031 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:01:01.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14917]: starting fuse 2021-07-11T04:01:01.042 INFO:teuthology.orchestra.run.gibba023.stdout:52 2021-07-11T04:01:01.042 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:01:01.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:01.043 DEBUG:teuthology.orchestra.run.gibba023:> sudo python3 -c ' 2021-07-11T04:01:01.044 DEBUG:teuthology.orchestra.run.gibba023:> import glob 2021-07-11T04:01:01.044 DEBUG:teuthology.orchestra.run.gibba023:> import re 2021-07-11T04:01:01.045 DEBUG:teuthology.orchestra.run.gibba023:> import os 2021-07-11T04:01:01.045 DEBUG:teuthology.orchestra.run.gibba023:> import subprocess 2021-07-11T04:01:01.046 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:01:01.046 DEBUG:teuthology.orchestra.run.gibba023:> def find_socket(client_name): 2021-07-11T04:01:01.047 DEBUG:teuthology.orchestra.run.gibba023:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-07-11T04:01:01.047 DEBUG:teuthology.orchestra.run.gibba023:> files = glob.glob(asok_path) 2021-07-11T04:01:01.047 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:01:01.048 DEBUG:teuthology.orchestra.run.gibba023:> # Given a non-glob path, it better be there 2021-07-11T04:01:01.048 DEBUG:teuthology.orchestra.run.gibba023:> if "*" not in asok_path: 2021-07-11T04:01:01.049 DEBUG:teuthology.orchestra.run.gibba023:> assert(len(files) == 1) 2021-07-11T04:01:01.049 DEBUG:teuthology.orchestra.run.gibba023:> return files[0] 2021-07-11T04:01:01.049 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:01:01.050 DEBUG:teuthology.orchestra.run.gibba023:> for f in files: 2021-07-11T04:01:01.050 DEBUG:teuthology.orchestra.run.gibba023:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:01:01.051 DEBUG:teuthology.orchestra.run.gibba023:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:01:01.051 DEBUG:teuthology.orchestra.run.gibba023:> return f 2021-07-11T04:01:01.052 DEBUG:teuthology.orchestra.run.gibba023:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:01:01.052 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:01:01.052 DEBUG:teuthology.orchestra.run.gibba023:> print(find_socket("client.1")) 2021-07-11T04:01:01.053 DEBUG:teuthology.orchestra.run.gibba023:> ' 2021-07-11T04:01:01.109 INFO:teuthology.orchestra.run.gibba023.stdout:/var/run/ceph/ceph-client.1.14917.asok 2021-07-11T04:01:01.112 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.14917.asok 2021-07-11T04:01:01.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:01.113 DEBUG:teuthology.orchestra.run.gibba023:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.14917.asok status 2021-07-11T04:01:01.162 INFO:teuthology.orchestra.run.gibba023.stdout:{ 2021-07-11T04:01:01.162 INFO:teuthology.orchestra.run.gibba023.stdout: "metadata": { 2021-07-11T04:01:01.163 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:01:01.163 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:01:01.163 INFO:teuthology.orchestra.run.gibba023.stdout: "entity_id": "1", 2021-07-11T04:01:01.163 INFO:teuthology.orchestra.run.gibba023.stdout: "hostname": "gibba023", 2021-07-11T04:01:01.164 INFO:teuthology.orchestra.run.gibba023.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-07-11T04:01:01.164 INFO:teuthology.orchestra.run.gibba023.stdout: "pid": "14917", 2021-07-11T04:01:01.164 INFO:teuthology.orchestra.run.gibba023.stdout: "root": "/" 2021-07-11T04:01:01.164 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:01:01.164 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_count": 0, 2021-07-11T04:01:01.165 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_pinned_count": 0, 2021-07-11T04:01:01.165 INFO:teuthology.orchestra.run.gibba023.stdout: "id": 5137, 2021-07-11T04:01:01.165 INFO:teuthology.orchestra.run.gibba023.stdout: "inst": { 2021-07-11T04:01:01.165 INFO:teuthology.orchestra.run.gibba023.stdout: "name": { 2021-07-11T04:01:01.166 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "client", 2021-07-11T04:01:01.166 INFO:teuthology.orchestra.run.gibba023.stdout: "num": 5137 2021-07-11T04:01:01.166 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:01:01.166 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:01:01.166 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:01:01.167 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:01:01.167 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 2765629900 2021-07-11T04:01:01.167 INFO:teuthology.orchestra.run.gibba023.stdout: } 2021-07-11T04:01:01.167 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:01:01.168 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:01:01.168 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:01:01.168 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:01:01.168 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 2765629900 2021-07-11T04:01:01.168 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:01:01.169 INFO:teuthology.orchestra.run.gibba023.stdout: "inst_str": "client.5137 172.21.2.123:0/2765629900", 2021-07-11T04:01:01.169 INFO:teuthology.orchestra.run.gibba023.stdout: "addr_str": "172.21.2.123:0/2765629900", 2021-07-11T04:01:01.169 INFO:teuthology.orchestra.run.gibba023.stdout: "inode_count": 1, 2021-07-11T04:01:01.169 INFO:teuthology.orchestra.run.gibba023.stdout: "mds_epoch": 50, 2021-07-11T04:01:01.170 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch": 108, 2021-07-11T04:01:01.170 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:01:01.170 INFO:teuthology.orchestra.run.gibba023.stdout: "blacklisted": false, 2021-07-11T04:01:01.170 INFO:teuthology.orchestra.run.gibba023.stdout: "fs_name": "cephfs" 2021-07-11T04:01:01.171 INFO:teuthology.orchestra.run.gibba023.stdout:} 2021-07-11T04:01:01.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:01.172 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:01.173 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:01.178 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:01:01.178 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:01:01.179 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:01.179 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:01.188 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2021-07-11T04:01:01.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 Processor -- start 2021-07-11T04:01:01.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- start start 2021-07-11T04:01:01.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcf700b1db0 0x7fcf700b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcf700a70c0 0x7fcf70144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf700b0350 con 0x7fcf701483a0 2021-07-11T04:01:01.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf700a90c0 con 0x7fcf700b1db0 2021-07-11T04:01:01.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf700b05f0 con 0x7fcf700a70c0 2021-07-11T04:01:01.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcf700b1db0 0x7fcf700b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:01.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fcf700b1db0 0x7fcf700b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48922/0 (socket says 172.21.2.104:48922) 2021-07-11T04:01:01.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 -- 172.21.2.104:0/4281507463 learned_addr learned my addr 172.21.2.104:0/4281507463 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:01.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 -- 172.21.2.104:0/4281507463 >> v2:172.21.2.105:3300/0 conn(0x7fcf700a70c0 msgr2=0x7fcf70144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:01.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 --2- 172.21.2.104:0/4281507463 >> v2:172.21.2.105:3300/0 conn(0x7fcf700a70c0 0x7fcf70144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 -- 172.21.2.104:0/4281507463 >> v1:172.21.2.105:6789/0 conn(0x7fcf701483a0 legacy=0x7fcf700b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:01.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 -- 172.21.2.104:0/4281507463 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf700b0890 con 0x7fcf700b1db0 2021-07-11T04:01:01.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 --2- 172.21.2.104:0/4281507463 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf700b1db0 0x7fcf700b0fd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7fcf70147730 tx=0x7fcf64006eb0).ready entity=mon.0 client_cookie=1f6acd98bc151b1b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:01.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7544b700 1 -- 172.21.2.104:0/4281507463 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6400d5c0 con 0x7fcf700b1db0 2021-07-11T04:01:01.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7544b700 1 -- 172.21.2.104:0/4281507463 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf64000e90 con 0x7fcf700b1db0 2021-07-11T04:01:01.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7544b700 1 -- 172.21.2.104:0/4281507463 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf6400d950 con 0x7fcf700b1db0 2021-07-11T04:01:01.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- 172.21.2.104:0/4281507463 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf700b1db0 msgr2=0x7fcf700b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:01.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 --2- 172.21.2.104:0/4281507463 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf700b1db0 0x7fcf700b0fd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7fcf70147730 tx=0x7fcf64006eb0).stop 2021-07-11T04:01:01.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- 172.21.2.104:0/4281507463 shutdown_connections 2021-07-11T04:01:01.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 --2- 172.21.2.104:0/4281507463 >> v2:172.21.2.105:3300/0 conn(0x7fcf700a70c0 0x7fcf70144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 --2- 172.21.2.104:0/4281507463 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf700b1db0 0x7fcf700b0fd0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:01.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- 172.21.2.104:0/4281507463 >> 172.21.2.104:0/4281507463 conn(0x7fcf7005f6b0 msgr2=0x7fcf7005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:01.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- 172.21.2.104:0/4281507463 shutdown_connections 2021-07-11T04:01:01.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- 172.21.2.104:0/4281507463 wait complete. 2021-07-11T04:01:01.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 Processor -- start 2021-07-11T04:01:01.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- start start 2021-07-11T04:01:01.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf700a70c0 0x7fcf700af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf700b1db0 0x7fcf7014e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf701483a0 0x7fcf7015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7015a070 con 0x7fcf700a70c0 2021-07-11T04:01:01.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf70146a90 con 0x7fcf701483a0 2021-07-11T04:01:01.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf786b1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf70146d30 con 0x7fcf700b1db0 2021-07-11T04:01:01.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf700a70c0 0x7fcf700af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:01.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf700a70c0 0x7fcf700af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48928/0 (socket says 172.21.2.104:48928) 2021-07-11T04:01:01.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 -- 172.21.2.104:0/3803874406 learned_addr learned my addr 172.21.2.104:0/3803874406 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:01.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 -- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf700b1db0 msgr2=0x7fcf7014e890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:01.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf700b1db0 0x7fcf7014e890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 -- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf701483a0 msgr2=0x7fcf7015a5d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:01.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf701483a0 0x7fcf7015a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.243+0000 7fcf7644d700 1 -- 172.21.2.104:0/3803874406 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6400b1e0 con 0x7fcf700a70c0 2021-07-11T04:01:01.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf7644d700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf700a70c0 0x7fcf700af9c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7fcf70147730 tx=0x7fcf64015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:01.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf637fe700 1 -- 172.21.2.104:0/3803874406 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf64021070 con 0x7fcf700a70c0 2021-07-11T04:01:01.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf637fe700 1 -- 172.21.2.104:0/3803874406 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf64000e90 con 0x7fcf700a70c0 2021-07-11T04:01:01.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf786b1700 1 -- 172.21.2.104:0/3803874406 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf70146fd0 con 0x7fcf700a70c0 2021-07-11T04:01:01.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf786b1700 1 -- 172.21.2.104:0/3803874406 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf70145b30 con 0x7fcf700a70c0 2021-07-11T04:01:01.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf637fe700 1 -- 172.21.2.104:0/3803874406 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcf64006be0 con 0x7fcf700a70c0 2021-07-11T04:01:01.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf786b1700 1 -- 172.21.2.104:0/3803874406 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf70057350 con 0x7fcf700a70c0 2021-07-11T04:01:01.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf637fe700 1 -- 172.21.2.104:0/3803874406 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcf6400f070 con 0x7fcf700a70c0 2021-07-11T04:01:01.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf637fe700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf500532d0 0x7fcf50055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf637fe700 1 -- 172.21.2.104:0/3803874406 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcf6406ca20 con 0x7fcf700a70c0 2021-07-11T04:01:01.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf75c4c700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf500532d0 0x7fcf50055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:01.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf75c4c700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf500532d0 0x7fcf50055790 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fcf6c002f10 tx=0x7fcf6c0068b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:01.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.247+0000 7fcf637fe700 1 -- 172.21.2.104:0/3803874406 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcf64043d00 con 0x7fcf700a70c0 2021-07-11T04:01:01.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.359+0000 7fcf786b1700 1 -- 172.21.2.104:0/3803874406 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7fcf70147e60 con 0x7fcf700a70c0 2021-07-11T04:01:01.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.359+0000 7fcf637fe700 1 -- 172.21.2.104:0/3803874406 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1936 (secure 0 0 0) 0x7fcf64047320 con 0x7fcf700a70c0 2021-07-11T04:01:01.362 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:01.362 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:01:01.362 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:01.363 INFO:teuthology.orchestra.run.gibba004.stdout: "total_bytes": 773094113280, 2021-07-11T04:01:01.363 INFO:teuthology.orchestra.run.gibba004.stdout: "total_avail_bytes": 764493283328, 2021-07-11T04:01:01.363 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_bytes": 10895360, 2021-07-11T04:01:01.363 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_bytes": 8600829952, 2021-07-11T04:01:01.363 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_ratio": 0.011125204153358936, 2021-07-11T04:01:01.364 INFO:teuthology.orchestra.run.gibba004.stdout: "num_osds": 8, 2021-07-11T04:01:01.364 INFO:teuthology.orchestra.run.gibba004.stdout: "num_per_pool_osds": 8, 2021-07-11T04:01:01.364 INFO:teuthology.orchestra.run.gibba004.stdout: "num_per_pool_omap_osds": 8 2021-07-11T04:01:01.364 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:01.365 INFO:teuthology.orchestra.run.gibba004.stdout: "stats_by_class": { 2021-07-11T04:01:01.365 INFO:teuthology.orchestra.run.gibba004.stdout: "ssd": { 2021-07-11T04:01:01.365 INFO:teuthology.orchestra.run.gibba004.stdout: "total_bytes": 773094113280, 2021-07-11T04:01:01.365 INFO:teuthology.orchestra.run.gibba004.stdout: "total_avail_bytes": 764493283328, 2021-07-11T04:01:01.366 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_bytes": 10895360, 2021-07-11T04:01:01.366 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_bytes": 8600829952, 2021-07-11T04:01:01.366 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_ratio": 0.011125204153358936 2021-07-11T04:01:01.367 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:01.367 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:01.367 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:01:01.367 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:01.367 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "device_health_metrics", 2021-07-11T04:01:01.368 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 1, 2021-07-11T04:01:01.368 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:01.368 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 0, 2021-07-11T04:01:01.368 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 2, 2021-07-11T04:01:01.368 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 0, 2021-07-11T04:01:01.369 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 0, 2021-07-11T04:01:01.369 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 0, 2021-07-11T04:01:01.369 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362919231488 2021-07-11T04:01:01.369 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:01.370 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:01.370 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:01.370 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "rbd", 2021-07-11T04:01:01.370 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 2, 2021-07-11T04:01:01.370 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:01.371 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 0, 2021-07-11T04:01:01.371 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 0, 2021-07-11T04:01:01.371 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 0, 2021-07-11T04:01:01.371 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 0, 2021-07-11T04:01:01.371 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 0, 2021-07-11T04:01:01.372 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362919231488 2021-07-11T04:01:01.372 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:01.372 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:01.372 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:01.373 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "cephfs_metadata", 2021-07-11T04:01:01.373 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 19, 2021-07-11T04:01:01.373 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:01.373 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 2741, 2021-07-11T04:01:01.374 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 21, 2021-07-11T04:01:01.374 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 64, 2021-07-11T04:01:01.374 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 65536, 2021-07-11T04:01:01.374 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 9.0290050991370663e-08, 2021-07-11T04:01:01.374 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362919231488 2021-07-11T04:01:01.375 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:01.375 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:01.375 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:01.375 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "cephfs_data", 2021-07-11T04:01:01.376 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 20, 2021-07-11T04:01:01.376 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:01.376 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 0, 2021-07-11T04:01:01.377 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 0, 2021-07-11T04:01:01.377 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 0, 2021-07-11T04:01:01.377 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 0, 2021-07-11T04:01:01.377 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 0, 2021-07-11T04:01:01.378 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362919231488 2021-07-11T04:01:01.378 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:01.378 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:01.378 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:01.379 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:01:01.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 -- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf500532d0 msgr2=0x7fcf50055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:01.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf500532d0 0x7fcf50055790 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fcf6c002f10 tx=0x7fcf6c0068b0).stop 2021-07-11T04:01:01.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 -- 172.21.2.104:0/3803874406 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf700a70c0 msgr2=0x7fcf700af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:01.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf700a70c0 0x7fcf700af9c0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7fcf70147730 tx=0x7fcf64015000).stop 2021-07-11T04:01:01.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 -- 172.21.2.104:0/3803874406 shutdown_connections 2021-07-11T04:01:01.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcf701483a0 0x7fcf7015a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcf700b1db0 0x7fcf7014e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcf500532d0 0x7fcf50055790 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:01.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 --2- 172.21.2.104:0/3803874406 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcf700a70c0 0x7fcf700af9c0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:01.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 -- 172.21.2.104:0/3803874406 >> 172.21.2.104:0/3803874406 conn(0x7fcf7005f6b0 msgr2=0x7fcf70142b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:01.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 -- 172.21.2.104:0/3803874406 shutdown_connections 2021-07-11T04:01:01.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.363+0000 7fcf786b1700 1 -- 172.21.2.104:0/3803874406 wait complete. 2021-07-11T04:01:01.383 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data size -f json-pretty 2021-07-11T04:01:01.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 Processor -- start 2021-07-11T04:01:01.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 -- start start 2021-07-11T04:01:01.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc0300a9ec0 0x7fc0300aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc03013f790 0x7fc03013fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc030061990 con 0x7fc0300603d0 2021-07-11T04:01:01.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0301400f0 con 0x7fc03013f790 2021-07-11T04:01:01.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc030061c30 con 0x7fc0300a9ec0 2021-07-11T04:01:01.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02e7fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc0300603d0 0x7fc03013ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45704/0 (socket says 172.21.2.104:45704) 2021-07-11T04:01:01.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02e7fc700 1 -- 172.21.2.104:0/1321248439 learned_addr learned my addr 172.21.2.104:0/1321248439 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:01.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02f7fe700 1 --2- 172.21.2.104:0/1321248439 >> v2:172.21.2.104:3300/0 conn(0x7fc03013f790 0x7fc03013fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:01.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02dffb700 1 -- 172.21.2.104:0/1321248439 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181460208 0 0) 0x7fc030061990 con 0x7fc0300603d0 2021-07-11T04:01:01.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02dffb700 1 -- 172.21.2.104:0/1321248439 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc010003260 con 0x7fc0300603d0 2021-07-11T04:01:01.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02f7fe700 1 -- 172.21.2.104:0/1321248439 >> v2:172.21.2.105:3300/0 conn(0x7fc0300a9ec0 msgr2=0x7fc0300aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:01.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02f7fe700 1 --2- 172.21.2.104:0/1321248439 >> v2:172.21.2.105:3300/0 conn(0x7fc0300a9ec0 0x7fc0300aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02f7fe700 1 -- 172.21.2.104:0/1321248439 >> v1:172.21.2.104:6789/0 conn(0x7fc0300603d0 legacy=0x7fc03013ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:01.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02f7fe700 1 -- 172.21.2.104:0/1321248439 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc030060b90 con 0x7fc03013f790 2021-07-11T04:01:01.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02f7fe700 1 --2- 172.21.2.104:0/1321248439 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc03013f790 0x7fc03013fc00 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7fc0300aaaf0 tx=0x7fc0280091d0).ready entity=mon.0 client_cookie=912a6be22b7a35c6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:01.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02dffb700 1 -- 172.21.2.104:0/1321248439 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc02800b680 con 0x7fc03013f790 2021-07-11T04:01:01.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02dffb700 1 -- 172.21.2.104:0/1321248439 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02800b7e0 con 0x7fc03013f790 2021-07-11T04:01:01.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc02dffb700 1 -- 172.21.2.104:0/1321248439 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc02800b950 con 0x7fc03013f790 2021-07-11T04:01:01.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 -- 172.21.2.104:0/1321248439 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc03013f790 msgr2=0x7fc03013fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:01.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 --2- 172.21.2.104:0/1321248439 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc03013f790 0x7fc03013fc00 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7fc0300aaaf0 tx=0x7fc0280091d0).stop 2021-07-11T04:01:01.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 -- 172.21.2.104:0/1321248439 shutdown_connections 2021-07-11T04:01:01.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 --2- 172.21.2.104:0/1321248439 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc03013f790 0x7fc03013fc00 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:01.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 --2- 172.21.2.104:0/1321248439 >> v2:172.21.2.105:3300/0 conn(0x7fc0300a9ec0 0x7fc0300aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 -- 172.21.2.104:0/1321248439 >> 172.21.2.104:0/1321248439 conn(0x7fc0300b2970 msgr2=0x7fc0300b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:01.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 -- 172.21.2.104:0/1321248439 shutdown_connections 2021-07-11T04:01:01.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 -- 172.21.2.104:0/1321248439 wait complete. 2021-07-11T04:01:01.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 Processor -- start 2021-07-11T04:01:01.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.435+0000 7fc03561d700 1 -- start start 2021-07-11T04:01:01.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc03561d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc0300603d0 0x7fc03014e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc03561d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc0300a9ec0 0x7fc030135fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc03561d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc03013f790 0x7fc0301364b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc03561d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03014eff0 con 0x7fc0300603d0 2021-07-11T04:01:01.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc03561d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc030141eb0 con 0x7fc03013f790 2021-07-11T04:01:01.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc03561d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc030142950 con 0x7fc0300a9ec0 2021-07-11T04:01:01.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc02effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc0300603d0 0x7fc03014e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:01.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc02effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc0300603d0 0x7fc03014e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48942/0 (socket says 172.21.2.104:48942) 2021-07-11T04:01:01.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc02effd700 1 -- 172.21.2.104:0/1968486106 learned_addr learned my addr 172.21.2.104:0/1968486106 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:01.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc02effd700 1 -- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc0300a9ec0 msgr2=0x7fc030135fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:01.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc02effd700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc0300a9ec0 0x7fc030135fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc02effd700 1 -- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc03013f790 msgr2=0x7fc0301364b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:01.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc02effd700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc03013f790 0x7fc0301364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc02effd700 1 -- 172.21.2.104:0/1968486106 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc028009470 con 0x7fc0300603d0 2021-07-11T04:01:01.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc02effd700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc0300603d0 0x7fc03014e780 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7fc018002f70 tx=0x7fc018006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:01.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc027fff700 1 -- 172.21.2.104:0/1968486106 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc018013460 con 0x7fc0300603d0 2021-07-11T04:01:01.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc027fff700 1 -- 172.21.2.104:0/1968486106 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0180135c0 con 0x7fc0300603d0 2021-07-11T04:01:01.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc027fff700 1 -- 172.21.2.104:0/1968486106 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc018013920 con 0x7fc0300603d0 2021-07-11T04:01:01.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc03561d700 1 -- 172.21.2.104:0/1968486106 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc030142f40 con 0x7fc0300603d0 2021-07-11T04:01:01.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc03561d700 1 -- 172.21.2.104:0/1968486106 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0300b00a0 con 0x7fc0300603d0 2021-07-11T04:01:01.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc03561d700 1 -- 172.21.2.104:0/1968486106 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc030142b50 con 0x7fc0300603d0 2021-07-11T04:01:01.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc027fff700 1 -- 172.21.2.104:0/1968486106 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc01801a060 con 0x7fc0300603d0 2021-07-11T04:01:01.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc027fff700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc0140531d0 0x7fc014055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc027fff700 1 -- 172.21.2.104:0/1968486106 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc01806af10 con 0x7fc0300603d0 2021-07-11T04:01:01.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.439+0000 7fc02e7fc700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc0140531d0 0x7fc014055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:01.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.443+0000 7fc02e7fc700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc0140531d0 0x7fc014055690 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fc020001110 tx=0x7fc0200076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:01.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.443+0000 7fc027fff700 1 -- 172.21.2.104:0/1968486106 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc018042250 con 0x7fc0300603d0 2021-07-11T04:01:01.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.523+0000 7fc03561d700 1 -- 172.21.2.104:0/1968486106 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"} v 0) v1 -- 0x7fc030057350 con 0x7fc0300603d0 2021-07-11T04:01:01.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.523+0000 7fc027fff700 1 -- 172.21.2.104:0/1968486106 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "size", "format": "json-pretty"}]=0 v108) v1 ==== 123+0+64 (secure 0 0 0) 0x7fc018018070 con 0x7fc0300603d0 2021-07-11T04:01:01.526 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:01.526 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:01:01.527 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": "cephfs_data", 2021-07-11T04:01:01.527 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_id": 20, 2021-07-11T04:01:01.527 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2 2021-07-11T04:01:01.527 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:01:01.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 -- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc0140531d0 msgr2=0x7fc014055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:01.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc0140531d0 0x7fc014055690 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fc020001110 tx=0x7fc0200076c0).stop 2021-07-11T04:01:01.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 -- 172.21.2.104:0/1968486106 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc0300603d0 msgr2=0x7fc03014e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:01.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc0300603d0 0x7fc03014e780 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7fc018002f70 tx=0x7fc018006ec0).stop 2021-07-11T04:01:01.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 -- 172.21.2.104:0/1968486106 shutdown_connections 2021-07-11T04:01:01.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc03013f790 0x7fc0301364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc0300a9ec0 0x7fc030135fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:01.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc0140531d0 0x7fc014055690 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:01.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 --2- 172.21.2.104:0/1968486106 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc0300603d0 0x7fc03014e780 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:01.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 -- 172.21.2.104:0/1968486106 >> 172.21.2.104:0/1968486106 conn(0x7fc0300b2970 msgr2=0x7fc0300b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:01.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 -- 172.21.2.104:0/1968486106 shutdown_connections 2021-07-11T04:01:01.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.527+0000 7fc03561d700 1 -- 172.21.2.104:0/1968486106 wait complete. 2021-07-11T04:01:01.533 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-07-11T04:01:01.533 DEBUG:teuthology.orchestra.run.gibba019:> cd /home/ubuntu/cephtest/mnt.0 && sudo df . 2021-07-11T04:01:01.560 INFO:teuthology.orchestra.run.gibba019.stdout:Filesystem 1K-blocks Used Available Use% Mounted on 2021-07-11T04:01:01.560 INFO:teuthology.orchestra.run.gibba019.stdout:ceph-fuse 354410496 0 354410496 0% /home/ubuntu/cephtest/mnt.0 2021-07-11T04:01:01.561 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:01:01.562 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:01.590 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:01.603 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:01.632 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:01.646 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:01:01.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:01.647 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:01.647 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:01.658 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:01:01.659 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:01:01.660 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T04:01:01.660 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:01:01.661 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:01.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15483]: fuse finished with error 0 and tester_r 0 2021-07-11T04:01:01.683 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:01.683 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:01.684 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:01.732 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T04:01:01.734 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:01:01.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:01:01.779 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:01.788 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:01.789 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:01.789 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:01.800 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:01:01.802 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:01:01.802 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba023.front.sepia.ceph.com... 2021-07-11T04:01:01.802 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:01:01.803 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:01.817 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[14917]: fuse finished with error 0 and tester_r 0 2021-07-11T04:01:01.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:01.819 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:01.819 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:01.824 INFO:teuthology.orchestra.run.gibba023.stdout:ext2/ext3 2021-07-11T04:01:01.824 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:01:01.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:01:01.935 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:01.940 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:01:01.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.995+0000 7f2bfe283700 1 Processor -- start 2021-07-11T04:01:01.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.995+0000 7f2bfe283700 1 -- start start 2021-07-11T04:01:01.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.995+0000 7f2bfe283700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2bf80ad0d0 0x7f2bf813f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.995+0000 7f2bfe283700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2bf80b0b30 0x7f2bf813bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:01.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.995+0000 7f2bfe283700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf80b0430 con 0x7f2bf80afc20 2021-07-11T04:01:01.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.995+0000 7f2bfe283700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf80ad4b0 con 0x7f2bf80ad0d0 2021-07-11T04:01:01.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.995+0000 7f2bfe283700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf80b5750 con 0x7f2bf80b0b30 2021-07-11T04:01:01.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf7fff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2bf80ad0d0 0x7f2bf813f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:01.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfc820700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2bf80b0b30 0x7f2bf813bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf7fff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2bf80ad0d0 0x7f2bf813f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:56508/0 (socket says 172.21.2.104:56508) 2021-07-11T04:01:02.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf7fff700 1 -- 172.21.2.104:0/3571132397 learned_addr learned my addr 172.21.2.104:0/3571132397 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:02.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf77fe700 1 --1- 172.21.2.104:0/3571132397 >> v1:172.21.2.105:6789/0 conn(0x7f2bf80afc20 0x7f2bf813f5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:45086/0 (socket says 172.21.2.104:45086) 2021-07-11T04:01:02.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf7fff700 1 -- 172.21.2.104:0/3571132397 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2bf80b0b30 msgr2=0x7f2bf813bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:02.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf7fff700 1 --2- 172.21.2.104:0/3571132397 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2bf80b0b30 0x7f2bf813bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf7fff700 1 -- 172.21.2.104:0/3571132397 >> v1:172.21.2.105:6789/0 conn(0x7f2bf80afc20 legacy=0x7f2bf813f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf7fff700 1 -- 172.21.2.104:0/3571132397 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf80b0660 con 0x7f2bf80ad0d0 2021-07-11T04:01:02.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf7fff700 1 --2- 172.21.2.104:0/3571132397 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2bf80ad0d0 0x7f2bf813f0d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f2be0002f70 tx=0x7f2be0006e40).ready entity=mon.1 client_cookie=2c389fc8c2227a1d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf6ffd700 1 -- 172.21.2.104:0/3571132397 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2be000bc90 con 0x7f2bf80ad0d0 2021-07-11T04:01:02.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf6ffd700 1 -- 172.21.2.104:0/3571132397 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be0000e90 con 0x7f2bf80ad0d0 2021-07-11T04:01:02.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bf6ffd700 1 -- 172.21.2.104:0/3571132397 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2be00153f0 con 0x7f2bf80ad0d0 2021-07-11T04:01:02.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 -- 172.21.2.104:0/3571132397 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2bf80ad0d0 msgr2=0x7f2bf813f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 --2- 172.21.2.104:0/3571132397 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2bf80ad0d0 0x7f2bf813f0d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f2be0002f70 tx=0x7f2be0006e40).stop 2021-07-11T04:01:02.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 -- 172.21.2.104:0/3571132397 shutdown_connections 2021-07-11T04:01:02.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 --2- 172.21.2.104:0/3571132397 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2bf80b0b30 0x7f2bf813bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 --2- 172.21.2.104:0/3571132397 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2bf80ad0d0 0x7f2bf813f0d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 -- 172.21.2.104:0/3571132397 >> 172.21.2.104:0/3571132397 conn(0x7f2bf8062c30 msgr2=0x7f2bf8063010 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:02.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 -- 172.21.2.104:0/3571132397 shutdown_connections 2021-07-11T04:01:02.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 -- 172.21.2.104:0/3571132397 wait complete. 2021-07-11T04:01:02.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 Processor -- start 2021-07-11T04:01:02.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 -- start start 2021-07-11T04:01:02.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2bf80ad0d0 0x7f2bf81395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2bf80afc20 0x7f2bf8151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bf80b0b30 0x7f2bf8067560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf80642e0 con 0x7f2bf80b0b30 2021-07-11T04:01:02.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf814e6f0 con 0x7f2bf80afc20 2021-07-11T04:01:02.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:01.999+0000 7f2bfe283700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf814e990 con 0x7f2bf80ad0d0 2021-07-11T04:01:02.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfc820700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bf80b0b30 0x7f2bf8067560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfc820700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bf80b0b30 0x7f2bf8067560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48956/0 (socket says 172.21.2.104:48956) 2021-07-11T04:01:02.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfc820700 1 -- 172.21.2.104:0/1890329127 learned_addr learned my addr 172.21.2.104:0/1890329127 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:02.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfc820700 1 -- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2bf80ad0d0 msgr2=0x7f2bf81395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfc820700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2bf80ad0d0 0x7f2bf81395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfc820700 1 -- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2bf80afc20 msgr2=0x7f2bf8151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfc820700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2bf80afc20 0x7f2bf8151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfc820700 1 -- 172.21.2.104:0/1890329127 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be000d4d0 con 0x7f2bf80b0b30 2021-07-11T04:01:02.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfc820700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bf80b0b30 0x7f2bf8067560 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f2bf000d000 tx=0x7f2bf000c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bf57fa700 1 -- 172.21.2.104:0/1890329127 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bf0015650 con 0x7f2bf80b0b30 2021-07-11T04:01:02.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfe283700 1 -- 172.21.2.104:0/1890329127 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf8064510 con 0x7f2bf80b0b30 2021-07-11T04:01:02.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfe283700 1 -- 172.21.2.104:0/1890329127 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf8140450 con 0x7f2bf80b0b30 2021-07-11T04:01:02.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bf57fa700 1 -- 172.21.2.104:0/1890329127 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf0010040 con 0x7f2bf80b0b30 2021-07-11T04:01:02.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bf57fa700 1 -- 172.21.2.104:0/1890329127 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2bf00159a0 con 0x7f2bf80b0b30 2021-07-11T04:01:02.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bfe283700 1 -- 172.21.2.104:0/1890329127 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd0003410 con 0x7f2bf80b0b30 2021-07-11T04:01:02.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bf57fa700 1 -- 172.21.2.104:0/1890329127 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2bf001c030 con 0x7f2bf80b0b30 2021-07-11T04:01:02.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bf57fa700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2bdc053270 0x7f2bdc055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bf57fa700 1 -- 172.21.2.104:0/1890329127 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2bf006b390 con 0x7f2bf80b0b30 2021-07-11T04:01:02.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bf7fff700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2bdc053270 0x7f2bdc055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.003+0000 7f2bf7fff700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2bdc053270 0x7f2bdc055730 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f2be0006fc0 tx=0x7f2be0018000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.007+0000 7f2bf57fa700 1 -- 172.21.2.104:0/1890329127 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2bf0042540 con 0x7f2bf80b0b30 2021-07-11T04:01:02.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 -- 172.21.2.104:0/1890329127 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f2bd00036c0 con 0x7f2bf80b0b30 2021-07-11T04:01:02.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bf57fa700 1 -- 172.21.2.104:0/1890329127 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) v1 ==== 81+0+33659 (secure 0 0 0) 0x7f2bf0045b60 con 0x7f2bf80b0b30 2021-07-11T04:01:02.089 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:02.090 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:01:02.090 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 108, 2021-07-11T04:01:02.091 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:01:02.091 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:01:02.091 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:00:56.672922+0000", 2021-07-11T04:01:02.091 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:01:02.091 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:01:02.092 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:01:02.092 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:01:02.093 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:01:02.093 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:01:02.093 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:01:02.093 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:01:02.094 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:01:02.094 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:02.094 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:01:02.094 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:01:02.095 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:01:02.095 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:01:02.095 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:01:02.095 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 20, 2021-07-11T04:01:02.095 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:01:02.096 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:01:02.096 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:01:02.097 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:01:02.097 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:01:02.097 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.097 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:01:02.098 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:01:02.098 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:01:02.098 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:01:02.098 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:01:02.098 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:02.099 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:02.099 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:02.099 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:02.099 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:02.100 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:02.100 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:01:02.101 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:01:02.101 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:01:02.101 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:01:02.101 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:01:02.101 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:02.102 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:02.102 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:02.102 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:02.102 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:02.103 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:02.103 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:02.104 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.104 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:01:02.104 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:02.104 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:02.105 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:02.105 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:02.105 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:02.105 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:02.106 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:02.106 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:02.106 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:02.106 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:02.107 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:02.107 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:02.107 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:02.108 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:02.108 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:02.108 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:02.108 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:02.109 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:02.109 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:02.109 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:02.109 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:02.110 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:02.110 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:02.110 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:02.110 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:02.111 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:02.111 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.111 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:02.112 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:02.112 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:02.112 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:02.113 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:02.113 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:02.114 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:02.114 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:02.114 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:02.115 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:02.115 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:02.115 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:01:02.115 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:01:02.115 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.116 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:02.116 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:01:02.116 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.116 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.117 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.117 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:01:02.119 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:01:02.119 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:01:02.119 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:01:02.120 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:01:02.120 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:02.120 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:02.120 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:02.120 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:02.121 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:02.121 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:02.122 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:01:02.122 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:01:02.122 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:01:02.122 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:01:02.123 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:01:02.123 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:02.123 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:02.123 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:02.123 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:02.124 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:02.125 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:02.125 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:02.125 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.126 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:01:02.126 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:02.126 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:02.126 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:02.127 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:02.127 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:02.127 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:02.127 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:02.127 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:02.128 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:02.128 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:02.128 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:02.128 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:02.129 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:02.129 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:02.129 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:02.130 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:02.130 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:02.130 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:02.130 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:02.131 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:02.132 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:02.132 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:02.132 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:02.133 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:02.133 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:02.133 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:02.133 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.133 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:02.134 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:02.134 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:02.134 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:02.134 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:02.135 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:02.135 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:02.135 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:02.135 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:02.135 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:02.136 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:02.136 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:01:02.136 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:02.136 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:01:02.137 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.137 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.137 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.137 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 19, 2021-07-11T04:01:02.137 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T04:01:02.138 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:00:53.655121+0000", 2021-07-11T04:01:02.138 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:01:02.138 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:01:02.139 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:02.139 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:02.139 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:02.140 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:02.140 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:02.141 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:02.141 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:01:02.141 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:01:02.141 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:01:02.141 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:01:02.142 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:01:02.142 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:02.142 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:02.142 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:02.143 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:02.143 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:02.143 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:02.143 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:02.143 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.144 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "107", 2021-07-11T04:01:02.144 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:02.144 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:02.144 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:02.145 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:02.145 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:02.145 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:02.145 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:02.145 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:02.146 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:02.146 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:02.146 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:02.146 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:02.147 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:02.147 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:02.147 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:02.147 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:02.147 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:02.148 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:02.148 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:02.148 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:02.148 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:02.149 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:02.149 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:02.149 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:02.149 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:02.149 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:02.150 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.150 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:02.150 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:02.150 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:02.150 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:02.151 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:02.151 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:02.151 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:02.151 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:02.152 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:02.152 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:02.152 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:02.152 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:01:02.152 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T04:01:02.153 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T04:01:02.153 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T04:01:02.153 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.153 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:02.153 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:01:02.154 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T04:01:02.154 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.154 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.154 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.155 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.155 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 20, 2021-07-11T04:01:02.155 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T04:01:02.155 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:00:54.748677+0000", 2021-07-11T04:01:02.156 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:01:02.156 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:01:02.156 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:02.156 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:02.157 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:02.157 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:02.157 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:02.157 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:02.158 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:01:02.158 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:01:02.158 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:01:02.158 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:01:02.158 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:01:02.159 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:02.159 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:02.159 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:02.159 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:02.159 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:02.160 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:02.160 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:02.160 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.160 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "108", 2021-07-11T04:01:02.161 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:02.161 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:02.161 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:02.161 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:02.162 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:02.162 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:02.162 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:02.162 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:02.162 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:02.163 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:02.163 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:02.163 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:02.163 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:02.164 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:02.164 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:02.164 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:02.164 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:02.164 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:02.165 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:02.165 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:02.165 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:02.165 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:02.165 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:02.166 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:02.166 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:02.166 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:02.166 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.167 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:02.167 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:02.171 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:02.172 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:02.172 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:02.172 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:02.173 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:02.173 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:02.173 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:02.173 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:02.174 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:02.174 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:01:02.174 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:02.174 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:01:02.174 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T04:01:02.175 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.175 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.175 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.175 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:02.176 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:01:02.176 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.176 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:01:02.176 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:01:02.177 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:02.177 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:02.177 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:02.177 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:02.178 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:02.178 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:02.178 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:02.179 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 106, 2021-07-11T04:01:02.179 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:02.179 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:02.179 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:02.180 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.180 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.180 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.180 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:01:02.180 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:02.181 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.181 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.181 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.182 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:01:02.182 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:02.182 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.183 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.183 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.183 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:02.183 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.184 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.184 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.184 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:01:02.184 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:02.184 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.185 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.185 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.185 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:01:02.185 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:02.185 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.186 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.186 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.186 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:02.186 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.187 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.187 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.187 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:01:02.188 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:02.188 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.189 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.189 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.189 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:01:02.189 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:02.189 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.190 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.190 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.190 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:02.190 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.191 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.191 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.191 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:01:02.191 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:02.191 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.192 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.193 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.193 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:01:02.193 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:02.193 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.193 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.194 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.194 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:01:02.194 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:01:02.194 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:01:02.195 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:01:02.195 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:02.195 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:02.195 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:02.196 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.196 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.196 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.196 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:01:02.196 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:01:02.197 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:02.197 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:02.197 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:02.198 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:02.198 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:02.198 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:02.199 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:02.199 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 106, 2021-07-11T04:01:02.199 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:02.199 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:02.200 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:02.200 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.200 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.200 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.200 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:01:02.201 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:02.201 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.201 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.201 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.201 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:01:02.202 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:02.202 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.202 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.203 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.203 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:02.203 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.203 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.204 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.204 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:01:02.204 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:02.204 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.205 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.205 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.205 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:01:02.205 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:02.205 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.206 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.206 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.206 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:02.206 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.207 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.207 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.207 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:01:02.207 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:02.208 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.208 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.208 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.208 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:01:02.209 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:02.209 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.209 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.210 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.210 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:02.210 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.210 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.210 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.211 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:01:02.211 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:02.211 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.211 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.212 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.212 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:01:02.212 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:02.212 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.212 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.213 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.213 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:01:02.213 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:01:02.214 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:01:02.214 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:01:02.214 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:02.214 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:02.214 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:02.215 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.215 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.215 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.215 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:01:02.216 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:01:02.216 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:02.216 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:02.216 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:02.216 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:02.217 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:02.217 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:02.217 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:02.218 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 105, 2021-07-11T04:01:02.218 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:02.218 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:02.219 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:02.219 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.219 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.220 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.220 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:01:02.220 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:02.220 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.220 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.221 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.221 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:01:02.221 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:02.221 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.221 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.222 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.222 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:02.222 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.222 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.223 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.223 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:01:02.223 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:02.224 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.224 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.224 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.224 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:01:02.224 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:02.225 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.225 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.225 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.225 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:02.225 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.226 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.226 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.226 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:01:02.226 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:02.227 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.227 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.227 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.227 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:01:02.227 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:02.228 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.228 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.228 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.229 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:02.229 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.229 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.229 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.230 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:01:02.230 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:02.230 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.230 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.231 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.231 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:01:02.231 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:02.231 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.231 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.232 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.232 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:01:02.232 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:01:02.232 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:01:02.233 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:01:02.233 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:02.233 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:02.233 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:02.234 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.234 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.234 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.234 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:01:02.235 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:01:02.235 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:02.235 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:02.235 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:02.235 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:02.236 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:02.236 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:02.236 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:02.236 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 106, 2021-07-11T04:01:02.237 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:02.237 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:02.237 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:02.237 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.237 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.238 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.238 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:01:02.238 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:02.239 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.239 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.239 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.239 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:01:02.240 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:02.240 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.240 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.240 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.241 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:02.241 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.241 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.241 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.241 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:01:02.242 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:02.242 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.242 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.242 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.243 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:01:02.243 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:02.243 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.243 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.244 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.244 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:02.244 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.244 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.245 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.245 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:01:02.245 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:02.245 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.245 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.246 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.246 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:01:02.246 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:02.246 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.247 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.247 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.247 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:02.247 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.247 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.248 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.248 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:01:02.248 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:02.248 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.249 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.249 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.249 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:01:02.249 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:02.250 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.250 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.251 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.251 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:01:02.251 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:01:02.251 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:01:02.251 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:01:02.252 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:02.252 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:02.252 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:02.252 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.252 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.253 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.253 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:01:02.253 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:01:02.253 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:02.254 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:02.254 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:02.254 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:02.254 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:02.255 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:02.255 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:02.255 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 105, 2021-07-11T04:01:02.255 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:02.255 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:02.256 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:02.256 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.256 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.256 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.257 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:01:02.257 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:02.257 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.257 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.257 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.258 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:01:02.258 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:02.258 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.258 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.259 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.259 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:02.259 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.259 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.260 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.260 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:01:02.260 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:02.260 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.260 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.261 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.261 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:01:02.261 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:02.261 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.261 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.262 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.262 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:02.262 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.262 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.263 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.263 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:01:02.263 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:02.263 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.263 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.264 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.264 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:01:02.264 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:02.264 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.264 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.265 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.265 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:02.265 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.265 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.266 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.266 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:01:02.266 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:02.266 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.266 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.267 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.267 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:01:02.267 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:02.267 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.267 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.268 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.268 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:01:02.268 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:01:02.268 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:01:02.268 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:01:02.269 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:02.269 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:02.269 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:02.269 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.270 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.270 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.270 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:01:02.270 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:01:02.270 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:02.271 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:02.271 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:02.271 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:02.271 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:02.271 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:02.272 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:02.272 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 105, 2021-07-11T04:01:02.272 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:02.272 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:02.273 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:02.273 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.273 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.273 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.274 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:01:02.274 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:02.274 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.274 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.274 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.275 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:01:02.275 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:02.275 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.275 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.276 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.276 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:02.276 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.276 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.276 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.277 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:01:02.277 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:02.277 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.277 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.277 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.278 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:01:02.278 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:02.278 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.278 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.279 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.279 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:02.279 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.279 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.279 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.280 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:01:02.280 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:02.280 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.280 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.281 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.281 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:01:02.281 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:02.282 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.282 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.282 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.282 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:02.283 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.283 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.283 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.283 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:01:02.284 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:02.284 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.284 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.284 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.284 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:01:02.285 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:02.285 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.285 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.285 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.286 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:01:02.286 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:01:02.286 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:01:02.286 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:01:02.286 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:02.287 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:02.287 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:02.287 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.287 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.287 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.288 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:01:02.288 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:01:02.288 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:02.288 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:02.289 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:02.289 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:02.289 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:02.289 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:02.289 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:02.290 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 106, 2021-07-11T04:01:02.290 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:02.290 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:02.290 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:02.290 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.291 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.291 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.291 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:01:02.291 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:02.291 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.292 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.292 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.292 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:01:02.292 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:02.293 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.293 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.293 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.293 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:02.293 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.294 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.294 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.294 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:01:02.294 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:02.295 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.295 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.295 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.295 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:01:02.295 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:02.296 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.296 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.296 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.296 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:02.296 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.297 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.297 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.297 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:01:02.297 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:02.297 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.298 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.298 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.298 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:01:02.298 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:02.299 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.299 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.299 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.299 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:02.299 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.300 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.300 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.300 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:01:02.300 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:02.300 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.301 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.301 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.301 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:01:02.301 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:02.301 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.302 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.302 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.302 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:01:02.302 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:01:02.303 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:01:02.303 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:01:02.303 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:02.303 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:02.303 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:02.304 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.304 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.304 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.304 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:01:02.305 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:01:02.305 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:02.305 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:02.305 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:02.305 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:02.306 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:02.306 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:02.306 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:02.306 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 106, 2021-07-11T04:01:02.306 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:02.307 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:02.307 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:02.307 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.307 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.308 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.308 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:01:02.308 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:02.308 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.308 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.309 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.309 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:01:02.309 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:02.309 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.309 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.310 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.310 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:02.310 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.310 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.311 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.311 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:01:02.311 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:02.311 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.311 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.312 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.312 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:01:02.312 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:02.312 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.312 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.313 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.313 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:02.313 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.313 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.313 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.314 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:01:02.314 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:02.314 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.314 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.315 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.315 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:01:02.315 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:02.315 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.315 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.316 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.316 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:02.316 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:02.316 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.317 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:02.317 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:01:02.317 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:02.317 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.317 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.318 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:02.318 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:01:02.318 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:02.318 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.319 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.319 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.319 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:01:02.319 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:01:02.320 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:01:02.320 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:01:02.320 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:02.320 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:02.320 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:02.321 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:02.321 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.321 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:02.321 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:01:02.322 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.322 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:01:02.322 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:02.322 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:02.322 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:02.323 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:02.323 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:02.323 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:01:02.323 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:02.323 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.324 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.324 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:01:02.324 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:02.324 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:02.325 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:02.325 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:02.325 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:02.325 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:01:02.325 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:02.326 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.326 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.326 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:01:02.326 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:02.327 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:02.327 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:02.327 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:02.327 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:02.327 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:01:02.328 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:02.328 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.328 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.328 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:01:02.328 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:02.329 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:02.329 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:02.329 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:02.329 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:02.329 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:01:02.330 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:02.330 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.330 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.331 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:01:02.331 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:02.331 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:02.331 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:02.331 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:02.332 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:02.332 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:01:02.332 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:02.332 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.332 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.333 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:01:02.333 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:02.333 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:02.333 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:02.333 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:02.334 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:02.334 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:01:02.334 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:02.334 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.335 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.335 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:01:02.335 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:02.335 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:02.335 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:02.336 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:02.336 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:02.336 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:01:02.336 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:02.336 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.337 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:02.337 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:01:02.337 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:02.337 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:02.338 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:02.338 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:02.338 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:02.338 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:01:02.339 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:02.339 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.339 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:02.339 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:01:02.339 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:01:02.340 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:01:02.340 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:01:02.340 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": {}, 2021-07-11T04:01:02.340 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:01:02.340 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:01:02.341 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:01:02.341 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:01:02.341 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:01:02.341 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:01:02.341 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:01:02.342 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:01:02.342 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:02.342 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:02.342 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:01:02.342 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:01:02.343 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:01:02.343 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:01:02.343 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:01:02.343 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:01:02.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 -- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2bdc053270 msgr2=0x7f2bdc055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2bdc053270 0x7f2bdc055730 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f2be0006fc0 tx=0x7f2be0018000).stop 2021-07-11T04:01:02.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 -- 172.21.2.104:0/1890329127 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bf80b0b30 msgr2=0x7f2bf8067560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bf80b0b30 0x7f2bf8067560 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f2bf000d000 tx=0x7f2bf000c040).stop 2021-07-11T04:01:02.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 -- 172.21.2.104:0/1890329127 shutdown_connections 2021-07-11T04:01:02.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2bdc053270 0x7f2bdc055730 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2bf80b0b30 0x7f2bf8067560 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2bf80afc20 0x7f2bf8151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 --2- 172.21.2.104:0/1890329127 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2bf80ad0d0 0x7f2bf81395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 -- 172.21.2.104:0/1890329127 >> 172.21.2.104:0/1890329127 conn(0x7f2bf8062c30 msgr2=0x7f2bf80b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:02.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 -- 172.21.2.104:0/1890329127 shutdown_connections 2021-07-11T04:01:02.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.087+0000 7f2bfe283700 1 -- 172.21.2.104:0/1890329127 wait complete. 2021-07-11T04:01:02.348 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:02.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.407+0000 7fcd2248e700 1 Processor -- start 2021-07-11T04:01:02.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.407+0000 7fcd2248e700 1 -- start start 2021-07-11T04:01:02.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.407+0000 7fcd2248e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcd1c0a76b0 0x7fcd1c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.407+0000 7fcd2248e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd1c0b0060 0x7fcd1c05fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.407+0000 7fcd2248e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c0a7a90 con 0x7fcd1c0b1970 2021-07-11T04:01:02.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.407+0000 7fcd2248e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c147620 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.407+0000 7fcd2248e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c1478c0 con 0x7fcd1c0b0060 2021-07-11T04:01:02.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd1bfff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcd1c0a76b0 0x7fcd1c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd20a2b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd1c0b0060 0x7fcd1c05fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd20a2b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd1c0b0060 0x7fcd1c05fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:52128/0 (socket says 172.21.2.104:52128) 2021-07-11T04:01:02.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd20a2b700 1 -- 172.21.2.104:0/605458099 learned_addr learned my addr 172.21.2.104:0/605458099 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:02.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd20a2b700 1 -- 172.21.2.104:0/605458099 >> v2:172.21.2.105:3300/0 conn(0x7fcd1c0a76b0 msgr2=0x7fcd1c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:02.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd20a2b700 1 --2- 172.21.2.104:0/605458099 >> v2:172.21.2.105:3300/0 conn(0x7fcd1c0a76b0 0x7fcd1c066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd20a2b700 1 -- 172.21.2.104:0/605458099 >> v1:172.21.2.105:6789/0 conn(0x7fcd1c0b1970 legacy=0x7fcd1c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd20a2b700 1 -- 172.21.2.104:0/605458099 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c147b60 con 0x7fcd1c0b0060 2021-07-11T04:01:02.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd1bfff700 1 --2- 172.21.2.104:0/605458099 >> v2:172.21.2.105:3300/0 conn(0x7fcd1c0a76b0 0x7fcd1c066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:01:02.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd20a2b700 1 --2- 172.21.2.104:0/605458099 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd1c0b0060 0x7fcd1c05fdc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fcd140032f0 tx=0x7fcd14009290).ready entity=mon.2 client_cookie=51b9df2e3cf0a95f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd1affd700 1 -- 172.21.2.104:0/605458099 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1400b8a0 con 0x7fcd1c0b0060 2021-07-11T04:01:02.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd1affd700 1 -- 172.21.2.104:0/605458099 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1400ba00 con 0x7fcd1c0b0060 2021-07-11T04:01:02.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd1affd700 1 -- 172.21.2.104:0/605458099 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd1400bb70 con 0x7fcd1c0b0060 2021-07-11T04:01:02.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd2248e700 1 -- 172.21.2.104:0/605458099 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd1c0b0060 msgr2=0x7fcd1c05fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd2248e700 1 --2- 172.21.2.104:0/605458099 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd1c0b0060 0x7fcd1c05fdc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fcd140032f0 tx=0x7fcd14009290).stop 2021-07-11T04:01:02.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd2248e700 1 -- 172.21.2.104:0/605458099 shutdown_connections 2021-07-11T04:01:02.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd2248e700 1 --2- 172.21.2.104:0/605458099 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd1c0b0060 0x7fcd1c05fdc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd2248e700 1 --2- 172.21.2.104:0/605458099 >> v2:172.21.2.105:3300/0 conn(0x7fcd1c0a76b0 0x7fcd1c066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd2248e700 1 -- 172.21.2.104:0/605458099 >> 172.21.2.104:0/605458099 conn(0x7fcd1c0b53e0 msgr2=0x7fcd1c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:02.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd2248e700 1 -- 172.21.2.104:0/605458099 shutdown_connections 2021-07-11T04:01:02.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd2248e700 1 -- 172.21.2.104:0/605458099 wait complete. 2021-07-11T04:01:02.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd2248e700 1 Processor -- start 2021-07-11T04:01:02.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.411+0000 7fcd2248e700 1 -- start start 2021-07-11T04:01:02.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd2248e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd1c0a76b0 0x7fcd1c0ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd2248e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd1c0b0060 0x7fcd1c064130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd2248e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd1c0b1970 0x7fcd1c064620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd2248e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c0ad880 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd2248e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c063030 con 0x7fcd1c0b0060 2021-07-11T04:01:02.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd2248e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c064b10 con 0x7fcd1c0b1970 2021-07-11T04:01:02.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd1c0a76b0 0x7fcd1c0ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd1c0a76b0 0x7fcd1c0ab680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48968/0 (socket says 172.21.2.104:48968) 2021-07-11T04:01:02.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1bfff700 1 -- 172.21.2.104:0/3514158345 learned_addr learned my addr 172.21.2.104:0/3514158345 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:02.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1bfff700 1 -- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd1c0b1970 msgr2=0x7fcd1c064620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1bfff700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd1c0b1970 0x7fcd1c064620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1bfff700 1 -- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd1c0b0060 msgr2=0x7fcd1c064130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1b7fe700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd1c0b0060 0x7fcd1c064130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1bfff700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd1c0b0060 0x7fcd1c064130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1bfff700 1 -- 172.21.2.104:0/3514158345 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c00b180 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1b7fe700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd1c0b0060 0x7fcd1c064130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:02.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1bfff700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd1c0a76b0 0x7fcd1c0ab680 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7fcd0c00c000 tx=0x7fcd0c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd197fa700 1 -- 172.21.2.104:0/3514158345 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd0c00ca40 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd197fa700 1 -- 172.21.2.104:0/3514158345 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c00cba0 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd197fa700 1 -- 172.21.2.104:0/3514158345 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcd0c0123f0 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd2248e700 1 -- 172.21.2.104:0/3514158345 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd14009530 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd2248e700 1 -- 172.21.2.104:0/3514158345 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c0adab0 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd2248e700 1 -- 172.21.2.104:0/3514158345 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c057350 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd197fa700 1 -- 172.21.2.104:0/3514158345 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcd0c012550 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd197fa700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccfc0531f0 0x7fccfc0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd197fa700 1 -- 172.21.2.104:0/3514158345 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fcd0c06bc90 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1b7fe700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccfc0531f0 0x7fccfc0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.415+0000 7fcd1b7fe700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccfc0531f0 0x7fccfc0556b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fcd10008000 tx=0x7fcd10009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.419+0000 7fcd197fa700 1 -- 172.21.2.104:0/3514158345 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcd0c042e60 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.523+0000 7fcd2248e700 1 -- 172.21.2.104:0/3514158345 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd1c149880 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.523+0000 7fcd197fa700 1 -- 172.21.2.104:0/3514158345 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+2484 (secure 0 0 0) 0x7fcd0c046480 con 0x7fcd1c0a76b0 2021-07-11T04:01:02.526 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:02.526 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":50,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/3293345379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3293345379},{"type":"v1","addr":"172.21.2.105:6835","nonce":3293345379}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/57099992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":57099992},{"type":"v1","addr":"172.21.2.104:6833","nonce":57099992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:55.667970+0000","modified":"2021-07-11T04:00:56.671726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7287":{"gid":7287,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/871079070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":871079070},{"type":"v1","addr":"172.21.2.104:6835","nonce":871079070}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-07-11T04:01:02.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.523+0000 7fcd2248e700 1 -- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccfc0531f0 msgr2=0x7fccfc0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.523+0000 7fcd2248e700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccfc0531f0 0x7fccfc0556b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fcd10008000 tx=0x7fcd10009040).stop 2021-07-11T04:01:02.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.523+0000 7fcd2248e700 1 -- 172.21.2.104:0/3514158345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd1c0a76b0 msgr2=0x7fcd1c0ab680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.523+0000 7fcd2248e700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd1c0a76b0 0x7fcd1c0ab680 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7fcd0c00c000 tx=0x7fcd0c006df0).stop 2021-07-11T04:01:02.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.527+0000 7fcd2248e700 1 -- 172.21.2.104:0/3514158345 shutdown_connections 2021-07-11T04:01:02.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.527+0000 7fcd2248e700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcd1c0b1970 0x7fcd1c064620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.527+0000 7fcd2248e700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcd1c0b0060 0x7fcd1c064130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.527+0000 7fcd2248e700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fccfc0531f0 0x7fccfc0556b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.527+0000 7fcd2248e700 1 --2- 172.21.2.104:0/3514158345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcd1c0a76b0 0x7fcd1c0ab680 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.527+0000 7fcd2248e700 1 -- 172.21.2.104:0/3514158345 >> 172.21.2.104:0/3514158345 conn(0x7fcd1c0b53e0 msgr2=0x7fcd1c066ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:02.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.527+0000 7fcd2248e700 1 -- 172.21.2.104:0/3514158345 shutdown_connections 2021-07-11T04:01:02.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.527+0000 7fcd2248e700 1 -- 172.21.2.104:0/3514158345 wait complete. 2021-07-11T04:01:02.530 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 50 2021-07-11T04:01:02.532 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T04:01:02.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 Processor -- start 2021-07-11T04:01:02.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 -- start start 2021-07-11T04:01:02.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc4100b1970 0x7fc41005fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4100a7a90 con 0x7fc410067180 2021-07-11T04:01:02.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc410147620 con 0x7fc4100a76b0 2021-07-11T04:01:02.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4101478c0 con 0x7fc4100b1970 2021-07-11T04:01:02.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc4148f7700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc410067180 0x7fc41005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45746/0 (socket says 172.21.2.104:45746) 2021-07-11T04:01:02.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc4148f7700 1 -- 172.21.2.104:0/383486664 learned_addr learned my addr 172.21.2.104:0/383486664 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:02.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758042394 0 0) 0x7fc4100a7a90 con 0x7fc410067180 2021-07-11T04:01:02.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3f8003260 con 0x7fc410067180 2021-07-11T04:01:02.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc4158f9700 1 --2- 172.21.2.104:0/383486664 >> v2:172.21.2.105:3300/0 conn(0x7fc4100b1970 0x7fc41005fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3724493746 0 0) 0x7fc3f8003260 con 0x7fc410067180 2021-07-11T04:01:02.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 >> v2:172.21.2.105:3300/0 conn(0x7fc4100b1970 msgr2=0x7fc41005fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:02.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 --2- 172.21.2.104:0/383486664 >> v2:172.21.2.105:3300/0 conn(0x7fc4100b1970 0x7fc41005fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 >> v1:172.21.2.105:6789/0 conn(0x7fc4100a76b0 legacy=0x7fc410066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:02.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410147b60 con 0x7fc410067180 2021-07-11T04:01:02.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fc40c001660 con 0x7fc410067180 2021-07-11T04:01:02.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 >> v1:172.21.2.104:6789/0 conn(0x7fc410067180 legacy=0x7fc41005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 --2- 172.21.2.104:0/383486664 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3f8003750 0x7fc3f8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 --2- 172.21.2.104:0/383486664 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3f8004500 0x7fc3f8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 --2- 172.21.2.104:0/383486664 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3f8004ec0 0x7fc3f800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4101478c0 con 0x7fc3f8004500 2021-07-11T04:01:02.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f800bda0 con 0x7fc3f8004ec0 2021-07-11T04:01:02.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f800bf60 con 0x7fc3f8003750 2021-07-11T04:01:02.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc4148f7700 1 --2- 172.21.2.104:0/383486664 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3f8004500 0x7fc3f8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc4148f7700 1 -- 172.21.2.104:0/383486664 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3f8003750 msgr2=0x7fc3f8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:02.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc4148f7700 1 --2- 172.21.2.104:0/383486664 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3f8003750 0x7fc3f8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc4148f7700 1 -- 172.21.2.104:0/383486664 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3f8004ec0 msgr2=0x7fc3f800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:02.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc4148f7700 1 --2- 172.21.2.104:0/383486664 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3f8004ec0 0x7fc3f800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc4158f9700 1 -- 172.21.2.104:0/383486664 reap_dead start 2021-07-11T04:01:02.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc4148f7700 1 -- 172.21.2.104:0/383486664 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f8003260 con 0x7fc3f8004500 2021-07-11T04:01:02.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc4148f7700 1 --2- 172.21.2.104:0/383486664 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3f8004500 0x7fc3f8004980 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7fc41006a180 tx=0x7fc4100a8c80).ready entity=mon.0 client_cookie=da362b344bc7caff server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40c0070d0 con 0x7fc3f8004500 2021-07-11T04:01:02.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c007230 con 0x7fc3f8004500 2021-07-11T04:01:02.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc403fff700 1 -- 172.21.2.104:0/383486664 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40c00c8c0 con 0x7fc3f8004500 2021-07-11T04:01:02.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 -- 172.21.2.104:0/383486664 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3f8004500 msgr2=0x7fc3f8004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 --2- 172.21.2.104:0/383486664 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3f8004500 0x7fc3f8004980 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7fc41006a180 tx=0x7fc4100a8c80).stop 2021-07-11T04:01:02.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 -- 172.21.2.104:0/383486664 shutdown_connections 2021-07-11T04:01:02.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 --2- 172.21.2.104:0/383486664 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3f8004500 0x7fc3f8004980 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 -- 172.21.2.104:0/383486664 >> 172.21.2.104:0/383486664 conn(0x7fc4100b53e0 msgr2=0x7fc4100b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:02.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 -- 172.21.2.104:0/383486664 shutdown_connections 2021-07-11T04:01:02.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.583+0000 7fc41735c700 1 -- 172.21.2.104:0/383486664 wait complete. 2021-07-11T04:01:02.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc41735c700 1 Processor -- start 2021-07-11T04:01:02.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc41735c700 1 -- start start 2021-07-11T04:01:02.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc41735c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3f8004500 0x7fc410145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc41735c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4100aa620 0x7fc4100aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc41735c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc4100af1a0 0x7fc410159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc41735c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc410147070 con 0x7fc4100aa620 2021-07-11T04:01:02.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc41735c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100ab1b0 con 0x7fc3f8004500 2021-07-11T04:01:02.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc41735c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4101468f0 con 0x7fc4100af1a0 2021-07-11T04:01:02.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4148f7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4100aa620 0x7fc4100aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4148f7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4100aa620 0x7fc4100aecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:48990/0 (socket says 172.21.2.104:48990) 2021-07-11T04:01:02.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4148f7700 1 -- 172.21.2.104:0/1569260665 learned_addr learned my addr 172.21.2.104:0/1569260665 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:02.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4158f9700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc4100af1a0 0x7fc410159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4150f8700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3f8004500 0x7fc410145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4148f7700 1 -- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc4100af1a0 msgr2=0x7fc410159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4148f7700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc4100af1a0 0x7fc410159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4148f7700 1 -- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3f8004500 msgr2=0x7fc410145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4148f7700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3f8004500 0x7fc410145cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4148f7700 1 -- 172.21.2.104:0/1569260665 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4100adfd0 con 0x7fc4100aa620 2021-07-11T04:01:02.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4148f7700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4100aa620 0x7fc4100aecb0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7fc41006a180 tx=0x7fc40c013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4027fc700 1 -- 172.21.2.104:0/1569260665 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40c00cd40 con 0x7fc4100aa620 2021-07-11T04:01:02.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4027fc700 1 -- 172.21.2.104:0/1569260665 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c00cea0 con 0x7fc4100aa620 2021-07-11T04:01:02.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc41735c700 1 -- 172.21.2.104:0/1569260665 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4100aaf10 con 0x7fc4100aa620 2021-07-11T04:01:02.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc41735c700 1 -- 172.21.2.104:0/1569260665 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4100aab90 con 0x7fc4100aa620 2021-07-11T04:01:02.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4027fc700 1 -- 172.21.2.104:0/1569260665 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc40c004710 con 0x7fc4100aa620 2021-07-11T04:01:02.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc41735c700 1 -- 172.21.2.104:0/1569260665 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc410057350 con 0x7fc4100aa620 2021-07-11T04:01:02.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4027fc700 1 -- 172.21.2.104:0/1569260665 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc40c01d030 con 0x7fc4100aa620 2021-07-11T04:01:02.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4027fc700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc3f00531d0 0x7fc3f0055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4027fc700 1 -- 172.21.2.104:0/1569260665 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc40c06be20 con 0x7fc4100aa620 2021-07-11T04:01:02.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.587+0000 7fc4150f8700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc3f00531d0 0x7fc3f0055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.591+0000 7fc4150f8700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc3f00531d0 0x7fc3f0055690 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fc404008000 tx=0x7fc404009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.591+0000 7fc4027fc700 1 -- 172.21.2.104:0/1569260665 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc40c043050 con 0x7fc4100aa620 2021-07-11T04:01:02.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.695+0000 7fc41735c700 1 -- 172.21.2.104:0/1569260665 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc41014b9c0 con 0x7fc4100aa620 2021-07-11T04:01:02.700 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T04:01:02.699+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T04:01:02.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc4027fc700 1 -- 172.21.2.104:0/1569260665 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v51) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc40c046670 con 0x7fc4100aa620 2021-07-11T04:01:02.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 -- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc3f00531d0 msgr2=0x7fc3f0055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc3f00531d0 0x7fc3f0055690 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fc404008000 tx=0x7fc404009040).stop 2021-07-11T04:01:02.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 -- 172.21.2.104:0/1569260665 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4100aa620 msgr2=0x7fc4100aecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4100aa620 0x7fc4100aecb0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7fc41006a180 tx=0x7fc40c013000).stop 2021-07-11T04:01:02.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 -- 172.21.2.104:0/1569260665 shutdown_connections 2021-07-11T04:01:02.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc4100af1a0 0x7fc410159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc3f00531d0 0x7fc3f0055690 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4100aa620 0x7fc4100aecb0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 --2- 172.21.2.104:0/1569260665 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3f8004500 0x7fc410145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 -- 172.21.2.104:0/1569260665 >> 172.21.2.104:0/1569260665 conn(0x7fc4100b53e0 msgr2=0x7fc41005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:02.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 -- 172.21.2.104:0/1569260665 shutdown_connections 2021-07-11T04:01:02.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.711+0000 7fc41735c700 1 -- 172.21.2.104:0/1569260665 wait complete. 2021-07-11T04:01:02.717 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T04:01:02.720 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:02.725 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T04:01:02.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69fe991700 1 Processor -- start 2021-07-11T04:01:02.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69fe991700 1 -- start start 2021-07-11T04:01:02.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69fe991700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69f80a76b0 0x7f69f8066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69fe991700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f69f80b1970 0x7f69f8066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69fe991700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f8067450 con 0x7f69f80b0060 2021-07-11T04:01:02.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69fe991700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69f80a7a90 con 0x7f69f80b1970 2021-07-11T04:01:02.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69fe991700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f8147620 con 0x7f69f80a76b0 2021-07-11T04:01:02.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69fcf2e700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f69f80b0060 0x7f69f805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45776/0 (socket says 172.21.2.104:45776) 2021-07-11T04:01:02.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69fcf2e700 1 -- 172.21.2.104:0/1400936889 learned_addr learned my addr 172.21.2.104:0/1400936889 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:02.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69f77fe700 1 --2- 172.21.2.104:0/1400936889 >> v2:172.21.2.104:3300/0 conn(0x7f69f80b1970 0x7f69f8066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69f6ffd700 1 -- 172.21.2.104:0/1400936889 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414886164 0 0) 0x7f69f8067450 con 0x7f69f80b0060 2021-07-11T04:01:02.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69f6ffd700 1 -- 172.21.2.104:0/1400936889 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d8003260 con 0x7f69f80b0060 2021-07-11T04:01:02.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69f77fe700 1 -- 172.21.2.104:0/1400936889 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69f80a76b0 msgr2=0x7f69f8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:02.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69f77fe700 1 --2- 172.21.2.104:0/1400936889 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69f80a76b0 0x7f69f8066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69f77fe700 1 -- 172.21.2.104:0/1400936889 >> v1:172.21.2.104:6789/0 conn(0x7f69f80b0060 legacy=0x7f69f805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69f77fe700 1 -- 172.21.2.104:0/1400936889 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f81478c0 con 0x7f69f80b1970 2021-07-11T04:01:02.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69f77fe700 1 --2- 172.21.2.104:0/1400936889 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69f80b1970 0x7f69f8066f60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f69e8002f70 tx=0x7f69e8006df0).ready entity=mon.0 client_cookie=e8a831e5d22a9d9f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69f6ffd700 1 -- 172.21.2.104:0/1400936889 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69e800d570 con 0x7f69f80b1970 2021-07-11T04:01:02.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69f6ffd700 1 -- 172.21.2.104:0/1400936889 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e8000e90 con 0x7f69f80b1970 2021-07-11T04:01:02.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69f6ffd700 1 -- 172.21.2.104:0/1400936889 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69e800d8c0 con 0x7f69f80b1970 2021-07-11T04:01:02.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69fe991700 1 -- 172.21.2.104:0/1400936889 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69f80b1970 msgr2=0x7f69f8066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.771+0000 7f69fe991700 1 --2- 172.21.2.104:0/1400936889 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69f80b1970 0x7f69f8066f60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f69e8002f70 tx=0x7f69e8006df0).stop 2021-07-11T04:01:02.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 -- 172.21.2.104:0/1400936889 shutdown_connections 2021-07-11T04:01:02.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 --2- 172.21.2.104:0/1400936889 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69f80b1970 0x7f69f8066f60 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 --2- 172.21.2.104:0/1400936889 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69f80a76b0 0x7f69f8066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 -- 172.21.2.104:0/1400936889 >> 172.21.2.104:0/1400936889 conn(0x7f69f80b53e0 msgr2=0x7f69f80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:02.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 -- 172.21.2.104:0/1400936889 shutdown_connections 2021-07-11T04:01:02.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 -- 172.21.2.104:0/1400936889 wait complete. 2021-07-11T04:01:02.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 Processor -- start 2021-07-11T04:01:02.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 -- start start 2021-07-11T04:01:02.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69f80a76b0 0x7f69f80aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69f80b0060 0x7f69f80aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69f80b1970 0x7f69f8159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f814eb80 con 0x7f69f80b1970 2021-07-11T04:01:02.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f814c6a0 con 0x7f69f80a76b0 2021-07-11T04:01:02.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f814c940 con 0x7f69f80b0060 2021-07-11T04:01:02.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fcf2e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69f80b1970 0x7f69f8159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fcf2e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69f80b1970 0x7f69f8159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49014/0 (socket says 172.21.2.104:49014) 2021-07-11T04:01:02.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fcf2e700 1 -- 172.21.2.104:0/4029842960 learned_addr learned my addr 172.21.2.104:0/4029842960 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:02.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fcf2e700 1 -- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69f80b0060 msgr2=0x7f69f80aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fcf2e700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69f80b0060 0x7f69f80aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fcf2e700 1 -- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69f80a76b0 msgr2=0x7f69f80aa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fcf2e700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69f80a76b0 0x7f69f80aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fcf2e700 1 -- 172.21.2.104:0/4029842960 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e800b180 con 0x7f69f80b1970 2021-07-11T04:01:02.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fcf2e700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69f80b1970 0x7f69f8159a60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f69f0001490 tx=0x7f69f0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69f57fa700 1 -- 172.21.2.104:0/4029842960 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f0005ba0 con 0x7f69f80b1970 2021-07-11T04:01:02.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69f57fa700 1 -- 172.21.2.104:0/4029842960 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69f0005d00 con 0x7f69f80b1970 2021-07-11T04:01:02.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 -- 172.21.2.104:0/4029842960 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f814cbe0 con 0x7f69f80b1970 2021-07-11T04:01:02.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69f57fa700 1 -- 172.21.2.104:0/4029842960 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f69f00072c0 con 0x7f69f80b1970 2021-07-11T04:01:02.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 -- 172.21.2.104:0/4029842960 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f814d080 con 0x7f69f80b1970 2021-07-11T04:01:02.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69fe991700 1 -- 172.21.2.104:0/4029842960 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f814cd70 con 0x7f69f80b1970 2021-07-11T04:01:02.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69f57fa700 1 -- 172.21.2.104:0/4029842960 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f69f001a060 con 0x7f69f80b1970 2021-07-11T04:01:02.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69f57fa700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69dc053380 0x7f69dc055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.775+0000 7f69f57fa700 1 -- 172.21.2.104:0/4029842960 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f69f006b270 con 0x7f69f80b1970 2021-07-11T04:01:02.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.779+0000 7f69f7fff700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69dc053380 0x7f69dc055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.779+0000 7f69f7fff700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69dc053380 0x7f69dc055840 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f69e0002f70 tx=0x7f69e0006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.779+0000 7f69f57fa700 1 -- 172.21.2.104:0/4029842960 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f69f0042450 con 0x7f69f80b1970 2021-07-11T04:01:02.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69fe991700 1 -- 172.21.2.104:0/4029842960 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69f8057350 con 0x7f69f80b1970 2021-07-11T04:01:02.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69f57fa700 1 -- 172.21.2.104:0/4029842960 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+2127 (secure 0 0 0) 0x7f69f0018070 con 0x7f69f80b1970 2021-07-11T04:01:02.889 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:02.890 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":51,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/3293345379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3293345379},{"type":"v1","addr":"172.21.2.105:6835","nonce":3293345379}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/57099992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":57099992},{"type":"v1","addr":"172.21.2.104:6833","nonce":57099992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48}],"filesystems":[{"mdsmap":{"epoch":51,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:00:55.667970+0000","modified":"2021-07-11T04:01:02.703506+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":109,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2021-07-11T04:01:02.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69fe991700 1 -- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69dc053380 msgr2=0x7f69dc055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69fe991700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69dc053380 0x7f69dc055840 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f69e0002f70 tx=0x7f69e0006a80).stop 2021-07-11T04:01:02.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69fe991700 1 -- 172.21.2.104:0/4029842960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69f80b1970 msgr2=0x7f69f8159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69fe991700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69f80b1970 0x7f69f8159a60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f69f0001490 tx=0x7f69f0007e00).stop 2021-07-11T04:01:02.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69fe991700 1 -- 172.21.2.104:0/4029842960 shutdown_connections 2021-07-11T04:01:02.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69fe991700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69dc053380 0x7f69dc055840 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69fe991700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69f80b1970 0x7f69f8159a60 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69fe991700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69f80b0060 0x7f69f80aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69fe991700 1 --2- 172.21.2.104:0/4029842960 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69f80a76b0 0x7f69f80aa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.887+0000 7f69fe991700 1 -- 172.21.2.104:0/4029842960 >> 172.21.2.104:0/4029842960 conn(0x7f69f80b53e0 msgr2=0x7f69f805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:02.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.891+0000 7f69fe991700 1 -- 172.21.2.104:0/4029842960 shutdown_connections 2021-07-11T04:01:02.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.891+0000 7f69fe991700 1 -- 172.21.2.104:0/4029842960 wait complete. 2021-07-11T04:01:02.894 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 51 2021-07-11T04:01:02.897 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T04:01:02.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 Processor -- start 2021-07-11T04:01:02.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 -- start start 2021-07-11T04:01:02.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0f88061d20 0x7f0f880af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f880b22f0 0x7f0f8813cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f880b28b0 con 0x7f0f880b1c60 2021-07-11T04:01:02.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f88062100 con 0x7f0f88061d20 2021-07-11T04:01:02.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f880b2ae0 con 0x7f0f880b22f0 2021-07-11T04:01:02.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8cfaa700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0f88061d20 0x7f0f880af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8cfaa700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0f88061d20 0x7f0f880af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49022/0 (socket says 172.21.2.104:49022) 2021-07-11T04:01:02.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8cfaa700 1 -- 172.21.2.104:0/1690480340 learned_addr learned my addr 172.21.2.104:0/1690480340 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:02.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8cfaa700 1 -- 172.21.2.104:0/1690480340 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f880b22f0 msgr2=0x7f0f8813cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:02.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8cfaa700 1 --2- 172.21.2.104:0/1690480340 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f880b22f0 0x7f0f8813cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8cfaa700 1 -- 172.21.2.104:0/1690480340 >> v1:172.21.2.105:6789/0 conn(0x7f0f880b1c60 legacy=0x7f0f880afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:02.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8cfaa700 1 -- 172.21.2.104:0/1690480340 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f880b2d80 con 0x7f0f88061d20 2021-07-11T04:01:02.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8cfaa700 1 --2- 172.21.2.104:0/1690480340 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f88061d20 0x7f0f880af690 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f0f880b6590 tx=0x7f0f70006eb0).ready entity=mon.0 client_cookie=29f72daa9dd22121 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f877fe700 1 -- 172.21.2.104:0/1690480340 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f7000d570 con 0x7f0f88061d20 2021-07-11T04:01:02.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f877fe700 1 -- 172.21.2.104:0/1690480340 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f70000e90 con 0x7f0f88061d20 2021-07-11T04:01:02.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f877fe700 1 -- 172.21.2.104:0/1690480340 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f7000d8c0 con 0x7f0f88061d20 2021-07-11T04:01:02.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 -- 172.21.2.104:0/1690480340 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f88061d20 msgr2=0x7f0f880af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 --2- 172.21.2.104:0/1690480340 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f88061d20 0x7f0f880af690 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f0f880b6590 tx=0x7f0f70006eb0).stop 2021-07-11T04:01:02.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 -- 172.21.2.104:0/1690480340 shutdown_connections 2021-07-11T04:01:02.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 --2- 172.21.2.104:0/1690480340 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f880b22f0 0x7f0f8813cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 --2- 172.21.2.104:0/1690480340 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f88061d20 0x7f0f880af690 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:02.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 -- 172.21.2.104:0/1690480340 >> 172.21.2.104:0/1690480340 conn(0x7f0f880639b0 msgr2=0x7f0f88063d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:02.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.947+0000 7f0f8f20e700 1 -- 172.21.2.104:0/1690480340 shutdown_connections 2021-07-11T04:01:02.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 -- 172.21.2.104:0/1690480340 wait complete. 2021-07-11T04:01:02.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 Processor -- start 2021-07-11T04:01:02.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 -- start start 2021-07-11T04:01:02.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f88061d20 0x7f0f8805fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f880b1c60 0x7f0f88067370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f880b22f0 0x7f0f8813f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f88065e70 con 0x7f0f880b22f0 2021-07-11T04:01:02.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f88065490 con 0x7f0f88061d20 2021-07-11T04:01:02.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f88065730 con 0x7f0f880b1c60 2021-07-11T04:01:02.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8d7ab700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f880b22f0 0x7f0f8813f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8d7ab700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f880b22f0 0x7f0f8813f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49032/0 (socket says 172.21.2.104:49032) 2021-07-11T04:01:02.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8d7ab700 1 -- 172.21.2.104:0/3132875145 learned_addr learned my addr 172.21.2.104:0/3132875145 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:02.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8d7ab700 1 -- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f880b1c60 msgr2=0x7f0f88067370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8d7ab700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f880b1c60 0x7f0f88067370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8d7ab700 1 -- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f88061d20 msgr2=0x7f0f8805fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:02.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8d7ab700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f88061d20 0x7f0f8805fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:02.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8d7ab700 1 -- 172.21.2.104:0/3132875145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7000b1e0 con 0x7f0f880b22f0 2021-07-11T04:01:02.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8d7ab700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f880b22f0 0x7f0f8813f810 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f0f800032f0 tx=0x7f0f800091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f85ffb700 1 -- 172.21.2.104:0/3132875145 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f8000f760 con 0x7f0f880b22f0 2021-07-11T04:01:02.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f85ffb700 1 -- 172.21.2.104:0/3132875145 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f8000f8c0 con 0x7f0f880b22f0 2021-07-11T04:01:02.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 -- 172.21.2.104:0/3132875145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f880659d0 con 0x7f0f880b22f0 2021-07-11T04:01:02.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f85ffb700 1 -- 172.21.2.104:0/3132875145 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f8000fc20 con 0x7f0f880b22f0 2021-07-11T04:01:02.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 -- 172.21.2.104:0/3132875145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8813f290 con 0x7f0f880b22f0 2021-07-11T04:01:02.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8f20e700 1 -- 172.21.2.104:0/3132875145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f88057350 con 0x7f0f880b22f0 2021-07-11T04:01:02.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f85ffb700 1 -- 172.21.2.104:0/3132875145 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0f8001a460 con 0x7f0f880b22f0 2021-07-11T04:01:02.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f85ffb700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f6c0532d0 0x7f0f6c055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:02.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f85ffb700 1 -- 172.21.2.104:0/3132875145 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f0f8006b870 con 0x7f0f880b22f0 2021-07-11T04:01:02.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.951+0000 7f0f8cfaa700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f6c0532d0 0x7f0f6c055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:02.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.955+0000 7f0f8cfaa700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f6c0532d0 0x7f0f6c055790 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f0f880b6590 tx=0x7f0f70015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:02.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:02.955+0000 7f0f85ffb700 1 -- 172.21.2.104:0/3132875145 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0f80042b00 con 0x7f0f880b22f0 2021-07-11T04:01:03.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.063+0000 7f0f8f20e700 1 -- 172.21.2.104:0/3132875145 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0f88065b60 con 0x7f0f880b22f0 2021-07-11T04:01:03.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.723+0000 7f0f85ffb700 1 -- 172.21.2.104:0/3132875145 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0f80046120 con 0x7f0f880b22f0 2021-07-11T04:01:03.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 -- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f6c0532d0 msgr2=0x7f0f6c055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:03.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f6c0532d0 0x7f0f6c055790 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f0f880b6590 tx=0x7f0f70015000).stop 2021-07-11T04:01:03.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 -- 172.21.2.104:0/3132875145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f880b22f0 msgr2=0x7f0f8813f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:03.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f880b22f0 0x7f0f8813f810 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f0f800032f0 tx=0x7f0f800091d0).stop 2021-07-11T04:01:03.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 -- 172.21.2.104:0/3132875145 shutdown_connections 2021-07-11T04:01:03.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f6c0532d0 0x7f0f6c055790 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:03.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f880b22f0 0x7f0f8813f810 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:03.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f880b1c60 0x7f0f88067370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:03.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 --2- 172.21.2.104:0/3132875145 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f88061d20 0x7f0f8805fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:03.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 -- 172.21.2.104:0/3132875145 >> 172.21.2.104:0/3132875145 conn(0x7f0f880639b0 msgr2=0x7f0f8813b600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:03.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 -- 172.21.2.104:0/3132875145 shutdown_connections 2021-07-11T04:01:03.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.727+0000 7f0f8f20e700 1 -- 172.21.2.104:0/3132875145 wait complete. 2021-07-11T04:01:03.734 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T04:01:03.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 Processor -- start 2021-07-11T04:01:03.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 -- start start 2021-07-11T04:01:03.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8a481483a0 0x7f8a480b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:03.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8a480a70c0 0x7f8a48144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:03.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a480b0350 con 0x7f8a480b1db0 2021-07-11T04:01:03.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a480a90c0 con 0x7f8a481483a0 2021-07-11T04:01:03.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a480b05f0 con 0x7f8a480a70c0 2021-07-11T04:01:03.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4d1ff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8a481483a0 0x7f8a480b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:03.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4d1ff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8a481483a0 0x7f8a480b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49040/0 (socket says 172.21.2.104:49040) 2021-07-11T04:01:03.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4d1ff700 1 -- 172.21.2.104:0/3914923465 learned_addr learned my addr 172.21.2.104:0/3914923465 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:03.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4d1ff700 1 -- 172.21.2.104:0/3914923465 >> v2:172.21.2.105:3300/0 conn(0x7f8a480a70c0 msgr2=0x7f8a48144720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:03.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4d1ff700 1 --2- 172.21.2.104:0/3914923465 >> v2:172.21.2.105:3300/0 conn(0x7f8a480a70c0 0x7f8a48144720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:03.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4d1ff700 1 -- 172.21.2.104:0/3914923465 >> v1:172.21.2.105:6789/0 conn(0x7f8a480b1db0 legacy=0x7f8a480b2190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:03.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4d1ff700 1 -- 172.21.2.104:0/3914923465 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a480b0890 con 0x7f8a481483a0 2021-07-11T04:01:03.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4d1ff700 1 --2- 172.21.2.104:0/3914923465 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a481483a0 0x7f8a480b0fd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f8a480ace60 tx=0x7f8a3c006eb0).ready entity=mon.0 client_cookie=3d2df17731d8b519 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:03.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4c9fe700 1 -- 172.21.2.104:0/3914923465 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c00d5c0 con 0x7f8a481483a0 2021-07-11T04:01:03.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4c9fe700 1 -- 172.21.2.104:0/3914923465 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c000e90 con 0x7f8a481483a0 2021-07-11T04:01:03.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4c9fe700 1 -- 172.21.2.104:0/3914923465 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c00d950 con 0x7f8a481483a0 2021-07-11T04:01:03.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 -- 172.21.2.104:0/3914923465 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a481483a0 msgr2=0x7f8a480b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:03.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 --2- 172.21.2.104:0/3914923465 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a481483a0 0x7f8a480b0fd0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f8a480ace60 tx=0x7f8a3c006eb0).stop 2021-07-11T04:01:03.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 -- 172.21.2.104:0/3914923465 shutdown_connections 2021-07-11T04:01:03.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 --2- 172.21.2.104:0/3914923465 >> v2:172.21.2.105:3300/0 conn(0x7f8a480a70c0 0x7f8a48144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:03.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 --2- 172.21.2.104:0/3914923465 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a481483a0 0x7f8a480b0fd0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:03.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 -- 172.21.2.104:0/3914923465 >> 172.21.2.104:0/3914923465 conn(0x7f8a4805f6b0 msgr2=0x7f8a4805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:03.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 -- 172.21.2.104:0/3914923465 shutdown_connections 2021-07-11T04:01:03.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 -- 172.21.2.104:0/3914923465 wait complete. 2021-07-11T04:01:03.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 Processor -- start 2021-07-11T04:01:03.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.787+0000 7f8a4fc64700 1 -- start start 2021-07-11T04:01:03.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4fc64700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a480a70c0 0x7f8a48147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:03.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4fc64700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a480b1db0 0x7f8a48156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:03.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4fc64700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a481483a0 0x7f8a481572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:03.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4fc64700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48145b60 con 0x7f8a480b1db0 2021-07-11T04:01:03.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4fc64700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48145dc0 con 0x7f8a481483a0 2021-07-11T04:01:03.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4fc64700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48146060 con 0x7f8a480a70c0 2021-07-11T04:01:03.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4d1ff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a480b1db0 0x7f8a48156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:03.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4d1ff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a480b1db0 0x7f8a48156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49046/0 (socket says 172.21.2.104:49046) 2021-07-11T04:01:03.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4d1ff700 1 -- 172.21.2.104:0/1731174742 learned_addr learned my addr 172.21.2.104:0/1731174742 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:03.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4d1ff700 1 -- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a480a70c0 msgr2=0x7f8a48147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:03.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4d1ff700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a480a70c0 0x7f8a48147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:03.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4d1ff700 1 -- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a481483a0 msgr2=0x7f8a481572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:03.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4d1ff700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a481483a0 0x7f8a481572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:03.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4d1ff700 1 -- 172.21.2.104:0/1731174742 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c00b1e0 con 0x7f8a480b1db0 2021-07-11T04:01:03.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4d1ff700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a480b1db0 0x7f8a48156df0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f8a480ace60 tx=0x7f8a3c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:03.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a3affd700 1 -- 172.21.2.104:0/1731174742 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c0119d0 con 0x7f8a480b1db0 2021-07-11T04:01:03.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a3affd700 1 -- 172.21.2.104:0/1731174742 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c011b30 con 0x7f8a480b1db0 2021-07-11T04:01:03.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a3affd700 1 -- 172.21.2.104:0/1731174742 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8a3c011e90 con 0x7f8a480b1db0 2021-07-11T04:01:03.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4fc64700 1 -- 172.21.2.104:0/1731174742 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a481467d0 con 0x7f8a480b1db0 2021-07-11T04:01:03.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4fc64700 1 -- 172.21.2.104:0/1731174742 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a48156920 con 0x7f8a480b1db0 2021-07-11T04:01:03.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4fc64700 1 -- 172.21.2.104:0/1731174742 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a48057350 con 0x7f8a480b1db0 2021-07-11T04:01:03.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a3affd700 1 -- 172.21.2.104:0/1731174742 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8a3c00f070 con 0x7f8a480b1db0 2021-07-11T04:01:03.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a3affd700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a280531f0 0x7f8a280556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:03.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a3affd700 1 -- 172.21.2.104:0/1731174742 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f8a3c06cde0 con 0x7f8a480b1db0 2021-07-11T04:01:03.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4da00700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a280531f0 0x7f8a280556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:03.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.791+0000 7f8a4da00700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a280531f0 0x7f8a280556b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f8a44002fb0 tx=0x7f8a440069c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:03.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.795+0000 7f8a3affd700 1 -- 172.21.2.104:0/1731174742 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8a3c0440d0 con 0x7f8a480b1db0 2021-07-11T04:01:03.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:03.875+0000 7f8a4fc64700 1 -- 172.21.2.104:0/1731174742 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8a48044650 con 0x7f8a480b1db0 2021-07-11T04:01:04.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a3affd700 1 -- 172.21.2.104:0/1731174742 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) v1 ==== 185+0+0 (secure 0 0 0) 0x7f8a3c0476f0 con 0x7f8a480b1db0 2021-07-11T04:01:04.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 -- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a280531f0 msgr2=0x7f8a280556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:04.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a280531f0 0x7f8a280556b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7f8a44002fb0 tx=0x7f8a440069c0).stop 2021-07-11T04:01:04.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 -- 172.21.2.104:0/1731174742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a480b1db0 msgr2=0x7f8a48156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:04.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a480b1db0 0x7f8a48156df0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f8a480ace60 tx=0x7f8a3c015000).stop 2021-07-11T04:01:04.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 -- 172.21.2.104:0/1731174742 shutdown_connections 2021-07-11T04:01:04.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8a481483a0 0x7f8a481572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:04.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8a280531f0 0x7f8a280556b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:04.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8a480b1db0 0x7f8a48156df0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:04.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 --2- 172.21.2.104:0/1731174742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8a480a70c0 0x7f8a48147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:04.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 -- 172.21.2.104:0/1731174742 >> 172.21.2.104:0/1731174742 conn(0x7f8a4805f6b0 msgr2=0x7f8a481479d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:04.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 -- 172.21.2.104:0/1731174742 shutdown_connections 2021-07-11T04:01:04.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.731+0000 7f8a4fc64700 1 -- 172.21.2.104:0/1731174742 wait complete. 2021-07-11T04:01:04.739 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T04:01:04.741 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T04:01:04.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 Processor -- start 2021-07-11T04:01:04.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 -- start start 2021-07-11T04:01:04.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c144000 0x7f928c14c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928c0aabb0 con 0x7f928c067300 2021-07-11T04:01:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928c0604e0 con 0x7f928c060100 2021-07-11T04:01:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928c066ea0 con 0x7f928c144000 2021-07-11T04:01:04.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928bfff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f928c067300 0x7f928c1438f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45822/0 (socket says 172.21.2.104:45822) 2021-07-11T04:01:04.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928bfff700 1 -- 172.21.2.104:0/1248669546 learned_addr learned my addr 172.21.2.104:0/1248669546 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:04.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573887495 0 0) 0x7f928c0aabb0 con 0x7f928c067300 2021-07-11T04:01:04.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f926c003260 con 0x7f928c067300 2021-07-11T04:01:04.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f929169d700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c144000 0x7f928c14c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:04.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3771981032 0 0) 0x7f926c003260 con 0x7f928c067300 2021-07-11T04:01:04.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c144000 msgr2=0x7f928c14c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:04.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c144000 0x7f928c14c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:04.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 >> v1:172.21.2.105:6789/0 conn(0x7f928c060100 legacy=0x7f928c0b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:04.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c0b33b0 con 0x7f928c067300 2021-07-11T04:01:04.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f9274001660 con 0x7f928c067300 2021-07-11T04:01:04.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 >> v1:172.21.2.104:6789/0 conn(0x7f928c067300 legacy=0x7f928c1438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:04.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f926c003800 0x7f926c003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c144000 0x7f926c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f926c004b10 0x7f926c004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c066ea0 con 0x7f926c004b10 2021-07-11T04:01:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c00b910 con 0x7f926c003800 2021-07-11T04:01:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c00bad0 con 0x7f928c144000 2021-07-11T04:01:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f929169d700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f926c004b10 0x7f926c004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9290e9c700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f926c003800 0x7f926c003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928bfff700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c144000 0x7f926c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f929169d700 1 -- 172.21.2.104:0/1248669546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c144000 msgr2=0x7f926c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f929169d700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c144000 0x7f926c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f929169d700 1 -- 172.21.2.104:0/1248669546 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f926c003800 msgr2=0x7f926c003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f929169d700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f926c003800 0x7f926c003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f929169d700 1 -- 172.21.2.104:0/1248669546 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c003260 con 0x7f926c004b10 2021-07-11T04:01:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f929169d700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f926c004b10 0x7f926c004f90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f928400a000 tx=0x7f9284008040).ready entity=mon.0 client_cookie=7a90d0db88bf7e9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f928400c070 con 0x7f926c004b10 2021-07-11T04:01:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f928400e620 con 0x7f926c004b10 2021-07-11T04:01:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f928b7fe700 1 -- 172.21.2.104:0/1248669546 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f928400e980 con 0x7f926c004b10 2021-07-11T04:01:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 -- 172.21.2.104:0/1248669546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f926c004b10 msgr2=0x7f926c004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 --2- 172.21.2.104:0/1248669546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f926c004b10 0x7f926c004f90 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f928400a000 tx=0x7f9284008040).stop 2021-07-11T04:01:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f929169d700 1 -- 172.21.2.104:0/1248669546 reap_dead start 2021-07-11T04:01:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 -- 172.21.2.104:0/1248669546 shutdown_connections 2021-07-11T04:01:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 -- 172.21.2.104:0/1248669546 >> 172.21.2.104:0/1248669546 conn(0x7f928c05cf30 msgr2=0x7f928c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 -- 172.21.2.104:0/1248669546 shutdown_connections 2021-07-11T04:01:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 -- 172.21.2.104:0/1248669546 wait complete. 2021-07-11T04:01:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 Processor -- start 2021-07-11T04:01:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.795+0000 7f9293100700 1 -- start start 2021-07-11T04:01:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9293100700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f928c142860 0x7f928c0a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9293100700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f928c0ab960 0x7f928c0aa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:04.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9293100700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c0afda0 0x7f928c0b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:04.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9293100700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c0ab720 con 0x7f928c142860 2021-07-11T04:01:04.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9293100700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c0ac6d0 con 0x7f928c0ab960 2021-07-11T04:01:04.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9293100700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f928c153740 con 0x7f928c0afda0 2021-07-11T04:01:04.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9290e9c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f928c142860 0x7f928c0a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:04.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9290e9c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f928c142860 0x7f928c0a9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49064/0 (socket says 172.21.2.104:49064) 2021-07-11T04:01:04.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9290e9c700 1 -- 172.21.2.104:0/3723214584 learned_addr learned my addr 172.21.2.104:0/3723214584 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:04.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f928bfff700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f928c0ab960 0x7f928c0aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:04.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f929169d700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c0afda0 0x7f928c0b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:04.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9290e9c700 1 -- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c0afda0 msgr2=0x7f928c0b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:04.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9290e9c700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c0afda0 0x7f928c0b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:04.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9290e9c700 1 -- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f928c0ab960 msgr2=0x7f928c0aa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:04.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9290e9c700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f928c0ab960 0x7f928c0aa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:04.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9290e9c700 1 -- 172.21.2.104:0/3723214584 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c0ac190 con 0x7f928c142860 2021-07-11T04:01:04.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f928bfff700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f928c0ab960 0x7f928c0aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:01:04.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f929169d700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c0afda0 0x7f928c0b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:04.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9290e9c700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f928c142860 0x7f928c0a9d20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f927c00b000 tx=0x7f927c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:04.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9289ffb700 1 -- 172.21.2.104:0/3723214584 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f927c015630 con 0x7f928c142860 2021-07-11T04:01:04.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9289ffb700 1 -- 172.21.2.104:0/3723214584 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c000e90 con 0x7f928c142860 2021-07-11T04:01:04.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9293100700 1 -- 172.21.2.104:0/3723214584 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928c0ac430 con 0x7f928c142860 2021-07-11T04:01:04.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9293100700 1 -- 172.21.2.104:0/3723214584 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928c0aa700 con 0x7f928c142860 2021-07-11T04:01:04.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9289ffb700 1 -- 172.21.2.104:0/3723214584 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f927c015980 con 0x7f928c142860 2021-07-11T04:01:04.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9293100700 1 -- 172.21.2.104:0/3723214584 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928c057350 con 0x7f928c142860 2021-07-11T04:01:04.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9289ffb700 1 -- 172.21.2.104:0/3723214584 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f927c015ae0 con 0x7f928c142860 2021-07-11T04:01:04.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9289ffb700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9270053350 0x7f9270055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:04.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f9289ffb700 1 -- 172.21.2.104:0/3723214584 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f927c06b210 con 0x7f928c142860 2021-07-11T04:01:04.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f928bfff700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9270053350 0x7f9270055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:04.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.799+0000 7f928bfff700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9270053350 0x7f9270055810 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f927400d000 tx=0x7f9274009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:04.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.803+0000 7f9289ffb700 1 -- 172.21.2.104:0/3723214584 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f927c0425a0 con 0x7f928c142860 2021-07-11T04:01:04.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:04.887+0000 7f9293100700 1 -- 172.21.2.104:0/3723214584 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f928c0acbe0 con 0x7f928c142860 2021-07-11T04:01:05.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.743+0000 7f9289ffb700 1 -- 172.21.2.104:0/3723214584 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) v1 ==== 173+0+0 (secure 0 0 0) 0x7f927c045bc0 con 0x7f928c142860 2021-07-11T04:01:05.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 -- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9270053350 msgr2=0x7f9270055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:05.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9270053350 0x7f9270055810 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f927400d000 tx=0x7f9274009040).stop 2021-07-11T04:01:05.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 -- 172.21.2.104:0/3723214584 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f928c142860 msgr2=0x7f928c0a9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:05.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f928c142860 0x7f928c0a9d20 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f927c00b000 tx=0x7f927c009040).stop 2021-07-11T04:01:05.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 -- 172.21.2.104:0/3723214584 shutdown_connections 2021-07-11T04:01:05.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f928c0afda0 0x7f928c0b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:05.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f928c0ab960 0x7f928c0aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:05.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9270053350 0x7f9270055810 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:05.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 --2- 172.21.2.104:0/3723214584 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f928c142860 0x7f928c0a9d20 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:05.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 -- 172.21.2.104:0/3723214584 >> 172.21.2.104:0/3723214584 conn(0x7f928c05cf30 msgr2=0x7f928c143590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:05.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 -- 172.21.2.104:0/3723214584 shutdown_connections 2021-07-11T04:01:05.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.747+0000 7f9293100700 1 -- 172.21.2.104:0/3723214584 wait complete. 2021-07-11T04:01:05.751 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T04:01:05.754 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-07-11T04:01:05.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 Processor -- start 2021-07-11T04:01:05.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- start start 2021-07-11T04:01:05.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4c040b1ec0 0x7f4c040b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f4c040affb0 0x7f4c040b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c040b0420 con 0x7f4c040a70c0 2021-07-11T04:01:05.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c040b2790 con 0x7f4c040b1ec0 2021-07-11T04:01:05.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c040b0650 con 0x7f4c040affb0 2021-07-11T04:01:05.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c02d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4c040a70c0 0x7f4c04141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45840/0 (socket says 172.21.2.104:45840) 2021-07-11T04:01:05.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c02d9d700 1 -- 172.21.2.104:0/3477035912 learned_addr learned my addr 172.21.2.104:0/3477035912 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:05.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c0259c700 1 --2- 172.21.2.104:0/3477035912 >> v2:172.21.2.104:3300/0 conn(0x7f4c040b1ec0 0x7f4c040b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:05.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c0259c700 1 -- 172.21.2.104:0/3477035912 >> v2:172.21.2.105:3300/0 conn(0x7f4c040affb0 msgr2=0x7f4c040b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:05.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c0259c700 1 --2- 172.21.2.104:0/3477035912 >> v2:172.21.2.105:3300/0 conn(0x7f4c040affb0 0x7f4c040b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c0259c700 1 -- 172.21.2.104:0/3477035912 >> v1:172.21.2.104:6789/0 conn(0x7f4c040a70c0 legacy=0x7f4c04141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:05.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c0259c700 1 -- 172.21.2.104:0/3477035912 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c040b08f0 con 0x7f4c040b1ec0 2021-07-11T04:01:05.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c0259c700 1 --2- 172.21.2.104:0/3477035912 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c040b1ec0 0x7f4c040b12d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f4bf8002f70 tx=0x7f4bf8006df0).ready entity=mon.0 client_cookie=fe6efe60d8450229 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:05.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c0159a700 1 -- 172.21.2.104:0/3477035912 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bf800d5c0 con 0x7f4c040b1ec0 2021-07-11T04:01:05.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c0159a700 1 -- 172.21.2.104:0/3477035912 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf8000e90 con 0x7f4c040b1ec0 2021-07-11T04:01:05.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c0159a700 1 -- 172.21.2.104:0/3477035912 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bf800d910 con 0x7f4c040b1ec0 2021-07-11T04:01:05.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3477035912 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c040b1ec0 msgr2=0x7f4c040b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:05.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 --2- 172.21.2.104:0/3477035912 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c040b1ec0 0x7f4c040b12d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f4bf8002f70 tx=0x7f4bf8006df0).stop 2021-07-11T04:01:05.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3477035912 shutdown_connections 2021-07-11T04:01:05.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 --2- 172.21.2.104:0/3477035912 >> v2:172.21.2.105:3300/0 conn(0x7f4c040affb0 0x7f4c040b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 --2- 172.21.2.104:0/3477035912 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c040b1ec0 0x7f4c040b12d0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:05.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3477035912 >> 172.21.2.104:0/3477035912 conn(0x7f4c0405f6b0 msgr2=0x7f4c0405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:05.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3477035912 shutdown_connections 2021-07-11T04:01:05.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3477035912 wait complete. 2021-07-11T04:01:05.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 Processor -- start 2021-07-11T04:01:05.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- start start 2021-07-11T04:01:05.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c040a70c0 0x7f4c04143080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c040affb0 0x7f4c04153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c040b1ec0 0x7f4c04154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0415af80 con 0x7f4c040b1ec0 2021-07-11T04:01:05.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c04149e80 con 0x7f4c040a70c0 2021-07-11T04:01:05.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0414a120 con 0x7f4c040affb0 2021-07-11T04:01:05.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c02d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c040b1ec0 0x7f4c04154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:05.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c02d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c040b1ec0 0x7f4c04154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49078/0 (socket says 172.21.2.104:49078) 2021-07-11T04:01:05.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c02d9d700 1 -- 172.21.2.104:0/3199260293 learned_addr learned my addr 172.21.2.104:0/3199260293 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:05.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c02d9d700 1 -- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c040affb0 msgr2=0x7f4c04153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:05.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c02d9d700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c040affb0 0x7f4c04153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c02d9d700 1 -- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c040a70c0 msgr2=0x7f4c04143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:05.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c02d9d700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c040a70c0 0x7f4c04143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c02d9d700 1 -- 172.21.2.104:0/3199260293 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf800b180 con 0x7f4c040b1ec0 2021-07-11T04:01:05.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c02d9d700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c040b1ec0 0x7f4c04154330 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f4c040ad2d0 tx=0x7f4bf4007bc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:05.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4bf37fe700 1 -- 172.21.2.104:0/3199260293 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bf400ce30 con 0x7f4c040b1ec0 2021-07-11T04:01:05.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4bf37fe700 1 -- 172.21.2.104:0/3199260293 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf40072e0 con 0x7f4c040b1ec0 2021-07-11T04:01:05.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4bf37fe700 1 -- 172.21.2.104:0/3199260293 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4bf4005800 con 0x7f4c040b1ec0 2021-07-11T04:01:05.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3199260293 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0414a3c0 con 0x7f4c040b1ec0 2021-07-11T04:01:05.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.811+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3199260293 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0414a860 con 0x7f4c040b1ec0 2021-07-11T04:01:05.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.815+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3199260293 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0414a550 con 0x7f4c040b1ec0 2021-07-11T04:01:05.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.815+0000 7f4bf37fe700 1 -- 172.21.2.104:0/3199260293 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4bf401a070 con 0x7f4c040b1ec0 2021-07-11T04:01:05.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.815+0000 7f4bf37fe700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4be00531f0 0x7f4be00556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.815+0000 7f4bf37fe700 1 -- 172.21.2.104:0/3199260293 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4bf406a490 con 0x7f4c040b1ec0 2021-07-11T04:01:05.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.815+0000 7f4c0259c700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4be00531f0 0x7f4be00556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:05.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.815+0000 7f4c0259c700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4be00531f0 0x7f4be00556b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f4bf8002f70 tx=0x7f4bf8014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:05.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.815+0000 7f4bf37fe700 1 -- 172.21.2.104:0/3199260293 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4bf4041ab0 con 0x7f4c040b1ec0 2021-07-11T04:01:05.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.903+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3199260293 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4c04057350 con 0x7f4c040b1ec0 2021-07-11T04:01:05.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4bf37fe700 1 -- 172.21.2.104:0/3199260293 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4bf40450d0 con 0x7f4c040b1ec0 2021-07-11T04:01:05.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4be00531f0 msgr2=0x7f4be00556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:05.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4be00531f0 0x7f4be00556b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f4bf8002f70 tx=0x7f4bf8014000).stop 2021-07-11T04:01:05.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3199260293 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c040b1ec0 msgr2=0x7f4c04154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:05.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c040b1ec0 0x7f4c04154330 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f4c040ad2d0 tx=0x7f4bf4007bc0).stop 2021-07-11T04:01:05.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3199260293 shutdown_connections 2021-07-11T04:01:05.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4be00531f0 0x7f4be00556b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:05.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c040b1ec0 0x7f4c04154330 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:05.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c040affb0 0x7f4c04153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 --2- 172.21.2.104:0/3199260293 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c040a70c0 0x7f4c04143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3199260293 >> 172.21.2.104:0/3199260293 conn(0x7f4c0405f6b0 msgr2=0x7f4c040b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:05.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3199260293 shutdown_connections 2021-07-11T04:01:05.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.907+0000 7f4c08b0e700 1 -- 172.21.2.104:0/3199260293 wait complete. 2021-07-11T04:01:05.915 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df' 2021-07-11T04:01:05.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.963+0000 7fdf80bde700 1 Processor -- start 2021-07-11T04:01:05.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.963+0000 7fdf80bde700 1 -- start start 2021-07-11T04:01:05.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.963+0000 7fdf80bde700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fdf7c0aa860 0x7fdf7c0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.963+0000 7fdf80bde700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fdf7c0a9da0 0x7fdf7c0aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.963+0000 7fdf80bde700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf7c0acf30 0x7fdf7c1434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.963+0000 7fdf80bde700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7c0abb50 con 0x7fdf7c0aa860 2021-07-11T04:01:05.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.963+0000 7fdf80bde700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7c0ac370 con 0x7fdf7c0a9da0 2021-07-11T04:01:05.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.963+0000 7fdf80bde700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0aac40 con 0x7fdf7c0acf30 2021-07-11T04:01:05.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fdf7c0aa860 0x7fdf7c0b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:05.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fdf7c0aa860 0x7fdf7c0b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49082/0 (socket says 172.21.2.104:49082) 2021-07-11T04:01:05.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 -- 172.21.2.104:0/1356130684 learned_addr learned my addr 172.21.2.104:0/1356130684 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:05.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 -- 172.21.2.104:0/1356130684 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf7c0acf30 msgr2=0x7fdf7c1434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:05.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 --2- 172.21.2.104:0/1356130684 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf7c0acf30 0x7fdf7c1434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 -- 172.21.2.104:0/1356130684 >> v2:172.21.2.105:3300/0 conn(0x7fdf7c0a9da0 msgr2=0x7fdf7c0aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:05.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 --2- 172.21.2.104:0/1356130684 >> v2:172.21.2.105:3300/0 conn(0x7fdf7c0a9da0 0x7fdf7c0aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 -- 172.21.2.104:0/1356130684 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c0b16b0 con 0x7fdf7c0aa860 2021-07-11T04:01:05.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 --2- 172.21.2.104:0/1356130684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf7c0aa860 0x7fdf7c0b11c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7fdf7c0b2e10 tx=0x7fdf64006f40).ready entity=mon.0 client_cookie=9a8ef0d1528f3b24 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:05.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7959a700 1 -- 172.21.2.104:0/1356130684 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf6400dc30 con 0x7fdf7c0aa860 2021-07-11T04:01:05.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7959a700 1 -- 172.21.2.104:0/1356130684 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64000e90 con 0x7fdf7c0aa860 2021-07-11T04:01:05.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7959a700 1 -- 172.21.2.104:0/1356130684 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf6400be90 con 0x7fdf7c0aa860 2021-07-11T04:01:05.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 -- 172.21.2.104:0/1356130684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf7c0aa860 msgr2=0x7fdf7c0b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:05.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 --2- 172.21.2.104:0/1356130684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf7c0aa860 0x7fdf7c0b11c0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7fdf7c0b2e10 tx=0x7fdf64006f40).stop 2021-07-11T04:01:05.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 -- 172.21.2.104:0/1356130684 shutdown_connections 2021-07-11T04:01:05.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 --2- 172.21.2.104:0/1356130684 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf7c0acf30 0x7fdf7c1434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 --2- 172.21.2.104:0/1356130684 >> v2:172.21.2.105:3300/0 conn(0x7fdf7c0a9da0 0x7fdf7c0aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 --2- 172.21.2.104:0/1356130684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf7c0aa860 0x7fdf7c0b11c0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:05.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 -- 172.21.2.104:0/1356130684 >> 172.21.2.104:0/1356130684 conn(0x7fdf7c0b64d0 msgr2=0x7fdf7c0b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:05.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 -- 172.21.2.104:0/1356130684 shutdown_connections 2021-07-11T04:01:05.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 -- 172.21.2.104:0/1356130684 wait complete. 2021-07-11T04:01:05.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 Processor -- start 2021-07-11T04:01:05.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 -- start start 2021-07-11T04:01:05.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf7c0a9da0 0x7fdf7c151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf7c0aa860 0x7fdf7c0646b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf7c0acf30 0x7fdf7c14e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c066b40 con 0x7fdf7c0a9da0 2021-07-11T04:01:05.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c063f90 con 0x7fdf7c0acf30 2021-07-11T04:01:05.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c066050 con 0x7fdf7c0aa860 2021-07-11T04:01:05.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf7c0a9da0 0x7fdf7c151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:05.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf7c0a9da0 0x7fdf7c151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49088/0 (socket says 172.21.2.104:49088) 2021-07-11T04:01:05.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 -- 172.21.2.104:0/40847210 learned_addr learned my addr 172.21.2.104:0/40847210 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:05.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 -- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf7c0aa860 msgr2=0x7fdf7c0646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:05.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf7c0aa860 0x7fdf7c0646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 -- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf7c0acf30 msgr2=0x7fdf7c14e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:05.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf7c0acf30 0x7fdf7c14e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:05.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 -- 172.21.2.104:0/40847210 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf6400b460 con 0x7fdf7c0a9da0 2021-07-11T04:01:05.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf7a59c700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf7c0a9da0 0x7fdf7c151950 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7fdf7c0b2e10 tx=0x7fdf7c0671e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:05.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf737fe700 1 -- 172.21.2.104:0/40847210 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf64023440 con 0x7fdf7c0a9da0 2021-07-11T04:01:05.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf737fe700 1 -- 172.21.2.104:0/40847210 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6401e040 con 0x7fdf7c0a9da0 2021-07-11T04:01:05.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf737fe700 1 -- 172.21.2.104:0/40847210 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf64023790 con 0x7fdf7c0a9da0 2021-07-11T04:01:05.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 -- 172.21.2.104:0/40847210 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c152230 con 0x7fdf7c0a9da0 2021-07-11T04:01:05.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.967+0000 7fdf80bde700 1 -- 172.21.2.104:0/40847210 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7c0664b0 con 0x7fdf7c0a9da0 2021-07-11T04:01:05.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.971+0000 7fdf80bde700 1 -- 172.21.2.104:0/40847210 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7c066210 con 0x7fdf7c0a9da0 2021-07-11T04:01:05.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.971+0000 7fdf737fe700 1 -- 172.21.2.104:0/40847210 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fdf64029020 con 0x7fdf7c0a9da0 2021-07-11T04:01:05.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.971+0000 7fdf737fe700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf60053350 0x7fdf60055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:05.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.971+0000 7fdf737fe700 1 -- 172.21.2.104:0/40847210 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fdf6406ed20 con 0x7fdf7c0a9da0 2021-07-11T04:01:05.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.971+0000 7fdf79d9b700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf60053350 0x7fdf60055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:05.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.971+0000 7fdf79d9b700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf60053350 0x7fdf60055810 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fdf6c002f70 tx=0x7fdf6c006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:05.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:05.971+0000 7fdf737fe700 1 -- 172.21.2.104:0/40847210 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fdf7c066210 con 0x7fdf7c0a9da0 2021-07-11T04:01:06.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.083+0000 7fdf80bde700 1 -- 172.21.2.104:0/40847210 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]} v 0) v1 -- 0x7fdf7c066210 con 0x7fdf7c0a9da0 2021-07-11T04:01:06.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf737fe700 1 -- 172.21.2.104:0/40847210 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_filtered_df"]}]=0 v332) v1 ==== 127+0+0 (secure 0 0 0) 0x7fdf7c066210 con 0x7fdf7c0a9da0 2021-07-11T04:01:06.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 -- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf60053350 msgr2=0x7fdf60055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:06.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf60053350 0x7fdf60055810 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fdf6c002f70 tx=0x7fdf6c006a80).stop 2021-07-11T04:01:06.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 -- 172.21.2.104:0/40847210 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf7c0a9da0 msgr2=0x7fdf7c151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:06.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf7c0a9da0 0x7fdf7c151950 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7fdf7c0b2e10 tx=0x7fdf7c0671e0).stop 2021-07-11T04:01:06.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 -- 172.21.2.104:0/40847210 shutdown_connections 2021-07-11T04:01:06.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf7c0acf30 0x7fdf7c14e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:06.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf7c0aa860 0x7fdf7c0646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:06.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf60053350 0x7fdf60055810 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:06.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 --2- 172.21.2.104:0/40847210 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf7c0a9da0 0x7fdf7c151950 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:06.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 -- 172.21.2.104:0/40847210 >> 172.21.2.104:0/40847210 conn(0x7fdf7c0b64d0 msgr2=0x7fdf7c0b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:06.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 -- 172.21.2.104:0/40847210 shutdown_connections 2021-07-11T04:01:06.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.755+0000 7fdf80bde700 1 -- 172.21.2.104:0/40847210 wait complete. 2021-07-11T04:01:06.764 INFO:tasks.cephfs_test_runner:test_filtered_df (tasks.cephfs.test_misc.TestMisc) ... ok 2021-07-11T04:01:06.765 INFO:tasks.cephfs_test_runner:Starting test: test_fs_new (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:01:06.766 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2021-07-11T04:01:06.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.815+0000 7f572d119700 1 Processor -- start 2021-07-11T04:01:06.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.815+0000 7f572d119700 1 -- start start 2021-07-11T04:01:06.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.815+0000 7f572d119700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f57280b1db0 0x7f57280b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:06.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.815+0000 7f572d119700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f57280a70c0 0x7f5728144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:06.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.815+0000 7f572d119700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57280b0350 con 0x7f57281483a0 2021-07-11T04:01:06.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.815+0000 7f572d119700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57280a90c0 con 0x7f57280b1db0 2021-07-11T04:01:06.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.815+0000 7f572d119700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57280b05f0 con 0x7f57280a70c0 2021-07-11T04:01:06.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572659c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f57281483a0 0x7f57280b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45864/0 (socket says 172.21.2.104:45864) 2021-07-11T04:01:06.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572659c700 1 -- 172.21.2.104:0/3535258133 learned_addr learned my addr 172.21.2.104:0/3535258133 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:06.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f5726d9d700 1 --2- 172.21.2.104:0/3535258133 >> v2:172.21.2.104:3300/0 conn(0x7f57280b1db0 0x7f57280b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:06.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f5725d9b700 1 -- 172.21.2.104:0/3535258133 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 86166033 0 0) 0x7f57280b0350 con 0x7f57281483a0 2021-07-11T04:01:06.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f5725d9b700 1 -- 172.21.2.104:0/3535258133 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5708003260 con 0x7f57281483a0 2021-07-11T04:01:06.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f5726d9d700 1 -- 172.21.2.104:0/3535258133 >> v2:172.21.2.105:3300/0 conn(0x7f57280a70c0 msgr2=0x7f5728144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:06.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f5726d9d700 1 --2- 172.21.2.104:0/3535258133 >> v2:172.21.2.105:3300/0 conn(0x7f57280a70c0 0x7f5728144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:06.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f5726d9d700 1 -- 172.21.2.104:0/3535258133 >> v1:172.21.2.104:6789/0 conn(0x7f57281483a0 legacy=0x7f57280b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:06.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f5726d9d700 1 -- 172.21.2.104:0/3535258133 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57280b0890 con 0x7f57280b1db0 2021-07-11T04:01:06.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f5726d9d700 1 --2- 172.21.2.104:0/3535258133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f57280b1db0 0x7f57280b0fd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f5728147730 tx=0x7f5718006df0).ready entity=mon.0 client_cookie=6016d8d0778ec6ee server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:06.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f5725d9b700 1 -- 172.21.2.104:0/3535258133 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f571800d5c0 con 0x7f57280b1db0 2021-07-11T04:01:06.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f5725d9b700 1 -- 172.21.2.104:0/3535258133 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5718000e90 con 0x7f57280b1db0 2021-07-11T04:01:06.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f5725d9b700 1 -- 172.21.2.104:0/3535258133 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f571800d910 con 0x7f57280b1db0 2021-07-11T04:01:06.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 -- 172.21.2.104:0/3535258133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f57280b1db0 msgr2=0x7f57280b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:06.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 --2- 172.21.2.104:0/3535258133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f57280b1db0 0x7f57280b0fd0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f5728147730 tx=0x7f5718006df0).stop 2021-07-11T04:01:06.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 -- 172.21.2.104:0/3535258133 shutdown_connections 2021-07-11T04:01:06.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 --2- 172.21.2.104:0/3535258133 >> v2:172.21.2.105:3300/0 conn(0x7f57280a70c0 0x7f5728144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:06.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 --2- 172.21.2.104:0/3535258133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f57280b1db0 0x7f57280b0fd0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:06.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 -- 172.21.2.104:0/3535258133 >> 172.21.2.104:0/3535258133 conn(0x7f572805f6b0 msgr2=0x7f572805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:06.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 -- 172.21.2.104:0/3535258133 shutdown_connections 2021-07-11T04:01:06.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 -- 172.21.2.104:0/3535258133 wait complete. 2021-07-11T04:01:06.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 Processor -- start 2021-07-11T04:01:06.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 -- start start 2021-07-11T04:01:06.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f57280a70c0 0x7f5728064820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:06.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57280b1db0 0x7f57280b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:06.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f57281483a0 0x7f57281567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:06.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5728064d10 con 0x7f57281483a0 2021-07-11T04:01:06.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5728062b40 con 0x7f57280a70c0 2021-07-11T04:01:06.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5728062de0 con 0x7f57280b1db0 2021-07-11T04:01:06.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572759e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f57281483a0 0x7f57281567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:06.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572759e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f57281483a0 0x7f57281567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49106/0 (socket says 172.21.2.104:49106) 2021-07-11T04:01:06.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572759e700 1 -- 172.21.2.104:0/1300634941 learned_addr learned my addr 172.21.2.104:0/1300634941 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:06.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572759e700 1 -- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57280b1db0 msgr2=0x7f57280b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:06.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572759e700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57280b1db0 0x7f57280b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:06.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572759e700 1 -- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f57280a70c0 msgr2=0x7f5728064820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:06.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572759e700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f57280a70c0 0x7f5728064820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:06.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572759e700 1 -- 172.21.2.104:0/1300634941 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571800b180 con 0x7f57281483a0 2021-07-11T04:01:06.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572759e700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f57281483a0 0x7f57281567a0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f57200032f0 tx=0x7f5720009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:06.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f571ffff700 1 -- 172.21.2.104:0/1300634941 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f572000d580 con 0x7f57281483a0 2021-07-11T04:01:06.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f571ffff700 1 -- 172.21.2.104:0/1300634941 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572000d6e0 con 0x7f57281483a0 2021-07-11T04:01:06.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 -- 172.21.2.104:0/1300634941 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5728064380 con 0x7f57281483a0 2021-07-11T04:01:06.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f571ffff700 1 -- 172.21.2.104:0/1300634941 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f572000da40 con 0x7f57281483a0 2021-07-11T04:01:06.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.819+0000 7f572d119700 1 -- 172.21.2.104:0/1300634941 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57280627c0 con 0x7f57281483a0 2021-07-11T04:01:06.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.823+0000 7f572d119700 1 -- 172.21.2.104:0/1300634941 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5728156e30 con 0x7f57281483a0 2021-07-11T04:01:06.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.823+0000 7f571ffff700 1 -- 172.21.2.104:0/1300634941 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f572000dba0 con 0x7f57281483a0 2021-07-11T04:01:06.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.823+0000 7f571ffff700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f570c053300 0x7f570c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:06.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.823+0000 7f571ffff700 1 -- 172.21.2.104:0/1300634941 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f572006b5a0 con 0x7f57281483a0 2021-07-11T04:01:06.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.823+0000 7f5726d9d700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f570c053300 0x7f570c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:06.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.823+0000 7f5726d9d700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f570c053300 0x7f570c0557c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f5728147730 tx=0x7f5718014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:06.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.823+0000 7f571ffff700 1 -- 172.21.2.104:0/1300634941 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5720042bc0 con 0x7f57281483a0 2021-07-11T04:01:06.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:06.935+0000 7f572d119700 1 -- 172.21.2.104:0/1300634941 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f5728062fe0 con 0x7f57281483a0 2021-07-11T04:01:07.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.759+0000 7f571ffff700 1 -- 172.21.2.104:0/1300634941 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v333) v1 ==== 125+0+0 (secure 0 0 0) 0x7f57200461e0 con 0x7f57281483a0 2021-07-11T04:01:07.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 -- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f570c053300 msgr2=0x7f570c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:07.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f570c053300 0x7f570c0557c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f5728147730 tx=0x7f5718014000).stop 2021-07-11T04:01:07.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 -- 172.21.2.104:0/1300634941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f57281483a0 msgr2=0x7f57281567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:07.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f57281483a0 0x7f57281567a0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f57200032f0 tx=0x7f5720009220).stop 2021-07-11T04:01:07.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 -- 172.21.2.104:0/1300634941 shutdown_connections 2021-07-11T04:01:07.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f570c053300 0x7f570c0557c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:07.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f57281483a0 0x7f57281567a0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:07.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f57280b1db0 0x7f57280b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:07.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 --2- 172.21.2.104:0/1300634941 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f57280a70c0 0x7f5728064820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:07.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 -- 172.21.2.104:0/1300634941 >> 172.21.2.104:0/1300634941 conn(0x7f572805f6b0 msgr2=0x7f5728147c80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:07.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 -- 172.21.2.104:0/1300634941 shutdown_connections 2021-07-11T04:01:07.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.763+0000 7f572d119700 1 -- 172.21.2.104:0/1300634941 wait complete. 2021-07-11T04:01:07.771 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T04:01:07.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.823+0000 7f83fed79700 1 Processor -- start 2021-07-11T04:01:07.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.823+0000 7f83fed79700 1 -- start start 2021-07-11T04:01:07.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.823+0000 7f83fed79700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f83f80b1ec0 0x7f83f80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:07.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.823+0000 7f83fed79700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f80b0420 con 0x7f83f80affb0 2021-07-11T04:01:07.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.823+0000 7f83fed79700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f80a77d0 con 0x7f83f80b22a0 2021-07-11T04:01:07.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.823+0000 7f83fed79700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f80b0650 con 0x7f83f80b1ec0 2021-07-11T04:01:07.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f7fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f83f80affb0 0x7f83f80a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45882/0 (socket says 172.21.2.104:45882) 2021-07-11T04:01:07.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f7fff700 1 -- 172.21.2.104:0/102327301 learned_addr learned my addr 172.21.2.104:0/102327301 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:07.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363905468 0 0) 0x7f83f80b0420 con 0x7f83f80affb0 2021-07-11T04:01:07.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83d8003260 con 0x7f83f80affb0 2021-07-11T04:01:07.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1704990933 0 0) 0x7f83d8003260 con 0x7f83f80affb0 2021-07-11T04:01:07.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 >> v2:172.21.2.105:3300/0 conn(0x7f83f80b1ec0 msgr2=0x7f83f80b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:07.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fcb15700 1 --2- 172.21.2.104:0/102327301 >> v2:172.21.2.105:3300/0 conn(0x7f83f80b1ec0 0x7f83f80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:07.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 --2- 172.21.2.104:0/102327301 >> v2:172.21.2.105:3300/0 conn(0x7f83f80b1ec0 0x7f83f80b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:07.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 >> v1:172.21.2.105:6789/0 conn(0x7f83f80b22a0 legacy=0x7f83f8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:07.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f80b08f0 con 0x7f83f80affb0 2021-07-11T04:01:07.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f83e8001660 con 0x7f83f80affb0 2021-07-11T04:01:07.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 >> v1:172.21.2.104:6789/0 conn(0x7f83f80affb0 legacy=0x7f83f80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:07.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 --2- 172.21.2.104:0/102327301 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83d8003820 0x7f83d8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:07.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 --2- 172.21.2.104:0/102327301 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83d8004500 0x7f83d8004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:07.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 --2- 172.21.2.104:0/102327301 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83d8005050 0x7f83d800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:07.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80b0650 con 0x7f83d8005050 2021-07-11T04:01:07.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d800bf40 con 0x7f83d8004500 2021-07-11T04:01:07.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d800c100 con 0x7f83d8003820 2021-07-11T04:01:07.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fd316700 1 --2- 172.21.2.104:0/102327301 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83d8005050 0x7f83d800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:07.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fd316700 1 -- 172.21.2.104:0/102327301 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83d8003820 msgr2=0x7f83d8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:07.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fd316700 1 --2- 172.21.2.104:0/102327301 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83d8003820 0x7f83d8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:07.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fd316700 1 -- 172.21.2.104:0/102327301 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83d8004500 msgr2=0x7f83d8004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:07.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fd316700 1 --2- 172.21.2.104:0/102327301 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83d8004500 0x7f83d8004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:07.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fd316700 1 -- 172.21.2.104:0/102327301 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d8003260 con 0x7f83d8005050 2021-07-11T04:01:07.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fd316700 1 -- 172.21.2.104:0/102327301 reap_dead start 2021-07-11T04:01:07.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fd316700 1 --2- 172.21.2.104:0/102327301 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83d8005050 0x7f83d800b750 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f83f00012f0 tx=0x7f83f0007d60).ready entity=mon.0 client_cookie=cee3f37df3814121 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:07.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83f00014c0 con 0x7f83d8005050 2021-07-11T04:01:07.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f0006870 con 0x7f83d8005050 2021-07-11T04:01:07.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f77fe700 1 -- 172.21.2.104:0/102327301 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83f0006bd0 con 0x7f83d8005050 2021-07-11T04:01:07.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 -- 172.21.2.104:0/102327301 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83d8005050 msgr2=0x7f83d800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:07.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 --2- 172.21.2.104:0/102327301 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83d8005050 0x7f83d800b750 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f83f00012f0 tx=0x7f83f0007d60).stop 2021-07-11T04:01:07.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 -- 172.21.2.104:0/102327301 shutdown_connections 2021-07-11T04:01:07.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 --2- 172.21.2.104:0/102327301 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83d8005050 0x7f83d800b750 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:07.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 -- 172.21.2.104:0/102327301 >> 172.21.2.104:0/102327301 conn(0x7f83f805f6b0 msgr2=0x7f83f805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:07.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 -- 172.21.2.104:0/102327301 shutdown_connections 2021-07-11T04:01:07.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 -- 172.21.2.104:0/102327301 wait complete. 2021-07-11T04:01:07.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 Processor -- start 2021-07-11T04:01:07.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 -- start start 2021-07-11T04:01:07.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83d8005050 0x7f83f80aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:07.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83f80b5690 0x7f83f814aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:07.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83f813ba60 0x7f83f813bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:07.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80ab570 con 0x7f83f80b5690 2021-07-11T04:01:07.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f813c3d0 con 0x7f83f813ba60 2021-07-11T04:01:07.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83fed79700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f80b5b10 con 0x7f83d8005050 2021-07-11T04:01:07.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83f80b5690 0x7f83f814aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:07.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83f80b5690 0x7f83f814aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49126/0 (socket says 172.21.2.104:49126) 2021-07-11T04:01:07.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.827+0000 7f83f7fff700 1 -- 172.21.2.104:0/1043350267 learned_addr learned my addr 172.21.2.104:0/1043350267 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:07.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f7fff700 1 -- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83d8005050 msgr2=0x7f83f80aa8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:07.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f7fff700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83d8005050 0x7f83f80aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:07.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f7fff700 1 -- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83f813ba60 msgr2=0x7f83f813bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:07.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f7fff700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83f813ba60 0x7f83f813bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:07.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f7fff700 1 -- 172.21.2.104:0/1043350267 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f8149e50 con 0x7f83f80b5690 2021-07-11T04:01:07.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f7fff700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83f80b5690 0x7f83f814aef0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f83f80b5e10 tx=0x7f83f80b0a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:07.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f5ffb700 1 -- 172.21.2.104:0/1043350267 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83e8007170 con 0x7f83f80b5690 2021-07-11T04:01:07.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f5ffb700 1 -- 172.21.2.104:0/1043350267 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e80072d0 con 0x7f83f80b5690 2021-07-11T04:01:07.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f5ffb700 1 -- 172.21.2.104:0/1043350267 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83e8004920 con 0x7f83f80b5690 2021-07-11T04:01:07.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83fed79700 1 -- 172.21.2.104:0/1043350267 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f80b32b0 con 0x7f83f80b5690 2021-07-11T04:01:07.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83fed79700 1 -- 172.21.2.104:0/1043350267 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f80ac490 con 0x7f83f80b5690 2021-07-11T04:01:07.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83fed79700 1 -- 172.21.2.104:0/1043350267 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f8057350 con 0x7f83f80b5690 2021-07-11T04:01:07.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f5ffb700 1 -- 172.21.2.104:0/1043350267 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f83e8004a80 con 0x7f83f80b5690 2021-07-11T04:01:07.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f5ffb700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83dc0532b0 0x7f83dc055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:07.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83f5ffb700 1 -- 172.21.2.104:0/1043350267 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f83e806ad20 con 0x7f83f80b5690 2021-07-11T04:01:07.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83fcb15700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83dc0532b0 0x7f83dc055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:07.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.831+0000 7f83fcb15700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83dc0532b0 0x7f83dc055770 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f83e0007000 tx=0x7f83e0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:07.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.835+0000 7f83f5ffb700 1 -- 172.21.2.104:0/1043350267 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f83e8042340 con 0x7f83f80b5690 2021-07-11T04:01:07.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.915+0000 7f83fed79700 1 -- 172.21.2.104:0/1043350267 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f83f814acd0 con 0x7f83f80b5690 2021-07-11T04:01:07.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.919+0000 7f83f5ffb700 1 -- 172.21.2.104:0/1043350267 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f83e8045960 con 0x7f83f80b5690 2021-07-11T04:01:07.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 -- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83dc0532b0 msgr2=0x7f83dc055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:07.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83dc0532b0 0x7f83dc055770 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f83e0007000 tx=0x7f83e0008040).stop 2021-07-11T04:01:07.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 -- 172.21.2.104:0/1043350267 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83f80b5690 msgr2=0x7f83f814aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:07.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83f80b5690 0x7f83f814aef0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f83f80b5e10 tx=0x7f83f80b0a80).stop 2021-07-11T04:01:07.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 -- 172.21.2.104:0/1043350267 shutdown_connections 2021-07-11T04:01:07.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83f813ba60 0x7f83f813bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:07.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83dc0532b0 0x7f83dc055770 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:07.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83f80b5690 0x7f83f814aef0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:07.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 --2- 172.21.2.104:0/1043350267 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83d8005050 0x7f83f80aa8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:07.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 -- 172.21.2.104:0/1043350267 >> 172.21.2.104:0/1043350267 conn(0x7f83f805f6b0 msgr2=0x7f83f813d760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:07.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 -- 172.21.2.104:0/1043350267 shutdown_connections 2021-07-11T04:01:07.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:07.923+0000 7f83fed79700 1 -- 172.21.2.104:0/1043350267 wait complete. 2021-07-11T04:01:07.929 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:01:07.930 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:07.938 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:07.946 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:07.960 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:07.973 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:01:07.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:07.974 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:07.975 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:08.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:01:08.009 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T04:01:08.009 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T04:01:08.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:08.010 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:08.011 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:08.019 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T04:01:08.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:01:08.021 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T04:01:08.021 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:01:08.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.075+0000 7fd59f708700 1 Processor -- start 2021-07-11T04:01:08.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.075+0000 7fd59f708700 1 -- start start 2021-07-11T04:01:08.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.075+0000 7fd59f708700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5980b1db0 0x7fd5980b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.075+0000 7fd59f708700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd5981483a0 0x7fd5980b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.075+0000 7fd59f708700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5980b0350 con 0x7fd5980a70c0 2021-07-11T04:01:08.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.075+0000 7fd59f708700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5980a90c0 con 0x7fd5981483a0 2021-07-11T04:01:08.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.075+0000 7fd59f708700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980b05f0 con 0x7fd5980b1db0 2021-07-11T04:01:08.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.075+0000 7fd59dca5700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd5980a70c0 0x7fd598144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45902/0 (socket says 172.21.2.104:45902) 2021-07-11T04:01:08.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.075+0000 7fd59dca5700 1 -- 172.21.2.104:0/3770291569 learned_addr learned my addr 172.21.2.104:0/3770291569 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:08.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.075+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979379008 0 0) 0x7fd5980b0350 con 0x7fd5980a70c0 2021-07-11T04:01:08.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd580003260 con 0x7fd5980a70c0 2021-07-11T04:01:08.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1759110946 0 0) 0x7fd580003260 con 0x7fd5980a70c0 2021-07-11T04:01:08.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5980b1db0 msgr2=0x7fd5980b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:08.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 --2- 172.21.2.104:0/3770291569 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5980b1db0 0x7fd5980b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 >> v2:172.21.2.105:3300/0 conn(0x7fd5981483a0 msgr2=0x7fd5980b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:08.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 --2- 172.21.2.104:0/3770291569 >> v2:172.21.2.105:3300/0 conn(0x7fd5981483a0 0x7fd5980b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980b0890 con 0x7fd5980a70c0 2021-07-11T04:01:08.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fd5900019e0 con 0x7fd5980a70c0 2021-07-11T04:01:08.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 >> v1:172.21.2.104:6789/0 conn(0x7fd5980a70c0 legacy=0x7fd598144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 --2- 172.21.2.104:0/3770291569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5800037d0 0x7fd580003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 --2- 172.21.2.104:0/3770291569 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5980b1db0 0x7fd580004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 --2- 172.21.2.104:0/3770291569 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd580004bf0 0x7fd58000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980b05f0 con 0x7fd5800037d0 2021-07-11T04:01:08.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd580005280 con 0x7fd580004bf0 2021-07-11T04:01:08.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58000bc40 con 0x7fd5980b1db0 2021-07-11T04:01:08.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59d4a4700 1 --2- 172.21.2.104:0/3770291569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5800037d0 0x7fd580003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59d4a4700 1 -- 172.21.2.104:0/3770291569 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5980b1db0 msgr2=0x7fd580004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:08.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59d4a4700 1 --2- 172.21.2.104:0/3770291569 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd5980b1db0 0x7fd580004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59d4a4700 1 -- 172.21.2.104:0/3770291569 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd580004bf0 msgr2=0x7fd58000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:08.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59d4a4700 1 --2- 172.21.2.104:0/3770291569 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd580004bf0 0x7fd58000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59d4a4700 1 -- 172.21.2.104:0/3770291569 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd580003260 con 0x7fd5800037d0 2021-07-11T04:01:08.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59d4a4700 1 --2- 172.21.2.104:0/3770291569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5800037d0 0x7fd580003bf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7fd58c002f70 tx=0x7fd58c006ca0).ready entity=mon.0 client_cookie=a57593097e3d1428 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd58c012410 con 0x7fd5800037d0 2021-07-11T04:01:08.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c012570 con 0x7fd5800037d0 2021-07-11T04:01:08.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58bfff700 1 -- 172.21.2.104:0/3770291569 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd58c0128d0 con 0x7fd5800037d0 2021-07-11T04:01:08.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 -- 172.21.2.104:0/3770291569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5800037d0 msgr2=0x7fd580003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 --2- 172.21.2.104:0/3770291569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd5800037d0 0x7fd580003bf0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7fd58c002f70 tx=0x7fd58c006ca0).stop 2021-07-11T04:01:08.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59dca5700 1 -- 172.21.2.104:0/3770291569 reap_dead start 2021-07-11T04:01:08.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 -- 172.21.2.104:0/3770291569 shutdown_connections 2021-07-11T04:01:08.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 -- 172.21.2.104:0/3770291569 >> 172.21.2.104:0/3770291569 conn(0x7fd59805f6b0 msgr2=0x7fd59805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:08.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 -- 172.21.2.104:0/3770291569 shutdown_connections 2021-07-11T04:01:08.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 -- 172.21.2.104:0/3770291569 wait complete. 2021-07-11T04:01:08.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 Processor -- start 2021-07-11T04:01:08.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 -- start start 2021-07-11T04:01:08.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd5980ad1e0 0x7fd598146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd598146be0 0x7fd598159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd59815a2f0 0x7fd59813f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd598145ad0 con 0x7fd59815a2f0 2021-07-11T04:01:08.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980ad5c0 con 0x7fd5980ad1e0 2021-07-11T04:01:08.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980ad970 con 0x7fd598146be0 2021-07-11T04:01:08.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59dca5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd59815a2f0 0x7fd59813f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59dca5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd59815a2f0 0x7fd59813f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49148/0 (socket says 172.21.2.104:49148) 2021-07-11T04:01:08.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59dca5700 1 -- 172.21.2.104:0/1543224213 learned_addr learned my addr 172.21.2.104:0/1543224213 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:08.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59d4a4700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd5980ad1e0 0x7fd598146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59cca3700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd598146be0 0x7fd598159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59dca5700 1 -- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd598146be0 msgr2=0x7fd598159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59dca5700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd598146be0 0x7fd598159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59dca5700 1 -- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd5980ad1e0 msgr2=0x7fd598146440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59dca5700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd5980ad1e0 0x7fd598146440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59dca5700 1 -- 172.21.2.104:0/1543224213 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5981470f0 con 0x7fd59815a2f0 2021-07-11T04:01:08.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59cca3700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd598146be0 0x7fd598159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:08.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59dca5700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd59815a2f0 0x7fd59813f840 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7fd590001bc0 tx=0x7fd5980b0a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58a7fc700 1 -- 172.21.2.104:0/1543224213 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd590004b70 con 0x7fd59815a2f0 2021-07-11T04:01:08.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58a7fc700 1 -- 172.21.2.104:0/1543224213 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590004cd0 con 0x7fd59815a2f0 2021-07-11T04:01:08.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 -- 172.21.2.104:0/1543224213 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd598145d00 con 0x7fd59815a2f0 2021-07-11T04:01:08.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd58a7fc700 1 -- 172.21.2.104:0/1543224213 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd5900072c0 con 0x7fd59815a2f0 2021-07-11T04:01:08.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.079+0000 7fd59f708700 1 -- 172.21.2.104:0/1543224213 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5980afa90 con 0x7fd59815a2f0 2021-07-11T04:01:08.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.083+0000 7fd59f708700 1 -- 172.21.2.104:0/1543224213 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd598057350 con 0x7fd59815a2f0 2021-07-11T04:01:08.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.083+0000 7fd58a7fc700 1 -- 172.21.2.104:0/1543224213 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd59001b070 con 0x7fd59815a2f0 2021-07-11T04:01:08.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.083+0000 7fd58a7fc700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5780532b0 0x7fd578055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.083+0000 7fd58a7fc700 1 -- 172.21.2.104:0/1543224213 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd59006a870 con 0x7fd59815a2f0 2021-07-11T04:01:08.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.083+0000 7fd59d4a4700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5780532b0 0x7fd578055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.083+0000 7fd59d4a4700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5780532b0 0x7fd578055770 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fd58c018000 tx=0x7fd58c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.083+0000 7fd58a7fc700 1 -- 172.21.2.104:0/1543224213 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd590041e90 con 0x7fd59815a2f0 2021-07-11T04:01:08.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.167+0000 7fd59f708700 1 -- 172.21.2.104:0/1543224213 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd598044650 con 0x7fd59815a2f0 2021-07-11T04:01:08.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.167+0000 7fd58a7fc700 1 -- 172.21.2.104:0/1543224213 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v111) v1 ==== 81+0+28835 (secure 0 0 0) 0x7fd5900454b0 con 0x7fd59815a2f0 2021-07-11T04:01:08.170 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:08.171 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:01:08.171 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 111, 2021-07-11T04:01:08.171 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:01:08.171 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:01:08.172 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:01:05.737351+0000", 2021-07-11T04:01:08.172 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:01:08.172 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:01:08.172 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:01:08.173 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:01:08.173 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:01:08.173 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:01:08.173 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:01:08.174 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:01:08.174 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:01:08.174 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:08.174 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:01:08.175 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:01:08.175 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:01:08.175 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:01:08.175 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:01:08.175 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 20, 2021-07-11T04:01:08.176 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:01:08.176 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:01:08.176 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:01:08.176 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:01:08.176 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:01:08.177 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.177 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:01:08.177 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:01:08.177 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:01:08.178 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:01:08.178 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:01:08.178 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:08.178 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:08.178 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:08.179 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:08.179 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:08.179 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:08.179 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:01:08.179 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:01:08.180 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:01:08.180 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:01:08.180 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:01:08.180 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:08.181 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:08.181 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:08.181 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:08.181 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:08.181 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:08.182 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:08.182 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.182 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:01:08.182 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:08.182 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:08.183 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:08.183 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:08.183 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:08.183 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:08.183 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:08.184 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:08.184 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:08.184 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:08.184 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:08.185 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:08.185 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:08.185 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:08.185 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:08.185 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:08.186 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:08.186 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:08.186 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:08.186 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:08.186 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:08.187 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:08.187 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:08.187 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:08.187 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:08.187 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:08.188 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.188 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:08.188 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:08.188 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:08.189 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:08.189 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:08.189 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:08.189 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:08.189 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:08.190 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:08.190 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:08.190 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:08.190 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:01:08.190 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:01:08.191 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.191 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:08.191 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:01:08.191 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.192 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.192 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.192 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:01:08.192 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:01:08.192 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:01:08.193 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:01:08.193 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:01:08.193 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:08.193 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:08.193 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:08.194 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:08.194 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:08.194 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:08.194 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:01:08.194 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:01:08.195 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:01:08.195 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:01:08.195 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:01:08.195 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:08.196 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:08.196 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:08.196 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:08.196 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:08.196 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:08.197 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:08.197 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.197 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:01:08.197 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:08.197 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:08.198 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:08.198 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:08.198 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:08.198 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:08.198 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:08.199 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:08.199 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:08.199 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:08.199 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:08.199 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:08.200 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:08.200 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:08.200 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:08.200 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:08.201 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:08.201 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:08.201 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:08.201 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:08.201 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:08.202 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:08.202 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:08.202 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:08.202 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:08.202 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:08.203 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.203 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:08.203 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:08.203 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:08.203 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:08.204 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:08.204 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:08.204 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:08.204 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:08.205 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:08.205 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:08.205 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:08.205 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:01:08.205 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:08.206 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:01:08.206 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.206 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.206 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:08.206 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:01:08.207 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.207 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:01:08.207 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:01:08.207 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:08.207 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:08.208 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:08.208 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:08.208 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:08.208 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:08.209 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:08.209 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 106, 2021-07-11T04:01:08.209 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:08.209 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:08.209 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:08.210 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.210 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.210 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.210 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:01:08.210 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:08.211 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.211 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.211 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.211 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:01:08.211 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:08.212 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.212 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.212 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.212 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:08.212 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.213 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.213 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.213 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:01:08.213 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:08.214 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.214 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.214 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.214 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:01:08.214 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:08.215 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.215 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.215 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.215 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:08.215 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.216 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.216 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.216 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:01:08.216 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:08.216 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.217 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.217 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.217 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:01:08.217 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:08.217 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.218 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.218 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.218 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:08.218 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.219 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.219 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.219 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:01:08.219 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:08.219 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.220 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.220 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.220 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:01:08.220 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:08.220 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.221 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.221 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.221 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:01:08.221 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:01:08.221 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:01:08.222 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:01:08.222 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:08.222 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:08.222 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:08.223 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.223 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.223 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.223 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:01:08.223 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:01:08.224 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:08.224 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:08.224 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:08.224 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:08.224 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:08.225 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:08.225 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:08.225 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 106, 2021-07-11T04:01:08.225 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:08.226 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:08.226 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:08.226 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.226 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.227 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.227 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:01:08.227 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:08.227 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.227 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.228 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.228 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:01:08.228 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:08.228 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.229 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.229 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.229 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:08.229 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.229 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.230 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.230 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:01:08.230 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:08.230 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.230 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.231 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.231 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:01:08.231 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:08.231 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.232 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.232 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.232 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:08.232 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.232 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.233 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.233 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:01:08.233 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:08.233 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.233 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.234 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.234 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:01:08.234 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:08.234 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.234 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.235 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.235 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:08.235 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.235 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.236 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.236 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:01:08.236 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:08.236 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.236 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.237 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.237 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:01:08.237 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:08.237 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.237 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.238 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.238 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:01:08.238 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:01:08.238 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:01:08.238 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:01:08.239 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:08.239 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:08.239 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:08.239 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.240 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.240 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.240 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:01:08.240 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:01:08.240 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:08.241 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:08.241 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:08.241 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:08.241 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:08.241 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:08.242 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:08.242 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 105, 2021-07-11T04:01:08.242 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:08.242 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:08.242 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:08.243 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.243 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.243 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.243 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:01:08.244 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:08.244 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.244 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.244 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.244 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:01:08.245 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:08.245 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.245 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.245 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.245 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:08.246 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.246 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.246 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.246 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:01:08.246 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:08.247 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.247 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.247 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.247 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:01:08.248 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:08.248 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.248 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.248 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.248 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:08.249 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.249 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.249 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.249 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:01:08.249 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:08.250 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.250 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.250 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.250 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:01:08.250 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:08.251 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.251 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.251 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.251 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:08.251 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.252 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.252 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.252 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:01:08.252 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:08.253 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.253 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.253 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.253 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:01:08.253 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:08.254 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.254 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.254 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.254 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:01:08.254 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:01:08.255 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:01:08.255 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:01:08.255 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:08.255 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:08.255 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:08.256 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.256 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.256 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.256 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:01:08.256 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:01:08.257 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:08.257 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:08.257 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:08.257 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:08.258 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:08.258 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:08.258 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:08.258 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 106, 2021-07-11T04:01:08.258 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:08.259 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:08.259 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:08.259 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.259 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.259 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.260 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:01:08.260 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:08.260 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.260 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.260 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.261 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:01:08.261 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:08.261 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.261 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.261 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.262 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:08.262 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.262 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.262 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.263 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:01:08.263 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:08.263 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.263 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.263 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.264 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:01:08.264 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:08.264 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.264 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.264 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.265 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:08.265 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.265 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.265 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.265 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:01:08.266 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:08.266 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.266 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.266 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.267 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:01:08.267 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:08.267 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.267 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.267 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.268 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:08.268 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.268 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.268 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.268 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:01:08.269 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:08.269 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.269 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.269 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.269 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:01:08.270 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:08.270 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.270 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.270 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.271 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:01:08.271 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:01:08.271 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:01:08.271 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:01:08.271 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:08.272 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:08.272 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:08.272 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.272 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.272 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.273 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:01:08.273 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:01:08.273 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:08.273 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:08.273 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:08.274 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:08.274 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:08.274 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:08.274 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:08.275 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 105, 2021-07-11T04:01:08.275 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:08.275 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:08.275 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:08.275 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.276 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.276 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.276 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:01:08.276 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:08.276 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.277 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.277 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.277 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:01:08.277 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:08.277 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.278 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.278 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.278 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:08.278 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.279 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.279 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.279 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:01:08.279 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:08.279 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.280 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.280 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.280 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:01:08.280 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:08.280 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.281 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.281 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.281 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:08.281 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.281 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.282 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.282 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:01:08.282 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:08.282 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.282 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.283 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.283 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:01:08.283 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:08.283 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.284 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.284 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.284 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:08.284 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.284 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.285 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.285 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:01:08.285 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:08.285 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.285 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.286 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.286 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:01:08.286 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:08.286 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.286 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.287 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.287 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:01:08.287 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:01:08.287 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:01:08.287 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:01:08.288 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:08.288 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:08.288 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:08.288 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.289 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.289 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.289 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:01:08.289 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:01:08.289 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:08.290 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:08.290 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:08.290 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:08.290 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:08.290 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:08.291 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:08.291 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 105, 2021-07-11T04:01:08.291 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:08.291 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:08.291 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:08.292 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.292 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.292 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.292 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:01:08.292 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:08.293 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.293 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.293 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.293 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:01:08.294 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:08.294 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.294 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.294 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.294 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:08.295 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.295 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.295 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.295 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:01:08.295 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:08.296 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.296 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.296 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.296 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:01:08.296 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:08.297 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.297 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.297 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.297 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:08.298 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.298 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.298 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.298 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:01:08.298 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:08.299 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.299 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.299 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.299 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:01:08.299 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:08.300 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.300 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.300 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.300 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:08.300 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.301 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.301 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.301 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:01:08.301 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:08.301 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.302 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.302 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.302 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:01:08.302 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:08.303 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.303 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.303 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.303 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:01:08.303 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:01:08.304 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:01:08.304 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:01:08.304 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:08.304 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:08.304 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:08.305 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.305 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.305 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.305 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:01:08.305 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:01:08.306 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:08.306 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:08.306 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:08.306 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:08.307 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:08.307 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:08.307 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:08.307 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 106, 2021-07-11T04:01:08.307 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:08.308 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:08.308 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:08.308 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.308 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.308 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.309 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:01:08.309 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:08.309 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.309 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.309 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.310 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:01:08.310 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:08.310 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.310 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.310 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.311 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:08.311 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.311 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.311 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.311 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:01:08.312 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:08.312 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.312 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.312 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.313 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:01:08.313 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:08.313 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.313 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.313 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.314 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:08.314 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.314 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.314 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.314 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:01:08.315 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:08.315 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.315 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.315 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.315 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:01:08.316 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:08.316 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.316 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.316 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.316 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:08.317 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.317 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.317 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.317 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:01:08.318 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:08.318 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.318 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.318 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.318 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:01:08.319 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:08.319 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.319 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.319 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.319 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:01:08.320 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:01:08.320 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:01:08.320 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:01:08.320 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:08.320 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:08.321 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:08.321 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.321 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.321 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.321 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:01:08.322 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:01:08.322 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:08.322 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:08.322 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:08.323 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:08.323 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:08.323 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:08.323 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:08.323 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 106, 2021-07-11T04:01:08.324 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:08.324 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:08.324 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:08.324 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.324 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.325 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.325 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:01:08.325 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:08.325 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.325 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.326 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.326 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:01:08.326 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:08.326 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.326 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.327 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.327 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:08.327 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.327 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.327 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.328 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:01:08.328 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:08.328 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.328 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.329 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.329 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:01:08.329 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:08.329 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.329 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.330 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.330 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:08.330 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.330 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.330 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.331 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:01:08.331 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:08.331 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.331 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.331 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.332 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:01:08.332 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:08.332 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.332 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.332 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.333 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:08.333 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:08.333 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.333 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:08.334 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:01:08.334 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:08.334 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.334 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.334 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:08.335 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:01:08.335 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:08.335 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.335 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.335 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.336 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:01:08.336 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:01:08.336 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:01:08.336 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:01:08.336 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:08.337 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:08.337 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:08.337 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:08.337 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.337 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:08.338 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:01:08.338 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.338 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:01:08.338 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:08.339 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:08.339 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:08.339 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:08.339 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:08.339 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:01:08.340 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:08.340 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.340 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.340 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:01:08.340 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:08.341 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:08.341 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:08.341 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:08.341 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:08.341 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:01:08.342 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:08.342 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.342 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.342 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:01:08.343 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:08.343 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:08.343 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:08.343 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:08.343 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:08.344 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:01:08.344 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:08.344 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.344 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.344 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:01:08.345 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:08.345 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:08.345 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:08.345 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:08.345 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:08.346 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:01:08.346 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:08.346 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.346 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.346 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:01:08.347 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:08.347 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:08.347 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:08.347 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:08.348 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:08.348 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:01:08.348 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:08.348 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.348 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.349 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:01:08.349 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:08.349 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:08.349 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:08.349 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:08.350 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:08.350 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:01:08.350 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:08.350 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.350 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.351 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:01:08.351 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:08.351 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:08.351 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:08.351 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:08.352 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:08.352 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:01:08.352 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:08.352 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.352 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:08.353 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:01:08.353 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:08.353 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:08.353 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:08.354 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:08.354 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:08.354 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:01:08.354 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:08.354 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.355 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:08.355 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:01:08.355 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:01:08.355 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:01:08.355 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:01:08.356 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T04:01:08.356 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6835/871079070": "2021-07-12T04:01:02.700926+0000", 2021-07-11T04:01:08.356 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6834/871079070": "2021-07-12T04:01:02.700926+0000" 2021-07-11T04:01:08.356 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.356 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:01:08.357 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:01:08.357 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:01:08.357 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:01:08.357 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:01:08.357 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:01:08.358 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:01:08.358 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:01:08.358 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:08.358 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:08.359 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:01:08.359 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:01:08.359 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:01:08.359 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:01:08.359 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:01:08.360 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:01:08.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 -- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5780532b0 msgr2=0x7fd578055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5780532b0 0x7fd578055770 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fd58c018000 tx=0x7fd58c015000).stop 2021-07-11T04:01:08.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 -- 172.21.2.104:0/1543224213 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd59815a2f0 msgr2=0x7fd59813f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd59815a2f0 0x7fd59813f840 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7fd590001bc0 tx=0x7fd5980b0a20).stop 2021-07-11T04:01:08.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 -- 172.21.2.104:0/1543224213 shutdown_connections 2021-07-11T04:01:08.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd5780532b0 0x7fd578055770 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd59815a2f0 0x7fd59813f840 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd598146be0 0x7fd598159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 --2- 172.21.2.104:0/1543224213 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd5980ad1e0 0x7fd598146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 -- 172.21.2.104:0/1543224213 >> 172.21.2.104:0/1543224213 conn(0x7fd59805f6b0 msgr2=0x7fd598142bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:08.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 -- 172.21.2.104:0/1543224213 shutdown_connections 2021-07-11T04:01:08.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.171+0000 7fd59f708700 1 -- 172.21.2.104:0/1543224213 wait complete. 2021-07-11T04:01:08.364 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:08.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.435+0000 7ff233da0700 1 Processor -- start 2021-07-11T04:01:08.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.435+0000 7ff233da0700 1 -- start start 2021-07-11T04:01:08.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.435+0000 7ff233da0700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff22c067180 0x7ff22c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.435+0000 7ff233da0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff22c0b1970 0x7ff22c05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.435+0000 7ff233da0700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff22c0a7a90 con 0x7ff22c0a76b0 2021-07-11T04:01:08.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.435+0000 7ff233da0700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c147620 con 0x7ff22c0b1970 2021-07-11T04:01:08.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.435+0000 7ff233da0700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff22c1478c0 con 0x7ff22c067180 2021-07-11T04:01:08.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.435+0000 7ff231b3c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff22c0a76b0 0x7ff22c066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:45920/0 (socket says 172.21.2.104:45920) 2021-07-11T04:01:08.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.435+0000 7ff231b3c700 1 -- 172.21.2.104:0/91549073 learned_addr learned my addr 172.21.2.104:0/91549073 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:08.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.435+0000 7ff23233d700 1 --2- 172.21.2.104:0/91549073 >> v2:172.21.2.104:3300/0 conn(0x7ff22c0b1970 0x7ff22c05fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff23233d700 1 -- 172.21.2.104:0/91549073 >> v2:172.21.2.105:3300/0 conn(0x7ff22c067180 msgr2=0x7ff22c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:08.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff23233d700 1 --2- 172.21.2.104:0/91549073 >> v2:172.21.2.105:3300/0 conn(0x7ff22c067180 0x7ff22c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff23233d700 1 -- 172.21.2.104:0/91549073 >> v1:172.21.2.104:6789/0 conn(0x7ff22c0a76b0 legacy=0x7ff22c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff23233d700 1 -- 172.21.2.104:0/91549073 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c147b60 con 0x7ff22c0b1970 2021-07-11T04:01:08.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff23233d700 1 --2- 172.21.2.104:0/91549073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff22c0b1970 0x7ff22c05fba0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7ff220001110 tx=0x7ff2240091d0).ready entity=mon.0 client_cookie=23759d456f7a6c9d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff230b3a700 1 -- 172.21.2.104:0/91549073 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff224009ea0 con 0x7ff22c0b1970 2021-07-11T04:01:08.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff230b3a700 1 -- 172.21.2.104:0/91549073 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22400b730 con 0x7ff22c0b1970 2021-07-11T04:01:08.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff230b3a700 1 -- 172.21.2.104:0/91549073 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff22400b8a0 con 0x7ff22c0b1970 2021-07-11T04:01:08.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- 172.21.2.104:0/91549073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff22c0b1970 msgr2=0x7ff22c05fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 --2- 172.21.2.104:0/91549073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff22c0b1970 0x7ff22c05fba0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7ff220001110 tx=0x7ff2240091d0).stop 2021-07-11T04:01:08.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- 172.21.2.104:0/91549073 shutdown_connections 2021-07-11T04:01:08.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 --2- 172.21.2.104:0/91549073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff22c0b1970 0x7ff22c05fba0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 --2- 172.21.2.104:0/91549073 >> v2:172.21.2.105:3300/0 conn(0x7ff22c067180 0x7ff22c05f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- 172.21.2.104:0/91549073 >> 172.21.2.104:0/91549073 conn(0x7ff22c0b53e0 msgr2=0x7ff22c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:08.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- 172.21.2.104:0/91549073 shutdown_connections 2021-07-11T04:01:08.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- 172.21.2.104:0/91549073 wait complete. 2021-07-11T04:01:08.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 Processor -- start 2021-07-11T04:01:08.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- start start 2021-07-11T04:01:08.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff22c067180 0x7ff22c14e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff22c0a76b0 0x7ff22c15a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff22c0b1970 0x7ff22c15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c14d350 con 0x7ff22c067180 2021-07-11T04:01:08.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c0aef30 con 0x7ff22c0a76b0 2021-07-11T04:01:08.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22c14d580 con 0x7ff22c0b1970 2021-07-11T04:01:08.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff231b3c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff22c067180 0x7ff22c14e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff231b3c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff22c067180 0x7ff22c14e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49158/0 (socket says 172.21.2.104:49158) 2021-07-11T04:01:08.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff231b3c700 1 -- 172.21.2.104:0/2721269135 learned_addr learned my addr 172.21.2.104:0/2721269135 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:08.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff231b3c700 1 -- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff22c0b1970 msgr2=0x7ff22c15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff231b3c700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff22c0b1970 0x7ff22c15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff231b3c700 1 -- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff22c0a76b0 msgr2=0x7ff22c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff231b3c700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff22c0a76b0 0x7ff22c15a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff231b3c700 1 -- 172.21.2.104:0/2721269135 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff224009470 con 0x7ff22c067180 2021-07-11T04:01:08.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff231b3c700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff22c067180 0x7ff22c14e970 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7ff2200011d0 tx=0x7ff220007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff21effd700 1 -- 172.21.2.104:0/2721269135 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2200072e0 con 0x7ff22c067180 2021-07-11T04:01:08.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff21effd700 1 -- 172.21.2.104:0/2721269135 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff220007440 con 0x7ff22c067180 2021-07-11T04:01:08.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- 172.21.2.104:0/2721269135 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22c14d820 con 0x7ff22c067180 2021-07-11T04:01:08.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff21effd700 1 -- 172.21.2.104:0/2721269135 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff220005800 con 0x7ff22c067180 2021-07-11T04:01:08.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- 172.21.2.104:0/2721269135 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22c14cf90 con 0x7ff22c067180 2021-07-11T04:01:08.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff233da0700 1 -- 172.21.2.104:0/2721269135 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff22c14d9b0 con 0x7ff22c067180 2021-07-11T04:01:08.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.439+0000 7ff21effd700 1 -- 172.21.2.104:0/2721269135 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff220010070 con 0x7ff22c067180 2021-07-11T04:01:08.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.443+0000 7ff21effd700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff20c053300 0x7ff20c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.443+0000 7ff21effd700 1 -- 172.21.2.104:0/2721269135 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff22006aa00 con 0x7ff22c067180 2021-07-11T04:01:08.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.443+0000 7ff23133b700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff20c053300 0x7ff20c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.443+0000 7ff23133b700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff20c053300 0x7ff20c0557c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7ff228002f70 tx=0x7ff228006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.443+0000 7ff21effd700 1 -- 172.21.2.104:0/2721269135 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff220042020 con 0x7ff22c067180 2021-07-11T04:01:08.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.551+0000 7ff233da0700 1 -- 172.21.2.104:0/2721269135 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff22c057350 con 0x7ff22c067180 2021-07-11T04:01:08.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.551+0000 7ff21effd700 1 -- 172.21.2.104:0/2721269135 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+1556 (secure 0 0 0) 0x7ff220018030 con 0x7ff22c067180 2021-07-11T04:01:08.554 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:08.554 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/3293345379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3293345379},{"type":"v1","addr":"172.21.2.105:6835","nonce":3293345379}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/57099992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":57099992},{"type":"v1","addr":"172.21.2.104:6833","nonce":57099992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6835/2809359847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2809359847},{"type":"v1","addr":"172.21.2.104:6835","nonce":2809359847}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":53}],"filesystems":[]} 2021-07-11T04:01:08.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.551+0000 7ff233da0700 1 -- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff20c053300 msgr2=0x7ff20c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.551+0000 7ff233da0700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff20c053300 0x7ff20c0557c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7ff228002f70 tx=0x7ff228006a80).stop 2021-07-11T04:01:08.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.551+0000 7ff233da0700 1 -- 172.21.2.104:0/2721269135 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff22c067180 msgr2=0x7ff22c14e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.551+0000 7ff233da0700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff22c067180 0x7ff22c14e970 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7ff2200011d0 tx=0x7ff220007be0).stop 2021-07-11T04:01:08.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.555+0000 7ff233da0700 1 -- 172.21.2.104:0/2721269135 shutdown_connections 2021-07-11T04:01:08.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.555+0000 7ff233da0700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff22c0b1970 0x7ff22c15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.555+0000 7ff233da0700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff22c0a76b0 0x7ff22c15a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.555+0000 7ff233da0700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff20c053300 0x7ff20c0557c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.555+0000 7ff233da0700 1 --2- 172.21.2.104:0/2721269135 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff22c067180 0x7ff22c14e970 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.555+0000 7ff233da0700 1 -- 172.21.2.104:0/2721269135 >> 172.21.2.104:0/2721269135 conn(0x7ff22c0b53e0 msgr2=0x7ff22c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:08.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.555+0000 7ff233da0700 1 -- 172.21.2.104:0/2721269135 shutdown_connections 2021-07-11T04:01:08.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.555+0000 7ff233da0700 1 -- 172.21.2.104:0/2721269135 wait complete. 2021-07-11T04:01:08.558 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 53 2021-07-11T04:01:08.561 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:08.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 Processor -- start 2021-07-11T04:01:08.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 -- start start 2021-07-11T04:01:08.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43e80a76b0 0x7f43e8066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f43e80b1970 0x7f43e8066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f43e80b0060 0x7f43e805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e8067450 con 0x7f43e80b1970 2021-07-11T04:01:08.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e80a7a90 con 0x7f43e80b0060 2021-07-11T04:01:08.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e8147620 con 0x7f43e80a76b0 2021-07-11T04:01:08.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e77fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f43e80b1970 0x7f43e8066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e77fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f43e80b1970 0x7f43e8066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49170/0 (socket says 172.21.2.104:49170) 2021-07-11T04:01:08.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e77fe700 1 -- 172.21.2.104:0/411401795 learned_addr learned my addr 172.21.2.104:0/411401795 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:08.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e77fe700 1 -- 172.21.2.104:0/411401795 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43e80a76b0 msgr2=0x7f43e8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:08.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e77fe700 1 --2- 172.21.2.104:0/411401795 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43e80a76b0 0x7f43e8066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e77fe700 1 -- 172.21.2.104:0/411401795 >> v2:172.21.2.105:3300/0 conn(0x7f43e80b0060 msgr2=0x7f43e805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:08.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e77fe700 1 --2- 172.21.2.104:0/411401795 >> v2:172.21.2.105:3300/0 conn(0x7f43e80b0060 0x7f43e805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e77fe700 1 -- 172.21.2.104:0/411401795 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e81478c0 con 0x7f43e80b1970 2021-07-11T04:01:08.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e77fe700 1 --2- 172.21.2.104:0/411401795 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43e80b1970 0x7f43e8066f60 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f43d8002f70 tx=0x7f43d8006eb0).ready entity=mon.0 client_cookie=9a11abe09411f9f9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e6ffd700 1 -- 172.21.2.104:0/411401795 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43d800d570 con 0x7f43e80b1970 2021-07-11T04:01:08.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e6ffd700 1 -- 172.21.2.104:0/411401795 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d8000e90 con 0x7f43e80b1970 2021-07-11T04:01:08.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43e6ffd700 1 -- 172.21.2.104:0/411401795 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43d800d900 con 0x7f43e80b1970 2021-07-11T04:01:08.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 -- 172.21.2.104:0/411401795 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43e80b1970 msgr2=0x7f43e8066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 --2- 172.21.2.104:0/411401795 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43e80b1970 0x7f43e8066f60 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f43d8002f70 tx=0x7f43d8006eb0).stop 2021-07-11T04:01:08.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 -- 172.21.2.104:0/411401795 shutdown_connections 2021-07-11T04:01:08.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 --2- 172.21.2.104:0/411401795 >> v2:172.21.2.105:3300/0 conn(0x7f43e80b0060 0x7f43e805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 --2- 172.21.2.104:0/411401795 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43e80b1970 0x7f43e8066f60 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 --2- 172.21.2.104:0/411401795 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43e80a76b0 0x7f43e8066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 -- 172.21.2.104:0/411401795 >> 172.21.2.104:0/411401795 conn(0x7f43e80b53e0 msgr2=0x7f43e80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:08.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 -- 172.21.2.104:0/411401795 shutdown_connections 2021-07-11T04:01:08.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 -- 172.21.2.104:0/411401795 wait complete. 2021-07-11T04:01:08.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 Processor -- start 2021-07-11T04:01:08.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.611+0000 7f43ee63e700 1 -- start start 2021-07-11T04:01:08.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43ee63e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43e80a76b0 0x7f43e80aa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43ee63e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43e80b0060 0x7f43e814d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43ee63e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43e80b1970 0x7f43e80ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43ee63e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80adcb0 con 0x7f43e80a76b0 2021-07-11T04:01:08.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43ee63e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80ab4a0 con 0x7f43e80b0060 2021-07-11T04:01:08.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43ee63e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80aec90 con 0x7f43e80b1970 2021-07-11T04:01:08.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43e80a76b0 0x7f43e80aa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43e80a76b0 0x7f43e80aa7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49172/0 (socket says 172.21.2.104:49172) 2021-07-11T04:01:08.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e7fff700 1 -- 172.21.2.104:0/1271211863 learned_addr learned my addr 172.21.2.104:0/1271211863 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:08.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e7fff700 1 -- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43e80b1970 msgr2=0x7f43e80ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e7fff700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43e80b1970 0x7f43e80ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e7fff700 1 -- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43e80b0060 msgr2=0x7f43e814d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e7fff700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43e80b0060 0x7f43e814d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e7fff700 1 -- 172.21.2.104:0/1271211863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d800b1e0 con 0x7f43e80a76b0 2021-07-11T04:01:08.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e7fff700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43e80a76b0 0x7f43e80aa7a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f43dc002f70 tx=0x7f43dc006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e57fa700 1 -- 172.21.2.104:0/1271211863 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43dc013460 con 0x7f43e80a76b0 2021-07-11T04:01:08.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e57fa700 1 -- 172.21.2.104:0/1271211863 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc0135c0 con 0x7f43e80a76b0 2021-07-11T04:01:08.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43ee63e700 1 -- 172.21.2.104:0/1271211863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e80adee0 con 0x7f43e80a76b0 2021-07-11T04:01:08.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43ee63e700 1 -- 172.21.2.104:0/1271211863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e814e540 con 0x7f43e80a76b0 2021-07-11T04:01:08.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e57fa700 1 -- 172.21.2.104:0/1271211863 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43dc013920 con 0x7f43e80a76b0 2021-07-11T04:01:08.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43ee63e700 1 -- 172.21.2.104:0/1271211863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e80ad900 con 0x7f43e80a76b0 2021-07-11T04:01:08.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e57fa700 1 -- 172.21.2.104:0/1271211863 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f43dc01a060 con 0x7f43e80a76b0 2021-07-11T04:01:08.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e57fa700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43c8053160 0x7f43c8055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e57fa700 1 -- 172.21.2.104:0/1271211863 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f43dc06ada0 con 0x7f43e80a76b0 2021-07-11T04:01:08.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e77fe700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43c8053160 0x7f43c8055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.615+0000 7f43e77fe700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43c8053160 0x7f43c8055620 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f43d8002f70 tx=0x7f43d8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.619+0000 7f43e57fa700 1 -- 172.21.2.104:0/1271211863 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f43dc0422b0 con 0x7f43e80a76b0 2021-07-11T04:01:08.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.723+0000 7f43ee63e700 1 -- 172.21.2.104:0/1271211863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43e8057350 con 0x7f43e80a76b0 2021-07-11T04:01:08.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43e57fa700 1 -- 172.21.2.104:0/1271211863 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+1556 (secure 0 0 0) 0x7f43dc0458d0 con 0x7f43e80a76b0 2021-07-11T04:01:08.727 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:08.727 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":53,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/3293345379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3293345379},{"type":"v1","addr":"172.21.2.105:6835","nonce":3293345379}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":49},{"gid":7284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/57099992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":57099992},{"type":"v1","addr":"172.21.2.104:6833","nonce":57099992}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":48},{"gid":7530,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6835/2809359847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2809359847},{"type":"v1","addr":"172.21.2.104:6835","nonce":2809359847}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":53}],"filesystems":[]} 2021-07-11T04:01:08.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 -- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43c8053160 msgr2=0x7f43c8055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43c8053160 0x7f43c8055620 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f43d8002f70 tx=0x7f43d8015000).stop 2021-07-11T04:01:08.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 -- 172.21.2.104:0/1271211863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43e80a76b0 msgr2=0x7f43e80aa7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43e80a76b0 0x7f43e80aa7a0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f43dc002f70 tx=0x7f43dc006ec0).stop 2021-07-11T04:01:08.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 -- 172.21.2.104:0/1271211863 shutdown_connections 2021-07-11T04:01:08.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f43e80b1970 0x7f43e80ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43e80b0060 0x7f43e814d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:08.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43c8053160 0x7f43c8055620 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 --2- 172.21.2.104:0/1271211863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43e80a76b0 0x7f43e80aa7a0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 -- 172.21.2.104:0/1271211863 >> 172.21.2.104:0/1271211863 conn(0x7f43e80b53e0 msgr2=0x7f43e805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:08.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 -- 172.21.2.104:0/1271211863 shutdown_connections 2021-07-11T04:01:08.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.727+0000 7f43ee63e700 1 -- 172.21.2.104:0/1271211863 wait complete. 2021-07-11T04:01:08.731 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 53 2021-07-11T04:01:08.734 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T04:01:08.735 INFO:tasks.ceph.mds.a:Stopping old one... 2021-07-11T04:01:08.735 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T04:01:08.735 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:01:08.735 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T04:01:08.736 INFO:tasks.ceph.mds.c:Stopping old one... 2021-07-11T04:01:08.736 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T04:01:08.736 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:01:08.737 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T04:01:08.737 INFO:tasks.ceph.mds.b:Stopping old one... 2021-07-11T04:01:08.737 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T04:01:08.738 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:01:08.740 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T04:01:08.741 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T04:01:08.742 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T04:01:08.743 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T04:01:08.744 INFO:tasks.ceph.mds.a:Started 2021-07-11T04:01:08.744 DEBUG:teuthology.parallel:result is None 2021-07-11T04:01:08.745 INFO:tasks.ceph.mds.c:Started 2021-07-11T04:01:08.745 DEBUG:teuthology.parallel:result is None 2021-07-11T04:01:08.748 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T04:01:08.749 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T04:01:08.751 INFO:tasks.ceph.mds.b:Started 2021-07-11T04:01:08.751 DEBUG:teuthology.parallel:result is None 2021-07-11T04:01:08.752 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T04:01:08.777 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T04:01:08.777 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T04:01:08.785 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T04:01:08.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.803+0000 7ff27ad81700 1 Processor -- start 2021-07-11T04:01:08.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.803+0000 7ff27ad81700 1 -- start start 2021-07-11T04:01:08.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff274061680 0x7ff2740b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff2740b4710 0x7ff2740ba200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2740b6910 con 0x7ff2741439a0 2021-07-11T04:01:08.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2740b9680 con 0x7ff274061680 2021-07-11T04:01:08.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2740b6bb0 con 0x7ff2740b4710 2021-07-11T04:01:08.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff279d7f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff274061680 0x7ff2740b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff279d7f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff274061680 0x7ff2740b9d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49204/0 (socket says 172.21.2.104:49204) 2021-07-11T04:01:08.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff279d7f700 1 -- 172.21.2.104:0/2828987591 learned_addr learned my addr 172.21.2.104:0/2828987591 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:08.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 --2- 172.21.2.104:0/2828987591 >> v2:172.21.2.105:3300/0 conn(0x7ff2740b4710 0x7ff2740ba200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 -- 172.21.2.104:0/2828987591 >> v2:172.21.2.104:3300/0 conn(0x7ff274061680 msgr2=0x7ff2740b9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:08.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 --2- 172.21.2.104:0/2828987591 >> v2:172.21.2.104:3300/0 conn(0x7ff274061680 0x7ff2740b9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 -- 172.21.2.104:0/2828987591 >> v1:172.21.2.105:6789/0 conn(0x7ff2741439a0 legacy=0x7ff27413fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 -- 172.21.2.104:0/2828987591 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2740b6e50 con 0x7ff2740b4710 2021-07-11T04:01:08.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff279d7f700 1 --2- 172.21.2.104:0/2828987591 >> v2:172.21.2.104:3300/0 conn(0x7ff274061680 0x7ff2740b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:01:08.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 --2- 172.21.2.104:0/2828987591 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff2740b4710 0x7ff2740ba200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7ff264002f70 tx=0x7ff264006df0).ready entity=mon.1 client_cookie=3a6579d24eacad8c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff278d7d700 1 -- 172.21.2.104:0/2828987591 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26400d570 con 0x7ff2740b4710 2021-07-11T04:01:08.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff278d7d700 1 -- 172.21.2.104:0/2828987591 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff264000e90 con 0x7ff2740b4710 2021-07-11T04:01:08.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff278d7d700 1 -- 172.21.2.104:0/2828987591 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26400d8c0 con 0x7ff2740b4710 2021-07-11T04:01:08.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- 172.21.2.104:0/2828987591 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff2740b4710 msgr2=0x7ff2740ba200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 --2- 172.21.2.104:0/2828987591 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff2740b4710 0x7ff2740ba200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7ff264002f70 tx=0x7ff264006df0).stop 2021-07-11T04:01:08.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- 172.21.2.104:0/2828987591 shutdown_connections 2021-07-11T04:01:08.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 --2- 172.21.2.104:0/2828987591 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff2740b4710 0x7ff2740ba200 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 --2- 172.21.2.104:0/2828987591 >> v2:172.21.2.104:3300/0 conn(0x7ff274061680 0x7ff2740b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- 172.21.2.104:0/2828987591 >> 172.21.2.104:0/2828987591 conn(0x7ff27406adf0 msgr2=0x7ff27405f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:08.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- 172.21.2.104:0/2828987591 shutdown_connections 2021-07-11T04:01:08.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- 172.21.2.104:0/2828987591 wait complete. 2021-07-11T04:01:08.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 Processor -- start 2021-07-11T04:01:08.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- start start 2021-07-11T04:01:08.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff274061680 0x7ff27413e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2741439a0 0x7ff2740b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff27413bdb0 0x7ff274138bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff274140820 con 0x7ff274061680 2021-07-11T04:01:08.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740b2b40 con 0x7ff27413bdb0 2021-07-11T04:01:08.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27413ed80 con 0x7ff2741439a0 2021-07-11T04:01:08.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff279d7f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff274061680 0x7ff27413e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff279d7f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff274061680 0x7ff27413e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49210/0 (socket says 172.21.2.104:49210) 2021-07-11T04:01:08.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2741439a0 0x7ff2740b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff279d7f700 1 -- 172.21.2.104:0/294572120 learned_addr learned my addr 172.21.2.104:0/294572120 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:08.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27a580700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff27413bdb0 0x7ff274138bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 -- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff27413bdb0 msgr2=0x7ff274138bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff27413bdb0 0x7ff274138bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 -- 172.21.2.104:0/294572120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff274061680 msgr2=0x7ff27413e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:08.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff274061680 0x7ff27413e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:08.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 -- 172.21.2.104:0/294572120 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27000b1e0 con 0x7ff2741439a0 2021-07-11T04:01:08.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27957e700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2741439a0 0x7ff2740b2260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7ff2740b2e50 tx=0x7ff264014000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff262ffd700 1 -- 172.21.2.104:0/294572120 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26400dea0 con 0x7ff2741439a0 2021-07-11T04:01:08.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff262ffd700 1 -- 172.21.2.104:0/294572120 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff264000e90 con 0x7ff2741439a0 2021-07-11T04:01:08.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- 172.21.2.104:0/294572120 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26400b180 con 0x7ff2741439a0 2021-07-11T04:01:08.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff262ffd700 1 -- 172.21.2.104:0/294572120 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff26401e630 con 0x7ff2741439a0 2021-07-11T04:01:08.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.807+0000 7ff27ad81700 1 -- 172.21.2.104:0/294572120 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27413f2c0 con 0x7ff2741439a0 2021-07-11T04:01:08.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.811+0000 7ff27ad81700 1 -- 172.21.2.104:0/294572120 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27413b270 con 0x7ff2741439a0 2021-07-11T04:01:08.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.811+0000 7ff262ffd700 1 -- 172.21.2.104:0/294572120 <== mon.2 v2:172.21.2.105:3301/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff264024020 con 0x7ff2741439a0 2021-07-11T04:01:08.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.811+0000 7ff262ffd700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2540531f0 0x7ff2540556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:08.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.811+0000 7ff262ffd700 1 -- 172.21.2.104:0/294572120 <== mon.2 v2:172.21.2.105:3301/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff26406c0f0 con 0x7ff2741439a0 2021-07-11T04:01:08.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.811+0000 7ff279d7f700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2540531f0 0x7ff2540556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:08.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.811+0000 7ff279d7f700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2540531f0 0x7ff2540556b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7ff27000b000 tx=0x7ff270006eb0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:08.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.811+0000 7ff262ffd700 1 -- 172.21.2.104:0/294572120 <== mon.2 v2:172.21.2.105:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff264043710 con 0x7ff2741439a0 2021-07-11T04:01:08.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:08.903+0000 7ff27ad81700 1 -- 172.21.2.104:0/294572120 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7ff27413b270 con 0x7ff2741439a0 2021-07-11T04:01:09.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.827+0000 7ff262ffd700 1 -- 172.21.2.104:0/294572120 <== mon.2 v2:172.21.2.105:3301/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v115) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff264046d30 con 0x7ff2741439a0 2021-07-11T04:01:09.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.831+0000 7ff27ad81700 1 -- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2540531f0 msgr2=0x7ff2540556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:09.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.831+0000 7ff27ad81700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2540531f0 0x7ff2540556b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7ff27000b000 tx=0x7ff270006eb0).stop 2021-07-11T04:01:09.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.831+0000 7ff27ad81700 1 -- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2741439a0 msgr2=0x7ff2740b2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:09.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.831+0000 7ff27ad81700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2741439a0 0x7ff2740b2260 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7ff2740b2e50 tx=0x7ff264014000).stop 2021-07-11T04:01:09.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.831+0000 7ff27ad81700 1 -- 172.21.2.104:0/294572120 shutdown_connections 2021-07-11T04:01:09.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.831+0000 7ff27ad81700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff27413bdb0 0x7ff274138bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:09.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.831+0000 7ff27ad81700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2741439a0 0x7ff2740b2260 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:09.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.831+0000 7ff27ad81700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2540531f0 0x7ff2540556b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:09.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.831+0000 7ff27ad81700 1 --2- 172.21.2.104:0/294572120 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff274061680 0x7ff27413e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:09.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.835+0000 7ff27ad81700 1 -- 172.21.2.104:0/294572120 >> 172.21.2.104:0/294572120 conn(0x7ff27406adf0 msgr2=0x7ff27405bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:09.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.835+0000 7ff27ad81700 1 -- 172.21.2.104:0/294572120 shutdown_connections 2021-07-11T04:01:09.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.835+0000 7ff27ad81700 1 -- 172.21.2.104:0/294572120 wait complete. 2021-07-11T04:01:09.838 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T04:01:09.847 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T04:01:09.848 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T04:01:09.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 Processor -- start 2021-07-11T04:01:09.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 -- start start 2021-07-11T04:01:09.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69840b1db0 0x7f69840b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:09.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f69841483a0 0x7f69840b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:09.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f69840a70c0 0x7f6984144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:09.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69840a90c0 con 0x7f69840a70c0 2021-07-11T04:01:09.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69840b0350 con 0x7f69841483a0 2021-07-11T04:01:09.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840b05f0 con 0x7f69840b1db0 2021-07-11T04:01:09.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698974d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f69840a70c0 0x7f6984144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:09.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698974d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f69840a70c0 0x7f6984144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49222/0 (socket says 172.21.2.104:49222) 2021-07-11T04:01:09.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698974d700 1 -- 172.21.2.104:0/1470353794 learned_addr learned my addr 172.21.2.104:0/1470353794 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:09.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698974d700 1 -- 172.21.2.104:0/1470353794 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69840b1db0 msgr2=0x7f69840b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:09.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698974d700 1 --2- 172.21.2.104:0/1470353794 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69840b1db0 0x7f69840b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:09.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698974d700 1 -- 172.21.2.104:0/1470353794 >> v2:172.21.2.105:3300/0 conn(0x7f69841483a0 msgr2=0x7f69840b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:09.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698974d700 1 --2- 172.21.2.104:0/1470353794 >> v2:172.21.2.105:3300/0 conn(0x7f69841483a0 0x7f69840b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:09.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698974d700 1 -- 172.21.2.104:0/1470353794 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840b0890 con 0x7f69840a70c0 2021-07-11T04:01:09.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698974d700 1 --2- 172.21.2.104:0/1470353794 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69840a70c0 0x7f6984144720 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f697c0032f0 tx=0x7f697c009290).ready entity=mon.0 client_cookie=d89dd3a8e0f21889 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:09.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f69837fe700 1 -- 172.21.2.104:0/1470353794 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f697c00b870 con 0x7f69840a70c0 2021-07-11T04:01:09.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f69837fe700 1 -- 172.21.2.104:0/1470353794 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c00b9d0 con 0x7f69840a70c0 2021-07-11T04:01:09.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f69837fe700 1 -- 172.21.2.104:0/1470353794 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f697c00bb40 con 0x7f69840a70c0 2021-07-11T04:01:09.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 -- 172.21.2.104:0/1470353794 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69840a70c0 msgr2=0x7f6984144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:09.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 --2- 172.21.2.104:0/1470353794 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69840a70c0 0x7f6984144720 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f697c0032f0 tx=0x7f697c009290).stop 2021-07-11T04:01:09.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 -- 172.21.2.104:0/1470353794 shutdown_connections 2021-07-11T04:01:09.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 --2- 172.21.2.104:0/1470353794 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69840a70c0 0x7f6984144720 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:09.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 --2- 172.21.2.104:0/1470353794 >> v2:172.21.2.105:3300/0 conn(0x7f69841483a0 0x7f69840b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:09.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 --2- 172.21.2.104:0/1470353794 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69840b1db0 0x7f69840b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:09.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 -- 172.21.2.104:0/1470353794 >> 172.21.2.104:0/1470353794 conn(0x7f698405f6b0 msgr2=0x7f698405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:09.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 -- 172.21.2.104:0/1470353794 shutdown_connections 2021-07-11T04:01:09.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 -- 172.21.2.104:0/1470353794 wait complete. 2021-07-11T04:01:09.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 Processor -- start 2021-07-11T04:01:09.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.963+0000 7f698b1b0700 1 -- start start 2021-07-11T04:01:09.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f698b1b0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69840a70c0 0x7f6984065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:09.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f698b1b0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69840b1db0 0x7f69840ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:09.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f698b1b0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69841483a0 0x7f6984156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:09.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f698b1b0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6984154430 con 0x7f69840a70c0 2021-07-11T04:01:09.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f698b1b0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6984067690 con 0x7f69841483a0 2021-07-11T04:01:09.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f698b1b0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840657c0 con 0x7f69840b1db0 2021-07-11T04:01:09.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6988f4c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69840a70c0 0x7f6984065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:09.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6988f4c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69840a70c0 0x7f6984065bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49226/0 (socket says 172.21.2.104:49226) 2021-07-11T04:01:09.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6988f4c700 1 -- 172.21.2.104:0/3698885069 learned_addr learned my addr 172.21.2.104:0/3698885069 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:09.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6988f4c700 1 -- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69840b1db0 msgr2=0x7f69840ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:09.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6988f4c700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69840b1db0 0x7f69840ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:09.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6988f4c700 1 -- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69841483a0 msgr2=0x7f6984156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:09.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6988f4c700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69841483a0 0x7f6984156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:09.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6988f4c700 1 -- 172.21.2.104:0/3698885069 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c009530 con 0x7f69840a70c0 2021-07-11T04:01:09.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6988f4c700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69840a70c0 0x7f6984065bf0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f69840ac290 tx=0x7f696c006ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:09.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6981ffb700 1 -- 172.21.2.104:0/3698885069 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f696c00f510 con 0x7f69840a70c0 2021-07-11T04:01:09.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6981ffb700 1 -- 172.21.2.104:0/3698885069 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f696c00f670 con 0x7f69840a70c0 2021-07-11T04:01:09.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f698b1b0700 1 -- 172.21.2.104:0/3698885069 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69840642b0 con 0x7f69840a70c0 2021-07-11T04:01:09.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6981ffb700 1 -- 172.21.2.104:0/3698885069 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f696c00f9d0 con 0x7f69840a70c0 2021-07-11T04:01:09.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f698b1b0700 1 -- 172.21.2.104:0/3698885069 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6984067310 con 0x7f69840a70c0 2021-07-11T04:01:09.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f698b1b0700 1 -- 172.21.2.104:0/3698885069 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69840659c0 con 0x7f69840a70c0 2021-07-11T04:01:09.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6981ffb700 1 -- 172.21.2.104:0/3698885069 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f696c018030 con 0x7f69840a70c0 2021-07-11T04:01:09.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6981ffb700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6968053380 0x7f6968055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:09.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6981ffb700 1 -- 172.21.2.104:0/3698885069 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f696c06a710 con 0x7f69840a70c0 2021-07-11T04:01:09.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6983fff700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6968053380 0x7f6968055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:09.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.967+0000 7f6983fff700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6968053380 0x7f6968055840 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f6974002f70 tx=0x7f6974006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:09.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:09.971+0000 7f6981ffb700 1 -- 172.21.2.104:0/3698885069 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f696c041e30 con 0x7f69840a70c0 2021-07-11T04:01:10.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 -- 172.21.2.104:0/3698885069 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6984057350 con 0x7f69840a70c0 2021-07-11T04:01:10.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f6981ffb700 1 -- 172.21.2.104:0/3698885069 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v29) v1 ==== 80+0+4255 (secure 0 0 0) 0x7f696c0210b0 con 0x7f69840a70c0 2021-07-11T04:01:10.084 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:10.084 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:01:10.084 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T04:01:10.084 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.085 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T04:01:10.085 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T04:01:10.085 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.085 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:01:10.085 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.086 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.086 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.086 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.086 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.086 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.087 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T04:01:10.087 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T04:01:10.087 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.087 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:01:10.088 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.088 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.088 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.088 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.089 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.089 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.089 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T04:01:10.089 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T04:01:10.089 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.090 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:01:10.090 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.090 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.090 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.091 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.091 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.091 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.091 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T04:01:10.091 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T04:01:10.092 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.092 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.092 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.092 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.092 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.093 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.093 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.093 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T04:01:10.093 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T04:01:10.094 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.094 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.094 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.094 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.095 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.095 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.095 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.095 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T04:01:10.095 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T04:01:10.096 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.096 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.096 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.096 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.097 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.097 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.097 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.097 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T04:01:10.098 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T04:01:10.098 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.098 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.098 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.098 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.099 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.099 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.099 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.099 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T04:01:10.100 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T04:01:10.100 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.100 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.100 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.101 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.101 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.101 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.101 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.101 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T04:01:10.102 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T04:01:10.102 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.102 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.102 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.103 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.103 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.103 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.103 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.104 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T04:01:10.104 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T04:01:10.104 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.104 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.104 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.105 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.105 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.105 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.105 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.106 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T04:01:10.106 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T04:01:10.106 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.106 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.106 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.107 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.107 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.107 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.107 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.107 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T04:01:10.108 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T04:01:10.108 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.108 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:01:10.108 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:01:10.109 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:01:10.109 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.109 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.109 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.109 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T04:01:10.110 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T04:01:10.110 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.110 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:01:10.110 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:01:10.111 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:01:10.111 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.111 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.111 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.112 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T04:01:10.112 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T04:01:10.112 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.112 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:01:10.112 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:10.113 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:10.113 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.113 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.113 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.113 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.114 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T04:01:10.114 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T04:01:10.114 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.114 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:01:10.115 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:01:10.115 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.115 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.115 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.115 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.116 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T04:01:10.116 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T04:01:10.116 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:10.116 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:01:10.117 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:01:10.117 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:10.117 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.117 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.118 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:10.118 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:01:10.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 -- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6968053380 msgr2=0x7f6968055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6968053380 0x7f6968055840 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f6974002f70 tx=0x7f6974006a10).stop 2021-07-11T04:01:10.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 -- 172.21.2.104:0/3698885069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69840a70c0 msgr2=0x7f6984065bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69840a70c0 0x7f6984065bf0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f69840ac290 tx=0x7f696c006ed0).stop 2021-07-11T04:01:10.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 -- 172.21.2.104:0/3698885069 shutdown_connections 2021-07-11T04:01:10.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f69841483a0 0x7f6984156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69840b1db0 0x7f69840ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6968053380 0x7f6968055840 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 --2- 172.21.2.104:0/3698885069 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69840a70c0 0x7f6984065bf0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 -- 172.21.2.104:0/3698885069 >> 172.21.2.104:0/3698885069 conn(0x7f698405f6b0 msgr2=0x7f69840aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:10.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 -- 172.21.2.104:0/3698885069 shutdown_connections 2021-07-11T04:01:10.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.083+0000 7f698b1b0700 1 -- 172.21.2.104:0/3698885069 wait complete. 2021-07-11T04:01:10.123 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:10.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.175+0000 7fb6aed4b700 1 Processor -- start 2021-07-11T04:01:10.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.175+0000 7fb6aed4b700 1 -- start start 2021-07-11T04:01:10.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.175+0000 7fb6aed4b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb6a80af960 0x7fb6a80b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.175+0000 7fb6aed4b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a80ad5a0 con 0x7fb6a80afd40 2021-07-11T04:01:10.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.175+0000 7fb6aed4b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a80ad060 con 0x7fb6a80b04f0 2021-07-11T04:01:10.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.175+0000 7fb6aed4b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a80af460 con 0x7fb6a80af960 2021-07-11T04:01:10.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6ad2e8700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb6a80afd40 0x7fb6a813fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46002/0 (socket says 172.21.2.104:46002) 2021-07-11T04:01:10.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6ad2e8700 1 -- 172.21.2.104:0/2391695849 learned_addr learned my addr 172.21.2.104:0/2391695849 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:10.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190018197 0 0) 0x7fb6a80ad5a0 con 0x7fb6a80afd40 2021-07-11T04:01:10.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb68c003260 con 0x7fb6a80afd40 2021-07-11T04:01:10.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6acae7700 1 --2- 172.21.2.104:0/2391695849 >> v2:172.21.2.105:3300/0 conn(0x7fb6a80af960 0x7fb6a80b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1416275703 0 0) 0x7fb68c003260 con 0x7fb6a80afd40 2021-07-11T04:01:10.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 >> v2:172.21.2.105:3300/0 conn(0x7fb6a80af960 msgr2=0x7fb6a80b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:10.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 --2- 172.21.2.104:0/2391695849 >> v2:172.21.2.105:3300/0 conn(0x7fb6a80af960 0x7fb6a80b1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 >> v1:172.21.2.105:6789/0 conn(0x7fb6a80b04f0 legacy=0x7fb6a80b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:10.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80ad300 con 0x7fb6a80afd40 2021-07-11T04:01:10.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fb6a00019e0 con 0x7fb6a80afd40 2021-07-11T04:01:10.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 >> v1:172.21.2.104:6789/0 conn(0x7fb6a80afd40 legacy=0x7fb6a813fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 --2- 172.21.2.104:0/2391695849 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb68c003820 0x7fb68c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 --2- 172.21.2.104:0/2391695849 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb68c004500 0x7fb68c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 --2- 172.21.2.104:0/2391695849 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb68c004f70 0x7fb68c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a80af460 con 0x7fb68c003820 2021-07-11T04:01:10.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c00be50 con 0x7fb68c004500 2021-07-11T04:01:10.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68c00c010 con 0x7fb68c004f70 2021-07-11T04:01:10.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6acae7700 1 --2- 172.21.2.104:0/2391695849 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb68c003820 0x7fb68c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a7fff700 1 --2- 172.21.2.104:0/2391695849 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb68c004500 0x7fb68c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6acae7700 1 -- 172.21.2.104:0/2391695849 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb68c004f70 msgr2=0x7fb68c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:10.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6acae7700 1 --2- 172.21.2.104:0/2391695849 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb68c004f70 0x7fb68c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6acae7700 1 -- 172.21.2.104:0/2391695849 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb68c004500 msgr2=0x7fb68c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:10.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6acae7700 1 --2- 172.21.2.104:0/2391695849 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb68c004500 0x7fb68c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6acae7700 1 -- 172.21.2.104:0/2391695849 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c003260 con 0x7fb68c003820 2021-07-11T04:01:10.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a7fff700 1 --2- 172.21.2.104:0/2391695849 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb68c004500 0x7fb68c004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:10.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6ad2e8700 1 -- 172.21.2.104:0/2391695849 reap_dead start 2021-07-11T04:01:10.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6acae7700 1 --2- 172.21.2.104:0/2391695849 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb68c003820 0x7fb68c003c20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7fb69c00a000 tx=0x7fb69c008040).ready entity=mon.0 client_cookie=f6dcf5566d6dfddc server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb69c00c070 con 0x7fb68c003820 2021-07-11T04:01:10.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb69c0133f0 con 0x7fb68c003820 2021-07-11T04:01:10.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6a77fe700 1 -- 172.21.2.104:0/2391695849 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb69c013750 con 0x7fb68c003820 2021-07-11T04:01:10.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2391695849 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb68c003820 msgr2=0x7fb68c003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 --2- 172.21.2.104:0/2391695849 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb68c003820 0x7fb68c003c20 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7fb69c00a000 tx=0x7fb69c008040).stop 2021-07-11T04:01:10.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2391695849 shutdown_connections 2021-07-11T04:01:10.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 --2- 172.21.2.104:0/2391695849 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb68c003820 0x7fb68c003c20 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2391695849 >> 172.21.2.104:0/2391695849 conn(0x7fb6a805cd20 msgr2=0x7fb6a8062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:10.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2391695849 shutdown_connections 2021-07-11T04:01:10.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2391695849 wait complete. 2021-07-11T04:01:10.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 Processor -- start 2021-07-11T04:01:10.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 -- start start 2021-07-11T04:01:10.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb68c003820 0x7fb6a80668a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb6a80a0640 0x7fb6a80a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb6a8155900 0x7fb6a8155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a80a1150 con 0x7fb6a8155900 2021-07-11T04:01:10.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a8156280 con 0x7fb6a80a0640 2021-07-11T04:01:10.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6aed4b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a80a1380 con 0x7fb68c003820 2021-07-11T04:01:10.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6ad2e8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb6a8155900 0x7fb6a8155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6ad2e8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb6a8155900 0x7fb6a8155d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49250/0 (socket says 172.21.2.104:49250) 2021-07-11T04:01:10.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.179+0000 7fb6ad2e8700 1 -- 172.21.2.104:0/2811813241 learned_addr learned my addr 172.21.2.104:0/2811813241 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:10.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6ad2e8700 1 -- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb68c003820 msgr2=0x7fb6a80668a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6ad2e8700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb68c003820 0x7fb6a80668a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6ad2e8700 1 -- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb6a80a0640 msgr2=0x7fb6a80a1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6ad2e8700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb6a80a0640 0x7fb6a80a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6ad2e8700 1 -- 172.21.2.104:0/2811813241 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a80a1620 con 0x7fb6a8155900 2021-07-11T04:01:10.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6ad2e8700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb6a8155900 0x7fb6a8155d90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7fb6a0001490 tx=0x7fb6a80ad1f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6a5ffb700 1 -- 172.21.2.104:0/2811813241 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6a0004ab0 con 0x7fb6a8155900 2021-07-11T04:01:10.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6a5ffb700 1 -- 172.21.2.104:0/2811813241 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a0004c10 con 0x7fb6a8155900 2021-07-11T04:01:10.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2811813241 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a80a18c0 con 0x7fb6a8155900 2021-07-11T04:01:10.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6a5ffb700 1 -- 172.21.2.104:0/2811813241 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb6a00070d0 con 0x7fb6a8155900 2021-07-11T04:01:10.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2811813241 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a80a2250 con 0x7fb6a8155900 2021-07-11T04:01:10.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2811813241 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a80b59d0 con 0x7fb6a8155900 2021-07-11T04:01:10.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6a5ffb700 1 -- 172.21.2.104:0/2811813241 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb6a001b070 con 0x7fb6a8155900 2021-07-11T04:01:10.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6a5ffb700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6900531a0 0x7fb690055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6a5ffb700 1 -- 172.21.2.104:0/2811813241 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb6a006a830 con 0x7fb6a8155900 2021-07-11T04:01:10.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6acae7700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6900531a0 0x7fb690055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.183+0000 7fb6acae7700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6900531a0 0x7fb690055660 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fb69c00a000 tx=0x7fb69c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.187+0000 7fb6a5ffb700 1 -- 172.21.2.104:0/2811813241 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb6a0041d90 con 0x7fb6a8155900 2021-07-11T04:01:10.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.267+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2811813241 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6a80a1a50 con 0x7fb6a8155900 2021-07-11T04:01:10.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.267+0000 7fb6a5ffb700 1 -- 172.21.2.104:0/2811813241 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+13191 (secure 0 0 0) 0x7fb6a00453b0 con 0x7fb6a8155900 2021-07-11T04:01:10.270 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:10.271 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":115,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:09.820344+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:10.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6900531a0 msgr2=0x7fb690055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6900531a0 0x7fb690055660 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7fb69c00a000 tx=0x7fb69c008040).stop 2021-07-11T04:01:10.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2811813241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb6a8155900 msgr2=0x7fb6a8155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb6a8155900 0x7fb6a8155d90 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7fb6a0001490 tx=0x7fb6a80ad1f0).stop 2021-07-11T04:01:10.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2811813241 shutdown_connections 2021-07-11T04:01:10.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb6900531a0 0x7fb690055660 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb6a8155900 0x7fb6a8155d90 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb6a80a0640 0x7fb6a80a1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 --2- 172.21.2.104:0/2811813241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb68c003820 0x7fb6a80668a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2811813241 >> 172.21.2.104:0/2811813241 conn(0x7fb6a805cd20 msgr2=0x7fb6a80b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:10.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2811813241 shutdown_connections 2021-07-11T04:01:10.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.271+0000 7fb6aed4b700 1 -- 172.21.2.104:0/2811813241 wait complete. 2021-07-11T04:01:10.278 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T04:01:10.279 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:01:10.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 Processor -- start 2021-07-11T04:01:10.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 -- start start 2021-07-11T04:01:10.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5c400b12d0 0x7f5c400affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5c400b22a0 0x7f5c40141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c400a77d0 con 0x7f5c400b1ec0 2021-07-11T04:01:10.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c400b1740 con 0x7f5c400b22a0 2021-07-11T04:01:10.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c400b0650 con 0x7f5c400b12d0 2021-07-11T04:01:10.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c3effd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f5c400b1ec0 0x7f5c400a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46022/0 (socket says 172.21.2.104:46022) 2021-07-11T04:01:10.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c3effd700 1 -- 172.21.2.104:0/2260879668 learned_addr learned my addr 172.21.2.104:0/2260879668 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:10.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c3f7fe700 1 --2- 172.21.2.104:0/2260879668 >> v2:172.21.2.104:3300/0 conn(0x7f5c400b22a0 0x7f5c40141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c3f7fe700 1 -- 172.21.2.104:0/2260879668 >> v2:172.21.2.105:3300/0 conn(0x7f5c400b12d0 msgr2=0x7f5c400affb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:10.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c3f7fe700 1 --2- 172.21.2.104:0/2260879668 >> v2:172.21.2.105:3300/0 conn(0x7f5c400b12d0 0x7f5c400affb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c3f7fe700 1 -- 172.21.2.104:0/2260879668 >> v1:172.21.2.104:6789/0 conn(0x7f5c400b1ec0 legacy=0x7f5c400a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c3f7fe700 1 -- 172.21.2.104:0/2260879668 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c400b08f0 con 0x7f5c400b22a0 2021-07-11T04:01:10.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c3f7fe700 1 --2- 172.21.2.104:0/2260879668 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c400b22a0 0x7f5c40141250 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7f5c380032f0 tx=0x7f5c380091d0).ready entity=mon.0 client_cookie=34dd4fed87e05cad server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c3dffb700 1 -- 172.21.2.104:0/2260879668 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c38009ea0 con 0x7f5c400b22a0 2021-07-11T04:01:10.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c3dffb700 1 -- 172.21.2.104:0/2260879668 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3800b870 con 0x7f5c400b22a0 2021-07-11T04:01:10.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c3dffb700 1 -- 172.21.2.104:0/2260879668 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c3800ba20 con 0x7f5c400b22a0 2021-07-11T04:01:10.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 -- 172.21.2.104:0/2260879668 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c400b22a0 msgr2=0x7f5c40141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 --2- 172.21.2.104:0/2260879668 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c400b22a0 0x7f5c40141250 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7f5c380032f0 tx=0x7f5c380091d0).stop 2021-07-11T04:01:10.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 -- 172.21.2.104:0/2260879668 shutdown_connections 2021-07-11T04:01:10.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 --2- 172.21.2.104:0/2260879668 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c400b22a0 0x7f5c40141250 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 --2- 172.21.2.104:0/2260879668 >> v2:172.21.2.105:3300/0 conn(0x7f5c400b12d0 0x7f5c400affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 -- 172.21.2.104:0/2260879668 >> 172.21.2.104:0/2260879668 conn(0x7f5c4005f6b0 msgr2=0x7f5c4005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:10.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 -- 172.21.2.104:0/2260879668 shutdown_connections 2021-07-11T04:01:10.340 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.331+0000 7f5c458b1700 1 -- 172.21.2.104:0/2260879668 wait complete. 2021-07-11T04:01:10.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c458b1700 1 Processor -- start 2021-07-11T04:01:10.341 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c458b1700 1 -- start start 2021-07-11T04:01:10.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c458b1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c400b12d0 0x7f5c40140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c458b1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c400b1ec0 0x7f5c400ad510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c458b1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c400b22a0 0x7f5c400ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c458b1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400b4cc0 con 0x7f5c400b1ec0 2021-07-11T04:01:10.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c458b1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400b5310 con 0x7f5c400b12d0 2021-07-11T04:01:10.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c458b1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c400b42e0 con 0x7f5c400b22a0 2021-07-11T04:01:10.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c3e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c400b1ec0 0x7f5c400ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c3e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c400b1ec0 0x7f5c400ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49264/0 (socket says 172.21.2.104:49264) 2021-07-11T04:01:10.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c3e7fc700 1 -- 172.21.2.104:0/1013889557 learned_addr learned my addr 172.21.2.104:0/1013889557 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:10.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c3e7fc700 1 -- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c400b22a0 msgr2=0x7f5c400ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c3e7fc700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c400b22a0 0x7f5c400ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c3e7fc700 1 -- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c400b12d0 msgr2=0x7f5c40140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c3e7fc700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c400b12d0 0x7f5c40140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c3e7fc700 1 -- 172.21.2.104:0/1013889557 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c38009470 con 0x7f5c400b1ec0 2021-07-11T04:01:10.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c3e7fc700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c400b1ec0 0x7f5c400ad510 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f5c34002fb0 tx=0x7f5c34006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c2ffff700 1 -- 172.21.2.104:0/1013889557 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c34013460 con 0x7f5c400b1ec0 2021-07-11T04:01:10.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c2ffff700 1 -- 172.21.2.104:0/1013889557 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c340135c0 con 0x7f5c400b1ec0 2021-07-11T04:01:10.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c458b1700 1 -- 172.21.2.104:0/1013889557 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c400b4580 con 0x7f5c400b1ec0 2021-07-11T04:01:10.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c458b1700 1 -- 172.21.2.104:0/1013889557 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c400b3210 con 0x7f5c400b1ec0 2021-07-11T04:01:10.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c2ffff700 1 -- 172.21.2.104:0/1013889557 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c34013920 con 0x7f5c400b1ec0 2021-07-11T04:01:10.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c458b1700 1 -- 172.21.2.104:0/1013889557 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c20003410 con 0x7f5c400b1ec0 2021-07-11T04:01:10.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c2ffff700 1 -- 172.21.2.104:0/1013889557 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5c3400f070 con 0x7f5c400b1ec0 2021-07-11T04:01:10.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c2ffff700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5c1c053300 0x7f5c1c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c2ffff700 1 -- 172.21.2.104:0/1013889557 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5c3406ab80 con 0x7f5c400b1ec0 2021-07-11T04:01:10.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.335+0000 7f5c3effd700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5c1c053300 0x7f5c1c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.339+0000 7f5c3effd700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5c1c053300 0x7f5c1c0557c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f5c400ad2d0 tx=0x7f5c300077d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.339+0000 7f5c2ffff700 1 -- 172.21.2.104:0/1013889557 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5c34042190 con 0x7f5c400b1ec0 2021-07-11T04:01:10.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 -- 172.21.2.104:0/1013889557 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5c20003c40 con 0x7f5c400b1ec0 2021-07-11T04:01:10.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c2ffff700 1 -- 172.21.2.104:0/1013889557 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v115) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5c340457b0 con 0x7f5c400b1ec0 2021-07-11T04:01:10.424 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:01:10.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 -- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5c1c053300 msgr2=0x7f5c1c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5c1c053300 0x7f5c1c0557c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f5c400ad2d0 tx=0x7f5c300077d0).stop 2021-07-11T04:01:10.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 -- 172.21.2.104:0/1013889557 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c400b1ec0 msgr2=0x7f5c400ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c400b1ec0 0x7f5c400ad510 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f5c34002fb0 tx=0x7f5c34006df0).stop 2021-07-11T04:01:10.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 -- 172.21.2.104:0/1013889557 shutdown_connections 2021-07-11T04:01:10.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c400b22a0 0x7f5c400ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5c1c053300 0x7f5c1c0557c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c400b1ec0 0x7f5c400ad510 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 --2- 172.21.2.104:0/1013889557 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c400b12d0 0x7f5c40140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 -- 172.21.2.104:0/1013889557 >> 172.21.2.104:0/1013889557 conn(0x7f5c4005f6b0 msgr2=0x7f5c40145870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:10.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 -- 172.21.2.104:0/1013889557 shutdown_connections 2021-07-11T04:01:10.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.423+0000 7f5c458b1700 1 -- 172.21.2.104:0/1013889557 wait complete. 2021-07-11T04:01:10.431 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:01:10.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0eb740700 1 Processor -- start 2021-07-11T04:01:10.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0eb740700 1 -- start start 2021-07-11T04:01:10.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0eb740700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0e4144440 0x7ff0e4144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0eb740700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e40604e0 con 0x7ff0e4065420 2021-07-11T04:01:10.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0eb740700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e40b23b0 con 0x7ff0e4060100 2021-07-11T04:01:10.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0eb740700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4065c80 con 0x7ff0e4144440 2021-07-11T04:01:10.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e9cdd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff0e4065420 0x7ff0e414b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46038/0 (socket says 172.21.2.104:46038) 2021-07-11T04:01:10.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e9cdd700 1 -- 172.21.2.104:0/3190558954 learned_addr learned my addr 172.21.2.104:0/3190558954 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:10.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984518117 0 0) 0x7ff0e40604e0 con 0x7ff0e4065420 2021-07-11T04:01:10.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0cc003260 con 0x7ff0e4065420 2021-07-11T04:01:10.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e8cdb700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0e4144440 0x7ff0e4144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1436146224 0 0) 0x7ff0cc003260 con 0x7ff0e4065420 2021-07-11T04:01:10.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0e4144440 msgr2=0x7ff0e4144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:10.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0e4144440 0x7ff0e4144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 >> v1:172.21.2.105:6789/0 conn(0x7ff0e4060100 legacy=0x7ff0e4064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40b2650 con 0x7ff0e4065420 2021-07-11T04:01:10.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7ff0dc0019e0 con 0x7ff0e4065420 2021-07-11T04:01:10.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 >> v1:172.21.2.104:6789/0 conn(0x7ff0e4065420 legacy=0x7ff0e414b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0cc003780 0x7ff0cc003b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0cc004500 0x7ff0cc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0cc004e70 0x7ff0cc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4065c80 con 0x7ff0cc004500 2021-07-11T04:01:10.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc00bc40 con 0x7ff0cc004e70 2021-07-11T04:01:10.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc00bdd0 con 0x7ff0cc003780 2021-07-11T04:01:10.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e8cdb700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0cc004500 0x7ff0cc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e94dc700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0cc003780 0x7ff0cc003b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e9cdd700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0cc004e70 0x7ff0cc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e8cdb700 1 -- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0cc003780 msgr2=0x7ff0cc003b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:10.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e8cdb700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0cc003780 0x7ff0cc003b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e8cdb700 1 -- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0cc004e70 msgr2=0x7ff0cc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:10.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e8cdb700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0cc004e70 0x7ff0cc00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e8cdb700 1 -- 172.21.2.104:0/3190558954 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc003260 con 0x7ff0cc004500 2021-07-11T04:01:10.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e9cdd700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0cc004e70 0x7ff0cc00b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:01:10.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e94dc700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0cc003780 0x7ff0cc003b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:01:10.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e8cdb700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0cc004500 0x7ff0cc004980 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7ff0d400c000 tx=0x7ff0d4006df0).ready entity=mon.0 client_cookie=fb0105811420af8a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0d400e830 con 0x7ff0cc004500 2021-07-11T04:01:10.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d400e990 con 0x7ff0cc004500 2021-07-11T04:01:10.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0d3fff700 1 -- 172.21.2.104:0/3190558954 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0d400ecf0 con 0x7ff0cc004500 2021-07-11T04:01:10.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0eb740700 1 -- 172.21.2.104:0/3190558954 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0cc004500 msgr2=0x7ff0cc004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0eb740700 1 --2- 172.21.2.104:0/3190558954 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0cc004500 0x7ff0cc004980 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7ff0d400c000 tx=0x7ff0d4006df0).stop 2021-07-11T04:01:10.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.483+0000 7ff0e9cdd700 1 -- 172.21.2.104:0/3190558954 reap_dead start 2021-07-11T04:01:10.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 -- 172.21.2.104:0/3190558954 shutdown_connections 2021-07-11T04:01:10.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 -- 172.21.2.104:0/3190558954 >> 172.21.2.104:0/3190558954 conn(0x7ff0e405cf60 msgr2=0x7ff0e405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:10.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 -- 172.21.2.104:0/3190558954 shutdown_connections 2021-07-11T04:01:10.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 -- 172.21.2.104:0/3190558954 wait complete. 2021-07-11T04:01:10.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 Processor -- start 2021-07-11T04:01:10.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 -- start start 2021-07-11T04:01:10.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0e4143370 0x7ff0e413a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0e4062580 0x7ff0e413ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0e40ab4d0 0x7ff0e40609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40aec60 con 0x7ff0e4062580 2021-07-11T04:01:10.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4143750 con 0x7ff0e4143370 2021-07-11T04:01:10.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e40aee90 con 0x7ff0e40ab4d0 2021-07-11T04:01:10.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e8cdb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0e4062580 0x7ff0e413ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e8cdb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0e4062580 0x7ff0e413ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49284/0 (socket says 172.21.2.104:49284) 2021-07-11T04:01:10.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e8cdb700 1 -- 172.21.2.104:0/804395033 learned_addr learned my addr 172.21.2.104:0/804395033 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:10.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e94dc700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0e4143370 0x7ff0e413a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e9cdd700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0e40ab4d0 0x7ff0e40609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e8cdb700 1 -- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0e40ab4d0 msgr2=0x7ff0e40609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e8cdb700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0e40ab4d0 0x7ff0e40609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e8cdb700 1 -- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0e4143370 msgr2=0x7ff0e413a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e8cdb700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0e4143370 0x7ff0e413a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e8cdb700 1 -- 172.21.2.104:0/804395033 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e40af130 con 0x7ff0e4062580 2021-07-11T04:01:10.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e94dc700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0e4143370 0x7ff0e413a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:01:10.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0e8cdb700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0e4062580 0x7ff0e413ac30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7ff0d400c000 tx=0x7ff0d4019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0d27fc700 1 -- 172.21.2.104:0/804395033 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0d401b7f0 con 0x7ff0e4062580 2021-07-11T04:01:10.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0d27fc700 1 -- 172.21.2.104:0/804395033 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d4000e90 con 0x7ff0e4062580 2021-07-11T04:01:10.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0d27fc700 1 -- 172.21.2.104:0/804395033 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0d401bb40 con 0x7ff0e4062580 2021-07-11T04:01:10.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 -- 172.21.2.104:0/804395033 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e40af3d0 con 0x7ff0e4062580 2021-07-11T04:01:10.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 -- 172.21.2.104:0/804395033 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e413b120 con 0x7ff0e4062580 2021-07-11T04:01:10.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0eb740700 1 -- 172.21.2.104:0/804395033 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e4060710 con 0x7ff0e4062580 2021-07-11T04:01:10.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.487+0000 7ff0d27fc700 1 -- 172.21.2.104:0/804395033 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff0d402a020 con 0x7ff0e4062580 2021-07-11T04:01:10.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.491+0000 7ff0d27fc700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0c4053330 0x7ff0c40557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.491+0000 7ff0d27fc700 1 -- 172.21.2.104:0/804395033 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff0d406c5d0 con 0x7ff0e4062580 2021-07-11T04:01:10.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.491+0000 7ff0e94dc700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0c4053330 0x7ff0c40557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.491+0000 7ff0e94dc700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0c4053330 0x7ff0c40557f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7ff0e000d000 tx=0x7ff0e0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.491+0000 7ff0d27fc700 1 -- 172.21.2.104:0/804395033 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff0d4043a50 con 0x7ff0e4062580 2021-07-11T04:01:10.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 -- 172.21.2.104:0/804395033 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff0e4057350 con 0x7ff0e4062580 2021-07-11T04:01:10.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0d27fc700 1 -- 172.21.2.104:0/804395033 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v115) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff0d4047070 con 0x7ff0e4062580 2021-07-11T04:01:10.576 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:10.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 -- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0c4053330 msgr2=0x7ff0c40557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0c4053330 0x7ff0c40557f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7ff0e000d000 tx=0x7ff0e0009040).stop 2021-07-11T04:01:10.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 -- 172.21.2.104:0/804395033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0e4062580 msgr2=0x7ff0e413ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0e4062580 0x7ff0e413ac30 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7ff0d400c000 tx=0x7ff0d4019000).stop 2021-07-11T04:01:10.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 -- 172.21.2.104:0/804395033 shutdown_connections 2021-07-11T04:01:10.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff0e40ab4d0 0x7ff0e40609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0c4053330 0x7ff0c40557f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff0e4062580 0x7ff0e413ac30 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 --2- 172.21.2.104:0/804395033 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff0e4143370 0x7ff0e413a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 -- 172.21.2.104:0/804395033 >> 172.21.2.104:0/804395033 conn(0x7ff0e405cf60 msgr2=0x7ff0e40b3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:10.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 -- 172.21.2.104:0/804395033 shutdown_connections 2021-07-11T04:01:10.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.575+0000 7ff0eb740700 1 -- 172.21.2.104:0/804395033 wait complete. 2021-07-11T04:01:10.583 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T04:01:10.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.631+0000 7ff2e4979700 1 Processor -- start 2021-07-11T04:01:10.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.631+0000 7ff2e4979700 1 -- start start 2021-07-11T04:01:10.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff2e0060100 0x7ff2e00b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff2e0067300 0x7ff2e01438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2e00b4c40 0x7ff2e0143de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e00aabb0 con 0x7ff2e0060100 2021-07-11T04:01:10.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e00604e0 con 0x7ff2e0067300 2021-07-11T04:01:10.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e0066ea0 con 0x7ff2e00b4c40 2021-07-11T04:01:10.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2de59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff2e0060100 0x7ff2e00b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2de59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff2e0060100 0x7ff2e00b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49288/0 (socket says 172.21.2.104:49288) 2021-07-11T04:01:10.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2de59c700 1 -- 172.21.2.104:0/1150100148 learned_addr learned my addr 172.21.2.104:0/1150100148 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:10.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2de59c700 1 -- 172.21.2.104:0/1150100148 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2e00b4c40 msgr2=0x7ff2e0143de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:10.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2de59c700 1 --2- 172.21.2.104:0/1150100148 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2e00b4c40 0x7ff2e0143de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2de59c700 1 -- 172.21.2.104:0/1150100148 >> v2:172.21.2.105:3300/0 conn(0x7ff2e0067300 msgr2=0x7ff2e01438f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:10.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2de59c700 1 --2- 172.21.2.104:0/1150100148 >> v2:172.21.2.105:3300/0 conn(0x7ff2e0067300 0x7ff2e01438f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2de59c700 1 -- 172.21.2.104:0/1150100148 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e01442d0 con 0x7ff2e0060100 2021-07-11T04:01:10.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2de59c700 1 --2- 172.21.2.104:0/1150100148 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff2e0060100 0x7ff2e00b4750 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7ff2d0002f70 tx=0x7ff2d0006eb0).ready entity=mon.0 client_cookie=4719fe88dac4bc7c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2dd59a700 1 -- 172.21.2.104:0/1150100148 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2d0010b60 con 0x7ff2e0060100 2021-07-11T04:01:10.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2dd59a700 1 -- 172.21.2.104:0/1150100148 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d0000e90 con 0x7ff2e0060100 2021-07-11T04:01:10.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2dd59a700 1 -- 172.21.2.104:0/1150100148 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2d00183d0 con 0x7ff2e0060100 2021-07-11T04:01:10.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- 172.21.2.104:0/1150100148 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff2e0060100 msgr2=0x7ff2e00b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 --2- 172.21.2.104:0/1150100148 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff2e0060100 0x7ff2e00b4750 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7ff2d0002f70 tx=0x7ff2d0006eb0).stop 2021-07-11T04:01:10.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- 172.21.2.104:0/1150100148 shutdown_connections 2021-07-11T04:01:10.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 --2- 172.21.2.104:0/1150100148 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2e00b4c40 0x7ff2e0143de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 --2- 172.21.2.104:0/1150100148 >> v2:172.21.2.105:3300/0 conn(0x7ff2e0067300 0x7ff2e01438f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 --2- 172.21.2.104:0/1150100148 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff2e0060100 0x7ff2e00b4750 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- 172.21.2.104:0/1150100148 >> 172.21.2.104:0/1150100148 conn(0x7ff2e005cf30 msgr2=0x7ff2e005f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:10.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- 172.21.2.104:0/1150100148 shutdown_connections 2021-07-11T04:01:10.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- 172.21.2.104:0/1150100148 wait complete. 2021-07-11T04:01:10.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 Processor -- start 2021-07-11T04:01:10.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- start start 2021-07-11T04:01:10.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff2e0060100 0x7ff2e0156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff2e0067300 0x7ff2e00b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2e00b4c40 0x7ff2e015a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e0159e30 con 0x7ff2e0067300 2021-07-11T04:01:10.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e00a75d0 con 0x7ff2e0060100 2021-07-11T04:01:10.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e00a7870 con 0x7ff2e00b4c40 2021-07-11T04:01:10.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2ddd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff2e0067300 0x7ff2e00b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2ddd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff2e0067300 0x7ff2e00b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49294/0 (socket says 172.21.2.104:49294) 2021-07-11T04:01:10.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2ddd9b700 1 -- 172.21.2.104:0/2726476571 learned_addr learned my addr 172.21.2.104:0/2726476571 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:10.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2ddd9b700 1 -- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2e00b4c40 msgr2=0x7ff2e015a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2ddd9b700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2e00b4c40 0x7ff2e015a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2ddd9b700 1 -- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff2e0060100 msgr2=0x7ff2e0156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2ddd9b700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff2e0060100 0x7ff2e0156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2ddd9b700 1 -- 172.21.2.104:0/2726476571 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d000b1e0 con 0x7ff2e0067300 2021-07-11T04:01:10.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2ddd9b700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff2e0067300 0x7ff2e00b0c70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7ff2c8002fb0 tx=0x7ff2c8006e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2d77fe700 1 -- 172.21.2.104:0/2726476571 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2c8013460 con 0x7ff2e0067300 2021-07-11T04:01:10.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2d77fe700 1 -- 172.21.2.104:0/2726476571 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c80135c0 con 0x7ff2e0067300 2021-07-11T04:01:10.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- 172.21.2.104:0/2726476571 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e015a060 con 0x7ff2e0067300 2021-07-11T04:01:10.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2d77fe700 1 -- 172.21.2.104:0/2726476571 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff2c8013920 con 0x7ff2e0067300 2021-07-11T04:01:10.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.635+0000 7ff2e4979700 1 -- 172.21.2.104:0/2726476571 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e00a70c0 con 0x7ff2e0067300 2021-07-11T04:01:10.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.639+0000 7ff2e4979700 1 -- 172.21.2.104:0/2726476571 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e015a1f0 con 0x7ff2e0067300 2021-07-11T04:01:10.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.639+0000 7ff2d77fe700 1 -- 172.21.2.104:0/2726476571 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff2c800f070 con 0x7ff2e0067300 2021-07-11T04:01:10.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.639+0000 7ff2d77fe700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2c4053210 0x7ff2c40556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.639+0000 7ff2d77fe700 1 -- 172.21.2.104:0/2726476571 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff2c806ab70 con 0x7ff2e0067300 2021-07-11T04:01:10.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.639+0000 7ff2de59c700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2c4053210 0x7ff2c40556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.639+0000 7ff2de59c700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2c4053210 0x7ff2c40556d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7ff2d0002f70 tx=0x7ff2d0017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.639+0000 7ff2d77fe700 1 -- 172.21.2.104:0/2726476571 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff2c8042180 con 0x7ff2e0067300 2021-07-11T04:01:10.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.723+0000 7ff2e4979700 1 -- 172.21.2.104:0/2726476571 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff2e0057350 con 0x7ff2e0067300 2021-07-11T04:01:10.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2d77fe700 1 -- 172.21.2.104:0/2726476571 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v115) v1 ==== 84+0+143 (secure 0 0 0) 0x7ff2c800f300 con 0x7ff2e0067300 2021-07-11T04:01:10.727 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:10.727 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T04:01:10.728 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.728 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T04:01:10.728 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T04:01:10.728 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:10.729 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:10.729 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T04:01:10.729 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T04:01:10.729 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:10.729 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T04:01:10.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 -- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2c4053210 msgr2=0x7ff2c40556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2c4053210 0x7ff2c40556d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7ff2d0002f70 tx=0x7ff2d0017000).stop 2021-07-11T04:01:10.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 -- 172.21.2.104:0/2726476571 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff2e0067300 msgr2=0x7ff2e00b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff2e0067300 0x7ff2e00b0c70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7ff2c8002fb0 tx=0x7ff2c8006e00).stop 2021-07-11T04:01:10.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 -- 172.21.2.104:0/2726476571 shutdown_connections 2021-07-11T04:01:10.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff2e00b4c40 0x7ff2e015a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff2c4053210 0x7ff2c40556d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff2e0067300 0x7ff2e00b0c70 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 --2- 172.21.2.104:0/2726476571 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff2e0060100 0x7ff2e0156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:10.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 -- 172.21.2.104:0/2726476571 >> 172.21.2.104:0/2726476571 conn(0x7ff2e005cf30 msgr2=0x7ff2e0065470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:10.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 -- 172.21.2.104:0/2726476571 shutdown_connections 2021-07-11T04:01:10.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.727+0000 7ff2e4979700 1 -- 172.21.2.104:0/2726476571 wait complete. 2021-07-11T04:01:10.734 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T04:01:10.735 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T04:01:10.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 Processor -- start 2021-07-11T04:01:10.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 -- start start 2021-07-11T04:01:10.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fab380afaf0 0x7fab380b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab380b5f40 con 0x7fab380ad040 2021-07-11T04:01:10.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab380ad420 con 0x7fab3813c9c0 2021-07-11T04:01:10.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab380aeb60 con 0x7fab380afaf0 2021-07-11T04:01:10.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab36d9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fab380afaf0 0x7fab380b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab36d9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fab380afaf0 0x7fab380b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49300/0 (socket says 172.21.2.104:49300) 2021-07-11T04:01:10.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab36d9d700 1 -- 172.21.2.104:0/3581906902 learned_addr learned my addr 172.21.2.104:0/3581906902 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:10.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab36d9d700 1 -- 172.21.2.104:0/3581906902 >> v1:172.21.2.105:6789/0 conn(0x7fab3813c9c0 legacy=0x7fab3813f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:10.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab36d9d700 1 -- 172.21.2.104:0/3581906902 >> v1:172.21.2.104:6789/0 conn(0x7fab380ad040 legacy=0x7fab3813c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab36d9d700 1 -- 172.21.2.104:0/3581906902 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab380aee00 con 0x7fab380afaf0 2021-07-11T04:01:10.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab36d9d700 1 --2- 172.21.2.104:0/3581906902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab380afaf0 0x7fab380b5a50 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7fab3805d010 tx=0x7fab30009b20).ready entity=mon.0 client_cookie=3c36a004dbad9182 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3559a700 1 -- 172.21.2.104:0/3581906902 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab3000cb60 con 0x7fab380afaf0 2021-07-11T04:01:10.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3559a700 1 -- 172.21.2.104:0/3581906902 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3000ccc0 con 0x7fab380afaf0 2021-07-11T04:01:10.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3559a700 1 -- 172.21.2.104:0/3581906902 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab30010600 con 0x7fab380afaf0 2021-07-11T04:01:10.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 -- 172.21.2.104:0/3581906902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab380afaf0 msgr2=0x7fab380b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 --2- 172.21.2.104:0/3581906902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab380afaf0 0x7fab380b5a50 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7fab3805d010 tx=0x7fab30009b20).stop 2021-07-11T04:01:10.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 -- 172.21.2.104:0/3581906902 shutdown_connections 2021-07-11T04:01:10.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 --2- 172.21.2.104:0/3581906902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab380afaf0 0x7fab380b5a50 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 -- 172.21.2.104:0/3581906902 >> 172.21.2.104:0/3581906902 conn(0x7fab3805c490 msgr2=0x7fab3805c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:10.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 -- 172.21.2.104:0/3581906902 shutdown_connections 2021-07-11T04:01:10.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.787+0000 7fab3cf81700 1 -- 172.21.2.104:0/3581906902 wait complete. 2021-07-11T04:01:10.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3cf81700 1 Processor -- start 2021-07-11T04:01:10.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3cf81700 1 -- start start 2021-07-11T04:01:10.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3cf81700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab380ad040 0x7fab3813fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3cf81700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab380afaf0 0x7fab38067170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3cf81700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab3813c9c0 0x7fab38067660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3cf81700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab380ad900 con 0x7fab3813c9c0 2021-07-11T04:01:10.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3cf81700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3805d760 con 0x7fab380ad040 2021-07-11T04:01:10.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3cf81700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab38140320 con 0x7fab380afaf0 2021-07-11T04:01:10.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab36d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab3813c9c0 0x7fab38067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab36d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab3813c9c0 0x7fab38067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49310/0 (socket says 172.21.2.104:49310) 2021-07-11T04:01:10.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab36d9d700 1 -- 172.21.2.104:0/3154838277 learned_addr learned my addr 172.21.2.104:0/3154838277 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:10.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab35d9b700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab380afaf0 0x7fab38067170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3659c700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab380ad040 0x7fab3813fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab36d9d700 1 -- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab380afaf0 msgr2=0x7fab38067170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab36d9d700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab380afaf0 0x7fab38067170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab36d9d700 1 -- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab380ad040 msgr2=0x7fab3813fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:10.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab36d9d700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab380ad040 0x7fab3813fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:10.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab36d9d700 1 -- 172.21.2.104:0/3154838277 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30009dc0 con 0x7fab3813c9c0 2021-07-11T04:01:10.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab36d9d700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab3813c9c0 0x7fab38067660 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7fab38064fd0 tx=0x7fab30017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab277fe700 1 -- 172.21.2.104:0/3154838277 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab3001a850 con 0x7fab3813c9c0 2021-07-11T04:01:10.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab277fe700 1 -- 172.21.2.104:0/3154838277 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3001a9b0 con 0x7fab3813c9c0 2021-07-11T04:01:10.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab277fe700 1 -- 172.21.2.104:0/3154838277 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fab3001ad10 con 0x7fab3813c9c0 2021-07-11T04:01:10.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3cf81700 1 -- 172.21.2.104:0/3154838277 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab381418f0 con 0x7fab3813c9c0 2021-07-11T04:01:10.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3cf81700 1 -- 172.21.2.104:0/3154838277 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab381421f0 con 0x7fab3813c9c0 2021-07-11T04:01:10.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3cf81700 1 -- 172.21.2.104:0/3154838277 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab38057350 con 0x7fab3813c9c0 2021-07-11T04:01:10.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab277fe700 1 -- 172.21.2.104:0/3154838277 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fab3000e040 con 0x7fab3813c9c0 2021-07-11T04:01:10.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab277fe700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab14053330 0x7fab140557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:10.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab277fe700 1 -- 172.21.2.104:0/3154838277 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fab3006c960 con 0x7fab3813c9c0 2021-07-11T04:01:10.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3659c700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab14053330 0x7fab140557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:10.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.791+0000 7fab3659c700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab14053330 0x7fab140557f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fab2c00b000 tx=0x7fab2c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:10.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.795+0000 7fab277fe700 1 -- 172.21.2.104:0/3154838277 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fab30043fd0 con 0x7fab3813c9c0 2021-07-11T04:01:10.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:10.875+0000 7fab3cf81700 1 -- 172.21.2.104:0/3154838277 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fab380ac990 con 0x7fab3813c9c0 2021-07-11T04:01:11.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab277fe700 1 -- 172.21.2.104:0/3154838277 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v116) v1 ==== 133+0+0 (secure 0 0 0) 0x7fab300098c0 con 0x7fab3813c9c0 2021-07-11T04:01:11.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 -- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab14053330 msgr2=0x7fab140557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:11.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab14053330 0x7fab140557f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fab2c00b000 tx=0x7fab2c009040).stop 2021-07-11T04:01:11.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 -- 172.21.2.104:0/3154838277 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab3813c9c0 msgr2=0x7fab38067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:11.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab3813c9c0 0x7fab38067660 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7fab38064fd0 tx=0x7fab30017000).stop 2021-07-11T04:01:11.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 -- 172.21.2.104:0/3154838277 shutdown_connections 2021-07-11T04:01:11.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fab14053330 0x7fab140557f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:11.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fab3813c9c0 0x7fab38067660 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:11.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fab380afaf0 0x7fab38067170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:11.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 --2- 172.21.2.104:0/3154838277 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fab380ad040 0x7fab3813fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:11.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 -- 172.21.2.104:0/3154838277 >> 172.21.2.104:0/3154838277 conn(0x7fab3805c490 msgr2=0x7fab380b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:11.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 -- 172.21.2.104:0/3154838277 shutdown_connections 2021-07-11T04:01:11.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.839+0000 7fab3cf81700 1 -- 172.21.2.104:0/3154838277 wait complete. 2021-07-11T04:01:11.843 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T04:01:11.848 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T04:01:11.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1ab398700 1 Processor -- start 2021-07-11T04:01:12.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1ab398700 1 -- start start 2021-07-11T04:01:12.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1ab398700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1a40b1ec0 0x7fb1a40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1ab398700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb1a40b22a0 0x7fb1a4141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1ab398700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a40a77d0 con 0x7fb1a40affb0 2021-07-11T04:01:12.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1ab398700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a40b03f0 con 0x7fb1a40b22a0 2021-07-11T04:01:12.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1ab398700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40b0650 con 0x7fb1a40b1ec0 2021-07-11T04:01:12.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1a8933700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7fb1a40affb0 0x7fb1a40a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:45454/0 (socket says 172.21.2.104:45454) 2021-07-11T04:01:12.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1a8933700 1 -- 172.21.2.104:0/3965337776 learned_addr learned my addr 172.21.2.104:0/3965337776 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:12.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1a9134700 1 --2- 172.21.2.104:0/3965337776 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1a40b1ec0 0x7fb1a40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:12.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1a9935700 1 --2- 172.21.2.104:0/3965337776 >> v2:172.21.2.105:3300/0 conn(0x7fb1a40b22a0 0x7fb1a4141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:12.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1a9935700 1 -- 172.21.2.104:0/3965337776 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1a40b1ec0 msgr2=0x7fb1a40b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:12.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1a9935700 1 --2- 172.21.2.104:0/3965337776 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1a40b1ec0 0x7fb1a40b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:12.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1a9935700 1 -- 172.21.2.104:0/3965337776 >> v1:172.21.2.105:6789/0 conn(0x7fb1a40affb0 legacy=0x7fb1a40a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:12.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1a9935700 1 -- 172.21.2.104:0/3965337776 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40b08f0 con 0x7fb1a40b22a0 2021-07-11T04:01:12.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.903+0000 7fb1a9935700 1 --2- 172.21.2.104:0/3965337776 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1a40b22a0 0x7fb1a4141250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb19c0032f0 tx=0x7fb19c0091d0).ready entity=mon.1 client_cookie=485ea7fb5b535062 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:12.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb197fff700 1 -- 172.21.2.104:0/3965337776 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb19c00b6c0 con 0x7fb1a40b22a0 2021-07-11T04:01:12.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb197fff700 1 -- 172.21.2.104:0/3965337776 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c00b820 con 0x7fb1a40b22a0 2021-07-11T04:01:12.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb197fff700 1 -- 172.21.2.104:0/3965337776 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb19c00b990 con 0x7fb1a40b22a0 2021-07-11T04:01:12.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- 172.21.2.104:0/3965337776 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1a40b22a0 msgr2=0x7fb1a4141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:12.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 --2- 172.21.2.104:0/3965337776 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1a40b22a0 0x7fb1a4141250 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb19c0032f0 tx=0x7fb19c0091d0).stop 2021-07-11T04:01:12.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- 172.21.2.104:0/3965337776 shutdown_connections 2021-07-11T04:01:12.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 --2- 172.21.2.104:0/3965337776 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1a40b22a0 0x7fb1a4141250 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:12.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 --2- 172.21.2.104:0/3965337776 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1a40b1ec0 0x7fb1a40b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:12.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- 172.21.2.104:0/3965337776 >> 172.21.2.104:0/3965337776 conn(0x7fb1a405f6b0 msgr2=0x7fb1a405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:12.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- 172.21.2.104:0/3965337776 shutdown_connections 2021-07-11T04:01:12.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- 172.21.2.104:0/3965337776 wait complete. 2021-07-11T04:01:12.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 Processor -- start 2021-07-11T04:01:12.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- start start 2021-07-11T04:01:12.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1a40affb0 0x7fb1a4146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1a40b1ec0 0x7fb1a40aa990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1a40b22a0 0x7fb1a41573b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a4156a80 con 0x7fb1a40b22a0 2021-07-11T04:01:12.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a40aae80 con 0x7fb1a40b1ec0 2021-07-11T04:01:12.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a415aa60 con 0x7fb1a40affb0 2021-07-11T04:01:12.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1a9935700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1a40b22a0 0x7fb1a41573b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:12.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1a9935700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1a40b22a0 0x7fb1a41573b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49324/0 (socket says 172.21.2.104:49324) 2021-07-11T04:01:12.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1a9935700 1 -- 172.21.2.104:0/1148813455 learned_addr learned my addr 172.21.2.104:0/1148813455 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:12.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1a9935700 1 -- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1a40affb0 msgr2=0x7fb1a4146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:12.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1a9935700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1a40affb0 0x7fb1a4146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:12.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1a9935700 1 -- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1a40b1ec0 msgr2=0x7fb1a40aa990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:12.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1a8933700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1a40b1ec0 0x7fb1a40aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:12.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1a9935700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1a40b1ec0 0x7fb1a40aa990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:12.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1a9935700 1 -- 172.21.2.104:0/1148813455 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb19c009470 con 0x7fb1a40b22a0 2021-07-11T04:01:12.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1a9935700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1a40b22a0 0x7fb1a41573b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7fb19c0032f0 tx=0x7fb19c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:12.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1967fc700 1 -- 172.21.2.104:0/1148813455 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb19c006ea0 con 0x7fb1a40b22a0 2021-07-11T04:01:12.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1967fc700 1 -- 172.21.2.104:0/1148813455 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19c011e90 con 0x7fb1a40b22a0 2021-07-11T04:01:12.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1967fc700 1 -- 172.21.2.104:0/1148813455 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb19c021670 con 0x7fb1a40b22a0 2021-07-11T04:01:12.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- 172.21.2.104:0/1148813455 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a415ad00 con 0x7fb1a40b22a0 2021-07-11T04:01:12.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- 172.21.2.104:0/1148813455 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a4156f10 con 0x7fb1a40b22a0 2021-07-11T04:01:12.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.907+0000 7fb1ab398700 1 -- 172.21.2.104:0/1148813455 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a4057350 con 0x7fb1a40b22a0 2021-07-11T04:01:12.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.911+0000 7fb1967fc700 1 -- 172.21.2.104:0/1148813455 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb19c028070 con 0x7fb1a40b22a0 2021-07-11T04:01:12.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.911+0000 7fb1967fc700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1840532b0 0x7fb184055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.911+0000 7fb1967fc700 1 -- 172.21.2.104:0/1148813455 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5603+0+0 (secure 0 0 0) 0x7fb19c06c1e0 con 0x7fb1a40b22a0 2021-07-11T04:01:12.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.911+0000 7fb1a9134700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1840532b0 0x7fb184055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:12.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.911+0000 7fb1a9134700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1840532b0 0x7fb184055770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fb19800c000 tx=0x7fb198006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:12.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.911+0000 7fb1967fc700 1 -- 172.21.2.104:0/1148813455 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb19c043640 con 0x7fb1a40b22a0 2021-07-11T04:01:12.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:11.995+0000 7fb1ab398700 1 -- 172.21.2.104:0/1148813455 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fb1a40ac760 con 0x7fb1a40b22a0 2021-07-11T04:01:12.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1967fc700 1 -- 172.21.2.104:0/1148813455 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v117) v1 ==== 125+0+0 (secure 0 0 0) 0x7fb19c046ba0 con 0x7fb1a40b22a0 2021-07-11T04:01:12.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 -- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1840532b0 msgr2=0x7fb184055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:12.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1840532b0 0x7fb184055770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fb19800c000 tx=0x7fb198006df0).stop 2021-07-11T04:01:12.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 -- 172.21.2.104:0/1148813455 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1a40b22a0 msgr2=0x7fb1a41573b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:12.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1a40b22a0 0x7fb1a41573b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7fb19c0032f0 tx=0x7fb19c016000).stop 2021-07-11T04:01:12.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 -- 172.21.2.104:0/1148813455 shutdown_connections 2021-07-11T04:01:12.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1840532b0 0x7fb184055770 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:12.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1a40b22a0 0x7fb1a41573b0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:12.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1a40b1ec0 0x7fb1a40aa990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:12.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 --2- 172.21.2.104:0/1148813455 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1a40affb0 0x7fb1a4146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:12.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 -- 172.21.2.104:0/1148813455 >> 172.21.2.104:0/1148813455 conn(0x7fb1a405f6b0 msgr2=0x7fb1a413c750 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:12.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 -- 172.21.2.104:0/1148813455 shutdown_connections 2021-07-11T04:01:12.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.851+0000 7fb1ab398700 1 -- 172.21.2.104:0/1148813455 wait complete. 2021-07-11T04:01:12.856 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T04:01:12.859 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T04:01:12.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27f03ac700 1 Processor -- start 2021-07-11T04:01:12.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27f03ac700 1 -- start start 2021-07-11T04:01:12.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27f03ac700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f27e80b7810 0x7f27e81445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27f03ac700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f27e80b1170 0x7f27e8144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27f03ac700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27e8064870 0x7f27e8064cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27f03ac700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e80b31d0 con 0x7f27e80b1170 2021-07-11T04:01:12.914 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27f03ac700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e80b7bf0 con 0x7f27e80b7810 2021-07-11T04:01:12.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27f03ac700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e80675c0 con 0x7f27e8064870 2021-07-11T04:01:12.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed947700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f27e80b1170 0x7f27e8144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:12.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed947700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f27e80b1170 0x7f27e8144a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49332/0 (socket says 172.21.2.104:49332) 2021-07-11T04:01:12.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed947700 1 -- 172.21.2.104:0/2014373986 learned_addr learned my addr 172.21.2.104:0/2014373986 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:12.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed947700 1 -- 172.21.2.104:0/2014373986 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27e8064870 msgr2=0x7f27e8064cd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:12.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed947700 1 --2- 172.21.2.104:0/2014373986 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27e8064870 0x7f27e8064cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:12.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed947700 1 -- 172.21.2.104:0/2014373986 >> v2:172.21.2.105:3300/0 conn(0x7f27e80b7810 msgr2=0x7f27e81445a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:12.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed947700 1 --2- 172.21.2.104:0/2014373986 >> v2:172.21.2.105:3300/0 conn(0x7f27e80b7810 0x7f27e81445a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:12.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed947700 1 -- 172.21.2.104:0/2014373986 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e80aae70 con 0x7f27e80b1170 2021-07-11T04:01:12.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed947700 1 --2- 172.21.2.104:0/2014373986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27e80b1170 0x7f27e8144a90 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f27d8002f70 tx=0x7f27d8006eb0).ready entity=mon.0 client_cookie=c6a10caf540e93d1 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:12.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed146700 1 -- 172.21.2.104:0/2014373986 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27d800d570 con 0x7f27e80b1170 2021-07-11T04:01:12.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed146700 1 -- 172.21.2.104:0/2014373986 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d8000e90 con 0x7f27e80b1170 2021-07-11T04:01:12.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27ed146700 1 -- 172.21.2.104:0/2014373986 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27d800d900 con 0x7f27e80b1170 2021-07-11T04:01:12.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27f03ac700 1 -- 172.21.2.104:0/2014373986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27e80b1170 msgr2=0x7f27e8144a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:12.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.911+0000 7f27f03ac700 1 --2- 172.21.2.104:0/2014373986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27e80b1170 0x7f27e8144a90 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7f27d8002f70 tx=0x7f27d8006eb0).stop 2021-07-11T04:01:12.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 -- 172.21.2.104:0/2014373986 shutdown_connections 2021-07-11T04:01:12.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 --2- 172.21.2.104:0/2014373986 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27e8064870 0x7f27e8064cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:12.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 --2- 172.21.2.104:0/2014373986 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27e80b1170 0x7f27e8144a90 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:12.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 --2- 172.21.2.104:0/2014373986 >> v2:172.21.2.105:3300/0 conn(0x7f27e80b7810 0x7f27e81445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:12.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 -- 172.21.2.104:0/2014373986 >> 172.21.2.104:0/2014373986 conn(0x7f27e805f6b0 msgr2=0x7f27e805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:12.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 -- 172.21.2.104:0/2014373986 shutdown_connections 2021-07-11T04:01:12.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 -- 172.21.2.104:0/2014373986 wait complete. 2021-07-11T04:01:12.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 Processor -- start 2021-07-11T04:01:12.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 -- start start 2021-07-11T04:01:12.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27e8064870 0x7f27e81522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27e80b1170 0x7f27e80afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27e80b7810 0x7f27e81560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e81527b0 con 0x7f27e80b7810 2021-07-11T04:01:12.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e81559a0 con 0x7f27e8064870 2021-07-11T04:01:12.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e8062730 con 0x7f27e80b1170 2021-07-11T04:01:12.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27ee949700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27e80b7810 0x7f27e81560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:12.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27ee949700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27e80b7810 0x7f27e81560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49336/0 (socket says 172.21.2.104:49336) 2021-07-11T04:01:12.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27ee949700 1 -- 172.21.2.104:0/2528486577 learned_addr learned my addr 172.21.2.104:0/2528486577 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:12.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27ee949700 1 -- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27e80b1170 msgr2=0x7f27e80afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:12.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27ee949700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27e80b1170 0x7f27e80afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:12.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27ee949700 1 -- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27e8064870 msgr2=0x7f27e81522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:12.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27ee949700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27e8064870 0x7f27e81522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:12.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27ee949700 1 -- 172.21.2.104:0/2528486577 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d800b1e0 con 0x7f27e80b7810 2021-07-11T04:01:12.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27ee949700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27e80b7810 0x7f27e81560a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f27e80628c0 tx=0x7f27e00091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:12.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27d77fe700 1 -- 172.21.2.104:0/2528486577 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27e0001520 con 0x7f27e80b7810 2021-07-11T04:01:12.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27d77fe700 1 -- 172.21.2.104:0/2528486577 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e0001680 con 0x7f27e80b7810 2021-07-11T04:01:12.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 -- 172.21.2.104:0/2528486577 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e80aff70 con 0x7f27e80b7810 2021-07-11T04:01:12.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27d77fe700 1 -- 172.21.2.104:0/2528486577 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27e00019e0 con 0x7f27e80b7810 2021-07-11T04:01:12.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 -- 172.21.2.104:0/2528486577 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e8061df0 con 0x7f27e80b7810 2021-07-11T04:01:12.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27f03ac700 1 -- 172.21.2.104:0/2528486577 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e8057350 con 0x7f27e80b7810 2021-07-11T04:01:12.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27d77fe700 1 -- 172.21.2.104:0/2528486577 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f27e0018030 con 0x7f27e80b7810 2021-07-11T04:01:12.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27d77fe700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f27c8053160 0x7f27c8055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:12.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27d77fe700 1 -- 172.21.2.104:0/2528486577 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f27e006ae30 con 0x7f27e80b7810 2021-07-11T04:01:12.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.915+0000 7f27ee148700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f27c8053160 0x7f27c8055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:12.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.919+0000 7f27ee148700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f27c8053160 0x7f27c8055620 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f27e4000ed0 tx=0x7f27e4006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:12.930 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:01:12.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:12.919+0000 7f27d77fe700 1 -- 172.21.2.104:0/2528486577 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f27e0042290 con 0x7f27e80b7810 2021-07-11T04:01:12.932 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:01:12.935 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:01:12.938 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:01:13.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.027+0000 7f27f03ac700 1 -- 172.21.2.104:0/2528486577 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f27e80698e0 con 0x7f27e80b7810 2021-07-11T04:01:13.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27d77fe700 1 -- 172.21.2.104:0/2528486577 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 21 and data pool 22 v55) v1 ==== 189+0+0 (secure 0 0 0) 0x7f27e00458b0 con 0x7f27e80b7810 2021-07-11T04:01:13.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 -- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f27c8053160 msgr2=0x7f27c8055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f27c8053160 0x7f27c8055620 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f27e4000ed0 tx=0x7f27e4006930).stop 2021-07-11T04:01:13.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 -- 172.21.2.104:0/2528486577 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27e80b7810 msgr2=0x7f27e81560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27e80b7810 0x7f27e81560a0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f27e80628c0 tx=0x7f27e00091a0).stop 2021-07-11T04:01:13.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 -- 172.21.2.104:0/2528486577 shutdown_connections 2021-07-11T04:01:13.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f27c8053160 0x7f27c8055620 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:13.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27e80b7810 0x7f27e81560a0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:13.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27e80b1170 0x7f27e80afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 --2- 172.21.2.104:0/2528486577 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27e8064870 0x7f27e81522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 -- 172.21.2.104:0/2528486577 >> 172.21.2.104:0/2528486577 conn(0x7f27e805f6b0 msgr2=0x7f27e8144d70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:13.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 -- 172.21.2.104:0/2528486577 shutdown_connections 2021-07-11T04:01:13.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.035+0000 7f27f03ac700 1 -- 172.21.2.104:0/2528486577 wait complete. 2021-07-11T04:01:13.039 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 21 and data pool 22 2021-07-11T04:01:13.044 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:13.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 Processor -- start 2021-07-11T04:01:13.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- start start 2021-07-11T04:01:13.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f50ec067180 0x7f50ec05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50ec0b1970 0x7f50ec05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ec0a7a90 con 0x7f50ec0a76b0 2021-07-11T04:01:13.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ec147620 con 0x7f50ec067180 2021-07-11T04:01:13.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec1478c0 con 0x7f50ec0b1970 2021-07-11T04:01:13.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f50ec067180 0x7f50ec05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:13.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f0f1a700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f50ec0a76b0 0x7f50ec066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46122/0 (socket says 172.21.2.104:46122) 2021-07-11T04:01:13.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f0f1a700 1 -- 172.21.2.104:0/1187347106 learned_addr learned my addr 172.21.2.104:0/1187347106 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:13.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 -- 172.21.2.104:0/1187347106 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50ec0b1970 msgr2=0x7f50ec05fba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:13.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 --2- 172.21.2.104:0/1187347106 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50ec0b1970 0x7f50ec05fba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 -- 172.21.2.104:0/1187347106 >> v1:172.21.2.104:6789/0 conn(0x7f50ec0a76b0 legacy=0x7f50ec066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 -- 172.21.2.104:0/1187347106 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ec147b60 con 0x7f50ec067180 2021-07-11T04:01:13.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 --2- 172.21.2.104:0/1187347106 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50ec067180 0x7f50ec05f6b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f50dc002f70 tx=0x7f50dc006df0).ready entity=mon.0 client_cookie=dbe3fee13b5883ba server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:13.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50eb7fe700 1 -- 172.21.2.104:0/1187347106 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50dc00d570 con 0x7f50ec067180 2021-07-11T04:01:13.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50eb7fe700 1 -- 172.21.2.104:0/1187347106 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc000e90 con 0x7f50ec067180 2021-07-11T04:01:13.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50eb7fe700 1 -- 172.21.2.104:0/1187347106 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50dc00d900 con 0x7f50ec067180 2021-07-11T04:01:13.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- 172.21.2.104:0/1187347106 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50ec067180 msgr2=0x7f50ec05f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 --2- 172.21.2.104:0/1187347106 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50ec067180 0x7f50ec05f6b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f50dc002f70 tx=0x7f50dc006df0).stop 2021-07-11T04:01:13.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- 172.21.2.104:0/1187347106 shutdown_connections 2021-07-11T04:01:13.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 --2- 172.21.2.104:0/1187347106 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50ec067180 0x7f50ec05f6b0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:13.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 --2- 172.21.2.104:0/1187347106 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50ec0b1970 0x7f50ec05fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- 172.21.2.104:0/1187347106 >> 172.21.2.104:0/1187347106 conn(0x7f50ec0b53e0 msgr2=0x7f50ec0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:13.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- 172.21.2.104:0/1187347106 shutdown_connections 2021-07-11T04:01:13.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- 172.21.2.104:0/1187347106 wait complete. 2021-07-11T04:01:13.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 Processor -- start 2021-07-11T04:01:13.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- start start 2021-07-11T04:01:13.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f50ec067180 0x7f50ec156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50ec0a76b0 0x7f50ec0ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50ec0b1970 0x7f50ec0aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0ae130 con 0x7f50ec0a76b0 2021-07-11T04:01:13.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0ae3d0 con 0x7f50ec067180 2021-07-11T04:01:13.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ec0ae670 con 0x7f50ec0b1970 2021-07-11T04:01:13.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50ec0a76b0 0x7f50ec0ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:13.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50ec0a76b0 0x7f50ec0ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49362/0 (socket says 172.21.2.104:49362) 2021-07-11T04:01:13.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 -- 172.21.2.104:0/17170771 learned_addr learned my addr 172.21.2.104:0/17170771 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:13.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 -- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50ec0b1970 msgr2=0x7f50ec0aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50ec0b1970 0x7f50ec0aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 -- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f50ec067180 msgr2=0x7f50ec156720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f50ec067180 0x7f50ec156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 -- 172.21.2.104:0/17170771 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc00b180 con 0x7f50ec0a76b0 2021-07-11T04:01:13.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50ebfff700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50ec0a76b0 0x7f50ec0ad660 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f50dc002f70 tx=0x7f50dc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:13.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50e9ffb700 1 -- 172.21.2.104:0/17170771 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50dc00de40 con 0x7f50ec0a76b0 2021-07-11T04:01:13.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50e9ffb700 1 -- 172.21.2.104:0/17170771 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50dc000e90 con 0x7f50ec0a76b0 2021-07-11T04:01:13.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- 172.21.2.104:0/17170771 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ec0ae910 con 0x7f50ec0a76b0 2021-07-11T04:01:13.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50e9ffb700 1 -- 172.21.2.104:0/17170771 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f50dc006aa0 con 0x7f50ec0a76b0 2021-07-11T04:01:13.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.095+0000 7f50f317e700 1 -- 172.21.2.104:0/17170771 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ec0aedb0 con 0x7f50ec0a76b0 2021-07-11T04:01:13.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.099+0000 7f50f317e700 1 -- 172.21.2.104:0/17170771 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ec057350 con 0x7f50ec0a76b0 2021-07-11T04:01:13.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.099+0000 7f50e9ffb700 1 -- 172.21.2.104:0/17170771 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f50dc023030 con 0x7f50ec0a76b0 2021-07-11T04:01:13.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.099+0000 7f50e9ffb700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50d00532b0 0x7f50d0055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.099+0000 7f50e9ffb700 1 -- 172.21.2.104:0/17170771 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f50dc06bdf0 con 0x7f50ec0a76b0 2021-07-11T04:01:13.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.099+0000 7f50f0f1a700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50d00532b0 0x7f50d0055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:13.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.099+0000 7f50f0f1a700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50d00532b0 0x7f50d0055770 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f50ec06a0e0 tx=0x7f50d4007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:13.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.099+0000 7f50e9ffb700 1 -- 172.21.2.104:0/17170771 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f50dc0430d0 con 0x7f50ec0a76b0 2021-07-11T04:01:13.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.183+0000 7f50f317e700 1 -- 172.21.2.104:0/17170771 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50ec0aab50 con 0x7f50ec0a76b0 2021-07-11T04:01:13.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.183+0000 7f50e9ffb700 1 -- 172.21.2.104:0/17170771 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f50dc0466f0 con 0x7f50ec0a76b0 2021-07-11T04:01:13.186 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:13.186 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":118,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:13.031308+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:01:10.890456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-07-11T04:01:12.008882+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:13.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 -- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50d00532b0 msgr2=0x7f50d0055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50d00532b0 0x7f50d0055770 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f50ec06a0e0 tx=0x7f50d4007800).stop 2021-07-11T04:01:13.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 -- 172.21.2.104:0/17170771 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50ec0a76b0 msgr2=0x7f50ec0ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50ec0a76b0 0x7f50ec0ad660 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f50dc002f70 tx=0x7f50dc015000).stop 2021-07-11T04:01:13.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 -- 172.21.2.104:0/17170771 shutdown_connections 2021-07-11T04:01:13.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f50ec0b1970 0x7f50ec0aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f50d00532b0 0x7f50d0055770 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:13.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f50ec0a76b0 0x7f50ec0ad660 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:13.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 --2- 172.21.2.104:0/17170771 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f50ec067180 0x7f50ec156720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 -- 172.21.2.104:0/17170771 >> 172.21.2.104:0/17170771 conn(0x7f50ec0b53e0 msgr2=0x7f50ec1447b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:13.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 -- 172.21.2.104:0/17170771 shutdown_connections 2021-07-11T04:01:13.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.187+0000 7f50f317e700 1 -- 172.21.2.104:0/17170771 wait complete. 2021-07-11T04:01:13.194 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:13.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.251+0000 7f5343588700 1 Processor -- start 2021-07-11T04:01:13.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.251+0000 7f5343588700 1 -- start start 2021-07-11T04:01:13.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.251+0000 7f5343588700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f533c067180 0x7f533c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.251+0000 7f5343588700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f533c0b1970 0x7f533c05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.251+0000 7f5343588700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f533c0a7a90 con 0x7f533c0a76b0 2021-07-11T04:01:13.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.251+0000 7f5343588700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f533c147620 con 0x7f533c0b1970 2021-07-11T04:01:13.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.251+0000 7f5343588700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c1478c0 con 0x7f533c067180 2021-07-11T04:01:13.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.251+0000 7f5341b25700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f533c0b1970 0x7f533c05fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:13.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.251+0000 7f5341324700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f533c0a76b0 0x7f533c066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46136/0 (socket says 172.21.2.104:46136) 2021-07-11T04:01:13.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341324700 1 -- 172.21.2.104:0/193757458 learned_addr learned my addr 172.21.2.104:0/193757458 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:13.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341b25700 1 -- 172.21.2.104:0/193757458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f533c067180 msgr2=0x7f533c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:13.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341b25700 1 --2- 172.21.2.104:0/193757458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f533c067180 0x7f533c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341b25700 1 -- 172.21.2.104:0/193757458 >> v1:172.21.2.104:6789/0 conn(0x7f533c0a76b0 legacy=0x7f533c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341b25700 1 -- 172.21.2.104:0/193757458 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533c147b60 con 0x7f533c0b1970 2021-07-11T04:01:13.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341b25700 1 --2- 172.21.2.104:0/193757458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f533c0b1970 0x7f533c05fba0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7f53340032f0 tx=0x7f53340091d0).ready entity=mon.0 client_cookie=e82eae4c00b30644 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:13.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f532bfff700 1 -- 172.21.2.104:0/193757458 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5334009ea0 con 0x7f533c0b1970 2021-07-11T04:01:13.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f532bfff700 1 -- 172.21.2.104:0/193757458 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f533400b8a0 con 0x7f533c0b1970 2021-07-11T04:01:13.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f532bfff700 1 -- 172.21.2.104:0/193757458 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f533400ba10 con 0x7f533c0b1970 2021-07-11T04:01:13.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- 172.21.2.104:0/193757458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f533c0b1970 msgr2=0x7f533c05fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 --2- 172.21.2.104:0/193757458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f533c0b1970 0x7f533c05fba0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7f53340032f0 tx=0x7f53340091d0).stop 2021-07-11T04:01:13.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- 172.21.2.104:0/193757458 shutdown_connections 2021-07-11T04:01:13.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 --2- 172.21.2.104:0/193757458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f533c0b1970 0x7f533c05fba0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:13.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 --2- 172.21.2.104:0/193757458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f533c067180 0x7f533c05f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- 172.21.2.104:0/193757458 >> 172.21.2.104:0/193757458 conn(0x7f533c0b53e0 msgr2=0x7f533c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:13.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- 172.21.2.104:0/193757458 shutdown_connections 2021-07-11T04:01:13.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- 172.21.2.104:0/193757458 wait complete. 2021-07-11T04:01:13.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 Processor -- start 2021-07-11T04:01:13.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- start start 2021-07-11T04:01:13.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f533c067180 0x7f533c0ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f533c0a76b0 0x7f533c0adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f533c0b1970 0x7f533c0ae520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c0ae0c0 con 0x7f533c067180 2021-07-11T04:01:13.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533c156c60 con 0x7f533c0b1970 2021-07-11T04:01:13.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533c157820 con 0x7f533c0a76b0 2021-07-11T04:01:13.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341324700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f533c067180 0x7f533c0ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:13.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341324700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f533c067180 0x7f533c0ad5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49376/0 (socket says 172.21.2.104:49376) 2021-07-11T04:01:13.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341324700 1 -- 172.21.2.104:0/1797730321 learned_addr learned my addr 172.21.2.104:0/1797730321 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:13.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341324700 1 -- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f533c0a76b0 msgr2=0x7f533c0adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341324700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f533c0a76b0 0x7f533c0adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341324700 1 -- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f533c0b1970 msgr2=0x7f533c0ae520 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:13.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341324700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f533c0b1970 0x7f533c0ae520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341324700 1 -- 172.21.2.104:0/1797730321 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5334009470 con 0x7f533c067180 2021-07-11T04:01:13.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5341324700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f533c067180 0x7f533c0ad5f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f533c0b4da0 tx=0x7f532c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:13.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f532a7fc700 1 -- 172.21.2.104:0/1797730321 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f532c00ce20 con 0x7f533c067180 2021-07-11T04:01:13.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f532a7fc700 1 -- 172.21.2.104:0/1797730321 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c0072e0 con 0x7f533c067180 2021-07-11T04:01:13.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f532a7fc700 1 -- 172.21.2.104:0/1797730321 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f532c0057e0 con 0x7f533c067180 2021-07-11T04:01:13.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- 172.21.2.104:0/1797730321 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c0b4f00 con 0x7f533c067180 2021-07-11T04:01:13.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- 172.21.2.104:0/1797730321 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533c1569c0 con 0x7f533c067180 2021-07-11T04:01:13.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f5343588700 1 -- 172.21.2.104:0/1797730321 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533c057350 con 0x7f533c067180 2021-07-11T04:01:13.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f532a7fc700 1 -- 172.21.2.104:0/1797730321 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f532c010070 con 0x7f533c067180 2021-07-11T04:01:13.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f532a7fc700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f531c053160 0x7f531c055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.255+0000 7f532a7fc700 1 -- 172.21.2.104:0/1797730321 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f532c06aa80 con 0x7f533c067180 2021-07-11T04:01:13.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.259+0000 7f5340b23700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f531c053160 0x7f531c055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:13.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.259+0000 7f5340b23700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f531c053160 0x7f531c055620 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f5338002f70 tx=0x7f5338006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:13.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.259+0000 7f532a7fc700 1 -- 172.21.2.104:0/1797730321 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f532c041dc0 con 0x7f533c067180 2021-07-11T04:01:13.344 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.343+0000 7f5343588700 1 -- 172.21.2.104:0/1797730321 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f533c0698e0 con 0x7f533c067180 2021-07-11T04:01:13.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.343+0000 7f532a7fc700 1 -- 172.21.2.104:0/1797730321 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f532c018030 con 0x7f533c067180 2021-07-11T04:01:13.346 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:13.346 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":118,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:13.031308+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:01:10.890456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-07-11T04:01:12.008882+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:13.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 -- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f531c053160 msgr2=0x7f531c055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f531c053160 0x7f531c055620 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f5338002f70 tx=0x7f5338006a80).stop 2021-07-11T04:01:13.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 -- 172.21.2.104:0/1797730321 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f533c067180 msgr2=0x7f533c0ad5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f533c067180 0x7f533c0ad5f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f533c0b4da0 tx=0x7f532c007c00).stop 2021-07-11T04:01:13.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 -- 172.21.2.104:0/1797730321 shutdown_connections 2021-07-11T04:01:13.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f533c0b1970 0x7f533c0ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f533c0a76b0 0x7f533c0adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f531c053160 0x7f531c055620 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:13.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 --2- 172.21.2.104:0/1797730321 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f533c067180 0x7f533c0ad5f0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:13.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 -- 172.21.2.104:0/1797730321 >> 172.21.2.104:0/1797730321 conn(0x7f533c0b53e0 msgr2=0x7f533c05fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:13.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 -- 172.21.2.104:0/1797730321 shutdown_connections 2021-07-11T04:01:13.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.347+0000 7f5343588700 1 -- 172.21.2.104:0/1797730321 wait complete. 2021-07-11T04:01:13.353 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T04:01:13.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.403+0000 7ff105282700 1 Processor -- start 2021-07-11T04:01:13.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.403+0000 7ff105282700 1 -- start start 2021-07-11T04:01:13.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.403+0000 7ff105282700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1000b1ec0 0x7ff1000b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff1000affb0 0x7ff1000b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff1000a70c0 0x7ff100141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1000b2790 con 0x7ff1000a70c0 2021-07-11T04:01:13.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1000b03f0 con 0x7ff1000affb0 2021-07-11T04:01:13.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1000b0650 con 0x7ff1000b1ec0 2021-07-11T04:01:13.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff1000a70c0 0x7ff100141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:13.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff1000a70c0 0x7ff100141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49386/0 (socket says 172.21.2.104:49386) 2021-07-11T04:01:13.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 -- 172.21.2.104:0/2133550221 learned_addr learned my addr 172.21.2.104:0/2133550221 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:13.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 -- 172.21.2.104:0/2133550221 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1000b1ec0 msgr2=0x7ff1000b12d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:13.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 --2- 172.21.2.104:0/2133550221 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1000b1ec0 0x7ff1000b12d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 -- 172.21.2.104:0/2133550221 >> v2:172.21.2.105:3300/0 conn(0x7ff1000affb0 msgr2=0x7ff1000b22a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:13.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 --2- 172.21.2.104:0/2133550221 >> v2:172.21.2.105:3300/0 conn(0x7ff1000affb0 0x7ff1000b22a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 -- 172.21.2.104:0/2133550221 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1000b08f0 con 0x7ff1000a70c0 2021-07-11T04:01:13.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 --2- 172.21.2.104:0/2133550221 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1000a70c0 0x7ff100141250 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7ff0f80032f0 tx=0x7ff0f8009290).ready entity=mon.0 client_cookie=32288a8a3e9da249 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:13.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0fdffb700 1 -- 172.21.2.104:0/2133550221 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0f800b870 con 0x7ff1000a70c0 2021-07-11T04:01:13.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0fdffb700 1 -- 172.21.2.104:0/2133550221 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f800b9d0 con 0x7ff1000a70c0 2021-07-11T04:01:13.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0fdffb700 1 -- 172.21.2.104:0/2133550221 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0f800bb80 con 0x7ff1000a70c0 2021-07-11T04:01:13.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- 172.21.2.104:0/2133550221 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1000a70c0 msgr2=0x7ff100141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 --2- 172.21.2.104:0/2133550221 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1000a70c0 0x7ff100141250 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7ff0f80032f0 tx=0x7ff0f8009290).stop 2021-07-11T04:01:13.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- 172.21.2.104:0/2133550221 shutdown_connections 2021-07-11T04:01:13.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 --2- 172.21.2.104:0/2133550221 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1000a70c0 0x7ff100141250 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:13.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 --2- 172.21.2.104:0/2133550221 >> v2:172.21.2.105:3300/0 conn(0x7ff1000affb0 0x7ff1000b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 --2- 172.21.2.104:0/2133550221 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1000b1ec0 0x7ff1000b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- 172.21.2.104:0/2133550221 >> 172.21.2.104:0/2133550221 conn(0x7ff10005f6b0 msgr2=0x7ff10005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:13.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- 172.21.2.104:0/2133550221 shutdown_connections 2021-07-11T04:01:13.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- 172.21.2.104:0/2133550221 wait complete. 2021-07-11T04:01:13.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 Processor -- start 2021-07-11T04:01:13.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- start start 2021-07-11T04:01:13.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1000a70c0 0x7ff10015aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1000affb0 0x7ff1001401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1000b1ec0 0x7ff100156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff100145420 con 0x7ff1000b1ec0 2021-07-11T04:01:13.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10015a0b0 con 0x7ff1000a70c0 2021-07-11T04:01:13.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10015a350 con 0x7ff1000affb0 2021-07-11T04:01:13.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1000b1ec0 0x7ff100156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:13.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1000b1ec0 0x7ff100156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49390/0 (socket says 172.21.2.104:49390) 2021-07-11T04:01:13.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 -- 172.21.2.104:0/3440131636 learned_addr learned my addr 172.21.2.104:0/3440131636 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:13.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 -- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1000affb0 msgr2=0x7ff1001401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1000affb0 0x7ff1001401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 -- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1000a70c0 msgr2=0x7ff10015aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:13.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1000a70c0 0x7ff10015aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:13.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 -- 172.21.2.104:0/3440131636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f8009530 con 0x7ff1000b1ec0 2021-07-11T04:01:13.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0ff7fe700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1000b1ec0 0x7ff100156a60 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7ff0f80032f0 tx=0x7ff0f8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:13.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0f7fff700 1 -- 172.21.2.104:0/3440131636 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0f8011c80 con 0x7ff1000b1ec0 2021-07-11T04:01:13.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0f7fff700 1 -- 172.21.2.104:0/3440131636 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8011de0 con 0x7ff1000b1ec0 2021-07-11T04:01:13.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff0f7fff700 1 -- 172.21.2.104:0/3440131636 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0f8020660 con 0x7ff1000b1ec0 2021-07-11T04:01:13.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- 172.21.2.104:0/3440131636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff10015a5f0 con 0x7ff1000b1ec0 2021-07-11T04:01:13.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.407+0000 7ff105282700 1 -- 172.21.2.104:0/3440131636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1000aabe0 con 0x7ff1000b1ec0 2021-07-11T04:01:13.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.411+0000 7ff105282700 1 -- 172.21.2.104:0/3440131636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff10015a780 con 0x7ff1000b1ec0 2021-07-11T04:01:13.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.411+0000 7ff0f7fff700 1 -- 172.21.2.104:0/3440131636 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff0f8029070 con 0x7ff1000b1ec0 2021-07-11T04:01:13.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.411+0000 7ff0f7fff700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e4053160 0x7ff0e4055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:13.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.411+0000 7ff0f7fff700 1 -- 172.21.2.104:0/3440131636 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff0f806c280 con 0x7ff1000b1ec0 2021-07-11T04:01:13.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.411+0000 7ff0feffd700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e4053160 0x7ff0e4055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:13.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.411+0000 7ff0feffd700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e4053160 0x7ff0e4055620 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7ff0e8000ed0 tx=0x7ff0e8006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:13.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.411+0000 7ff0f7fff700 1 -- 172.21.2.104:0/3440131636 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff0f8043560 con 0x7ff1000b1ec0 2021-07-11T04:01:13.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:13.519+0000 7ff105282700 1 -- 172.21.2.104:0/3440131636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff100057350 con 0x7ff1000b1ec0 2021-07-11T04:01:14.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff0f7fff700 1 -- 172.21.2.104:0/3440131636 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff0f8015020 con 0x7ff1000b1ec0 2021-07-11T04:01:14.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 -- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e4053160 msgr2=0x7ff0e4055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e4053160 0x7ff0e4055620 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7ff0e8000ed0 tx=0x7ff0e8006930).stop 2021-07-11T04:01:14.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 -- 172.21.2.104:0/3440131636 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1000b1ec0 msgr2=0x7ff100156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1000b1ec0 0x7ff100156a60 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7ff0f80032f0 tx=0x7ff0f8016000).stop 2021-07-11T04:01:14.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 -- 172.21.2.104:0/3440131636 shutdown_connections 2021-07-11T04:01:14.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e4053160 0x7ff0e4055620 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1000b1ec0 0x7ff100156a60 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff1000affb0 0x7ff1001401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 --2- 172.21.2.104:0/3440131636 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1000a70c0 0x7ff10015aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 -- 172.21.2.104:0/3440131636 >> 172.21.2.104:0/3440131636 conn(0x7ff10005f6b0 msgr2=0x7ff10013e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:14.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 -- 172.21.2.104:0/3440131636 shutdown_connections 2021-07-11T04:01:14.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.351+0000 7ff105282700 1 -- 172.21.2.104:0/3440131636 wait complete. 2021-07-11T04:01:14.360 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:14.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f483d34f700 1 Processor -- start 2021-07-11T04:01:14.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f483d34f700 1 -- start start 2021-07-11T04:01:14.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f483d34f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4838067180 0x7f483805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f483d34f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f48380b1970 0x7f483805fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f483d34f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48380a7a90 con 0x7f48380a76b0 2021-07-11T04:01:14.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f483d34f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4838147620 con 0x7f4838067180 2021-07-11T04:01:14.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f483d34f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48381478c0 con 0x7f48380b1970 2021-07-11T04:01:14.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f4836ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f48380a76b0 0x7f4838066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46164/0 (socket says 172.21.2.104:46164) 2021-07-11T04:01:14.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f4836ffd700 1 -- 172.21.2.104:0/2550549227 learned_addr learned my addr 172.21.2.104:0/2550549227 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:14.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f48367fc700 1 --2- 172.21.2.104:0/2550549227 >> v2:172.21.2.104:3300/0 conn(0x7f4838067180 0x7f483805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f48367fc700 1 -- 172.21.2.104:0/2550549227 >> v2:172.21.2.105:3300/0 conn(0x7f48380b1970 msgr2=0x7f483805fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:14.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f48367fc700 1 --2- 172.21.2.104:0/2550549227 >> v2:172.21.2.105:3300/0 conn(0x7f48380b1970 0x7f483805fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f48367fc700 1 -- 172.21.2.104:0/2550549227 >> v1:172.21.2.104:6789/0 conn(0x7f48380a76b0 legacy=0x7f4838066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f48367fc700 1 -- 172.21.2.104:0/2550549227 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4838147b60 con 0x7f4838067180 2021-07-11T04:01:14.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f48367fc700 1 --2- 172.21.2.104:0/2550549227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4838067180 0x7f483805f6b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f4828002f70 tx=0x7f4828006df0).ready entity=mon.0 client_cookie=b49c0bfe8f417984 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f4835ffb700 1 -- 172.21.2.104:0/2550549227 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f482800d5c0 con 0x7f4838067180 2021-07-11T04:01:14.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.415+0000 7f4835ffb700 1 -- 172.21.2.104:0/2550549227 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4828000e90 con 0x7f4838067180 2021-07-11T04:01:14.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4835ffb700 1 -- 172.21.2.104:0/2550549227 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f482800d950 con 0x7f4838067180 2021-07-11T04:01:14.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- 172.21.2.104:0/2550549227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4838067180 msgr2=0x7f483805f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 --2- 172.21.2.104:0/2550549227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4838067180 0x7f483805f6b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7f4828002f70 tx=0x7f4828006df0).stop 2021-07-11T04:01:14.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- 172.21.2.104:0/2550549227 shutdown_connections 2021-07-11T04:01:14.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 --2- 172.21.2.104:0/2550549227 >> v2:172.21.2.105:3300/0 conn(0x7f48380b1970 0x7f483805fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 --2- 172.21.2.104:0/2550549227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4838067180 0x7f483805f6b0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- 172.21.2.104:0/2550549227 >> 172.21.2.104:0/2550549227 conn(0x7f48380b53e0 msgr2=0x7f48380b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:14.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- 172.21.2.104:0/2550549227 shutdown_connections 2021-07-11T04:01:14.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- 172.21.2.104:0/2550549227 wait complete. 2021-07-11T04:01:14.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 Processor -- start 2021-07-11T04:01:14.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- start start 2021-07-11T04:01:14.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4838067180 0x7f4838143bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48380a76b0 0x7f483815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48380b1970 0x7f4838156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48380a9ab0 con 0x7f4838067180 2021-07-11T04:01:14.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4838156640 con 0x7f48380a76b0 2021-07-11T04:01:14.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4838156860 con 0x7f48380b1970 2021-07-11T04:01:14.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4836ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4838067180 0x7f4838143bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4836ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4838067180 0x7f4838143bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49404/0 (socket says 172.21.2.104:49404) 2021-07-11T04:01:14.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4836ffd700 1 -- 172.21.2.104:0/2474740159 learned_addr learned my addr 172.21.2.104:0/2474740159 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:14.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4836ffd700 1 -- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48380b1970 msgr2=0x7f4838156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4836ffd700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48380b1970 0x7f4838156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4836ffd700 1 -- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48380a76b0 msgr2=0x7f483815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4836ffd700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48380a76b0 0x7f483815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4836ffd700 1 -- 172.21.2.104:0/2474740159 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482800b180 con 0x7f4838067180 2021-07-11T04:01:14.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4836ffd700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4838067180 0x7f4838143bb0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f483806a0e0 tx=0x7f482c007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4827fff700 1 -- 172.21.2.104:0/2474740159 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f482c00ce30 con 0x7f4838067180 2021-07-11T04:01:14.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4827fff700 1 -- 172.21.2.104:0/2474740159 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c0072e0 con 0x7f4838067180 2021-07-11T04:01:14.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- 172.21.2.104:0/2474740159 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483815ab20 con 0x7f4838067180 2021-07-11T04:01:14.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4827fff700 1 -- 172.21.2.104:0/2474740159 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f482c005800 con 0x7f4838067180 2021-07-11T04:01:14.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- 172.21.2.104:0/2474740159 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48380a9d40 con 0x7f4838067180 2021-07-11T04:01:14.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f483d34f700 1 -- 172.21.2.104:0/2474740159 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4838057350 con 0x7f4838067180 2021-07-11T04:01:14.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4827fff700 1 -- 172.21.2.104:0/2474740159 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f482c010070 con 0x7f4838067180 2021-07-11T04:01:14.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4827fff700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4814053160 0x7f4814055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.419+0000 7f4827fff700 1 -- 172.21.2.104:0/2474740159 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f482c06ab70 con 0x7f4838067180 2021-07-11T04:01:14.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.423+0000 7f48367fc700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4814053160 0x7f4814055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.423+0000 7f48367fc700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4814053160 0x7f4814055620 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f4828002f70 tx=0x7f4828014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.423+0000 7f4827fff700 1 -- 172.21.2.104:0/2474740159 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f482c041eb0 con 0x7f4838067180 2021-07-11T04:01:14.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 -- 172.21.2.104:0/2474740159 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4838156a20 con 0x7f4838067180 2021-07-11T04:01:14.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f4827fff700 1 -- 172.21.2.104:0/2474740159 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f4838156a20 con 0x7f4838067180 2021-07-11T04:01:14.533 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:14.533 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3151473850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3151473850},{"type":"v1","addr":"172.21.2.105:6835","nonce":3151473850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/4259840781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":4259840781},{"type":"v1","addr":"172.21.2.104:6834","nonce":4259840781}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:13.031418+0000","modified":"2021-07-11T04:01:14.344351+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7620":{"gid":7620,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/1367521056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1367521056},{"type":"v1","addr":"172.21.2.104:6835","nonce":1367521056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-07-11T04:01:14.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 -- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4814053160 msgr2=0x7f4814055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4814053160 0x7f4814055620 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f4828002f70 tx=0x7f4828014000).stop 2021-07-11T04:01:14.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 -- 172.21.2.104:0/2474740159 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4838067180 msgr2=0x7f4838143bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4838067180 0x7f4838143bb0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f483806a0e0 tx=0x7f482c007c40).stop 2021-07-11T04:01:14.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 -- 172.21.2.104:0/2474740159 shutdown_connections 2021-07-11T04:01:14.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48380b1970 0x7f4838156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48380a76b0 0x7f483815a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4814053160 0x7f4814055620 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 --2- 172.21.2.104:0/2474740159 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4838067180 0x7f4838143bb0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 -- 172.21.2.104:0/2474740159 >> 172.21.2.104:0/2474740159 conn(0x7f48380b53e0 msgr2=0x7f4838148c00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:14.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 -- 172.21.2.104:0/2474740159 shutdown_connections 2021-07-11T04:01:14.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.531+0000 7f483d34f700 1 -- 172.21.2.104:0/2474740159 wait complete. 2021-07-11T04:01:14.537 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 57 2021-07-11T04:01:14.541 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:14.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.591+0000 7fc378841700 1 Processor -- start 2021-07-11T04:01:14.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.591+0000 7fc378841700 1 -- start start 2021-07-11T04:01:14.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.591+0000 7fc378841700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc3740a76b0 0x7fc374066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.591+0000 7fc378841700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc3740b1970 0x7fc374066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.591+0000 7fc378841700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3740b0060 0x7fc37405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.591+0000 7fc378841700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc374067450 con 0x7fc3740b1970 2021-07-11T04:01:14.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.591+0000 7fc378841700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3740a7a90 con 0x7fc3740a76b0 2021-07-11T04:01:14.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.591+0000 7fc378841700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc374147620 con 0x7fc3740b0060 2021-07-11T04:01:14.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.591+0000 7fc371d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc3740b1970 0x7fc374066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.591+0000 7fc371d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc3740b1970 0x7fc374066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49410/0 (socket says 172.21.2.104:49410) 2021-07-11T04:01:14.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.591+0000 7fc371d9b700 1 -- 172.21.2.104:0/1897372295 learned_addr learned my addr 172.21.2.104:0/1897372295 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:14.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc371d9b700 1 -- 172.21.2.104:0/1897372295 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3740b0060 msgr2=0x7fc37405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:14.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc371d9b700 1 --2- 172.21.2.104:0/1897372295 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3740b0060 0x7fc37405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc371d9b700 1 -- 172.21.2.104:0/1897372295 >> v2:172.21.2.105:3300/0 conn(0x7fc3740a76b0 msgr2=0x7fc374066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:14.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc371d9b700 1 --2- 172.21.2.104:0/1897372295 >> v2:172.21.2.105:3300/0 conn(0x7fc3740a76b0 0x7fc374066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc371d9b700 1 -- 172.21.2.104:0/1897372295 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3741478c0 con 0x7fc3740b1970 2021-07-11T04:01:14.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc371d9b700 1 --2- 172.21.2.104:0/1897372295 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3740b1970 0x7fc374066f60 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7fc36c0032f0 tx=0x7fc36c009290).ready entity=mon.0 client_cookie=c778ff8e93c327ca server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37159a700 1 -- 172.21.2.104:0/1897372295 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc36c00b870 con 0x7fc3740b1970 2021-07-11T04:01:14.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37159a700 1 -- 172.21.2.104:0/1897372295 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c00b9d0 con 0x7fc3740b1970 2021-07-11T04:01:14.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37159a700 1 -- 172.21.2.104:0/1897372295 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc36c00bb40 con 0x7fc3740b1970 2021-07-11T04:01:14.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 -- 172.21.2.104:0/1897372295 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3740b1970 msgr2=0x7fc374066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 --2- 172.21.2.104:0/1897372295 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3740b1970 0x7fc374066f60 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7fc36c0032f0 tx=0x7fc36c009290).stop 2021-07-11T04:01:14.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 -- 172.21.2.104:0/1897372295 shutdown_connections 2021-07-11T04:01:14.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 --2- 172.21.2.104:0/1897372295 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3740b0060 0x7fc37405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 --2- 172.21.2.104:0/1897372295 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3740b1970 0x7fc374066f60 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 --2- 172.21.2.104:0/1897372295 >> v2:172.21.2.105:3300/0 conn(0x7fc3740a76b0 0x7fc374066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 -- 172.21.2.104:0/1897372295 >> 172.21.2.104:0/1897372295 conn(0x7fc3740b53e0 msgr2=0x7fc3740b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:14.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 -- 172.21.2.104:0/1897372295 shutdown_connections 2021-07-11T04:01:14.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 -- 172.21.2.104:0/1897372295 wait complete. 2021-07-11T04:01:14.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 Processor -- start 2021-07-11T04:01:14.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 -- start start 2021-07-11T04:01:14.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3740a76b0 0x7fc3740abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3740b0060 0x7fc37414e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3740b1970 0x7fc3740aae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740ae060 con 0x7fc3740a76b0 2021-07-11T04:01:14.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3740ad960 con 0x7fc3740b0060 2021-07-11T04:01:14.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37414e9b0 con 0x7fc3740b1970 2021-07-11T04:01:14.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3740a76b0 0x7fc3740abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3740a76b0 0x7fc3740abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49418/0 (socket says 172.21.2.104:49418) 2021-07-11T04:01:14.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37259c700 1 -- 172.21.2.104:0/3766198194 learned_addr learned my addr 172.21.2.104:0/3766198194 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:14.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc371d9b700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3740b0060 0x7fc37414e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc372d9d700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3740b1970 0x7fc3740aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37259c700 1 -- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3740b1970 msgr2=0x7fc3740aae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37259c700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3740b1970 0x7fc3740aae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37259c700 1 -- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3740b0060 msgr2=0x7fc37414e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37259c700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3740b0060 0x7fc37414e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37259c700 1 -- 172.21.2.104:0/3766198194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c009530 con 0x7fc3740a76b0 2021-07-11T04:01:14.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc37259c700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3740a76b0 0x7fc3740abf90 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7fc368002f70 tx=0x7fc368006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc3637fe700 1 -- 172.21.2.104:0/3766198194 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc368013460 con 0x7fc3740a76b0 2021-07-11T04:01:14.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc3637fe700 1 -- 172.21.2.104:0/3766198194 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3680135c0 con 0x7fc3740a76b0 2021-07-11T04:01:14.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 -- 172.21.2.104:0/3766198194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37414e710 con 0x7fc3740a76b0 2021-07-11T04:01:14.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc3637fe700 1 -- 172.21.2.104:0/3766198194 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc368013920 con 0x7fc3740a76b0 2021-07-11T04:01:14.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.595+0000 7fc378841700 1 -- 172.21.2.104:0/3766198194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3740af570 con 0x7fc3740a76b0 2021-07-11T04:01:14.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.599+0000 7fc378841700 1 -- 172.21.2.104:0/3766198194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3740addc0 con 0x7fc3740a76b0 2021-07-11T04:01:14.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.599+0000 7fc3637fe700 1 -- 172.21.2.104:0/3766198194 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc36800f070 con 0x7fc3740a76b0 2021-07-11T04:01:14.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.599+0000 7fc3637fe700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc350053180 0x7fc350055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.599+0000 7fc3637fe700 1 -- 172.21.2.104:0/3766198194 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc36806af20 con 0x7fc3740a76b0 2021-07-11T04:01:14.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.599+0000 7fc371d9b700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc350053180 0x7fc350055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.599+0000 7fc371d9b700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc350053180 0x7fc350055640 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fc36c018000 tx=0x7fc36c016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.599+0000 7fc3637fe700 1 -- 172.21.2.104:0/3766198194 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc3740addc0 con 0x7fc3740a76b0 2021-07-11T04:01:14.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 -- 172.21.2.104:0/3766198194 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc374057350 con 0x7fc3740a76b0 2021-07-11T04:01:14.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc3637fe700 1 -- 172.21.2.104:0/3766198194 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16052 (secure 0 0 0) 0x7fc368042150 con 0x7fc3740a76b0 2021-07-11T04:01:14.684 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:14.684 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":119,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:14.034771+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:01:10.890456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-07-11T04:01:12.008882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:14.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 -- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc350053180 msgr2=0x7fc350055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc350053180 0x7fc350055640 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fc36c018000 tx=0x7fc36c016000).stop 2021-07-11T04:01:14.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 -- 172.21.2.104:0/3766198194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3740a76b0 msgr2=0x7fc3740abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3740a76b0 0x7fc3740abf90 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7fc368002f70 tx=0x7fc368006ec0).stop 2021-07-11T04:01:14.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 -- 172.21.2.104:0/3766198194 shutdown_connections 2021-07-11T04:01:14.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3740b1970 0x7fc3740aae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3740b0060 0x7fc37414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc350053180 0x7fc350055640 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 --2- 172.21.2.104:0/3766198194 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3740a76b0 0x7fc3740abf90 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 -- 172.21.2.104:0/3766198194 >> 172.21.2.104:0/3766198194 conn(0x7fc3740b53e0 msgr2=0x7fc37405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:14.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 -- 172.21.2.104:0/3766198194 shutdown_connections 2021-07-11T04:01:14.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.683+0000 7fc378841700 1 -- 172.21.2.104:0/3766198194 wait complete. 2021-07-11T04:01:14.691 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:01:14.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1c231c700 1 Processor -- start 2021-07-11T04:01:14.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1c231c700 1 -- start start 2021-07-11T04:01:14.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1c231c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1bc0a76b0 0x7fb1bc066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1c231c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb1bc0b1970 0x7fb1bc066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1c231c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1bc067450 con 0x7fb1bc0b0060 2021-07-11T04:01:14.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1c231c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1bc0a7a90 con 0x7fb1bc0b1970 2021-07-11T04:01:14.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1c231c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc147620 con 0x7fb1bc0a76b0 2021-07-11T04:01:14.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1bb7fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb1bc0b1970 0x7fb1bc066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1bb7fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb1bc0b1970 0x7fb1bc066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:56986/0 (socket says 172.21.2.104:56986) 2021-07-11T04:01:14.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1bb7fe700 1 -- 172.21.2.104:0/3032380038 learned_addr learned my addr 172.21.2.104:0/3032380038 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:14.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1bbfff700 1 --2- 172.21.2.104:0/3032380038 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1bc0a76b0 0x7fb1bc066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1bb7fe700 1 -- 172.21.2.104:0/3032380038 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1bc0a76b0 msgr2=0x7fb1bc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:14.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1bb7fe700 1 --2- 172.21.2.104:0/3032380038 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1bc0a76b0 0x7fb1bc066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1bb7fe700 1 -- 172.21.2.104:0/3032380038 >> v1:172.21.2.105:6789/0 conn(0x7fb1bc0b0060 legacy=0x7fb1bc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1bb7fe700 1 -- 172.21.2.104:0/3032380038 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc1478c0 con 0x7fb1bc0b1970 2021-07-11T04:01:14.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1bb7fe700 1 --2- 172.21.2.104:0/3032380038 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1bc0b1970 0x7fb1bc066f60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fb1a4002f70 tx=0x7fb1a4006eb0).ready entity=mon.1 client_cookie=61e2843d35271d48 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1baffd700 1 -- 172.21.2.104:0/3032380038 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1a400d570 con 0x7fb1bc0b1970 2021-07-11T04:01:14.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1baffd700 1 -- 172.21.2.104:0/3032380038 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a4000e90 con 0x7fb1bc0b1970 2021-07-11T04:01:14.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1c231c700 1 -- 172.21.2.104:0/3032380038 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1bc0b1970 msgr2=0x7fb1bc066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.747+0000 7fb1c231c700 1 --2- 172.21.2.104:0/3032380038 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1bc0b1970 0x7fb1bc066f60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fb1a4002f70 tx=0x7fb1a4006eb0).stop 2021-07-11T04:01:14.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 -- 172.21.2.104:0/3032380038 shutdown_connections 2021-07-11T04:01:14.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 --2- 172.21.2.104:0/3032380038 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1bc0b1970 0x7fb1bc066f60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 --2- 172.21.2.104:0/3032380038 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1bc0a76b0 0x7fb1bc066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 -- 172.21.2.104:0/3032380038 >> 172.21.2.104:0/3032380038 conn(0x7fb1bc0b53e0 msgr2=0x7fb1bc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:14.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 -- 172.21.2.104:0/3032380038 shutdown_connections 2021-07-11T04:01:14.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 -- 172.21.2.104:0/3032380038 wait complete. 2021-07-11T04:01:14.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 Processor -- start 2021-07-11T04:01:14.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 -- start start 2021-07-11T04:01:14.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1bc0a76b0 0x7fb1bc0adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1bc0b0060 0x7fb1bc156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1bc0b1970 0x7fb1bc156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0ad1c0 con 0x7fb1bc0b1970 2021-07-11T04:01:14.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0af4b0 con 0x7fb1bc0b0060 2021-07-11T04:01:14.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc0af750 con 0x7fb1bc0a76b0 2021-07-11T04:01:14.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c08b9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1bc0b1970 0x7fb1bc156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c08b9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1bc0b1970 0x7fb1bc156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49434/0 (socket says 172.21.2.104:49434) 2021-07-11T04:01:14.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c08b9700 1 -- 172.21.2.104:0/112318958 learned_addr learned my addr 172.21.2.104:0/112318958 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:14.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c08b9700 1 -- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1bc0a76b0 msgr2=0x7fb1bc0adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c08b9700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1bc0a76b0 0x7fb1bc0adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c08b9700 1 -- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1bc0b0060 msgr2=0x7fb1bc156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c08b9700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1bc0b0060 0x7fb1bc156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c08b9700 1 -- 172.21.2.104:0/112318958 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a400b1e0 con 0x7fb1bc0b1970 2021-07-11T04:01:14.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c08b9700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1bc0b1970 0x7fb1bc156930 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7fb1bc0b6460 tx=0x7fb1b4007f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1b97fa700 1 -- 172.21.2.104:0/112318958 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b400e4a0 con 0x7fb1bc0b1970 2021-07-11T04:01:14.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1b97fa700 1 -- 172.21.2.104:0/112318958 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b400e600 con 0x7fb1bc0b1970 2021-07-11T04:01:14.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1b97fa700 1 -- 172.21.2.104:0/112318958 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1b400e960 con 0x7fb1bc0b1970 2021-07-11T04:01:14.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 -- 172.21.2.104:0/112318958 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc0a9d40 con 0x7fb1bc0b1970 2021-07-11T04:01:14.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 -- 172.21.2.104:0/112318958 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc156e20 con 0x7fb1bc0b1970 2021-07-11T04:01:14.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1c231c700 1 -- 172.21.2.104:0/112318958 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1bc0ae800 con 0x7fb1bc0b1970 2021-07-11T04:01:14.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1b97fa700 1 -- 172.21.2.104:0/112318958 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb1b4019030 con 0x7fb1bc0b1970 2021-07-11T04:01:14.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1b97fa700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1a00531a0 0x7fb1a0055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.751+0000 7fb1b97fa700 1 -- 172.21.2.104:0/112318958 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb1b406ad20 con 0x7fb1bc0b1970 2021-07-11T04:01:14.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.755+0000 7fb1bbfff700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1a00531a0 0x7fb1a0055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.755+0000 7fb1bbfff700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1a00531a0 0x7fb1a0055660 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fb1ac00c000 tx=0x7fb1ac006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.755+0000 7fb1b97fa700 1 -- 172.21.2.104:0/112318958 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb1b4041f50 con 0x7fb1bc0b1970 2021-07-11T04:01:14.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.867+0000 7fb1c231c700 1 -- 172.21.2.104:0/112318958 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fb1bc057350 con 0x7fb1bc0b1970 2021-07-11T04:01:14.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1b97fa700 1 -- 172.21.2.104:0/112318958 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v30) v1 ==== 184+0+0 (secure 0 0 0) 0x7fb1b4045570 con 0x7fb1bc0b1970 2021-07-11T04:01:14.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 -- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1a00531a0 msgr2=0x7fb1a0055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1a00531a0 0x7fb1a0055660 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7fb1ac00c000 tx=0x7fb1ac006df0).stop 2021-07-11T04:01:14.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 -- 172.21.2.104:0/112318958 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1bc0b1970 msgr2=0x7fb1bc156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1bc0b1970 0x7fb1bc156930 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7fb1bc0b6460 tx=0x7fb1b4007f20).stop 2021-07-11T04:01:14.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 -- 172.21.2.104:0/112318958 shutdown_connections 2021-07-11T04:01:14.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1a00531a0 0x7fb1a0055660 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1bc0b1970 0x7fb1bc156930 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1bc0b0060 0x7fb1bc156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 --2- 172.21.2.104:0/112318958 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1bc0a76b0 0x7fb1bc0adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:14.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 -- 172.21.2.104:0/112318958 >> 172.21.2.104:0/112318958 conn(0x7fb1bc0b53e0 msgr2=0x7fb1bc05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:14.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 -- 172.21.2.104:0/112318958 shutdown_connections 2021-07-11T04:01:14.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.871+0000 7fb1c231c700 1 -- 172.21.2.104:0/112318958 wait complete. 2021-07-11T04:01:14.877 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T04:01:14.880 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:01:14.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a98269700 1 Processor -- start 2021-07-11T04:01:14.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a98269700 1 -- start start 2021-07-11T04:01:14.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a98269700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1a900a76b0 0x7f1a90066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a98269700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b0060 0x7f1a9005fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a98269700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a900a7a90 con 0x7f1a900b1970 2021-07-11T04:01:14.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a98269700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a90147620 con 0x7f1a900a76b0 2021-07-11T04:01:14.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a98269700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a901478c0 con 0x7f1a900b0060 2021-07-11T04:01:14.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a95804700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f1a900b1970 0x7f1a9005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:45578/0 (socket says 172.21.2.104:45578) 2021-07-11T04:01:14.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a95804700 1 -- 172.21.2.104:0/2737262479 learned_addr learned my addr 172.21.2.104:0/2737262479 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:14.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a96806700 1 --2- 172.21.2.104:0/2737262479 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b0060 0x7f1a9005fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a96005700 1 --2- 172.21.2.104:0/2737262479 >> v2:172.21.2.105:3300/0 conn(0x7f1a900a76b0 0x7f1a90066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a95003700 1 -- 172.21.2.104:0/2737262479 <== mon.1 v1:172.21.2.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686335000 0 0) 0x7f1a900a7a90 con 0x7f1a900b1970 2021-07-11T04:01:14.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a95003700 1 -- 172.21.2.104:0/2737262479 --> v1:172.21.2.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a78003260 con 0x7f1a900b1970 2021-07-11T04:01:14.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a96806700 1 -- 172.21.2.104:0/2737262479 >> v2:172.21.2.105:3300/0 conn(0x7f1a900a76b0 msgr2=0x7f1a90066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:14.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a96806700 1 --2- 172.21.2.104:0/2737262479 >> v2:172.21.2.105:3300/0 conn(0x7f1a900a76b0 0x7f1a90066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a96806700 1 -- 172.21.2.104:0/2737262479 >> v1:172.21.2.105:6789/0 conn(0x7f1a900b1970 legacy=0x7f1a9005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a96806700 1 -- 172.21.2.104:0/2737262479 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a90147b60 con 0x7f1a900b0060 2021-07-11T04:01:14.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.935+0000 7f1a96806700 1 --2- 172.21.2.104:0/2737262479 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b0060 0x7f1a9005fdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1a880032f0 tx=0x7f1a880091d0).ready entity=mon.2 client_cookie=4d23531edfc039af server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95003700 1 -- 172.21.2.104:0/2737262479 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8800b600 con 0x7f1a900b0060 2021-07-11T04:01:14.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95003700 1 -- 172.21.2.104:0/2737262479 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8800b760 con 0x7f1a900b0060 2021-07-11T04:01:14.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95003700 1 -- 172.21.2.104:0/2737262479 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8800b8d0 con 0x7f1a900b0060 2021-07-11T04:01:14.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 -- 172.21.2.104:0/2737262479 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b0060 msgr2=0x7f1a9005fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 --2- 172.21.2.104:0/2737262479 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b0060 0x7f1a9005fdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1a880032f0 tx=0x7f1a880091d0).stop 2021-07-11T04:01:14.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 -- 172.21.2.104:0/2737262479 shutdown_connections 2021-07-11T04:01:14.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 --2- 172.21.2.104:0/2737262479 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b0060 0x7f1a9005fdc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 --2- 172.21.2.104:0/2737262479 >> v2:172.21.2.105:3300/0 conn(0x7f1a900a76b0 0x7f1a90066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 -- 172.21.2.104:0/2737262479 >> 172.21.2.104:0/2737262479 conn(0x7f1a900b53e0 msgr2=0x7f1a900b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:14.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 -- 172.21.2.104:0/2737262479 shutdown_connections 2021-07-11T04:01:14.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 -- 172.21.2.104:0/2737262479 wait complete. 2021-07-11T04:01:14.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 Processor -- start 2021-07-11T04:01:14.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 -- start start 2021-07-11T04:01:14.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1a900a76b0 0x7f1a900abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1a900b0060 0x7f1a900ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b1970 0x7f1a900aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a900af0f0 con 0x7f1a900b0060 2021-07-11T04:01:14.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a900ab3f0 con 0x7f1a900a76b0 2021-07-11T04:01:14.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a900aa5d0 con 0x7f1a900b1970 2021-07-11T04:01:14.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95804700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1a900b0060 0x7f1a900ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95804700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1a900b0060 0x7f1a900ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49448/0 (socket says 172.21.2.104:49448) 2021-07-11T04:01:14.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95804700 1 -- 172.21.2.104:0/3862669953 learned_addr learned my addr 172.21.2.104:0/3862669953 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:14.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a96806700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b1970 0x7f1a900aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a96005700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1a900a76b0 0x7f1a900abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95804700 1 -- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b1970 msgr2=0x7f1a900aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95804700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b1970 0x7f1a900aec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95804700 1 -- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1a900a76b0 msgr2=0x7f1a900abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:14.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95804700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1a900a76b0 0x7f1a900abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:14.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95804700 1 -- 172.21.2.104:0/3862669953 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a88009470 con 0x7f1a900b0060 2021-07-11T04:01:14.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a96806700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b1970 0x7f1a900aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:01:14.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a95804700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1a900b0060 0x7f1a900ae710 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f1a84001110 tx=0x7f1a84007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a837fe700 1 -- 172.21.2.104:0/3862669953 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a8400ce80 con 0x7f1a900b0060 2021-07-11T04:01:14.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a837fe700 1 -- 172.21.2.104:0/3862669953 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a840070d0 con 0x7f1a900b0060 2021-07-11T04:01:14.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 -- 172.21.2.104:0/3862669953 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a900aa870 con 0x7f1a900b0060 2021-07-11T04:01:14.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a98269700 1 -- 172.21.2.104:0/3862669953 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a900af7e0 con 0x7f1a900b0060 2021-07-11T04:01:14.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.939+0000 7f1a837fe700 1 -- 172.21.2.104:0/3862669953 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1a84007430 con 0x7f1a900b0060 2021-07-11T04:01:14.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.943+0000 7f1a98269700 1 -- 172.21.2.104:0/3862669953 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a90057350 con 0x7f1a900b0060 2021-07-11T04:01:14.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.943+0000 7f1a837fe700 1 -- 172.21.2.104:0/3862669953 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1a8401a060 con 0x7f1a900b0060 2021-07-11T04:01:14.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.943+0000 7f1a837fe700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1a70053270 0x7f1a70055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:14.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.943+0000 7f1a837fe700 1 -- 172.21.2.104:0/3862669953 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1a8406ac80 con 0x7f1a900b0060 2021-07-11T04:01:14.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.943+0000 7f1a96005700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1a70053270 0x7f1a70055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:14.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.943+0000 7f1a96005700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1a70053270 0x7f1a70055730 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f1a8c00b000 tx=0x7f1a8c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:14.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:14.943+0000 7f1a837fe700 1 -- 172.21.2.104:0/3862669953 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1a84071080 con 0x7f1a900b0060 2021-07-11T04:01:15.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.055+0000 7f1a98269700 1 -- 172.21.2.104:0/3862669953 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f1a900698e0 con 0x7f1a900b0060 2021-07-11T04:01:15.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a837fe700 1 -- 172.21.2.104:0/3862669953 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v31) v1 ==== 184+0+0 (secure 0 0 0) 0x7f1a84018030 con 0x7f1a900b0060 2021-07-11T04:01:15.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 -- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1a70053270 msgr2=0x7f1a70055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1a70053270 0x7f1a70055730 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f1a8c00b000 tx=0x7f1a8c006df0).stop 2021-07-11T04:01:15.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 -- 172.21.2.104:0/3862669953 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1a900b0060 msgr2=0x7f1a900ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1a900b0060 0x7f1a900ae710 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f1a84001110 tx=0x7f1a84007a80).stop 2021-07-11T04:01:15.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 -- 172.21.2.104:0/3862669953 shutdown_connections 2021-07-11T04:01:15.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1a900b1970 0x7f1a900aec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1a70053270 0x7f1a70055730 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1a900b0060 0x7f1a900ae710 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 --2- 172.21.2.104:0/3862669953 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1a900a76b0 0x7f1a900abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 -- 172.21.2.104:0/3862669953 >> 172.21.2.104:0/3862669953 conn(0x7f1a900b53e0 msgr2=0x7f1a90148a00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:15.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 -- 172.21.2.104:0/3862669953 shutdown_connections 2021-07-11T04:01:15.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.059+0000 7f1a98269700 1 -- 172.21.2.104:0/3862669953 wait complete. 2021-07-11T04:01:15.065 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T04:01:15.068 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:15.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 Processor -- start 2021-07-11T04:01:15.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 -- start start 2021-07-11T04:01:15.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f28c80ad150 0x7f28c80b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80ac370 con 0x7f28c80a9da0 2021-07-11T04:01:15.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28c80aac40 con 0x7f28c80aa860 2021-07-11T04:01:15.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28c80abb50 con 0x7f28c80ad150 2021-07-11T04:01:15.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c6d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f28c80a9da0 0x7f28c8140340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46222/0 (socket says 172.21.2.104:46222) 2021-07-11T04:01:15.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c6d9d700 1 -- 172.21.2.104:0/4063730311 learned_addr learned my addr 172.21.2.104:0/4063730311 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:15.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262947797 0 0) 0x7f28c80ac370 con 0x7f28c80a9da0 2021-07-11T04:01:15.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28ac003260 con 0x7f28c80a9da0 2021-07-11T04:01:15.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c5d9b700 1 --2- 172.21.2.104:0/4063730311 >> v2:172.21.2.105:3300/0 conn(0x7f28c80ad150 0x7f28c80b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1096169735 0 0) 0x7f28ac003260 con 0x7f28c80a9da0 2021-07-11T04:01:15.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 >> v2:172.21.2.105:3300/0 conn(0x7f28c80ad150 msgr2=0x7f28c80b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:15.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 --2- 172.21.2.104:0/4063730311 >> v2:172.21.2.105:3300/0 conn(0x7f28c80ad150 0x7f28c80b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 >> v1:172.21.2.105:6789/0 conn(0x7f28c80aa860 legacy=0x7f28c80aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:15.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80b16b0 con 0x7f28c80a9da0 2021-07-11T04:01:15.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f28c00019e0 con 0x7f28c80a9da0 2021-07-11T04:01:15.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 >> v1:172.21.2.104:6789/0 conn(0x7f28c80a9da0 legacy=0x7f28c8140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 --2- 172.21.2.104:0/4063730311 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28ac003820 0x7f28ac003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 --2- 172.21.2.104:0/4063730311 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28ac004500 0x7f28ac004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 --2- 172.21.2.104:0/4063730311 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28ac004f70 0x7f28ac00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac00be20 con 0x7f28ac003820 2021-07-11T04:01:15.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac00bf90 con 0x7f28ac004500 2021-07-11T04:01:15.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ac00c150 con 0x7f28ac004f70 2021-07-11T04:01:15.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c659c700 1 --2- 172.21.2.104:0/4063730311 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28ac003820 0x7f28ac003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c659c700 1 -- 172.21.2.104:0/4063730311 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28ac004f70 msgr2=0x7f28ac00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:15.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c659c700 1 --2- 172.21.2.104:0/4063730311 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28ac004f70 0x7f28ac00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c659c700 1 -- 172.21.2.104:0/4063730311 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28ac004500 msgr2=0x7f28ac004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:15.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c659c700 1 --2- 172.21.2.104:0/4063730311 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28ac004500 0x7f28ac004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c659c700 1 -- 172.21.2.104:0/4063730311 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ac003260 con 0x7f28ac003820 2021-07-11T04:01:15.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c6d9d700 1 -- 172.21.2.104:0/4063730311 reap_dead start 2021-07-11T04:01:15.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c659c700 1 --2- 172.21.2.104:0/4063730311 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28ac003820 0x7f28ac003c20 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f28bc001fa0 tx=0x7f28bc007a00).ready entity=mon.0 client_cookie=28bd77b63b0769b2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28bc007070 con 0x7f28ac003820 2021-07-11T04:01:15.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28bc0071d0 con 0x7f28ac003820 2021-07-11T04:01:15.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28c559a700 1 -- 172.21.2.104:0/4063730311 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28bc006510 con 0x7f28ac003820 2021-07-11T04:01:15.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 -- 172.21.2.104:0/4063730311 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28ac003820 msgr2=0x7f28ac003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 --2- 172.21.2.104:0/4063730311 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28ac003820 0x7f28ac003c20 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f28bc001fa0 tx=0x7f28bc007a00).stop 2021-07-11T04:01:15.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 -- 172.21.2.104:0/4063730311 shutdown_connections 2021-07-11T04:01:15.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 --2- 172.21.2.104:0/4063730311 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28ac003820 0x7f28ac003c20 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 -- 172.21.2.104:0/4063730311 >> 172.21.2.104:0/4063730311 conn(0x7f28c80b64d0 msgr2=0x7f28c80b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:15.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 -- 172.21.2.104:0/4063730311 shutdown_connections 2021-07-11T04:01:15.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.119+0000 7f28ccd37700 1 -- 172.21.2.104:0/4063730311 wait complete. 2021-07-11T04:01:15.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28ccd37700 1 Processor -- start 2021-07-11T04:01:15.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28ccd37700 1 -- start start 2021-07-11T04:01:15.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28ccd37700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28ac003820 0x7f28c80a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28ccd37700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28c80a2cc0 0x7f28c8151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28ccd37700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28c80a69a0 0x7f28c8152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28ccd37700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80a36d0 con 0x7f28c80a69a0 2021-07-11T04:01:15.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28ccd37700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80a4a50 con 0x7f28ac003820 2021-07-11T04:01:15.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28ccd37700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28c80a3270 con 0x7f28c80a2cc0 2021-07-11T04:01:15.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28c6d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28c80a69a0 0x7f28c8152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28c6d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28c80a69a0 0x7f28c8152360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49466/0 (socket says 172.21.2.104:49466) 2021-07-11T04:01:15.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28c6d9d700 1 -- 172.21.2.104:0/3581230339 learned_addr learned my addr 172.21.2.104:0/3581230339 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:15.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28c6d9d700 1 -- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28c80a2cc0 msgr2=0x7f28c8151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28c6d9d700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28c80a2cc0 0x7f28c8151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28c6d9d700 1 -- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28ac003820 msgr2=0x7f28c80a64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28c6d9d700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28ac003820 0x7f28c80a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28c6d9d700 1 -- 172.21.2.104:0/3581230339 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80667a0 con 0x7f28c80a69a0 2021-07-11T04:01:15.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28c6d9d700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28c80a69a0 0x7f28c8152360 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f28c0001490 tx=0x7f28c0001ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28b77fe700 1 -- 172.21.2.104:0/3581230339 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c0004ab0 con 0x7f28c80a69a0 2021-07-11T04:01:15.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28b77fe700 1 -- 172.21.2.104:0/3581230339 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0004c10 con 0x7f28c80a69a0 2021-07-11T04:01:15.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28ccd37700 1 -- 172.21.2.104:0/3581230339 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c80a4830 con 0x7f28c80a69a0 2021-07-11T04:01:15.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28ccd37700 1 -- 172.21.2.104:0/3581230339 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c80a4c80 con 0x7f28c80a69a0 2021-07-11T04:01:15.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28b77fe700 1 -- 172.21.2.104:0/3581230339 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28c00070d0 con 0x7f28c80a69a0 2021-07-11T04:01:15.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28ccd37700 1 -- 172.21.2.104:0/3581230339 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c8057350 con 0x7f28c80a69a0 2021-07-11T04:01:15.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28b77fe700 1 -- 172.21.2.104:0/3581230339 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f28c001c070 con 0x7f28c80a69a0 2021-07-11T04:01:15.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28b77fe700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28a4053350 0x7f28a4055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28b77fe700 1 -- 172.21.2.104:0/3581230339 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f28c006a950 con 0x7f28c80a69a0 2021-07-11T04:01:15.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.123+0000 7f28c659c700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28a4053350 0x7f28a4055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.127+0000 7f28c659c700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28a4053350 0x7f28a4055810 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f28bc001fa0 tx=0x7f28bc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.127+0000 7f28b77fe700 1 -- 172.21.2.104:0/3581230339 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f28c0041c90 con 0x7f28c80a69a0 2021-07-11T04:01:15.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.231+0000 7f28ccd37700 1 -- 172.21.2.104:0/3581230339 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28c80698e0 con 0x7f28c80a69a0 2021-07-11T04:01:15.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28b77fe700 1 -- 172.21.2.104:0/3581230339 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f28c00452b0 con 0x7f28c80a69a0 2021-07-11T04:01:15.235 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:15.235 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3151473850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3151473850},{"type":"v1","addr":"172.21.2.105:6835","nonce":3151473850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/4259840781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":4259840781},{"type":"v1","addr":"172.21.2.104:6834","nonce":4259840781}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:13.031418+0000","modified":"2021-07-11T04:01:14.344351+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7620":{"gid":7620,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/1367521056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1367521056},{"type":"v1","addr":"172.21.2.104:6835","nonce":1367521056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-07-11T04:01:15.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 -- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28a4053350 msgr2=0x7f28a4055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28a4053350 0x7f28a4055810 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f28bc001fa0 tx=0x7f28bc015000).stop 2021-07-11T04:01:15.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 -- 172.21.2.104:0/3581230339 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28c80a69a0 msgr2=0x7f28c8152360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28c80a69a0 0x7f28c8152360 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f28c0001490 tx=0x7f28c0001ac0).stop 2021-07-11T04:01:15.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 -- 172.21.2.104:0/3581230339 shutdown_connections 2021-07-11T04:01:15.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28a4053350 0x7f28a4055810 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28c80a69a0 0x7f28c8152360 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28c80a2cc0 0x7f28c8151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 --2- 172.21.2.104:0/3581230339 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28ac003820 0x7f28c80a64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 -- 172.21.2.104:0/3581230339 >> 172.21.2.104:0/3581230339 conn(0x7f28c80b64d0 msgr2=0x7f28c80b5970 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:15.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 -- 172.21.2.104:0/3581230339 shutdown_connections 2021-07-11T04:01:15.239 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.235+0000 7f28ccd37700 1 -- 172.21.2.104:0/3581230339 wait complete. 2021-07-11T04:01:15.239 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 57 2021-07-11T04:01:15.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:01:13.031418+0000', 'modified': '2021-07-11T04:01:14.344351+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7620}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7620': {'gid': 7620, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/1367521056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 1367521056}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 1367521056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:01:15.544 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:01:15.544 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:01:15.610 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":7620,"want_state":"up:active","state":"up:active","rank_uptime":2.573644931,"mdsmap_epoch":57,"osdmap_epoch":119,"osdmap_epoch_barrier":119,"uptime":6.8274420539999996} 2021-07-11T04:01:15.611 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 7620, "mdsmap_epoch": 57, "osdmap_epoch": 119, "osdmap_epoch_barrier": 119, "rank_uptime": 2.573644931, "state": "up:active", "uptime": 6.827442054, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:01:15.611 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:15.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f853994e700 1 Processor -- start 2021-07-11T04:01:15.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f853994e700 1 -- start start 2021-07-11T04:01:15.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f853994e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ad150 0x7f85340b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f853994e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85340ac370 con 0x7f85340a9da0 2021-07-11T04:01:15.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f853994e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85340aac40 con 0x7f85340aa860 2021-07-11T04:01:15.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f853994e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85340abb50 con 0x7f85340ad150 2021-07-11T04:01:15.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85337fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f85340a9da0 0x7f8534140340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46244/0 (socket says 172.21.2.104:46244) 2021-07-11T04:01:15.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85337fe700 1 -- 172.21.2.104:0/3301514482 learned_addr learned my addr 172.21.2.104:0/3301514482 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:15.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321853156 0 0) 0x7f85340ac370 con 0x7f85340a9da0 2021-07-11T04:01:15.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8518003260 con 0x7f85340a9da0 2021-07-11T04:01:15.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85327fc700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ad150 0x7f85340b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2980082104 0 0) 0x7f8518003260 con 0x7f85340a9da0 2021-07-11T04:01:15.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ad150 msgr2=0x7f85340b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:15.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ad150 0x7f85340b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 >> v1:172.21.2.105:6789/0 conn(0x7f85340aa860 legacy=0x7f85340aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85340b16b0 con 0x7f85340a9da0 2021-07-11T04:01:15.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f852c0019e0 con 0x7f85340a9da0 2021-07-11T04:01:15.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 >> v1:172.21.2.104:6789/0 conn(0x7f85340a9da0 legacy=0x7f8534140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8518003780 0x7f8518003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ad150 0x7f8518004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8518004a40 0x7f8518004ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851800daf0 con 0x7f8518004a40 2021-07-11T04:01:15.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8518009770 con 0x7f8518003780 2021-07-11T04:01:15.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8518009930 con 0x7f85340ad150 2021-07-11T04:01:15.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85337fe700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8518004a40 0x7f8518004ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8532ffd700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8518003780 0x7f8518003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85327fc700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ad150 0x7f8518004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85337fe700 1 -- 172.21.2.104:0/3301514482 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ad150 msgr2=0x7f8518004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:15.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85337fe700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ad150 0x7f8518004500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85337fe700 1 -- 172.21.2.104:0/3301514482 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8518003780 msgr2=0x7f8518003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:15.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85337fe700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8518003780 0x7f8518003ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85337fe700 1 -- 172.21.2.104:0/3301514482 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8518003260 con 0x7f8518004a40 2021-07-11T04:01:15.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85337fe700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8518004a40 0x7f8518004ec0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f852c0016f0 tx=0x7f85340ac290).ready entity=mon.0 client_cookie=f6bef5fc8b171fb9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f852c00cc50 con 0x7f8518004a40 2021-07-11T04:01:15.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c00cdb0 con 0x7f8518004a40 2021-07-11T04:01:15.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f8531ffb700 1 -- 172.21.2.104:0/3301514482 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f852c005b80 con 0x7f8518004a40 2021-07-11T04:01:15.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f853994e700 1 -- 172.21.2.104:0/3301514482 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8518004a40 msgr2=0x7f8518004ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f853994e700 1 --2- 172.21.2.104:0/3301514482 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8518004a40 0x7f8518004ec0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f852c0016f0 tx=0x7f85340ac290).stop 2021-07-11T04:01:15.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.663+0000 7f85337fe700 1 -- 172.21.2.104:0/3301514482 reap_dead start 2021-07-11T04:01:15.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 -- 172.21.2.104:0/3301514482 shutdown_connections 2021-07-11T04:01:15.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 -- 172.21.2.104:0/3301514482 >> 172.21.2.104:0/3301514482 conn(0x7f85340b64d0 msgr2=0x7f85340b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:15.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 -- 172.21.2.104:0/3301514482 shutdown_connections 2021-07-11T04:01:15.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 -- 172.21.2.104:0/3301514482 wait complete. 2021-07-11T04:01:15.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 Processor -- start 2021-07-11T04:01:15.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 -- start start 2021-07-11T04:01:15.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8534063a20 0x7f8534066050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ae350 0x7f8534066540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f853413e680 0x7f853413eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8534066a30 con 0x7f853413e680 2021-07-11T04:01:15.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8534065350 con 0x7f8534063a20 2021-07-11T04:01:15.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8534063e00 con 0x7f85340ae350 2021-07-11T04:01:15.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f85337fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f853413e680 0x7f853413eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f85337fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f853413e680 0x7f853413eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49488/0 (socket says 172.21.2.104:49488) 2021-07-11T04:01:15.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f85337fe700 1 -- 172.21.2.104:0/3374516998 learned_addr learned my addr 172.21.2.104:0/3374516998 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:15.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f8532ffd700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8534063a20 0x7f8534066050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f85327fc700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ae350 0x7f8534066540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f85337fe700 1 -- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ae350 msgr2=0x7f8534066540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f85337fe700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ae350 0x7f8534066540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f85337fe700 1 -- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8534063a20 msgr2=0x7f8534066050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f85337fe700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8534063a20 0x7f8534066050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f85337fe700 1 -- 172.21.2.104:0/3374516998 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85340ae7f0 con 0x7f853413e680 2021-07-11T04:01:15.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f85337fe700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f853413e680 0x7f853413eb30 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f8534066b70 tx=0x7f852c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f8523fff700 1 -- 172.21.2.104:0/3374516998 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f852c004dc0 con 0x7f853413e680 2021-07-11T04:01:15.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f8523fff700 1 -- 172.21.2.104:0/3374516998 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c018070 con 0x7f853413e680 2021-07-11T04:01:15.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 -- 172.21.2.104:0/3374516998 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8534065580 con 0x7f853413e680 2021-07-11T04:01:15.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f8523fff700 1 -- 172.21.2.104:0/3374516998 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f852c0072c0 con 0x7f853413e680 2021-07-11T04:01:15.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 -- 172.21.2.104:0/3374516998 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85340ada10 con 0x7f853413e680 2021-07-11T04:01:15.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f853994e700 1 -- 172.21.2.104:0/3374516998 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8534057350 con 0x7f853413e680 2021-07-11T04:01:15.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f8523fff700 1 -- 172.21.2.104:0/3374516998 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f852c010070 con 0x7f853413e680 2021-07-11T04:01:15.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f8523fff700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8510053270 0x7f8510055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.667+0000 7f8523fff700 1 -- 172.21.2.104:0/3374516998 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f852c06c050 con 0x7f853413e680 2021-07-11T04:01:15.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.671+0000 7f8532ffd700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8510053270 0x7f8510055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.671+0000 7f8532ffd700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8510053270 0x7f8510055730 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f852400c000 tx=0x7f8524009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.671+0000 7f8523fff700 1 -- 172.21.2.104:0/3374516998 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f852c043390 con 0x7f853413e680 2021-07-11T04:01:15.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 -- 172.21.2.104:0/3374516998 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85340698e0 con 0x7f853413e680 2021-07-11T04:01:15.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f8523fff700 1 -- 172.21.2.104:0/3374516998 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f852c0469b0 con 0x7f853413e680 2021-07-11T04:01:15.756 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:15.756 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":119,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:14.034771+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:01:10.890456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-07-11T04:01:12.008882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:15.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 -- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8510053270 msgr2=0x7f8510055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8510053270 0x7f8510055730 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f852400c000 tx=0x7f8524009040).stop 2021-07-11T04:01:15.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 -- 172.21.2.104:0/3374516998 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f853413e680 msgr2=0x7f853413eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f853413e680 0x7f853413eb30 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f8534066b70 tx=0x7f852c015000).stop 2021-07-11T04:01:15.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 -- 172.21.2.104:0/3374516998 shutdown_connections 2021-07-11T04:01:15.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8510053270 0x7f8510055730 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f853413e680 0x7f853413eb30 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f85340ae350 0x7f8534066540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 --2- 172.21.2.104:0/3374516998 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8534063a20 0x7f8534066050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 -- 172.21.2.104:0/3374516998 >> 172.21.2.104:0/3374516998 conn(0x7f85340b64d0 msgr2=0x7f85340b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:15.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 -- 172.21.2.104:0/3374516998 shutdown_connections 2021-07-11T04:01:15.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.755+0000 7f853994e700 1 -- 172.21.2.104:0/3374516998 wait complete. 2021-07-11T04:01:15.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:01:10.890456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:01:12.008882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:01:15.765 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:01:15.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.815+0000 7fca3a534700 1 Processor -- start 2021-07-11T04:01:15.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.815+0000 7fca3a534700 1 -- start start 2021-07-11T04:01:15.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fca340b1ec0 0x7fca340b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fca340affb0 0x7fca340b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fca340a70c0 0x7fca34141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca340b0420 con 0x7fca340b1ec0 2021-07-11T04:01:15.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca340b2790 con 0x7fca340a70c0 2021-07-11T04:01:15.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca340b0650 con 0x7fca340affb0 2021-07-11T04:01:15.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca33fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fca340b1ec0 0x7fca340b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca33fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fca340b1ec0 0x7fca340b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49492/0 (socket says 172.21.2.104:49492) 2021-07-11T04:01:15.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca33fff700 1 -- 172.21.2.104:0/940517065 learned_addr learned my addr 172.21.2.104:0/940517065 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:15.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca33fff700 1 -- 172.21.2.104:0/940517065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fca340affb0 msgr2=0x7fca340b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:15.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca33fff700 1 --2- 172.21.2.104:0/940517065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fca340affb0 0x7fca340b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca33fff700 1 -- 172.21.2.104:0/940517065 >> v2:172.21.2.105:3300/0 conn(0x7fca340a70c0 msgr2=0x7fca34141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:15.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca33fff700 1 --2- 172.21.2.104:0/940517065 >> v2:172.21.2.105:3300/0 conn(0x7fca340a70c0 0x7fca34141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca33fff700 1 -- 172.21.2.104:0/940517065 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca340b08f0 con 0x7fca340b1ec0 2021-07-11T04:01:15.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca33fff700 1 --2- 172.21.2.104:0/940517065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fca340b1ec0 0x7fca340b12d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7fca1c002f70 tx=0x7fca1c006eb0).ready entity=mon.0 client_cookie=dde71c11e82ec74f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca32ffd700 1 -- 172.21.2.104:0/940517065 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca1c00d570 con 0x7fca340b1ec0 2021-07-11T04:01:15.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca32ffd700 1 -- 172.21.2.104:0/940517065 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c000e90 con 0x7fca340b1ec0 2021-07-11T04:01:15.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca32ffd700 1 -- 172.21.2.104:0/940517065 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca1c00d900 con 0x7fca340b1ec0 2021-07-11T04:01:15.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- 172.21.2.104:0/940517065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fca340b1ec0 msgr2=0x7fca340b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 --2- 172.21.2.104:0/940517065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fca340b1ec0 0x7fca340b12d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7fca1c002f70 tx=0x7fca1c006eb0).stop 2021-07-11T04:01:15.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- 172.21.2.104:0/940517065 shutdown_connections 2021-07-11T04:01:15.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 --2- 172.21.2.104:0/940517065 >> v2:172.21.2.105:3300/0 conn(0x7fca340a70c0 0x7fca34141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 --2- 172.21.2.104:0/940517065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fca340affb0 0x7fca340b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 --2- 172.21.2.104:0/940517065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fca340b1ec0 0x7fca340b12d0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- 172.21.2.104:0/940517065 >> 172.21.2.104:0/940517065 conn(0x7fca3405f6b0 msgr2=0x7fca3405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:15.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- 172.21.2.104:0/940517065 shutdown_connections 2021-07-11T04:01:15.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- 172.21.2.104:0/940517065 wait complete. 2021-07-11T04:01:15.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 Processor -- start 2021-07-11T04:01:15.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- start start 2021-07-11T04:01:15.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fca340a70c0 0x7fca340ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fca340affb0 0x7fca341400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fca340b1ec0 0x7fca34157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca34157970 con 0x7fca340affb0 2021-07-11T04:01:15.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca340aeae0 con 0x7fca340a70c0 2021-07-11T04:01:15.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca340ad3b0 con 0x7fca340b1ec0 2021-07-11T04:01:15.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca337fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fca340affb0 0x7fca341400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca337fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fca340affb0 0x7fca341400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49502/0 (socket says 172.21.2.104:49502) 2021-07-11T04:01:15.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca337fe700 1 -- 172.21.2.104:0/372903424 learned_addr learned my addr 172.21.2.104:0/372903424 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:15.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca337fe700 1 -- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fca340b1ec0 msgr2=0x7fca34157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca337fe700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fca340b1ec0 0x7fca34157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca337fe700 1 -- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fca340a70c0 msgr2=0x7fca340ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca337fe700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fca340a70c0 0x7fca340ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca337fe700 1 -- 172.21.2.104:0/372903424 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c00b1e0 con 0x7fca340affb0 2021-07-11T04:01:15.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca337fe700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fca340affb0 0x7fca341400a0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7fca24002f70 tx=0x7fca24006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca317fa700 1 -- 172.21.2.104:0/372903424 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca24013460 con 0x7fca340affb0 2021-07-11T04:01:15.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca317fa700 1 -- 172.21.2.104:0/372903424 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca240135c0 con 0x7fca340affb0 2021-07-11T04:01:15.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- 172.21.2.104:0/372903424 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca340ad650 con 0x7fca340affb0 2021-07-11T04:01:15.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca317fa700 1 -- 172.21.2.104:0/372903424 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fca24013920 con 0x7fca340affb0 2021-07-11T04:01:15.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.819+0000 7fca3a534700 1 -- 172.21.2.104:0/372903424 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca340add90 con 0x7fca340affb0 2021-07-11T04:01:15.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.823+0000 7fca3a534700 1 -- 172.21.2.104:0/372903424 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca34057350 con 0x7fca340affb0 2021-07-11T04:01:15.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.823+0000 7fca317fa700 1 -- 172.21.2.104:0/372903424 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fca2401a060 con 0x7fca340affb0 2021-07-11T04:01:15.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.823+0000 7fca317fa700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca18053350 0x7fca18055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.823+0000 7fca317fa700 1 -- 172.21.2.104:0/372903424 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fca2406afc0 con 0x7fca340affb0 2021-07-11T04:01:15.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.823+0000 7fca33fff700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca18053350 0x7fca18055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.823+0000 7fca33fff700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca18053350 0x7fca18055810 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7fca1c002f70 tx=0x7fca1c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.823+0000 7fca317fa700 1 -- 172.21.2.104:0/372903424 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fca240421f0 con 0x7fca340affb0 2021-07-11T04:01:15.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.903+0000 7fca3a534700 1 -- 172.21.2.104:0/372903424 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fca340ac760 con 0x7fca340affb0 2021-07-11T04:01:15.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.903+0000 7fca317fa700 1 -- 172.21.2.104:0/372903424 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v119) v1 ==== 110+0+10 (secure 0 0 0) 0x7fca24018020 con 0x7fca340affb0 2021-07-11T04:01:15.907 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:01:15.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 -- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca18053350 msgr2=0x7fca18055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca18053350 0x7fca18055810 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7fca1c002f70 tx=0x7fca1c015000).stop 2021-07-11T04:01:15.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 -- 172.21.2.104:0/372903424 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fca340affb0 msgr2=0x7fca341400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fca340affb0 0x7fca341400a0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7fca24002f70 tx=0x7fca24006e40).stop 2021-07-11T04:01:15.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 -- 172.21.2.104:0/372903424 shutdown_connections 2021-07-11T04:01:15.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fca340b1ec0 0x7fca34157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fca18053350 0x7fca18055810 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fca340affb0 0x7fca341400a0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 --2- 172.21.2.104:0/372903424 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fca340a70c0 0x7fca340ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 -- 172.21.2.104:0/372903424 >> 172.21.2.104:0/372903424 conn(0x7fca3405f6b0 msgr2=0x7fca3413d900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:15.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 -- 172.21.2.104:0/372903424 shutdown_connections 2021-07-11T04:01:15.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.907+0000 7fca3a534700 1 -- 172.21.2.104:0/372903424 wait complete. 2021-07-11T04:01:15.915 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:01:15.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 Processor -- start 2021-07-11T04:01:15.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 -- start start 2021-07-11T04:01:15.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f00280b1ec0 0x7f00280b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f00280affb0 0x7f00280b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00280a70c0 0x7f0028141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00280b0420 con 0x7f00280affb0 2021-07-11T04:01:15.971 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00280b2790 con 0x7f00280b1ec0 2021-07-11T04:01:15.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00280b0650 con 0x7f00280a70c0 2021-07-11T04:01:15.972 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f00267fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f00280affb0 0x7f00280b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f00267fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f00280affb0 0x7f00280b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49510/0 (socket says 172.21.2.104:49510) 2021-07-11T04:01:15.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f00267fc700 1 -- 172.21.2.104:0/3822159772 learned_addr learned my addr 172.21.2.104:0/3822159772 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:15.973 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f00267fc700 1 -- 172.21.2.104:0/3822159772 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00280a70c0 msgr2=0x7f0028141250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:15.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f00267fc700 1 --2- 172.21.2.104:0/3822159772 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00280a70c0 0x7f0028141250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.974 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f00267fc700 1 -- 172.21.2.104:0/3822159772 >> v2:172.21.2.105:3300/0 conn(0x7f00280b1ec0 msgr2=0x7f00280b12d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:15.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f00267fc700 1 --2- 172.21.2.104:0/3822159772 >> v2:172.21.2.105:3300/0 conn(0x7f00280b1ec0 0x7f00280b12d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.975 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f00267fc700 1 -- 172.21.2.104:0/3822159772 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00280b08f0 con 0x7f00280affb0 2021-07-11T04:01:15.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f00267fc700 1 --2- 172.21.2.104:0/3822159772 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00280affb0 0x7f00280b22a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f0018002f70 tx=0x7f0018006eb0).ready entity=mon.0 client_cookie=2b6654a568e5d828 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f0025ffb700 1 -- 172.21.2.104:0/3822159772 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001800d570 con 0x7f00280affb0 2021-07-11T04:01:15.976 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f0025ffb700 1 -- 172.21.2.104:0/3822159772 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0018000e90 con 0x7f00280affb0 2021-07-11T04:01:15.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f0025ffb700 1 -- 172.21.2.104:0/3822159772 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f001800d900 con 0x7f00280affb0 2021-07-11T04:01:15.977 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 -- 172.21.2.104:0/3822159772 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00280affb0 msgr2=0x7f00280b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 --2- 172.21.2.104:0/3822159772 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00280affb0 0x7f00280b22a0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f0018002f70 tx=0x7f0018006eb0).stop 2021-07-11T04:01:15.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 -- 172.21.2.104:0/3822159772 shutdown_connections 2021-07-11T04:01:15.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 --2- 172.21.2.104:0/3822159772 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00280a70c0 0x7f0028141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 --2- 172.21.2.104:0/3822159772 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00280affb0 0x7f00280b22a0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 --2- 172.21.2.104:0/3822159772 >> v2:172.21.2.105:3300/0 conn(0x7f00280b1ec0 0x7f00280b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:15.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 -- 172.21.2.104:0/3822159772 >> 172.21.2.104:0/3822159772 conn(0x7f002805f6b0 msgr2=0x7f002805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:15.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 -- 172.21.2.104:0/3822159772 shutdown_connections 2021-07-11T04:01:15.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 -- 172.21.2.104:0/3822159772 wait complete. 2021-07-11T04:01:15.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 Processor -- start 2021-07-11T04:01:15.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.967+0000 7f002d294700 1 -- start start 2021-07-11T04:01:15.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f002d294700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00280a70c0 0x7f0028146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f002d294700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f00280affb0 0x7f002814ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f002d294700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00280b1ec0 0x7f00280aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f002d294700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00280ac1f0 con 0x7f00280b1ec0 2021-07-11T04:01:15.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f002d294700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00280aa4b0 con 0x7f00280affb0 2021-07-11T04:01:15.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f002d294700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002814b2d0 con 0x7f00280a70c0 2021-07-11T04:01:15.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f00277fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00280b1ec0 0x7f00280aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f00277fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00280b1ec0 0x7f00280aaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49516/0 (socket says 172.21.2.104:49516) 2021-07-11T04:01:15.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f00277fe700 1 -- 172.21.2.104:0/3583881586 learned_addr learned my addr 172.21.2.104:0/3583881586 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:15.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f00267fc700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f00280affb0 0x7f002814ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f00277fe700 1 -- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00280a70c0 msgr2=0x7f0028146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f0026ffd700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00280a70c0 0x7f0028146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f00277fe700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00280a70c0 0x7f0028146f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f00277fe700 1 -- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f00280affb0 msgr2=0x7f002814ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:15.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f00277fe700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f00280affb0 0x7f002814ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:15.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f00277fe700 1 -- 172.21.2.104:0/3583881586 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001800b1e0 con 0x7f00280b1ec0 2021-07-11T04:01:15.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f00267fc700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f00280affb0 0x7f002814ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:15.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f00277fe700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00280b1ec0 0x7f00280aaf10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f00280b18e0 tx=0x7f00200091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f0017fff700 1 -- 172.21.2.104:0/3583881586 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0020001520 con 0x7f00280b1ec0 2021-07-11T04:01:15.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f0017fff700 1 -- 172.21.2.104:0/3583881586 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0020001680 con 0x7f00280b1ec0 2021-07-11T04:01:15.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f002d294700 1 -- 172.21.2.104:0/3583881586 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00280aa250 con 0x7f00280b1ec0 2021-07-11T04:01:15.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f002d294700 1 -- 172.21.2.104:0/3583881586 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0028145220 con 0x7f00280b1ec0 2021-07-11T04:01:15.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f0017fff700 1 -- 172.21.2.104:0/3583881586 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00200019e0 con 0x7f00280b1ec0 2021-07-11T04:01:15.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f002d294700 1 -- 172.21.2.104:0/3583881586 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00281453e0 con 0x7f00280b1ec0 2021-07-11T04:01:15.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f0017fff700 1 -- 172.21.2.104:0/3583881586 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0020018030 con 0x7f00280b1ec0 2021-07-11T04:01:15.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f0017fff700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0004053300 0x7f00040557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:15.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f0017fff700 1 -- 172.21.2.104:0/3583881586 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f002006af20 con 0x7f00280b1ec0 2021-07-11T04:01:15.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f0026ffd700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0004053300 0x7f00040557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:15.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.971+0000 7f0026ffd700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0004053300 0x7f00040557c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f001c007000 tx=0x7f001c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:15.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:15.975+0000 7f0017fff700 1 -- 172.21.2.104:0/3583881586 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0020045e60 con 0x7f00280b1ec0 2021-07-11T04:01:16.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 -- 172.21.2.104:0/3583881586 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0028057350 con 0x7f00280b1ec0 2021-07-11T04:01:16.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f0017fff700 1 -- 172.21.2.104:0/3583881586 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v119) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0020042200 con 0x7f00280b1ec0 2021-07-11T04:01:16.056 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:16.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 -- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0004053300 msgr2=0x7f00040557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0004053300 0x7f00040557c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f001c007000 tx=0x7f001c008040).stop 2021-07-11T04:01:16.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 -- 172.21.2.104:0/3583881586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00280b1ec0 msgr2=0x7f00280aaf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00280b1ec0 0x7f00280aaf10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f00280b18e0 tx=0x7f00200091a0).stop 2021-07-11T04:01:16.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 -- 172.21.2.104:0/3583881586 shutdown_connections 2021-07-11T04:01:16.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0004053300 0x7f00040557c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00280b1ec0 0x7f00280aaf10 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f00280affb0 0x7f002814ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 --2- 172.21.2.104:0/3583881586 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00280a70c0 0x7f0028146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 -- 172.21.2.104:0/3583881586 >> 172.21.2.104:0/3583881586 conn(0x7f002805f6b0 msgr2=0x7f002813e5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:16.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 -- 172.21.2.104:0/3583881586 shutdown_connections 2021-07-11T04:01:16.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.055+0000 7f002d294700 1 -- 172.21.2.104:0/3583881586 wait complete. 2021-07-11T04:01:16.064 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:01:16.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f7416f15700 1 Processor -- start 2021-07-11T04:01:16.116 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f7416f15700 1 -- start start 2021-07-11T04:01:16.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f7416f15700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74100b12d0 0x7f74100affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f7416f15700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f74100b22a0 0x7f7410141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f7416f15700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74100a77d0 con 0x7f74100b1ec0 2021-07-11T04:01:16.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f7416f15700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74100b1740 con 0x7f74100b22a0 2021-07-11T04:01:16.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f7416f15700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74100b0650 con 0x7f74100b12d0 2021-07-11T04:01:16.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f7414cb1700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f74100b1ec0 0x7f74100a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46288/0 (socket says 172.21.2.104:46288) 2021-07-11T04:01:16.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f7414cb1700 1 -- 172.21.2.104:0/804598592 learned_addr learned my addr 172.21.2.104:0/804598592 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:16.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f74154b2700 1 --2- 172.21.2.104:0/804598592 >> v2:172.21.2.104:3300/0 conn(0x7f74100b22a0 0x7f7410141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:16.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f74154b2700 1 -- 172.21.2.104:0/804598592 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74100b12d0 msgr2=0x7f74100affb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:16.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f74154b2700 1 --2- 172.21.2.104:0/804598592 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74100b12d0 0x7f74100affb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f74154b2700 1 -- 172.21.2.104:0/804598592 >> v1:172.21.2.104:6789/0 conn(0x7f74100b1ec0 legacy=0x7f74100a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f74154b2700 1 -- 172.21.2.104:0/804598592 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74100b08f0 con 0x7f74100b22a0 2021-07-11T04:01:16.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f74154b2700 1 --2- 172.21.2.104:0/804598592 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74100b22a0 0x7f7410141250 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f74080032f0 tx=0x7f74080091d0).ready entity=mon.0 client_cookie=f92f2e332c4b8720 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:16.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f740f7fe700 1 -- 172.21.2.104:0/804598592 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7408009ea0 con 0x7f74100b22a0 2021-07-11T04:01:16.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f740f7fe700 1 -- 172.21.2.104:0/804598592 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f740800b8a0 con 0x7f74100b22a0 2021-07-11T04:01:16.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f740f7fe700 1 -- 172.21.2.104:0/804598592 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f740800ba50 con 0x7f74100b22a0 2021-07-11T04:01:16.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f7416f15700 1 -- 172.21.2.104:0/804598592 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74100b22a0 msgr2=0x7f7410141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.115+0000 7f7416f15700 1 --2- 172.21.2.104:0/804598592 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74100b22a0 0x7f7410141250 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f74080032f0 tx=0x7f74080091d0).stop 2021-07-11T04:01:16.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 -- 172.21.2.104:0/804598592 shutdown_connections 2021-07-11T04:01:16.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 --2- 172.21.2.104:0/804598592 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74100b22a0 0x7f7410141250 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 --2- 172.21.2.104:0/804598592 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74100b12d0 0x7f74100affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 -- 172.21.2.104:0/804598592 >> 172.21.2.104:0/804598592 conn(0x7f741005f6b0 msgr2=0x7f741005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:16.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 -- 172.21.2.104:0/804598592 shutdown_connections 2021-07-11T04:01:16.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 -- 172.21.2.104:0/804598592 wait complete. 2021-07-11T04:01:16.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 Processor -- start 2021-07-11T04:01:16.124 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 -- start start 2021-07-11T04:01:16.126 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74100b12d0 0x7f74100ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74100b1ec0 0x7f741015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74100b22a0 0x7f741015abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74100ac270 con 0x7f74100b1ec0 2021-07-11T04:01:16.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74100b4e70 con 0x7f74100b22a0 2021-07-11T04:01:16.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74100b5110 con 0x7f74100b12d0 2021-07-11T04:01:16.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74100b1ec0 0x7f741015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:16.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74100b1ec0 0x7f741015a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49528/0 (socket says 172.21.2.104:49528) 2021-07-11T04:01:16.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740ffff700 1 -- 172.21.2.104:0/1792331612 learned_addr learned my addr 172.21.2.104:0/1792331612 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:16.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740ffff700 1 -- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74100b12d0 msgr2=0x7f74100ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740ffff700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74100b12d0 0x7f74100ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740ffff700 1 -- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74100b22a0 msgr2=0x7f741015abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740ffff700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74100b22a0 0x7f741015abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740ffff700 1 -- 172.21.2.104:0/1792331612 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408009470 con 0x7f74100b1ec0 2021-07-11T04:01:16.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740ffff700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74100b1ec0 0x7f741015a6d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f7400002fb0 tx=0x7f7400006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:16.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740dffb700 1 -- 172.21.2.104:0/1792331612 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7400013460 con 0x7f74100b1ec0 2021-07-11T04:01:16.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740dffb700 1 -- 172.21.2.104:0/1792331612 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74000135c0 con 0x7f74100b1ec0 2021-07-11T04:01:16.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 -- 172.21.2.104:0/1792331612 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741015a230 con 0x7f74100b1ec0 2021-07-11T04:01:16.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740dffb700 1 -- 172.21.2.104:0/1792331612 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7400013920 con 0x7f74100b1ec0 2021-07-11T04:01:16.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 -- 172.21.2.104:0/1792331612 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74100b4b00 con 0x7f74100b1ec0 2021-07-11T04:01:16.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f7416f15700 1 -- 172.21.2.104:0/1792331612 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741015a3c0 con 0x7f74100b1ec0 2021-07-11T04:01:16.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740dffb700 1 -- 172.21.2.104:0/1792331612 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f740000f070 con 0x7f74100b1ec0 2021-07-11T04:01:16.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740dffb700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73f4053300 0x7f73f40557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.119+0000 7f740dffb700 1 -- 172.21.2.104:0/1792331612 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f740006af40 con 0x7f74100b1ec0 2021-07-11T04:01:16.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.123+0000 7f7414cb1700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73f4053300 0x7f73f40557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:16.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.123+0000 7f7414cb1700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73f4053300 0x7f73f40557c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f741006a0f0 tx=0x7f73f80077d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:16.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.123+0000 7f740dffb700 1 -- 172.21.2.104:0/1792331612 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7400068cf0 con 0x7f74100b1ec0 2021-07-11T04:01:16.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.203+0000 7f7416f15700 1 -- 172.21.2.104:0/1792331612 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74100b52d0 con 0x7f74100b1ec0 2021-07-11T04:01:16.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f740dffb700 1 -- 172.21.2.104:0/1792331612 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+10 (secure 0 0 0) 0x7f74000470a0 con 0x7f74100b1ec0 2021-07-11T04:01:16.207 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:16.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 -- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73f4053300 msgr2=0x7f73f40557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73f4053300 0x7f73f40557c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f741006a0f0 tx=0x7f73f80077d0).stop 2021-07-11T04:01:16.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 -- 172.21.2.104:0/1792331612 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74100b1ec0 msgr2=0x7f741015a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74100b1ec0 0x7f741015a6d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f7400002fb0 tx=0x7f7400006df0).stop 2021-07-11T04:01:16.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 -- 172.21.2.104:0/1792331612 shutdown_connections 2021-07-11T04:01:16.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f74100b22a0 0x7f741015abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f73f4053300 0x7f73f40557c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f74100b1ec0 0x7f741015a6d0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 --2- 172.21.2.104:0/1792331612 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f74100b12d0 0x7f74100ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 -- 172.21.2.104:0/1792331612 >> 172.21.2.104:0/1792331612 conn(0x7f741005f6b0 msgr2=0x7f74100a8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:16.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 -- 172.21.2.104:0/1792331612 shutdown_connections 2021-07-11T04:01:16.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.207+0000 7f7416f15700 1 -- 172.21.2.104:0/1792331612 wait complete. 2021-07-11T04:01:16.214 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:01:16.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 Processor -- start 2021-07-11T04:01:16.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 -- start start 2021-07-11T04:01:16.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f447413c9c0 0x7f44740b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44740b5f40 con 0x7f44740afaf0 2021-07-11T04:01:16.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44740ad420 con 0x7f44740ad040 2021-07-11T04:01:16.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44740aeb60 con 0x7f447413c9c0 2021-07-11T04:01:16.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f4473fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f447413c9c0 0x7f44740b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:16.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447965e700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f44740afaf0 0x7f447413f070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46302/0 (socket says 172.21.2.104:46302) 2021-07-11T04:01:16.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447965e700 1 -- 172.21.2.104:0/8776970 learned_addr learned my addr 172.21.2.104:0/8776970 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:16.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f4473fff700 1 -- 172.21.2.104:0/8776970 >> v1:172.21.2.105:6789/0 conn(0x7f44740ad040 legacy=0x7f447413c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:16.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f4473fff700 1 -- 172.21.2.104:0/8776970 >> v1:172.21.2.104:6789/0 conn(0x7f44740afaf0 legacy=0x7f447413f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f4473fff700 1 -- 172.21.2.104:0/8776970 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44740aee00 con 0x7f447413c9c0 2021-07-11T04:01:16.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f4473fff700 1 --2- 172.21.2.104:0/8776970 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f447413c9c0 0x7f44740b5a50 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7f447405d010 tx=0x7f4464006e80).ready entity=mon.0 client_cookie=db5ebe1664abe783 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:16.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f44737fe700 1 -- 172.21.2.104:0/8776970 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f446400ba30 con 0x7f447413c9c0 2021-07-11T04:01:16.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f44737fe700 1 -- 172.21.2.104:0/8776970 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4464000e90 con 0x7f447413c9c0 2021-07-11T04:01:16.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f44737fe700 1 -- 172.21.2.104:0/8776970 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f446400bd80 con 0x7f447413c9c0 2021-07-11T04:01:16.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 -- 172.21.2.104:0/8776970 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f447413c9c0 msgr2=0x7f44740b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 --2- 172.21.2.104:0/8776970 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f447413c9c0 0x7f44740b5a50 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7f447405d010 tx=0x7f4464006e80).stop 2021-07-11T04:01:16.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 -- 172.21.2.104:0/8776970 shutdown_connections 2021-07-11T04:01:16.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 --2- 172.21.2.104:0/8776970 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f447413c9c0 0x7f44740b5a50 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 -- 172.21.2.104:0/8776970 >> 172.21.2.104:0/8776970 conn(0x7f447405c490 msgr2=0x7f447405c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:16.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 -- 172.21.2.104:0/8776970 shutdown_connections 2021-07-11T04:01:16.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.267+0000 7f447b0c1700 1 -- 172.21.2.104:0/8776970 wait complete. 2021-07-11T04:01:16.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447b0c1700 1 Processor -- start 2021-07-11T04:01:16.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447b0c1700 1 -- start start 2021-07-11T04:01:16.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447b0c1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f44740ad040 0x7f44740b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447b0c1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f44740afaf0 0x7f4474140320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447b0c1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f447413c9c0 0x7f4474134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447b0c1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4474141a60 con 0x7f447413c9c0 2021-07-11T04:01:16.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447b0c1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4474140810 con 0x7f44740afaf0 2021-07-11T04:01:16.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447b0c1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4474140aa0 con 0x7f44740ad040 2021-07-11T04:01:16.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447965e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f447413c9c0 0x7f4474134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:16.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447965e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f447413c9c0 0x7f4474134230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49538/0 (socket says 172.21.2.104:49538) 2021-07-11T04:01:16.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447965e700 1 -- 172.21.2.104:0/3101606861 learned_addr learned my addr 172.21.2.104:0/3101606861 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:16.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447965e700 1 -- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f44740ad040 msgr2=0x7f44740b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447965e700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f44740ad040 0x7f44740b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447965e700 1 -- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f44740afaf0 msgr2=0x7f4474140320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447965e700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f44740afaf0 0x7f4474140320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447965e700 1 -- 172.21.2.104:0/3101606861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446400d410 con 0x7f447413c9c0 2021-07-11T04:01:16.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447965e700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f447413c9c0 0x7f4474134230 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f4474062900 tx=0x7f446c00a2a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:16.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f4471ffb700 1 -- 172.21.2.104:0/3101606861 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f446c006070 con 0x7f447413c9c0 2021-07-11T04:01:16.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f4471ffb700 1 -- 172.21.2.104:0/3101606861 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c00e620 con 0x7f447413c9c0 2021-07-11T04:01:16.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447b0c1700 1 -- 172.21.2.104:0/3101606861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4474142340 con 0x7f447413c9c0 2021-07-11T04:01:16.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f4471ffb700 1 -- 172.21.2.104:0/3101606861 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f446c00e9c0 con 0x7f447413c9c0 2021-07-11T04:01:16.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447b0c1700 1 -- 172.21.2.104:0/3101606861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4474142880 con 0x7f447413c9c0 2021-07-11T04:01:16.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f447b0c1700 1 -- 172.21.2.104:0/3101606861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4474142a40 con 0x7f447413c9c0 2021-07-11T04:01:16.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f4471ffb700 1 -- 172.21.2.104:0/3101606861 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f446c00d040 con 0x7f447413c9c0 2021-07-11T04:01:16.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f4471ffb700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f44580531f0 0x7f44580556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f4471ffb700 1 -- 172.21.2.104:0/3101606861 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f446c06b820 con 0x7f447413c9c0 2021-07-11T04:01:16.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.271+0000 7f4478e5d700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f44580531f0 0x7f44580556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:16.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.275+0000 7f4478e5d700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f44580531f0 0x7f44580556b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f445c002f90 tx=0x7f445c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:16.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.275+0000 7f4471ffb700 1 -- 172.21.2.104:0/3101606861 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f446c042b60 con 0x7f447413c9c0 2021-07-11T04:01:16.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.355+0000 7f447b0c1700 1 -- 172.21.2.104:0/3101606861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4474057350 con 0x7f447413c9c0 2021-07-11T04:01:16.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.355+0000 7f4471ffb700 1 -- 172.21.2.104:0/3101606861 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+10 (secure 0 0 0) 0x7f446c046180 con 0x7f447413c9c0 2021-07-11T04:01:16.358 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:16.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 -- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f44580531f0 msgr2=0x7f44580556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f44580531f0 0x7f44580556b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f445c002f90 tx=0x7f445c0069b0).stop 2021-07-11T04:01:16.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 -- 172.21.2.104:0/3101606861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f447413c9c0 msgr2=0x7f4474134230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f447413c9c0 0x7f4474134230 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f4474062900 tx=0x7f446c00a2a0).stop 2021-07-11T04:01:16.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 -- 172.21.2.104:0/3101606861 shutdown_connections 2021-07-11T04:01:16.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f44580531f0 0x7f44580556b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f447413c9c0 0x7f4474134230 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f44740afaf0 0x7f4474140320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 --2- 172.21.2.104:0/3101606861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f44740ad040 0x7f44740b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 -- 172.21.2.104:0/3101606861 >> 172.21.2.104:0/3101606861 conn(0x7f447405c490 msgr2=0x7f447413a820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:16.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 -- 172.21.2.104:0/3101606861 shutdown_connections 2021-07-11T04:01:16.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.359+0000 7f447b0c1700 1 -- 172.21.2.104:0/3101606861 wait complete. 2021-07-11T04:01:16.366 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:01:16.366 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:16.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28be97700 1 Processor -- start 2021-07-11T04:01:16.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28be97700 1 -- start start 2021-07-11T04:01:16.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28be97700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe2840a76b0 0x7fe284066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28be97700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe2840b0060 0x7fe28405fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28be97700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2840a7a90 con 0x7fe2840b1970 2021-07-11T04:01:16.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28be97700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe284147620 con 0x7fe2840b0060 2021-07-11T04:01:16.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28be97700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2841478c0 con 0x7fe2840a76b0 2021-07-11T04:01:16.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28a434700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe2840b0060 0x7fe28405fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:16.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28a434700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe2840b0060 0x7fe28405fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49546/0 (socket says 172.21.2.104:49546) 2021-07-11T04:01:16.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28a434700 1 -- 172.21.2.104:0/1098535656 learned_addr learned my addr 172.21.2.104:0/1098535656 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:16.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28a434700 1 -- 172.21.2.104:0/1098535656 >> v2:172.21.2.105:3300/0 conn(0x7fe2840a76b0 msgr2=0x7fe284066a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:16.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28a434700 1 --2- 172.21.2.104:0/1098535656 >> v2:172.21.2.105:3300/0 conn(0x7fe2840a76b0 0x7fe284066a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28a434700 1 -- 172.21.2.104:0/1098535656 >> v1:172.21.2.105:6789/0 conn(0x7fe2840b1970 legacy=0x7fe28405f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:16.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.415+0000 7fe28a434700 1 -- 172.21.2.104:0/1098535656 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe284147b60 con 0x7fe2840b0060 2021-07-11T04:01:16.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28a434700 1 --2- 172.21.2.104:0/1098535656 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2840b0060 0x7fe28405fdc0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7fe28406a0e0 tx=0x7fe27c009290).ready entity=mon.0 client_cookie=b9da0edbcea5ab6c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:16.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe288c31700 1 -- 172.21.2.104:0/1098535656 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe27c00b830 con 0x7fe2840b0060 2021-07-11T04:01:16.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe288c31700 1 -- 172.21.2.104:0/1098535656 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c00b990 con 0x7fe2840b0060 2021-07-11T04:01:16.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe288c31700 1 -- 172.21.2.104:0/1098535656 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe27c00bb40 con 0x7fe2840b0060 2021-07-11T04:01:16.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- 172.21.2.104:0/1098535656 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2840b0060 msgr2=0x7fe28405fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 --2- 172.21.2.104:0/1098535656 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2840b0060 0x7fe28405fdc0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7fe28406a0e0 tx=0x7fe27c009290).stop 2021-07-11T04:01:16.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- 172.21.2.104:0/1098535656 shutdown_connections 2021-07-11T04:01:16.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 --2- 172.21.2.104:0/1098535656 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2840b0060 0x7fe28405fdc0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 --2- 172.21.2.104:0/1098535656 >> v2:172.21.2.105:3300/0 conn(0x7fe2840a76b0 0x7fe284066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- 172.21.2.104:0/1098535656 >> 172.21.2.104:0/1098535656 conn(0x7fe2840b53e0 msgr2=0x7fe2840b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:16.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- 172.21.2.104:0/1098535656 shutdown_connections 2021-07-11T04:01:16.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- 172.21.2.104:0/1098535656 wait complete. 2021-07-11T04:01:16.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 Processor -- start 2021-07-11T04:01:16.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- start start 2021-07-11T04:01:16.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2840a76b0 0x7fe2840abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe2840b0060 0x7fe2840ae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2840b1970 0x7fe2840ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2840ace90 con 0x7fe2840a76b0 2021-07-11T04:01:16.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2840aab60 con 0x7fe2840b0060 2021-07-11T04:01:16.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2840aae00 con 0x7fe2840b1970 2021-07-11T04:01:16.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe289c33700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2840a76b0 0x7fe2840abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:16.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe289c33700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2840a76b0 0x7fe2840abf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49552/0 (socket says 172.21.2.104:49552) 2021-07-11T04:01:16.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe289c33700 1 -- 172.21.2.104:0/1013060376 learned_addr learned my addr 172.21.2.104:0/1013060376 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:16.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe289c33700 1 -- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2840b1970 msgr2=0x7fe2840ac860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:16.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe289c33700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2840b1970 0x7fe2840ac860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe289c33700 1 -- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe2840b0060 msgr2=0x7fe2840ae5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:16.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe289c33700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe2840b0060 0x7fe2840ae5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe289c33700 1 -- 172.21.2.104:0/1013060376 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c009530 con 0x7fe2840a76b0 2021-07-11T04:01:16.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe289c33700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2840a76b0 0x7fe2840abf80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7fe274002fb0 tx=0x7fe274006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:16.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe272ffd700 1 -- 172.21.2.104:0/1013060376 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe274013460 con 0x7fe2840a76b0 2021-07-11T04:01:16.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- 172.21.2.104:0/1013060376 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2840adeb0 con 0x7fe2840a76b0 2021-07-11T04:01:16.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe272ffd700 1 -- 172.21.2.104:0/1013060376 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2740135c0 con 0x7fe2840a76b0 2021-07-11T04:01:16.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- 172.21.2.104:0/1013060376 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2840aa2a0 con 0x7fe2840a76b0 2021-07-11T04:01:16.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe272ffd700 1 -- 172.21.2.104:0/1013060376 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe274013920 con 0x7fe2840a76b0 2021-07-11T04:01:16.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe28be97700 1 -- 172.21.2.104:0/1013060376 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe284057350 con 0x7fe2840a76b0 2021-07-11T04:01:16.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.419+0000 7fe272ffd700 1 -- 172.21.2.104:0/1013060376 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe27400f070 con 0x7fe2840a76b0 2021-07-11T04:01:16.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.423+0000 7fe272ffd700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2640531f0 0x7fe2640556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:16.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.423+0000 7fe272ffd700 1 -- 172.21.2.104:0/1013060376 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe27406aee0 con 0x7fe2840a76b0 2021-07-11T04:01:16.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.423+0000 7fe289432700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2640531f0 0x7fe2640556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:16.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.423+0000 7fe289432700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2640531f0 0x7fe2640556b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fe280000ed0 tx=0x7fe2800068b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:16.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.423+0000 7fe272ffd700 1 -- 172.21.2.104:0/1013060376 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe274042220 con 0x7fe2840a76b0 2021-07-11T04:01:16.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 -- 172.21.2.104:0/1013060376 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2840ae260 con 0x7fe2840a76b0 2021-07-11T04:01:16.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe272ffd700 1 -- 172.21.2.104:0/1013060376 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2495 (secure 0 0 0) 0x7fe274045840 con 0x7fe2840a76b0 2021-07-11T04:01:16.532 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:16.532 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3151473850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3151473850},{"type":"v1","addr":"172.21.2.105:6835","nonce":3151473850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/4259840781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":4259840781},{"type":"v1","addr":"172.21.2.104:6834","nonce":4259840781}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:13.031418+0000","modified":"2021-07-11T04:01:14.344351+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7620":{"gid":7620,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/1367521056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1367521056},{"type":"v1","addr":"172.21.2.104:6835","nonce":1367521056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-07-11T04:01:16.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 -- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2640531f0 msgr2=0x7fe2640556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2640531f0 0x7fe2640556b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7fe280000ed0 tx=0x7fe2800068b0).stop 2021-07-11T04:01:16.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 -- 172.21.2.104:0/1013060376 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2840a76b0 msgr2=0x7fe2840abf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:16.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2840a76b0 0x7fe2840abf80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7fe274002fb0 tx=0x7fe274006df0).stop 2021-07-11T04:01:16.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 -- 172.21.2.104:0/1013060376 shutdown_connections 2021-07-11T04:01:16.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe2840b1970 0x7fe2840ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe2840b0060 0x7fe2840ae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:16.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2640531f0 0x7fe2640556b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 --2- 172.21.2.104:0/1013060376 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe2840a76b0 0x7fe2840abf80 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:16.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 -- 172.21.2.104:0/1013060376 >> 172.21.2.104:0/1013060376 conn(0x7fe2840b53e0 msgr2=0x7fe284066820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:16.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 -- 172.21.2.104:0/1013060376 shutdown_connections 2021-07-11T04:01:16.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:16.531+0000 7fe28be97700 1 -- 172.21.2.104:0/1013060376 wait complete. 2021-07-11T04:01:16.536 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 57 2021-07-11T04:01:16.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:01:13.031418+0000', 'modified': '2021-07-11T04:01:14.344351+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7620}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7620': {'gid': 7620, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/1367521056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 1367521056}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 1367521056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:01:16.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:01:16.842 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:01:16.923 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":7620,"want_state":"up:active","state":"up:active","rank_uptime":3.8871645460000002,"mdsmap_epoch":57,"osdmap_epoch":119,"osdmap_epoch_barrier":119,"uptime":8.1409618590000008} 2021-07-11T04:01:16.924 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 7620, "mdsmap_epoch": 57, "osdmap_epoch": 119, "osdmap_epoch_barrier": 119, "rank_uptime": 3.887164546, "state": "up:active", "uptime": 8.140961859, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:01:16.924 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:01:16.924 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T04:01:16.925 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T04:01:16.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:16.926 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:16.936 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:01:16.991 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:16.992 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:16.997 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:16.998 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:16.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:16.999 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:01:17.003 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:01:17.004 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:17.047 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:01:17.059 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:17.060 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:17.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15566]: starting ceph client 2021-07-11T04:01:17.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T04:01:17.081+0000 7f66ed333080 -1 init, newargv = 0x5618b34b2310 newargc=9 2021-07-11T04:01:17.092 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15566]: starting fuse 2021-07-11T04:01:17.110 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:17.111 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:17.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:17.113 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:01:17.183 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T04:01:17.184 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:01:17.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:17.185 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T04:01:17.186 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T04:01:17.186 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T04:01:17.187 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T04:01:17.187 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T04:01:17.188 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:01:17.188 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T04:01:17.190 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T04:01:17.190 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T04:01:17.191 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:01:17.191 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T04:01:17.192 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T04:01:17.192 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T04:01:17.193 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T04:01:17.193 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:01:17.194 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T04:01:17.194 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:01:17.195 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:01:17.195 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T04:01:17.196 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:01:17.196 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:01:17.197 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T04:01:17.197 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T04:01:17.265 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15566.asok 2021-07-11T04:01:17.267 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15566.asok 2021-07-11T04:01:17.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:17.269 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15566.asok status 2021-07-11T04:01:17.393 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T04:01:17.393 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T04:01:17.394 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:01:17.394 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:01:17.394 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T04:01:17.394 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T04:01:17.395 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T04:01:17.395 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15566", 2021-07-11T04:01:17.395 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T04:01:17.395 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:01:17.395 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T04:01:17.396 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T04:01:17.396 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 4937, 2021-07-11T04:01:17.396 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T04:01:17.396 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T04:01:17.397 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T04:01:17.397 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 4937 2021-07-11T04:01:17.397 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:01:17.397 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:01:17.397 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:01:17.398 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:01:17.398 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 293007817 2021-07-11T04:01:17.398 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T04:01:17.398 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:01:17.398 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:01:17.399 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:01:17.399 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:01:17.399 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 293007817 2021-07-11T04:01:17.399 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:01:17.400 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.4937 172.21.2.119:0/293007817", 2021-07-11T04:01:17.400 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/293007817", 2021-07-11T04:01:17.400 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T04:01:17.400 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 57, 2021-07-11T04:01:17.401 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 119, 2021-07-11T04:01:17.401 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:01:17.401 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T04:01:17.401 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T04:01:17.401 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T04:01:17.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:17.403 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:17.404 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:17.412 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:01:17.413 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:01:17.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:17.414 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:17.468 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:17.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 Processor -- start 2021-07-11T04:01:17.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 -- start start 2021-07-11T04:01:17.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cd80973d0 0x7f1cd805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1cd813f560 0x7f1cd813fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd80b2a40 con 0x7f1cd80b2660 2021-07-11T04:01:17.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd8097b80 con 0x7f1cd813f560 2021-07-11T04:01:17.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8097e20 con 0x7f1cd80973d0 2021-07-11T04:01:17.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cde2c8700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1cd80b2660 0x7f1cd8096cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46328/0 (socket says 172.21.2.104:46328) 2021-07-11T04:01:17.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cde2c8700 1 -- 172.21.2.104:0/709076670 learned_addr learned my addr 172.21.2.104:0/709076670 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:17.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cdeac9700 1 --2- 172.21.2.104:0/709076670 >> v2:172.21.2.104:3300/0 conn(0x7f1cd813f560 0x7f1cd813fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cdd2c6700 1 -- 172.21.2.104:0/709076670 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856188757 0 0) 0x7f1cd80b2a40 con 0x7f1cd80b2660 2021-07-11T04:01:17.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cdd2c6700 1 -- 172.21.2.104:0/709076670 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc0003260 con 0x7f1cd80b2660 2021-07-11T04:01:17.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cdeac9700 1 -- 172.21.2.104:0/709076670 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cd80973d0 msgr2=0x7f1cd805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:17.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cdeac9700 1 --2- 172.21.2.104:0/709076670 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cd80973d0 0x7f1cd805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cdeac9700 1 -- 172.21.2.104:0/709076670 >> v1:172.21.2.104:6789/0 conn(0x7f1cd80b2660 legacy=0x7f1cd8096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cdeac9700 1 -- 172.21.2.104:0/709076670 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd80980c0 con 0x7f1cd813f560 2021-07-11T04:01:17.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cdeac9700 1 --2- 172.21.2.104:0/709076670 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cd813f560 0x7f1cd813fa10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f1cd00032f0 tx=0x7f1cd00091d0).ready entity=mon.0 client_cookie=4576a4e1caac5b88 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:17.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cdd2c6700 1 -- 172.21.2.104:0/709076670 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cd000c9c0 con 0x7f1cd813f560 2021-07-11T04:01:17.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cdd2c6700 1 -- 172.21.2.104:0/709076670 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd000cb20 con 0x7f1cd813f560 2021-07-11T04:01:17.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1cdd2c6700 1 -- 172.21.2.104:0/709076670 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1cd000ce80 con 0x7f1cd813f560 2021-07-11T04:01:17.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 -- 172.21.2.104:0/709076670 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cd813f560 msgr2=0x7f1cd813fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 --2- 172.21.2.104:0/709076670 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cd813f560 0x7f1cd813fa10 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f1cd00032f0 tx=0x7f1cd00091d0).stop 2021-07-11T04:01:17.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 -- 172.21.2.104:0/709076670 shutdown_connections 2021-07-11T04:01:17.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 --2- 172.21.2.104:0/709076670 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cd813f560 0x7f1cd813fa10 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 --2- 172.21.2.104:0/709076670 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cd80973d0 0x7f1cd805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 -- 172.21.2.104:0/709076670 >> 172.21.2.104:0/709076670 conn(0x7f1cd8062f10 msgr2=0x7f1cd80632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:17.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 -- 172.21.2.104:0/709076670 shutdown_connections 2021-07-11T04:01:17.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 -- 172.21.2.104:0/709076670 wait complete. 2021-07-11T04:01:17.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 Processor -- start 2021-07-11T04:01:17.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.527+0000 7f1ce052c700 1 -- start start 2021-07-11T04:01:17.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ce052c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cd80973d0 0x7f1cd81530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ce052c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cd80b2660 0x7f1cd805c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ce052c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cd813f560 0x7f1cd8139c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ce052c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8152470 con 0x7f1cd80973d0 2021-07-11T04:01:17.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ce052c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd8152710 con 0x7f1cd80b2660 2021-07-11T04:01:17.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ce052c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd81529b0 con 0x7f1cd813f560 2021-07-11T04:01:17.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cde2c8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cd80973d0 0x7f1cd81530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cde2c8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cd80973d0 0x7f1cd81530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49568/0 (socket says 172.21.2.104:49568) 2021-07-11T04:01:17.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cde2c8700 1 -- 172.21.2.104:0/301244787 learned_addr learned my addr 172.21.2.104:0/301244787 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:17.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cddac7700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cd80b2660 0x7f1cd805c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cdeac9700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cd813f560 0x7f1cd8139c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cde2c8700 1 -- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cd813f560 msgr2=0x7f1cd8139c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cde2c8700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cd813f560 0x7f1cd8139c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cde2c8700 1 -- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cd80b2660 msgr2=0x7f1cd805c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cde2c8700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cd80b2660 0x7f1cd805c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cde2c8700 1 -- 172.21.2.104:0/301244787 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd0009470 con 0x7f1cd80973d0 2021-07-11T04:01:17.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cddac7700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cd80b2660 0x7f1cd805c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:17.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cde2c8700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cd80973d0 0x7f1cd81530f0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f1ccc001110 tx=0x7f1ccc007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:17.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ccb7fe700 1 -- 172.21.2.104:0/301244787 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ccc0072e0 con 0x7f1cd80973d0 2021-07-11T04:01:17.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ccb7fe700 1 -- 172.21.2.104:0/301244787 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc007440 con 0x7f1cd80973d0 2021-07-11T04:01:17.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ce052c700 1 -- 172.21.2.104:0/301244787 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd8152c50 con 0x7f1cd80973d0 2021-07-11T04:01:17.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ccb7fe700 1 -- 172.21.2.104:0/301244787 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ccc005800 con 0x7f1cd80973d0 2021-07-11T04:01:17.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ce052c700 1 -- 172.21.2.104:0/301244787 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd813b720 con 0x7f1cd80973d0 2021-07-11T04:01:17.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ce052c700 1 -- 172.21.2.104:0/301244787 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd8057350 con 0x7f1cd80973d0 2021-07-11T04:01:17.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ccb7fe700 1 -- 172.21.2.104:0/301244787 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1ccc010070 con 0x7f1cd80973d0 2021-07-11T04:01:17.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ccb7fe700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cb8053350 0x7f1cb8055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1ccb7fe700 1 -- 172.21.2.104:0/301244787 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1ccc06ad60 con 0x7f1cd80973d0 2021-07-11T04:01:17.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.531+0000 7f1cddac7700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cb8053350 0x7f1cb8055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.535+0000 7f1cddac7700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cb8053350 0x7f1cb8055810 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f1cd4007000 tx=0x7f1cd4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:17.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.535+0000 7f1ccb7fe700 1 -- 172.21.2.104:0/301244787 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1ccc041f90 con 0x7f1cd80973d0 2021-07-11T04:01:17.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.615+0000 7f1ce052c700 1 -- 172.21.2.104:0/301244787 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cd8061450 con 0x7f1cd80973d0 2021-07-11T04:01:17.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.615+0000 7f1ccb7fe700 1 -- 172.21.2.104:0/301244787 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f1ccc0455b0 con 0x7f1cd80973d0 2021-07-11T04:01:17.617 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:17.618 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":119,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:14.034771+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:01:10.890456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2021-07-11T04:01:12.008882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:17.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.615+0000 7f1ce052c700 1 -- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cb8053350 msgr2=0x7f1cb8055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.615+0000 7f1ce052c700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cb8053350 0x7f1cb8055810 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f1cd4007000 tx=0x7f1cd4008040).stop 2021-07-11T04:01:17.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.615+0000 7f1ce052c700 1 -- 172.21.2.104:0/301244787 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cd80973d0 msgr2=0x7f1cd81530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.615+0000 7f1ce052c700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cd80973d0 0x7f1cd81530f0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f1ccc001110 tx=0x7f1ccc007ad0).stop 2021-07-11T04:01:17.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.619+0000 7f1ce052c700 1 -- 172.21.2.104:0/301244787 shutdown_connections 2021-07-11T04:01:17.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.619+0000 7f1ce052c700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cd813f560 0x7f1cd8139c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.619+0000 7f1ce052c700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cd80b2660 0x7f1cd805c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.619+0000 7f1ce052c700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1cb8053350 0x7f1cb8055810 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.619+0000 7f1ce052c700 1 --2- 172.21.2.104:0/301244787 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cd80973d0 0x7f1cd81530f0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.619+0000 7f1ce052c700 1 -- 172.21.2.104:0/301244787 >> 172.21.2.104:0/301244787 conn(0x7f1cd8062f10 msgr2=0x7f1cd80b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:17.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.619+0000 7f1ce052c700 1 -- 172.21.2.104:0/301244787 shutdown_connections 2021-07-11T04:01:17.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.619+0000 7f1ce052c700 1 -- 172.21.2.104:0/301244787 wait complete. 2021-07-11T04:01:17.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:01:10.890456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:01:12.008882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:01:17.626 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:01:17.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.683+0000 7f0f36135700 1 Processor -- start 2021-07-11T04:01:17.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.683+0000 7f0f36135700 1 -- start start 2021-07-11T04:01:17.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.683+0000 7f0f36135700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0f300b3550 0x7f0f30067110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.683+0000 7f0f36135700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f30144ce0 0x7f0f30141130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.683+0000 7f0f36135700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f30066500 con 0x7f0f30064980 2021-07-11T04:01:17.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.683+0000 7f0f36135700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f300b3ea0 con 0x7f0f300b3550 2021-07-11T04:01:17.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.683+0000 7f0f36135700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f30140d40 con 0x7f0f30144ce0 2021-07-11T04:01:17.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f35934700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f30144ce0 0x7f0f30141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f35133700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f0f30064980 0x7f0f30066a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:45712/0 (socket says 172.21.2.104:45712) 2021-07-11T04:01:17.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f34932700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0f300b3550 0x7f0f30067110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f35133700 1 -- 172.21.2.104:0/4067263754 learned_addr learned my addr 172.21.2.104:0/4067263754 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:17.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f34932700 1 -- 172.21.2.104:0/4067263754 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f30144ce0 msgr2=0x7f0f30141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:17.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f34932700 1 --2- 172.21.2.104:0/4067263754 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f30144ce0 0x7f0f30141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f34932700 1 -- 172.21.2.104:0/4067263754 >> v1:172.21.2.105:6789/0 conn(0x7f0f30064980 legacy=0x7f0f30066a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f34932700 1 -- 172.21.2.104:0/4067263754 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f300b46c0 con 0x7f0f300b3550 2021-07-11T04:01:17.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f34932700 1 --2- 172.21.2.104:0/4067263754 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f300b3550 0x7f0f30067110 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f0f3013b3f0 tx=0x7f0f280091d0).ready entity=mon.1 client_cookie=e6f743674afaefd2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:17.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f1ffff700 1 -- 172.21.2.104:0/4067263754 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f2800b680 con 0x7f0f300b3550 2021-07-11T04:01:17.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f1ffff700 1 -- 172.21.2.104:0/4067263754 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2800b7e0 con 0x7f0f300b3550 2021-07-11T04:01:17.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f1ffff700 1 -- 172.21.2.104:0/4067263754 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f2800b950 con 0x7f0f300b3550 2021-07-11T04:01:17.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f36135700 1 -- 172.21.2.104:0/4067263754 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f300b3550 msgr2=0x7f0f30067110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f36135700 1 --2- 172.21.2.104:0/4067263754 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f300b3550 0x7f0f30067110 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f0f3013b3f0 tx=0x7f0f280091d0).stop 2021-07-11T04:01:17.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f36135700 1 -- 172.21.2.104:0/4067263754 shutdown_connections 2021-07-11T04:01:17.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f36135700 1 --2- 172.21.2.104:0/4067263754 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f30144ce0 0x7f0f30141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f36135700 1 --2- 172.21.2.104:0/4067263754 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f300b3550 0x7f0f30067110 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f36135700 1 -- 172.21.2.104:0/4067263754 >> 172.21.2.104:0/4067263754 conn(0x7f0f3005f6b0 msgr2=0x7f0f3005fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:17.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f36135700 1 -- 172.21.2.104:0/4067263754 shutdown_connections 2021-07-11T04:01:17.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.687+0000 7f0f36135700 1 -- 172.21.2.104:0/4067263754 wait complete. 2021-07-11T04:01:17.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f36135700 1 Processor -- start 2021-07-11T04:01:17.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f36135700 1 -- start start 2021-07-11T04:01:17.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f36135700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f30064980 0x7f0f300aedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f36135700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f300b3550 0x7f0f3013a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f36135700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f30144ce0 0x7f0f3013ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f36135700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f301427e0 con 0x7f0f30064980 2021-07-11T04:01:17.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f36135700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f300af750 con 0x7f0f300b3550 2021-07-11T04:01:17.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f36135700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f301421c0 con 0x7f0f30144ce0 2021-07-11T04:01:17.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f35133700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f30064980 0x7f0f300aedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f35133700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f30064980 0x7f0f300aedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49580/0 (socket says 172.21.2.104:49580) 2021-07-11T04:01:17.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f35133700 1 -- 172.21.2.104:0/3688423985 learned_addr learned my addr 172.21.2.104:0/3688423985 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:17.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f35133700 1 -- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f30144ce0 msgr2=0x7f0f3013ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f35133700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f30144ce0 0x7f0f3013ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f35133700 1 -- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f300b3550 msgr2=0x7f0f3013a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f35133700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f300b3550 0x7f0f3013a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f35133700 1 -- 172.21.2.104:0/3688423985 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f28009470 con 0x7f0f30064980 2021-07-11T04:01:17.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f35133700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f30064980 0x7f0f300aedb0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f0f300b7310 tx=0x7f0f20007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:17.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f1e7fc700 1 -- 172.21.2.104:0/3688423985 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f2000ce30 con 0x7f0f30064980 2021-07-11T04:01:17.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f1e7fc700 1 -- 172.21.2.104:0/3688423985 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f200072e0 con 0x7f0f30064980 2021-07-11T04:01:17.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f1e7fc700 1 -- 172.21.2.104:0/3688423985 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0f200057e0 con 0x7f0f30064980 2021-07-11T04:01:17.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f36135700 1 -- 172.21.2.104:0/3688423985 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f30142460 con 0x7f0f30064980 2021-07-11T04:01:17.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f36135700 1 -- 172.21.2.104:0/3688423985 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f301484d0 con 0x7f0f30064980 2021-07-11T04:01:17.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f36135700 1 -- 172.21.2.104:0/3688423985 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f30142950 con 0x7f0f30064980 2021-07-11T04:01:17.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f1e7fc700 1 -- 172.21.2.104:0/3688423985 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0f20010070 con 0x7f0f30064980 2021-07-11T04:01:17.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f1e7fc700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f10053240 0x7f0f10055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f1e7fc700 1 -- 172.21.2.104:0/3688423985 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0f2006ab20 con 0x7f0f30064980 2021-07-11T04:01:17.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.691+0000 7f0f34932700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f10053240 0x7f0f10055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.695+0000 7f0f34932700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f10053240 0x7f0f10055700 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f0f3013b3f0 tx=0x7f0f28016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:17.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.695+0000 7f0f1e7fc700 1 -- 172.21.2.104:0/3688423985 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0f30142950 con 0x7f0f30064980 2021-07-11T04:01:17.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 -- 172.21.2.104:0/3688423985 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0f30142920 con 0x7f0f30064980 2021-07-11T04:01:17.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f1e7fc700 1 -- 172.21.2.104:0/3688423985 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v119) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0f20041e60 con 0x7f0f30064980 2021-07-11T04:01:17.777 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:01:17.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 -- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f10053240 msgr2=0x7f0f10055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f10053240 0x7f0f10055700 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f0f3013b3f0 tx=0x7f0f28016000).stop 2021-07-11T04:01:17.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 -- 172.21.2.104:0/3688423985 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f30064980 msgr2=0x7f0f300aedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f30064980 0x7f0f300aedb0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f0f300b7310 tx=0x7f0f20007c00).stop 2021-07-11T04:01:17.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 -- 172.21.2.104:0/3688423985 shutdown_connections 2021-07-11T04:01:17.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0f30144ce0 0x7f0f3013ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0f300b3550 0x7f0f3013a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0f10053240 0x7f0f10055700 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 --2- 172.21.2.104:0/3688423985 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0f30064980 0x7f0f300aedb0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 -- 172.21.2.104:0/3688423985 >> 172.21.2.104:0/3688423985 conn(0x7f0f3005f6b0 msgr2=0x7f0f300a70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:17.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 -- 172.21.2.104:0/3688423985 shutdown_connections 2021-07-11T04:01:17.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.775+0000 7f0f36135700 1 -- 172.21.2.104:0/3688423985 wait complete. 2021-07-11T04:01:17.784 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:01:17.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.839+0000 7f6f96f70700 1 Processor -- start 2021-07-11T04:01:17.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.839+0000 7f6f96f70700 1 -- start start 2021-07-11T04:01:17.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f900acb70 0x7f6f900b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6f90140ba0 0x7f6f900b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f900ac7c0 con 0x7f6f900b10c0 2021-07-11T04:01:17.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f900ac020 con 0x7f6f90140ba0 2021-07-11T04:01:17.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90141010 con 0x7f6f900acb70 2021-07-11T04:01:17.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f94d0c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6f900b10c0 0x7f6f9013d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46356/0 (socket says 172.21.2.104:46356) 2021-07-11T04:01:17.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f94d0c700 1 -- 172.21.2.104:0/352970615 learned_addr learned my addr 172.21.2.104:0/352970615 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:17.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f9550d700 1 --2- 172.21.2.104:0/352970615 >> v2:172.21.2.104:3300/0 conn(0x7f6f90140ba0 0x7f6f900b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8f7fe700 1 -- 172.21.2.104:0/352970615 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 194682732 0 0) 0x7f6f900ac7c0 con 0x7f6f900b10c0 2021-07-11T04:01:17.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8f7fe700 1 -- 172.21.2.104:0/352970615 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f7c003260 con 0x7f6f900b10c0 2021-07-11T04:01:17.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f9550d700 1 -- 172.21.2.104:0/352970615 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f900acb70 msgr2=0x7f6f900b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:17.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f9550d700 1 --2- 172.21.2.104:0/352970615 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f900acb70 0x7f6f900b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f9550d700 1 -- 172.21.2.104:0/352970615 >> v1:172.21.2.104:6789/0 conn(0x7f6f900b10c0 legacy=0x7f6f9013d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f9550d700 1 -- 172.21.2.104:0/352970615 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f900ac280 con 0x7f6f90140ba0 2021-07-11T04:01:17.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f9550d700 1 --2- 172.21.2.104:0/352970615 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f90140ba0 0x7f6f900b24e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f6f880032f0 tx=0x7f6f880091d0).ready entity=mon.0 client_cookie=f76ef6b0ce53b21b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:17.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8f7fe700 1 -- 172.21.2.104:0/352970615 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f8800b6c0 con 0x7f6f90140ba0 2021-07-11T04:01:17.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8f7fe700 1 -- 172.21.2.104:0/352970615 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8800b820 con 0x7f6f90140ba0 2021-07-11T04:01:17.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8f7fe700 1 -- 172.21.2.104:0/352970615 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f8800b990 con 0x7f6f90140ba0 2021-07-11T04:01:17.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- 172.21.2.104:0/352970615 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f90140ba0 msgr2=0x7f6f900b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 --2- 172.21.2.104:0/352970615 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f90140ba0 0x7f6f900b24e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f6f880032f0 tx=0x7f6f880091d0).stop 2021-07-11T04:01:17.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- 172.21.2.104:0/352970615 shutdown_connections 2021-07-11T04:01:17.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 --2- 172.21.2.104:0/352970615 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f90140ba0 0x7f6f900b24e0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 --2- 172.21.2.104:0/352970615 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f900acb70 0x7f6f900b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- 172.21.2.104:0/352970615 >> 172.21.2.104:0/352970615 conn(0x7f6f9005f6b0 msgr2=0x7f6f9005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:17.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- 172.21.2.104:0/352970615 shutdown_connections 2021-07-11T04:01:17.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- 172.21.2.104:0/352970615 wait complete. 2021-07-11T04:01:17.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 Processor -- start 2021-07-11T04:01:17.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- start start 2021-07-11T04:01:17.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f900acb70 0x7f6f900ab5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b10c0 0x7f6f9015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f90140ba0 0x7f6f9014d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.855 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9015abc0 con 0x7f6f900b10c0 2021-07-11T04:01:17.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f9014e370 con 0x7f6f900acb70 2021-07-11T04:01:17.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90062bf0 con 0x7f6f90140ba0 2021-07-11T04:01:17.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b10c0 0x7f6f9015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b10c0 0x7f6f9015a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49596/0 (socket says 172.21.2.104:49596) 2021-07-11T04:01:17.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8ffff700 1 -- 172.21.2.104:0/12107659 learned_addr learned my addr 172.21.2.104:0/12107659 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:17.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8ffff700 1 -- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f90140ba0 msgr2=0x7f6f9014d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8ffff700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f90140ba0 0x7f6f9014d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8ffff700 1 -- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f900acb70 msgr2=0x7f6f900ab5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8ffff700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f900acb70 0x7f6f900ab5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8ffff700 1 -- 172.21.2.104:0/12107659 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f88009470 con 0x7f6f900b10c0 2021-07-11T04:01:17.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8ffff700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b10c0 0x7f6f9015a6d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f6f80002f70 tx=0x7f6f80006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:17.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8dffb700 1 -- 172.21.2.104:0/12107659 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f80013460 con 0x7f6f900b10c0 2021-07-11T04:01:17.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8dffb700 1 -- 172.21.2.104:0/12107659 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f800135c0 con 0x7f6f900b10c0 2021-07-11T04:01:17.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- 172.21.2.104:0/12107659 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f90062e90 con 0x7f6f900b10c0 2021-07-11T04:01:17.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f8dffb700 1 -- 172.21.2.104:0/12107659 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f80013920 con 0x7f6f900b10c0 2021-07-11T04:01:17.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.843+0000 7f6f96f70700 1 -- 172.21.2.104:0/12107659 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f900b35c0 con 0x7f6f900b10c0 2021-07-11T04:01:17.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.847+0000 7f6f96f70700 1 -- 172.21.2.104:0/12107659 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f90063020 con 0x7f6f900b10c0 2021-07-11T04:01:17.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.847+0000 7f6f8dffb700 1 -- 172.21.2.104:0/12107659 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6f8001a060 con 0x7f6f900b10c0 2021-07-11T04:01:17.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.847+0000 7f6f8dffb700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f70053210 0x7f6f700556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.847+0000 7f6f8dffb700 1 -- 172.21.2.104:0/12107659 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6f8006ae70 con 0x7f6f900b10c0 2021-07-11T04:01:17.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.847+0000 7f6f94d0c700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f70053210 0x7f6f700556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.847+0000 7f6f94d0c700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f70053210 0x7f6f700556d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f6f84001110 tx=0x7f6f840076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:17.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.847+0000 7f6f8dffb700 1 -- 172.21.2.104:0/12107659 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6f800421b0 con 0x7f6f900b10c0 2021-07-11T04:01:17.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 -- 172.21.2.104:0/12107659 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6f90057350 con 0x7f6f900b10c0 2021-07-11T04:01:17.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f8dffb700 1 -- 172.21.2.104:0/12107659 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v119) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6f80018070 con 0x7f6f900b10c0 2021-07-11T04:01:17.933 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:17.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 -- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f70053210 msgr2=0x7f6f700556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f70053210 0x7f6f700556d0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f6f84001110 tx=0x7f6f840076c0).stop 2021-07-11T04:01:17.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 -- 172.21.2.104:0/12107659 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b10c0 msgr2=0x7f6f9015a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b10c0 0x7f6f9015a6d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f6f80002f70 tx=0x7f6f80006ec0).stop 2021-07-11T04:01:17.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 -- 172.21.2.104:0/12107659 shutdown_connections 2021-07-11T04:01:17.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f90140ba0 0x7f6f9014d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f70053210 0x7f6f700556d0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b10c0 0x7f6f9015a6d0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 --2- 172.21.2.104:0/12107659 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f900acb70 0x7f6f900ab5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:17.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 -- 172.21.2.104:0/12107659 >> 172.21.2.104:0/12107659 conn(0x7f6f9005f6b0 msgr2=0x7f6f900a9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:17.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 -- 172.21.2.104:0/12107659 shutdown_connections 2021-07-11T04:01:17.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.931+0000 7f6f96f70700 1 -- 172.21.2.104:0/12107659 wait complete. 2021-07-11T04:01:17.940 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:01:17.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.991+0000 7f640f4ef700 1 Processor -- start 2021-07-11T04:01:17.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.991+0000 7f640f4ef700 1 -- start start 2021-07-11T04:01:17.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640f4ef700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f64080a95d0 0x7f64080a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640f4ef700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64080b2c00 0x7f64080b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:17.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640f4ef700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6408139bc0 con 0x7f64080aca50 2021-07-11T04:01:17.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640f4ef700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6408140500 con 0x7f64080a95d0 2021-07-11T04:01:17.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640f4ef700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64080b0130 con 0x7f64080b2c00 2021-07-11T04:01:17.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640da8c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64080b2c00 0x7f64080b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640d28b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f64080a95d0 0x7f64080a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:17.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640da8c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64080b2c00 0x7f64080b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:52770/0 (socket says 172.21.2.104:52770) 2021-07-11T04:01:17.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640da8c700 1 -- 172.21.2.104:0/2558714207 learned_addr learned my addr 172.21.2.104:0/2558714207 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:17.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640d28b700 1 -- 172.21.2.104:0/2558714207 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64080b2c00 msgr2=0x7f64080b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:17.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640d28b700 1 --2- 172.21.2.104:0/2558714207 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64080b2c00 0x7f64080b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:17.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640d28b700 1 -- 172.21.2.104:0/2558714207 >> v1:172.21.2.105:6789/0 conn(0x7f64080aca50 legacy=0x7f64080a8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:17.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640d28b700 1 -- 172.21.2.104:0/2558714207 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64080b03d0 con 0x7f64080a95d0 2021-07-11T04:01:17.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640d28b700 1 --2- 172.21.2.104:0/2558714207 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64080a95d0 0x7f64080a99b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f6404002f70 tx=0x7f6404006df0).ready entity=mon.1 client_cookie=36653f040883fe40 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:17.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f63f7fff700 1 -- 172.21.2.104:0/2558714207 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640400d570 con 0x7f64080a95d0 2021-07-11T04:01:18.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f63f7fff700 1 -- 172.21.2.104:0/2558714207 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6404000e90 con 0x7f64080a95d0 2021-07-11T04:01:18.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f63f7fff700 1 -- 172.21.2.104:0/2558714207 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640400d8c0 con 0x7f64080a95d0 2021-07-11T04:01:18.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640f4ef700 1 -- 172.21.2.104:0/2558714207 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64080a95d0 msgr2=0x7f64080a99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.995+0000 7f640f4ef700 1 --2- 172.21.2.104:0/2558714207 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64080a95d0 0x7f64080a99b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f6404002f70 tx=0x7f6404006df0).stop 2021-07-11T04:01:18.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 -- 172.21.2.104:0/2558714207 shutdown_connections 2021-07-11T04:01:18.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 --2- 172.21.2.104:0/2558714207 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64080b2c00 0x7f64080b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 --2- 172.21.2.104:0/2558714207 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64080a95d0 0x7f64080a99b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 -- 172.21.2.104:0/2558714207 >> 172.21.2.104:0/2558714207 conn(0x7f640805f6b0 msgr2=0x7f640805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:18.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 -- 172.21.2.104:0/2558714207 shutdown_connections 2021-07-11T04:01:18.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 -- 172.21.2.104:0/2558714207 wait complete. 2021-07-11T04:01:18.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 Processor -- start 2021-07-11T04:01:18.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 -- start start 2021-07-11T04:01:18.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64080a95d0 0x7f64080ab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64080aca50 0x7f6408157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64080b2c00 0x7f64080a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080b3bb0 con 0x7f64080b2c00 2021-07-11T04:01:18.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64080a7160 con 0x7f64080a95d0 2021-07-11T04:01:18.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6408157750 con 0x7f64080aca50 2021-07-11T04:01:18.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640da8c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64080b2c00 0x7f64080a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:18.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640da8c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64080b2c00 0x7f64080a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49610/0 (socket says 172.21.2.104:49610) 2021-07-11T04:01:18.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640da8c700 1 -- 172.21.2.104:0/3212649909 learned_addr learned my addr 172.21.2.104:0/3212649909 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:18.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640da8c700 1 -- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64080aca50 msgr2=0x7f6408157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640da8c700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64080aca50 0x7f6408157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:18.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640da8c700 1 -- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64080a95d0 msgr2=0x7f64080ab5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640da8c700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64080a95d0 0x7f64080ab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:18.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640da8c700 1 -- 172.21.2.104:0/3212649909 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640400b180 con 0x7f64080b2c00 2021-07-11T04:01:18.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640da8c700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64080b2c00 0x7f64080a75c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f640000c000 tx=0x7f6400009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:18.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f63f67fc700 1 -- 172.21.2.104:0/3212649909 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640000c6a0 con 0x7f64080b2c00 2021-07-11T04:01:18.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f63f67fc700 1 -- 172.21.2.104:0/3212649909 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640000c800 con 0x7f64080b2c00 2021-07-11T04:01:18.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f63f67fc700 1 -- 172.21.2.104:0/3212649909 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f640000cb60 con 0x7f64080b2c00 2021-07-11T04:01:18.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 -- 172.21.2.104:0/3212649909 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64080abf00 con 0x7f64080b2c00 2021-07-11T04:01:18.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 -- 172.21.2.104:0/3212649909 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64080b3740 con 0x7f64080b2c00 2021-07-11T04:01:18.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f640f4ef700 1 -- 172.21.2.104:0/3212649909 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6408057350 con 0x7f64080b2c00 2021-07-11T04:01:18.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f63f67fc700 1 -- 172.21.2.104:0/3212649909 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6400022070 con 0x7f64080b2c00 2021-07-11T04:01:18.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f63f67fc700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f63e8053300 0x7f63e80557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:17.999+0000 7f63f67fc700 1 -- 172.21.2.104:0/3212649909 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f640006bce0 con 0x7f64080b2c00 2021-07-11T04:01:18.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.003+0000 7f640d28b700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f63e8053300 0x7f63e80557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:18.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.003+0000 7f640d28b700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f63e8053300 0x7f63e80557c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f6404002f70 tx=0x7f6404014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:18.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.003+0000 7f63f67fc700 1 -- 172.21.2.104:0/3212649909 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6400042f10 con 0x7f64080b2c00 2021-07-11T04:01:18.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.083+0000 7f640f4ef700 1 -- 172.21.2.104:0/3212649909 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6408044650 con 0x7f64080b2c00 2021-07-11T04:01:18.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.083+0000 7f63f67fc700 1 -- 172.21.2.104:0/3212649909 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v119) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6400016020 con 0x7f64080b2c00 2021-07-11T04:01:18.086 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:18.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 -- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f63e8053300 msgr2=0x7f63e80557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f63e8053300 0x7f63e80557c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f6404002f70 tx=0x7f6404014000).stop 2021-07-11T04:01:18.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 -- 172.21.2.104:0/3212649909 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64080b2c00 msgr2=0x7f64080a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64080b2c00 0x7f64080a75c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f640000c000 tx=0x7f6400009290).stop 2021-07-11T04:01:18.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 -- 172.21.2.104:0/3212649909 shutdown_connections 2021-07-11T04:01:18.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f63e8053300 0x7f63e80557c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64080b2c00 0x7f64080a75c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64080aca50 0x7f6408157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:18.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 --2- 172.21.2.104:0/3212649909 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64080a95d0 0x7f64080ab5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:18.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 -- 172.21.2.104:0/3212649909 >> 172.21.2.104:0/3212649909 conn(0x7f640805f6b0 msgr2=0x7f64081434b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:18.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 -- 172.21.2.104:0/3212649909 shutdown_connections 2021-07-11T04:01:18.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.087+0000 7f640f4ef700 1 -- 172.21.2.104:0/3212649909 wait complete. 2021-07-11T04:01:18.093 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:01:18.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 Processor -- start 2021-07-11T04:01:18.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 -- start start 2021-07-11T04:01:18.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc5b40b12d0 0x7fc5b40affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b40a77d0 con 0x7fc5b40b22a0 2021-07-11T04:01:18.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b40b0650 con 0x7fc5b40b1ec0 2021-07-11T04:01:18.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b40b1740 con 0x7fc5b40b12d0 2021-07-11T04:01:18.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5ba1ed700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc5b40b22a0 0x7fc5b4141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46386/0 (socket says 172.21.2.104:46386) 2021-07-11T04:01:18.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5ba1ed700 1 -- 172.21.2.104:0/620404972 learned_addr learned my addr 172.21.2.104:0/620404972 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:18.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5b91eb700 1 --2- 172.21.2.104:0/620404972 >> v2:172.21.2.104:3300/0 conn(0x7fc5b40b12d0 0x7fc5b40affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:18.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5b91eb700 1 -- 172.21.2.104:0/620404972 >> v1:172.21.2.105:6789/0 conn(0x7fc5b40b1ec0 legacy=0x7fc5b40a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:18.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5b89ea700 1 -- 172.21.2.104:0/620404972 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070154155 0 0) 0x7fc5b40a77d0 con 0x7fc5b40b22a0 2021-07-11T04:01:18.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5b91eb700 1 -- 172.21.2.104:0/620404972 >> v1:172.21.2.104:6789/0 conn(0x7fc5b40b22a0 legacy=0x7fc5b4141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5b91eb700 1 -- 172.21.2.104:0/620404972 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b40b08f0 con 0x7fc5b40b12d0 2021-07-11T04:01:18.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5b91eb700 1 --2- 172.21.2.104:0/620404972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b40b12d0 0x7fc5b40affb0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fc5b40b2f50 tx=0x7fc5b0006df0).ready entity=mon.0 client_cookie=f8795c2f6d44c53d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:18.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5b89ea700 1 -- 172.21.2.104:0/620404972 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5b000d5c0 con 0x7fc5b40b12d0 2021-07-11T04:01:18.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5b89ea700 1 -- 172.21.2.104:0/620404972 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b0000e90 con 0x7fc5b40b12d0 2021-07-11T04:01:18.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5b89ea700 1 -- 172.21.2.104:0/620404972 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5b000d910 con 0x7fc5b40b12d0 2021-07-11T04:01:18.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 -- 172.21.2.104:0/620404972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b40b12d0 msgr2=0x7fc5b40affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 --2- 172.21.2.104:0/620404972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b40b12d0 0x7fc5b40affb0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7fc5b40b2f50 tx=0x7fc5b0006df0).stop 2021-07-11T04:01:18.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 -- 172.21.2.104:0/620404972 shutdown_connections 2021-07-11T04:01:18.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 --2- 172.21.2.104:0/620404972 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b40b12d0 0x7fc5b40affb0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 -- 172.21.2.104:0/620404972 >> 172.21.2.104:0/620404972 conn(0x7fc5b405f6b0 msgr2=0x7fc5b405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:18.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 -- 172.21.2.104:0/620404972 shutdown_connections 2021-07-11T04:01:18.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 -- 172.21.2.104:0/620404972 wait complete. 2021-07-11T04:01:18.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 Processor -- start 2021-07-11T04:01:18.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.147+0000 7fc5bbc50700 1 -- start start 2021-07-11T04:01:18.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5bbc50700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b40b12d0 0x7fc5b413a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5bbc50700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5b40b1ec0 0x7fc5b40a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5bbc50700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5b40b22a0 0x7fc5b40a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5bbc50700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40af0a0 con 0x7fc5b40b12d0 2021-07-11T04:01:18.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5bbc50700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40af7c0 con 0x7fc5b40b1ec0 2021-07-11T04:01:18.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5bbc50700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b40afa20 con 0x7fc5b40b22a0 2021-07-11T04:01:18.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5b99ec700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b40b12d0 0x7fc5b413a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:18.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5b99ec700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b40b12d0 0x7fc5b413a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49626/0 (socket says 172.21.2.104:49626) 2021-07-11T04:01:18.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5b99ec700 1 -- 172.21.2.104:0/2107613388 learned_addr learned my addr 172.21.2.104:0/2107613388 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:18.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5b91eb700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5b40b1ec0 0x7fc5b40a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:18.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5b99ec700 1 -- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5b40b22a0 msgr2=0x7fc5b40a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5b99ec700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5b40b22a0 0x7fc5b40a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:18.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5b99ec700 1 -- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5b40b1ec0 msgr2=0x7fc5b40a9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5b99ec700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5b40b1ec0 0x7fc5b40a9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5b99ec700 1 -- 172.21.2.104:0/2107613388 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b000b180 con 0x7fc5b40b12d0 2021-07-11T04:01:18.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5b99ec700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b40b12d0 0x7fc5b413a980 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7fc5a8000f70 tx=0x7fc5a8007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:18.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5a6ffd700 1 -- 172.21.2.104:0/2107613388 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5a800ce80 con 0x7fc5b40b12d0 2021-07-11T04:01:18.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5a6ffd700 1 -- 172.21.2.104:0/2107613388 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a8013450 con 0x7fc5b40b12d0 2021-07-11T04:01:18.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5bbc50700 1 -- 172.21.2.104:0/2107613388 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b40af2d0 con 0x7fc5b40b12d0 2021-07-11T04:01:18.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5a6ffd700 1 -- 172.21.2.104:0/2107613388 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5a80137b0 con 0x7fc5b40b12d0 2021-07-11T04:01:18.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5bbc50700 1 -- 172.21.2.104:0/2107613388 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b413b430 con 0x7fc5b40b12d0 2021-07-11T04:01:18.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5bbc50700 1 -- 172.21.2.104:0/2107613388 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b40aa630 con 0x7fc5b40b12d0 2021-07-11T04:01:18.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5a6ffd700 1 -- 172.21.2.104:0/2107613388 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc5a801a060 con 0x7fc5b40b12d0 2021-07-11T04:01:18.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5a6ffd700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053350 0x7fc594055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5a6ffd700 1 -- 172.21.2.104:0/2107613388 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc5a806af80 con 0x7fc5b40b12d0 2021-07-11T04:01:18.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.151+0000 7fc5b91eb700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053350 0x7fc594055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:18.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.155+0000 7fc5b91eb700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053350 0x7fc594055810 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fc5b0013000 tx=0x7fc5b0012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:18.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.155+0000 7fc5a6ffd700 1 -- 172.21.2.104:0/2107613388 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc5a80421b0 con 0x7fc5b40b12d0 2021-07-11T04:01:18.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.235+0000 7fc5bbc50700 1 -- 172.21.2.104:0/2107613388 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5b4057350 con 0x7fc5b40b12d0 2021-07-11T04:01:18.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5a6ffd700 1 -- 172.21.2.104:0/2107613388 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v119) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc5a8018070 con 0x7fc5b40b12d0 2021-07-11T04:01:18.239 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:18.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 -- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053350 msgr2=0x7fc594055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053350 0x7fc594055810 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fc5b0013000 tx=0x7fc5b0012000).stop 2021-07-11T04:01:18.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 -- 172.21.2.104:0/2107613388 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b40b12d0 msgr2=0x7fc5b413a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.240 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b40b12d0 0x7fc5b413a980 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7fc5a8000f70 tx=0x7fc5a8007c30).stop 2021-07-11T04:01:18.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 -- 172.21.2.104:0/2107613388 shutdown_connections 2021-07-11T04:01:18.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5b40b22a0 0x7fc5b40a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:18.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5b40b1ec0 0x7fc5b40a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.241 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc594053350 0x7fc594055810 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 --2- 172.21.2.104:0/2107613388 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5b40b12d0 0x7fc5b413a980 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 -- 172.21.2.104:0/2107613388 >> 172.21.2.104:0/2107613388 conn(0x7fc5b405f6b0 msgr2=0x7fc5b413ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:18.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 -- 172.21.2.104:0/2107613388 shutdown_connections 2021-07-11T04:01:18.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.239+0000 7fc5bbc50700 1 -- 172.21.2.104:0/2107613388 wait complete. 2021-07-11T04:01:18.246 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:01:18.247 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:18.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.295+0000 7f21823ba700 1 Processor -- start 2021-07-11T04:01:18.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.295+0000 7f21823ba700 1 -- start start 2021-07-11T04:01:18.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f217c0973d0 0x7f217c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f217c13f560 0x7f217c13fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c0b2a40 con 0x7f217c0b2660 2021-07-11T04:01:18.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217c097b80 con 0x7f217c13f560 2021-07-11T04:01:18.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c097e20 con 0x7f217c0973d0 2021-07-11T04:01:18.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f217b7fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f217c0973d0 0x7f217c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:18.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f2180957700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f217c13f560 0x7f217c13fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:18.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f2180957700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f217c13f560 0x7f217c13fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:57192/0 (socket says 172.21.2.104:57192) 2021-07-11T04:01:18.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f217b7fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f217c0973d0 0x7f217c05f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:52798/0 (socket says 172.21.2.104:52798) 2021-07-11T04:01:18.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f2180957700 1 -- 172.21.2.104:0/3592550516 learned_addr learned my addr 172.21.2.104:0/3592550516 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:18.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f217b7fe700 1 -- 172.21.2.104:0/3592550516 >> v2:172.21.2.105:3300/0 conn(0x7f217c13f560 msgr2=0x7f217c13fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:18.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f217b7fe700 1 --2- 172.21.2.104:0/3592550516 >> v2:172.21.2.105:3300/0 conn(0x7f217c13f560 0x7f217c13fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f217b7fe700 1 -- 172.21.2.104:0/3592550516 >> v1:172.21.2.105:6789/0 conn(0x7f217c0b2660 legacy=0x7f217c096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f217b7fe700 1 -- 172.21.2.104:0/3592550516 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0980c0 con 0x7f217c0973d0 2021-07-11T04:01:18.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f217b7fe700 1 --2- 172.21.2.104:0/3592550516 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f217c0973d0 0x7f217c05f6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f216c002f70 tx=0x7f216c006df0).ready entity=mon.2 client_cookie=a87d9b7ffae42d05 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:18.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f217affd700 1 -- 172.21.2.104:0/3592550516 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f216c00d540 con 0x7f217c0973d0 2021-07-11T04:01:18.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f217affd700 1 -- 172.21.2.104:0/3592550516 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c000e90 con 0x7f217c0973d0 2021-07-11T04:01:18.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f217affd700 1 -- 172.21.2.104:0/3592550516 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f216c00d890 con 0x7f217c0973d0 2021-07-11T04:01:18.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 -- 172.21.2.104:0/3592550516 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f217c0973d0 msgr2=0x7f217c05f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 --2- 172.21.2.104:0/3592550516 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f217c0973d0 0x7f217c05f6b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f216c002f70 tx=0x7f216c006df0).stop 2021-07-11T04:01:18.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 -- 172.21.2.104:0/3592550516 shutdown_connections 2021-07-11T04:01:18.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 --2- 172.21.2.104:0/3592550516 >> v2:172.21.2.105:3300/0 conn(0x7f217c13f560 0x7f217c13fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 --2- 172.21.2.104:0/3592550516 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f217c0973d0 0x7f217c05f6b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 -- 172.21.2.104:0/3592550516 >> 172.21.2.104:0/3592550516 conn(0x7f217c062f10 msgr2=0x7f217c0632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:18.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 -- 172.21.2.104:0/3592550516 shutdown_connections 2021-07-11T04:01:18.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.299+0000 7f21823ba700 1 -- 172.21.2.104:0/3592550516 wait complete. 2021-07-11T04:01:18.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21823ba700 1 Processor -- start 2021-07-11T04:01:18.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21823ba700 1 -- start start 2021-07-11T04:01:18.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21823ba700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f217c0973d0 0x7f217c0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21823ba700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f217c0b2660 0x7f217c156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21823ba700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f217c13f560 0x7f217c157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21823ba700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0ac6b0 con 0x7f217c0b2660 2021-07-11T04:01:18.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21823ba700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c157940 con 0x7f217c0973d0 2021-07-11T04:01:18.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21823ba700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c064600 con 0x7f217c13f560 2021-07-11T04:01:18.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f217b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f217c0b2660 0x7f217c156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:18.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f217b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f217c0b2660 0x7f217c156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49638/0 (socket says 172.21.2.104:49638) 2021-07-11T04:01:18.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f217b7fe700 1 -- 172.21.2.104:0/1835831423 learned_addr learned my addr 172.21.2.104:0/1835831423 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:18.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f217b7fe700 1 -- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f217c13f560 msgr2=0x7f217c157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f217b7fe700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f217c13f560 0x7f217c157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:18.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f217b7fe700 1 -- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f217c0973d0 msgr2=0x7f217c0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f217b7fe700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f217c0973d0 0x7f217c0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:18.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f217b7fe700 1 -- 172.21.2.104:0/1835831423 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c00c290 con 0x7f217c0b2660 2021-07-11T04:01:18.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f217b7fe700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f217c0b2660 0x7f217c156f60 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f216c002f70 tx=0x7f216c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:18.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21797fa700 1 -- 172.21.2.104:0/1835831423 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f216c010ce0 con 0x7f217c0b2660 2021-07-11T04:01:18.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21797fa700 1 -- 172.21.2.104:0/1835831423 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216c000e90 con 0x7f217c0b2660 2021-07-11T04:01:18.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21823ba700 1 -- 172.21.2.104:0/1835831423 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c065ba0 con 0x7f217c0b2660 2021-07-11T04:01:18.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21797fa700 1 -- 172.21.2.104:0/1835831423 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f216c01f5f0 con 0x7f217c0b2660 2021-07-11T04:01:18.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21823ba700 1 -- 172.21.2.104:0/1835831423 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c0660e0 con 0x7f217c0b2660 2021-07-11T04:01:18.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21823ba700 1 -- 172.21.2.104:0/1835831423 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c057350 con 0x7f217c0b2660 2021-07-11T04:01:18.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21797fa700 1 -- 172.21.2.104:0/1835831423 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f216c01d070 con 0x7f217c0b2660 2021-07-11T04:01:18.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21797fa700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2160053270 0x7f2160055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:18.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f21797fa700 1 -- 172.21.2.104:0/1835831423 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f216c06bf80 con 0x7f217c0b2660 2021-07-11T04:01:18.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.303+0000 7f217bfff700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2160053270 0x7f2160055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:18.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.307+0000 7f217bfff700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2160053270 0x7f2160055730 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f217c0b20e0 tx=0x7f2164007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:18.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.307+0000 7f21797fa700 1 -- 172.21.2.104:0/1835831423 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f216c0431b0 con 0x7f217c0b2660 2021-07-11T04:01:18.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 -- 172.21.2.104:0/1835831423 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f217c095300 con 0x7f217c0b2660 2021-07-11T04:01:18.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21797fa700 1 -- 172.21.2.104:0/1835831423 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+2495 (secure 0 0 0) 0x7f216c014070 con 0x7f217c0b2660 2021-07-11T04:01:18.416 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:18.416 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":57,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4901,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/3151473850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":3151473850},{"type":"v1","addr":"172.21.2.105:6835","nonce":3151473850}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54},{"gid":7617,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/4259840781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":4259840781},{"type":"v1","addr":"172.21.2.104:6834","nonce":4259840781}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":54}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:13.031418+0000","modified":"2021-07-11T04:01:14.344351+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7620},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7620":{"gid":7620,"name":"c","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/1367521056","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1367521056},{"type":"v1","addr":"172.21.2.104:6835","nonce":1367521056}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2021-07-11T04:01:18.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 -- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2160053270 msgr2=0x7f2160055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2160053270 0x7f2160055730 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f217c0b20e0 tx=0x7f2164007810).stop 2021-07-11T04:01:18.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 -- 172.21.2.104:0/1835831423 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f217c0b2660 msgr2=0x7f217c156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:18.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f217c0b2660 0x7f217c156f60 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f216c002f70 tx=0x7f216c015000).stop 2021-07-11T04:01:18.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 -- 172.21.2.104:0/1835831423 shutdown_connections 2021-07-11T04:01:18.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f217c13f560 0x7f217c157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:18.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2160053270 0x7f2160055730 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f217c0b2660 0x7f217c156f60 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:18.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 --2- 172.21.2.104:0/1835831423 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f217c0973d0 0x7f217c0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:18.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 -- 172.21.2.104:0/1835831423 >> 172.21.2.104:0/1835831423 conn(0x7f217c062f10 msgr2=0x7f217c0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:18.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 -- 172.21.2.104:0/1835831423 shutdown_connections 2021-07-11T04:01:18.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:18.415+0000 7f21823ba700 1 -- 172.21.2.104:0/1835831423 wait complete. 2021-07-11T04:01:18.420 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 57 2021-07-11T04:01:18.725 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 57, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:01:13.031418+0000', 'modified': '2021-07-11T04:01:14.344351+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7620}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7620': {'gid': 7620, 'name': 'c', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/1367521056', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 1367521056}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 1367521056}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:01:18.726 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:01:18.726 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:01:18.793 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":7620,"want_state":"up:active","state":"up:active","rank_uptime":5.7565034339999999,"mdsmap_epoch":57,"osdmap_epoch":119,"osdmap_epoch_barrier":119,"uptime":10.010298395} 2021-07-11T04:01:18.794 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 7620, "mdsmap_epoch": 57, "osdmap_epoch": 119, "osdmap_epoch_barrier": 119, "rank_uptime": 5.756503434, "state": "up:active", "uptime": 10.010298395, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:01:18.794 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:01:18.794 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-07-11T04:01:18.795 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba023.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-07-11T04:01:18.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:18.796 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:18.806 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:01:18.858 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:18.859 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:18.865 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:18.865 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:18.866 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:18.867 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:01:18.870 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:01:18.871 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:18.914 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:01:18.923 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:18.923 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:18.928 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:18.929 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:18.930 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:18.930 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:01:18.953 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15000]: starting ceph client 2021-07-11T04:01:18.954 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:2021-07-11T04:01:18.953+0000 7fc047985080 -1 init, newargv = 0x55ecbae00bc0 newargc=9 2021-07-11T04:01:18.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15000]: starting fuse 2021-07-11T04:01:20.059 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:01:20.067 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:20.068 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:20.118 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:20.119 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:20.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:20.120 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:01:20.124 INFO:teuthology.orchestra.run.gibba023.stdout:52 2021-07-11T04:01:20.124 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:01:20.125 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:20.125 DEBUG:teuthology.orchestra.run.gibba023:> sudo python3 -c ' 2021-07-11T04:01:20.126 DEBUG:teuthology.orchestra.run.gibba023:> import glob 2021-07-11T04:01:20.126 DEBUG:teuthology.orchestra.run.gibba023:> import re 2021-07-11T04:01:20.126 DEBUG:teuthology.orchestra.run.gibba023:> import os 2021-07-11T04:01:20.127 DEBUG:teuthology.orchestra.run.gibba023:> import subprocess 2021-07-11T04:01:20.127 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:01:20.128 DEBUG:teuthology.orchestra.run.gibba023:> def find_socket(client_name): 2021-07-11T04:01:20.128 DEBUG:teuthology.orchestra.run.gibba023:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-07-11T04:01:20.128 DEBUG:teuthology.orchestra.run.gibba023:> files = glob.glob(asok_path) 2021-07-11T04:01:20.129 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:01:20.129 DEBUG:teuthology.orchestra.run.gibba023:> # Given a non-glob path, it better be there 2021-07-11T04:01:20.130 DEBUG:teuthology.orchestra.run.gibba023:> if "*" not in asok_path: 2021-07-11T04:01:20.130 DEBUG:teuthology.orchestra.run.gibba023:> assert(len(files) == 1) 2021-07-11T04:01:20.130 DEBUG:teuthology.orchestra.run.gibba023:> return files[0] 2021-07-11T04:01:20.131 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:01:20.131 DEBUG:teuthology.orchestra.run.gibba023:> for f in files: 2021-07-11T04:01:20.132 DEBUG:teuthology.orchestra.run.gibba023:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:01:20.132 DEBUG:teuthology.orchestra.run.gibba023:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:01:20.132 DEBUG:teuthology.orchestra.run.gibba023:> return f 2021-07-11T04:01:20.133 DEBUG:teuthology.orchestra.run.gibba023:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:01:20.133 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:01:20.134 DEBUG:teuthology.orchestra.run.gibba023:> print(find_socket("client.1")) 2021-07-11T04:01:20.134 DEBUG:teuthology.orchestra.run.gibba023:> ' 2021-07-11T04:01:20.155 INFO:teuthology.orchestra.run.gibba023.stdout:/var/run/ceph/ceph-client.1.15000.asok 2021-07-11T04:01:20.157 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15000.asok 2021-07-11T04:01:20.158 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:20.158 DEBUG:teuthology.orchestra.run.gibba023:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15000.asok status 2021-07-11T04:01:20.263 INFO:teuthology.orchestra.run.gibba023.stdout:{ 2021-07-11T04:01:20.263 INFO:teuthology.orchestra.run.gibba023.stdout: "metadata": { 2021-07-11T04:01:20.264 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:01:20.264 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:01:20.264 INFO:teuthology.orchestra.run.gibba023.stdout: "entity_id": "1", 2021-07-11T04:01:20.264 INFO:teuthology.orchestra.run.gibba023.stdout: "hostname": "gibba023", 2021-07-11T04:01:20.265 INFO:teuthology.orchestra.run.gibba023.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-07-11T04:01:20.265 INFO:teuthology.orchestra.run.gibba023.stdout: "pid": "15000", 2021-07-11T04:01:20.265 INFO:teuthology.orchestra.run.gibba023.stdout: "root": "/" 2021-07-11T04:01:20.265 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:01:20.265 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_count": 0, 2021-07-11T04:01:20.266 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_pinned_count": 0, 2021-07-11T04:01:20.266 INFO:teuthology.orchestra.run.gibba023.stdout: "id": 4952, 2021-07-11T04:01:20.266 INFO:teuthology.orchestra.run.gibba023.stdout: "inst": { 2021-07-11T04:01:20.266 INFO:teuthology.orchestra.run.gibba023.stdout: "name": { 2021-07-11T04:01:20.266 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "client", 2021-07-11T04:01:20.267 INFO:teuthology.orchestra.run.gibba023.stdout: "num": 4952 2021-07-11T04:01:20.267 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:01:20.267 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:01:20.267 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:01:20.268 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:01:20.268 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 4094451698 2021-07-11T04:01:20.268 INFO:teuthology.orchestra.run.gibba023.stdout: } 2021-07-11T04:01:20.268 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:01:20.269 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:01:20.269 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:01:20.269 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:01:20.269 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 4094451698 2021-07-11T04:01:20.269 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:01:20.270 INFO:teuthology.orchestra.run.gibba023.stdout: "inst_str": "client.4952 172.21.2.123:0/4094451698", 2021-07-11T04:01:20.270 INFO:teuthology.orchestra.run.gibba023.stdout: "addr_str": "172.21.2.123:0/4094451698", 2021-07-11T04:01:20.270 INFO:teuthology.orchestra.run.gibba023.stdout: "inode_count": 1, 2021-07-11T04:01:20.270 INFO:teuthology.orchestra.run.gibba023.stdout: "mds_epoch": 57, 2021-07-11T04:01:20.271 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch": 119, 2021-07-11T04:01:20.271 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:01:20.271 INFO:teuthology.orchestra.run.gibba023.stdout: "blacklisted": false, 2021-07-11T04:01:20.271 INFO:teuthology.orchestra.run.gibba023.stdout: "fs_name": "cephfs" 2021-07-11T04:01:20.271 INFO:teuthology.orchestra.run.gibba023.stdout:} 2021-07-11T04:01:20.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:20.273 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:20.274 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:20.279 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:01:20.280 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:01:20.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:20.280 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:20.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:20.330 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:20.330 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:20.352 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:01:20.353 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:01:20.353 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:20.354 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:20.354 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:20.366 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:01:20.367 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:01:20.367 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T04:01:20.368 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:01:20.368 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:20.422 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15566]: fuse finished with error 0 and tester_r 0 2021-07-11T04:01:20.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:20.424 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:20.425 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:20.429 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T04:01:20.429 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:01:20.430 INFO:teuthology.orchestra.run:waiting for 900 2021-07-11T04:01:20.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:01:20.493 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:20.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:20.499 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:20.499 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:20.511 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:01:20.512 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:01:20.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:20.513 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:20.513 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:20.524 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:01:20.525 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:01:20.526 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba023.front.sepia.ceph.com... 2021-07-11T04:01:20.526 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:01:20.527 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:20.581 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15000]: fuse finished with error 0 and tester_r 0 2021-07-11T04:01:20.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:20.582 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:20.583 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:20.587 INFO:teuthology.orchestra.run.gibba023.stdout:ext2/ext3 2021-07-11T04:01:20.587 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:01:20.588 INFO:teuthology.orchestra.run:waiting for 900 2021-07-11T04:01:26.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:01:26.691 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:26.702 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T04:01:26.702 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:01:26.703 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T04:01:26.703 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:01:26.704 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T04:01:26.704 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:01:26.712 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T04:01:26.713 DEBUG:teuthology.parallel:result is None 2021-07-11T04:01:26.714 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T04:01:26.715 DEBUG:teuthology.parallel:result is None 2021-07-11T04:01:26.716 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T04:01:26.716 DEBUG:teuthology.parallel:result is None 2021-07-11T04:01:26.717 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2021-07-11T04:01:26.717 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2021-07-11T04:01:26.718 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2021-07-11T04:01:26.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f8c6d6e1700 1 Processor -- start 2021-07-11T04:01:26.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7faac0a0c700 1 Processor -- start 2021-07-11T04:01:26.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b4d7a8700 1 Processor -- start 2021-07-11T04:01:26.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b4d7a8700 1 -- start start 2021-07-11T04:01:26.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7faac0a0c700 1 -- start start 2021-07-11T04:01:26.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f8c6d6e1700 1 -- start start 2021-07-11T04:01:26.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b4d7a8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b480b3240 0x7f6b48066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b4d7a8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6b480b0e70 0x7f6b48066b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b4d7a8700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b480b3620 con 0x7f6b480b1840 2021-07-11T04:01:26.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b4d7a8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b481436f0 con 0x7f6b480b0e70 2021-07-11T04:01:26.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b4d7a8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b48065ca0 con 0x7f6b480b3240 2021-07-11T04:01:26.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b477fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6b480b0e70 0x7f6b48066b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:26.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b477fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6b480b0e70 0x7f6b48066b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49644/0 (socket says 172.21.2.104:49644) 2021-07-11T04:01:26.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b477fe700 1 -- 172.21.2.104:0/1277023702 learned_addr learned my addr 172.21.2.104:0/1277023702 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:26.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7faac0a0c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faabc066e00 0x7faabc0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f8c6d6e1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680b04f0 0x7f8c680b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f8c6d6e1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8c680afd40 0x7f8c6813fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7faac0a0c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faabc0aaf10 0x7faabc143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f8c6d6e1700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c680ad5a0 con 0x7f8c680af960 2021-07-11T04:01:26.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7faac0a0c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc0671e0 con 0x7faabc060d00 2021-07-11T04:01:26.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f8c6d6e1700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c680af460 con 0x7f8c680afd40 2021-07-11T04:01:26.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7faac0a0c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faabc0ab340 con 0x7faabc0aaf10 2021-07-11T04:01:26.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7faac0a0c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0a9450 con 0x7faabc066e00 2021-07-11T04:01:26.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f8c6d6e1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680ad060 con 0x7f8c680b04f0 2021-07-11T04:01:26.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b477fe700 1 -- 172.21.2.104:0/1277023702 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b480b3240 msgr2=0x7f6b48066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:26.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b477fe700 1 --2- 172.21.2.104:0/1277023702 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b480b3240 0x7f6b48066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b477fe700 1 -- 172.21.2.104:0/1277023702 >> v1:172.21.2.104:6789/0 conn(0x7f6b480b1840 legacy=0x7f6b48142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:26.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f6b477fe700 1 -- 172.21.2.104:0/1277023702 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b48065f40 con 0x7f6b480b0e70 2021-07-11T04:01:26.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7f8c677fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8c680afd40 0x7f8c6813fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:26.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7faabad9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faabc0aaf10 0x7faabc143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:26.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7faab9d9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7faabc060d00 0x7faabc0aa800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46422/0 (socket says 172.21.2.104:46422) 2021-07-11T04:01:26.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.816+0000 7faab9d9b700 1 -- 172.21.2.104:0/2213688783 learned_addr learned my addr 172.21.2.104:0/2213688783 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:26.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8c680afd40 0x7f8c6813fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49658/0 (socket says 172.21.2.104:49658) 2021-07-11T04:01:26.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 -- 172.21.2.104:0/3174949164 learned_addr learned my addr 172.21.2.104:0/3174949164 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:26.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b477fe700 1 --2- 172.21.2.104:0/1277023702 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b480b0e70 0x7f6b48066b70 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f6b400032f0 tx=0x7f6b40009290).ready entity=mon.0 client_cookie=addfb7cbbdbb02dd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:26.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faabad9d700 1 -- 172.21.2.104:0/2213688783 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faabc066e00 msgr2=0x7faabc0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:26.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faabad9d700 1 --2- 172.21.2.104:0/2213688783 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faabc066e00 0x7faabc0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faabad9d700 1 -- 172.21.2.104:0/2213688783 >> v1:172.21.2.104:6789/0 conn(0x7faabc060d00 legacy=0x7faabc0aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:26.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faabad9d700 1 -- 172.21.2.104:0/2213688783 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc0b03b0 con 0x7faabc0aaf10 2021-07-11T04:01:26.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faab959a700 1 -- 172.21.2.104:0/2213688783 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471712353 0 0) 0x7faabc0671e0 con 0x7faabc060d00 2021-07-11T04:01:26.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b45ffb700 1 -- 172.21.2.104:0/1277023702 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4000b8a0 con 0x7f6b480b0e70 2021-07-11T04:01:26.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 -- 172.21.2.104:0/3174949164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680b04f0 msgr2=0x7f8c680b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:26.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 --2- 172.21.2.104:0/3174949164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680b04f0 0x7f8c680b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 -- 172.21.2.104:0/3174949164 >> v1:172.21.2.105:6789/0 conn(0x7f8c680af960 legacy=0x7f8c680b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:26.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b45ffb700 1 -- 172.21.2.104:0/1277023702 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b4000ba00 con 0x7f6b480b0e70 2021-07-11T04:01:26.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 -- 172.21.2.104:0/3174949164 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c680ad300 con 0x7f8c680afd40 2021-07-11T04:01:26.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b45ffb700 1 -- 172.21.2.104:0/1277023702 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b4000bb70 con 0x7f6b480b0e70 2021-07-11T04:01:26.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/1277023702 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b480b0e70 msgr2=0x7f6b48066b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:26.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 --2- 172.21.2.104:0/1277023702 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b480b0e70 0x7f6b48066b70 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f6b400032f0 tx=0x7f6b40009290).stop 2021-07-11T04:01:26.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faabad9d700 1 --2- 172.21.2.104:0/2213688783 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faabc0aaf10 0x7faabc143460 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7faab40032f0 tx=0x7faab40091d0).ready entity=mon.0 client_cookie=921e7b72d5fde9ab server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:26.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 --2- 172.21.2.104:0/3174949164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680afd40 0x7f8c6813fa90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f8c680b04b0 tx=0x7f8c60009290).ready entity=mon.0 client_cookie=fc5dee0d83fb8d67 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:26.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faab959a700 1 -- 172.21.2.104:0/2213688783 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faab4009ea0 con 0x7faabc0aaf10 2021-07-11T04:01:26.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faab959a700 1 -- 172.21.2.104:0/2213688783 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab400b8a0 con 0x7faabc0aaf10 2021-07-11T04:01:26.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faab959a700 1 -- 172.21.2.104:0/2213688783 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faab400ba10 con 0x7faabc0aaf10 2021-07-11T04:01:26.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- 172.21.2.104:0/2213688783 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faabc0aaf10 msgr2=0x7faabc143460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:26.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 --2- 172.21.2.104:0/2213688783 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faabc0aaf10 0x7faabc143460 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7faab40032f0 tx=0x7faab40091d0).stop 2021-07-11T04:01:26.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c65ffb700 1 -- 172.21.2.104:0/3174949164 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c6000b860 con 0x7f8c680afd40 2021-07-11T04:01:26.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c65ffb700 1 -- 172.21.2.104:0/3174949164 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6000b9c0 con 0x7f8c680afd40 2021-07-11T04:01:26.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c65ffb700 1 -- 172.21.2.104:0/3174949164 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c6000bb70 con 0x7f8c680afd40 2021-07-11T04:01:26.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/3174949164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680afd40 msgr2=0x7f8c6813fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:26.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 --2- 172.21.2.104:0/3174949164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680afd40 0x7f8c6813fa90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7f8c680b04b0 tx=0x7f8c60009290).stop 2021-07-11T04:01:26.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/1277023702 shutdown_connections 2021-07-11T04:01:26.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 --2- 172.21.2.104:0/1277023702 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b480b0e70 0x7f6b48066b70 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:26.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 --2- 172.21.2.104:0/1277023702 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b480b3240 0x7f6b48066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/1277023702 >> 172.21.2.104:0/1277023702 conn(0x7f6b4805c490 msgr2=0x7f6b4805c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:26.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/1277023702 shutdown_connections 2021-07-11T04:01:26.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/1277023702 wait complete. 2021-07-11T04:01:26.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- 172.21.2.104:0/2213688783 shutdown_connections 2021-07-11T04:01:26.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 --2- 172.21.2.104:0/2213688783 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faabc0aaf10 0x7faabc143460 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:26.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 --2- 172.21.2.104:0/2213688783 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faabc066e00 0x7faabc0afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- 172.21.2.104:0/2213688783 >> 172.21.2.104:0/2213688783 conn(0x7faabc0b53e0 msgr2=0x7faabc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:26.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- 172.21.2.104:0/2213688783 shutdown_connections 2021-07-11T04:01:26.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- 172.21.2.104:0/2213688783 wait complete. 2021-07-11T04:01:26.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/3174949164 shutdown_connections 2021-07-11T04:01:26.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 --2- 172.21.2.104:0/3174949164 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680afd40 0x7f8c6813fa90 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:26.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 --2- 172.21.2.104:0/3174949164 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680b04f0 0x7f8c680b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/3174949164 >> 172.21.2.104:0/3174949164 conn(0x7f8c6805cd20 msgr2=0x7f8c68062fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:26.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/3174949164 shutdown_connections 2021-07-11T04:01:26.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/3174949164 wait complete. 2021-07-11T04:01:26.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 Processor -- start 2021-07-11T04:01:26.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- start start 2021-07-11T04:01:26.853 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 Processor -- start 2021-07-11T04:01:26.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- start start 2021-07-11T04:01:26.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 Processor -- start 2021-07-11T04:01:26.854 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- start start 2021-07-11T04:01:26.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b480b0e70 0x7f6b480b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b480b1840 0x7f6b480ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b480b3240 0x7f6b480ab840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b480aa580 con 0x7f6b480b1840 2021-07-11T04:01:26.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b480ac390 con 0x7f6b480b3240 2021-07-11T04:01:26.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b480aa2e0 con 0x7f6b480b0e70 2021-07-11T04:01:26.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b467fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b480b1840 0x7f6b480ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:26.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b467fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b480b1840 0x7f6b480ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49664/0 (socket says 172.21.2.104:49664) 2021-07-11T04:01:26.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b467fc700 1 -- 172.21.2.104:0/2099240237 learned_addr learned my addr 172.21.2.104:0/2099240237 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:26.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faabc060d00 0x7faabc0ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faabc066e00 0x7faabc05ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faabc0aaf10 0x7faabc1431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0ad9b0 con 0x7faabc060d00 2021-07-11T04:01:26.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc1561c0 con 0x7faabc0aaf10 2021-07-11T04:01:26.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0adbe0 con 0x7faabc066e00 2021-07-11T04:01:26.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b467fc700 1 -- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b480b0e70 msgr2=0x7f6b480b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:26.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b467fc700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b480b0e70 0x7f6b480b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b467fc700 1 -- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b480b3240 msgr2=0x7f6b480ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:26.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b467fc700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b480b3240 0x7f6b480ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b467fc700 1 -- 172.21.2.104:0/2099240237 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b40009530 con 0x7f6b480b1840 2021-07-11T04:01:26.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680af960 0x7f8c68144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680afd40 0x7f8c68140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faaba59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faabc060d00 0x7faabc0ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:26.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680b04f0 0x7f8c681414c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680a3f10 con 0x7f8c680b04f0 2021-07-11T04:01:26.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faaba59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faabc060d00 0x7faabc0ae080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49668/0 (socket says 172.21.2.104:49668) 2021-07-11T04:01:26.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680a41f0 con 0x7f8c680af960 2021-07-11T04:01:26.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c681419b0 con 0x7f8c680afd40 2021-07-11T04:01:26.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faaba59c700 1 -- 172.21.2.104:0/1255449136 learned_addr learned my addr 172.21.2.104:0/1255449136 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:26.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b467fc700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b480b1840 0x7f6b480ab350 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f6b480b38c0 tx=0x7f6b38007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:26.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680b04f0 0x7f8c681414c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:26.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680b04f0 0x7f8c681414c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49678/0 (socket says 172.21.2.104:49678) 2021-07-11T04:01:26.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 -- 172.21.2.104:0/4283478484 learned_addr learned my addr 172.21.2.104:0/4283478484 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:26.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b37fff700 1 -- 172.21.2.104:0/2099240237 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b3800ce30 con 0x7f6b480b1840 2021-07-11T04:01:26.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b37fff700 1 -- 172.21.2.104:0/2099240237 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b3800f470 con 0x7f6b480b1840 2021-07-11T04:01:26.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/2099240237 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b480ac680 con 0x7f6b480b1840 2021-07-11T04:01:26.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faaba59c700 1 -- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faabc066e00 msgr2=0x7faabc05ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:26.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faaba59c700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faabc066e00 0x7faabc05ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faaba59c700 1 -- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faabc0aaf10 msgr2=0x7faabc1431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:26.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faaba59c700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faabc0aaf10 0x7faabc1431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faaba59c700 1 -- 172.21.2.104:0/1255449136 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab4009470 con 0x7faabc060d00 2021-07-11T04:01:26.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b37fff700 1 -- 172.21.2.104:0/2099240237 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b3800f7d0 con 0x7f6b480b1840 2021-07-11T04:01:26.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 -- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680afd40 msgr2=0x7f8c68140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:26.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680afd40 0x7f8c68140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/2099240237 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b480aae10 con 0x7f6b480b1840 2021-07-11T04:01:26.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faaba59c700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faabc060d00 0x7faabc0ae080 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7faaac002f70 tx=0x7faaac006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:26.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 -- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680af960 msgr2=0x7f8c68144560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:26.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680af960 0x7f8c68144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:26.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 -- 172.21.2.104:0/4283478484 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c60009530 con 0x7f8c680b04f0 2021-07-11T04:01:26.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c677fe700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680b04f0 0x7f8c681414c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f8c68141b40 tx=0x7f8c60016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:26.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faab37fe700 1 -- 172.21.2.104:0/1255449136 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaac013460 con 0x7faabc060d00 2021-07-11T04:01:26.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c5ffff700 1 -- 172.21.2.104:0/4283478484 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c60011c80 con 0x7f8c680b04f0 2021-07-11T04:01:26.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c5ffff700 1 -- 172.21.2.104:0/4283478484 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c60011de0 con 0x7f8c680b04f0 2021-07-11T04:01:26.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faab37fe700 1 -- 172.21.2.104:0/1255449136 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaac0135c0 con 0x7faabc060d00 2021-07-11T04:01:26.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/4283478484 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c680ac5c0 con 0x7f8c680b04f0 2021-07-11T04:01:26.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- 172.21.2.104:0/1255449136 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faabc156460 con 0x7faabc060d00 2021-07-11T04:01:26.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c5ffff700 1 -- 172.21.2.104:0/4283478484 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c60020620 con 0x7f8c680b04f0 2021-07-11T04:01:26.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faab37fe700 1 -- 172.21.2.104:0/1255449136 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaac013920 con 0x7faabc060d00 2021-07-11T04:01:26.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/4283478484 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c680a37d0 con 0x7f8c680b04f0 2021-07-11T04:01:26.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- 172.21.2.104:0/1255449136 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc142a20 con 0x7faabc060d00 2021-07-11T04:01:26.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/2099240237 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b480ac810 con 0x7f6b480b1840 2021-07-11T04:01:26.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b37fff700 1 -- 172.21.2.104:0/2099240237 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6b38019030 con 0x7f6b480b1840 2021-07-11T04:01:26.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/4283478484 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c680a3c10 con 0x7f8c680b04f0 2021-07-11T04:01:26.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faac0a0c700 1 -- 172.21.2.104:0/1255449136 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faabc0adda0 con 0x7faabc060d00 2021-07-11T04:01:26.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b37fff700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b24053270 0x7f6b24055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b37fff700 1 -- 172.21.2.104:0/2099240237 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6b3806a990 con 0x7f6b480b1840 2021-07-11T04:01:26.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c5ffff700 1 -- 172.21.2.104:0/4283478484 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8c60026020 con 0x7f8c680b04f0 2021-07-11T04:01:26.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c5ffff700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053270 0x7f8c4c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f8c5ffff700 1 -- 172.21.2.104:0/4283478484 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f8c6006c2c0 con 0x7f8c680b04f0 2021-07-11T04:01:26.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faab37fe700 1 -- 172.21.2.104:0/1255449136 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7faaac00f070 con 0x7faabc060d00 2021-07-11T04:01:26.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7f6b46ffd700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b24053270 0x7f6b24055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:26.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.820+0000 7faab37fe700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaa00531f0 0x7faaa00556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:26.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.824+0000 7faab37fe700 1 -- 172.21.2.104:0/1255449136 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6069+0+0 (secure 0 0 0) 0x7faaac06ae70 con 0x7faabc060d00 2021-07-11T04:01:26.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.824+0000 7f8c66ffd700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053270 0x7f8c4c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:26.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.824+0000 7faab9d9b700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaa00531f0 0x7faaa00556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:26.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.824+0000 7f6b46ffd700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b24053270 0x7f6b24055730 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f6b3c002f70 tx=0x7f6b3c0069e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:26.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.824+0000 7f8c66ffd700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053270 0x7f8c4c055730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f8c58002f90 tx=0x7f8c580069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:26.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.824+0000 7faab9d9b700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaa00531f0 0x7faaa00556b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7faaa4000ed0 tx=0x7faaa40076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:26.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.824+0000 7f6b37fff700 1 -- 172.21.2.104:0/2099240237 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6b38041bf0 con 0x7f6b480b1840 2021-07-11T04:01:26.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.828+0000 7f8c5ffff700 1 -- 172.21.2.104:0/4283478484 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8c60043600 con 0x7f8c680b04f0 2021-07-11T04:01:26.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.828+0000 7faab37fe700 1 -- 172.21.2.104:0/1255449136 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7faaac0421b0 con 0x7faabc060d00 2021-07-11T04:01:26.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.936+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/2099240237 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f6b480b2a90 con 0x7f6b480b1840 2021-07-11T04:01:26.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.940+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/4283478484 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7f8c6805d760 con 0x7f8c680b04f0 2021-07-11T04:01:26.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:26.944+0000 7faac0a0c700 1 -- 172.21.2.104:0/1255449136 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7faabc057350 con 0x7faabc060d00 2021-07-11T04:01:27.375 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T04:01:27.376+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T04:01:27.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b37fff700 1 -- 172.21.2.104:0/2099240237 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 7617 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7f6b38045210 con 0x7f6b480b1840 2021-07-11T04:01:27.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c5ffff700 1 -- 172.21.2.104:0/4283478484 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 4901 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7f8c60015020 con 0x7f8c680b04f0 2021-07-11T04:01:27.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faab37fe700 1 -- 172.21.2.104:0/1255449136 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 7620 v58) v1 ==== 95+0+0 (secure 0 0 0) 0x7faaac0457d0 con 0x7faabc060d00 2021-07-11T04:01:27.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053270 msgr2=0x7f8c4c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:27.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053270 0x7f8c4c055730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f8c58002f90 tx=0x7f8c580069b0).stop 2021-07-11T04:01:27.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/4283478484 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680b04f0 msgr2=0x7f8c681414c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:27.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680b04f0 0x7f8c681414c0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f8c68141b40 tx=0x7f8c60016000).stop 2021-07-11T04:01:27.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 -- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaa00531f0 msgr2=0x7faaa00556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:27.391 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaa00531f0 0x7faaa00556b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7faaa4000ed0 tx=0x7faaa40076c0).stop 2021-07-11T04:01:27.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 -- 172.21.2.104:0/1255449136 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faabc060d00 msgr2=0x7faabc0ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:27.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faabc060d00 0x7faabc0ae080 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7faaac002f70 tx=0x7faaac006ec0).stop 2021-07-11T04:01:27.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b24053270 msgr2=0x7f6b24055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:27.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b24053270 0x7f6b24055730 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f6b3c002f70 tx=0x7f6b3c0069e0).stop 2021-07-11T04:01:27.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/2099240237 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b480b1840 msgr2=0x7f6b480ab350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:27.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b480b1840 0x7f6b480ab350 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7f6b480b38c0 tx=0x7f6b38007c50).stop 2021-07-11T04:01:27.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 -- 172.21.2.104:0/1255449136 shutdown_connections 2021-07-11T04:01:27.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faabc0aaf10 0x7faabc1431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:27.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faabc066e00 0x7faabc05ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:27.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faaa00531f0 0x7faaa00556b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:27.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 --2- 172.21.2.104:0/1255449136 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faabc060d00 0x7faabc0ae080 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:27.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 -- 172.21.2.104:0/1255449136 >> 172.21.2.104:0/1255449136 conn(0x7faabc0b53e0 msgr2=0x7faabc141340 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:27.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/4283478484 shutdown_connections 2021-07-11T04:01:27.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053270 0x7f8c4c055730 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:27.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680b04f0 0x7f8c681414c0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:27.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680afd40 0x7f8c68140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:27.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 --2- 172.21.2.104:0/4283478484 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680af960 0x7f8c68144560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:27.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/4283478484 >> 172.21.2.104:0/4283478484 conn(0x7f8c6805cd20 msgr2=0x7f8c680b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:27.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/2099240237 shutdown_connections 2021-07-11T04:01:27.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b480b3240 0x7f6b480ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:27.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b24053270 0x7f6b24055730 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:27.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b480b1840 0x7f6b480ab350 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:27.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 --2- 172.21.2.104:0/2099240237 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b480b0e70 0x7f6b480b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:27.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/2099240237 >> 172.21.2.104:0/2099240237 conn(0x7f6b4805c490 msgr2=0x7f6b48143830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:27.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 -- 172.21.2.104:0/1255449136 shutdown_connections 2021-07-11T04:01:27.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/4283478484 shutdown_connections 2021-07-11T04:01:27.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f8c6d6e1700 1 -- 172.21.2.104:0/4283478484 wait complete. 2021-07-11T04:01:27.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7faac0a0c700 1 -- 172.21.2.104:0/1255449136 wait complete. 2021-07-11T04:01:27.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/2099240237 shutdown_connections 2021-07-11T04:01:27.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.388+0000 7f6b4d7a8700 1 -- 172.21.2.104:0/2099240237 wait complete. 2021-07-11T04:01:27.404 INFO:teuthology.orchestra.run.gibba004.stderr:failed mds gid 4901 2021-07-11T04:01:27.404 INFO:teuthology.orchestra.run.gibba004.stderr:failed mds gid 7617 2021-07-11T04:01:27.405 INFO:teuthology.orchestra.run.gibba004.stderr:failed mds gid 7620 2021-07-11T04:01:27.406 DEBUG:teuthology.parallel:result is '' 2021-07-11T04:01:27.407 DEBUG:teuthology.parallel:result is '' 2021-07-11T04:01:27.408 DEBUG:teuthology.parallel:result is '' 2021-07-11T04:01:27.409 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T04:01:27.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5693dc700 1 Processor -- start 2021-07-11T04:01:27.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5693dc700 1 -- start start 2021-07-11T04:01:27.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5693dc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5640b29a0 0x7fe5640b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:27.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5693dc700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5640aeb30 con 0x7fe56413ff70 2021-07-11T04:01:27.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5693dc700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5640ae890 con 0x7fe5640b59f0 2021-07-11T04:01:27.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5693dc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56413f830 con 0x7fe5640b29a0 2021-07-11T04:01:27.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5637fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe56413ff70 0x7fe5640b17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46456/0 (socket says 172.21.2.104:46456) 2021-07-11T04:01:27.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5637fe700 1 -- 172.21.2.104:0/3133825589 learned_addr learned my addr 172.21.2.104:0/3133825589 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:27.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658374848 0 0) 0x7fe5640aeb30 con 0x7fe56413ff70 2021-07-11T04:01:27.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe544003260 con 0x7fe56413ff70 2021-07-11T04:01:27.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe562ffd700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5640b29a0 0x7fe5640b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:27.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 696104074 0 0) 0x7fe544003260 con 0x7fe56413ff70 2021-07-11T04:01:27.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5640b29a0 msgr2=0x7fe5640b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:27.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5640b29a0 0x7fe5640b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:27.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 >> v1:172.21.2.105:6789/0 conn(0x7fe5640b59f0 legacy=0x7fe5640ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:27.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56413fad0 con 0x7fe56413ff70 2021-07-11T04:01:27.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fe55c0019e0 con 0x7fe56413ff70 2021-07-11T04:01:27.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 >> v1:172.21.2.104:6789/0 conn(0x7fe56413ff70 legacy=0x7fe5640b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:27.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe544003780 0x7fe544003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:27.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe544004500 0x7fe544004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:27.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5640b29a0 0x7fe544004f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:27.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56413f830 con 0x7fe544003780 2021-07-11T04:01:27.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe544005800 con 0x7fe544004500 2021-07-11T04:01:27.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440059c0 con 0x7fe5640b29a0 2021-07-11T04:01:27.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe562ffd700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe544003780 0x7fe544003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:27.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5637fe700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5640b29a0 0x7fe544004f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:27.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5627fc700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe544004500 0x7fe544004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:27.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe562ffd700 1 -- 172.21.2.104:0/3133825589 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5640b29a0 msgr2=0x7fe544004f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:27.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe562ffd700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe5640b29a0 0x7fe544004f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:27.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe562ffd700 1 -- 172.21.2.104:0/3133825589 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe544004500 msgr2=0x7fe544004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:27.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe562ffd700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe544004500 0x7fe544004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:27.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe562ffd700 1 -- 172.21.2.104:0/3133825589 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe544003260 con 0x7fe544003780 2021-07-11T04:01:27.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe562ffd700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe544003780 0x7fe544003ba0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7fe55400a000 tx=0x7fe554008040).ready entity=mon.0 client_cookie=b17ca2911539f429 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:27.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe554013670 con 0x7fe544003780 2021-07-11T04:01:27.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5540137d0 con 0x7fe544003780 2021-07-11T04:01:27.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe561ffb700 1 -- 172.21.2.104:0/3133825589 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe554013b30 con 0x7fe544003780 2021-07-11T04:01:27.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5693dc700 1 -- 172.21.2.104:0/3133825589 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe544003780 msgr2=0x7fe544003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:27.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5693dc700 1 --2- 172.21.2.104:0/3133825589 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe544003780 0x7fe544003ba0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7fe55400a000 tx=0x7fe554008040).stop 2021-07-11T04:01:27.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.464+0000 7fe5637fe700 1 -- 172.21.2.104:0/3133825589 reap_dead start 2021-07-11T04:01:27.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 -- 172.21.2.104:0/3133825589 shutdown_connections 2021-07-11T04:01:27.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 -- 172.21.2.104:0/3133825589 >> 172.21.2.104:0/3133825589 conn(0x7fe56406a110 msgr2=0x7fe56405cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:27.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 -- 172.21.2.104:0/3133825589 shutdown_connections 2021-07-11T04:01:27.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 -- 172.21.2.104:0/3133825589 wait complete. 2021-07-11T04:01:27.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 Processor -- start 2021-07-11T04:01:27.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 -- start start 2021-07-11T04:01:27.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe564063d90 0x7fe5640a7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:27.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe56415a3d0 0x7fe56415a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:27.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe564156e70 0x7fe5641572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:27.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe564064170 con 0x7fe564063d90 2021-07-11T04:01:27.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe564159f70 con 0x7fe564156e70 2021-07-11T04:01:27.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe564066aa0 con 0x7fe56415a3d0 2021-07-11T04:01:27.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe562ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe564063d90 0x7fe5640a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:27.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe562ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe564063d90 0x7fe5640a7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49698/0 (socket says 172.21.2.104:49698) 2021-07-11T04:01:27.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe562ffd700 1 -- 172.21.2.104:0/1171033463 learned_addr learned my addr 172.21.2.104:0/1171033463 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:27.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe562ffd700 1 -- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe56415a3d0 msgr2=0x7fe56415a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:27.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe562ffd700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe56415a3d0 0x7fe56415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:27.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe562ffd700 1 -- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe564156e70 msgr2=0x7fe5641572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:27.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe562ffd700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe564156e70 0x7fe5641572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:27.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe562ffd700 1 -- 172.21.2.104:0/1171033463 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe564066560 con 0x7fe564063d90 2021-07-11T04:01:27.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe562ffd700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe564063d90 0x7fe5640a7360 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7fe56415a2b0 tx=0x7fe554008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:27.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe55bfff700 1 -- 172.21.2.104:0/1171033463 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe55400b180 con 0x7fe564063d90 2021-07-11T04:01:27.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe55bfff700 1 -- 172.21.2.104:0/1171033463 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe554000e90 con 0x7fe564063d90 2021-07-11T04:01:27.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 -- 172.21.2.104:0/1171033463 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe564064170 con 0x7fe564063d90 2021-07-11T04:01:27.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe55bfff700 1 -- 172.21.2.104:0/1171033463 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe55401d410 con 0x7fe564063d90 2021-07-11T04:01:27.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 -- 172.21.2.104:0/1171033463 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5640aabe0 con 0x7fe564063d90 2021-07-11T04:01:27.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe5693dc700 1 -- 172.21.2.104:0/1171033463 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe56415a0b0 con 0x7fe564063d90 2021-07-11T04:01:27.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe55bfff700 1 -- 172.21.2.104:0/1171033463 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe55401d570 con 0x7fe564063d90 2021-07-11T04:01:27.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe55bfff700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe548053240 0x7fe548055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:27.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.468+0000 7fe55bfff700 1 -- 172.21.2.104:0/1171033463 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fe55406cea0 con 0x7fe564063d90 2021-07-11T04:01:27.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.472+0000 7fe5627fc700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe548053240 0x7fe548055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:27.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.472+0000 7fe5627fc700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe548053240 0x7fe548055700 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fe54c00b000 tx=0x7fe54c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:27.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.472+0000 7fe55bfff700 1 -- 172.21.2.104:0/1171033463 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe554044020 con 0x7fe564063d90 2021-07-11T04:01:27.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:27.584+0000 7fe5693dc700 1 -- 172.21.2.104:0/1171033463 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe564057350 con 0x7fe564063d90 2021-07-11T04:01:28.390 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe55bfff700 1 -- 172.21.2.104:0/1171033463 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe554047640 con 0x7fe564063d90 2021-07-11T04:01:28.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 -- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe548053240 msgr2=0x7fe548055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:28.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe548053240 0x7fe548055700 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fe54c00b000 tx=0x7fe54c009040).stop 2021-07-11T04:01:28.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 -- 172.21.2.104:0/1171033463 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe564063d90 msgr2=0x7fe5640a7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:28.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe564063d90 0x7fe5640a7360 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7fe56415a2b0 tx=0x7fe554008040).stop 2021-07-11T04:01:28.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 -- 172.21.2.104:0/1171033463 shutdown_connections 2021-07-11T04:01:28.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe564156e70 0x7fe5641572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:28.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe56415a3d0 0x7fe56415a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:28.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe548053240 0x7fe548055700 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:28.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 --2- 172.21.2.104:0/1171033463 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe564063d90 0x7fe5640a7360 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:28.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 -- 172.21.2.104:0/1171033463 >> 172.21.2.104:0/1171033463 conn(0x7fe56406a110 msgr2=0x7fe5640b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:28.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 -- 172.21.2.104:0/1171033463 shutdown_connections 2021-07-11T04:01:28.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.392+0000 7fe5693dc700 1 -- 172.21.2.104:0/1171033463 wait complete. 2021-07-11T04:01:28.402 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T04:01:28.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f786db76700 1 Processor -- start 2021-07-11T04:01:28.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f786db76700 1 -- start start 2021-07-11T04:01:28.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f786db76700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f78680b0fd0 0x7f7868144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:28.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f786db76700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78680b0350 con 0x7f78680b1db0 2021-07-11T04:01:28.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f786db76700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78680b05f0 con 0x7f78681483a0 2021-07-11T04:01:28.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f786db76700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78680a90c0 con 0x7f78680b0fd0 2021-07-11T04:01:28.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78677fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f78680b1db0 0x7f78680b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46478/0 (socket says 172.21.2.104:46478) 2021-07-11T04:01:28.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78677fe700 1 -- 172.21.2.104:0/2721360399 learned_addr learned my addr 172.21.2.104:0/2721360399 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:28.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145760788 0 0) 0x7f78680b0350 con 0x7f78680b1db0 2021-07-11T04:01:28.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7854003260 con 0x7f78680b1db0 2021-07-11T04:01:28.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2465463498 0 0) 0x7f7854003260 con 0x7f78680b1db0 2021-07-11T04:01:28.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f78680b0fd0 msgr2=0x7f7868144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:28.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 --2- 172.21.2.104:0/2721360399 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f78680b0fd0 0x7f7868144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:28.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 >> v1:172.21.2.105:6789/0 conn(0x7f78681483a0 legacy=0x7f78680a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:28.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680b0890 con 0x7f78680b1db0 2021-07-11T04:01:28.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f7858001660 con 0x7f78680b1db0 2021-07-11T04:01:28.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 >> v1:172.21.2.104:6789/0 conn(0x7f78680b1db0 legacy=0x7f78680b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:28.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 --2- 172.21.2.104:0/2721360399 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f78680b0fd0 0x7f7854003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:28.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 --2- 172.21.2.104:0/2721360399 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7854003ef0 0x7f7854004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:28.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 --2- 172.21.2.104:0/2721360399 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7854004bf0 0x7f785400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:28.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78540052c0 con 0x7f7854004bf0 2021-07-11T04:01:28.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785400bc40 con 0x7f7854003ef0 2021-07-11T04:01:28.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785400bd80 con 0x7f78680b0fd0 2021-07-11T04:01:28.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f7867fff700 1 --2- 172.21.2.104:0/2721360399 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7854004bf0 0x7f785400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:28.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f7866ffd700 1 --2- 172.21.2.104:0/2721360399 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7854003ef0 0x7f7854004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:28.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f7867fff700 1 -- 172.21.2.104:0/2721360399 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f78680b0fd0 msgr2=0x7f7854003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:28.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f7867fff700 1 --2- 172.21.2.104:0/2721360399 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f78680b0fd0 0x7f7854003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:28.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f7867fff700 1 -- 172.21.2.104:0/2721360399 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7854003ef0 msgr2=0x7f7854004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:28.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f7867fff700 1 --2- 172.21.2.104:0/2721360399 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7854003ef0 0x7f7854004500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:28.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.456+0000 7f7867fff700 1 -- 172.21.2.104:0/2721360399 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7854003260 con 0x7f7854004bf0 2021-07-11T04:01:28.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f7867fff700 1 --2- 172.21.2.104:0/2721360399 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7854004bf0 0x7f785400b750 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f78600032f0 tx=0x7f7860006f90).ready entity=mon.0 client_cookie=61fd399467ab0d89 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:28.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7860011400 con 0x7f7854004bf0 2021-07-11T04:01:28.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7860011560 con 0x7f7854004bf0 2021-07-11T04:01:28.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78667fc700 1 -- 172.21.2.104:0/2721360399 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78600116d0 con 0x7f7854004bf0 2021-07-11T04:01:28.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- 172.21.2.104:0/2721360399 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7854004bf0 msgr2=0x7f785400b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:28.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 --2- 172.21.2.104:0/2721360399 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7854004bf0 0x7f785400b750 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f78600032f0 tx=0x7f7860006f90).stop 2021-07-11T04:01:28.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f7867fff700 1 -- 172.21.2.104:0/2721360399 reap_dead start 2021-07-11T04:01:28.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- 172.21.2.104:0/2721360399 shutdown_connections 2021-07-11T04:01:28.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- 172.21.2.104:0/2721360399 >> 172.21.2.104:0/2721360399 conn(0x7f786805f6b0 msgr2=0x7f786805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:28.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- 172.21.2.104:0/2721360399 shutdown_connections 2021-07-11T04:01:28.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- 172.21.2.104:0/2721360399 wait complete. 2021-07-11T04:01:28.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 Processor -- start 2021-07-11T04:01:28.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- start start 2021-07-11T04:01:28.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7868146c10 0x7f786814ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:28.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f78680acea0 0x7f786814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:28.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7868156780 0x7f7868156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:28.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78681424d0 con 0x7f7868146c10 2021-07-11T04:01:28.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786814b730 con 0x7f78680acea0 2021-07-11T04:01:28.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7868146ff0 con 0x7f7868156780 2021-07-11T04:01:28.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78677fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7868146c10 0x7f786814ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:28.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78677fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7868146c10 0x7f786814ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49718/0 (socket says 172.21.2.104:49718) 2021-07-11T04:01:28.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78677fe700 1 -- 172.21.2.104:0/1730790065 learned_addr learned my addr 172.21.2.104:0/1730790065 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:28.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78677fe700 1 -- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7868156780 msgr2=0x7f7868156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:28.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78677fe700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7868156780 0x7f7868156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:28.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78677fe700 1 -- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f78680acea0 msgr2=0x7f786814b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:28.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78677fe700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f78680acea0 0x7f786814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:28.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78677fe700 1 -- 172.21.2.104:0/1730790065 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7868145920 con 0x7f7868146c10 2021-07-11T04:01:28.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f78677fe700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7868146c10 0x7f786814ad50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f78680613a0 tx=0x7f78680b04e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:28.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f7864ff9700 1 -- 172.21.2.104:0/1730790065 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7858007170 con 0x7f7868146c10 2021-07-11T04:01:28.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f7864ff9700 1 -- 172.21.2.104:0/1730790065 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78580072d0 con 0x7f7868146c10 2021-07-11T04:01:28.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- 172.21.2.104:0/1730790065 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7868145bc0 con 0x7f7868146c10 2021-07-11T04:01:28.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f7864ff9700 1 -- 172.21.2.104:0/1730790065 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f78580049e0 con 0x7f7868146c10 2021-07-11T04:01:28.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- 172.21.2.104:0/1730790065 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7868146100 con 0x7f7868146c10 2021-07-11T04:01:28.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f786db76700 1 -- 172.21.2.104:0/1730790065 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78680ad620 con 0x7f7868146c10 2021-07-11T04:01:28.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f7864ff9700 1 -- 172.21.2.104:0/1730790065 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f785801b070 con 0x7f7868146c10 2021-07-11T04:01:28.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f7864ff9700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78480531f0 0x7f78480556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:28.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.460+0000 7f7864ff9700 1 -- 172.21.2.104:0/1730790065 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f785806adb0 con 0x7f7868146c10 2021-07-11T04:01:28.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.464+0000 7f7866ffd700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78480531f0 0x7f78480556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:28.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.464+0000 7f7866ffd700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78480531f0 0x7f78480556b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f785c008000 tx=0x7f785c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:28.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.464+0000 7f7864ff9700 1 -- 172.21.2.104:0/1730790065 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f78580420a0 con 0x7f7868146c10 2021-07-11T04:01:28.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:28.548+0000 7f786db76700 1 -- 172.21.2.104:0/1730790065 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7868057350 con 0x7f7868146c10 2021-07-11T04:01:29.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.400+0000 7f7864ff9700 1 -- 172.21.2.104:0/1730790065 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) v1 ==== 185+0+0 (secure 0 0 0) 0x7f78580456c0 con 0x7f7868146c10 2021-07-11T04:01:29.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 -- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78480531f0 msgr2=0x7f78480556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:29.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78480531f0 0x7f78480556b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f785c008000 tx=0x7f785c009040).stop 2021-07-11T04:01:29.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 -- 172.21.2.104:0/1730790065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7868146c10 msgr2=0x7f786814ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:29.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7868146c10 0x7f786814ad50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f78680613a0 tx=0x7f78680b04e0).stop 2021-07-11T04:01:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 -- 172.21.2.104:0/1730790065 shutdown_connections 2021-07-11T04:01:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7868156780 0x7f7868156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f78680acea0 0x7f786814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f78480531f0 0x7f78480556b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:29.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 --2- 172.21.2.104:0/1730790065 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7868146c10 0x7f786814ad50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:29.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 -- 172.21.2.104:0/1730790065 >> 172.21.2.104:0/1730790065 conn(0x7f786805f6b0 msgr2=0x7f78680b49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:29.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 -- 172.21.2.104:0/1730790065 shutdown_connections 2021-07-11T04:01:29.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.404+0000 7f786db76700 1 -- 172.21.2.104:0/1730790065 wait complete. 2021-07-11T04:01:29.406 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T04:01:29.410 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T04:01:29.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 Processor -- start 2021-07-11T04:01:29.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- start start 2021-07-11T04:01:29.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f82a00b12d0 0x7f82a00affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:29.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f82a00b22a0 0x7f82a0141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:29.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a00a77d0 con 0x7f82a00b1ec0 2021-07-11T04:01:29.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a00b1740 con 0x7f82a00b22a0 2021-07-11T04:01:29.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a00b0650 con 0x7f82a00b12d0 2021-07-11T04:01:29.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a6c8f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f82a00b22a0 0x7f82a0141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:29.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a6c8f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f82a00b22a0 0x7f82a0141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49726/0 (socket says 172.21.2.104:49726) 2021-07-11T04:01:29.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a6c8f700 1 -- 172.21.2.104:0/3548277397 learned_addr learned my addr 172.21.2.104:0/3548277397 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:29.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a6c8f700 1 -- 172.21.2.104:0/3548277397 >> v2:172.21.2.105:3300/0 conn(0x7f82a00b12d0 msgr2=0x7f82a00affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:29.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a5c8d700 1 --2- 172.21.2.104:0/3548277397 >> v2:172.21.2.105:3300/0 conn(0x7f82a00b12d0 0x7f82a00affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:29.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a6c8f700 1 --2- 172.21.2.104:0/3548277397 >> v2:172.21.2.105:3300/0 conn(0x7f82a00b12d0 0x7f82a00affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:29.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a6c8f700 1 -- 172.21.2.104:0/3548277397 >> v1:172.21.2.105:6789/0 conn(0x7f82a00b1ec0 legacy=0x7f82a00a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:29.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a6c8f700 1 -- 172.21.2.104:0/3548277397 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a00b08f0 con 0x7f82a00b22a0 2021-07-11T04:01:29.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a5c8d700 1 --2- 172.21.2.104:0/3548277397 >> v2:172.21.2.105:3300/0 conn(0x7f82a00b12d0 0x7f82a00affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:29.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a6c8f700 1 --2- 172.21.2.104:0/3548277397 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82a00b22a0 0x7f82a0141250 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f82a00ad2d0 tx=0x7f8298009290).ready entity=mon.0 client_cookie=15d5edaf4877db48 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:29.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a548c700 1 -- 172.21.2.104:0/3548277397 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f829800b830 con 0x7f82a00b22a0 2021-07-11T04:01:29.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a548c700 1 -- 172.21.2.104:0/3548277397 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829800b990 con 0x7f82a00b22a0 2021-07-11T04:01:29.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a548c700 1 -- 172.21.2.104:0/3548277397 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f829800bb40 con 0x7f82a00b22a0 2021-07-11T04:01:29.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- 172.21.2.104:0/3548277397 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82a00b22a0 msgr2=0x7f82a0141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:29.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 --2- 172.21.2.104:0/3548277397 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82a00b22a0 0x7f82a0141250 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f82a00ad2d0 tx=0x7f8298009290).stop 2021-07-11T04:01:29.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- 172.21.2.104:0/3548277397 shutdown_connections 2021-07-11T04:01:29.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 --2- 172.21.2.104:0/3548277397 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82a00b22a0 0x7f82a0141250 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:29.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 --2- 172.21.2.104:0/3548277397 >> v2:172.21.2.105:3300/0 conn(0x7f82a00b12d0 0x7f82a00affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:29.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- 172.21.2.104:0/3548277397 >> 172.21.2.104:0/3548277397 conn(0x7f82a005f6b0 msgr2=0x7f82a005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:29.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- 172.21.2.104:0/3548277397 shutdown_connections 2021-07-11T04:01:29.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- 172.21.2.104:0/3548277397 wait complete. 2021-07-11T04:01:29.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 Processor -- start 2021-07-11T04:01:29.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- start start 2021-07-11T04:01:29.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82a00b12d0 0x7f82a00b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:29.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82a00b1ec0 0x7f82a00b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:29.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82a00b22a0 0x7f82a00b5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:29.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00abd40 con 0x7f82a00b12d0 2021-07-11T04:01:29.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a014b6d0 con 0x7f82a00b22a0 2021-07-11T04:01:29.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a86f2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0149030 con 0x7f82a00b1ec0 2021-07-11T04:01:29.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a648e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82a00b12d0 0x7f82a00b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:29.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a648e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82a00b12d0 0x7f82a00b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49732/0 (socket says 172.21.2.104:49732) 2021-07-11T04:01:29.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.464+0000 7f82a648e700 1 -- 172.21.2.104:0/1568253542 learned_addr learned my addr 172.21.2.104:0/1568253542 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:29.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f82a648e700 1 -- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82a00b1ec0 msgr2=0x7f82a00b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:29.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f82a648e700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82a00b1ec0 0x7f82a00b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:29.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f82a648e700 1 -- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82a00b22a0 msgr2=0x7f82a00b5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:29.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f82a648e700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82a00b22a0 0x7f82a00b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:29.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f82a648e700 1 -- 172.21.2.104:0/1568253542 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8298009530 con 0x7f82a00b12d0 2021-07-11T04:01:29.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f82a648e700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82a00b12d0 0x7f82a00b42a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f82a00b3eb0 tx=0x7f829c007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:29.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f828f7fe700 1 -- 172.21.2.104:0/1568253542 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f829c00ce70 con 0x7f82a00b12d0 2021-07-11T04:01:29.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f828f7fe700 1 -- 172.21.2.104:0/1568253542 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c007240 con 0x7f82a00b12d0 2021-07-11T04:01:29.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f82a86f2700 1 -- 172.21.2.104:0/1568253542 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a014b470 con 0x7f82a00b12d0 2021-07-11T04:01:29.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f82a86f2700 1 -- 172.21.2.104:0/1568253542 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a0149260 con 0x7f82a00b12d0 2021-07-11T04:01:29.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f828f7fe700 1 -- 172.21.2.104:0/1568253542 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f829c006510 con 0x7f82a00b12d0 2021-07-11T04:01:29.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f82a86f2700 1 -- 172.21.2.104:0/1568253542 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a0057350 con 0x7f82a00b12d0 2021-07-11T04:01:29.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f828f7fe700 1 -- 172.21.2.104:0/1568253542 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f829c010030 con 0x7f82a00b12d0 2021-07-11T04:01:29.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f828f7fe700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8280053350 0x7f8280055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:29.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f828f7fe700 1 -- 172.21.2.104:0/1568253542 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f829c06b1f0 con 0x7f82a00b12d0 2021-07-11T04:01:29.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f82a5c8d700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8280053350 0x7f8280055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:29.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.468+0000 7f82a5c8d700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8280053350 0x7f8280055810 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f8290007000 tx=0x7f8290008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:29.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.472+0000 7f828f7fe700 1 -- 172.21.2.104:0/1568253542 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f829c042520 con 0x7f82a00b12d0 2021-07-11T04:01:29.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:29.552+0000 7f82a86f2700 1 -- 172.21.2.104:0/1568253542 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f82a00ac760 con 0x7f82a00b12d0 2021-07-11T04:01:30.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f828f7fe700 1 -- 172.21.2.104:0/1568253542 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v124) v1 ==== 133+0+0 (secure 0 0 0) 0x7f829c045b40 con 0x7f82a00b12d0 2021-07-11T04:01:30.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 -- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8280053350 msgr2=0x7f8280055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:30.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8280053350 0x7f8280055810 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f8290007000 tx=0x7f8290008040).stop 2021-07-11T04:01:30.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 -- 172.21.2.104:0/1568253542 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82a00b12d0 msgr2=0x7f82a00b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:30.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82a00b12d0 0x7f82a00b42a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f82a00b3eb0 tx=0x7f829c007c30).stop 2021-07-11T04:01:30.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 -- 172.21.2.104:0/1568253542 shutdown_connections 2021-07-11T04:01:30.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f82a00b22a0 0x7f82a00b5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:30.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f82a00b1ec0 0x7f82a00b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:30.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8280053350 0x7f8280055810 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:30.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 --2- 172.21.2.104:0/1568253542 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f82a00b12d0 0x7f82a00b42a0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:30.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 -- 172.21.2.104:0/1568253542 >> 172.21.2.104:0/1568253542 conn(0x7f82a005f6b0 msgr2=0x7f82a013f070 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:30.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 -- 172.21.2.104:0/1568253542 shutdown_connections 2021-07-11T04:01:30.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:30.412+0000 7f82a86f2700 1 -- 172.21.2.104:0/1568253542 wait complete. 2021-07-11T04:01:30.417 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T04:01:30.420 DEBUG:teuthology.orchestra.run.gibba019:> sudo rados -p cephfs_metadata put key /etc/fstab 2021-07-11T04:01:30.446 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.437+0000 7f02a0a77b80 1 Processor -- start 2021-07-11T04:01:30.447 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.437+0000 7f02a0a77b80 1 -- start start 2021-07-11T04:01:30.447 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af0a730 0x560c9af1cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:30.447 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x560c9af10080 0x560c9af1d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:30.447 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x560c9af10950 0x560c9af27980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:30.448 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9ae2e900 con 0x560c9af10080 2021-07-11T04:01:30.448 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9adfe380 con 0x560c9af0a730 2021-07-11T04:01:30.448 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c9af27f20 con 0x560c9af10950 2021-07-11T04:01:30.448 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029eeb5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af0a730 0x560c9af1cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:30.449 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029eeb5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af0a730 0x560c9af1cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.119:35578/0 (socket says 172.21.2.119:35578) 2021-07-11T04:01:30.449 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029eeb5700 1 -- 172.21.2.119:0/2047293991 learned_addr learned my addr 172.21.2.119:0/2047293991 (peer_addr_for_me v2:172.21.2.119:0/0) 2021-07-11T04:01:30.449 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029f6b6700 1 --2- 172.21.2.119:0/2047293991 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x560c9af10950 0x560c9af27980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:30.449 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029eeb5700 1 -- 172.21.2.119:0/2047293991 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x560c9af10950 msgr2=0x560c9af27980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:30.449 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029eeb5700 1 --2- 172.21.2.119:0/2047293991 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x560c9af10950 0x560c9af27980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:30.450 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029eeb5700 1 -- 172.21.2.119:0/2047293991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x560c9af10080 msgr2=0x560c9af1d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:30.450 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029eeb5700 1 --2- 172.21.2.119:0/2047293991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x560c9af10080 0x560c9af1d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:30.450 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029eeb5700 1 -- 172.21.2.119:0/2047293991 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0290005d30 con 0x560c9af0a730 2021-07-11T04:01:30.450 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029eeb5700 1 --2- 172.21.2.119:0/2047293991 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af0a730 0x560c9af1cc60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f0290002f70 tx=0x7f0290014000).ready entity=mon.1 client_cookie=91c56bd7a942d3dc server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:30.451 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029ceb1700 1 -- 172.21.2.119:0/2047293991 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f029001a070 con 0x560c9af0a730 2021-07-11T04:01:30.451 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029ceb1700 1 -- 172.21.2.119:0/2047293991 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0290000e90 con 0x560c9af0a730 2021-07-11T04:01:30.451 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029ceb1700 1 -- 172.21.2.119:0/2047293991 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f029000bb10 con 0x560c9af0a730 2021-07-11T04:01:30.451 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- 172.21.2.119:0/2047293991 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af0a730 msgr2=0x560c9af1cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:30.452 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 --2- 172.21.2.119:0/2047293991 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af0a730 0x560c9af1cc60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f0290002f70 tx=0x7f0290014000).stop 2021-07-11T04:01:30.452 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- 172.21.2.119:0/2047293991 shutdown_connections 2021-07-11T04:01:30.452 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 --2- 172.21.2.119:0/2047293991 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x560c9af10950 0x560c9af27980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:30.452 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 --2- 172.21.2.119:0/2047293991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x560c9af10080 0x560c9af1d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:30.452 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 --2- 172.21.2.119:0/2047293991 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af0a730 0x560c9af1cc60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:30.453 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- 172.21.2.119:0/2047293991 >> 172.21.2.119:0/2047293991 conn(0x560c9af05f00 msgr2=0x560c9af0ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:30.453 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- 172.21.2.119:0/2047293991 shutdown_connections 2021-07-11T04:01:30.453 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- 172.21.2.119:0/2047293991 wait complete. 2021-07-11T04:01:30.453 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 Processor -- start 2021-07-11T04:01:30.453 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- start start 2021-07-11T04:01:30.454 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x560c9af0a730 0x560c9af27770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:30.454 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af10080 0x560c9af22840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:30.454 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x560c9af10950 0x560c9af22d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:30.454 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9af082b0 con 0x560c9af0a730 2021-07-11T04:01:30.455 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c9af234a0 con 0x560c9af10080 2021-07-11T04:01:30.455 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c9af23660 con 0x560c9af10950 2021-07-11T04:01:30.458 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029f6b6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x560c9af10950 0x560c9af22d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:30.458 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029f6b6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x560c9af10950 0x560c9af22d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.119:51854/0 (socket says 172.21.2.119:51854) 2021-07-11T04:01:30.458 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029f6b6700 1 -- 172.21.2.119:0/255627748 learned_addr learned my addr 172.21.2.119:0/255627748 (peer_addr_for_me v2:172.21.2.119:0/0) 2021-07-11T04:01:30.458 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029e6b4700 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af10080 0x560c9af22840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:30.459 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029eeb5700 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x560c9af0a730 0x560c9af27770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:30.459 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029e6b4700 1 -- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x560c9af10950 msgr2=0x560c9af22d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:30.459 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029e6b4700 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x560c9af10950 0x560c9af22d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:30.459 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029e6b4700 1 -- 172.21.2.119:0/255627748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x560c9af0a730 msgr2=0x560c9af27770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:30.460 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029e6b4700 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x560c9af0a730 0x560c9af27770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:30.460 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029e6b4700 1 -- 172.21.2.119:0/255627748 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c9ae25fd0 con 0x560c9af10080 2021-07-11T04:01:30.460 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029f6b6700 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x560c9af10950 0x560c9af22d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:01:30.460 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f029e6b4700 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af10080 0x560c9af22840 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f029400b000 tx=0x7f029400c040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:30.461 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f028ffff700 1 -- 172.21.2.119:0/255627748 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f029400f040 con 0x560c9af10080 2021-07-11T04:01:30.461 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f028ffff700 1 -- 172.21.2.119:0/255627748 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02940067a0 con 0x560c9af10080 2021-07-11T04:01:30.461 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f028ffff700 1 -- 172.21.2.119:0/255627748 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0294006b00 con 0x560c9af10080 2021-07-11T04:01:30.461 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- 172.21.2.119:0/255627748 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c9ae229c0 con 0x560c9af10080 2021-07-11T04:01:30.461 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.441+0000 7f02a0a77b80 1 -- 172.21.2.119:0/255627748 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c9adf6f00 con 0x560c9af10080 2021-07-11T04:01:30.462 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.445+0000 7f028ffff700 1 -- 172.21.2.119:0/255627748 <== mon.1 v2:172.21.2.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0294006c60 con 0x560c9af10080 2021-07-11T04:01:30.462 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.445+0000 7f028ffff700 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0280051df0 0x7f02800542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:30.462 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.445+0000 7f028ffff700 1 -- 172.21.2.119:0/255627748 <== mon.1 v2:172.21.2.105:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6071+0+0 (secure 0 0 0) 0x7f029406abc0 con 0x560c9af10080 2021-07-11T04:01:30.462 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.445+0000 7f02a0a77b80 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x560c9af30160 0x560c9af32550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:30.463 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.445+0000 7f02a0a77b80 1 -- 172.21.2.119:0/255627748 --> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] -- osd_op(unknown.0.0:1 23.0 23:0620e6f9:::key:head [writefull 0~617 in=617b] snapc 0=[] ondisk+write+known_if_redirected e124) v8 -- 0x560c9af32ae0 con 0x560c9af30160 2021-07-11T04:01:30.463 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.445+0000 7f029eeb5700 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0280051df0 0x7f02800542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:30.463 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.445+0000 7f029f6b6700 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x560c9af30160 0x560c9af32550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:30.463 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.445+0000 7f029eeb5700 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0280051df0 0x7f02800542b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f0290020000 tx=0x7f029001a000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:30.463 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:30.445+0000 7f029f6b6700 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x560c9af30160 0x560c9af32550 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:31.427 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f028ffff700 1 -- 172.21.2.119:0/255627748 <== osd.0 v2:172.21.2.104:6800/15411 1 ==== osd_map(125..125 src has 1..125) v4 ==== 320+0+0 (crc 0 0 0) 0x7f0298004b40 con 0x560c9af30160 2021-07-11T04:01:31.428 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f029f6b6700 1 -- 172.21.2.119:0/255627748 <== osd.0 v2:172.21.2.104:6800/15411 2 ==== osd_op_reply(1 key [writefull 0~617] v125'1 uv1 ondisk = 0) v8 ==== 147+0+0 (crc 0 0 0) 0x7f0298004d00 con 0x560c9af30160 2021-07-11T04:01:31.429 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f02a0a77b80 1 -- 172.21.2.119:0/255627748 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x560c9af30160 msgr2=0x560c9af32550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:31.429 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f02a0a77b80 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x560c9af30160 0x560c9af32550 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:31.430 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f02a0a77b80 1 -- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0280051df0 msgr2=0x7f02800542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:31.430 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f02a0a77b80 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0280051df0 0x7f02800542b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f0290020000 tx=0x7f029001a000).stop 2021-07-11T04:01:31.431 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f02a0a77b80 1 -- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af10080 msgr2=0x560c9af22840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:31.431 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f02a0a77b80 1 --2- 172.21.2.119:0/255627748 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x560c9af10080 0x560c9af22840 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f029400b000 tx=0x7f029400c040).stop 2021-07-11T04:01:31.431 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f029f6b6700 1 -- 172.21.2.119:0/255627748 reap_dead start 2021-07-11T04:01:31.431 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f02a0a77b80 1 -- 172.21.2.119:0/255627748 shutdown_connections 2021-07-11T04:01:31.432 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f02a0a77b80 1 -- 172.21.2.119:0/255627748 >> 172.21.2.119:0/255627748 conn(0x560c9af05f00 msgr2=0x560c9af1ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:31.432 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f02a0a77b80 1 -- 172.21.2.119:0/255627748 shutdown_connections 2021-07-11T04:01:31.432 INFO:teuthology.orchestra.run.gibba019.stderr:2021-07-11T04:01:31.425+0000 7f02a0a77b80 1 -- 172.21.2.119:0/255627748 wait complete. 2021-07-11T04:01:31.437 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2021-07-11T04:01:31.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.488+0000 7f6c93d79700 1 Processor -- start 2021-07-11T04:01:31.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.488+0000 7f6c93d79700 1 -- start start 2021-07-11T04:01:31.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6c8c0affd0 0x7f6c8c1416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:31.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6c8c0ac510 0x7f6c8c141b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:31.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8c142080 con 0x7f6c8c0b2200 2021-07-11T04:01:31.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8c145250 con 0x7f6c8c0ac510 2021-07-11T04:01:31.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c1454f0 con 0x7f6c8c0affd0 2021-07-11T04:01:31.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c92316700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6c8c0ac510 0x7f6c8c141b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:31.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c92316700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6c8c0ac510 0x7f6c8c141b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49740/0 (socket says 172.21.2.104:49740) 2021-07-11T04:01:31.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c92316700 1 -- 172.21.2.104:0/1791186090 learned_addr learned my addr 172.21.2.104:0/1791186090 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:31.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c92316700 1 -- 172.21.2.104:0/1791186090 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6c8c0affd0 msgr2=0x7f6c8c1416a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:31.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c92316700 1 --2- 172.21.2.104:0/1791186090 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6c8c0affd0 0x7f6c8c1416a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:31.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c92316700 1 -- 172.21.2.104:0/1791186090 >> v1:172.21.2.105:6789/0 conn(0x7f6c8c0b2200 legacy=0x7f6c8c0abe00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:31.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c92316700 1 -- 172.21.2.104:0/1791186090 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c141200 con 0x7f6c8c0ac510 2021-07-11T04:01:31.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c92316700 1 --2- 172.21.2.104:0/1791186090 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6c8c0ac510 0x7f6c8c141b90 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f6c8c0afc90 tx=0x7f6c84009290).ready entity=mon.0 client_cookie=f7b8c45a27e6e1e6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:31.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c90b13700 1 -- 172.21.2.104:0/1791186090 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c8400b860 con 0x7f6c8c0ac510 2021-07-11T04:01:31.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c90b13700 1 -- 172.21.2.104:0/1791186090 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8400b9c0 con 0x7f6c8c0ac510 2021-07-11T04:01:31.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c90b13700 1 -- 172.21.2.104:0/1791186090 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c8400bb70 con 0x7f6c8c0ac510 2021-07-11T04:01:31.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- 172.21.2.104:0/1791186090 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6c8c0ac510 msgr2=0x7f6c8c141b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:31.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 --2- 172.21.2.104:0/1791186090 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6c8c0ac510 0x7f6c8c141b90 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7f6c8c0afc90 tx=0x7f6c84009290).stop 2021-07-11T04:01:31.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- 172.21.2.104:0/1791186090 shutdown_connections 2021-07-11T04:01:31.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 --2- 172.21.2.104:0/1791186090 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6c8c0ac510 0x7f6c8c141b90 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:31.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 --2- 172.21.2.104:0/1791186090 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6c8c0affd0 0x7f6c8c1416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:31.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- 172.21.2.104:0/1791186090 >> 172.21.2.104:0/1791186090 conn(0x7f6c8c05f6b0 msgr2=0x7f6c8c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:31.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- 172.21.2.104:0/1791186090 shutdown_connections 2021-07-11T04:01:31.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- 172.21.2.104:0/1791186090 wait complete. 2021-07-11T04:01:31.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 Processor -- start 2021-07-11T04:01:31.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- start start 2021-07-11T04:01:31.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6c8c0ac510 0x7f6c8c0673c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:31.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6c8c0affd0 0x7f6c8c1424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:31.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6c8c0b2200 0x7f6c8c0b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:31.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c150f20 con 0x7f6c8c0affd0 2021-07-11T04:01:31.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c144640 con 0x7f6c8c0ac510 2021-07-11T04:01:31.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c1443a0 con 0x7f6c8c0b2200 2021-07-11T04:01:31.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c91314700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6c8c0affd0 0x7f6c8c1424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:31.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c91314700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6c8c0affd0 0x7f6c8c1424f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49748/0 (socket says 172.21.2.104:49748) 2021-07-11T04:01:31.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c91314700 1 -- 172.21.2.104:0/813672572 learned_addr learned my addr 172.21.2.104:0/813672572 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:31.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c91314700 1 -- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6c8c0b2200 msgr2=0x7f6c8c0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:31.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c91314700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6c8c0b2200 0x7f6c8c0b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:31.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c91314700 1 -- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6c8c0ac510 msgr2=0x7f6c8c0673c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:31.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c91314700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6c8c0ac510 0x7f6c8c0673c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:31.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c91314700 1 -- 172.21.2.104:0/813672572 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c84009530 con 0x7f6c8c0affd0 2021-07-11T04:01:31.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c91314700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6c8c0affd0 0x7f6c8c1424f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f6c88002fb0 tx=0x7f6c88006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:31.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c7affd700 1 -- 172.21.2.104:0/813672572 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c88013460 con 0x7f6c8c0affd0 2021-07-11T04:01:31.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c7affd700 1 -- 172.21.2.104:0/813672572 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c880135c0 con 0x7f6c8c0affd0 2021-07-11T04:01:31.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c7affd700 1 -- 172.21.2.104:0/813672572 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6c88013920 con 0x7f6c8c0affd0 2021-07-11T04:01:31.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- 172.21.2.104:0/813672572 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8c151180 con 0x7f6c8c0affd0 2021-07-11T04:01:31.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- 172.21.2.104:0/813672572 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c0b4850 con 0x7f6c8c0affd0 2021-07-11T04:01:31.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.492+0000 7f6c93d79700 1 -- 172.21.2.104:0/813672572 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8c143e20 con 0x7f6c8c0affd0 2021-07-11T04:01:31.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.496+0000 7f6c7affd700 1 -- 172.21.2.104:0/813672572 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6c8800f070 con 0x7f6c8c0affd0 2021-07-11T04:01:31.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.496+0000 7f6c7affd700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6c6c0532d0 0x7f6c6c055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:31.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.496+0000 7f6c7affd700 1 -- 172.21.2.104:0/813672572 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6071+0+0 (secure 0 0 0) 0x7f6c8806b000 con 0x7f6c8c0affd0 2021-07-11T04:01:31.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.496+0000 7f6c91b15700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6c6c0532d0 0x7f6c6c055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:31.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.496+0000 7f6c91b15700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6c6c0532d0 0x7f6c6c055790 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f6c7c000ed0 tx=0x7f6c7c0068b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:31.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.496+0000 7f6c7affd700 1 -- 172.21.2.104:0/813672572 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6c8c143e20 con 0x7f6c8c0affd0 2021-07-11T04:01:31.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 -- 172.21.2.104:0/813672572 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f6c8c143e20 con 0x7f6c8c0affd0 2021-07-11T04:01:31.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c7affd700 1 -- 172.21.2.104:0/813672572 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1906 (secure 0 0 0) 0x7f6c8c143e20 con 0x7f6c8c0affd0 2021-07-11T04:01:31.607 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:31.608 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:01:31.608 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:31.608 INFO:teuthology.orchestra.run.gibba004.stdout: "total_bytes": 773094113280, 2021-07-11T04:01:31.608 INFO:teuthology.orchestra.run.gibba004.stdout: "total_avail_bytes": 764491800576, 2021-07-11T04:01:31.609 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_bytes": 12378112, 2021-07-11T04:01:31.609 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_bytes": 8602312704, 2021-07-11T04:01:31.610 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_ratio": 0.011127122677862644, 2021-07-11T04:01:31.610 INFO:teuthology.orchestra.run.gibba004.stdout: "num_osds": 8, 2021-07-11T04:01:31.610 INFO:teuthology.orchestra.run.gibba004.stdout: "num_per_pool_osds": 8, 2021-07-11T04:01:31.610 INFO:teuthology.orchestra.run.gibba004.stdout: "num_per_pool_omap_osds": 8 2021-07-11T04:01:31.611 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:31.611 INFO:teuthology.orchestra.run.gibba004.stdout: "stats_by_class": { 2021-07-11T04:01:31.611 INFO:teuthology.orchestra.run.gibba004.stdout: "ssd": { 2021-07-11T04:01:31.611 INFO:teuthology.orchestra.run.gibba004.stdout: "total_bytes": 773094113280, 2021-07-11T04:01:31.611 INFO:teuthology.orchestra.run.gibba004.stdout: "total_avail_bytes": 764491800576, 2021-07-11T04:01:31.612 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_bytes": 12378112, 2021-07-11T04:01:31.612 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_bytes": 8602312704, 2021-07-11T04:01:31.613 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_ratio": 0.011127122677862644 2021-07-11T04:01:31.613 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:31.613 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:31.613 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:01:31.613 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:31.614 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "device_health_metrics", 2021-07-11T04:01:31.614 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 1, 2021-07-11T04:01:31.614 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:31.614 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 0, 2021-07-11T04:01:31.615 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 2, 2021-07-11T04:01:31.615 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 0, 2021-07-11T04:01:31.615 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 0, 2021-07-11T04:01:31.615 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 0, 2021-07-11T04:01:31.615 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362918510592 2021-07-11T04:01:31.616 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:31.616 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:31.616 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:31.616 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "rbd", 2021-07-11T04:01:31.617 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 2, 2021-07-11T04:01:31.617 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:31.617 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 0, 2021-07-11T04:01:31.617 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 0, 2021-07-11T04:01:31.618 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 0, 2021-07-11T04:01:31.618 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 0, 2021-07-11T04:01:31.618 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 0, 2021-07-11T04:01:31.618 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362918510592 2021-07-11T04:01:31.619 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:31.619 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:31.619 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:31.620 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "cephfs_data", 2021-07-11T04:01:31.620 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 22, 2021-07-11T04:01:31.620 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:31.620 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 0, 2021-07-11T04:01:31.621 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 0, 2021-07-11T04:01:31.621 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 0, 2021-07-11T04:01:31.621 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 0, 2021-07-11T04:01:31.621 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 0, 2021-07-11T04:01:31.621 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362918510592 2021-07-11T04:01:31.622 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:31.622 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:31.622 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:31.623 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "cephfs_metadata", 2021-07-11T04:01:31.623 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 23, 2021-07-11T04:01:31.623 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:31.624 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 0, 2021-07-11T04:01:31.624 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 0, 2021-07-11T04:01:31.624 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 0, 2021-07-11T04:01:31.624 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 0, 2021-07-11T04:01:31.624 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 0, 2021-07-11T04:01:31.625 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362918510592 2021-07-11T04:01:31.625 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:31.625 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:31.625 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:31.625 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:01:31.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 -- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6c6c0532d0 msgr2=0x7f6c6c055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:31.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6c6c0532d0 0x7f6c6c055790 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f6c7c000ed0 tx=0x7f6c7c0068b0).stop 2021-07-11T04:01:31.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 -- 172.21.2.104:0/813672572 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6c8c0affd0 msgr2=0x7f6c8c1424f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:31.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6c8c0affd0 0x7f6c8c1424f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f6c88002fb0 tx=0x7f6c88006e10).stop 2021-07-11T04:01:31.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 -- 172.21.2.104:0/813672572 shutdown_connections 2021-07-11T04:01:31.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6c8c0b2200 0x7f6c8c0b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:31.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6c6c0532d0 0x7f6c6c055790 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:31.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6c8c0affd0 0x7f6c8c1424f0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:31.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 --2- 172.21.2.104:0/813672572 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6c8c0ac510 0x7f6c8c0673c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:31.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 -- 172.21.2.104:0/813672572 >> 172.21.2.104:0/813672572 conn(0x7f6c8c05f6b0 msgr2=0x7f6c8c0b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:31.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 -- 172.21.2.104:0/813672572 shutdown_connections 2021-07-11T04:01:31.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:31.608+0000 7f6c93d79700 1 -- 172.21.2.104:0/813672572 wait complete. 2021-07-11T04:01:31.631 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30)... 2021-07-11T04:01:36.631 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2021-07-11T04:01:36.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6e626700 1 Processor -- start 2021-07-11T04:01:36.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6e626700 1 -- start start 2021-07-11T04:01:36.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6e626700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e681483a0 0x7f2e680b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6e626700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2e680a70c0 0x7f2e68144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6e626700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e680b0350 con 0x7f2e680b1db0 2021-07-11T04:01:36.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6e626700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e680a90c0 con 0x7f2e680a70c0 2021-07-11T04:01:36.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6e626700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e680b05f0 con 0x7f2e681483a0 2021-07-11T04:01:36.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e67fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f2e680b1db0 0x7f2e680b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46520/0 (socket says 172.21.2.104:46520) 2021-07-11T04:01:36.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e67fff700 1 -- 172.21.2.104:0/3847647001 learned_addr learned my addr 172.21.2.104:0/3847647001 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:36.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1061471357 0 0) 0x7f2e680b0350 con 0x7f2e680b1db0 2021-07-11T04:01:36.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e48003260 con 0x7f2e680b1db0 2021-07-11T04:01:36.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e677fe700 1 --2- 172.21.2.104:0/3847647001 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e681483a0 0x7f2e680b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:36.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6cbc3700 1 --2- 172.21.2.104:0/3847647001 >> v2:172.21.2.105:3300/0 conn(0x7f2e680a70c0 0x7f2e68144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:36.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3559142175 0 0) 0x7f2e48003260 con 0x7f2e680b1db0 2021-07-11T04:01:36.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e681483a0 msgr2=0x7f2e680b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:36.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 --2- 172.21.2.104:0/3847647001 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e681483a0 0x7f2e680b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:36.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 >> v2:172.21.2.105:3300/0 conn(0x7f2e680a70c0 msgr2=0x7f2e68144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:36.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 --2- 172.21.2.104:0/3847647001 >> v2:172.21.2.105:3300/0 conn(0x7f2e680a70c0 0x7f2e68144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:36.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e680b0890 con 0x7f2e680b1db0 2021-07-11T04:01:36.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f2e58001660 con 0x7f2e680b1db0 2021-07-11T04:01:36.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6cbc3700 1 --2- 172.21.2.104:0/3847647001 >> v2:172.21.2.105:3300/0 conn(0x7f2e680a70c0 0x7f2e68144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:36.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 >> v1:172.21.2.104:6789/0 conn(0x7f2e680b1db0 legacy=0x7f2e680b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:36.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 --2- 172.21.2.104:0/3847647001 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e480037d0 0x7f2e48003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 --2- 172.21.2.104:0/3847647001 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e681483a0 0x7f2e48004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 --2- 172.21.2.104:0/3847647001 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e48004bf0 0x7f2e4800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e680b05f0 con 0x7f2e48004bf0 2021-07-11T04:01:36.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e48005280 con 0x7f2e480037d0 2021-07-11T04:01:36.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4800bc40 con 0x7f2e681483a0 2021-07-11T04:01:36.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6cbc3700 1 --2- 172.21.2.104:0/3847647001 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e48004bf0 0x7f2e4800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:36.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6cbc3700 1 -- 172.21.2.104:0/3847647001 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e681483a0 msgr2=0x7f2e48004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:36.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6cbc3700 1 --2- 172.21.2.104:0/3847647001 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e681483a0 0x7f2e48004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:36.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6cbc3700 1 -- 172.21.2.104:0/3847647001 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e480037d0 msgr2=0x7f2e48003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:36.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6cbc3700 1 --2- 172.21.2.104:0/3847647001 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e480037d0 0x7f2e48003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:36.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6cbc3700 1 -- 172.21.2.104:0/3847647001 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e48003260 con 0x7f2e48004bf0 2021-07-11T04:01:36.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6cbc3700 1 --2- 172.21.2.104:0/3847647001 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e48004bf0 0x7f2e4800b750 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f2e60007000 tx=0x7f2e60008040).ready entity=mon.0 client_cookie=c038a7058ddb6dc0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:36.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e6000c710 con 0x7f2e48004bf0 2021-07-11T04:01:36.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6000c870 con 0x7f2e48004bf0 2021-07-11T04:01:36.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e66ffd700 1 -- 172.21.2.104:0/3847647001 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e6000c9e0 con 0x7f2e48004bf0 2021-07-11T04:01:36.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6e626700 1 -- 172.21.2.104:0/3847647001 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e48004bf0 msgr2=0x7f2e4800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:36.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6e626700 1 --2- 172.21.2.104:0/3847647001 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e48004bf0 0x7f2e4800b750 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f2e60007000 tx=0x7f2e60008040).stop 2021-07-11T04:01:36.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.688+0000 7f2e6cbc3700 1 -- 172.21.2.104:0/3847647001 reap_dead start 2021-07-11T04:01:36.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 -- 172.21.2.104:0/3847647001 shutdown_connections 2021-07-11T04:01:36.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 -- 172.21.2.104:0/3847647001 >> 172.21.2.104:0/3847647001 conn(0x7f2e6805f6b0 msgr2=0x7f2e6805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:36.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 -- 172.21.2.104:0/3847647001 shutdown_connections 2021-07-11T04:01:36.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 -- 172.21.2.104:0/3847647001 wait complete. 2021-07-11T04:01:36.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 Processor -- start 2021-07-11T04:01:36.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 -- start start 2021-07-11T04:01:36.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e68141f70 0x7f2e6814afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e6813e6a0 0x7f2e6813fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e6813d7e0 0x7f2e6813dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6813eff0 con 0x7f2e68141f70 2021-07-11T04:01:36.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e68142350 con 0x7f2e6813e6a0 2021-07-11T04:01:36.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e68148d60 con 0x7f2e6813d7e0 2021-07-11T04:01:36.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e67fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e68141f70 0x7f2e6814afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:36.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e67fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e68141f70 0x7f2e6814afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49764/0 (socket says 172.21.2.104:49764) 2021-07-11T04:01:36.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e67fff700 1 -- 172.21.2.104:0/3375199610 learned_addr learned my addr 172.21.2.104:0/3375199610 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:36.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e67fff700 1 -- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e6813d7e0 msgr2=0x7f2e6813dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:36.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e67fff700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e6813d7e0 0x7f2e6813dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:36.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e67fff700 1 -- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e6813e6a0 msgr2=0x7f2e6813fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:36.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e67fff700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e6813e6a0 0x7f2e6813fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:36.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e67fff700 1 -- 172.21.2.104:0/3375199610 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e6813f220 con 0x7f2e68141f70 2021-07-11T04:01:36.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e67fff700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e68141f70 0x7f2e6814afe0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f2e58001370 tx=0x7f2e680b0a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:36.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e657fa700 1 -- 172.21.2.104:0/3375199610 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e58007170 con 0x7f2e68141f70 2021-07-11T04:01:36.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e657fa700 1 -- 172.21.2.104:0/3375199610 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e580072d0 con 0x7f2e68141f70 2021-07-11T04:01:36.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e657fa700 1 -- 172.21.2.104:0/3375199610 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2e580049e0 con 0x7f2e68141f70 2021-07-11T04:01:36.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 -- 172.21.2.104:0/3375199610 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e6814acf0 con 0x7f2e68141f70 2021-07-11T04:01:36.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 -- 172.21.2.104:0/3375199610 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e6813e150 con 0x7f2e68141f70 2021-07-11T04:01:36.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.692+0000 7f2e6e626700 1 -- 172.21.2.104:0/3375199610 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e68057350 con 0x7f2e68141f70 2021-07-11T04:01:36.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.696+0000 7f2e657fa700 1 -- 172.21.2.104:0/3375199610 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2e58010070 con 0x7f2e68141f70 2021-07-11T04:01:36.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.696+0000 7f2e657fa700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e4c053210 0x7f2e4c0556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.696+0000 7f2e657fa700 1 -- 172.21.2.104:0/3375199610 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6071+0+0 (secure 0 0 0) 0x7f2e5806ae50 con 0x7f2e68141f70 2021-07-11T04:01:36.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.696+0000 7f2e677fe700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e4c053210 0x7f2e4c0556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:36.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.696+0000 7f2e677fe700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e4c053210 0x7f2e4c0556d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f2e50007000 tx=0x7f2e50008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:36.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.696+0000 7f2e657fa700 1 -- 172.21.2.104:0/3375199610 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2e58042190 con 0x7f2e68141f70 2021-07-11T04:01:36.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.804+0000 7f2e6e626700 1 -- 172.21.2.104:0/3375199610 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f2e68044650 con 0x7f2e68141f70 2021-07-11T04:01:36.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e657fa700 1 -- 172.21.2.104:0/3375199610 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1932 (secure 0 0 0) 0x7f2e58019030 con 0x7f2e68141f70 2021-07-11T04:01:36.807 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:36.807 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:01:36.808 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:36.808 INFO:teuthology.orchestra.run.gibba004.stdout: "total_bytes": 773094113280, 2021-07-11T04:01:36.808 INFO:teuthology.orchestra.run.gibba004.stdout: "total_avail_bytes": 764491472896, 2021-07-11T04:01:36.808 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_bytes": 12705792, 2021-07-11T04:01:36.808 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_bytes": 8602640384, 2021-07-11T04:01:36.809 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_ratio": 0.011127546429634094, 2021-07-11T04:01:36.809 INFO:teuthology.orchestra.run.gibba004.stdout: "num_osds": 8, 2021-07-11T04:01:36.809 INFO:teuthology.orchestra.run.gibba004.stdout: "num_per_pool_osds": 8, 2021-07-11T04:01:36.809 INFO:teuthology.orchestra.run.gibba004.stdout: "num_per_pool_omap_osds": 8 2021-07-11T04:01:36.810 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:36.810 INFO:teuthology.orchestra.run.gibba004.stdout: "stats_by_class": { 2021-07-11T04:01:36.810 INFO:teuthology.orchestra.run.gibba004.stdout: "ssd": { 2021-07-11T04:01:36.810 INFO:teuthology.orchestra.run.gibba004.stdout: "total_bytes": 773094113280, 2021-07-11T04:01:36.811 INFO:teuthology.orchestra.run.gibba004.stdout: "total_avail_bytes": 764491472896, 2021-07-11T04:01:36.811 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_bytes": 12705792, 2021-07-11T04:01:36.811 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_bytes": 8602640384, 2021-07-11T04:01:36.811 INFO:teuthology.orchestra.run.gibba004.stdout: "total_used_raw_ratio": 0.011127546429634094 2021-07-11T04:01:36.811 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:36.812 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:36.812 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:01:36.812 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:36.812 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "device_health_metrics", 2021-07-11T04:01:36.813 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 1, 2021-07-11T04:01:36.813 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:36.813 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 0, 2021-07-11T04:01:36.813 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 2, 2021-07-11T04:01:36.813 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 0, 2021-07-11T04:01:36.814 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 0, 2021-07-11T04:01:36.814 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 0, 2021-07-11T04:01:36.814 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362918346752 2021-07-11T04:01:36.814 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:36.815 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:36.815 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:36.815 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "rbd", 2021-07-11T04:01:36.815 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 2, 2021-07-11T04:01:36.815 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:36.816 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 0, 2021-07-11T04:01:36.816 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 0, 2021-07-11T04:01:36.816 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 0, 2021-07-11T04:01:36.816 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 0, 2021-07-11T04:01:36.817 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 0, 2021-07-11T04:01:36.817 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362918346752 2021-07-11T04:01:36.817 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:36.817 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:36.817 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:36.818 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "cephfs_data", 2021-07-11T04:01:36.818 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 22, 2021-07-11T04:01:36.818 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:36.818 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 0, 2021-07-11T04:01:36.819 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 0, 2021-07-11T04:01:36.819 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 0, 2021-07-11T04:01:36.819 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 0, 2021-07-11T04:01:36.819 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 0, 2021-07-11T04:01:36.819 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362918346752 2021-07-11T04:01:36.820 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:36.820 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:36.820 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:36.820 INFO:teuthology.orchestra.run.gibba004.stdout: "name": "cephfs_metadata", 2021-07-11T04:01:36.820 INFO:teuthology.orchestra.run.gibba004.stdout: "id": 23, 2021-07-11T04:01:36.821 INFO:teuthology.orchestra.run.gibba004.stdout: "stats": { 2021-07-11T04:01:36.821 INFO:teuthology.orchestra.run.gibba004.stdout: "stored": 617, 2021-07-11T04:01:36.821 INFO:teuthology.orchestra.run.gibba004.stdout: "objects": 1, 2021-07-11T04:01:36.821 INFO:teuthology.orchestra.run.gibba004.stdout: "kb_used": 8, 2021-07-11T04:01:36.822 INFO:teuthology.orchestra.run.gibba004.stdout: "bytes_used": 8192, 2021-07-11T04:01:36.822 INFO:teuthology.orchestra.run.gibba004.stdout: "percent_used": 1.1286285683809183e-08, 2021-07-11T04:01:36.822 INFO:teuthology.orchestra.run.gibba004.stdout: "max_avail": 362918346752 2021-07-11T04:01:36.822 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:36.822 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:36.823 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:36.823 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:01:36.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 -- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e4c053210 msgr2=0x7f2e4c0556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:36.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e4c053210 0x7f2e4c0556d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f2e50007000 tx=0x7f2e50008040).stop 2021-07-11T04:01:36.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 -- 172.21.2.104:0/3375199610 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e68141f70 msgr2=0x7f2e6814afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:36.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e68141f70 0x7f2e6814afe0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f2e58001370 tx=0x7f2e680b0a20).stop 2021-07-11T04:01:36.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 -- 172.21.2.104:0/3375199610 shutdown_connections 2021-07-11T04:01:36.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2e6813d7e0 0x7f2e6813dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:36.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2e6813e6a0 0x7f2e6813fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:36.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2e4c053210 0x7f2e4c0556d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:36.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 --2- 172.21.2.104:0/3375199610 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2e68141f70 0x7f2e6814afe0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:36.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 -- 172.21.2.104:0/3375199610 >> 172.21.2.104:0/3375199610 conn(0x7f2e6805f6b0 msgr2=0x7f2e680b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:36.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 -- 172.21.2.104:0/3375199610 shutdown_connections 2021-07-11T04:01:36.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.808+0000 7f2e6e626700 1 -- 172.21.2.104:0/3375199610 wait complete. 2021-07-11T04:01:36.828 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s 2021-07-11T04:01:36.828 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2021-07-11T04:01:36.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 Processor -- start 2021-07-11T04:01:36.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- start start 2021-07-11T04:01:36.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8cac0b1ec0 0x7f8cac0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8cac0affb0 0x7f8cac0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac0b0420 con 0x7f8cac0a70c0 2021-07-11T04:01:36.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0b2790 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0b0650 con 0x7f8cac0affb0 2021-07-11T04:01:36.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8caaffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8cac0b1ec0 0x7f8cac0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:36.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8caaffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8cac0b1ec0 0x7f8cac0b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49772/0 (socket says 172.21.2.104:49772) 2021-07-11T04:01:36.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8caaffd700 1 -- 172.21.2.104:0/731453781 learned_addr learned my addr 172.21.2.104:0/731453781 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:36.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8caaffd700 1 -- 172.21.2.104:0/731453781 >> v2:172.21.2.105:3300/0 conn(0x7f8cac0affb0 msgr2=0x7f8cac0b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:36.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8caaffd700 1 --2- 172.21.2.104:0/731453781 >> v2:172.21.2.105:3300/0 conn(0x7f8cac0affb0 0x7f8cac0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:36.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8caaffd700 1 -- 172.21.2.104:0/731453781 >> v1:172.21.2.104:6789/0 conn(0x7f8cac0a70c0 legacy=0x7f8cac141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:36.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8caaffd700 1 -- 172.21.2.104:0/731453781 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0b08f0 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8caaffd700 1 --2- 172.21.2.104:0/731453781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cac0b1ec0 0x7f8cac0b12d0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f8c9c002f70 tx=0x7f8c9c006eb0).ready entity=mon.0 client_cookie=3c0febff825eb743 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:36.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8ca9ffb700 1 -- 172.21.2.104:0/731453781 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c9c00d610 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8ca9ffb700 1 -- 172.21.2.104:0/731453781 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c000e90 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- 172.21.2.104:0/731453781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cac0b1ec0 msgr2=0x7f8cac0b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:36.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 --2- 172.21.2.104:0/731453781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cac0b1ec0 0x7f8cac0b12d0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7f8c9c002f70 tx=0x7f8c9c006eb0).stop 2021-07-11T04:01:36.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8ca9ffb700 1 -- 172.21.2.104:0/731453781 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c9c00d9a0 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- 172.21.2.104:0/731453781 shutdown_connections 2021-07-11T04:01:36.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 --2- 172.21.2.104:0/731453781 >> v2:172.21.2.105:3300/0 conn(0x7f8cac0affb0 0x7f8cac0b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:36.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 --2- 172.21.2.104:0/731453781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cac0b1ec0 0x7f8cac0b12d0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:36.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- 172.21.2.104:0/731453781 >> 172.21.2.104:0/731453781 conn(0x7f8cac05f6b0 msgr2=0x7f8cac05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:36.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- 172.21.2.104:0/731453781 shutdown_connections 2021-07-11T04:01:36.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- 172.21.2.104:0/731453781 wait complete. 2021-07-11T04:01:36.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 Processor -- start 2021-07-11T04:01:36.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- start start 2021-07-11T04:01:36.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8cac0a70c0 0x7f8cac0b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8cac0affb0 0x7f8cac149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cac0b1ec0 0x7f8cac14a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac14b730 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0b5910 con 0x7f8cac0a70c0 2021-07-11T04:01:36.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cb18be700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac14b920 con 0x7f8cac0affb0 2021-07-11T04:01:36.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cab7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cac0b1ec0 0x7f8cac14a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:36.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cab7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cac0b1ec0 0x7f8cac14a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49780/0 (socket says 172.21.2.104:49780) 2021-07-11T04:01:36.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cab7fe700 1 -- 172.21.2.104:0/756063781 learned_addr learned my addr 172.21.2.104:0/756063781 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:36.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cab7fe700 1 -- 172.21.2.104:0/756063781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8cac0affb0 msgr2=0x7f8cac149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:36.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cab7fe700 1 --2- 172.21.2.104:0/756063781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8cac0affb0 0x7f8cac149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:36.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cab7fe700 1 -- 172.21.2.104:0/756063781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8cac0a70c0 msgr2=0x7f8cac0b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:36.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cab7fe700 1 --2- 172.21.2.104:0/756063781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8cac0a70c0 0x7f8cac0b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:36.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.884+0000 7f8cab7fe700 1 -- 172.21.2.104:0/756063781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c9c00b1e0 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8cab7fe700 1 --2- 172.21.2.104:0/756063781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8cac0b1ec0 0x7f8cac14a450 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7f8cac0ad2d0 tx=0x7f8ca4007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:36.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8c9bfff700 1 -- 172.21.2.104:0/756063781 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ca400e4a0 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8c9bfff700 1 -- 172.21.2.104:0/756063781 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca400e600 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8c9bfff700 1 -- 172.21.2.104:0/756063781 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8ca400e960 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8cb18be700 1 -- 172.21.2.104:0/756063781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac0b3210 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8cb18be700 1 -- 172.21.2.104:0/756063781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac0b3750 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8cb18be700 1 -- 172.21.2.104:0/756063781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac057350 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8c9bfff700 1 -- 172.21.2.104:0/756063781 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8ca4019030 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8c9bfff700 1 --2- 172.21.2.104:0/756063781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c880532b0 0x7f8c88055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:36.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8c9bfff700 1 -- 172.21.2.104:0/756063781 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6071+0+0 (secure 0 0 0) 0x7f8ca406abc0 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8caaffd700 1 --2- 172.21.2.104:0/756063781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c880532b0 0x7f8c88055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:36.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8caaffd700 1 --2- 172.21.2.104:0/756063781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c880532b0 0x7f8c88055770 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f8c9c002f70 tx=0x7f8c9c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:36.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.888+0000 7f8c9bfff700 1 -- 172.21.2.104:0/756063781 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8ca4041f00 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.996+0000 7f8cb18be700 1 -- 172.21.2.104:0/756063781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8cac0ac760 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:36.996+0000 7f8c9bfff700 1 -- 172.21.2.104:0/756063781 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=-22 pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. v59) v1 ==== 209+0+0 (secure 0 0 0) 0x7f8ca4017070 con 0x7f8cac0b1ec0 2021-07-11T04:01:36.997 INFO:teuthology.orchestra.run.gibba004.stderr:Error EINVAL: pool 'cephfs_metadata' already contains some objects. Use an empty pool instead. 2021-07-11T04:01:36.998 DEBUG:teuthology.orchestra.run:got remote process result: 22 2021-07-11T04:01:37.000 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T04:01:37.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f2993067700 1 Processor -- start 2021-07-11T04:01:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f2993067700 1 -- start start 2021-07-11T04:01:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f2993067700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f298c0a76b0 0x7f298c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f2993067700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f298c0b0060 0x7f298c05fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f2993067700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f298c0a7a90 con 0x7f298c0b1970 2021-07-11T04:01:37.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f2993067700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f298c147620 con 0x7f298c0b0060 2021-07-11T04:01:37.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f2993067700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298c1478c0 con 0x7f298c0a76b0 2021-07-11T04:01:37.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f298bfff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f298c0b1970 0x7f298c05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46556/0 (socket says 172.21.2.104:46556) 2021-07-11T04:01:37.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f298bfff700 1 -- 172.21.2.104:0/2633766721 learned_addr learned my addr 172.21.2.104:0/2633766721 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:37.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337442191 0 0) 0x7f298c0a7a90 con 0x7f298c0b1970 2021-07-11T04:01:37.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f296c003260 con 0x7f298c0b1970 2021-07-11T04:01:37.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f2990e03700 1 --2- 172.21.2.104:0/2633766721 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f298c0a76b0 0x7f298c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.052+0000 7f2991604700 1 --2- 172.21.2.104:0/2633766721 >> v2:172.21.2.105:3300/0 conn(0x7f298c0b0060 0x7f298c05fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3472625121 0 0) 0x7f296c003260 con 0x7f298c0b1970 2021-07-11T04:01:37.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f298c0a76b0 msgr2=0x7f298c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:37.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 --2- 172.21.2.104:0/2633766721 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f298c0a76b0 0x7f298c066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:37.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 >> v2:172.21.2.105:3300/0 conn(0x7f298c0b0060 msgr2=0x7f298c05fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:37.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 --2- 172.21.2.104:0/2633766721 >> v2:172.21.2.105:3300/0 conn(0x7f298c0b0060 0x7f298c05fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:37.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c147b60 con 0x7f298c0b1970 2021-07-11T04:01:37.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f2974001660 con 0x7f298c0b1970 2021-07-11T04:01:37.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 >> v1:172.21.2.104:6789/0 conn(0x7f298c0b1970 legacy=0x7f298c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:37.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 --2- 172.21.2.104:0/2633766721 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f296c0037d0 0x7f296c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 --2- 172.21.2.104:0/2633766721 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f296c004500 0x7f296c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 --2- 172.21.2.104:0/2633766721 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f298c0a76b0 0x7f296c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c147620 con 0x7f296c004500 2021-07-11T04:01:37.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c1478c0 con 0x7f296c0037d0 2021-07-11T04:01:37.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c00b820 con 0x7f298c0a76b0 2021-07-11T04:01:37.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298bfff700 1 --2- 172.21.2.104:0/2633766721 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f296c004500 0x7f296c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298bfff700 1 -- 172.21.2.104:0/2633766721 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f298c0a76b0 msgr2=0x7f296c004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:37.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298bfff700 1 --2- 172.21.2.104:0/2633766721 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f298c0a76b0 0x7f296c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:37.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298bfff700 1 -- 172.21.2.104:0/2633766721 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f296c0037d0 msgr2=0x7f296c003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:37.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298bfff700 1 --2- 172.21.2.104:0/2633766721 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f296c0037d0 0x7f296c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:37.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298bfff700 1 -- 172.21.2.104:0/2633766721 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c147b60 con 0x7f296c004500 2021-07-11T04:01:37.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298bfff700 1 --2- 172.21.2.104:0/2633766721 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f296c004500 0x7f296c004980 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f2974001370 tx=0x7f2974001e40).ready entity=mon.0 client_cookie=8f8c3f0042c9d66f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:37.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f29740070d0 con 0x7f296c004500 2021-07-11T04:01:37.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2974007230 con 0x7f296c004500 2021-07-11T04:01:37.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f298b7fe700 1 -- 172.21.2.104:0/2633766721 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f297400c980 con 0x7f296c004500 2021-07-11T04:01:37.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 -- 172.21.2.104:0/2633766721 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f296c004500 msgr2=0x7f296c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:37.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 --2- 172.21.2.104:0/2633766721 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f296c004500 0x7f296c004980 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f2974001370 tx=0x7f2974001e40).stop 2021-07-11T04:01:37.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2991604700 1 -- 172.21.2.104:0/2633766721 reap_dead start 2021-07-11T04:01:37.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 -- 172.21.2.104:0/2633766721 shutdown_connections 2021-07-11T04:01:37.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 -- 172.21.2.104:0/2633766721 >> 172.21.2.104:0/2633766721 conn(0x7f298c0b53e0 msgr2=0x7f298c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:37.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 -- 172.21.2.104:0/2633766721 shutdown_connections 2021-07-11T04:01:37.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 -- 172.21.2.104:0/2633766721 wait complete. 2021-07-11T04:01:37.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 Processor -- start 2021-07-11T04:01:37.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 -- start start 2021-07-11T04:01:37.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f298c148ba0 0x7f298c0ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f298c0abda0 0x7f298c1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f298c156eb0 0x7f298c159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c0aae00 con 0x7f298c148ba0 2021-07-11T04:01:37.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c0aa520 con 0x7f298c156eb0 2021-07-11T04:01:37.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298c064710 con 0x7f298c0abda0 2021-07-11T04:01:37.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2990e03700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f298c148ba0 0x7f298c0ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2990e03700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f298c148ba0 0x7f298c0ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49800/0 (socket says 172.21.2.104:49800) 2021-07-11T04:01:37.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2990e03700 1 -- 172.21.2.104:0/588688100 learned_addr learned my addr 172.21.2.104:0/588688100 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:37.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2991604700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f298c156eb0 0x7f298c159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2990e03700 1 -- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f298c0abda0 msgr2=0x7f298c1569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:37.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2990e03700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f298c0abda0 0x7f298c1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:37.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2990e03700 1 -- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f298c156eb0 msgr2=0x7f298c159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:37.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2990e03700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f298c156eb0 0x7f298c159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:37.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2990e03700 1 -- 172.21.2.104:0/588688100 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f298c0aa280 con 0x7f298c148ba0 2021-07-11T04:01:37.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2991604700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f298c156eb0 0x7f298c159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:37.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2990e03700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f298c148ba0 0x7f298c0ab8b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f297c00a000 tx=0x7f297c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:37.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2989ffb700 1 -- 172.21.2.104:0/588688100 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f297c016070 con 0x7f298c148ba0 2021-07-11T04:01:37.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2989ffb700 1 -- 172.21.2.104:0/588688100 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c000e90 con 0x7f298c148ba0 2021-07-11T04:01:37.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 -- 172.21.2.104:0/588688100 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c0aab60 con 0x7f298c148ba0 2021-07-11T04:01:37.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2989ffb700 1 -- 172.21.2.104:0/588688100 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f297c011a40 con 0x7f298c148ba0 2021-07-11T04:01:37.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.056+0000 7f2993067700 1 -- 172.21.2.104:0/588688100 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298c0a92c0 con 0x7f298c148ba0 2021-07-11T04:01:37.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.060+0000 7f2993067700 1 -- 172.21.2.104:0/588688100 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298c057350 con 0x7f298c148ba0 2021-07-11T04:01:37.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.060+0000 7f2989ffb700 1 -- 172.21.2.104:0/588688100 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f297c011ba0 con 0x7f298c148ba0 2021-07-11T04:01:37.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.060+0000 7f2989ffb700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2970053330 0x7f29700557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.060+0000 7f2989ffb700 1 -- 172.21.2.104:0/588688100 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6071+0+0 (secure 0 0 0) 0x7f297c06b460 con 0x7f298c148ba0 2021-07-11T04:01:37.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.060+0000 7f298bfff700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2970053330 0x7f29700557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.060+0000 7f298bfff700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2970053330 0x7f29700557f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f2974001370 tx=0x7f2974015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:37.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.060+0000 7f2989ffb700 1 -- 172.21.2.104:0/588688100 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f297c0427a0 con 0x7f298c148ba0 2021-07-11T04:01:37.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.168+0000 7f2993067700 1 -- 172.21.2.104:0/588688100 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f298c0698e0 con 0x7f298c148ba0 2021-07-11T04:01:37.170 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T04:01:37.172+0000 7f2069947700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T04:01:37.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2989ffb700 1 -- 172.21.2.104:0/588688100 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 23 and data pool 22 v60) v1 ==== 189+0+0 (secure 0 0 0) 0x7f297c045dc0 con 0x7f298c148ba0 2021-07-11T04:01:37.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 -- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2970053330 msgr2=0x7f29700557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:37.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2970053330 0x7f29700557f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f2974001370 tx=0x7f2974015000).stop 2021-07-11T04:01:37.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 -- 172.21.2.104:0/588688100 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f298c148ba0 msgr2=0x7f298c0ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:37.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f298c148ba0 0x7f298c0ab8b0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f297c00a000 tx=0x7f297c008040).stop 2021-07-11T04:01:37.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 -- 172.21.2.104:0/588688100 shutdown_connections 2021-07-11T04:01:37.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f298c156eb0 0x7f298c159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:37.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f298c0abda0 0x7f298c1569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:37.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2970053330 0x7f29700557f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:37.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 --2- 172.21.2.104:0/588688100 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f298c148ba0 0x7f298c0ab8b0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:37.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 -- 172.21.2.104:0/588688100 >> 172.21.2.104:0/588688100 conn(0x7f298c0b53e0 msgr2=0x7f298c149df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:37.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 -- 172.21.2.104:0/588688100 shutdown_connections 2021-07-11T04:01:37.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.176+0000 7f2993067700 1 -- 172.21.2.104:0/588688100 wait complete. 2021-07-11T04:01:37.179 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 23 and data pool 22 2021-07-11T04:01:37.183 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T04:01:37.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cdb427700 1 Processor -- start 2021-07-11T04:01:37.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cdb427700 1 -- start start 2021-07-11T04:01:37.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cdb427700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd4149840 0x7f3cd414bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cdb427700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd40af6b0 con 0x7f3cd4143640 2021-07-11T04:01:37.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cdb427700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd4143e90 con 0x7f3cd40ac180 2021-07-11T04:01:37.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cdb427700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40b6220 con 0x7f3cd4149840 2021-07-11T04:01:37.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd91c3700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f3cd4143640 0x7f3cd413d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46574/0 (socket says 172.21.2.104:46574) 2021-07-11T04:01:37.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd91c3700 1 -- 172.21.2.104:0/2380785838 learned_addr learned my addr 172.21.2.104:0/2380785838 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:37.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321321811 0 0) 0x7f3cd40af6b0 con 0x7f3cd4143640 2021-07-11T04:01:37.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cbc003260 con 0x7f3cd4143640 2021-07-11T04:01:37.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd99c4700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd4149840 0x7f3cd414bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1464111375 0 0) 0x7f3cbc003260 con 0x7f3cd4143640 2021-07-11T04:01:37.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd4149840 msgr2=0x7f3cd414bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:37.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd4149840 0x7f3cd414bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:37.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 >> v1:172.21.2.105:6789/0 conn(0x7f3cd40ac180 legacy=0x7f3cd40ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:37.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd4144130 con 0x7f3cd4143640 2021-07-11T04:01:37.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f3cc4001660 con 0x7f3cd4143640 2021-07-11T04:01:37.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 >> v1:172.21.2.104:6789/0 conn(0x7f3cd4143640 legacy=0x7f3cd413d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:37.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3cbc003780 0x7f3cbc003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd4149840 0x7f3cbc004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cbc0049f0 0x7f3cbc004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc00fc10 con 0x7f3cbc0049f0 2021-07-11T04:01:37.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc00fd50 con 0x7f3cbc003780 2021-07-11T04:01:37.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cbc00fee0 con 0x7f3cd4149840 2021-07-11T04:01:37.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd99c4700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cbc0049f0 0x7f3cbc004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd91c3700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3cbc003780 0x7f3cbc003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd89c2700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd4149840 0x7f3cbc004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd99c4700 1 -- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd4149840 msgr2=0x7f3cbc004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:37.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd99c4700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd4149840 0x7f3cbc004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:37.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd99c4700 1 -- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3cbc003780 msgr2=0x7f3cbc003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:37.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd99c4700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3cbc003780 0x7f3cbc003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:37.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd99c4700 1 -- 172.21.2.104:0/2380785838 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cbc003260 con 0x7f3cbc0049f0 2021-07-11T04:01:37.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd91c3700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3cbc003780 0x7f3cbc003ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:01:37.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.236+0000 7f3cd99c4700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cbc0049f0 0x7f3cbc004e70 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f3ccc00c000 tx=0x7f3ccc0091d0).ready entity=mon.0 client_cookie=c1b6bea70265e877 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:37.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ccc00e7e0 con 0x7f3cbc0049f0 2021-07-11T04:01:37.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc00e940 con 0x7f3cbc0049f0 2021-07-11T04:01:37.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cc3fff700 1 -- 172.21.2.104:0/2380785838 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ccc00eca0 con 0x7f3cbc0049f0 2021-07-11T04:01:37.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- 172.21.2.104:0/2380785838 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cbc0049f0 msgr2=0x7f3cbc004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:37.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 --2- 172.21.2.104:0/2380785838 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cbc0049f0 0x7f3cbc004e70 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f3ccc00c000 tx=0x7f3ccc0091d0).stop 2021-07-11T04:01:37.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd99c4700 1 -- 172.21.2.104:0/2380785838 reap_dead start 2021-07-11T04:01:37.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- 172.21.2.104:0/2380785838 shutdown_connections 2021-07-11T04:01:37.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- 172.21.2.104:0/2380785838 >> 172.21.2.104:0/2380785838 conn(0x7f3cd40b5570 msgr2=0x7f3cd40b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:37.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- 172.21.2.104:0/2380785838 shutdown_connections 2021-07-11T04:01:37.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- 172.21.2.104:0/2380785838 wait complete. 2021-07-11T04:01:37.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 Processor -- start 2021-07-11T04:01:37.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- start start 2021-07-11T04:01:37.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd41340a0 0x7f3cd40b4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3cd40b42a0 0x7f3cd40b3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cd40b3660 0x7f3cd40b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd40b3fd0 con 0x7f3cd40b3660 2021-07-11T04:01:37.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4134480 con 0x7f3cd40b42a0 2021-07-11T04:01:37.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd4063700 con 0x7f3cd41340a0 2021-07-11T04:01:37.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd99c4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cd40b3660 0x7f3cd40b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd99c4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cd40b3660 0x7f3cd40b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49822/0 (socket says 172.21.2.104:49822) 2021-07-11T04:01:37.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd99c4700 1 -- 172.21.2.104:0/3528979132 learned_addr learned my addr 172.21.2.104:0/3528979132 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:37.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd89c2700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3cd40b42a0 0x7f3cd40b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd91c3700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd41340a0 0x7f3cd40b4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd99c4700 1 -- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd41340a0 msgr2=0x7f3cd40b4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:37.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd99c4700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd41340a0 0x7f3cd40b4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:37.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd99c4700 1 -- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3cd40b42a0 msgr2=0x7f3cd40b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:37.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd99c4700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3cd40b42a0 0x7f3cd40b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:37.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd99c4700 1 -- 172.21.2.104:0/3528979132 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd4132530 con 0x7f3cd40b3660 2021-07-11T04:01:37.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cd99c4700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cd40b3660 0x7f3cd40b3ae0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f3cc4001110 tx=0x7f3ccc01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:37.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cc27fc700 1 -- 172.21.2.104:0/3528979132 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ccc01d7a0 con 0x7f3cd40b3660 2021-07-11T04:01:37.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cc27fc700 1 -- 172.21.2.104:0/3528979132 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc01d900 con 0x7f3cd40b3660 2021-07-11T04:01:37.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cc27fc700 1 -- 172.21.2.104:0/3528979132 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3ccc01dc60 con 0x7f3cd40b3660 2021-07-11T04:01:37.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- 172.21.2.104:0/3528979132 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd40669a0 con 0x7f3cd40b3660 2021-07-11T04:01:37.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- 172.21.2.104:0/3528979132 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd4067720 con 0x7f3cd40b3660 2021-07-11T04:01:37.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cdb427700 1 -- 172.21.2.104:0/3528979132 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd40b48c0 con 0x7f3cd40b3660 2021-07-11T04:01:37.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.240+0000 7f3cc27fc700 1 -- 172.21.2.104:0/3528979132 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3ccc01b450 con 0x7f3cd40b3660 2021-07-11T04:01:37.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.244+0000 7f3cc27fc700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb4053300 0x7f3cb40557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:37.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.244+0000 7f3cc27fc700 1 -- 172.21.2.104:0/3528979132 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f3ccc06cf70 con 0x7f3cd40b3660 2021-07-11T04:01:37.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.244+0000 7f3cd91c3700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb4053300 0x7f3cb40557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:37.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.244+0000 7f3cd91c3700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb4053300 0x7f3cb40557c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f3cc400d000 tx=0x7f3cc4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:37.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.244+0000 7f3cc27fc700 1 -- 172.21.2.104:0/3528979132 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3ccc044200 con 0x7f3cd40b3660 2021-07-11T04:01:37.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:37.356+0000 7f3cdb427700 1 -- 172.21.2.104:0/3528979132 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3cd4132390 con 0x7f3cd40b3660 2021-07-11T04:01:38.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cc27fc700 1 -- 172.21.2.104:0/3528979132 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3cd4132390 con 0x7f3cd40b3660 2021-07-11T04:01:38.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 -- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb4053300 msgr2=0x7f3cb40557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:38.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb4053300 0x7f3cb40557c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f3cc400d000 tx=0x7f3cc4009040).stop 2021-07-11T04:01:38.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 -- 172.21.2.104:0/3528979132 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cd40b3660 msgr2=0x7f3cd40b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:38.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cd40b3660 0x7f3cd40b3ae0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f3cc4001110 tx=0x7f3ccc01a000).stop 2021-07-11T04:01:38.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 -- 172.21.2.104:0/3528979132 shutdown_connections 2021-07-11T04:01:38.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3cb4053300 0x7f3cb40557c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:38.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3cd40b3660 0x7f3cd40b3ae0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:38.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3cd40b42a0 0x7f3cd40b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:38.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 --2- 172.21.2.104:0/3528979132 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3cd41340a0 0x7f3cd40b4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:38.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 -- 172.21.2.104:0/3528979132 >> 172.21.2.104:0/3528979132 conn(0x7f3cd40b5570 msgr2=0x7f3cd4060e10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:38.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 -- 172.21.2.104:0/3528979132 shutdown_connections 2021-07-11T04:01:38.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.184+0000 7f3cdb427700 1 -- 172.21.2.104:0/3528979132 wait complete. 2021-07-11T04:01:38.191 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T04:01:38.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f9736707700 1 Processor -- start 2021-07-11T04:01:38.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f9736707700 1 -- start start 2021-07-11T04:01:38.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f9736707700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9730060910 0x7f97300acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:38.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f9736707700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97300aa5d0 0x7f973013c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:38.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f9736707700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97300b6ce0 con 0x7f97300b6900 2021-07-11T04:01:38.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f9736707700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97300611e0 con 0x7f9730060910 2021-07-11T04:01:38.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f9736707700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9730061480 con 0x7f97300aa5d0 2021-07-11T04:01:38.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f972f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9730060910 0x7f97300acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:38.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f972f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9730060910 0x7f97300acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49828/0 (socket says 172.21.2.104:49828) 2021-07-11T04:01:38.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f972f7fe700 1 -- 172.21.2.104:0/3992663803 learned_addr learned my addr 172.21.2.104:0/3992663803 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:38.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f972ffff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f97300b6900 0x7f97300a9ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46594/0 (socket says 172.21.2.104:46594) 2021-07-11T04:01:38.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f972f7fe700 1 -- 172.21.2.104:0/3992663803 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97300aa5d0 msgr2=0x7f973013c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:38.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f972f7fe700 1 --2- 172.21.2.104:0/3992663803 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97300aa5d0 0x7f973013c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:38.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f972f7fe700 1 -- 172.21.2.104:0/3992663803 >> v1:172.21.2.104:6789/0 conn(0x7f97300b6900 legacy=0x7f97300a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:38.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.244+0000 7f972effd700 1 -- 172.21.2.104:0/3992663803 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188186137 0 0) 0x7f97300b6ce0 con 0x7f97300b6900 2021-07-11T04:01:38.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972f7fe700 1 -- 172.21.2.104:0/3992663803 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730061720 con 0x7f9730060910 2021-07-11T04:01:38.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972f7fe700 1 --2- 172.21.2.104:0/3992663803 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9730060910 0x7f97300acd50 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f9724002f70 tx=0x7f9724006eb0).ready entity=mon.0 client_cookie=8b8045b576c2619a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:38.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972effd700 1 -- 172.21.2.104:0/3992663803 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f972400d5c0 con 0x7f9730060910 2021-07-11T04:01:38.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972effd700 1 -- 172.21.2.104:0/3992663803 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9724000e90 con 0x7f9730060910 2021-07-11T04:01:38.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972effd700 1 -- 172.21.2.104:0/3992663803 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f972400d910 con 0x7f9730060910 2021-07-11T04:01:38.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- 172.21.2.104:0/3992663803 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9730060910 msgr2=0x7f97300acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:38.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 --2- 172.21.2.104:0/3992663803 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9730060910 0x7f97300acd50 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f9724002f70 tx=0x7f9724006eb0).stop 2021-07-11T04:01:38.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- 172.21.2.104:0/3992663803 shutdown_connections 2021-07-11T04:01:38.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 --2- 172.21.2.104:0/3992663803 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9730060910 0x7f97300acd50 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:38.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 --2- 172.21.2.104:0/3992663803 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97300aa5d0 0x7f973013c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:38.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- 172.21.2.104:0/3992663803 >> 172.21.2.104:0/3992663803 conn(0x7f97300b3480 msgr2=0x7f97300b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:38.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- 172.21.2.104:0/3992663803 shutdown_connections 2021-07-11T04:01:38.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- 172.21.2.104:0/3992663803 wait complete. 2021-07-11T04:01:38.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 Processor -- start 2021-07-11T04:01:38.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- start start 2021-07-11T04:01:38.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9730060910 0x7f97300af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:38.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97300aa5d0 0x7f9730063c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:38.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97300b6900 0x7f973014ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:38.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9730063290 con 0x7f9730060910 2021-07-11T04:01:38.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973013f660 con 0x7f97300b6900 2021-07-11T04:01:38.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97300634c0 con 0x7f97300aa5d0 2021-07-11T04:01:38.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9730060910 0x7f97300af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:38.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9730060910 0x7f97300af7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49832/0 (socket says 172.21.2.104:49832) 2021-07-11T04:01:38.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972ffff700 1 -- 172.21.2.104:0/2180653055 learned_addr learned my addr 172.21.2.104:0/2180653055 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:38.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972ffff700 1 -- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97300aa5d0 msgr2=0x7f9730063c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:38.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972ffff700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97300aa5d0 0x7f9730063c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:38.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972ffff700 1 -- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97300b6900 msgr2=0x7f973014ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:38.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972ffff700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97300b6900 0x7f973014ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:38.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972ffff700 1 -- 172.21.2.104:0/2180653055 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972400b1e0 con 0x7f9730060910 2021-07-11T04:01:38.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972ffff700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9730060910 0x7f97300af7e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f9720000ed0 tx=0x7f9720007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:38.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972d7fa700 1 -- 172.21.2.104:0/2180653055 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f972000ce80 con 0x7f9730060910 2021-07-11T04:01:38.260 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972d7fa700 1 -- 172.21.2.104:0/2180653055 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9720007170 con 0x7f9730060910 2021-07-11T04:01:38.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972d7fa700 1 -- 172.21.2.104:0/2180653055 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9720004730 con 0x7f9730060910 2021-07-11T04:01:38.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- 172.21.2.104:0/2180653055 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9730063760 con 0x7f9730060910 2021-07-11T04:01:38.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- 172.21.2.104:0/2180653055 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9730065fc0 con 0x7f9730060910 2021-07-11T04:01:38.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f9736707700 1 -- 172.21.2.104:0/2180653055 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97300638f0 con 0x7f9730060910 2021-07-11T04:01:38.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.248+0000 7f972d7fa700 1 -- 172.21.2.104:0/2180653055 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9720004890 con 0x7f9730060910 2021-07-11T04:01:38.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.252+0000 7f972d7fa700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97100532d0 0x7f9710055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:38.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.252+0000 7f972d7fa700 1 -- 172.21.2.104:0/2180653055 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f972006afb0 con 0x7f9730060910 2021-07-11T04:01:38.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.252+0000 7f972f7fe700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97100532d0 0x7f9710055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:38.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.252+0000 7f972f7fe700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97100532d0 0x7f9710055790 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f9724002f70 tx=0x7f9724015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:38.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.252+0000 7f972d7fa700 1 -- 172.21.2.104:0/2180653055 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9720042190 con 0x7f9730060910 2021-07-11T04:01:38.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:38.336+0000 7f9736707700 1 -- 172.21.2.104:0/2180653055 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9730057350 con 0x7f9730060910 2021-07-11T04:01:39.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.188+0000 7f972d7fa700 1 -- 172.21.2.104:0/2180653055 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v128) v1 ==== 185+0+0 (secure 0 0 0) 0x7f97200490e0 con 0x7f9730060910 2021-07-11T04:01:39.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 -- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97100532d0 msgr2=0x7f9710055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:39.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97100532d0 0x7f9710055790 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f9724002f70 tx=0x7f9724015000).stop 2021-07-11T04:01:39.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 -- 172.21.2.104:0/2180653055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9730060910 msgr2=0x7f97300af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:39.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9730060910 0x7f97300af7e0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f9720000ed0 tx=0x7f9720007c50).stop 2021-07-11T04:01:39.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 -- 172.21.2.104:0/2180653055 shutdown_connections 2021-07-11T04:01:39.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f97300b6900 0x7f973014ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:39.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f97300aa5d0 0x7f9730063c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:39.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f97100532d0 0x7f9710055790 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:39.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 --2- 172.21.2.104:0/2180653055 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9730060910 0x7f97300af7e0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:39.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 -- 172.21.2.104:0/2180653055 >> 172.21.2.104:0/2180653055 conn(0x7f97300b3480 msgr2=0x7f973013c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:39.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 -- 172.21.2.104:0/2180653055 shutdown_connections 2021-07-11T04:01:39.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.192+0000 7f9736707700 1 -- 172.21.2.104:0/2180653055 wait complete. 2021-07-11T04:01:39.195 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T04:01:39.199 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T04:01:39.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.252+0000 7f16cf697700 1 Processor -- start 2021-07-11T04:01:39.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.252+0000 7f16cf697700 1 -- start start 2021-07-11T04:01:39.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.252+0000 7f16cf697700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f16c80603d0 0x7f16c80aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:39.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.252+0000 7f16cf697700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c813f790 0x7f16c813fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:39.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.252+0000 7f16cf697700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c8061990 con 0x7f16c80a9ec0 2021-07-11T04:01:39.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.252+0000 7f16cf697700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c81400f0 con 0x7f16c80603d0 2021-07-11T04:01:39.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.252+0000 7f16cf697700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c8061c30 con 0x7f16c813f790 2021-07-11T04:01:39.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16ccc32700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f16c80603d0 0x7f16c80aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:39.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cdc34700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c813f790 0x7f16c813fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:39.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cdc34700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c813f790 0x7f16c813fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:53006/0 (socket says 172.21.2.104:53006) 2021-07-11T04:01:39.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cdc34700 1 -- 172.21.2.104:0/3047734459 learned_addr learned my addr 172.21.2.104:0/3047734459 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:39.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cdc34700 1 -- 172.21.2.104:0/3047734459 >> v2:172.21.2.105:3300/0 conn(0x7f16c80603d0 msgr2=0x7f16c80aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:39.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cdc34700 1 --2- 172.21.2.104:0/3047734459 >> v2:172.21.2.105:3300/0 conn(0x7f16c80603d0 0x7f16c80aa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:39.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cdc34700 1 -- 172.21.2.104:0/3047734459 >> v1:172.21.2.105:6789/0 conn(0x7f16c80a9ec0 legacy=0x7f16c813ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:39.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cdc34700 1 -- 172.21.2.104:0/3047734459 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c8060b90 con 0x7f16c813f790 2021-07-11T04:01:39.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cdc34700 1 --2- 172.21.2.104:0/3047734459 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c813f790 0x7f16c813fc00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f16c00032f0 tx=0x7f16c000b530).ready entity=mon.2 client_cookie=6ce6f773ab6559d6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:39.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16bbfff700 1 -- 172.21.2.104:0/3047734459 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16c00153f0 con 0x7f16c813f790 2021-07-11T04:01:39.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16bbfff700 1 -- 172.21.2.104:0/3047734459 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c0015550 con 0x7f16c813f790 2021-07-11T04:01:39.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16bbfff700 1 -- 172.21.2.104:0/3047734459 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16c00158b0 con 0x7f16c813f790 2021-07-11T04:01:39.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cf697700 1 -- 172.21.2.104:0/3047734459 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c813f790 msgr2=0x7f16c813fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:39.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cf697700 1 --2- 172.21.2.104:0/3047734459 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c813f790 0x7f16c813fc00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f16c00032f0 tx=0x7f16c000b530).stop 2021-07-11T04:01:39.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cf697700 1 -- 172.21.2.104:0/3047734459 shutdown_connections 2021-07-11T04:01:39.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cf697700 1 --2- 172.21.2.104:0/3047734459 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c813f790 0x7f16c813fc00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:39.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cf697700 1 --2- 172.21.2.104:0/3047734459 >> v2:172.21.2.105:3300/0 conn(0x7f16c80603d0 0x7f16c80aa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:39.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cf697700 1 -- 172.21.2.104:0/3047734459 >> 172.21.2.104:0/3047734459 conn(0x7f16c80b2970 msgr2=0x7f16c80b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:39.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cf697700 1 -- 172.21.2.104:0/3047734459 shutdown_connections 2021-07-11T04:01:39.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.256+0000 7f16cf697700 1 -- 172.21.2.104:0/3047734459 wait complete. 2021-07-11T04:01:39.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cf697700 1 Processor -- start 2021-07-11T04:01:39.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cf697700 1 -- start start 2021-07-11T04:01:39.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cf697700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c80603d0 0x7f16c80ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:39.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cf697700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f16c80a9ec0 0x7f16c8143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:39.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cf697700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16c813f790 0x7f16c8143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:39.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cf697700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c80b0270 con 0x7f16c813f790 2021-07-11T04:01:39.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cf697700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c8141c20 con 0x7f16c80a9ec0 2021-07-11T04:01:39.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cf697700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c8141240 con 0x7f16c80603d0 2021-07-11T04:01:39.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cdc34700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16c813f790 0x7f16c8143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:39.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cdc34700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16c813f790 0x7f16c8143760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49850/0 (socket says 172.21.2.104:49850) 2021-07-11T04:01:39.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cdc34700 1 -- 172.21.2.104:0/2843200400 learned_addr learned my addr 172.21.2.104:0/2843200400 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:39.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16ccc32700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f16c80a9ec0 0x7f16c8143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:39.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cd433700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c80603d0 0x7f16c80ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:39.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cdc34700 1 -- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c80603d0 msgr2=0x7f16c80ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:39.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cdc34700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c80603d0 0x7f16c80ad500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:39.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cdc34700 1 -- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f16c80a9ec0 msgr2=0x7f16c8143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:39.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cdc34700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f16c80a9ec0 0x7f16c8143270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:39.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cdc34700 1 -- 172.21.2.104:0/2843200400 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c000b7d0 con 0x7f16c813f790 2021-07-11T04:01:39.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cd433700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c80603d0 0x7f16c80ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:39.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cdc34700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16c813f790 0x7f16c8143760 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f16c00032f0 tx=0x7f16c0019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:39.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16ba7fc700 1 -- 172.21.2.104:0/2843200400 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16c0009040 con 0x7f16c813f790 2021-07-11T04:01:39.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16ba7fc700 1 -- 172.21.2.104:0/2843200400 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c0014560 con 0x7f16c813f790 2021-07-11T04:01:39.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cf697700 1 -- 172.21.2.104:0/2843200400 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c81414e0 con 0x7f16c813f790 2021-07-11T04:01:39.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16ba7fc700 1 -- 172.21.2.104:0/2843200400 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16c00148c0 con 0x7f16c813f790 2021-07-11T04:01:39.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cf697700 1 -- 172.21.2.104:0/2843200400 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c80aeca0 con 0x7f16c813f790 2021-07-11T04:01:39.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cf697700 1 -- 172.21.2.104:0/2843200400 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c8057350 con 0x7f16c813f790 2021-07-11T04:01:39.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16ba7fc700 1 -- 172.21.2.104:0/2843200400 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f16c0025030 con 0x7f16c813f790 2021-07-11T04:01:39.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16ba7fc700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16a80531f0 0x7f16a80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:39.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16ba7fc700 1 -- 172.21.2.104:0/2843200400 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f16c006ca20 con 0x7f16c813f790 2021-07-11T04:01:39.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.260+0000 7f16cd433700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16a80531f0 0x7f16a80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:39.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.264+0000 7f16cd433700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16a80531f0 0x7f16a80556b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f16bc008000 tx=0x7f16bc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:39.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.264+0000 7f16ba7fc700 1 -- 172.21.2.104:0/2843200400 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f16c0043e40 con 0x7f16c813f790 2021-07-11T04:01:39.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:39.348+0000 7f16cf697700 1 -- 172.21.2.104:0/2843200400 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f16c80ac9f0 con 0x7f16c813f790 2021-07-11T04:01:40.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.356+0000 7f16ba7fc700 1 -- 172.21.2.104:0/2843200400 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v129) v1 ==== 133+0+0 (secure 0 0 0) 0x7f16c0023020 con 0x7f16c813f790 2021-07-11T04:01:40.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 -- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16a80531f0 msgr2=0x7f16a80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16a80531f0 0x7f16a80556b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f16bc008000 tx=0x7f16bc009040).stop 2021-07-11T04:01:40.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 -- 172.21.2.104:0/2843200400 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16c813f790 msgr2=0x7f16c8143760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16c813f790 0x7f16c8143760 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f16c00032f0 tx=0x7f16c0019000).stop 2021-07-11T04:01:40.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 -- 172.21.2.104:0/2843200400 shutdown_connections 2021-07-11T04:01:40.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f16a80531f0 0x7f16a80556b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:40.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f16c813f790 0x7f16c8143760 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:40.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f16c80a9ec0 0x7f16c8143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:40.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 --2- 172.21.2.104:0/2843200400 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16c80603d0 0x7f16c80ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:40.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 -- 172.21.2.104:0/2843200400 >> 172.21.2.104:0/2843200400 conn(0x7f16c80b2970 msgr2=0x7f16c80b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:40.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 -- 172.21.2.104:0/2843200400 shutdown_connections 2021-07-11T04:01:40.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.360+0000 7f16cf697700 1 -- 172.21.2.104:0/2843200400 wait complete. 2021-07-11T04:01:40.363 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T04:01:40.367 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2021-07-11T04:01:40.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba880b9700 1 Processor -- start 2021-07-11T04:01:40.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba880b9700 1 -- start start 2021-07-11T04:01:40.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba880b9700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fba800b12d0 0x7fba800affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba880b9700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba800a77d0 con 0x7fba800b1ec0 2021-07-11T04:01:40.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba880b9700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba800b0650 con 0x7fba800b22a0 2021-07-11T04:01:40.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba880b9700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba800b1740 con 0x7fba800b12d0 2021-07-11T04:01:40.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba85e55700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fba800b1ec0 0x7fba800a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46622/0 (socket says 172.21.2.104:46622) 2021-07-11T04:01:40.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba85e55700 1 -- 172.21.2.104:0/2753373349 learned_addr learned my addr 172.21.2.104:0/2753373349 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:40.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46715916 0 0) 0x7fba800a77d0 con 0x7fba800b1ec0 2021-07-11T04:01:40.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba68003260 con 0x7fba800b1ec0 2021-07-11T04:01:40.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba85654700 1 --2- 172.21.2.104:0/2753373349 >> v2:172.21.2.105:3300/0 conn(0x7fba800b12d0 0x7fba800affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:40.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1385586424 0 0) 0x7fba68003260 con 0x7fba800b1ec0 2021-07-11T04:01:40.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 >> v2:172.21.2.105:3300/0 conn(0x7fba800b12d0 msgr2=0x7fba800affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:40.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 --2- 172.21.2.104:0/2753373349 >> v2:172.21.2.105:3300/0 conn(0x7fba800b12d0 0x7fba800affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:40.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 >> v1:172.21.2.105:6789/0 conn(0x7fba800b22a0 legacy=0x7fba80141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:40.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba800b08f0 con 0x7fba800b1ec0 2021-07-11T04:01:40.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fba7c001660 con 0x7fba800b1ec0 2021-07-11T04:01:40.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 >> v1:172.21.2.104:6789/0 conn(0x7fba800b1ec0 legacy=0x7fba800a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 --2- 172.21.2.104:0/2753373349 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba68003750 0x7fba68003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 --2- 172.21.2.104:0/2753373349 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba68004500 0x7fba68004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 --2- 172.21.2.104:0/2753373349 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba68004f90 0x7fba6800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba800b1740 con 0x7fba68004500 2021-07-11T04:01:40.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6800be80 con 0x7fba68003750 2021-07-11T04:01:40.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6800c040 con 0x7fba68004f90 2021-07-11T04:01:40.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba85654700 1 --2- 172.21.2.104:0/2753373349 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba68004500 0x7fba68004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:40.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba85654700 1 -- 172.21.2.104:0/2753373349 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba68004f90 msgr2=0x7fba6800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:40.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba85654700 1 --2- 172.21.2.104:0/2753373349 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba68004f90 0x7fba6800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:40.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba85654700 1 -- 172.21.2.104:0/2753373349 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba68003750 msgr2=0x7fba68003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:40.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba85654700 1 --2- 172.21.2.104:0/2753373349 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba68003750 0x7fba68003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:40.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba85654700 1 -- 172.21.2.104:0/2753373349 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba68003260 con 0x7fba68004500 2021-07-11T04:01:40.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba86656700 1 -- 172.21.2.104:0/2753373349 reap_dead start 2021-07-11T04:01:40.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba85654700 1 --2- 172.21.2.104:0/2753373349 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba68004500 0x7fba68004980 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fba70007000 tx=0x7fba70008040).ready entity=mon.0 client_cookie=dbcf9c9cc72b0e58 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:40.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba70012470 con 0x7fba68004500 2021-07-11T04:01:40.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7000c030 con 0x7fba68004500 2021-07-11T04:01:40.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba84e53700 1 -- 172.21.2.104:0/2753373349 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba700127c0 con 0x7fba68004500 2021-07-11T04:01:40.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba880b9700 1 -- 172.21.2.104:0/2753373349 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba68004500 msgr2=0x7fba68004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.424+0000 7fba880b9700 1 --2- 172.21.2.104:0/2753373349 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba68004500 0x7fba68004980 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fba70007000 tx=0x7fba70008040).stop 2021-07-11T04:01:40.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 -- 172.21.2.104:0/2753373349 shutdown_connections 2021-07-11T04:01:40.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 --2- 172.21.2.104:0/2753373349 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba68004500 0x7fba68004980 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:40.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 -- 172.21.2.104:0/2753373349 >> 172.21.2.104:0/2753373349 conn(0x7fba8005f6b0 msgr2=0x7fba8005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:40.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 -- 172.21.2.104:0/2753373349 shutdown_connections 2021-07-11T04:01:40.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 -- 172.21.2.104:0/2753373349 wait complete. 2021-07-11T04:01:40.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 Processor -- start 2021-07-11T04:01:40.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 -- start start 2021-07-11T04:01:40.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba68004500 0x7fba800b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba800b56e0 0x7fba80156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba80157260 0x7fba8015a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba80156980 con 0x7fba800b56e0 2021-07-11T04:01:40.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba80159f10 con 0x7fba80157260 2021-07-11T04:01:40.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8014bb50 con 0x7fba68004500 2021-07-11T04:01:40.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba85654700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba800b56e0 0x7fba80156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:40.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba85654700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba800b56e0 0x7fba80156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49866/0 (socket says 172.21.2.104:49866) 2021-07-11T04:01:40.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba85654700 1 -- 172.21.2.104:0/344204769 learned_addr learned my addr 172.21.2.104:0/344204769 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:40.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba85654700 1 -- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba68004500 msgr2=0x7fba800b6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba85654700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba68004500 0x7fba800b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:40.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba85654700 1 -- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba80157260 msgr2=0x7fba8015a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba85654700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba80157260 0x7fba8015a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:40.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba85654700 1 -- 172.21.2.104:0/344204769 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba800aba80 con 0x7fba800b56e0 2021-07-11T04:01:40.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba85654700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba800b56e0 0x7fba80156d70 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7fba800b5d00 tx=0x7fba70008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:40.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba6effd700 1 -- 172.21.2.104:0/344204769 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba7001c6f0 con 0x7fba800b56e0 2021-07-11T04:01:40.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba6effd700 1 -- 172.21.2.104:0/344204769 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba70000e90 con 0x7fba800b56e0 2021-07-11T04:01:40.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba6effd700 1 -- 172.21.2.104:0/344204769 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fba7001ca40 con 0x7fba800b56e0 2021-07-11T04:01:40.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 -- 172.21.2.104:0/344204769 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba800abd20 con 0x7fba800b56e0 2021-07-11T04:01:40.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 -- 172.21.2.104:0/344204769 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba800b4740 con 0x7fba800b56e0 2021-07-11T04:01:40.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba880b9700 1 -- 172.21.2.104:0/344204769 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba800ac1c0 con 0x7fba800b56e0 2021-07-11T04:01:40.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba6effd700 1 -- 172.21.2.104:0/344204769 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fba7001cba0 con 0x7fba800b56e0 2021-07-11T04:01:40.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba6effd700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba60053260 0x7fba60055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.428+0000 7fba6effd700 1 -- 172.21.2.104:0/344204769 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6071+0+0 (secure 0 0 0) 0x7fba7006c890 con 0x7fba800b56e0 2021-07-11T04:01:40.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.432+0000 7fba85e55700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba60053260 0x7fba60055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:40.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.432+0000 7fba85e55700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba60053260 0x7fba60055720 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fba800b2f50 tx=0x7fba800b0a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:40.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.432+0000 7fba6effd700 1 -- 172.21.2.104:0/344204769 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fba70043ac0 con 0x7fba800b56e0 2021-07-11T04:01:40.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.540+0000 7fba880b9700 1 -- 172.21.2.104:0/344204769 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fba80057350 con 0x7fba800b56e0 2021-07-11T04:01:40.542 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T04:01:40.540+0000 7f2069947700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T04:01:40.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba6effd700 1 -- 172.21.2.104:0/344204769 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 22 v62) v1 ==== 174+0+0 (secure 0 0 0) 0x7fba700470e0 con 0x7fba800b56e0 2021-07-11T04:01:40.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 -- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba60053260 msgr2=0x7fba60055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba60053260 0x7fba60055720 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7fba800b2f50 tx=0x7fba800b0a80).stop 2021-07-11T04:01:40.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 -- 172.21.2.104:0/344204769 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba800b56e0 msgr2=0x7fba80156d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba800b56e0 0x7fba80156d70 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7fba800b5d00 tx=0x7fba70008040).stop 2021-07-11T04:01:40.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 -- 172.21.2.104:0/344204769 shutdown_connections 2021-07-11T04:01:40.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fba80157260 0x7fba8015a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:40.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fba60053260 0x7fba60055720 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:40.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fba800b56e0 0x7fba80156d70 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:40.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 --2- 172.21.2.104:0/344204769 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fba68004500 0x7fba800b6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:40.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 -- 172.21.2.104:0/344204769 >> 172.21.2.104:0/344204769 conn(0x7fba8005f6b0 msgr2=0x7fba800af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:40.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 -- 172.21.2.104:0/344204769 shutdown_connections 2021-07-11T04:01:40.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.548+0000 7fba880b9700 1 -- 172.21.2.104:0/344204769 wait complete. 2021-07-11T04:01:40.551 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 24 and data pool 22 2021-07-11T04:01:40.556 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:01:40.556 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:40.564 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:40.572 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:40.586 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:40.599 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:01:40.599 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:40.600 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:40.600 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:40.610 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T04:01:40.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:01:40.611 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T04:01:40.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:01:40.612 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:40.660 INFO:teuthology.orchestra.run.gibba019.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T04:01:40.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:01:40.662 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:40.662 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:40.663 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:40.671 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T04:01:40.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:01:40.673 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T04:01:40.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:01:40.674 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:40.724 INFO:teuthology.orchestra.run.gibba023.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T04:01:40.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:01:40.725 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:01:40.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.780+0000 7fdf2d79b700 1 Processor -- start 2021-07-11T04:01:40.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.780+0000 7fdf2d79b700 1 -- start start 2021-07-11T04:01:40.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.780+0000 7fdf2d79b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fdf281483a0 0x7fdf280b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.780+0000 7fdf2d79b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf280b0350 con 0x7fdf280b1db0 2021-07-11T04:01:40.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.780+0000 7fdf2d79b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf280b05f0 con 0x7fdf280a70c0 2021-07-11T04:01:40.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.780+0000 7fdf2d79b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf280a90c0 con 0x7fdf281483a0 2021-07-11T04:01:40.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.780+0000 7fdf26ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fdf280b1db0 0x7fdf280b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46642/0 (socket says 172.21.2.104:46642) 2021-07-11T04:01:40.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.780+0000 7fdf26ffd700 1 -- 172.21.2.104:0/760079515 learned_addr learned my addr 172.21.2.104:0/760079515 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:40.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.780+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048091232 0 0) 0x7fdf280b0350 con 0x7fdf280b1db0 2021-07-11T04:01:40.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.780+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf08003260 con 0x7fdf280b1db0 2021-07-11T04:01:40.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf267fc700 1 --2- 172.21.2.104:0/760079515 >> v2:172.21.2.105:3300/0 conn(0x7fdf281483a0 0x7fdf280b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:40.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 709089922 0 0) 0x7fdf08003260 con 0x7fdf280b1db0 2021-07-11T04:01:40.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 >> v2:172.21.2.105:3300/0 conn(0x7fdf281483a0 msgr2=0x7fdf280b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:40.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 --2- 172.21.2.104:0/760079515 >> v2:172.21.2.105:3300/0 conn(0x7fdf281483a0 0x7fdf280b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:40.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 >> v1:172.21.2.105:6789/0 conn(0x7fdf280a70c0 legacy=0x7fdf28144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:40.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf280b0890 con 0x7fdf280b1db0 2021-07-11T04:01:40.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fdf10001660 con 0x7fdf280b1db0 2021-07-11T04:01:40.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 >> v1:172.21.2.104:6789/0 conn(0x7fdf280b1db0 legacy=0x7fdf280b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 --2- 172.21.2.104:0/760079515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf08003850 0x7fdf08003c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 --2- 172.21.2.104:0/760079515 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf08004500 0x7fdf08004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 --2- 172.21.2.104:0/760079515 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf08005070 0x7fdf0800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0800be70 con 0x7fdf08003850 2021-07-11T04:01:40.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0800bfe0 con 0x7fdf08005070 2021-07-11T04:01:40.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0800c1a0 con 0x7fdf08004500 2021-07-11T04:01:40.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf26ffd700 1 --2- 172.21.2.104:0/760079515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf08003850 0x7fdf08003c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:40.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf26ffd700 1 -- 172.21.2.104:0/760079515 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf08004500 msgr2=0x7fdf08004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:40.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf26ffd700 1 --2- 172.21.2.104:0/760079515 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf08004500 0x7fdf08004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:40.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf26ffd700 1 -- 172.21.2.104:0/760079515 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf08005070 msgr2=0x7fdf0800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:40.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf26ffd700 1 --2- 172.21.2.104:0/760079515 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf08005070 0x7fdf0800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:40.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf26ffd700 1 -- 172.21.2.104:0/760079515 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf08003260 con 0x7fdf08003850 2021-07-11T04:01:40.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf277fe700 1 -- 172.21.2.104:0/760079515 reap_dead start 2021-07-11T04:01:40.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf26ffd700 1 --2- 172.21.2.104:0/760079515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf08003850 0x7fdf08003c50 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7fdf28147730 tx=0x7fdf280b0a20).ready entity=mon.0 client_cookie=7d678216b77f2887 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:40.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf100070d0 con 0x7fdf08003850 2021-07-11T04:01:40.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf10007230 con 0x7fdf08003850 2021-07-11T04:01:40.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf25ffb700 1 -- 172.21.2.104:0/760079515 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf1000c8f0 con 0x7fdf08003850 2021-07-11T04:01:40.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- 172.21.2.104:0/760079515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf08003850 msgr2=0x7fdf08003c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 --2- 172.21.2.104:0/760079515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf08003850 0x7fdf08003c50 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7fdf28147730 tx=0x7fdf280b0a20).stop 2021-07-11T04:01:40.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- 172.21.2.104:0/760079515 shutdown_connections 2021-07-11T04:01:40.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 --2- 172.21.2.104:0/760079515 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf08003850 0x7fdf08003c50 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:40.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- 172.21.2.104:0/760079515 >> 172.21.2.104:0/760079515 conn(0x7fdf2805f6b0 msgr2=0x7fdf2805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:40.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- 172.21.2.104:0/760079515 shutdown_connections 2021-07-11T04:01:40.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- 172.21.2.104:0/760079515 wait complete. 2021-07-11T04:01:40.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 Processor -- start 2021-07-11T04:01:40.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- start start 2021-07-11T04:01:40.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf08003850 0x7fdf28141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf28146ed0 0x7fdf28156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf281574c0 0x7fdf281455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28156940 con 0x7fdf28146ed0 2021-07-11T04:01:40.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28148e10 con 0x7fdf281574c0 2021-07-11T04:01:40.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281490b0 con 0x7fdf08003850 2021-07-11T04:01:40.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf267fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf28146ed0 0x7fdf28156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:40.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf267fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf28146ed0 0x7fdf28156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49888/0 (socket says 172.21.2.104:49888) 2021-07-11T04:01:40.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf267fc700 1 -- 172.21.2.104:0/3250777100 learned_addr learned my addr 172.21.2.104:0/3250777100 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:40.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf267fc700 1 -- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf08003850 msgr2=0x7fdf28141d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf267fc700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf08003850 0x7fdf28141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:40.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf267fc700 1 -- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf281574c0 msgr2=0x7fdf281455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:40.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf267fc700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf281574c0 0x7fdf281455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:40.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf267fc700 1 -- 172.21.2.104:0/3250777100 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28156b30 con 0x7fdf28146ed0 2021-07-11T04:01:40.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf267fc700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf28146ed0 0x7fdf28156fd0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7fdf1800a000 tx=0x7fdf18008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:40.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf1ffff700 1 -- 172.21.2.104:0/3250777100 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf18016070 con 0x7fdf28146ed0 2021-07-11T04:01:40.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf1ffff700 1 -- 172.21.2.104:0/3250777100 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf18000e90 con 0x7fdf28146ed0 2021-07-11T04:01:40.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- 172.21.2.104:0/3250777100 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf28148b00 con 0x7fdf28146ed0 2021-07-11T04:01:40.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- 172.21.2.104:0/3250777100 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf28142820 con 0x7fdf28146ed0 2021-07-11T04:01:40.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf1ffff700 1 -- 172.21.2.104:0/3250777100 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf18011970 con 0x7fdf28146ed0 2021-07-11T04:01:40.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.784+0000 7fdf2d79b700 1 -- 172.21.2.104:0/3250777100 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf280aca50 con 0x7fdf28146ed0 2021-07-11T04:01:40.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.788+0000 7fdf1ffff700 1 -- 172.21.2.104:0/3250777100 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fdf18011ad0 con 0x7fdf28146ed0 2021-07-11T04:01:40.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.788+0000 7fdf1ffff700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf0c053270 0x7fdf0c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:40.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.788+0000 7fdf1ffff700 1 -- 172.21.2.104:0/3250777100 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fdf1806b3c0 con 0x7fdf28146ed0 2021-07-11T04:01:40.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.788+0000 7fdf26ffd700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf0c053270 0x7fdf0c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:40.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.788+0000 7fdf26ffd700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf0c053270 0x7fdf0c055730 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fdf28147730 tx=0x7fdf10013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:40.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.788+0000 7fdf1ffff700 1 -- 172.21.2.104:0/3250777100 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fdf180425a0 con 0x7fdf28146ed0 2021-07-11T04:01:40.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.872+0000 7fdf2d79b700 1 -- 172.21.2.104:0/3250777100 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fdf28149270 con 0x7fdf28146ed0 2021-07-11T04:01:40.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.872+0000 7fdf1ffff700 1 -- 172.21.2.104:0/3250777100 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v130) v1 ==== 81+0+33826 (secure 0 0 0) 0x7fdf18045bc0 con 0x7fdf28146ed0 2021-07-11T04:01:40.874 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:40.875 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:01:40.875 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 130, 2021-07-11T04:01:40.875 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:01:40.875 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:01:40.876 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:01:40.544600+0000", 2021-07-11T04:01:40.876 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:01:40.876 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:01:40.876 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:01:40.877 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:01:40.877 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:01:40.877 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:01:40.877 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:01:40.877 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:01:40.878 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:01:40.878 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:40.878 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:01:40.879 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:01:40.879 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:01:40.879 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:01:40.879 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:01:40.879 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 24, 2021-07-11T04:01:40.880 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:01:40.880 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:01:40.880 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:01:40.880 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:01:40.880 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:01:40.881 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.881 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:01:40.881 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:01:40.881 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:01:40.882 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:01:40.882 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:01:40.883 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:40.883 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:40.883 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:40.883 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:40.883 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:40.884 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:40.884 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:01:40.884 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:01:40.884 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:01:40.885 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:01:40.885 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:01:40.885 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:40.885 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:40.886 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:40.886 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:40.886 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:40.886 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:40.887 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:40.887 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.887 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:01:40.887 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:40.888 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:40.888 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:40.888 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:40.888 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:40.888 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:40.889 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:40.889 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:40.889 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:40.889 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:40.889 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:40.890 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:40.890 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:40.890 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:40.890 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:40.891 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:40.891 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:40.891 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:40.891 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:40.892 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:40.892 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:40.892 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:40.892 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:40.892 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:40.893 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:40.893 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:40.893 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.893 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:40.894 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:40.894 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:40.894 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:40.894 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:40.895 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:40.895 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:40.895 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:40.895 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:40.896 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:40.896 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:40.896 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:01:40.896 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:01:40.896 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.897 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:40.897 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:01:40.897 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.897 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.898 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.898 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:01:40.898 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:01:40.898 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:01:40.898 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:01:40.899 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:01:40.899 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:40.899 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:40.899 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:40.900 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:40.900 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:40.900 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:40.900 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:01:40.900 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:01:40.901 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:01:40.901 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:01:40.901 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:01:40.901 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:40.902 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:40.902 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:40.902 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:40.902 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:40.902 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:40.903 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:40.903 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.903 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:01:40.903 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:40.903 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:40.904 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:40.904 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:40.904 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:40.905 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:40.905 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:40.905 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:40.905 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:40.906 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:40.906 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:40.906 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:40.906 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:40.907 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:40.907 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:40.907 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:40.907 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:40.907 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:40.908 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:40.908 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:40.908 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:40.908 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:40.908 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:40.909 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:40.909 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:40.909 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:40.909 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.909 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:40.910 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:40.910 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:40.910 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:40.910 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:40.911 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:40.911 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:40.911 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:40.911 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:40.911 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:40.912 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:40.912 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:01:40.912 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:40.912 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:01:40.913 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.913 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.913 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.913 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 22, 2021-07-11T04:01:40.913 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T04:01:40.914 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:01:12.008882+0000", 2021-07-11T04:01:40.914 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:01:40.914 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:01:40.914 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:40.915 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:40.915 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:40.915 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:40.915 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:40.916 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:40.916 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:01:40.916 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:01:40.916 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:01:40.917 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:01:40.917 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:01:40.917 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:40.917 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:40.917 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:40.918 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:40.918 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:40.918 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:40.918 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:40.919 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.919 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "130", 2021-07-11T04:01:40.919 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:40.919 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:40.920 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:40.920 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:40.920 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:40.920 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:40.920 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:40.921 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:40.921 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:40.921 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:40.921 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:40.922 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:40.922 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:40.922 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:40.922 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:40.922 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:40.923 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:40.923 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:40.923 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:40.923 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:40.923 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:40.924 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:40.924 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:40.924 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:40.924 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:40.925 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:40.925 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.925 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:40.925 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:40.926 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:40.926 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:40.926 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:40.926 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:40.927 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:40.927 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:40.927 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:40.927 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:40.927 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:40.928 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:01:40.928 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:40.928 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:01:40.928 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T04:01:40.929 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.929 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.929 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.929 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.929 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 24, 2021-07-11T04:01:40.930 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T04:01:40.930 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:01:39.359875+0000", 2021-07-11T04:01:40.930 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 32769, 2021-07-11T04:01:40.930 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool,creating", 2021-07-11T04:01:40.931 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:40.931 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:40.931 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:40.931 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:40.931 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:40.932 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:40.932 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:01:40.932 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:01:40.932 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:01:40.933 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:01:40.933 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:01:40.933 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:40.933 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:40.933 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:40.934 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:40.934 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:40.934 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:40.934 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:40.935 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.935 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "130", 2021-07-11T04:01:40.935 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:40.935 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:40.936 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:40.936 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:40.936 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:40.936 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:40.937 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:40.937 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:40.937 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:40.937 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:40.937 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:40.938 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:40.938 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:40.938 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:40.938 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:40.939 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:40.939 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:40.939 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:40.939 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:40.939 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:40.940 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:40.940 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:40.940 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:40.940 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:40.941 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:40.941 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:40.941 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.941 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:40.942 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:40.942 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:40.942 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:40.942 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:40.943 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:40.943 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:40.943 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:40.943 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:40.943 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:40.944 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:40.944 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:01:40.944 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T04:01:40.944 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T04:01:40.944 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T04:01:40.945 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.945 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:40.945 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:01:40.945 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T04:01:40.946 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.946 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.946 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.947 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:40.947 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:01:40.947 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.947 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:01:40.947 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:01:40.948 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:40.948 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:40.948 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:40.948 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:40.949 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:40.949 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:40.949 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:40.949 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 129, 2021-07-11T04:01:40.950 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:40.950 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:40.950 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:40.950 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.950 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.951 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.951 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:01:40.951 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:40.951 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.952 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.952 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.952 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:01:40.952 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:40.952 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.953 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.953 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.953 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:40.953 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.954 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.954 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.954 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:01:40.954 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:40.954 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.955 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.955 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.955 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:01:40.955 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:40.956 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.956 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.956 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.956 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:40.957 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.957 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.957 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.957 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:01:40.958 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:40.958 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.958 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.958 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.958 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:01:40.959 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:40.959 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.959 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.959 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.960 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:40.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.960 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.960 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:01:40.961 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:40.961 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.961 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.961 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.961 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:01:40.962 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:40.962 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.962 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.962 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.963 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:01:40.963 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:01:40.963 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:01:40.963 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:01:40.963 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:40.964 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:40.964 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:40.964 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.964 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.965 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.965 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:01:40.965 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:01:40.965 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:40.965 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:40.966 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:40.966 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:40.966 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:40.967 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:40.967 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:40.967 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 129, 2021-07-11T04:01:40.967 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:40.967 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:40.968 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:40.968 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.968 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.968 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.968 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:01:40.969 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:40.969 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.969 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.969 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.970 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:01:40.970 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:40.970 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.970 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.971 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.971 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:40.971 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.971 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.971 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.972 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:01:40.972 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:40.973 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.973 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.973 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.973 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:01:40.973 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:40.974 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.974 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.974 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.974 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:40.975 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.975 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.975 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.976 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:01:40.976 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:40.976 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.976 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.976 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.977 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:01:40.977 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:40.977 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.977 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.978 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.978 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:40.978 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.978 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.979 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.979 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:01:40.979 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:40.979 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.979 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.980 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.980 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:01:40.980 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:40.980 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.981 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.981 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.981 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:01:40.981 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:01:40.981 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:01:40.982 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:01:40.982 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:40.982 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:40.982 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:40.983 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.983 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.983 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.983 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:01:40.984 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:01:40.984 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:40.984 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:40.984 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:40.985 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:40.985 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:40.985 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:40.985 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:40.986 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 124, 2021-07-11T04:01:40.986 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:40.986 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:40.986 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:40.987 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.987 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.987 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.987 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:01:40.987 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:40.988 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.988 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.988 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.988 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:01:40.988 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:40.989 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.989 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.989 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.989 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:40.990 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.990 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.990 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.990 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:01:40.990 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:40.991 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.991 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.991 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.991 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:01:40.992 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:40.992 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.992 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.992 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.992 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:40.993 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.993 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.993 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.993 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:01:40.993 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:40.994 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.994 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.994 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.994 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:01:40.995 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:40.995 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.995 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.995 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.996 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:40.996 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:40.996 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.996 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:40.997 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:01:40.997 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:40.997 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.997 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:40.997 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:40.998 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:01:40.998 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:40.998 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:40.998 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:40.999 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:40.999 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:01:40.999 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:01:40.999 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:01:40.999 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:01:41.000 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:41.000 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:41.000 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:41.000 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.001 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.001 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.001 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:01:41.001 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:01:41.001 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:41.002 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:41.002 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:41.002 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:41.002 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:41.003 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:41.003 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:41.003 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 124, 2021-07-11T04:01:41.003 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:41.003 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:41.004 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:41.004 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.004 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.004 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.004 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:01:41.005 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:41.005 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.005 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.005 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.006 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:01:41.006 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:41.006 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.007 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.007 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.007 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:41.007 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.007 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.008 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.008 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:01:41.008 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:41.008 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.008 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.009 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.009 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:01:41.009 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:41.009 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.010 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.010 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.010 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:41.010 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.011 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.011 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.011 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:01:41.011 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:41.012 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.012 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.012 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.012 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:01:41.012 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:41.013 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.013 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.013 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.013 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:41.013 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.014 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.014 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.014 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:01:41.014 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:41.014 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.015 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.015 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.015 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:01:41.015 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:41.016 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.016 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.016 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.017 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:01:41.017 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:01:41.017 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:01:41.017 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:01:41.017 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:41.018 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:41.018 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:41.018 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.018 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.019 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.019 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:01:41.019 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:01:41.019 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:41.020 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:41.020 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:41.020 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:41.020 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:41.020 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:41.021 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:41.021 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 124, 2021-07-11T04:01:41.021 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:41.021 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:41.022 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:41.022 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.022 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.022 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.023 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:01:41.023 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:41.023 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.023 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.024 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.024 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:01:41.024 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:41.024 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.025 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.025 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.025 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:41.025 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.025 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.026 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.026 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:01:41.026 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:41.027 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.027 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.027 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.027 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:01:41.028 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:41.028 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.028 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.028 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.028 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:41.029 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.029 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.029 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.029 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:01:41.030 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:41.030 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.030 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.030 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.031 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:01:41.031 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:41.031 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.031 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.032 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.032 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:41.032 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.032 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.032 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.033 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:01:41.033 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:41.033 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.033 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.034 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.034 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:01:41.034 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:41.034 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.035 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.035 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.035 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:01:41.035 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:01:41.035 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:01:41.036 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:01:41.036 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:41.036 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:41.036 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:41.037 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.037 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.037 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.037 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:01:41.038 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:01:41.038 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:41.038 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:41.038 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:41.039 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:41.039 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:41.039 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:41.039 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:41.039 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 124, 2021-07-11T04:01:41.040 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:41.040 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:41.040 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:41.040 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.041 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.041 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.041 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:01:41.041 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:41.041 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.042 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.042 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.042 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:01:41.042 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:41.043 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.043 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.043 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.043 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:41.044 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.044 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.044 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.044 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:01:41.045 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:41.045 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.045 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.045 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.045 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:01:41.046 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:41.046 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.046 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.046 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.047 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:41.047 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.048 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.048 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.048 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:01:41.048 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:41.049 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.049 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.049 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.049 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:01:41.049 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:41.050 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.050 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.050 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.051 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:41.051 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.051 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.051 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.052 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:01:41.052 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:41.052 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.052 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.052 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.053 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:01:41.053 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:41.053 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.053 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.054 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.054 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:01:41.054 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:01:41.054 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:01:41.055 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:01:41.055 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:41.055 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:41.055 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:41.056 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.056 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.056 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.056 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:01:41.056 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:01:41.057 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:41.057 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:41.057 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:41.058 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:41.058 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:41.058 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:41.058 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:41.058 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 117, 2021-07-11T04:01:41.059 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:41.059 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:41.059 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:41.059 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.060 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.060 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.060 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:01:41.060 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:41.060 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.061 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.061 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.061 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:01:41.061 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:41.062 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.062 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.062 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.062 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:41.062 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.063 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.063 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.063 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:01:41.063 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:41.064 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.064 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.064 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.064 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:01:41.064 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:41.065 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.065 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.065 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.065 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:41.066 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.066 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.066 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.066 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:01:41.066 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:41.067 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.067 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.067 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.068 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:01:41.068 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:41.068 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.068 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.069 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.069 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:41.069 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.069 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.070 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.070 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:01:41.070 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:41.070 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.071 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.071 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.071 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:01:41.071 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:41.072 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.072 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.072 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.072 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:01:41.073 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:01:41.073 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:01:41.073 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:01:41.073 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:41.074 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:41.074 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:41.074 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.074 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.075 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.075 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:01:41.075 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:01:41.075 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:41.076 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:41.076 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:41.076 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:41.076 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:41.077 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:41.077 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:41.077 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 129, 2021-07-11T04:01:41.077 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:41.078 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:41.078 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:41.078 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.079 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.079 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.079 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:01:41.079 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:41.079 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.080 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.080 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.080 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:01:41.080 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:41.081 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.081 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.081 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.081 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:41.081 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.082 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.082 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.082 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:01:41.082 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:41.083 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.083 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.083 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.083 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:01:41.084 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:41.084 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.084 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.084 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.084 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:41.085 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.085 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.085 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.085 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:01:41.086 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:41.086 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.086 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.086 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.086 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:01:41.087 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:41.087 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.087 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.087 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.088 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:41.088 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:41.088 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.089 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:41.089 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:01:41.089 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:41.089 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.089 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.090 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:41.090 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:01:41.090 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:41.090 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.091 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.091 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.091 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:01:41.091 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:01:41.091 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:01:41.092 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:01:41.092 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:41.092 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:41.092 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:41.093 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:41.093 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.093 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:41.093 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:01:41.094 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.094 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:01:41.094 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:41.094 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:41.094 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:41.095 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:41.095 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:41.095 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:01:41.095 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:41.096 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.096 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.096 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:01:41.096 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:41.096 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:41.097 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:41.097 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:41.097 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:41.097 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:01:41.098 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:41.098 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.098 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.099 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:01:41.099 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:41.099 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:41.099 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:41.100 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:41.100 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:41.100 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:01:41.100 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:41.100 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.101 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.101 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:01:41.101 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:41.101 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:41.102 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:41.102 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:41.102 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:41.102 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:01:41.103 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:41.103 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.103 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.103 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:01:41.104 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:41.104 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:41.104 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:41.104 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:41.105 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:41.105 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:01:41.105 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:41.105 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.105 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.106 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:01:41.106 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:41.106 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:41.106 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:41.107 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:41.107 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:41.107 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:01:41.107 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:41.108 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.108 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.108 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:01:41.108 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:41.109 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:41.109 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:41.109 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:41.109 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:41.109 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:01:41.110 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:41.110 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.110 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:41.110 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:01:41.111 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:41.111 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:41.111 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:41.111 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:41.111 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:41.112 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:01:41.112 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:41.112 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.112 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:41.113 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:01:41.113 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:01:41.113 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:01:41.113 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:01:41.114 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T04:01:41.114 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6835/1367521056": "2021-07-12T04:01:26.954459+0000", 2021-07-11T04:01:41.114 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6833/1367521056": "2021-07-12T04:01:26.954459+0000" 2021-07-11T04:01:41.114 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.114 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:01:41.115 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:01:41.115 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:01:41.115 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:01:41.115 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:01:41.116 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:01:41.116 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:01:41.116 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:01:41.116 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:41.116 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:41.117 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:01:41.117 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:01:41.117 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:01:41.117 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:01:41.118 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:01:41.118 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:01:41.119 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 -- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf0c053270 msgr2=0x7fdf0c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf0c053270 0x7fdf0c055730 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fdf28147730 tx=0x7fdf10013000).stop 2021-07-11T04:01:41.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 -- 172.21.2.104:0/3250777100 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf28146ed0 msgr2=0x7fdf28156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf28146ed0 0x7fdf28156fd0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7fdf1800a000 tx=0x7fdf18008040).stop 2021-07-11T04:01:41.120 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 -- 172.21.2.104:0/3250777100 shutdown_connections 2021-07-11T04:01:41.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf281574c0 0x7fdf281455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:41.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf0c053270 0x7fdf0c055730 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:41.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf28146ed0 0x7fdf28156fd0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:41.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 --2- 172.21.2.104:0/3250777100 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf08003850 0x7fdf28141d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:41.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 -- 172.21.2.104:0/3250777100 >> 172.21.2.104:0/3250777100 conn(0x7fdf2805f6b0 msgr2=0x7fdf280b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:41.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 -- 172.21.2.104:0/3250777100 shutdown_connections 2021-07-11T04:01:41.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:40.876+0000 7fdf2d79b700 1 -- 172.21.2.104:0/3250777100 wait complete. 2021-07-11T04:01:41.123 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:41.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9f0279700 1 Processor -- start 2021-07-11T04:01:41.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9f0279700 1 -- start start 2021-07-11T04:01:41.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9f0279700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9e8066d80 0x7fb9e814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9f0279700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb9e8067160 0x7fb9e8149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9f0279700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e814b9b0 con 0x7fb9e814a3d0 2021-07-11T04:01:41.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9f0279700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e80b4f10 con 0x7fb9e8067160 2021-07-11T04:01:41.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9f0279700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80aac20 con 0x7fb9e8066d80 2021-07-11T04:01:41.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9ed814700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb9e8067160 0x7fb9e8149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:41.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9ee015700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9e8066d80 0x7fb9e814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:41.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9ed814700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb9e8067160 0x7fb9e8149ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:57454/0 (socket says 172.21.2.104:57454) 2021-07-11T04:01:41.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9ed814700 1 -- 172.21.2.104:0/2493466349 learned_addr learned my addr 172.21.2.104:0/2493466349 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:41.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9ed814700 1 -- 172.21.2.104:0/2493466349 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9e8066d80 msgr2=0x7fb9e814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:41.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9ed814700 1 --2- 172.21.2.104:0/2493466349 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9e8066d80 0x7fb9e814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:41.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9ed814700 1 -- 172.21.2.104:0/2493466349 >> v1:172.21.2.105:6789/0 conn(0x7fb9e814a3d0 legacy=0x7fb9e8147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9ed814700 1 -- 172.21.2.104:0/2493466349 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e80aaec0 con 0x7fb9e8067160 2021-07-11T04:01:41.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.196+0000 7fb9ed814700 1 --2- 172.21.2.104:0/2493466349 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9e8067160 0x7fb9e8149ee0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fb9e4002f70 tx=0x7fb9e4006eb0).ready entity=mon.1 client_cookie=472ab797ee4d2989 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:41.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ed013700 1 -- 172.21.2.104:0/2493466349 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9e400d570 con 0x7fb9e8067160 2021-07-11T04:01:41.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ed013700 1 -- 172.21.2.104:0/2493466349 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e4000e90 con 0x7fb9e8067160 2021-07-11T04:01:41.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ed013700 1 -- 172.21.2.104:0/2493466349 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9e400d900 con 0x7fb9e8067160 2021-07-11T04:01:41.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- 172.21.2.104:0/2493466349 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9e8067160 msgr2=0x7fb9e8149ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.204 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 --2- 172.21.2.104:0/2493466349 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9e8067160 0x7fb9e8149ee0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fb9e4002f70 tx=0x7fb9e4006eb0).stop 2021-07-11T04:01:41.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- 172.21.2.104:0/2493466349 shutdown_connections 2021-07-11T04:01:41.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 --2- 172.21.2.104:0/2493466349 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9e8067160 0x7fb9e8149ee0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:41.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 --2- 172.21.2.104:0/2493466349 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9e8066d80 0x7fb9e814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:41.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- 172.21.2.104:0/2493466349 >> 172.21.2.104:0/2493466349 conn(0x7fb9e805c490 msgr2=0x7fb9e805c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:41.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- 172.21.2.104:0/2493466349 shutdown_connections 2021-07-11T04:01:41.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- 172.21.2.104:0/2493466349 wait complete. 2021-07-11T04:01:41.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 Processor -- start 2021-07-11T04:01:41.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- start start 2021-07-11T04:01:41.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9e8066d80 0x7fb9e8143970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9e8067160 0x7fb9e80b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9e814a3d0 0x7fb9e8159ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e81440d0 con 0x7fb9e814a3d0 2021-07-11T04:01:41.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80b4350 con 0x7fb9e8066d80 2021-07-11T04:01:41.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80b2ba0 con 0x7fb9e8067160 2021-07-11T04:01:41.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ee816700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9e814a3d0 0x7fb9e8159ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:41.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ee816700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9e814a3d0 0x7fb9e8159ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49904/0 (socket says 172.21.2.104:49904) 2021-07-11T04:01:41.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ee816700 1 -- 172.21.2.104:0/3665667886 learned_addr learned my addr 172.21.2.104:0/3665667886 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:41.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ee816700 1 -- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9e8067160 msgr2=0x7fb9e80b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ee816700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9e8067160 0x7fb9e80b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:41.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ee816700 1 -- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9e8066d80 msgr2=0x7fb9e8143970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ee816700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9e8066d80 0x7fb9e8143970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:41.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ee816700 1 -- 172.21.2.104:0/3665667886 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e400b1e0 con 0x7fb9e814a3d0 2021-07-11T04:01:41.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9ee816700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9e814a3d0 0x7fb9e8159ca0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7fb9e80ad8f0 tx=0x7fb9e0007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:41.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9db7fe700 1 -- 172.21.2.104:0/3665667886 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9e0006b70 con 0x7fb9e814a3d0 2021-07-11T04:01:41.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9db7fe700 1 -- 172.21.2.104:0/3665667886 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e0006cd0 con 0x7fb9e814a3d0 2021-07-11T04:01:41.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- 172.21.2.104:0/3665667886 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e80b2e40 con 0x7fb9e814a3d0 2021-07-11T04:01:41.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- 172.21.2.104:0/3665667886 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e80b3910 con 0x7fb9e814a3d0 2021-07-11T04:01:41.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9db7fe700 1 -- 172.21.2.104:0/3665667886 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb9e0007220 con 0x7fb9e814a3d0 2021-07-11T04:01:41.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9f0279700 1 -- 172.21.2.104:0/3665667886 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c4003410 con 0x7fb9e814a3d0 2021-07-11T04:01:41.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9db7fe700 1 -- 172.21.2.104:0/3665667886 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb9e0010030 con 0x7fb9e814a3d0 2021-07-11T04:01:41.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9db7fe700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9c8053380 0x7fb9c8055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.200+0000 7fb9db7fe700 1 -- 172.21.2.104:0/3665667886 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fb9e006b5e0 con 0x7fb9e814a3d0 2021-07-11T04:01:41.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.204+0000 7fb9ee015700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9c8053380 0x7fb9c8055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:41.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.204+0000 7fb9ee015700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9c8053380 0x7fb9c8055840 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fb9dc00c000 tx=0x7fb9dc006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:41.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.204+0000 7fb9db7fe700 1 -- 172.21.2.104:0/3665667886 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb9e00427c0 con 0x7fb9e814a3d0 2021-07-11T04:01:41.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 -- 172.21.2.104:0/3665667886 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9c4003c40 con 0x7fb9e814a3d0 2021-07-11T04:01:41.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9db7fe700 1 -- 172.21.2.104:0/3665667886 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+1426 (secure 0 0 0) 0x7fb9e0045de0 con 0x7fb9e814a3d0 2021-07-11T04:01:41.312 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:41.312 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":62,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:40.544726+0000","modified":"2021-07-11T04:01:40.544778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-07-11T04:01:41.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 -- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9c8053380 msgr2=0x7fb9c8055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9c8053380 0x7fb9c8055840 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fb9dc00c000 tx=0x7fb9dc006df0).stop 2021-07-11T04:01:41.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 -- 172.21.2.104:0/3665667886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9e814a3d0 msgr2=0x7fb9e8159ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9e814a3d0 0x7fb9e8159ca0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7fb9e80ad8f0 tx=0x7fb9e0007ef0).stop 2021-07-11T04:01:41.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 -- 172.21.2.104:0/3665667886 shutdown_connections 2021-07-11T04:01:41.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb9c8053380 0x7fb9c8055840 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:41.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb9e814a3d0 0x7fb9e8159ca0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:41.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb9e8067160 0x7fb9e80b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:41.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 --2- 172.21.2.104:0/3665667886 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb9e8066d80 0x7fb9e8143970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:41.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 -- 172.21.2.104:0/3665667886 >> 172.21.2.104:0/3665667886 conn(0x7fb9e805c490 msgr2=0x7fb9e81464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:41.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 -- 172.21.2.104:0/3665667886 shutdown_connections 2021-07-11T04:01:41.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.312+0000 7fb9f0279700 1 -- 172.21.2.104:0/3665667886 wait complete. 2021-07-11T04:01:41.316 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 62 2021-07-11T04:01:41.319 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T04:01:41.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 Processor -- start 2021-07-11T04:01:41.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- start start 2021-07-11T04:01:41.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd55c060d60 0x7fd55c0ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd55c1411e0 0x7fd55c141670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd55c0a9ef0 con 0x7fd55c0ac600 2021-07-11T04:01:41.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd55c0619b0 con 0x7fd55c060d60 2021-07-11T04:01:41.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c0b20b0 con 0x7fd55c1411e0 2021-07-11T04:01:41.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55affd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd55c060d60 0x7fd55c0ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:41.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55affd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd55c060d60 0x7fd55c0ac9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49910/0 (socket says 172.21.2.104:49910) 2021-07-11T04:01:41.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55affd700 1 -- 172.21.2.104:0/1925482230 learned_addr learned my addr 172.21.2.104:0/1925482230 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:41.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55affd700 1 -- 172.21.2.104:0/1925482230 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd55c1411e0 msgr2=0x7fd55c141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:41.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55affd700 1 --2- 172.21.2.104:0/1925482230 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd55c1411e0 0x7fd55c141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:41.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55affd700 1 -- 172.21.2.104:0/1925482230 >> v1:172.21.2.104:6789/0 conn(0x7fd55c0ac600 legacy=0x7fd55c066c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55affd700 1 -- 172.21.2.104:0/1925482230 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55c0b2350 con 0x7fd55c060d60 2021-07-11T04:01:41.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55affd700 1 --2- 172.21.2.104:0/1925482230 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd55c060d60 0x7fd55c0ac9e0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7fd550002f70 tx=0x7fd550006eb0).ready entity=mon.0 client_cookie=6ebd262f4d391211 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:41.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55a7fc700 1 -- 172.21.2.104:0/1925482230 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd55000d5c0 con 0x7fd55c060d60 2021-07-11T04:01:41.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55a7fc700 1 -- 172.21.2.104:0/1925482230 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd550000e90 con 0x7fd55c060d60 2021-07-11T04:01:41.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55a7fc700 1 -- 172.21.2.104:0/1925482230 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd55000d950 con 0x7fd55c060d60 2021-07-11T04:01:41.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- 172.21.2.104:0/1925482230 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd55c060d60 msgr2=0x7fd55c0ac9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 --2- 172.21.2.104:0/1925482230 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd55c060d60 0x7fd55c0ac9e0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7fd550002f70 tx=0x7fd550006eb0).stop 2021-07-11T04:01:41.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- 172.21.2.104:0/1925482230 shutdown_connections 2021-07-11T04:01:41.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 --2- 172.21.2.104:0/1925482230 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd55c060d60 0x7fd55c0ac9e0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:41.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 --2- 172.21.2.104:0/1925482230 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd55c1411e0 0x7fd55c141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:41.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- 172.21.2.104:0/1925482230 >> 172.21.2.104:0/1925482230 conn(0x7fd55c0b64d0 msgr2=0x7fd55c0b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:41.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- 172.21.2.104:0/1925482230 shutdown_connections 2021-07-11T04:01:41.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- 172.21.2.104:0/1925482230 wait complete. 2021-07-11T04:01:41.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 Processor -- start 2021-07-11T04:01:41.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- start start 2021-07-11T04:01:41.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd55c060d60 0x7fd55c0660e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd55c0ac600 0x7fd55c143ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd55c1411e0 0x7fd55c143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c142040 con 0x7fd55c060d60 2021-07-11T04:01:41.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c065a60 con 0x7fd55c1411e0 2021-07-11T04:01:41.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd5621bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c064990 con 0x7fd55c0ac600 2021-07-11T04:01:41.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd55c060d60 0x7fd55c0660e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:41.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd55c060d60 0x7fd55c0660e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49918/0 (socket says 172.21.2.104:49918) 2021-07-11T04:01:41.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55b7fe700 1 -- 172.21.2.104:0/2724620804 learned_addr learned my addr 172.21.2.104:0/2724620804 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:41.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55bfff700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd55c1411e0 0x7fd55c143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:41.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55b7fe700 1 -- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd55c0ac600 msgr2=0x7fd55c143ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55b7fe700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd55c0ac600 0x7fd55c143ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:41.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55b7fe700 1 -- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd55c1411e0 msgr2=0x7fd55c143fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:41.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55b7fe700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd55c1411e0 0x7fd55c143fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:41.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.372+0000 7fd55b7fe700 1 -- 172.21.2.104:0/2724620804 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd55000b1e0 con 0x7fd55c060d60 2021-07-11T04:01:41.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd55bfff700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd55c1411e0 0x7fd55c143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:41.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd55b7fe700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd55c060d60 0x7fd55c0660e0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fd55c063220 tx=0x7fd54c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:41.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd558ff9700 1 -- 172.21.2.104:0/2724620804 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd54c00ce30 con 0x7fd55c060d60 2021-07-11T04:01:41.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd558ff9700 1 -- 172.21.2.104:0/2724620804 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54c00f470 con 0x7fd55c060d60 2021-07-11T04:01:41.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd5621bb700 1 -- 172.21.2.104:0/2724620804 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55c1444a0 con 0x7fd55c060d60 2021-07-11T04:01:41.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd558ff9700 1 -- 172.21.2.104:0/2724620804 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd54c00f7d0 con 0x7fd55c060d60 2021-07-11T04:01:41.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd5621bb700 1 -- 172.21.2.104:0/2724620804 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55c0657c0 con 0x7fd55c060d60 2021-07-11T04:01:41.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd5621bb700 1 -- 172.21.2.104:0/2724620804 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55c057350 con 0x7fd55c060d60 2021-07-11T04:01:41.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd558ff9700 1 -- 172.21.2.104:0/2724620804 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd54c019030 con 0x7fd55c060d60 2021-07-11T04:01:41.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd558ff9700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd53c0531f0 0x7fd53c0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:41.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd558ff9700 1 -- 172.21.2.104:0/2724620804 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fd54c06a980 con 0x7fd55c060d60 2021-07-11T04:01:41.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd55affd700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd53c0531f0 0x7fd53c0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:41.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd55affd700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd53c0531f0 0x7fd53c0556b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fd550002f70 tx=0x7fd550015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:41.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.376+0000 7fd558ff9700 1 -- 172.21.2.104:0/2724620804 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd54c041c10 con 0x7fd55c060d60 2021-07-11T04:01:41.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:41.484+0000 7fd5621bb700 1 -- 172.21.2.104:0/2724620804 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd55c0ae350 con 0x7fd55c060d60 2021-07-11T04:01:42.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd558ff9700 1 -- 172.21.2.104:0/2724620804 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd54c045230 con 0x7fd55c060d60 2021-07-11T04:01:42.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 -- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd53c0531f0 msgr2=0x7fd53c0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:42.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd53c0531f0 0x7fd53c0556b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fd550002f70 tx=0x7fd550015000).stop 2021-07-11T04:01:42.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 -- 172.21.2.104:0/2724620804 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd55c060d60 msgr2=0x7fd55c0660e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:42.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd55c060d60 0x7fd55c0660e0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fd55c063220 tx=0x7fd54c007c50).stop 2021-07-11T04:01:42.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 -- 172.21.2.104:0/2724620804 shutdown_connections 2021-07-11T04:01:42.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd55c1411e0 0x7fd55c143fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:42.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd55c0ac600 0x7fd55c143ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:42.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd53c0531f0 0x7fd53c0556b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:42.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 --2- 172.21.2.104:0/2724620804 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd55c060d60 0x7fd55c0660e0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:42.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 -- 172.21.2.104:0/2724620804 >> 172.21.2.104:0/2724620804 conn(0x7fd55c0b64d0 msgr2=0x7fd55c060560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:42.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 -- 172.21.2.104:0/2724620804 shutdown_connections 2021-07-11T04:01:42.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.364+0000 7fd5621bb700 1 -- 172.21.2.104:0/2724620804 wait complete. 2021-07-11T04:01:42.368 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T04:01:42.372 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:42.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 Processor -- start 2021-07-11T04:01:42.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 -- start start 2021-07-11T04:01:42.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d8c0a76b0 0x7f8d8c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8d8c0b1970 0x7f8d8c066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8c067450 con 0x7f8d8c0b0060 2021-07-11T04:01:42.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8c0a7a90 con 0x7f8d8c0b1970 2021-07-11T04:01:42.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c147620 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d89d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8d8c0b1970 0x7f8d8c066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:42.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d89d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8d8c0b1970 0x7f8d8c066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49926/0 (socket says 172.21.2.104:49926) 2021-07-11T04:01:42.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d89d9b700 1 -- 172.21.2.104:0/474121360 learned_addr learned my addr 172.21.2.104:0/474121360 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:42.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d89d9b700 1 -- 172.21.2.104:0/474121360 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d8c0a76b0 msgr2=0x7f8d8c066a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:42.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d89d9b700 1 --2- 172.21.2.104:0/474121360 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d8c0a76b0 0x7f8d8c066a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:42.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d89d9b700 1 -- 172.21.2.104:0/474121360 >> v1:172.21.2.105:6789/0 conn(0x7f8d8c0b0060 legacy=0x7f8d8c05f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:42.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d89d9b700 1 -- 172.21.2.104:0/474121360 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c1478c0 con 0x7f8d8c0b1970 2021-07-11T04:01:42.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d89d9b700 1 --2- 172.21.2.104:0/474121360 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d8c0b1970 0x7f8d8c066f60 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f8d8c0b6460 tx=0x7f8d74006eb0).ready entity=mon.0 client_cookie=23736387cb49c354 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:42.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d8959a700 1 -- 172.21.2.104:0/474121360 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d7400d570 con 0x7f8d8c0b1970 2021-07-11T04:01:42.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d8959a700 1 -- 172.21.2.104:0/474121360 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d74000e90 con 0x7f8d8c0b1970 2021-07-11T04:01:42.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d8959a700 1 -- 172.21.2.104:0/474121360 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d7400d8c0 con 0x7f8d8c0b1970 2021-07-11T04:01:42.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 -- 172.21.2.104:0/474121360 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d8c0b1970 msgr2=0x7f8d8c066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:42.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 --2- 172.21.2.104:0/474121360 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d8c0b1970 0x7f8d8c066f60 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f8d8c0b6460 tx=0x7f8d74006eb0).stop 2021-07-11T04:01:42.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 -- 172.21.2.104:0/474121360 shutdown_connections 2021-07-11T04:01:42.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 --2- 172.21.2.104:0/474121360 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d8c0b1970 0x7f8d8c066f60 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:42.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 --2- 172.21.2.104:0/474121360 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d8c0a76b0 0x7f8d8c066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:42.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 -- 172.21.2.104:0/474121360 >> 172.21.2.104:0/474121360 conn(0x7f8d8c0b53e0 msgr2=0x7f8d8c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:42.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 -- 172.21.2.104:0/474121360 shutdown_connections 2021-07-11T04:01:42.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 -- 172.21.2.104:0/474121360 wait complete. 2021-07-11T04:01:42.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 Processor -- start 2021-07-11T04:01:42.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.432+0000 7f8d90a73700 1 -- start start 2021-07-11T04:01:42.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d90a73700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d8c0a76b0 0x7f8d8c14d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d90a73700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d8c0b0060 0x7f8d8c0a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d90a73700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d8c0b1970 0x7f8d8c0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d90a73700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0aef20 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d90a73700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0a9a60 con 0x7f8d8c0b1970 2021-07-11T04:01:42.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d90a73700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0ae540 con 0x7f8d8c0b0060 2021-07-11T04:01:42.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d8a59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d8c0a76b0 0x7f8d8c14d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:42.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d8a59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d8c0a76b0 0x7f8d8c14d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49928/0 (socket says 172.21.2.104:49928) 2021-07-11T04:01:42.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d8a59c700 1 -- 172.21.2.104:0/3111092658 learned_addr learned my addr 172.21.2.104:0/3111092658 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:42.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d8ad9d700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d8c0b1970 0x7f8d8c0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:42.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d8a59c700 1 -- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d8c0b0060 msgr2=0x7f8d8c0a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:42.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d8a59c700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d8c0b0060 0x7f8d8c0a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:42.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d8a59c700 1 -- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d8c0b1970 msgr2=0x7f8d8c0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:42.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d8a59c700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d8c0b1970 0x7f8d8c0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:42.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d8a59c700 1 -- 172.21.2.104:0/3111092658 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7400b1e0 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d8ad9d700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d8c0b1970 0x7f8d8c0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:42.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d8a59c700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d8c0a76b0 0x7f8d8c14d290 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f8d7c002fb0 tx=0x7f8d7c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:42.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d837fe700 1 -- 172.21.2.104:0/3111092658 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d7c013460 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d837fe700 1 -- 172.21.2.104:0/3111092658 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c0135c0 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d90a73700 1 -- 172.21.2.104:0/3111092658 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c0ae7e0 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d90a73700 1 -- 172.21.2.104:0/3111092658 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c0a9720 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d837fe700 1 -- 172.21.2.104:0/3111092658 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8d7c013920 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d90a73700 1 -- 172.21.2.104:0/3111092658 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c0af490 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d837fe700 1 -- 172.21.2.104:0/3111092658 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8d7c00f070 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d837fe700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d70053380 0x7f8d70055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d837fe700 1 -- 172.21.2.104:0/3111092658 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f8d7c06afe0 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d89d9b700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d70053380 0x7f8d70055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:42.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.436+0000 7f8d89d9b700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d70053380 0x7f8d70055840 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f8d8c0b6460 tx=0x7f8d74015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:42.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.440+0000 7f8d837fe700 1 -- 172.21.2.104:0/3111092658 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8d7c042220 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 -- 172.21.2.104:0/3111092658 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d8c057350 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d837fe700 1 -- 172.21.2.104:0/3111092658 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+1426 (secure 0 0 0) 0x7f8d7c045840 con 0x7f8d8c0a76b0 2021-07-11T04:01:42.547 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:42.548 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":63,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[{"mdsmap":{"epoch":63,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:40.544726+0000","modified":"2021-07-11T04:01:42.355913+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[],"up":{},"failed":[],"damaged":[],"stopped":[],"info":{},"data_pools":[22],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2021-07-11T04:01:42.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 -- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d70053380 msgr2=0x7f8d70055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:42.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d70053380 0x7f8d70055840 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f8d8c0b6460 tx=0x7f8d74015000).stop 2021-07-11T04:01:42.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 -- 172.21.2.104:0/3111092658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d8c0a76b0 msgr2=0x7f8d8c14d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:42.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d8c0a76b0 0x7f8d8c14d290 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f8d7c002fb0 tx=0x7f8d7c006df0).stop 2021-07-11T04:01:42.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 -- 172.21.2.104:0/3111092658 shutdown_connections 2021-07-11T04:01:42.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8d8c0b1970 0x7f8d8c0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:42.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8d8c0b0060 0x7f8d8c0a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:42.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8d70053380 0x7f8d70055840 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:42.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 --2- 172.21.2.104:0/3111092658 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8d8c0a76b0 0x7f8d8c14d290 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:42.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 -- 172.21.2.104:0/3111092658 >> 172.21.2.104:0/3111092658 conn(0x7f8d8c0b53e0 msgr2=0x7f8d8c148e50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:42.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 -- 172.21.2.104:0/3111092658 shutdown_connections 2021-07-11T04:01:42.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.548+0000 7f8d90a73700 1 -- 172.21.2.104:0/3111092658 wait complete. 2021-07-11T04:01:42.553 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 63 2021-07-11T04:01:42.555 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T04:01:42.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 Processor -- start 2021-07-11T04:01:42.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 -- start start 2021-07-11T04:01:42.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa74014ab80 0x7fa74005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa7400aff10 0x7fa74014af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa740061360 con 0x7fa7400ac9b0 2021-07-11T04:01:42.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74014b5e0 con 0x7fa7400aff10 2021-07-11T04:01:42.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74014b880 con 0x7fa74014ab80 2021-07-11T04:01:42.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa73e7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa7400aff10 0x7fa74014af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:42.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa73e7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa7400aff10 0x7fa74014af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49940/0 (socket says 172.21.2.104:49940) 2021-07-11T04:01:42.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa73e7fc700 1 -- 172.21.2.104:0/658169236 learned_addr learned my addr 172.21.2.104:0/658169236 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:42.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa73e7fc700 1 -- 172.21.2.104:0/658169236 >> v2:172.21.2.105:3300/0 conn(0x7fa74014ab80 msgr2=0x7fa74005f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:42.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa73e7fc700 1 --2- 172.21.2.104:0/658169236 >> v2:172.21.2.105:3300/0 conn(0x7fa74014ab80 0x7fa74005f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:42.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa73e7fc700 1 -- 172.21.2.104:0/658169236 >> v1:172.21.2.105:6789/0 conn(0x7fa7400ac9b0 legacy=0x7fa7400af1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:42.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa73e7fc700 1 -- 172.21.2.104:0/658169236 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7400aed20 con 0x7fa7400aff10 2021-07-11T04:01:42.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa73e7fc700 1 --2- 172.21.2.104:0/658169236 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7400aff10 0x7fa74014af60 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7fa740062060 tx=0x7fa728006eb0).ready entity=mon.0 client_cookie=397e4e8840b34b3b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:42.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa73dffb700 1 -- 172.21.2.104:0/658169236 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa72800d5c0 con 0x7fa7400aff10 2021-07-11T04:01:42.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa73dffb700 1 -- 172.21.2.104:0/658169236 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa728000e90 con 0x7fa7400aff10 2021-07-11T04:01:42.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa73dffb700 1 -- 172.21.2.104:0/658169236 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa72800d910 con 0x7fa7400aff10 2021-07-11T04:01:42.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 -- 172.21.2.104:0/658169236 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7400aff10 msgr2=0x7fa74014af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:42.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 --2- 172.21.2.104:0/658169236 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7400aff10 0x7fa74014af60 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7fa740062060 tx=0x7fa728006eb0).stop 2021-07-11T04:01:42.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 -- 172.21.2.104:0/658169236 shutdown_connections 2021-07-11T04:01:42.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 --2- 172.21.2.104:0/658169236 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7400aff10 0x7fa74014af60 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:42.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 --2- 172.21.2.104:0/658169236 >> v2:172.21.2.105:3300/0 conn(0x7fa74014ab80 0x7fa74005f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:42.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 -- 172.21.2.104:0/658169236 >> 172.21.2.104:0/658169236 conn(0x7fa7400b5570 msgr2=0x7fa7400b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:42.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 -- 172.21.2.104:0/658169236 shutdown_connections 2021-07-11T04:01:42.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.608+0000 7fa74593f700 1 -- 172.21.2.104:0/658169236 wait complete. 2021-07-11T04:01:42.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa74593f700 1 Processor -- start 2021-07-11T04:01:42.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa74593f700 1 -- start start 2021-07-11T04:01:42.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa74593f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7400ac9b0 0x7fa7400b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa74593f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7400aff10 0x7fa740159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa74593f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa74014ab80 0x7fa74015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa74593f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7400b5180 con 0x7fa7400aff10 2021-07-11T04:01:42.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa74593f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7400b49a0 con 0x7fa74014ab80 2021-07-11T04:01:42.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa74593f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7400b3ef0 con 0x7fa7400ac9b0 2021-07-11T04:01:42.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa73e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7400aff10 0x7fa740159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:42.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa73e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7400aff10 0x7fa740159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49942/0 (socket says 172.21.2.104:49942) 2021-07-11T04:01:42.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa73e7fc700 1 -- 172.21.2.104:0/3873565887 learned_addr learned my addr 172.21.2.104:0/3873565887 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:42.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa73e7fc700 1 -- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7400ac9b0 msgr2=0x7fa7400b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:42.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa73e7fc700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7400ac9b0 0x7fa7400b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:42.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa73e7fc700 1 -- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa74014ab80 msgr2=0x7fa74015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:42.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa73f7fe700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa74014ab80 0x7fa74015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:42.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa73e7fc700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa74014ab80 0x7fa74015a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:42.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa73e7fc700 1 -- 172.21.2.104:0/3873565887 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72800b1e0 con 0x7fa7400aff10 2021-07-11T04:01:42.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa73e7fc700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7400aff10 0x7fa740159cc0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7fa740062060 tx=0x7fa728015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:42.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa737fff700 1 -- 172.21.2.104:0/3873565887 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa728006be0 con 0x7fa7400aff10 2021-07-11T04:01:42.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa737fff700 1 -- 172.21.2.104:0/3873565887 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa728002f70 con 0x7fa7400aff10 2021-07-11T04:01:42.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa74593f700 1 -- 172.21.2.104:0/3873565887 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7400b4500 con 0x7fa7400aff10 2021-07-11T04:01:42.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa737fff700 1 -- 172.21.2.104:0/3873565887 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa72801d450 con 0x7fa7400aff10 2021-07-11T04:01:42.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa74593f700 1 -- 172.21.2.104:0/3873565887 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa740140f40 con 0x7fa7400aff10 2021-07-11T04:01:42.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa74593f700 1 -- 172.21.2.104:0/3873565887 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7400b40f0 con 0x7fa7400aff10 2021-07-11T04:01:42.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa737fff700 1 -- 172.21.2.104:0/3873565887 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa72800f070 con 0x7fa7400aff10 2021-07-11T04:01:42.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa737fff700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa724053300 0x7fa7240557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:42.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa737fff700 1 -- 172.21.2.104:0/3873565887 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fa72806ce60 con 0x7fa7400aff10 2021-07-11T04:01:42.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.612+0000 7fa73effd700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa724053300 0x7fa7240557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:42.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.616+0000 7fa73effd700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa724053300 0x7fa7240557c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fa730000ed0 tx=0x7fa730006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:42.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.616+0000 7fa737fff700 1 -- 172.21.2.104:0/3873565887 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa728044150 con 0x7fa7400aff10 2021-07-11T04:01:42.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:42.724+0000 7fa74593f700 1 -- 172.21.2.104:0/3873565887 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa7400b4690 con 0x7fa7400aff10 2021-07-11T04:01:42.993 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:01:43.003 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:01:43.013 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:01:43.031 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:01:43.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa737fff700 1 -- 172.21.2.104:0/3873565887 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa728047770 con 0x7fa7400aff10 2021-07-11T04:01:43.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 -- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa724053300 msgr2=0x7fa7240557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:43.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa724053300 0x7fa7240557c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fa730000ed0 tx=0x7fa730006930).stop 2021-07-11T04:01:43.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 -- 172.21.2.104:0/3873565887 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7400aff10 msgr2=0x7fa740159cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:43.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7400aff10 0x7fa740159cc0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7fa740062060 tx=0x7fa728015000).stop 2021-07-11T04:01:43.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 -- 172.21.2.104:0/3873565887 shutdown_connections 2021-07-11T04:01:43.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa74014ab80 0x7fa74015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:43.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa724053300 0x7fa7240557c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:43.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa7400aff10 0x7fa740159cc0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:43.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 --2- 172.21.2.104:0/3873565887 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa7400ac9b0 0x7fa7400b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:43.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 -- 172.21.2.104:0/3873565887 >> 172.21.2.104:0/3873565887 conn(0x7fa7400b5570 msgr2=0x7fa74014a270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:43.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 -- 172.21.2.104:0/3873565887 shutdown_connections 2021-07-11T04:01:43.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.576+0000 7fa74593f700 1 -- 172.21.2.104:0/3873565887 wait complete. 2021-07-11T04:01:43.583 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T04:01:43.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f8483d58700 1 Processor -- start 2021-07-11T04:01:43.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f8483d58700 1 -- start start 2021-07-11T04:01:43.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f8483d58700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f847c1483a0 0x7f847c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:43.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f8483d58700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f847c0a70c0 0x7f847c144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:43.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f8483d58700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c0b0350 con 0x7f847c0b1db0 2021-07-11T04:01:43.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f8483d58700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847c0a90c0 con 0x7f847c0a70c0 2021-07-11T04:01:43.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f8483d58700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0b05f0 con 0x7f847c1483a0 2021-07-11T04:01:43.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f84812f3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f847c1483a0 0x7f847c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:43.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f84822f5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f847c0a70c0 0x7f847c144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:43.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f84812f3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f847c1483a0 0x7f847c0b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:53118/0 (socket says 172.21.2.104:53118) 2021-07-11T04:01:43.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f84822f5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f847c0a70c0 0x7f847c144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:57512/0 (socket says 172.21.2.104:57512) 2021-07-11T04:01:43.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f84812f3700 1 -- 172.21.2.104:0/658110145 learned_addr learned my addr 172.21.2.104:0/658110145 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:43.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f8481af4700 1 --1- 172.21.2.104:0/658110145 >> v1:172.21.2.105:6789/0 conn(0x7f847c0b1db0 0x7f847c0b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:46088/0 (socket says 172.21.2.104:46088) 2021-07-11T04:01:43.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f84812f3700 1 -- 172.21.2.104:0/658110145 >> v2:172.21.2.105:3300/0 conn(0x7f847c0a70c0 msgr2=0x7f847c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:43.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f84812f3700 1 --2- 172.21.2.104:0/658110145 >> v2:172.21.2.105:3300/0 conn(0x7f847c0a70c0 0x7f847c144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:43.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f84812f3700 1 -- 172.21.2.104:0/658110145 >> v1:172.21.2.105:6789/0 conn(0x7f847c0b1db0 legacy=0x7f847c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:43.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f84812f3700 1 -- 172.21.2.104:0/658110145 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c0b0890 con 0x7f847c1483a0 2021-07-11T04:01:43.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.664+0000 7f84812f3700 1 --2- 172.21.2.104:0/658110145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f847c1483a0 0x7f847c0b0fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f846c002f70 tx=0x7f846c006eb0).ready entity=mon.2 client_cookie=b197d13605f3def7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:43.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8480af2700 1 -- 172.21.2.104:0/658110145 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846c00d570 con 0x7f847c1483a0 2021-07-11T04:01:43.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8480af2700 1 -- 172.21.2.104:0/658110145 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c000e90 con 0x7f847c1483a0 2021-07-11T04:01:43.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8480af2700 1 -- 172.21.2.104:0/658110145 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846c00d900 con 0x7f847c1483a0 2021-07-11T04:01:43.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 -- 172.21.2.104:0/658110145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f847c1483a0 msgr2=0x7f847c0b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:43.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 --2- 172.21.2.104:0/658110145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f847c1483a0 0x7f847c0b0fd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f846c002f70 tx=0x7f846c006eb0).stop 2021-07-11T04:01:43.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 -- 172.21.2.104:0/658110145 shutdown_connections 2021-07-11T04:01:43.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 --2- 172.21.2.104:0/658110145 >> v2:172.21.2.105:3300/0 conn(0x7f847c0a70c0 0x7f847c144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:43.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 --2- 172.21.2.104:0/658110145 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f847c1483a0 0x7f847c0b0fd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:43.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 -- 172.21.2.104:0/658110145 >> 172.21.2.104:0/658110145 conn(0x7f847c05f6b0 msgr2=0x7f847c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:43.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 -- 172.21.2.104:0/658110145 shutdown_connections 2021-07-11T04:01:43.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 -- 172.21.2.104:0/658110145 wait complete. 2021-07-11T04:01:43.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 Processor -- start 2021-07-11T04:01:43.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 -- start start 2021-07-11T04:01:43.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f847c0a70c0 0x7f847c145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:43.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f847c0b1db0 0x7f847c0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:43.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f847c1483a0 0x7f847c0acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:43.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0ad390 con 0x7f847c0b1db0 2021-07-11T04:01:43.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0ac900 con 0x7f847c1483a0 2021-07-11T04:01:43.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c0ad5b0 con 0x7f847c0a70c0 2021-07-11T04:01:43.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f84812f3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f847c0b1db0 0x7f847c0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:43.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f84812f3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f847c0b1db0 0x7f847c0aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49958/0 (socket says 172.21.2.104:49958) 2021-07-11T04:01:43.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f84812f3700 1 -- 172.21.2.104:0/1232807066 learned_addr learned my addr 172.21.2.104:0/1232807066 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:43.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f84812f3700 1 -- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f847c0a70c0 msgr2=0x7f847c145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:43.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f84812f3700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f847c0a70c0 0x7f847c145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:43.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f84812f3700 1 -- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f847c1483a0 msgr2=0x7f847c0acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:43.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f84812f3700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f847c1483a0 0x7f847c0acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:43.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f84812f3700 1 -- 172.21.2.104:0/1232807066 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c00b1e0 con 0x7f847c0b1db0 2021-07-11T04:01:43.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f84812f3700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f847c0b1db0 0x7f847c0aed40 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f846c002f70 tx=0x7f846c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:43.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f846affd700 1 -- 172.21.2.104:0/1232807066 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846c021070 con 0x7f847c0b1db0 2021-07-11T04:01:43.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f846affd700 1 -- 172.21.2.104:0/1232807066 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c000e90 con 0x7f847c0b1db0 2021-07-11T04:01:43.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f846affd700 1 -- 172.21.2.104:0/1232807066 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f846c0118d0 con 0x7f847c0b1db0 2021-07-11T04:01:43.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 -- 172.21.2.104:0/1232807066 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847c156b10 con 0x7f847c0b1db0 2021-07-11T04:01:43.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.668+0000 7f8483d58700 1 -- 172.21.2.104:0/1232807066 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847c0ad850 con 0x7f847c0b1db0 2021-07-11T04:01:43.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.672+0000 7f8483d58700 1 -- 172.21.2.104:0/1232807066 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847c057350 con 0x7f847c0b1db0 2021-07-11T04:01:43.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.672+0000 7f846affd700 1 -- 172.21.2.104:0/1232807066 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f846c00f070 con 0x7f847c0b1db0 2021-07-11T04:01:43.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.672+0000 7f846affd700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f845c053270 0x7f845c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:43.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.672+0000 7f846affd700 1 -- 172.21.2.104:0/1232807066 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f846c06c7b0 con 0x7f847c0b1db0 2021-07-11T04:01:43.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.672+0000 7f8481af4700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f845c053270 0x7f845c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:43.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.672+0000 7f8481af4700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f845c053270 0x7f845c055730 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f847c0613a0 tx=0x7f8478007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:43.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.672+0000 7f846affd700 1 -- 172.21.2.104:0/1232807066 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f846c0439d0 con 0x7f847c0b1db0 2021-07-11T04:01:43.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:43.756+0000 7f8483d58700 1 -- 172.21.2.104:0/1232807066 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f847c0a7e20 con 0x7f847c0b1db0 2021-07-11T04:01:44.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.584+0000 7f846affd700 1 -- 172.21.2.104:0/1232807066 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) v1 ==== 185+0+0 (secure 0 0 0) 0x7f846c046ff0 con 0x7f847c0b1db0 2021-07-11T04:01:44.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 -- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f845c053270 msgr2=0x7f845c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:44.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f845c053270 0x7f845c055730 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f847c0613a0 tx=0x7f8478007800).stop 2021-07-11T04:01:44.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 -- 172.21.2.104:0/1232807066 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f847c0b1db0 msgr2=0x7f847c0aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:44.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f847c0b1db0 0x7f847c0aed40 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f846c002f70 tx=0x7f846c015000).stop 2021-07-11T04:01:44.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 -- 172.21.2.104:0/1232807066 shutdown_connections 2021-07-11T04:01:44.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f847c1483a0 0x7f847c0acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:44.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f845c053270 0x7f845c055730 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:44.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f847c0b1db0 0x7f847c0aed40 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:44.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 --2- 172.21.2.104:0/1232807066 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f847c0a70c0 0x7f847c145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:44.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 -- 172.21.2.104:0/1232807066 >> 172.21.2.104:0/1232807066 conn(0x7f847c05f6b0 msgr2=0x7f847c0a7540 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:44.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 -- 172.21.2.104:0/1232807066 shutdown_connections 2021-07-11T04:01:44.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.588+0000 7f8483d58700 1 -- 172.21.2.104:0/1232807066 wait complete. 2021-07-11T04:01:44.591 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T04:01:44.595 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T04:01:44.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f1872eaf700 1 Processor -- start 2021-07-11T04:01:44.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f1872eaf700 1 -- start start 2021-07-11T04:01:44.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f1872eaf700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c0b1db0 0x7f186c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:44.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f1872eaf700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186c0b0350 con 0x7f186c0a70c0 2021-07-11T04:01:44.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f1872eaf700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186c0b05f0 con 0x7f186c1483a0 2021-07-11T04:01:44.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f1872eaf700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c0a90c0 con 0x7f186c0b1db0 2021-07-11T04:01:44.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f187144c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f186c0a70c0 0x7f186c144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46736/0 (socket says 172.21.2.104:46736) 2021-07-11T04:01:44.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f187144c700 1 -- 172.21.2.104:0/758323034 learned_addr learned my addr 172.21.2.104:0/758323034 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:44.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077197795 0 0) 0x7f186c0b0350 con 0x7f186c0a70c0 2021-07-11T04:01:44.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1858003260 con 0x7f186c0a70c0 2021-07-11T04:01:44.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f1870c4b700 1 --2- 172.21.2.104:0/758323034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c0b1db0 0x7f186c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:44.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2218124278 0 0) 0x7f1858003260 con 0x7f186c0a70c0 2021-07-11T04:01:44.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c0b1db0 msgr2=0x7f186c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:44.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.648+0000 7f186b7fe700 1 --2- 172.21.2.104:0/758323034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c0b1db0 0x7f186c0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:44.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 >> v1:172.21.2.105:6789/0 conn(0x7f186c1483a0 legacy=0x7f186c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:44.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c0b0890 con 0x7f186c0a70c0 2021-07-11T04:01:44.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f18640019e0 con 0x7f186c0a70c0 2021-07-11T04:01:44.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 >> v1:172.21.2.104:6789/0 conn(0x7f186c0a70c0 legacy=0x7f186c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:44.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 --2- 172.21.2.104:0/758323034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c0b1db0 0x7f1858003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:44.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 --2- 172.21.2.104:0/758323034 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1858003d40 0x7f1858004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:44.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 --2- 172.21.2.104:0/758323034 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f18580049f0 0x7f1858004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:44.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185800b850 con 0x7f18580049f0 2021-07-11T04:01:44.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185800b9c0 con 0x7f1858003d40 2021-07-11T04:01:44.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185800bb80 con 0x7f186c0b1db0 2021-07-11T04:01:44.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 --2- 172.21.2.104:0/758323034 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f18580049f0 0x7f1858004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:44.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 -- 172.21.2.104:0/758323034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c0b1db0 msgr2=0x7f1858003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:44.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 --2- 172.21.2.104:0/758323034 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c0b1db0 0x7f1858003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:44.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 -- 172.21.2.104:0/758323034 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1858003d40 msgr2=0x7f1858004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:44.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 --2- 172.21.2.104:0/758323034 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1858003d40 0x7f1858004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:44.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 -- 172.21.2.104:0/758323034 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1858003260 con 0x7f18580049f0 2021-07-11T04:01:44.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 --2- 172.21.2.104:0/758323034 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f18580049f0 0x7f1858004e70 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7f186c061440 tx=0x7f186c0b0780).ready entity=mon.0 client_cookie=b0647b9f68511cbb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:44.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f186400ccc0 con 0x7f18580049f0 2021-07-11T04:01:44.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186400ce20 con 0x7f18580049f0 2021-07-11T04:01:44.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f186b7fe700 1 -- 172.21.2.104:0/758323034 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1864005b80 con 0x7f18580049f0 2021-07-11T04:01:44.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 -- 172.21.2.104:0/758323034 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f18580049f0 msgr2=0x7f1858004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:44.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 --2- 172.21.2.104:0/758323034 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f18580049f0 0x7f1858004e70 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7f186c061440 tx=0x7f186c0b0780).stop 2021-07-11T04:01:44.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 -- 172.21.2.104:0/758323034 reap_dead start 2021-07-11T04:01:44.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 -- 172.21.2.104:0/758323034 shutdown_connections 2021-07-11T04:01:44.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 -- 172.21.2.104:0/758323034 >> 172.21.2.104:0/758323034 conn(0x7f186c05f6b0 msgr2=0x7f186c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:44.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 -- 172.21.2.104:0/758323034 shutdown_connections 2021-07-11T04:01:44.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 -- 172.21.2.104:0/758323034 wait complete. 2021-07-11T04:01:44.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 Processor -- start 2021-07-11T04:01:44.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 -- start start 2021-07-11T04:01:44.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f186c0ae170 0x7f186c0627c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:44.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c062cb0 0x7f186c063130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:44.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0a9440 0x7f186c0a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:44.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c0ab1a0 con 0x7f186c0a9440 2021-07-11T04:01:44.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c0ae550 con 0x7f186c0ae170 2021-07-11T04:01:44.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c0ac0b0 con 0x7f186c062cb0 2021-07-11T04:01:44.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0a9440 0x7f186c0a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:44.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0a9440 0x7f186c0a98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49980/0 (socket says 172.21.2.104:49980) 2021-07-11T04:01:44.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 -- 172.21.2.104:0/4128053045 learned_addr learned my addr 172.21.2.104:0/4128053045 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:44.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 -- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c062cb0 msgr2=0x7f186c063130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:44.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c062cb0 0x7f186c063130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:44.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 -- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f186c0ae170 msgr2=0x7f186c0627c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:44.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f186c0ae170 0x7f186c0627c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:44.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 -- 172.21.2.104:0/4128053045 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c0ab3d0 con 0x7f186c0a9440 2021-07-11T04:01:44.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f187144c700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0a9440 0x7f186c0a98f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f186c0b3170 tx=0x7f1864013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:44.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1869ffb700 1 -- 172.21.2.104:0/4128053045 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1864001fd0 con 0x7f186c0a9440 2021-07-11T04:01:44.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1869ffb700 1 -- 172.21.2.104:0/4128053045 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1864018030 con 0x7f186c0a9440 2021-07-11T04:01:44.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1869ffb700 1 -- 172.21.2.104:0/4128053045 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f18640074d0 con 0x7f186c0a9440 2021-07-11T04:01:44.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 -- 172.21.2.104:0/4128053045 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186c0ac350 con 0x7f186c0a9440 2021-07-11T04:01:44.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.652+0000 7f1872eaf700 1 -- 172.21.2.104:0/4128053045 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c067670 con 0x7f186c0a9440 2021-07-11T04:01:44.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.656+0000 7f1872eaf700 1 -- 172.21.2.104:0/4128053045 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186c0b5380 con 0x7f186c0a9440 2021-07-11T04:01:44.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.656+0000 7f1869ffb700 1 -- 172.21.2.104:0/4128053045 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1864026070 con 0x7f186c0a9440 2021-07-11T04:01:44.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.656+0000 7f1869ffb700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f184c0532b0 0x7f184c055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:44.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.656+0000 7f1869ffb700 1 -- 172.21.2.104:0/4128053045 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f186406b970 con 0x7f186c0a9440 2021-07-11T04:01:44.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.656+0000 7f1870c4b700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f184c0532b0 0x7f184c055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:44.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.656+0000 7f1870c4b700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f184c0532b0 0x7f184c055770 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f185c00a000 tx=0x7f185c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:44.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.656+0000 7f1869ffb700 1 -- 172.21.2.104:0/4128053045 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1864042d00 con 0x7f186c0a9440 2021-07-11T04:01:44.738 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:44.740+0000 7f1872eaf700 1 -- 172.21.2.104:0/4128053045 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f186c0aaea0 con 0x7f186c0a9440 2021-07-11T04:01:45.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1869ffb700 1 -- 172.21.2.104:0/4128053045 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) v1 ==== 173+0+0 (secure 0 0 0) 0x7f1864046320 con 0x7f186c0a9440 2021-07-11T04:01:45.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 -- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f184c0532b0 msgr2=0x7f184c055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:45.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f184c0532b0 0x7f184c055770 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f185c00a000 tx=0x7f185c008040).stop 2021-07-11T04:01:45.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 -- 172.21.2.104:0/4128053045 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0a9440 msgr2=0x7f186c0a98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:45.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0a9440 0x7f186c0a98f0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f186c0b3170 tx=0x7f1864013000).stop 2021-07-11T04:01:45.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 -- 172.21.2.104:0/4128053045 shutdown_connections 2021-07-11T04:01:45.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f184c0532b0 0x7f184c055770 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:45.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f186c0a9440 0x7f186c0a98f0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:45.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f186c062cb0 0x7f186c063130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:45.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 --2- 172.21.2.104:0/4128053045 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f186c0ae170 0x7f186c0627c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:45.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 -- 172.21.2.104:0/4128053045 >> 172.21.2.104:0/4128053045 conn(0x7f186c05f6b0 msgr2=0x7f186c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:45.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 -- 172.21.2.104:0/4128053045 shutdown_connections 2021-07-11T04:01:45.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.596+0000 7f1872eaf700 1 -- 172.21.2.104:0/4128053045 wait complete. 2021-07-11T04:01:45.601 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T04:01:45.603 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-07-11T04:01:45.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48d74c3700 1 Processor -- start 2021-07-11T04:01:45.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48d74c3700 1 -- start start 2021-07-11T04:01:45.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48d74c3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d00b1ec0 0x7f48d00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48d74c3700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d00a77d0 con 0x7f48d00affb0 2021-07-11T04:01:45.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48d74c3700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d00b03f0 con 0x7f48d00b22a0 2021-07-11T04:01:45.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48d74c3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00b0650 con 0x7f48d00b1ec0 2021-07-11T04:01:45.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48d4a5e700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f48d00affb0 0x7f48d00a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46756/0 (socket says 172.21.2.104:46756) 2021-07-11T04:01:45.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48d4a5e700 1 -- 172.21.2.104:0/695418011 learned_addr learned my addr 172.21.2.104:0/695418011 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:45.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813631976 0 0) 0x7f48d00a77d0 con 0x7f48d00affb0 2021-07-11T04:01:45.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b8003260 con 0x7f48d00affb0 2021-07-11T04:01:45.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3711996500 0 0) 0x7f48b8003260 con 0x7f48d00affb0 2021-07-11T04:01:45.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d00b1ec0 msgr2=0x7f48d00b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:45.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d00b1ec0 0x7f48d00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:45.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 >> v1:172.21.2.105:6789/0 conn(0x7f48d00b22a0 legacy=0x7f48d0141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:45.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d00b08f0 con 0x7f48d00affb0 2021-07-11T04:01:45.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f48c0001660 con 0x7f48d00affb0 2021-07-11T04:01:45.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 >> v1:172.21.2.104:6789/0 conn(0x7f48d00affb0 legacy=0x7f48d00a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:45.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d00b1ec0 0x7f48b8003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48b8003ef0 0x7f48b8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48b8004bf0 0x7f48b800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00b0650 con 0x7f48b8004bf0 2021-07-11T04:01:45.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b80052f0 con 0x7f48b8003ef0 2021-07-11T04:01:45.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b800bc40 con 0x7f48d00b1ec0 2021-07-11T04:01:45.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.656+0000 7f48d5a60700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48b8004bf0 0x7f48b800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:45.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d4a5e700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48b8003ef0 0x7f48b8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:45.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d525f700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d00b1ec0 0x7f48b8003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:45.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d5a60700 1 -- 172.21.2.104:0/695418011 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d00b1ec0 msgr2=0x7f48b8003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:45.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d5a60700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d00b1ec0 0x7f48b8003800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:45.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d5a60700 1 -- 172.21.2.104:0/695418011 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48b8003ef0 msgr2=0x7f48b8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:45.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d5a60700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48b8003ef0 0x7f48b8004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:45.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d5a60700 1 -- 172.21.2.104:0/695418011 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b8003260 con 0x7f48b8004bf0 2021-07-11T04:01:45.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d4a5e700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48b8003ef0 0x7f48b8004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:01:45.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d5a60700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48b8004bf0 0x7f48b800b750 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f48c8002320 tx=0x7f48c8007d80).ready entity=mon.0 client_cookie=8021b87b4b289387 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:45.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48c8002510 con 0x7f48b8004bf0 2021-07-11T04:01:45.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c8006890 con 0x7f48b8004bf0 2021-07-11T04:01:45.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48bffff700 1 -- 172.21.2.104:0/695418011 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48c8006bf0 con 0x7f48b8004bf0 2021-07-11T04:01:45.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 -- 172.21.2.104:0/695418011 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48b8004bf0 msgr2=0x7f48b800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:45.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 --2- 172.21.2.104:0/695418011 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48b8004bf0 0x7f48b800b750 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f48c8002320 tx=0x7f48c8007d80).stop 2021-07-11T04:01:45.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d5a60700 1 -- 172.21.2.104:0/695418011 reap_dead start 2021-07-11T04:01:45.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 -- 172.21.2.104:0/695418011 shutdown_connections 2021-07-11T04:01:45.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 -- 172.21.2.104:0/695418011 >> 172.21.2.104:0/695418011 conn(0x7f48d005f6b0 msgr2=0x7f48d005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:45.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 -- 172.21.2.104:0/695418011 shutdown_connections 2021-07-11T04:01:45.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 -- 172.21.2.104:0/695418011 wait complete. 2021-07-11T04:01:45.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 Processor -- start 2021-07-11T04:01:45.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 -- start start 2021-07-11T04:01:45.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48d00a95c0 0x7f48d01574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48d0157990 0x7f48d015a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d015a770 0x7f48d015ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d00aa680 con 0x7f48d0157990 2021-07-11T04:01:45.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d015b110 con 0x7f48d00a95c0 2021-07-11T04:01:45.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d014b490 con 0x7f48d015a770 2021-07-11T04:01:45.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d4a5e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48d0157990 0x7f48d015a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:45.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d4a5e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48d0157990 0x7f48d015a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:49998/0 (socket says 172.21.2.104:49998) 2021-07-11T04:01:45.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d4a5e700 1 -- 172.21.2.104:0/942777481 learned_addr learned my addr 172.21.2.104:0/942777481 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:45.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d525f700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48d00a95c0 0x7f48d01574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:45.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d5a60700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d015a770 0x7f48d015ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:45.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d4a5e700 1 -- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d015a770 msgr2=0x7f48d015ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:45.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d4a5e700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d015a770 0x7f48d015ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:45.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d4a5e700 1 -- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48d00a95c0 msgr2=0x7f48d01574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:45.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d4a5e700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48d00a95c0 0x7f48d01574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:45.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d4a5e700 1 -- 172.21.2.104:0/942777481 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d014b6c0 con 0x7f48d0157990 2021-07-11T04:01:45.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d525f700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48d00a95c0 0x7f48d01574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:45.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d4a5e700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48d0157990 0x7f48d015a280 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f48d015b250 tx=0x7f48d00b2a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:45.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48be7fc700 1 -- 172.21.2.104:0/942777481 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48c0010680 con 0x7f48d0157990 2021-07-11T04:01:45.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48be7fc700 1 -- 172.21.2.104:0/942777481 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c0010040 con 0x7f48d0157990 2021-07-11T04:01:45.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 -- 172.21.2.104:0/942777481 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d0156d30 con 0x7f48d0157990 2021-07-11T04:01:45.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48be7fc700 1 -- 172.21.2.104:0/942777481 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f48c00109d0 con 0x7f48d0157990 2021-07-11T04:01:45.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.660+0000 7f48d74c3700 1 -- 172.21.2.104:0/942777481 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d0148f50 con 0x7f48d0157990 2021-07-11T04:01:45.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.664+0000 7f48d74c3700 1 -- 172.21.2.104:0/942777481 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d0057350 con 0x7f48d0157990 2021-07-11T04:01:45.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.664+0000 7f48be7fc700 1 -- 172.21.2.104:0/942777481 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f48c001b020 con 0x7f48d0157990 2021-07-11T04:01:45.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.664+0000 7f48be7fc700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48b00532b0 0x7f48b0055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.664+0000 7f48be7fc700 1 -- 172.21.2.104:0/942777481 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f48c006abe0 con 0x7f48d0157990 2021-07-11T04:01:45.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.664+0000 7f48d525f700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48b00532b0 0x7f48b0055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:45.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.664+0000 7f48d525f700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48b00532b0 0x7f48b0055770 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f48cc008000 tx=0x7f48cc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:45.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.664+0000 7f48be7fc700 1 -- 172.21.2.104:0/942777481 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f48c0042200 con 0x7f48d0157990 2021-07-11T04:01:45.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.748+0000 7f48d74c3700 1 -- 172.21.2.104:0/942777481 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f48d01571d0 con 0x7f48d0157990 2021-07-11T04:01:45.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48be7fc700 1 -- 172.21.2.104:0/942777481 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f48c0045820 con 0x7f48d0157990 2021-07-11T04:01:45.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 -- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48b00532b0 msgr2=0x7f48b0055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:45.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48b00532b0 0x7f48b0055770 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f48cc008000 tx=0x7f48cc008040).stop 2021-07-11T04:01:45.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 -- 172.21.2.104:0/942777481 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48d0157990 msgr2=0x7f48d015a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:45.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48d0157990 0x7f48d015a280 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f48d015b250 tx=0x7f48d00b2a00).stop 2021-07-11T04:01:45.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 -- 172.21.2.104:0/942777481 shutdown_connections 2021-07-11T04:01:45.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48d015a770 0x7f48d015ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:45.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48b00532b0 0x7f48b0055770 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:45.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48d0157990 0x7f48d015a280 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:45.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 --2- 172.21.2.104:0/942777481 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48d00a95c0 0x7f48d01574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:45.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 -- 172.21.2.104:0/942777481 >> 172.21.2.104:0/942777481 conn(0x7f48d005f6b0 msgr2=0x7f48d014e220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:45.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 -- 172.21.2.104:0/942777481 shutdown_connections 2021-07-11T04:01:45.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.756+0000 7f48d74c3700 1 -- 172.21.2.104:0/942777481 wait complete. 2021-07-11T04:01:45.762 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new' 2021-07-11T04:01:45.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830ac7d700 1 Processor -- start 2021-07-11T04:01:45.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830ac7d700 1 -- start start 2021-07-11T04:01:45.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830ac7d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f83040b1db0 0x7f83040b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830ac7d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83041483a0 0x7f83040b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830ac7d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f83040a70c0 0x7f8304144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830ac7d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83040a90c0 con 0x7f83040a70c0 2021-07-11T04:01:45.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830ac7d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83040b0350 con 0x7f83040b1db0 2021-07-11T04:01:45.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830ac7d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83040b05f0 con 0x7f83041483a0 2021-07-11T04:01:45.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830921a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f83040a70c0 0x7f8304144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:45.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830921a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f83040a70c0 0x7f8304144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50004/0 (socket says 172.21.2.104:50004) 2021-07-11T04:01:45.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830921a700 1 -- 172.21.2.104:0/1315257302 learned_addr learned my addr 172.21.2.104:0/1315257302 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:45.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830921a700 1 -- 172.21.2.104:0/1315257302 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83041483a0 msgr2=0x7f83040b2190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:45.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830921a700 1 --2- 172.21.2.104:0/1315257302 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83041483a0 0x7f83040b2190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:45.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830921a700 1 -- 172.21.2.104:0/1315257302 >> v2:172.21.2.105:3300/0 conn(0x7f83040b1db0 msgr2=0x7f83040b0fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:45.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830921a700 1 --2- 172.21.2.104:0/1315257302 >> v2:172.21.2.105:3300/0 conn(0x7f83040b1db0 0x7f83040b0fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:45.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.812+0000 7f830921a700 1 -- 172.21.2.104:0/1315257302 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83040b0890 con 0x7f83040a70c0 2021-07-11T04:01:45.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830921a700 1 --2- 172.21.2.104:0/1315257302 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83040a70c0 0x7f8304144720 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f82fc0032f0 tx=0x7f82fc009290).ready entity=mon.0 client_cookie=75be5a4371386308 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:45.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f83037fe700 1 -- 172.21.2.104:0/1315257302 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82fc00b870 con 0x7f83040a70c0 2021-07-11T04:01:45.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f83037fe700 1 -- 172.21.2.104:0/1315257302 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82fc00b9d0 con 0x7f83040a70c0 2021-07-11T04:01:45.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f83037fe700 1 -- 172.21.2.104:0/1315257302 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82fc00bb80 con 0x7f83040a70c0 2021-07-11T04:01:45.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- 172.21.2.104:0/1315257302 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83040a70c0 msgr2=0x7f8304144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:45.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 --2- 172.21.2.104:0/1315257302 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83040a70c0 0x7f8304144720 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f82fc0032f0 tx=0x7f82fc009290).stop 2021-07-11T04:01:45.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- 172.21.2.104:0/1315257302 shutdown_connections 2021-07-11T04:01:45.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 --2- 172.21.2.104:0/1315257302 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83040a70c0 0x7f8304144720 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:45.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 --2- 172.21.2.104:0/1315257302 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83041483a0 0x7f83040b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:45.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 --2- 172.21.2.104:0/1315257302 >> v2:172.21.2.105:3300/0 conn(0x7f83040b1db0 0x7f83040b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:45.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- 172.21.2.104:0/1315257302 >> 172.21.2.104:0/1315257302 conn(0x7f830405f6b0 msgr2=0x7f830405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:45.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- 172.21.2.104:0/1315257302 shutdown_connections 2021-07-11T04:01:45.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- 172.21.2.104:0/1315257302 wait complete. 2021-07-11T04:01:45.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 Processor -- start 2021-07-11T04:01:45.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- start start 2021-07-11T04:01:45.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83040a70c0 0x7f83040668e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83040b1db0 0x7f83040aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83041483a0 0x7f8304144a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8304157630 con 0x7f83040b1db0 2021-07-11T04:01:45.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8304065a50 con 0x7f83040a70c0 2021-07-11T04:01:45.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8304064350 con 0x7f83041483a0 2021-07-11T04:01:45.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8303fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83040b1db0 0x7f83040aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:45.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8303fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83040b1db0 0x7f83040aba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50010/0 (socket says 172.21.2.104:50010) 2021-07-11T04:01:45.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8303fff700 1 -- 172.21.2.104:0/444934300 learned_addr learned my addr 172.21.2.104:0/444934300 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:45.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8303fff700 1 -- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83041483a0 msgr2=0x7f8304144a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:45.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8303fff700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83041483a0 0x7f8304144a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:45.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8303fff700 1 -- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83040a70c0 msgr2=0x7f83040668e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:45.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8303fff700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83040a70c0 0x7f83040668e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:45.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8303fff700 1 -- 172.21.2.104:0/444934300 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc009530 con 0x7f83040b1db0 2021-07-11T04:01:45.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8303fff700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83040b1db0 0x7f83040aba60 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f82f4000ed0 tx=0x7f82f4006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:45.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8301ffb700 1 -- 172.21.2.104:0/444934300 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82f4009e00 con 0x7f83040b1db0 2021-07-11T04:01:45.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8301ffb700 1 -- 172.21.2.104:0/444934300 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f4012410 con 0x7f83040b1db0 2021-07-11T04:01:45.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- 172.21.2.104:0/444934300 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83040645f0 con 0x7f83040b1db0 2021-07-11T04:01:45.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8301ffb700 1 -- 172.21.2.104:0/444934300 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82f4012770 con 0x7f83040b1db0 2021-07-11T04:01:45.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- 172.21.2.104:0/444934300 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83040657c0 con 0x7f83040b1db0 2021-07-11T04:01:45.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f830ac7d700 1 -- 172.21.2.104:0/444934300 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8304057350 con 0x7f83040b1db0 2021-07-11T04:01:45.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8301ffb700 1 -- 172.21.2.104:0/444934300 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f82f400f030 con 0x7f83040b1db0 2021-07-11T04:01:45.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8301ffb700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82e80531d0 0x7f82e8055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:45.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.816+0000 7f8301ffb700 1 -- 172.21.2.104:0/444934300 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f82f406ad90 con 0x7f83040b1db0 2021-07-11T04:01:45.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.820+0000 7f8308a19700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82e80531d0 0x7f82e8055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:45.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.820+0000 7f8308a19700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82e80531d0 0x7f82e8055690 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f82ec002fb0 tx=0x7f82ec006950).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:45.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.820+0000 7f8301ffb700 1 -- 172.21.2.104:0/444934300 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f82f40423b0 con 0x7f83040b1db0 2021-07-11T04:01:45.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:45.932+0000 7f830ac7d700 1 -- 172.21.2.104:0/444934300 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]} v 0) v1 -- 0x7f8304044650 con 0x7f83040b1db0 2021-07-11T04:01:46.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f8301ffb700 1 -- 172.21.2.104:0/444934300 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_fs_new"]}]=0 v366) v1 ==== 122+0+0 (secure 0 0 0) 0x7f82f40459d0 con 0x7f83040b1db0 2021-07-11T04:01:46.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 -- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82e80531d0 msgr2=0x7f82e8055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:46.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82e80531d0 0x7f82e8055690 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f82ec002fb0 tx=0x7f82ec006950).stop 2021-07-11T04:01:46.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 -- 172.21.2.104:0/444934300 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83040b1db0 msgr2=0x7f83040aba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:46.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83040b1db0 0x7f83040aba60 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f82f4000ed0 tx=0x7f82f4006d60).stop 2021-07-11T04:01:46.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 -- 172.21.2.104:0/444934300 shutdown_connections 2021-07-11T04:01:46.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f83041483a0 0x7f8304144a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:46.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f82e80531d0 0x7f82e8055690 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:46.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f83040b1db0 0x7f83040aba60 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:46.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 --2- 172.21.2.104:0/444934300 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f83040a70c0 0x7f83040668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:46.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 -- 172.21.2.104:0/444934300 >> 172.21.2.104:0/444934300 conn(0x7f830405f6b0 msgr2=0x7f83040b3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:46.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 -- 172.21.2.104:0/444934300 shutdown_connections 2021-07-11T04:01:46.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.604+0000 7f830ac7d700 1 -- 172.21.2.104:0/444934300 wait complete. 2021-07-11T04:01:46.612 INFO:tasks.cephfs_test_runner:test_fs_new (tasks.cephfs.test_misc.TestMisc) ... ok 2021-07-11T04:01:46.613 INFO:tasks.cephfs_test_runner:Starting test: test_getattr_caps (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:01:46.613 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2021-07-11T04:01:46.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.664+0000 7f45ffd33700 1 Processor -- start 2021-07-11T04:01:46.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.664+0000 7f45ffd33700 1 -- start start 2021-07-11T04:01:46.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f45f8060910 0x7f45f80acd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:46.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f80b6ce0 con 0x7f45f80aa5d0 2021-07-11T04:01:46.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f80611e0 con 0x7f45f80b6900 2021-07-11T04:01:46.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f805cba0 con 0x7f45f8060910 2021-07-11T04:01:46.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fd2ce700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f45f8060910 0x7f45f80acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:46.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fd2ce700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f45f8060910 0x7f45f80acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50020/0 (socket says 172.21.2.104:50020) 2021-07-11T04:01:46.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fd2ce700 1 -- 172.21.2.104:0/1784113051 learned_addr learned my addr 172.21.2.104:0/1784113051 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:46.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fd2ce700 1 -- 172.21.2.104:0/1784113051 >> v1:172.21.2.105:6789/0 conn(0x7f45f80b6900 legacy=0x7f45f80a9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:46.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fd2ce700 1 -- 172.21.2.104:0/1784113051 >> v1:172.21.2.104:6789/0 conn(0x7f45f80aa5d0 legacy=0x7f45f805c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:46.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fd2ce700 1 -- 172.21.2.104:0/1784113051 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f8061480 con 0x7f45f8060910 2021-07-11T04:01:46.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fd2ce700 1 --2- 172.21.2.104:0/1784113051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45f8060910 0x7f45f80acd50 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f45f805dc30 tx=0x7f45e8006eb0).ready entity=mon.0 client_cookie=cf7af30963eb828f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:46.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fcacd700 1 -- 172.21.2.104:0/1784113051 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45e800d610 con 0x7f45f8060910 2021-07-11T04:01:46.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fcacd700 1 -- 172.21.2.104:0/1784113051 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e8000e90 con 0x7f45f8060910 2021-07-11T04:01:46.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fcacd700 1 -- 172.21.2.104:0/1784113051 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45e800d960 con 0x7f45f8060910 2021-07-11T04:01:46.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- 172.21.2.104:0/1784113051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45f8060910 msgr2=0x7f45f80acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:46.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 --2- 172.21.2.104:0/1784113051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45f8060910 0x7f45f80acd50 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f45f805dc30 tx=0x7f45e8006eb0).stop 2021-07-11T04:01:46.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- 172.21.2.104:0/1784113051 shutdown_connections 2021-07-11T04:01:46.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 --2- 172.21.2.104:0/1784113051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45f8060910 0x7f45f80acd50 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:46.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- 172.21.2.104:0/1784113051 >> 172.21.2.104:0/1784113051 conn(0x7f45f80b3480 msgr2=0x7f45f80b5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:46.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- 172.21.2.104:0/1784113051 shutdown_connections 2021-07-11T04:01:46.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- 172.21.2.104:0/1784113051 wait complete. 2021-07-11T04:01:46.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 Processor -- start 2021-07-11T04:01:46.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- start start 2021-07-11T04:01:46.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45f8060910 0x7f45f80b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:46.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45f80aa5d0 0x7f45f813b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:46.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f45f80b6900 0x7f45f813ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:46.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f813f540 con 0x7f45f8060910 2021-07-11T04:01:46.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f814dd60 con 0x7f45f80b6900 2021-07-11T04:01:46.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45ffd33700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f813eb60 con 0x7f45f80aa5d0 2021-07-11T04:01:46.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fdacf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45f8060910 0x7f45f80b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:46.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fdacf700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45f8060910 0x7f45f80b1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50028/0 (socket says 172.21.2.104:50028) 2021-07-11T04:01:46.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.668+0000 7f45fdacf700 1 -- 172.21.2.104:0/3149076008 learned_addr learned my addr 172.21.2.104:0/3149076008 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:46.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45fdacf700 1 -- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45f80aa5d0 msgr2=0x7f45f813b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:46.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45fdacf700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45f80aa5d0 0x7f45f813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:46.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45fdacf700 1 -- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f45f80b6900 msgr2=0x7f45f813ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:46.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45fdacf700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f45f80b6900 0x7f45f813ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:46.678 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45fdacf700 1 -- 172.21.2.104:0/3149076008 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e800b1e0 con 0x7f45f8060910 2021-07-11T04:01:46.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45fdacf700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45f8060910 0x7f45f80b1c30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f45f4002ef0 tx=0x7f45f4006d50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:46.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45eeffd700 1 -- 172.21.2.104:0/3149076008 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45f4012410 con 0x7f45f8060910 2021-07-11T04:01:46.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45eeffd700 1 -- 172.21.2.104:0/3149076008 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f4012570 con 0x7f45f8060910 2021-07-11T04:01:46.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45eeffd700 1 -- 172.21.2.104:0/3149076008 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f45f40128d0 con 0x7f45f8060910 2021-07-11T04:01:46.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45ffd33700 1 -- 172.21.2.104:0/3149076008 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f813ee00 con 0x7f45f8060910 2021-07-11T04:01:46.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45ffd33700 1 -- 172.21.2.104:0/3149076008 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f80a9750 con 0x7f45f8060910 2021-07-11T04:01:46.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45ffd33700 1 -- 172.21.2.104:0/3149076008 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f814dad0 con 0x7f45f8060910 2021-07-11T04:01:46.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45eeffd700 1 -- 172.21.2.104:0/3149076008 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f45f400f030 con 0x7f45f8060910 2021-07-11T04:01:46.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45eeffd700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f45e0053330 0x7f45e00557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:46.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45eeffd700 1 -- 172.21.2.104:0/3149076008 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f45f406af80 con 0x7f45f8060910 2021-07-11T04:01:46.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45fd2ce700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f45e0053330 0x7f45e00557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:46.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.672+0000 7f45fd2ce700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f45e0053330 0x7f45e00557f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f45f805dc30 tx=0x7f45e8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:46.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.676+0000 7f45eeffd700 1 -- 172.21.2.104:0/3149076008 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f45f4042490 con 0x7f45f8060910 2021-07-11T04:01:46.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:46.788+0000 7f45ffd33700 1 -- 172.21.2.104:0/3149076008 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f45f8057350 con 0x7f45f8060910 2021-07-11T04:01:47.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.608+0000 7f45eeffd700 1 -- 172.21.2.104:0/3149076008 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v367) v1 ==== 131+0+0 (secure 0 0 0) 0x7f45f4045ab0 con 0x7f45f8060910 2021-07-11T04:01:47.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 -- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f45e0053330 msgr2=0x7f45e00557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f45e0053330 0x7f45e00557f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f45f805dc30 tx=0x7f45e8015000).stop 2021-07-11T04:01:47.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 -- 172.21.2.104:0/3149076008 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45f8060910 msgr2=0x7f45f80b1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45f8060910 0x7f45f80b1c30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f45f4002ef0 tx=0x7f45f4006d50).stop 2021-07-11T04:01:47.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 -- 172.21.2.104:0/3149076008 shutdown_connections 2021-07-11T04:01:47.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f45f80b6900 0x7f45f813ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:47.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45f80aa5d0 0x7f45f813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:47.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f45e0053330 0x7f45e00557f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:47.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 --2- 172.21.2.104:0/3149076008 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45f8060910 0x7f45f80b1c30 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:47.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 -- 172.21.2.104:0/3149076008 >> 172.21.2.104:0/3149076008 conn(0x7f45f80b3480 msgr2=0x7f45f805c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:47.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 -- 172.21.2.104:0/3149076008 shutdown_connections 2021-07-11T04:01:47.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.612+0000 7f45ffd33700 1 -- 172.21.2.104:0/3149076008 wait complete. 2021-07-11T04:01:47.620 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T04:01:47.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.672+0000 7f35ebe7d700 1 Processor -- start 2021-07-11T04:01:47.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.672+0000 7f35ebe7d700 1 -- start start 2021-07-11T04:01:47.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f35e40b1ec0 0x7f35e40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35e40b22a0 0x7f35e4141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e40b0420 con 0x7f35e40affb0 2021-07-11T04:01:47.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e40a77d0 con 0x7f35e40b1ec0 2021-07-11T04:01:47.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40b0650 con 0x7f35e40b22a0 2021-07-11T04:01:47.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e9c19700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f35e40b1ec0 0x7f35e40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:47.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e9418700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f35e40affb0 0x7f35e40a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46800/0 (socket says 172.21.2.104:46800) 2021-07-11T04:01:47.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e9418700 1 -- 172.21.2.104:0/3184954377 learned_addr learned my addr 172.21.2.104:0/3184954377 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:47.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e9c19700 1 -- 172.21.2.104:0/3184954377 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35e40b22a0 msgr2=0x7f35e4141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:47.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e9c19700 1 --2- 172.21.2.104:0/3184954377 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35e40b22a0 0x7f35e4141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:47.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e9c19700 1 -- 172.21.2.104:0/3184954377 >> v1:172.21.2.104:6789/0 conn(0x7f35e40affb0 legacy=0x7f35e40a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e9c19700 1 -- 172.21.2.104:0/3184954377 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e40b08f0 con 0x7f35e40b1ec0 2021-07-11T04:01:47.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e9c19700 1 --2- 172.21.2.104:0/3184954377 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35e40b1ec0 0x7f35e40b12d0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f35e0002f70 tx=0x7f35e0006df0).ready entity=mon.0 client_cookie=a0027852844bb0ab server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:47.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e8c17700 1 -- 172.21.2.104:0/3184954377 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35e000d570 con 0x7f35e40b1ec0 2021-07-11T04:01:47.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e8c17700 1 -- 172.21.2.104:0/3184954377 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e0000e90 con 0x7f35e40b1ec0 2021-07-11T04:01:47.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e8c17700 1 -- 172.21.2.104:0/3184954377 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35e000d8c0 con 0x7f35e40b1ec0 2021-07-11T04:01:47.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- 172.21.2.104:0/3184954377 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35e40b1ec0 msgr2=0x7f35e40b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 --2- 172.21.2.104:0/3184954377 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35e40b1ec0 0x7f35e40b12d0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f35e0002f70 tx=0x7f35e0006df0).stop 2021-07-11T04:01:47.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- 172.21.2.104:0/3184954377 shutdown_connections 2021-07-11T04:01:47.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 --2- 172.21.2.104:0/3184954377 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35e40b22a0 0x7f35e4141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:47.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 --2- 172.21.2.104:0/3184954377 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35e40b1ec0 0x7f35e40b12d0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:47.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- 172.21.2.104:0/3184954377 >> 172.21.2.104:0/3184954377 conn(0x7f35e405f6b0 msgr2=0x7f35e405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:47.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- 172.21.2.104:0/3184954377 shutdown_connections 2021-07-11T04:01:47.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- 172.21.2.104:0/3184954377 wait complete. 2021-07-11T04:01:47.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 Processor -- start 2021-07-11T04:01:47.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- start start 2021-07-11T04:01:47.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35e40affb0 0x7f35e413fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35e40b1ec0 0x7f35e414b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35e40b22a0 0x7f35e4156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e414abd0 con 0x7f35e40b1ec0 2021-07-11T04:01:47.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e414bb20 con 0x7f35e40b22a0 2021-07-11T04:01:47.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35ebe7d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e414a930 con 0x7f35e40affb0 2021-07-11T04:01:47.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e9418700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35e40b1ec0 0x7f35e414b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:47.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e9418700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35e40b1ec0 0x7f35e414b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50040/0 (socket says 172.21.2.104:50040) 2021-07-11T04:01:47.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.676+0000 7f35e9418700 1 -- 172.21.2.104:0/1976391633 learned_addr learned my addr 172.21.2.104:0/1976391633 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:47.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35e9418700 1 -- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35e40affb0 msgr2=0x7f35e413fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35e9418700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35e40affb0 0x7f35e413fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:47.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35e9418700 1 -- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35e40b22a0 msgr2=0x7f35e4156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35e9418700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35e40b22a0 0x7f35e4156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:47.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35e9418700 1 -- 172.21.2.104:0/1976391633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e000b180 con 0x7f35e40b1ec0 2021-07-11T04:01:47.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35e9418700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35e40b1ec0 0x7f35e414b630 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f35e40b2f50 tx=0x7f35d4007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:47.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35d2ffd700 1 -- 172.21.2.104:0/1976391633 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d400ce30 con 0x7f35e40b1ec0 2021-07-11T04:01:47.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35d2ffd700 1 -- 172.21.2.104:0/1976391633 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d40072e0 con 0x7f35e40b1ec0 2021-07-11T04:01:47.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35d2ffd700 1 -- 172.21.2.104:0/1976391633 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f35d4005800 con 0x7f35e40b1ec0 2021-07-11T04:01:47.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35ebe7d700 1 -- 172.21.2.104:0/1976391633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e414aef0 con 0x7f35e40b1ec0 2021-07-11T04:01:47.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35ebe7d700 1 -- 172.21.2.104:0/1976391633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e414a050 con 0x7f35e40b1ec0 2021-07-11T04:01:47.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35ebe7d700 1 -- 172.21.2.104:0/1976391633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e41571a0 con 0x7f35e40b1ec0 2021-07-11T04:01:47.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35d2ffd700 1 -- 172.21.2.104:0/1976391633 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f35d4010070 con 0x7f35e40b1ec0 2021-07-11T04:01:47.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35d2ffd700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35c4053240 0x7f35c4055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35d2ffd700 1 -- 172.21.2.104:0/1976391633 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f35d406a970 con 0x7f35e40b1ec0 2021-07-11T04:01:47.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35e9c19700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35c4053240 0x7f35c4055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:47.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.680+0000 7f35e9c19700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35c4053240 0x7f35c4055700 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f35e0002f70 tx=0x7f35e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:47.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.684+0000 7f35d2ffd700 1 -- 172.21.2.104:0/1976391633 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f35d4041e80 con 0x7f35e40b1ec0 2021-07-11T04:01:47.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.764+0000 7f35ebe7d700 1 -- 172.21.2.104:0/1976391633 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f35e4057350 con 0x7f35e40b1ec0 2021-07-11T04:01:47.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.768+0000 7f35d2ffd700 1 -- 172.21.2.104:0/1976391633 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f35d40454a0 con 0x7f35e40b1ec0 2021-07-11T04:01:47.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.768+0000 7f35ebe7d700 1 -- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35c4053240 msgr2=0x7f35c4055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.768+0000 7f35ebe7d700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35c4053240 0x7f35c4055700 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f35e0002f70 tx=0x7f35e0015000).stop 2021-07-11T04:01:47.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.768+0000 7f35ebe7d700 1 -- 172.21.2.104:0/1976391633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35e40b1ec0 msgr2=0x7f35e414b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.768+0000 7f35ebe7d700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35e40b1ec0 0x7f35e414b630 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f35e40b2f50 tx=0x7f35d4007c40).stop 2021-07-11T04:01:47.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.768+0000 7f35ebe7d700 1 -- 172.21.2.104:0/1976391633 shutdown_connections 2021-07-11T04:01:47.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.768+0000 7f35ebe7d700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f35e40b22a0 0x7f35e4156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:47.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.768+0000 7f35ebe7d700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f35c4053240 0x7f35c4055700 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:47.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.768+0000 7f35ebe7d700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f35e40b1ec0 0x7f35e414b630 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:47.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.768+0000 7f35ebe7d700 1 --2- 172.21.2.104:0/1976391633 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f35e40affb0 0x7f35e413fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:47.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.768+0000 7f35ebe7d700 1 -- 172.21.2.104:0/1976391633 >> 172.21.2.104:0/1976391633 conn(0x7f35e405f6b0 msgr2=0x7f35e413ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:47.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.772+0000 7f35ebe7d700 1 -- 172.21.2.104:0/1976391633 shutdown_connections 2021-07-11T04:01:47.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.772+0000 7f35ebe7d700 1 -- 172.21.2.104:0/1976391633 wait complete. 2021-07-11T04:01:47.777 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:01:47.778 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:47.786 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:47.794 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:47.813 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:01:47.827 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:01:47.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:47.828 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:47.828 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:47.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:01:47.833 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T04:01:47.833 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T04:01:47.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:47.834 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:47.835 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:47.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:01:47.839 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T04:01:47.839 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T04:01:47.839 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:01:47.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.916+0000 7efea36a5700 1 Processor -- start 2021-07-11T04:01:47.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.916+0000 7efea36a5700 1 -- start start 2021-07-11T04:01:47.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7efe9c0aa120 0x7efe9c0ae440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7efe9c0b0880 0x7efe9c1405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9c0b2af0 con 0x7efe9c0b04a0 2021-07-11T04:01:47.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c0b2770 con 0x7efe9c0b0880 2021-07-11T04:01:47.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c0b2d20 con 0x7efe9c0aa120 2021-07-11T04:01:47.921 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea1c42700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7efe9c0b0880 0x7efe9c1405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:47.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea1c42700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7efe9c0b0880 0x7efe9c1405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50050/0 (socket says 172.21.2.104:50050) 2021-07-11T04:01:47.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea1c42700 1 -- 172.21.2.104:0/3341887952 learned_addr learned my addr 172.21.2.104:0/3341887952 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:47.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea1441700 1 --1- 172.21.2.104:0/3341887952 >> v1:172.21.2.104:6789/0 conn(0x7efe9c0b04a0 0x7efe9c0add30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46814/0 (socket says 172.21.2.104:46814) 2021-07-11T04:01:47.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea0c40700 1 --2- 172.21.2.104:0/3341887952 >> v2:172.21.2.105:3300/0 conn(0x7efe9c0aa120 0x7efe9c0ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:47.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea1c42700 1 -- 172.21.2.104:0/3341887952 >> v2:172.21.2.105:3300/0 conn(0x7efe9c0aa120 msgr2=0x7efe9c0ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:47.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea1c42700 1 --2- 172.21.2.104:0/3341887952 >> v2:172.21.2.105:3300/0 conn(0x7efe9c0aa120 0x7efe9c0ae440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:47.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea1c42700 1 -- 172.21.2.104:0/3341887952 >> v1:172.21.2.104:6789/0 conn(0x7efe9c0b04a0 legacy=0x7efe9c0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea1c42700 1 -- 172.21.2.104:0/3341887952 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0a9b70 con 0x7efe9c0b0880 2021-07-11T04:01:47.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea0c40700 1 --2- 172.21.2.104:0/3341887952 >> v2:172.21.2.105:3300/0 conn(0x7efe9c0aa120 0x7efe9c0ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:47.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea1c42700 1 --2- 172.21.2.104:0/3341887952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efe9c0b0880 0x7efe9c1405a0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7efe90001110 tx=0x7efe9400b530).ready entity=mon.0 client_cookie=51aa287c2de88235 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:47.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efe8ffff700 1 -- 172.21.2.104:0/3341887952 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe9400be70 con 0x7efe9c0b0880 2021-07-11T04:01:47.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efe8ffff700 1 -- 172.21.2.104:0/3341887952 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe94010470 con 0x7efe9c0b0880 2021-07-11T04:01:47.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efe8ffff700 1 -- 172.21.2.104:0/3341887952 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe940107d0 con 0x7efe9c0b0880 2021-07-11T04:01:47.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- 172.21.2.104:0/3341887952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efe9c0b0880 msgr2=0x7efe9c1405a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 --2- 172.21.2.104:0/3341887952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efe9c0b0880 0x7efe9c1405a0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7efe90001110 tx=0x7efe9400b530).stop 2021-07-11T04:01:47.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- 172.21.2.104:0/3341887952 shutdown_connections 2021-07-11T04:01:47.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 --2- 172.21.2.104:0/3341887952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efe9c0b0880 0x7efe9c1405a0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:47.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 --2- 172.21.2.104:0/3341887952 >> v2:172.21.2.105:3300/0 conn(0x7efe9c0aa120 0x7efe9c0ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:47.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- 172.21.2.104:0/3341887952 >> 172.21.2.104:0/3341887952 conn(0x7efe9c0607c0 msgr2=0x7efe9c060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:47.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- 172.21.2.104:0/3341887952 shutdown_connections 2021-07-11T04:01:47.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- 172.21.2.104:0/3341887952 wait complete. 2021-07-11T04:01:47.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 Processor -- start 2021-07-11T04:01:47.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- start start 2021-07-11T04:01:47.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efe9c0aa120 0x7efe9c0a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efe9c0b04a0 0x7efe9c13fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efe9c0b0880 0x7efe9c140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c05efa0 con 0x7efe9c0b04a0 2021-07-11T04:01:47.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0a3770 con 0x7efe9c0aa120 2021-07-11T04:01:47.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea36a5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c066b40 con 0x7efe9c0b0880 2021-07-11T04:01:47.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea0c40700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efe9c0b04a0 0x7efe9c13fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:47.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea0c40700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efe9c0b04a0 0x7efe9c13fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50056/0 (socket says 172.21.2.104:50056) 2021-07-11T04:01:47.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.920+0000 7efea0c40700 1 -- 172.21.2.104:0/1477980822 learned_addr learned my addr 172.21.2.104:0/1477980822 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:47.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efea0c40700 1 -- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efe9c0b0880 msgr2=0x7efe9c140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efea0c40700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efe9c0b0880 0x7efe9c140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:47.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efea0c40700 1 -- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efe9c0aa120 msgr2=0x7efe9c0a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:47.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efea0c40700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efe9c0aa120 0x7efe9c0a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:47.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efea0c40700 1 -- 172.21.2.104:0/1477980822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9400b7d0 con 0x7efe9c0b04a0 2021-07-11T04:01:47.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efea0c40700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efe9c0b04a0 0x7efe9c13fe10 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7efe98007000 tx=0x7efe98008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:47.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efe8e7fc700 1 -- 172.21.2.104:0/1477980822 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe98010580 con 0x7efe9c0b04a0 2021-07-11T04:01:47.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efe8e7fc700 1 -- 172.21.2.104:0/1477980822 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe980106e0 con 0x7efe9c0b04a0 2021-07-11T04:01:47.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efe8e7fc700 1 -- 172.21.2.104:0/1477980822 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7efe98010a40 con 0x7efe9c0b04a0 2021-07-11T04:01:47.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efea36a5700 1 -- 172.21.2.104:0/1477980822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c064860 con 0x7efe9c0b04a0 2021-07-11T04:01:47.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efea36a5700 1 -- 172.21.2.104:0/1477980822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c0677c0 con 0x7efe9c0b04a0 2021-07-11T04:01:47.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efea36a5700 1 -- 172.21.2.104:0/1477980822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9c057350 con 0x7efe9c0b04a0 2021-07-11T04:01:47.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efe8e7fc700 1 -- 172.21.2.104:0/1477980822 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7efe98010ba0 con 0x7efe9c0b04a0 2021-07-11T04:01:47.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efe8e7fc700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efe7c0532d0 0x7efe7c055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:47.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efe8e7fc700 1 -- 172.21.2.104:0/1477980822 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efe9806b090 con 0x7efe9c0b04a0 2021-07-11T04:01:47.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efea1441700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efe7c0532d0 0x7efe7c055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:47.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.924+0000 7efea1441700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efe7c0532d0 0x7efe7c055790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7efe900011d0 tx=0x7efe90007780).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:47.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:47.928+0000 7efe8e7fc700 1 -- 172.21.2.104:0/1477980822 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7efe980426b0 con 0x7efe9c0b04a0 2021-07-11T04:01:48.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 -- 172.21.2.104:0/1477980822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7efe9c044650 con 0x7efe9c0b04a0 2021-07-11T04:01:48.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efe8e7fc700 1 -- 172.21.2.104:0/1477980822 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v134) v1 ==== 81+0+28837 (secure 0 0 0) 0x7efe98045cd0 con 0x7efe9c0b04a0 2021-07-11T04:01:48.008 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:48.008 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:01:48.008 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 134, 2021-07-11T04:01:48.008 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:01:48.009 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:01:48.009 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:01:45.582896+0000", 2021-07-11T04:01:48.009 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:01:48.009 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:01:48.010 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:01:48.010 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:01:48.010 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:01:48.010 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:01:48.011 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:01:48.011 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:01:48.011 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:01:48.012 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:48.012 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:01:48.012 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:01:48.012 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:01:48.012 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:01:48.013 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:01:48.013 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 24, 2021-07-11T04:01:48.013 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:01:48.013 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:01:48.014 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:01:48.014 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:01:48.014 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:01:48.014 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.014 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:01:48.015 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:01:48.015 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:01:48.015 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:01:48.015 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:01:48.015 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:48.016 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:48.016 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:48.016 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:48.016 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:48.016 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:48.017 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:01:48.017 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:01:48.017 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:01:48.017 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:01:48.018 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:01:48.018 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:48.018 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:48.018 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:48.018 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:48.019 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:48.019 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:48.019 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:48.019 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.020 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:01:48.020 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:48.020 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:48.020 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:48.020 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:48.021 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:48.021 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:48.021 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:48.022 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:48.022 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:48.022 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:48.022 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:48.023 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:48.023 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:48.023 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:48.023 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:48.023 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:48.024 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:48.024 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:48.024 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:48.024 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:48.024 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:48.025 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:48.025 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:48.025 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:48.025 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:48.026 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:48.026 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.026 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:48.026 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:48.026 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:48.027 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:48.027 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:48.027 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:48.027 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:48.027 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:48.028 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:48.028 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:48.028 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:48.028 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:01:48.028 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:01:48.029 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.029 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:48.029 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:01:48.029 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.030 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.030 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.030 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:01:48.030 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:01:48.030 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:01:48.031 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:01:48.031 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:01:48.031 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:01:48.031 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:01:48.032 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:01:48.032 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:01:48.032 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:01:48.032 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:01:48.033 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:01:48.033 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:01:48.033 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:01:48.033 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:01:48.033 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:01:48.034 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:01:48.034 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:01:48.034 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:01:48.034 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:01:48.035 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:01:48.035 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:01:48.035 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:01:48.035 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.035 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:01:48.036 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:01:48.036 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:01:48.036 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:01:48.036 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:01:48.036 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:01:48.037 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:01:48.037 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:01:48.037 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:01:48.037 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:01:48.037 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:01:48.038 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:01:48.038 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:01:48.038 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:01:48.038 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:01:48.039 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:01:48.039 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:01:48.039 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:01:48.039 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:01:48.039 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:01:48.040 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:01:48.040 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:01:48.040 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:01:48.040 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:01:48.040 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:01:48.041 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:01:48.041 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:01:48.041 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.041 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:01:48.042 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:01:48.042 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:01:48.042 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:01:48.043 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:01:48.043 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:01:48.043 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:01:48.043 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:01:48.043 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:01:48.044 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:01:48.044 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:01:48.044 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:01:48.044 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:01:48.044 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:01:48.045 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.045 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.045 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:48.045 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:01:48.045 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.046 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:01:48.046 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:01:48.046 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:48.046 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:48.047 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:48.047 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:48.047 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:48.047 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:48.047 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:48.048 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 129, 2021-07-11T04:01:48.048 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:48.048 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:48.048 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:48.048 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.049 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.049 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.049 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:01:48.049 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:48.049 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.050 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.050 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.050 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:01:48.050 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:48.051 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.051 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.051 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.051 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:48.051 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.052 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.052 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.052 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:01:48.053 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:48.053 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.053 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.053 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.053 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:01:48.054 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:48.054 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.054 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.054 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.055 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:48.055 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.055 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.055 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.055 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:01:48.056 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:48.056 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.056 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.056 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.056 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:01:48.057 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:48.057 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.057 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.057 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.057 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:48.058 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.058 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.058 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.058 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:01:48.059 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:48.059 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.059 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.059 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.059 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:01:48.060 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:01:48.060 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.060 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.060 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.060 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:01:48.061 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:01:48.061 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:01:48.061 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:01:48.061 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:48.061 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:48.062 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:48.062 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.062 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.063 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.063 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:01:48.063 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:01:48.063 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:48.064 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:48.064 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:48.064 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:48.064 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:48.064 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:48.065 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:48.065 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 129, 2021-07-11T04:01:48.065 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:48.065 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:48.065 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:48.066 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.066 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.066 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.066 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:01:48.067 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:48.067 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.067 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.067 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.067 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:01:48.068 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:48.068 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.068 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.068 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.069 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:48.069 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.069 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.069 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.069 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:01:48.070 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:48.070 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.070 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.070 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.071 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:01:48.071 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:48.071 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.071 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.071 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.072 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:48.072 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.072 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.072 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.072 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:01:48.073 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:48.073 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.073 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.074 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.074 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:01:48.074 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:48.074 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.075 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.075 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.075 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:48.075 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.075 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.076 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.076 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:01:48.076 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:48.076 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.076 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.077 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.077 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:01:48.077 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:01:48.077 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.078 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.078 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.078 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:01:48.078 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:01:48.078 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:01:48.079 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:01:48.079 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:48.079 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:48.079 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:48.079 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.080 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.080 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.080 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:01:48.080 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:01:48.080 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:48.081 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:48.081 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:48.081 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:48.081 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:48.081 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:48.082 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:48.082 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 124, 2021-07-11T04:01:48.082 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:48.082 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:48.083 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:48.083 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.083 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.084 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.084 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:01:48.084 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:48.084 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.084 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.085 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.085 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:01:48.085 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:48.085 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.085 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.086 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.086 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:48.086 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.086 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.087 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.087 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:01:48.087 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:48.087 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.087 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.088 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.088 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:01:48.088 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:48.088 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.088 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.089 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.089 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:48.089 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.089 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.089 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.090 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:01:48.090 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:48.090 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.090 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.091 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.091 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:01:48.091 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:48.091 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.091 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.092 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.092 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:48.092 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.092 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.092 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.093 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:01:48.093 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:48.093 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.093 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.094 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.094 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:01:48.094 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:01:48.094 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.095 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.095 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.095 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:01:48.095 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:01:48.095 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:01:48.096 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:01:48.096 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:48.096 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:48.096 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:48.096 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.097 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.097 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.097 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:01:48.097 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:01:48.097 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:48.098 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:48.098 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:48.098 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:48.098 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:48.099 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:48.099 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:48.099 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 130, 2021-07-11T04:01:48.099 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:48.099 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:48.100 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:48.100 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.100 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.100 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.100 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:01:48.101 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:48.101 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.101 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.101 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.101 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:01:48.102 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:48.102 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.102 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.102 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.103 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:48.103 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.103 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.103 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.104 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:01:48.104 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:48.104 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.104 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.105 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.105 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:01:48.105 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:48.105 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.105 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.106 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.106 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:48.106 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.106 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.106 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.107 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:01:48.107 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:48.107 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.107 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.108 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.108 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:01:48.108 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:48.108 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.108 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.109 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.109 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:48.109 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.109 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.109 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.110 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:01:48.110 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:48.110 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.110 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.111 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.111 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:01:48.111 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:01:48.111 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.111 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.112 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.112 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:01:48.112 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:01:48.112 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:01:48.112 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:01:48.113 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:48.113 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:48.113 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:48.113 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.113 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.114 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.114 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:01:48.115 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:01:48.115 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:48.115 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:48.115 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:48.115 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:48.116 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:48.116 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:48.116 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:48.116 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 124, 2021-07-11T04:01:48.116 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:48.117 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:48.117 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:48.117 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.117 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.117 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.118 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:01:48.118 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:48.118 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.118 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.119 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.119 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:01:48.119 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:48.119 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.119 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.120 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.120 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:48.120 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.120 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.120 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.121 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:01:48.121 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:48.121 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.121 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.121 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.122 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:01:48.122 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:48.122 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.122 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.123 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.123 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:48.123 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.123 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.123 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.124 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:01:48.124 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:48.124 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.124 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.125 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.125 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:01:48.125 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:48.125 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.126 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.126 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.126 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:48.126 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.126 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.127 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.127 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:01:48.127 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:48.127 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.127 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.128 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.128 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:01:48.128 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:01:48.128 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.129 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.129 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.129 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:01:48.129 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:01:48.129 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:01:48.130 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:01:48.130 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:48.130 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:48.130 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:48.130 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.131 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.131 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.131 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:01:48.131 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:01:48.132 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:48.132 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:48.132 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:48.132 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:48.132 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:48.133 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:48.133 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:48.133 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 130, 2021-07-11T04:01:48.133 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:48.133 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:48.134 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:48.134 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.134 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.135 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.135 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:01:48.135 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:48.135 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.136 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.136 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.136 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:01:48.136 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:48.136 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.137 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.137 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.137 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:48.137 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.138 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.138 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.138 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:01:48.138 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:48.138 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.139 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.139 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.139 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:01:48.139 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:48.139 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.140 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.140 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.140 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:48.140 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.141 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.141 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.141 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:01:48.141 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:48.141 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.142 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.142 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.142 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:01:48.142 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:48.142 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.143 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.143 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.143 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:48.143 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.143 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.144 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.144 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:01:48.144 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:48.144 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.144 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.145 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.145 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:01:48.145 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:01:48.146 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.147 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.147 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.147 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:01:48.147 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:01:48.148 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:01:48.148 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:01:48.148 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:48.148 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:48.148 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:48.149 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.149 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.149 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.149 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:01:48.150 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:01:48.150 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:48.150 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:48.150 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:48.151 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:48.151 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:48.151 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:48.151 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:48.152 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 130, 2021-07-11T04:01:48.152 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:48.152 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:48.152 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:48.152 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.153 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.153 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.153 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:01:48.153 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:48.153 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.154 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.154 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.154 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:01:48.154 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:48.155 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.155 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.155 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.155 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:48.156 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.156 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.156 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.156 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:01:48.156 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:48.157 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.157 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.157 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.157 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:01:48.157 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:48.158 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.158 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.158 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.158 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:48.159 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.159 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.159 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.159 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:01:48.159 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:48.160 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.160 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.160 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.160 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:01:48.161 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:48.161 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.161 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.161 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.161 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:48.162 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.162 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.162 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.162 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:01:48.163 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:48.163 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.163 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.163 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.163 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:01:48.164 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:01:48.164 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.164 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.164 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.164 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:01:48.165 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:01:48.165 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:01:48.165 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:01:48.166 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:48.166 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:48.166 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:48.166 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.166 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.167 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.167 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:01:48.167 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:01:48.167 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:01:48.168 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:01:48.168 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:01:48.168 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:01:48.168 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:01:48.168 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:01:48.169 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:01:48.169 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 129, 2021-07-11T04:01:48.169 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:01:48.169 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:01:48.169 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:01:48.170 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.170 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.170 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.170 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:01:48.171 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:48.171 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.171 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.171 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.172 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:01:48.172 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:48.172 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.172 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.172 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.173 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:01:48.173 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.173 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.173 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.173 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:01:48.174 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:48.174 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.174 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.174 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.175 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:01:48.175 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:48.175 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.175 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.175 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.176 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:01:48.176 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.176 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.176 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.176 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:01:48.177 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:48.177 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.177 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.177 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.178 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:01:48.178 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:48.178 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.178 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.178 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.179 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:01:48.179 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:01:48.179 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.179 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:01:48.180 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:01:48.180 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:48.180 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.180 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.181 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:01:48.181 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:01:48.181 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:01:48.181 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.181 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.182 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.182 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:01:48.182 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:01:48.182 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:01:48.182 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:01:48.183 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:01:48.183 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:01:48.183 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:01:48.183 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:48.184 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.184 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:48.184 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:01:48.184 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.184 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:01:48.185 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:48.185 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:48.185 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:48.185 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:48.186 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:48.186 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:01:48.186 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:48.186 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.186 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.187 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:01:48.187 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:48.187 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:48.187 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:48.187 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:48.188 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:48.188 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:01:48.188 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:48.188 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.188 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.189 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:01:48.189 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:48.189 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:48.189 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:48.190 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:48.190 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:48.190 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:01:48.190 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:48.191 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.191 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.191 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:01:48.191 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:48.191 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:48.192 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:48.192 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:48.192 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:48.192 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:01:48.192 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:48.193 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.193 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.193 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:01:48.193 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:48.194 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:48.194 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:48.194 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:48.194 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:48.194 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:01:48.195 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:48.195 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.195 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.195 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:01:48.196 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:48.197 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:48.197 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:48.197 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:48.197 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:48.197 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:01:48.198 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:48.198 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.198 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.198 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:01:48.199 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:48.199 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:48.199 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:48.199 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:48.199 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:48.200 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:01:48.200 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:48.200 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.200 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:48.201 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:01:48.201 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:01:48.201 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:01:48.201 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:01:48.201 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:01:48.202 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:01:48.202 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:01:48.202 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:01:48.202 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.203 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:01:48.203 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:01:48.203 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:01:48.203 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:01:48.203 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:01:48.204 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T04:01:48.204 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6835/1367521056": "2021-07-12T04:01:26.954459+0000", 2021-07-11T04:01:48.204 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6833/1367521056": "2021-07-12T04:01:26.954459+0000" 2021-07-11T04:01:48.204 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.205 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:01:48.205 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:01:48.205 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:01:48.205 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:01:48.206 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:01:48.206 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:01:48.206 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:01:48.206 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:01:48.206 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:48.207 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:48.207 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:01:48.207 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:01:48.207 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:01:48.208 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:01:48.208 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:01:48.208 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:01:48.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 -- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efe7c0532d0 msgr2=0x7efe7c055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efe7c0532d0 0x7efe7c055790 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7efe900011d0 tx=0x7efe90007780).stop 2021-07-11T04:01:48.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 -- 172.21.2.104:0/1477980822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efe9c0b04a0 msgr2=0x7efe9c13fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efe9c0b04a0 0x7efe9c13fe10 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7efe98007000 tx=0x7efe98008040).stop 2021-07-11T04:01:48.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 -- 172.21.2.104:0/1477980822 shutdown_connections 2021-07-11T04:01:48.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7efe9c0b0880 0x7efe9c140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:48.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7efe7c0532d0 0x7efe7c055790 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7efe9c0b04a0 0x7efe9c13fe10 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 --2- 172.21.2.104:0/1477980822 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7efe9c0aa120 0x7efe9c0a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:48.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 -- 172.21.2.104:0/1477980822 >> 172.21.2.104:0/1477980822 conn(0x7efe9c0607c0 msgr2=0x7efe9c05fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:48.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 -- 172.21.2.104:0/1477980822 shutdown_connections 2021-07-11T04:01:48.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.008+0000 7efea36a5700 1 -- 172.21.2.104:0/1477980822 wait complete. 2021-07-11T04:01:48.213 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:48.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f82042f8700 1 Processor -- start 2021-07-11T04:01:48.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f82042f8700 1 -- start start 2021-07-11T04:01:48.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f82042f8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81fc0a76b0 0x7f81fc066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f82042f8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f81fc0b1970 0x7f81fc066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f82042f8700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81fc067450 con 0x7f81fc0b0060 2021-07-11T04:01:48.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f82042f8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc0a7a90 con 0x7f81fc0b1970 2021-07-11T04:01:48.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f82042f8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc147620 con 0x7f81fc0a76b0 2021-07-11T04:01:48.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f8201893700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f81fc0b1970 0x7f81fc066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f8202895700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f81fc0b0060 0x7f81fc05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46832/0 (socket says 172.21.2.104:46832) 2021-07-11T04:01:48.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f8202895700 1 -- 172.21.2.104:0/1034324964 learned_addr learned my addr 172.21.2.104:0/1034324964 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:48.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f8201893700 1 -- 172.21.2.104:0/1034324964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81fc0a76b0 msgr2=0x7f81fc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:48.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f8201893700 1 --2- 172.21.2.104:0/1034324964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81fc0a76b0 0x7f81fc066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:48.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f8201893700 1 -- 172.21.2.104:0/1034324964 >> v1:172.21.2.104:6789/0 conn(0x7f81fc0b0060 legacy=0x7f81fc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.268+0000 7f8201893700 1 -- 172.21.2.104:0/1034324964 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc1478c0 con 0x7f81fc0b1970 2021-07-11T04:01:48.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201893700 1 --2- 172.21.2.104:0/1034324964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81fc0b1970 0x7f81fc066f60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f81f4001490 tx=0x7f81ec006df0).ready entity=mon.0 client_cookie=e7d0610023e2c9eb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:48.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201092700 1 -- 172.21.2.104:0/1034324964 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81ec00d570 con 0x7f81fc0b1970 2021-07-11T04:01:48.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201092700 1 -- 172.21.2.104:0/1034324964 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec000e90 con 0x7f81fc0b1970 2021-07-11T04:01:48.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201092700 1 -- 172.21.2.104:0/1034324964 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81ec00d8c0 con 0x7f81fc0b1970 2021-07-11T04:01:48.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- 172.21.2.104:0/1034324964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81fc0b1970 msgr2=0x7f81fc066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.274 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 --2- 172.21.2.104:0/1034324964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81fc0b1970 0x7f81fc066f60 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f81f4001490 tx=0x7f81ec006df0).stop 2021-07-11T04:01:48.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- 172.21.2.104:0/1034324964 shutdown_connections 2021-07-11T04:01:48.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 --2- 172.21.2.104:0/1034324964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81fc0b1970 0x7f81fc066f60 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 --2- 172.21.2.104:0/1034324964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81fc0a76b0 0x7f81fc066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:48.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- 172.21.2.104:0/1034324964 >> 172.21.2.104:0/1034324964 conn(0x7f81fc0b53e0 msgr2=0x7f81fc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:48.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- 172.21.2.104:0/1034324964 shutdown_connections 2021-07-11T04:01:48.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- 172.21.2.104:0/1034324964 wait complete. 2021-07-11T04:01:48.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 Processor -- start 2021-07-11T04:01:48.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- start start 2021-07-11T04:01:48.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81fc0a76b0 0x7f81fc0af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81fc0b0060 0x7f81fc0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81fc0b1970 0x7f81fc0aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0abe50 con 0x7f81fc0b0060 2021-07-11T04:01:48.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0ac080 con 0x7f81fc0a76b0 2021-07-11T04:01:48.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0ae570 con 0x7f81fc0b1970 2021-07-11T04:01:48.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201893700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81fc0b0060 0x7f81fc0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201893700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81fc0b0060 0x7f81fc0ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50066/0 (socket says 172.21.2.104:50066) 2021-07-11T04:01:48.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201893700 1 -- 172.21.2.104:0/260989964 learned_addr learned my addr 172.21.2.104:0/260989964 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:48.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201893700 1 -- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81fc0b1970 msgr2=0x7f81fc0aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201893700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81fc0b1970 0x7f81fc0aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:48.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201893700 1 -- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81fc0a76b0 msgr2=0x7f81fc0af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201893700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81fc0a76b0 0x7f81fc0af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:48.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201893700 1 -- 172.21.2.104:0/260989964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ec00b180 con 0x7f81fc0b0060 2021-07-11T04:01:48.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f8201893700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81fc0b0060 0x7f81fc0ac860 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f81f4001490 tx=0x7f81ec014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:48.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f81eb7fe700 1 -- 172.21.2.104:0/260989964 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81ec002f50 con 0x7f81fc0b0060 2021-07-11T04:01:48.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f81eb7fe700 1 -- 172.21.2.104:0/260989964 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec00de40 con 0x7f81fc0b0060 2021-07-11T04:01:48.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- 172.21.2.104:0/260989964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc0ae810 con 0x7f81fc0b0060 2021-07-11T04:01:48.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- 172.21.2.104:0/260989964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc0aa770 con 0x7f81fc0b0060 2021-07-11T04:01:48.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f81eb7fe700 1 -- 172.21.2.104:0/260989964 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f81ec01e620 con 0x7f81fc0b0060 2021-07-11T04:01:48.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.272+0000 7f82042f8700 1 -- 172.21.2.104:0/260989964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81fc0ae9a0 con 0x7f81fc0b0060 2021-07-11T04:01:48.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.276+0000 7f81eb7fe700 1 -- 172.21.2.104:0/260989964 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f81ec00f030 con 0x7f81fc0b0060 2021-07-11T04:01:48.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.276+0000 7f81eb7fe700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81dc0532d0 0x7f81dc055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.276+0000 7f81eb7fe700 1 -- 172.21.2.104:0/260989964 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f81ec06c330 con 0x7f81fc0b0060 2021-07-11T04:01:48.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.276+0000 7f8202094700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81dc0532d0 0x7f81dc055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.276+0000 7f8202094700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81dc0532d0 0x7f81dc055790 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f81f8002f70 tx=0x7f81f8006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:48.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.276+0000 7f81eb7fe700 1 -- 172.21.2.104:0/260989964 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f81ec0438f0 con 0x7f81fc0b0060 2021-07-11T04:01:48.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.388+0000 7f82042f8700 1 -- 172.21.2.104:0/260989964 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81fc0ae9a0 con 0x7f81fc0b0060 2021-07-11T04:01:48.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.388+0000 7f81eb7fe700 1 -- 172.21.2.104:0/260989964 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+504 (secure 0 0 0) 0x7f81ec046f10 con 0x7f81fc0b0060 2021-07-11T04:01:48.396 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:48.396 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2021-07-11T04:01:48.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 -- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81dc0532d0 msgr2=0x7f81dc055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81dc0532d0 0x7f81dc055790 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f81f8002f70 tx=0x7f81f8006a80).stop 2021-07-11T04:01:48.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 -- 172.21.2.104:0/260989964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81fc0b0060 msgr2=0x7f81fc0ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81fc0b0060 0x7f81fc0ac860 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f81f4001490 tx=0x7f81ec014000).stop 2021-07-11T04:01:48.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 -- 172.21.2.104:0/260989964 shutdown_connections 2021-07-11T04:01:48.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81fc0b1970 0x7f81fc0aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:48.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81dc0532d0 0x7f81dc055790 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81fc0b0060 0x7f81fc0ac860 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 --2- 172.21.2.104:0/260989964 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81fc0a76b0 0x7f81fc0af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:48.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 -- 172.21.2.104:0/260989964 >> 172.21.2.104:0/260989964 conn(0x7f81fc0b53e0 msgr2=0x7f81fc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:48.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 -- 172.21.2.104:0/260989964 shutdown_connections 2021-07-11T04:01:48.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.392+0000 7f82042f8700 1 -- 172.21.2.104:0/260989964 wait complete. 2021-07-11T04:01:48.401 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 64 2021-07-11T04:01:48.401 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:48.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3f0b16700 1 Processor -- start 2021-07-11T04:01:48.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3f0b16700 1 -- start start 2021-07-11T04:01:48.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3f0b16700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd3ec13fff0 0x7fd3ec140480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3f0b16700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ec067500 con 0x7fd3ec067120 2021-07-11T04:01:48.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3f0b16700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ec0af4d0 con 0x7fd3ec0b96e0 2021-07-11T04:01:48.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3f0b16700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ec143080 con 0x7fd3ec13fff0 2021-07-11T04:01:48.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ea59c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd3ec067120 0x7fd3ec1439b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46842/0 (socket says 172.21.2.104:46842) 2021-07-11T04:01:48.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ea59c700 1 -- 172.21.2.104:0/1608369927 learned_addr learned my addr 172.21.2.104:0/1608369927 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:48.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3756891621 0 0) 0x7fd3ec067500 con 0x7fd3ec067120 2021-07-11T04:01:48.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3cc003260 con 0x7fd3ec067120 2021-07-11T04:01:48.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ead9d700 1 --2- 172.21.2.104:0/1608369927 >> v2:172.21.2.105:3300/0 conn(0x7fd3ec13fff0 0x7fd3ec140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1799592043 0 0) 0x7fd3cc003260 con 0x7fd3ec067120 2021-07-11T04:01:48.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 >> v2:172.21.2.105:3300/0 conn(0x7fd3ec13fff0 msgr2=0x7fd3ec140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:48.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 --2- 172.21.2.104:0/1608369927 >> v2:172.21.2.105:3300/0 conn(0x7fd3ec13fff0 0x7fd3ec140480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 >> v1:172.21.2.105:6789/0 conn(0x7fd3ec0b96e0 legacy=0x7fd3ec0aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:48.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec143270 con 0x7fd3ec067120 2021-07-11T04:01:48.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fd3d4001680 con 0x7fd3ec067120 2021-07-11T04:01:48.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 >> v1:172.21.2.104:6789/0 conn(0x7fd3ec067120 legacy=0x7fd3ec1439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 --2- 172.21.2.104:0/1608369927 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3cc003820 0x7fd3cc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 --2- 172.21.2.104:0/1608369927 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc004500 0x7fd3cc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 --2- 172.21.2.104:0/1608369927 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc004f70 0x7fd3cc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc00be20 con 0x7fd3cc004f70 2021-07-11T04:01:48.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc00bf90 con 0x7fd3cc003820 2021-07-11T04:01:48.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc00c150 con 0x7fd3cc004500 2021-07-11T04:01:48.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ead9d700 1 --2- 172.21.2.104:0/1608369927 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc004f70 0x7fd3cc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ea59c700 1 --2- 172.21.2.104:0/1608369927 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3cc003820 0x7fd3cc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ead9d700 1 -- 172.21.2.104:0/1608369927 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc004500 msgr2=0x7fd3cc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:48.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ead9d700 1 --2- 172.21.2.104:0/1608369927 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc004500 0x7fd3cc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:48.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ead9d700 1 -- 172.21.2.104:0/1608369927 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3cc003820 msgr2=0x7fd3cc003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:48.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ead9d700 1 --2- 172.21.2.104:0/1608369927 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3cc003820 0x7fd3cc003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ead9d700 1 -- 172.21.2.104:0/1608369927 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc003260 con 0x7fd3cc004f70 2021-07-11T04:01:48.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ea59c700 1 --2- 172.21.2.104:0/1608369927 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3cc003820 0x7fd3cc003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:48.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ead9d700 1 -- 172.21.2.104:0/1608369927 reap_dead start 2021-07-11T04:01:48.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3ead9d700 1 --2- 172.21.2.104:0/1608369927 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc004f70 0x7fd3cc00b750 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7fd3e400a000 tx=0x7fd3e4008040).ready entity=mon.0 client_cookie=b770bd83b7941d83 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:48.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3e400c070 con 0x7fd3cc004f70 2021-07-11T04:01:48.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e400e810 con 0x7fd3cc004f70 2021-07-11T04:01:48.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3e959a700 1 -- 172.21.2.104:0/1608369927 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3e400eb70 con 0x7fd3cc004f70 2021-07-11T04:01:48.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3f0b16700 1 -- 172.21.2.104:0/1608369927 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc004f70 msgr2=0x7fd3cc00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.452+0000 7fd3f0b16700 1 --2- 172.21.2.104:0/1608369927 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc004f70 0x7fd3cc00b750 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7fd3e400a000 tx=0x7fd3e4008040).stop 2021-07-11T04:01:48.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 -- 172.21.2.104:0/1608369927 shutdown_connections 2021-07-11T04:01:48.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 --2- 172.21.2.104:0/1608369927 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3cc004f70 0x7fd3cc00b750 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 -- 172.21.2.104:0/1608369927 >> 172.21.2.104:0/1608369927 conn(0x7fd3ec05f9f0 msgr2=0x7fd3ec0616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:48.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 -- 172.21.2.104:0/1608369927 shutdown_connections 2021-07-11T04:01:48.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 -- 172.21.2.104:0/1608369927 wait complete. 2021-07-11T04:01:48.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 Processor -- start 2021-07-11T04:01:48.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 -- start start 2021-07-11T04:01:48.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc004f70 0x7fd3ec1421f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3ec13bfb0 0x7fd3ec155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3ec155cf0 0x7fd3ec13ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec0b88d0 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec149df0 con 0x7fd3ec155cf0 2021-07-11T04:01:48.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec1553d0 con 0x7fd3cc004f70 2021-07-11T04:01:48.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e9d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3ec13bfb0 0x7fd3ec155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e9d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3ec13bfb0 0x7fd3ec155800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50088/0 (socket says 172.21.2.104:50088) 2021-07-11T04:01:48.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e9d9b700 1 -- 172.21.2.104:0/3410157935 learned_addr learned my addr 172.21.2.104:0/3410157935 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:48.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3ead9d700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3ec155cf0 0x7fd3ec13ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3ea59c700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc004f70 0x7fd3ec1421f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e9d9b700 1 -- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc004f70 msgr2=0x7fd3ec1421f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e9d9b700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc004f70 0x7fd3ec1421f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e9d9b700 1 -- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3ec155cf0 msgr2=0x7fd3ec13ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e9d9b700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3ec155cf0 0x7fd3ec13ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e9d9b700 1 -- 172.21.2.104:0/3410157935 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec0b73d0 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3ead9d700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3ec155cf0 0x7fd3ec13ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:48.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e9d9b700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3ec13bfb0 0x7fd3ec155800 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7fd3dc001fa0 tx=0x7fd3dc007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:48.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e37fe700 1 -- 172.21.2.104:0/3410157935 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3dc00ce80 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e37fe700 1 -- 172.21.2.104:0/3410157935 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc000e90 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 -- 172.21.2.104:0/3410157935 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ec147890 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e37fe700 1 -- 172.21.2.104:0/3410157935 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd3dc007430 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 -- 172.21.2.104:0/3410157935 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ec0b0650 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3f0b16700 1 -- 172.21.2.104:0/3410157935 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ec0b9490 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e37fe700 1 -- 172.21.2.104:0/3410157935 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd3dc01a060 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e37fe700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3d00531f0 0x7fd3d00556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.456+0000 7fd3e37fe700 1 -- 172.21.2.104:0/3410157935 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd3dc06ab40 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.460+0000 7fd3ea59c700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3d00531f0 0x7fd3d00556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.460+0000 7fd3ea59c700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3d00531f0 0x7fd3d00556b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fd3d4009000 tx=0x7fd3d4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:48.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.460+0000 7fd3e37fe700 1 -- 172.21.2.104:0/3410157935 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd3dc042050 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.568+0000 7fd3f0b16700 1 -- 172.21.2.104:0/3410157935 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3ec155510 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.568+0000 7fd3e37fe700 1 -- 172.21.2.104:0/3410157935 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+504 (secure 0 0 0) 0x7fd3dc06ae60 con 0x7fd3ec13bfb0 2021-07-11T04:01:48.569 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:48.570 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[],"filesystems":[]} 2021-07-11T04:01:48.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.568+0000 7fd3f0b16700 1 -- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3d00531f0 msgr2=0x7fd3d00556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.568+0000 7fd3f0b16700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3d00531f0 0x7fd3d00556b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fd3d4009000 tx=0x7fd3d4009040).stop 2021-07-11T04:01:48.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.568+0000 7fd3f0b16700 1 -- 172.21.2.104:0/3410157935 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3ec13bfb0 msgr2=0x7fd3ec155800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.568+0000 7fd3f0b16700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3ec13bfb0 0x7fd3ec155800 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7fd3dc001fa0 tx=0x7fd3dc007c30).stop 2021-07-11T04:01:48.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.572+0000 7fd3f0b16700 1 -- 172.21.2.104:0/3410157935 shutdown_connections 2021-07-11T04:01:48.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.572+0000 7fd3f0b16700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd3ec155cf0 0x7fd3ec13ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.572+0000 7fd3f0b16700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd3d00531f0 0x7fd3d00556b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.572+0000 7fd3f0b16700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd3ec13bfb0 0x7fd3ec155800 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.572+0000 7fd3f0b16700 1 --2- 172.21.2.104:0/3410157935 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd3cc004f70 0x7fd3ec1421f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.572+0000 7fd3f0b16700 1 -- 172.21.2.104:0/3410157935 >> 172.21.2.104:0/3410157935 conn(0x7fd3ec05f9f0 msgr2=0x7fd3ec0b5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:48.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.572+0000 7fd3f0b16700 1 -- 172.21.2.104:0/3410157935 shutdown_connections 2021-07-11T04:01:48.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.572+0000 7fd3f0b16700 1 -- 172.21.2.104:0/3410157935 wait complete. 2021-07-11T04:01:48.573 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 64 2021-07-11T04:01:48.577 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T04:01:48.577 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T04:01:48.578 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T04:01:48.578 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T04:01:48.579 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T04:01:48.579 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T04:01:48.581 INFO:tasks.ceph.mds.a:Started 2021-07-11T04:01:48.582 DEBUG:teuthology.parallel:result is None 2021-07-11T04:01:48.583 INFO:tasks.ceph.mds.b:Started 2021-07-11T04:01:48.583 INFO:tasks.ceph.mds.c:Started 2021-07-11T04:01:48.584 DEBUG:teuthology.parallel:result is None 2021-07-11T04:01:48.584 DEBUG:teuthology.parallel:result is None 2021-07-11T04:01:48.585 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T04:01:48.619 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T04:01:48.619 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T04:01:48.622 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T04:01:48.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.636+0000 7fb7d953e700 1 Processor -- start 2021-07-11T04:01:48.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.636+0000 7fb7d953e700 1 -- start start 2021-07-11T04:01:48.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d953e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb7d40ad870 0x7fb7d40b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d953e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d40ace10 con 0x7fb7d40aca30 2021-07-11T04:01:48.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d953e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d40b5160 con 0x7fb7d40b1430 2021-07-11T04:01:48.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d953e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d40adee0 con 0x7fb7d40ad870 2021-07-11T04:01:48.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d3fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb7d40aca30 0x7fb7d41434c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46892/0 (socket says 172.21.2.104:46892) 2021-07-11T04:01:48.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d3fff700 1 -- 172.21.2.104:0/374187075 learned_addr learned my addr 172.21.2.104:0/374187075 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:48.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d37fe700 1 --2- 172.21.2.104:0/374187075 >> v2:172.21.2.105:3300/0 conn(0x7fb7d40ad870 0x7fb7d40b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106698255 0 0) 0x7fb7d40ace10 con 0x7fb7d40aca30 2021-07-11T04:01:48.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7b4003260 con 0x7fb7d40aca30 2021-07-11T04:01:48.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1602914035 0 0) 0x7fb7b4003260 con 0x7fb7d40aca30 2021-07-11T04:01:48.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 >> v2:172.21.2.105:3300/0 conn(0x7fb7d40ad870 msgr2=0x7fb7d40b55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:48.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 --2- 172.21.2.104:0/374187075 >> v2:172.21.2.105:3300/0 conn(0x7fb7d40ad870 0x7fb7d40b55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 >> v1:172.21.2.105:6789/0 conn(0x7fb7d40b1430 legacy=0x7fb7d4067000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d40ae180 con 0x7fb7d40aca30 2021-07-11T04:01:48.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fb7c4001660 con 0x7fb7d40aca30 2021-07-11T04:01:48.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 >> v1:172.21.2.104:6789/0 conn(0x7fb7d40aca30 legacy=0x7fb7d41434c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b40037a0 0x7fb7b4003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 0x7fb7b4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7b4004c90 0x7fb7b400fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d40adee0 con 0x7fb7b40040b0 2021-07-11T04:01:48.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b4010250 con 0x7fb7b4004c90 2021-07-11T04:01:48.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b4010390 con 0x7fb7b40037a0 2021-07-11T04:01:48.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d37fe700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 0x7fb7b4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d8d3d700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7b4004c90 0x7fb7b400fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d3fff700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b40037a0 0x7fb7b4003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d37fe700 1 -- 172.21.2.104:0/374187075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b40037a0 msgr2=0x7fb7b4003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:48.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d37fe700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7b40037a0 0x7fb7b4003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d37fe700 1 -- 172.21.2.104:0/374187075 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7b4004c90 msgr2=0x7fb7b400fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:48.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d37fe700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7b4004c90 0x7fb7b400fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d37fe700 1 -- 172.21.2.104:0/374187075 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b4003260 con 0x7fb7b40040b0 2021-07-11T04:01:48.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d8d3d700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7b4004c90 0x7fb7b400fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:01:48.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d8d3d700 1 -- 172.21.2.104:0/374187075 reap_dead start 2021-07-11T04:01:48.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d37fe700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 0x7fb7b4004500 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0x7fb7bc00c000 tx=0x7fb7bc006df0).ready entity=mon.0 client_cookie=bc0a43971b90de5e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:48.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7bc00e800 con 0x7fb7b40040b0 2021-07-11T04:01:48.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc00e960 con 0x7fb7b40040b0 2021-07-11T04:01:48.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d2ffd700 1 -- 172.21.2.104:0/374187075 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7bc00ecc0 con 0x7fb7b40040b0 2021-07-11T04:01:48.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d953e700 1 -- 172.21.2.104:0/374187075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 msgr2=0x7fb7b4004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d953e700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 0x7fb7b4004500 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0x7fb7bc00c000 tx=0x7fb7bc006df0).stop 2021-07-11T04:01:48.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d953e700 1 -- 172.21.2.104:0/374187075 shutdown_connections 2021-07-11T04:01:48.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d953e700 1 --2- 172.21.2.104:0/374187075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 0x7fb7b4004500 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d953e700 1 -- 172.21.2.104:0/374187075 >> 172.21.2.104:0/374187075 conn(0x7fb7d4062560 msgr2=0x7fb7d4062940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:48.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d953e700 1 -- 172.21.2.104:0/374187075 shutdown_connections 2021-07-11T04:01:48.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.640+0000 7fb7d953e700 1 -- 172.21.2.104:0/374187075 wait complete. 2021-07-11T04:01:48.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d953e700 1 Processor -- start 2021-07-11T04:01:48.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d953e700 1 -- start start 2021-07-11T04:01:48.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d953e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 0x7fb7d4143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d953e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7d4143e20 0x7fb7d4063d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d953e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7d4064270 0x7fb7d413b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d953e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d413dc60 con 0x7fb7b40040b0 2021-07-11T04:01:48.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d953e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d413d1c0 con 0x7fb7d4143e20 2021-07-11T04:01:48.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d953e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d4142d80 con 0x7fb7d4064270 2021-07-11T04:01:48.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 0x7fb7d4143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 0x7fb7d4143650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50138/0 (socket says 172.21.2.104:50138) 2021-07-11T04:01:48.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d3fff700 1 -- 172.21.2.104:0/3411418336 learned_addr learned my addr 172.21.2.104:0/3411418336 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:48.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d8d3d700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7d4064270 0x7fb7d413b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d37fe700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7d4143e20 0x7fb7d4063d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d3fff700 1 -- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7d4064270 msgr2=0x7fb7d413b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d3fff700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7d4064270 0x7fb7d413b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d3fff700 1 -- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7d4143e20 msgr2=0x7fb7d4063d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:48.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d3fff700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7d4143e20 0x7fb7d4063d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:48.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d3fff700 1 -- 172.21.2.104:0/3411418336 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d413d3f0 con 0x7fb7b40040b0 2021-07-11T04:01:48.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d3fff700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 0x7fb7d4143650 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7fb7c400d000 tx=0x7fb7d40ae310).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:48.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d17fa700 1 -- 172.21.2.104:0/3411418336 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7c40116b0 con 0x7fb7b40040b0 2021-07-11T04:01:48.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d17fa700 1 -- 172.21.2.104:0/3411418336 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c4011810 con 0x7fb7b40040b0 2021-07-11T04:01:48.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d17fa700 1 -- 172.21.2.104:0/3411418336 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb7c4011b70 con 0x7fb7b40040b0 2021-07-11T04:01:48.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d953e700 1 -- 172.21.2.104:0/3411418336 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d413d890 con 0x7fb7b40040b0 2021-07-11T04:01:48.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d953e700 1 -- 172.21.2.104:0/3411418336 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d4143020 con 0x7fb7b40040b0 2021-07-11T04:01:48.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d953e700 1 -- 172.21.2.104:0/3411418336 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d413d580 con 0x7fb7b40040b0 2021-07-11T04:01:48.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d17fa700 1 -- 172.21.2.104:0/3411418336 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb7c401b030 con 0x7fb7b40040b0 2021-07-11T04:01:48.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d17fa700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb7b8053300 0x7fb7b80557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:48.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d17fa700 1 -- 172.21.2.104:0/3411418336 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb7c406ac00 con 0x7fb7b40040b0 2021-07-11T04:01:48.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d37fe700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb7b8053300 0x7fb7b80557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:48.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.644+0000 7fb7d37fe700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb7b8053300 0x7fb7b80557c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fb7bc01e000 tx=0x7fb7bc019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:48.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.648+0000 7fb7d17fa700 1 -- 172.21.2.104:0/3411418336 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb7c4070020 con 0x7fb7b40040b0 2021-07-11T04:01:48.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:48.740+0000 7fb7d953e700 1 -- 172.21.2.104:0/3411418336 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fb7d4057350 con 0x7fb7b40040b0 2021-07-11T04:01:49.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d17fa700 1 -- 172.21.2.104:0/3411418336 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v135) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb7c4019070 con 0x7fb7b40040b0 2021-07-11T04:01:49.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 -- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb7b8053300 msgr2=0x7fb7b80557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:49.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb7b8053300 0x7fb7b80557c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fb7bc01e000 tx=0x7fb7bc019000).stop 2021-07-11T04:01:49.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 -- 172.21.2.104:0/3411418336 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 msgr2=0x7fb7d4143650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:49.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 0x7fb7d4143650 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7fb7c400d000 tx=0x7fb7d40ae310).stop 2021-07-11T04:01:49.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 -- 172.21.2.104:0/3411418336 shutdown_connections 2021-07-11T04:01:49.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7d4064270 0x7fb7d413b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:49.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7d4143e20 0x7fb7d4063d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:49.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb7b8053300 0x7fb7b80557c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:49.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 --2- 172.21.2.104:0/3411418336 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb7b40040b0 0x7fb7d4143650 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:49.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 -- 172.21.2.104:0/3411418336 >> 172.21.2.104:0/3411418336 conn(0x7fb7d4062560 msgr2=0x7fb7d40b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:49.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 -- 172.21.2.104:0/3411418336 shutdown_connections 2021-07-11T04:01:49.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.660+0000 7fb7d953e700 1 -- 172.21.2.104:0/3411418336 wait complete. 2021-07-11T04:01:49.665 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T04:01:49.671 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T04:01:49.672 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T04:01:49.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 Processor -- start 2021-07-11T04:01:49.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 -- start start 2021-07-11T04:01:49.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7faa040b04a0 0x7faa040add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:49.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faa040aa120 0x7faa040ae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:49.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa040b0880 0x7faa04143710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:49.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa040b2af0 con 0x7faa040aa120 2021-07-11T04:01:49.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa040b2770 con 0x7faa040b04a0 2021-07-11T04:01:49.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa040b2d20 con 0x7faa040b0880 2021-07-11T04:01:49.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa02ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faa040aa120 0x7faa040ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:49.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa02ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faa040aa120 0x7faa040ae220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50152/0 (socket says 172.21.2.104:50152) 2021-07-11T04:01:49.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa02ffd700 1 -- 172.21.2.104:0/3713099635 learned_addr learned my addr 172.21.2.104:0/3713099635 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:49.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa02ffd700 1 -- 172.21.2.104:0/3713099635 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa040b0880 msgr2=0x7faa04143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:49.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa02ffd700 1 --2- 172.21.2.104:0/3713099635 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa040b0880 0x7faa04143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:49.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa02ffd700 1 -- 172.21.2.104:0/3713099635 >> v2:172.21.2.105:3300/0 conn(0x7faa040b04a0 msgr2=0x7faa040add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:49.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa02ffd700 1 --2- 172.21.2.104:0/3713099635 >> v2:172.21.2.105:3300/0 conn(0x7faa040b04a0 0x7faa040add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:49.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa02ffd700 1 -- 172.21.2.104:0/3713099635 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa040a9b70 con 0x7faa040aa120 2021-07-11T04:01:49.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa02ffd700 1 --2- 172.21.2.104:0/3713099635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa040aa120 0x7faa040ae220 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7fa9ec002f70 tx=0x7fa9ec006f40).ready entity=mon.0 client_cookie=56e9a0fbaceadc98 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:49.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa027fc700 1 -- 172.21.2.104:0/3713099635 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ec00da60 con 0x7faa040aa120 2021-07-11T04:01:49.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa027fc700 1 -- 172.21.2.104:0/3713099635 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec000e90 con 0x7faa040aa120 2021-07-11T04:01:49.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa027fc700 1 -- 172.21.2.104:0/3713099635 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9ec00ddf0 con 0x7faa040aa120 2021-07-11T04:01:49.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 -- 172.21.2.104:0/3713099635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa040aa120 msgr2=0x7faa040ae220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:49.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 --2- 172.21.2.104:0/3713099635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa040aa120 0x7faa040ae220 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7fa9ec002f70 tx=0x7fa9ec006f40).stop 2021-07-11T04:01:49.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 -- 172.21.2.104:0/3713099635 shutdown_connections 2021-07-11T04:01:49.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 --2- 172.21.2.104:0/3713099635 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa040b0880 0x7faa04143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:49.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 --2- 172.21.2.104:0/3713099635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa040aa120 0x7faa040ae220 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:49.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 --2- 172.21.2.104:0/3713099635 >> v2:172.21.2.105:3300/0 conn(0x7faa040b04a0 0x7faa040add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:49.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 -- 172.21.2.104:0/3713099635 >> 172.21.2.104:0/3713099635 conn(0x7faa040607c0 msgr2=0x7faa04060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:49.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 -- 172.21.2.104:0/3713099635 shutdown_connections 2021-07-11T04:01:49.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 -- 172.21.2.104:0/3713099635 wait complete. 2021-07-11T04:01:49.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 Processor -- start 2021-07-11T04:01:49.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.772+0000 7faa0a1e7700 1 -- start start 2021-07-11T04:01:49.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa0a1e7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa040aa120 0x7faa041440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:49.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa0a1e7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faa040b04a0 0x7faa04141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:49.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa0a1e7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa040b0880 0x7faa04141e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:49.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa0a1e7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa04067080 con 0x7faa040aa120 2021-07-11T04:01:49.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa0a1e7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa04066ac0 con 0x7faa040b04a0 2021-07-11T04:01:49.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa0a1e7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa040672b0 con 0x7faa040b0880 2021-07-11T04:01:49.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa037fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa040aa120 0x7faa041440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:49.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa037fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa040aa120 0x7faa041440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50154/0 (socket says 172.21.2.104:50154) 2021-07-11T04:01:49.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa037fe700 1 -- 172.21.2.104:0/1674219729 learned_addr learned my addr 172.21.2.104:0/1674219729 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:49.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa037fe700 1 -- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa040b0880 msgr2=0x7faa04141e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:49.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa037fe700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa040b0880 0x7faa04141e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:49.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa037fe700 1 -- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faa040b04a0 msgr2=0x7faa04141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:49.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa037fe700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faa040b04a0 0x7faa04141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:49.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa037fe700 1 -- 172.21.2.104:0/1674219729 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ec00b460 con 0x7faa040aa120 2021-07-11T04:01:49.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa037fe700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa040aa120 0x7faa041440b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7fa9f4002f70 tx=0x7fa9f4006ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:49.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa00ff9700 1 -- 172.21.2.104:0/1674219729 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9f400f510 con 0x7faa040aa120 2021-07-11T04:01:49.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa00ff9700 1 -- 172.21.2.104:0/1674219729 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f400f670 con 0x7faa040aa120 2021-07-11T04:01:49.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa0a1e7700 1 -- 172.21.2.104:0/1674219729 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa04142340 con 0x7faa040aa120 2021-07-11T04:01:49.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa00ff9700 1 -- 172.21.2.104:0/1674219729 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9f400f9d0 con 0x7faa040aa120 2021-07-11T04:01:49.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa0a1e7700 1 -- 172.21.2.104:0/1674219729 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa040659a0 con 0x7faa040aa120 2021-07-11T04:01:49.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa0a1e7700 1 -- 172.21.2.104:0/1674219729 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa04057350 con 0x7faa040aa120 2021-07-11T04:01:49.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa00ff9700 1 -- 172.21.2.104:0/1674219729 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa9f4018030 con 0x7faa040aa120 2021-07-11T04:01:49.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa00ff9700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9e8053300 0x7fa9e80557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:49.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa00ff9700 1 -- 172.21.2.104:0/1674219729 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa9f406a7e0 con 0x7faa040aa120 2021-07-11T04:01:49.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa02ffd700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9e8053300 0x7fa9e80557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:49.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.776+0000 7faa02ffd700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9e8053300 0x7fa9e80557c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fa9ec002f70 tx=0x7fa9ec01a000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:49.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.780+0000 7faa00ff9700 1 -- 172.21.2.104:0/1674219729 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa9f4041df0 con 0x7faa040aa120 2021-07-11T04:01:49.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.888+0000 7faa0a1e7700 1 -- 172.21.2.104:0/1674219729 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7faa04044650 con 0x7faa040aa120 2021-07-11T04:01:49.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.888+0000 7faa00ff9700 1 -- 172.21.2.104:0/1674219729 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fa9f4045410 con 0x7faa040aa120 2021-07-11T04:01:49.889 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:49.890 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:01:49.890 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T04:01:49.890 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.890 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T04:01:49.891 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T04:01:49.891 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.891 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:01:49.891 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.892 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.892 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.892 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.892 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.892 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.893 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T04:01:49.893 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T04:01:49.893 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.893 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:01:49.894 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.894 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.894 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.894 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.895 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.895 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.895 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T04:01:49.895 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T04:01:49.895 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.896 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:01:49.896 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.896 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.896 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.897 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.897 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.897 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.897 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T04:01:49.897 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T04:01:49.898 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.898 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.898 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.898 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.899 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.899 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.899 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.899 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T04:01:49.899 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T04:01:49.900 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.900 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.900 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.900 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.901 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.901 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.901 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.901 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T04:01:49.901 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T04:01:49.902 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.902 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.902 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.902 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.902 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.903 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.903 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.903 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T04:01:49.903 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T04:01:49.904 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.904 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.904 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.904 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.904 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.905 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.905 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.905 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T04:01:49.905 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T04:01:49.905 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.906 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.906 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.906 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.906 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.907 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.907 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.908 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T04:01:49.908 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T04:01:49.908 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.908 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.909 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.909 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.909 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.909 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.909 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.910 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T04:01:49.910 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T04:01:49.910 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.910 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.911 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.911 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.911 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.911 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.911 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.912 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T04:01:49.912 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T04:01:49.912 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.912 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.912 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.913 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.913 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.913 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.913 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.914 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T04:01:49.914 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T04:01:49.914 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.914 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:01:49.915 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:01:49.915 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:01:49.915 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.915 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.915 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.916 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T04:01:49.916 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T04:01:49.916 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.916 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:01:49.916 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:01:49.917 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:01:49.917 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.917 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.917 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.917 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T04:01:49.918 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T04:01:49.918 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.918 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:01:49.918 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:01:49.919 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:01:49.919 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.919 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.919 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.919 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.920 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T04:01:49.920 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T04:01:49.920 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.920 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:01:49.921 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:01:49.921 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.921 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.921 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:49.921 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:49.922 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T04:01:49.922 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T04:01:49.922 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:01:49.922 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:01:49.922 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:01:49.923 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:01:49.923 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.923 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:49.923 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:01:49.924 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:01:49.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 -- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9e8053300 msgr2=0x7fa9e80557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:49.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9e8053300 0x7fa9e80557c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fa9ec002f70 tx=0x7fa9ec01a000).stop 2021-07-11T04:01:49.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 -- 172.21.2.104:0/1674219729 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa040aa120 msgr2=0x7faa041440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:49.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa040aa120 0x7faa041440b0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7fa9f4002f70 tx=0x7fa9f4006ed0).stop 2021-07-11T04:01:49.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 -- 172.21.2.104:0/1674219729 shutdown_connections 2021-07-11T04:01:49.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa040b0880 0x7faa04141e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:49.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faa040b04a0 0x7faa04141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:49.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa9e8053300 0x7fa9e80557c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:49.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 --2- 172.21.2.104:0/1674219729 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa040aa120 0x7faa041440b0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:49.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 -- 172.21.2.104:0/1674219729 >> 172.21.2.104:0/1674219729 conn(0x7faa040607c0 msgr2=0x7faa040b5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:49.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 -- 172.21.2.104:0/1674219729 shutdown_connections 2021-07-11T04:01:49.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.892+0000 7faa0a1e7700 1 -- 172.21.2.104:0/1674219729 wait complete. 2021-07-11T04:01:49.928 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:49.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd751240700 1 Processor -- start 2021-07-11T04:01:49.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd751240700 1 -- start start 2021-07-11T04:01:49.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd751240700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd74c0b7810 0x7fd74c1445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:49.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd751240700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74c0b1170 0x7fd74c144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:49.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd751240700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74c0b31d0 con 0x7fd74c064870 2021-07-11T04:01:49.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd751240700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74c0b7bf0 con 0x7fd74c0b7810 2021-07-11T04:01:49.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd751240700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c0675c0 con 0x7fd74c0b1170 2021-07-11T04:01:49.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd74b59e700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fd74c064870 0x7fd74c064cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46930/0 (socket says 172.21.2.104:46930) 2021-07-11T04:01:49.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd74b59e700 1 -- 172.21.2.104:0/2993768575 learned_addr learned my addr 172.21.2.104:0/2993768575 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:49.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd74ad9d700 1 --2- 172.21.2.104:0/2993768575 >> v2:172.21.2.104:3300/0 conn(0x7fd74c0b7810 0x7fd74c1445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:49.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd749d9b700 1 -- 172.21.2.104:0/2993768575 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123823593 0 0) 0x7fd74c0b31d0 con 0x7fd74c064870 2021-07-11T04:01:49.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.980+0000 7fd749d9b700 1 -- 172.21.2.104:0/2993768575 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd72c003260 con 0x7fd74c064870 2021-07-11T04:01:49.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 -- 172.21.2.104:0/2993768575 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74c0b1170 msgr2=0x7fd74c144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:49.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 --2- 172.21.2.104:0/2993768575 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74c0b1170 0x7fd74c144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:49.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 -- 172.21.2.104:0/2993768575 >> v1:172.21.2.104:6789/0 conn(0x7fd74c064870 legacy=0x7fd74c064cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:49.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 -- 172.21.2.104:0/2993768575 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c0aae70 con 0x7fd74c0b7810 2021-07-11T04:01:49.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 --2- 172.21.2.104:0/2993768575 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd74c0b7810 0x7fd74c1445a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7fd73c002f70 tx=0x7fd73c006df0).ready entity=mon.0 client_cookie=bb84c3282440f0bf server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:49.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd749d9b700 1 -- 172.21.2.104:0/2993768575 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd73c00d570 con 0x7fd74c0b7810 2021-07-11T04:01:49.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd749d9b700 1 -- 172.21.2.104:0/2993768575 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c000e90 con 0x7fd74c0b7810 2021-07-11T04:01:49.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd749d9b700 1 -- 172.21.2.104:0/2993768575 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd73c00d8c0 con 0x7fd74c0b7810 2021-07-11T04:01:49.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 -- 172.21.2.104:0/2993768575 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd74c0b7810 msgr2=0x7fd74c1445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:49.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 --2- 172.21.2.104:0/2993768575 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd74c0b7810 0x7fd74c1445a0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7fd73c002f70 tx=0x7fd73c006df0).stop 2021-07-11T04:01:49.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 -- 172.21.2.104:0/2993768575 shutdown_connections 2021-07-11T04:01:49.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 --2- 172.21.2.104:0/2993768575 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74c0b1170 0x7fd74c144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:49.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 --2- 172.21.2.104:0/2993768575 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd74c0b7810 0x7fd74c1445a0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:49.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 -- 172.21.2.104:0/2993768575 >> 172.21.2.104:0/2993768575 conn(0x7fd74c05f6b0 msgr2=0x7fd74c05fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:49.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 -- 172.21.2.104:0/2993768575 shutdown_connections 2021-07-11T04:01:49.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 -- 172.21.2.104:0/2993768575 wait complete. 2021-07-11T04:01:50.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 Processor -- start 2021-07-11T04:01:50.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 -- start start 2021-07-11T04:01:50.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd74c064870 0x7fd74c0af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74c0b1170 0x7fd74c143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd74c0b7810 0x7fd74c141bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c061810 con 0x7fd74c064870 2021-07-11T04:01:50.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c1563b0 con 0x7fd74c0b7810 2021-07-11T04:01:50.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c061c70 con 0x7fd74c0b1170 2021-07-11T04:01:50.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd74c064870 0x7fd74c0af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd74c064870 0x7fd74c0af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50164/0 (socket says 172.21.2.104:50164) 2021-07-11T04:01:50.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 -- 172.21.2.104:0/861147916 learned_addr learned my addr 172.21.2.104:0/861147916 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:50.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 -- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74c0b1170 msgr2=0x7fd74c143cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74c0b1170 0x7fd74c143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 -- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd74c0b7810 msgr2=0x7fd74c141bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd74c0b7810 0x7fd74c141bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 -- 172.21.2.104:0/861147916 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd73c00b180 con 0x7fd74c064870 2021-07-11T04:01:50.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd74ad9d700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd74c064870 0x7fd74c0af930 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7fd73c002f70 tx=0x7fd73c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd743fff700 1 -- 172.21.2.104:0/861147916 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd73c012be0 con 0x7fd74c064870 2021-07-11T04:01:50.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd743fff700 1 -- 172.21.2.104:0/861147916 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73c012d40 con 0x7fd74c064870 2021-07-11T04:01:50.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 -- 172.21.2.104:0/861147916 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74c0aff60 con 0x7fd74c064870 2021-07-11T04:01:50.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd751240700 1 -- 172.21.2.104:0/861147916 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74c1565a0 con 0x7fd74c064870 2021-07-11T04:01:50.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.984+0000 7fd743fff700 1 -- 172.21.2.104:0/861147916 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd73c01e620 con 0x7fd74c064870 2021-07-11T04:01:50.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.988+0000 7fd751240700 1 -- 172.21.2.104:0/861147916 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74c156760 con 0x7fd74c064870 2021-07-11T04:01:50.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.988+0000 7fd743fff700 1 -- 172.21.2.104:0/861147916 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd73c024020 con 0x7fd74c064870 2021-07-11T04:01:50.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.988+0000 7fd743fff700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7300532b0 0x7fd730055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.988+0000 7fd743fff700 1 -- 172.21.2.104:0/861147916 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd73c06c4c0 con 0x7fd74c064870 2021-07-11T04:01:50.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.988+0000 7fd74a59c700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7300532b0 0x7fd730055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.988+0000 7fd74a59c700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7300532b0 0x7fd730055770 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fd734002f70 tx=0x7fd734006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:49.988+0000 7fd743fff700 1 -- 172.21.2.104:0/861147916 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd73c043a20 con 0x7fd74c064870 2021-07-11T04:01:50.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 -- 172.21.2.104:0/861147916 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd74c156760 con 0x7fd74c064870 2021-07-11T04:01:50.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd743fff700 1 -- 172.21.2.104:0/861147916 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+13191 (secure 0 0 0) 0x7fd73c047040 con 0x7fd74c064870 2021-07-11T04:01:50.072 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:50.072 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":135,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:49.648605+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:50.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 -- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7300532b0 msgr2=0x7fd730055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7300532b0 0x7fd730055770 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fd734002f70 tx=0x7fd734006a80).stop 2021-07-11T04:01:50.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 -- 172.21.2.104:0/861147916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd74c064870 msgr2=0x7fd74c0af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd74c064870 0x7fd74c0af930 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7fd73c002f70 tx=0x7fd73c014000).stop 2021-07-11T04:01:50.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 -- 172.21.2.104:0/861147916 shutdown_connections 2021-07-11T04:01:50.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd74c0b7810 0x7fd74c141bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd74c0b1170 0x7fd74c143cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd7300532b0 0x7fd730055770 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 --2- 172.21.2.104:0/861147916 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd74c064870 0x7fd74c0af930 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 -- 172.21.2.104:0/861147916 >> 172.21.2.104:0/861147916 conn(0x7fd74c05f6b0 msgr2=0x7fd74c0b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:50.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 -- 172.21.2.104:0/861147916 shutdown_connections 2021-07-11T04:01:50.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.072+0000 7fd751240700 1 -- 172.21.2.104:0/861147916 wait complete. 2021-07-11T04:01:50.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T04:01:50.080 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:01:50.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6eca9700 1 Processor -- start 2021-07-11T04:01:50.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6eca9700 1 -- start start 2021-07-11T04:01:50.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6eca9700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8b680a9ec0 0x7f8b680aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6eca9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b680603d0 0x7f8b6813ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6eca9700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b68061990 con 0x7f8b6813d1c0 2021-07-11T04:01:50.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6eca9700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b68061c30 con 0x7f8b680a9ec0 2021-07-11T04:01:50.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6eca9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b68060b90 con 0x7f8b680603d0 2021-07-11T04:01:50.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6d246700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8b6813d1c0 0x7f8b6813f790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46940/0 (socket says 172.21.2.104:46940) 2021-07-11T04:01:50.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6d246700 1 -- 172.21.2.104:0/1290707958 learned_addr learned my addr 172.21.2.104:0/1290707958 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:50.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6ca45700 1 --2- 172.21.2.104:0/1290707958 >> v2:172.21.2.104:3300/0 conn(0x7f8b680a9ec0 0x7f8b680aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6ca45700 1 -- 172.21.2.104:0/1290707958 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b680603d0 msgr2=0x7f8b6813ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:50.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6ca45700 1 --2- 172.21.2.104:0/1290707958 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b680603d0 0x7f8b6813ccd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6ca45700 1 -- 172.21.2.104:0/1290707958 >> v1:172.21.2.104:6789/0 conn(0x7f8b6813d1c0 legacy=0x7f8b6813f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6ca45700 1 -- 172.21.2.104:0/1290707958 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b68060e30 con 0x7f8b680a9ec0 2021-07-11T04:01:50.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6ca45700 1 --2- 172.21.2.104:0/1290707958 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b680a9ec0 0x7f8b680aa2a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f8b6805f870 tx=0x7f8b58007f00).ready entity=mon.0 client_cookie=e828b030edefc170 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b677fe700 1 -- 172.21.2.104:0/1290707958 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b58010510 con 0x7f8b680a9ec0 2021-07-11T04:01:50.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b677fe700 1 -- 172.21.2.104:0/1290707958 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b58000e90 con 0x7f8b680a9ec0 2021-07-11T04:01:50.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b677fe700 1 -- 172.21.2.104:0/1290707958 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b58010670 con 0x7f8b680a9ec0 2021-07-11T04:01:50.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6eca9700 1 -- 172.21.2.104:0/1290707958 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b680a9ec0 msgr2=0x7f8b680aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.132+0000 7f8b6eca9700 1 --2- 172.21.2.104:0/1290707958 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b680a9ec0 0x7f8b680aa2a0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f8b6805f870 tx=0x7f8b58007f00).stop 2021-07-11T04:01:50.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 -- 172.21.2.104:0/1290707958 shutdown_connections 2021-07-11T04:01:50.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 --2- 172.21.2.104:0/1290707958 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b680603d0 0x7f8b6813ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 --2- 172.21.2.104:0/1290707958 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b680a9ec0 0x7f8b680aa2a0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 -- 172.21.2.104:0/1290707958 >> 172.21.2.104:0/1290707958 conn(0x7f8b680b2970 msgr2=0x7f8b680b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:50.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 -- 172.21.2.104:0/1290707958 shutdown_connections 2021-07-11T04:01:50.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 -- 172.21.2.104:0/1290707958 wait complete. 2021-07-11T04:01:50.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 Processor -- start 2021-07-11T04:01:50.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 -- start start 2021-07-11T04:01:50.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b680603d0 0x7f8b68143550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b680a9ec0 0x7f8b6813d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b6813d1c0 0x7f8b6813db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b68139f40 con 0x7f8b680603d0 2021-07-11T04:01:50.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b68139720 con 0x7f8b680a9ec0 2021-07-11T04:01:50.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b68142920 con 0x7f8b6813d1c0 2021-07-11T04:01:50.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6ca45700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b680603d0 0x7f8b68143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6ca45700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b680603d0 0x7f8b68143550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50178/0 (socket says 172.21.2.104:50178) 2021-07-11T04:01:50.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6ca45700 1 -- 172.21.2.104:0/424058130 learned_addr learned my addr 172.21.2.104:0/424058130 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:50.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6ca45700 1 -- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b6813d1c0 msgr2=0x7f8b6813db20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:50.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6ca45700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b6813d1c0 0x7f8b6813db20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6ca45700 1 -- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b680a9ec0 msgr2=0x7f8b6813d630 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:50.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6ca45700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b680a9ec0 0x7f8b6813d630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6ca45700 1 -- 172.21.2.104:0/424058130 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5800c200 con 0x7f8b680603d0 2021-07-11T04:01:50.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6ca45700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b680603d0 0x7f8b68143550 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f8b68142ab0 tx=0x7f8b58017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b65ffb700 1 -- 172.21.2.104:0/424058130 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b58007540 con 0x7f8b680603d0 2021-07-11T04:01:50.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b65ffb700 1 -- 172.21.2.104:0/424058130 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b580056f0 con 0x7f8b680603d0 2021-07-11T04:01:50.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 -- 172.21.2.104:0/424058130 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6813e010 con 0x7f8b680603d0 2021-07-11T04:01:50.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b65ffb700 1 -- 172.21.2.104:0/424058130 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8b58005a50 con 0x7f8b680603d0 2021-07-11T04:01:50.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 -- 172.21.2.104:0/424058130 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6813ffc0 con 0x7f8b680603d0 2021-07-11T04:01:50.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b6eca9700 1 -- 172.21.2.104:0/424058130 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b681398e0 con 0x7f8b680603d0 2021-07-11T04:01:50.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b65ffb700 1 -- 172.21.2.104:0/424058130 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8b58029070 con 0x7f8b680603d0 2021-07-11T04:01:50.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b65ffb700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b4c053210 0x7f8b4c0556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b65ffb700 1 -- 172.21.2.104:0/424058130 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8b5806bc40 con 0x7f8b680603d0 2021-07-11T04:01:50.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.136+0000 7f8b67fff700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b4c053210 0x7f8b4c0556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.140+0000 7f8b67fff700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b4c053210 0x7f8b4c0556d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f8b50002fb0 tx=0x7f8b50006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.140+0000 7f8b65ffb700 1 -- 172.21.2.104:0/424058130 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8b681398e0 con 0x7f8b680603d0 2021-07-11T04:01:50.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 -- 172.21.2.104:0/424058130 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8b681398e0 con 0x7f8b680603d0 2021-07-11T04:01:50.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b65ffb700 1 -- 172.21.2.104:0/424058130 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v135) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8b681398e0 con 0x7f8b680603d0 2021-07-11T04:01:50.223 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:01:50.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 -- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b4c053210 msgr2=0x7f8b4c0556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b4c053210 0x7f8b4c0556d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f8b50002fb0 tx=0x7f8b50006930).stop 2021-07-11T04:01:50.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 -- 172.21.2.104:0/424058130 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b680603d0 msgr2=0x7f8b68143550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b680603d0 0x7f8b68143550 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f8b68142ab0 tx=0x7f8b58017000).stop 2021-07-11T04:01:50.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 -- 172.21.2.104:0/424058130 shutdown_connections 2021-07-11T04:01:50.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8b6813d1c0 0x7f8b6813db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8b680a9ec0 0x7f8b6813d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8b4c053210 0x7f8b4c0556d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 --2- 172.21.2.104:0/424058130 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8b680603d0 0x7f8b68143550 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 -- 172.21.2.104:0/424058130 >> 172.21.2.104:0/424058130 conn(0x7f8b680b2970 msgr2=0x7f8b6813a470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:50.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 -- 172.21.2.104:0/424058130 shutdown_connections 2021-07-11T04:01:50.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.224+0000 7f8b6eca9700 1 -- 172.21.2.104:0/424058130 wait complete. 2021-07-11T04:01:50.230 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:01:50.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 Processor -- start 2021-07-11T04:01:50.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 -- start start 2021-07-11T04:01:50.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7fb00b12d0 0x7f7fb00affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f7fb00b22a0 0x7f7fb0141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb00a77d0 con 0x7f7fb00b1ec0 2021-07-11T04:01:50.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb00b1740 con 0x7f7fb00b12d0 2021-07-11T04:01:50.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb00b0650 con 0x7f7fb00b22a0 2021-07-11T04:01:50.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7faf7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7fb00b12d0 0x7f7fb00affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7faf7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7fb00b12d0 0x7f7fb00affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50190/0 (socket says 172.21.2.104:50190) 2021-07-11T04:01:50.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7faf7fe700 1 -- 172.21.2.104:0/1916150363 learned_addr learned my addr 172.21.2.104:0/1916150363 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:50.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7faf7fe700 1 -- 172.21.2.104:0/1916150363 >> v2:172.21.2.105:3300/0 conn(0x7f7fb00b22a0 msgr2=0x7f7fb0141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:50.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7faf7fe700 1 --2- 172.21.2.104:0/1916150363 >> v2:172.21.2.105:3300/0 conn(0x7f7fb00b22a0 0x7f7fb0141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7faf7fe700 1 -- 172.21.2.104:0/1916150363 >> v1:172.21.2.105:6789/0 conn(0x7f7fb00b1ec0 legacy=0x7f7fb00a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:50.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7faf7fe700 1 -- 172.21.2.104:0/1916150363 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb00b08f0 con 0x7f7fb00b12d0 2021-07-11T04:01:50.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7faf7fe700 1 --2- 172.21.2.104:0/1916150363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fb00b12d0 0x7f7fb00affb0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f7fb00ad2d0 tx=0x7f7fa0006eb0).ready entity=mon.0 client_cookie=eee4e0542cf6d69 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7faeffd700 1 -- 172.21.2.104:0/1916150363 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa000d5c0 con 0x7f7fb00b12d0 2021-07-11T04:01:50.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7faeffd700 1 -- 172.21.2.104:0/1916150363 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa0000e90 con 0x7f7fb00b12d0 2021-07-11T04:01:50.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7faeffd700 1 -- 172.21.2.104:0/1916150363 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa000d910 con 0x7f7fb00b12d0 2021-07-11T04:01:50.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 -- 172.21.2.104:0/1916150363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fb00b12d0 msgr2=0x7f7fb00affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 --2- 172.21.2.104:0/1916150363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fb00b12d0 0x7f7fb00affb0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f7fb00ad2d0 tx=0x7f7fa0006eb0).stop 2021-07-11T04:01:50.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 -- 172.21.2.104:0/1916150363 shutdown_connections 2021-07-11T04:01:50.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 --2- 172.21.2.104:0/1916150363 >> v2:172.21.2.105:3300/0 conn(0x7f7fb00b22a0 0x7f7fb0141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 --2- 172.21.2.104:0/1916150363 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fb00b12d0 0x7f7fb00affb0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 -- 172.21.2.104:0/1916150363 >> 172.21.2.104:0/1916150363 conn(0x7f7fb005f6b0 msgr2=0x7f7fb005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:50.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 -- 172.21.2.104:0/1916150363 shutdown_connections 2021-07-11T04:01:50.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.284+0000 7f7fb6693700 1 -- 172.21.2.104:0/1916150363 wait complete. 2021-07-11T04:01:50.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb6693700 1 Processor -- start 2021-07-11T04:01:50.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb6693700 1 -- start start 2021-07-11T04:01:50.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb6693700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fb00b12d0 0x7f7fb00aa780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb6693700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7fb00b1ec0 0x7f7fb014aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb6693700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fb00b22a0 0x7f7fb014afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb6693700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0148060 con 0x7f7fb00b22a0 2021-07-11T04:01:50.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb6693700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb00ac150 con 0x7f7fb00b1ec0 2021-07-11T04:01:50.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb6693700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb0148290 con 0x7f7fb00b12d0 2021-07-11T04:01:50.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb4c30700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fb00b22a0 0x7f7fb014afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb4c30700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fb00b22a0 0x7f7fb014afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50196/0 (socket says 172.21.2.104:50196) 2021-07-11T04:01:50.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb4c30700 1 -- 172.21.2.104:0/3978625043 learned_addr learned my addr 172.21.2.104:0/3978625043 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:50.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb4c30700 1 -- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fb00b12d0 msgr2=0x7f7fb00aa780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb4c30700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fb00b12d0 0x7f7fb00aa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb4c30700 1 -- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7fb00b1ec0 msgr2=0x7f7fb014aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb4c30700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7fb00b1ec0 0x7f7fb014aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb4c30700 1 -- 172.21.2.104:0/3978625043 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa000b1e0 con 0x7f7fb00b22a0 2021-07-11T04:01:50.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb4c30700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fb00b22a0 0x7f7fb014afc0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f7fa80032f0 tx=0x7f7fa80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fad7fa700 1 -- 172.21.2.104:0/3978625043 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa800f760 con 0x7f7fb00b22a0 2021-07-11T04:01:50.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fad7fa700 1 -- 172.21.2.104:0/3978625043 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa800f8c0 con 0x7f7fb00b22a0 2021-07-11T04:01:50.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb6693700 1 -- 172.21.2.104:0/3978625043 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb0148530 con 0x7f7fb00b22a0 2021-07-11T04:01:50.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fad7fa700 1 -- 172.21.2.104:0/3978625043 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fa800fc20 con 0x7f7fb00b22a0 2021-07-11T04:01:50.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb6693700 1 -- 172.21.2.104:0/3978625043 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb00ab3c0 con 0x7f7fb00b22a0 2021-07-11T04:01:50.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fb6693700 1 -- 172.21.2.104:0/3978625043 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb01486c0 con 0x7f7fb00b22a0 2021-07-11T04:01:50.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fad7fa700 1 -- 172.21.2.104:0/3978625043 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f7fa801a460 con 0x7f7fb00b22a0 2021-07-11T04:01:50.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fad7fa700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7f940531f0 0x7f7f940556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7fad7fa700 1 -- 172.21.2.104:0/3978625043 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7fa806b5f0 con 0x7f7fb00b22a0 2021-07-11T04:01:50.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.288+0000 7f7faffff700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7f940531f0 0x7f7f940556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.292+0000 7f7faffff700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7f940531f0 0x7f7f940556b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f7f98000f70 tx=0x7f7f980077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.292+0000 7f7fad7fa700 1 -- 172.21.2.104:0/3978625043 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7fa8042c00 con 0x7f7fb00b22a0 2021-07-11T04:01:50.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 -- 172.21.2.104:0/3978625043 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7fb0057350 con 0x7f7fb00b22a0 2021-07-11T04:01:50.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fad7fa700 1 -- 172.21.2.104:0/3978625043 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v135) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7fa8046220 con 0x7f7fb00b22a0 2021-07-11T04:01:50.373 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:50.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 -- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7f940531f0 msgr2=0x7f7f940556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7f940531f0 0x7f7f940556b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f7f98000f70 tx=0x7f7f980077f0).stop 2021-07-11T04:01:50.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 -- 172.21.2.104:0/3978625043 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fb00b22a0 msgr2=0x7f7fb014afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fb00b22a0 0x7f7fb014afc0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f7fa80032f0 tx=0x7f7fa80091d0).stop 2021-07-11T04:01:50.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 -- 172.21.2.104:0/3978625043 shutdown_connections 2021-07-11T04:01:50.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7f940531f0 0x7f7f940556b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fb00b22a0 0x7f7fb014afc0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7fb00b1ec0 0x7f7fb014aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 --2- 172.21.2.104:0/3978625043 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fb00b12d0 0x7f7fb00aa780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 -- 172.21.2.104:0/3978625043 >> 172.21.2.104:0/3978625043 conn(0x7f7fb005f6b0 msgr2=0x7f7fb00af670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:50.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 -- 172.21.2.104:0/3978625043 shutdown_connections 2021-07-11T04:01:50.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.372+0000 7f7fb6693700 1 -- 172.21.2.104:0/3978625043 wait complete. 2021-07-11T04:01:50.380 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T04:01:50.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 Processor -- start 2021-07-11T04:01:50.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 -- start start 2021-07-11T04:01:50.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4b6c0b1db0 0x7f4b6c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f4b6c1483a0 0x7f4b6c0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b6c0b0350 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6c0a90c0 con 0x7f4b6c0b1db0 2021-07-11T04:01:50.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6c0b05f0 con 0x7f4b6c1483a0 2021-07-11T04:01:50.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b70e7c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4b6c0a70c0 0x7f4b6c144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46972/0 (socket says 172.21.2.104:46972) 2021-07-11T04:01:50.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b70e7c700 1 -- 172.21.2.104:0/704630374 learned_addr learned my addr 172.21.2.104:0/704630374 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:50.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b6bfff700 1 --2- 172.21.2.104:0/704630374 >> v2:172.21.2.104:3300/0 conn(0x7f4b6c0b1db0 0x7f4b6c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b6affd700 1 -- 172.21.2.104:0/704630374 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987933337 0 0) 0x7f4b6c0b0350 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b6affd700 1 -- 172.21.2.104:0/704630374 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b4c003260 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b6bfff700 1 -- 172.21.2.104:0/704630374 >> v2:172.21.2.105:3300/0 conn(0x7f4b6c1483a0 msgr2=0x7f4b6c0b2190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:50.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b6bfff700 1 --2- 172.21.2.104:0/704630374 >> v2:172.21.2.105:3300/0 conn(0x7f4b6c1483a0 0x7f4b6c0b2190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b6bfff700 1 -- 172.21.2.104:0/704630374 >> v1:172.21.2.104:6789/0 conn(0x7f4b6c0a70c0 legacy=0x7f4b6c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b6bfff700 1 -- 172.21.2.104:0/704630374 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6c0b0890 con 0x7f4b6c0b1db0 2021-07-11T04:01:50.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b6bfff700 1 --2- 172.21.2.104:0/704630374 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b6c0b1db0 0x7f4b6c0b0fd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f4b6c147730 tx=0x7f4b5c006df0).ready entity=mon.0 client_cookie=d0263ee2bd3f7af8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b6affd700 1 -- 172.21.2.104:0/704630374 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b5c00d5c0 con 0x7f4b6c0b1db0 2021-07-11T04:01:50.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b6affd700 1 -- 172.21.2.104:0/704630374 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c000e90 con 0x7f4b6c0b1db0 2021-07-11T04:01:50.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b6affd700 1 -- 172.21.2.104:0/704630374 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b5c00d950 con 0x7f4b6c0b1db0 2021-07-11T04:01:50.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 -- 172.21.2.104:0/704630374 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b6c0b1db0 msgr2=0x7f4b6c0b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 --2- 172.21.2.104:0/704630374 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b6c0b1db0 0x7f4b6c0b0fd0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f4b6c147730 tx=0x7f4b5c006df0).stop 2021-07-11T04:01:50.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 -- 172.21.2.104:0/704630374 shutdown_connections 2021-07-11T04:01:50.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 --2- 172.21.2.104:0/704630374 >> v2:172.21.2.105:3300/0 conn(0x7f4b6c1483a0 0x7f4b6c0b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 --2- 172.21.2.104:0/704630374 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b6c0b1db0 0x7f4b6c0b0fd0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 -- 172.21.2.104:0/704630374 >> 172.21.2.104:0/704630374 conn(0x7f4b6c05f6b0 msgr2=0x7f4b6c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:50.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 -- 172.21.2.104:0/704630374 shutdown_connections 2021-07-11T04:01:50.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 -- 172.21.2.104:0/704630374 wait complete. 2021-07-11T04:01:50.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 Processor -- start 2021-07-11T04:01:50.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.432+0000 7f4b728df700 1 -- start start 2021-07-11T04:01:50.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b728df700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b6c0a70c0 0x7f4b6c0b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b728df700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b6c0b1db0 0x7f4b6c1469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b728df700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b6c1483a0 0x7f4b6c145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b728df700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c0a9860 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b728df700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c1455a0 con 0x7f4b6c0b1db0 2021-07-11T04:01:50.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b728df700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6c0aab70 con 0x7f4b6c1483a0 2021-07-11T04:01:50.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b6bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b6c0a70c0 0x7f4b6c0b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b6bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b6c0a70c0 0x7f4b6c0b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50206/0 (socket says 172.21.2.104:50206) 2021-07-11T04:01:50.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b6bfff700 1 -- 172.21.2.104:0/4239624208 learned_addr learned my addr 172.21.2.104:0/4239624208 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:50.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b6bfff700 1 -- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b6c1483a0 msgr2=0x7f4b6c145b60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:50.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b6bfff700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b6c1483a0 0x7f4b6c145b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b6bfff700 1 -- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b6c0b1db0 msgr2=0x7f4b6c1469e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:50.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b6bfff700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b6c0b1db0 0x7f4b6c1469e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b6bfff700 1 -- 172.21.2.104:0/4239624208 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c00b180 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b6bfff700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b6c0a70c0 0x7f4b6c0b3640 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f4b6c0a9c10 tx=0x7f4b5c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b697fa700 1 -- 172.21.2.104:0/4239624208 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b5c012a30 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b697fa700 1 -- 172.21.2.104:0/4239624208 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c012b90 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b728df700 1 -- 172.21.2.104:0/4239624208 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b6c0a95c0 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b728df700 1 -- 172.21.2.104:0/4239624208 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6c146050 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b697fa700 1 -- 172.21.2.104:0/4239624208 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b5c01e430 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b728df700 1 -- 172.21.2.104:0/4239624208 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b6c0aace0 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b697fa700 1 -- 172.21.2.104:0/4239624208 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4b5c00f030 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b697fa700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b500532d0 0x7f4b50055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b697fa700 1 -- 172.21.2.104:0/4239624208 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4b5c06c1e0 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b6b7fe700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b500532d0 0x7f4b50055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.436+0000 7f4b6b7fe700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b500532d0 0x7f4b50055790 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f4b54002ef0 tx=0x7f4b54006890).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.440+0000 7f4b697fa700 1 -- 172.21.2.104:0/4239624208 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4b6c0aace0 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 -- 172.21.2.104:0/4239624208 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4b6c0aace0 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b697fa700 1 -- 172.21.2.104:0/4239624208 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v135) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4b6c0aace0 con 0x7f4b6c0a70c0 2021-07-11T04:01:50.523 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:50.524 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T04:01:50.524 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:50.524 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T04:01:50.524 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T04:01:50.525 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:01:50.525 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:01:50.525 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T04:01:50.525 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T04:01:50.526 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:01:50.526 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T04:01:50.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 -- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b500532d0 msgr2=0x7f4b50055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b500532d0 0x7f4b50055790 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f4b54002ef0 tx=0x7f4b54006890).stop 2021-07-11T04:01:50.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 -- 172.21.2.104:0/4239624208 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b6c0a70c0 msgr2=0x7f4b6c0b3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b6c0a70c0 0x7f4b6c0b3640 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f4b6c0a9c10 tx=0x7f4b5c014000).stop 2021-07-11T04:01:50.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 -- 172.21.2.104:0/4239624208 shutdown_connections 2021-07-11T04:01:50.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b6c1483a0 0x7f4b6c145b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b6c0b1db0 0x7f4b6c1469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b500532d0 0x7f4b50055790 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 --2- 172.21.2.104:0/4239624208 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b6c0a70c0 0x7f4b6c0b3640 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 -- 172.21.2.104:0/4239624208 >> 172.21.2.104:0/4239624208 conn(0x7f4b6c05f6b0 msgr2=0x7f4b6c0a7550 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:50.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 -- 172.21.2.104:0/4239624208 shutdown_connections 2021-07-11T04:01:50.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.524+0000 7f4b728df700 1 -- 172.21.2.104:0/4239624208 wait complete. 2021-07-11T04:01:50.531 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T04:01:50.531 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T04:01:50.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f425ec1d700 1 Processor -- start 2021-07-11T04:01:50.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f425ec1d700 1 -- start start 2021-07-11T04:01:50.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f425ec1d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f42580b1ec0 0x7f42580b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f425ec1d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f42580affb0 0x7f42580b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f425ec1d700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42580b0420 con 0x7f42580a70c0 2021-07-11T04:01:50.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f425ec1d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42580b2790 con 0x7f42580affb0 2021-07-11T04:01:50.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f425ec1d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42580b0650 con 0x7f42580b1ec0 2021-07-11T04:01:50.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f4257fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f42580affb0 0x7f42580b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f4257fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f42580affb0 0x7f42580b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50216/0 (socket says 172.21.2.104:50216) 2021-07-11T04:01:50.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f4257fff700 1 -- 172.21.2.104:0/84208535 learned_addr learned my addr 172.21.2.104:0/84208535 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:50.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f4257fff700 1 -- 172.21.2.104:0/84208535 >> v2:172.21.2.105:3300/0 conn(0x7f42580b1ec0 msgr2=0x7f42580b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:50.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f4257fff700 1 --2- 172.21.2.104:0/84208535 >> v2:172.21.2.105:3300/0 conn(0x7f42580b1ec0 0x7f42580b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f4257fff700 1 -- 172.21.2.104:0/84208535 >> v1:172.21.2.105:6789/0 conn(0x7f42580a70c0 legacy=0x7f4258141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:50.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f4257fff700 1 -- 172.21.2.104:0/84208535 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42580b08f0 con 0x7f42580affb0 2021-07-11T04:01:50.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f4257fff700 1 --2- 172.21.2.104:0/84208535 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42580affb0 0x7f42580b22a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7f42580ad2d0 tx=0x7f4248006eb0).ready entity=mon.0 client_cookie=2fd474098acb1b9b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f42577fe700 1 -- 172.21.2.104:0/84208535 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f424800d5c0 con 0x7f42580affb0 2021-07-11T04:01:50.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f42577fe700 1 -- 172.21.2.104:0/84208535 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4248000e90 con 0x7f42580affb0 2021-07-11T04:01:50.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f42577fe700 1 -- 172.21.2.104:0/84208535 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f424800d910 con 0x7f42580affb0 2021-07-11T04:01:50.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f425ec1d700 1 -- 172.21.2.104:0/84208535 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42580affb0 msgr2=0x7f42580b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.584+0000 7f425ec1d700 1 --2- 172.21.2.104:0/84208535 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42580affb0 0x7f42580b22a0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7f42580ad2d0 tx=0x7f4248006eb0).stop 2021-07-11T04:01:50.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 -- 172.21.2.104:0/84208535 shutdown_connections 2021-07-11T04:01:50.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 --2- 172.21.2.104:0/84208535 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42580affb0 0x7f42580b22a0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:50.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 --2- 172.21.2.104:0/84208535 >> v2:172.21.2.105:3300/0 conn(0x7f42580b1ec0 0x7f42580b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 -- 172.21.2.104:0/84208535 >> 172.21.2.104:0/84208535 conn(0x7f425805f6b0 msgr2=0x7f425805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:50.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 -- 172.21.2.104:0/84208535 shutdown_connections 2021-07-11T04:01:50.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 -- 172.21.2.104:0/84208535 wait complete. 2021-07-11T04:01:50.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 Processor -- start 2021-07-11T04:01:50.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 -- start start 2021-07-11T04:01:50.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42580a70c0 0x7f42580aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42580affb0 0x7f4258148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f42580b1ec0 0x7f425814b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4258148c90 con 0x7f42580affb0 2021-07-11T04:01:50.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425814a2b0 con 0x7f42580b1ec0 2021-07-11T04:01:50.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4258149800 con 0x7f42580a70c0 2021-07-11T04:01:50.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4257fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42580affb0 0x7f4258148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4257fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42580affb0 0x7f4258148040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50224/0 (socket says 172.21.2.104:50224) 2021-07-11T04:01:50.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4257fff700 1 -- 172.21.2.104:0/3048994778 learned_addr learned my addr 172.21.2.104:0/3048994778 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:50.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4257fff700 1 -- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42580a70c0 msgr2=0x7f42580aaeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4257fff700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42580a70c0 0x7f42580aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4257fff700 1 -- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f42580b1ec0 msgr2=0x7f425814b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:50.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4257fff700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f42580b1ec0 0x7f425814b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:50.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4257fff700 1 -- 172.21.2.104:0/3048994778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424800b1e0 con 0x7f42580affb0 2021-07-11T04:01:50.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4257fff700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42580affb0 0x7f4258148040 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f42580ad2d0 tx=0x7f4248015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4255ffb700 1 -- 172.21.2.104:0/3048994778 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4248021070 con 0x7f42580affb0 2021-07-11T04:01:50.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4255ffb700 1 -- 172.21.2.104:0/3048994778 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4248000e90 con 0x7f42580affb0 2021-07-11T04:01:50.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 -- 172.21.2.104:0/3048994778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4258149aa0 con 0x7f42580affb0 2021-07-11T04:01:50.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 -- 172.21.2.104:0/3048994778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4258148530 con 0x7f42580affb0 2021-07-11T04:01:50.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4255ffb700 1 -- 172.21.2.104:0/3048994778 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4248006be0 con 0x7f42580affb0 2021-07-11T04:01:50.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f425ec1d700 1 -- 172.21.2.104:0/3048994778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4258149c30 con 0x7f42580affb0 2021-07-11T04:01:50.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4255ffb700 1 -- 172.21.2.104:0/3048994778 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f424800f070 con 0x7f42580affb0 2021-07-11T04:01:50.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4255ffb700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f423c0531a0 0x7f423c055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:50.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.588+0000 7f4255ffb700 1 -- 172.21.2.104:0/3048994778 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f424806c660 con 0x7f42580affb0 2021-07-11T04:01:50.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.592+0000 7f425c9b9700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f423c0531a0 0x7f423c055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:50.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.592+0000 7f425c9b9700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f423c0531a0 0x7f423c055660 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f4240002f70 tx=0x7f4240006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:50.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.592+0000 7f4255ffb700 1 -- 172.21.2.104:0/3048994778 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4248043bc0 con 0x7f42580affb0 2021-07-11T04:01:50.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:50.672+0000 7f425ec1d700 1 -- 172.21.2.104:0/3048994778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f425814a470 con 0x7f42580affb0 2021-07-11T04:01:51.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f4255ffb700 1 -- 172.21.2.104:0/3048994778 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v136) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4248002f50 con 0x7f42580affb0 2021-07-11T04:01:51.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 -- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f423c0531a0 msgr2=0x7f423c055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:51.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f423c0531a0 0x7f423c055660 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f4240002f70 tx=0x7f4240006a10).stop 2021-07-11T04:01:51.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 -- 172.21.2.104:0/3048994778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42580affb0 msgr2=0x7f4258148040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:51.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42580affb0 0x7f4258148040 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7f42580ad2d0 tx=0x7f4248015000).stop 2021-07-11T04:01:51.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 -- 172.21.2.104:0/3048994778 shutdown_connections 2021-07-11T04:01:51.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f42580b1ec0 0x7f425814b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:51.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f423c0531a0 0x7f423c055660 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:51.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f42580affb0 0x7f4258148040 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:51.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 --2- 172.21.2.104:0/3048994778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f42580a70c0 0x7f42580aaeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:51.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 -- 172.21.2.104:0/3048994778 >> 172.21.2.104:0/3048994778 conn(0x7f425805f6b0 msgr2=0x7f425813f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:51.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 -- 172.21.2.104:0/3048994778 shutdown_connections 2021-07-11T04:01:51.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.684+0000 7f425ec1d700 1 -- 172.21.2.104:0/3048994778 wait complete. 2021-07-11T04:01:51.692 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T04:01:51.694 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T04:01:51.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27b7073700 1 Processor -- start 2021-07-11T04:01:51.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27b7073700 1 -- start start 2021-07-11T04:01:51.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27b7073700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f27b00b12d0 0x7f27b00affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:51.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27b7073700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b00a77d0 con 0x7f27b00b1ec0 2021-07-11T04:01:51.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27b7073700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b00b0650 con 0x7f27b00b22a0 2021-07-11T04:01:51.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27b7073700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b00b1740 con 0x7f27b00b12d0 2021-07-11T04:01:51.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27b4e0f700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f27b00b1ec0 0x7f27b00a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:46996/0 (socket says 172.21.2.104:46996) 2021-07-11T04:01:51.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27b4e0f700 1 -- 172.21.2.104:0/1020689540 learned_addr learned my addr 172.21.2.104:0/1020689540 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:51.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27affff700 1 --2- 172.21.2.104:0/1020689540 >> v2:172.21.2.104:3300/0 conn(0x7f27b00b12d0 0x7f27b00affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:51.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27affff700 1 -- 172.21.2.104:0/1020689540 >> v1:172.21.2.105:6789/0 conn(0x7f27b00b22a0 legacy=0x7f27b0141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:51.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27affff700 1 -- 172.21.2.104:0/1020689540 >> v1:172.21.2.104:6789/0 conn(0x7f27b00b1ec0 legacy=0x7f27b00a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:51.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27affff700 1 -- 172.21.2.104:0/1020689540 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b00b08f0 con 0x7f27b00b12d0 2021-07-11T04:01:51.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27affff700 1 --2- 172.21.2.104:0/1020689540 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27b00b12d0 0x7f27b00affb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f27b00b2f50 tx=0x7f2798006df0).ready entity=mon.0 client_cookie=5abf51aea21b8d45 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:51.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27af7fe700 1 -- 172.21.2.104:0/1020689540 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f279800d5c0 con 0x7f27b00b12d0 2021-07-11T04:01:51.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27af7fe700 1 -- 172.21.2.104:0/1020689540 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2798000e90 con 0x7f27b00b12d0 2021-07-11T04:01:51.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27af7fe700 1 -- 172.21.2.104:0/1020689540 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f279800d910 con 0x7f27b00b12d0 2021-07-11T04:01:51.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27b7073700 1 -- 172.21.2.104:0/1020689540 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27b00b12d0 msgr2=0x7f27b00affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:51.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.748+0000 7f27b7073700 1 --2- 172.21.2.104:0/1020689540 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27b00b12d0 0x7f27b00affb0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f27b00b2f50 tx=0x7f2798006df0).stop 2021-07-11T04:01:51.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 -- 172.21.2.104:0/1020689540 shutdown_connections 2021-07-11T04:01:51.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 --2- 172.21.2.104:0/1020689540 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27b00b12d0 0x7f27b00affb0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:51.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 -- 172.21.2.104:0/1020689540 >> 172.21.2.104:0/1020689540 conn(0x7f27b005f6b0 msgr2=0x7f27b005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:51.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 -- 172.21.2.104:0/1020689540 shutdown_connections 2021-07-11T04:01:51.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 -- 172.21.2.104:0/1020689540 wait complete. 2021-07-11T04:01:51.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 Processor -- start 2021-07-11T04:01:51.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 -- start start 2021-07-11T04:01:51.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27b00b12d0 0x7f27b0140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:51.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27b00b1ec0 0x7f27b00ab890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:51.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27b00b22a0 0x7f27b00abd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:51.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b014d860 con 0x7f27b00b12d0 2021-07-11T04:01:51.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00a9420 con 0x7f27b00b22a0 2021-07-11T04:01:51.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00ac270 con 0x7f27b00b1ec0 2021-07-11T04:01:51.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b4e0f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27b00b12d0 0x7f27b0140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:51.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b4e0f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27b00b12d0 0x7f27b0140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50236/0 (socket says 172.21.2.104:50236) 2021-07-11T04:01:51.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b4e0f700 1 -- 172.21.2.104:0/2050360801 learned_addr learned my addr 172.21.2.104:0/2050360801 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:51.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b4e0f700 1 -- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27b00b1ec0 msgr2=0x7f27b00ab890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:51.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b4e0f700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27b00b1ec0 0x7f27b00ab890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:51.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b4e0f700 1 -- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27b00b22a0 msgr2=0x7f27b00abd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:51.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b4e0f700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27b00b22a0 0x7f27b00abd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:51.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b4e0f700 1 -- 172.21.2.104:0/2050360801 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279800b180 con 0x7f27b00b12d0 2021-07-11T04:01:51.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b4e0f700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27b00b12d0 0x7f27b0140150 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f27b00ad2d0 tx=0x7f27a0007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:51.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27adffb700 1 -- 172.21.2.104:0/2050360801 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27a000ce30 con 0x7f27b00b12d0 2021-07-11T04:01:51.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27adffb700 1 -- 172.21.2.104:0/2050360801 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a00072e0 con 0x7f27b00b12d0 2021-07-11T04:01:51.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 -- 172.21.2.104:0/2050360801 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b00b32b0 con 0x7f27b00b12d0 2021-07-11T04:01:51.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27adffb700 1 -- 172.21.2.104:0/2050360801 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f27a0005800 con 0x7f27b00b12d0 2021-07-11T04:01:51.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 -- 172.21.2.104:0/2050360801 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b00a9650 con 0x7f27b00b12d0 2021-07-11T04:01:51.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27b7073700 1 -- 172.21.2.104:0/2050360801 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b0057350 con 0x7f27b00b12d0 2021-07-11T04:01:51.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27adffb700 1 -- 172.21.2.104:0/2050360801 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f27a0010070 con 0x7f27b00b12d0 2021-07-11T04:01:51.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27adffb700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2794053160 0x7f2794055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:51.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.752+0000 7f27adffb700 1 -- 172.21.2.104:0/2050360801 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5603+0+0 (secure 0 0 0) 0x7f27a006aa60 con 0x7f27b00b12d0 2021-07-11T04:01:51.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.756+0000 7f27affff700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2794053160 0x7f2794055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:51.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.756+0000 7f27affff700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2794053160 0x7f2794055620 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f27b00b2f50 tx=0x7f2798012000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:51.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.756+0000 7f27adffb700 1 -- 172.21.2.104:0/2050360801 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f27a0041e60 con 0x7f27b00b12d0 2021-07-11T04:01:51.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:51.840+0000 7f27b7073700 1 -- 172.21.2.104:0/2050360801 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f27b00b43e0 con 0x7f27b00b12d0 2021-07-11T04:01:52.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27adffb700 1 -- 172.21.2.104:0/2050360801 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v137) v1 ==== 125+0+0 (secure 0 0 0) 0x7f27a0018030 con 0x7f27b00b12d0 2021-07-11T04:01:52.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 -- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2794053160 msgr2=0x7f2794055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:52.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2794053160 0x7f2794055620 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f27b00b2f50 tx=0x7f2798012000).stop 2021-07-11T04:01:52.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 -- 172.21.2.104:0/2050360801 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27b00b12d0 msgr2=0x7f27b0140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:52.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27b00b12d0 0x7f27b0140150 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f27b00ad2d0 tx=0x7f27a0007c40).stop 2021-07-11T04:01:52.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 -- 172.21.2.104:0/2050360801 shutdown_connections 2021-07-11T04:01:52.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f27b00b22a0 0x7f27b00abd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:52.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f27b00b1ec0 0x7f27b00ab890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:52.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2794053160 0x7f2794055620 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:52.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 --2- 172.21.2.104:0/2050360801 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f27b00b12d0 0x7f27b0140150 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:52.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 -- 172.21.2.104:0/2050360801 >> 172.21.2.104:0/2050360801 conn(0x7f27b005f6b0 msgr2=0x7f27b00af540 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:52.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 -- 172.21.2.104:0/2050360801 shutdown_connections 2021-07-11T04:01:52.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.696+0000 7f27b7073700 1 -- 172.21.2.104:0/2050360801 wait complete. 2021-07-11T04:01:52.700 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T04:01:52.704 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T04:01:52.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa5949de700 1 Processor -- start 2021-07-11T04:01:52.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa5949de700 1 -- start start 2021-07-11T04:01:52.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa5949de700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa59005f6b0 0x7fa5900a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa5949de700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa5900a8b60 0x7fa590143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa5949de700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5900b14c0 con 0x7fa590143540 2021-07-11T04:01:52.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa5949de700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5900ab3e0 con 0x7fa59005f6b0 2021-07-11T04:01:52.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa5949de700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5900b1b90 con 0x7fa5900a8b60 2021-07-11T04:01:52.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa58dd9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa59005f6b0 0x7fa5900a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:52.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa58dd9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fa59005f6b0 0x7fa5900a9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50242/0 (socket says 172.21.2.104:50242) 2021-07-11T04:01:52.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa58dd9b700 1 -- 172.21.2.104:0/671226137 learned_addr learned my addr 172.21.2.104:0/671226137 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:52.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa58dd9b700 1 -- 172.21.2.104:0/671226137 >> v2:172.21.2.105:3300/0 conn(0x7fa5900a8b60 msgr2=0x7fa590143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:52.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa58dd9b700 1 --2- 172.21.2.104:0/671226137 >> v2:172.21.2.105:3300/0 conn(0x7fa5900a8b60 0x7fa590143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:52.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa58dd9b700 1 -- 172.21.2.104:0/671226137 >> v1:172.21.2.104:6789/0 conn(0x7fa590143540 legacy=0x7fa5900b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:52.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa58dd9b700 1 -- 172.21.2.104:0/671226137 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa590143fa0 con 0x7fa59005f6b0 2021-07-11T04:01:52.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.756+0000 7fa58d59a700 1 -- 172.21.2.104:0/671226137 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807862897 0 0) 0x7fa5900b14c0 con 0x7fa590143540 2021-07-11T04:01:52.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58dd9b700 1 --2- 172.21.2.104:0/671226137 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa59005f6b0 0x7fa5900a9300 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7fa584002f70 tx=0x7fa584006eb0).ready entity=mon.0 client_cookie=cbe0bc8cef760960 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:52.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58d59a700 1 -- 172.21.2.104:0/671226137 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58400d610 con 0x7fa59005f6b0 2021-07-11T04:01:52.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58d59a700 1 -- 172.21.2.104:0/671226137 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa584000e90 con 0x7fa59005f6b0 2021-07-11T04:01:52.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58d59a700 1 -- 172.21.2.104:0/671226137 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58400d960 con 0x7fa59005f6b0 2021-07-11T04:01:52.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- 172.21.2.104:0/671226137 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa59005f6b0 msgr2=0x7fa5900a9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:52.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 --2- 172.21.2.104:0/671226137 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa59005f6b0 0x7fa5900a9300 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7fa584002f70 tx=0x7fa584006eb0).stop 2021-07-11T04:01:52.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- 172.21.2.104:0/671226137 shutdown_connections 2021-07-11T04:01:52.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 --2- 172.21.2.104:0/671226137 >> v2:172.21.2.105:3300/0 conn(0x7fa5900a8b60 0x7fa590143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:52.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 --2- 172.21.2.104:0/671226137 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa59005f6b0 0x7fa5900a9300 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:52.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- 172.21.2.104:0/671226137 >> 172.21.2.104:0/671226137 conn(0x7fa5900b53e0 msgr2=0x7fa5900b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:52.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- 172.21.2.104:0/671226137 shutdown_connections 2021-07-11T04:01:52.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- 172.21.2.104:0/671226137 wait complete. 2021-07-11T04:01:52.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 Processor -- start 2021-07-11T04:01:52.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- start start 2021-07-11T04:01:52.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa59005f6b0 0x7fa5900a9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5900a8b60 0x7fa59013b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa590143540 0x7fa59013bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5900b2fe0 con 0x7fa5900a8b60 2021-07-11T04:01:52.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa590156a20 con 0x7fa59005f6b0 2021-07-11T04:01:52.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59015aea0 con 0x7fa590143540 2021-07-11T04:01:52.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58dd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5900a8b60 0x7fa59013b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:52.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58dd9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5900a8b60 0x7fa59013b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50248/0 (socket says 172.21.2.104:50248) 2021-07-11T04:01:52.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58dd9b700 1 -- 172.21.2.104:0/2059354280 learned_addr learned my addr 172.21.2.104:0/2059354280 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:52.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58dd9b700 1 -- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa590143540 msgr2=0x7fa59013bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:52.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58dd9b700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa590143540 0x7fa59013bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:52.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58dd9b700 1 -- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa59005f6b0 msgr2=0x7fa5900a9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:52.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58dd9b700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa59005f6b0 0x7fa5900a9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:52.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58dd9b700 1 -- 172.21.2.104:0/2059354280 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58400b1e0 con 0x7fa5900a8b60 2021-07-11T04:01:52.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa58dd9b700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5900a8b60 0x7fa59013b880 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7fa5901482a0 tx=0x7fa584015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:52.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa57f7fe700 1 -- 172.21.2.104:0/2059354280 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa584021070 con 0x7fa5900a8b60 2021-07-11T04:01:52.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa57f7fe700 1 -- 172.21.2.104:0/2059354280 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa584000e90 con 0x7fa5900a8b60 2021-07-11T04:01:52.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- 172.21.2.104:0/2059354280 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa590148380 con 0x7fa5900a8b60 2021-07-11T04:01:52.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa57f7fe700 1 -- 172.21.2.104:0/2059354280 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa58400d290 con 0x7fa5900a8b60 2021-07-11T04:01:52.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- 172.21.2.104:0/2059354280 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59013c260 con 0x7fa5900a8b60 2021-07-11T04:01:52.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa5949de700 1 -- 172.21.2.104:0/2059354280 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa590148820 con 0x7fa5900a8b60 2021-07-11T04:01:52.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa57f7fe700 1 -- 172.21.2.104:0/2059354280 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa58400f070 con 0x7fa5900a8b60 2021-07-11T04:01:52.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.760+0000 7fa57f7fe700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa56c0531a0 0x7fa56c055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.764+0000 7fa57f7fe700 1 -- 172.21.2.104:0/2059354280 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5953+0+0 (secure 0 0 0) 0x7fa58406c900 con 0x7fa5900a8b60 2021-07-11T04:01:52.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.764+0000 7fa58e59c700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa56c0531a0 0x7fa56c055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:52.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.764+0000 7fa58e59c700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa56c0531a0 0x7fa56c055660 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fa580000bc0 tx=0x7fa5800077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:52.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.764+0000 7fa57f7fe700 1 -- 172.21.2.104:0/2059354280 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa584043cb0 con 0x7fa5900a8b60 2021-07-11T04:01:52.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.872+0000 7fa5949de700 1 -- 172.21.2.104:0/2059354280 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fa590057350 con 0x7fa5900a8b60 2021-07-11T04:01:52.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa57f7fe700 1 -- 172.21.2.104:0/2059354280 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 25 and data pool 26 v66) v1 ==== 189+0+0 (secure 0 0 0) 0x7fa5840472d0 con 0x7fa5900a8b60 2021-07-11T04:01:52.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 -- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa56c0531a0 msgr2=0x7fa56c055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:52.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa56c0531a0 0x7fa56c055660 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fa580000bc0 tx=0x7fa5800077f0).stop 2021-07-11T04:01:52.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 -- 172.21.2.104:0/2059354280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5900a8b60 msgr2=0x7fa59013b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:52.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5900a8b60 0x7fa59013b880 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7fa5901482a0 tx=0x7fa584015000).stop 2021-07-11T04:01:52.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 -- 172.21.2.104:0/2059354280 shutdown_connections 2021-07-11T04:01:52.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa590143540 0x7fa59013bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:52.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa56c0531a0 0x7fa56c055660 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:52.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa5900a8b60 0x7fa59013b880 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:52.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 --2- 172.21.2.104:0/2059354280 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa59005f6b0 0x7fa5900a9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:52.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 -- 172.21.2.104:0/2059354280 >> 172.21.2.104:0/2059354280 conn(0x7fa5900b53e0 msgr2=0x7fa590066d10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:52.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 -- 172.21.2.104:0/2059354280 shutdown_connections 2021-07-11T04:01:52.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.880+0000 7fa5949de700 1 -- 172.21.2.104:0/2059354280 wait complete. 2021-07-11T04:01:52.887 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 25 and data pool 26 2021-07-11T04:01:52.889 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:52.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01cd08700 1 Processor -- start 2021-07-11T04:01:52.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01cd08700 1 -- start start 2021-07-11T04:01:52.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01cd08700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb0180973d0 0x7fb01805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01cd08700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb01813f560 0x7fb01813f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01cd08700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0180b2a40 con 0x7fb0180b2660 2021-07-11T04:01:52.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01cd08700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb018097b80 con 0x7fb0180973d0 2021-07-11T04:01:52.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01cd08700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb018097e20 con 0x7fb01813f560 2021-07-11T04:01:52.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb015d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb0180973d0 0x7fb01805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:52.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb015d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb0180973d0 0x7fb01805f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50268/0 (socket says 172.21.2.104:50268) 2021-07-11T04:01:52.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb015d9b700 1 -- 172.21.2.104:0/52283595 learned_addr learned my addr 172.21.2.104:0/52283595 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:52.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb016d9d700 1 --2- 172.21.2.104:0/52283595 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb01813f560 0x7fb01813f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:52.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb015d9b700 1 -- 172.21.2.104:0/52283595 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb01813f560 msgr2=0x7fb01813f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:52.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb015d9b700 1 --2- 172.21.2.104:0/52283595 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb01813f560 0x7fb01813f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:52.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb015d9b700 1 -- 172.21.2.104:0/52283595 >> v1:172.21.2.105:6789/0 conn(0x7fb0180b2660 legacy=0x7fb018096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb015d9b700 1 -- 172.21.2.104:0/52283595 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180980c0 con 0x7fb0180973d0 2021-07-11T04:01:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb015d9b700 1 --2- 172.21.2.104:0/52283595 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0180973d0 0x7fb01805f6b0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7fb0180b3350 tx=0x7fb000006eb0).ready entity=mon.0 client_cookie=f52d40e4874d7d2d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01559a700 1 -- 172.21.2.104:0/52283595 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb00000d570 con 0x7fb0180973d0 2021-07-11T04:01:52.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01559a700 1 -- 172.21.2.104:0/52283595 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb000000e90 con 0x7fb0180973d0 2021-07-11T04:01:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01559a700 1 -- 172.21.2.104:0/52283595 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb00000d8c0 con 0x7fb0180973d0 2021-07-11T04:01:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01cd08700 1 -- 172.21.2.104:0/52283595 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0180973d0 msgr2=0x7fb01805f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.940+0000 7fb01cd08700 1 --2- 172.21.2.104:0/52283595 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0180973d0 0x7fb01805f6b0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7fb0180b3350 tx=0x7fb000006eb0).stop 2021-07-11T04:01:52.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 -- 172.21.2.104:0/52283595 shutdown_connections 2021-07-11T04:01:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 --2- 172.21.2.104:0/52283595 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb01813f560 0x7fb01813f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 --2- 172.21.2.104:0/52283595 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0180973d0 0x7fb01805f6b0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 -- 172.21.2.104:0/52283595 >> 172.21.2.104:0/52283595 conn(0x7fb018062f10 msgr2=0x7fb0180632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 -- 172.21.2.104:0/52283595 shutdown_connections 2021-07-11T04:01:52.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 -- 172.21.2.104:0/52283595 wait complete. 2021-07-11T04:01:52.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 Processor -- start 2021-07-11T04:01:52.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 -- start start 2021-07-11T04:01:52.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0180973d0 0x7fb0180abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0180b2660 0x7fb0180649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb01813f560 0x7fb018150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180645f0 con 0x7fb0180973d0 2021-07-11T04:01:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb018064390 con 0x7fb01813f560 2021-07-11T04:01:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb018154410 con 0x7fb0180b2660 2021-07-11T04:01:52.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0180973d0 0x7fb0180abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:52.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0180973d0 0x7fb0180abfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50274/0 (socket says 172.21.2.104:50274) 2021-07-11T04:01:52.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01659c700 1 -- 172.21.2.104:0/3962196943 learned_addr learned my addr 172.21.2.104:0/3962196943 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:52.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb016d9d700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb01813f560 0x7fb018150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:52.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb015d9b700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0180b2660 0x7fb0180649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01659c700 1 -- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0180b2660 msgr2=0x7fb0180649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01659c700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0180b2660 0x7fb0180649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01659c700 1 -- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb01813f560 msgr2=0x7fb018150320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01659c700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb01813f560 0x7fb018150320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:52.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01659c700 1 -- 172.21.2.104:0/3962196943 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00000b1e0 con 0x7fb0180973d0 2021-07-11T04:01:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01659c700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0180973d0 0x7fb0180abfa0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7fb008001fa0 tx=0x7fb008007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb00f7fe700 1 -- 172.21.2.104:0/3962196943 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb008007030 con 0x7fb0180973d0 2021-07-11T04:01:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 -- 172.21.2.104:0/3962196943 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0181546b0 con 0x7fb0180973d0 2021-07-11T04:01:52.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb00f7fe700 1 -- 172.21.2.104:0/3962196943 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb008007190 con 0x7fb0180973d0 2021-07-11T04:01:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 -- 172.21.2.104:0/3962196943 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb018063f10 con 0x7fb0180973d0 2021-07-11T04:01:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb00f7fe700 1 -- 172.21.2.104:0/3962196943 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb008006510 con 0x7fb0180973d0 2021-07-11T04:01:52.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.944+0000 7fb01cd08700 1 -- 172.21.2.104:0/3962196943 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb018057350 con 0x7fb0180973d0 2021-07-11T04:01:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.948+0000 7fb00f7fe700 1 -- 172.21.2.104:0/3962196943 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb008010030 con 0x7fb0180973d0 2021-07-11T04:01:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.948+0000 7fb00f7fe700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faffc053270 0x7faffc055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.948+0000 7fb00f7fe700 1 -- 172.21.2.104:0/3962196943 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb00806b250 con 0x7fb0180973d0 2021-07-11T04:01:52.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.948+0000 7fb015d9b700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faffc053270 0x7faffc055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:52.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.948+0000 7fb015d9b700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faffc053270 0x7faffc055730 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fb00001b000 tx=0x7fb000015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:52.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:52.948+0000 7fb00f7fe700 1 -- 172.21.2.104:0/3962196943 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb008042400 con 0x7fb0180973d0 2021-07-11T04:01:53.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.032+0000 7fb01cd08700 1 -- 172.21.2.104:0/3962196943 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb018061450 con 0x7fb0180973d0 2021-07-11T04:01:53.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.032+0000 7fb00f7fe700 1 -- 172.21.2.104:0/3962196943 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16065 (secure 0 0 0) 0x7fb008045a20 con 0x7fb0180973d0 2021-07-11T04:01:53.035 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:53.035 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":138,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:52.876878+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:01:50.686994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-07-11T04:01:51.853293+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:53.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 -- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faffc053270 msgr2=0x7faffc055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:53.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faffc053270 0x7faffc055730 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fb00001b000 tx=0x7fb000015000).stop 2021-07-11T04:01:53.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 -- 172.21.2.104:0/3962196943 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0180973d0 msgr2=0x7fb0180abfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:53.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0180973d0 0x7fb0180abfa0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7fb008001fa0 tx=0x7fb008007c30).stop 2021-07-11T04:01:53.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 -- 172.21.2.104:0/3962196943 shutdown_connections 2021-07-11T04:01:53.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb01813f560 0x7fb018150320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:53.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb0180b2660 0x7fb0180649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:53.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faffc053270 0x7faffc055730 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:53.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 --2- 172.21.2.104:0/3962196943 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb0180973d0 0x7fb0180abfa0 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:53.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 -- 172.21.2.104:0/3962196943 >> 172.21.2.104:0/3962196943 conn(0x7fb018062f10 msgr2=0x7fb0180b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:53.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 -- 172.21.2.104:0/3962196943 shutdown_connections 2021-07-11T04:01:53.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.036+0000 7fb01cd08700 1 -- 172.21.2.104:0/3962196943 wait complete. 2021-07-11T04:01:53.042 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:53.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.096+0000 7f520a581700 1 Processor -- start 2021-07-11T04:01:53.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.096+0000 7f520a581700 1 -- start start 2021-07-11T04:01:53.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.096+0000 7f520a581700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52040a76b0 0x7f5204066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.096+0000 7f520a581700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f52040b0060 0x7f520405fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.096+0000 7f520a581700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52040a7a90 con 0x7f52040b1970 2021-07-11T04:01:53.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.096+0000 7f520a581700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5204147620 con 0x7f52040b0060 2021-07-11T04:01:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.096+0000 7f520a581700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52041478c0 con 0x7f52040a76b0 2021-07-11T04:01:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.096+0000 7f5208b1e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f52040b0060 0x7f520405fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.096+0000 7f5208b1e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f52040b0060 0x7f520405fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50284/0 (socket says 172.21.2.104:50284) 2021-07-11T04:01:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.096+0000 7f5208b1e700 1 -- 172.21.2.104:0/3923217133 learned_addr learned my addr 172.21.2.104:0/3923217133 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:53.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f5208b1e700 1 -- 172.21.2.104:0/3923217133 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52040a76b0 msgr2=0x7f5204066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f5208b1e700 1 --2- 172.21.2.104:0/3923217133 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52040a76b0 0x7f5204066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f5208b1e700 1 -- 172.21.2.104:0/3923217133 >> v1:172.21.2.105:6789/0 conn(0x7f52040b1970 legacy=0x7f520405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f5208b1e700 1 -- 172.21.2.104:0/3923217133 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5204147b60 con 0x7f52040b0060 2021-07-11T04:01:53.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f5208b1e700 1 --2- 172.21.2.104:0/3923217133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52040b0060 0x7f520405fdc0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f52040b6460 tx=0x7f51fc009290).ready entity=mon.0 client_cookie=49413d598f44290 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f5202ffd700 1 -- 172.21.2.104:0/3923217133 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51fc00b860 con 0x7f52040b0060 2021-07-11T04:01:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f5202ffd700 1 -- 172.21.2.104:0/3923217133 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc00b9c0 con 0x7f52040b0060 2021-07-11T04:01:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f5202ffd700 1 -- 172.21.2.104:0/3923217133 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51fc00bb70 con 0x7f52040b0060 2021-07-11T04:01:53.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- 172.21.2.104:0/3923217133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52040b0060 msgr2=0x7f520405fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 --2- 172.21.2.104:0/3923217133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52040b0060 0x7f520405fdc0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f52040b6460 tx=0x7f51fc009290).stop 2021-07-11T04:01:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- 172.21.2.104:0/3923217133 shutdown_connections 2021-07-11T04:01:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 --2- 172.21.2.104:0/3923217133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52040b0060 0x7f520405fdc0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 --2- 172.21.2.104:0/3923217133 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52040a76b0 0x7f5204066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:53.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- 172.21.2.104:0/3923217133 >> 172.21.2.104:0/3923217133 conn(0x7f52040b53e0 msgr2=0x7f52040b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:53.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- 172.21.2.104:0/3923217133 shutdown_connections 2021-07-11T04:01:53.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- 172.21.2.104:0/3923217133 wait complete. 2021-07-11T04:01:53.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 Processor -- start 2021-07-11T04:01:53.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- start start 2021-07-11T04:01:53.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f52040a76b0 0x7f5204145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52040b0060 0x7f52040adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52040b1970 0x7f52040ae4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52040ad670 con 0x7f52040b0060 2021-07-11T04:01:53.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52040ac110 con 0x7f52040a76b0 2021-07-11T04:01:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52040aa0c0 con 0x7f52040b1970 2021-07-11T04:01:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52037fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52040b0060 0x7f52040adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52037fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52040b0060 0x7f52040adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50286/0 (socket says 172.21.2.104:50286) 2021-07-11T04:01:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52037fe700 1 -- 172.21.2.104:0/3815427654 learned_addr learned my addr 172.21.2.104:0/3815427654 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:53.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52037fe700 1 -- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52040b1970 msgr2=0x7f52040ae4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52037fe700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52040b1970 0x7f52040ae4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52037fe700 1 -- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f52040a76b0 msgr2=0x7f5204145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52037fe700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f52040a76b0 0x7f5204145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:53.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52037fe700 1 -- 172.21.2.104:0/3815427654 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc009530 con 0x7f52040b0060 2021-07-11T04:01:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52037fe700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52040b0060 0x7f52040adfd0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f51ec002f10 tx=0x7f51ec006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52017fa700 1 -- 172.21.2.104:0/3815427654 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51ec00f510 con 0x7f52040b0060 2021-07-11T04:01:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52017fa700 1 -- 172.21.2.104:0/3815427654 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec00f670 con 0x7f52040b0060 2021-07-11T04:01:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- 172.21.2.104:0/3815427654 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52040abe70 con 0x7f52040b0060 2021-07-11T04:01:53.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f52017fa700 1 -- 172.21.2.104:0/3815427654 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f51ec00f7e0 con 0x7f52040b0060 2021-07-11T04:01:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- 172.21.2.104:0/3815427654 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52040a95e0 con 0x7f52040b0060 2021-07-11T04:01:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.100+0000 7f520a581700 1 -- 172.21.2.104:0/3815427654 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5204057350 con 0x7f52040b0060 2021-07-11T04:01:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.104+0000 7f52017fa700 1 -- 172.21.2.104:0/3815427654 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f51ec017020 con 0x7f52040b0060 2021-07-11T04:01:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.104+0000 7f52017fa700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51e80531f0 0x7f51e80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.104+0000 7f52017fa700 1 -- 172.21.2.104:0/3815427654 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f51ec06a680 con 0x7f52040b0060 2021-07-11T04:01:53.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.104+0000 7f5203fff700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51e80531f0 0x7f51e80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:53.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.104+0000 7f5203fff700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51e80531f0 0x7f51e80556b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f51f4002f70 tx=0x7f51f4006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:53.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.104+0000 7f52017fa700 1 -- 172.21.2.104:0/3815427654 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f51ec0419c0 con 0x7f52040b0060 2021-07-11T04:01:53.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.184+0000 7f520a581700 1 -- 172.21.2.104:0/3815427654 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52040698e0 con 0x7f52040b0060 2021-07-11T04:01:53.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f52017fa700 1 -- 172.21.2.104:0/3815427654 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f51ec023ee0 con 0x7f52040b0060 2021-07-11T04:01:53.186 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:53.187 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":138,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:52.876878+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:01:50.686994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-07-11T04:01:51.853293+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:53.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 -- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51e80531f0 msgr2=0x7f51e80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:53.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51e80531f0 0x7f51e80556b0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f51f4002f70 tx=0x7f51f4006a80).stop 2021-07-11T04:01:53.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 -- 172.21.2.104:0/3815427654 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52040b0060 msgr2=0x7f52040adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:53.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52040b0060 0x7f52040adfd0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f51ec002f10 tx=0x7f51ec006d70).stop 2021-07-11T04:01:53.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 -- 172.21.2.104:0/3815427654 shutdown_connections 2021-07-11T04:01:53.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f52040b1970 0x7f52040ae4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:53.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51e80531f0 0x7f51e80556b0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:53.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f52040b0060 0x7f52040adfd0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:53.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 --2- 172.21.2.104:0/3815427654 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f52040a76b0 0x7f5204145fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:53.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 -- 172.21.2.104:0/3815427654 >> 172.21.2.104:0/3815427654 conn(0x7f52040b53e0 msgr2=0x7f5204060330 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:53.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 -- 172.21.2.104:0/3815427654 shutdown_connections 2021-07-11T04:01:53.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.188+0000 7f520a581700 1 -- 172.21.2.104:0/3815427654 wait complete. 2021-07-11T04:01:53.194 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T04:01:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54f1eb0700 1 Processor -- start 2021-07-11T04:01:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54f1eb0700 1 -- start start 2021-07-11T04:01:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54f1eb0700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f54ec060100 0x7f54ec064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54f1eb0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54ec0b38d0 0x7f54ec065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54f1eb0700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54ec0604e0 con 0x7f54ec144440 2021-07-11T04:01:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54f1eb0700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54ec0656f0 con 0x7f54ec060100 2021-07-11T04:01:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54f1eb0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec065c80 con 0x7f54ec0b38d0 2021-07-11T04:01:53.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54eb7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f54ec060100 0x7f54ec064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:53.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54eb7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f54ec060100 0x7f54ec064d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50294/0 (socket says 172.21.2.104:50294) 2021-07-11T04:01:53.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54eb7fe700 1 -- 172.21.2.104:0/503151381 learned_addr learned my addr 172.21.2.104:0/503151381 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:53.262 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54eb7fe700 1 -- 172.21.2.104:0/503151381 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54ec0b38d0 msgr2=0x7f54ec065200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:53.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54eb7fe700 1 --2- 172.21.2.104:0/503151381 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54ec0b38d0 0x7f54ec065200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:53.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54eb7fe700 1 -- 172.21.2.104:0/503151381 >> v1:172.21.2.105:6789/0 conn(0x7f54ec144440 legacy=0x7f54ec144970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:53.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54eb7fe700 1 -- 172.21.2.104:0/503151381 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec0b23b0 con 0x7f54ec060100 2021-07-11T04:01:53.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54eb7fe700 1 --2- 172.21.2.104:0/503151381 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54ec060100 0x7f54ec064d10 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f54ec05d460 tx=0x7f54d4006eb0).ready entity=mon.0 client_cookie=2642b46bb19bd41e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:53.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54ea7fc700 1 -- 172.21.2.104:0/503151381 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54d400d570 con 0x7f54ec060100 2021-07-11T04:01:53.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54ea7fc700 1 -- 172.21.2.104:0/503151381 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d4000e90 con 0x7f54ec060100 2021-07-11T04:01:53.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54ea7fc700 1 -- 172.21.2.104:0/503151381 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54d400d8c0 con 0x7f54ec060100 2021-07-11T04:01:53.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54f1eb0700 1 -- 172.21.2.104:0/503151381 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54ec060100 msgr2=0x7f54ec064d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:53.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.256+0000 7f54f1eb0700 1 --2- 172.21.2.104:0/503151381 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54ec060100 0x7f54ec064d10 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f54ec05d460 tx=0x7f54d4006eb0).stop 2021-07-11T04:01:53.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 -- 172.21.2.104:0/503151381 shutdown_connections 2021-07-11T04:01:53.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 --2- 172.21.2.104:0/503151381 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54ec0b38d0 0x7f54ec065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:53.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 --2- 172.21.2.104:0/503151381 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54ec060100 0x7f54ec064d10 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:53.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 -- 172.21.2.104:0/503151381 >> 172.21.2.104:0/503151381 conn(0x7f54ec05cf60 msgr2=0x7f54ec05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:53.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 -- 172.21.2.104:0/503151381 shutdown_connections 2021-07-11T04:01:53.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 -- 172.21.2.104:0/503151381 wait complete. 2021-07-11T04:01:53.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 Processor -- start 2021-07-11T04:01:53.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 -- start start 2021-07-11T04:01:53.266 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54ec060100 0x7f54ec141370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54ec0b38d0 0x7f54ec1397a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54ec0ab4d0 0x7f54ec139c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec060cd0 con 0x7f54ec060100 2021-07-11T04:01:53.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec0b1780 con 0x7f54ec0ab4d0 2021-07-11T04:01:53.267 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54ec060940 con 0x7f54ec0b38d0 2021-07-11T04:01:53.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54eb7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54ec060100 0x7f54ec141370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:53.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54eb7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54ec060100 0x7f54ec141370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50302/0 (socket says 172.21.2.104:50302) 2021-07-11T04:01:53.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54eb7fe700 1 -- 172.21.2.104:0/1664098855 learned_addr learned my addr 172.21.2.104:0/1664098855 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:53.268 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54eb7fe700 1 -- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54ec0b38d0 msgr2=0x7f54ec1397a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:53.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54eb7fe700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54ec0b38d0 0x7f54ec1397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:53.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54eb7fe700 1 -- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54ec0ab4d0 msgr2=0x7f54ec139c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:53.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54eb7fe700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54ec0ab4d0 0x7f54ec139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:53.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54eb7fe700 1 -- 172.21.2.104:0/1664098855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d400b1e0 con 0x7f54ec060100 2021-07-11T04:01:53.269 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54eb7fe700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54ec060100 0x7f54ec141370 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f54ec05d460 tx=0x7f54d4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:53.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54e8ff9700 1 -- 172.21.2.104:0/1664098855 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54d4021070 con 0x7f54ec060100 2021-07-11T04:01:53.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54e8ff9700 1 -- 172.21.2.104:0/1664098855 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54d4000e90 con 0x7f54ec060100 2021-07-11T04:01:53.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54e8ff9700 1 -- 172.21.2.104:0/1664098855 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54d4011a50 con 0x7f54ec060100 2021-07-11T04:01:53.270 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 -- 172.21.2.104:0/1664098855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54ec060f00 con 0x7f54ec060100 2021-07-11T04:01:53.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 -- 172.21.2.104:0/1664098855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54ec0ad570 con 0x7f54ec060100 2021-07-11T04:01:53.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54f1eb0700 1 -- 172.21.2.104:0/1664098855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54ec13a320 con 0x7f54ec060100 2021-07-11T04:01:53.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54e8ff9700 1 -- 172.21.2.104:0/1664098855 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f54d400f070 con 0x7f54ec060100 2021-07-11T04:01:53.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54e8ff9700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54d0053380 0x7f54d0055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:53.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.260+0000 7f54e8ff9700 1 -- 172.21.2.104:0/1664098855 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f54d406c930 con 0x7f54ec060100 2021-07-11T04:01:53.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.264+0000 7f54eaffd700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54d0053380 0x7f54d0055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:53.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.264+0000 7f54eaffd700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54d0053380 0x7f54d0055840 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f54dc000ed0 tx=0x7f54dc006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:53.273 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.264+0000 7f54e8ff9700 1 -- 172.21.2.104:0/1664098855 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f54d4043c70 con 0x7f54ec060100 2021-07-11T04:01:53.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:53.372+0000 7f54f1eb0700 1 -- 172.21.2.104:0/1664098855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f54ec060710 con 0x7f54ec060100 2021-07-11T04:01:54.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.368+0000 7f54e8ff9700 1 -- 172.21.2.104:0/1664098855 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7f54d4047290 con 0x7f54ec060100 2021-07-11T04:01:54.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 -- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54d0053380 msgr2=0x7f54d0055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54d0053380 0x7f54d0055840 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f54dc000ed0 tx=0x7f54dc006930).stop 2021-07-11T04:01:54.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 -- 172.21.2.104:0/1664098855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54ec060100 msgr2=0x7f54ec141370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54ec060100 0x7f54ec141370 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f54ec05d460 tx=0x7f54d4015000).stop 2021-07-11T04:01:54.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 -- 172.21.2.104:0/1664098855 shutdown_connections 2021-07-11T04:01:54.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54ec0ab4d0 0x7f54ec139c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54ec0b38d0 0x7f54ec1397a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f54d0053380 0x7f54d0055840 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 --2- 172.21.2.104:0/1664098855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54ec060100 0x7f54ec141370 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 -- 172.21.2.104:0/1664098855 >> 172.21.2.104:0/1664098855 conn(0x7f54ec05cf60 msgr2=0x7f54ec1436f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:54.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 -- 172.21.2.104:0/1664098855 shutdown_connections 2021-07-11T04:01:54.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.372+0000 7f54f1eb0700 1 -- 172.21.2.104:0/1664098855 wait complete. 2021-07-11T04:01:54.378 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:54.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.428+0000 7ffb58f9e700 1 Processor -- start 2021-07-11T04:01:54.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.428+0000 7ffb58f9e700 1 -- start start 2021-07-11T04:01:54.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ffb540a76b0 0x7ffb54066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ffb540b0060 0x7ffb5405fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb540a7a90 con 0x7ffb540b1970 2021-07-11T04:01:54.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb54147620 con 0x7ffb540b0060 2021-07-11T04:01:54.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb541478c0 con 0x7ffb540a76b0 2021-07-11T04:01:54.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb51d9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ffb540b1970 0x7ffb5405f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47076/0 (socket says 172.21.2.104:47076) 2021-07-11T04:01:54.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb51d9b700 1 -- 172.21.2.104:0/2102469290 learned_addr learned my addr 172.21.2.104:0/2102469290 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:54.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb52d9d700 1 --2- 172.21.2.104:0/2102469290 >> v2:172.21.2.104:3300/0 conn(0x7ffb540b0060 0x7ffb5405fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb5159a700 1 -- 172.21.2.104:0/2102469290 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387245436 0 0) 0x7ffb540a7a90 con 0x7ffb540b1970 2021-07-11T04:01:54.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb52d9d700 1 -- 172.21.2.104:0/2102469290 >> v2:172.21.2.105:3300/0 conn(0x7ffb540a76b0 msgr2=0x7ffb54066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:54.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb52d9d700 1 --2- 172.21.2.104:0/2102469290 >> v2:172.21.2.105:3300/0 conn(0x7ffb540a76b0 0x7ffb54066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb52d9d700 1 -- 172.21.2.104:0/2102469290 >> v1:172.21.2.104:6789/0 conn(0x7ffb540b1970 legacy=0x7ffb5405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb52d9d700 1 -- 172.21.2.104:0/2102469290 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb54147b60 con 0x7ffb540b0060 2021-07-11T04:01:54.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb52d9d700 1 --2- 172.21.2.104:0/2102469290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffb540b0060 0x7ffb5405fdc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7ffb4c0032f0 tx=0x7ffb4c0091d0).ready entity=mon.0 client_cookie=377b1e310a0a895f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:54.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb5159a700 1 -- 172.21.2.104:0/2102469290 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb4c009ea0 con 0x7ffb540b0060 2021-07-11T04:01:54.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb5159a700 1 -- 172.21.2.104:0/2102469290 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c00b870 con 0x7ffb540b0060 2021-07-11T04:01:54.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb5159a700 1 -- 172.21.2.104:0/2102469290 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb4c00b9e0 con 0x7ffb540b0060 2021-07-11T04:01:54.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- 172.21.2.104:0/2102469290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffb540b0060 msgr2=0x7ffb5405fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 --2- 172.21.2.104:0/2102469290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffb540b0060 0x7ffb5405fdc0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7ffb4c0032f0 tx=0x7ffb4c0091d0).stop 2021-07-11T04:01:54.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- 172.21.2.104:0/2102469290 shutdown_connections 2021-07-11T04:01:54.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 --2- 172.21.2.104:0/2102469290 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffb540b0060 0x7ffb5405fdc0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 --2- 172.21.2.104:0/2102469290 >> v2:172.21.2.105:3300/0 conn(0x7ffb540a76b0 0x7ffb54066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- 172.21.2.104:0/2102469290 >> 172.21.2.104:0/2102469290 conn(0x7ffb540b53e0 msgr2=0x7ffb540b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:54.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- 172.21.2.104:0/2102469290 shutdown_connections 2021-07-11T04:01:54.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- 172.21.2.104:0/2102469290 wait complete. 2021-07-11T04:01:54.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 Processor -- start 2021-07-11T04:01:54.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- start start 2021-07-11T04:01:54.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffb540a76b0 0x7ffb540b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffb540b0060 0x7ffb540b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffb540b1970 0x7ffb54159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5414c930 con 0x7ffb540b0060 2021-07-11T04:01:54.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5414bce0 con 0x7ffb540a76b0 2021-07-11T04:01:54.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb54157320 con 0x7ffb540b1970 2021-07-11T04:01:54.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb51d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffb540b0060 0x7ffb540b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb51d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffb540b0060 0x7ffb540b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50314/0 (socket says 172.21.2.104:50314) 2021-07-11T04:01:54.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb51d9b700 1 -- 172.21.2.104:0/1497092979 learned_addr learned my addr 172.21.2.104:0/1497092979 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:54.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb51d9b700 1 -- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffb540b1970 msgr2=0x7ffb54159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb51d9b700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffb540b1970 0x7ffb54159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb51d9b700 1 -- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffb540a76b0 msgr2=0x7ffb540b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb51d9b700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffb540a76b0 0x7ffb540b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb51d9b700 1 -- 172.21.2.104:0/1497092979 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c009470 con 0x7ffb540b0060 2021-07-11T04:01:54.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb51d9b700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffb540b0060 0x7ffb540b44e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7ffb44000bc0 tx=0x7ffb44007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:54.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb4b7fe700 1 -- 172.21.2.104:0/1497092979 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb44001d70 con 0x7ffb540b0060 2021-07-11T04:01:54.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb4b7fe700 1 -- 172.21.2.104:0/1497092979 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4400ce80 con 0x7ffb540b0060 2021-07-11T04:01:54.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb58f9e700 1 -- 172.21.2.104:0/1497092979 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb541575c0 con 0x7ffb540b0060 2021-07-11T04:01:54.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.432+0000 7ffb4b7fe700 1 -- 172.21.2.104:0/1497092979 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ffb440074d0 con 0x7ffb540b0060 2021-07-11T04:01:54.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.436+0000 7ffb58f9e700 1 -- 172.21.2.104:0/1497092979 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5414c5b0 con 0x7ffb540b0060 2021-07-11T04:01:54.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.436+0000 7ffb58f9e700 1 -- 172.21.2.104:0/1497092979 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb54057350 con 0x7ffb540b0060 2021-07-11T04:01:54.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.436+0000 7ffb4b7fe700 1 -- 172.21.2.104:0/1497092979 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ffb4401a060 con 0x7ffb540b0060 2021-07-11T04:01:54.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.436+0000 7ffb4b7fe700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffb380531f0 0x7ffb380556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.436+0000 7ffb4b7fe700 1 -- 172.21.2.104:0/1497092979 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ffb4406ad40 con 0x7ffb540b0060 2021-07-11T04:01:54.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.436+0000 7ffb5259c700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffb380531f0 0x7ffb380556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.436+0000 7ffb5259c700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffb380531f0 0x7ffb380556b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7ffb3c002f70 tx=0x7ffb3c006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:54.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.440+0000 7ffb4b7fe700 1 -- 172.21.2.104:0/1497092979 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ffb44045e60 con 0x7ffb540b0060 2021-07-11T04:01:54.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.544+0000 7ffb58f9e700 1 -- 172.21.2.104:0/1497092979 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb5414a910 con 0x7ffb540b0060 2021-07-11T04:01:54.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb4b7fe700 1 -- 172.21.2.104:0/1497092979 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2492 (secure 0 0 0) 0x7ffb44045e60 con 0x7ffb540b0060 2021-07-11T04:01:54.547 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:54.547 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5335,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/1814518027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":1814518027},{"type":"v1","addr":"172.21.2.105:6835","nonce":1814518027}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/155502403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":155502403},{"type":"v1","addr":"172.21.2.104:6834","nonce":155502403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:52.876988+0000","modified":"2021-07-11T04:01:54.362899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/2616834266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":2616834266},{"type":"v1","addr":"172.21.2.104:6835","nonce":2616834266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-07-11T04:01:54.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 -- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffb380531f0 msgr2=0x7ffb380556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffb380531f0 0x7ffb380556b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7ffb3c002f70 tx=0x7ffb3c006a80).stop 2021-07-11T04:01:54.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 -- 172.21.2.104:0/1497092979 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffb540b0060 msgr2=0x7ffb540b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffb540b0060 0x7ffb540b44e0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7ffb44000bc0 tx=0x7ffb44007c30).stop 2021-07-11T04:01:54.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 -- 172.21.2.104:0/1497092979 shutdown_connections 2021-07-11T04:01:54.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ffb540b1970 0x7ffb54159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ffb380531f0 0x7ffb380556b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ffb540b0060 0x7ffb540b44e0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 --2- 172.21.2.104:0/1497092979 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ffb540a76b0 0x7ffb540b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 -- 172.21.2.104:0/1497092979 >> 172.21.2.104:0/1497092979 conn(0x7ffb540b53e0 msgr2=0x7ffb54146040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:54.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 -- 172.21.2.104:0/1497092979 shutdown_connections 2021-07-11T04:01:54.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.548+0000 7ffb58f9e700 1 -- 172.21.2.104:0/1497092979 wait complete. 2021-07-11T04:01:54.551 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 68 2021-07-11T04:01:54.554 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:54.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0bafd700 1 Processor -- start 2021-07-11T04:01:54.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0bafd700 1 -- start start 2021-07-11T04:01:54.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0bafd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e04067180 0x7f1e0405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0bafd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1e040b1970 0x7f1e0405fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0bafd700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e040a7a90 con 0x7f1e040a76b0 2021-07-11T04:01:54.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0bafd700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e04147620 con 0x7f1e040b1970 2021-07-11T04:01:54.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0bafd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e041478c0 con 0x7f1e04067180 2021-07-11T04:01:54.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e09899700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1e040a76b0 0x7f1e04066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47088/0 (socket says 172.21.2.104:47088) 2021-07-11T04:01:54.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e09899700 1 -- 172.21.2.104:0/2015076828 learned_addr learned my addr 172.21.2.104:0/2015076828 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:54.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264420389 0 0) 0x7f1e040a7a90 con 0x7f1e040a76b0 2021-07-11T04:01:54.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dec003260 con 0x7f1e040a76b0 2021-07-11T04:01:54.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0a09a700 1 --2- 172.21.2.104:0/2015076828 >> v2:172.21.2.105:3300/0 conn(0x7f1e040b1970 0x7f1e0405fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e09098700 1 --2- 172.21.2.104:0/2015076828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e04067180 0x7f1e0405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 275278466 0 0) 0x7f1dec003260 con 0x7f1e040a76b0 2021-07-11T04:01:54.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e04067180 msgr2=0x7f1e0405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:54.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 --2- 172.21.2.104:0/2015076828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e04067180 0x7f1e0405f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 >> v2:172.21.2.105:3300/0 conn(0x7f1e040b1970 msgr2=0x7f1e0405fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:54.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 --2- 172.21.2.104:0/2015076828 >> v2:172.21.2.105:3300/0 conn(0x7f1e040b1970 0x7f1e0405fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04147b60 con 0x7f1e040a76b0 2021-07-11T04:01:54.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f1e00001680 con 0x7f1e040a76b0 2021-07-11T04:01:54.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 >> v1:172.21.2.104:6789/0 conn(0x7f1e040a76b0 legacy=0x7f1e04066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 --2- 172.21.2.104:0/2015076828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1dec003750 0x7f1dec003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 --2- 172.21.2.104:0/2015076828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e04067180 0x7f1dec004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 --2- 172.21.2.104:0/2015076828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1dec0049f0 0x7f1dec004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e04147620 con 0x7f1dec0049f0 2021-07-11T04:01:54.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e041478c0 con 0x7f1dec003750 2021-07-11T04:01:54.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec00b750 con 0x7f1e04067180 2021-07-11T04:01:54.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0a09a700 1 --2- 172.21.2.104:0/2015076828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1dec0049f0 0x7f1dec004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0a09a700 1 -- 172.21.2.104:0/2015076828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e04067180 msgr2=0x7f1dec004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:54.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0a09a700 1 --2- 172.21.2.104:0/2015076828 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e04067180 0x7f1dec004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0a09a700 1 -- 172.21.2.104:0/2015076828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1dec003750 msgr2=0x7f1dec003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:54.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0a09a700 1 --2- 172.21.2.104:0/2015076828 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1dec003750 0x7f1dec003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.608+0000 7f1e0a09a700 1 -- 172.21.2.104:0/2015076828 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e04147b60 con 0x7f1dec0049f0 2021-07-11T04:01:54.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0a09a700 1 --2- 172.21.2.104:0/2015076828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1dec0049f0 0x7f1dec004e70 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7f1dfc00a000 tx=0x7f1dfc008040).ready entity=mon.0 client_cookie=8eb402925d5c13c5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:54.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dfc00c070 con 0x7f1dec0049f0 2021-07-11T04:01:54.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc00e5f0 con 0x7f1dec0049f0 2021-07-11T04:01:54.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e08897700 1 -- 172.21.2.104:0/2015076828 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1dfc00e950 con 0x7f1dec0049f0 2021-07-11T04:01:54.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- 172.21.2.104:0/2015076828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1dec0049f0 msgr2=0x7f1dec004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 --2- 172.21.2.104:0/2015076828 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1dec0049f0 0x7f1dec004e70 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7f1dfc00a000 tx=0x7f1dfc008040).stop 2021-07-11T04:01:54.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0a09a700 1 -- 172.21.2.104:0/2015076828 reap_dead start 2021-07-11T04:01:54.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- 172.21.2.104:0/2015076828 shutdown_connections 2021-07-11T04:01:54.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- 172.21.2.104:0/2015076828 >> 172.21.2.104:0/2015076828 conn(0x7f1e040b53e0 msgr2=0x7f1e040b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:54.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- 172.21.2.104:0/2015076828 shutdown_connections 2021-07-11T04:01:54.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- 172.21.2.104:0/2015076828 wait complete. 2021-07-11T04:01:54.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 Processor -- start 2021-07-11T04:01:54.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- start start 2021-07-11T04:01:54.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e040ac860 0x7f1e040b4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e040b4970 0x7f1e040aa770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1e040ae550 0x7f1e04159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040acc40 con 0x7f1e040ac860 2021-07-11T04:01:54.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040abc10 con 0x7f1e040ae550 2021-07-11T04:01:54.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040abeb0 con 0x7f1e040b4970 2021-07-11T04:01:54.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e09899700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e040ac860 0x7f1e040b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e09899700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e040ac860 0x7f1e040b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50332/0 (socket says 172.21.2.104:50332) 2021-07-11T04:01:54.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e09899700 1 -- 172.21.2.104:0/3439988747 learned_addr learned my addr 172.21.2.104:0/3439988747 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:54.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e09899700 1 -- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e040b4970 msgr2=0x7f1e040aa770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e09899700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e040b4970 0x7f1e040aa770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e09899700 1 -- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1e040ae550 msgr2=0x7f1e04159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e09899700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1e040ae550 0x7f1e04159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e09899700 1 -- 172.21.2.104:0/3439988747 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e040ac150 con 0x7f1e040ac860 2021-07-11T04:01:54.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e09899700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e040ac860 0x7f1e040b4230 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f1e00001390 tx=0x7f1e00001bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:54.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1df6ffd700 1 -- 172.21.2.104:0/3439988747 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e00007170 con 0x7f1e040ac860 2021-07-11T04:01:54.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1df6ffd700 1 -- 172.21.2.104:0/3439988747 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e000072d0 con 0x7f1e040ac860 2021-07-11T04:01:54.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1df6ffd700 1 -- 172.21.2.104:0/3439988747 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e000049e0 con 0x7f1e040ac860 2021-07-11T04:01:54.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- 172.21.2.104:0/3439988747 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e040ab850 con 0x7f1e040ac860 2021-07-11T04:01:54.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- 172.21.2.104:0/3439988747 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e040aeb80 con 0x7f1e040ac860 2021-07-11T04:01:54.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1e0bafd700 1 -- 172.21.2.104:0/3439988747 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e04057350 con 0x7f1e040ac860 2021-07-11T04:01:54.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1df6ffd700 1 -- 172.21.2.104:0/3439988747 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1e00010070 con 0x7f1e040ac860 2021-07-11T04:01:54.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1df6ffd700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1de4053160 0x7f1de4055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.612+0000 7f1df6ffd700 1 -- 172.21.2.104:0/3439988747 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1e0006b170 con 0x7f1e040ac860 2021-07-11T04:01:54.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.616+0000 7f1e09098700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1de4053160 0x7f1de4055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.616+0000 7f1e09098700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1de4053160 0x7f1de4055620 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f1df800a000 tx=0x7f1df8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:54.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.616+0000 7f1df6ffd700 1 -- 172.21.2.104:0/3439988747 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1e000424b0 con 0x7f1e040ac860 2021-07-11T04:01:54.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 -- 172.21.2.104:0/3439988747 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e040b2dc0 con 0x7f1e040ac860 2021-07-11T04:01:54.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1df6ffd700 1 -- 172.21.2.104:0/3439988747 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f1e00045ad0 con 0x7f1e040ac860 2021-07-11T04:01:54.701 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:54.701 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":139,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:53.880137+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:01:50.686994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-07-11T04:01:51.853293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:54.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 -- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1de4053160 msgr2=0x7f1de4055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1de4053160 0x7f1de4055620 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f1df800a000 tx=0x7f1df8008040).stop 2021-07-11T04:01:54.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 -- 172.21.2.104:0/3439988747 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e040ac860 msgr2=0x7f1e040b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e040ac860 0x7f1e040b4230 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7f1e00001390 tx=0x7f1e00001bd0).stop 2021-07-11T04:01:54.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 -- 172.21.2.104:0/3439988747 shutdown_connections 2021-07-11T04:01:54.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1e040ae550 0x7f1e04159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e040b4970 0x7f1e040aa770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1de4053160 0x7f1de4055620 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 --2- 172.21.2.104:0/3439988747 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e040ac860 0x7f1e040b4230 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 -- 172.21.2.104:0/3439988747 >> 172.21.2.104:0/3439988747 conn(0x7f1e040b53e0 msgr2=0x7f1e0414a940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:54.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 -- 172.21.2.104:0/3439988747 shutdown_connections 2021-07-11T04:01:54.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.700+0000 7f1e0bafd700 1 -- 172.21.2.104:0/3439988747 wait complete. 2021-07-11T04:01:54.708 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:01:54.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 Processor -- start 2021-07-11T04:01:54.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 -- start start 2021-07-11T04:01:54.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2a340a76b0 0x7f2a34066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2a340b1970 0x7f2a34066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a34067450 con 0x7f2a340b0060 2021-07-11T04:01:54.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a340a7a90 con 0x7f2a340b1970 2021-07-11T04:01:54.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a34147620 con 0x7f2a340a76b0 2021-07-11T04:01:54.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a388f8700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f2a340b0060 0x7f2a3405f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47108/0 (socket says 172.21.2.104:47108) 2021-07-11T04:01:54.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a388f8700 1 -- 172.21.2.104:0/2721664219 learned_addr learned my addr 172.21.2.104:0/2721664219 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:54.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a337fe700 1 --2- 172.21.2.104:0/2721664219 >> v2:172.21.2.104:3300/0 conn(0x7f2a340b1970 0x7f2a34066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a32ffd700 1 -- 172.21.2.104:0/2721664219 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349755142 0 0) 0x7f2a34067450 con 0x7f2a340b0060 2021-07-11T04:01:54.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a32ffd700 1 -- 172.21.2.104:0/2721664219 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a14003260 con 0x7f2a340b0060 2021-07-11T04:01:54.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a337fe700 1 -- 172.21.2.104:0/2721664219 >> v2:172.21.2.105:3300/0 conn(0x7f2a340a76b0 msgr2=0x7f2a34066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:54.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a33fff700 1 --2- 172.21.2.104:0/2721664219 >> v2:172.21.2.105:3300/0 conn(0x7f2a340a76b0 0x7f2a34066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a337fe700 1 --2- 172.21.2.104:0/2721664219 >> v2:172.21.2.105:3300/0 conn(0x7f2a340a76b0 0x7f2a34066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a337fe700 1 -- 172.21.2.104:0/2721664219 >> v1:172.21.2.104:6789/0 conn(0x7f2a340b0060 legacy=0x7f2a3405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a337fe700 1 -- 172.21.2.104:0/2721664219 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a341478c0 con 0x7f2a340b1970 2021-07-11T04:01:54.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a33fff700 1 --2- 172.21.2.104:0/2721664219 >> v2:172.21.2.105:3300/0 conn(0x7f2a340a76b0 0x7f2a34066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:01:54.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a337fe700 1 --2- 172.21.2.104:0/2721664219 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a340b1970 0x7f2a34066f60 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7f2a1c002f70 tx=0x7f2a1c006df0).ready entity=mon.0 client_cookie=b8fb1520f466c979 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:54.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a32ffd700 1 -- 172.21.2.104:0/2721664219 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a1c00d570 con 0x7f2a340b1970 2021-07-11T04:01:54.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a32ffd700 1 -- 172.21.2.104:0/2721664219 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c000e90 con 0x7f2a340b1970 2021-07-11T04:01:54.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a32ffd700 1 -- 172.21.2.104:0/2721664219 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a1c00d8c0 con 0x7f2a340b1970 2021-07-11T04:01:54.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 -- 172.21.2.104:0/2721664219 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a340b1970 msgr2=0x7f2a34066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 --2- 172.21.2.104:0/2721664219 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a340b1970 0x7f2a34066f60 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7f2a1c002f70 tx=0x7f2a1c006df0).stop 2021-07-11T04:01:54.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 -- 172.21.2.104:0/2721664219 shutdown_connections 2021-07-11T04:01:54.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 --2- 172.21.2.104:0/2721664219 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a340b1970 0x7f2a34066f60 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 --2- 172.21.2.104:0/2721664219 >> v2:172.21.2.105:3300/0 conn(0x7f2a340a76b0 0x7f2a34066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 -- 172.21.2.104:0/2721664219 >> 172.21.2.104:0/2721664219 conn(0x7f2a340b53e0 msgr2=0x7f2a340b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:54.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 -- 172.21.2.104:0/2721664219 shutdown_connections 2021-07-11T04:01:54.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.804+0000 7f2a3a35b700 1 -- 172.21.2.104:0/2721664219 wait complete. 2021-07-11T04:01:54.824 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a3a35b700 1 Processor -- start 2021-07-11T04:01:54.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a3a35b700 1 -- start start 2021-07-11T04:01:54.825 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a3a35b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a340a76b0 0x7f2a34062910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a3a35b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2a340b0060 0x7f2a340ab690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a3a35b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2a340b1970 0x7f2a34065cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.826 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a3a35b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a34064a80 con 0x7f2a340a76b0 2021-07-11T04:01:54.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a3a35b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340aad40 con 0x7f2a340b1970 2021-07-11T04:01:54.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a3a35b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340aafe0 con 0x7f2a340b0060 2021-07-11T04:01:54.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a33fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a340a76b0 0x7f2a34062910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a33fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a340a76b0 0x7f2a34062910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50348/0 (socket says 172.21.2.104:50348) 2021-07-11T04:01:54.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a33fff700 1 -- 172.21.2.104:0/79294992 learned_addr learned my addr 172.21.2.104:0/79294992 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:54.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a33fff700 1 -- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2a340b0060 msgr2=0x7f2a340ab690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a33fff700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2a340b0060 0x7f2a340ab690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a33fff700 1 -- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2a340b1970 msgr2=0x7f2a34065cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a33fff700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2a340b1970 0x7f2a34065cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a33fff700 1 -- 172.21.2.104:0/79294992 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c00b180 con 0x7f2a340a76b0 2021-07-11T04:01:54.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a33fff700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a340a76b0 0x7f2a34062910 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f2a24007000 tx=0x7f2a24008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:54.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a317fa700 1 -- 172.21.2.104:0/79294992 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a24010580 con 0x7f2a340a76b0 2021-07-11T04:01:54.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a317fa700 1 -- 172.21.2.104:0/79294992 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a240106e0 con 0x7f2a340a76b0 2021-07-11T04:01:54.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a3a35b700 1 -- 172.21.2.104:0/79294992 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a340aa300 con 0x7f2a340a76b0 2021-07-11T04:01:54.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a3a35b700 1 -- 172.21.2.104:0/79294992 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a340abcd0 con 0x7f2a340a76b0 2021-07-11T04:01:54.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a317fa700 1 -- 172.21.2.104:0/79294992 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a24010a40 con 0x7f2a340a76b0 2021-07-11T04:01:54.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a3a35b700 1 -- 172.21.2.104:0/79294992 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a340ab1a0 con 0x7f2a340a76b0 2021-07-11T04:01:54.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a317fa700 1 -- 172.21.2.104:0/79294992 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2a24010ba0 con 0x7f2a340a76b0 2021-07-11T04:01:54.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a317fa700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a18053240 0x7f2a18055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a317fa700 1 -- 172.21.2.104:0/79294992 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2a2406b7f0 con 0x7f2a340a76b0 2021-07-11T04:01:54.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.808+0000 7f2a337fe700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a18053240 0x7f2a18055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.812+0000 7f2a337fe700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a18053240 0x7f2a18055700 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f2a1c002f70 tx=0x7f2a1c014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:54.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.812+0000 7f2a317fa700 1 -- 172.21.2.104:0/79294992 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2a240429c0 con 0x7f2a340a76b0 2021-07-11T04:01:54.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.920+0000 7f2a3a35b700 1 -- 172.21.2.104:0/79294992 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2a340aa490 con 0x7f2a340a76b0 2021-07-11T04:01:54.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a317fa700 1 -- 172.21.2.104:0/79294992 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v32) v1 ==== 184+0+0 (secure 0 0 0) 0x7f2a24045fe0 con 0x7f2a340a76b0 2021-07-11T04:01:54.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 -- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a18053240 msgr2=0x7f2a18055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a18053240 0x7f2a18055700 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f2a1c002f70 tx=0x7f2a1c014000).stop 2021-07-11T04:01:54.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 -- 172.21.2.104:0/79294992 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a340a76b0 msgr2=0x7f2a34062910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a340a76b0 0x7f2a34062910 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f2a24007000 tx=0x7f2a24008040).stop 2021-07-11T04:01:54.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 -- 172.21.2.104:0/79294992 shutdown_connections 2021-07-11T04:01:54.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2a340b1970 0x7f2a34065cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2a340b0060 0x7f2a340ab690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a18053240 0x7f2a18055700 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 --2- 172.21.2.104:0/79294992 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a340a76b0 0x7f2a34062910 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:54.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 -- 172.21.2.104:0/79294992 >> 172.21.2.104:0/79294992 conn(0x7f2a340b53e0 msgr2=0x7f2a34145830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:54.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 -- 172.21.2.104:0/79294992 shutdown_connections 2021-07-11T04:01:54.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.924+0000 7f2a3a35b700 1 -- 172.21.2.104:0/79294992 wait complete. 2021-07-11T04:01:54.930 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T04:01:54.933 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:01:54.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 Processor -- start 2021-07-11T04:01:54.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 -- start start 2021-07-11T04:01:54.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccc00a76b0 0x7fccc0066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc00a7a90 con 0x7fccc00b0060 2021-07-11T04:01:54.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc0147620 con 0x7fccc00b1970 2021-07-11T04:01:54.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc01478c0 con 0x7fccc00a76b0 2021-07-11T04:01:54.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc68b6700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fccc00b0060 0x7fccc0151f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47124/0 (socket says 172.21.2.104:47124) 2021-07-11T04:01:54.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc68b6700 1 -- 172.21.2.104:0/2700040297 learned_addr learned my addr 172.21.2.104:0/2700040297 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:54.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1872319405 0 0) 0x7fccc00a7a90 con 0x7fccc00b0060 2021-07-11T04:01:54.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcca8003260 con 0x7fccc00b0060 2021-07-11T04:01:54.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 449258802 0 0) 0x7fcca8003260 con 0x7fccc00b0060 2021-07-11T04:01:54.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccc00a76b0 msgr2=0x7fccc0066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:54.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 --2- 172.21.2.104:0/2700040297 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccc00a76b0 0x7fccc0066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 >> v1:172.21.2.105:6789/0 conn(0x7fccc00b1970 legacy=0x7fccc005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:54.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0147b60 con 0x7fccc00b0060 2021-07-11T04:01:54.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fccb80019e0 con 0x7fccc00b0060 2021-07-11T04:01:54.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 >> v1:172.21.2.104:6789/0 conn(0x7fccc00b0060 legacy=0x7fccc0151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 --2- 172.21.2.104:0/2700040297 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccc00a76b0 0x7fcca8003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 --2- 172.21.2.104:0/2700040297 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcca8003ef0 0x7fcca8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 --2- 172.21.2.104:0/2700040297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca8004bf0 0x7fcca800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:54.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc01478c0 con 0x7fcca8004bf0 2021-07-11T04:01:54.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca80052f0 con 0x7fcca8003ef0 2021-07-11T04:01:54.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca800bc40 con 0x7fccc00a76b0 2021-07-11T04:01:54.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc68b6700 1 --2- 172.21.2.104:0/2700040297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca8004bf0 0x7fcca800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:54.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc68b6700 1 -- 172.21.2.104:0/2700040297 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccc00a76b0 msgr2=0x7fcca8003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:54.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc68b6700 1 --2- 172.21.2.104:0/2700040297 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccc00a76b0 0x7fcca8003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc68b6700 1 -- 172.21.2.104:0/2700040297 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcca8003ef0 msgr2=0x7fcca8004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:54.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc68b6700 1 --2- 172.21.2.104:0/2700040297 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcca8003ef0 0x7fcca8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:54.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc68b6700 1 -- 172.21.2.104:0/2700040297 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8003260 con 0x7fcca8004bf0 2021-07-11T04:01:54.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc68b6700 1 --2- 172.21.2.104:0/2700040297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca8004bf0 0x7fcca800b750 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7fccc0148420 tx=0x7fccb80020e0).ready entity=mon.0 client_cookie=9e772c5b4a28130a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:54.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccb800cc00 con 0x7fcca8004bf0 2021-07-11T04:01:54.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb800cd60 con 0x7fcca8004bf0 2021-07-11T04:01:54.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc50b3700 1 -- 172.21.2.104:0/2700040297 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccb8005b80 con 0x7fcca8004bf0 2021-07-11T04:01:54.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 -- 172.21.2.104:0/2700040297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca8004bf0 msgr2=0x7fcca800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:54.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 --2- 172.21.2.104:0/2700040297 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcca8004bf0 0x7fcca800b750 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7fccc0148420 tx=0x7fccb80020e0).stop 2021-07-11T04:01:54.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc68b6700 1 -- 172.21.2.104:0/2700040297 reap_dead start 2021-07-11T04:01:54.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 -- 172.21.2.104:0/2700040297 shutdown_connections 2021-07-11T04:01:55.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 -- 172.21.2.104:0/2700040297 >> 172.21.2.104:0/2700040297 conn(0x7fccc00b53e0 msgr2=0x7fccc00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:55.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 -- 172.21.2.104:0/2700040297 shutdown_connections 2021-07-11T04:01:55.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.988+0000 7fccc8319700 1 -- 172.21.2.104:0/2700040297 wait complete. 2021-07-11T04:01:55.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc8319700 1 Processor -- start 2021-07-11T04:01:55.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc8319700 1 -- start start 2021-07-11T04:01:55.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc8319700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccc014dc10 0x7fccc014dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc8319700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccc014e6b0 0x7fccc0146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc8319700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccc014c3e0 0x7fccc0146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc8319700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc01470e0 con 0x7fccc014c3e0 2021-07-11T04:01:55.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc8319700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc014c9c0 con 0x7fccc014e6b0 2021-07-11T04:01:55.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc8319700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc014d660 con 0x7fccc014dc10 2021-07-11T04:01:55.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc68b6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccc014c3e0 0x7fccc0146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc68b6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccc014c3e0 0x7fccc0146bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50370/0 (socket says 172.21.2.104:50370) 2021-07-11T04:01:55.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc68b6700 1 -- 172.21.2.104:0/2668462458 learned_addr learned my addr 172.21.2.104:0/2668462458 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:55.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc68b6700 1 -- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccc014dc10 msgr2=0x7fccc014dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc68b6700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccc014dc10 0x7fccc014dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc68b6700 1 -- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccc014e6b0 msgr2=0x7fccc0146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc68b6700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccc014e6b0 0x7fccc0146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.003 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc68b6700 1 -- 172.21.2.104:0/2668462458 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc014d900 con 0x7fccc014c3e0 2021-07-11T04:01:55.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc68b6700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccc014c3e0 0x7fccc0146bf0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7fccc00ad510 tx=0x7fccb8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:55.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccb37fe700 1 -- 172.21.2.104:0/2668462458 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccb8018030 con 0x7fccc014c3e0 2021-07-11T04:01:55.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccb37fe700 1 -- 172.21.2.104:0/2668462458 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb8004b50 con 0x7fccc014c3e0 2021-07-11T04:01:55.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccb37fe700 1 -- 172.21.2.104:0/2668462458 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fccb80020e0 con 0x7fccc014c3e0 2021-07-11T04:01:55.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc8319700 1 -- 172.21.2.104:0/2668462458 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc014cbf0 con 0x7fccc014c3e0 2021-07-11T04:01:55.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc8319700 1 -- 172.21.2.104:0/2668462458 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc0156510 con 0x7fccc014c3e0 2021-07-11T04:01:55.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc8319700 1 -- 172.21.2.104:0/2668462458 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc014cd80 con 0x7fccc014c3e0 2021-07-11T04:01:55.006 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccb37fe700 1 -- 172.21.2.104:0/2668462458 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fccb8026070 con 0x7fccc014c3e0 2021-07-11T04:01:55.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccb37fe700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcca00532d0 0x7fcca0055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccb37fe700 1 -- 172.21.2.104:0/2668462458 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fccb806bb60 con 0x7fccc014c3e0 2021-07-11T04:01:55.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.992+0000 7fccc60b5700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcca00532d0 0x7fcca0055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.996+0000 7fccc60b5700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcca00532d0 0x7fcca0055790 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fccbc002f70 tx=0x7fccbc006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:55.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:54.996+0000 7fccb37fe700 1 -- 172.21.2.104:0/2668462458 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fccb8042d90 con 0x7fccc014c3e0 2021-07-11T04:01:55.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.104+0000 7fccc8319700 1 -- 172.21.2.104:0/2668462458 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7fccc0057350 con 0x7fccc014c3e0 2021-07-11T04:01:55.108 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.108+0000 7fccb37fe700 1 -- 172.21.2.104:0/2668462458 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7fccb801a030 con 0x7fccc014c3e0 2021-07-11T04:01:55.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.108+0000 7fccc8319700 1 -- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcca00532d0 msgr2=0x7fcca0055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.108+0000 7fccc8319700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcca00532d0 0x7fcca0055790 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fccbc002f70 tx=0x7fccbc006a10).stop 2021-07-11T04:01:55.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.112+0000 7fccc8319700 1 -- 172.21.2.104:0/2668462458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccc014c3e0 msgr2=0x7fccc0146bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.112+0000 7fccc8319700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccc014c3e0 0x7fccc0146bf0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7fccc00ad510 tx=0x7fccb8015000).stop 2021-07-11T04:01:55.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.112+0000 7fccc8319700 1 -- 172.21.2.104:0/2668462458 shutdown_connections 2021-07-11T04:01:55.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.112+0000 7fccc8319700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcca00532d0 0x7fcca0055790 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.112+0000 7fccc8319700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fccc014c3e0 0x7fccc0146bf0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.112+0000 7fccc8319700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fccc014e6b0 0x7fccc0146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.112+0000 7fccc8319700 1 --2- 172.21.2.104:0/2668462458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fccc014dc10 0x7fccc014dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.112+0000 7fccc8319700 1 -- 172.21.2.104:0/2668462458 >> 172.21.2.104:0/2668462458 conn(0x7fccc00b53e0 msgr2=0x7fccc00a7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:55.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.112+0000 7fccc8319700 1 -- 172.21.2.104:0/2668462458 shutdown_connections 2021-07-11T04:01:55.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.112+0000 7fccc8319700 1 -- 172.21.2.104:0/2668462458 wait complete. 2021-07-11T04:01:55.114 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T04:01:55.117 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:55.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.168+0000 7f008f2b5700 1 Processor -- start 2021-07-11T04:01:55.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.168+0000 7f008f2b5700 1 -- start start 2021-07-11T04:01:55.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.168+0000 7f008f2b5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f00881393d0 0x7f00880b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.168+0000 7f008f2b5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0088142840 0x7f0088142ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.168+0000 7f008f2b5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00880604e0 con 0x7f0088060100 2021-07-11T04:01:55.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.168+0000 7f008f2b5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00880b16b0 con 0x7f00881393d0 2021-07-11T04:01:55.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.168+0000 7f008f2b5700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00880ac400 con 0x7f0088142840 2021-07-11T04:01:55.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.168+0000 7f008c850700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f00881393d0 0x7f00880b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.168+0000 7f008c850700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f00881393d0 0x7f00880b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50378/0 (socket says 172.21.2.104:50378) 2021-07-11T04:01:55.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.168+0000 7f008c850700 1 -- 172.21.2.104:0/4174170041 learned_addr learned my addr 172.21.2.104:0/4174170041 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:55.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008c850700 1 -- 172.21.2.104:0/4174170041 >> v2:172.21.2.105:3300/0 conn(0x7f0088142840 msgr2=0x7f0088142ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:55.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008c850700 1 --2- 172.21.2.104:0/4174170041 >> v2:172.21.2.105:3300/0 conn(0x7f0088142840 0x7f0088142ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008c850700 1 -- 172.21.2.104:0/4174170041 >> v1:172.21.2.105:6789/0 conn(0x7f0088060100 legacy=0x7f0088138cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:55.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008c850700 1 -- 172.21.2.104:0/4174170041 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880ac6a0 con 0x7f00881393d0 2021-07-11T04:01:55.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008c850700 1 --2- 172.21.2.104:0/4174170041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00881393d0 0x7f00880b11c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f0088061a70 tx=0x7f0084006f40).ready entity=mon.0 client_cookie=b6a00fbe3255d116 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:55.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f0077fff700 1 -- 172.21.2.104:0/4174170041 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0084010070 con 0x7f00881393d0 2021-07-11T04:01:55.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f0077fff700 1 -- 172.21.2.104:0/4174170041 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084000e90 con 0x7f00881393d0 2021-07-11T04:01:55.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f0077fff700 1 -- 172.21.2.104:0/4174170041 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0084008860 con 0x7f00881393d0 2021-07-11T04:01:55.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- 172.21.2.104:0/4174170041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00881393d0 msgr2=0x7f00880b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 --2- 172.21.2.104:0/4174170041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00881393d0 0x7f00880b11c0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f0088061a70 tx=0x7f0084006f40).stop 2021-07-11T04:01:55.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- 172.21.2.104:0/4174170041 shutdown_connections 2021-07-11T04:01:55.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 --2- 172.21.2.104:0/4174170041 >> v2:172.21.2.105:3300/0 conn(0x7f0088142840 0x7f0088142ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 --2- 172.21.2.104:0/4174170041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f00881393d0 0x7f00880b11c0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- 172.21.2.104:0/4174170041 >> 172.21.2.104:0/4174170041 conn(0x7f0088061070 msgr2=0x7f008805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:55.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- 172.21.2.104:0/4174170041 shutdown_connections 2021-07-11T04:01:55.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- 172.21.2.104:0/4174170041 wait complete. 2021-07-11T04:01:55.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 Processor -- start 2021-07-11T04:01:55.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- start start 2021-07-11T04:01:55.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0088060100 0x7f00880b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00881393d0 0x7f008814aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0088142840 0x7f008814b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008813de00 con 0x7f0088142840 2021-07-11T04:01:55.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008813d030 con 0x7f0088060100 2021-07-11T04:01:55.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f008814d260 con 0x7f00881393d0 2021-07-11T04:01:55.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008d852700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0088142840 0x7f008814b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008d852700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0088142840 0x7f008814b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50382/0 (socket says 172.21.2.104:50382) 2021-07-11T04:01:55.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008d852700 1 -- 172.21.2.104:0/1367893190 learned_addr learned my addr 172.21.2.104:0/1367893190 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:55.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008d852700 1 -- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00881393d0 msgr2=0x7f008814aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008d852700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00881393d0 0x7f008814aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008d852700 1 -- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0088060100 msgr2=0x7f00880b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008d852700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0088060100 0x7f00880b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008d852700 1 -- 172.21.2.104:0/1367893190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0084008240 con 0x7f0088142840 2021-07-11T04:01:55.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008d852700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0088142840 0x7f008814b3b0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f008813dbf0 tx=0x7f00800090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:55.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f00767fc700 1 -- 172.21.2.104:0/1367893190 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00800014e0 con 0x7f0088142840 2021-07-11T04:01:55.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f00767fc700 1 -- 172.21.2.104:0/1367893190 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0080001640 con 0x7f0088142840 2021-07-11T04:01:55.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- 172.21.2.104:0/1367893190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0088140570 con 0x7f0088142840 2021-07-11T04:01:55.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- 172.21.2.104:0/1367893190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008813e790 con 0x7f0088142840 2021-07-11T04:01:55.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f00767fc700 1 -- 172.21.2.104:0/1367893190 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f00800019a0 con 0x7f0088142840 2021-07-11T04:01:55.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.172+0000 7f008f2b5700 1 -- 172.21.2.104:0/1367893190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008813ee70 con 0x7f0088142840 2021-07-11T04:01:55.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.176+0000 7f00767fc700 1 -- 172.21.2.104:0/1367893190 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0080018030 con 0x7f0088142840 2021-07-11T04:01:55.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.176+0000 7f00767fc700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00680532d0 0x7f0068055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.176+0000 7f00767fc700 1 -- 172.21.2.104:0/1367893190 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f008006af00 con 0x7f0088142840 2021-07-11T04:01:55.190 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.176+0000 7f008d051700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00680532d0 0x7f0068055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.176+0000 7f008d051700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00680532d0 0x7f0068055790 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f0078002fb0 tx=0x7f00780069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:55.191 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.176+0000 7f00767fc700 1 -- 172.21.2.104:0/1367893190 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f00800421e0 con 0x7f0088142840 2021-07-11T04:01:55.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.284+0000 7f008f2b5700 1 -- 172.21.2.104:0/1367893190 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0088057350 con 0x7f0088142840 2021-07-11T04:01:55.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.284+0000 7f00767fc700 1 -- 172.21.2.104:0/1367893190 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f0080016070 con 0x7f0088142840 2021-07-11T04:01:55.286 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:55.286 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5335,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/1814518027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":1814518027},{"type":"v1","addr":"172.21.2.105:6835","nonce":1814518027}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/155502403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":155502403},{"type":"v1","addr":"172.21.2.104:6834","nonce":155502403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:52.876988+0000","modified":"2021-07-11T04:01:54.362899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/2616834266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":2616834266},{"type":"v1","addr":"172.21.2.104:6835","nonce":2616834266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-07-11T04:01:55.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 -- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00680532d0 msgr2=0x7f0068055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00680532d0 0x7f0068055790 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f0078002fb0 tx=0x7f00780069b0).stop 2021-07-11T04:01:55.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 -- 172.21.2.104:0/1367893190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0088142840 msgr2=0x7f008814b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0088142840 0x7f008814b3b0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f008813dbf0 tx=0x7f00800090f0).stop 2021-07-11T04:01:55.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 -- 172.21.2.104:0/1367893190 shutdown_connections 2021-07-11T04:01:55.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f00680532d0 0x7f0068055790 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0088142840 0x7f008814b3b0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f00881393d0 0x7f008814aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 --2- 172.21.2.104:0/1367893190 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0088060100 0x7f00880b4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 -- 172.21.2.104:0/1367893190 >> 172.21.2.104:0/1367893190 conn(0x7f0088061070 msgr2=0x7f00880b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:55.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 -- 172.21.2.104:0/1367893190 shutdown_connections 2021-07-11T04:01:55.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.288+0000 7f008f2b5700 1 -- 172.21.2.104:0/1367893190 wait complete. 2021-07-11T04:01:55.291 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 68 2021-07-11T04:01:55.596 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:01:52.876988+0000', 'modified': '2021-07-11T04:01:54.362899+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8022}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8022': {'gid': 8022, 'name': 'a', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/2616834266', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 2616834266}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 2616834266}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:01:55.597 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:01:55.597 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-07-11T04:01:55.662 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":8022,"want_state":"up:active","state":"up:active","rank_uptime":2.7800433309999999,"mdsmap_epoch":68,"osdmap_epoch":139,"osdmap_epoch_barrier":139,"uptime":7.0378728410000004} 2021-07-11T04:01:55.662 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 8022, "mdsmap_epoch": 68, "osdmap_epoch": 139, "osdmap_epoch_barrier": 139, "rank_uptime": 2.780043331, "state": "up:active", "uptime": 7.037872841, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:01:55.663 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:55.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.712+0000 7f12f3da6700 1 Processor -- start 2021-07-11T04:01:55.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.712+0000 7f12f3da6700 1 -- start start 2021-07-11T04:01:55.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12ec0a76b0 0x7f12ec066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f12ec0b1970 0x7f12ec066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ec067450 con 0x7f12ec0b0060 2021-07-11T04:01:55.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ec0a7a90 con 0x7f12ec0b1970 2021-07-11T04:01:55.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec147620 con 0x7f12ec0a76b0 2021-07-11T04:01:55.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f2343700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f12ec0b0060 0x7f12ec05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47154/0 (socket says 172.21.2.104:47154) 2021-07-11T04:01:55.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f2343700 1 -- 172.21.2.104:0/3119390260 learned_addr learned my addr 172.21.2.104:0/3119390260 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:55.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248558319 0 0) 0x7f12ec067450 con 0x7f12ec0b0060 2021-07-11T04:01:55.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12d4003260 con 0x7f12ec0b0060 2021-07-11T04:01:55.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f1341700 1 --2- 172.21.2.104:0/3119390260 >> v2:172.21.2.105:3300/0 conn(0x7f12ec0b1970 0x7f12ec066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f1b42700 1 --2- 172.21.2.104:0/3119390260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12ec0a76b0 0x7f12ec066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1636519729 0 0) 0x7f12d4003260 con 0x7f12ec0b0060 2021-07-11T04:01:55.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12ec0a76b0 msgr2=0x7f12ec066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:55.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 --2- 172.21.2.104:0/3119390260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12ec0a76b0 0x7f12ec066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 >> v2:172.21.2.105:3300/0 conn(0x7f12ec0b1970 msgr2=0x7f12ec066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:55.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 --2- 172.21.2.104:0/3119390260 >> v2:172.21.2.105:3300/0 conn(0x7f12ec0b1970 0x7f12ec066f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec1478c0 con 0x7f12ec0b0060 2021-07-11T04:01:55.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f12e40019e0 con 0x7f12ec0b0060 2021-07-11T04:01:55.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 >> v1:172.21.2.104:6789/0 conn(0x7f12ec0b0060 legacy=0x7f12ec05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 --2- 172.21.2.104:0/3119390260 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12d40037d0 0x7f12d4003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 --2- 172.21.2.104:0/3119390260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12ec0a76b0 0x7f12d4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 --2- 172.21.2.104:0/3119390260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d40049f0 0x7f12d4004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec0a7a90 con 0x7f12d40049f0 2021-07-11T04:01:55.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec147620 con 0x7f12d40037d0 2021-07-11T04:01:55.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d400b880 con 0x7f12ec0a76b0 2021-07-11T04:01:55.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f2343700 1 --2- 172.21.2.104:0/3119390260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d40049f0 0x7f12d4004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f2343700 1 -- 172.21.2.104:0/3119390260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12ec0a76b0 msgr2=0x7f12d4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:55.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f2343700 1 --2- 172.21.2.104:0/3119390260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12ec0a76b0 0x7f12d4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f2343700 1 -- 172.21.2.104:0/3119390260 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12d40037d0 msgr2=0x7f12d4003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:55.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f2343700 1 --2- 172.21.2.104:0/3119390260 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12d40037d0 0x7f12d4003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f2343700 1 -- 172.21.2.104:0/3119390260 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d4003260 con 0x7f12d40049f0 2021-07-11T04:01:55.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f2343700 1 --2- 172.21.2.104:0/3119390260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d40049f0 0x7f12d4004e70 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f12e40016f0 tx=0x7f12e4002100).ready entity=mon.0 client_cookie=54b7f43a9f29b18 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:55.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e400cb80 con 0x7f12d40049f0 2021-07-11T04:01:55.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e400cce0 con 0x7f12d40049f0 2021-07-11T04:01:55.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f0b40700 1 -- 172.21.2.104:0/3119390260 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e4005b80 con 0x7f12d40049f0 2021-07-11T04:01:55.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 -- 172.21.2.104:0/3119390260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d40049f0 msgr2=0x7f12d4004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 --2- 172.21.2.104:0/3119390260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d40049f0 0x7f12d4004e70 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f12e40016f0 tx=0x7f12e4002100).stop 2021-07-11T04:01:55.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f2343700 1 -- 172.21.2.104:0/3119390260 reap_dead start 2021-07-11T04:01:55.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 -- 172.21.2.104:0/3119390260 shutdown_connections 2021-07-11T04:01:55.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 -- 172.21.2.104:0/3119390260 >> 172.21.2.104:0/3119390260 conn(0x7f12ec0b53e0 msgr2=0x7f12ec0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:55.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 -- 172.21.2.104:0/3119390260 shutdown_connections 2021-07-11T04:01:55.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 -- 172.21.2.104:0/3119390260 wait complete. 2021-07-11T04:01:55.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 Processor -- start 2021-07-11T04:01:55.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.716+0000 7f12f3da6700 1 -- start start 2021-07-11T04:01:55.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f3da6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12ec14c0f0 0x7f12ec159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f3da6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12ec0ac520 0x7f12ec15a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f3da6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12ec0aafd0 0x7f12ec1442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f3da6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec14c4d0 con 0x7f12ec0aafd0 2021-07-11T04:01:55.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f3da6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec0ae7b0 con 0x7f12ec0ac520 2021-07-11T04:01:55.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f3da6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec0ac250 con 0x7f12ec14c0f0 2021-07-11T04:01:55.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f2343700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12ec0aafd0 0x7f12ec1442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f2343700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12ec0aafd0 0x7f12ec1442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50398/0 (socket says 172.21.2.104:50398) 2021-07-11T04:01:55.734 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f2343700 1 -- 172.21.2.104:0/1891662427 learned_addr learned my addr 172.21.2.104:0/1891662427 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:55.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f2343700 1 -- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12ec14c0f0 msgr2=0x7f12ec159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f2343700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12ec14c0f0 0x7f12ec159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.735 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f2343700 1 -- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12ec0ac520 msgr2=0x7f12ec15a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f2343700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12ec0ac520 0x7f12ec15a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f2343700 1 -- 172.21.2.104:0/1891662427 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec0ab5d0 con 0x7f12ec0aafd0 2021-07-11T04:01:55.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f2343700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12ec0aafd0 0x7f12ec1442a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f12ec0a8ee0 tx=0x7f12e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:55.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12deffd700 1 -- 172.21.2.104:0/1891662427 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e40074d0 con 0x7f12ec0aafd0 2021-07-11T04:01:55.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12deffd700 1 -- 172.21.2.104:0/1891662427 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e4007630 con 0x7f12ec0aafd0 2021-07-11T04:01:55.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f3da6700 1 -- 172.21.2.104:0/1891662427 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ec0ab870 con 0x7f12ec0aafd0 2021-07-11T04:01:55.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12deffd700 1 -- 172.21.2.104:0/1891662427 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12e4007aa0 con 0x7f12ec0aafd0 2021-07-11T04:01:55.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f3da6700 1 -- 172.21.2.104:0/1891662427 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ec0abdb0 con 0x7f12ec0aafd0 2021-07-11T04:01:55.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f3da6700 1 -- 172.21.2.104:0/1891662427 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ec0abf70 con 0x7f12ec0aafd0 2021-07-11T04:01:55.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12deffd700 1 -- 172.21.2.104:0/1891662427 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f12e4010070 con 0x7f12ec0aafd0 2021-07-11T04:01:55.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12deffd700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12cc053330 0x7f12cc0557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12deffd700 1 -- 172.21.2.104:0/1891662427 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f12e406bca0 con 0x7f12ec0aafd0 2021-07-11T04:01:55.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.720+0000 7f12f1b42700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12cc053330 0x7f12cc0557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.724+0000 7f12f1b42700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12cc053330 0x7f12cc0557f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f12e000a000 tx=0x7f12e0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:55.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.724+0000 7f12deffd700 1 -- 172.21.2.104:0/1891662427 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f12e4042ed0 con 0x7f12ec0aafd0 2021-07-11T04:01:55.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.804+0000 7f12f3da6700 1 -- 172.21.2.104:0/1891662427 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12ec057350 con 0x7f12ec0aafd0 2021-07-11T04:01:55.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.804+0000 7f12deffd700 1 -- 172.21.2.104:0/1891662427 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f12e40464f0 con 0x7f12ec0aafd0 2021-07-11T04:01:55.805 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:55.806 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":139,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:53.880137+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:01:50.686994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-07-11T04:01:51.853293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:55.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 -- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12cc053330 msgr2=0x7f12cc0557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12cc053330 0x7f12cc0557f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f12e000a000 tx=0x7f12e0008040).stop 2021-07-11T04:01:55.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 -- 172.21.2.104:0/1891662427 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12ec0aafd0 msgr2=0x7f12ec1442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12ec0aafd0 0x7f12ec1442a0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f12ec0a8ee0 tx=0x7f12e4015000).stop 2021-07-11T04:01:55.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 -- 172.21.2.104:0/1891662427 shutdown_connections 2021-07-11T04:01:55.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12cc053330 0x7f12cc0557f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12ec0aafd0 0x7f12ec1442a0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12ec0ac520 0x7f12ec15a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 --2- 172.21.2.104:0/1891662427 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12ec14c0f0 0x7f12ec159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 -- 172.21.2.104:0/1891662427 >> 172.21.2.104:0/1891662427 conn(0x7f12ec0b53e0 msgr2=0x7f12ec0a7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:55.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 -- 172.21.2.104:0/1891662427 shutdown_connections 2021-07-11T04:01:55.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.808+0000 7f12f3da6700 1 -- 172.21.2.104:0/1891662427 wait complete. 2021-07-11T04:01:55.813 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:01:50.686994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:01:51.853293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:01:55.814 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:01:55.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 Processor -- start 2021-07-11T04:01:55.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 -- start start 2021-07-11T04:01:55.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faa1c0b3710 0x7faa1c0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7faa1c0a9c10 0x7faa1c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c143920 con 0x7faa1c0b05b0 2021-07-11T04:01:55.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c143bc0 con 0x7faa1c0b3710 2021-07-11T04:01:55.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c143e60 con 0x7faa1c0a9c10 2021-07-11T04:01:55.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa1a7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faa1c0b3710 0x7faa1c0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa1affd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7faa1c0b05b0 0x7faa1c13d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47174/0 (socket says 172.21.2.104:47174) 2021-07-11T04:01:55.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa1affd700 1 -- 172.21.2.104:0/160099076 learned_addr learned my addr 172.21.2.104:0/160099076 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:55.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa1a7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faa1c0b3710 0x7faa1c0b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50410/0 (socket says 172.21.2.104:50410) 2021-07-11T04:01:55.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa1a7fc700 1 -- 172.21.2.104:0/160099076 >> v2:172.21.2.105:3300/0 conn(0x7faa1c0a9c10 msgr2=0x7faa1c0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:55.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa1a7fc700 1 --2- 172.21.2.104:0/160099076 >> v2:172.21.2.105:3300/0 conn(0x7faa1c0a9c10 0x7faa1c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa1a7fc700 1 -- 172.21.2.104:0/160099076 >> v1:172.21.2.104:6789/0 conn(0x7faa1c0b05b0 legacy=0x7faa1c13d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa1a7fc700 1 -- 172.21.2.104:0/160099076 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c144100 con 0x7faa1c0b3710 2021-07-11T04:01:55.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa1a7fc700 1 --2- 172.21.2.104:0/160099076 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa1c0b3710 0x7faa1c0b0990 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7faa0c002f70 tx=0x7faa0c006df0).ready entity=mon.0 client_cookie=d7a110cfc02cb9cc server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:55.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa19ffb700 1 -- 172.21.2.104:0/160099076 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa0c00d5c0 con 0x7faa1c0b3710 2021-07-11T04:01:55.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa19ffb700 1 -- 172.21.2.104:0/160099076 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0c000e90 con 0x7faa1c0b3710 2021-07-11T04:01:55.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa19ffb700 1 -- 172.21.2.104:0/160099076 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa0c00d910 con 0x7faa1c0b3710 2021-07-11T04:01:55.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 -- 172.21.2.104:0/160099076 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa1c0b3710 msgr2=0x7faa1c0b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 --2- 172.21.2.104:0/160099076 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa1c0b3710 0x7faa1c0b0990 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7faa0c002f70 tx=0x7faa0c006df0).stop 2021-07-11T04:01:55.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 -- 172.21.2.104:0/160099076 shutdown_connections 2021-07-11T04:01:55.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 --2- 172.21.2.104:0/160099076 >> v2:172.21.2.105:3300/0 conn(0x7faa1c0a9c10 0x7faa1c0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 --2- 172.21.2.104:0/160099076 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa1c0b3710 0x7faa1c0b0990 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 -- 172.21.2.104:0/160099076 >> 172.21.2.104:0/160099076 conn(0x7faa1c0607c0 msgr2=0x7faa1c060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:55.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 -- 172.21.2.104:0/160099076 shutdown_connections 2021-07-11T04:01:55.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.868+0000 7faa218a1700 1 -- 172.21.2.104:0/160099076 wait complete. 2021-07-11T04:01:55.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa218a1700 1 Processor -- start 2021-07-11T04:01:55.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa218a1700 1 -- start start 2021-07-11T04:01:55.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa218a1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa1c0a9c10 0x7faa1c064c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa218a1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faa1c0b05b0 0x7faa1c13cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa218a1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa1c0b3710 0x7faa1c13cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa218a1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c14ea80 con 0x7faa1c0b3710 2021-07-11T04:01:55.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa218a1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c13f6f0 con 0x7faa1c0b05b0 2021-07-11T04:01:55.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa218a1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c14ecb0 con 0x7faa1c0a9c10 2021-07-11T04:01:55.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa1b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa1c0b3710 0x7faa1c13cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa1b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa1c0b3710 0x7faa1c13cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50416/0 (socket says 172.21.2.104:50416) 2021-07-11T04:01:55.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa1b7fe700 1 -- 172.21.2.104:0/795589390 learned_addr learned my addr 172.21.2.104:0/795589390 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:55.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa1b7fe700 1 -- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa1c0a9c10 msgr2=0x7faa1c064c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa1b7fe700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa1c0a9c10 0x7faa1c064c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa1b7fe700 1 -- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faa1c0b05b0 msgr2=0x7faa1c13cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa1b7fe700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faa1c0b05b0 0x7faa1c13cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa1b7fe700 1 -- 172.21.2.104:0/795589390 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c00b180 con 0x7faa1c0b3710 2021-07-11T04:01:55.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa1b7fe700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa1c0b3710 0x7faa1c13cfc0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7faa140032f0 tx=0x7faa14009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:55.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa13fff700 1 -- 172.21.2.104:0/795589390 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1400d580 con 0x7faa1c0b3710 2021-07-11T04:01:55.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa13fff700 1 -- 172.21.2.104:0/795589390 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1400d6e0 con 0x7faa1c0b3710 2021-07-11T04:01:55.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa218a1700 1 -- 172.21.2.104:0/795589390 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c14ef50 con 0x7faa1c0b3710 2021-07-11T04:01:55.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa13fff700 1 -- 172.21.2.104:0/795589390 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa1400da40 con 0x7faa1c0b3710 2021-07-11T04:01:55.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa218a1700 1 -- 172.21.2.104:0/795589390 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c13e450 con 0x7faa1c0b3710 2021-07-11T04:01:55.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa218a1700 1 -- 172.21.2.104:0/795589390 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c13fb20 con 0x7faa1c0b3710 2021-07-11T04:01:55.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa13fff700 1 -- 172.21.2.104:0/795589390 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7faa1400dba0 con 0x7faa1c0b3710 2021-07-11T04:01:55.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa13fff700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa000531f0 0x7faa000556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:55.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa13fff700 1 -- 172.21.2.104:0/795589390 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7faa1406b850 con 0x7faa1c0b3710 2021-07-11T04:01:55.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.872+0000 7faa1affd700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa000531f0 0x7faa000556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:55.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.876+0000 7faa1affd700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa000531f0 0x7faa000556b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7faa1c06aa50 tx=0x7faa04007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:55.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.876+0000 7faa13fff700 1 -- 172.21.2.104:0/795589390 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7faa1c13fb20 con 0x7faa1c0b3710 2021-07-11T04:01:55.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.956+0000 7faa218a1700 1 -- 172.21.2.104:0/795589390 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7faa1c13fb20 con 0x7faa1c0b3710 2021-07-11T04:01:55.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.956+0000 7faa13fff700 1 -- 172.21.2.104:0/795589390 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v139) v1 ==== 110+0+10 (secure 0 0 0) 0x7faa1c13fb20 con 0x7faa1c0b3710 2021-07-11T04:01:55.958 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:01:55.959 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 -- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa000531f0 msgr2=0x7faa000556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa000531f0 0x7faa000556b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7faa1c06aa50 tx=0x7faa04007800).stop 2021-07-11T04:01:55.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 -- 172.21.2.104:0/795589390 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa1c0b3710 msgr2=0x7faa1c13cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:55.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa1c0b3710 0x7faa1c13cfc0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7faa140032f0 tx=0x7faa14009220).stop 2021-07-11T04:01:55.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 -- 172.21.2.104:0/795589390 shutdown_connections 2021-07-11T04:01:55.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa000531f0 0x7faa000556b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faa1c0b3710 0x7faa1c13cfc0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:55.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faa1c0b05b0 0x7faa1c13cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 --2- 172.21.2.104:0/795589390 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faa1c0a9c10 0x7faa1c064c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:55.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 -- 172.21.2.104:0/795589390 >> 172.21.2.104:0/795589390 conn(0x7faa1c0607c0 msgr2=0x7faa1c0b4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:55.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 -- 172.21.2.104:0/795589390 shutdown_connections 2021-07-11T04:01:55.962 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:55.960+0000 7faa218a1700 1 -- 172.21.2.104:0/795589390 wait complete. 2021-07-11T04:01:55.965 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:01:56.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 Processor -- start 2021-07-11T04:01:56.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 -- start start 2021-07-11T04:01:56.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4e80b1ec0 0x7ff4e80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e80a77d0 con 0x7ff4e80affb0 2021-07-11T04:01:56.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e80b03f0 con 0x7ff4e80b22a0 2021-07-11T04:01:56.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80b0650 con 0x7ff4e80b1ec0 2021-07-11T04:01:56.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e67fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff4e80affb0 0x7ff4e80a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47192/0 (socket says 172.21.2.104:47192) 2021-07-11T04:01:56.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e67fc700 1 -- 172.21.2.104:0/3066567533 learned_addr learned my addr 172.21.2.104:0/3066567533 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:56.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953830902 0 0) 0x7ff4e80a77d0 con 0x7ff4e80affb0 2021-07-11T04:01:56.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4c8003260 con 0x7ff4e80affb0 2021-07-11T04:01:56.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e6ffd700 1 --2- 172.21.2.104:0/3066567533 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4e80b1ec0 0x7ff4e80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 804739297 0 0) 0x7ff4c8003260 con 0x7ff4e80affb0 2021-07-11T04:01:56.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4e80b1ec0 msgr2=0x7ff4e80b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:56.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 --2- 172.21.2.104:0/3066567533 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4e80b1ec0 0x7ff4e80b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 >> v1:172.21.2.105:6789/0 conn(0x7ff4e80b22a0 legacy=0x7ff4e8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:56.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e80b08f0 con 0x7ff4e80affb0 2021-07-11T04:01:56.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7ff4d0001660 con 0x7ff4e80affb0 2021-07-11T04:01:56.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 >> v1:172.21.2.104:6789/0 conn(0x7ff4e80affb0 legacy=0x7ff4e80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 --2- 172.21.2.104:0/3066567533 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4e80b1ec0 0x7ff4c8003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 --2- 172.21.2.104:0/3066567533 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4c8003e20 0x7ff4c8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 --2- 172.21.2.104:0/3066567533 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff4c8004b10 0x7ff4c8004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80b0650 con 0x7ff4c8003e20 2021-07-11T04:01:56.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c800b960 con 0x7ff4c8004b10 2021-07-11T04:01:56.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c800bb20 con 0x7ff4e80b1ec0 2021-07-11T04:01:56.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e67fc700 1 --2- 172.21.2.104:0/3066567533 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4c8003e20 0x7ff4c8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e67fc700 1 -- 172.21.2.104:0/3066567533 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4e80b1ec0 msgr2=0x7ff4c8003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:56.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e67fc700 1 --2- 172.21.2.104:0/3066567533 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4e80b1ec0 0x7ff4c8003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e67fc700 1 -- 172.21.2.104:0/3066567533 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff4c8004b10 msgr2=0x7ff4c8004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:56.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e67fc700 1 --2- 172.21.2.104:0/3066567533 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff4c8004b10 0x7ff4c8004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e67fc700 1 -- 172.21.2.104:0/3066567533 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c8003260 con 0x7ff4c8003e20 2021-07-11T04:01:56.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e67fc700 1 --2- 172.21.2.104:0/3066567533 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4c8003e20 0x7ff4c8004500 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7ff4e806a190 tx=0x7ff4e80b2a00).ready entity=mon.0 client_cookie=31a895cf1f64f112 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4d00070d0 con 0x7ff4c8003e20 2021-07-11T04:01:56.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d0007230 con 0x7ff4c8003e20 2021-07-11T04:01:56.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e5ffb700 1 -- 172.21.2.104:0/3066567533 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4d000c9c0 con 0x7ff4c8003e20 2021-07-11T04:01:56.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 -- 172.21.2.104:0/3066567533 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4c8003e20 msgr2=0x7ff4c8004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 --2- 172.21.2.104:0/3066567533 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4c8003e20 0x7ff4c8004500 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7ff4e806a190 tx=0x7ff4e80b2a00).stop 2021-07-11T04:01:56.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4e77fe700 1 -- 172.21.2.104:0/3066567533 reap_dead start 2021-07-11T04:01:56.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 -- 172.21.2.104:0/3066567533 shutdown_connections 2021-07-11T04:01:56.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 -- 172.21.2.104:0/3066567533 >> 172.21.2.104:0/3066567533 conn(0x7ff4e805f6b0 msgr2=0x7ff4e805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:56.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 -- 172.21.2.104:0/3066567533 shutdown_connections 2021-07-11T04:01:56.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 -- 172.21.2.104:0/3066567533 wait complete. 2021-07-11T04:01:56.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 Processor -- start 2021-07-11T04:01:56.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.020+0000 7ff4ed387700 1 -- start start 2021-07-11T04:01:56.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4ed387700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4e80adf20 0x7ff4e814a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4ed387700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff4e8149e20 0x7ff4e814a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4ed387700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4e814b650 0x7ff4e8139810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4ed387700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80ae300 con 0x7ff4e814b650 2021-07-11T04:01:56.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4ed387700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e814ad50 con 0x7ff4e8149e20 2021-07-11T04:01:56.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4ed387700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80ac1c0 con 0x7ff4e80adf20 2021-07-11T04:01:56.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4e77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4e814b650 0x7ff4e8139810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4e77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4e814b650 0x7ff4e8139810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50432/0 (socket says 172.21.2.104:50432) 2021-07-11T04:01:56.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4e77fe700 1 -- 172.21.2.104:0/3684240212 learned_addr learned my addr 172.21.2.104:0/3684240212 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:56.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4e77fe700 1 -- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4e80adf20 msgr2=0x7ff4e814a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4e77fe700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4e80adf20 0x7ff4e814a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4e77fe700 1 -- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff4e8149e20 msgr2=0x7ff4e814a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4e77fe700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff4e8149e20 0x7ff4e814a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4e77fe700 1 -- 172.21.2.104:0/3684240212 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e80ac460 con 0x7ff4e814b650 2021-07-11T04:01:56.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4e77fe700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4e814b650 0x7ff4e8139810 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7ff4d0001110 tx=0x7ff4e0007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4dffff700 1 -- 172.21.2.104:0/3684240212 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4e0005880 con 0x7ff4e814b650 2021-07-11T04:01:56.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4dffff700 1 -- 172.21.2.104:0/3684240212 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e00059e0 con 0x7ff4e814b650 2021-07-11T04:01:56.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4ed387700 1 -- 172.21.2.104:0/3684240212 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e80ab020 con 0x7ff4e814b650 2021-07-11T04:01:56.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4ed387700 1 -- 172.21.2.104:0/3684240212 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e80ab560 con 0x7ff4e814b650 2021-07-11T04:01:56.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4dffff700 1 -- 172.21.2.104:0/3684240212 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff4e0005d80 con 0x7ff4e814b650 2021-07-11T04:01:56.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4ed387700 1 -- 172.21.2.104:0/3684240212 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4c0003410 con 0x7ff4e814b650 2021-07-11T04:01:56.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4dffff700 1 -- 172.21.2.104:0/3684240212 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff4e0010070 con 0x7ff4e814b650 2021-07-11T04:01:56.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4dffff700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4cc053380 0x7ff4cc055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4dffff700 1 -- 172.21.2.104:0/3684240212 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff4e006abe0 con 0x7ff4e814b650 2021-07-11T04:01:56.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4e6ffd700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4cc053380 0x7ff4cc055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.024+0000 7ff4e6ffd700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4cc053380 0x7ff4cc055840 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7ff4d8007000 tx=0x7ff4d8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.028+0000 7ff4dffff700 1 -- 172.21.2.104:0/3684240212 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff4e0041db0 con 0x7ff4e814b650 2021-07-11T04:01:56.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.108+0000 7ff4ed387700 1 -- 172.21.2.104:0/3684240212 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff4c0003c40 con 0x7ff4e814b650 2021-07-11T04:01:56.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.108+0000 7ff4dffff700 1 -- 172.21.2.104:0/3684240212 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v139) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff4e00453d0 con 0x7ff4e814b650 2021-07-11T04:01:56.110 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:56.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 -- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4cc053380 msgr2=0x7ff4cc055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.111 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4cc053380 0x7ff4cc055840 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7ff4d8007000 tx=0x7ff4d8008040).stop 2021-07-11T04:01:56.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 -- 172.21.2.104:0/3684240212 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4e814b650 msgr2=0x7ff4e8139810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4e814b650 0x7ff4e8139810 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7ff4d0001110 tx=0x7ff4e0007e50).stop 2021-07-11T04:01:56.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 -- 172.21.2.104:0/3684240212 shutdown_connections 2021-07-11T04:01:56.112 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff4cc053380 0x7ff4cc055840 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff4e814b650 0x7ff4e8139810 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff4e8149e20 0x7ff4e814a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.113 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 --2- 172.21.2.104:0/3684240212 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff4e80adf20 0x7ff4e814a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 -- 172.21.2.104:0/3684240212 >> 172.21.2.104:0/3684240212 conn(0x7ff4e805f6b0 msgr2=0x7ff4e8140270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:56.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 -- 172.21.2.104:0/3684240212 shutdown_connections 2021-07-11T04:01:56.114 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.112+0000 7ff4ed387700 1 -- 172.21.2.104:0/3684240212 wait complete. 2021-07-11T04:01:56.117 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:01:56.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 Processor -- start 2021-07-11T04:01:56.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 -- start start 2021-07-11T04:01:56.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc9f8144440 0x7fc9f8144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc9f8065420 0x7fc9f8144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f8145350 con 0x7fc9f8060100 2021-07-11T04:01:56.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f8065c80 con 0x7fc9f8144440 2021-07-11T04:01:56.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f80604e0 con 0x7fc9f8065420 2021-07-11T04:01:56.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9f7fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc9f8144440 0x7fc9f8144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9f7fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc9f8144440 0x7fc9f8144970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50442/0 (socket says 172.21.2.104:50442) 2021-07-11T04:01:56.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9f7fff700 1 -- 172.21.2.104:0/4212530241 learned_addr learned my addr 172.21.2.104:0/4212530241 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:56.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9f7fff700 1 -- 172.21.2.104:0/4212530241 >> v2:172.21.2.105:3300/0 conn(0x7fc9f8065420 msgr2=0x7fc9f8144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:56.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9f7fff700 1 --2- 172.21.2.104:0/4212530241 >> v2:172.21.2.105:3300/0 conn(0x7fc9f8065420 0x7fc9f8144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9f7fff700 1 -- 172.21.2.104:0/4212530241 >> v1:172.21.2.105:6789/0 conn(0x7fc9f8060100 legacy=0x7fc9f8064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:56.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9f7fff700 1 -- 172.21.2.104:0/4212530241 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f80b23b0 con 0x7fc9f8144440 2021-07-11T04:01:56.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9f7fff700 1 --2- 172.21.2.104:0/4212530241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9f8144440 0x7fc9f8144970 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fc9f805d460 tx=0x7fc9e0006eb0).ready entity=mon.0 client_cookie=1b5c977eca4a256e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9f77fe700 1 -- 172.21.2.104:0/4212530241 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9e000d5c0 con 0x7fc9f8144440 2021-07-11T04:01:56.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9f77fe700 1 -- 172.21.2.104:0/4212530241 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e0000e90 con 0x7fc9f8144440 2021-07-11T04:01:56.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9f77fe700 1 -- 172.21.2.104:0/4212530241 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9e000d910 con 0x7fc9f8144440 2021-07-11T04:01:56.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 -- 172.21.2.104:0/4212530241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9f8144440 msgr2=0x7fc9f8144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 --2- 172.21.2.104:0/4212530241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9f8144440 0x7fc9f8144970 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7fc9f805d460 tx=0x7fc9e0006eb0).stop 2021-07-11T04:01:56.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 -- 172.21.2.104:0/4212530241 shutdown_connections 2021-07-11T04:01:56.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 --2- 172.21.2.104:0/4212530241 >> v2:172.21.2.105:3300/0 conn(0x7fc9f8065420 0x7fc9f8144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 --2- 172.21.2.104:0/4212530241 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9f8144440 0x7fc9f8144970 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 -- 172.21.2.104:0/4212530241 >> 172.21.2.104:0/4212530241 conn(0x7fc9f805cf60 msgr2=0x7fc9f805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:56.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 -- 172.21.2.104:0/4212530241 shutdown_connections 2021-07-11T04:01:56.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 -- 172.21.2.104:0/4212530241 wait complete. 2021-07-11T04:01:56.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 Processor -- start 2021-07-11T04:01:56.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.172+0000 7fc9fefd5700 1 -- start start 2021-07-11T04:01:56.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9fefd5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9f8060100 0x7fc9f80b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9fefd5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9f8065420 0x7fc9f80b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9fefd5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc9f8143390 0x7fc9f81363a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9fefd5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80afb80 con 0x7fc9f8065420 2021-07-11T04:01:56.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9fefd5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80b5a80 con 0x7fc9f8143390 2021-07-11T04:01:56.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9fefd5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80afd70 con 0x7fc9f8060100 2021-07-11T04:01:56.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9f8065420 0x7fc9f80b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9f8065420 0x7fc9f80b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50448/0 (socket says 172.21.2.104:50448) 2021-07-11T04:01:56.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f7fff700 1 -- 172.21.2.104:0/3737287284 learned_addr learned my addr 172.21.2.104:0/3737287284 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:56.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f7fff700 1 -- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9f8060100 msgr2=0x7fc9f80b0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f7fff700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9f8060100 0x7fc9f80b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f7fff700 1 -- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc9f8143390 msgr2=0x7fc9f81363a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f7fff700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc9f8143390 0x7fc9f81363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f7fff700 1 -- 172.21.2.104:0/3737287284 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e000b1e0 con 0x7fc9f8065420 2021-07-11T04:01:56.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f7fff700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9f8065420 0x7fc9f80b0880 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fc9f805d460 tx=0x7fc9e0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f5ffb700 1 -- 172.21.2.104:0/3737287284 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9e0021070 con 0x7fc9f8065420 2021-07-11T04:01:56.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f5ffb700 1 -- 172.21.2.104:0/3737287284 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e0000e90 con 0x7fc9f8065420 2021-07-11T04:01:56.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9fefd5700 1 -- 172.21.2.104:0/3737287284 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f80b51e0 con 0x7fc9f8065420 2021-07-11T04:01:56.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f5ffb700 1 -- 172.21.2.104:0/3737287284 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9e0002f50 con 0x7fc9f8065420 2021-07-11T04:01:56.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9fefd5700 1 -- 172.21.2.104:0/3737287284 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f80b7210 con 0x7fc9f8065420 2021-07-11T04:01:56.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9fefd5700 1 -- 172.21.2.104:0/3737287284 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f80b5560 con 0x7fc9f8065420 2021-07-11T04:01:56.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f5ffb700 1 -- 172.21.2.104:0/3737287284 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc9e000f070 con 0x7fc9f8065420 2021-07-11T04:01:56.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f5ffb700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9dc053270 0x7fc9dc055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9f5ffb700 1 -- 172.21.2.104:0/3737287284 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc9e006c910 con 0x7fc9f8065420 2021-07-11T04:01:56.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.176+0000 7fc9fcd71700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9dc053270 0x7fc9dc055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.180+0000 7fc9fcd71700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9dc053270 0x7fc9dc055730 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fc9e8002f90 tx=0x7fc9e80069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.180+0000 7fc9f5ffb700 1 -- 172.21.2.104:0/3737287284 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc9e0043c50 con 0x7fc9f8065420 2021-07-11T04:01:56.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.260+0000 7fc9fefd5700 1 -- 172.21.2.104:0/3737287284 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc9f8057350 con 0x7fc9f8065420 2021-07-11T04:01:56.261 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.260+0000 7fc9f5ffb700 1 -- 172.21.2.104:0/3737287284 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc9e0047270 con 0x7fc9f8065420 2021-07-11T04:01:56.262 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:56.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 -- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9dc053270 msgr2=0x7fc9dc055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9dc053270 0x7fc9dc055730 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fc9e8002f90 tx=0x7fc9e80069b0).stop 2021-07-11T04:01:56.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 -- 172.21.2.104:0/3737287284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9f8065420 msgr2=0x7fc9f80b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.263 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9f8065420 0x7fc9f80b0880 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fc9f805d460 tx=0x7fc9e0015000).stop 2021-07-11T04:01:56.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 -- 172.21.2.104:0/3737287284 shutdown_connections 2021-07-11T04:01:56.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc9f8143390 0x7fc9f81363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9dc053270 0x7fc9dc055730 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc9f8065420 0x7fc9f80b0880 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.264 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 --2- 172.21.2.104:0/3737287284 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc9f8060100 0x7fc9f80b0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 -- 172.21.2.104:0/3737287284 >> 172.21.2.104:0/3737287284 conn(0x7fc9f805cf60 msgr2=0x7fc9f8140040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:56.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 -- 172.21.2.104:0/3737287284 shutdown_connections 2021-07-11T04:01:56.265 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.264+0000 7fc9fefd5700 1 -- 172.21.2.104:0/3737287284 wait complete. 2021-07-11T04:01:56.269 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:01:56.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.320+0000 7fbae99a7700 1 Processor -- start 2021-07-11T04:01:56.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.320+0000 7fbae99a7700 1 -- start start 2021-07-11T04:01:56.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.320+0000 7fbae99a7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbae40a9ec0 0x7fbae40aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.320+0000 7fbae99a7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fbae413f790 0x7fbae413fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.320+0000 7fbae99a7700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae4061990 con 0x7fbae40603d0 2021-07-11T04:01:56.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.320+0000 7fbae99a7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae41400f0 con 0x7fbae40a9ec0 2021-07-11T04:01:56.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.320+0000 7fbae99a7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae4061c30 con 0x7fbae413f790 2021-07-11T04:01:56.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.320+0000 7fbae2ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbae40a9ec0 0x7fbae40aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.320+0000 7fbae27fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fbae40603d0 0x7fbae413ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47226/0 (socket says 172.21.2.104:47226) 2021-07-11T04:01:56.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.320+0000 7fbae27fc700 1 -- 172.21.2.104:0/1882178810 learned_addr learned my addr 172.21.2.104:0/1882178810 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:56.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 -- 172.21.2.104:0/1882178810 >> v2:172.21.2.105:3300/0 conn(0x7fbae413f790 msgr2=0x7fbae413fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:56.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 --2- 172.21.2.104:0/1882178810 >> v2:172.21.2.105:3300/0 conn(0x7fbae413f790 0x7fbae413fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae1ffb700 1 -- 172.21.2.104:0/1882178810 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871806188 0 0) 0x7fbae4061990 con 0x7fbae40603d0 2021-07-11T04:01:56.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 -- 172.21.2.104:0/1882178810 >> v1:172.21.2.104:6789/0 conn(0x7fbae40603d0 legacy=0x7fbae413ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 -- 172.21.2.104:0/1882178810 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae4060b90 con 0x7fbae40a9ec0 2021-07-11T04:01:56.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 --2- 172.21.2.104:0/1882178810 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbae40a9ec0 0x7fbae40aa2a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fbae40aaaf0 tx=0x7fbad4006d80).ready entity=mon.0 client_cookie=4340781b6e19d462 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae1ffb700 1 -- 172.21.2.104:0/1882178810 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbad400bb30 con 0x7fbae40a9ec0 2021-07-11T04:01:56.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae1ffb700 1 -- 172.21.2.104:0/1882178810 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad4000e90 con 0x7fbae40a9ec0 2021-07-11T04:01:56.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae1ffb700 1 -- 172.21.2.104:0/1882178810 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbad400be80 con 0x7fbae40a9ec0 2021-07-11T04:01:56.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 -- 172.21.2.104:0/1882178810 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbae40a9ec0 msgr2=0x7fbae40aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 --2- 172.21.2.104:0/1882178810 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbae40a9ec0 0x7fbae40aa2a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7fbae40aaaf0 tx=0x7fbad4006d80).stop 2021-07-11T04:01:56.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 -- 172.21.2.104:0/1882178810 shutdown_connections 2021-07-11T04:01:56.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 --2- 172.21.2.104:0/1882178810 >> v2:172.21.2.105:3300/0 conn(0x7fbae413f790 0x7fbae413fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 --2- 172.21.2.104:0/1882178810 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbae40a9ec0 0x7fbae40aa2a0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 -- 172.21.2.104:0/1882178810 >> 172.21.2.104:0/1882178810 conn(0x7fbae40b2970 msgr2=0x7fbae40b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:56.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 -- 172.21.2.104:0/1882178810 shutdown_connections 2021-07-11T04:01:56.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 -- 172.21.2.104:0/1882178810 wait complete. 2021-07-11T04:01:56.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 Processor -- start 2021-07-11T04:01:56.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 -- start start 2021-07-11T04:01:56.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbae40603d0 0x7fbae40b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbae40a9ec0 0x7fbae4063ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbae413f790 0x7fbae40644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4143470 con 0x7fbae40603d0 2021-07-11T04:01:56.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae4063c70 con 0x7fbae413f790 2021-07-11T04:01:56.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae40649d0 con 0x7fbae40a9ec0 2021-07-11T04:01:56.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbae40603d0 0x7fbae40b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbae40603d0 0x7fbae40b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50460/0 (socket says 172.21.2.104:50460) 2021-07-11T04:01:56.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 -- 172.21.2.104:0/1819038242 learned_addr learned my addr 172.21.2.104:0/1819038242 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:56.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 -- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbae40a9ec0 msgr2=0x7fbae4063ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbae40a9ec0 0x7fbae4063ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 -- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbae413f790 msgr2=0x7fbae40644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbae413f790 0x7fbae40644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 -- 172.21.2.104:0/1819038242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad400d410 con 0x7fbae40603d0 2021-07-11T04:01:56.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae2ffd700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbae40603d0 0x7fbae40b0ae0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7fbae40aaaf0 tx=0x7fbad4019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbadbfff700 1 -- 172.21.2.104:0/1819038242 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbad4021070 con 0x7fbae40603d0 2021-07-11T04:01:56.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbadbfff700 1 -- 172.21.2.104:0/1819038242 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad4000e90 con 0x7fbae40603d0 2021-07-11T04:01:56.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 -- 172.21.2.104:0/1819038242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae41436d0 con 0x7fbae40603d0 2021-07-11T04:01:56.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbae99a7700 1 -- 172.21.2.104:0/1819038242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae4066e10 con 0x7fbae40603d0 2021-07-11T04:01:56.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.324+0000 7fbadbfff700 1 -- 172.21.2.104:0/1819038242 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbad4009040 con 0x7fbae40603d0 2021-07-11T04:01:56.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.328+0000 7fbae99a7700 1 -- 172.21.2.104:0/1819038242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae4057350 con 0x7fbae40603d0 2021-07-11T04:01:56.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.328+0000 7fbadbfff700 1 -- 172.21.2.104:0/1819038242 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbad401b030 con 0x7fbae40603d0 2021-07-11T04:01:56.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.328+0000 7fbadbfff700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbac8053240 0x7fbac8055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.328+0000 7fbadbfff700 1 -- 172.21.2.104:0/1819038242 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbad406ca50 con 0x7fbae40603d0 2021-07-11T04:01:56.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.328+0000 7fbae27fc700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbac8053240 0x7fbac8055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.328+0000 7fbae27fc700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbac8053240 0x7fbac8055700 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fbacc000bc0 tx=0x7fbacc0011a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.328+0000 7fbadbfff700 1 -- 172.21.2.104:0/1819038242 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbad4043c80 con 0x7fbae40603d0 2021-07-11T04:01:56.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 -- 172.21.2.104:0/1819038242 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbae40ac9f0 con 0x7fbae40603d0 2021-07-11T04:01:56.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbadbfff700 1 -- 172.21.2.104:0/1819038242 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbad40472a0 con 0x7fbae40603d0 2021-07-11T04:01:56.411 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:56.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 -- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbac8053240 msgr2=0x7fbac8055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbac8053240 0x7fbac8055700 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fbacc000bc0 tx=0x7fbacc0011a0).stop 2021-07-11T04:01:56.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 -- 172.21.2.104:0/1819038242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbae40603d0 msgr2=0x7fbae40b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbae40603d0 0x7fbae40b0ae0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7fbae40aaaf0 tx=0x7fbad4019000).stop 2021-07-11T04:01:56.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 -- 172.21.2.104:0/1819038242 shutdown_connections 2021-07-11T04:01:56.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbae413f790 0x7fbae40644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbae40a9ec0 0x7fbae4063ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbac8053240 0x7fbac8055700 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 --2- 172.21.2.104:0/1819038242 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbae40603d0 0x7fbae40b0ae0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 -- 172.21.2.104:0/1819038242 >> 172.21.2.104:0/1819038242 conn(0x7fbae40b2970 msgr2=0x7fbae413f030 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:56.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 -- 172.21.2.104:0/1819038242 shutdown_connections 2021-07-11T04:01:56.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.412+0000 7fbae99a7700 1 -- 172.21.2.104:0/1819038242 wait complete. 2021-07-11T04:01:56.418 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:01:56.419 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:56.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.468+0000 7f647bf61700 1 Processor -- start 2021-07-11T04:01:56.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.468+0000 7f647bf61700 1 -- start start 2021-07-11T04:01:56.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.468+0000 7f647bf61700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f64740b7810 0x7f64741445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.468+0000 7f647bf61700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64740b1170 0x7f6474144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.468+0000 7f647bf61700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64740b31d0 con 0x7f6474064870 2021-07-11T04:01:56.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.468+0000 7f647bf61700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64740b7bf0 con 0x7f64740b7810 2021-07-11T04:01:56.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.468+0000 7f647bf61700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64740675c0 con 0x7f64740b1170 2021-07-11T04:01:56.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.468+0000 7f6479cfd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f64740b7810 0x7f64741445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.468+0000 7f6479cfd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f64740b7810 0x7f64741445a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50470/0 (socket says 172.21.2.104:50470) 2021-07-11T04:01:56.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.468+0000 7f6479cfd700 1 -- 172.21.2.104:0/1792142017 learned_addr learned my addr 172.21.2.104:0/1792142017 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:56.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f6479cfd700 1 -- 172.21.2.104:0/1792142017 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64740b1170 msgr2=0x7f6474144a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:56.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f6479cfd700 1 --2- 172.21.2.104:0/1792142017 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64740b1170 0x7f6474144a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f6479cfd700 1 -- 172.21.2.104:0/1792142017 >> v1:172.21.2.105:6789/0 conn(0x7f6474064870 legacy=0x7f6474064cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:56.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f6479cfd700 1 -- 172.21.2.104:0/1792142017 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64740aae70 con 0x7f64740b7810 2021-07-11T04:01:56.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f6479cfd700 1 --2- 172.21.2.104:0/1792142017 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64740b7810 0x7f64741445a0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7f6474060740 tx=0x7f6470006eb0).ready entity=mon.0 client_cookie=29589161c5ff9864 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f6478cfb700 1 -- 172.21.2.104:0/1792142017 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647000d570 con 0x7f64740b7810 2021-07-11T04:01:56.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f6478cfb700 1 -- 172.21.2.104:0/1792142017 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6470000e90 con 0x7f64740b7810 2021-07-11T04:01:56.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f6478cfb700 1 -- 172.21.2.104:0/1792142017 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f647000d8c0 con 0x7f64740b7810 2021-07-11T04:01:56.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 -- 172.21.2.104:0/1792142017 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64740b7810 msgr2=0x7f64741445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 --2- 172.21.2.104:0/1792142017 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64740b7810 0x7f64741445a0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7f6474060740 tx=0x7f6470006eb0).stop 2021-07-11T04:01:56.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 -- 172.21.2.104:0/1792142017 shutdown_connections 2021-07-11T04:01:56.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 --2- 172.21.2.104:0/1792142017 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f64740b1170 0x7f6474144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 --2- 172.21.2.104:0/1792142017 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64740b7810 0x7f64741445a0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 -- 172.21.2.104:0/1792142017 >> 172.21.2.104:0/1792142017 conn(0x7f647405f6b0 msgr2=0x7f647405fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:56.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 -- 172.21.2.104:0/1792142017 shutdown_connections 2021-07-11T04:01:56.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 -- 172.21.2.104:0/1792142017 wait complete. 2021-07-11T04:01:56.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 Processor -- start 2021-07-11T04:01:56.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 -- start start 2021-07-11T04:01:56.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6474064870 0x7f6474062320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64740b1170 0x7f6474157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64740b7810 0x7f64741580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647415ac30 con 0x7f64740b7810 2021-07-11T04:01:56.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6474145040 con 0x7f64740b1170 2021-07-11T04:01:56.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6474157630 con 0x7f6474064870 2021-07-11T04:01:56.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647a4fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64740b7810 0x7f64741580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647a4fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64740b7810 0x7f64741580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50474/0 (socket says 172.21.2.104:50474) 2021-07-11T04:01:56.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647a4fe700 1 -- 172.21.2.104:0/3952361798 learned_addr learned my addr 172.21.2.104:0/3952361798 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:56.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647a4fe700 1 -- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6474064870 msgr2=0x7f6474062320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647a4fe700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6474064870 0x7f6474062320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647a4fe700 1 -- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64740b1170 msgr2=0x7f6474157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647a4fe700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64740b1170 0x7f6474157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647a4fe700 1 -- 172.21.2.104:0/3952361798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647000b1e0 con 0x7f64740b7810 2021-07-11T04:01:56.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647a4fe700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64740b7810 0x7f64741580d0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f646c003310 tx=0x7f646c0091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f646affd700 1 -- 172.21.2.104:0/3952361798 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f646c001500 con 0x7f64740b7810 2021-07-11T04:01:56.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f646affd700 1 -- 172.21.2.104:0/3952361798 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f646c001660 con 0x7f64740b7810 2021-07-11T04:01:56.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 -- 172.21.2.104:0/3952361798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6474156710 con 0x7f64740b7810 2021-07-11T04:01:56.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f647bf61700 1 -- 172.21.2.104:0/3952361798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6474156c50 con 0x7f64740b7810 2021-07-11T04:01:56.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.472+0000 7f646affd700 1 -- 172.21.2.104:0/3952361798 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f646c0019c0 con 0x7f64740b7810 2021-07-11T04:01:56.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.476+0000 7f647bf61700 1 -- 172.21.2.104:0/3952361798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6450003410 con 0x7f64740b7810 2021-07-11T04:01:56.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.476+0000 7f646affd700 1 -- 172.21.2.104:0/3952361798 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f646c018030 con 0x7f64740b7810 2021-07-11T04:01:56.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.476+0000 7f646affd700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f645c053300 0x7f645c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:56.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.476+0000 7f646affd700 1 -- 172.21.2.104:0/3952361798 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f646c06af40 con 0x7f64740b7810 2021-07-11T04:01:56.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.476+0000 7f6479cfd700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f645c053300 0x7f645c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:56.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.476+0000 7f6479cfd700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f645c053300 0x7f645c0557c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f6474060740 tx=0x7f6470015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:56.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.476+0000 7f646affd700 1 -- 172.21.2.104:0/3952361798 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f646c042220 con 0x7f64740b7810 2021-07-11T04:01:56.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.584+0000 7f647bf61700 1 -- 172.21.2.104:0/3952361798 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6450003c40 con 0x7f64740b7810 2021-07-11T04:01:56.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.584+0000 7f646affd700 1 -- 172.21.2.104:0/3952361798 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f646c045840 con 0x7f64740b7810 2021-07-11T04:01:56.586 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:56.586 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5335,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/1814518027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":1814518027},{"type":"v1","addr":"172.21.2.105:6835","nonce":1814518027}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/155502403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":155502403},{"type":"v1","addr":"172.21.2.104:6834","nonce":155502403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:52.876988+0000","modified":"2021-07-11T04:01:54.362899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/2616834266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":2616834266},{"type":"v1","addr":"172.21.2.104:6835","nonce":2616834266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-07-11T04:01:56.587 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 -- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f645c053300 msgr2=0x7f645c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f645c053300 0x7f645c0557c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f6474060740 tx=0x7f6470015000).stop 2021-07-11T04:01:56.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 -- 172.21.2.104:0/3952361798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64740b7810 msgr2=0x7f64741580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:56.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64740b7810 0x7f64741580d0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f646c003310 tx=0x7f646c0091a0).stop 2021-07-11T04:01:56.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 -- 172.21.2.104:0/3952361798 shutdown_connections 2021-07-11T04:01:56.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f645c053300 0x7f645c0557c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f64740b7810 0x7f64741580d0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:56.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f64740b1170 0x7f6474157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 --2- 172.21.2.104:0/3952361798 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6474064870 0x7f6474062320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:56.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 -- 172.21.2.104:0/3952361798 >> 172.21.2.104:0/3952361798 conn(0x7f647405f6b0 msgr2=0x7f6474061040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:56.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 -- 172.21.2.104:0/3952361798 shutdown_connections 2021-07-11T04:01:56.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:56.588+0000 7f647bf61700 1 -- 172.21.2.104:0/3952361798 wait complete. 2021-07-11T04:01:56.590 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 68 2021-07-11T04:01:56.895 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:01:52.876988+0000', 'modified': '2021-07-11T04:01:54.362899+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8022}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8022': {'gid': 8022, 'name': 'a', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/2616834266', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 2616834266}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 2616834266}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:01:56.896 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:01:56.897 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-07-11T04:01:56.956 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":8022,"want_state":"up:active","state":"up:active","rank_uptime":4.0745021020000003,"mdsmap_epoch":68,"osdmap_epoch":139,"osdmap_epoch_barrier":139,"uptime":8.3323478019999992} 2021-07-11T04:01:56.957 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 8022, "mdsmap_epoch": 68, "osdmap_epoch": 139, "osdmap_epoch_barrier": 139, "rank_uptime": 4.074502102, "state": "up:active", "uptime": 8.332347802, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:01:56.957 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:01:56.958 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T04:01:56.958 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T04:01:56.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:56.959 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:56.970 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:01:57.022 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:57.023 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:57.029 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:57.029 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:57.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:57.031 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:01:57.035 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:01:57.035 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:57.078 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:01:57.087 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:57.087 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:57.092 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:57.093 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:57.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:57.094 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:01:57.116 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T04:01:57.110+0000 7fa6e03ec080 -1 init, newargv = 0x564451f9ac40 newargc=9 2021-07-11T04:01:57.116 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15692]: starting ceph client 2021-07-11T04:01:57.130 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15692]: starting fuse 2021-07-11T04:01:58.199 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:01:58.212 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:58.212 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:58.265 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:58.266 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:58.267 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:58.267 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:01:58.272 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T04:01:58.273 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:01:58.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:58.274 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T04:01:58.274 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T04:01:58.275 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T04:01:58.275 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T04:01:58.275 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T04:01:58.276 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:01:58.276 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T04:01:58.277 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T04:01:58.277 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T04:01:58.277 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:01:58.278 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T04:01:58.278 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T04:01:58.279 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T04:01:58.279 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T04:01:58.279 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:01:58.280 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T04:01:58.280 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:01:58.281 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:01:58.281 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T04:01:58.282 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:01:58.282 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:01:58.282 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T04:01:58.283 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T04:01:58.314 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15692.asok 2021-07-11T04:01:58.316 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15692.asok 2021-07-11T04:01:58.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:58.317 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15692.asok status 2021-07-11T04:01:58.420 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T04:01:58.420 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T04:01:58.421 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:01:58.421 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:01:58.421 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T04:01:58.421 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T04:01:58.422 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T04:01:58.422 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15692", 2021-07-11T04:01:58.422 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T04:01:58.422 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:01:58.422 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T04:01:58.423 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T04:01:58.423 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 8202, 2021-07-11T04:01:58.423 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T04:01:58.423 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T04:01:58.423 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T04:01:58.424 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 8202 2021-07-11T04:01:58.424 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:01:58.424 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:01:58.424 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:01:58.425 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:01:58.425 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 654054262 2021-07-11T04:01:58.425 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T04:01:58.425 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:01:58.425 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:01:58.426 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:01:58.428 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:01:58.428 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 654054262 2021-07-11T04:01:58.428 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:01:58.429 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.8202 172.21.2.119:0/654054262", 2021-07-11T04:01:58.429 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/654054262", 2021-07-11T04:01:58.429 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T04:01:58.429 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 68, 2021-07-11T04:01:58.430 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 139, 2021-07-11T04:01:58.430 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:01:58.430 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T04:01:58.430 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T04:01:58.430 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T04:01:58.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:58.432 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:01:58.433 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:58.442 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:01:58.443 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:01:58.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:58.444 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:01:58.500 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:01:58.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 Processor -- start 2021-07-11T04:01:58.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 -- start start 2021-07-11T04:01:58.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40540aa860 0x7f40540b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f40540a9da0 0x7f40540aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40540ac370 con 0x7f40540acf30 2021-07-11T04:01:58.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40540abb50 con 0x7f40540aa860 2021-07-11T04:01:58.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40540aac40 con 0x7f40540a9da0 2021-07-11T04:01:58.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f4058b41700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40540aa860 0x7f40540b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:58.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f4058b41700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40540aa860 0x7f40540b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50482/0 (socket says 172.21.2.104:50482) 2021-07-11T04:01:58.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f4058b41700 1 -- 172.21.2.104:0/2711876015 learned_addr learned my addr 172.21.2.104:0/2711876015 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:58.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f4053fff700 1 --2- 172.21.2.104:0/2711876015 >> v2:172.21.2.105:3300/0 conn(0x7f40540a9da0 0x7f40540aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:58.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f4058b41700 1 -- 172.21.2.104:0/2711876015 >> v2:172.21.2.105:3300/0 conn(0x7f40540a9da0 msgr2=0x7f40540aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:58.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f4058b41700 1 --2- 172.21.2.104:0/2711876015 >> v2:172.21.2.105:3300/0 conn(0x7f40540a9da0 0x7f40540aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:58.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f4058b41700 1 -- 172.21.2.104:0/2711876015 >> v1:172.21.2.104:6789/0 conn(0x7f40540acf30 legacy=0x7f40541434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f4058b41700 1 -- 172.21.2.104:0/2711876015 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40540b16b0 con 0x7f40540aa860 2021-07-11T04:01:58.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f4058b41700 1 --2- 172.21.2.104:0/2711876015 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40540aa860 0x7f40540b11c0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f4044002f70 tx=0x7f4044006eb0).ready entity=mon.0 client_cookie=fbe254a920c30a01 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:58.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f40537fe700 1 -- 172.21.2.104:0/2711876015 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f404400d610 con 0x7f40540aa860 2021-07-11T04:01:58.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f40537fe700 1 -- 172.21.2.104:0/2711876015 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4044000e90 con 0x7f40540aa860 2021-07-11T04:01:58.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f40537fe700 1 -- 172.21.2.104:0/2711876015 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f404400d9a0 con 0x7f40540aa860 2021-07-11T04:01:58.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 -- 172.21.2.104:0/2711876015 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40540aa860 msgr2=0x7f40540b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 --2- 172.21.2.104:0/2711876015 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40540aa860 0x7f40540b11c0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f4044002f70 tx=0x7f4044006eb0).stop 2021-07-11T04:01:58.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 -- 172.21.2.104:0/2711876015 shutdown_connections 2021-07-11T04:01:58.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 --2- 172.21.2.104:0/2711876015 >> v2:172.21.2.105:3300/0 conn(0x7f40540a9da0 0x7f40540aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:58.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 --2- 172.21.2.104:0/2711876015 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40540aa860 0x7f40540b11c0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:58.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 -- 172.21.2.104:0/2711876015 >> 172.21.2.104:0/2711876015 conn(0x7f40540b64d0 msgr2=0x7f40540b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:58.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 -- 172.21.2.104:0/2711876015 shutdown_connections 2021-07-11T04:01:58.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.556+0000 7f405ada5700 1 -- 172.21.2.104:0/2711876015 wait complete. 2021-07-11T04:01:58.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f405ada5700 1 Processor -- start 2021-07-11T04:01:58.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f405ada5700 1 -- start start 2021-07-11T04:01:58.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f405ada5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40540a9da0 0x7f40541472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f405ada5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40540aa860 0x7f40540636e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f405ada5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40540acf30 0x7f4054063bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f405ada5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4054140bd0 con 0x7f40540acf30 2021-07-11T04:01:58.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f405ada5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4054141aa0 con 0x7f40540aa860 2021-07-11T04:01:58.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f405ada5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4054145af0 con 0x7f40540a9da0 2021-07-11T04:01:58.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4059342700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40540acf30 0x7f4054063bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:58.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4059342700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40540acf30 0x7f4054063bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50492/0 (socket says 172.21.2.104:50492) 2021-07-11T04:01:58.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4059342700 1 -- 172.21.2.104:0/2981136824 learned_addr learned my addr 172.21.2.104:0/2981136824 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:58.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4059342700 1 -- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40540a9da0 msgr2=0x7f40541472b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4059342700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40540a9da0 0x7f40541472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4059342700 1 -- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40540aa860 msgr2=0x7f40540636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4059342700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40540aa860 0x7f40540636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4059342700 1 -- 172.21.2.104:0/2981136824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404400b1e0 con 0x7f40540acf30 2021-07-11T04:01:58.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4059342700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40540acf30 0x7f4054063bd0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f40540b63e0 tx=0x7f404c0013c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:58.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4051ffb700 1 -- 172.21.2.104:0/2981136824 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f404c006aa0 con 0x7f40540acf30 2021-07-11T04:01:58.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4051ffb700 1 -- 172.21.2.104:0/2981136824 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404c006c00 con 0x7f40540acf30 2021-07-11T04:01:58.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f405ada5700 1 -- 172.21.2.104:0/2981136824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4054147c20 con 0x7f40540acf30 2021-07-11T04:01:58.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4051ffb700 1 -- 172.21.2.104:0/2981136824 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f404c007030 con 0x7f40540acf30 2021-07-11T04:01:58.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f405ada5700 1 -- 172.21.2.104:0/2981136824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4054065d80 con 0x7f40540acf30 2021-07-11T04:01:58.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f405ada5700 1 -- 172.21.2.104:0/2981136824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4054141840 con 0x7f40540acf30 2021-07-11T04:01:58.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4051ffb700 1 -- 172.21.2.104:0/2981136824 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f404c010030 con 0x7f40540acf30 2021-07-11T04:01:58.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4051ffb700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40340531a0 0x7f4034055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4051ffb700 1 -- 172.21.2.104:0/2981136824 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f404c06b510 con 0x7f40540acf30 2021-07-11T04:01:58.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.560+0000 7f4058b41700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40340531a0 0x7f4034055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:58.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.564+0000 7f4058b41700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40340531a0 0x7f4034055660 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f4044002f70 tx=0x7f4044015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:58.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.564+0000 7f4051ffb700 1 -- 172.21.2.104:0/2981136824 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f404c042740 con 0x7f40540acf30 2021-07-11T04:01:58.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.644+0000 7f405ada5700 1 -- 172.21.2.104:0/2981136824 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4054057350 con 0x7f40540acf30 2021-07-11T04:01:58.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f4051ffb700 1 -- 172.21.2.104:0/2981136824 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f404c045d60 con 0x7f40540acf30 2021-07-11T04:01:58.647 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:58.647 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":139,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:53.880137+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:01:50.686994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-07-11T04:01:51.853293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:01:58.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 -- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40340531a0 msgr2=0x7f4034055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40340531a0 0x7f4034055660 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f4044002f70 tx=0x7f4044015000).stop 2021-07-11T04:01:58.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 -- 172.21.2.104:0/2981136824 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40540acf30 msgr2=0x7f4054063bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40540acf30 0x7f4054063bd0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f40540b63e0 tx=0x7f404c0013c0).stop 2021-07-11T04:01:58.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 -- 172.21.2.104:0/2981136824 shutdown_connections 2021-07-11T04:01:58.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40340531a0 0x7f4034055660 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:58.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40540acf30 0x7f4054063bd0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:58.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40540aa860 0x7f40540636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 --2- 172.21.2.104:0/2981136824 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f40540a9da0 0x7f40541472b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 -- 172.21.2.104:0/2981136824 >> 172.21.2.104:0/2981136824 conn(0x7f40540b64d0 msgr2=0x7f40540605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:58.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 -- 172.21.2.104:0/2981136824 shutdown_connections 2021-07-11T04:01:58.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.648+0000 7f405ada5700 1 -- 172.21.2.104:0/2981136824 wait complete. 2021-07-11T04:01:58.655 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:01:50.686994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:01:51.853293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:01:58.656 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:01:58.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf9acf6700 1 Processor -- start 2021-07-11T04:01:58.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf9acf6700 1 -- start start 2021-07-11T04:01:58.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf9acf6700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fdf940b05b0 0x7fdf940b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf9acf6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf940b3710 0x7fdf9413d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf9acf6700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf94143920 con 0x7fdf9413da40 2021-07-11T04:01:58.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf9acf6700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf94143bc0 con 0x7fdf940b05b0 2021-07-11T04:01:58.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf9acf6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf94143e60 con 0x7fdf940b3710 2021-07-11T04:01:58.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf99293700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fdf9413da40 0x7fdf940b10b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47264/0 (socket says 172.21.2.104:47264) 2021-07-11T04:01:58.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf99293700 1 -- 172.21.2.104:0/4287528894 learned_addr learned my addr 172.21.2.104:0/4287528894 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:58.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757521141 0 0) 0x7fdf94143920 con 0x7fdf9413da40 2021-07-11T04:01:58.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf74003260 con 0x7fdf9413da40 2021-07-11T04:01:58.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1932894887 0 0) 0x7fdf74003260 con 0x7fdf9413da40 2021-07-11T04:01:58.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf940b3710 msgr2=0x7fdf9413d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:58.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 --2- 172.21.2.104:0/4287528894 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf940b3710 0x7fdf9413d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 >> v2:172.21.2.105:3300/0 conn(0x7fdf940b05b0 msgr2=0x7fdf940b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:58.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 --2- 172.21.2.104:0/4287528894 >> v2:172.21.2.105:3300/0 conn(0x7fdf940b05b0 0x7fdf940b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf94144100 con 0x7fdf9413da40 2021-07-11T04:01:58.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fdf8c0019e0 con 0x7fdf9413da40 2021-07-11T04:01:58.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 >> v1:172.21.2.104:6789/0 conn(0x7fdf9413da40 legacy=0x7fdf940b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 --2- 172.21.2.104:0/4287528894 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf740037d0 0x7fdf74003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 --2- 172.21.2.104:0/4287528894 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf74004500 0x7fdf74004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 --2- 172.21.2.104:0/4287528894 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf940b3710 0x7fdf7400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf94143bc0 con 0x7fdf74004500 2021-07-11T04:01:58.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf94143e60 con 0x7fdf740037d0 2021-07-11T04:01:58.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf74005300 con 0x7fdf940b3710 2021-07-11T04:01:58.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.708+0000 7fdf93fff700 1 --2- 172.21.2.104:0/4287528894 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf74004500 0x7fdf74004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:58.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf93fff700 1 -- 172.21.2.104:0/4287528894 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf940b3710 msgr2=0x7fdf7400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:58.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf93fff700 1 --2- 172.21.2.104:0/4287528894 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf940b3710 0x7fdf7400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf93fff700 1 -- 172.21.2.104:0/4287528894 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf740037d0 msgr2=0x7fdf74003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:58.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf93fff700 1 --2- 172.21.2.104:0/4287528894 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf740037d0 0x7fdf74003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf93fff700 1 -- 172.21.2.104:0/4287528894 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74003260 con 0x7fdf74004500 2021-07-11T04:01:58.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf93fff700 1 --2- 172.21.2.104:0/4287528894 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf74004500 0x7fdf74004980 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7fdf84002f70 tx=0x7fdf84006ca0).ready entity=mon.0 client_cookie=9802c8095d2d16b7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:58.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf84012410 con 0x7fdf74004500 2021-07-11T04:01:58.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf84012570 con 0x7fdf74004500 2021-07-11T04:01:58.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf937fe700 1 -- 172.21.2.104:0/4287528894 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf840128d0 con 0x7fdf74004500 2021-07-11T04:01:58.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- 172.21.2.104:0/4287528894 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf74004500 msgr2=0x7fdf74004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 --2- 172.21.2.104:0/4287528894 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf74004500 0x7fdf74004980 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7fdf84002f70 tx=0x7fdf84006ca0).stop 2021-07-11T04:01:58.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf99293700 1 -- 172.21.2.104:0/4287528894 reap_dead start 2021-07-11T04:01:58.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- 172.21.2.104:0/4287528894 shutdown_connections 2021-07-11T04:01:58.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- 172.21.2.104:0/4287528894 >> 172.21.2.104:0/4287528894 conn(0x7fdf940607c0 msgr2=0x7fdf94060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:58.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- 172.21.2.104:0/4287528894 shutdown_connections 2021-07-11T04:01:58.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- 172.21.2.104:0/4287528894 wait complete. 2021-07-11T04:01:58.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 Processor -- start 2021-07-11T04:01:58.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- start start 2021-07-11T04:01:58.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf94141bf0 0x7fdf9414f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf94142b20 0x7fdf9414f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf940ab2b0 0x7fdf940ae720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf94141fd0 con 0x7fdf940ab2b0 2021-07-11T04:01:58.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf94153000 con 0x7fdf94141bf0 2021-07-11T04:01:58.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf941532a0 con 0x7fdf94142b20 2021-07-11T04:01:58.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf99293700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf940ab2b0 0x7fdf940ae720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:58.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf99293700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf940ab2b0 0x7fdf940ae720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50512/0 (socket says 172.21.2.104:50512) 2021-07-11T04:01:58.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf99293700 1 -- 172.21.2.104:0/701243368 learned_addr learned my addr 172.21.2.104:0/701243368 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:58.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf99293700 1 -- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf94142b20 msgr2=0x7fdf9414f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf99293700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf94142b20 0x7fdf9414f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf99293700 1 -- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf94141bf0 msgr2=0x7fdf9414f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf99293700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf94141bf0 0x7fdf9414f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf99293700 1 -- 172.21.2.104:0/701243368 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf940aab70 con 0x7fdf940ab2b0 2021-07-11T04:01:58.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf99293700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf940ab2b0 0x7fdf940ae720 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7fdf941531c0 tx=0x7fdf94143ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:58.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf91ffb700 1 -- 172.21.2.104:0/701243368 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf8c004b70 con 0x7fdf940ab2b0 2021-07-11T04:01:58.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf91ffb700 1 -- 172.21.2.104:0/701243368 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c004cd0 con 0x7fdf940ab2b0 2021-07-11T04:01:58.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- 172.21.2.104:0/701243368 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf940aae10 con 0x7fdf940ab2b0 2021-07-11T04:01:58.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- 172.21.2.104:0/701243368 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf94066b80 con 0x7fdf940ab2b0 2021-07-11T04:01:58.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf91ffb700 1 -- 172.21.2.104:0/701243368 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fdf8c0072c0 con 0x7fdf940ab2b0 2021-07-11T04:01:58.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf9acf6700 1 -- 172.21.2.104:0/701243368 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf940aafa0 con 0x7fdf940ab2b0 2021-07-11T04:01:58.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.712+0000 7fdf91ffb700 1 -- 172.21.2.104:0/701243368 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fdf8c01b070 con 0x7fdf940ab2b0 2021-07-11T04:01:58.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.716+0000 7fdf91ffb700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf78053270 0x7fdf78055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.716+0000 7fdf91ffb700 1 -- 172.21.2.104:0/701243368 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdf8c06ab30 con 0x7fdf940ab2b0 2021-07-11T04:01:58.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.716+0000 7fdf98a92700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf78053270 0x7fdf78055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:58.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.716+0000 7fdf98a92700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf78053270 0x7fdf78055730 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fdf7c002f70 tx=0x7fdf7c006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:58.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.716+0000 7fdf91ffb700 1 -- 172.21.2.104:0/701243368 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fdf8c041e10 con 0x7fdf940ab2b0 2021-07-11T04:01:58.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 -- 172.21.2.104:0/701243368 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdf94057350 con 0x7fdf940ab2b0 2021-07-11T04:01:58.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf91ffb700 1 -- 172.21.2.104:0/701243368 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v139) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdf8c019020 con 0x7fdf940ab2b0 2021-07-11T04:01:58.799 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:01:58.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 -- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf78053270 msgr2=0x7fdf78055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf78053270 0x7fdf78055730 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7fdf7c002f70 tx=0x7fdf7c006a80).stop 2021-07-11T04:01:58.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 -- 172.21.2.104:0/701243368 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf940ab2b0 msgr2=0x7fdf940ae720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf940ab2b0 0x7fdf940ae720 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7fdf941531c0 tx=0x7fdf94143ab0).stop 2021-07-11T04:01:58.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 -- 172.21.2.104:0/701243368 shutdown_connections 2021-07-11T04:01:58.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fdf78053270 0x7fdf78055730 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:58.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fdf940ab2b0 0x7fdf940ae720 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:58.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fdf94142b20 0x7fdf9414f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 --2- 172.21.2.104:0/701243368 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fdf94141bf0 0x7fdf9414f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 -- 172.21.2.104:0/701243368 >> 172.21.2.104:0/701243368 conn(0x7fdf940607c0 msgr2=0x7fdf941455b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:58.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 -- 172.21.2.104:0/701243368 shutdown_connections 2021-07-11T04:01:58.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.800+0000 7fdf9acf6700 1 -- 172.21.2.104:0/701243368 wait complete. 2021-07-11T04:01:58.806 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:01:58.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1abd64700 1 Processor -- start 2021-07-11T04:01:58.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1abd64700 1 -- start start 2021-07-11T04:01:58.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1abd64700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa1a4060100 0x7fa1a4064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1abd64700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a40604e0 con 0x7fa1a40b38d0 2021-07-11T04:01:58.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1abd64700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a40b23b0 con 0x7fa1a4144440 2021-07-11T04:01:58.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1abd64700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a4065c80 con 0x7fa1a4060100 2021-07-11T04:01:58.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1aa301700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fa1a40b38d0 0x7fa1a414b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47286/0 (socket says 172.21.2.104:47286) 2021-07-11T04:01:58.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1aa301700 1 -- 172.21.2.104:0/311109720 learned_addr learned my addr 172.21.2.104:0/311109720 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:58.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1653796359 0 0) 0x7fa1a40604e0 con 0x7fa1a40b38d0 2021-07-11T04:01:58.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa18c003260 con 0x7fa1a40b38d0 2021-07-11T04:01:58.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a9b00700 1 --2- 172.21.2.104:0/311109720 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa1a4060100 0x7fa1a4064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:58.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 937505035 0 0) 0x7fa18c003260 con 0x7fa1a40b38d0 2021-07-11T04:01:58.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa1a4060100 msgr2=0x7fa1a4064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:58.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 --2- 172.21.2.104:0/311109720 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa1a4060100 0x7fa1a4064d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:58.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 >> v1:172.21.2.105:6789/0 conn(0x7fa1a4144440 legacy=0x7fa1a4144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:58.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a40b2650 con 0x7fa1a40b38d0 2021-07-11T04:01:58.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fa19c0019e0 con 0x7fa1a40b38d0 2021-07-11T04:01:58.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 >> v1:172.21.2.104:6789/0 conn(0x7fa1a40b38d0 legacy=0x7fa1a414b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 --2- 172.21.2.104:0/311109720 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa18c003800 0x7fa18c003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 --2- 172.21.2.104:0/311109720 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa18c004500 0x7fa18c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 --2- 172.21.2.104:0/311109720 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa18c004f90 0x7fa18c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a4065c80 con 0x7fa18c004500 2021-07-11T04:01:58.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c00bdd0 con 0x7fa18c004f90 2021-07-11T04:01:58.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c00bf90 con 0x7fa18c003800 2021-07-11T04:01:58.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a92ff700 1 --2- 172.21.2.104:0/311109720 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa18c004500 0x7fa18c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:58.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a92ff700 1 -- 172.21.2.104:0/311109720 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa18c003800 msgr2=0x7fa18c003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:58.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a92ff700 1 --2- 172.21.2.104:0/311109720 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa18c003800 0x7fa18c003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a92ff700 1 -- 172.21.2.104:0/311109720 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa18c004f90 msgr2=0x7fa18c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:58.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a92ff700 1 --2- 172.21.2.104:0/311109720 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa18c004f90 0x7fa18c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a92ff700 1 -- 172.21.2.104:0/311109720 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c003260 con 0x7fa18c004500 2021-07-11T04:01:58.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a92ff700 1 --2- 172.21.2.104:0/311109720 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa18c004500 0x7fa18c004980 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7fa198001fa0 tx=0x7fa198007a00).ready entity=mon.0 client_cookie=9e83b3c4b7123416 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:58.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa198007070 con 0x7fa18c004500 2021-07-11T04:01:58.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1980071d0 con 0x7fa18c004500 2021-07-11T04:01:58.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1a8afe700 1 -- 172.21.2.104:0/311109720 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa198006510 con 0x7fa18c004500 2021-07-11T04:01:58.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1abd64700 1 -- 172.21.2.104:0/311109720 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa18c004500 msgr2=0x7fa18c004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.860+0000 7fa1abd64700 1 --2- 172.21.2.104:0/311109720 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa18c004500 0x7fa18c004980 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7fa198001fa0 tx=0x7fa198007a00).stop 2021-07-11T04:01:58.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1aa301700 1 -- 172.21.2.104:0/311109720 reap_dead start 2021-07-11T04:01:58.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 -- 172.21.2.104:0/311109720 shutdown_connections 2021-07-11T04:01:58.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 -- 172.21.2.104:0/311109720 >> 172.21.2.104:0/311109720 conn(0x7fa1a405cf60 msgr2=0x7fa1a405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:58.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 -- 172.21.2.104:0/311109720 shutdown_connections 2021-07-11T04:01:58.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 -- 172.21.2.104:0/311109720 wait complete. 2021-07-11T04:01:58.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 Processor -- start 2021-07-11T04:01:58.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 -- start start 2021-07-11T04:01:58.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa1a40b0f80 0x7fa1a4153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa1a4141af0 0x7fa1a4153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa1a4140bc0 0x7fa1a413a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40b1360 con 0x7fa1a40b0f80 2021-07-11T04:01:58.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a4140620 con 0x7fa1a4140bc0 2021-07-11T04:01:58.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a413e980 con 0x7fa1a4141af0 2021-07-11T04:01:58.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1a9b00700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa1a40b0f80 0x7fa1a4153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:58.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1a9b00700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa1a40b0f80 0x7fa1a4153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50528/0 (socket says 172.21.2.104:50528) 2021-07-11T04:01:58.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1a9b00700 1 -- 172.21.2.104:0/3522156786 learned_addr learned my addr 172.21.2.104:0/3522156786 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:58.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1a9b00700 1 -- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa1a4141af0 msgr2=0x7fa1a4153c70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:58.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1a9b00700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa1a4141af0 0x7fa1a4153c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1a9b00700 1 -- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa1a4140bc0 msgr2=0x7fa1a413a820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:58.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1a9b00700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa1a4140bc0 0x7fa1a413a820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1a9b00700 1 -- 172.21.2.104:0/3522156786 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a4140880 con 0x7fa1a40b0f80 2021-07-11T04:01:58.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1a9b00700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa1a40b0f80 0x7fa1a4153780 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fa1a000a000 tx=0x7fa1a0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:58.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa196ffd700 1 -- 172.21.2.104:0/3522156786 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1a0016070 con 0x7fa1a40b0f80 2021-07-11T04:01:58.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa196ffd700 1 -- 172.21.2.104:0/3522156786 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a0000e90 con 0x7fa1a40b0f80 2021-07-11T04:01:58.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 -- 172.21.2.104:0/3522156786 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a4141670 con 0x7fa1a40b0f80 2021-07-11T04:01:58.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa196ffd700 1 -- 172.21.2.104:0/3522156786 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa1a0011970 con 0x7fa1a40b0f80 2021-07-11T04:01:58.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 -- 172.21.2.104:0/3522156786 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a4140240 con 0x7fa1a40b0f80 2021-07-11T04:01:58.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa1abd64700 1 -- 172.21.2.104:0/3522156786 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a4141800 con 0x7fa1a40b0f80 2021-07-11T04:01:58.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa196ffd700 1 -- 172.21.2.104:0/3522156786 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa1a0011ad0 con 0x7fa1a40b0f80 2021-07-11T04:01:58.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa196ffd700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa1840531f0 0x7fa1840556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:58.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.864+0000 7fa196ffd700 1 -- 172.21.2.104:0/3522156786 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa1a006b0d0 con 0x7fa1a40b0f80 2021-07-11T04:01:58.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.868+0000 7fa1a92ff700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa1840531f0 0x7fa1840556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:58.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.868+0000 7fa1a92ff700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa1840531f0 0x7fa1840556b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fa198001fa0 tx=0x7fa198015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:58.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.868+0000 7fa196ffd700 1 -- 172.21.2.104:0/3522156786 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa1a0042410 con 0x7fa1a40b0f80 2021-07-11T04:01:58.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 -- 172.21.2.104:0/3522156786 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa1a4057350 con 0x7fa1a40b0f80 2021-07-11T04:01:58.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa196ffd700 1 -- 172.21.2.104:0/3522156786 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v139) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa1a0045a30 con 0x7fa1a40b0f80 2021-07-11T04:01:58.951 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:58.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 -- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa1840531f0 msgr2=0x7fa1840556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa1840531f0 0x7fa1840556b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7fa198001fa0 tx=0x7fa198015000).stop 2021-07-11T04:01:58.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 -- 172.21.2.104:0/3522156786 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa1a40b0f80 msgr2=0x7fa1a4153780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:58.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa1a40b0f80 0x7fa1a4153780 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fa1a000a000 tx=0x7fa1a0008040).stop 2021-07-11T04:01:58.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 -- 172.21.2.104:0/3522156786 shutdown_connections 2021-07-11T04:01:58.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa1a4140bc0 0x7fa1a413a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa1a4141af0 0x7fa1a4153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:58.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa1840531f0 0x7fa1840556b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:58.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 --2- 172.21.2.104:0/3522156786 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa1a40b0f80 0x7fa1a4153780 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:58.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 -- 172.21.2.104:0/3522156786 >> 172.21.2.104:0/3522156786 conn(0x7fa1a405cf60 msgr2=0x7fa1a40b5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:58.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 -- 172.21.2.104:0/3522156786 shutdown_connections 2021-07-11T04:01:58.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:58.952+0000 7fa1abd64700 1 -- 172.21.2.104:0/3522156786 wait complete. 2021-07-11T04:01:58.959 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:01:59.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 Processor -- start 2021-07-11T04:01:59.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 -- start start 2021-07-11T04:01:59.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6a68060100 0x7f6a68064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6a68144440 0x7f6a68065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a680604e0 con 0x7f6a680b38d0 2021-07-11T04:01:59.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a680656f0 con 0x7f6a68060100 2021-07-11T04:01:59.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a68065c80 con 0x7f6a68144440 2021-07-11T04:01:59.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a677fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6a68060100 0x7f6a68064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:59.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a677fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6a68060100 0x7f6a68064d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50536/0 (socket says 172.21.2.104:50536) 2021-07-11T04:01:59.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a677fe700 1 -- 172.21.2.104:0/3325901967 learned_addr learned my addr 172.21.2.104:0/3325901967 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:59.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a677fe700 1 -- 172.21.2.104:0/3325901967 >> v2:172.21.2.105:3300/0 conn(0x7f6a68144440 msgr2=0x7f6a68065200 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:01:59.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a677fe700 1 --2- 172.21.2.104:0/3325901967 >> v2:172.21.2.105:3300/0 conn(0x7f6a68144440 0x7f6a68065200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a677fe700 1 -- 172.21.2.104:0/3325901967 >> v1:172.21.2.105:6789/0 conn(0x7f6a680b38d0 legacy=0x7f6a68144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:59.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a677fe700 1 -- 172.21.2.104:0/3325901967 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a680b23b0 con 0x7f6a68060100 2021-07-11T04:01:59.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a677fe700 1 --2- 172.21.2.104:0/3325901967 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6a68060100 0x7f6a68064d10 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f6a6805d460 tx=0x7f6a50006eb0).ready entity=mon.0 client_cookie=83d58453de714d68 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:59.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a667fc700 1 -- 172.21.2.104:0/3325901967 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a5000d5c0 con 0x7f6a68060100 2021-07-11T04:01:59.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a667fc700 1 -- 172.21.2.104:0/3325901967 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a50000e90 con 0x7f6a68060100 2021-07-11T04:01:59.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a667fc700 1 -- 172.21.2.104:0/3325901967 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a5000d910 con 0x7f6a68060100 2021-07-11T04:01:59.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 -- 172.21.2.104:0/3325901967 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6a68060100 msgr2=0x7f6a68064d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 --2- 172.21.2.104:0/3325901967 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6a68060100 0x7f6a68064d10 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f6a6805d460 tx=0x7f6a50006eb0).stop 2021-07-11T04:01:59.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 -- 172.21.2.104:0/3325901967 shutdown_connections 2021-07-11T04:01:59.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 --2- 172.21.2.104:0/3325901967 >> v2:172.21.2.105:3300/0 conn(0x7f6a68144440 0x7f6a68065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 --2- 172.21.2.104:0/3325901967 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6a68060100 0x7f6a68064d10 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:59.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 -- 172.21.2.104:0/3325901967 >> 172.21.2.104:0/3325901967 conn(0x7f6a6805cf60 msgr2=0x7f6a6805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:59.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 -- 172.21.2.104:0/3325901967 shutdown_connections 2021-07-11T04:01:59.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 -- 172.21.2.104:0/3325901967 wait complete. 2021-07-11T04:01:59.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 Processor -- start 2021-07-11T04:01:59.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.012+0000 7f6a6dc62700 1 -- start start 2021-07-11T04:01:59.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a6dc62700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6a68060100 0x7f6a68062180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a6dc62700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6a680b38d0 0x7f6a68152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a6dc62700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6a68140ee0 0x7f6a6814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a6dc62700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a681524b0 con 0x7f6a680b38d0 2021-07-11T04:01:59.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a6dc62700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a680aece0 con 0x7f6a68140ee0 2021-07-11T04:01:59.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a6dc62700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a68141740 con 0x7f6a68060100 2021-07-11T04:01:59.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a66ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6a680b38d0 0x7f6a68152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:59.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a66ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6a680b38d0 0x7f6a68152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50544/0 (socket says 172.21.2.104:50544) 2021-07-11T04:01:59.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a66ffd700 1 -- 172.21.2.104:0/208979803 learned_addr learned my addr 172.21.2.104:0/208979803 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:59.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a66ffd700 1 -- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6a68060100 msgr2=0x7f6a68062180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a66ffd700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6a68060100 0x7f6a68062180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a66ffd700 1 -- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6a68140ee0 msgr2=0x7f6a6814f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a66ffd700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6a68140ee0 0x7f6a6814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a66ffd700 1 -- 172.21.2.104:0/208979803 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5000b1e0 con 0x7f6a680b38d0 2021-07-11T04:01:59.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a66ffd700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6a680b38d0 0x7f6a68152de0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f6a58000ed0 tx=0x7f6a58006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:59.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a64ff9700 1 -- 172.21.2.104:0/208979803 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a58012410 con 0x7f6a680b38d0 2021-07-11T04:01:59.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a64ff9700 1 -- 172.21.2.104:0/208979803 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a58012570 con 0x7f6a680b38d0 2021-07-11T04:01:59.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a6dc62700 1 -- 172.21.2.104:0/208979803 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a681526a0 con 0x7f6a680b38d0 2021-07-11T04:01:59.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a6dc62700 1 -- 172.21.2.104:0/208979803 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a680aff90 con 0x7f6a680b38d0 2021-07-11T04:01:59.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a64ff9700 1 -- 172.21.2.104:0/208979803 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6a580128d0 con 0x7f6a680b38d0 2021-07-11T04:01:59.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a6dc62700 1 -- 172.21.2.104:0/208979803 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a68057350 con 0x7f6a680b38d0 2021-07-11T04:01:59.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a64ff9700 1 -- 172.21.2.104:0/208979803 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6a5800f030 con 0x7f6a680b38d0 2021-07-11T04:01:59.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a64ff9700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6a4c053350 0x7f6a4c055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a64ff9700 1 -- 172.21.2.104:0/208979803 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6a5806b1b0 con 0x7f6a680b38d0 2021-07-11T04:01:59.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.016+0000 7f6a677fe700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6a4c053350 0x7f6a4c055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:59.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.020+0000 7f6a677fe700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6a4c053350 0x7f6a4c055810 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f6a6805d460 tx=0x7f6a50015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:59.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.020+0000 7f6a64ff9700 1 -- 172.21.2.104:0/208979803 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6a580423e0 con 0x7f6a680b38d0 2021-07-11T04:01:59.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 -- 172.21.2.104:0/208979803 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a680698e0 con 0x7f6a680b38d0 2021-07-11T04:01:59.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a64ff9700 1 -- 172.21.2.104:0/208979803 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6a58045a00 con 0x7f6a680b38d0 2021-07-11T04:01:59.101 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:59.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 -- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6a4c053350 msgr2=0x7f6a4c055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6a4c053350 0x7f6a4c055810 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f6a6805d460 tx=0x7f6a50015000).stop 2021-07-11T04:01:59.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 -- 172.21.2.104:0/208979803 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6a680b38d0 msgr2=0x7f6a68152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6a680b38d0 0x7f6a68152de0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f6a58000ed0 tx=0x7f6a58006d70).stop 2021-07-11T04:01:59.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 -- 172.21.2.104:0/208979803 shutdown_connections 2021-07-11T04:01:59.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6a68140ee0 0x7f6a6814f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6a4c053350 0x7f6a4c055810 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:59.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6a680b38d0 0x7f6a68152de0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:59.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 --2- 172.21.2.104:0/208979803 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6a68060100 0x7f6a68062180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 -- 172.21.2.104:0/208979803 >> 172.21.2.104:0/208979803 conn(0x7f6a6805cf60 msgr2=0x7f6a6813f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:59.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 -- 172.21.2.104:0/208979803 shutdown_connections 2021-07-11T04:01:59.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.100+0000 7f6a6dc62700 1 -- 172.21.2.104:0/208979803 wait complete. 2021-07-11T04:01:59.108 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:01:59.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb86ee700 1 Processor -- start 2021-07-11T04:01:59.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb86ee700 1 -- start start 2021-07-11T04:01:59.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb86ee700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cb00b12d0 0x7f1cb00affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb86ee700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00a77d0 con 0x7f1cb00b1ec0 2021-07-11T04:01:59.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb86ee700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cb00b0650 con 0x7f1cb00b22a0 2021-07-11T04:01:59.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb86ee700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00b1740 con 0x7f1cb00b12d0 2021-07-11T04:01:59.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb648a700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1cb00b1ec0 0x7f1cb00a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47316/0 (socket says 172.21.2.104:47316) 2021-07-11T04:01:59.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb648a700 1 -- 172.21.2.104:0/2692894900 learned_addr learned my addr 172.21.2.104:0/2692894900 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:59.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798590460 0 0) 0x7f1cb00a77d0 con 0x7f1cb00b1ec0 2021-07-11T04:01:59.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c98003260 con 0x7f1cb00b1ec0 2021-07-11T04:01:59.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3907494113 0 0) 0x7f1c98003260 con 0x7f1cb00b1ec0 2021-07-11T04:01:59.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cb00b12d0 msgr2=0x7f1cb00affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:59.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 --2- 172.21.2.104:0/2692894900 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cb00b12d0 0x7f1cb00affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 >> v1:172.21.2.105:6789/0 conn(0x7f1cb00b22a0 legacy=0x7f1cb0141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:59.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb00b08f0 con 0x7f1cb00b1ec0 2021-07-11T04:01:59.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f1ca4001660 con 0x7f1cb00b1ec0 2021-07-11T04:01:59.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 >> v1:172.21.2.104:6789/0 conn(0x7f1cb00b1ec0 legacy=0x7f1cb00a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 --2- 172.21.2.104:0/2692894900 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1c98003780 0x7f1c98003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 --2- 172.21.2.104:0/2692894900 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cb00b12d0 0x7f1c98004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 --2- 172.21.2.104:0/2692894900 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1c98004b10 0x7f1c98004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb00b1740 con 0x7f1c98003780 2021-07-11T04:01:59.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9800b940 con 0x7f1c98004b10 2021-07-11T04:01:59.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9800bb00 con 0x7f1cb00b12d0 2021-07-11T04:01:59.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.160+0000 7f1cb648a700 1 --2- 172.21.2.104:0/2692894900 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1c98003780 0x7f1c98003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:59.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 -- 172.21.2.104:0/2692894900 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cb00b12d0 msgr2=0x7f1c98004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:59.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 --2- 172.21.2.104:0/2692894900 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cb00b12d0 0x7f1c98004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 -- 172.21.2.104:0/2692894900 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1c98004b10 msgr2=0x7f1c98004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:59.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 --2- 172.21.2.104:0/2692894900 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1c98004b10 0x7f1c98004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 -- 172.21.2.104:0/2692894900 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c98003260 con 0x7f1c98003780 2021-07-11T04:01:59.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 --2- 172.21.2.104:0/2692894900 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1c98003780 0x7f1c98003b80 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f1cb006a190 tx=0x7f1cb00b07e0).ready entity=mon.0 client_cookie=d60c54b6aeed8df3 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:59.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ca40070d0 con 0x7f1c98003780 2021-07-11T04:01:59.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca4007230 con 0x7f1c98003780 2021-07-11T04:01:59.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb5488700 1 -- 172.21.2.104:0/2692894900 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ca400c9c0 con 0x7f1c98003780 2021-07-11T04:01:59.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2692894900 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1c98003780 msgr2=0x7f1c98003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 --2- 172.21.2.104:0/2692894900 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1c98003780 0x7f1c98003b80 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f1cb006a190 tx=0x7f1cb00b07e0).stop 2021-07-11T04:01:59.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb6c8b700 1 -- 172.21.2.104:0/2692894900 reap_dead start 2021-07-11T04:01:59.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2692894900 shutdown_connections 2021-07-11T04:01:59.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2692894900 >> 172.21.2.104:0/2692894900 conn(0x7f1cb005f6b0 msgr2=0x7f1cb005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:59.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2692894900 shutdown_connections 2021-07-11T04:01:59.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2692894900 wait complete. 2021-07-11T04:01:59.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 Processor -- start 2021-07-11T04:01:59.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- start start 2021-07-11T04:01:59.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cb0147a30 0x7f1cb014a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cb014a9b0 0x7f1cb00aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cb014ae30 0x7f1cb00aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb0149360 con 0x7f1cb0147a30 2021-07-11T04:01:59.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb0148c70 con 0x7f1cb014a9b0 2021-07-11T04:01:59.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cb0149720 con 0x7f1cb014ae30 2021-07-11T04:01:59.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cb0147a30 0x7f1cb014a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:59.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cb0147a30 0x7f1cb014a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50560/0 (socket says 172.21.2.104:50560) 2021-07-11T04:01:59.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 -- 172.21.2.104:0/2711465532 learned_addr learned my addr 172.21.2.104:0/2711465532 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:59.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 -- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cb014ae30 msgr2=0x7f1cb00aa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cb014ae30 0x7f1cb00aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 -- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cb014a9b0 msgr2=0x7f1cb00aa3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cb014a9b0 0x7f1cb00aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 -- 172.21.2.104:0/2711465532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb0149ee0 con 0x7f1cb0147a30 2021-07-11T04:01:59.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb648a700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cb0147a30 0x7f1cb014a4c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f1cb006a190 tx=0x7f1ca4013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:59.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1ca37fe700 1 -- 172.21.2.104:0/2711465532 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ca401a070 con 0x7f1cb0147a30 2021-07-11T04:01:59.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1ca37fe700 1 -- 172.21.2.104:0/2711465532 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ca40049e0 con 0x7f1cb0147a30 2021-07-11T04:01:59.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2711465532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cb00ab5b0 con 0x7f1cb0147a30 2021-07-11T04:01:59.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1ca37fe700 1 -- 172.21.2.104:0/2711465532 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ca4001ab0 con 0x7f1cb0147a30 2021-07-11T04:01:59.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2711465532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cb0147fb0 con 0x7f1cb0147a30 2021-07-11T04:01:59.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.164+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2711465532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb0148e70 con 0x7f1cb0147a30 2021-07-11T04:01:59.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.168+0000 7f1ca37fe700 1 -- 172.21.2.104:0/2711465532 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1ca4010070 con 0x7f1cb0147a30 2021-07-11T04:01:59.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.168+0000 7f1ca37fe700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1c900532b0 0x7f1c90055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.168+0000 7f1ca37fe700 1 -- 172.21.2.104:0/2711465532 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1ca406b7c0 con 0x7f1cb0147a30 2021-07-11T04:01:59.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.168+0000 7f1cb5c89700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1c900532b0 0x7f1c90055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:59.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.168+0000 7f1cb5c89700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1c900532b0 0x7f1c90055770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f1cac002f70 tx=0x7f1cac006a10).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:59.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.168+0000 7f1ca37fe700 1 -- 172.21.2.104:0/2711465532 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1ca4042b00 con 0x7f1cb0147a30 2021-07-11T04:01:59.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2711465532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1cb0057350 con 0x7f1cb0147a30 2021-07-11T04:01:59.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1ca37fe700 1 -- 172.21.2.104:0/2711465532 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1ca4046120 con 0x7f1cb0147a30 2021-07-11T04:01:59.252 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:01:59.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1c900532b0 msgr2=0x7f1c90055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1c900532b0 0x7f1c90055770 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f1cac002f70 tx=0x7f1cac006a10).stop 2021-07-11T04:01:59.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2711465532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cb0147a30 msgr2=0x7f1cb014a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cb0147a30 0x7f1cb014a4c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f1cb006a190 tx=0x7f1ca4013000).stop 2021-07-11T04:01:59.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2711465532 shutdown_connections 2021-07-11T04:01:59.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1cb014ae30 0x7f1cb00aa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1cb014a9b0 0x7f1cb00aa3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1c900532b0 0x7f1c90055770 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:59.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 --2- 172.21.2.104:0/2711465532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1cb0147a30 0x7f1cb014a4c0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:59.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2711465532 >> 172.21.2.104:0/2711465532 conn(0x7f1cb005f6b0 msgr2=0x7f1cb00a70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:59.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2711465532 shutdown_connections 2021-07-11T04:01:59.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.252+0000 7f1cb86ee700 1 -- 172.21.2.104:0/2711465532 wait complete. 2021-07-11T04:01:59.260 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:01:59.260 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:01:59.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 Processor -- start 2021-07-11T04:01:59.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 -- start start 2021-07-11T04:01:59.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f43380b1170 0x7f4338141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4338141920 0x7f4338141db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43380b31d0 con 0x7f43380b7810 2021-07-11T04:01:59.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43381422a0 con 0x7f4338141920 2021-07-11T04:01:59.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43380b7bf0 con 0x7f43380b1170 2021-07-11T04:01:59.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433659c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f43380b7810 0x7f43381445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47336/0 (socket says 172.21.2.104:47336) 2021-07-11T04:01:59.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433659c700 1 -- 172.21.2.104:0/2775318122 learned_addr learned my addr 172.21.2.104:0/2775318122 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:01:59.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f4336d9d700 1 --2- 172.21.2.104:0/2775318122 >> v2:172.21.2.104:3300/0 conn(0x7f4338141920 0x7f4338141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:59.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433559a700 1 -- 172.21.2.104:0/2775318122 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276592267 0 0) 0x7f43380b31d0 con 0x7f43380b7810 2021-07-11T04:01:59.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433559a700 1 -- 172.21.2.104:0/2775318122 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f431c003260 con 0x7f43380b7810 2021-07-11T04:01:59.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f4336d9d700 1 -- 172.21.2.104:0/2775318122 >> v2:172.21.2.105:3300/0 conn(0x7f43380b1170 msgr2=0x7f4338141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:01:59.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f4336d9d700 1 --2- 172.21.2.104:0/2775318122 >> v2:172.21.2.105:3300/0 conn(0x7f43380b1170 0x7f4338141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f4336d9d700 1 -- 172.21.2.104:0/2775318122 >> v1:172.21.2.104:6789/0 conn(0x7f43380b7810 legacy=0x7f43381445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f4336d9d700 1 -- 172.21.2.104:0/2775318122 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43380675c0 con 0x7f4338141920 2021-07-11T04:01:59.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f4336d9d700 1 --2- 172.21.2.104:0/2775318122 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4338141920 0x7f4338141db0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f43300032f0 tx=0x7f43300091d0).ready entity=mon.0 client_cookie=c47a953b85cfefba server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:59.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433559a700 1 -- 172.21.2.104:0/2775318122 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f433000b680 con 0x7f4338141920 2021-07-11T04:01:59.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433559a700 1 -- 172.21.2.104:0/2775318122 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433000b7e0 con 0x7f4338141920 2021-07-11T04:01:59.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433559a700 1 -- 172.21.2.104:0/2775318122 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f433000b950 con 0x7f4338141920 2021-07-11T04:01:59.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 -- 172.21.2.104:0/2775318122 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4338141920 msgr2=0x7f4338141db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 --2- 172.21.2.104:0/2775318122 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4338141920 0x7f4338141db0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f43300032f0 tx=0x7f43300091d0).stop 2021-07-11T04:01:59.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 -- 172.21.2.104:0/2775318122 shutdown_connections 2021-07-11T04:01:59.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 --2- 172.21.2.104:0/2775318122 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4338141920 0x7f4338141db0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:59.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 --2- 172.21.2.104:0/2775318122 >> v2:172.21.2.105:3300/0 conn(0x7f43380b1170 0x7f4338141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 -- 172.21.2.104:0/2775318122 >> 172.21.2.104:0/2775318122 conn(0x7f433805f6b0 msgr2=0x7f433805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:59.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 -- 172.21.2.104:0/2775318122 shutdown_connections 2021-07-11T04:01:59.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 -- 172.21.2.104:0/2775318122 wait complete. 2021-07-11T04:01:59.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 Processor -- start 2021-07-11T04:01:59.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.312+0000 7f433cd6d700 1 -- start start 2021-07-11T04:01:59.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433cd6d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43380b1170 0x7f4338139460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433cd6d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43380b7810 0x7f4338143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433cd6d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4338141920 0x7f4338143a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433cd6d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4338133be0 con 0x7f43380b1170 2021-07-11T04:01:59.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433cd6d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4338144370 con 0x7f43380b7810 2021-07-11T04:01:59.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433cd6d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4338136220 con 0x7f4338141920 2021-07-11T04:01:59.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43380b1170 0x7f4338139460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:59.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433659c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43380b1170 0x7f4338139460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50574/0 (socket says 172.21.2.104:50574) 2021-07-11T04:01:59.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433659c700 1 -- 172.21.2.104:0/1503798335 learned_addr learned my addr 172.21.2.104:0/1503798335 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:01:59.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433659c700 1 -- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4338141920 msgr2=0x7f4338143a10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:01:59.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433659c700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4338141920 0x7f4338143a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433659c700 1 -- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43380b7810 msgr2=0x7f4338143520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433659c700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43380b7810 0x7f4338143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433659c700 1 -- 172.21.2.104:0/1503798335 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4330009470 con 0x7f43380b1170 2021-07-11T04:01:59.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433659c700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43380b1170 0x7f4338139460 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f4338136720 tx=0x7f4328007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:59.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f43277fe700 1 -- 172.21.2.104:0/1503798335 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43280072e0 con 0x7f43380b1170 2021-07-11T04:01:59.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f43277fe700 1 -- 172.21.2.104:0/1503798335 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4328007440 con 0x7f43380b1170 2021-07-11T04:01:59.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f43277fe700 1 -- 172.21.2.104:0/1503798335 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4328005800 con 0x7f43380b1170 2021-07-11T04:01:59.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433cd6d700 1 -- 172.21.2.104:0/1503798335 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4338142e10 con 0x7f43380b1170 2021-07-11T04:01:59.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433cd6d700 1 -- 172.21.2.104:0/1503798335 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4338143080 con 0x7f43380b1170 2021-07-11T04:01:59.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f433cd6d700 1 -- 172.21.2.104:0/1503798335 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4338057350 con 0x7f43380b1170 2021-07-11T04:01:59.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f43277fe700 1 -- 172.21.2.104:0/1503798335 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4328010070 con 0x7f43380b1170 2021-07-11T04:01:59.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f43277fe700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43140531f0 0x7f43140556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:01:59.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f43277fe700 1 -- 172.21.2.104:0/1503798335 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f432806abb0 con 0x7f43380b1170 2021-07-11T04:01:59.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f4335d9b700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43140531f0 0x7f43140556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:01:59.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.316+0000 7f4335d9b700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43140531f0 0x7f43140556b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f432c002f70 tx=0x7f432c006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:01:59.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.320+0000 7f43277fe700 1 -- 172.21.2.104:0/1503798335 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4328071080 con 0x7f43380b1170 2021-07-11T04:01:59.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 -- 172.21.2.104:0/1503798335 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f43380698e0 con 0x7f43380b1170 2021-07-11T04:01:59.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f43277fe700 1 -- 172.21.2.104:0/1503798335 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f4328018070 con 0x7f43380b1170 2021-07-11T04:01:59.430 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:01:59.430 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5335,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/1814518027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":1814518027},{"type":"v1","addr":"172.21.2.105:6835","nonce":1814518027}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/155502403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":155502403},{"type":"v1","addr":"172.21.2.104:6834","nonce":155502403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:52.876988+0000","modified":"2021-07-11T04:01:54.362899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/2616834266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":2616834266},{"type":"v1","addr":"172.21.2.104:6835","nonce":2616834266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-07-11T04:01:59.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 -- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43140531f0 msgr2=0x7f43140556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43140531f0 0x7f43140556b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f432c002f70 tx=0x7f432c006a80).stop 2021-07-11T04:01:59.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 -- 172.21.2.104:0/1503798335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43380b1170 msgr2=0x7f4338139460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:01:59.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43380b1170 0x7f4338139460 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f4338136720 tx=0x7f4328007ad0).stop 2021-07-11T04:01:59.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 -- 172.21.2.104:0/1503798335 shutdown_connections 2021-07-11T04:01:59.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4338141920 0x7f4338143a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f43380b7810 0x7f4338143520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:01:59.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43140531f0 0x7f43140556b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:59.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 --2- 172.21.2.104:0/1503798335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f43380b1170 0x7f4338139460 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:01:59.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 -- 172.21.2.104:0/1503798335 >> 172.21.2.104:0/1503798335 conn(0x7f433805f6b0 msgr2=0x7f43380b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:01:59.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 -- 172.21.2.104:0/1503798335 shutdown_connections 2021-07-11T04:01:59.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:01:59.428+0000 7f433cd6d700 1 -- 172.21.2.104:0/1503798335 wait complete. 2021-07-11T04:01:59.436 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 68 2021-07-11T04:01:59.741 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:01:52.876988+0000', 'modified': '2021-07-11T04:01:54.362899+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8022}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8022': {'gid': 8022, 'name': 'a', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/2616834266', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 2616834266}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 2616834266}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:01:59.741 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:01:59.742 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-07-11T04:01:59.807 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":8022,"want_state":"up:active","state":"up:active","rank_uptime":6.9252808520000002,"mdsmap_epoch":68,"osdmap_epoch":139,"osdmap_epoch_barrier":139,"uptime":11.183109696000001} 2021-07-11T04:01:59.808 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 8022, "mdsmap_epoch": 68, "osdmap_epoch": 139, "osdmap_epoch_barrier": 139, "rank_uptime": 6.925280852, "state": "up:active", "uptime": 11.183109696, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:01:59.808 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:01:59.809 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-07-11T04:01:59.809 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba023.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-07-11T04:01:59.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:59.810 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:59.821 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:01:59.876 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:59.877 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:59.883 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:59.884 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:59.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:59.886 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:01:59.891 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:01:59.891 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:01:59.934 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:01:59.944 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:01:59.945 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:01:59.951 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:01:59.951 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:01:59.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:01:59.954 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:01:59.972 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:2021-07-11T04:01:59.970+0000 7f5008209080 -1 init, newargv = 0x55ac3c64d2c0 newargc=9 2021-07-11T04:01:59.973 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15103]: starting ceph client 2021-07-11T04:01:59.986 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15103]: starting fuse 2021-07-11T04:02:01.078 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:02:01.090 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:01.090 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:01.138 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:01.139 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:01.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:01.141 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:02:01.146 INFO:teuthology.orchestra.run.gibba023.stdout:52 2021-07-11T04:02:01.147 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:02:01.148 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:01.149 DEBUG:teuthology.orchestra.run.gibba023:> sudo python3 -c ' 2021-07-11T04:02:01.149 DEBUG:teuthology.orchestra.run.gibba023:> import glob 2021-07-11T04:02:01.150 DEBUG:teuthology.orchestra.run.gibba023:> import re 2021-07-11T04:02:01.151 DEBUG:teuthology.orchestra.run.gibba023:> import os 2021-07-11T04:02:01.153 DEBUG:teuthology.orchestra.run.gibba023:> import subprocess 2021-07-11T04:02:01.153 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:01.154 DEBUG:teuthology.orchestra.run.gibba023:> def find_socket(client_name): 2021-07-11T04:02:01.155 DEBUG:teuthology.orchestra.run.gibba023:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-07-11T04:02:01.156 DEBUG:teuthology.orchestra.run.gibba023:> files = glob.glob(asok_path) 2021-07-11T04:02:01.157 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:01.158 DEBUG:teuthology.orchestra.run.gibba023:> # Given a non-glob path, it better be there 2021-07-11T04:02:01.159 DEBUG:teuthology.orchestra.run.gibba023:> if "*" not in asok_path: 2021-07-11T04:02:01.160 DEBUG:teuthology.orchestra.run.gibba023:> assert(len(files) == 1) 2021-07-11T04:02:01.161 DEBUG:teuthology.orchestra.run.gibba023:> return files[0] 2021-07-11T04:02:01.161 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:01.163 DEBUG:teuthology.orchestra.run.gibba023:> for f in files: 2021-07-11T04:02:01.164 DEBUG:teuthology.orchestra.run.gibba023:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:02:01.164 DEBUG:teuthology.orchestra.run.gibba023:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:02:01.165 DEBUG:teuthology.orchestra.run.gibba023:> return f 2021-07-11T04:02:01.166 DEBUG:teuthology.orchestra.run.gibba023:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:02:01.167 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:01.168 DEBUG:teuthology.orchestra.run.gibba023:> print(find_socket("client.1")) 2021-07-11T04:02:01.169 DEBUG:teuthology.orchestra.run.gibba023:> ' 2021-07-11T04:02:01.212 INFO:teuthology.orchestra.run.gibba023.stdout:/var/run/ceph/ceph-client.1.15103.asok 2021-07-11T04:02:01.214 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15103.asok 2021-07-11T04:02:01.215 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:01.216 DEBUG:teuthology.orchestra.run.gibba023:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15103.asok status 2021-07-11T04:02:01.262 INFO:teuthology.orchestra.run.gibba023.stdout:{ 2021-07-11T04:02:01.263 INFO:teuthology.orchestra.run.gibba023.stdout: "metadata": { 2021-07-11T04:02:01.263 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:02:01.263 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:02:01.263 INFO:teuthology.orchestra.run.gibba023.stdout: "entity_id": "1", 2021-07-11T04:02:01.264 INFO:teuthology.orchestra.run.gibba023.stdout: "hostname": "gibba023", 2021-07-11T04:02:01.264 INFO:teuthology.orchestra.run.gibba023.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-07-11T04:02:01.264 INFO:teuthology.orchestra.run.gibba023.stdout: "pid": "15103", 2021-07-11T04:02:01.265 INFO:teuthology.orchestra.run.gibba023.stdout: "root": "/" 2021-07-11T04:02:01.265 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:01.265 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_count": 0, 2021-07-11T04:02:01.265 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_pinned_count": 0, 2021-07-11T04:02:01.266 INFO:teuthology.orchestra.run.gibba023.stdout: "id": 8250, 2021-07-11T04:02:01.266 INFO:teuthology.orchestra.run.gibba023.stdout: "inst": { 2021-07-11T04:02:01.266 INFO:teuthology.orchestra.run.gibba023.stdout: "name": { 2021-07-11T04:02:01.266 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "client", 2021-07-11T04:02:01.267 INFO:teuthology.orchestra.run.gibba023.stdout: "num": 8250 2021-07-11T04:02:01.267 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:01.267 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:02:01.267 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:02:01.267 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:02:01.268 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 3865639210 2021-07-11T04:02:01.268 INFO:teuthology.orchestra.run.gibba023.stdout: } 2021-07-11T04:02:01.268 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:01.268 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:02:01.269 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:02:01.269 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:02:01.269 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 3865639210 2021-07-11T04:02:01.269 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:01.269 INFO:teuthology.orchestra.run.gibba023.stdout: "inst_str": "client.8250 172.21.2.123:0/3865639210", 2021-07-11T04:02:01.270 INFO:teuthology.orchestra.run.gibba023.stdout: "addr_str": "172.21.2.123:0/3865639210", 2021-07-11T04:02:01.270 INFO:teuthology.orchestra.run.gibba023.stdout: "inode_count": 1, 2021-07-11T04:02:01.270 INFO:teuthology.orchestra.run.gibba023.stdout: "mds_epoch": 68, 2021-07-11T04:02:01.270 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch": 139, 2021-07-11T04:02:01.271 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:02:01.271 INFO:teuthology.orchestra.run.gibba023.stdout: "blacklisted": false, 2021-07-11T04:02:01.271 INFO:teuthology.orchestra.run.gibba023.stdout: "fs_name": "cephfs" 2021-07-11T04:02:01.271 INFO:teuthology.orchestra.run.gibba023.stdout:} 2021-07-11T04:02:01.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:01.274 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:01.274 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:01.280 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:02:01.280 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:01.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:01.281 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:01.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:01.291 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:01.291 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:01.297 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:02:01.297 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:01.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:01.298 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:01.298 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:01.304 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:02:01.304 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:01.305 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba023.front.sepia.ceph.com... 2021-07-11T04:02:01.305 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:02:01.306 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:01.312 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15103]: fuse finished with error 0 and tester_r 0 2021-07-11T04:02:01.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:01.313 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:01.313 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:01.366 INFO:teuthology.orchestra.run.gibba023.stdout:ext2/ext3 2021-07-11T04:02:01.366 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:02:01.367 INFO:teuthology.orchestra.run:waiting for 900 2021-07-11T04:02:01.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:02:01.373 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:01.378 DEBUG:teuthology.orchestra.run.gibba004:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T04:02:01.382 DEBUG:teuthology.orchestra.run.gibba005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T04:02:01.384 DEBUG:teuthology.orchestra.run.gibba019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T04:02:01.388 DEBUG:teuthology.orchestra.run.gibba023:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T04:02:01.392 INFO:teuthology.orchestra.run.gibba004.stdout:[global] 2021-07-11T04:02:01.392 INFO:teuthology.orchestra.run.gibba004.stdout: chdir = "" 2021-07-11T04:02:01.392 INFO:teuthology.orchestra.run.gibba004.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T04:02:01.393 INFO:teuthology.orchestra.run.gibba004.stdout: auth supported = cephx 2021-07-11T04:02:01.393 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.393 INFO:teuthology.orchestra.run.gibba004.stdout: filestore xattr use omap = true 2021-07-11T04:02:01.393 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.393 INFO:teuthology.orchestra.run.gibba004.stdout: mon clock drift allowed = 1.000 2021-07-11T04:02:01.394 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.394 INFO:teuthology.orchestra.run.gibba004.stdout: osd crush chooseleaf type = 0 2021-07-11T04:02:01.394 INFO:teuthology.orchestra.run.gibba004.stdout: auth debug = true 2021-07-11T04:02:01.394 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.395 INFO:teuthology.orchestra.run.gibba004.stdout: ms die on old message = true 2021-07-11T04:02:01.395 INFO:teuthology.orchestra.run.gibba004.stdout: ms die on bug = true 2021-07-11T04:02:01.395 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.395 INFO:teuthology.orchestra.run.gibba004.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T04:02:01.395 INFO:teuthology.orchestra.run.gibba004.stdout: mon pg warn max object skew = 0 2021-07-11T04:02:01.396 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.396 INFO:teuthology.orchestra.run.gibba004.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T04:02:01.396 INFO:teuthology.orchestra.run.gibba004.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T04:02:01.396 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.397 INFO:teuthology.orchestra.run.gibba004.stdout: osd pool default size = 2 2021-07-11T04:02:01.397 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.397 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd allow primary affinity = true 2021-07-11T04:02:01.397 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd allow pg remap = true 2021-07-11T04:02:01.397 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on legacy crush tunables = false 2021-07-11T04:02:01.398 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on crush straw calc version zero = false 2021-07-11T04:02:01.398 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on no sortbitwise = false 2021-07-11T04:02:01.398 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on osd down out interval zero = false 2021-07-11T04:02:01.398 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on too few osds = false 2021-07-11T04:02:01.399 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T04:02:01.399 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T04:02:01.400 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.400 INFO:teuthology.orchestra.run.gibba004.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T04:02:01.400 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.400 INFO:teuthology.orchestra.run.gibba004.stdout: osd default data pool replay window = 5 2021-07-11T04:02:01.401 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.401 INFO:teuthology.orchestra.run.gibba004.stdout: mon allow pool delete = true 2021-07-11T04:02:01.401 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.401 INFO:teuthology.orchestra.run.gibba004.stdout: mon cluster log file level = debug 2021-07-11T04:02:01.402 INFO:teuthology.orchestra.run.gibba004.stdout: debug asserts on shutdown = true 2021-07-11T04:02:01.402 INFO:teuthology.orchestra.run.gibba004.stdout: mon health detail to clog = false 2021-07-11T04:02:01.402 INFO:teuthology.orchestra.run.gibba004.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T04:02:01.402 INFO:teuthology.orchestra.run.gibba004.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T04:02:01.403 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.403 INFO:teuthology.orchestra.run.gibba004.stdout:[osd] 2021-07-11T04:02:01.403 INFO:teuthology.orchestra.run.gibba004.stdout: osd journal size = 100 2021-07-11T04:02:01.403 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.403 INFO:teuthology.orchestra.run.gibba004.stdout: osd scrub load threshold = 5.0 2021-07-11T04:02:01.404 INFO:teuthology.orchestra.run.gibba004.stdout: osd scrub max interval = 600 2021-07-11T04:02:01.404 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.404 INFO:teuthology.orchestra.run.gibba004.stdout: osd recover clone overlap = true 2021-07-11T04:02:01.404 INFO:teuthology.orchestra.run.gibba004.stdout: osd recovery max chunk = 1048576 2021-07-11T04:02:01.405 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.405 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug shutdown = true 2021-07-11T04:02:01.405 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug op order = true 2021-07-11T04:02:01.405 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug verify stray on activate = true 2021-07-11T04:02:01.405 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.406 INFO:teuthology.orchestra.run.gibba004.stdout: osd open classes on start = true 2021-07-11T04:02:01.406 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug pg log writeout = true 2021-07-11T04:02:01.406 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.406 INFO:teuthology.orchestra.run.gibba004.stdout: osd deep scrub update digest min age = 30 2021-07-11T04:02:01.407 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.407 INFO:teuthology.orchestra.run.gibba004.stdout: osd map max advance = 10 2021-07-11T04:02:01.407 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.407 INFO:teuthology.orchestra.run.gibba004.stdout: journal zero on create = true 2021-07-11T04:02:01.407 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.408 INFO:teuthology.orchestra.run.gibba004.stdout: filestore ondisk finisher threads = 3 2021-07-11T04:02:01.408 INFO:teuthology.orchestra.run.gibba004.stdout: filestore apply finisher threads = 3 2021-07-11T04:02:01.408 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.408 INFO:teuthology.orchestra.run.gibba004.stdout: bdev debug aio = true 2021-07-11T04:02:01.409 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug misdirected ops = true 2021-07-11T04:02:01.409 INFO:teuthology.orchestra.run.gibba004.stdout: bluestore block size = 96636764160 2021-07-11T04:02:01.409 INFO:teuthology.orchestra.run.gibba004.stdout: bluestore compression mode = aggressive 2021-07-11T04:02:01.409 INFO:teuthology.orchestra.run.gibba004.stdout: bluestore fsck on mount = True 2021-07-11T04:02:01.409 INFO:teuthology.orchestra.run.gibba004.stdout: debug bluefs = 20 2021-07-11T04:02:01.410 INFO:teuthology.orchestra.run.gibba004.stdout: debug bluestore = 20 2021-07-11T04:02:01.410 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T04:02:01.410 INFO:teuthology.orchestra.run.gibba004.stdout: debug osd = 20 2021-07-11T04:02:01.410 INFO:teuthology.orchestra.run.gibba004.stdout: debug rocksdb = 10 2021-07-11T04:02:01.411 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T04:02:01.411 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd full ratio = 0.9 2021-07-11T04:02:01.411 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd nearfull ratio = 0.8 2021-07-11T04:02:01.411 INFO:teuthology.orchestra.run.gibba004.stdout: osd failsafe full ratio = 0.95 2021-07-11T04:02:01.411 INFO:teuthology.orchestra.run.gibba004.stdout: osd objectstore = bluestore 2021-07-11T04:02:01.412 INFO:teuthology.orchestra.run.gibba004.stdout: osd op complaint time = 180 2021-07-11T04:02:01.412 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.412 INFO:teuthology.orchestra.run.gibba004.stdout:[mgr] 2021-07-11T04:02:01.412 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T04:02:01.412 INFO:teuthology.orchestra.run.gibba004.stdout: debug mgr = 20 2021-07-11T04:02:01.413 INFO:teuthology.orchestra.run.gibba004.stdout: debug mon = 20 2021-07-11T04:02:01.413 INFO:teuthology.orchestra.run.gibba004.stdout: debug auth = 20 2021-07-11T04:02:01.413 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:01.413 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:01.414 INFO:teuthology.orchestra.run.gibba004.stdout: mgr/telemetry/nag = false 2021-07-11T04:02:01.414 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.414 INFO:teuthology.orchestra.run.gibba004.stdout:[mon] 2021-07-11T04:02:01.414 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T04:02:01.415 INFO:teuthology.orchestra.run.gibba004.stdout: debug mon = 20 2021-07-11T04:02:01.415 INFO:teuthology.orchestra.run.gibba004.stdout: debug paxos = 20 2021-07-11T04:02:01.415 INFO:teuthology.orchestra.run.gibba004.stdout: debug auth = 20 2021-07-11T04:02:01.415 INFO:teuthology.orchestra.run.gibba004.stdout: mon data avail warn = 5 2021-07-11T04:02:01.416 INFO:teuthology.orchestra.run.gibba004.stdout: mon mgr mkfs grace = 240 2021-07-11T04:02:01.416 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:01.416 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd reporter subtree level = osd 2021-07-11T04:02:01.416 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd prime pg temp = true 2021-07-11T04:02:01.416 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:01.417 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.417 INFO:teuthology.orchestra.run.gibba004.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T04:02:01.417 INFO:teuthology.orchestra.run.gibba004.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T04:02:01.417 INFO:teuthology.orchestra.run.gibba004.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T04:02:01.418 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.418 INFO:teuthology.orchestra.run.gibba004.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T04:02:01.418 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T04:02:01.418 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T04:02:01.418 INFO:teuthology.orchestra.run.gibba004.stdout: mon op complaint time = 120 2021-07-11T04:02:01.419 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.419 INFO:teuthology.orchestra.run.gibba004.stdout:[client] 2021-07-11T04:02:01.419 INFO:teuthology.orchestra.run.gibba004.stdout: rgw cache enabled = true 2021-07-11T04:02:01.419 INFO:teuthology.orchestra.run.gibba004.stdout: rgw enable ops log = true 2021-07-11T04:02:01.420 INFO:teuthology.orchestra.run.gibba004.stdout: rgw enable usage log = true 2021-07-11T04:02:01.420 INFO:teuthology.orchestra.run.gibba004.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T04:02:01.420 INFO:teuthology.orchestra.run.gibba004.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T04:02:01.420 INFO:teuthology.orchestra.run.gibba004.stdout: client mount timeout = 600 2021-07-11T04:02:01.420 INFO:teuthology.orchestra.run.gibba004.stdout: debug client = 20 2021-07-11T04:02:01.421 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T04:02:01.421 INFO:teuthology.orchestra.run.gibba004.stdout: rados mon op timeout = 15m 2021-07-11T04:02:01.421 INFO:teuthology.orchestra.run.gibba004.stdout: rados osd op timeout = 15m 2021-07-11T04:02:01.421 INFO:teuthology.orchestra.run.gibba004.stdout: client debug getattr caps = true 2021-07-11T04:02:01.422 INFO:teuthology.orchestra.run.gibba004.stdout:[mon.a] 2021-07-11T04:02:01.422 INFO:teuthology.orchestra.run.gibba004.stdout:[mon.b] 2021-07-11T04:02:01.422 INFO:teuthology.orchestra.run.gibba004.stdout:[mon.c] 2021-07-11T04:02:01.422 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.a] 2021-07-11T04:02:01.422 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.c] 2021-07-11T04:02:01.423 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.b] 2021-07-11T04:02:01.423 INFO:teuthology.orchestra.run.gibba004.stdout:[mds] 2021-07-11T04:02:01.423 INFO:teuthology.orchestra.run.gibba004.stdout: debug mds = 20 2021-07-11T04:02:01.423 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T04:02:01.424 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal frag = True 2021-07-11T04:02:01.424 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal fragment size max = 10000 2021-07-11T04:02:01.424 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal merge size = 5 2021-07-11T04:02:01.424 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal split bits = 3 2021-07-11T04:02:01.424 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal split size = 100 2021-07-11T04:02:01.425 INFO:teuthology.orchestra.run.gibba004.stdout: mds debug frag = True 2021-07-11T04:02:01.425 INFO:teuthology.orchestra.run.gibba004.stdout: mds debug scatterstat = True 2021-07-11T04:02:01.425 INFO:teuthology.orchestra.run.gibba004.stdout: mds op complaint time = 180 2021-07-11T04:02:01.425 INFO:teuthology.orchestra.run.gibba004.stdout: mds verify scatter = True 2021-07-11T04:02:01.425 INFO:teuthology.orchestra.run.gibba004.stdout: osd op complaint time = 180 2021-07-11T04:02:01.426 INFO:teuthology.orchestra.run.gibba004.stdout: rados mon op timeout = 15m 2021-07-11T04:02:01.426 INFO:teuthology.orchestra.run.gibba004.stdout: rados osd op timeout = 15m 2021-07-11T04:02:01.429 INFO:teuthology.orchestra.run.gibba019.stdout:[global] 2021-07-11T04:02:01.429 INFO:teuthology.orchestra.run.gibba019.stdout: chdir = "" 2021-07-11T04:02:01.430 INFO:teuthology.orchestra.run.gibba019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T04:02:01.430 INFO:teuthology.orchestra.run.gibba019.stdout: auth supported = cephx 2021-07-11T04:02:01.430 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.430 INFO:teuthology.orchestra.run.gibba019.stdout: filestore xattr use omap = true 2021-07-11T04:02:01.431 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.431 INFO:teuthology.orchestra.run.gibba019.stdout: mon clock drift allowed = 1.000 2021-07-11T04:02:01.431 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.431 INFO:teuthology.orchestra.run.gibba019.stdout: osd crush chooseleaf type = 0 2021-07-11T04:02:01.432 INFO:teuthology.orchestra.run.gibba019.stdout: auth debug = true 2021-07-11T04:02:01.432 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.432 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on old message = true 2021-07-11T04:02:01.432 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on bug = true 2021-07-11T04:02:01.433 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.433 INFO:teuthology.orchestra.run.gibba019.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T04:02:01.433 INFO:teuthology.orchestra.run.gibba019.stdout: mon pg warn max object skew = 0 2021-07-11T04:02:01.433 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.433 INFO:teuthology.orchestra.run.gibba019.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T04:02:01.434 INFO:teuthology.orchestra.run.gibba019.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T04:02:01.434 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.434 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default size = 2 2021-07-11T04:02:01.434 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.435 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow primary affinity = true 2021-07-11T04:02:01.435 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow pg remap = true 2021-07-11T04:02:01.435 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on legacy crush tunables = false 2021-07-11T04:02:01.435 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on crush straw calc version zero = false 2021-07-11T04:02:01.435 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on no sortbitwise = false 2021-07-11T04:02:01.436 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on osd down out interval zero = false 2021-07-11T04:02:01.436 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on too few osds = false 2021-07-11T04:02:01.436 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T04:02:01.436 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T04:02:01.437 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.437 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T04:02:01.437 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.437 INFO:teuthology.orchestra.run.gibba019.stdout: osd default data pool replay window = 5 2021-07-11T04:02:01.438 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.438 INFO:teuthology.orchestra.run.gibba019.stdout: mon allow pool delete = true 2021-07-11T04:02:01.438 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.438 INFO:teuthology.orchestra.run.gibba019.stdout: mon cluster log file level = debug 2021-07-11T04:02:01.439 INFO:teuthology.orchestra.run.gibba019.stdout: debug asserts on shutdown = true 2021-07-11T04:02:01.439 INFO:teuthology.orchestra.run.gibba019.stdout: mon health detail to clog = false 2021-07-11T04:02:01.439 INFO:teuthology.orchestra.run.gibba019.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T04:02:01.439 INFO:teuthology.orchestra.run.gibba019.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T04:02:01.439 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.440 INFO:teuthology.orchestra.run.gibba019.stdout:[osd] 2021-07-11T04:02:01.440 INFO:teuthology.orchestra.run.gibba019.stdout: osd journal size = 100 2021-07-11T04:02:01.440 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.440 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub load threshold = 5.0 2021-07-11T04:02:01.441 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub max interval = 600 2021-07-11T04:02:01.441 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.441 INFO:teuthology.orchestra.run.gibba019.stdout: osd recover clone overlap = true 2021-07-11T04:02:01.441 INFO:teuthology.orchestra.run.gibba019.stdout: osd recovery max chunk = 1048576 2021-07-11T04:02:01.442 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.442 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug shutdown = true 2021-07-11T04:02:01.442 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug op order = true 2021-07-11T04:02:01.442 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug verify stray on activate = true 2021-07-11T04:02:01.442 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.443 INFO:teuthology.orchestra.run.gibba019.stdout: osd open classes on start = true 2021-07-11T04:02:01.443 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug pg log writeout = true 2021-07-11T04:02:01.443 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.443 INFO:teuthology.orchestra.run.gibba019.stdout: osd deep scrub update digest min age = 30 2021-07-11T04:02:01.444 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.444 INFO:teuthology.orchestra.run.gibba019.stdout: osd map max advance = 10 2021-07-11T04:02:01.444 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.444 INFO:teuthology.orchestra.run.gibba019.stdout: journal zero on create = true 2021-07-11T04:02:01.444 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.445 INFO:teuthology.orchestra.run.gibba019.stdout: filestore ondisk finisher threads = 3 2021-07-11T04:02:01.445 INFO:teuthology.orchestra.run.gibba019.stdout: filestore apply finisher threads = 3 2021-07-11T04:02:01.445 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.445 INFO:teuthology.orchestra.run.gibba019.stdout: bdev debug aio = true 2021-07-11T04:02:01.446 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug misdirected ops = true 2021-07-11T04:02:01.446 INFO:teuthology.orchestra.run.gibba019.stdout: bluestore block size = 96636764160 2021-07-11T04:02:01.446 INFO:teuthology.orchestra.run.gibba019.stdout: bluestore compression mode = aggressive 2021-07-11T04:02:01.446 INFO:teuthology.orchestra.run.gibba019.stdout: bluestore fsck on mount = True 2021-07-11T04:02:01.446 INFO:teuthology.orchestra.run.gibba019.stdout: debug bluefs = 20 2021-07-11T04:02:01.447 INFO:teuthology.orchestra.run.gibba019.stdout: debug bluestore = 20 2021-07-11T04:02:01.447 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T04:02:01.447 INFO:teuthology.orchestra.run.gibba019.stdout: debug osd = 20 2021-07-11T04:02:01.447 INFO:teuthology.orchestra.run.gibba019.stdout: debug rocksdb = 10 2021-07-11T04:02:01.448 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T04:02:01.448 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd full ratio = 0.9 2021-07-11T04:02:01.448 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd nearfull ratio = 0.8 2021-07-11T04:02:01.448 INFO:teuthology.orchestra.run.gibba019.stdout: osd failsafe full ratio = 0.95 2021-07-11T04:02:01.448 INFO:teuthology.orchestra.run.gibba019.stdout: osd objectstore = bluestore 2021-07-11T04:02:01.449 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-07-11T04:02:01.449 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.449 INFO:teuthology.orchestra.run.gibba019.stdout:[mgr] 2021-07-11T04:02:01.449 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T04:02:01.450 INFO:teuthology.orchestra.run.gibba019.stdout: debug mgr = 20 2021-07-11T04:02:01.450 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-07-11T04:02:01.450 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-07-11T04:02:01.450 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:01.450 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:01.451 INFO:teuthology.orchestra.run.gibba019.stdout: mgr/telemetry/nag = false 2021-07-11T04:02:01.451 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.451 INFO:teuthology.orchestra.run.gibba019.stdout:[mon] 2021-07-11T04:02:01.451 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T04:02:01.452 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-07-11T04:02:01.452 INFO:teuthology.orchestra.run.gibba019.stdout: debug paxos = 20 2021-07-11T04:02:01.452 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-07-11T04:02:01.452 INFO:teuthology.orchestra.run.gibba019.stdout: mon data avail warn = 5 2021-07-11T04:02:01.453 INFO:teuthology.orchestra.run.gibba019.stdout: mon mgr mkfs grace = 240 2021-07-11T04:02:01.453 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:01.453 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd reporter subtree level = osd 2021-07-11T04:02:01.453 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd prime pg temp = true 2021-07-11T04:02:01.453 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:01.454 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.454 INFO:teuthology.orchestra.run.gibba019.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T04:02:01.454 INFO:teuthology.orchestra.run.gibba019.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T04:02:01.454 INFO:teuthology.orchestra.run.gibba019.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T04:02:01.455 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.455 INFO:teuthology.orchestra.run.gibba019.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T04:02:01.455 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T04:02:01.455 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T04:02:01.455 INFO:teuthology.orchestra.run.gibba019.stdout: mon op complaint time = 120 2021-07-11T04:02:01.456 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:01.456 INFO:teuthology.orchestra.run.gibba019.stdout:[client] 2021-07-11T04:02:01.456 INFO:teuthology.orchestra.run.gibba019.stdout: rgw cache enabled = true 2021-07-11T04:02:01.456 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable ops log = true 2021-07-11T04:02:01.456 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable usage log = true 2021-07-11T04:02:01.457 INFO:teuthology.orchestra.run.gibba019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T04:02:01.457 INFO:teuthology.orchestra.run.gibba019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T04:02:01.457 INFO:teuthology.orchestra.run.gibba019.stdout: client mount timeout = 600 2021-07-11T04:02:01.457 INFO:teuthology.orchestra.run.gibba019.stdout: debug client = 20 2021-07-11T04:02:01.458 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T04:02:01.458 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-07-11T04:02:01.458 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-07-11T04:02:01.458 INFO:teuthology.orchestra.run.gibba019.stdout: client debug getattr caps = true 2021-07-11T04:02:01.458 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.a] 2021-07-11T04:02:01.459 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.b] 2021-07-11T04:02:01.459 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.c] 2021-07-11T04:02:01.459 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.a] 2021-07-11T04:02:01.459 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.c] 2021-07-11T04:02:01.460 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.b] 2021-07-11T04:02:01.460 INFO:teuthology.orchestra.run.gibba019.stdout:[mds] 2021-07-11T04:02:01.460 INFO:teuthology.orchestra.run.gibba019.stdout: debug mds = 20 2021-07-11T04:02:01.460 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T04:02:01.460 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal frag = True 2021-07-11T04:02:01.461 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal fragment size max = 10000 2021-07-11T04:02:01.461 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal merge size = 5 2021-07-11T04:02:01.461 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split bits = 3 2021-07-11T04:02:01.461 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split size = 100 2021-07-11T04:02:01.462 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug frag = True 2021-07-11T04:02:01.462 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug scatterstat = True 2021-07-11T04:02:01.462 INFO:teuthology.orchestra.run.gibba019.stdout: mds op complaint time = 180 2021-07-11T04:02:01.462 INFO:teuthology.orchestra.run.gibba019.stdout: mds verify scatter = True 2021-07-11T04:02:01.463 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-07-11T04:02:01.463 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-07-11T04:02:01.463 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-07-11T04:02:01.463 INFO:teuthology.orchestra.run.gibba023.stdout:[global] 2021-07-11T04:02:01.464 INFO:teuthology.orchestra.run.gibba023.stdout: chdir = "" 2021-07-11T04:02:01.464 INFO:teuthology.orchestra.run.gibba023.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T04:02:01.464 INFO:teuthology.orchestra.run.gibba023.stdout: auth supported = cephx 2021-07-11T04:02:01.464 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.464 INFO:teuthology.orchestra.run.gibba023.stdout: filestore xattr use omap = true 2021-07-11T04:02:01.465 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.465 INFO:teuthology.orchestra.run.gibba023.stdout: mon clock drift allowed = 1.000 2021-07-11T04:02:01.465 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.465 INFO:teuthology.orchestra.run.gibba023.stdout: osd crush chooseleaf type = 0 2021-07-11T04:02:01.466 INFO:teuthology.orchestra.run.gibba023.stdout: auth debug = true 2021-07-11T04:02:01.466 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.466 INFO:teuthology.orchestra.run.gibba023.stdout: ms die on old message = true 2021-07-11T04:02:01.466 INFO:teuthology.orchestra.run.gibba023.stdout: ms die on bug = true 2021-07-11T04:02:01.466 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.467 INFO:teuthology.orchestra.run.gibba023.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T04:02:01.467 INFO:teuthology.orchestra.run.gibba023.stdout: mon pg warn max object skew = 0 2021-07-11T04:02:01.467 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.467 INFO:teuthology.orchestra.run.gibba023.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T04:02:01.468 INFO:teuthology.orchestra.run.gibba023.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T04:02:01.468 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.468 INFO:teuthology.orchestra.run.gibba023.stdout: osd pool default size = 2 2021-07-11T04:02:01.468 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.468 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd allow primary affinity = true 2021-07-11T04:02:01.469 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd allow pg remap = true 2021-07-11T04:02:01.469 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on legacy crush tunables = false 2021-07-11T04:02:01.469 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on crush straw calc version zero = false 2021-07-11T04:02:01.469 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on no sortbitwise = false 2021-07-11T04:02:01.470 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on osd down out interval zero = false 2021-07-11T04:02:01.470 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on too few osds = false 2021-07-11T04:02:01.470 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T04:02:01.470 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T04:02:01.470 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.471 INFO:teuthology.orchestra.run.gibba023.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T04:02:01.471 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.471 INFO:teuthology.orchestra.run.gibba023.stdout: osd default data pool replay window = 5 2021-07-11T04:02:01.471 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.472 INFO:teuthology.orchestra.run.gibba023.stdout: mon allow pool delete = true 2021-07-11T04:02:01.472 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.472 INFO:teuthology.orchestra.run.gibba023.stdout: mon cluster log file level = debug 2021-07-11T04:02:01.472 INFO:teuthology.orchestra.run.gibba023.stdout: debug asserts on shutdown = true 2021-07-11T04:02:01.472 INFO:teuthology.orchestra.run.gibba023.stdout: mon health detail to clog = false 2021-07-11T04:02:01.473 INFO:teuthology.orchestra.run.gibba023.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T04:02:01.473 INFO:teuthology.orchestra.run.gibba023.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T04:02:01.473 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.473 INFO:teuthology.orchestra.run.gibba023.stdout:[osd] 2021-07-11T04:02:01.474 INFO:teuthology.orchestra.run.gibba023.stdout: osd journal size = 100 2021-07-11T04:02:01.474 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.474 INFO:teuthology.orchestra.run.gibba023.stdout: osd scrub load threshold = 5.0 2021-07-11T04:02:01.474 INFO:teuthology.orchestra.run.gibba023.stdout: osd scrub max interval = 600 2021-07-11T04:02:01.474 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.475 INFO:teuthology.orchestra.run.gibba023.stdout: osd recover clone overlap = true 2021-07-11T04:02:01.475 INFO:teuthology.orchestra.run.gibba023.stdout: osd recovery max chunk = 1048576 2021-07-11T04:02:01.475 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.475 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug shutdown = true 2021-07-11T04:02:01.476 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug op order = true 2021-07-11T04:02:01.476 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug verify stray on activate = true 2021-07-11T04:02:01.476 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.476 INFO:teuthology.orchestra.run.gibba023.stdout: osd open classes on start = true 2021-07-11T04:02:01.476 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug pg log writeout = true 2021-07-11T04:02:01.477 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.477 INFO:teuthology.orchestra.run.gibba023.stdout: osd deep scrub update digest min age = 30 2021-07-11T04:02:01.477 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.477 INFO:teuthology.orchestra.run.gibba023.stdout: osd map max advance = 10 2021-07-11T04:02:01.477 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.478 INFO:teuthology.orchestra.run.gibba023.stdout: journal zero on create = true 2021-07-11T04:02:01.478 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.478 INFO:teuthology.orchestra.run.gibba023.stdout: filestore ondisk finisher threads = 3 2021-07-11T04:02:01.478 INFO:teuthology.orchestra.run.gibba023.stdout: filestore apply finisher threads = 3 2021-07-11T04:02:01.479 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.479 INFO:teuthology.orchestra.run.gibba023.stdout: bdev debug aio = true 2021-07-11T04:02:01.479 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug misdirected ops = true 2021-07-11T04:02:01.479 INFO:teuthology.orchestra.run.gibba023.stdout: bluestore block size = 96636764160 2021-07-11T04:02:01.480 INFO:teuthology.orchestra.run.gibba023.stdout: bluestore compression mode = aggressive 2021-07-11T04:02:01.480 INFO:teuthology.orchestra.run.gibba023.stdout: bluestore fsck on mount = True 2021-07-11T04:02:01.480 INFO:teuthology.orchestra.run.gibba023.stdout: debug bluefs = 20 2021-07-11T04:02:01.480 INFO:teuthology.orchestra.run.gibba023.stdout: debug bluestore = 20 2021-07-11T04:02:01.481 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T04:02:01.481 INFO:teuthology.orchestra.run.gibba023.stdout: debug osd = 20 2021-07-11T04:02:01.481 INFO:teuthology.orchestra.run.gibba023.stdout: debug rocksdb = 10 2021-07-11T04:02:01.481 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T04:02:01.481 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd full ratio = 0.9 2021-07-11T04:02:01.482 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd nearfull ratio = 0.8 2021-07-11T04:02:01.482 INFO:teuthology.orchestra.run.gibba023.stdout: osd failsafe full ratio = 0.95 2021-07-11T04:02:01.482 INFO:teuthology.orchestra.run.gibba023.stdout: osd objectstore = bluestore 2021-07-11T04:02:01.482 INFO:teuthology.orchestra.run.gibba023.stdout: osd op complaint time = 180 2021-07-11T04:02:01.483 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.483 INFO:teuthology.orchestra.run.gibba023.stdout:[mgr] 2021-07-11T04:02:01.483 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T04:02:01.483 INFO:teuthology.orchestra.run.gibba023.stdout: debug mgr = 20 2021-07-11T04:02:01.483 INFO:teuthology.orchestra.run.gibba023.stdout: debug mon = 20 2021-07-11T04:02:01.484 INFO:teuthology.orchestra.run.gibba023.stdout: debug auth = 20 2021-07-11T04:02:01.484 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:01.484 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:01.484 INFO:teuthology.orchestra.run.gibba023.stdout: mgr/telemetry/nag = false 2021-07-11T04:02:01.485 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.485 INFO:teuthology.orchestra.run.gibba023.stdout:[mon] 2021-07-11T04:02:01.485 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T04:02:01.485 INFO:teuthology.orchestra.run.gibba023.stdout: debug mon = 20 2021-07-11T04:02:01.485 INFO:teuthology.orchestra.run.gibba023.stdout: debug paxos = 20 2021-07-11T04:02:01.486 INFO:teuthology.orchestra.run.gibba023.stdout: debug auth = 20 2021-07-11T04:02:01.486 INFO:teuthology.orchestra.run.gibba023.stdout: mon data avail warn = 5 2021-07-11T04:02:01.486 INFO:teuthology.orchestra.run.gibba023.stdout: mon mgr mkfs grace = 240 2021-07-11T04:02:01.486 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:01.487 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd reporter subtree level = osd 2021-07-11T04:02:01.487 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd prime pg temp = true 2021-07-11T04:02:01.487 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:01.487 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.487 INFO:teuthology.orchestra.run.gibba023.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T04:02:01.488 INFO:teuthology.orchestra.run.gibba023.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T04:02:01.488 INFO:teuthology.orchestra.run.gibba023.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T04:02:01.488 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.488 INFO:teuthology.orchestra.run.gibba023.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T04:02:01.489 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T04:02:01.489 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T04:02:01.489 INFO:teuthology.orchestra.run.gibba023.stdout: mon op complaint time = 120 2021-07-11T04:02:01.489 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:01.489 INFO:teuthology.orchestra.run.gibba023.stdout:[client] 2021-07-11T04:02:01.490 INFO:teuthology.orchestra.run.gibba023.stdout: rgw cache enabled = true 2021-07-11T04:02:01.490 INFO:teuthology.orchestra.run.gibba023.stdout: rgw enable ops log = true 2021-07-11T04:02:01.490 INFO:teuthology.orchestra.run.gibba023.stdout: rgw enable usage log = true 2021-07-11T04:02:01.490 INFO:teuthology.orchestra.run.gibba023.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T04:02:01.490 INFO:teuthology.orchestra.run.gibba023.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T04:02:01.491 INFO:teuthology.orchestra.run.gibba023.stdout: client mount timeout = 600 2021-07-11T04:02:01.491 INFO:teuthology.orchestra.run.gibba023.stdout: debug client = 20 2021-07-11T04:02:01.491 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T04:02:01.491 INFO:teuthology.orchestra.run.gibba023.stdout: rados mon op timeout = 15m 2021-07-11T04:02:01.492 INFO:teuthology.orchestra.run.gibba023.stdout: rados osd op timeout = 15m 2021-07-11T04:02:01.492 INFO:teuthology.orchestra.run.gibba023.stdout: client debug getattr caps = true 2021-07-11T04:02:01.492 INFO:teuthology.orchestra.run.gibba023.stdout:[mon.a] 2021-07-11T04:02:01.492 INFO:teuthology.orchestra.run.gibba023.stdout:[mon.b] 2021-07-11T04:02:01.492 INFO:teuthology.orchestra.run.gibba023.stdout:[mon.c] 2021-07-11T04:02:01.493 INFO:teuthology.orchestra.run.gibba023.stdout:[mds.a] 2021-07-11T04:02:01.493 INFO:teuthology.orchestra.run.gibba023.stdout:[mds.c] 2021-07-11T04:02:01.493 INFO:teuthology.orchestra.run.gibba023.stdout:[mds.b] 2021-07-11T04:02:01.493 INFO:teuthology.orchestra.run.gibba023.stdout:[mds] 2021-07-11T04:02:01.494 INFO:teuthology.orchestra.run.gibba023.stdout: debug mds = 20 2021-07-11T04:02:01.494 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T04:02:01.494 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal frag = True 2021-07-11T04:02:01.494 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal fragment size max = 10000 2021-07-11T04:02:01.494 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal merge size = 5 2021-07-11T04:02:01.495 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal split bits = 3 2021-07-11T04:02:01.495 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal split size = 100 2021-07-11T04:02:01.495 INFO:teuthology.orchestra.run.gibba023.stdout: mds debug frag = True 2021-07-11T04:02:01.495 INFO:teuthology.orchestra.run.gibba023.stdout: mds debug scatterstat = True 2021-07-11T04:02:01.496 INFO:teuthology.orchestra.run.gibba023.stdout: mds op complaint time = 180 2021-07-11T04:02:01.496 INFO:teuthology.orchestra.run.gibba023.stdout: mds verify scatter = True 2021-07-11T04:02:01.496 INFO:teuthology.orchestra.run.gibba023.stdout: osd op complaint time = 180 2021-07-11T04:02:01.496 INFO:teuthology.orchestra.run.gibba023.stdout: rados mon op timeout = 15m 2021-07-11T04:02:01.496 INFO:teuthology.orchestra.run.gibba023.stdout: rados osd op timeout = 15m 2021-07-11T04:02:01.497 INFO:teuthology.orchestra.run.gibba005.stdout:[global] 2021-07-11T04:02:01.497 INFO:teuthology.orchestra.run.gibba005.stdout: chdir = "" 2021-07-11T04:02:01.497 INFO:teuthology.orchestra.run.gibba005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T04:02:01.497 INFO:teuthology.orchestra.run.gibba005.stdout: auth supported = cephx 2021-07-11T04:02:01.498 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.498 INFO:teuthology.orchestra.run.gibba005.stdout: filestore xattr use omap = true 2021-07-11T04:02:01.498 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.498 INFO:teuthology.orchestra.run.gibba005.stdout: mon clock drift allowed = 1.000 2021-07-11T04:02:01.499 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.499 INFO:teuthology.orchestra.run.gibba005.stdout: osd crush chooseleaf type = 0 2021-07-11T04:02:01.499 INFO:teuthology.orchestra.run.gibba005.stdout: auth debug = true 2021-07-11T04:02:01.499 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.499 INFO:teuthology.orchestra.run.gibba005.stdout: ms die on old message = true 2021-07-11T04:02:01.500 INFO:teuthology.orchestra.run.gibba005.stdout: ms die on bug = true 2021-07-11T04:02:01.500 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.500 INFO:teuthology.orchestra.run.gibba005.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T04:02:01.500 INFO:teuthology.orchestra.run.gibba005.stdout: mon pg warn max object skew = 0 2021-07-11T04:02:01.501 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.501 INFO:teuthology.orchestra.run.gibba005.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T04:02:01.501 INFO:teuthology.orchestra.run.gibba005.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T04:02:01.501 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.501 INFO:teuthology.orchestra.run.gibba005.stdout: osd pool default size = 2 2021-07-11T04:02:01.502 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.502 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd allow primary affinity = true 2021-07-11T04:02:01.502 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd allow pg remap = true 2021-07-11T04:02:01.502 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on legacy crush tunables = false 2021-07-11T04:02:01.503 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on crush straw calc version zero = false 2021-07-11T04:02:01.503 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on no sortbitwise = false 2021-07-11T04:02:01.503 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on osd down out interval zero = false 2021-07-11T04:02:01.503 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on too few osds = false 2021-07-11T04:02:01.503 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T04:02:01.504 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T04:02:01.504 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.504 INFO:teuthology.orchestra.run.gibba005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T04:02:01.504 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.505 INFO:teuthology.orchestra.run.gibba005.stdout: osd default data pool replay window = 5 2021-07-11T04:02:01.505 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.505 INFO:teuthology.orchestra.run.gibba005.stdout: mon allow pool delete = true 2021-07-11T04:02:01.505 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.505 INFO:teuthology.orchestra.run.gibba005.stdout: mon cluster log file level = debug 2021-07-11T04:02:01.506 INFO:teuthology.orchestra.run.gibba005.stdout: debug asserts on shutdown = true 2021-07-11T04:02:01.506 INFO:teuthology.orchestra.run.gibba005.stdout: mon health detail to clog = false 2021-07-11T04:02:01.506 INFO:teuthology.orchestra.run.gibba005.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T04:02:01.506 INFO:teuthology.orchestra.run.gibba005.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T04:02:01.507 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.507 INFO:teuthology.orchestra.run.gibba005.stdout:[osd] 2021-07-11T04:02:01.507 INFO:teuthology.orchestra.run.gibba005.stdout: osd journal size = 100 2021-07-11T04:02:01.507 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.507 INFO:teuthology.orchestra.run.gibba005.stdout: osd scrub load threshold = 5.0 2021-07-11T04:02:01.508 INFO:teuthology.orchestra.run.gibba005.stdout: osd scrub max interval = 600 2021-07-11T04:02:01.508 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.508 INFO:teuthology.orchestra.run.gibba005.stdout: osd recover clone overlap = true 2021-07-11T04:02:01.508 INFO:teuthology.orchestra.run.gibba005.stdout: osd recovery max chunk = 1048576 2021-07-11T04:02:01.509 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.509 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug shutdown = true 2021-07-11T04:02:01.509 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug op order = true 2021-07-11T04:02:01.509 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug verify stray on activate = true 2021-07-11T04:02:01.509 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.510 INFO:teuthology.orchestra.run.gibba005.stdout: osd open classes on start = true 2021-07-11T04:02:01.510 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug pg log writeout = true 2021-07-11T04:02:01.510 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.510 INFO:teuthology.orchestra.run.gibba005.stdout: osd deep scrub update digest min age = 30 2021-07-11T04:02:01.511 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.511 INFO:teuthology.orchestra.run.gibba005.stdout: osd map max advance = 10 2021-07-11T04:02:01.511 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.511 INFO:teuthology.orchestra.run.gibba005.stdout: journal zero on create = true 2021-07-11T04:02:01.511 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.512 INFO:teuthology.orchestra.run.gibba005.stdout: filestore ondisk finisher threads = 3 2021-07-11T04:02:01.512 INFO:teuthology.orchestra.run.gibba005.stdout: filestore apply finisher threads = 3 2021-07-11T04:02:01.512 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.512 INFO:teuthology.orchestra.run.gibba005.stdout: bdev debug aio = true 2021-07-11T04:02:01.512 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug misdirected ops = true 2021-07-11T04:02:01.513 INFO:teuthology.orchestra.run.gibba005.stdout: bluestore block size = 96636764160 2021-07-11T04:02:01.513 INFO:teuthology.orchestra.run.gibba005.stdout: bluestore compression mode = aggressive 2021-07-11T04:02:01.513 INFO:teuthology.orchestra.run.gibba005.stdout: bluestore fsck on mount = True 2021-07-11T04:02:01.513 INFO:teuthology.orchestra.run.gibba005.stdout: debug bluefs = 20 2021-07-11T04:02:01.514 INFO:teuthology.orchestra.run.gibba005.stdout: debug bluestore = 20 2021-07-11T04:02:01.514 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T04:02:01.514 INFO:teuthology.orchestra.run.gibba005.stdout: debug osd = 20 2021-07-11T04:02:01.514 INFO:teuthology.orchestra.run.gibba005.stdout: debug rocksdb = 10 2021-07-11T04:02:01.514 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T04:02:01.515 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd full ratio = 0.9 2021-07-11T04:02:01.515 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd nearfull ratio = 0.8 2021-07-11T04:02:01.515 INFO:teuthology.orchestra.run.gibba005.stdout: osd failsafe full ratio = 0.95 2021-07-11T04:02:01.515 INFO:teuthology.orchestra.run.gibba005.stdout: osd objectstore = bluestore 2021-07-11T04:02:01.516 INFO:teuthology.orchestra.run.gibba005.stdout: osd op complaint time = 180 2021-07-11T04:02:01.516 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.516 INFO:teuthology.orchestra.run.gibba005.stdout:[mgr] 2021-07-11T04:02:01.516 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T04:02:01.516 INFO:teuthology.orchestra.run.gibba005.stdout: debug mgr = 20 2021-07-11T04:02:01.517 INFO:teuthology.orchestra.run.gibba005.stdout: debug mon = 20 2021-07-11T04:02:01.517 INFO:teuthology.orchestra.run.gibba005.stdout: debug auth = 20 2021-07-11T04:02:01.517 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:01.517 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:01.518 INFO:teuthology.orchestra.run.gibba005.stdout: mgr/telemetry/nag = false 2021-07-11T04:02:01.518 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.518 INFO:teuthology.orchestra.run.gibba005.stdout:[mon] 2021-07-11T04:02:01.518 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T04:02:01.518 INFO:teuthology.orchestra.run.gibba005.stdout: debug mon = 20 2021-07-11T04:02:01.519 INFO:teuthology.orchestra.run.gibba005.stdout: debug paxos = 20 2021-07-11T04:02:01.519 INFO:teuthology.orchestra.run.gibba005.stdout: debug auth = 20 2021-07-11T04:02:01.519 INFO:teuthology.orchestra.run.gibba005.stdout: mon data avail warn = 5 2021-07-11T04:02:01.519 INFO:teuthology.orchestra.run.gibba005.stdout: mon mgr mkfs grace = 240 2021-07-11T04:02:01.520 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:01.520 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd reporter subtree level = osd 2021-07-11T04:02:01.520 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd prime pg temp = true 2021-07-11T04:02:01.520 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:01.520 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.521 INFO:teuthology.orchestra.run.gibba005.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T04:02:01.521 INFO:teuthology.orchestra.run.gibba005.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T04:02:01.521 INFO:teuthology.orchestra.run.gibba005.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T04:02:01.521 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.521 INFO:teuthology.orchestra.run.gibba005.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T04:02:01.522 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T04:02:01.522 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T04:02:01.522 INFO:teuthology.orchestra.run.gibba005.stdout: mon op complaint time = 120 2021-07-11T04:02:01.522 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:01.523 INFO:teuthology.orchestra.run.gibba005.stdout:[client] 2021-07-11T04:02:01.523 INFO:teuthology.orchestra.run.gibba005.stdout: rgw cache enabled = true 2021-07-11T04:02:01.523 INFO:teuthology.orchestra.run.gibba005.stdout: rgw enable ops log = true 2021-07-11T04:02:01.523 INFO:teuthology.orchestra.run.gibba005.stdout: rgw enable usage log = true 2021-07-11T04:02:01.523 INFO:teuthology.orchestra.run.gibba005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T04:02:01.524 INFO:teuthology.orchestra.run.gibba005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T04:02:01.524 INFO:teuthology.orchestra.run.gibba005.stdout: client mount timeout = 600 2021-07-11T04:02:01.524 INFO:teuthology.orchestra.run.gibba005.stdout: debug client = 20 2021-07-11T04:02:01.524 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T04:02:01.525 INFO:teuthology.orchestra.run.gibba005.stdout: rados mon op timeout = 15m 2021-07-11T04:02:01.525 INFO:teuthology.orchestra.run.gibba005.stdout: rados osd op timeout = 15m 2021-07-11T04:02:01.525 INFO:teuthology.orchestra.run.gibba005.stdout: client debug getattr caps = true 2021-07-11T04:02:01.525 INFO:teuthology.orchestra.run.gibba005.stdout:[mon.a] 2021-07-11T04:02:01.525 INFO:teuthology.orchestra.run.gibba005.stdout:[mon.b] 2021-07-11T04:02:01.526 INFO:teuthology.orchestra.run.gibba005.stdout:[mon.c] 2021-07-11T04:02:01.526 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.a] 2021-07-11T04:02:01.526 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.c] 2021-07-11T04:02:01.526 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.b] 2021-07-11T04:02:01.527 INFO:teuthology.orchestra.run.gibba005.stdout:[mds] 2021-07-11T04:02:01.527 INFO:teuthology.orchestra.run.gibba005.stdout: debug mds = 20 2021-07-11T04:02:01.527 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T04:02:01.527 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal frag = True 2021-07-11T04:02:01.527 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal fragment size max = 10000 2021-07-11T04:02:01.528 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal merge size = 5 2021-07-11T04:02:01.528 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal split bits = 3 2021-07-11T04:02:01.528 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal split size = 100 2021-07-11T04:02:01.528 INFO:teuthology.orchestra.run.gibba005.stdout: mds debug frag = True 2021-07-11T04:02:01.529 INFO:teuthology.orchestra.run.gibba005.stdout: mds debug scatterstat = True 2021-07-11T04:02:01.529 INFO:teuthology.orchestra.run.gibba005.stdout: mds op complaint time = 180 2021-07-11T04:02:01.529 INFO:teuthology.orchestra.run.gibba005.stdout: mds verify scatter = True 2021-07-11T04:02:01.529 INFO:teuthology.orchestra.run.gibba005.stdout: osd op complaint time = 180 2021-07-11T04:02:01.529 INFO:teuthology.orchestra.run.gibba005.stdout: rados mon op timeout = 15m 2021-07-11T04:02:01.530 INFO:teuthology.orchestra.run.gibba005.stdout: rados osd op timeout = 15m 2021-07-11T04:02:01.530 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:01.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f3631d6e700 1 Processor -- start 2021-07-11T04:02:01.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f3631d6e700 1 -- start start 2021-07-11T04:02:01.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f3631d6e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f362c0b5820 0x7f362c0677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f3631d6e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0b5c00 con 0x7f362c145140 2021-07-11T04:02:01.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f3631d6e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c0b00b0 con 0x7f362c0a8d60 2021-07-11T04:02:01.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f3631d6e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362c0b0350 con 0x7f362c0b5820 2021-07-11T04:02:01.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f362b7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f362c0b5820 0x7f362c0677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:01.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f362b7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f362c0b5820 0x7f362c0677f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50582/0 (socket says 172.21.2.104:50582) 2021-07-11T04:02:01.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f362b7fe700 1 -- 172.21.2.104:0/4265968141 learned_addr learned my addr 172.21.2.104:0/4265968141 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:01.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f362b7fe700 1 -- 172.21.2.104:0/4265968141 >> v1:172.21.2.105:6789/0 conn(0x7f362c0a8d60 legacy=0x7f362c144a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:01.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f362b7fe700 1 -- 172.21.2.104:0/4265968141 >> v1:172.21.2.104:6789/0 conn(0x7f362c145140 legacy=0x7f362c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.588+0000 7f362b7fe700 1 -- 172.21.2.104:0/4265968141 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0b05f0 con 0x7f362c0b5820 2021-07-11T04:02:01.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362b7fe700 1 --2- 172.21.2.104:0/4265968141 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0b5820 0x7f362c0677f0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f362c05d9a0 tx=0x7f361c006eb0).ready entity=mon.0 client_cookie=113c1a99bcfcd1d9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:01.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362a7fc700 1 -- 172.21.2.104:0/4265968141 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361c00d610 con 0x7f362c0b5820 2021-07-11T04:02:01.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362a7fc700 1 -- 172.21.2.104:0/4265968141 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c000e90 con 0x7f362c0b5820 2021-07-11T04:02:01.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362a7fc700 1 -- 172.21.2.104:0/4265968141 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361c00d960 con 0x7f362c0b5820 2021-07-11T04:02:01.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- 172.21.2.104:0/4265968141 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0b5820 msgr2=0x7f362c0677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 --2- 172.21.2.104:0/4265968141 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0b5820 0x7f362c0677f0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f362c05d9a0 tx=0x7f361c006eb0).stop 2021-07-11T04:02:01.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- 172.21.2.104:0/4265968141 shutdown_connections 2021-07-11T04:02:01.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 --2- 172.21.2.104:0/4265968141 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0b5820 0x7f362c0677f0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:01.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- 172.21.2.104:0/4265968141 >> 172.21.2.104:0/4265968141 conn(0x7f362c05bd40 msgr2=0x7f362c05c150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:01.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- 172.21.2.104:0/4265968141 shutdown_connections 2021-07-11T04:02:01.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- 172.21.2.104:0/4265968141 wait complete. 2021-07-11T04:02:01.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 Processor -- start 2021-07-11T04:02:01.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- start start 2021-07-11T04:02:01.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0a8d60 0x7f362c066f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f362c0b5820 0x7f362c1426c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f362c145140 0x7f362c142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c156b40 con 0x7f362c0a8d60 2021-07-11T04:02:01.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1462d0 con 0x7f362c0b5820 2021-07-11T04:02:01.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c064460 con 0x7f362c145140 2021-07-11T04:02:01.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0a8d60 0x7f362c066f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:01.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0a8d60 0x7f362c066f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50588/0 (socket says 172.21.2.104:50588) 2021-07-11T04:02:01.601 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362b7fe700 1 -- 172.21.2.104:0/1654841177 learned_addr learned my addr 172.21.2.104:0/1654841177 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:01.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362b7fe700 1 -- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f362c145140 msgr2=0x7f362c142bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362b7fe700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f362c145140 0x7f362c142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362b7fe700 1 -- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f362c0b5820 msgr2=0x7f362c1426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362b7fe700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f362c0b5820 0x7f362c1426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362b7fe700 1 -- 172.21.2.104:0/1654841177 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c00b1e0 con 0x7f362c0a8d60 2021-07-11T04:02:01.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f362b7fe700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0a8d60 0x7f362c066f50 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f362c0ab9d0 tx=0x7f361c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:01.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3628ff9700 1 -- 172.21.2.104:0/1654841177 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361c021070 con 0x7f362c0a8d60 2021-07-11T04:02:01.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3628ff9700 1 -- 172.21.2.104:0/1654841177 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c000e90 con 0x7f362c0a8d60 2021-07-11T04:02:01.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3628ff9700 1 -- 172.21.2.104:0/1654841177 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f361c002f50 con 0x7f362c0a8d60 2021-07-11T04:02:01.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- 172.21.2.104:0/1654841177 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c0b1010 con 0x7f362c0a8d60 2021-07-11T04:02:01.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- 172.21.2.104:0/1654841177 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c0b1550 con 0x7f362c0a8d60 2021-07-11T04:02:01.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3631d6e700 1 -- 172.21.2.104:0/1654841177 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c0b1710 con 0x7f362c0a8d60 2021-07-11T04:02:01.604 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.592+0000 7f3628ff9700 1 -- 172.21.2.104:0/1654841177 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f361c00f070 con 0x7f362c0a8d60 2021-07-11T04:02:01.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.596+0000 7f3628ff9700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3610053180 0x7f3610055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.596+0000 7f3628ff9700 1 -- 172.21.2.104:0/1654841177 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f361c06ca10 con 0x7f362c0a8d60 2021-07-11T04:02:01.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.596+0000 7f362affd700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3610053180 0x7f3610055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:01.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.596+0000 7f362affd700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3610053180 0x7f3610055640 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f3614002f90 tx=0x7f36140069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:01.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.596+0000 7f3628ff9700 1 -- 172.21.2.104:0/1654841177 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f361c043c40 con 0x7f362c0a8d60 2021-07-11T04:02:01.676 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.676+0000 7f3631d6e700 1 -- 172.21.2.104:0/1654841177 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f362c057350 con 0x7f362c0a8d60 2021-07-11T04:02:01.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.676+0000 7f3628ff9700 1 -- 172.21.2.104:0/1654841177 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f361c047260 con 0x7f362c0a8d60 2021-07-11T04:02:01.677 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:01.678 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":139,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:01:53.880137+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:01:50.686994+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2021-07-11T04:01:51.853293+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:01.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 -- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3610053180 msgr2=0x7f3610055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3610053180 0x7f3610055640 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f3614002f90 tx=0x7f36140069b0).stop 2021-07-11T04:02:01.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 -- 172.21.2.104:0/1654841177 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0a8d60 msgr2=0x7f362c066f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0a8d60 0x7f362c066f50 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f362c0ab9d0 tx=0x7f361c015000).stop 2021-07-11T04:02:01.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 -- 172.21.2.104:0/1654841177 shutdown_connections 2021-07-11T04:02:01.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f362c145140 0x7f362c142bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f362c0b5820 0x7f362c1426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3610053180 0x7f3610055640 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:01.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 --2- 172.21.2.104:0/1654841177 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f362c0a8d60 0x7f362c066f50 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:01.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 -- 172.21.2.104:0/1654841177 >> 172.21.2.104:0/1654841177 conn(0x7f362c05bd40 msgr2=0x7f362c0ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:01.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 -- 172.21.2.104:0/1654841177 shutdown_connections 2021-07-11T04:02:01.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.680+0000 7f3631d6e700 1 -- 172.21.2.104:0/1654841177 wait complete. 2021-07-11T04:02:01.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:01:50.686994+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:01:51.853293+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:02:01.686 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:02:01.739 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f02007b5700 1 Processor -- start 2021-07-11T04:02:01.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f02007b5700 1 -- start start 2021-07-11T04:02:01.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f02007b5700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f01f80b1950 0x7f01f80a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f02007b5700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f814ac10 con 0x7f01f81475a0 2021-07-11T04:02:01.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f02007b5700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f80b2940 con 0x7f01f80b2560 2021-07-11T04:02:01.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f02007b5700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f80a8cc0 con 0x7f01f80b1950 2021-07-11T04:02:01.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fed52700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f01f81475a0 0x7f01f80ab7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47368/0 (socket says 172.21.2.104:47368) 2021-07-11T04:02:01.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fed52700 1 -- 172.21.2.104:0/1119473031 learned_addr learned my addr 172.21.2.104:0/1119473031 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:01.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828962993 0 0) 0x7f01f814ac10 con 0x7f01f81475a0 2021-07-11T04:02:01.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e0003260 con 0x7f01f81475a0 2021-07-11T04:02:01.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fdd50700 1 --2- 172.21.2.104:0/1119473031 >> v2:172.21.2.105:3300/0 conn(0x7f01f80b1950 0x7f01f80a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:01.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 74970168 0 0) 0x7f01e0003260 con 0x7f01f81475a0 2021-07-11T04:02:01.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 >> v2:172.21.2.105:3300/0 conn(0x7f01f80b1950 msgr2=0x7f01f80a7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:01.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 --2- 172.21.2.104:0/1119473031 >> v2:172.21.2.105:3300/0 conn(0x7f01f80b1950 0x7f01f80a7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:01.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 >> v1:172.21.2.105:6789/0 conn(0x7f01f80b2560 legacy=0x7f01f814b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:01.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f80b0910 con 0x7f01f81475a0 2021-07-11T04:02:01.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f01f00019e0 con 0x7f01f81475a0 2021-07-11T04:02:01.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 >> v1:172.21.2.104:6789/0 conn(0x7f01f81475a0 legacy=0x7f01f80ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 --2- 172.21.2.104:0/1119473031 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01e0003820 0x7f01e0003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 --2- 172.21.2.104:0/1119473031 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f01e0004500 0x7f01e0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 --2- 172.21.2.104:0/1119473031 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 0x7f01e000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f80a8cc0 con 0x7f01e0005050 2021-07-11T04:02:01.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e000bf40 con 0x7f01e0004500 2021-07-11T04:02:01.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e000c100 con 0x7f01e0003820 2021-07-11T04:02:01.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fed52700 1 --2- 172.21.2.104:0/1119473031 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 0x7f01e000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:01.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fed52700 1 -- 172.21.2.104:0/1119473031 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01e0003820 msgr2=0x7f01e0003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:01.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fed52700 1 --2- 172.21.2.104:0/1119473031 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01e0003820 0x7f01e0003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fed52700 1 -- 172.21.2.104:0/1119473031 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f01e0004500 msgr2=0x7f01e0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:01.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fed52700 1 --2- 172.21.2.104:0/1119473031 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f01e0004500 0x7f01e0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fed52700 1 -- 172.21.2.104:0/1119473031 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0003260 con 0x7f01e0005050 2021-07-11T04:02:01.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fed52700 1 -- 172.21.2.104:0/1119473031 reap_dead start 2021-07-11T04:02:01.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fed52700 1 --2- 172.21.2.104:0/1119473031 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 0x7f01e000b750 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f01f80afe40 tx=0x7f01f80b0aa0).ready entity=mon.0 client_cookie=4c5f11d0b99a2cb4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:01.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01f000cc60 con 0x7f01e0005050 2021-07-11T04:02:01.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f000cdc0 con 0x7f01e0005050 2021-07-11T04:02:01.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f01fd54f700 1 -- 172.21.2.104:0/1119473031 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01f0005b80 con 0x7f01e0005050 2021-07-11T04:02:01.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f02007b5700 1 -- 172.21.2.104:0/1119473031 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 msgr2=0x7f01e000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.740+0000 7f02007b5700 1 --2- 172.21.2.104:0/1119473031 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 0x7f01e000b750 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f01f80afe40 tx=0x7f01f80b0aa0).stop 2021-07-11T04:02:01.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 -- 172.21.2.104:0/1119473031 shutdown_connections 2021-07-11T04:02:01.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 --2- 172.21.2.104:0/1119473031 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 0x7f01e000b750 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:01.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 -- 172.21.2.104:0/1119473031 >> 172.21.2.104:0/1119473031 conn(0x7f01f805ca20 msgr2=0x7f01f805ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:01.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 -- 172.21.2.104:0/1119473031 shutdown_connections 2021-07-11T04:02:01.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 -- 172.21.2.104:0/1119473031 wait complete. 2021-07-11T04:02:01.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 Processor -- start 2021-07-11T04:02:01.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 -- start start 2021-07-11T04:02:01.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 0x7f01f813e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f01f8146f70 0x7f01f80a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f80a9dd0 0x7f01f8145440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f80aa250 con 0x7f01e0005050 2021-07-11T04:02:01.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f814d2e0 con 0x7f01f8146f70 2021-07-11T04:02:01.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f814c5f0 con 0x7f01f80a9dd0 2021-07-11T04:02:01.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01fe551700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 0x7f01f813e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:01.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01fe551700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 0x7f01f813e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50608/0 (socket says 172.21.2.104:50608) 2021-07-11T04:02:01.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01fe551700 1 -- 172.21.2.104:0/1072407832 learned_addr learned my addr 172.21.2.104:0/1072407832 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:01.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01fe551700 1 -- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f80a9dd0 msgr2=0x7f01f8145440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01fe551700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f80a9dd0 0x7f01f8145440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01fe551700 1 -- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f01f8146f70 msgr2=0x7f01f80a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01fe551700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f01f8146f70 0x7f01f80a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01fe551700 1 -- 172.21.2.104:0/1072407832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f814c350 con 0x7f01e0005050 2021-07-11T04:02:01.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01fe551700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 0x7f01f813e440 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f01f81469a0 tx=0x7f01f4007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:01.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01e77fe700 1 -- 172.21.2.104:0/1072407832 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01f400ce70 con 0x7f01e0005050 2021-07-11T04:02:01.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01e77fe700 1 -- 172.21.2.104:0/1072407832 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f4000e90 con 0x7f01e0005050 2021-07-11T04:02:01.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 -- 172.21.2.104:0/1072407832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f814d540 con 0x7f01e0005050 2021-07-11T04:02:01.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 -- 172.21.2.104:0/1072407832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f813ed60 con 0x7f01e0005050 2021-07-11T04:02:01.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01e77fe700 1 -- 172.21.2.104:0/1072407832 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f01f4007430 con 0x7f01e0005050 2021-07-11T04:02:01.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f02007b5700 1 -- 172.21.2.104:0/1072407832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f8057350 con 0x7f01e0005050 2021-07-11T04:02:01.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01e77fe700 1 -- 172.21.2.104:0/1072407832 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f01f401a060 con 0x7f01e0005050 2021-07-11T04:02:01.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01e77fe700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d8053270 0x7f01d8055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.744+0000 7f01e77fe700 1 -- 172.21.2.104:0/1072407832 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f01f406adc0 con 0x7f01e0005050 2021-07-11T04:02:01.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.748+0000 7f01fdd50700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d8053270 0x7f01d8055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:01.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.748+0000 7f01fdd50700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d8053270 0x7f01d8055730 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f01e8007000 tx=0x7f01e8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:01.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.748+0000 7f01e77fe700 1 -- 172.21.2.104:0/1072407832 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f01f4041ff0 con 0x7f01e0005050 2021-07-11T04:02:01.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 -- 172.21.2.104:0/1072407832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f01f814d740 con 0x7f01e0005050 2021-07-11T04:02:01.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f01e77fe700 1 -- 172.21.2.104:0/1072407832 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v139) v1 ==== 110+0+10 (secure 0 0 0) 0x7f01f4018020 con 0x7f01e0005050 2021-07-11T04:02:01.831 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:02:01.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 -- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d8053270 msgr2=0x7f01d8055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d8053270 0x7f01d8055730 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f01e8007000 tx=0x7f01e8008040).stop 2021-07-11T04:02:01.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 -- 172.21.2.104:0/1072407832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 msgr2=0x7f01f813e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 0x7f01f813e440 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7f01f81469a0 tx=0x7f01f4007c30).stop 2021-07-11T04:02:01.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 -- 172.21.2.104:0/1072407832 shutdown_connections 2021-07-11T04:02:01.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f01f80a9dd0 0x7f01f8145440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f01f8146f70 0x7f01f80a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f01d8053270 0x7f01d8055730 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:01.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 --2- 172.21.2.104:0/1072407832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f01e0005050 0x7f01f813e440 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:01.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 -- 172.21.2.104:0/1072407832 >> 172.21.2.104:0/1072407832 conn(0x7f01f805ca20 msgr2=0x7f01f805ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:01.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 -- 172.21.2.104:0/1072407832 shutdown_connections 2021-07-11T04:02:01.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.832+0000 7f02007b5700 1 -- 172.21.2.104:0/1072407832 wait complete. 2021-07-11T04:02:01.838 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:02:01.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 Processor -- start 2021-07-11T04:02:01.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 -- start start 2021-07-11T04:02:01.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc99c0b2560 0x7fc99c0a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c14ac10 con 0x7fc99c1475a0 2021-07-11T04:02:01.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc99c0b2940 con 0x7fc99c0b1950 2021-07-11T04:02:01.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc99c0a8cc0 con 0x7fc99c0b2560 2021-07-11T04:02:01.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a0e4a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc99c0b2560 0x7fc99c0a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:01.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a0e4a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc99c0b2560 0x7fc99c0a7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50616/0 (socket says 172.21.2.104:50616) 2021-07-11T04:02:01.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a164b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc99c1475a0 0x7fc99c0ab7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47384/0 (socket says 172.21.2.104:47384) 2021-07-11T04:02:01.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a0e4a700 1 -- 172.21.2.104:0/2664374075 learned_addr learned my addr 172.21.2.104:0/2664374075 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:01.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a0e4a700 1 -- 172.21.2.104:0/2664374075 >> v1:172.21.2.105:6789/0 conn(0x7fc99c0b1950 legacy=0x7fc99c14b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:01.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a0e4a700 1 -- 172.21.2.104:0/2664374075 >> v1:172.21.2.104:6789/0 conn(0x7fc99c1475a0 legacy=0x7fc99c0ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a0e4a700 1 -- 172.21.2.104:0/2664374075 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c0b0910 con 0x7fc99c0b2560 2021-07-11T04:02:01.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a0e4a700 1 --2- 172.21.2.104:0/2664374075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc99c0b2560 0x7fc99c0a7d00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7fc99c0afe40 tx=0x7fc984006eb0).ready entity=mon.0 client_cookie=411730e52eba9a8f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:01.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc99b7fe700 1 -- 172.21.2.104:0/2664374075 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc98400d610 con 0x7fc99c0b2560 2021-07-11T04:02:01.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc99b7fe700 1 -- 172.21.2.104:0/2664374075 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc984000e90 con 0x7fc99c0b2560 2021-07-11T04:02:01.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc99b7fe700 1 -- 172.21.2.104:0/2664374075 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc98400d960 con 0x7fc99c0b2560 2021-07-11T04:02:01.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 -- 172.21.2.104:0/2664374075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc99c0b2560 msgr2=0x7fc99c0a7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 --2- 172.21.2.104:0/2664374075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc99c0b2560 0x7fc99c0a7d00 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7fc99c0afe40 tx=0x7fc984006eb0).stop 2021-07-11T04:02:01.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 -- 172.21.2.104:0/2664374075 shutdown_connections 2021-07-11T04:02:01.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 --2- 172.21.2.104:0/2664374075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc99c0b2560 0x7fc99c0a7d00 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:01.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 -- 172.21.2.104:0/2664374075 >> 172.21.2.104:0/2664374075 conn(0x7fc99c05ca20 msgr2=0x7fc99c05ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:01.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 -- 172.21.2.104:0/2664374075 shutdown_connections 2021-07-11T04:02:01.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.892+0000 7fc9a30ae700 1 -- 172.21.2.104:0/2664374075 wait complete. 2021-07-11T04:02:01.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a30ae700 1 Processor -- start 2021-07-11T04:02:01.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a30ae700 1 -- start start 2021-07-11T04:02:01.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a30ae700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc99c0b1950 0x7fc99c147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a30ae700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc99c0b2560 0x7fc99c0a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a30ae700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc99c1475a0 0x7fc99c0a9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a30ae700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0aac70 con 0x7fc99c0b2560 2021-07-11T04:02:01.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a30ae700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0ab7f0 con 0x7fc99c1475a0 2021-07-11T04:02:01.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a30ae700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc99c0a9f40 con 0x7fc99c0b1950 2021-07-11T04:02:01.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc99bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc99c0b2560 0x7fc99c0a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:01.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc99bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc99c0b2560 0x7fc99c0a9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50622/0 (socket says 172.21.2.104:50622) 2021-07-11T04:02:01.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc99bfff700 1 -- 172.21.2.104:0/3983231465 learned_addr learned my addr 172.21.2.104:0/3983231465 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:01.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc99bfff700 1 -- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc99c0b1950 msgr2=0x7fc99c147d10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:01.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc99bfff700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc99c0b1950 0x7fc99c147d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc99bfff700 1 -- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc99c1475a0 msgr2=0x7fc99c0a9a50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:01.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc99bfff700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc99c1475a0 0x7fc99c0a9a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc99bfff700 1 -- 172.21.2.104:0/3983231465 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98400b1e0 con 0x7fc99c0b2560 2021-07-11T04:02:01.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc99bfff700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc99c0b2560 0x7fc99c0a9560 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fc98c002f90 tx=0x7fc98c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:01.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc999ffb700 1 -- 172.21.2.104:0/3983231465 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc98c013460 con 0x7fc99c0b2560 2021-07-11T04:02:01.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc999ffb700 1 -- 172.21.2.104:0/3983231465 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98c0135c0 con 0x7fc99c0b2560 2021-07-11T04:02:01.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a30ae700 1 -- 172.21.2.104:0/3983231465 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc99c0b42c0 con 0x7fc99c0b2560 2021-07-11T04:02:01.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc999ffb700 1 -- 172.21.2.104:0/3983231465 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc98c013920 con 0x7fc99c0b2560 2021-07-11T04:02:01.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a30ae700 1 -- 172.21.2.104:0/3983231465 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc99c0abcc0 con 0x7fc99c0b2560 2021-07-11T04:02:01.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a30ae700 1 -- 172.21.2.104:0/3983231465 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99c0abe80 con 0x7fc99c0b2560 2021-07-11T04:02:01.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc999ffb700 1 -- 172.21.2.104:0/3983231465 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc98c01a060 con 0x7fc99c0b2560 2021-07-11T04:02:01.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc999ffb700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9800531d0 0x7fc980055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:01.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc999ffb700 1 -- 172.21.2.104:0/3983231465 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc98c06ada0 con 0x7fc99c0b2560 2021-07-11T04:02:01.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.896+0000 7fc9a0e4a700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9800531d0 0x7fc980055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:01.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.900+0000 7fc9a0e4a700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9800531d0 0x7fc980055690 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fc99c0afe40 tx=0x7fc984015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:01.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.900+0000 7fc999ffb700 1 -- 172.21.2.104:0/3983231465 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc98c0420e0 con 0x7fc99c0b2560 2021-07-11T04:02:01.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 -- 172.21.2.104:0/3983231465 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc99c057350 con 0x7fc99c0b2560 2021-07-11T04:02:01.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc999ffb700 1 -- 172.21.2.104:0/3983231465 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v139) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc98c018020 con 0x7fc99c0b2560 2021-07-11T04:02:01.980 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:01.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 -- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9800531d0 msgr2=0x7fc980055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9800531d0 0x7fc980055690 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fc99c0afe40 tx=0x7fc984015000).stop 2021-07-11T04:02:01.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 -- 172.21.2.104:0/3983231465 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc99c0b2560 msgr2=0x7fc99c0a9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:01.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc99c0b2560 0x7fc99c0a9560 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fc98c002f90 tx=0x7fc98c006df0).stop 2021-07-11T04:02:01.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 -- 172.21.2.104:0/3983231465 shutdown_connections 2021-07-11T04:02:01.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc99c1475a0 0x7fc99c0a9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc9800531d0 0x7fc980055690 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:01.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc99c0b2560 0x7fc99c0a9560 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:01.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 --2- 172.21.2.104:0/3983231465 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc99c0b1950 0x7fc99c147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:01.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 -- 172.21.2.104:0/3983231465 >> 172.21.2.104:0/3983231465 conn(0x7fc99c05ca20 msgr2=0x7fc99c145440 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:01.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 -- 172.21.2.104:0/3983231465 shutdown_connections 2021-07-11T04:02:01.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:01.980+0000 7fc9a30ae700 1 -- 172.21.2.104:0/3983231465 wait complete. 2021-07-11T04:02:01.987 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:02:02.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 Processor -- start 2021-07-11T04:02:02.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 -- start start 2021-07-11T04:02:02.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6e140b1950 0x7f6e140a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6e141475a0 0x7f6e140ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1414ac10 con 0x7f6e140b2560 2021-07-11T04:02:02.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e140a8cc0 con 0x7f6e141475a0 2021-07-11T04:02:02.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e14147a10 con 0x7f6e140b1950 2021-07-11T04:02:02.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e197c9700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6e141475a0 0x7f6e140ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:02.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e197c9700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6e141475a0 0x7f6e140ab7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50634/0 (socket says 172.21.2.104:50634) 2021-07-11T04:02:02.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e197c9700 1 -- 172.21.2.104:0/2184102742 learned_addr learned my addr 172.21.2.104:0/2184102742 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:02.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e197c9700 1 -- 172.21.2.104:0/2184102742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6e140b1950 msgr2=0x7f6e140a7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:02.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e197c9700 1 --2- 172.21.2.104:0/2184102742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6e140b1950 0x7f6e140a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e197c9700 1 -- 172.21.2.104:0/2184102742 >> v1:172.21.2.105:6789/0 conn(0x7f6e140b2560 legacy=0x7f6e1414b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:02.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e197c9700 1 -- 172.21.2.104:0/2184102742 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e140b2940 con 0x7f6e141475a0 2021-07-11T04:02:02.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e197c9700 1 --2- 172.21.2.104:0/2184102742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6e141475a0 0x7f6e140ab7b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7f6e1414b1c0 tx=0x7f6e0c009290).ready entity=mon.0 client_cookie=68e04c6417eef4ba server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:02.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e137fe700 1 -- 172.21.2.104:0/2184102742 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e0c00b860 con 0x7f6e141475a0 2021-07-11T04:02:02.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e137fe700 1 -- 172.21.2.104:0/2184102742 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c00b9c0 con 0x7f6e141475a0 2021-07-11T04:02:02.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e137fe700 1 -- 172.21.2.104:0/2184102742 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e0c00bb70 con 0x7f6e141475a0 2021-07-11T04:02:02.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 -- 172.21.2.104:0/2184102742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6e141475a0 msgr2=0x7f6e140ab7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 --2- 172.21.2.104:0/2184102742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6e141475a0 0x7f6e140ab7b0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7f6e1414b1c0 tx=0x7f6e0c009290).stop 2021-07-11T04:02:02.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 -- 172.21.2.104:0/2184102742 shutdown_connections 2021-07-11T04:02:02.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 --2- 172.21.2.104:0/2184102742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6e141475a0 0x7f6e140ab7b0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 --2- 172.21.2.104:0/2184102742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6e140b1950 0x7f6e140a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 -- 172.21.2.104:0/2184102742 >> 172.21.2.104:0/2184102742 conn(0x7f6e1405ca20 msgr2=0x7f6e1405ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:02.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 -- 172.21.2.104:0/2184102742 shutdown_connections 2021-07-11T04:02:02.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.040+0000 7f6e1b22c700 1 -- 172.21.2.104:0/2184102742 wait complete. 2021-07-11T04:02:02.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e1b22c700 1 Processor -- start 2021-07-11T04:02:02.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e1b22c700 1 -- start start 2021-07-11T04:02:02.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e1b22c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6e140b1950 0x7f6e14143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e1b22c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6e140b2560 0x7f6e1415a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e1b22c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6e141475a0 0x7f6e1415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e1b22c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e14159d50 con 0x7f6e141475a0 2021-07-11T04:02:02.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e1b22c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140aa910 con 0x7f6e140b1950 2021-07-11T04:02:02.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e1b22c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140aabb0 con 0x7f6e140b2560 2021-07-11T04:02:02.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e197c9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6e141475a0 0x7f6e1415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:02.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e197c9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6e141475a0 0x7f6e1415a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50636/0 (socket says 172.21.2.104:50636) 2021-07-11T04:02:02.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e197c9700 1 -- 172.21.2.104:0/1116503182 learned_addr learned my addr 172.21.2.104:0/1116503182 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:02.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e13fff700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6e140b2560 0x7f6e1415a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:02.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e197c9700 1 -- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6e140b2560 msgr2=0x7f6e1415a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e197c9700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6e140b2560 0x7f6e1415a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e197c9700 1 -- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6e140b1950 msgr2=0x7f6e14143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e18fc8700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6e140b1950 0x7f6e14143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:02.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e197c9700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6e140b1950 0x7f6e14143510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e197c9700 1 -- 172.21.2.104:0/1116503182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c009530 con 0x7f6e141475a0 2021-07-11T04:02:02.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e197c9700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6e141475a0 0x7f6e1415a670 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f6e1414b1c0 tx=0x7f6e0c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:02.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e11ffb700 1 -- 172.21.2.104:0/1116503182 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e0c011c80 con 0x7f6e141475a0 2021-07-11T04:02:02.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e11ffb700 1 -- 172.21.2.104:0/1116503182 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0c011de0 con 0x7f6e141475a0 2021-07-11T04:02:02.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e11ffb700 1 -- 172.21.2.104:0/1116503182 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6e0c01f610 con 0x7f6e141475a0 2021-07-11T04:02:02.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e1b22c700 1 -- 172.21.2.104:0/1116503182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e140aae50 con 0x7f6e141475a0 2021-07-11T04:02:02.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e1b22c700 1 -- 172.21.2.104:0/1116503182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e14156780 con 0x7f6e141475a0 2021-07-11T04:02:02.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e1b22c700 1 -- 172.21.2.104:0/1116503182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e14057350 con 0x7f6e141475a0 2021-07-11T04:02:02.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e11ffb700 1 -- 172.21.2.104:0/1116503182 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6e0c00f070 con 0x7f6e141475a0 2021-07-11T04:02:02.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e11ffb700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6df80531f0 0x7f6df80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e11ffb700 1 -- 172.21.2.104:0/1116503182 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6e0c06bfd0 con 0x7f6e141475a0 2021-07-11T04:02:02.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.044+0000 7f6e18fc8700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6df80531f0 0x7f6df80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:02.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.048+0000 7f6e18fc8700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6df80531f0 0x7f6df80556b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f6e04008000 tx=0x7f6e04009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:02.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.048+0000 7f6e11ffb700 1 -- 172.21.2.104:0/1116503182 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6e0c0433c0 con 0x7f6e141475a0 2021-07-11T04:02:02.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.128+0000 7f6e1b22c700 1 -- 172.21.2.104:0/1116503182 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6e14069a00 con 0x7f6e141475a0 2021-07-11T04:02:02.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e11ffb700 1 -- 172.21.2.104:0/1116503182 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6e0c0469e0 con 0x7f6e141475a0 2021-07-11T04:02:02.131 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:02.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 -- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6df80531f0 msgr2=0x7f6df80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6df80531f0 0x7f6df80556b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f6e04008000 tx=0x7f6e04009040).stop 2021-07-11T04:02:02.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 -- 172.21.2.104:0/1116503182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6e141475a0 msgr2=0x7f6e1415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6e141475a0 0x7f6e1415a670 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f6e1414b1c0 tx=0x7f6e0c016000).stop 2021-07-11T04:02:02.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 -- 172.21.2.104:0/1116503182 shutdown_connections 2021-07-11T04:02:02.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6df80531f0 0x7f6df80556b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6e141475a0 0x7f6e1415a670 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6e140b2560 0x7f6e1415a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 --2- 172.21.2.104:0/1116503182 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6e140b1950 0x7f6e14143510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 -- 172.21.2.104:0/1116503182 >> 172.21.2.104:0/1116503182 conn(0x7f6e1405ca20 msgr2=0x7f6e14144610 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:02.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 -- 172.21.2.104:0/1116503182 shutdown_connections 2021-07-11T04:02:02.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.132+0000 7f6e1b22c700 1 -- 172.21.2.104:0/1116503182 wait complete. 2021-07-11T04:02:02.138 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:02:02.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5f3f8700 1 Processor -- start 2021-07-11T04:02:02.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5f3f8700 1 -- start start 2021-07-11T04:02:02.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5f3f8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813c960 0x7fbd580b0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5f3f8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fbd580b2fb0 0x7fbd580a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5f3f8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbd5813fbb0 0x7fbd580b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5f3f8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd580b0810 con 0x7fbd5813fbb0 2021-07-11T04:02:02.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5f3f8700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd580a9190 con 0x7fbd580b2fb0 2021-07-11T04:02:02.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5f3f8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580a9420 con 0x7fbd5813c960 2021-07-11T04:02:02.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5d995700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbd5813fbb0 0x7fbd580b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:02.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5d995700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbd5813fbb0 0x7fbd580b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50646/0 (socket says 172.21.2.104:50646) 2021-07-11T04:02:02.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5d995700 1 -- 172.21.2.104:0/1808927457 learned_addr learned my addr 172.21.2.104:0/1808927457 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:02.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5d995700 1 -- 172.21.2.104:0/1808927457 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813c960 msgr2=0x7fbd580b0320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:02.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5d995700 1 --2- 172.21.2.104:0/1808927457 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813c960 0x7fbd580b0320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5d995700 1 -- 172.21.2.104:0/1808927457 >> v2:172.21.2.105:3300/0 conn(0x7fbd580b2fb0 msgr2=0x7fbd580a8ca0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:02.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5d995700 1 --2- 172.21.2.104:0/1808927457 >> v2:172.21.2.105:3300/0 conn(0x7fbd580b2fb0 0x7fbd580a8ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5d995700 1 -- 172.21.2.104:0/1808927457 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5813f1d0 con 0x7fbd5813fbb0 2021-07-11T04:02:02.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5d995700 1 --2- 172.21.2.104:0/1808927457 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd5813fbb0 0x7fbd580b2370 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7fbd500032f0 tx=0x7fbd50009bb0).ready entity=mon.0 client_cookie=36a557e667c7177b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:02.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd4bfff700 1 -- 172.21.2.104:0/1808927457 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd500094d0 con 0x7fbd5813fbb0 2021-07-11T04:02:02.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd4bfff700 1 -- 172.21.2.104:0/1808927457 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd50009630 con 0x7fbd5813fbb0 2021-07-11T04:02:02.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd4bfff700 1 -- 172.21.2.104:0/1808927457 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd50007950 con 0x7fbd5813fbb0 2021-07-11T04:02:02.200 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/1808927457 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd5813fbb0 msgr2=0x7fbd580b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.192+0000 7fbd5f3f8700 1 --2- 172.21.2.104:0/1808927457 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd5813fbb0 0x7fbd580b2370 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 rx=0x7fbd500032f0 tx=0x7fbd50009bb0).stop 2021-07-11T04:02:02.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/1808927457 shutdown_connections 2021-07-11T04:02:02.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 --2- 172.21.2.104:0/1808927457 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd5813fbb0 0x7fbd580b2370 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.201 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 --2- 172.21.2.104:0/1808927457 >> v2:172.21.2.105:3300/0 conn(0x7fbd580b2fb0 0x7fbd580a8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 --2- 172.21.2.104:0/1808927457 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813c960 0x7fbd580b0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/1808927457 >> 172.21.2.104:0/1808927457 conn(0x7fbd5805f6b0 msgr2=0x7fbd5805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:02.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/1808927457 shutdown_connections 2021-07-11T04:02:02.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/1808927457 wait complete. 2021-07-11T04:02:02.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 Processor -- start 2021-07-11T04:02:02.203 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 -- start start 2021-07-11T04:02:02.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbd580b2fb0 0x7fbd5813b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.205 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813c960 0x7fbd58144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd5813fbb0 0x7fbd58138a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58143400 con 0x7fbd5813fbb0 2021-07-11T04:02:02.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58138340 con 0x7fbd580b2fb0 2021-07-11T04:02:02.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58138570 con 0x7fbd5813c960 2021-07-11T04:02:02.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5d995700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd5813fbb0 0x7fbd58138a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:02.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5d995700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd5813fbb0 0x7fbd58138a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50650/0 (socket says 172.21.2.104:50650) 2021-07-11T04:02:02.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5d995700 1 -- 172.21.2.104:0/3472992506 learned_addr learned my addr 172.21.2.104:0/3472992506 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:02.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5d995700 1 -- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813c960 msgr2=0x7fbd58144de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5d995700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813c960 0x7fbd58144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5d995700 1 -- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbd580b2fb0 msgr2=0x7fbd5813b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5d995700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbd580b2fb0 0x7fbd5813b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5d995700 1 -- 172.21.2.104:0/3472992506 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5000d040 con 0x7fbd5813fbb0 2021-07-11T04:02:02.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5d995700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd5813fbb0 0x7fbd58138a70 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7fbd500032f0 tx=0x7fbd580af0a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:02.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd4a7fc700 1 -- 172.21.2.104:0/3472992506 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd50009870 con 0x7fbd5813fbb0 2021-07-11T04:02:02.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd4a7fc700 1 -- 172.21.2.104:0/3472992506 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5000c6e0 con 0x7fbd5813fbb0 2021-07-11T04:02:02.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/3472992506 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd580b4a60 con 0x7fbd5813fbb0 2021-07-11T04:02:02.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd4a7fc700 1 -- 172.21.2.104:0/3472992506 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd5000ca40 con 0x7fbd5813fbb0 2021-07-11T04:02:02.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/3472992506 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd58143630 con 0x7fbd5813fbb0 2021-07-11T04:02:02.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd4a7fc700 1 -- 172.21.2.104:0/3472992506 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbd50024020 con 0x7fbd5813fbb0 2021-07-11T04:02:02.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/3472992506 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd580b4bf0 con 0x7fbd5813fbb0 2021-07-11T04:02:02.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd4a7fc700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd38053300 0x7fbd380557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.196+0000 7fbd4a7fc700 1 -- 172.21.2.104:0/3472992506 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fbd5006c900 con 0x7fbd5813fbb0 2021-07-11T04:02:02.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.200+0000 7fbd5d194700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd38053300 0x7fbd380557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:02.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.200+0000 7fbd5d194700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd38053300 0x7fbd380557c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fbd54002fb0 tx=0x7fbd540069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:02.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.200+0000 7fbd4a7fc700 1 -- 172.21.2.104:0/3472992506 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbd50043c40 con 0x7fbd5813fbb0 2021-07-11T04:02:02.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/3472992506 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd58057350 con 0x7fbd5813fbb0 2021-07-11T04:02:02.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd4a7fc700 1 -- 172.21.2.104:0/3472992506 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+10 (secure 0 0 0) 0x7fbd50047260 con 0x7fbd5813fbb0 2021-07-11T04:02:02.284 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:02.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd38053300 msgr2=0x7fbd380557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd38053300 0x7fbd380557c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7fbd54002fb0 tx=0x7fbd540069d0).stop 2021-07-11T04:02:02.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/3472992506 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd5813fbb0 msgr2=0x7fbd58138a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd5813fbb0 0x7fbd58138a70 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7fbd500032f0 tx=0x7fbd580af0a0).stop 2021-07-11T04:02:02.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/3472992506 shutdown_connections 2021-07-11T04:02:02.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd38053300 0x7fbd380557c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd5813fbb0 0x7fbd58138a70 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813c960 0x7fbd58144de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 --2- 172.21.2.104:0/3472992506 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbd580b2fb0 0x7fbd5813b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/3472992506 >> 172.21.2.104:0/3472992506 conn(0x7fbd5805f6b0 msgr2=0x7fbd58066e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:02.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/3472992506 shutdown_connections 2021-07-11T04:02:02.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.284+0000 7fbd5f3f8700 1 -- 172.21.2.104:0/3472992506 wait complete. 2021-07-11T04:02:02.291 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:02:02.292 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:02.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77c9e1700 1 Processor -- start 2021-07-11T04:02:02.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77c9e1700 1 -- start start 2021-07-11T04:02:02.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77c9e1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff77805f9f0 0x7ff7780b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77c9e1700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff7780601a0 0x7ff778144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77c9e1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77805fdd0 con 0x7ff77813e5d0 2021-07-11T04:02:02.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77c9e1700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff778065df0 con 0x7ff7780601a0 2021-07-11T04:02:02.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77c9e1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7780ab420 con 0x7ff77805f9f0 2021-07-11T04:02:02.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff776d9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7ff77813e5d0 0x7ff77805bd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47430/0 (socket says 172.21.2.104:47430) 2021-07-11T04:02:02.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff776d9d700 1 -- 172.21.2.104:0/3744759004 learned_addr learned my addr 172.21.2.104:0/3744759004 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:02.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165372652 0 0) 0x7ff77805fdd0 con 0x7ff77813e5d0 2021-07-11T04:02:02.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff75c003260 con 0x7ff77813e5d0 2021-07-11T04:02:02.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4155051642 0 0) 0x7ff75c003260 con 0x7ff77813e5d0 2021-07-11T04:02:02.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff77805f9f0 msgr2=0x7ff7780b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:02.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 --2- 172.21.2.104:0/3744759004 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff77805f9f0 0x7ff7780b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 >> v2:172.21.2.105:3300/0 conn(0x7ff7780601a0 msgr2=0x7ff778144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:02.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 --2- 172.21.2.104:0/3744759004 >> v2:172.21.2.105:3300/0 conn(0x7ff7780601a0 0x7ff778144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77805cea0 con 0x7ff77813e5d0 2021-07-11T04:02:02.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7ff7700019e0 con 0x7ff77813e5d0 2021-07-11T04:02:02.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 >> v1:172.21.2.104:6789/0 conn(0x7ff77813e5d0 legacy=0x7ff77805bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 --2- 172.21.2.104:0/3744759004 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0037d0 0x7ff75c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 --2- 172.21.2.104:0/3744759004 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff77805f9f0 0x7ff75c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 --2- 172.21.2.104:0/3744759004 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff75c004bf0 0x7ff75c00fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7780ab420 con 0x7ff75c0037d0 2021-07-11T04:02:02.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c005280 con 0x7ff75c004bf0 2021-07-11T04:02:02.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c010050 con 0x7ff77805f9f0 2021-07-11T04:02:02.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77659c700 1 --2- 172.21.2.104:0/3744759004 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0037d0 0x7ff75c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:02.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77659c700 1 -- 172.21.2.104:0/3744759004 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff77805f9f0 msgr2=0x7ff75c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:02.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77659c700 1 --2- 172.21.2.104:0/3744759004 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff77805f9f0 0x7ff75c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77659c700 1 -- 172.21.2.104:0/3744759004 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff75c004bf0 msgr2=0x7ff75c00fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:02.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77659c700 1 --2- 172.21.2.104:0/3744759004 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff75c004bf0 0x7ff75c00fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77659c700 1 -- 172.21.2.104:0/3744759004 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c003260 con 0x7ff75c0037d0 2021-07-11T04:02:02.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77659c700 1 --2- 172.21.2.104:0/3744759004 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0037d0 0x7ff75c003bf0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7ff76c002f70 tx=0x7ff76c006ca0).ready entity=mon.0 client_cookie=f4296b7ac5fccf24 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:02.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff76c012410 con 0x7ff75c0037d0 2021-07-11T04:02:02.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c012570 con 0x7ff75c0037d0 2021-07-11T04:02:02.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77559a700 1 -- 172.21.2.104:0/3744759004 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff76c0128d0 con 0x7ff75c0037d0 2021-07-11T04:02:02.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77c9e1700 1 -- 172.21.2.104:0/3744759004 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0037d0 msgr2=0x7ff75c003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff77c9e1700 1 --2- 172.21.2.104:0/3744759004 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff75c0037d0 0x7ff75c003bf0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 rx=0x7ff76c002f70 tx=0x7ff76c006ca0).stop 2021-07-11T04:02:02.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.344+0000 7ff776d9d700 1 -- 172.21.2.104:0/3744759004 reap_dead start 2021-07-11T04:02:02.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 -- 172.21.2.104:0/3744759004 shutdown_connections 2021-07-11T04:02:02.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 -- 172.21.2.104:0/3744759004 >> 172.21.2.104:0/3744759004 conn(0x7ff7780b7790 msgr2=0x7ff7780b7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:02.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 -- 172.21.2.104:0/3744759004 shutdown_connections 2021-07-11T04:02:02.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 -- 172.21.2.104:0/3744759004 wait complete. 2021-07-11T04:02:02.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 Processor -- start 2021-07-11T04:02:02.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 -- start start 2021-07-11T04:02:02.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff77813a100 0x7ff778136940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff778143660 0x7ff778136e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff778144000 0x7ff778144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77813a4e0 con 0x7ff778144000 2021-07-11T04:02:02.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff778142c30 con 0x7ff77813a100 2021-07-11T04:02:02.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff778142e90 con 0x7ff778143660 2021-07-11T04:02:02.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff776d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff778144000 0x7ff778144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:02.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff776d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff778144000 0x7ff778144480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50674/0 (socket says 172.21.2.104:50674) 2021-07-11T04:02:02.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff776d9d700 1 -- 172.21.2.104:0/486490504 learned_addr learned my addr 172.21.2.104:0/486490504 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:02.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff776d9d700 1 -- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff778143660 msgr2=0x7ff778136e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff776d9d700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff778143660 0x7ff778136e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff776d9d700 1 -- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff77813a100 msgr2=0x7ff778136940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff776d9d700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff77813a100 0x7ff778136940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff776d9d700 1 -- 172.21.2.104:0/486490504 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff778143130 con 0x7ff778144000 2021-07-11T04:02:02.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff776d9d700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff778144000 0x7ff778144480 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7ff770001bc0 tx=0x7ff7700019d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:02.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff7677fe700 1 -- 172.21.2.104:0/486490504 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff770004b70 con 0x7ff778144000 2021-07-11T04:02:02.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff7677fe700 1 -- 172.21.2.104:0/486490504 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff770004cd0 con 0x7ff778144000 2021-07-11T04:02:02.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff7677fe700 1 -- 172.21.2.104:0/486490504 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff770007360 con 0x7ff778144000 2021-07-11T04:02:02.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 -- 172.21.2.104:0/486490504 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff778152a70 con 0x7ff778144000 2021-07-11T04:02:02.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 -- 172.21.2.104:0/486490504 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff778137320 con 0x7ff778144000 2021-07-11T04:02:02.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff77c9e1700 1 -- 172.21.2.104:0/486490504 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7781432c0 con 0x7ff778144000 2021-07-11T04:02:02.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff7677fe700 1 -- 172.21.2.104:0/486490504 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff7700074c0 con 0x7ff778144000 2021-07-11T04:02:02.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff7677fe700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7540531f0 0x7ff7540556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:02.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.348+0000 7ff7677fe700 1 -- 172.21.2.104:0/486490504 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ff77006b120 con 0x7ff778144000 2021-07-11T04:02:02.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.352+0000 7ff77659c700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7540531f0 0x7ff7540556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:02.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.352+0000 7ff77659c700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7540531f0 0x7ff7540556b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7ff76c002f70 tx=0x7ff76c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:02.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.352+0000 7ff7677fe700 1 -- 172.21.2.104:0/486490504 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff770042350 con 0x7ff778144000 2021-07-11T04:02:02.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 -- 172.21.2.104:0/486490504 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff778057350 con 0x7ff778144000 2021-07-11T04:02:02.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff7677fe700 1 -- 172.21.2.104:0/486490504 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2492 (secure 0 0 0) 0x7ff770045970 con 0x7ff778144000 2021-07-11T04:02:02.461 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:02.461 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5335,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/1814518027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":1814518027},{"type":"v1","addr":"172.21.2.105:6835","nonce":1814518027}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/155502403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":155502403},{"type":"v1","addr":"172.21.2.104:6834","nonce":155502403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:52.876988+0000","modified":"2021-07-11T04:01:54.362899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/2616834266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":2616834266},{"type":"v1","addr":"172.21.2.104:6835","nonce":2616834266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-07-11T04:02:02.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 -- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7540531f0 msgr2=0x7ff7540556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7540531f0 0x7ff7540556b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7ff76c002f70 tx=0x7ff76c015000).stop 2021-07-11T04:02:02.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 -- 172.21.2.104:0/486490504 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff778144000 msgr2=0x7ff778144480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:02.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff778144000 0x7ff778144480 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7ff770001bc0 tx=0x7ff7700019d0).stop 2021-07-11T04:02:02.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 -- 172.21.2.104:0/486490504 shutdown_connections 2021-07-11T04:02:02.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff7540531f0 0x7ff7540556b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff778144000 0x7ff778144480 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:02.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff778143660 0x7ff778136e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 --2- 172.21.2.104:0/486490504 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff77813a100 0x7ff778136940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:02.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 -- 172.21.2.104:0/486490504 >> 172.21.2.104:0/486490504 conn(0x7ff7780b7790 msgr2=0x7ff77805c690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:02.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 -- 172.21.2.104:0/486490504 shutdown_connections 2021-07-11T04:02:02.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:02.460+0000 7ff77c9e1700 1 -- 172.21.2.104:0/486490504 wait complete. 2021-07-11T04:02:02.469 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 68 2021-07-11T04:02:02.773 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:01:52.876988+0000', 'modified': '2021-07-11T04:01:54.362899+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8022}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8022': {'gid': 8022, 'name': 'a', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/2616834266', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6833', 'nonce': 2616834266}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 2616834266}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:02:02.774 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:02:02.775 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2021-07-11T04:02:02.840 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":8022,"want_state":"up:active","state":"up:active","rank_uptime":9.9575765179999998,"mdsmap_epoch":68,"osdmap_epoch":139,"osdmap_epoch_barrier":139,"uptime":14.21540675} 2021-07-11T04:02:02.840 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 8022, "mdsmap_epoch": 68, "osdmap_epoch": 139, "osdmap_epoch_barrier": 139, "rank_uptime": 9.957576518, "state": "up:active", "uptime": 14.21540675, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:02:02.841 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:02:02.841 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-07-11T04:02:02.842 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba023.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-07-11T04:02:02.842 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:02.842 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:02.854 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:02:02.866 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:02.867 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:02.873 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:02.874 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:02.874 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:02.875 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:02:02.920 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:02:02.920 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:02.923 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:02:02.934 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:02.935 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:02.967 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15183]: starting ceph client 2021-07-11T04:02:02.968 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:2021-07-11T04:02:02.962+0000 7f5696100080 -1 init, newargv = 0x55c1cddb5a10 newargc=9 2021-07-11T04:02:02.979 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:02.981 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:02.981 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:02.982 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:02:02.984 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15183]: starting fuse 2021-07-11T04:02:02.987 INFO:teuthology.orchestra.run.gibba023.stdout:52 2021-07-11T04:02:02.987 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:02:02.987 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:02.988 DEBUG:teuthology.orchestra.run.gibba023:> sudo python3 -c ' 2021-07-11T04:02:02.988 DEBUG:teuthology.orchestra.run.gibba023:> import glob 2021-07-11T04:02:02.989 DEBUG:teuthology.orchestra.run.gibba023:> import re 2021-07-11T04:02:02.989 DEBUG:teuthology.orchestra.run.gibba023:> import os 2021-07-11T04:02:02.989 DEBUG:teuthology.orchestra.run.gibba023:> import subprocess 2021-07-11T04:02:02.990 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:02.991 DEBUG:teuthology.orchestra.run.gibba023:> def find_socket(client_name): 2021-07-11T04:02:02.991 DEBUG:teuthology.orchestra.run.gibba023:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-07-11T04:02:02.992 DEBUG:teuthology.orchestra.run.gibba023:> files = glob.glob(asok_path) 2021-07-11T04:02:02.992 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:02.992 DEBUG:teuthology.orchestra.run.gibba023:> # Given a non-glob path, it better be there 2021-07-11T04:02:02.993 DEBUG:teuthology.orchestra.run.gibba023:> if "*" not in asok_path: 2021-07-11T04:02:02.993 DEBUG:teuthology.orchestra.run.gibba023:> assert(len(files) == 1) 2021-07-11T04:02:02.994 DEBUG:teuthology.orchestra.run.gibba023:> return files[0] 2021-07-11T04:02:02.994 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:02.994 DEBUG:teuthology.orchestra.run.gibba023:> for f in files: 2021-07-11T04:02:02.995 DEBUG:teuthology.orchestra.run.gibba023:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:02:02.995 DEBUG:teuthology.orchestra.run.gibba023:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:02:02.996 DEBUG:teuthology.orchestra.run.gibba023:> return f 2021-07-11T04:02:02.996 DEBUG:teuthology.orchestra.run.gibba023:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:02:02.996 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:02.997 DEBUG:teuthology.orchestra.run.gibba023:> print(find_socket("client.1")) 2021-07-11T04:02:02.997 DEBUG:teuthology.orchestra.run.gibba023:> ' 2021-07-11T04:02:03.052 INFO:teuthology.orchestra.run.gibba023.stdout:/var/run/ceph/ceph-client.1.15183.asok 2021-07-11T04:02:03.054 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15183.asok 2021-07-11T04:02:03.054 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:03.055 DEBUG:teuthology.orchestra.run.gibba023:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15183.asok status 2021-07-11T04:02:03.105 INFO:teuthology.orchestra.run.gibba023.stdout:{ 2021-07-11T04:02:03.105 INFO:teuthology.orchestra.run.gibba023.stdout: "metadata": { 2021-07-11T04:02:03.105 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:02:03.105 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:02:03.106 INFO:teuthology.orchestra.run.gibba023.stdout: "entity_id": "1", 2021-07-11T04:02:03.106 INFO:teuthology.orchestra.run.gibba023.stdout: "hostname": "gibba023", 2021-07-11T04:02:03.106 INFO:teuthology.orchestra.run.gibba023.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-07-11T04:02:03.106 INFO:teuthology.orchestra.run.gibba023.stdout: "pid": "15183", 2021-07-11T04:02:03.107 INFO:teuthology.orchestra.run.gibba023.stdout: "root": "/" 2021-07-11T04:02:03.107 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:03.107 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_count": 0, 2021-07-11T04:02:03.107 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_pinned_count": 0, 2021-07-11T04:02:03.107 INFO:teuthology.orchestra.run.gibba023.stdout: "id": 5371, 2021-07-11T04:02:03.108 INFO:teuthology.orchestra.run.gibba023.stdout: "inst": { 2021-07-11T04:02:03.108 INFO:teuthology.orchestra.run.gibba023.stdout: "name": { 2021-07-11T04:02:03.108 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "client", 2021-07-11T04:02:03.108 INFO:teuthology.orchestra.run.gibba023.stdout: "num": 5371 2021-07-11T04:02:03.108 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:03.109 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:02:03.109 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:02:03.109 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:02:03.109 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 2513306309 2021-07-11T04:02:03.110 INFO:teuthology.orchestra.run.gibba023.stdout: } 2021-07-11T04:02:03.110 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:03.110 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:02:03.110 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:02:03.110 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:02:03.111 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 2513306309 2021-07-11T04:02:03.111 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:03.111 INFO:teuthology.orchestra.run.gibba023.stdout: "inst_str": "client.5371 172.21.2.123:0/2513306309", 2021-07-11T04:02:03.111 INFO:teuthology.orchestra.run.gibba023.stdout: "addr_str": "172.21.2.123:0/2513306309", 2021-07-11T04:02:03.111 INFO:teuthology.orchestra.run.gibba023.stdout: "inode_count": 1, 2021-07-11T04:02:03.112 INFO:teuthology.orchestra.run.gibba023.stdout: "mds_epoch": 68, 2021-07-11T04:02:03.112 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch": 139, 2021-07-11T04:02:03.112 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:02:03.112 INFO:teuthology.orchestra.run.gibba023.stdout: "blacklisted": false, 2021-07-11T04:02:03.113 INFO:teuthology.orchestra.run.gibba023.stdout: "fs_name": "cephfs" 2021-07-11T04:02:03.113 INFO:teuthology.orchestra.run.gibba023.stdout:} 2021-07-11T04:02:03.114 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:03.115 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:03.115 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:03.120 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:02:03.121 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:03.121 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:03.121 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:03.173 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:03.174 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:03.174 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:03.185 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:02:03.187 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:02:03.187 DEBUG:teuthology.orchestra.run.gibba019:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2021-07-11T04:02:03.188 DEBUG:teuthology.orchestra.run.gibba019:> import time 2021-07-11T04:02:03.188 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:02:03.188 DEBUG:teuthology.orchestra.run.gibba019:> with open("/home/ubuntu/cephtest/mnt.0/testfile", '"'"'w'"'"') as f: 2021-07-11T04:02:03.189 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content'"'"') 2021-07-11T04:02:03.189 DEBUG:teuthology.orchestra.run.gibba019:> f.flush() 2021-07-11T04:02:03.190 DEBUG:teuthology.orchestra.run.gibba019:> f.write('"'"'content2'"'"') 2021-07-11T04:02:03.190 DEBUG:teuthology.orchestra.run.gibba019:> while True: 2021-07-11T04:02:03.190 DEBUG:teuthology.orchestra.run.gibba019:> time.sleep(1) 2021-07-11T04:02:03.191 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T04:02:03.194 DEBUG:teuthology.orchestra.run.gibba019:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2021-07-11T04:02:03.210 INFO:teuthology.orchestra.run.gibba019.stderr:ls: cannot access '/home/ubuntu/cephtest/mnt.0/testfile': No such file or directory 2021-07-11T04:02:03.211 DEBUG:teuthology.orchestra.run:got remote process result: 2 2021-07-11T04:02:04.213 DEBUG:teuthology.orchestra.run.gibba019:> sudo ls /home/ubuntu/cephtest/mnt.0/testfile 2021-07-11T04:02:04.225 INFO:teuthology.orchestra.run.gibba019.stdout:/home/ubuntu/cephtest/mnt.0/testfile 2021-07-11T04:02:04.226 DEBUG:tasks.cephfs.mount:File testfile became visible from 0 after 1s 2021-07-11T04:02:04.227 DEBUG:teuthology.orchestra.run.gibba023:> sudo ls /home/ubuntu/cephtest/mnt.1/testfile 2021-07-11T04:02:04.272 INFO:teuthology.orchestra.run.gibba023.stdout:/home/ubuntu/cephtest/mnt.1/testfile 2021-07-11T04:02:04.273 DEBUG:tasks.cephfs.mount:File testfile became visible from 1 after 0s 2021-07-11T04:02:04.274 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-07-11T04:02:04.274 DEBUG:teuthology.orchestra.run.gibba023:> cd /home/ubuntu/cephtest/mnt.1 && sudo cat testfile 2021-07-11T04:02:04.334 INFO:teuthology.orchestra.run.gibba023.stdout:content 2021-07-11T04:02:04.346 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:02:04.346 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:04.359 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:04.373 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:04.383 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:04.412 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:02:04.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:04.413 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:04.414 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:04.423 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:02:04.425 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:02:04.425 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T04:02:04.425 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:02:04.426 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:04.487 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15692]: fuse finished with error 0 and tester_r 0 2021-07-11T04:02:04.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:04.490 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:04.490 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:04.499 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T04:02:04.501 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:02:04.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:02:04.534 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:04.538 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:04.538 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:04.539 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:04.550 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:02:04.551 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:04.552 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba023.front.sepia.ceph.com... 2021-07-11T04:02:04.552 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:02:04.552 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:04.566 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15183]: fuse finished with error 0 and tester_r 0 2021-07-11T04:02:04.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:04.568 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:04.568 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:04.616 INFO:teuthology.orchestra.run.gibba023.stdout:ext2/ext3 2021-07-11T04:02:04.617 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:02:04.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:02:04.728 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:04.733 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:02:04.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.792+0000 7f5bb421f700 1 Processor -- start 2021-07-11T04:02:04.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.792+0000 7f5bb421f700 1 -- start start 2021-07-11T04:02:04.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.792+0000 7f5bb421f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5bac0b2560 0x7f5bac0a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:04.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.792+0000 7f5bb421f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5bac1475a0 0x7f5bac0ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.792+0000 7f5bb421f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bac14ac10 con 0x7f5bac0b1950 2021-07-11T04:02:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.792+0000 7f5bb421f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bac0a8cc0 con 0x7f5bac0b2560 2021-07-11T04:02:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.792+0000 7f5bb421f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac147a10 con 0x7f5bac1475a0 2021-07-11T04:02:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb17ba700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f5bac0b1950 0x7f5bac14b2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47446/0 (socket says 172.21.2.104:47446) 2021-07-11T04:02:04.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb17ba700 1 -- 172.21.2.104:0/218814163 learned_addr learned my addr 172.21.2.104:0/218814163 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:04.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 --2- 172.21.2.104:0/218814163 >> v2:172.21.2.104:3300/0 conn(0x7f5bac0b2560 0x7f5bac0a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:04.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 -- 172.21.2.104:0/218814163 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5bac1475a0 msgr2=0x7f5bac0ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:04.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 --2- 172.21.2.104:0/218814163 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5bac1475a0 0x7f5bac0ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:04.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 -- 172.21.2.104:0/218814163 >> v1:172.21.2.104:6789/0 conn(0x7f5bac0b1950 legacy=0x7f5bac14b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:04.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 -- 172.21.2.104:0/218814163 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac0b2940 con 0x7f5bac0b2560 2021-07-11T04:02:04.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 --2- 172.21.2.104:0/218814163 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5bac0b2560 0x7f5bac0a7d00 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7f5ba0002f70 tx=0x7f5ba0006df0).ready entity=mon.0 client_cookie=d82cdc32f1a92df9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:04.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb0fb9700 1 -- 172.21.2.104:0/218814163 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba000d570 con 0x7f5bac0b2560 2021-07-11T04:02:04.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb0fb9700 1 -- 172.21.2.104:0/218814163 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0000e90 con 0x7f5bac0b2560 2021-07-11T04:02:04.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb0fb9700 1 -- 172.21.2.104:0/218814163 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba000d8c0 con 0x7f5bac0b2560 2021-07-11T04:02:04.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 -- 172.21.2.104:0/218814163 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5bac0b2560 msgr2=0x7f5bac0a7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:04.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 --2- 172.21.2.104:0/218814163 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5bac0b2560 0x7f5bac0a7d00 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7f5ba0002f70 tx=0x7f5ba0006df0).stop 2021-07-11T04:02:04.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 -- 172.21.2.104:0/218814163 shutdown_connections 2021-07-11T04:02:04.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 --2- 172.21.2.104:0/218814163 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5bac1475a0 0x7f5bac0ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:04.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 --2- 172.21.2.104:0/218814163 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5bac0b2560 0x7f5bac0a7d00 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:04.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 -- 172.21.2.104:0/218814163 >> 172.21.2.104:0/218814163 conn(0x7f5bac05ca20 msgr2=0x7f5bac05ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:04.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 -- 172.21.2.104:0/218814163 shutdown_connections 2021-07-11T04:02:04.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 -- 172.21.2.104:0/218814163 wait complete. 2021-07-11T04:02:04.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 Processor -- start 2021-07-11T04:02:04.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 -- start start 2021-07-11T04:02:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5bac0b1950 0x7f5bac0acc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5bac0b2560 0x7f5bac15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:04.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5bac1475a0 0x7f5bac15a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac15abd0 con 0x7f5bac0b1950 2021-07-11T04:02:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0aeaf0 con 0x7f5bac1475a0 2021-07-11T04:02:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb421f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac0aed90 con 0x7f5bac0b2560 2021-07-11T04:02:04.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5bac0b1950 0x7f5bac0acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5bac0b1950 0x7f5bac0acc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50684/0 (socket says 172.21.2.104:50684) 2021-07-11T04:02:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 -- 172.21.2.104:0/997140884 learned_addr learned my addr 172.21.2.104:0/997140884 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 -- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5bac0b2560 msgr2=0x7f5bac15a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:04.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5bac0b2560 0x7f5bac15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 -- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5bac1475a0 msgr2=0x7f5bac15a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5bac1475a0 0x7f5bac15a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 -- 172.21.2.104:0/997140884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba000b180 con 0x7f5bac0b1950 2021-07-11T04:02:04.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.796+0000 7f5bb1fbb700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5bac0b1950 0x7f5bac0acc40 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f5ba0002f70 tx=0x7f5ba0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5b9effd700 1 -- 172.21.2.104:0/997140884 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba0011990 con 0x7f5bac0b1950 2021-07-11T04:02:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5b9effd700 1 -- 172.21.2.104:0/997140884 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0011af0 con 0x7f5bac0b1950 2021-07-11T04:02:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5bb421f700 1 -- 172.21.2.104:0/997140884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac0af030 con 0x7f5bac0b1950 2021-07-11T04:02:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5b9effd700 1 -- 172.21.2.104:0/997140884 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5ba0011e50 con 0x7f5bac0b1950 2021-07-11T04:02:04.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5bb421f700 1 -- 172.21.2.104:0/997140884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac159c80 con 0x7f5bac0b1950 2021-07-11T04:02:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5bb421f700 1 -- 172.21.2.104:0/997140884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac057350 con 0x7f5bac0b1950 2021-07-11T04:02:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5b9effd700 1 -- 172.21.2.104:0/997140884 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5ba000f070 con 0x7f5bac0b1950 2021-07-11T04:02:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5b9effd700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b8c053270 0x7f5b8c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:04.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5b9effd700 1 -- 172.21.2.104:0/997140884 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5ba006cf40 con 0x7f5bac0b1950 2021-07-11T04:02:04.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5bb17ba700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b8c053270 0x7f5b8c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:04.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5bb17ba700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b8c053270 0x7f5b8c055730 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f5bac0a94b0 tx=0x7f5ba8007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:04.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.800+0000 7f5b9effd700 1 -- 172.21.2.104:0/997140884 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5ba00441e0 con 0x7f5bac0b1950 2021-07-11T04:02:04.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.880+0000 7f5bb421f700 1 -- 172.21.2.104:0/997140884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5bac067b60 con 0x7f5bac0b1950 2021-07-11T04:02:04.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.880+0000 7f5b9effd700 1 -- 172.21.2.104:0/997140884 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v139) v1 ==== 81+0+33659 (secure 0 0 0) 0x7f5ba0047800 con 0x7f5bac0b1950 2021-07-11T04:02:04.883 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:04.883 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:02:04.884 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 139, 2021-07-11T04:02:04.884 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:02:04.884 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:02:04.884 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:01:53.880137+0000", 2021-07-11T04:02:04.885 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:02:04.885 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:02:04.885 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:02:04.885 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:02:04.886 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:02:04.886 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:02:04.887 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:02:04.887 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:02:04.887 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:02:04.887 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:04.888 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:02:04.888 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:02:04.889 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:02:04.890 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:02:04.890 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:02:04.890 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 26, 2021-07-11T04:02:04.890 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:02:04.891 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:02:04.891 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:02:04.891 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:02:04.891 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:02:04.892 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.892 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:02:04.892 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:02:04.892 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:02:04.893 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:04.893 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:04.893 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:04.894 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:04.894 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:04.894 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:04.894 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:04.895 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:04.895 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:02:04.895 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:02:04.895 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:02:04.896 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:02:04.896 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:02:04.896 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:04.896 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:04.896 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:04.897 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:04.897 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:04.897 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:04.897 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:04.898 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.898 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:02:04.898 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:04.898 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:04.898 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:04.899 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:04.899 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:04.899 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:04.899 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:04.900 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:04.900 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:04.900 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:04.901 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:04.901 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:04.901 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:04.901 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:04.902 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:04.902 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:04.902 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:04.902 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:04.902 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:04.903 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:04.903 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:04.904 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:04.904 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:04.904 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:04.905 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:04.905 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:04.905 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.905 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:04.905 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:04.906 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:04.906 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:04.906 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:04.906 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:04.907 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:04.907 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:04.907 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:04.907 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:04.907 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:04.908 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:02:04.908 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:02:04.908 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.908 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:04.909 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:02:04.909 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.909 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.909 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.909 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:02:04.910 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:02:04.910 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:02:04.911 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:04.911 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:04.911 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:04.911 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:04.911 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:04.912 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:04.912 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:04.912 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:04.912 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:02:04.913 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:02:04.913 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:02:04.913 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:02:04.913 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:02:04.913 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:04.914 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:04.914 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:04.915 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:04.915 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:04.915 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:04.915 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:04.916 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.916 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:02:04.916 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:04.916 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:04.916 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:04.917 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:04.917 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:04.917 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:04.917 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:04.918 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:04.918 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:04.918 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:04.918 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:04.919 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:04.919 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:04.919 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:04.919 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:04.919 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:04.920 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:04.920 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:04.920 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:04.921 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:04.921 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:04.921 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:04.921 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:04.922 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:04.922 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:04.922 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:04.922 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.923 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:04.923 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:04.923 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:04.923 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:04.923 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:04.924 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:04.924 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:04.924 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:04.925 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:04.925 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:04.925 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:04.925 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:02:04.926 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:04.926 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:02:04.926 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.926 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.927 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.927 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 25, 2021-07-11T04:02:04.927 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T04:02:04.927 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:01:50.686994+0000", 2021-07-11T04:02:04.928 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:04.928 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:04.928 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:04.928 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:04.928 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:04.929 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:04.929 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:04.929 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:04.929 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:02:04.930 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:02:04.930 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:02:04.930 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:02:04.930 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:02:04.930 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:04.931 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:04.931 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:04.932 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:04.932 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:04.932 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:04.932 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:04.932 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.933 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "138", 2021-07-11T04:02:04.933 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:04.933 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:04.933 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:04.934 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:04.934 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:04.934 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:04.934 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:04.934 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:04.935 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:04.935 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:04.936 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:04.936 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:04.936 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:04.936 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:04.936 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:04.937 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:04.937 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:04.937 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:04.937 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:04.938 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:04.938 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:04.938 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:04.938 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:04.939 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:04.939 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:04.939 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:04.939 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.939 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:04.940 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:04.940 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:04.940 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:04.940 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:04.941 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:04.941 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:04.941 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:04.942 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:04.942 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:04.942 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:04.942 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:02:04.943 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T04:02:04.943 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T04:02:04.943 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T04:02:04.943 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.943 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:04.944 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:02:04.944 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T04:02:04.944 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.944 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.945 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.945 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.945 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 26, 2021-07-11T04:02:04.946 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T04:02:04.946 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:01:51.853293+0000", 2021-07-11T04:02:04.946 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:04.946 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:04.947 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:04.947 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:04.947 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:04.947 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:04.947 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:04.948 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:04.948 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:02:04.948 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:02:04.948 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:02:04.949 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:02:04.949 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:02:04.949 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:04.949 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:04.949 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:04.950 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:04.950 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:04.950 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:04.950 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:04.950 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.951 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "139", 2021-07-11T04:02:04.951 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:04.951 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:04.951 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:04.952 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:04.952 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:04.952 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:04.952 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:04.952 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:04.953 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:04.953 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:04.953 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:04.953 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:04.954 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:04.954 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:04.954 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:04.954 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:04.954 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:04.955 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:04.955 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:04.955 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:04.955 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:04.956 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:04.956 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:04.956 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:04.956 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:04.957 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:04.957 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.957 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:04.957 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:04.957 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:04.958 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:04.958 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:04.958 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:04.958 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:04.958 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:04.959 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:04.959 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:04.959 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:04.959 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:02:04.960 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:04.960 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:02:04.960 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T04:02:04.960 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.961 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.961 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.961 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:04.961 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:02:04.961 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.962 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:02:04.962 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:02:04.962 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:04.962 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:04.963 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:04.963 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:04.963 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:04.963 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:04.964 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:04.964 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 136, 2021-07-11T04:02:04.964 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:04.964 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:04.964 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:04.965 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:04.965 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.965 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:04.965 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:02:04.966 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:04.966 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.966 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.966 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:04.967 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:02:04.967 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:04.967 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.967 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:04.967 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.968 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:04.968 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:04.968 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.968 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:04.969 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:02:04.969 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:04.969 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.969 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.970 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:04.970 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:02:04.970 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:04.970 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.971 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:04.971 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.971 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:04.971 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:04.971 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.972 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:04.972 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:02:04.972 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:04.972 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.973 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.973 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:04.973 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:02:04.973 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:04.974 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.974 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:04.974 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.974 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:04.974 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:04.975 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.975 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:04.975 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:02:04.975 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:04.975 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.976 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.976 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:04.976 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:02:04.976 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:04.977 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.977 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:04.977 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.977 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:02:04.977 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:02:04.978 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:02:04.978 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:02:04.978 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:04.978 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:04.979 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:04.979 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:04.979 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.979 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.979 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:02:04.980 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:02:04.980 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:04.980 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:04.980 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:04.980 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:04.981 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:04.981 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:04.981 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:04.981 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 137, 2021-07-11T04:02:04.982 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:04.982 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:04.982 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:04.982 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:04.982 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.983 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:04.983 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:02:04.983 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:04.983 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.983 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.984 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:04.984 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:02:04.984 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:04.984 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.985 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:04.985 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.985 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:04.985 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:04.985 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.986 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:04.986 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:02:04.986 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:04.986 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.986 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.987 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:04.987 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:02:04.987 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:04.987 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.988 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:04.988 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.988 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:04.988 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:04.988 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.989 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:04.989 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:02:04.989 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:04.989 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.989 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.990 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:04.990 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:02:04.990 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:04.990 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.991 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:04.991 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.991 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:04.991 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:04.991 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.992 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:04.992 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:02:04.992 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:04.992 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.992 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.993 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:04.993 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:02:04.993 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:04.993 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:04.994 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:04.994 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.994 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:02:04.994 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:02:04.994 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:02:04.995 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:02:04.995 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:04.995 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:04.995 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:04.995 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:04.996 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:04.996 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.996 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:02:04.996 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:02:04.996 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:04.997 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:04.997 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:04.997 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:04.997 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:04.998 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:04.998 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:04.998 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 137, 2021-07-11T04:02:04.998 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:04.998 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:04.999 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:04.999 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:04.999 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:04.999 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:04.999 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:02:05.000 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:05.000 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.000 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.000 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.001 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:02:05.001 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:05.001 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.001 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.001 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.002 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:05.002 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.002 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.002 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.003 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:02:05.003 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:05.003 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.003 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.003 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.004 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:02:05.004 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:05.004 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.004 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.004 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.005 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:05.005 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.005 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.005 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.006 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:02:05.006 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:05.006 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.006 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.006 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.007 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:02:05.007 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:05.007 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.007 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.007 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.008 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:05.008 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.008 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.008 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.009 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:02:05.009 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:05.009 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.009 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.009 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.010 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:02:05.010 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:05.010 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.010 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.010 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.011 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:02:05.011 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:02:05.011 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:02:05.011 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:02:05.012 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:05.012 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:05.012 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:05.012 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.012 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.013 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.013 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:02:05.013 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:02:05.013 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:05.013 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:05.014 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:05.014 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:05.014 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:05.014 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:05.015 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:05.015 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 136, 2021-07-11T04:02:05.015 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:05.015 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:05.015 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:05.016 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.016 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.016 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.016 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:02:05.017 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:05.017 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.017 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.017 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.017 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:02:05.018 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:05.018 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.018 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.018 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.018 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:05.019 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.019 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.019 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.019 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:02:05.020 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:05.020 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.020 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.020 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.020 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:02:05.021 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:05.021 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.021 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.021 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.021 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:05.022 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.022 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.022 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.022 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:02:05.023 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:05.023 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.023 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.023 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.023 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:02:05.024 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:05.024 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.024 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.024 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.024 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:05.025 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.025 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.025 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.025 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:02:05.025 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:05.026 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.026 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.026 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.026 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:02:05.027 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:05.027 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.027 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.027 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.027 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:02:05.028 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:02:05.028 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:02:05.028 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:02:05.028 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:05.028 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:05.029 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:05.029 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.029 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.029 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.030 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:02:05.030 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:02:05.030 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:05.030 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:05.030 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:05.031 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:05.031 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:05.031 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:05.031 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:05.032 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 137, 2021-07-11T04:02:05.032 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:05.032 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:05.032 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:05.032 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.033 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.033 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.033 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:02:05.033 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:05.033 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.034 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.034 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.034 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:02:05.034 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:05.035 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.035 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.035 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.035 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:05.035 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.036 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.036 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.036 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:02:05.036 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:05.037 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.037 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.037 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.037 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:02:05.037 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:05.038 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.038 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.038 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.038 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:05.038 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.039 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.039 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.039 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:02:05.039 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:05.040 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.040 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.040 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.040 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:02:05.040 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:05.041 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.041 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.041 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.041 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:05.041 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.042 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.042 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.042 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:02:05.042 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:05.043 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.043 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.043 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.043 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:02:05.043 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:05.044 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.044 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.044 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.044 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:02:05.044 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:02:05.045 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:02:05.045 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:02:05.045 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:05.045 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:05.045 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:05.046 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.046 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.046 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.046 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:02:05.047 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:02:05.047 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:05.047 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:05.047 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:05.047 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:05.048 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:05.048 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:05.048 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:05.048 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 130, 2021-07-11T04:02:05.048 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:05.049 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:05.049 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:05.049 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.049 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.050 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.050 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:02:05.050 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:05.050 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.050 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.051 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.051 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:02:05.051 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:05.051 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.051 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.052 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.052 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:05.052 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.052 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.053 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.053 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:02:05.053 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:05.053 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.053 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.054 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.054 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:02:05.054 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:05.054 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.055 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.055 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.055 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:05.055 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.055 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.056 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.056 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:02:05.056 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:05.056 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.056 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.057 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.057 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:02:05.057 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:05.057 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.058 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.058 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.058 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:05.058 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.058 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.059 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.059 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:02:05.059 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:05.059 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.059 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.060 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.060 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:02:05.060 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:05.060 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.061 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.061 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.061 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:02:05.061 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:02:05.061 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:02:05.062 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:02:05.062 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:05.062 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:05.062 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:05.063 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.063 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.063 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.063 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:02:05.063 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:02:05.064 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:05.064 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:05.064 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:05.064 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:05.065 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:05.065 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:05.065 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:05.065 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 137, 2021-07-11T04:02:05.065 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:05.066 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:05.066 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:05.066 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.066 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.066 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.067 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:02:05.067 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:05.067 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.067 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.068 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.068 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:02:05.068 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:05.068 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.068 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.069 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.069 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:05.069 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.069 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.070 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.070 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:02:05.070 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:05.070 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.070 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.071 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.071 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:02:05.071 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:05.071 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.071 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.072 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.072 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:05.072 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.072 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.073 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.073 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:02:05.073 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:05.073 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.073 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.074 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.074 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:02:05.074 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:05.074 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.074 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.075 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.075 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:05.075 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.075 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.076 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.076 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:02:05.076 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:05.076 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.076 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.077 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.077 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:02:05.077 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:05.077 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.078 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.078 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.078 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:02:05.078 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:02:05.078 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:02:05.079 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:02:05.079 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:05.079 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:05.079 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:05.080 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.080 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.080 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.080 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:02:05.080 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:02:05.081 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:05.081 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:05.081 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:05.081 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:05.081 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:05.082 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:05.082 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:05.082 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 137, 2021-07-11T04:02:05.082 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:05.083 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:05.083 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:05.083 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.083 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.083 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.084 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:02:05.084 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:05.084 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.084 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.084 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.085 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:02:05.085 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:05.085 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.085 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.086 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.086 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:05.086 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.086 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.086 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.087 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:02:05.087 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:05.087 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.087 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.087 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.088 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:02:05.088 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:05.088 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.088 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.089 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.089 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:05.089 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.089 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.089 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.090 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:02:05.090 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:05.090 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.090 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.090 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.091 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:02:05.091 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:05.091 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.091 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.092 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.092 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:05.092 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:05.092 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.092 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:05.093 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:02:05.093 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:05.093 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.093 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.094 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:05.094 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:02:05.094 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:05.094 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.094 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.095 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.095 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:02:05.095 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:02:05.095 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:02:05.096 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:02:05.096 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:05.096 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:05.096 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:05.096 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:05.097 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.097 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:05.097 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:02:05.097 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.097 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:02:05.098 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:05.098 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:05.098 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:05.098 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:05.099 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:05.099 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:02:05.099 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:05.099 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.099 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.100 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:02:05.100 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:05.100 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:05.100 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:05.100 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:05.101 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:05.101 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:02:05.101 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:05.101 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.101 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.102 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:02:05.102 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:05.102 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:05.102 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:05.103 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:05.103 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:05.103 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:02:05.103 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:05.103 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.104 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.104 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:02:05.104 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:05.104 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:05.104 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:05.105 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:05.105 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:05.105 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:02:05.105 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:05.106 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.106 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.106 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:02:05.106 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:05.106 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:05.107 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:05.107 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:05.107 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:05.107 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:02:05.108 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:05.108 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.108 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.108 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:02:05.108 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:05.109 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:05.109 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:05.109 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:05.109 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:05.110 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:02:05.110 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:05.110 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.110 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.110 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:02:05.111 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:05.111 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:05.111 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:05.111 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:05.112 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:05.112 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:02:05.112 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:05.112 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.112 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:05.113 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:02:05.113 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:05.113 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:05.113 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:05.114 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:05.114 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:05.114 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:02:05.114 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:05.114 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.115 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:05.115 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:02:05.115 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:02:05.115 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:02:05.116 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:02:05.116 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": {}, 2021-07-11T04:02:05.116 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:02:05.116 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:02:05.116 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:02:05.117 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:02:05.117 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:02:05.117 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:02:05.117 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:02:05.117 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:02:05.118 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:05.118 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:05.118 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:02:05.118 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:02:05.119 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:02:05.119 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:02:05.119 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:02:05.119 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:02:05.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 -- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b8c053270 msgr2=0x7f5b8c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:05.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b8c053270 0x7f5b8c055730 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f5bac0a94b0 tx=0x7f5ba8007800).stop 2021-07-11T04:02:05.121 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 -- 172.21.2.104:0/997140884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5bac0b1950 msgr2=0x7f5bac0acc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:05.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5bac0b1950 0x7f5bac0acc40 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 rx=0x7f5ba0002f70 tx=0x7f5ba0015000).stop 2021-07-11T04:02:05.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 -- 172.21.2.104:0/997140884 shutdown_connections 2021-07-11T04:02:05.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5bac1475a0 0x7f5bac15a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:05.122 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5bac0b2560 0x7f5bac15a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:05.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b8c053270 0x7f5b8c055730 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:05.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 --2- 172.21.2.104:0/997140884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5bac0b1950 0x7f5bac0acc40 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:05.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 -- 172.21.2.104:0/997140884 >> 172.21.2.104:0/997140884 conn(0x7f5bac05ca20 msgr2=0x7f5bac147b50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:05.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 -- 172.21.2.104:0/997140884 shutdown_connections 2021-07-11T04:02:05.123 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:04.884+0000 7f5bb421f700 1 -- 172.21.2.104:0/997140884 wait complete. 2021-07-11T04:02:05.124 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:05.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.180+0000 7f923d8f9700 1 Processor -- start 2021-07-11T04:02:05.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.180+0000 7f923d8f9700 1 -- start start 2021-07-11T04:02:05.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.180+0000 7f923d8f9700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f92380b5820 0x7f92380677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.180+0000 7f923d8f9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9238145140 0x7f92380b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.180+0000 7f923d8f9700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92380b5c00 con 0x7f92380a8d60 2021-07-11T04:02:05.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.180+0000 7f923d8f9700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92380b00b0 con 0x7f92380b5820 2021-07-11T04:02:05.183 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.180+0000 7f923d8f9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92380b0350 con 0x7f9238145140 2021-07-11T04:02:05.184 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.180+0000 7f9236ffd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f92380b5820 0x7f92380677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:05.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f92377fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9238145140 0x7f92380b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:05.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f9236ffd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f92380b5820 0x7f92380677f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:58252/0 (socket says 172.21.2.104:58252) 2021-07-11T04:02:05.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f92367fc700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f92380a8d60 0x7f9238144a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:46834/0 (socket says 172.21.2.104:46834) 2021-07-11T04:02:05.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f9236ffd700 1 -- 172.21.2.104:0/153397933 learned_addr learned my addr 172.21.2.104:0/153397933 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:05.185 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f92377fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9238145140 0x7f92380b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:53858/0 (socket says 172.21.2.104:53858) 2021-07-11T04:02:05.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f9236ffd700 1 -- 172.21.2.104:0/153397933 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9238145140 msgr2=0x7f92380b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:05.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f9236ffd700 1 --2- 172.21.2.104:0/153397933 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9238145140 0x7f92380b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:05.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f9236ffd700 1 -- 172.21.2.104:0/153397933 >> v1:172.21.2.105:6789/0 conn(0x7f92380a8d60 legacy=0x7f9238144a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:05.186 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f9236ffd700 1 -- 172.21.2.104:0/153397933 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92380b05f0 con 0x7f92380b5820 2021-07-11T04:02:05.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f92377fe700 1 --2- 172.21.2.104:0/153397933 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9238145140 0x7f92380b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:02:05.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f9236ffd700 1 --2- 172.21.2.104:0/153397933 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f92380b5820 0x7f92380677f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9220002f70 tx=0x7f9220006eb0).ready entity=mon.1 client_cookie=c7aab0140f747f14 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:05.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f9235ffb700 1 -- 172.21.2.104:0/153397933 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f922000d570 con 0x7f92380b5820 2021-07-11T04:02:05.187 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f9235ffb700 1 -- 172.21.2.104:0/153397933 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9220000e90 con 0x7f92380b5820 2021-07-11T04:02:05.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f9235ffb700 1 -- 172.21.2.104:0/153397933 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f922000d900 con 0x7f92380b5820 2021-07-11T04:02:05.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 -- 172.21.2.104:0/153397933 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f92380b5820 msgr2=0x7f92380677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:05.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 --2- 172.21.2.104:0/153397933 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f92380b5820 0x7f92380677f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9220002f70 tx=0x7f9220006eb0).stop 2021-07-11T04:02:05.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 -- 172.21.2.104:0/153397933 shutdown_connections 2021-07-11T04:02:05.188 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 --2- 172.21.2.104:0/153397933 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9238145140 0x7f92380b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:05.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 --2- 172.21.2.104:0/153397933 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f92380b5820 0x7f92380677f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:05.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 -- 172.21.2.104:0/153397933 >> 172.21.2.104:0/153397933 conn(0x7f923805bd40 msgr2=0x7f923805c150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:05.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 -- 172.21.2.104:0/153397933 shutdown_connections 2021-07-11T04:02:05.189 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 -- 172.21.2.104:0/153397933 wait complete. 2021-07-11T04:02:05.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 Processor -- start 2021-07-11T04:02:05.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 -- start start 2021-07-11T04:02:05.192 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f92380a8d60 0x7f9238147650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f92380b5820 0x7f9238145ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9238145140 0x7f9238143860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238146a10 con 0x7f92380a8d60 2021-07-11T04:02:05.193 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92381493e0 con 0x7f92380b5820 2021-07-11T04:02:05.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.184+0000 7f923d8f9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923814a100 con 0x7f9238145140 2021-07-11T04:02:05.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f9236ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f92380a8d60 0x7f9238147650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:05.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f9236ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f92380a8d60 0x7f9238147650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50698/0 (socket says 172.21.2.104:50698) 2021-07-11T04:02:05.194 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f9236ffd700 1 -- 172.21.2.104:0/1983442863 learned_addr learned my addr 172.21.2.104:0/1983442863 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:05.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f9236ffd700 1 -- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9238145140 msgr2=0x7f9238143860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:05.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f9236ffd700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9238145140 0x7f9238143860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:05.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f9236ffd700 1 -- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f92380b5820 msgr2=0x7f9238145ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:05.195 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f9236ffd700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f92380b5820 0x7f9238145ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:05.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f9236ffd700 1 -- 172.21.2.104:0/1983442863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9230009470 con 0x7f92380a8d60 2021-07-11T04:02:05.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f9236ffd700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f92380a8d60 0x7f9238147650 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f9238147e80 tx=0x7f9220015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:05.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f922ffff700 1 -- 172.21.2.104:0/1983442863 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9220021070 con 0x7f92380a8d60 2021-07-11T04:02:05.196 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f922ffff700 1 -- 172.21.2.104:0/1983442863 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9220000e90 con 0x7f92380a8d60 2021-07-11T04:02:05.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f923d8f9700 1 -- 172.21.2.104:0/1983442863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922000b1e0 con 0x7f92380a8d60 2021-07-11T04:02:05.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f923d8f9700 1 -- 172.21.2.104:0/1983442863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9238149870 con 0x7f92380a8d60 2021-07-11T04:02:05.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f922ffff700 1 -- 172.21.2.104:0/1983442863 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9220011950 con 0x7f92380a8d60 2021-07-11T04:02:05.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f923d8f9700 1 -- 172.21.2.104:0/1983442863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9238057350 con 0x7f92380a8d60 2021-07-11T04:02:05.197 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f922ffff700 1 -- 172.21.2.104:0/1983442863 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f922000f070 con 0x7f92380a8d60 2021-07-11T04:02:05.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f922ffff700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f921c053240 0x7f921c055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f922ffff700 1 -- 172.21.2.104:0/1983442863 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f922006c850 con 0x7f92380a8d60 2021-07-11T04:02:05.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f92367fc700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f921c053240 0x7f921c055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:05.198 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.188+0000 7f92367fc700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f921c053240 0x7f921c055700 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f9238060d80 tx=0x7f9228007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:05.199 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.192+0000 7f922ffff700 1 -- 172.21.2.104:0/1983442863 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9220043a80 con 0x7f92380a8d60 2021-07-11T04:02:05.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 -- 172.21.2.104:0/1983442863 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9238156b60 con 0x7f92380a8d60 2021-07-11T04:02:05.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f922ffff700 1 -- 172.21.2.104:0/1983442863 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2492 (secure 0 0 0) 0x7f92200470a0 con 0x7f92380a8d60 2021-07-11T04:02:05.297 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:05.297 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":68,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5335,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/1814518027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":1814518027},{"type":"v1","addr":"172.21.2.105:6835","nonce":1814518027}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/155502403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":155502403},{"type":"v1","addr":"172.21.2.104:6834","nonce":155502403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:52.876988+0000","modified":"2021-07-11T04:01:54.362899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8022},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8022":{"gid":8022,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/2616834266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":2616834266},{"type":"v1","addr":"172.21.2.104:6835","nonce":2616834266}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-07-11T04:02:05.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 -- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f921c053240 msgr2=0x7f921c055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:05.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f921c053240 0x7f921c055700 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f9238060d80 tx=0x7f9228007800).stop 2021-07-11T04:02:05.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 -- 172.21.2.104:0/1983442863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f92380a8d60 msgr2=0x7f9238147650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:05.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f92380a8d60 0x7f9238147650 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f9238147e80 tx=0x7f9220015000).stop 2021-07-11T04:02:05.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 -- 172.21.2.104:0/1983442863 shutdown_connections 2021-07-11T04:02:05.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9238145140 0x7f9238143860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:05.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f92380b5820 0x7f9238145ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:05.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f921c053240 0x7f921c055700 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:05.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 --2- 172.21.2.104:0/1983442863 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f92380a8d60 0x7f9238147650 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:05.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 -- 172.21.2.104:0/1983442863 >> 172.21.2.104:0/1983442863 conn(0x7f923805bd40 msgr2=0x7f92380b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:05.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 -- 172.21.2.104:0/1983442863 shutdown_connections 2021-07-11T04:02:05.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.296+0000 7f923d8f9700 1 -- 172.21.2.104:0/1983442863 wait complete. 2021-07-11T04:02:05.302 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 68 2021-07-11T04:02:05.304 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T04:02:05.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e4aa16700 1 Processor -- start 2021-07-11T04:02:05.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e4aa16700 1 -- start start 2021-07-11T04:02:05.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e4aa16700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e440b5820 0x7f1e440677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e4aa16700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1e44145140 0x7f1e440b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e4aa16700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e440b5c00 con 0x7f1e440a8d60 2021-07-11T04:02:05.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e4aa16700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e440b00b0 con 0x7f1e44145140 2021-07-11T04:02:05.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e4aa16700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440b0350 con 0x7f1e440b5820 2021-07-11T04:02:05.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e48fb3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1e44145140 0x7f1e440b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:05.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e48fb3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1e44145140 0x7f1e440b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50710/0 (socket says 172.21.2.104:50710) 2021-07-11T04:02:05.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e48fb3700 1 -- 172.21.2.104:0/3349238093 learned_addr learned my addr 172.21.2.104:0/3349238093 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:05.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e48fb3700 1 -- 172.21.2.104:0/3349238093 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e440b5820 msgr2=0x7f1e440677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:05.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e48fb3700 1 --2- 172.21.2.104:0/3349238093 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e440b5820 0x7f1e440677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:05.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e48fb3700 1 -- 172.21.2.104:0/3349238093 >> v1:172.21.2.105:6789/0 conn(0x7f1e440a8d60 legacy=0x7f1e44144a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:05.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e48fb3700 1 -- 172.21.2.104:0/3349238093 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440b05f0 con 0x7f1e44145140 2021-07-11T04:02:05.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e48fb3700 1 --2- 172.21.2.104:0/3349238093 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e44145140 0x7f1e440b1ff0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7f1e440b0c10 tx=0x7f1e3c009290).ready entity=mon.0 client_cookie=d0ddc225da76f7b8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:05.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e42ffd700 1 -- 172.21.2.104:0/3349238093 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e3c00b860 con 0x7f1e44145140 2021-07-11T04:02:05.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e42ffd700 1 -- 172.21.2.104:0/3349238093 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3c00b9c0 con 0x7f1e44145140 2021-07-11T04:02:05.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e42ffd700 1 -- 172.21.2.104:0/3349238093 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e3c00bb70 con 0x7f1e44145140 2021-07-11T04:02:05.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e4aa16700 1 -- 172.21.2.104:0/3349238093 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e44145140 msgr2=0x7f1e440b1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:05.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.356+0000 7f1e4aa16700 1 --2- 172.21.2.104:0/3349238093 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e44145140 0x7f1e440b1ff0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7f1e440b0c10 tx=0x7f1e3c009290).stop 2021-07-11T04:02:05.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 -- 172.21.2.104:0/3349238093 shutdown_connections 2021-07-11T04:02:05.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 --2- 172.21.2.104:0/3349238093 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e44145140 0x7f1e440b1ff0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:05.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 --2- 172.21.2.104:0/3349238093 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e440b5820 0x7f1e440677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:05.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 -- 172.21.2.104:0/3349238093 >> 172.21.2.104:0/3349238093 conn(0x7f1e4405bd40 msgr2=0x7f1e4405c150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:05.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 -- 172.21.2.104:0/3349238093 shutdown_connections 2021-07-11T04:02:05.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 -- 172.21.2.104:0/3349238093 wait complete. 2021-07-11T04:02:05.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 Processor -- start 2021-07-11T04:02:05.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 -- start start 2021-07-11T04:02:05.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1e440a8d60 0x7f1e440aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e440b5820 0x7f1e440ab3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e44145140 0x7f1e44148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e44157220 con 0x7f1e440b5820 2021-07-11T04:02:05.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e44157440 con 0x7f1e440a8d60 2021-07-11T04:02:05.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e441483e0 con 0x7f1e44145140 2021-07-11T04:02:05.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e437fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e440b5820 0x7f1e440ab3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:05.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e437fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e440b5820 0x7f1e440ab3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50716/0 (socket says 172.21.2.104:50716) 2021-07-11T04:02:05.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e437fe700 1 -- 172.21.2.104:0/2795765569 learned_addr learned my addr 172.21.2.104:0/2795765569 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:05.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e437fe700 1 -- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e44145140 msgr2=0x7f1e44148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:05.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e437fe700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e44145140 0x7f1e44148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:05.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e437fe700 1 -- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1e440a8d60 msgr2=0x7f1e440aaf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:05.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e437fe700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1e440a8d60 0x7f1e440aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:05.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e437fe700 1 -- 172.21.2.104:0/2795765569 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c009530 con 0x7f1e440b5820 2021-07-11T04:02:05.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e437fe700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e440b5820 0x7f1e440ab3f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f1e2c002f90 tx=0x7f1e2c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:05.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e417fa700 1 -- 172.21.2.104:0/2795765569 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e2c013460 con 0x7f1e440b5820 2021-07-11T04:02:05.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e417fa700 1 -- 172.21.2.104:0/2795765569 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c0135c0 con 0x7f1e440b5820 2021-07-11T04:02:05.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 -- 172.21.2.104:0/2795765569 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e440a93e0 con 0x7f1e440b5820 2021-07-11T04:02:05.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 -- 172.21.2.104:0/2795765569 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e440a9680 con 0x7f1e440b5820 2021-07-11T04:02:05.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e417fa700 1 -- 172.21.2.104:0/2795765569 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1e2c013920 con 0x7f1e440b5820 2021-07-11T04:02:05.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e4aa16700 1 -- 172.21.2.104:0/2795765569 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e44145770 con 0x7f1e440b5820 2021-07-11T04:02:05.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e417fa700 1 -- 172.21.2.104:0/2795765569 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1e2c01a060 con 0x7f1e440b5820 2021-07-11T04:02:05.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e417fa700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1e28053260 0x7f1e28055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:05.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.360+0000 7f1e417fa700 1 -- 172.21.2.104:0/2795765569 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1e2c06aeb0 con 0x7f1e440b5820 2021-07-11T04:02:05.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.364+0000 7f1e43fff700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1e28053260 0x7f1e28055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:05.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.364+0000 7f1e43fff700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1e28053260 0x7f1e28055720 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f1e34002f70 tx=0x7f1e34006a90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:05.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.364+0000 7f1e417fa700 1 -- 172.21.2.104:0/2795765569 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1e2c042120 con 0x7f1e440b5820 2021-07-11T04:02:05.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:05.472+0000 7f1e4aa16700 1 -- 172.21.2.104:0/2795765569 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1e44057350 con 0x7f1e440b5820 2021-07-11T04:02:06.403 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T04:02:06.400+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T04:02:06.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e417fa700 1 -- 172.21.2.104:0/2795765569 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1e2c018070 con 0x7f1e440b5820 2021-07-11T04:02:06.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 -- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1e28053260 msgr2=0x7f1e28055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:06.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1e28053260 0x7f1e28055720 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f1e34002f70 tx=0x7f1e34006a90).stop 2021-07-11T04:02:06.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 -- 172.21.2.104:0/2795765569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e440b5820 msgr2=0x7f1e440ab3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:06.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e440b5820 0x7f1e440ab3f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f1e2c002f90 tx=0x7f1e2c006df0).stop 2021-07-11T04:02:06.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 -- 172.21.2.104:0/2795765569 shutdown_connections 2021-07-11T04:02:06.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1e44145140 0x7f1e44148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:06.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1e28053260 0x7f1e28055720 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:06.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1e440b5820 0x7f1e440ab3f0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:06.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 --2- 172.21.2.104:0/2795765569 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1e440a8d60 0x7f1e440aaf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:06.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 -- 172.21.2.104:0/2795765569 >> 172.21.2.104:0/2795765569 conn(0x7f1e4405bd40 msgr2=0x7f1e440b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:06.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 -- 172.21.2.104:0/2795765569 shutdown_connections 2021-07-11T04:02:06.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.416+0000 7f1e4aa16700 1 -- 172.21.2.104:0/2795765569 wait complete. 2021-07-11T04:02:06.419 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T04:02:06.423 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:06.431 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T04:02:06.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.472+0000 7f631e725700 1 Processor -- start 2021-07-11T04:02:06.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.472+0000 7f631e725700 1 -- start start 2021-07-11T04:02:06.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f631805f9f0 0x7f63180b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f63180601a0 0x7f6318144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f631813e5d0 0x7f631813ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631813ef30 con 0x7f631813e5d0 2021-07-11T04:02:06.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6318065df0 con 0x7f63180601a0 2021-07-11T04:02:06.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63180ab420 con 0x7f631805f9f0 2021-07-11T04:02:06.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631ccc2700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f631813e5d0 0x7f631813ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:06.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631ccc2700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f631813e5d0 0x7f631813ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50738/0 (socket says 172.21.2.104:50738) 2021-07-11T04:02:06.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631ccc2700 1 -- 172.21.2.104:0/1562503144 learned_addr learned my addr 172.21.2.104:0/1562503144 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:06.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f63177fe700 1 --2- 172.21.2.104:0/1562503144 >> v2:172.21.2.105:3300/0 conn(0x7f63180601a0 0x7f6318144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:06.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6317fff700 1 --2- 172.21.2.104:0/1562503144 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f631805f9f0 0x7f63180b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:06.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631ccc2700 1 -- 172.21.2.104:0/1562503144 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f631805f9f0 msgr2=0x7f63180b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:06.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631ccc2700 1 --2- 172.21.2.104:0/1562503144 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f631805f9f0 0x7f63180b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:06.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631ccc2700 1 -- 172.21.2.104:0/1562503144 >> v2:172.21.2.105:3300/0 conn(0x7f63180601a0 msgr2=0x7f6318144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:06.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631ccc2700 1 --2- 172.21.2.104:0/1562503144 >> v2:172.21.2.105:3300/0 conn(0x7f63180601a0 0x7f6318144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:06.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631ccc2700 1 -- 172.21.2.104:0/1562503144 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631805fdd0 con 0x7f631813e5d0 2021-07-11T04:02:06.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631ccc2700 1 --2- 172.21.2.104:0/1562503144 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f631813e5d0 0x7f631813ea40 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f63180ab560 tx=0x7f6310009290).ready entity=mon.0 client_cookie=120b7a9c970fdae6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:06.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6316ffd700 1 -- 172.21.2.104:0/1562503144 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f631000b830 con 0x7f631813e5d0 2021-07-11T04:02:06.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6316ffd700 1 -- 172.21.2.104:0/1562503144 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631000b990 con 0x7f631813e5d0 2021-07-11T04:02:06.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6316ffd700 1 -- 172.21.2.104:0/1562503144 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f631000bb00 con 0x7f631813e5d0 2021-07-11T04:02:06.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- 172.21.2.104:0/1562503144 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f631813e5d0 msgr2=0x7f631813ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:06.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 --2- 172.21.2.104:0/1562503144 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f631813e5d0 0x7f631813ea40 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f63180ab560 tx=0x7f6310009290).stop 2021-07-11T04:02:06.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- 172.21.2.104:0/1562503144 shutdown_connections 2021-07-11T04:02:06.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 --2- 172.21.2.104:0/1562503144 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f631813e5d0 0x7f631813ea40 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:06.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 --2- 172.21.2.104:0/1562503144 >> v2:172.21.2.105:3300/0 conn(0x7f63180601a0 0x7f6318144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:06.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 --2- 172.21.2.104:0/1562503144 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f631805f9f0 0x7f63180b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:06.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- 172.21.2.104:0/1562503144 >> 172.21.2.104:0/1562503144 conn(0x7f63180b7790 msgr2=0x7f63180b7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:06.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- 172.21.2.104:0/1562503144 shutdown_connections 2021-07-11T04:02:06.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- 172.21.2.104:0/1562503144 wait complete. 2021-07-11T04:02:06.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 Processor -- start 2021-07-11T04:02:06.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- start start 2021-07-11T04:02:06.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f631805f9f0 0x7f63180b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f63180601a0 0x7f6318138a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f631813e5d0 0x7f6318138f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6318067360 con 0x7f631805f9f0 2021-07-11T04:02:06.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631813b690 con 0x7f63180601a0 2021-07-11T04:02:06.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f631e725700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631813f180 con 0x7f631813e5d0 2021-07-11T04:02:06.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6317fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f631805f9f0 0x7f63180b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:06.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6317fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f631805f9f0 0x7f63180b56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50744/0 (socket says 172.21.2.104:50744) 2021-07-11T04:02:06.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6317fff700 1 -- 172.21.2.104:0/518002822 learned_addr learned my addr 172.21.2.104:0/518002822 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:06.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6317fff700 1 -- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f631813e5d0 msgr2=0x7f6318138f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:06.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6317fff700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f631813e5d0 0x7f6318138f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:06.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6317fff700 1 -- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f63180601a0 msgr2=0x7f6318138a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:06.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6317fff700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f63180601a0 0x7f6318138a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:06.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6317fff700 1 -- 172.21.2.104:0/518002822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6310009530 con 0x7f631805f9f0 2021-07-11T04:02:06.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.476+0000 7f6317fff700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f631805f9f0 0x7f63180b56d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f6318141ec0 tx=0x7f6308008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:06.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f63157fa700 1 -- 172.21.2.104:0/518002822 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6308011780 con 0x7f631805f9f0 2021-07-11T04:02:06.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f63157fa700 1 -- 172.21.2.104:0/518002822 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63080118e0 con 0x7f631805f9f0 2021-07-11T04:02:06.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f631e725700 1 -- 172.21.2.104:0/518002822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631813f420 con 0x7f631805f9f0 2021-07-11T04:02:06.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f631e725700 1 -- 172.21.2.104:0/518002822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6318139410 con 0x7f631805f9f0 2021-07-11T04:02:06.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f63157fa700 1 -- 172.21.2.104:0/518002822 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6308011c40 con 0x7f631805f9f0 2021-07-11T04:02:06.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f631e725700 1 -- 172.21.2.104:0/518002822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6318057350 con 0x7f631805f9f0 2021-07-11T04:02:06.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f63157fa700 1 -- 172.21.2.104:0/518002822 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f630801c460 con 0x7f631805f9f0 2021-07-11T04:02:06.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f63157fa700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f62f80532d0 0x7f62f8055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f63157fa700 1 -- 172.21.2.104:0/518002822 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f630806b850 con 0x7f631805f9f0 2021-07-11T04:02:06.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f63177fe700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f62f80532d0 0x7f62f8055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:06.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f63177fe700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f62f80532d0 0x7f62f8055790 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f630c00a000 tx=0x7f630c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:06.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.480+0000 7f63157fa700 1 -- 172.21.2.104:0/518002822 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f63080429d0 con 0x7f631805f9f0 2021-07-11T04:02:06.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 -- 172.21.2.104:0/518002822 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6318069990 con 0x7f631805f9f0 2021-07-11T04:02:06.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f63157fa700 1 -- 172.21.2.104:0/518002822 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+2132 (secure 0 0 0) 0x7f6308045ff0 con 0x7f631805f9f0 2021-07-11T04:02:06.592 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:06.592 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":69,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5335,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/1814518027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":1814518027},{"type":"v1","addr":"172.21.2.105:6835","nonce":1814518027}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/155502403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":155502403},{"type":"v1","addr":"172.21.2.104:6834","nonce":155502403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65}],"filesystems":[{"mdsmap":{"epoch":69,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:01:52.876988+0000","modified":"2021-07-11T04:02:06.404400+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":140,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":13}]} 2021-07-11T04:02:06.593 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 -- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f62f80532d0 msgr2=0x7f62f8055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:06.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f62f80532d0 0x7f62f8055790 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f630c00a000 tx=0x7f630c008040).stop 2021-07-11T04:02:06.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 -- 172.21.2.104:0/518002822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f631805f9f0 msgr2=0x7f63180b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:06.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f631805f9f0 0x7f63180b56d0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f6318141ec0 tx=0x7f6308008040).stop 2021-07-11T04:02:06.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 -- 172.21.2.104:0/518002822 shutdown_connections 2021-07-11T04:02:06.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f631813e5d0 0x7f6318138f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:06.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f63180601a0 0x7f6318138a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:06.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f62f80532d0 0x7f62f8055790 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:06.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 --2- 172.21.2.104:0/518002822 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f631805f9f0 0x7f63180b56d0 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:06.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 -- 172.21.2.104:0/518002822 >> 172.21.2.104:0/518002822 conn(0x7f63180b7790 msgr2=0x7f63180b62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:06.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 -- 172.21.2.104:0/518002822 shutdown_connections 2021-07-11T04:02:06.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.592+0000 7f631e725700 1 -- 172.21.2.104:0/518002822 wait complete. 2021-07-11T04:02:06.597 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 69 2021-07-11T04:02:06.599 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T04:02:06.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 Processor -- start 2021-07-11T04:02:06.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 -- start start 2021-07-11T04:02:06.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f65480b5820 0x7f65480677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6548145140 0x7f65480b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65480b5c00 con 0x7f65480a8d60 2021-07-11T04:02:06.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65480b00b0 con 0x7f65480b5820 2021-07-11T04:02:06.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65480b0350 con 0x7f6548145140 2021-07-11T04:02:06.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654d560700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f65480b5820 0x7f65480677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:06.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654cd5f700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f65480a8d60 0x7f6548144a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47522/0 (socket says 172.21.2.104:47522) 2021-07-11T04:02:06.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654cd5f700 1 -- 172.21.2.104:0/4288470472 learned_addr learned my addr 172.21.2.104:0/4288470472 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:06.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654d560700 1 -- 172.21.2.104:0/4288470472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6548145140 msgr2=0x7f65480b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:06.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654d560700 1 --2- 172.21.2.104:0/4288470472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6548145140 0x7f65480b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:06.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654d560700 1 -- 172.21.2.104:0/4288470472 >> v1:172.21.2.104:6789/0 conn(0x7f65480a8d60 legacy=0x7f6548144a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:06.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654d560700 1 -- 172.21.2.104:0/4288470472 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65480b05f0 con 0x7f65480b5820 2021-07-11T04:02:06.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654d560700 1 --2- 172.21.2.104:0/4288470472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65480b5820 0x7f65480677f0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f6544002f70 tx=0x7f6544006df0).ready entity=mon.0 client_cookie=11d483cb66df8f44 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:06.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f653bfff700 1 -- 172.21.2.104:0/4288470472 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f654400d570 con 0x7f65480b5820 2021-07-11T04:02:06.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f653bfff700 1 -- 172.21.2.104:0/4288470472 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6544000e90 con 0x7f65480b5820 2021-07-11T04:02:06.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f653bfff700 1 -- 172.21.2.104:0/4288470472 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f654400d8c0 con 0x7f65480b5820 2021-07-11T04:02:06.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 -- 172.21.2.104:0/4288470472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65480b5820 msgr2=0x7f65480677f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:06.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 --2- 172.21.2.104:0/4288470472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65480b5820 0x7f65480677f0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7f6544002f70 tx=0x7f6544006df0).stop 2021-07-11T04:02:06.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 -- 172.21.2.104:0/4288470472 shutdown_connections 2021-07-11T04:02:06.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 --2- 172.21.2.104:0/4288470472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6548145140 0x7f65480b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:06.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 --2- 172.21.2.104:0/4288470472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65480b5820 0x7f65480677f0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:06.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 -- 172.21.2.104:0/4288470472 >> 172.21.2.104:0/4288470472 conn(0x7f654805bd40 msgr2=0x7f654805c150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:06.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 -- 172.21.2.104:0/4288470472 shutdown_connections 2021-07-11T04:02:06.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.652+0000 7f654f7c4700 1 -- 172.21.2.104:0/4288470472 wait complete. 2021-07-11T04:02:06.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654f7c4700 1 Processor -- start 2021-07-11T04:02:06.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654f7c4700 1 -- start start 2021-07-11T04:02:06.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654f7c4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65480a8d60 0x7f65481478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654f7c4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65480b5820 0x7f65481496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654f7c4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6548145140 0x7f6548149bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654f7c4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6548157230 con 0x7f6548145140 2021-07-11T04:02:06.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654f7c4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65481491f0 con 0x7f65480a8d60 2021-07-11T04:02:06.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654f7c4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6548148f50 con 0x7f65480b5820 2021-07-11T04:02:06.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654dd61700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6548145140 0x7f6548149bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:06.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654dd61700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6548145140 0x7f6548149bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50762/0 (socket says 172.21.2.104:50762) 2021-07-11T04:02:06.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654dd61700 1 -- 172.21.2.104:0/972016078 learned_addr learned my addr 172.21.2.104:0/972016078 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:06.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654dd61700 1 -- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65480b5820 msgr2=0x7f65481496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:06.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654dd61700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65480b5820 0x7f65481496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:06.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654dd61700 1 -- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65480a8d60 msgr2=0x7f65481478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:06.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654dd61700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65480a8d60 0x7f65481478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:06.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654dd61700 1 -- 172.21.2.104:0/972016078 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654400b180 con 0x7f6548145140 2021-07-11T04:02:06.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654dd61700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6548145140 0x7f6548149bd0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f65400032f0 tx=0x7f65400091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:06.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f653a7fc700 1 -- 172.21.2.104:0/972016078 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f654000d4a0 con 0x7f6548145140 2021-07-11T04:02:06.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f653a7fc700 1 -- 172.21.2.104:0/972016078 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654000d600 con 0x7f6548145140 2021-07-11T04:02:06.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654f7c4700 1 -- 172.21.2.104:0/972016078 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6548157430 con 0x7f6548145140 2021-07-11T04:02:06.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f653a7fc700 1 -- 172.21.2.104:0/972016078 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f654000d960 con 0x7f6548145140 2021-07-11T04:02:06.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654f7c4700 1 -- 172.21.2.104:0/972016078 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654814a110 con 0x7f6548145140 2021-07-11T04:02:06.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654f7c4700 1 -- 172.21.2.104:0/972016078 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6548057350 con 0x7f6548145140 2021-07-11T04:02:06.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f653a7fc700 1 -- 172.21.2.104:0/972016078 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6540018030 con 0x7f6548145140 2021-07-11T04:02:06.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f653a7fc700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6528053300 0x7f65280557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:06.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f653a7fc700 1 -- 172.21.2.104:0/972016078 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f654006ae10 con 0x7f6548145140 2021-07-11T04:02:06.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.656+0000 7f654d560700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6528053300 0x7f65280557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:06.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.660+0000 7f654d560700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6528053300 0x7f65280557c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f6544002f70 tx=0x7f6544015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:06.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.660+0000 7f653a7fc700 1 -- 172.21.2.104:0/972016078 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f65400421b0 con 0x7f6548145140 2021-07-11T04:02:06.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:06.768+0000 7f654f7c4700 1 -- 172.21.2.104:0/972016078 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6548069990 con 0x7f6548145140 2021-07-11T04:02:07.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f653a7fc700 1 -- 172.21.2.104:0/972016078 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7f65400457d0 con 0x7f6548145140 2021-07-11T04:02:07.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 -- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6528053300 msgr2=0x7f65280557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:07.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6528053300 0x7f65280557c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f6544002f70 tx=0x7f6544015000).stop 2021-07-11T04:02:07.424 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 -- 172.21.2.104:0/972016078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6548145140 msgr2=0x7f6548149bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:07.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6548145140 0x7f6548149bd0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f65400032f0 tx=0x7f65400091a0).stop 2021-07-11T04:02:07.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 -- 172.21.2.104:0/972016078 shutdown_connections 2021-07-11T04:02:07.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6528053300 0x7f65280557c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:07.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6548145140 0x7f6548149bd0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:07.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65480b5820 0x7f65481496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:07.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 --2- 172.21.2.104:0/972016078 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65480a8d60 0x7f65481478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:07.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 -- 172.21.2.104:0/972016078 >> 172.21.2.104:0/972016078 conn(0x7f654805bd40 msgr2=0x7f65480b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:07.426 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 -- 172.21.2.104:0/972016078 shutdown_connections 2021-07-11T04:02:07.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.424+0000 7f654f7c4700 1 -- 172.21.2.104:0/972016078 wait complete. 2021-07-11T04:02:07.431 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T04:02:07.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.484+0000 7f6fda613700 1 Processor -- start 2021-07-11T04:02:07.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.484+0000 7f6fda613700 1 -- start start 2021-07-11T04:02:07.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.484+0000 7f6fda613700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6fd40b5800 0x7f6fd40af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:07.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.484+0000 7f6fda613700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6fd40afed0 0x7f6fd40b0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:07.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.484+0000 7f6fda613700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd40b5be0 con 0x7f6fd4146a30 2021-07-11T04:02:07.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.484+0000 7f6fda613700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd40a9030 con 0x7f6fd40b5800 2021-07-11T04:02:07.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.484+0000 7f6fda613700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd40b0f10 con 0x7f6fd40afed0 2021-07-11T04:02:07.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.484+0000 7f6fd3fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6fd40b5800 0x7f6fd40af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:07.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.484+0000 7f6fd3fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6fd40b5800 0x7f6fd40af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50766/0 (socket says 172.21.2.104:50766) 2021-07-11T04:02:07.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.484+0000 7f6fd37fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6fd4146a30 0x7f6fd4146eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47536/0 (socket says 172.21.2.104:47536) 2021-07-11T04:02:07.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.484+0000 7f6fd3fff700 1 -- 172.21.2.104:0/1418615635 learned_addr learned my addr 172.21.2.104:0/1418615635 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:07.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 -- 172.21.2.104:0/1418615635 >> v2:172.21.2.105:3300/0 conn(0x7f6fd40afed0 msgr2=0x7f6fd40b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:07.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 --2- 172.21.2.104:0/1418615635 >> v2:172.21.2.105:3300/0 conn(0x7f6fd40afed0 0x7f6fd40b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:07.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 -- 172.21.2.104:0/1418615635 >> v1:172.21.2.104:6789/0 conn(0x7f6fd4146a30 legacy=0x7f6fd4146eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:07.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 -- 172.21.2.104:0/1418615635 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd40b11b0 con 0x7f6fd40b5800 2021-07-11T04:02:07.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 --2- 172.21.2.104:0/1418615635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fd40b5800 0x7f6fd40af430 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f6fc4002f70 tx=0x7f6fc4006eb0).ready entity=mon.0 client_cookie=ecfe959b48295509 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:07.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd2ffd700 1 -- 172.21.2.104:0/1418615635 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fc400d610 con 0x7f6fd40b5800 2021-07-11T04:02:07.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd2ffd700 1 -- 172.21.2.104:0/1418615635 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc4000e90 con 0x7f6fd40b5800 2021-07-11T04:02:07.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd2ffd700 1 -- 172.21.2.104:0/1418615635 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fc400d9a0 con 0x7f6fd40b5800 2021-07-11T04:02:07.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 -- 172.21.2.104:0/1418615635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fd40b5800 msgr2=0x7f6fd40af430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:07.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 --2- 172.21.2.104:0/1418615635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fd40b5800 0x7f6fd40af430 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f6fc4002f70 tx=0x7f6fc4006eb0).stop 2021-07-11T04:02:07.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 -- 172.21.2.104:0/1418615635 shutdown_connections 2021-07-11T04:02:07.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 --2- 172.21.2.104:0/1418615635 >> v2:172.21.2.105:3300/0 conn(0x7f6fd40afed0 0x7f6fd40b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:07.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 --2- 172.21.2.104:0/1418615635 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fd40b5800 0x7f6fd40af430 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:07.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 -- 172.21.2.104:0/1418615635 >> 172.21.2.104:0/1418615635 conn(0x7f6fd405bd40 msgr2=0x7f6fd405c120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:07.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 -- 172.21.2.104:0/1418615635 shutdown_connections 2021-07-11T04:02:07.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 -- 172.21.2.104:0/1418615635 wait complete. 2021-07-11T04:02:07.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 Processor -- start 2021-07-11T04:02:07.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 -- start start 2021-07-11T04:02:07.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fd40afed0 0x7f6fd414d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:07.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fd40b5800 0x7f6fd414ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:07.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fd4146a30 0x7f6fd414b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:07.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd4140cd0 con 0x7f6fd40afed0 2021-07-11T04:02:07.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd4148fb0 con 0x7f6fd4146a30 2021-07-11T04:02:07.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd40632a0 con 0x7f6fd40b5800 2021-07-11T04:02:07.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fd40afed0 0x7f6fd414d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:07.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fd40afed0 0x7f6fd414d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50774/0 (socket says 172.21.2.104:50774) 2021-07-11T04:02:07.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 -- 172.21.2.104:0/1595850343 learned_addr learned my addr 172.21.2.104:0/1595850343 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:07.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 -- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fd40b5800 msgr2=0x7f6fd414ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:07.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fd40b5800 0x7f6fd414ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:07.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 -- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fd4146a30 msgr2=0x7f6fd414b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:07.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fd4146a30 0x7f6fd414b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:07.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 -- 172.21.2.104:0/1595850343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc400b1e0 con 0x7f6fd40afed0 2021-07-11T04:02:07.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd3fff700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fd40afed0 0x7f6fd414d2d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f6fc4002f70 tx=0x7f6fc4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:07.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd17fa700 1 -- 172.21.2.104:0/1595850343 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fc4021070 con 0x7f6fd40afed0 2021-07-11T04:02:07.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd17fa700 1 -- 172.21.2.104:0/1595850343 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc4000e90 con 0x7f6fd40afed0 2021-07-11T04:02:07.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fd17fa700 1 -- 172.21.2.104:0/1595850343 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fc400dea0 con 0x7f6fd40afed0 2021-07-11T04:02:07.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 -- 172.21.2.104:0/1595850343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd41494d0 con 0x7f6fd40afed0 2021-07-11T04:02:07.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.488+0000 7f6fda613700 1 -- 172.21.2.104:0/1595850343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd4149700 con 0x7f6fd40afed0 2021-07-11T04:02:07.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.492+0000 7f6fda613700 1 -- 172.21.2.104:0/1595850343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd4149b40 con 0x7f6fd40afed0 2021-07-11T04:02:07.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.492+0000 7f6fd17fa700 1 -- 172.21.2.104:0/1595850343 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6fc400f070 con 0x7f6fd40afed0 2021-07-11T04:02:07.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.492+0000 7f6fd17fa700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb80532b0 0x7f6fb8055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:07.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.492+0000 7f6fd17fa700 1 -- 172.21.2.104:0/1595850343 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f6fc406c8a0 con 0x7f6fd40afed0 2021-07-11T04:02:07.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.492+0000 7f6fd37fe700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb80532b0 0x7f6fb8055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:07.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.492+0000 7f6fd37fe700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb80532b0 0x7f6fb8055770 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f6fd40b1530 tx=0x7f6fbc007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:07.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.492+0000 7f6fd17fa700 1 -- 172.21.2.104:0/1595850343 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6fc4043b90 con 0x7f6fd40afed0 2021-07-11T04:02:07.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:07.576+0000 7f6fda613700 1 -- 172.21.2.104:0/1595850343 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6fd4057350 con 0x7f6fd40afed0 2021-07-11T04:02:08.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.424+0000 7f6fd17fa700 1 -- 172.21.2.104:0/1595850343 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v141) v1 ==== 185+0+0 (secure 0 0 0) 0x7f6fc40471b0 con 0x7f6fd40afed0 2021-07-11T04:02:08.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 -- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb80532b0 msgr2=0x7f6fb8055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:08.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb80532b0 0x7f6fb8055770 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f6fd40b1530 tx=0x7f6fbc007800).stop 2021-07-11T04:02:08.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 -- 172.21.2.104:0/1595850343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fd40afed0 msgr2=0x7f6fd414d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:08.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fd40afed0 0x7f6fd414d2d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f6fc4002f70 tx=0x7f6fc4015000).stop 2021-07-11T04:02:08.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 -- 172.21.2.104:0/1595850343 shutdown_connections 2021-07-11T04:02:08.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fd4146a30 0x7f6fd414b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:08.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fd40b5800 0x7f6fd414ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:08.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fb80532b0 0x7f6fb8055770 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:08.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 --2- 172.21.2.104:0/1595850343 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fd40afed0 0x7f6fd414d2d0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:08.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 -- 172.21.2.104:0/1595850343 >> 172.21.2.104:0/1595850343 conn(0x7f6fd405bd40 msgr2=0x7f6fd40b0340 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:08.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 -- 172.21.2.104:0/1595850343 shutdown_connections 2021-07-11T04:02:08.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.428+0000 7f6fda613700 1 -- 172.21.2.104:0/1595850343 wait complete. 2021-07-11T04:02:08.432 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T04:02:08.438 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T04:02:08.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 Processor -- start 2021-07-11T04:02:08.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 -- start start 2021-07-11T04:02:08.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f09140b41a0 0x7f091405bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:08.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f09140673f0 0x7f09141418a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:08.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091405cd20 con 0x7f09140b2110 2021-07-11T04:02:08.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09140b4c90 con 0x7f09140673f0 2021-07-11T04:02:08.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09140aad60 con 0x7f09140b41a0 2021-07-11T04:02:08.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091259c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f09140b2110 0x7f09140b4580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47550/0 (socket says 172.21.2.104:47550) 2021-07-11T04:02:08.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091259c700 1 -- 172.21.2.104:0/2937410941 learned_addr learned my addr 172.21.2.104:0/2937410941 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:08.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091359e700 1 --2- 172.21.2.104:0/2937410941 >> v2:172.21.2.104:3300/0 conn(0x7f09140673f0 0x7f09141418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:08.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091359e700 1 -- 172.21.2.104:0/2937410941 >> v2:172.21.2.105:3300/0 conn(0x7f09140b41a0 msgr2=0x7f091405bd40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:08.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091359e700 1 --2- 172.21.2.104:0/2937410941 >> v2:172.21.2.105:3300/0 conn(0x7f09140b41a0 0x7f091405bd40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:08.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091359e700 1 -- 172.21.2.104:0/2937410941 >> v1:172.21.2.104:6789/0 conn(0x7f09140b2110 legacy=0x7f09140b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:08.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091359e700 1 -- 172.21.2.104:0/2937410941 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09140ab000 con 0x7f09140673f0 2021-07-11T04:02:08.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f0911d9b700 1 -- 172.21.2.104:0/2937410941 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721182249 0 0) 0x7f091405cd20 con 0x7f09140b2110 2021-07-11T04:02:08.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091359e700 1 --2- 172.21.2.104:0/2937410941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09140673f0 0x7f09141418a0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f090c0032f0 tx=0x7f090c0091d0).ready entity=mon.0 client_cookie=ea4c85ade28a70c4 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:08.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f0911d9b700 1 -- 172.21.2.104:0/2937410941 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090c009ea0 con 0x7f09140673f0 2021-07-11T04:02:08.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f0911d9b700 1 -- 172.21.2.104:0/2937410941 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c00b870 con 0x7f09140673f0 2021-07-11T04:02:08.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f0911d9b700 1 -- 172.21.2.104:0/2937410941 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090c00ba20 con 0x7f09140673f0 2021-07-11T04:02:08.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 -- 172.21.2.104:0/2937410941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09140673f0 msgr2=0x7f09141418a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:08.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 --2- 172.21.2.104:0/2937410941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09140673f0 0x7f09141418a0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f090c0032f0 tx=0x7f090c0091d0).stop 2021-07-11T04:02:08.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 -- 172.21.2.104:0/2937410941 shutdown_connections 2021-07-11T04:02:08.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 --2- 172.21.2.104:0/2937410941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09140673f0 0x7f09141418a0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:08.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 --2- 172.21.2.104:0/2937410941 >> v2:172.21.2.105:3300/0 conn(0x7f09140b41a0 0x7f091405bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:08.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 -- 172.21.2.104:0/2937410941 >> 172.21.2.104:0/2937410941 conn(0x7f09140b7770 msgr2=0x7f09140b7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:08.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 -- 172.21.2.104:0/2937410941 shutdown_connections 2021-07-11T04:02:08.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 -- 172.21.2.104:0/2937410941 wait complete. 2021-07-11T04:02:08.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 Processor -- start 2021-07-11T04:02:08.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.492+0000 7f091913a700 1 -- start start 2021-07-11T04:02:08.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091913a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09140673f0 0x7f09140adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:08.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091913a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09140b2110 0x7f09140ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:08.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091913a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09140b41a0 0x7f091413edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:08.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091913a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091414e9b0 con 0x7f09140b2110 2021-07-11T04:02:08.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091913a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091413d2b0 con 0x7f09140b41a0 2021-07-11T04:02:08.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091913a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091413c140 con 0x7f09140673f0 2021-07-11T04:02:08.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09140b2110 0x7f09140ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:08.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09140b2110 0x7f09140ae160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50786/0 (socket says 172.21.2.104:50786) 2021-07-11T04:02:08.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091259c700 1 -- 172.21.2.104:0/2013196162 learned_addr learned my addr 172.21.2.104:0/2013196162 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:08.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091259c700 1 -- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09140673f0 msgr2=0x7f09140adc70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:08.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091259c700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09140673f0 0x7f09140adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:08.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091259c700 1 -- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09140b41a0 msgr2=0x7f091413edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:08.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091259c700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09140b41a0 0x7f091413edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:08.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091259c700 1 -- 172.21.2.104:0/2013196162 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c009470 con 0x7f09140b2110 2021-07-11T04:02:08.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091259c700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09140b2110 0x7f09140ae160 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f0904000bc0 tx=0x7f0904007a80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:08.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f0903fff700 1 -- 172.21.2.104:0/2013196162 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f090400ce50 con 0x7f09140b2110 2021-07-11T04:02:08.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f0903fff700 1 -- 172.21.2.104:0/2013196162 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09040072e0 con 0x7f09140b2110 2021-07-11T04:02:08.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091913a700 1 -- 172.21.2.104:0/2013196162 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091413cd80 con 0x7f09140b2110 2021-07-11T04:02:08.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091913a700 1 -- 172.21.2.104:0/2013196162 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091413bb10 con 0x7f09140b2110 2021-07-11T04:02:08.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f0903fff700 1 -- 172.21.2.104:0/2013196162 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0904005800 con 0x7f09140b2110 2021-07-11T04:02:08.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f091913a700 1 -- 172.21.2.104:0/2013196162 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091413c870 con 0x7f09140b2110 2021-07-11T04:02:08.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f0903fff700 1 -- 172.21.2.104:0/2013196162 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f090401a060 con 0x7f09140b2110 2021-07-11T04:02:08.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f0903fff700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08f0053210 0x7f08f00556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:08.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f0903fff700 1 -- 172.21.2.104:0/2013196162 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f090406ab30 con 0x7f09140b2110 2021-07-11T04:02:08.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f0912d9d700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08f0053210 0x7f08f00556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:08.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.496+0000 7f0912d9d700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08f0053210 0x7f08f00556d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f0908000ed0 tx=0x7f0908006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:08.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.500+0000 7f0903fff700 1 -- 172.21.2.104:0/2013196162 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0904041fd0 con 0x7f09140b2110 2021-07-11T04:02:08.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:08.580+0000 7f091913a700 1 -- 172.21.2.104:0/2013196162 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f091414eb30 con 0x7f09140b2110 2021-07-11T04:02:09.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f0903fff700 1 -- 172.21.2.104:0/2013196162 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v142) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0904018020 con 0x7f09140b2110 2021-07-11T04:02:09.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 -- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08f0053210 msgr2=0x7f08f00556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:09.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08f0053210 0x7f08f00556d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f0908000ed0 tx=0x7f0908006930).stop 2021-07-11T04:02:09.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 -- 172.21.2.104:0/2013196162 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09140b2110 msgr2=0x7f09140ae160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:09.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09140b2110 0x7f09140ae160 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f0904000bc0 tx=0x7f0904007a80).stop 2021-07-11T04:02:09.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 -- 172.21.2.104:0/2013196162 shutdown_connections 2021-07-11T04:02:09.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f09140b41a0 0x7f091413edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:09.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f08f0053210 0x7f08f00556d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f09140b2110 0x7f09140ae160 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 --2- 172.21.2.104:0/2013196162 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f09140673f0 0x7f09140adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:09.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 -- 172.21.2.104:0/2013196162 >> 172.21.2.104:0/2013196162 conn(0x7f09140b7770 msgr2=0x7f091413d770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:09.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 -- 172.21.2.104:0/2013196162 shutdown_connections 2021-07-11T04:02:09.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.436+0000 7f091913a700 1 -- 172.21.2.104:0/2013196162 wait complete. 2021-07-11T04:02:09.440 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T04:02:09.445 DEBUG:teuthology.orchestra.run.gibba004:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T04:02:09.447 DEBUG:teuthology.orchestra.run.gibba005:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T04:02:09.450 DEBUG:teuthology.orchestra.run.gibba019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T04:02:09.453 DEBUG:teuthology.orchestra.run.gibba023:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2021-07-11T04:02:09.458 INFO:teuthology.orchestra.run.gibba004.stdout:[global] 2021-07-11T04:02:09.459 INFO:teuthology.orchestra.run.gibba004.stdout: chdir = "" 2021-07-11T04:02:09.459 INFO:teuthology.orchestra.run.gibba004.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T04:02:09.459 INFO:teuthology.orchestra.run.gibba004.stdout: auth supported = cephx 2021-07-11T04:02:09.459 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.460 INFO:teuthology.orchestra.run.gibba004.stdout: filestore xattr use omap = true 2021-07-11T04:02:09.460 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.460 INFO:teuthology.orchestra.run.gibba004.stdout: mon clock drift allowed = 1.000 2021-07-11T04:02:09.460 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.461 INFO:teuthology.orchestra.run.gibba004.stdout: osd crush chooseleaf type = 0 2021-07-11T04:02:09.461 INFO:teuthology.orchestra.run.gibba004.stdout: auth debug = true 2021-07-11T04:02:09.461 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.461 INFO:teuthology.orchestra.run.gibba004.stdout: ms die on old message = true 2021-07-11T04:02:09.462 INFO:teuthology.orchestra.run.gibba004.stdout: ms die on bug = true 2021-07-11T04:02:09.462 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.462 INFO:teuthology.orchestra.run.gibba004.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T04:02:09.462 INFO:teuthology.orchestra.run.gibba004.stdout: mon pg warn max object skew = 0 2021-07-11T04:02:09.463 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.463 INFO:teuthology.orchestra.run.gibba004.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T04:02:09.463 INFO:teuthology.orchestra.run.gibba004.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T04:02:09.463 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.463 INFO:teuthology.orchestra.run.gibba004.stdout: osd pool default size = 2 2021-07-11T04:02:09.464 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.464 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd allow primary affinity = true 2021-07-11T04:02:09.464 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd allow pg remap = true 2021-07-11T04:02:09.464 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on legacy crush tunables = false 2021-07-11T04:02:09.464 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on crush straw calc version zero = false 2021-07-11T04:02:09.465 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on no sortbitwise = false 2021-07-11T04:02:09.465 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on osd down out interval zero = false 2021-07-11T04:02:09.465 INFO:teuthology.orchestra.run.gibba004.stdout: mon warn on too few osds = false 2021-07-11T04:02:09.465 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T04:02:09.466 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T04:02:09.466 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.466 INFO:teuthology.orchestra.run.gibba004.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T04:02:09.466 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.466 INFO:teuthology.orchestra.run.gibba004.stdout: osd default data pool replay window = 5 2021-07-11T04:02:09.467 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.467 INFO:teuthology.orchestra.run.gibba004.stdout: mon allow pool delete = true 2021-07-11T04:02:09.467 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.467 INFO:teuthology.orchestra.run.gibba004.stdout: mon cluster log file level = debug 2021-07-11T04:02:09.468 INFO:teuthology.orchestra.run.gibba004.stdout: debug asserts on shutdown = true 2021-07-11T04:02:09.468 INFO:teuthology.orchestra.run.gibba004.stdout: mon health detail to clog = false 2021-07-11T04:02:09.468 INFO:teuthology.orchestra.run.gibba004.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T04:02:09.468 INFO:teuthology.orchestra.run.gibba004.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T04:02:09.469 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.469 INFO:teuthology.orchestra.run.gibba004.stdout:[osd] 2021-07-11T04:02:09.469 INFO:teuthology.orchestra.run.gibba004.stdout: osd journal size = 100 2021-07-11T04:02:09.469 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.469 INFO:teuthology.orchestra.run.gibba004.stdout: osd scrub load threshold = 5.0 2021-07-11T04:02:09.470 INFO:teuthology.orchestra.run.gibba004.stdout: osd scrub max interval = 600 2021-07-11T04:02:09.470 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.470 INFO:teuthology.orchestra.run.gibba004.stdout: osd recover clone overlap = true 2021-07-11T04:02:09.470 INFO:teuthology.orchestra.run.gibba004.stdout: osd recovery max chunk = 1048576 2021-07-11T04:02:09.471 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.471 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug shutdown = true 2021-07-11T04:02:09.471 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug op order = true 2021-07-11T04:02:09.471 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug verify stray on activate = true 2021-07-11T04:02:09.471 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.472 INFO:teuthology.orchestra.run.gibba004.stdout: osd open classes on start = true 2021-07-11T04:02:09.472 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug pg log writeout = true 2021-07-11T04:02:09.472 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.472 INFO:teuthology.orchestra.run.gibba004.stdout: osd deep scrub update digest min age = 30 2021-07-11T04:02:09.472 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.473 INFO:teuthology.orchestra.run.gibba004.stdout: osd map max advance = 10 2021-07-11T04:02:09.473 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.473 INFO:teuthology.orchestra.run.gibba004.stdout: journal zero on create = true 2021-07-11T04:02:09.473 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.473 INFO:teuthology.orchestra.run.gibba004.stdout: filestore ondisk finisher threads = 3 2021-07-11T04:02:09.474 INFO:teuthology.orchestra.run.gibba004.stdout: filestore apply finisher threads = 3 2021-07-11T04:02:09.474 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.474 INFO:teuthology.orchestra.run.gibba004.stdout: bdev debug aio = true 2021-07-11T04:02:09.474 INFO:teuthology.orchestra.run.gibba004.stdout: osd debug misdirected ops = true 2021-07-11T04:02:09.475 INFO:teuthology.orchestra.run.gibba004.stdout: bluestore block size = 96636764160 2021-07-11T04:02:09.475 INFO:teuthology.orchestra.run.gibba004.stdout: bluestore compression mode = aggressive 2021-07-11T04:02:09.475 INFO:teuthology.orchestra.run.gibba004.stdout: bluestore fsck on mount = True 2021-07-11T04:02:09.475 INFO:teuthology.orchestra.run.gibba004.stdout: debug bluefs = 20 2021-07-11T04:02:09.476 INFO:teuthology.orchestra.run.gibba004.stdout: debug bluestore = 20 2021-07-11T04:02:09.476 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T04:02:09.476 INFO:teuthology.orchestra.run.gibba004.stdout: debug osd = 20 2021-07-11T04:02:09.476 INFO:teuthology.orchestra.run.gibba004.stdout: debug rocksdb = 10 2021-07-11T04:02:09.476 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T04:02:09.477 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd full ratio = 0.9 2021-07-11T04:02:09.477 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd nearfull ratio = 0.8 2021-07-11T04:02:09.477 INFO:teuthology.orchestra.run.gibba004.stdout: osd failsafe full ratio = 0.95 2021-07-11T04:02:09.477 INFO:teuthology.orchestra.run.gibba004.stdout: osd objectstore = bluestore 2021-07-11T04:02:09.478 INFO:teuthology.orchestra.run.gibba004.stdout: osd op complaint time = 180 2021-07-11T04:02:09.478 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.478 INFO:teuthology.orchestra.run.gibba004.stdout:[mgr] 2021-07-11T04:02:09.478 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T04:02:09.478 INFO:teuthology.orchestra.run.gibba004.stdout: debug mgr = 20 2021-07-11T04:02:09.479 INFO:teuthology.orchestra.run.gibba004.stdout: debug mon = 20 2021-07-11T04:02:09.479 INFO:teuthology.orchestra.run.gibba004.stdout: debug auth = 20 2021-07-11T04:02:09.479 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:09.479 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:09.479 INFO:teuthology.orchestra.run.gibba004.stdout: mgr/telemetry/nag = false 2021-07-11T04:02:09.480 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.480 INFO:teuthology.orchestra.run.gibba004.stdout:[mon] 2021-07-11T04:02:09.480 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T04:02:09.480 INFO:teuthology.orchestra.run.gibba004.stdout: debug mon = 20 2021-07-11T04:02:09.480 INFO:teuthology.orchestra.run.gibba004.stdout: debug paxos = 20 2021-07-11T04:02:09.481 INFO:teuthology.orchestra.run.gibba004.stdout: debug auth = 20 2021-07-11T04:02:09.481 INFO:teuthology.orchestra.run.gibba004.stdout: mon data avail warn = 5 2021-07-11T04:02:09.481 INFO:teuthology.orchestra.run.gibba004.stdout: mon mgr mkfs grace = 240 2021-07-11T04:02:09.481 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:09.482 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd reporter subtree level = osd 2021-07-11T04:02:09.482 INFO:teuthology.orchestra.run.gibba004.stdout: mon osd prime pg temp = true 2021-07-11T04:02:09.482 INFO:teuthology.orchestra.run.gibba004.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:09.482 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.482 INFO:teuthology.orchestra.run.gibba004.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T04:02:09.483 INFO:teuthology.orchestra.run.gibba004.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T04:02:09.483 INFO:teuthology.orchestra.run.gibba004.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T04:02:09.483 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.483 INFO:teuthology.orchestra.run.gibba004.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T04:02:09.484 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T04:02:09.484 INFO:teuthology.orchestra.run.gibba004.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T04:02:09.484 INFO:teuthology.orchestra.run.gibba004.stdout: mon op complaint time = 120 2021-07-11T04:02:09.484 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:09.484 INFO:teuthology.orchestra.run.gibba004.stdout:[client] 2021-07-11T04:02:09.485 INFO:teuthology.orchestra.run.gibba004.stdout: rgw cache enabled = true 2021-07-11T04:02:09.485 INFO:teuthology.orchestra.run.gibba004.stdout: rgw enable ops log = true 2021-07-11T04:02:09.485 INFO:teuthology.orchestra.run.gibba004.stdout: rgw enable usage log = true 2021-07-11T04:02:09.485 INFO:teuthology.orchestra.run.gibba004.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T04:02:09.485 INFO:teuthology.orchestra.run.gibba004.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T04:02:09.486 INFO:teuthology.orchestra.run.gibba004.stdout: client mount timeout = 600 2021-07-11T04:02:09.486 INFO:teuthology.orchestra.run.gibba004.stdout: debug client = 20 2021-07-11T04:02:09.486 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T04:02:09.486 INFO:teuthology.orchestra.run.gibba004.stdout: rados mon op timeout = 15m 2021-07-11T04:02:09.487 INFO:teuthology.orchestra.run.gibba004.stdout: rados osd op timeout = 15m 2021-07-11T04:02:09.487 INFO:teuthology.orchestra.run.gibba004.stdout:[mon.a] 2021-07-11T04:02:09.487 INFO:teuthology.orchestra.run.gibba004.stdout:[mon.b] 2021-07-11T04:02:09.487 INFO:teuthology.orchestra.run.gibba004.stdout:[mon.c] 2021-07-11T04:02:09.487 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.a] 2021-07-11T04:02:09.488 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.c] 2021-07-11T04:02:09.488 INFO:teuthology.orchestra.run.gibba004.stdout:[mds.b] 2021-07-11T04:02:09.488 INFO:teuthology.orchestra.run.gibba004.stdout:[mds] 2021-07-11T04:02:09.488 INFO:teuthology.orchestra.run.gibba004.stdout: debug mds = 20 2021-07-11T04:02:09.488 INFO:teuthology.orchestra.run.gibba004.stdout: debug ms = 1 2021-07-11T04:02:09.489 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal frag = True 2021-07-11T04:02:09.489 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal fragment size max = 10000 2021-07-11T04:02:09.489 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal merge size = 5 2021-07-11T04:02:09.489 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal split bits = 3 2021-07-11T04:02:09.490 INFO:teuthology.orchestra.run.gibba004.stdout: mds bal split size = 100 2021-07-11T04:02:09.490 INFO:teuthology.orchestra.run.gibba004.stdout: mds debug frag = True 2021-07-11T04:02:09.490 INFO:teuthology.orchestra.run.gibba004.stdout: mds debug scatterstat = True 2021-07-11T04:02:09.490 INFO:teuthology.orchestra.run.gibba004.stdout: mds op complaint time = 180 2021-07-11T04:02:09.490 INFO:teuthology.orchestra.run.gibba004.stdout: mds verify scatter = True 2021-07-11T04:02:09.491 INFO:teuthology.orchestra.run.gibba004.stdout: osd op complaint time = 180 2021-07-11T04:02:09.491 INFO:teuthology.orchestra.run.gibba004.stdout: rados mon op timeout = 15m 2021-07-11T04:02:09.491 INFO:teuthology.orchestra.run.gibba004.stdout: rados osd op timeout = 15m 2021-07-11T04:02:09.494 INFO:teuthology.orchestra.run.gibba023.stdout:[global] 2021-07-11T04:02:09.495 INFO:teuthology.orchestra.run.gibba023.stdout: chdir = "" 2021-07-11T04:02:09.495 INFO:teuthology.orchestra.run.gibba023.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T04:02:09.495 INFO:teuthology.orchestra.run.gibba023.stdout: auth supported = cephx 2021-07-11T04:02:09.495 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.496 INFO:teuthology.orchestra.run.gibba023.stdout: filestore xattr use omap = true 2021-07-11T04:02:09.496 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.496 INFO:teuthology.orchestra.run.gibba023.stdout: mon clock drift allowed = 1.000 2021-07-11T04:02:09.496 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.497 INFO:teuthology.orchestra.run.gibba023.stdout: osd crush chooseleaf type = 0 2021-07-11T04:02:09.497 INFO:teuthology.orchestra.run.gibba023.stdout: auth debug = true 2021-07-11T04:02:09.497 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.497 INFO:teuthology.orchestra.run.gibba023.stdout: ms die on old message = true 2021-07-11T04:02:09.497 INFO:teuthology.orchestra.run.gibba023.stdout: ms die on bug = true 2021-07-11T04:02:09.498 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.498 INFO:teuthology.orchestra.run.gibba023.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T04:02:09.498 INFO:teuthology.orchestra.run.gibba023.stdout: mon pg warn max object skew = 0 2021-07-11T04:02:09.498 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.499 INFO:teuthology.orchestra.run.gibba023.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T04:02:09.499 INFO:teuthology.orchestra.run.gibba023.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T04:02:09.499 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.499 INFO:teuthology.orchestra.run.gibba023.stdout: osd pool default size = 2 2021-07-11T04:02:09.500 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.500 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd allow primary affinity = true 2021-07-11T04:02:09.500 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd allow pg remap = true 2021-07-11T04:02:09.500 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on legacy crush tunables = false 2021-07-11T04:02:09.500 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on crush straw calc version zero = false 2021-07-11T04:02:09.501 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on no sortbitwise = false 2021-07-11T04:02:09.501 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on osd down out interval zero = false 2021-07-11T04:02:09.501 INFO:teuthology.orchestra.run.gibba023.stdout: mon warn on too few osds = false 2021-07-11T04:02:09.501 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T04:02:09.502 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T04:02:09.502 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.502 INFO:teuthology.orchestra.run.gibba023.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T04:02:09.502 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.503 INFO:teuthology.orchestra.run.gibba023.stdout: osd default data pool replay window = 5 2021-07-11T04:02:09.503 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.503 INFO:teuthology.orchestra.run.gibba023.stdout: mon allow pool delete = true 2021-07-11T04:02:09.503 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.504 INFO:teuthology.orchestra.run.gibba023.stdout: mon cluster log file level = debug 2021-07-11T04:02:09.504 INFO:teuthology.orchestra.run.gibba023.stdout: debug asserts on shutdown = true 2021-07-11T04:02:09.504 INFO:teuthology.orchestra.run.gibba023.stdout: mon health detail to clog = false 2021-07-11T04:02:09.504 INFO:teuthology.orchestra.run.gibba023.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T04:02:09.504 INFO:teuthology.orchestra.run.gibba023.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T04:02:09.505 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.505 INFO:teuthology.orchestra.run.gibba023.stdout:[osd] 2021-07-11T04:02:09.505 INFO:teuthology.orchestra.run.gibba023.stdout: osd journal size = 100 2021-07-11T04:02:09.505 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.506 INFO:teuthology.orchestra.run.gibba023.stdout: osd scrub load threshold = 5.0 2021-07-11T04:02:09.506 INFO:teuthology.orchestra.run.gibba023.stdout: osd scrub max interval = 600 2021-07-11T04:02:09.506 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.506 INFO:teuthology.orchestra.run.gibba023.stdout: osd recover clone overlap = true 2021-07-11T04:02:09.506 INFO:teuthology.orchestra.run.gibba023.stdout: osd recovery max chunk = 1048576 2021-07-11T04:02:09.507 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.507 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug shutdown = true 2021-07-11T04:02:09.507 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug op order = true 2021-07-11T04:02:09.507 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug verify stray on activate = true 2021-07-11T04:02:09.508 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.508 INFO:teuthology.orchestra.run.gibba023.stdout: osd open classes on start = true 2021-07-11T04:02:09.508 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug pg log writeout = true 2021-07-11T04:02:09.508 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.508 INFO:teuthology.orchestra.run.gibba023.stdout: osd deep scrub update digest min age = 30 2021-07-11T04:02:09.509 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.509 INFO:teuthology.orchestra.run.gibba023.stdout: osd map max advance = 10 2021-07-11T04:02:09.509 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.509 INFO:teuthology.orchestra.run.gibba023.stdout: journal zero on create = true 2021-07-11T04:02:09.509 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.510 INFO:teuthology.orchestra.run.gibba023.stdout: filestore ondisk finisher threads = 3 2021-07-11T04:02:09.510 INFO:teuthology.orchestra.run.gibba023.stdout: filestore apply finisher threads = 3 2021-07-11T04:02:09.510 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.510 INFO:teuthology.orchestra.run.gibba023.stdout: bdev debug aio = true 2021-07-11T04:02:09.511 INFO:teuthology.orchestra.run.gibba023.stdout: osd debug misdirected ops = true 2021-07-11T04:02:09.511 INFO:teuthology.orchestra.run.gibba023.stdout: bluestore block size = 96636764160 2021-07-11T04:02:09.511 INFO:teuthology.orchestra.run.gibba023.stdout: bluestore compression mode = aggressive 2021-07-11T04:02:09.511 INFO:teuthology.orchestra.run.gibba023.stdout: bluestore fsck on mount = True 2021-07-11T04:02:09.511 INFO:teuthology.orchestra.run.gibba023.stdout: debug bluefs = 20 2021-07-11T04:02:09.512 INFO:teuthology.orchestra.run.gibba023.stdout: debug bluestore = 20 2021-07-11T04:02:09.512 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T04:02:09.512 INFO:teuthology.orchestra.run.gibba023.stdout: debug osd = 20 2021-07-11T04:02:09.512 INFO:teuthology.orchestra.run.gibba023.stdout: debug rocksdb = 10 2021-07-11T04:02:09.513 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T04:02:09.513 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd full ratio = 0.9 2021-07-11T04:02:09.513 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd nearfull ratio = 0.8 2021-07-11T04:02:09.513 INFO:teuthology.orchestra.run.gibba023.stdout: osd failsafe full ratio = 0.95 2021-07-11T04:02:09.513 INFO:teuthology.orchestra.run.gibba023.stdout: osd objectstore = bluestore 2021-07-11T04:02:09.514 INFO:teuthology.orchestra.run.gibba023.stdout: osd op complaint time = 180 2021-07-11T04:02:09.514 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.514 INFO:teuthology.orchestra.run.gibba023.stdout:[mgr] 2021-07-11T04:02:09.514 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T04:02:09.515 INFO:teuthology.orchestra.run.gibba023.stdout: debug mgr = 20 2021-07-11T04:02:09.515 INFO:teuthology.orchestra.run.gibba023.stdout: debug mon = 20 2021-07-11T04:02:09.515 INFO:teuthology.orchestra.run.gibba023.stdout: debug auth = 20 2021-07-11T04:02:09.515 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:09.515 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:09.516 INFO:teuthology.orchestra.run.gibba023.stdout: mgr/telemetry/nag = false 2021-07-11T04:02:09.516 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.516 INFO:teuthology.orchestra.run.gibba023.stdout:[mon] 2021-07-11T04:02:09.516 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T04:02:09.516 INFO:teuthology.orchestra.run.gibba023.stdout: debug mon = 20 2021-07-11T04:02:09.517 INFO:teuthology.orchestra.run.gibba023.stdout: debug paxos = 20 2021-07-11T04:02:09.517 INFO:teuthology.orchestra.run.gibba023.stdout: debug auth = 20 2021-07-11T04:02:09.517 INFO:teuthology.orchestra.run.gibba023.stdout: mon data avail warn = 5 2021-07-11T04:02:09.517 INFO:teuthology.orchestra.run.gibba023.stdout: mon mgr mkfs grace = 240 2021-07-11T04:02:09.518 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:09.518 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd reporter subtree level = osd 2021-07-11T04:02:09.518 INFO:teuthology.orchestra.run.gibba023.stdout: mon osd prime pg temp = true 2021-07-11T04:02:09.518 INFO:teuthology.orchestra.run.gibba023.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:09.518 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.519 INFO:teuthology.orchestra.run.gibba023.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T04:02:09.519 INFO:teuthology.orchestra.run.gibba023.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T04:02:09.519 INFO:teuthology.orchestra.run.gibba023.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T04:02:09.519 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.519 INFO:teuthology.orchestra.run.gibba023.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T04:02:09.520 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T04:02:09.520 INFO:teuthology.orchestra.run.gibba023.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T04:02:09.520 INFO:teuthology.orchestra.run.gibba023.stdout: mon op complaint time = 120 2021-07-11T04:02:09.520 INFO:teuthology.orchestra.run.gibba023.stdout: 2021-07-11T04:02:09.521 INFO:teuthology.orchestra.run.gibba023.stdout:[client] 2021-07-11T04:02:09.521 INFO:teuthology.orchestra.run.gibba023.stdout: rgw cache enabled = true 2021-07-11T04:02:09.521 INFO:teuthology.orchestra.run.gibba023.stdout: rgw enable ops log = true 2021-07-11T04:02:09.521 INFO:teuthology.orchestra.run.gibba023.stdout: rgw enable usage log = true 2021-07-11T04:02:09.521 INFO:teuthology.orchestra.run.gibba023.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T04:02:09.522 INFO:teuthology.orchestra.run.gibba023.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T04:02:09.522 INFO:teuthology.orchestra.run.gibba023.stdout: client mount timeout = 600 2021-07-11T04:02:09.522 INFO:teuthology.orchestra.run.gibba023.stdout: debug client = 20 2021-07-11T04:02:09.523 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T04:02:09.523 INFO:teuthology.orchestra.run.gibba023.stdout: rados mon op timeout = 15m 2021-07-11T04:02:09.523 INFO:teuthology.orchestra.run.gibba023.stdout: rados osd op timeout = 15m 2021-07-11T04:02:09.523 INFO:teuthology.orchestra.run.gibba023.stdout:[mon.a] 2021-07-11T04:02:09.523 INFO:teuthology.orchestra.run.gibba023.stdout:[mon.b] 2021-07-11T04:02:09.524 INFO:teuthology.orchestra.run.gibba023.stdout:[mon.c] 2021-07-11T04:02:09.524 INFO:teuthology.orchestra.run.gibba023.stdout:[mds.a] 2021-07-11T04:02:09.524 INFO:teuthology.orchestra.run.gibba023.stdout:[mds.c] 2021-07-11T04:02:09.524 INFO:teuthology.orchestra.run.gibba023.stdout:[mds.b] 2021-07-11T04:02:09.525 INFO:teuthology.orchestra.run.gibba023.stdout:[mds] 2021-07-11T04:02:09.525 INFO:teuthology.orchestra.run.gibba023.stdout: debug mds = 20 2021-07-11T04:02:09.525 INFO:teuthology.orchestra.run.gibba023.stdout: debug ms = 1 2021-07-11T04:02:09.525 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal frag = True 2021-07-11T04:02:09.525 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal fragment size max = 10000 2021-07-11T04:02:09.526 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal merge size = 5 2021-07-11T04:02:09.526 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal split bits = 3 2021-07-11T04:02:09.526 INFO:teuthology.orchestra.run.gibba023.stdout: mds bal split size = 100 2021-07-11T04:02:09.526 INFO:teuthology.orchestra.run.gibba023.stdout: mds debug frag = True 2021-07-11T04:02:09.527 INFO:teuthology.orchestra.run.gibba023.stdout: mds debug scatterstat = True 2021-07-11T04:02:09.527 INFO:teuthology.orchestra.run.gibba023.stdout: mds op complaint time = 180 2021-07-11T04:02:09.527 INFO:teuthology.orchestra.run.gibba023.stdout: mds verify scatter = True 2021-07-11T04:02:09.527 INFO:teuthology.orchestra.run.gibba023.stdout: osd op complaint time = 180 2021-07-11T04:02:09.527 INFO:teuthology.orchestra.run.gibba023.stdout: rados mon op timeout = 15m 2021-07-11T04:02:09.528 INFO:teuthology.orchestra.run.gibba023.stdout: rados osd op timeout = 15m 2021-07-11T04:02:09.528 INFO:teuthology.orchestra.run.gibba019.stdout:[global] 2021-07-11T04:02:09.528 INFO:teuthology.orchestra.run.gibba019.stdout: chdir = "" 2021-07-11T04:02:09.528 INFO:teuthology.orchestra.run.gibba019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T04:02:09.529 INFO:teuthology.orchestra.run.gibba019.stdout: auth supported = cephx 2021-07-11T04:02:09.529 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.529 INFO:teuthology.orchestra.run.gibba019.stdout: filestore xattr use omap = true 2021-07-11T04:02:09.529 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.529 INFO:teuthology.orchestra.run.gibba019.stdout: mon clock drift allowed = 1.000 2021-07-11T04:02:09.530 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.530 INFO:teuthology.orchestra.run.gibba019.stdout: osd crush chooseleaf type = 0 2021-07-11T04:02:09.530 INFO:teuthology.orchestra.run.gibba019.stdout: auth debug = true 2021-07-11T04:02:09.530 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.531 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on old message = true 2021-07-11T04:02:09.531 INFO:teuthology.orchestra.run.gibba019.stdout: ms die on bug = true 2021-07-11T04:02:09.531 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.531 INFO:teuthology.orchestra.run.gibba019.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T04:02:09.531 INFO:teuthology.orchestra.run.gibba019.stdout: mon pg warn max object skew = 0 2021-07-11T04:02:09.532 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.532 INFO:teuthology.orchestra.run.gibba019.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T04:02:09.532 INFO:teuthology.orchestra.run.gibba019.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T04:02:09.532 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.532 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default size = 2 2021-07-11T04:02:09.533 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.533 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow primary affinity = true 2021-07-11T04:02:09.533 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd allow pg remap = true 2021-07-11T04:02:09.533 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on legacy crush tunables = false 2021-07-11T04:02:09.534 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on crush straw calc version zero = false 2021-07-11T04:02:09.534 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on no sortbitwise = false 2021-07-11T04:02:09.534 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on osd down out interval zero = false 2021-07-11T04:02:09.534 INFO:teuthology.orchestra.run.gibba019.stdout: mon warn on too few osds = false 2021-07-11T04:02:09.535 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T04:02:09.535 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T04:02:09.535 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.535 INFO:teuthology.orchestra.run.gibba019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T04:02:09.535 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.536 INFO:teuthology.orchestra.run.gibba019.stdout: osd default data pool replay window = 5 2021-07-11T04:02:09.536 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.536 INFO:teuthology.orchestra.run.gibba019.stdout: mon allow pool delete = true 2021-07-11T04:02:09.536 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.536 INFO:teuthology.orchestra.run.gibba019.stdout: mon cluster log file level = debug 2021-07-11T04:02:09.537 INFO:teuthology.orchestra.run.gibba019.stdout: debug asserts on shutdown = true 2021-07-11T04:02:09.537 INFO:teuthology.orchestra.run.gibba019.stdout: mon health detail to clog = false 2021-07-11T04:02:09.537 INFO:teuthology.orchestra.run.gibba019.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T04:02:09.537 INFO:teuthology.orchestra.run.gibba019.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T04:02:09.538 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.538 INFO:teuthology.orchestra.run.gibba019.stdout:[osd] 2021-07-11T04:02:09.538 INFO:teuthology.orchestra.run.gibba019.stdout: osd journal size = 100 2021-07-11T04:02:09.538 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.538 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub load threshold = 5.0 2021-07-11T04:02:09.539 INFO:teuthology.orchestra.run.gibba019.stdout: osd scrub max interval = 600 2021-07-11T04:02:09.539 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.539 INFO:teuthology.orchestra.run.gibba019.stdout: osd recover clone overlap = true 2021-07-11T04:02:09.539 INFO:teuthology.orchestra.run.gibba019.stdout: osd recovery max chunk = 1048576 2021-07-11T04:02:09.539 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.540 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug shutdown = true 2021-07-11T04:02:09.540 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug op order = true 2021-07-11T04:02:09.540 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug verify stray on activate = true 2021-07-11T04:02:09.540 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.541 INFO:teuthology.orchestra.run.gibba019.stdout: osd open classes on start = true 2021-07-11T04:02:09.541 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug pg log writeout = true 2021-07-11T04:02:09.541 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.541 INFO:teuthology.orchestra.run.gibba019.stdout: osd deep scrub update digest min age = 30 2021-07-11T04:02:09.542 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.542 INFO:teuthology.orchestra.run.gibba019.stdout: osd map max advance = 10 2021-07-11T04:02:09.542 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.542 INFO:teuthology.orchestra.run.gibba019.stdout: journal zero on create = true 2021-07-11T04:02:09.542 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.543 INFO:teuthology.orchestra.run.gibba019.stdout: filestore ondisk finisher threads = 3 2021-07-11T04:02:09.543 INFO:teuthology.orchestra.run.gibba019.stdout: filestore apply finisher threads = 3 2021-07-11T04:02:09.543 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.543 INFO:teuthology.orchestra.run.gibba019.stdout: bdev debug aio = true 2021-07-11T04:02:09.544 INFO:teuthology.orchestra.run.gibba019.stdout: osd debug misdirected ops = true 2021-07-11T04:02:09.544 INFO:teuthology.orchestra.run.gibba019.stdout: bluestore block size = 96636764160 2021-07-11T04:02:09.544 INFO:teuthology.orchestra.run.gibba019.stdout: bluestore compression mode = aggressive 2021-07-11T04:02:09.544 INFO:teuthology.orchestra.run.gibba019.stdout: bluestore fsck on mount = True 2021-07-11T04:02:09.544 INFO:teuthology.orchestra.run.gibba019.stdout: debug bluefs = 20 2021-07-11T04:02:09.545 INFO:teuthology.orchestra.run.gibba019.stdout: debug bluestore = 20 2021-07-11T04:02:09.545 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T04:02:09.545 INFO:teuthology.orchestra.run.gibba019.stdout: debug osd = 20 2021-07-11T04:02:09.545 INFO:teuthology.orchestra.run.gibba019.stdout: debug rocksdb = 10 2021-07-11T04:02:09.545 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T04:02:09.546 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd full ratio = 0.9 2021-07-11T04:02:09.546 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd nearfull ratio = 0.8 2021-07-11T04:02:09.546 INFO:teuthology.orchestra.run.gibba019.stdout: osd failsafe full ratio = 0.95 2021-07-11T04:02:09.546 INFO:teuthology.orchestra.run.gibba019.stdout: osd objectstore = bluestore 2021-07-11T04:02:09.547 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-07-11T04:02:09.547 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.547 INFO:teuthology.orchestra.run.gibba019.stdout:[mgr] 2021-07-11T04:02:09.547 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T04:02:09.547 INFO:teuthology.orchestra.run.gibba019.stdout: debug mgr = 20 2021-07-11T04:02:09.548 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-07-11T04:02:09.548 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-07-11T04:02:09.548 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:09.548 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:09.548 INFO:teuthology.orchestra.run.gibba019.stdout: mgr/telemetry/nag = false 2021-07-11T04:02:09.549 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.549 INFO:teuthology.orchestra.run.gibba019.stdout:[mon] 2021-07-11T04:02:09.549 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T04:02:09.549 INFO:teuthology.orchestra.run.gibba019.stdout: debug mon = 20 2021-07-11T04:02:09.549 INFO:teuthology.orchestra.run.gibba019.stdout: debug paxos = 20 2021-07-11T04:02:09.550 INFO:teuthology.orchestra.run.gibba019.stdout: debug auth = 20 2021-07-11T04:02:09.550 INFO:teuthology.orchestra.run.gibba019.stdout: mon data avail warn = 5 2021-07-11T04:02:09.550 INFO:teuthology.orchestra.run.gibba019.stdout: mon mgr mkfs grace = 240 2021-07-11T04:02:09.550 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:09.551 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd reporter subtree level = osd 2021-07-11T04:02:09.551 INFO:teuthology.orchestra.run.gibba019.stdout: mon osd prime pg temp = true 2021-07-11T04:02:09.551 INFO:teuthology.orchestra.run.gibba019.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:09.551 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.551 INFO:teuthology.orchestra.run.gibba019.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T04:02:09.552 INFO:teuthology.orchestra.run.gibba019.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T04:02:09.552 INFO:teuthology.orchestra.run.gibba019.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T04:02:09.552 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.552 INFO:teuthology.orchestra.run.gibba019.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T04:02:09.553 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T04:02:09.553 INFO:teuthology.orchestra.run.gibba019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T04:02:09.553 INFO:teuthology.orchestra.run.gibba019.stdout: mon op complaint time = 120 2021-07-11T04:02:09.553 INFO:teuthology.orchestra.run.gibba019.stdout: 2021-07-11T04:02:09.553 INFO:teuthology.orchestra.run.gibba019.stdout:[client] 2021-07-11T04:02:09.554 INFO:teuthology.orchestra.run.gibba019.stdout: rgw cache enabled = true 2021-07-11T04:02:09.554 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable ops log = true 2021-07-11T04:02:09.554 INFO:teuthology.orchestra.run.gibba019.stdout: rgw enable usage log = true 2021-07-11T04:02:09.554 INFO:teuthology.orchestra.run.gibba019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T04:02:09.555 INFO:teuthology.orchestra.run.gibba019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T04:02:09.555 INFO:teuthology.orchestra.run.gibba019.stdout: client mount timeout = 600 2021-07-11T04:02:09.555 INFO:teuthology.orchestra.run.gibba019.stdout: debug client = 20 2021-07-11T04:02:09.555 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T04:02:09.555 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-07-11T04:02:09.556 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-07-11T04:02:09.556 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.a] 2021-07-11T04:02:09.556 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.b] 2021-07-11T04:02:09.556 INFO:teuthology.orchestra.run.gibba019.stdout:[mon.c] 2021-07-11T04:02:09.556 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.a] 2021-07-11T04:02:09.557 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.c] 2021-07-11T04:02:09.557 INFO:teuthology.orchestra.run.gibba019.stdout:[mds.b] 2021-07-11T04:02:09.557 INFO:teuthology.orchestra.run.gibba019.stdout:[mds] 2021-07-11T04:02:09.557 INFO:teuthology.orchestra.run.gibba019.stdout: debug mds = 20 2021-07-11T04:02:09.558 INFO:teuthology.orchestra.run.gibba019.stdout: debug ms = 1 2021-07-11T04:02:09.558 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal frag = True 2021-07-11T04:02:09.558 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal fragment size max = 10000 2021-07-11T04:02:09.558 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal merge size = 5 2021-07-11T04:02:09.558 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split bits = 3 2021-07-11T04:02:09.559 INFO:teuthology.orchestra.run.gibba019.stdout: mds bal split size = 100 2021-07-11T04:02:09.559 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug frag = True 2021-07-11T04:02:09.559 INFO:teuthology.orchestra.run.gibba019.stdout: mds debug scatterstat = True 2021-07-11T04:02:09.559 INFO:teuthology.orchestra.run.gibba019.stdout: mds op complaint time = 180 2021-07-11T04:02:09.559 INFO:teuthology.orchestra.run.gibba019.stdout: mds verify scatter = True 2021-07-11T04:02:09.560 INFO:teuthology.orchestra.run.gibba019.stdout: osd op complaint time = 180 2021-07-11T04:02:09.560 INFO:teuthology.orchestra.run.gibba019.stdout: rados mon op timeout = 15m 2021-07-11T04:02:09.560 INFO:teuthology.orchestra.run.gibba019.stdout: rados osd op timeout = 15m 2021-07-11T04:02:09.560 INFO:teuthology.orchestra.run.gibba005.stdout:[global] 2021-07-11T04:02:09.561 INFO:teuthology.orchestra.run.gibba005.stdout: chdir = "" 2021-07-11T04:02:09.561 INFO:teuthology.orchestra.run.gibba005.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-07-11T04:02:09.561 INFO:teuthology.orchestra.run.gibba005.stdout: auth supported = cephx 2021-07-11T04:02:09.561 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.562 INFO:teuthology.orchestra.run.gibba005.stdout: filestore xattr use omap = true 2021-07-11T04:02:09.562 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.562 INFO:teuthology.orchestra.run.gibba005.stdout: mon clock drift allowed = 1.000 2021-07-11T04:02:09.562 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.562 INFO:teuthology.orchestra.run.gibba005.stdout: osd crush chooseleaf type = 0 2021-07-11T04:02:09.563 INFO:teuthology.orchestra.run.gibba005.stdout: auth debug = true 2021-07-11T04:02:09.563 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.563 INFO:teuthology.orchestra.run.gibba005.stdout: ms die on old message = true 2021-07-11T04:02:09.563 INFO:teuthology.orchestra.run.gibba005.stdout: ms die on bug = true 2021-07-11T04:02:09.563 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.564 INFO:teuthology.orchestra.run.gibba005.stdout: mon max pg per osd = 10000 # >= luminous 2021-07-11T04:02:09.564 INFO:teuthology.orchestra.run.gibba005.stdout: mon pg warn max object skew = 0 2021-07-11T04:02:09.564 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.564 INFO:teuthology.orchestra.run.gibba005.stdout: # disable pg_autoscaler by default for new pools 2021-07-11T04:02:09.565 INFO:teuthology.orchestra.run.gibba005.stdout: osd_pool_default_pg_autoscale_mode = off 2021-07-11T04:02:09.565 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.565 INFO:teuthology.orchestra.run.gibba005.stdout: osd pool default size = 2 2021-07-11T04:02:09.565 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.565 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd allow primary affinity = true 2021-07-11T04:02:09.566 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd allow pg remap = true 2021-07-11T04:02:09.566 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on legacy crush tunables = false 2021-07-11T04:02:09.566 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on crush straw calc version zero = false 2021-07-11T04:02:09.566 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on no sortbitwise = false 2021-07-11T04:02:09.567 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on osd down out interval zero = false 2021-07-11T04:02:09.567 INFO:teuthology.orchestra.run.gibba005.stdout: mon warn on too few osds = false 2021-07-11T04:02:09.567 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-07-11T04:02:09.567 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_pool_no_redundancy = false 2021-07-11T04:02:09.567 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.568 INFO:teuthology.orchestra.run.gibba005.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-07-11T04:02:09.568 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.568 INFO:teuthology.orchestra.run.gibba005.stdout: osd default data pool replay window = 5 2021-07-11T04:02:09.568 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.569 INFO:teuthology.orchestra.run.gibba005.stdout: mon allow pool delete = true 2021-07-11T04:02:09.569 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.569 INFO:teuthology.orchestra.run.gibba005.stdout: mon cluster log file level = debug 2021-07-11T04:02:09.569 INFO:teuthology.orchestra.run.gibba005.stdout: debug asserts on shutdown = true 2021-07-11T04:02:09.569 INFO:teuthology.orchestra.run.gibba005.stdout: mon health detail to clog = false 2021-07-11T04:02:09.570 INFO:teuthology.orchestra.run.gibba005.stdout: mon host = "172.21.2.104,172.21.2.105,[v2:172.21.2.105:3301,v1:172.21.2.105:6790]" 2021-07-11T04:02:09.570 INFO:teuthology.orchestra.run.gibba005.stdout: fsid = 57c05e2e-f3b6-453c-878e-145bef41bd6c 2021-07-11T04:02:09.570 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.570 INFO:teuthology.orchestra.run.gibba005.stdout:[osd] 2021-07-11T04:02:09.571 INFO:teuthology.orchestra.run.gibba005.stdout: osd journal size = 100 2021-07-11T04:02:09.571 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.571 INFO:teuthology.orchestra.run.gibba005.stdout: osd scrub load threshold = 5.0 2021-07-11T04:02:09.571 INFO:teuthology.orchestra.run.gibba005.stdout: osd scrub max interval = 600 2021-07-11T04:02:09.571 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.572 INFO:teuthology.orchestra.run.gibba005.stdout: osd recover clone overlap = true 2021-07-11T04:02:09.572 INFO:teuthology.orchestra.run.gibba005.stdout: osd recovery max chunk = 1048576 2021-07-11T04:02:09.572 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.572 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug shutdown = true 2021-07-11T04:02:09.572 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug op order = true 2021-07-11T04:02:09.573 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug verify stray on activate = true 2021-07-11T04:02:09.573 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.573 INFO:teuthology.orchestra.run.gibba005.stdout: osd open classes on start = true 2021-07-11T04:02:09.573 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug pg log writeout = true 2021-07-11T04:02:09.574 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.574 INFO:teuthology.orchestra.run.gibba005.stdout: osd deep scrub update digest min age = 30 2021-07-11T04:02:09.574 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.574 INFO:teuthology.orchestra.run.gibba005.stdout: osd map max advance = 10 2021-07-11T04:02:09.574 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.575 INFO:teuthology.orchestra.run.gibba005.stdout: journal zero on create = true 2021-07-11T04:02:09.575 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.575 INFO:teuthology.orchestra.run.gibba005.stdout: filestore ondisk finisher threads = 3 2021-07-11T04:02:09.575 INFO:teuthology.orchestra.run.gibba005.stdout: filestore apply finisher threads = 3 2021-07-11T04:02:09.575 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.576 INFO:teuthology.orchestra.run.gibba005.stdout: bdev debug aio = true 2021-07-11T04:02:09.576 INFO:teuthology.orchestra.run.gibba005.stdout: osd debug misdirected ops = true 2021-07-11T04:02:09.576 INFO:teuthology.orchestra.run.gibba005.stdout: bluestore block size = 96636764160 2021-07-11T04:02:09.576 INFO:teuthology.orchestra.run.gibba005.stdout: bluestore compression mode = aggressive 2021-07-11T04:02:09.576 INFO:teuthology.orchestra.run.gibba005.stdout: bluestore fsck on mount = True 2021-07-11T04:02:09.577 INFO:teuthology.orchestra.run.gibba005.stdout: debug bluefs = 20 2021-07-11T04:02:09.577 INFO:teuthology.orchestra.run.gibba005.stdout: debug bluestore = 20 2021-07-11T04:02:09.577 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T04:02:09.577 INFO:teuthology.orchestra.run.gibba005.stdout: debug osd = 20 2021-07-11T04:02:09.578 INFO:teuthology.orchestra.run.gibba005.stdout: debug rocksdb = 10 2021-07-11T04:02:09.578 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd backfillfull_ratio = 0.85 2021-07-11T04:02:09.578 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd full ratio = 0.9 2021-07-11T04:02:09.578 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd nearfull ratio = 0.8 2021-07-11T04:02:09.579 INFO:teuthology.orchestra.run.gibba005.stdout: osd failsafe full ratio = 0.95 2021-07-11T04:02:09.579 INFO:teuthology.orchestra.run.gibba005.stdout: osd objectstore = bluestore 2021-07-11T04:02:09.579 INFO:teuthology.orchestra.run.gibba005.stdout: osd op complaint time = 180 2021-07-11T04:02:09.579 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.579 INFO:teuthology.orchestra.run.gibba005.stdout:[mgr] 2021-07-11T04:02:09.580 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T04:02:09.580 INFO:teuthology.orchestra.run.gibba005.stdout: debug mgr = 20 2021-07-11T04:02:09.580 INFO:teuthology.orchestra.run.gibba005.stdout: debug mon = 20 2021-07-11T04:02:09.580 INFO:teuthology.orchestra.run.gibba005.stdout: debug auth = 20 2021-07-11T04:02:09.581 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:09.581 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:09.581 INFO:teuthology.orchestra.run.gibba005.stdout: mgr/telemetry/nag = false 2021-07-11T04:02:09.581 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.581 INFO:teuthology.orchestra.run.gibba005.stdout:[mon] 2021-07-11T04:02:09.582 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T04:02:09.582 INFO:teuthology.orchestra.run.gibba005.stdout: debug mon = 20 2021-07-11T04:02:09.582 INFO:teuthology.orchestra.run.gibba005.stdout: debug paxos = 20 2021-07-11T04:02:09.582 INFO:teuthology.orchestra.run.gibba005.stdout: debug auth = 20 2021-07-11T04:02:09.583 INFO:teuthology.orchestra.run.gibba005.stdout: mon data avail warn = 5 2021-07-11T04:02:09.583 INFO:teuthology.orchestra.run.gibba005.stdout: mon mgr mkfs grace = 240 2021-07-11T04:02:09.583 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min pgs per osd = 4 2021-07-11T04:02:09.583 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd reporter subtree level = osd 2021-07-11T04:02:09.583 INFO:teuthology.orchestra.run.gibba005.stdout: mon osd prime pg temp = true 2021-07-11T04:02:09.584 INFO:teuthology.orchestra.run.gibba005.stdout: mon reweight min bytes per osd = 10 2021-07-11T04:02:09.584 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.584 INFO:teuthology.orchestra.run.gibba005.stdout: # rotate auth tickets quickly to exercise renewal paths 2021-07-11T04:02:09.584 INFO:teuthology.orchestra.run.gibba005.stdout: auth mon ticket ttl = 660 # 11m 2021-07-11T04:02:09.585 INFO:teuthology.orchestra.run.gibba005.stdout: auth service ticket ttl = 240 # 4m 2021-07-11T04:02:09.585 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.585 INFO:teuthology.orchestra.run.gibba005.stdout: # don't complain about insecure global_id in the test suite 2021-07-11T04:02:09.585 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_insecure_global_id_reclaim = false 2021-07-11T04:02:09.586 INFO:teuthology.orchestra.run.gibba005.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2021-07-11T04:02:09.586 INFO:teuthology.orchestra.run.gibba005.stdout: mon op complaint time = 120 2021-07-11T04:02:09.586 INFO:teuthology.orchestra.run.gibba005.stdout: 2021-07-11T04:02:09.586 INFO:teuthology.orchestra.run.gibba005.stdout:[client] 2021-07-11T04:02:09.586 INFO:teuthology.orchestra.run.gibba005.stdout: rgw cache enabled = true 2021-07-11T04:02:09.587 INFO:teuthology.orchestra.run.gibba005.stdout: rgw enable ops log = true 2021-07-11T04:02:09.587 INFO:teuthology.orchestra.run.gibba005.stdout: rgw enable usage log = true 2021-07-11T04:02:09.587 INFO:teuthology.orchestra.run.gibba005.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-07-11T04:02:09.587 INFO:teuthology.orchestra.run.gibba005.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-07-11T04:02:09.588 INFO:teuthology.orchestra.run.gibba005.stdout: client mount timeout = 600 2021-07-11T04:02:09.588 INFO:teuthology.orchestra.run.gibba005.stdout: debug client = 20 2021-07-11T04:02:09.588 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T04:02:09.588 INFO:teuthology.orchestra.run.gibba005.stdout: rados mon op timeout = 15m 2021-07-11T04:02:09.589 INFO:teuthology.orchestra.run.gibba005.stdout: rados osd op timeout = 15m 2021-07-11T04:02:09.589 INFO:teuthology.orchestra.run.gibba005.stdout:[mon.a] 2021-07-11T04:02:09.589 INFO:teuthology.orchestra.run.gibba005.stdout:[mon.b] 2021-07-11T04:02:09.589 INFO:teuthology.orchestra.run.gibba005.stdout:[mon.c] 2021-07-11T04:02:09.589 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.a] 2021-07-11T04:02:09.590 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.c] 2021-07-11T04:02:09.590 INFO:teuthology.orchestra.run.gibba005.stdout:[mds.b] 2021-07-11T04:02:09.590 INFO:teuthology.orchestra.run.gibba005.stdout:[mds] 2021-07-11T04:02:09.590 INFO:teuthology.orchestra.run.gibba005.stdout: debug mds = 20 2021-07-11T04:02:09.590 INFO:teuthology.orchestra.run.gibba005.stdout: debug ms = 1 2021-07-11T04:02:09.591 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal frag = True 2021-07-11T04:02:09.591 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal fragment size max = 10000 2021-07-11T04:02:09.591 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal merge size = 5 2021-07-11T04:02:09.591 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal split bits = 3 2021-07-11T04:02:09.591 INFO:teuthology.orchestra.run.gibba005.stdout: mds bal split size = 100 2021-07-11T04:02:09.592 INFO:teuthology.orchestra.run.gibba005.stdout: mds debug frag = True 2021-07-11T04:02:09.592 INFO:teuthology.orchestra.run.gibba005.stdout: mds debug scatterstat = True 2021-07-11T04:02:09.592 INFO:teuthology.orchestra.run.gibba005.stdout: mds op complaint time = 180 2021-07-11T04:02:09.592 INFO:teuthology.orchestra.run.gibba005.stdout: mds verify scatter = True 2021-07-11T04:02:09.593 INFO:teuthology.orchestra.run.gibba005.stdout: osd op complaint time = 180 2021-07-11T04:02:09.593 INFO:teuthology.orchestra.run.gibba005.stdout: rados mon op timeout = 15m 2021-07-11T04:02:09.593 INFO:teuthology.orchestra.run.gibba005.stdout: rados osd op timeout = 15m 2021-07-11T04:02:09.594 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-07-11T04:02:09.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 Processor -- start 2021-07-11T04:02:09.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 -- start start 2021-07-11T04:02:09.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f3ddc141960 0x7f3ddc0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ddc0a77d0 con 0x7f3ddc0b1ec0 2021-07-11T04:02:09.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ddc0b0650 con 0x7f3ddc0b12d0 2021-07-11T04:02:09.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ddc0b1740 con 0x7f3ddc141960 2021-07-11T04:02:09.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3ddaffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f3ddc0b1ec0 0x7f3ddc0a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47562/0 (socket says 172.21.2.104:47562) 2021-07-11T04:02:09.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3ddaffd700 1 -- 172.21.2.104:0/264990905 learned_addr learned my addr 172.21.2.104:0/264990905 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:09.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630010634 0 0) 0x7f3ddc0a77d0 con 0x7f3ddc0b1ec0 2021-07-11T04:02:09.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dbc003260 con 0x7f3ddc0b1ec0 2021-07-11T04:02:09.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3ddb7fe700 1 --2- 172.21.2.104:0/264990905 >> v2:172.21.2.105:3300/0 conn(0x7f3ddc141960 0x7f3ddc0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:09.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3134453468 0 0) 0x7f3dbc003260 con 0x7f3ddc0b1ec0 2021-07-11T04:02:09.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 >> v2:172.21.2.105:3300/0 conn(0x7f3ddc141960 msgr2=0x7f3ddc0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:09.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 --2- 172.21.2.104:0/264990905 >> v2:172.21.2.105:3300/0 conn(0x7f3ddc141960 0x7f3ddc0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 >> v1:172.21.2.105:6789/0 conn(0x7f3ddc0b12d0 legacy=0x7f3ddc141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:09.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc0b08f0 con 0x7f3ddc0b1ec0 2021-07-11T04:02:09.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f3dc4001660 con 0x7f3ddc0b1ec0 2021-07-11T04:02:09.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 >> v1:172.21.2.104:6789/0 conn(0x7f3ddc0b1ec0 legacy=0x7f3ddc0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:09.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 --2- 172.21.2.104:0/264990905 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3dbc003750 0x7f3dbc003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 --2- 172.21.2.104:0/264990905 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dbc004500 0x7f3dbc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 --2- 172.21.2.104:0/264990905 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dbc004ec0 0x7f3dbc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0b1740 con 0x7f3dbc004500 2021-07-11T04:02:09.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc00bda0 con 0x7f3dbc003750 2021-07-11T04:02:09.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dbc00bf60 con 0x7f3dbc004ec0 2021-07-11T04:02:09.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dda7fc700 1 --2- 172.21.2.104:0/264990905 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dbc004500 0x7f3dbc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:09.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3ddb7fe700 1 --2- 172.21.2.104:0/264990905 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dbc004ec0 0x7f3dbc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:09.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3ddaffd700 1 --2- 172.21.2.104:0/264990905 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3dbc003750 0x7f3dbc003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:09.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dda7fc700 1 -- 172.21.2.104:0/264990905 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dbc004ec0 msgr2=0x7f3dbc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:09.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dda7fc700 1 --2- 172.21.2.104:0/264990905 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dbc004ec0 0x7f3dbc00b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dda7fc700 1 -- 172.21.2.104:0/264990905 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3dbc003750 msgr2=0x7f3dbc003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:09.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dda7fc700 1 --2- 172.21.2.104:0/264990905 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3dbc003750 0x7f3dbc003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dda7fc700 1 -- 172.21.2.104:0/264990905 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc003260 con 0x7f3dbc004500 2021-07-11T04:02:09.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3ddb7fe700 1 -- 172.21.2.104:0/264990905 reap_dead start 2021-07-11T04:02:09.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dda7fc700 1 --2- 172.21.2.104:0/264990905 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dbc004500 0x7f3dbc004980 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f3ddc0b2f50 tx=0x7f3dcc007a00).ready entity=mon.0 client_cookie=8ade619f703f3098 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:09.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dcc007d10 con 0x7f3dbc004500 2021-07-11T04:02:09.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc007030 con 0x7f3dbc004500 2021-07-11T04:02:09.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3dd9ffb700 1 -- 172.21.2.104:0/264990905 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dcc007390 con 0x7f3dbc004500 2021-07-11T04:02:09.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 -- 172.21.2.104:0/264990905 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dbc004500 msgr2=0x7f3dbc004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:09.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 --2- 172.21.2.104:0/264990905 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dbc004500 0x7f3dbc004980 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f3ddc0b2f50 tx=0x7f3dcc007a00).stop 2021-07-11T04:02:09.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 -- 172.21.2.104:0/264990905 shutdown_connections 2021-07-11T04:02:09.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 --2- 172.21.2.104:0/264990905 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3dbc004500 0x7f3dbc004980 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 -- 172.21.2.104:0/264990905 >> 172.21.2.104:0/264990905 conn(0x7f3ddc05f6b0 msgr2=0x7f3ddc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:09.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 -- 172.21.2.104:0/264990905 shutdown_connections 2021-07-11T04:02:09.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.652+0000 7f3de1554700 1 -- 172.21.2.104:0/264990905 wait complete. 2021-07-11T04:02:09.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3de1554700 1 Processor -- start 2021-07-11T04:02:09.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3de1554700 1 -- start start 2021-07-11T04:02:09.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3de1554700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dbc004500 0x7f3ddc14ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3de1554700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ddc14a9a0 0x7f3ddc157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3de1554700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3ddc0aa7e0 0x7f3ddc0aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3de1554700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc0aa3b0 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3de1554700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc148b30 con 0x7f3ddc0aa7e0 2021-07-11T04:02:09.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3de1554700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ddc148170 con 0x7f3dbc004500 2021-07-11T04:02:09.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dda7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ddc14a9a0 0x7f3ddc157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:09.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3ddaffd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dbc004500 0x7f3ddc14ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:09.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dda7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ddc14a9a0 0x7f3ddc157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50810/0 (socket says 172.21.2.104:50810) 2021-07-11T04:02:09.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dda7fc700 1 -- 172.21.2.104:0/3522018785 learned_addr learned my addr 172.21.2.104:0/3522018785 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:09.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3ddaffd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dbc004500 0x7f3ddc14ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:53970/0 (socket says 172.21.2.104:53970) 2021-07-11T04:02:09.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3ddb7fe700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3ddc0aa7e0 0x7f3ddc0aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:09.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dda7fc700 1 -- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dbc004500 msgr2=0x7f3ddc14ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:09.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dda7fc700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dbc004500 0x7f3ddc14ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dda7fc700 1 -- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3ddc0aa7e0 msgr2=0x7f3ddc0aac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:09.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dda7fc700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3ddc0aa7e0 0x7f3ddc0aac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dda7fc700 1 -- 172.21.2.104:0/3522018785 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc148410 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.671 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dda7fc700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ddc14a9a0 0x7f3ddc157390 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f3ddc0b2f50 tx=0x7f3dcc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:09.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dd3fff700 1 -- 172.21.2.104:0/3522018785 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dcc007430 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dd3fff700 1 -- 172.21.2.104:0/3522018785 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc006510 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dd3fff700 1 -- 172.21.2.104:0/3522018785 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3dcc006870 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3de1554700 1 -- 172.21.2.104:0/3522018785 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ddc1486b0 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3de1554700 1 -- 172.21.2.104:0/3522018785 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ddc0ab150 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3de1554700 1 -- 172.21.2.104:0/3522018785 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ddc0aa520 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dd3fff700 1 -- 172.21.2.104:0/3522018785 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3dcc010070 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dd3fff700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3dc0053270 0x7f3dc0055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3dd3fff700 1 -- 172.21.2.104:0/3522018785 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3dcc06bd60 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.656+0000 7f3ddaffd700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3dc0053270 0x7f3dc0055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:09.674 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.660+0000 7f3ddaffd700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3dc0053270 0x7f3dc0055730 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f3dc4005000 tx=0x7f3ddc0b0a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:09.675 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.660+0000 7f3dd3fff700 1 -- 172.21.2.104:0/3522018785 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3ddc0aa520 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.744+0000 7f3de1554700 1 -- 172.21.2.104:0/3522018785 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3ddc057350 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3dd3fff700 1 -- 172.21.2.104:0/3522018785 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3dcc025070 con 0x7f3ddc14a9a0 2021-07-11T04:02:09.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 -- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3dc0053270 msgr2=0x7f3dc0055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:09.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3dc0053270 0x7f3dc0055730 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f3dc4005000 tx=0x7f3ddc0b0a80).stop 2021-07-11T04:02:09.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 -- 172.21.2.104:0/3522018785 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ddc14a9a0 msgr2=0x7f3ddc157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:09.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ddc14a9a0 0x7f3ddc157390 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f3ddc0b2f50 tx=0x7f3dcc015000).stop 2021-07-11T04:02:09.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 -- 172.21.2.104:0/3522018785 shutdown_connections 2021-07-11T04:02:09.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3ddc0aa7e0 0x7f3ddc0aac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3dc0053270 0x7f3dc0055730 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3ddc14a9a0 0x7f3ddc157390 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 --2- 172.21.2.104:0/3522018785 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3dbc004500 0x7f3ddc14ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 -- 172.21.2.104:0/3522018785 >> 172.21.2.104:0/3522018785 conn(0x7f3ddc05f6b0 msgr2=0x7f3ddc14be40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:09.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 -- 172.21.2.104:0/3522018785 shutdown_connections 2021-07-11T04:02:09.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.748+0000 7f3de1554700 1 -- 172.21.2.104:0/3522018785 wait complete. 2021-07-11T04:02:09.756 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps' 2021-07-11T04:02:09.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 Processor -- start 2021-07-11T04:02:09.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 -- start start 2021-07-11T04:02:09.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7df8067180 0x7f7df805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f7df80b1970 0x7f7df805fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df80a7a90 con 0x7f7df80a76b0 2021-07-11T04:02:09.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df8147620 con 0x7f7df8067180 2021-07-11T04:02:09.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df81478c0 con 0x7f7df80b1970 2021-07-11T04:02:09.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7df77fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7df8067180 0x7f7df805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:09.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7df77fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7df8067180 0x7f7df805f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50818/0 (socket says 172.21.2.104:50818) 2021-07-11T04:02:09.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7df77fe700 1 -- 172.21.2.104:0/1373251003 learned_addr learned my addr 172.21.2.104:0/1373251003 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:09.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7df77fe700 1 -- 172.21.2.104:0/1373251003 >> v2:172.21.2.105:3300/0 conn(0x7f7df80b1970 msgr2=0x7f7df805fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:09.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7df77fe700 1 --2- 172.21.2.104:0/1373251003 >> v2:172.21.2.105:3300/0 conn(0x7f7df80b1970 0x7f7df805fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:09.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7df77fe700 1 -- 172.21.2.104:0/1373251003 >> v1:172.21.2.105:6789/0 conn(0x7f7df80a76b0 legacy=0x7f7df8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:09.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7df77fe700 1 -- 172.21.2.104:0/1373251003 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df8147b60 con 0x7f7df8067180 2021-07-11T04:02:09.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7df77fe700 1 --2- 172.21.2.104:0/1373251003 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7df8067180 0x7f7df805f6b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f7df806a0e0 tx=0x7f7de0006eb0).ready entity=mon.0 client_cookie=5b5167798d7c79e3 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:09.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7df6ffd700 1 -- 172.21.2.104:0/1373251003 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7de000d5c0 con 0x7f7df8067180 2021-07-11T04:02:09.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7df6ffd700 1 -- 172.21.2.104:0/1373251003 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de0000e90 con 0x7f7df8067180 2021-07-11T04:02:09.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7df6ffd700 1 -- 172.21.2.104:0/1373251003 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7de000d910 con 0x7f7df8067180 2021-07-11T04:02:09.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/1373251003 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7df8067180 msgr2=0x7f7df805f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:09.813 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 --2- 172.21.2.104:0/1373251003 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7df8067180 0x7f7df805f6b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f7df806a0e0 tx=0x7f7de0006eb0).stop 2021-07-11T04:02:09.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/1373251003 shutdown_connections 2021-07-11T04:02:09.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 --2- 172.21.2.104:0/1373251003 >> v2:172.21.2.105:3300/0 conn(0x7f7df80b1970 0x7f7df805fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:09.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 --2- 172.21.2.104:0/1373251003 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7df8067180 0x7f7df805f6b0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:09.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/1373251003 >> 172.21.2.104:0/1373251003 conn(0x7f7df80b53e0 msgr2=0x7f7df80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:09.814 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/1373251003 shutdown_connections 2021-07-11T04:02:09.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.808+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/1373251003 wait complete. 2021-07-11T04:02:09.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7dfe3c3700 1 Processor -- start 2021-07-11T04:02:09.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7dfe3c3700 1 -- start start 2021-07-11T04:02:09.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7dfe3c3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7df8067180 0x7f7df80ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7dfe3c3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7df80a76b0 0x7f7df80ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7dfe3c3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7df80b1970 0x7f7df80ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7dfe3c3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df80ac100 con 0x7f7df80a76b0 2021-07-11T04:02:09.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7dfe3c3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df80ac6a0 con 0x7f7df80b1970 2021-07-11T04:02:09.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7dfe3c3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df80ac940 con 0x7f7df8067180 2021-07-11T04:02:09.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7df80a76b0 0x7f7df80ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:09.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7df80a76b0 0x7f7df80ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50822/0 (socket says 172.21.2.104:50822) 2021-07-11T04:02:09.819 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df77fe700 1 -- 172.21.2.104:0/91988777 learned_addr learned my addr 172.21.2.104:0/91988777 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:09.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df77fe700 1 -- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7df8067180 msgr2=0x7f7df80ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:09.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df77fe700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7df8067180 0x7f7df80ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:09.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df77fe700 1 -- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7df80b1970 msgr2=0x7f7df80ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:09.820 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df77fe700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7df80b1970 0x7f7df80ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:09.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df77fe700 1 -- 172.21.2.104:0/91988777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de000b1e0 con 0x7f7df80a76b0 2021-07-11T04:02:09.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df77fe700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7df80a76b0 0x7f7df80ae190 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f7df806a0e0 tx=0x7f7de0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:09.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df57fa700 1 -- 172.21.2.104:0/91988777 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7de0021070 con 0x7f7df80a76b0 2021-07-11T04:02:09.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df57fa700 1 -- 172.21.2.104:0/91988777 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de0000e90 con 0x7f7df80a76b0 2021-07-11T04:02:09.821 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/91988777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df80aeb70 con 0x7f7df80a76b0 2021-07-11T04:02:09.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df57fa700 1 -- 172.21.2.104:0/91988777 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7de0006be0 con 0x7f7df80a76b0 2021-07-11T04:02:09.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/91988777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df80abd80 con 0x7f7df80a76b0 2021-07-11T04:02:09.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/91988777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df80aaa60 con 0x7f7df80a76b0 2021-07-11T04:02:09.822 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df57fa700 1 -- 172.21.2.104:0/91988777 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f7de000f070 con 0x7f7df80a76b0 2021-07-11T04:02:09.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df57fa700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7ddc053210 0x7f7ddc0556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:09.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.812+0000 7f7df57fa700 1 -- 172.21.2.104:0/91988777 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7de006c770 con 0x7f7df80a76b0 2021-07-11T04:02:09.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.816+0000 7f7df7fff700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7ddc053210 0x7f7ddc0556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:09.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.816+0000 7f7df7fff700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7ddc053210 0x7f7ddc0556d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f7de8000f70 tx=0x7f7de80077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:09.823 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.816+0000 7f7df57fa700 1 -- 172.21.2.104:0/91988777 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7df80aaa60 con 0x7f7df80a76b0 2021-07-11T04:02:09.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:09.928+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/91988777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]} v 0) v1 -- 0x7f7df80aaa60 con 0x7f7df80a76b0 2021-07-11T04:02:10.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.444+0000 7f7df57fa700 1 -- 172.21.2.104:0/91988777 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_getattr_caps"]}]=0 v388) v1 ==== 128+0+0 (secure 0 0 0) 0x7f7df80aaa60 con 0x7f7df80a76b0 2021-07-11T04:02:10.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7ddc053210 msgr2=0x7f7ddc0556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:10.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7ddc053210 0x7f7ddc0556d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f7de8000f70 tx=0x7f7de80077f0).stop 2021-07-11T04:02:10.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/91988777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7df80a76b0 msgr2=0x7f7df80ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:10.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7df80a76b0 0x7f7df80ae190 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f7df806a0e0 tx=0x7f7de0015000).stop 2021-07-11T04:02:10.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/91988777 shutdown_connections 2021-07-11T04:02:10.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7df80b1970 0x7f7df80ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:10.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7ddc053210 0x7f7ddc0556d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:10.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7df80a76b0 0x7f7df80ae190 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:10.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 --2- 172.21.2.104:0/91988777 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7df8067180 0x7f7df80ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:10.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/91988777 >> 172.21.2.104:0/91988777 conn(0x7f7df80b53e0 msgr2=0x7f7df80b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:10.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/91988777 shutdown_connections 2021-07-11T04:02:10.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.448+0000 7f7dfe3c3700 1 -- 172.21.2.104:0/91988777 wait complete. 2021-07-11T04:02:10.454 INFO:tasks.cephfs_test_runner:test_getattr_caps (tasks.cephfs.test_misc.TestMisc) ... ok 2021-07-11T04:02:10.455 INFO:tasks.cephfs_test_runner:Starting test: test_root_rctime (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:02:10.456 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2021-07-11T04:02:10.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.516+0000 7f8931b6e700 1 Processor -- start 2021-07-11T04:02:10.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.516+0000 7f8931b6e700 1 -- start start 2021-07-11T04:02:10.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f892c1483a0 0x7f892c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:10.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f892c0a70c0 0x7f892c144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:10.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f892c0b0350 con 0x7f892c0b1db0 2021-07-11T04:02:10.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892c0a90c0 con 0x7f892c0a70c0 2021-07-11T04:02:10.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892c0b05f0 con 0x7f892c1483a0 2021-07-11T04:02:10.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892b7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f892c0b1db0 0x7f892c0b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47596/0 (socket says 172.21.2.104:47596) 2021-07-11T04:02:10.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892b7fe700 1 -- 172.21.2.104:0/103100546 learned_addr learned my addr 172.21.2.104:0/103100546 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:10.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892bfff700 1 --2- 172.21.2.104:0/103100546 >> v2:172.21.2.104:3300/0 conn(0x7f892c0a70c0 0x7f892c144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:10.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892bfff700 1 -- 172.21.2.104:0/103100546 >> v2:172.21.2.105:3300/0 conn(0x7f892c1483a0 msgr2=0x7f892c0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:10.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892bfff700 1 --2- 172.21.2.104:0/103100546 >> v2:172.21.2.105:3300/0 conn(0x7f892c1483a0 0x7f892c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:10.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892bfff700 1 -- 172.21.2.104:0/103100546 >> v1:172.21.2.104:6789/0 conn(0x7f892c0b1db0 legacy=0x7f892c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:10.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892bfff700 1 -- 172.21.2.104:0/103100546 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c0b0890 con 0x7f892c0a70c0 2021-07-11T04:02:10.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892bfff700 1 --2- 172.21.2.104:0/103100546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f892c0a70c0 0x7f892c144720 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f892c147730 tx=0x7f89240091d0).ready entity=mon.0 client_cookie=88117469439fb877 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:10.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892a7fc700 1 -- 172.21.2.104:0/103100546 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8924009ea0 con 0x7f892c0a70c0 2021-07-11T04:02:10.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892a7fc700 1 -- 172.21.2.104:0/103100546 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892400b860 con 0x7f892c0a70c0 2021-07-11T04:02:10.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892a7fc700 1 -- 172.21.2.104:0/103100546 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f892400b9d0 con 0x7f892c0a70c0 2021-07-11T04:02:10.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- 172.21.2.104:0/103100546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f892c0a70c0 msgr2=0x7f892c144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:10.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 --2- 172.21.2.104:0/103100546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f892c0a70c0 0x7f892c144720 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f892c147730 tx=0x7f89240091d0).stop 2021-07-11T04:02:10.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- 172.21.2.104:0/103100546 shutdown_connections 2021-07-11T04:02:10.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 --2- 172.21.2.104:0/103100546 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f892c0a70c0 0x7f892c144720 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:10.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 --2- 172.21.2.104:0/103100546 >> v2:172.21.2.105:3300/0 conn(0x7f892c1483a0 0x7f892c0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:10.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- 172.21.2.104:0/103100546 >> 172.21.2.104:0/103100546 conn(0x7f892c05f6b0 msgr2=0x7f892c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:10.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- 172.21.2.104:0/103100546 shutdown_connections 2021-07-11T04:02:10.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- 172.21.2.104:0/103100546 wait complete. 2021-07-11T04:02:10.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 Processor -- start 2021-07-11T04:02:10.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- start start 2021-07-11T04:02:10.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f892c0a70c0 0x7f892c0af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:10.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f892c0b1db0 0x7f892c1455a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:10.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f892c1483a0 0x7f892c146430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:10.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0abc10 con 0x7f892c0b1db0 2021-07-11T04:02:10.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c145a90 con 0x7f892c0a70c0 2021-07-11T04:02:10.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0aaf90 con 0x7f892c1483a0 2021-07-11T04:02:10.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f892c0b1db0 0x7f892c1455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:10.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f892c0b1db0 0x7f892c1455a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50836/0 (socket says 172.21.2.104:50836) 2021-07-11T04:02:10.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892affd700 1 -- 172.21.2.104:0/317028316 learned_addr learned my addr 172.21.2.104:0/317028316 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:10.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892affd700 1 -- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f892c1483a0 msgr2=0x7f892c146430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:10.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892affd700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f892c1483a0 0x7f892c146430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:10.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892affd700 1 -- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f892c0a70c0 msgr2=0x7f892c0af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:10.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892affd700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f892c0a70c0 0x7f892c0af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:10.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892affd700 1 -- 172.21.2.104:0/317028316 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8924009470 con 0x7f892c0b1db0 2021-07-11T04:02:10.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f892affd700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f892c0b1db0 0x7f892c1455a0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f891c002f70 tx=0x7f891c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:10.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8928ff9700 1 -- 172.21.2.104:0/317028316 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f891c013460 con 0x7f892c0b1db0 2021-07-11T04:02:10.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8928ff9700 1 -- 172.21.2.104:0/317028316 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c0135c0 con 0x7f892c0b1db0 2021-07-11T04:02:10.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- 172.21.2.104:0/317028316 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892c0ab230 con 0x7f892c0b1db0 2021-07-11T04:02:10.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8931b6e700 1 -- 172.21.2.104:0/317028316 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892c0ab770 con 0x7f892c0b1db0 2021-07-11T04:02:10.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.520+0000 7f8928ff9700 1 -- 172.21.2.104:0/317028316 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f891c013920 con 0x7f892c0b1db0 2021-07-11T04:02:10.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.524+0000 7f8931b6e700 1 -- 172.21.2.104:0/317028316 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892c0ac2b0 con 0x7f892c0b1db0 2021-07-11T04:02:10.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.524+0000 7f8928ff9700 1 -- 172.21.2.104:0/317028316 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f891c01a060 con 0x7f892c0b1db0 2021-07-11T04:02:10.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.524+0000 7f8928ff9700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8910053350 0x7f8910055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:10.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.524+0000 7f8928ff9700 1 -- 172.21.2.104:0/317028316 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f891c06ace0 con 0x7f892c0b1db0 2021-07-11T04:02:10.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.524+0000 7f892b7fe700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8910053350 0x7f8910055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:10.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.524+0000 7f892b7fe700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8910053350 0x7f8910055810 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f892c0ace60 tx=0x7f8914007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:10.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.524+0000 7f8928ff9700 1 -- 172.21.2.104:0/317028316 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f892c0ac2b0 con 0x7f892c0b1db0 2021-07-11T04:02:10.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:10.636+0000 7f8931b6e700 1 -- 172.21.2.104:0/317028316 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f892c0ac2b0 con 0x7f892c0b1db0 2021-07-11T04:02:11.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.448+0000 7f8928ff9700 1 -- 172.21.2.104:0/317028316 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v389) v1 ==== 130+0+0 (secure 0 0 0) 0x7f892c0ac2b0 con 0x7f892c0b1db0 2021-07-11T04:02:11.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 -- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8910053350 msgr2=0x7f8910055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:11.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8910053350 0x7f8910055810 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f892c0ace60 tx=0x7f8914007800).stop 2021-07-11T04:02:11.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 -- 172.21.2.104:0/317028316 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f892c0b1db0 msgr2=0x7f892c1455a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:11.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f892c0b1db0 0x7f892c1455a0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 rx=0x7f891c002f70 tx=0x7f891c006df0).stop 2021-07-11T04:02:11.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 -- 172.21.2.104:0/317028316 shutdown_connections 2021-07-11T04:02:11.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f892c1483a0 0x7f892c146430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:11.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8910053350 0x7f8910055810 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:11.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f892c0b1db0 0x7f892c1455a0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:11.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 --2- 172.21.2.104:0/317028316 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f892c0a70c0 0x7f892c0af410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:11.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 -- 172.21.2.104:0/317028316 >> 172.21.2.104:0/317028316 conn(0x7f892c05f6b0 msgr2=0x7f892c0b3920 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:11.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 -- 172.21.2.104:0/317028316 shutdown_connections 2021-07-11T04:02:11.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.452+0000 7f8931b6e700 1 -- 172.21.2.104:0/317028316 wait complete. 2021-07-11T04:02:11.459 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T04:02:11.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.512+0000 7f0a1901a700 1 Processor -- start 2021-07-11T04:02:11.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.512+0000 7f0a1901a700 1 -- start start 2021-07-11T04:02:11.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0a140b4690 0x7f0a14140940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:11.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a14143b60 con 0x7f0a140b4180 2021-07-11T04:02:11.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a14143e00 con 0x7f0a14146950 2021-07-11T04:02:11.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a14147510 con 0x7f0a140b4690 2021-07-11T04:02:11.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1259c700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f0a140b4180 0x7f0a140b4ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47612/0 (socket says 172.21.2.104:47612) 2021-07-11T04:02:11.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1259c700 1 -- 172.21.2.104:0/1853583138 learned_addr learned my addr 172.21.2.104:0/1853583138 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:11.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 --2- 172.21.2.104:0/1853583138 >> v2:172.21.2.104:3300/0 conn(0x7f0a140b4690 0x7f0a14140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:11.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 -- 172.21.2.104:0/1853583138 >> v1:172.21.2.105:6789/0 conn(0x7f0a14146950 legacy=0x7f0a14140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:11.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 -- 172.21.2.104:0/1853583138 >> v1:172.21.2.104:6789/0 conn(0x7f0a140b4180 legacy=0x7f0a140b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:11.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 -- 172.21.2.104:0/1853583138 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a140b0c90 con 0x7f0a140b4690 2021-07-11T04:02:11.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 --2- 172.21.2.104:0/1853583138 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140b4690 0x7f0a14140940 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f0a14060c90 tx=0x7f0a0c0091d0).ready entity=mon.0 client_cookie=bfe4e8519b10f66 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:11.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a11d9b700 1 -- 172.21.2.104:0/1853583138 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a0c009ea0 con 0x7f0a140b4690 2021-07-11T04:02:11.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a11d9b700 1 -- 172.21.2.104:0/1853583138 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c00b860 con 0x7f0a140b4690 2021-07-11T04:02:11.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a11d9b700 1 -- 172.21.2.104:0/1853583138 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a0c00b9d0 con 0x7f0a140b4690 2021-07-11T04:02:11.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- 172.21.2.104:0/1853583138 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140b4690 msgr2=0x7f0a14140940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:11.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 --2- 172.21.2.104:0/1853583138 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140b4690 0x7f0a14140940 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7f0a14060c90 tx=0x7f0a0c0091d0).stop 2021-07-11T04:02:11.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- 172.21.2.104:0/1853583138 shutdown_connections 2021-07-11T04:02:11.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 --2- 172.21.2.104:0/1853583138 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a140b4690 0x7f0a14140940 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:11.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- 172.21.2.104:0/1853583138 >> 172.21.2.104:0/1853583138 conn(0x7f0a1405c490 msgr2=0x7f0a1405c870 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:11.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- 172.21.2.104:0/1853583138 shutdown_connections 2021-07-11T04:02:11.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- 172.21.2.104:0/1853583138 wait complete. 2021-07-11T04:02:11.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 Processor -- start 2021-07-11T04:02:11.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- start start 2021-07-11T04:02:11.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a140b4180 0x7f0a140b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:11.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a140b4690 0x7f0a140ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:11.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a14146950 0x7f0a140ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:11.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a14145ff0 con 0x7f0a14146950 2021-07-11T04:02:11.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140aea60 con 0x7f0a140b4180 2021-07-11T04:02:11.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a140b7680 con 0x7f0a140b4690 2021-07-11T04:02:11.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a14146950 0x7f0a140ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:11.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a14146950 0x7f0a140ae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50854/0 (socket says 172.21.2.104:50854) 2021-07-11T04:02:11.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 -- 172.21.2.104:0/1461373680 learned_addr learned my addr 172.21.2.104:0/1461373680 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:11.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 -- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a140b4690 msgr2=0x7f0a140ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:11.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a140b4690 0x7f0a140ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:11.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 -- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a140b4180 msgr2=0x7f0a140b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:11.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a140b4180 0x7f0a140b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:11.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 -- 172.21.2.104:0/1461373680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c009470 con 0x7f0a14146950 2021-07-11T04:02:11.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1359e700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a14146950 0x7f0a140ae570 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7f0a140aeba0 tx=0x7f0a0c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:11.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a0bfff700 1 -- 172.21.2.104:0/1461373680 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a0c021480 con 0x7f0a14146950 2021-07-11T04:02:11.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a0bfff700 1 -- 172.21.2.104:0/1461373680 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0c0215e0 con 0x7f0a14146950 2021-07-11T04:02:11.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- 172.21.2.104:0/1461373680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a141458f0 con 0x7f0a14146950 2021-07-11T04:02:11.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a0bfff700 1 -- 172.21.2.104:0/1461373680 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a0c021940 con 0x7f0a14146950 2021-07-11T04:02:11.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.516+0000 7f0a1901a700 1 -- 172.21.2.104:0/1461373680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a140adb40 con 0x7f0a14146950 2021-07-11T04:02:11.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.520+0000 7f0a1901a700 1 -- 172.21.2.104:0/1461373680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a14057350 con 0x7f0a14146950 2021-07-11T04:02:11.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.520+0000 7f0a0bfff700 1 -- 172.21.2.104:0/1461373680 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0a0c00f070 con 0x7f0a14146950 2021-07-11T04:02:11.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.520+0000 7f0a0bfff700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f8053270 0x7f09f8055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:11.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.520+0000 7f0a0bfff700 1 -- 172.21.2.104:0/1461373680 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0a0c06c410 con 0x7f0a14146950 2021-07-11T04:02:11.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.520+0000 7f0a12d9d700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f8053270 0x7f09f8055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:11.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.520+0000 7f0a12d9d700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f8053270 0x7f09f8055730 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f0a04002f10 tx=0x7f0a04006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:11.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.520+0000 7f0a0bfff700 1 -- 172.21.2.104:0/1461373680 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0a0c043a30 con 0x7f0a14146950 2021-07-11T04:02:11.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.604+0000 7f0a1901a700 1 -- 172.21.2.104:0/1461373680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0a140b73e0 con 0x7f0a14146950 2021-07-11T04:02:11.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a0bfff700 1 -- 172.21.2.104:0/1461373680 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0a0c0260c0 con 0x7f0a14146950 2021-07-11T04:02:11.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 -- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f8053270 msgr2=0x7f09f8055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:11.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f8053270 0x7f09f8055730 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f0a04002f10 tx=0x7f0a04006930).stop 2021-07-11T04:02:11.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 -- 172.21.2.104:0/1461373680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a14146950 msgr2=0x7f0a140ae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:11.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a14146950 0x7f0a140ae570 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7f0a140aeba0 tx=0x7f0a0c016000).stop 2021-07-11T04:02:11.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 -- 172.21.2.104:0/1461373680 shutdown_connections 2021-07-11T04:02:11.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f09f8053270 0x7f09f8055730 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:11.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a14146950 0x7f0a140ae570 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:11.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a140b4690 0x7f0a140ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:11.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 --2- 172.21.2.104:0/1461373680 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a140b4180 0x7f0a140b69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:11.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 -- 172.21.2.104:0/1461373680 >> 172.21.2.104:0/1461373680 conn(0x7f0a1405c490 msgr2=0x7f0a1413e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:11.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 -- 172.21.2.104:0/1461373680 shutdown_connections 2021-07-11T04:02:11.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.608+0000 7f0a1901a700 1 -- 172.21.2.104:0/1461373680 wait complete. 2021-07-11T04:02:11.618 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:02:11.619 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:11.627 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:11.636 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:11.650 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:11.664 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:02:11.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:11.665 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:11.666 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:11.676 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T04:02:11.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:02:11.677 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T04:02:11.678 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:11.678 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:11.679 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:11.712 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T04:02:11.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:02:11.714 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:11.714 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:02:11.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b57407700 1 Processor -- start 2021-07-11T04:02:11.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b57407700 1 -- start start 2021-07-11T04:02:11.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b57407700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b500b1db0 0x7f5b500b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:11.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b57407700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5b500a70c0 0x7f5b50144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:11.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b57407700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b500b0350 con 0x7f5b501483a0 2021-07-11T04:02:11.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b57407700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b500a90c0 con 0x7f5b500a70c0 2021-07-11T04:02:11.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b57407700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b500b05f0 con 0x7f5b500b1db0 2021-07-11T04:02:11.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b559a4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5b500a70c0 0x7f5b50144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:11.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b559a4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5b500a70c0 0x7f5b50144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50858/0 (socket says 172.21.2.104:50858) 2021-07-11T04:02:11.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b559a4700 1 -- 172.21.2.104:0/2610426218 learned_addr learned my addr 172.21.2.104:0/2610426218 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:11.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b559a4700 1 -- 172.21.2.104:0/2610426218 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b500b1db0 msgr2=0x7f5b500b0fd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:11.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b559a4700 1 --2- 172.21.2.104:0/2610426218 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b500b1db0 0x7f5b500b0fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:11.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b559a4700 1 -- 172.21.2.104:0/2610426218 >> v1:172.21.2.105:6789/0 conn(0x7f5b501483a0 legacy=0x7f5b500b2190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:11.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b559a4700 1 -- 172.21.2.104:0/2610426218 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b500b0890 con 0x7f5b500a70c0 2021-07-11T04:02:11.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b559a4700 1 --2- 172.21.2.104:0/2610426218 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b500a70c0 0x7f5b50144720 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7f5b500613a0 tx=0x7f5b48009290).ready entity=mon.0 client_cookie=48c3dc7fc2744c97 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:11.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b43fff700 1 -- 172.21.2.104:0/2610426218 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4800b860 con 0x7f5b500a70c0 2021-07-11T04:02:11.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b43fff700 1 -- 172.21.2.104:0/2610426218 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4800b9c0 con 0x7f5b500a70c0 2021-07-11T04:02:11.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b43fff700 1 -- 172.21.2.104:0/2610426218 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4800bb70 con 0x7f5b500a70c0 2021-07-11T04:02:11.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b57407700 1 -- 172.21.2.104:0/2610426218 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b500a70c0 msgr2=0x7f5b50144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:11.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.772+0000 7f5b57407700 1 --2- 172.21.2.104:0/2610426218 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b500a70c0 0x7f5b50144720 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7f5b500613a0 tx=0x7f5b48009290).stop 2021-07-11T04:02:11.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 -- 172.21.2.104:0/2610426218 shutdown_connections 2021-07-11T04:02:11.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 --2- 172.21.2.104:0/2610426218 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b500a70c0 0x7f5b50144720 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:11.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 --2- 172.21.2.104:0/2610426218 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b500b1db0 0x7f5b500b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:11.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 -- 172.21.2.104:0/2610426218 >> 172.21.2.104:0/2610426218 conn(0x7f5b5005f6b0 msgr2=0x7f5b5005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:11.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 -- 172.21.2.104:0/2610426218 shutdown_connections 2021-07-11T04:02:11.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 -- 172.21.2.104:0/2610426218 wait complete. 2021-07-11T04:02:11.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 Processor -- start 2021-07-11T04:02:11.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 -- start start 2021-07-11T04:02:11.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b500a70c0 0x7f5b50062b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:11.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b500b1db0 0x7f5b500ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:11.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b501483a0 0x7f5b5015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:11.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b500aa930 con 0x7f5b500b1db0 2021-07-11T04:02:11.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b500ab520 con 0x7f5b500a70c0 2021-07-11T04:02:11.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b500aab60 con 0x7f5b501483a0 2021-07-11T04:02:11.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b549a2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b500b1db0 0x7f5b500ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:11.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b549a2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b500b1db0 0x7f5b500ac0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50866/0 (socket says 172.21.2.104:50866) 2021-07-11T04:02:11.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b549a2700 1 -- 172.21.2.104:0/395506821 learned_addr learned my addr 172.21.2.104:0/395506821 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:11.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b549a2700 1 -- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b501483a0 msgr2=0x7f5b5015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:11.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b549a2700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b501483a0 0x7f5b5015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:11.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b549a2700 1 -- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b500a70c0 msgr2=0x7f5b50062b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:11.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b549a2700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b500a70c0 0x7f5b50062b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:11.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b549a2700 1 -- 172.21.2.104:0/395506821 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b48009530 con 0x7f5b500b1db0 2021-07-11T04:02:11.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b549a2700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b500b1db0 0x7f5b500ac0f0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f5b4c000ed0 tx=0x7f5b4c006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:11.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b427fc700 1 -- 172.21.2.104:0/395506821 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4c009e00 con 0x7f5b500b1db0 2021-07-11T04:02:11.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b427fc700 1 -- 172.21.2.104:0/395506821 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b4c012410 con 0x7f5b500b1db0 2021-07-11T04:02:11.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 -- 172.21.2.104:0/395506821 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b500ab7c0 con 0x7f5b500b1db0 2021-07-11T04:02:11.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 -- 172.21.2.104:0/395506821 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b500ac5e0 con 0x7f5b500b1db0 2021-07-11T04:02:11.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b427fc700 1 -- 172.21.2.104:0/395506821 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5b4c012770 con 0x7f5b500b1db0 2021-07-11T04:02:11.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b57407700 1 -- 172.21.2.104:0/395506821 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b500aad20 con 0x7f5b500b1db0 2021-07-11T04:02:11.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b427fc700 1 -- 172.21.2.104:0/395506821 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5b4c00f030 con 0x7f5b500b1db0 2021-07-11T04:02:11.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b427fc700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b30053240 0x7f5b30055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:11.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.776+0000 7f5b427fc700 1 -- 172.21.2.104:0/395506821 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5b4c06af00 con 0x7f5b500b1db0 2021-07-11T04:02:11.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.780+0000 7f5b551a3700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b30053240 0x7f5b30055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:11.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.780+0000 7f5b551a3700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b30053240 0x7f5b30055700 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f5b44002fb0 tx=0x7f5b440069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:11.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.780+0000 7f5b427fc700 1 -- 172.21.2.104:0/395506821 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5b4c042410 con 0x7f5b500b1db0 2021-07-11T04:02:11.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.860+0000 7f5b57407700 1 -- 172.21.2.104:0/395506821 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5b50057350 con 0x7f5b500b1db0 2021-07-11T04:02:11.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b427fc700 1 -- 172.21.2.104:0/395506821 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v142) v1 ==== 81+0+28837 (secure 0 0 0) 0x7f5b4c045a30 con 0x7f5b500b1db0 2021-07-11T04:02:11.863 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:11.863 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:02:11.863 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 142, 2021-07-11T04:02:11.864 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:02:11.864 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:02:11.864 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:02:09.426678+0000", 2021-07-11T04:02:11.864 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:02:11.864 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:02:11.865 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:02:11.865 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:02:11.865 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:02:11.866 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:02:11.866 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:02:11.866 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:02:11.867 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:02:11.867 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:11.867 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:02:11.867 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:02:11.868 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:02:11.868 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:02:11.868 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:02:11.868 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 26, 2021-07-11T04:02:11.868 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:02:11.869 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:02:11.869 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:02:11.869 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:02:11.869 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:02:11.870 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.870 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:02:11.870 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:02:11.870 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:02:11.870 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:11.871 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:11.871 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:11.871 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:11.871 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:11.872 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:11.872 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:11.872 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:11.872 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:02:11.872 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:02:11.873 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:02:11.873 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:02:11.873 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:02:11.873 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:11.874 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:11.874 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:11.874 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:11.874 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:11.874 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:11.875 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:11.875 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.875 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:02:11.875 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:11.876 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:11.877 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:11.877 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:11.877 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:11.877 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:11.877 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:11.878 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:11.878 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:11.878 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:11.879 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:11.879 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:11.879 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:11.879 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:11.880 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:11.880 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:11.880 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:11.880 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:11.880 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:11.881 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:11.881 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:11.881 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:11.881 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:11.882 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:11.882 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:11.882 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:11.882 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.882 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:11.883 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:11.883 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:11.883 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:11.883 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:11.884 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:11.884 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:11.884 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:11.884 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:11.884 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:11.885 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:11.885 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:02:11.885 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:02:11.885 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.886 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:11.886 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:02:11.886 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.886 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.887 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.887 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:02:11.887 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:02:11.887 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:02:11.888 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:11.888 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:11.888 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:11.888 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:11.889 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:11.889 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:11.889 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:11.889 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:11.889 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:02:11.890 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:02:11.890 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:02:11.890 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:02:11.890 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:02:11.891 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:11.891 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:11.891 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:11.891 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:11.891 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:11.892 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:11.892 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:11.892 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.892 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:02:11.892 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:11.893 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:11.893 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:11.893 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:11.893 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:11.894 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:11.894 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:11.894 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:11.894 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:11.894 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:11.895 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:11.895 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:11.895 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:11.895 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:11.896 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:11.896 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:11.896 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:11.896 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:11.896 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:11.897 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:11.898 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:11.898 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:11.898 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:11.898 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:11.898 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:11.899 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:11.899 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.899 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:11.899 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:11.900 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:11.900 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:11.900 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:11.900 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:11.900 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:11.901 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:11.901 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:11.901 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:11.901 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:11.902 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:02:11.902 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:11.902 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:02:11.902 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.902 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.903 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:11.903 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:02:11.903 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.903 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:02:11.904 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:02:11.904 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:11.904 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:11.904 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:11.904 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:11.905 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:11.905 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:11.905 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:11.905 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 136, 2021-07-11T04:02:11.905 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:11.906 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:11.906 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:11.906 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.906 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.907 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.907 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:02:11.907 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:11.907 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.907 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.908 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.908 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:02:11.908 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:11.908 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.908 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.909 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.909 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:11.909 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.909 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.910 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.910 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:02:11.910 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:11.910 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.910 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.911 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.911 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:02:11.911 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:11.911 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.912 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.912 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.912 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:11.912 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.912 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.913 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.913 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:02:11.913 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:11.913 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.913 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.914 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.914 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:02:11.914 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:11.914 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.915 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.915 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.915 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:11.915 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.915 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.916 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.916 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:02:11.916 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:11.916 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.916 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.917 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.917 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:02:11.917 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:11.917 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.918 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.918 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.918 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:02:11.918 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:02:11.918 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:02:11.919 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:02:11.919 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:11.919 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:11.919 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:11.919 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.920 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.920 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.920 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:02:11.920 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:02:11.921 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:11.921 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:11.921 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:11.921 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:11.921 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:11.922 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:11.922 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:11.922 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 137, 2021-07-11T04:02:11.922 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:11.923 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:11.923 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:11.923 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.923 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.923 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.924 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:02:11.924 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:11.924 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.924 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.924 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.925 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:02:11.925 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:11.925 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.925 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.926 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.926 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:11.926 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.926 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.926 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.927 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:02:11.927 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:11.927 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.927 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.928 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.928 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:02:11.928 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:11.928 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.928 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.929 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.929 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:11.929 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.929 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.930 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.930 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:02:11.930 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:11.931 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.931 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.931 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.931 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:02:11.931 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:11.932 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.932 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.932 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.932 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:11.933 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.933 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.933 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.933 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:02:11.933 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:11.934 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.934 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.934 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.934 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:02:11.935 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:11.935 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.935 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.935 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.935 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:02:11.936 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:02:11.936 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:02:11.936 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:02:11.936 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:11.937 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:11.937 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:11.937 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.937 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.937 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.938 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:02:11.938 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:02:11.938 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:11.938 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:11.939 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:11.939 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:11.939 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:11.939 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:11.939 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:11.940 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 137, 2021-07-11T04:02:11.940 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:11.940 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:11.940 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:11.941 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.941 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.941 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.941 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:02:11.941 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:11.942 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.942 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.942 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.942 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:02:11.943 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:11.943 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.943 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.943 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.944 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:11.944 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.944 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.944 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.944 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:02:11.945 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:11.945 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.945 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.945 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.946 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:02:11.946 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:11.946 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.946 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.946 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.947 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:11.947 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.947 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.947 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.948 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:02:11.948 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:11.948 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.948 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.948 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.949 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:02:11.949 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:11.949 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.949 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.949 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.950 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:11.950 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.950 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.950 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.951 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:02:11.951 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:11.951 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.951 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.951 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.952 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:02:11.952 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:11.952 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.952 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.952 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.953 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:02:11.953 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:02:11.953 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:02:11.953 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:02:11.954 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:11.954 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:11.954 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:11.954 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.954 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.955 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.955 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:02:11.955 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:02:11.955 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:11.955 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:11.956 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:11.956 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:11.956 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:11.956 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:11.957 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:11.957 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 136, 2021-07-11T04:02:11.957 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:11.957 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:11.957 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:11.958 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.958 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.958 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.958 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:02:11.959 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:11.959 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.959 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.959 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.959 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:02:11.960 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:11.960 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.960 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.960 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.961 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:11.961 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.961 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.961 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.961 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:02:11.962 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:11.962 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.962 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.962 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.962 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:02:11.963 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:11.963 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.963 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.963 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.964 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:11.964 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.964 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.964 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.964 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:02:11.965 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:11.965 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.965 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.965 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.965 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:02:11.966 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:11.966 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.966 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.966 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.967 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:11.967 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.967 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.967 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.967 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:02:11.968 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:11.968 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.968 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.968 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.969 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:02:11.969 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:11.969 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.969 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.969 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.970 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:02:11.970 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:02:11.970 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:02:11.970 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:02:11.971 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:11.971 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:11.971 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:11.971 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.971 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.972 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.972 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:02:11.972 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:02:11.972 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:11.973 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:11.973 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:11.973 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:11.973 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:11.973 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:11.974 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:11.974 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 137, 2021-07-11T04:02:11.974 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:11.974 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:11.974 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:11.975 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.975 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.975 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.975 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:02:11.976 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:11.976 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.976 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.976 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.976 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:02:11.977 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:11.977 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.977 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.977 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.978 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:11.978 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.978 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.978 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.978 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:02:11.979 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:11.979 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.979 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.980 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.980 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:02:11.980 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:11.980 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.981 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.981 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.981 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:11.981 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.981 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.982 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.982 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:02:11.982 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:11.982 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.983 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.983 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.983 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:02:11.983 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:11.983 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.984 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.984 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.984 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:11.984 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.984 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.985 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.985 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:02:11.985 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:11.985 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.986 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.986 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.986 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:02:11.986 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:11.986 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:11.987 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.987 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.987 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:02:11.987 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:02:11.988 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:02:11.988 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:02:11.988 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:11.988 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:11.988 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:11.989 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:11.989 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.990 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.990 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:02:11.991 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:02:11.991 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:11.991 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:11.992 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:11.992 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:11.993 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:11.993 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:11.994 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:11.994 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 130, 2021-07-11T04:02:11.995 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:11.995 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:11.996 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:11.996 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:11.996 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.997 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:11.997 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:02:11.998 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:11.998 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:11.999 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:11.999 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:11.999 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:02:12.000 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:12.000 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.000 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.000 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.001 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:12.001 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:12.001 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.002 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:12.002 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:02:12.002 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:12.003 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.003 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.003 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:12.004 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:02:12.004 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:12.005 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.005 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.006 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.006 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:12.006 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:12.007 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.007 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:12.007 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:02:12.008 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:12.008 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.008 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.009 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:12.009 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:02:12.009 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:12.010 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.010 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.010 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.011 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:12.011 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:12.012 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.012 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:12.012 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:02:12.013 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:12.013 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.013 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.014 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:12.014 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:02:12.014 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:12.015 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.015 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.015 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.016 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:02:12.016 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:02:12.016 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:02:12.017 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:02:12.017 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:12.017 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:12.018 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:12.018 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.019 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.019 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.019 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:02:12.020 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:02:12.020 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:12.021 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:12.021 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:12.021 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:12.022 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:12.022 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:12.023 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:12.023 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 137, 2021-07-11T04:02:12.023 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:12.024 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:12.024 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:12.025 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:12.025 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.025 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:12.026 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:02:12.026 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:12.027 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.027 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.027 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:12.027 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:02:12.028 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:12.028 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.028 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.028 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.028 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:12.029 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:12.029 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.029 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:12.029 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:02:12.029 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:12.030 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.030 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.030 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:12.030 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:02:12.030 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:12.031 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.031 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.031 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.031 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:12.032 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:12.032 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.032 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:12.032 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:02:12.032 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:12.033 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.033 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.033 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:12.033 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:02:12.033 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:12.034 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.034 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.034 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.034 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:12.034 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:12.035 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.035 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:12.035 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:02:12.035 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:12.035 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.036 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.036 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:12.036 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:02:12.036 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:12.037 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.037 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.037 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.037 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:02:12.037 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:02:12.038 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:02:12.038 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:02:12.038 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:12.038 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:12.038 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:12.039 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.039 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.039 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.039 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:02:12.040 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:02:12.040 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:12.040 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:12.040 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:12.040 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:12.041 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:12.041 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:12.041 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:12.041 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 137, 2021-07-11T04:02:12.041 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:12.042 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:12.042 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:12.042 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:12.042 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.042 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:12.043 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:02:12.043 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:12.043 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.043 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.044 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:12.044 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:02:12.044 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:12.044 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.044 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.045 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.045 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:12.045 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:12.045 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.045 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:12.046 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:02:12.046 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:12.046 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.046 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.046 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:12.047 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:02:12.047 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:12.047 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.047 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.048 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.048 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:12.048 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:12.048 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.048 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:12.049 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:02:12.049 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:12.049 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.049 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.049 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:12.050 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:02:12.050 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:12.050 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.050 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.051 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.051 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:12.051 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:12.051 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.051 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:12.052 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:02:12.052 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:12.052 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.052 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.052 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:12.053 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:02:12.053 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:12.053 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.053 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.053 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.054 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:02:12.054 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:02:12.054 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:02:12.054 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:02:12.054 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:12.055 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:12.055 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:12.055 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:12.055 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.056 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:12.056 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:02:12.056 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.056 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:02:12.056 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:12.057 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:12.057 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:12.057 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:12.057 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:12.057 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:02:12.058 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:12.058 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.058 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.058 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:02:12.058 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:12.059 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:12.059 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:12.059 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:12.059 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:12.059 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:02:12.060 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:12.060 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.060 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.060 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:02:12.060 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:12.061 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:12.061 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:12.061 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:12.061 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:12.062 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:02:12.062 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:12.062 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.062 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.062 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:02:12.063 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:12.063 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:12.063 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:12.063 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:12.063 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:12.064 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:02:12.064 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:12.064 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.064 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.064 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:02:12.065 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:12.065 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:12.065 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:12.065 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:12.066 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:12.066 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:02:12.066 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:12.066 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.066 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.067 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:02:12.067 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:12.067 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:12.067 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:12.067 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:12.068 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:12.068 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:02:12.068 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:12.068 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.068 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.069 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:02:12.069 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:12.069 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:12.069 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:12.069 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:12.070 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:12.070 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:02:12.070 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:12.070 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.071 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:12.071 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:02:12.071 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:12.071 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:12.071 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:12.072 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:12.072 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:12.072 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:02:12.072 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:12.072 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.073 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:12.073 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:02:12.073 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:02:12.073 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:02:12.073 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:02:12.074 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T04:02:12.074 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6835/2616834266": "2021-07-12T04:02:05.474650+0000", 2021-07-11T04:02:12.074 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.104:6833/2616834266": "2021-07-12T04:02:05.474650+0000" 2021-07-11T04:02:12.074 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.074 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:02:12.075 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:02:12.075 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:02:12.075 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:02:12.075 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:02:12.075 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:02:12.076 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:02:12.076 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:02:12.076 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:12.076 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:12.076 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:02:12.077 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:02:12.077 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:02:12.077 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:02:12.077 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:02:12.077 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:02:12.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 -- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b30053240 msgr2=0x7f5b30055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b30053240 0x7f5b30055700 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f5b44002fb0 tx=0x7f5b440069d0).stop 2021-07-11T04:02:12.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 -- 172.21.2.104:0/395506821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b500b1db0 msgr2=0x7f5b500ac0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b500b1db0 0x7f5b500ac0f0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f5b4c000ed0 tx=0x7f5b4c006d60).stop 2021-07-11T04:02:12.080 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 -- 172.21.2.104:0/395506821 shutdown_connections 2021-07-11T04:02:12.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5b501483a0 0x7f5b5015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5b30053240 0x7f5b30055700 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5b500b1db0 0x7f5b500ac0f0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 --2- 172.21.2.104:0/395506821 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5b500a70c0 0x7f5b50062b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 -- 172.21.2.104:0/395506821 >> 172.21.2.104:0/395506821 conn(0x7f5b5005f6b0 msgr2=0x7f5b500b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:12.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 -- 172.21.2.104:0/395506821 shutdown_connections 2021-07-11T04:02:12.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:11.864+0000 7f5b57407700 1 -- 172.21.2.104:0/395506821 wait complete. 2021-07-11T04:02:12.083 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:12.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 Processor -- start 2021-07-11T04:02:12.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 -- start start 2021-07-11T04:02:12.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f45580a76b0 0x7f4558066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45580b0060 0x7f455805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45580a7a90 con 0x7f45580b1970 2021-07-11T04:02:12.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4558147620 con 0x7f45580a76b0 2021-07-11T04:02:12.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45581478c0 con 0x7f45580b0060 2021-07-11T04:02:12.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f4557fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f45580a76b0 0x7f4558066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f4557fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f45580a76b0 0x7f4558066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50872/0 (socket says 172.21.2.104:50872) 2021-07-11T04:02:12.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f4557fff700 1 -- 172.21.2.104:0/3433676742 learned_addr learned my addr 172.21.2.104:0/3433676742 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:12.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f4557fff700 1 -- 172.21.2.104:0/3433676742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45580b0060 msgr2=0x7f455805fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:12.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f4557fff700 1 --2- 172.21.2.104:0/3433676742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45580b0060 0x7f455805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f4557fff700 1 -- 172.21.2.104:0/3433676742 >> v1:172.21.2.105:6789/0 conn(0x7f45580b1970 legacy=0x7f455805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:12.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f4557fff700 1 -- 172.21.2.104:0/3433676742 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558147b60 con 0x7f45580a76b0 2021-07-11T04:02:12.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f4557fff700 1 --2- 172.21.2.104:0/3433676742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45580a76b0 0x7f4558066a70 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f455806a0e0 tx=0x7f4548006eb0).ready entity=mon.0 client_cookie=bd0e70a009d13d5d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:12.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f45577fe700 1 -- 172.21.2.104:0/3433676742 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f454800d570 con 0x7f45580a76b0 2021-07-11T04:02:12.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f45577fe700 1 -- 172.21.2.104:0/3433676742 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4548000e90 con 0x7f45580a76b0 2021-07-11T04:02:12.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f45577fe700 1 -- 172.21.2.104:0/3433676742 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f454800d8c0 con 0x7f45580a76b0 2021-07-11T04:02:12.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 -- 172.21.2.104:0/3433676742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45580a76b0 msgr2=0x7f4558066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 --2- 172.21.2.104:0/3433676742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45580a76b0 0x7f4558066a70 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f455806a0e0 tx=0x7f4548006eb0).stop 2021-07-11T04:02:12.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 -- 172.21.2.104:0/3433676742 shutdown_connections 2021-07-11T04:02:12.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 --2- 172.21.2.104:0/3433676742 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45580b0060 0x7f455805fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 --2- 172.21.2.104:0/3433676742 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45580a76b0 0x7f4558066a70 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 -- 172.21.2.104:0/3433676742 >> 172.21.2.104:0/3433676742 conn(0x7f45580b53e0 msgr2=0x7f45580b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:12.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 -- 172.21.2.104:0/3433676742 shutdown_connections 2021-07-11T04:02:12.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.140+0000 7f455e6a7700 1 -- 172.21.2.104:0/3433676742 wait complete. 2021-07-11T04:02:12.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455e6a7700 1 Processor -- start 2021-07-11T04:02:12.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455e6a7700 1 -- start start 2021-07-11T04:02:12.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455e6a7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45580a76b0 0x7f45580aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455e6a7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f45580b0060 0x7f45580ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455e6a7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45580b1970 0x7f45580aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455e6a7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45580a91f0 con 0x7f45580b1970 2021-07-11T04:02:12.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455e6a7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45580add00 con 0x7f45580b0060 2021-07-11T04:02:12.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455e6a7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45580ad6d0 con 0x7f45580a76b0 2021-07-11T04:02:12.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455cc44700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45580b1970 0x7f45580aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455cc44700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45580b1970 0x7f45580aee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50882/0 (socket says 172.21.2.104:50882) 2021-07-11T04:02:12.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455cc44700 1 -- 172.21.2.104:0/4000368684 learned_addr learned my addr 172.21.2.104:0/4000368684 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:12.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455cc44700 1 -- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45580a76b0 msgr2=0x7f45580aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455cc44700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45580a76b0 0x7f45580aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455cc44700 1 -- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f45580b0060 msgr2=0x7f45580ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455cc44700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f45580b0060 0x7f45580ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455cc44700 1 -- 172.21.2.104:0/4000368684 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454800b1e0 con 0x7f45580b1970 2021-07-11T04:02:12.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455cc44700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45580b1970 0x7f45580aee10 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f45500032f0 tx=0x7f4550009250).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:12.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f4555ffb700 1 -- 172.21.2.104:0/4000368684 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f455000f7e0 con 0x7f45580b1970 2021-07-11T04:02:12.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f4555ffb700 1 -- 172.21.2.104:0/4000368684 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455000f940 con 0x7f45580b1970 2021-07-11T04:02:12.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455e6a7700 1 -- 172.21.2.104:0/4000368684 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45580a9420 con 0x7f45580b1970 2021-07-11T04:02:12.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f4555ffb700 1 -- 172.21.2.104:0/4000368684 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f455000fca0 con 0x7f45580b1970 2021-07-11T04:02:12.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455e6a7700 1 -- 172.21.2.104:0/4000368684 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45580ae160 con 0x7f45580b1970 2021-07-11T04:02:12.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f455e6a7700 1 -- 172.21.2.104:0/4000368684 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45580a95b0 con 0x7f45580b1970 2021-07-11T04:02:12.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f4555ffb700 1 -- 172.21.2.104:0/4000368684 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4550010480 con 0x7f45580b1970 2021-07-11T04:02:12.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f4555ffb700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f453c053300 0x7f453c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f4555ffb700 1 -- 172.21.2.104:0/4000368684 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f455006b9a0 con 0x7f45580b1970 2021-07-11T04:02:12.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.144+0000 7f4557fff700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f453c053300 0x7f453c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.148+0000 7f4557fff700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f453c053300 0x7f453c0557c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f455806a0e0 tx=0x7f4548015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:12.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.148+0000 7f4555ffb700 1 -- 172.21.2.104:0/4000368684 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4550042e50 con 0x7f45580b1970 2021-07-11T04:02:12.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 -- 172.21.2.104:0/4000368684 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f45580ade70 con 0x7f45580b1970 2021-07-11T04:02:12.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f4555ffb700 1 -- 172.21.2.104:0/4000368684 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+1559 (secure 0 0 0) 0x7f455000fe00 con 0x7f45580b1970 2021-07-11T04:02:12.255 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:12.256 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5335,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/1814518027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":1814518027},{"type":"v1","addr":"172.21.2.105:6835","nonce":1814518027}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/155502403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":155502403},{"type":"v1","addr":"172.21.2.104:6834","nonce":155502403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6835/1688901480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1688901480},{"type":"v1","addr":"172.21.2.104:6835","nonce":1688901480}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[]} 2021-07-11T04:02:12.256 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 -- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f453c053300 msgr2=0x7f453c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f453c053300 0x7f453c0557c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f455806a0e0 tx=0x7f4548015000).stop 2021-07-11T04:02:12.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 -- 172.21.2.104:0/4000368684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45580b1970 msgr2=0x7f45580aee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45580b1970 0x7f45580aee10 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7f45500032f0 tx=0x7f4550009250).stop 2021-07-11T04:02:12.257 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 -- 172.21.2.104:0/4000368684 shutdown_connections 2021-07-11T04:02:12.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f453c053300 0x7f453c0557c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f45580b1970 0x7f45580aee10 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f45580b0060 0x7f45580ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 --2- 172.21.2.104:0/4000368684 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f45580a76b0 0x7f45580aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.258 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 -- 172.21.2.104:0/4000368684 >> 172.21.2.104:0/4000368684 conn(0x7f45580b53e0 msgr2=0x7f455814a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:12.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 -- 172.21.2.104:0/4000368684 shutdown_connections 2021-07-11T04:02:12.259 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.256+0000 7f455e6a7700 1 -- 172.21.2.104:0/4000368684 wait complete. 2021-07-11T04:02:12.259 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 71 2021-07-11T04:02:12.263 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:12.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.312+0000 7fdde040f700 1 Processor -- start 2021-07-11T04:02:12.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.312+0000 7fdde040f700 1 -- start start 2021-07-11T04:02:12.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.312+0000 7fdde040f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fddd80a76b0 0x7fddd8066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.312+0000 7fdde040f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fddd80b1970 0x7fddd8066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.312+0000 7fdde040f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fddd80b0060 0x7fddd805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.312+0000 7fdde040f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd8067450 con 0x7fddd80b0060 2021-07-11T04:02:12.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.312+0000 7fdde040f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd80a7a90 con 0x7fddd80a76b0 2021-07-11T04:02:12.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.312+0000 7fdde040f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd8147620 con 0x7fddd80b1970 2021-07-11T04:02:12.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.312+0000 7fddde9ac700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fddd80b0060 0x7fddd805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.312+0000 7fddde9ac700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fddd80b0060 0x7fddd805f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50888/0 (socket says 172.21.2.104:50888) 2021-07-11T04:02:12.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.312+0000 7fddde9ac700 1 -- 172.21.2.104:0/3052641276 learned_addr learned my addr 172.21.2.104:0/3052641276 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:12.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fddde9ac700 1 -- 172.21.2.104:0/3052641276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fddd80b1970 msgr2=0x7fddd8066f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:12.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fddde9ac700 1 --2- 172.21.2.104:0/3052641276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fddd80b1970 0x7fddd8066f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fddde9ac700 1 -- 172.21.2.104:0/3052641276 >> v2:172.21.2.105:3300/0 conn(0x7fddd80a76b0 msgr2=0x7fddd8066a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:12.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fddde9ac700 1 --2- 172.21.2.104:0/3052641276 >> v2:172.21.2.105:3300/0 conn(0x7fddd80a76b0 0x7fddd8066a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fddde9ac700 1 -- 172.21.2.104:0/3052641276 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd81478c0 con 0x7fddd80b0060 2021-07-11T04:02:12.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fddde9ac700 1 --2- 172.21.2.104:0/3052641276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fddd80b0060 0x7fddd805f6b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7fddd00032f0 tx=0x7fddd0009290).ready entity=mon.0 client_cookie=737f2014702b8224 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:12.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd1a9700 1 -- 172.21.2.104:0/3052641276 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddd000b870 con 0x7fddd80b0060 2021-07-11T04:02:12.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd1a9700 1 -- 172.21.2.104:0/3052641276 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd000b9d0 con 0x7fddd80b0060 2021-07-11T04:02:12.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd1a9700 1 -- 172.21.2.104:0/3052641276 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddd000bb80 con 0x7fddd80b0060 2021-07-11T04:02:12.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 -- 172.21.2.104:0/3052641276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fddd80b0060 msgr2=0x7fddd805f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 --2- 172.21.2.104:0/3052641276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fddd80b0060 0x7fddd805f6b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 rx=0x7fddd00032f0 tx=0x7fddd0009290).stop 2021-07-11T04:02:12.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 -- 172.21.2.104:0/3052641276 shutdown_connections 2021-07-11T04:02:12.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 --2- 172.21.2.104:0/3052641276 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fddd80b0060 0x7fddd805f6b0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 --2- 172.21.2.104:0/3052641276 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fddd80b1970 0x7fddd8066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 --2- 172.21.2.104:0/3052641276 >> v2:172.21.2.105:3300/0 conn(0x7fddd80a76b0 0x7fddd8066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 -- 172.21.2.104:0/3052641276 >> 172.21.2.104:0/3052641276 conn(0x7fddd80b53e0 msgr2=0x7fddd80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:12.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 -- 172.21.2.104:0/3052641276 shutdown_connections 2021-07-11T04:02:12.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 -- 172.21.2.104:0/3052641276 wait complete. 2021-07-11T04:02:12.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 Processor -- start 2021-07-11T04:02:12.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 -- start start 2021-07-11T04:02:12.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fddd80a76b0 0x7fddd805f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fddd80b0060 0x7fddd8144670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fddd80b1970 0x7fddd8144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd80af600 con 0x7fddd80b0060 2021-07-11T04:02:12.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd80aec40 con 0x7fddd80b1970 2021-07-11T04:02:12.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd814e2b0 con 0x7fddd80a76b0 2021-07-11T04:02:12.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd9aa700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fddd80b0060 0x7fddd8144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd9aa700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fddd80b0060 0x7fddd8144670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50896/0 (socket says 172.21.2.104:50896) 2021-07-11T04:02:12.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd9aa700 1 -- 172.21.2.104:0/1879941690 learned_addr learned my addr 172.21.2.104:0/1879941690 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:12.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd9aa700 1 -- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fddd80a76b0 msgr2=0x7fddd805f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd9aa700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fddd80a76b0 0x7fddd805f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd9aa700 1 -- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fddd80b1970 msgr2=0x7fddd8144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd9aa700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fddd80b1970 0x7fddd8144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd9aa700 1 -- 172.21.2.104:0/1879941690 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd0009530 con 0x7fddd80b0060 2021-07-11T04:02:12.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdddd9aa700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fddd80b0060 0x7fddd8144670 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7fddd80af740 tx=0x7fddcc006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:12.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fddcb7fe700 1 -- 172.21.2.104:0/1879941690 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc012410 con 0x7fddd80b0060 2021-07-11T04:02:12.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fddcb7fe700 1 -- 172.21.2.104:0/1879941690 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc012570 con 0x7fddd80b0060 2021-07-11T04:02:12.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 -- 172.21.2.104:0/1879941690 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd80aeec0 con 0x7fddd80b0060 2021-07-11T04:02:12.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fddcb7fe700 1 -- 172.21.2.104:0/1879941690 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fddcc0128d0 con 0x7fddd80b0060 2021-07-11T04:02:12.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.316+0000 7fdde040f700 1 -- 172.21.2.104:0/1879941690 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd80ab4b0 con 0x7fddd80b0060 2021-07-11T04:02:12.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.320+0000 7fdde040f700 1 -- 172.21.2.104:0/1879941690 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd8057350 con 0x7fddd80b0060 2021-07-11T04:02:12.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.320+0000 7fddcb7fe700 1 -- 172.21.2.104:0/1879941690 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fddcc00f030 con 0x7fddd80b0060 2021-07-11T04:02:12.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.320+0000 7fddcb7fe700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fddb8053160 0x7fddb8055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.320+0000 7fddcb7fe700 1 -- 172.21.2.104:0/1879941690 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fddcc06aee0 con 0x7fddd80b0060 2021-07-11T04:02:12.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.320+0000 7fddde1ab700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fddb8053160 0x7fddb8055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.320+0000 7fddde1ab700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fddb8053160 0x7fddb8055620 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fddd4002fb0 tx=0x7fddd40069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:12.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.320+0000 7fddcb7fe700 1 -- 172.21.2.104:0/1879941690 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fddcc0423f0 con 0x7fddd80b0060 2021-07-11T04:02:12.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.428+0000 7fdde040f700 1 -- 172.21.2.104:0/1879941690 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddd80af360 con 0x7fddd80b0060 2021-07-11T04:02:12.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.428+0000 7fddcb7fe700 1 -- 172.21.2.104:0/1879941690 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+1559 (secure 0 0 0) 0x7fddcc045a10 con 0x7fddd80b0060 2021-07-11T04:02:12.429 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:12.430 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5335,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.105:6835/1814518027","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":1814518027},{"type":"v1","addr":"172.21.2.105:6835","nonce":1814518027}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8019,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6834/155502403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":155502403},{"type":"v1","addr":"172.21.2.104:6834","nonce":155502403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":65},{"gid":8319,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6835/1688901480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6833","nonce":1688901480},{"type":"v1","addr":"172.21.2.104:6835","nonce":1688901480}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":71}],"filesystems":[]} 2021-07-11T04:02:12.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.428+0000 7fdde040f700 1 -- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fddb8053160 msgr2=0x7fddb8055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.428+0000 7fdde040f700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fddb8053160 0x7fddb8055620 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7fddd4002fb0 tx=0x7fddd40069b0).stop 2021-07-11T04:02:12.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.428+0000 7fdde040f700 1 -- 172.21.2.104:0/1879941690 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fddd80b0060 msgr2=0x7fddd8144670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.428+0000 7fdde040f700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fddd80b0060 0x7fddd8144670 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7fddd80af740 tx=0x7fddcc006d70).stop 2021-07-11T04:02:12.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.432+0000 7fdde040f700 1 -- 172.21.2.104:0/1879941690 shutdown_connections 2021-07-11T04:02:12.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.432+0000 7fdde040f700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fddd80b1970 0x7fddd8144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.432+0000 7fdde040f700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fddb8053160 0x7fddb8055620 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.432+0000 7fdde040f700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fddd80b0060 0x7fddd8144670 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.432+0000 7fdde040f700 1 --2- 172.21.2.104:0/1879941690 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fddd80a76b0 0x7fddd805f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.432+0000 7fdde040f700 1 -- 172.21.2.104:0/1879941690 >> 172.21.2.104:0/1879941690 conn(0x7fddd80b53e0 msgr2=0x7fddd8145f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:12.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.432+0000 7fdde040f700 1 -- 172.21.2.104:0/1879941690 shutdown_connections 2021-07-11T04:02:12.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.432+0000 7fdde040f700 1 -- 172.21.2.104:0/1879941690 wait complete. 2021-07-11T04:02:12.433 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 71 2021-07-11T04:02:12.437 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T04:02:12.437 INFO:tasks.ceph.mds.a:Stopping old one... 2021-07-11T04:02:12.437 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T04:02:12.438 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:02:12.438 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T04:02:12.438 INFO:tasks.ceph.mds.c:Stopping old one... 2021-07-11T04:02:12.439 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T04:02:12.439 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:02:12.439 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T04:02:12.439 INFO:tasks.ceph.mds.b:Stopping old one... 2021-07-11T04:02:12.440 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T04:02:12.440 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:02:12.443 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T04:02:12.443 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T04:02:12.445 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T04:02:12.445 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T04:02:12.446 INFO:tasks.ceph.mds.a:Started 2021-07-11T04:02:12.447 DEBUG:teuthology.parallel:result is None 2021-07-11T04:02:12.447 INFO:tasks.ceph.mds.c:Started 2021-07-11T04:02:12.448 DEBUG:teuthology.parallel:result is None 2021-07-11T04:02:12.452 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T04:02:12.452 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T04:02:12.454 INFO:tasks.ceph.mds.b:Started 2021-07-11T04:02:12.454 DEBUG:teuthology.parallel:result is None 2021-07-11T04:02:12.455 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T04:02:12.478 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T04:02:12.479 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T04:02:12.489 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T04:02:12.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f8134700 1 Processor -- start 2021-07-11T04:02:12.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f8134700 1 -- start start 2021-07-11T04:02:12.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f8134700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0141b40 0x7fb1f014cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f8134700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f00b31d0 con 0x7fb1f00b7810 2021-07-11T04:02:12.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f8134700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1f00b7bf0 con 0x7fb1f00b1170 2021-07-11T04:02:12.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f8134700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00675c0 con 0x7fb1f0141b40 2021-07-11T04:02:12.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f5ed0700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb1f00b7810 0x7fb1f01445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47694/0 (socket says 172.21.2.104:47694) 2021-07-11T04:02:12.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f5ed0700 1 -- 172.21.2.104:0/3802909573 learned_addr learned my addr 172.21.2.104:0/3802909573 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:12.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276207863 0 0) 0x7fb1f00b31d0 con 0x7fb1f00b7810 2021-07-11T04:02:12.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d4003260 con 0x7fb1f00b7810 2021-07-11T04:02:12.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2398690875 0 0) 0x7fb1d4003260 con 0x7fb1f00b7810 2021-07-11T04:02:12.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0141b40 msgr2=0x7fb1f014cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:12.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0141b40 0x7fb1f014cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:12.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 >> v1:172.21.2.105:6789/0 conn(0x7fb1f00b1170 legacy=0x7fb1f0141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:12.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f00aae70 con 0x7fb1f00b7810 2021-07-11T04:02:12.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fb1e0001660 con 0x7fb1f00b7810 2021-07-11T04:02:12.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 >> v1:172.21.2.104:6789/0 conn(0x7fb1f00b7810 legacy=0x7fb1f01445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1d4003780 0x7fb1d4003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0141b40 0x7fb1d4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1d4004b10 0x7fb1d4004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00675c0 con 0x7fb1d4003780 2021-07-11T04:02:12.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d400b910 con 0x7fb1d4004b10 2021-07-11T04:02:12.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.508+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d400bad0 con 0x7fb1f0141b40 2021-07-11T04:02:12.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f5ed0700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1d4003780 0x7fb1d4003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1d4004b10 0x7fb1d4004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f56cf700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0141b40 0x7fb1d4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f5ed0700 1 -- 172.21.2.104:0/3802909573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0141b40 msgr2=0x7fb1d4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:12.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f5ed0700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0141b40 0x7fb1d4004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f5ed0700 1 -- 172.21.2.104:0/3802909573 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1d4004b10 msgr2=0x7fb1d4004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:12.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f5ed0700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1d4004b10 0x7fb1d4004f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f5ed0700 1 -- 172.21.2.104:0/3802909573 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d4003260 con 0x7fb1d4003780 2021-07-11T04:02:12.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1d4004b10 0x7fb1d4004f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:02:12.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f5ed0700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1d4003780 0x7fb1d4003b80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7fb1f0060740 tx=0x7fb1f0066a60).ready entity=mon.0 client_cookie=d69a00bc56e8189b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:12.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e0007270 con 0x7fb1d4003780 2021-07-11T04:02:12.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e00073d0 con 0x7fb1d4003780 2021-07-11T04:02:12.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f4ece700 1 -- 172.21.2.104:0/3802909573 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e000cae0 con 0x7fb1d4003780 2021-07-11T04:02:12.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 -- 172.21.2.104:0/3802909573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1d4003780 msgr2=0x7fb1d4003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 --2- 172.21.2.104:0/3802909573 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1d4003780 0x7fb1d4003b80 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7fb1f0060740 tx=0x7fb1f0066a60).stop 2021-07-11T04:02:12.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 -- 172.21.2.104:0/3802909573 reap_dead start 2021-07-11T04:02:12.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 -- 172.21.2.104:0/3802909573 shutdown_connections 2021-07-11T04:02:12.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 -- 172.21.2.104:0/3802909573 >> 172.21.2.104:0/3802909573 conn(0x7fb1f005f6b0 msgr2=0x7fb1f005fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:12.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 -- 172.21.2.104:0/3802909573 shutdown_connections 2021-07-11T04:02:12.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 -- 172.21.2.104:0/3802909573 wait complete. 2021-07-11T04:02:12.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 Processor -- start 2021-07-11T04:02:12.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 -- start start 2021-07-11T04:02:12.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1f00b4260 0x7fb1f00a8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0159f70 0x7fb1f015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f0144510 0x7fb1f0144990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f015a8c0 con 0x7fb1f0144510 2021-07-11T04:02:12.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f00b4640 con 0x7fb1f00b4260 2021-07-11T04:02:12.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f0159b10 con 0x7fb1f0159f70 2021-07-11T04:02:12.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f0144510 0x7fb1f0144990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f0144510 0x7fb1f0144990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50940/0 (socket says 172.21.2.104:50940) 2021-07-11T04:02:12.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 -- 172.21.2.104:0/1972303250 learned_addr learned my addr 172.21.2.104:0/1972303250 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:12.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f5ed0700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1f00b4260 0x7fb1f00a8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f56cf700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0159f70 0x7fb1f015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 -- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0159f70 msgr2=0x7fb1f015a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0159f70 0x7fb1f015a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 -- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1f00b4260 msgr2=0x7fb1f00a8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:12.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1f00b4260 0x7fb1f00a8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:12.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 -- 172.21.2.104:0/1972303250 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f00a8fa0 con 0x7fb1f0144510 2021-07-11T04:02:12.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f66d1700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f0144510 0x7fb1f0144990 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7fb1ec000b60 tx=0x7fb1ec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:12.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1e6ffd700 1 -- 172.21.2.104:0/1972303250 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1ec00bda0 con 0x7fb1f0144510 2021-07-11T04:02:12.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1e6ffd700 1 -- 172.21.2.104:0/1972303250 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ec013430 con 0x7fb1f0144510 2021-07-11T04:02:12.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1e6ffd700 1 -- 172.21.2.104:0/1972303250 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1ec0137b0 con 0x7fb1f0144510 2021-07-11T04:02:12.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 -- 172.21.2.104:0/1972303250 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f00a8a60 con 0x7fb1f0144510 2021-07-11T04:02:12.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.512+0000 7fb1f8134700 1 -- 172.21.2.104:0/1972303250 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f0147260 con 0x7fb1f0144510 2021-07-11T04:02:12.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.516+0000 7fb1f8134700 1 -- 172.21.2.104:0/1972303250 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f0057350 con 0x7fb1f0144510 2021-07-11T04:02:12.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.516+0000 7fb1e6ffd700 1 -- 172.21.2.104:0/1972303250 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb1ec013910 con 0x7fb1f0144510 2021-07-11T04:02:12.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.516+0000 7fb1e6ffd700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d8053160 0x7fb1d8055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:12.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.516+0000 7fb1e6ffd700 1 -- 172.21.2.104:0/1972303250 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb1ec06acd0 con 0x7fb1f0144510 2021-07-11T04:02:12.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.516+0000 7fb1f5ed0700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d8053160 0x7fb1d8055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:12.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.516+0000 7fb1f5ed0700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d8053160 0x7fb1d8055620 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fb1e001b000 tx=0x7fb1e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:12.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.516+0000 7fb1e6ffd700 1 -- 172.21.2.104:0/1972303250 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb1ec0421e0 con 0x7fb1f0144510 2021-07-11T04:02:12.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:12.608+0000 7fb1f8134700 1 -- 172.21.2.104:0/1972303250 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fb1f00698e0 con 0x7fb1f0144510 2021-07-11T04:02:13.082 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:02:13.087 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:02:13.091 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:02:13.094 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:02:13.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1e6ffd700 1 -- 172.21.2.104:0/1972303250 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v144) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb1ec045800 con 0x7fb1f0144510 2021-07-11T04:02:13.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 -- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d8053160 msgr2=0x7fb1d8055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d8053160 0x7fb1d8055620 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7fb1e001b000 tx=0x7fb1e0015000).stop 2021-07-11T04:02:13.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 -- 172.21.2.104:0/1972303250 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f0144510 msgr2=0x7fb1f0144990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f0144510 0x7fb1f0144990 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 rx=0x7fb1ec000b60 tx=0x7fb1ec008040).stop 2021-07-11T04:02:13.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 -- 172.21.2.104:0/1972303250 shutdown_connections 2021-07-11T04:02:13.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d8053160 0x7fb1d8055620 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f0144510 0x7fb1f0144990 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f0159f70 0x7fb1f015a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 --2- 172.21.2.104:0/1972303250 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1f00b4260 0x7fb1f00a8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 -- 172.21.2.104:0/1972303250 >> 172.21.2.104:0/1972303250 conn(0x7fb1f005f6b0 msgr2=0x7fb1f005fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:13.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 -- 172.21.2.104:0/1972303250 shutdown_connections 2021-07-11T04:02:13.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.500+0000 7fb1f8134700 1 -- 172.21.2.104:0/1972303250 wait complete. 2021-07-11T04:02:13.505 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T04:02:13.511 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T04:02:13.511 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T04:02:13.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9beee700 1 Processor -- start 2021-07-11T04:02:13.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9beee700 1 -- start start 2021-07-11T04:02:13.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9beee700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae941483a0 0x7fae940b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9beee700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fae940a70c0 0x7fae94144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9beee700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae940b0350 con 0x7fae940b1db0 2021-07-11T04:02:13.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9beee700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae940a90c0 con 0x7fae940a70c0 2021-07-11T04:02:13.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9beee700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae940b05f0 con 0x7fae941483a0 2021-07-11T04:02:13.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae99c8a700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fae940b1db0 0x7fae940b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47714/0 (socket says 172.21.2.104:47714) 2021-07-11T04:02:13.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae99c8a700 1 -- 172.21.2.104:0/3236569049 learned_addr learned my addr 172.21.2.104:0/3236569049 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:13.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391992090 0 0) 0x7fae940b0350 con 0x7fae940b1db0 2021-07-11T04:02:13.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae7c003260 con 0x7fae940b1db0 2021-07-11T04:02:13.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9a48b700 1 --2- 172.21.2.104:0/3236569049 >> v2:172.21.2.105:3300/0 conn(0x7fae940a70c0 0x7fae94144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae99489700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae941483a0 0x7fae940b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 573536280 0 0) 0x7fae7c003260 con 0x7fae940b1db0 2021-07-11T04:02:13.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae941483a0 msgr2=0x7fae940b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:13.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae941483a0 0x7fae940b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 >> v2:172.21.2.105:3300/0 conn(0x7fae940a70c0 msgr2=0x7fae94144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:13.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 --2- 172.21.2.104:0/3236569049 >> v2:172.21.2.105:3300/0 conn(0x7fae940a70c0 0x7fae94144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940b0890 con 0x7fae940b1db0 2021-07-11T04:02:13.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fae90001660 con 0x7fae940b1db0 2021-07-11T04:02:13.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 >> v1:172.21.2.104:6789/0 conn(0x7fae940b1db0 legacy=0x7fae940b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae941483a0 0x7fae7c0037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fae7c003d50 0x7fae7c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae7c0049f0 0x7fae7c004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae940b05f0 con 0x7fae7c0049f0 2021-07-11T04:02:13.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c00b7b0 con 0x7fae7c003d50 2021-07-11T04:02:13.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c00b970 con 0x7fae941483a0 2021-07-11T04:02:13.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9a48b700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae7c0049f0 0x7fae7c004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae99c8a700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae941483a0 0x7fae7c0037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae99489700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fae7c003d50 0x7fae7c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9a48b700 1 -- 172.21.2.104:0/3236569049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae941483a0 msgr2=0x7fae7c0037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:13.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9a48b700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae941483a0 0x7fae7c0037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9a48b700 1 -- 172.21.2.104:0/3236569049 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fae7c003d50 msgr2=0x7fae7c004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:13.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9a48b700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fae7c003d50 0x7fae7c004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9a48b700 1 -- 172.21.2.104:0/3236569049 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c003260 con 0x7fae7c0049f0 2021-07-11T04:02:13.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9a48b700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae7c0049f0 0x7fae7c004e70 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7fae8c00c000 tx=0x7fae8c0091d0).ready entity=mon.0 client_cookie=b55975c13320d5a5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:13.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae8c00e7e0 con 0x7fae7c0049f0 2021-07-11T04:02:13.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c00e940 con 0x7fae7c0049f0 2021-07-11T04:02:13.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae98c88700 1 -- 172.21.2.104:0/3236569049 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae8c00eca0 con 0x7fae7c0049f0 2021-07-11T04:02:13.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9beee700 1 -- 172.21.2.104:0/3236569049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae7c0049f0 msgr2=0x7fae7c004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9beee700 1 --2- 172.21.2.104:0/3236569049 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae7c0049f0 0x7fae7c004e70 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7fae8c00c000 tx=0x7fae8c0091d0).stop 2021-07-11T04:02:13.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.564+0000 7fae9a48b700 1 -- 172.21.2.104:0/3236569049 reap_dead start 2021-07-11T04:02:13.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 -- 172.21.2.104:0/3236569049 shutdown_connections 2021-07-11T04:02:13.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 -- 172.21.2.104:0/3236569049 >> 172.21.2.104:0/3236569049 conn(0x7fae9405f6b0 msgr2=0x7fae9405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:13.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 -- 172.21.2.104:0/3236569049 shutdown_connections 2021-07-11T04:02:13.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 -- 172.21.2.104:0/3236569049 wait complete. 2021-07-11T04:02:13.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 Processor -- start 2021-07-11T04:02:13.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 -- start start 2021-07-11T04:02:13.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae940ad1e0 0x7fae94156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fae94146a90 0x7fae9413f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae9413fba0 0x7fae94159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae94147580 con 0x7fae9413fba0 2021-07-11T04:02:13.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae940afa70 con 0x7fae94146a90 2021-07-11T04:02:13.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae940ad5c0 con 0x7fae940ad1e0 2021-07-11T04:02:13.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9a48b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae9413fba0 0x7fae94159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9a48b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae9413fba0 0x7fae94159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50962/0 (socket says 172.21.2.104:50962) 2021-07-11T04:02:13.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9a48b700 1 -- 172.21.2.104:0/3740996952 learned_addr learned my addr 172.21.2.104:0/3740996952 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:13.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae99489700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fae94146a90 0x7fae9413f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae99c8a700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae940ad1e0 0x7fae94156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9a48b700 1 -- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae940ad1e0 msgr2=0x7fae94156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9a48b700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae940ad1e0 0x7fae94156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9a48b700 1 -- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fae94146a90 msgr2=0x7fae9413f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9a48b700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fae94146a90 0x7fae9413f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9a48b700 1 -- 172.21.2.104:0/3740996952 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae94140430 con 0x7fae9413fba0 2021-07-11T04:02:13.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae99489700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fae94146a90 0x7fae9413f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:02:13.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9a48b700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae9413fba0 0x7fae94159e60 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7fae94145de0 tx=0x7fae8c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:13.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae86ffd700 1 -- 172.21.2.104:0/3740996952 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae8c01d990 con 0x7fae9413fba0 2021-07-11T04:02:13.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae86ffd700 1 -- 172.21.2.104:0/3740996952 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c01daf0 con 0x7fae9413fba0 2021-07-11T04:02:13.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 -- 172.21.2.104:0/3740996952 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae940ad8b0 con 0x7fae9413fba0 2021-07-11T04:02:13.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 -- 172.21.2.104:0/3740996952 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae940afca0 con 0x7fae9413fba0 2021-07-11T04:02:13.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae86ffd700 1 -- 172.21.2.104:0/3740996952 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fae8c01de50 con 0x7fae9413fba0 2021-07-11T04:02:13.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae9beee700 1 -- 172.21.2.104:0/3740996952 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae94057350 con 0x7fae9413fba0 2021-07-11T04:02:13.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae86ffd700 1 -- 172.21.2.104:0/3740996952 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fae8c01b450 con 0x7fae9413fba0 2021-07-11T04:02:13.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae86ffd700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fae74053380 0x7fae74055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.568+0000 7fae86ffd700 1 -- 172.21.2.104:0/3740996952 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fae8c06c4b0 con 0x7fae9413fba0 2021-07-11T04:02:13.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.572+0000 7fae99c8a700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fae74053380 0x7fae74055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.572+0000 7fae99c8a700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fae74053380 0x7fae74055840 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fae9000d000 tx=0x7fae90009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:13.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.572+0000 7fae86ffd700 1 -- 172.21.2.104:0/3740996952 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fae8c0439f0 con 0x7fae9413fba0 2021-07-11T04:02:13.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 -- 172.21.2.104:0/3740996952 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fae94067810 con 0x7fae9413fba0 2021-07-11T04:02:13.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae86ffd700 1 -- 172.21.2.104:0/3740996952 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v33) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fae8c016090 con 0x7fae9413fba0 2021-07-11T04:02:13.685 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:13.685 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:02:13.685 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T04:02:13.685 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.686 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T04:02:13.686 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T04:02:13.686 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.686 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:02:13.687 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.687 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.687 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.687 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.688 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.688 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.688 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T04:02:13.688 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T04:02:13.688 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.689 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:02:13.689 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.689 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.689 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.689 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.690 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.690 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.690 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T04:02:13.690 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T04:02:13.691 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.691 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:02:13.691 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.691 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.692 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.692 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.692 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.692 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.692 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T04:02:13.693 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T04:02:13.693 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.693 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.694 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.695 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.695 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.695 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.695 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.695 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T04:02:13.696 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T04:02:13.696 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.696 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.696 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.697 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.697 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.697 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.697 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.698 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T04:02:13.698 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T04:02:13.698 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.698 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.698 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.699 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.699 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.699 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.699 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.700 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T04:02:13.700 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T04:02:13.700 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.700 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.700 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.701 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.701 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.701 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.701 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.702 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T04:02:13.702 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T04:02:13.702 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.702 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.702 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.703 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.703 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.703 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.703 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.704 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T04:02:13.704 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T04:02:13.704 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.706 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.706 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.706 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.706 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.707 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.707 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.707 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T04:02:13.707 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T04:02:13.708 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.708 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.708 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.708 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.708 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.709 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.709 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.709 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T04:02:13.709 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T04:02:13.709 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.710 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.710 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.710 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.710 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.711 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.711 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.711 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T04:02:13.711 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T04:02:13.711 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.712 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:02:13.712 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:02:13.712 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:02:13.712 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.713 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.713 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.713 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T04:02:13.713 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T04:02:13.713 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.714 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:02:13.714 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:02:13.714 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:02:13.714 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.715 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.715 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.715 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T04:02:13.715 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T04:02:13.715 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.716 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:02:13.716 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:13.716 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:13.716 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.717 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.717 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.717 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.717 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T04:02:13.717 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T04:02:13.718 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.718 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:02:13.718 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:02:13.718 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.719 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.719 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:13.719 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:13.719 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T04:02:13.719 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T04:02:13.720 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:13.720 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:02:13.720 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:02:13.720 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:13.720 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.721 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:13.721 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:13.721 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:02:13.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 -- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fae74053380 msgr2=0x7fae74055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fae74053380 0x7fae74055840 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fae9000d000 tx=0x7fae90009040).stop 2021-07-11T04:02:13.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 -- 172.21.2.104:0/3740996952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae9413fba0 msgr2=0x7fae94159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae9413fba0 0x7fae94159e60 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7fae94145de0 tx=0x7fae8c01a000).stop 2021-07-11T04:02:13.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 -- 172.21.2.104:0/3740996952 shutdown_connections 2021-07-11T04:02:13.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fae74053380 0x7fae74055840 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fae9413fba0 0x7fae94159e60 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fae94146a90 0x7fae9413f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 --2- 172.21.2.104:0/3740996952 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fae940ad1e0 0x7fae94156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 -- 172.21.2.104:0/3740996952 >> 172.21.2.104:0/3740996952 conn(0x7fae9405f6b0 msgr2=0x7fae940b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:13.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 -- 172.21.2.104:0/3740996952 shutdown_connections 2021-07-11T04:02:13.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.684+0000 7fae9beee700 1 -- 172.21.2.104:0/3740996952 wait complete. 2021-07-11T04:02:13.727 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:13.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 Processor -- start 2021-07-11T04:02:13.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 -- start start 2021-07-11T04:02:13.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9400a76b0 0x7fe940066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe9400b1970 0x7fe940066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe940067450 con 0x7fe9400b0060 2021-07-11T04:02:13.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9400a7a90 con 0x7fe9400b1970 2021-07-11T04:02:13.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe940147620 con 0x7fe9400a76b0 2021-07-11T04:02:13.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94562b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe9400b0060 0x7fe94005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47738/0 (socket says 172.21.2.104:47738) 2021-07-11T04:02:13.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94562b700 1 -- 172.21.2.104:0/3876329387 learned_addr learned my addr 172.21.2.104:0/3876329387 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:13.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054811520 0 0) 0x7fe940067450 con 0x7fe9400b0060 2021-07-11T04:02:13.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe92c003260 con 0x7fe9400b0060 2021-07-11T04:02:13.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3409539612 0 0) 0x7fe92c003260 con 0x7fe9400b0060 2021-07-11T04:02:13.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9400a76b0 msgr2=0x7fe940066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:13.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 --2- 172.21.2.104:0/3876329387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9400a76b0 0x7fe940066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 >> v2:172.21.2.105:3300/0 conn(0x7fe9400b1970 msgr2=0x7fe940066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:13.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 --2- 172.21.2.104:0/3876329387 >> v2:172.21.2.105:3300/0 conn(0x7fe9400b1970 0x7fe940066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9401478c0 con 0x7fe9400b0060 2021-07-11T04:02:13.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fe9380019e0 con 0x7fe9400b0060 2021-07-11T04:02:13.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 >> v1:172.21.2.104:6789/0 conn(0x7fe9400b0060 legacy=0x7fe94005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 --2- 172.21.2.104:0/3876329387 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe92c0037d0 0x7fe92c003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 --2- 172.21.2.104:0/3876329387 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe92c004500 0x7fe92c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 --2- 172.21.2.104:0/3876329387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9400a76b0 0x7fe92c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400a7a90 con 0x7fe92c0037d0 2021-07-11T04:02:13.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe940147620 con 0x7fe92c004500 2021-07-11T04:02:13.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92c005300 con 0x7fe9400a76b0 2021-07-11T04:02:13.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe944e2a700 1 --2- 172.21.2.104:0/3876329387 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe92c0037d0 0x7fe92c003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe944e2a700 1 -- 172.21.2.104:0/3876329387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9400a76b0 msgr2=0x7fe92c00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:13.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe944e2a700 1 --2- 172.21.2.104:0/3876329387 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9400a76b0 0x7fe92c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe944e2a700 1 -- 172.21.2.104:0/3876329387 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe92c004500 msgr2=0x7fe92c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:13.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe944e2a700 1 --2- 172.21.2.104:0/3876329387 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe92c004500 0x7fe92c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe944e2a700 1 -- 172.21.2.104:0/3876329387 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c003260 con 0x7fe92c0037d0 2021-07-11T04:02:13.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe944e2a700 1 --2- 172.21.2.104:0/3876329387 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe92c0037d0 0x7fe92c003bf0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7fe930002f70 tx=0x7fe930006ca0).ready entity=mon.0 client_cookie=e87f817e7422eab0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:13.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe930012410 con 0x7fe92c0037d0 2021-07-11T04:02:13.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe930012570 con 0x7fe92c0037d0 2021-07-11T04:02:13.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe93f7fe700 1 -- 172.21.2.104:0/3876329387 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9300128d0 con 0x7fe92c0037d0 2021-07-11T04:02:13.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 -- 172.21.2.104:0/3876329387 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe92c0037d0 msgr2=0x7fe92c003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 --2- 172.21.2.104:0/3876329387 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe92c0037d0 0x7fe92c003bf0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7fe930002f70 tx=0x7fe930006ca0).stop 2021-07-11T04:02:13.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94562b700 1 -- 172.21.2.104:0/3876329387 reap_dead start 2021-07-11T04:02:13.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 -- 172.21.2.104:0/3876329387 shutdown_connections 2021-07-11T04:02:13.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 -- 172.21.2.104:0/3876329387 >> 172.21.2.104:0/3876329387 conn(0x7fe9400b53e0 msgr2=0x7fe9400b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:13.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 -- 172.21.2.104:0/3876329387 shutdown_connections 2021-07-11T04:02:13.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 -- 172.21.2.104:0/3876329387 wait complete. 2021-07-11T04:02:13.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 Processor -- start 2021-07-11T04:02:13.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.780+0000 7fe94708e700 1 -- start start 2021-07-11T04:02:13.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe94708e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9400ad5c0 0x7fe9400ad9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe94708e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9400ade90 0x7fe9400aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe94708e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9400a9e80 0x7fe9400ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe94708e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe940157560 con 0x7fe9400ad5c0 2021-07-11T04:02:13.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe94708e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400a9740 con 0x7fe9400a9e80 2021-07-11T04:02:13.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe94708e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9400a99e0 con 0x7fe9400ade90 2021-07-11T04:02:13.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe944e2a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9400ad5c0 0x7fe9400ad9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe944e2a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9400ad5c0 0x7fe9400ad9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50978/0 (socket says 172.21.2.104:50978) 2021-07-11T04:02:13.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe944e2a700 1 -- 172.21.2.104:0/2285203518 learned_addr learned my addr 172.21.2.104:0/2285203518 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:13.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe944e2a700 1 -- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9400ade90 msgr2=0x7fe9400aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe944e2a700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9400ade90 0x7fe9400aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe944e2a700 1 -- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9400a9e80 msgr2=0x7fe9400ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe944e2a700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9400a9e80 0x7fe9400ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe944e2a700 1 -- 172.21.2.104:0/2285203518 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe940157290 con 0x7fe9400ad5c0 2021-07-11T04:02:13.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe944e2a700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9400ad5c0 0x7fe9400ad9a0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7fe930002f70 tx=0x7fe930015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:13.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe93dffb700 1 -- 172.21.2.104:0/2285203518 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9300117b0 con 0x7fe9400ad5c0 2021-07-11T04:02:13.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe93dffb700 1 -- 172.21.2.104:0/2285203518 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe930011910 con 0x7fe9400ad5c0 2021-07-11T04:02:13.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe94708e700 1 -- 172.21.2.104:0/2285203518 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9400ac1a0 con 0x7fe9400ad5c0 2021-07-11T04:02:13.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe93dffb700 1 -- 172.21.2.104:0/2285203518 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe930011c70 con 0x7fe9400ad5c0 2021-07-11T04:02:13.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe94708e700 1 -- 172.21.2.104:0/2285203518 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe940066610 con 0x7fe9400ad5c0 2021-07-11T04:02:13.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe94708e700 1 -- 172.21.2.104:0/2285203518 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9400aae60 con 0x7fe9400ad5c0 2021-07-11T04:02:13.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe93dffb700 1 -- 172.21.2.104:0/2285203518 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe93000f030 con 0x7fe9400ad5c0 2021-07-11T04:02:13.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe93dffb700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe920053260 0x7fe920055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe93dffb700 1 -- 172.21.2.104:0/2285203518 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe93006bcd0 con 0x7fe9400ad5c0 2021-07-11T04:02:13.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe93ffff700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe920053260 0x7fe920055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.784+0000 7fe93ffff700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe920053260 0x7fe920055720 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fe934002f70 tx=0x7fe934006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:13.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.788+0000 7fe93dffb700 1 -- 172.21.2.104:0/2285203518 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe930043340 con 0x7fe9400ad5c0 2021-07-11T04:02:13.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 -- 172.21.2.104:0/2285203518 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe940157760 con 0x7fe9400ad5c0 2021-07-11T04:02:13.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe93dffb700 1 -- 172.21.2.104:0/2285203518 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+13191 (secure 0 0 0) 0x7fe930022070 con 0x7fe9400ad5c0 2021-07-11T04:02:13.869 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:13.870 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":144,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:13.490546+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:13.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 -- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe920053260 msgr2=0x7fe920055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe920053260 0x7fe920055720 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fe934002f70 tx=0x7fe934006a80).stop 2021-07-11T04:02:13.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 -- 172.21.2.104:0/2285203518 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9400ad5c0 msgr2=0x7fe9400ad9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9400ad5c0 0x7fe9400ad9a0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 rx=0x7fe930002f70 tx=0x7fe930015000).stop 2021-07-11T04:02:13.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 -- 172.21.2.104:0/2285203518 shutdown_connections 2021-07-11T04:02:13.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9400a9e80 0x7fe9400ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9400ade90 0x7fe9400aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe920053260 0x7fe920055720 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 --2- 172.21.2.104:0/2285203518 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9400ad5c0 0x7fe9400ad9a0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 -- 172.21.2.104:0/2285203518 >> 172.21.2.104:0/2285203518 conn(0x7fe9400b53e0 msgr2=0x7fe940149c30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:13.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 -- 172.21.2.104:0/2285203518 shutdown_connections 2021-07-11T04:02:13.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.868+0000 7fe94708e700 1 -- 172.21.2.104:0/2285203518 wait complete. 2021-07-11T04:02:13.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T04:02:13.881 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:02:13.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.936+0000 7f40865dc700 1 Processor -- start 2021-07-11T04:02:13.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.936+0000 7f40865dc700 1 -- start start 2021-07-11T04:02:13.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f408005faa0 0x7f40801438d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40800ac500 0x7f4080143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408005fe80 con 0x7f40800607d0 2021-07-11T04:02:13.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4080143240 con 0x7f40800ac500 2021-07-11T04:02:13.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40800acc60 con 0x7f408005faa0 2021-07-11T04:02:13.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40800ac500 0x7f4080143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f40800ac500 0x7f4080143dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50986/0 (socket says 172.21.2.104:50986) 2021-07-11T04:02:13.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 -- 172.21.2.104:0/1311172687 learned_addr learned my addr 172.21.2.104:0/1311172687 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:13.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 -- 172.21.2.104:0/1311172687 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f408005faa0 msgr2=0x7f40801438d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:13.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 --2- 172.21.2.104:0/1311172687 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f408005faa0 0x7f40801438d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 -- 172.21.2.104:0/1311172687 >> v1:172.21.2.105:6789/0 conn(0x7f40800607d0 legacy=0x7f4080146810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:13.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 -- 172.21.2.104:0/1311172687 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40800acf00 con 0x7f40800ac500 2021-07-11T04:02:13.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 --2- 172.21.2.104:0/1311172687 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40800ac500 0x7f4080143dc0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f408005cb70 tx=0x7f4078009290).ready entity=mon.0 client_cookie=baa1b92eb4856137 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:13.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f407effd700 1 -- 172.21.2.104:0/1311172687 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407800b860 con 0x7f40800ac500 2021-07-11T04:02:13.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f407effd700 1 -- 172.21.2.104:0/1311172687 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407800b9c0 con 0x7f40800ac500 2021-07-11T04:02:13.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f407effd700 1 -- 172.21.2.104:0/1311172687 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f407800bb70 con 0x7f40800ac500 2021-07-11T04:02:13.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- 172.21.2.104:0/1311172687 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40800ac500 msgr2=0x7f4080143dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 --2- 172.21.2.104:0/1311172687 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40800ac500 0x7f4080143dc0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f408005cb70 tx=0x7f4078009290).stop 2021-07-11T04:02:13.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- 172.21.2.104:0/1311172687 shutdown_connections 2021-07-11T04:02:13.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 --2- 172.21.2.104:0/1311172687 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40800ac500 0x7f4080143dc0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:13.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 --2- 172.21.2.104:0/1311172687 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f408005faa0 0x7f40801438d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- 172.21.2.104:0/1311172687 >> 172.21.2.104:0/1311172687 conn(0x7f40800b2970 msgr2=0x7f40800b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:13.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- 172.21.2.104:0/1311172687 shutdown_connections 2021-07-11T04:02:13.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- 172.21.2.104:0/1311172687 wait complete. 2021-07-11T04:02:13.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 Processor -- start 2021-07-11T04:02:13.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- start start 2021-07-11T04:02:13.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f408005faa0 0x7f408013c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40800607d0 0x7f4080152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40800ac500 0x7f4080152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40800b3cc0 con 0x7f40800ac500 2021-07-11T04:02:13.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4080152430 con 0x7f40800607d0 2021-07-11T04:02:13.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40800b6160 con 0x7f408005faa0 2021-07-11T04:02:13.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40800ac500 0x7f4080152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40800ac500 0x7f4080152d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:50996/0 (socket says 172.21.2.104:50996) 2021-07-11T04:02:13.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 -- 172.21.2.104:0/2486241408 learned_addr learned my addr 172.21.2.104:0/2486241408 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:13.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 -- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f408005faa0 msgr2=0x7f408013c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f408005faa0 0x7f408013c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 -- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40800607d0 msgr2=0x7f4080152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:13.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40800607d0 0x7f4080152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:13.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 -- 172.21.2.104:0/2486241408 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4078009530 con 0x7f40800ac500 2021-07-11T04:02:13.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f4084b79700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40800ac500 0x7f4080152d80 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f408005cb70 tx=0x7f4078016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:13.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f407d7fa700 1 -- 172.21.2.104:0/2486241408 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4078011cc0 con 0x7f40800ac500 2021-07-11T04:02:13.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f407d7fa700 1 -- 172.21.2.104:0/2486241408 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4078011e20 con 0x7f40800ac500 2021-07-11T04:02:13.954 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f407d7fa700 1 -- 172.21.2.104:0/2486241408 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4078020620 con 0x7f40800ac500 2021-07-11T04:02:13.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- 172.21.2.104:0/2486241408 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40800b6400 con 0x7f40800ac500 2021-07-11T04:02:13.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.940+0000 7f40865dc700 1 -- 172.21.2.104:0/2486241408 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40800b58e0 con 0x7f40800ac500 2021-07-11T04:02:13.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.944+0000 7f40865dc700 1 -- 172.21.2.104:0/2486241408 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40801525f0 con 0x7f40800ac500 2021-07-11T04:02:13.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.944+0000 7f407d7fa700 1 -- 172.21.2.104:0/2486241408 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4078026020 con 0x7f40800ac500 2021-07-11T04:02:13.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.944+0000 7f407d7fa700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40600531f0 0x7f40600556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:13.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.944+0000 7f407d7fa700 1 -- 172.21.2.104:0/2486241408 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f407806bef0 con 0x7f40800ac500 2021-07-11T04:02:13.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.944+0000 7f407ffff700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40600531f0 0x7f40600556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:13.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.944+0000 7f407ffff700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40600531f0 0x7f40600556b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f4074002fb0 tx=0x7f40740069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:13.958 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:13.944+0000 7f407d7fa700 1 -- 172.21.2.104:0/2486241408 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f40801525f0 con 0x7f40800ac500 2021-07-11T04:02:14.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 -- 172.21.2.104:0/2486241408 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f40801525f0 con 0x7f40800ac500 2021-07-11T04:02:14.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f407d7fa700 1 -- 172.21.2.104:0/2486241408 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v144) v1 ==== 110+0+10 (secure 0 0 0) 0x7f40801525f0 con 0x7f40800ac500 2021-07-11T04:02:14.027 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:02:14.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 -- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40600531f0 msgr2=0x7f40600556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40600531f0 0x7f40600556b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f4074002fb0 tx=0x7f40740069b0).stop 2021-07-11T04:02:14.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 -- 172.21.2.104:0/2486241408 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40800ac500 msgr2=0x7f4080152d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40800ac500 0x7f4080152d80 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f408005cb70 tx=0x7f4078016000).stop 2021-07-11T04:02:14.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 -- 172.21.2.104:0/2486241408 shutdown_connections 2021-07-11T04:02:14.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f40600531f0 0x7f40600556b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:14.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f40800ac500 0x7f4080152d80 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:14.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f40800607d0 0x7f4080152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 --2- 172.21.2.104:0/2486241408 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f408005faa0 0x7f408013c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 -- 172.21.2.104:0/2486241408 >> 172.21.2.104:0/2486241408 conn(0x7f40800b2970 msgr2=0x7f4080141a40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:14.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 -- 172.21.2.104:0/2486241408 shutdown_connections 2021-07-11T04:02:14.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.028+0000 7f40865dc700 1 -- 172.21.2.104:0/2486241408 wait complete. 2021-07-11T04:02:14.034 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:02:14.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 Processor -- start 2021-07-11T04:02:14.087 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 -- start start 2021-07-11T04:02:14.088 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f475c0b1ec0 0x7f475c0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f475c0b22a0 0x7f475c141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f475c0b0420 con 0x7f475c0affb0 2021-07-11T04:02:14.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475c0a77d0 con 0x7f475c0b22a0 2021-07-11T04:02:14.089 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f475c0b0650 con 0x7f475c0b1ec0 2021-07-11T04:02:14.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f475bfff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f475c0b22a0 0x7f475c141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:14.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f475bfff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f475c0b22a0 0x7f475c141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51004/0 (socket says 172.21.2.104:51004) 2021-07-11T04:02:14.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f475bfff700 1 -- 172.21.2.104:0/3768482812 learned_addr learned my addr 172.21.2.104:0/3768482812 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:14.090 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f475bfff700 1 -- 172.21.2.104:0/3768482812 >> v2:172.21.2.105:3300/0 conn(0x7f475c0b1ec0 msgr2=0x7f475c0b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:14.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f475bfff700 1 --2- 172.21.2.104:0/3768482812 >> v2:172.21.2.105:3300/0 conn(0x7f475c0b1ec0 0x7f475c0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f475bfff700 1 -- 172.21.2.104:0/3768482812 >> v1:172.21.2.104:6789/0 conn(0x7f475c0affb0 legacy=0x7f475c0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f475bfff700 1 -- 172.21.2.104:0/3768482812 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c0b08f0 con 0x7f475c0b22a0 2021-07-11T04:02:14.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f475bfff700 1 --2- 172.21.2.104:0/3768482812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f475c0b22a0 0x7f475c141250 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f47540032f0 tx=0x7f4754009290).ready entity=mon.0 client_cookie=8e26146393732c9d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:14.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f475a7fc700 1 -- 172.21.2.104:0/3768482812 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f475400b870 con 0x7f475c0b22a0 2021-07-11T04:02:14.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f475a7fc700 1 -- 172.21.2.104:0/3768482812 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475400b9d0 con 0x7f475c0b22a0 2021-07-11T04:02:14.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f475a7fc700 1 -- 172.21.2.104:0/3768482812 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f475400bb40 con 0x7f475c0b22a0 2021-07-11T04:02:14.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 -- 172.21.2.104:0/3768482812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f475c0b22a0 msgr2=0x7f475c141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 --2- 172.21.2.104:0/3768482812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f475c0b22a0 0x7f475c141250 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f47540032f0 tx=0x7f4754009290).stop 2021-07-11T04:02:14.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 -- 172.21.2.104:0/3768482812 shutdown_connections 2021-07-11T04:02:14.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 --2- 172.21.2.104:0/3768482812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f475c0b22a0 0x7f475c141250 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:14.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 --2- 172.21.2.104:0/3768482812 >> v2:172.21.2.105:3300/0 conn(0x7f475c0b1ec0 0x7f475c0b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 -- 172.21.2.104:0/3768482812 >> 172.21.2.104:0/3768482812 conn(0x7f475c05f6b0 msgr2=0x7f475c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:14.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 -- 172.21.2.104:0/3768482812 shutdown_connections 2021-07-11T04:02:14.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.088+0000 7f4761ee8700 1 -- 172.21.2.104:0/3768482812 wait complete. 2021-07-11T04:02:14.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4761ee8700 1 Processor -- start 2021-07-11T04:02:14.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4761ee8700 1 -- start start 2021-07-11T04:02:14.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4761ee8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f475c0affb0 0x7f475c0b5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4761ee8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f475c0b1ec0 0x7f475c14a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4761ee8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f475c0b22a0 0x7f475c14b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4761ee8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c0b4f20 con 0x7f475c0b1ec0 2021-07-11T04:02:14.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4761ee8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f475c0aae50 con 0x7f475c0b22a0 2021-07-11T04:02:14.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4761ee8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475c0ab0f0 con 0x7f475c0affb0 2021-07-11T04:02:14.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f475affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f475c0b1ec0 0x7f475c14a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:14.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f475affd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f475c0b1ec0 0x7f475c14a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51010/0 (socket says 172.21.2.104:51010) 2021-07-11T04:02:14.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f475affd700 1 -- 172.21.2.104:0/988287597 learned_addr learned my addr 172.21.2.104:0/988287597 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:14.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f475affd700 1 -- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f475c0affb0 msgr2=0x7f475c0b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f475affd700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f475c0affb0 0x7f475c0b5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f475affd700 1 -- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f475c0b22a0 msgr2=0x7f475c14b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f475affd700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f475c0b22a0 0x7f475c14b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f475affd700 1 -- 172.21.2.104:0/988287597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4754009530 con 0x7f475c0b1ec0 2021-07-11T04:02:14.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f475affd700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f475c0b1ec0 0x7f475c14a040 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f475c0ad2d0 tx=0x7f474c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:14.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4758ff9700 1 -- 172.21.2.104:0/988287597 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f474c00ce30 con 0x7f475c0b1ec0 2021-07-11T04:02:14.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4758ff9700 1 -- 172.21.2.104:0/988287597 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474c00f470 con 0x7f475c0b1ec0 2021-07-11T04:02:14.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4761ee8700 1 -- 172.21.2.104:0/988287597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c0ab390 con 0x7f475c0b1ec0 2021-07-11T04:02:14.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4761ee8700 1 -- 172.21.2.104:0/988287597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475c14b900 con 0x7f475c0b1ec0 2021-07-11T04:02:14.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4758ff9700 1 -- 172.21.2.104:0/988287597 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f474c00f7d0 con 0x7f475c0b1ec0 2021-07-11T04:02:14.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4761ee8700 1 -- 172.21.2.104:0/988287597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475c057350 con 0x7f475c0b1ec0 2021-07-11T04:02:14.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4758ff9700 1 -- 172.21.2.104:0/988287597 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f474c019030 con 0x7f475c0b1ec0 2021-07-11T04:02:14.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4758ff9700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4740053240 0x7f4740055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f4758ff9700 1 -- 172.21.2.104:0/988287597 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f474c06a680 con 0x7f475c0b1ec0 2021-07-11T04:02:14.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.092+0000 7f475b7fe700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4740053240 0x7f4740055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:14.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.096+0000 7f475b7fe700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4740053240 0x7f4740055700 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f4744002f70 tx=0x7f4744006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:14.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.096+0000 7f4758ff9700 1 -- 172.21.2.104:0/988287597 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f474c041cf0 con 0x7f475c0b1ec0 2021-07-11T04:02:14.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.176+0000 7f4761ee8700 1 -- 172.21.2.104:0/988287597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f475c1405d0 con 0x7f475c0b1ec0 2021-07-11T04:02:14.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.176+0000 7f4758ff9700 1 -- 172.21.2.104:0/988287597 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v144) v1 ==== 92+0+10 (secure 0 0 0) 0x7f474c045310 con 0x7f475c0b1ec0 2021-07-11T04:02:14.178 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 -- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4740053240 msgr2=0x7f4740055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4740053240 0x7f4740055700 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f4744002f70 tx=0x7f4744006a80).stop 2021-07-11T04:02:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 -- 172.21.2.104:0/988287597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f475c0b1ec0 msgr2=0x7f475c14a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f475c0b1ec0 0x7f475c14a040 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f475c0ad2d0 tx=0x7f474c007c50).stop 2021-07-11T04:02:14.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 -- 172.21.2.104:0/988287597 shutdown_connections 2021-07-11T04:02:14.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f475c0b22a0 0x7f475c14b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4740053240 0x7f4740055700 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:14.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f475c0b1ec0 0x7f475c14a040 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:14.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 --2- 172.21.2.104:0/988287597 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f475c0affb0 0x7f475c0b5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 -- 172.21.2.104:0/988287597 >> 172.21.2.104:0/988287597 conn(0x7f475c05f6b0 msgr2=0x7f475c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:14.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 -- 172.21.2.104:0/988287597 shutdown_connections 2021-07-11T04:02:14.182 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.180+0000 7f4761ee8700 1 -- 172.21.2.104:0/988287597 wait complete. 2021-07-11T04:02:14.185 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T04:02:14.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c59dad700 1 Processor -- start 2021-07-11T04:02:14.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c59dad700 1 -- start start 2021-07-11T04:02:14.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c59dad700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4c54067300 0x7f4c541438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.237 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c59dad700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c540aabb0 con 0x7f4c54143de0 2021-07-11T04:02:14.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c59dad700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c540604e0 con 0x7f4c54060100 2021-07-11T04:02:14.238 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c59dad700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c54066ea0 con 0x7f4c54067300 2021-07-11T04:02:14.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c52ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4c54067300 0x7f4c541438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:14.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c53fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4c54143de0 0x7f4c5414b850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47784/0 (socket says 172.21.2.104:47784) 2021-07-11T04:02:14.242 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c53fff700 1 -- 172.21.2.104:0/3868636946 learned_addr learned my addr 172.21.2.104:0/3868636946 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:14.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c52ffd700 1 -- 172.21.2.104:0/3868636946 >> v1:172.21.2.105:6789/0 conn(0x7f4c54060100 legacy=0x7f4c540b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:14.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c52ffd700 1 -- 172.21.2.104:0/3868636946 >> v1:172.21.2.104:6789/0 conn(0x7f4c54143de0 legacy=0x7f4c5414b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c52ffd700 1 -- 172.21.2.104:0/3868636946 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c540b33b0 con 0x7f4c54067300 2021-07-11T04:02:14.243 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c52ffd700 1 --2- 172.21.2.104:0/3868636946 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c54067300 0x7f4c541438f0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f4c5405f220 tx=0x7f4c44006df0).ready entity=mon.0 client_cookie=bdff1d1af7a60ae2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:14.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c527fc700 1 -- 172.21.2.104:0/3868636946 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c4400d5c0 con 0x7f4c54067300 2021-07-11T04:02:14.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c527fc700 1 -- 172.21.2.104:0/3868636946 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c44000e90 con 0x7f4c54067300 2021-07-11T04:02:14.244 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c527fc700 1 -- 172.21.2.104:0/3868636946 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c4400d910 con 0x7f4c54067300 2021-07-11T04:02:14.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c59dad700 1 -- 172.21.2.104:0/3868636946 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c54067300 msgr2=0x7f4c541438f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.236+0000 7f4c59dad700 1 --2- 172.21.2.104:0/3868636946 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c54067300 0x7f4c541438f0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x7f4c5405f220 tx=0x7f4c44006df0).stop 2021-07-11T04:02:14.245 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 -- 172.21.2.104:0/3868636946 shutdown_connections 2021-07-11T04:02:14.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 --2- 172.21.2.104:0/3868636946 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c54067300 0x7f4c541438f0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:14.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 -- 172.21.2.104:0/3868636946 >> 172.21.2.104:0/3868636946 conn(0x7f4c5405cf30 msgr2=0x7f4c5405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:14.246 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 -- 172.21.2.104:0/3868636946 shutdown_connections 2021-07-11T04:02:14.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 -- 172.21.2.104:0/3868636946 wait complete. 2021-07-11T04:02:14.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 Processor -- start 2021-07-11T04:02:14.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 -- start start 2021-07-11T04:02:14.247 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c54060100 0x7f4c540b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c54067300 0x7f4c54142220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c54143de0 0x7f4c54142710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5409f100 con 0x7f4c54067300 2021-07-11T04:02:14.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540ae160 con 0x7f4c54143de0 2021-07-11T04:02:14.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c540ae400 con 0x7f4c54060100 2021-07-11T04:02:14.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c52ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c54067300 0x7f4c54142220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:14.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c52ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c54067300 0x7f4c54142220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51020/0 (socket says 172.21.2.104:51020) 2021-07-11T04:02:14.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c52ffd700 1 -- 172.21.2.104:0/1163293227 learned_addr learned my addr 172.21.2.104:0/1163293227 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:14.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c52ffd700 1 -- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c54060100 msgr2=0x7f4c540b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c52ffd700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c54060100 0x7f4c540b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c52ffd700 1 -- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c54143de0 msgr2=0x7f4c54142710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c52ffd700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c54143de0 0x7f4c54142710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c52ffd700 1 -- 172.21.2.104:0/1163293227 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4400b180 con 0x7f4c54067300 2021-07-11T04:02:14.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c52ffd700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c54067300 0x7f4c54142220 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f4c5405f220 tx=0x7f4c44012000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:14.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c50ff9700 1 -- 172.21.2.104:0/1163293227 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c44002f50 con 0x7f4c54067300 2021-07-11T04:02:14.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c50ff9700 1 -- 172.21.2.104:0/1163293227 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c44017030 con 0x7f4c54067300 2021-07-11T04:02:14.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 -- 172.21.2.104:0/1163293227 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c540ae6a0 con 0x7f4c54067300 2021-07-11T04:02:14.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 -- 172.21.2.104:0/1163293227 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c540a0580 con 0x7f4c54067300 2021-07-11T04:02:14.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c50ff9700 1 -- 172.21.2.104:0/1163293227 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4c4401d470 con 0x7f4c54067300 2021-07-11T04:02:14.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c59dad700 1 -- 172.21.2.104:0/1163293227 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c54142da0 con 0x7f4c54067300 2021-07-11T04:02:14.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c50ff9700 1 -- 172.21.2.104:0/1163293227 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4c44024030 con 0x7f4c54067300 2021-07-11T04:02:14.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c50ff9700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c380531d0 0x7f4c38055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c50ff9700 1 -- 172.21.2.104:0/1163293227 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4c4406c080 con 0x7f4c54067300 2021-07-11T04:02:14.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.240+0000 7f4c537fe700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c380531d0 0x7f4c38055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:14.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.244+0000 7f4c537fe700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c380531d0 0x7f4c38055690 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f4c3c002f90 tx=0x7f4c3c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:14.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.244+0000 7f4c50ff9700 1 -- 172.21.2.104:0/1163293227 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4c440435e0 con 0x7f4c54067300 2021-07-11T04:02:14.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 -- 172.21.2.104:0/1163293227 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4c540aded0 con 0x7f4c54067300 2021-07-11T04:02:14.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c50ff9700 1 -- 172.21.2.104:0/1163293227 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v144) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4c44022090 con 0x7f4c54067300 2021-07-11T04:02:14.329 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:14.329 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T04:02:14.329 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:14.330 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T04:02:14.330 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T04:02:14.330 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:14.331 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:14.331 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T04:02:14.331 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T04:02:14.331 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:14.332 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T04:02:14.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 -- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c380531d0 msgr2=0x7f4c38055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c380531d0 0x7f4c38055690 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f4c3c002f90 tx=0x7f4c3c0069b0).stop 2021-07-11T04:02:14.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 -- 172.21.2.104:0/1163293227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c54067300 msgr2=0x7f4c54142220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c54067300 0x7f4c54142220 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x7f4c5405f220 tx=0x7f4c44012000).stop 2021-07-11T04:02:14.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 -- 172.21.2.104:0/1163293227 shutdown_connections 2021-07-11T04:02:14.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4c54143de0 0x7f4c54142710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4c380531d0 0x7f4c38055690 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:14.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4c54067300 0x7f4c54142220 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:14.338 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 --2- 172.21.2.104:0/1163293227 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4c54060100 0x7f4c540b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.339 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 -- 172.21.2.104:0/1163293227 >> 172.21.2.104:0/1163293227 conn(0x7f4c5405cf30 msgr2=0x7f4c540b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:14.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 -- 172.21.2.104:0/1163293227 shutdown_connections 2021-07-11T04:02:14.343 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.328+0000 7f4c59dad700 1 -- 172.21.2.104:0/1163293227 wait complete. 2021-07-11T04:02:14.343 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T04:02:14.344 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T04:02:14.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.396+0000 7f653616c700 1 Processor -- start 2021-07-11T04:02:14.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.396+0000 7f653616c700 1 -- start start 2021-07-11T04:02:14.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.396+0000 7f653616c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f65300b1ec0 0x7f65300b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.396+0000 7f653616c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65300affb0 0x7f65300b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.396+0000 7f653616c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65300b0420 con 0x7f65300a70c0 2021-07-11T04:02:14.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.396+0000 7f653616c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65300b2790 con 0x7f65300b1ec0 2021-07-11T04:02:14.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.396+0000 7f653616c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65300b0650 con 0x7f65300affb0 2021-07-11T04:02:14.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.396+0000 7f652f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f65300b1ec0 0x7f65300b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:14.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.396+0000 7f652f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f65300b1ec0 0x7f65300b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51028/0 (socket says 172.21.2.104:51028) 2021-07-11T04:02:14.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.396+0000 7f652f7fe700 1 -- 172.21.2.104:0/748948915 learned_addr learned my addr 172.21.2.104:0/748948915 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:14.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652f7fe700 1 -- 172.21.2.104:0/748948915 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65300affb0 msgr2=0x7f65300b22a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:14.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652f7fe700 1 --2- 172.21.2.104:0/748948915 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65300affb0 0x7f65300b22a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652f7fe700 1 -- 172.21.2.104:0/748948915 >> v1:172.21.2.105:6789/0 conn(0x7f65300a70c0 legacy=0x7f6530141250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:14.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652f7fe700 1 -- 172.21.2.104:0/748948915 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65300b08f0 con 0x7f65300b1ec0 2021-07-11T04:02:14.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652f7fe700 1 --2- 172.21.2.104:0/748948915 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65300b1ec0 0x7f65300b12d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7f653006a0f0 tx=0x7f6520006eb0).ready entity=mon.0 client_cookie=c933f892c13c30b8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:14.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652e7fc700 1 -- 172.21.2.104:0/748948915 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f652000d570 con 0x7f65300b1ec0 2021-07-11T04:02:14.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652e7fc700 1 -- 172.21.2.104:0/748948915 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6520000e90 con 0x7f65300b1ec0 2021-07-11T04:02:14.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652e7fc700 1 -- 172.21.2.104:0/748948915 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f652000d8c0 con 0x7f65300b1ec0 2021-07-11T04:02:14.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 -- 172.21.2.104:0/748948915 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65300b1ec0 msgr2=0x7f65300b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 --2- 172.21.2.104:0/748948915 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65300b1ec0 0x7f65300b12d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 rx=0x7f653006a0f0 tx=0x7f6520006eb0).stop 2021-07-11T04:02:14.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 -- 172.21.2.104:0/748948915 shutdown_connections 2021-07-11T04:02:14.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 --2- 172.21.2.104:0/748948915 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65300affb0 0x7f65300b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 --2- 172.21.2.104:0/748948915 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65300b1ec0 0x7f65300b12d0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:14.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 -- 172.21.2.104:0/748948915 >> 172.21.2.104:0/748948915 conn(0x7f653005f6b0 msgr2=0x7f653005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:14.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 -- 172.21.2.104:0/748948915 shutdown_connections 2021-07-11T04:02:14.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 -- 172.21.2.104:0/748948915 wait complete. 2021-07-11T04:02:14.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 Processor -- start 2021-07-11T04:02:14.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 -- start start 2021-07-11T04:02:14.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65300a70c0 0x7f6530156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65300affb0 0x7f6530156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65300b1ec0 0x7f65300a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65300afc60 con 0x7f65300affb0 2021-07-11T04:02:14.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6530157490 con 0x7f65300b1ec0 2021-07-11T04:02:14.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65301576b0 con 0x7f65300a70c0 2021-07-11T04:02:14.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65300affb0 0x7f6530156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:14.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65300affb0 0x7f6530156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51036/0 (socket says 172.21.2.104:51036) 2021-07-11T04:02:14.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652effd700 1 -- 172.21.2.104:0/2695946 learned_addr learned my addr 172.21.2.104:0/2695946 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:14.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652effd700 1 -- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65300a70c0 msgr2=0x7f6530156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652effd700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65300a70c0 0x7f6530156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652effd700 1 -- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65300b1ec0 msgr2=0x7f65300a9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:14.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652effd700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65300b1ec0 0x7f65300a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:14.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652effd700 1 -- 172.21.2.104:0/2695946 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f652000b1e0 con 0x7f65300affb0 2021-07-11T04:02:14.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652effd700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65300affb0 0x7f6530156fa0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f65300ac4e0 tx=0x7f6518006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:14.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652cff9700 1 -- 172.21.2.104:0/2695946 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6518013460 con 0x7f65300affb0 2021-07-11T04:02:14.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652cff9700 1 -- 172.21.2.104:0/2695946 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65180135c0 con 0x7f65300affb0 2021-07-11T04:02:14.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 -- 172.21.2.104:0/2695946 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6530157950 con 0x7f65300affb0 2021-07-11T04:02:14.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f652cff9700 1 -- 172.21.2.104:0/2695946 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6518013920 con 0x7f65300affb0 2021-07-11T04:02:14.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.400+0000 7f653616c700 1 -- 172.21.2.104:0/2695946 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65301428a0 con 0x7f65300affb0 2021-07-11T04:02:14.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.404+0000 7f653616c700 1 -- 172.21.2.104:0/2695946 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6530141fe0 con 0x7f65300affb0 2021-07-11T04:02:14.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.404+0000 7f652cff9700 1 -- 172.21.2.104:0/2695946 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f651800f070 con 0x7f65300affb0 2021-07-11T04:02:14.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.404+0000 7f652cff9700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65140532d0 0x7f6514055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:14.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.404+0000 7f652cff9700 1 -- 172.21.2.104:0/2695946 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f651806ad10 con 0x7f65300affb0 2021-07-11T04:02:14.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.404+0000 7f652f7fe700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65140532d0 0x7f6514055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:14.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.404+0000 7f652f7fe700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65140532d0 0x7f6514055790 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f653006a0f0 tx=0x7f6520015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:14.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.404+0000 7f652cff9700 1 -- 172.21.2.104:0/2695946 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6518042270 con 0x7f65300affb0 2021-07-11T04:02:14.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:14.488+0000 7f653616c700 1 -- 172.21.2.104:0/2695946 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f6530057350 con 0x7f65300affb0 2021-07-11T04:02:15.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f652cff9700 1 -- 172.21.2.104:0/2695946 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v145) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6518045890 con 0x7f65300affb0 2021-07-11T04:02:15.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 -- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65140532d0 msgr2=0x7f6514055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:15.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65140532d0 0x7f6514055790 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f653006a0f0 tx=0x7f6520015000).stop 2021-07-11T04:02:15.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 -- 172.21.2.104:0/2695946 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65300affb0 msgr2=0x7f6530156fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:15.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65300affb0 0x7f6530156fa0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f65300ac4e0 tx=0x7f6518006e10).stop 2021-07-11T04:02:15.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 -- 172.21.2.104:0/2695946 shutdown_connections 2021-07-11T04:02:15.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f65300b1ec0 0x7f65300a9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:15.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f65140532d0 0x7f6514055790 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:15.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f65300affb0 0x7f6530156fa0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:15.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 --2- 172.21.2.104:0/2695946 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f65300a70c0 0x7f6530156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:15.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 -- 172.21.2.104:0/2695946 >> 172.21.2.104:0/2695946 conn(0x7f653005f6b0 msgr2=0x7f653013d830 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:15.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 -- 172.21.2.104:0/2695946 shutdown_connections 2021-07-11T04:02:15.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.476+0000 7f653616c700 1 -- 172.21.2.104:0/2695946 wait complete. 2021-07-11T04:02:15.483 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T04:02:15.485 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T04:02:15.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 Processor -- start 2021-07-11T04:02:15.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- start start 2021-07-11T04:02:15.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbd580a9ec0 0x7fbd580aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:15.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813f790 0x7fbd5813fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:15.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd58061990 con 0x7fbd580603d0 2021-07-11T04:02:15.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd581400f0 con 0x7fbd580a9ec0 2021-07-11T04:02:15.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58061c30 con 0x7fbd5813f790 2021-07-11T04:02:15.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd56ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbd580a9ec0 0x7fbd580aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:15.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd56ffd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbd580a9ec0 0x7fbd580aa2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51042/0 (socket says 172.21.2.104:51042) 2021-07-11T04:02:15.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd56ffd700 1 -- 172.21.2.104:0/1548637834 learned_addr learned my addr 172.21.2.104:0/1548637834 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:15.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd56ffd700 1 -- 172.21.2.104:0/1548637834 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813f790 msgr2=0x7fbd5813fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:15.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd56ffd700 1 --2- 172.21.2.104:0/1548637834 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813f790 0x7fbd5813fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:15.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd56ffd700 1 -- 172.21.2.104:0/1548637834 >> v1:172.21.2.104:6789/0 conn(0x7fbd580603d0 legacy=0x7fbd5813ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:15.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd56ffd700 1 -- 172.21.2.104:0/1548637834 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd58060b90 con 0x7fbd580a9ec0 2021-07-11T04:02:15.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd56ffd700 1 --2- 172.21.2.104:0/1548637834 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd580a9ec0 0x7fbd580aa2a0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7fbd48001110 tx=0x7fbd40006e40).ready entity=mon.0 client_cookie=aa21d9f000518b14 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:15.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd55ffb700 1 -- 172.21.2.104:0/1548637834 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd4000bbe0 con 0x7fbd580a9ec0 2021-07-11T04:02:15.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd55ffb700 1 -- 172.21.2.104:0/1548637834 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd40000e90 con 0x7fbd580a9ec0 2021-07-11T04:02:15.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd55ffb700 1 -- 172.21.2.104:0/1548637834 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd400153f0 con 0x7fbd580a9ec0 2021-07-11T04:02:15.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- 172.21.2.104:0/1548637834 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd580a9ec0 msgr2=0x7fbd580aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:15.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 --2- 172.21.2.104:0/1548637834 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd580a9ec0 0x7fbd580aa2a0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 rx=0x7fbd48001110 tx=0x7fbd40006e40).stop 2021-07-11T04:02:15.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- 172.21.2.104:0/1548637834 shutdown_connections 2021-07-11T04:02:15.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 --2- 172.21.2.104:0/1548637834 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813f790 0x7fbd5813fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:15.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 --2- 172.21.2.104:0/1548637834 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd580a9ec0 0x7fbd580aa2a0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:15.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- 172.21.2.104:0/1548637834 >> 172.21.2.104:0/1548637834 conn(0x7fbd580b2970 msgr2=0x7fbd580b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:15.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- 172.21.2.104:0/1548637834 shutdown_connections 2021-07-11T04:02:15.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- 172.21.2.104:0/1548637834 wait complete. 2021-07-11T04:02:15.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 Processor -- start 2021-07-11T04:02:15.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- start start 2021-07-11T04:02:15.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbd580603d0 0x7fbd5814b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:15.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd580a9ec0 0x7fbd580b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:15.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813f790 0x7fbd580b4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:15.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580a77a0 con 0x7fbd580a9ec0 2021-07-11T04:02:15.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580b4800 con 0x7fbd580603d0 2021-07-11T04:02:15.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd5d555700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580a8cc0 con 0x7fbd5813f790 2021-07-11T04:02:15.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd567fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd580a9ec0 0x7fbd580b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:15.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd567fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd580a9ec0 0x7fbd580b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51050/0 (socket says 172.21.2.104:51050) 2021-07-11T04:02:15.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd567fc700 1 -- 172.21.2.104:0/4223961104 learned_addr learned my addr 172.21.2.104:0/4223961104 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:15.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd567fc700 1 -- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813f790 msgr2=0x7fbd580b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:15.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd567fc700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813f790 0x7fbd580b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:15.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd567fc700 1 -- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbd580603d0 msgr2=0x7fbd5814b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:15.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd567fc700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbd580603d0 0x7fbd5814b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:15.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd567fc700 1 -- 172.21.2.104:0/4223961104 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4000d4d0 con 0x7fbd580a9ec0 2021-07-11T04:02:15.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.540+0000 7fbd567fc700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd580a9ec0 0x7fbd580b3e20 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fbd480011d0 tx=0x7fbd48007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:15.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd4ffff700 1 -- 172.21.2.104:0/4223961104 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd48007240 con 0x7fbd580a9ec0 2021-07-11T04:02:15.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd4ffff700 1 -- 172.21.2.104:0/4223961104 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd480073a0 con 0x7fbd580a9ec0 2021-07-11T04:02:15.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd5d555700 1 -- 172.21.2.104:0/4223961104 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd580a8f60 con 0x7fbd580a9ec0 2021-07-11T04:02:15.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd4ffff700 1 -- 172.21.2.104:0/4223961104 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbd48006700 con 0x7fbd580a9ec0 2021-07-11T04:02:15.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd5d555700 1 -- 172.21.2.104:0/4223961104 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd58149e70 con 0x7fbd580a9ec0 2021-07-11T04:02:15.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd5d555700 1 -- 172.21.2.104:0/4223961104 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd58057350 con 0x7fbd580a9ec0 2021-07-11T04:02:15.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd4ffff700 1 -- 172.21.2.104:0/4223961104 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbd48010030 con 0x7fbd580a9ec0 2021-07-11T04:02:15.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd4ffff700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd3c053300 0x7fbd3c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:15.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd4ffff700 1 -- 172.21.2.104:0/4223961104 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5603+0+0 (secure 0 0 0) 0x7fbd4806b2d0 con 0x7fbd580a9ec0 2021-07-11T04:02:15.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd56ffd700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd3c053300 0x7fbd3c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:15.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd56ffd700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd3c053300 0x7fbd3c0557c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fbd40006fc0 tx=0x7fbd40018000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:15.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.544+0000 7fbd4ffff700 1 -- 172.21.2.104:0/4223961104 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbd480426d0 con 0x7fbd580a9ec0 2021-07-11T04:02:15.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:15.628+0000 7fbd5d555700 1 -- 172.21.2.104:0/4223961104 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fbd5813f540 con 0x7fbd580a9ec0 2021-07-11T04:02:16.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.488+0000 7fbd4ffff700 1 -- 172.21.2.104:0/4223961104 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v146) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbd48045cf0 con 0x7fbd580a9ec0 2021-07-11T04:02:16.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 -- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd3c053300 msgr2=0x7fbd3c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd3c053300 0x7fbd3c0557c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fbd40006fc0 tx=0x7fbd40018000).stop 2021-07-11T04:02:16.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 -- 172.21.2.104:0/4223961104 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd580a9ec0 msgr2=0x7fbd580b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd580a9ec0 0x7fbd580b3e20 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7fbd480011d0 tx=0x7fbd48007c00).stop 2021-07-11T04:02:16.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 -- 172.21.2.104:0/4223961104 shutdown_connections 2021-07-11T04:02:16.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbd5813f790 0x7fbd580b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbd3c053300 0x7fbd3c0557c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbd580a9ec0 0x7fbd580b3e20 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 --2- 172.21.2.104:0/4223961104 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbd580603d0 0x7fbd5814b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 -- 172.21.2.104:0/4223961104 >> 172.21.2.104:0/4223961104 conn(0x7fbd580b2970 msgr2=0x7fbd5813e310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:16.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 -- 172.21.2.104:0/4223961104 shutdown_connections 2021-07-11T04:02:16.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.492+0000 7fbd5d555700 1 -- 172.21.2.104:0/4223961104 wait complete. 2021-07-11T04:02:16.494 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T04:02:16.500 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T04:02:16.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 Processor -- start 2021-07-11T04:02:16.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 -- start start 2021-07-11T04:02:16.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff6480a76b0 0x7ff648066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7ff6480b0060 0x7ff64805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6480a7a90 con 0x7ff6480b1970 2021-07-11T04:02:16.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff648147620 con 0x7ff6480a76b0 2021-07-11T04:02:16.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6481478c0 con 0x7ff6480b0060 2021-07-11T04:02:16.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64659c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff6480a76b0 0x7ff648066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64659c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff6480a76b0 0x7ff648066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51056/0 (socket says 172.21.2.104:51056) 2021-07-11T04:02:16.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64659c700 1 -- 172.21.2.104:0/3090944123 learned_addr learned my addr 172.21.2.104:0/3090944123 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:16.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff646d9d700 1 --2- 172.21.2.104:0/3090944123 >> v2:172.21.2.105:3300/0 conn(0x7ff6480b0060 0x7ff64805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64659c700 1 -- 172.21.2.104:0/3090944123 >> v2:172.21.2.105:3300/0 conn(0x7ff6480b0060 msgr2=0x7ff64805fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:16.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64659c700 1 --2- 172.21.2.104:0/3090944123 >> v2:172.21.2.105:3300/0 conn(0x7ff6480b0060 0x7ff64805fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64659c700 1 -- 172.21.2.104:0/3090944123 >> v1:172.21.2.105:6789/0 conn(0x7ff6480b1970 legacy=0x7ff64805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64659c700 1 -- 172.21.2.104:0/3090944123 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff648147b60 con 0x7ff6480a76b0 2021-07-11T04:02:16.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64659c700 1 --2- 172.21.2.104:0/3090944123 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6480a76b0 0x7ff648066a70 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7ff638002f70 tx=0x7ff638006eb0).ready entity=mon.0 client_cookie=9ffa0339adee0c68 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:16.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64559a700 1 -- 172.21.2.104:0/3090944123 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff63800d5c0 con 0x7ff6480a76b0 2021-07-11T04:02:16.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64559a700 1 -- 172.21.2.104:0/3090944123 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff638000e90 con 0x7ff6480a76b0 2021-07-11T04:02:16.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64559a700 1 -- 172.21.2.104:0/3090944123 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff63800d950 con 0x7ff6480a76b0 2021-07-11T04:02:16.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 -- 172.21.2.104:0/3090944123 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6480a76b0 msgr2=0x7ff648066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 --2- 172.21.2.104:0/3090944123 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6480a76b0 0x7ff648066a70 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7ff638002f70 tx=0x7ff638006eb0).stop 2021-07-11T04:02:16.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 -- 172.21.2.104:0/3090944123 shutdown_connections 2021-07-11T04:02:16.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 --2- 172.21.2.104:0/3090944123 >> v2:172.21.2.105:3300/0 conn(0x7ff6480b0060 0x7ff64805fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 --2- 172.21.2.104:0/3090944123 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6480a76b0 0x7ff648066a70 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 -- 172.21.2.104:0/3090944123 >> 172.21.2.104:0/3090944123 conn(0x7ff6480b53e0 msgr2=0x7ff6480b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:16.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 -- 172.21.2.104:0/3090944123 shutdown_connections 2021-07-11T04:02:16.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.556+0000 7ff64c9df700 1 -- 172.21.2.104:0/3090944123 wait complete. 2021-07-11T04:02:16.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64c9df700 1 Processor -- start 2021-07-11T04:02:16.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64c9df700 1 -- start start 2021-07-11T04:02:16.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64c9df700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6480a76b0 0x7ff6480ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64c9df700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6480b0060 0x7ff648156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64c9df700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6480b1970 0x7ff648156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64c9df700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480ac100 con 0x7ff6480b0060 2021-07-11T04:02:16.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64c9df700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480aae10 con 0x7ff6480b1970 2021-07-11T04:02:16.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64c9df700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6480aa220 con 0x7ff6480a76b0 2021-07-11T04:02:16.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff645d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6480b0060 0x7ff648156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff645d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6480b0060 0x7ff648156610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51064/0 (socket says 172.21.2.104:51064) 2021-07-11T04:02:16.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff645d9b700 1 -- 172.21.2.104:0/4083261564 learned_addr learned my addr 172.21.2.104:0/4083261564 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:16.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff646d9d700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6480b1970 0x7ff648156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64659c700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6480a76b0 0x7ff6480ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff645d9b700 1 -- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6480a76b0 msgr2=0x7ff6480ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff645d9b700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6480a76b0 0x7ff6480ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff645d9b700 1 -- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6480b1970 msgr2=0x7ff648156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff645d9b700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6480b1970 0x7ff648156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff645d9b700 1 -- 172.21.2.104:0/4083261564 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63800b1e0 con 0x7ff6480b0060 2021-07-11T04:02:16.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff646d9d700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6480b1970 0x7ff648156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:02:16.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff645d9b700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6480b0060 0x7ff648156610 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7ff6480acac0 tx=0x7ff630007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:16.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff63f7fe700 1 -- 172.21.2.104:0/4083261564 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff63000ce20 con 0x7ff6480b0060 2021-07-11T04:02:16.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff63f7fe700 1 -- 172.21.2.104:0/4083261564 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63000f470 con 0x7ff6480b0060 2021-07-11T04:02:16.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64c9df700 1 -- 172.21.2.104:0/4083261564 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6480aa970 con 0x7ff6480b0060 2021-07-11T04:02:16.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff63f7fe700 1 -- 172.21.2.104:0/4083261564 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff63000f7d0 con 0x7ff6480b0060 2021-07-11T04:02:16.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64c9df700 1 -- 172.21.2.104:0/4083261564 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6480a9cd0 con 0x7ff6480b0060 2021-07-11T04:02:16.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64c9df700 1 -- 172.21.2.104:0/4083261564 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6480aa420 con 0x7ff6480b0060 2021-07-11T04:02:16.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff63f7fe700 1 -- 172.21.2.104:0/4083261564 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff63001a060 con 0x7ff6480b0060 2021-07-11T04:02:16.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff63f7fe700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff62c0532d0 0x7ff62c055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff63f7fe700 1 -- 172.21.2.104:0/4083261564 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5953+0+0 (secure 0 0 0) 0x7ff63006ab80 con 0x7ff6480b0060 2021-07-11T04:02:16.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.560+0000 7ff64659c700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff62c0532d0 0x7ff62c055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.564+0000 7ff64659c700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff62c0532d0 0x7ff62c055790 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7ff638019000 tx=0x7ff638015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:16.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.564+0000 7ff63f7fe700 1 -- 172.21.2.104:0/4083261564 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff630041f30 con 0x7ff6480b0060 2021-07-11T04:02:16.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.672+0000 7ff64c9df700 1 -- 172.21.2.104:0/4083261564 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7ff648057350 con 0x7ff6480b0060 2021-07-11T04:02:16.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff63f7fe700 1 -- 172.21.2.104:0/4083261564 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 27 and data pool 28 v73) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff630018030 con 0x7ff6480b0060 2021-07-11T04:02:16.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 -- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff62c0532d0 msgr2=0x7ff62c055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.682 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff62c0532d0 0x7ff62c055790 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7ff638019000 tx=0x7ff638015000).stop 2021-07-11T04:02:16.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 -- 172.21.2.104:0/4083261564 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6480b0060 msgr2=0x7ff648156610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6480b0060 0x7ff648156610 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7ff6480acac0 tx=0x7ff630007bf0).stop 2021-07-11T04:02:16.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 -- 172.21.2.104:0/4083261564 shutdown_connections 2021-07-11T04:02:16.683 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff6480b1970 0x7ff648156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff62c0532d0 0x7ff62c055790 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff6480b0060 0x7ff648156610 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 --2- 172.21.2.104:0/4083261564 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff6480a76b0 0x7ff6480ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 -- 172.21.2.104:0/4083261564 >> 172.21.2.104:0/4083261564 conn(0x7ff6480b53e0 msgr2=0x7ff64805f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:16.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 -- 172.21.2.104:0/4083261564 shutdown_connections 2021-07-11T04:02:16.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.680+0000 7ff64c9df700 1 -- 172.21.2.104:0/4083261564 wait complete. 2021-07-11T04:02:16.685 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 27 and data pool 28 2021-07-11T04:02:16.690 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:16.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f9590e700 1 Processor -- start 2021-07-11T04:02:16.740 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f9590e700 1 -- start start 2021-07-11T04:02:16.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f9590e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6f900a76b0 0x7f6f90066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f9590e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6f900b0060 0x7f6f9005fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f9590e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f900a7a90 con 0x7f6f900b1970 2021-07-11T04:02:16.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f9590e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f90147620 con 0x7f6f900b0060 2021-07-11T04:02:16.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f9590e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f901478c0 con 0x7f6f900a76b0 2021-07-11T04:02:16.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f8f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6f900b0060 0x7f6f9005fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f8f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6f900b0060 0x7f6f9005fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51074/0 (socket says 172.21.2.104:51074) 2021-07-11T04:02:16.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f8f7fe700 1 -- 172.21.2.104:0/1083848154 learned_addr learned my addr 172.21.2.104:0/1083848154 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:16.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f8f7fe700 1 -- 172.21.2.104:0/1083848154 >> v2:172.21.2.105:3300/0 conn(0x7f6f900a76b0 msgr2=0x7f6f90066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:16.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f8f7fe700 1 --2- 172.21.2.104:0/1083848154 >> v2:172.21.2.105:3300/0 conn(0x7f6f900a76b0 0x7f6f90066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f8f7fe700 1 -- 172.21.2.104:0/1083848154 >> v1:172.21.2.105:6789/0 conn(0x7f6f900b1970 legacy=0x7f6f9005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:16.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f8f7fe700 1 -- 172.21.2.104:0/1083848154 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f90147b60 con 0x7f6f900b0060 2021-07-11T04:02:16.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f8f7fe700 1 --2- 172.21.2.104:0/1083848154 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b0060 0x7f6f9005fdc0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f6f9006a0e0 tx=0x7f6f88009290).ready entity=mon.0 client_cookie=8de60aa13bf1d2a2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:16.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f8dffb700 1 -- 172.21.2.104:0/1083848154 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f8800b830 con 0x7f6f900b0060 2021-07-11T04:02:16.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f8dffb700 1 -- 172.21.2.104:0/1083848154 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8800b990 con 0x7f6f900b0060 2021-07-11T04:02:16.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f8dffb700 1 -- 172.21.2.104:0/1083848154 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f8800bb00 con 0x7f6f900b0060 2021-07-11T04:02:16.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f9590e700 1 -- 172.21.2.104:0/1083848154 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b0060 msgr2=0x7f6f9005fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.740+0000 7f6f9590e700 1 --2- 172.21.2.104:0/1083848154 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b0060 0x7f6f9005fdc0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 rx=0x7f6f9006a0e0 tx=0x7f6f88009290).stop 2021-07-11T04:02:16.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 -- 172.21.2.104:0/1083848154 shutdown_connections 2021-07-11T04:02:16.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 --2- 172.21.2.104:0/1083848154 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b0060 0x7f6f9005fdc0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 --2- 172.21.2.104:0/1083848154 >> v2:172.21.2.105:3300/0 conn(0x7f6f900a76b0 0x7f6f90066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 -- 172.21.2.104:0/1083848154 >> 172.21.2.104:0/1083848154 conn(0x7f6f900b53e0 msgr2=0x7f6f900b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:16.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 -- 172.21.2.104:0/1083848154 shutdown_connections 2021-07-11T04:02:16.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 -- 172.21.2.104:0/1083848154 wait complete. 2021-07-11T04:02:16.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 Processor -- start 2021-07-11T04:02:16.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 -- start start 2021-07-11T04:02:16.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f900a76b0 0x7f6f900aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b0060 0x7f6f900b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f900b1970 0x7f6f900a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90156820 con 0x7f6f900b0060 2021-07-11T04:02:16.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f900ac6a0 con 0x7f6f900a76b0 2021-07-11T04:02:16.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f90156a10 con 0x7f6f900b1970 2021-07-11T04:02:16.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f8e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b0060 0x7f6f900b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f8e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b0060 0x7f6f900b44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51080/0 (socket says 172.21.2.104:51080) 2021-07-11T04:02:16.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f8e7fc700 1 -- 172.21.2.104:0/2223048197 learned_addr learned my addr 172.21.2.104:0/2223048197 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:16.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f8effd700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f900a76b0 0x7f6f900aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f8f7fe700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f900b1970 0x7f6f900a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f8e7fc700 1 -- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f900b1970 msgr2=0x7f6f900a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f8e7fc700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f900b1970 0x7f6f900a9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f8e7fc700 1 -- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f900a76b0 msgr2=0x7f6f900aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f8e7fc700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f900a76b0 0x7f6f900aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f8e7fc700 1 -- 172.21.2.104:0/2223048197 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f88009530 con 0x7f6f900b0060 2021-07-11T04:02:16.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f8e7fc700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b0060 0x7f6f900b44a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f6f78002f10 tx=0x7f6f78006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:16.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f87fff700 1 -- 172.21.2.104:0/2223048197 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f78011400 con 0x7f6f900b0060 2021-07-11T04:02:16.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f87fff700 1 -- 172.21.2.104:0/2223048197 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f78011560 con 0x7f6f900b0060 2021-07-11T04:02:16.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f87fff700 1 -- 172.21.2.104:0/2223048197 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6f780116d0 con 0x7f6f900b0060 2021-07-11T04:02:16.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 -- 172.21.2.104:0/2223048197 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f90156cb0 con 0x7f6f900b0060 2021-07-11T04:02:16.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 -- 172.21.2.104:0/2223048197 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f900abf00 con 0x7f6f900b0060 2021-07-11T04:02:16.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f9590e700 1 -- 172.21.2.104:0/2223048197 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f90057350 con 0x7f6f900b0060 2021-07-11T04:02:16.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f87fff700 1 -- 172.21.2.104:0/2223048197 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6f7800f070 con 0x7f6f900b0060 2021-07-11T04:02:16.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f87fff700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f740531d0 0x7f6f74055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.744+0000 7f6f87fff700 1 -- 172.21.2.104:0/2223048197 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6f7806ab90 con 0x7f6f900b0060 2021-07-11T04:02:16.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.748+0000 7f6f8effd700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f740531d0 0x7f6f74055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.748+0000 7f6f8effd700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f740531d0 0x7f6f74055690 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f6f8000c000 tx=0x7f6f80008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:16.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.748+0000 7f6f87fff700 1 -- 172.21.2.104:0/2223048197 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6f78041ed0 con 0x7f6f900b0060 2021-07-11T04:02:16.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 -- 172.21.2.104:0/2223048197 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f900ac860 con 0x7f6f900b0060 2021-07-11T04:02:16.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f87fff700 1 -- 172.21.2.104:0/2223048197 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f6f780454f0 con 0x7f6f900b0060 2021-07-11T04:02:16.832 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:16.832 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":149,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:16.677257+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:02:14.500311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-07-11T04:02:15.641534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:16.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 -- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f740531d0 msgr2=0x7f6f74055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f740531d0 0x7f6f74055690 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f6f8000c000 tx=0x7f6f80008040).stop 2021-07-11T04:02:16.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 -- 172.21.2.104:0/2223048197 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b0060 msgr2=0x7f6f900b44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b0060 0x7f6f900b44a0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 rx=0x7f6f78002f10 tx=0x7f6f78006d60).stop 2021-07-11T04:02:16.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 -- 172.21.2.104:0/2223048197 shutdown_connections 2021-07-11T04:02:16.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6f900b1970 0x7f6f900a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6f740531d0 0x7f6f74055690 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6f900b0060 0x7f6f900b44a0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 --2- 172.21.2.104:0/2223048197 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6f900a76b0 0x7f6f900aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 -- 172.21.2.104:0/2223048197 >> 172.21.2.104:0/2223048197 conn(0x7f6f900b53e0 msgr2=0x7f6f90066820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:16.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 -- 172.21.2.104:0/2223048197 shutdown_connections 2021-07-11T04:02:16.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.832+0000 7f6f9590e700 1 -- 172.21.2.104:0/2223048197 wait complete. 2021-07-11T04:02:16.839 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:16.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 Processor -- start 2021-07-11T04:02:16.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 -- start start 2021-07-11T04:02:16.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f820c0a76b0 0x7f820c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f820c0b1970 0x7f820c066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f820c0b0060 0x7f820c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c067450 con 0x7f820c0a76b0 2021-07-11T04:02:16.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820c0a7a90 con 0x7f820c0b1970 2021-07-11T04:02:16.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c147620 con 0x7f820c0b0060 2021-07-11T04:02:16.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8211fb6700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f820c0a76b0 0x7f820c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8211fb6700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f820c0a76b0 0x7f820c066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51086/0 (socket says 172.21.2.104:51086) 2021-07-11T04:02:16.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8211fb6700 1 -- 172.21.2.104:0/3900835134 learned_addr learned my addr 172.21.2.104:0/3900835134 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:16.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8211fb6700 1 -- 172.21.2.104:0/3900835134 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f820c0b0060 msgr2=0x7f820c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:16.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8211fb6700 1 --2- 172.21.2.104:0/3900835134 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f820c0b0060 0x7f820c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8211fb6700 1 -- 172.21.2.104:0/3900835134 >> v2:172.21.2.105:3300/0 conn(0x7f820c0b1970 msgr2=0x7f820c066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:16.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8211fb6700 1 --2- 172.21.2.104:0/3900835134 >> v2:172.21.2.105:3300/0 conn(0x7f820c0b1970 0x7f820c066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8211fb6700 1 -- 172.21.2.104:0/3900835134 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c1478c0 con 0x7f820c0a76b0 2021-07-11T04:02:16.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8211fb6700 1 --2- 172.21.2.104:0/3900835134 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f820c0a76b0 0x7f820c066a70 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f8200002f70 tx=0x7f8200006eb0).ready entity=mon.0 client_cookie=3aaa8de73beb3ae5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:16.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8210fb4700 1 -- 172.21.2.104:0/3900835134 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f820000d570 con 0x7f820c0a76b0 2021-07-11T04:02:16.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8210fb4700 1 -- 172.21.2.104:0/3900835134 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8200000e90 con 0x7f820c0a76b0 2021-07-11T04:02:16.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f8210fb4700 1 -- 172.21.2.104:0/3900835134 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f820000d900 con 0x7f820c0a76b0 2021-07-11T04:02:16.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 -- 172.21.2.104:0/3900835134 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f820c0a76b0 msgr2=0x7f820c066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 --2- 172.21.2.104:0/3900835134 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f820c0a76b0 0x7f820c066a70 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7f8200002f70 tx=0x7f8200006eb0).stop 2021-07-11T04:02:16.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 -- 172.21.2.104:0/3900835134 shutdown_connections 2021-07-11T04:02:16.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 --2- 172.21.2.104:0/3900835134 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f820c0b0060 0x7f820c05f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 --2- 172.21.2.104:0/3900835134 >> v2:172.21.2.105:3300/0 conn(0x7f820c0b1970 0x7f820c066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 --2- 172.21.2.104:0/3900835134 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f820c0a76b0 0x7f820c066a70 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 -- 172.21.2.104:0/3900835134 >> 172.21.2.104:0/3900835134 conn(0x7f820c0b53e0 msgr2=0x7f820c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:16.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 -- 172.21.2.104:0/3900835134 shutdown_connections 2021-07-11T04:02:16.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.892+0000 7f821421a700 1 -- 172.21.2.104:0/3900835134 wait complete. 2021-07-11T04:02:16.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f821421a700 1 Processor -- start 2021-07-11T04:02:16.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f821421a700 1 -- start start 2021-07-11T04:02:16.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f821421a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f820c0a76b0 0x7f820c14dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f821421a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f820c0b0060 0x7f820c15a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f821421a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f820c0b1970 0x7f820c15a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f821421a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c14e830 con 0x7f820c0b0060 2021-07-11T04:02:16.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f821421a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c159f00 con 0x7f820c0a76b0 2021-07-11T04:02:16.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f821421a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0ac520 con 0x7f820c0b1970 2021-07-11T04:02:16.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f82117b5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f820c0b0060 0x7f820c15a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f82117b5700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f820c0b0060 0x7f820c15a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51096/0 (socket says 172.21.2.104:51096) 2021-07-11T04:02:16.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f82117b5700 1 -- 172.21.2.104:0/3324953426 learned_addr learned my addr 172.21.2.104:0/3324953426 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:16.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f82117b5700 1 -- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f820c0b1970 msgr2=0x7f820c15a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f82117b5700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f820c0b1970 0x7f820c15a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f82117b5700 1 -- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f820c0a76b0 msgr2=0x7f820c14dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f82117b5700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f820c0a76b0 0x7f820c14dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f82117b5700 1 -- 172.21.2.104:0/3324953426 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820000b1e0 con 0x7f820c0b0060 2021-07-11T04:02:16.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f82117b5700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f820c0b0060 0x7f820c15a330 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f820c14e970 tx=0x7f8208006da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:16.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f81feffd700 1 -- 172.21.2.104:0/3324953426 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8208011400 con 0x7f820c0b0060 2021-07-11T04:02:16.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f81feffd700 1 -- 172.21.2.104:0/3324953426 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8208011560 con 0x7f820c0b0060 2021-07-11T04:02:16.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f821421a700 1 -- 172.21.2.104:0/3324953426 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c0ab5d0 con 0x7f820c0b0060 2021-07-11T04:02:16.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f81feffd700 1 -- 172.21.2.104:0/3324953426 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f82080116d0 con 0x7f820c0b0060 2021-07-11T04:02:16.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f821421a700 1 -- 172.21.2.104:0/3324953426 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c0abb10 con 0x7f820c0b0060 2021-07-11T04:02:16.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f821421a700 1 -- 172.21.2.104:0/3324953426 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820c0a9d50 con 0x7f820c0b0060 2021-07-11T04:02:16.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f81feffd700 1 -- 172.21.2.104:0/3324953426 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8208017020 con 0x7f820c0b0060 2021-07-11T04:02:16.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f81feffd700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81ec053160 0x7f81ec055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:16.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f81feffd700 1 -- 172.21.2.104:0/3324953426 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f820806acd0 con 0x7f820c0b0060 2021-07-11T04:02:16.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.896+0000 7f8211fb6700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81ec053160 0x7f81ec055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:16.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.900+0000 7f8211fb6700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81ec053160 0x7f81ec055620 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8200002f70 tx=0x7f8200015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:16.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.900+0000 7f81feffd700 1 -- 172.21.2.104:0/3324953426 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8208041f00 con 0x7f820c0b0060 2021-07-11T04:02:16.980 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.980+0000 7f821421a700 1 -- 172.21.2.104:0/3324953426 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f820c057350 con 0x7f820c0b0060 2021-07-11T04:02:16.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.980+0000 7f81feffd700 1 -- 172.21.2.104:0/3324953426 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f8208015030 con 0x7f820c0b0060 2021-07-11T04:02:16.981 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:16.981 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":149,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:16.677257+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:02:14.500311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-07-11T04:02:15.641534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:16.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.980+0000 7f821421a700 1 -- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81ec053160 msgr2=0x7f81ec055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.980+0000 7f821421a700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81ec053160 0x7f81ec055620 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8200002f70 tx=0x7f8200015000).stop 2021-07-11T04:02:16.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.980+0000 7f821421a700 1 -- 172.21.2.104:0/3324953426 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f820c0b0060 msgr2=0x7f820c15a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:16.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.980+0000 7f821421a700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f820c0b0060 0x7f820c15a330 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7f820c14e970 tx=0x7f8208006da0).stop 2021-07-11T04:02:16.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.984+0000 7f821421a700 1 -- 172.21.2.104:0/3324953426 shutdown_connections 2021-07-11T04:02:16.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.984+0000 7f821421a700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f820c0b1970 0x7f820c15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.984+0000 7f821421a700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81ec053160 0x7f81ec055620 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.984+0000 7f821421a700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f820c0b0060 0x7f820c15a330 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:16.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.984+0000 7f821421a700 1 --2- 172.21.2.104:0/3324953426 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f820c0a76b0 0x7f820c14dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:16.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.984+0000 7f821421a700 1 -- 172.21.2.104:0/3324953426 >> 172.21.2.104:0/3324953426 conn(0x7f820c0b53e0 msgr2=0x7f820c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:16.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.984+0000 7f821421a700 1 -- 172.21.2.104:0/3324953426 shutdown_connections 2021-07-11T04:02:16.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:16.984+0000 7f821421a700 1 -- 172.21.2.104:0/3324953426 wait complete. 2021-07-11T04:02:16.989 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T04:02:17.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.040+0000 7fe8a0784700 1 Processor -- start 2021-07-11T04:02:17.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.040+0000 7fe8a0784700 1 -- start start 2021-07-11T04:02:17.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.040+0000 7fe8a0784700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe8980b1ec0 0x7fe8980b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.040+0000 7fe8a0784700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8980b22a0 0x7fe898141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.040+0000 7fe8a0784700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8980b0420 con 0x7fe8980affb0 2021-07-11T04:02:17.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.040+0000 7fe8a0784700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8980a77d0 con 0x7fe8980b1ec0 2021-07-11T04:02:17.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.040+0000 7fe8a0784700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980b0650 con 0x7fe8980b22a0 2021-07-11T04:02:17.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe8980affb0 0x7fe8980a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47870/0 (socket says 172.21.2.104:47870) 2021-07-11T04:02:17.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 -- 172.21.2.104:0/1141160252 learned_addr learned my addr 172.21.2.104:0/1141160252 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:17.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493370860 0 0) 0x7fe8980b0420 con 0x7fe8980affb0 2021-07-11T04:02:17.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe880003260 con 0x7fe8980affb0 2021-07-11T04:02:17.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89e520700 1 --2- 172.21.2.104:0/1141160252 >> v2:172.21.2.105:3300/0 conn(0x7fe8980b1ec0 0x7fe8980b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89ed21700 1 --2- 172.21.2.104:0/1141160252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8980b22a0 0x7fe898141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2982814719 0 0) 0x7fe880003260 con 0x7fe8980affb0 2021-07-11T04:02:17.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8980b22a0 msgr2=0x7fe898141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:17.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 --2- 172.21.2.104:0/1141160252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8980b22a0 0x7fe898141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 >> v2:172.21.2.105:3300/0 conn(0x7fe8980b1ec0 msgr2=0x7fe8980b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:17.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 --2- 172.21.2.104:0/1141160252 >> v2:172.21.2.105:3300/0 conn(0x7fe8980b1ec0 0x7fe8980b12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8980b08f0 con 0x7fe8980affb0 2021-07-11T04:02:17.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fe894001660 con 0x7fe8980affb0 2021-07-11T04:02:17.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 >> v1:172.21.2.104:6789/0 conn(0x7fe8980affb0 legacy=0x7fe8980a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:17.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 --2- 172.21.2.104:0/1141160252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8800037d0 0x7fe880003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 --2- 172.21.2.104:0/1141160252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe880004500 0x7fe880004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 --2- 172.21.2.104:0/1141160252 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe880004ec0 0x7fe88000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980a77d0 con 0x7fe880004500 2021-07-11T04:02:17.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980b0650 con 0x7fe880004ec0 2021-07-11T04:02:17.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88000bdd0 con 0x7fe8800037d0 2021-07-11T04:02:17.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89ed21700 1 --2- 172.21.2.104:0/1141160252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8980b22a0 0x7fe898141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:17.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 --2- 172.21.2.104:0/1141160252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe880004500 0x7fe880004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 -- 172.21.2.104:0/1141160252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8800037d0 msgr2=0x7fe880003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:17.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 --2- 172.21.2.104:0/1141160252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe8800037d0 0x7fe880003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 -- 172.21.2.104:0/1141160252 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe880004ec0 msgr2=0x7fe88000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:17.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 --2- 172.21.2.104:0/1141160252 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe880004ec0 0x7fe88000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 -- 172.21.2.104:0/1141160252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8980b08f0 con 0x7fe880004500 2021-07-11T04:02:17.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 --2- 172.21.2.104:0/1141160252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe880004500 0x7fe880004980 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fe894001040 tx=0x7fe89400c3c0).ready entity=mon.0 client_cookie=466471dcbf4e0e0a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:17.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8940070d0 con 0x7fe880004500 2021-07-11T04:02:17.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe894007230 con 0x7fe880004500 2021-07-11T04:02:17.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89d51e700 1 -- 172.21.2.104:0/1141160252 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89400c9b0 con 0x7fe880004500 2021-07-11T04:02:17.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 -- 172.21.2.104:0/1141160252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe880004500 msgr2=0x7fe880004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:17.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 --2- 172.21.2.104:0/1141160252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe880004500 0x7fe880004980 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 rx=0x7fe894001040 tx=0x7fe89400c3c0).stop 2021-07-11T04:02:17.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89ed21700 1 -- 172.21.2.104:0/1141160252 reap_dead start 2021-07-11T04:02:17.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 -- 172.21.2.104:0/1141160252 shutdown_connections 2021-07-11T04:02:17.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 -- 172.21.2.104:0/1141160252 >> 172.21.2.104:0/1141160252 conn(0x7fe89805f6b0 msgr2=0x7fe89805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:17.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 -- 172.21.2.104:0/1141160252 shutdown_connections 2021-07-11T04:02:17.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 -- 172.21.2.104:0/1141160252 wait complete. 2021-07-11T04:02:17.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 Processor -- start 2021-07-11T04:02:17.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 -- start start 2021-07-11T04:02:17.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8980aea60 0x7fe898156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8980abaa0 0x7fe898157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe89814b800 0x7fe898148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980aee40 con 0x7fe8980abaa0 2021-07-11T04:02:17.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980ab1a0 con 0x7fe8980aea60 2021-07-11T04:02:17.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe8a0784700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8980aadc0 con 0x7fe89814b800 2021-07-11T04:02:17.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8980abaa0 0x7fe898157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8980abaa0 0x7fe898157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51114/0 (socket says 172.21.2.104:51114) 2021-07-11T04:02:17.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.044+0000 7fe89dd1f700 1 -- 172.21.2.104:0/2861676784 learned_addr learned my addr 172.21.2.104:0/2861676784 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:17.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe89dd1f700 1 -- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe89814b800 msgr2=0x7fe898148c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:17.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe89dd1f700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe89814b800 0x7fe898148c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe89dd1f700 1 -- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8980aea60 msgr2=0x7fe898156f20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:17.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe89dd1f700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8980aea60 0x7fe898156f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe89dd1f700 1 -- 172.21.2.104:0/2861676784 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8980aab20 con 0x7fe8980abaa0 2021-07-11T04:02:17.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe89dd1f700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8980abaa0 0x7fe898157410 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fe894001040 tx=0x7fe894015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:17.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe8877fe700 1 -- 172.21.2.104:0/2861676784 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe89400ce10 con 0x7fe8980abaa0 2021-07-11T04:02:17.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe8877fe700 1 -- 172.21.2.104:0/2861676784 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe894018070 con 0x7fe8980abaa0 2021-07-11T04:02:17.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe8877fe700 1 -- 172.21.2.104:0/2861676784 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8940047d0 con 0x7fe8980abaa0 2021-07-11T04:02:17.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe8a0784700 1 -- 172.21.2.104:0/2861676784 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8980ab400 con 0x7fe8980abaa0 2021-07-11T04:02:17.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe8a0784700 1 -- 172.21.2.104:0/2861676784 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89814b4f0 con 0x7fe8980abaa0 2021-07-11T04:02:17.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe8a0784700 1 -- 172.21.2.104:0/2861676784 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe898157aa0 con 0x7fe8980abaa0 2021-07-11T04:02:17.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe8877fe700 1 -- 172.21.2.104:0/2861676784 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe894024020 con 0x7fe8980abaa0 2021-07-11T04:02:17.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe8877fe700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe878053240 0x7fe878055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe8877fe700 1 -- 172.21.2.104:0/2861676784 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe89406c0e0 con 0x7fe8980abaa0 2021-07-11T04:02:17.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe89e520700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe878053240 0x7fe878055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.048+0000 7fe89e520700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe878053240 0x7fe878055700 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fe88800a000 tx=0x7fe888008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:17.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.052+0000 7fe8877fe700 1 -- 172.21.2.104:0/2861676784 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe894043310 con 0x7fe8980abaa0 2021-07-11T04:02:17.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.160+0000 7fe8a0784700 1 -- 172.21.2.104:0/2861676784 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe8980ab590 con 0x7fe8980abaa0 2021-07-11T04:02:17.688 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8877fe700 1 -- 172.21.2.104:0/2861676784 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe894046930 con 0x7fe8980abaa0 2021-07-11T04:02:17.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 -- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe878053240 msgr2=0x7fe878055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:17.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe878053240 0x7fe878055700 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fe88800a000 tx=0x7fe888008040).stop 2021-07-11T04:02:17.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 -- 172.21.2.104:0/2861676784 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8980abaa0 msgr2=0x7fe898157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:17.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8980abaa0 0x7fe898157410 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 rx=0x7fe894001040 tx=0x7fe894015000).stop 2021-07-11T04:02:17.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 -- 172.21.2.104:0/2861676784 shutdown_connections 2021-07-11T04:02:17.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe89814b800 0x7fe898148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe878053240 0x7fe878055700 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe8980abaa0 0x7fe898157410 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 --2- 172.21.2.104:0/2861676784 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe8980aea60 0x7fe898156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 -- 172.21.2.104:0/2861676784 >> 172.21.2.104:0/2861676784 conn(0x7fe89805f6b0 msgr2=0x7fe89813d900 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:17.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 -- 172.21.2.104:0/2861676784 shutdown_connections 2021-07-11T04:02:17.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.688+0000 7fe8a0784700 1 -- 172.21.2.104:0/2861676784 wait complete. 2021-07-11T04:02:17.702 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:17.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.752+0000 7fb1fc6bb700 1 Processor -- start 2021-07-11T04:02:17.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 -- start start 2021-07-11T04:02:17.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f40a76b0 0x7fb1f4066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb1f40b1970 0x7fb1f4066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb1f40b0060 0x7fb1f405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f4067450 con 0x7fb1f40b1970 2021-07-11T04:02:17.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1f40a7a90 con 0x7fb1f40b0060 2021-07-11T04:02:17.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f4147620 con 0x7fb1f40a76b0 2021-07-11T04:02:17.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9c56700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb1f40b1970 0x7fb1f4066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9c56700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fb1f40b1970 0x7fb1f4066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51124/0 (socket says 172.21.2.104:51124) 2021-07-11T04:02:17.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9c56700 1 -- 172.21.2.104:0/286454362 learned_addr learned my addr 172.21.2.104:0/286454362 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:17.759 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9c56700 1 -- 172.21.2.104:0/286454362 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f40a76b0 msgr2=0x7fb1f4066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:17.760 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9c56700 1 --2- 172.21.2.104:0/286454362 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f40a76b0 0x7fb1f4066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9c56700 1 -- 172.21.2.104:0/286454362 >> v2:172.21.2.105:3300/0 conn(0x7fb1f40b0060 msgr2=0x7fb1f405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:17.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9c56700 1 --2- 172.21.2.104:0/286454362 >> v2:172.21.2.105:3300/0 conn(0x7fb1f40b0060 0x7fb1f405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9c56700 1 -- 172.21.2.104:0/286454362 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f41478c0 con 0x7fb1f40b1970 2021-07-11T04:02:17.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9c56700 1 --2- 172.21.2.104:0/286454362 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f40b1970 0x7fb1f4066f60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7fb1e4002f70 tx=0x7fb1e4006eb0).ready entity=mon.0 client_cookie=a913adc871a09014 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:17.761 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9455700 1 -- 172.21.2.104:0/286454362 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e400d570 con 0x7fb1f40b1970 2021-07-11T04:02:17.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9455700 1 -- 172.21.2.104:0/286454362 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e4000e90 con 0x7fb1f40b1970 2021-07-11T04:02:17.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1f9455700 1 -- 172.21.2.104:0/286454362 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1e400d900 con 0x7fb1f40b1970 2021-07-11T04:02:17.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/286454362 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f40b1970 msgr2=0x7fb1f4066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:17.762 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 --2- 172.21.2.104:0/286454362 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f40b1970 0x7fb1f4066f60 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 rx=0x7fb1e4002f70 tx=0x7fb1e4006eb0).stop 2021-07-11T04:02:17.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/286454362 shutdown_connections 2021-07-11T04:02:17.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 --2- 172.21.2.104:0/286454362 >> v2:172.21.2.105:3300/0 conn(0x7fb1f40b0060 0x7fb1f405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 --2- 172.21.2.104:0/286454362 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f40b1970 0x7fb1f4066f60 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 --2- 172.21.2.104:0/286454362 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f40a76b0 0x7fb1f4066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/286454362 >> 172.21.2.104:0/286454362 conn(0x7fb1f40b53e0 msgr2=0x7fb1f40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:17.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/286454362 shutdown_connections 2021-07-11T04:02:17.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/286454362 wait complete. 2021-07-11T04:02:17.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 Processor -- start 2021-07-11T04:02:17.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.756+0000 7fb1fc6bb700 1 -- start start 2021-07-11T04:02:17.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fc6bb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f40a76b0 0x7fb1f40abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fc6bb700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f40b0060 0x7fb1f40ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fc6bb700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1f40b1970 0x7fb1f40a9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fc6bb700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f4066880 con 0x7fb1f40a76b0 2021-07-11T04:02:17.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fc6bb700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f40b4340 con 0x7fb1f40b1970 2021-07-11T04:02:17.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fc6bb700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1f40ae670 con 0x7fb1f40b0060 2021-07-11T04:02:17.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fa457700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f40a76b0 0x7fb1f40abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fa457700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f40a76b0 0x7fb1f40abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51126/0 (socket says 172.21.2.104:51126) 2021-07-11T04:02:17.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fa457700 1 -- 172.21.2.104:0/2489154417 learned_addr learned my addr 172.21.2.104:0/2489154417 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:17.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1f9c56700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f40b0060 0x7fb1f40ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fa457700 1 -- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f40b0060 msgr2=0x7fb1f40ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:17.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fac58700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1f40b1970 0x7fb1f40a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fa457700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f40b0060 0x7fb1f40ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fa457700 1 -- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1f40b1970 msgr2=0x7fb1f40a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:17.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fa457700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1f40b1970 0x7fb1f40a9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fa457700 1 -- 172.21.2.104:0/2489154417 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e400b1e0 con 0x7fb1f40a76b0 2021-07-11T04:02:17.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fac58700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1f40b1970 0x7fb1f40a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:17.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fa457700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f40a76b0 0x7fb1f40abe40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7fb1f0002f70 tx=0x7fb1f0006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:17.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1e37fe700 1 -- 172.21.2.104:0/2489154417 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1f0013460 con 0x7fb1f40a76b0 2021-07-11T04:02:17.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1e37fe700 1 -- 172.21.2.104:0/2489154417 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f00135c0 con 0x7fb1f40a76b0 2021-07-11T04:02:17.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/2489154417 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1f40b4570 con 0x7fb1f40a76b0 2021-07-11T04:02:17.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/2489154417 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1f40b4ab0 con 0x7fb1f40a76b0 2021-07-11T04:02:17.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1e37fe700 1 -- 172.21.2.104:0/2489154417 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb1f0013920 con 0x7fb1f40a76b0 2021-07-11T04:02:17.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/2489154417 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1f4057350 con 0x7fb1f40a76b0 2021-07-11T04:02:17.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1e37fe700 1 -- 172.21.2.104:0/2489154417 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb1f001a060 con 0x7fb1f40a76b0 2021-07-11T04:02:17.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1e37fe700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d4053160 0x7fb1d4055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1e37fe700 1 -- 172.21.2.104:0/2489154417 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb1f006b010 con 0x7fb1f40a76b0 2021-07-11T04:02:17.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.760+0000 7fb1f9c56700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d4053160 0x7fb1d4055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.764+0000 7fb1f9c56700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d4053160 0x7fb1d4055620 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fb1e4019000 tx=0x7fb1e4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:17.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.764+0000 7fb1e37fe700 1 -- 172.21.2.104:0/2489154417 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fb1f0042240 con 0x7fb1f40a76b0 2021-07-11T04:02:17.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/2489154417 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1f40a94f0 con 0x7fb1f40a76b0 2021-07-11T04:02:17.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1e37fe700 1 -- 172.21.2.104:0/2489154417 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2486 (secure 0 0 0) 0x7fb1f0045860 con 0x7fb1f40a76b0 2021-07-11T04:02:17.872 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:17.872 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/696025904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":696025904},{"type":"v1","addr":"172.21.2.104:6833","nonce":696025904}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8412,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/438569403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":438569403},{"type":"v1","addr":"172.21.2.104:6835","nonce":438569403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:16.677336+0000","modified":"2021-07-11T04:02:17.681052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8415},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8415":{"gid":8415,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.2.105:6835/130583442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":130583442},{"type":"v1","addr":"172.21.2.105:6835","nonce":130583442}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-07-11T04:02:17.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d4053160 msgr2=0x7fb1d4055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:17.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d4053160 0x7fb1d4055620 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fb1e4019000 tx=0x7fb1e4015000).stop 2021-07-11T04:02:17.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/2489154417 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f40a76b0 msgr2=0x7fb1f40abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:17.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f40a76b0 0x7fb1f40abe40 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 rx=0x7fb1f0002f70 tx=0x7fb1f0006ec0).stop 2021-07-11T04:02:17.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/2489154417 shutdown_connections 2021-07-11T04:02:17.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb1f40b1970 0x7fb1f40a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb1f40b0060 0x7fb1f40ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb1d4053160 0x7fb1d4055620 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 --2- 172.21.2.104:0/2489154417 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb1f40a76b0 0x7fb1f40abe40 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/2489154417 >> 172.21.2.104:0/2489154417 conn(0x7fb1f40b53e0 msgr2=0x7fb1f405fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:17.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/2489154417 shutdown_connections 2021-07-11T04:02:17.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.872+0000 7fb1fc6bb700 1 -- 172.21.2.104:0/2489154417 wait complete. 2021-07-11T04:02:17.878 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 74 2021-07-11T04:02:17.880 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:17.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.928+0000 7f6db6fab700 1 Processor -- start 2021-07-11T04:02:17.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.928+0000 7f6db6fab700 1 -- start start 2021-07-11T04:02:17.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6db00a8b60 0x7f6db013d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db00b14c0 con 0x7f6db0143540 2021-07-11T04:02:17.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db00b1b90 con 0x7f6db005f6b0 2021-07-11T04:02:17.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db00ab3e0 con 0x7f6db00a8b60 2021-07-11T04:02:17.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6db00a8b60 0x7f6db013d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f6db00a8b60 0x7f6db013d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51134/0 (socket says 172.21.2.104:51134) 2021-07-11T04:02:17.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 -- 172.21.2.104:0/102446981 learned_addr learned my addr 172.21.2.104:0/102446981 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:17.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 -- 172.21.2.104:0/102446981 >> v1:172.21.2.105:6789/0 conn(0x7f6db005f6b0 legacy=0x7f6db013d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:17.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 -- 172.21.2.104:0/102446981 >> v1:172.21.2.104:6789/0 conn(0x7f6db0143540 legacy=0x7f6db00b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:17.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 -- 172.21.2.104:0/102446981 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0143fa0 con 0x7f6db00a8b60 2021-07-11T04:02:17.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 --2- 172.21.2.104:0/102446981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6db00a8b60 0x7f6db013d940 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7f6db00ac320 tx=0x7f6da800b530).ready entity=mon.0 client_cookie=6ef46b29a57ef08d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:17.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6daf7fe700 1 -- 172.21.2.104:0/102446981 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6da8010470 con 0x7f6db00a8b60 2021-07-11T04:02:17.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6daf7fe700 1 -- 172.21.2.104:0/102446981 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da80105d0 con 0x7f6db00a8b60 2021-07-11T04:02:17.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6daf7fe700 1 -- 172.21.2.104:0/102446981 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6da8010930 con 0x7f6db00a8b60 2021-07-11T04:02:17.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- 172.21.2.104:0/102446981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6db00a8b60 msgr2=0x7f6db013d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:17.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 --2- 172.21.2.104:0/102446981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6db00a8b60 0x7f6db013d940 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 rx=0x7f6db00ac320 tx=0x7f6da800b530).stop 2021-07-11T04:02:17.937 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- 172.21.2.104:0/102446981 shutdown_connections 2021-07-11T04:02:17.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 --2- 172.21.2.104:0/102446981 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6db00a8b60 0x7f6db013d940 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:17.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- 172.21.2.104:0/102446981 >> 172.21.2.104:0/102446981 conn(0x7f6db00b53e0 msgr2=0x7f6db00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:17.938 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- 172.21.2.104:0/102446981 shutdown_connections 2021-07-11T04:02:17.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- 172.21.2.104:0/102446981 wait complete. 2021-07-11T04:02:17.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 Processor -- start 2021-07-11T04:02:17.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- start start 2021-07-11T04:02:17.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6db005f6b0 0x7f6db013ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.939 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6db00a8b60 0x7f6db01391f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6db0143540 0x7f6db01396e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db013a120 con 0x7f6db0143540 2021-07-11T04:02:17.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db013a5f0 con 0x7f6db005f6b0 2021-07-11T04:02:17.940 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db6fab700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db013a350 con 0x7f6db00a8b60 2021-07-11T04:02:17.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6db0143540 0x7f6db01396e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6db0143540 0x7f6db01396e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51142/0 (socket says 172.21.2.104:51142) 2021-07-11T04:02:17.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 -- 172.21.2.104:0/2809739339 learned_addr learned my addr 172.21.2.104:0/2809739339 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:17.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 -- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6db00a8b60 msgr2=0x7f6db01391f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:17.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6db00a8b60 0x7f6db01391f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 -- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6db005f6b0 msgr2=0x7f6db013ab90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:17.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6db005f6b0 0x7f6db013ab90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:17.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.932+0000 7f6db5548700 1 -- 172.21.2.104:0/2809739339 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da800b7d0 con 0x7f6db0143540 2021-07-11T04:02:17.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6db5548700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6db0143540 0x7f6db01396e0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7f6db00ac320 tx=0x7f6db0144130).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:17.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6dadffb700 1 -- 172.21.2.104:0/2809739339 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6da801d070 con 0x7f6db0143540 2021-07-11T04:02:17.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6dadffb700 1 -- 172.21.2.104:0/2809739339 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da8018800 con 0x7f6db0143540 2021-07-11T04:02:17.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6db6fab700 1 -- 172.21.2.104:0/2809739339 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db013bcf0 con 0x7f6db0143540 2021-07-11T04:02:17.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6dadffb700 1 -- 172.21.2.104:0/2809739339 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6da8018b60 con 0x7f6db0143540 2021-07-11T04:02:17.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6db6fab700 1 -- 172.21.2.104:0/2809739339 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db013dab0 con 0x7f6db0143540 2021-07-11T04:02:17.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6db6fab700 1 -- 172.21.2.104:0/2809739339 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db0057350 con 0x7f6db0143540 2021-07-11T04:02:17.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6dadffb700 1 -- 172.21.2.104:0/2809739339 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6da8029070 con 0x7f6db0143540 2021-07-11T04:02:17.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6dadffb700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d940531f0 0x7f6d940556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:17.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6dadffb700 1 -- 172.21.2.104:0/2809739339 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6da806cf10 con 0x7f6db0143540 2021-07-11T04:02:17.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6db4d47700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d940531f0 0x7f6d940556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:17.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.936+0000 7f6db4d47700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d940531f0 0x7f6d940556b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f6db00b6100 tx=0x7f6da0007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:17.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:17.940+0000 7f6dadffb700 1 -- 172.21.2.104:0/2809739339 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6da8044140 con 0x7f6db0143540 2021-07-11T04:02:18.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.020+0000 7f6db6fab700 1 -- 172.21.2.104:0/2809739339 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6db00aaf50 con 0x7f6db0143540 2021-07-11T04:02:18.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.020+0000 7f6dadffb700 1 -- 172.21.2.104:0/2809739339 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f6da8027030 con 0x7f6db0143540 2021-07-11T04:02:18.021 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:18.021 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":149,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:16.677257+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:02:14.500311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-07-11T04:02:15.641534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:18.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.020+0000 7f6db6fab700 1 -- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d940531f0 msgr2=0x7f6d940556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.020+0000 7f6db6fab700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d940531f0 0x7f6d940556b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f6db00b6100 tx=0x7f6da0007810).stop 2021-07-11T04:02:18.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.020+0000 7f6db6fab700 1 -- 172.21.2.104:0/2809739339 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6db0143540 msgr2=0x7f6db01396e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.020+0000 7f6db6fab700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6db0143540 0x7f6db01396e0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 rx=0x7f6db00ac320 tx=0x7f6db0144130).stop 2021-07-11T04:02:18.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.020+0000 7f6db6fab700 1 -- 172.21.2.104:0/2809739339 shutdown_connections 2021-07-11T04:02:18.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.024+0000 7f6db6fab700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6d940531f0 0x7f6d940556b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.024+0000 7f6db6fab700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6db0143540 0x7f6db01396e0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.024+0000 7f6db6fab700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6db00a8b60 0x7f6db01391f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.024+0000 7f6db6fab700 1 --2- 172.21.2.104:0/2809739339 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6db005f6b0 0x7f6db013ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.024+0000 7f6db6fab700 1 -- 172.21.2.104:0/2809739339 >> 172.21.2.104:0/2809739339 conn(0x7f6db00b53e0 msgr2=0x7f6db00aa6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:18.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.024+0000 7f6db6fab700 1 -- 172.21.2.104:0/2809739339 shutdown_connections 2021-07-11T04:02:18.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.024+0000 7f6db6fab700 1 -- 172.21.2.104:0/2809739339 wait complete. 2021-07-11T04:02:18.029 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:02:18.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062387f700 1 Processor -- start 2021-07-11T04:02:18.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062387f700 1 -- start start 2021-07-11T04:02:18.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062387f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f061c141b40 0x7f061c14cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062387f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061c0b31d0 con 0x7f061c0b7810 2021-07-11T04:02:18.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062387f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061c0b7bf0 con 0x7f061c0b1170 2021-07-11T04:02:18.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062387f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061c0675c0 con 0x7f061c141b40 2021-07-11T04:02:18.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062161b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f061c0b7810 0x7f061c1445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47916/0 (socket says 172.21.2.104:47916) 2021-07-11T04:02:18.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062161b700 1 -- 172.21.2.104:0/1288646133 learned_addr learned my addr 172.21.2.104:0/1288646133 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:18.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442401851 0 0) 0x7f061c0b31d0 con 0x7f061c0b7810 2021-07-11T04:02:18.091 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0604003260 con 0x7f061c0b7810 2021-07-11T04:02:18.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f0621e1c700 1 --2- 172.21.2.104:0/1288646133 >> v2:172.21.2.105:3300/0 conn(0x7f061c141b40 0x7f061c14cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:18.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1819416120 0 0) 0x7f0604003260 con 0x7f061c0b7810 2021-07-11T04:02:18.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 >> v2:172.21.2.105:3300/0 conn(0x7f061c141b40 msgr2=0x7f061c14cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:18.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 --2- 172.21.2.104:0/1288646133 >> v2:172.21.2.105:3300/0 conn(0x7f061c141b40 0x7f061c14cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 >> v1:172.21.2.105:6789/0 conn(0x7f061c0b1170 legacy=0x7f061c141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:18.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c0aae70 con 0x7f061c0b7810 2021-07-11T04:02:18.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f0618001660 con 0x7f061c0b7810 2021-07-11T04:02:18.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 >> v1:172.21.2.104:6789/0 conn(0x7f061c0b7810 legacy=0x7f061c1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 --2- 172.21.2.104:0/1288646133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 0x7f0604003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 --2- 172.21.2.104:0/1288646133 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0604004500 0x7f0604004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 --2- 172.21.2.104:0/1288646133 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0604004f70 0x7f060400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c0675c0 con 0x7f0604003820 2021-07-11T04:02:18.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060400be50 con 0x7f0604004f70 2021-07-11T04:02:18.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f060400c010 con 0x7f0604004500 2021-07-11T04:02:18.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062161b700 1 --2- 172.21.2.104:0/1288646133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 0x7f0604003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:18.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062161b700 1 -- 172.21.2.104:0/1288646133 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0604004500 msgr2=0x7f0604004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:18.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062161b700 1 --2- 172.21.2.104:0/1288646133 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0604004500 0x7f0604004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062161b700 1 -- 172.21.2.104:0/1288646133 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0604004f70 msgr2=0x7f060400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:18.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062161b700 1 --2- 172.21.2.104:0/1288646133 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0604004f70 0x7f060400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062161b700 1 -- 172.21.2.104:0/1288646133 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0604003260 con 0x7f0604003820 2021-07-11T04:02:18.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f0621e1c700 1 -- 172.21.2.104:0/1288646133 reap_dead start 2021-07-11T04:02:18.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062161b700 1 --2- 172.21.2.104:0/1288646133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 0x7f0604003c20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f061c060740 tx=0x7f061c065ad0).ready entity=mon.0 client_cookie=fa6bd38d90706268 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:18.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06180070d0 con 0x7f0604003820 2021-07-11T04:02:18.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0618007230 con 0x7f0604003820 2021-07-11T04:02:18.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f060bfff700 1 -- 172.21.2.104:0/1288646133 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f061800c8f0 con 0x7f0604003820 2021-07-11T04:02:18.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062387f700 1 -- 172.21.2.104:0/1288646133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 msgr2=0x7f0604003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.084+0000 7f062387f700 1 --2- 172.21.2.104:0/1288646133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 0x7f0604003c20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 rx=0x7f061c060740 tx=0x7f061c065ad0).stop 2021-07-11T04:02:18.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 -- 172.21.2.104:0/1288646133 shutdown_connections 2021-07-11T04:02:18.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 --2- 172.21.2.104:0/1288646133 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 0x7f0604003c20 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 -- 172.21.2.104:0/1288646133 >> 172.21.2.104:0/1288646133 conn(0x7f061c05f6b0 msgr2=0x7f061c05fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:18.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 -- 172.21.2.104:0/1288646133 shutdown_connections 2021-07-11T04:02:18.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 -- 172.21.2.104:0/1288646133 wait complete. 2021-07-11T04:02:18.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 Processor -- start 2021-07-11T04:02:18.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 -- start start 2021-07-11T04:02:18.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 0x7f061c061070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f061c0a9810 0x7f061c0a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f061c159b10 0x7f061c159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c061c30 con 0x7f0604003820 2021-07-11T04:02:18.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c156fc0 con 0x7f061c0a9810 2021-07-11T04:02:18.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c062680 con 0x7f061c159b10 2021-07-11T04:02:18.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062161b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 0x7f061c061070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:18.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062161b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 0x7f061c061070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51160/0 (socket says 172.21.2.104:51160) 2021-07-11T04:02:18.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062161b700 1 -- 172.21.2.104:0/4219366042 learned_addr learned my addr 172.21.2.104:0/4219366042 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:18.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062161b700 1 -- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f061c159b10 msgr2=0x7f061c159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062161b700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f061c159b10 0x7f061c159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062161b700 1 -- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f061c0a9810 msgr2=0x7f061c0a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062161b700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f061c0a9810 0x7f061c0a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062161b700 1 -- 172.21.2.104:0/4219366042 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c061e20 con 0x7f0604003820 2021-07-11T04:02:18.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062161b700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 0x7f061c061070 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7f061c0a9420 tx=0x7f0618015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:18.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f060a7fc700 1 -- 172.21.2.104:0/4219366042 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0618004520 con 0x7f0604003820 2021-07-11T04:02:18.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f060a7fc700 1 -- 172.21.2.104:0/4219366042 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0618004680 con 0x7f0604003820 2021-07-11T04:02:18.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 -- 172.21.2.104:0/4219366042 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c1573f0 con 0x7f0604003820 2021-07-11T04:02:18.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 -- 172.21.2.104:0/4219366042 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c061560 con 0x7f0604003820 2021-07-11T04:02:18.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f060a7fc700 1 -- 172.21.2.104:0/4219366042 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f06180049e0 con 0x7f0604003820 2021-07-11T04:02:18.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f062387f700 1 -- 172.21.2.104:0/4219366042 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c061fb0 con 0x7f0604003820 2021-07-11T04:02:18.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f060a7fc700 1 -- 172.21.2.104:0/4219366042 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0618010070 con 0x7f0604003820 2021-07-11T04:02:18.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f060a7fc700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05fc0531d0 0x7f05fc055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.088+0000 7f060a7fc700 1 -- 172.21.2.104:0/4219366042 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f061806c360 con 0x7f0604003820 2021-07-11T04:02:18.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.092+0000 7f0620e1a700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05fc0531d0 0x7f05fc055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:18.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.092+0000 7f0620e1a700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05fc0531d0 0x7f05fc055690 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f060c000f70 tx=0x7f060c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:18.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.092+0000 7f060a7fc700 1 -- 172.21.2.104:0/4219366042 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0618043640 con 0x7f0604003820 2021-07-11T04:02:18.202 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.204+0000 7f062387f700 1 -- 172.21.2.104:0/4219366042 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f061c061fb0 con 0x7f0604003820 2021-07-11T04:02:18.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f060a7fc700 1 -- 172.21.2.104:0/4219366042 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v34) v1 ==== 184+0+0 (secure 0 0 0) 0x7f0618046c60 con 0x7f0604003820 2021-07-11T04:02:18.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 -- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05fc0531d0 msgr2=0x7f05fc055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05fc0531d0 0x7f05fc055690 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f060c000f70 tx=0x7f060c0069b0).stop 2021-07-11T04:02:18.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 -- 172.21.2.104:0/4219366042 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 msgr2=0x7f061c061070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 0x7f061c061070 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 rx=0x7f061c0a9420 tx=0x7f0618015000).stop 2021-07-11T04:02:18.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 -- 172.21.2.104:0/4219366042 shutdown_connections 2021-07-11T04:02:18.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f061c159b10 0x7f061c159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f061c0a9810 0x7f061c0a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f05fc0531d0 0x7f05fc055690 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 --2- 172.21.2.104:0/4219366042 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0604003820 0x7f061c061070 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 -- 172.21.2.104:0/4219366042 >> 172.21.2.104:0/4219366042 conn(0x7f061c05f6b0 msgr2=0x7f061c0b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:18.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 -- 172.21.2.104:0/4219366042 shutdown_connections 2021-07-11T04:02:18.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.208+0000 7f062387f700 1 -- 172.21.2.104:0/4219366042 wait complete. 2021-07-11T04:02:18.212 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T04:02:18.215 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:02:18.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 Processor -- start 2021-07-11T04:02:18.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- start start 2021-07-11T04:02:18.271 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f28840b1970 0x7f288405fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28840a7a90 con 0x7f28840a76b0 2021-07-11T04:02:18.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2884147620 con 0x7f2884067180 2021-07-11T04:02:18.272 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28841478c0 con 0x7f28840b1970 2021-07-11T04:02:18.275 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f2889d0b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f28840b1970 0x7f288405fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:18.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f2889d0b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f28840b1970 0x7f288405fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51170/0 (socket says 172.21.2.104:51170) 2021-07-11T04:02:18.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f2889d0b700 1 -- 172.21.2.104:0/3070218684 learned_addr learned my addr 172.21.2.104:0/3070218684 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:18.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f2889d0b700 1 -- 172.21.2.104:0/3070218684 >> v1:172.21.2.105:6789/0 conn(0x7f2884067180 legacy=0x7f288405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:18.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f2889d0b700 1 -- 172.21.2.104:0/3070218684 >> v1:172.21.2.104:6789/0 conn(0x7f28840a76b0 legacy=0x7f2884066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f2889d0b700 1 -- 172.21.2.104:0/3070218684 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2884147b60 con 0x7f28840b1970 2021-07-11T04:02:18.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f2889d0b700 1 --2- 172.21.2.104:0/3070218684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28840b1970 0x7f288405fdc0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7f288406a180 tx=0x7f287c009290).ready entity=mon.0 client_cookie=c5db033dc3fac3b7 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:18.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f2877fff700 1 -- 172.21.2.104:0/3070218684 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287c00b830 con 0x7f28840b1970 2021-07-11T04:02:18.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f2877fff700 1 -- 172.21.2.104:0/3070218684 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c00b990 con 0x7f28840b1970 2021-07-11T04:02:18.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f2877fff700 1 -- 172.21.2.104:0/3070218684 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287c00bb40 con 0x7f28840b1970 2021-07-11T04:02:18.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- 172.21.2.104:0/3070218684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28840b1970 msgr2=0x7f288405fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 --2- 172.21.2.104:0/3070218684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28840b1970 0x7f288405fdc0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7f288406a180 tx=0x7f287c009290).stop 2021-07-11T04:02:18.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- 172.21.2.104:0/3070218684 shutdown_connections 2021-07-11T04:02:18.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 --2- 172.21.2.104:0/3070218684 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28840b1970 0x7f288405fdc0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- 172.21.2.104:0/3070218684 >> 172.21.2.104:0/3070218684 conn(0x7f28840b53e0 msgr2=0x7f28840b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:18.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- 172.21.2.104:0/3070218684 shutdown_connections 2021-07-11T04:02:18.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- 172.21.2.104:0/3070218684 wait complete. 2021-07-11T04:02:18.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 Processor -- start 2021-07-11T04:02:18.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- start start 2021-07-11T04:02:18.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2884067180 0x7f28840b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28840a76b0 0x7f28840ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28840b1970 0x7f28840aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28840abcd0 con 0x7f2884067180 2021-07-11T04:02:18.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28840ab720 con 0x7f28840a76b0 2021-07-11T04:02:18.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288b76e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28840ab970 con 0x7f28840b1970 2021-07-11T04:02:18.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288950a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2884067180 0x7f28840b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:18.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288950a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2884067180 0x7f28840b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51174/0 (socket says 172.21.2.104:51174) 2021-07-11T04:02:18.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.272+0000 7f288950a700 1 -- 172.21.2.104:0/1286237648 learned_addr learned my addr 172.21.2.104:0/1286237648 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:18.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f288950a700 1 -- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28840b1970 msgr2=0x7f28840aca10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:18.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f288950a700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28840b1970 0x7f28840aca10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f288950a700 1 -- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28840a76b0 msgr2=0x7f28840ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f288950a700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28840a76b0 0x7f28840ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f288950a700 1 -- 172.21.2.104:0/1286237648 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c009530 con 0x7f2884067180 2021-07-11T04:02:18.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f288950a700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2884067180 0x7f28840b2750 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f288406a0e0 tx=0x7f2878007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:18.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f28767fc700 1 -- 172.21.2.104:0/1286237648 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287800ce30 con 0x7f2884067180 2021-07-11T04:02:18.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f28767fc700 1 -- 172.21.2.104:0/1286237648 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287800f470 con 0x7f2884067180 2021-07-11T04:02:18.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f288b76e700 1 -- 172.21.2.104:0/1286237648 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28840abf00 con 0x7f2884067180 2021-07-11T04:02:18.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f28767fc700 1 -- 172.21.2.104:0/1286237648 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f287800f7d0 con 0x7f2884067180 2021-07-11T04:02:18.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f288b76e700 1 -- 172.21.2.104:0/1286237648 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28840ae080 con 0x7f2884067180 2021-07-11T04:02:18.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f288b76e700 1 -- 172.21.2.104:0/1286237648 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28840ac090 con 0x7f2884067180 2021-07-11T04:02:18.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f28767fc700 1 -- 172.21.2.104:0/1286237648 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2878019030 con 0x7f2884067180 2021-07-11T04:02:18.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f28767fc700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2864053180 0x7f2864055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f28767fc700 1 -- 172.21.2.104:0/1286237648 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f287806a8e0 con 0x7f2884067180 2021-07-11T04:02:18.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f2888d09700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2864053180 0x7f2864055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:18.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f2888d09700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2864053180 0x7f2864055640 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f2880002f90 tx=0x7f28800069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:18.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.276+0000 7f28767fc700 1 -- 172.21.2.104:0/1286237648 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2878041bc0 con 0x7f2884067180 2021-07-11T04:02:18.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.388+0000 7f288b76e700 1 -- 172.21.2.104:0/1286237648 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f2884057350 con 0x7f2884067180 2021-07-11T04:02:18.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.392+0000 7f28767fc700 1 -- 172.21.2.104:0/1286237648 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v35) v1 ==== 184+0+0 (secure 0 0 0) 0x7f28780451e0 con 0x7f2884067180 2021-07-11T04:02:18.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 -- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2864053180 msgr2=0x7f2864055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2864053180 0x7f2864055640 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f2880002f90 tx=0x7f28800069b0).stop 2021-07-11T04:02:18.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 -- 172.21.2.104:0/1286237648 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2884067180 msgr2=0x7f28840b2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2884067180 0x7f28840b2750 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f288406a0e0 tx=0x7f2878007c50).stop 2021-07-11T04:02:18.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 -- 172.21.2.104:0/1286237648 shutdown_connections 2021-07-11T04:02:18.396 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28840b1970 0x7f28840aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28840a76b0 0x7f28840ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2864053180 0x7f2864055640 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 --2- 172.21.2.104:0/1286237648 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2884067180 0x7f28840b2750 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 -- 172.21.2.104:0/1286237648 >> 172.21.2.104:0/1286237648 conn(0x7f28840b53e0 msgr2=0x7f2884148c90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:18.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 -- 172.21.2.104:0/1286237648 shutdown_connections 2021-07-11T04:02:18.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.396+0000 7f288b76e700 1 -- 172.21.2.104:0/1286237648 wait complete. 2021-07-11T04:02:18.398 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T04:02:18.401 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:18.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a399ab700 1 Processor -- start 2021-07-11T04:02:18.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a399ab700 1 -- start start 2021-07-11T04:02:18.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a399ab700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a340a76b0 0x7f0a34066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a399ab700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0a340b1970 0x7f0a34066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a399ab700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0a340b0060 0x7f0a3405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a399ab700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a34067450 con 0x7f0a340b1970 2021-07-11T04:02:18.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a399ab700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a340a7a90 con 0x7f0a340b0060 2021-07-11T04:02:18.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a399ab700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a34147620 con 0x7f0a340a76b0 2021-07-11T04:02:18.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a327fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0a340b1970 0x7f0a34066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:18.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a327fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0a340b1970 0x7f0a34066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51184/0 (socket says 172.21.2.104:51184) 2021-07-11T04:02:18.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a327fc700 1 -- 172.21.2.104:0/3411358507 learned_addr learned my addr 172.21.2.104:0/3411358507 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:18.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a327fc700 1 -- 172.21.2.104:0/3411358507 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a340a76b0 msgr2=0x7f0a34066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:18.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a327fc700 1 --2- 172.21.2.104:0/3411358507 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a340a76b0 0x7f0a34066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a327fc700 1 -- 172.21.2.104:0/3411358507 >> v2:172.21.2.105:3300/0 conn(0x7f0a340b0060 msgr2=0x7f0a3405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:18.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a327fc700 1 --2- 172.21.2.104:0/3411358507 >> v2:172.21.2.105:3300/0 conn(0x7f0a340b0060 0x7f0a3405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a327fc700 1 -- 172.21.2.104:0/3411358507 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a341478c0 con 0x7f0a340b1970 2021-07-11T04:02:18.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a327fc700 1 --2- 172.21.2.104:0/3411358507 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a340b1970 0x7f0a34066f60 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7f0a1c002f70 tx=0x7f0a1c006eb0).ready entity=mon.0 client_cookie=1bb77772e528be53 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:18.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a31ffb700 1 -- 172.21.2.104:0/3411358507 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1c00d570 con 0x7f0a340b1970 2021-07-11T04:02:18.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a31ffb700 1 -- 172.21.2.104:0/3411358507 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1c000e90 con 0x7f0a340b1970 2021-07-11T04:02:18.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a31ffb700 1 -- 172.21.2.104:0/3411358507 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1c00d900 con 0x7f0a340b1970 2021-07-11T04:02:18.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a399ab700 1 -- 172.21.2.104:0/3411358507 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a340b1970 msgr2=0x7f0a34066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.452+0000 7f0a399ab700 1 --2- 172.21.2.104:0/3411358507 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a340b1970 0x7f0a34066f60 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7f0a1c002f70 tx=0x7f0a1c006eb0).stop 2021-07-11T04:02:18.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 -- 172.21.2.104:0/3411358507 shutdown_connections 2021-07-11T04:02:18.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 --2- 172.21.2.104:0/3411358507 >> v2:172.21.2.105:3300/0 conn(0x7f0a340b0060 0x7f0a3405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 --2- 172.21.2.104:0/3411358507 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a340b1970 0x7f0a34066f60 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 --2- 172.21.2.104:0/3411358507 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a340a76b0 0x7f0a34066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 -- 172.21.2.104:0/3411358507 >> 172.21.2.104:0/3411358507 conn(0x7f0a340b53e0 msgr2=0x7f0a340b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:18.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 -- 172.21.2.104:0/3411358507 shutdown_connections 2021-07-11T04:02:18.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 -- 172.21.2.104:0/3411358507 wait complete. 2021-07-11T04:02:18.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 Processor -- start 2021-07-11T04:02:18.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 -- start start 2021-07-11T04:02:18.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a340a76b0 0x7f0a340abe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a340b0060 0x7f0a340ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a340b1970 0x7f0a340a9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a34066880 con 0x7f0a340b0060 2021-07-11T04:02:18.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340b4340 con 0x7f0a340a76b0 2021-07-11T04:02:18.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340ae670 con 0x7f0a340b1970 2021-07-11T04:02:18.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a327fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a340b0060 0x7f0a340ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:18.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a327fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a340b0060 0x7f0a340ae180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51192/0 (socket says 172.21.2.104:51192) 2021-07-11T04:02:18.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a327fc700 1 -- 172.21.2.104:0/1631357303 learned_addr learned my addr 172.21.2.104:0/1631357303 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:18.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a327fc700 1 -- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a340b1970 msgr2=0x7f0a340a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a327fc700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a340b1970 0x7f0a340a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a327fc700 1 -- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a340a76b0 msgr2=0x7f0a340abe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a327fc700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a340a76b0 0x7f0a340abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a327fc700 1 -- 172.21.2.104:0/1631357303 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c00b1e0 con 0x7f0a340b0060 2021-07-11T04:02:18.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a327fc700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a340b0060 0x7f0a340ae180 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7f0a1c002f70 tx=0x7f0a1c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:18.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a2bfff700 1 -- 172.21.2.104:0/1631357303 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1c021070 con 0x7f0a340b0060 2021-07-11T04:02:18.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a2bfff700 1 -- 172.21.2.104:0/1631357303 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1c000e90 con 0x7f0a340b0060 2021-07-11T04:02:18.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 -- 172.21.2.104:0/1631357303 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a340b4570 con 0x7f0a340b0060 2021-07-11T04:02:18.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a2bfff700 1 -- 172.21.2.104:0/1631357303 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0a1c00de80 con 0x7f0a340b0060 2021-07-11T04:02:18.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 -- 172.21.2.104:0/1631357303 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a340b4ab0 con 0x7f0a340b0060 2021-07-11T04:02:18.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a399ab700 1 -- 172.21.2.104:0/1631357303 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a34057350 con 0x7f0a340b0060 2021-07-11T04:02:18.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a2bfff700 1 -- 172.21.2.104:0/1631357303 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0a1c00f070 con 0x7f0a340b0060 2021-07-11T04:02:18.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a2bfff700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a18053160 0x7f0a18055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:18.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.456+0000 7f0a2bfff700 1 -- 172.21.2.104:0/1631357303 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0a1c06c9b0 con 0x7f0a340b0060 2021-07-11T04:02:18.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.460+0000 7f0a32ffd700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a18053160 0x7f0a18055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:18.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.460+0000 7f0a32ffd700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a18053160 0x7f0a18055620 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f0a24002f70 tx=0x7f0a24006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:18.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.460+0000 7f0a2bfff700 1 -- 172.21.2.104:0/1631357303 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0a1c043c90 con 0x7f0a340b0060 2021-07-11T04:02:18.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 -- 172.21.2.104:0/1631357303 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a340a94f0 con 0x7f0a340b0060 2021-07-11T04:02:18.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a2bfff700 1 -- 172.21.2.104:0/1631357303 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f0a1c0472b0 con 0x7f0a340b0060 2021-07-11T04:02:18.569 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:18.569 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/696025904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":696025904},{"type":"v1","addr":"172.21.2.104:6833","nonce":696025904}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8412,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/438569403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":438569403},{"type":"v1","addr":"172.21.2.104:6835","nonce":438569403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:16.677336+0000","modified":"2021-07-11T04:02:17.681052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8415},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8415":{"gid":8415,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.2.105:6835/130583442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":130583442},{"type":"v1","addr":"172.21.2.105:6835","nonce":130583442}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-07-11T04:02:18.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 -- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a18053160 msgr2=0x7f0a18055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a18053160 0x7f0a18055620 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f0a24002f70 tx=0x7f0a24006a80).stop 2021-07-11T04:02:18.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 -- 172.21.2.104:0/1631357303 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a340b0060 msgr2=0x7f0a340ae180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:18.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a340b0060 0x7f0a340ae180 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7f0a1c002f70 tx=0x7f0a1c015000).stop 2021-07-11T04:02:18.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 -- 172.21.2.104:0/1631357303 shutdown_connections 2021-07-11T04:02:18.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0a340b1970 0x7f0a340a9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0a18053160 0x7f0a18055620 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0a340b0060 0x7f0a340ae180 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:18.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 --2- 172.21.2.104:0/1631357303 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0a340a76b0 0x7f0a340abe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:18.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 -- 172.21.2.104:0/1631357303 >> 172.21.2.104:0/1631357303 conn(0x7f0a340b53e0 msgr2=0x7f0a3405fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:18.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 -- 172.21.2.104:0/1631357303 shutdown_connections 2021-07-11T04:02:18.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:18.568+0000 7f0a399ab700 1 -- 172.21.2.104:0/1631357303 wait complete. 2021-07-11T04:02:18.574 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 74 2021-07-11T04:02:18.881 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:02:16.677336+0000', 'modified': '2021-07-11T04:02:17.681052+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8415}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8415': {'gid': 8415, 'name': 'b', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.105:6835/130583442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.105:6834', 'nonce': 130583442}, {'type': 'v1', 'addr': '172.21.2.105:6835', 'nonce': 130583442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:02:18.881 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:02:18.882 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2021-07-11T04:02:18.945 INFO:teuthology.orchestra.run.gibba005.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":8415,"want_state":"up:active","state":"up:active","rank_uptime":2.262094378,"mdsmap_epoch":74,"osdmap_epoch":149,"osdmap_epoch_barrier":149,"uptime":6.4510338049999998} 2021-07-11T04:02:18.946 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 8415, "mdsmap_epoch": 74, "osdmap_epoch": 149, "osdmap_epoch_barrier": 149, "rank_uptime": 2.262094378, "state": "up:active", "uptime": 6.451033805, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:02:18.947 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:19.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b3db4700 1 Processor -- start 2021-07-11T04:02:19.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b3db4700 1 -- start start 2021-07-11T04:02:19.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b3db4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f66ac067460 0x7f66ac146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b3db4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 0x7f66ac0ab7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b3db4700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66ac0aa030 con 0x7f66ac0b08f0 2021-07-11T04:02:19.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b3db4700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66ac067840 con 0x7f66ac067460 2021-07-11T04:02:19.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b3db4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0b1c20 con 0x7f66ac0b0330 2021-07-11T04:02:19.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b1b50700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f66ac067460 0x7f66ac146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b1b50700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f66ac067460 0x7f66ac146bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51196/0 (socket says 172.21.2.104:51196) 2021-07-11T04:02:19.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b1b50700 1 -- 172.21.2.104:0/3260846460 learned_addr learned my addr 172.21.2.104:0/3260846460 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:19.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b2351700 1 --2- 172.21.2.104:0/3260846460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 0x7f66ac0ab7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b1b50700 1 -- 172.21.2.104:0/3260846460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 msgr2=0x7f66ac0ab7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:19.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b1b50700 1 --2- 172.21.2.104:0/3260846460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 0x7f66ac0ab7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b1b50700 1 -- 172.21.2.104:0/3260846460 >> v1:172.21.2.105:6789/0 conn(0x7f66ac0b08f0 legacy=0x7f66ac1470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:19.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b1b50700 1 -- 172.21.2.104:0/3260846460 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac0ab340 con 0x7f66ac067460 2021-07-11T04:02:19.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.004+0000 7f66b2351700 1 --2- 172.21.2.104:0/3260846460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 0x7f66ac0ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:19.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b1b50700 1 --2- 172.21.2.104:0/3260846460 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66ac067460 0x7f66ac146bd0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7f66ac062740 tx=0x7f66a8006eb0).ready entity=mon.0 client_cookie=4ccd45603dd78320 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b0b4e700 1 -- 172.21.2.104:0/3260846460 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a800d570 con 0x7f66ac067460 2021-07-11T04:02:19.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b0b4e700 1 -- 172.21.2.104:0/3260846460 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a8000e90 con 0x7f66ac067460 2021-07-11T04:02:19.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b0b4e700 1 -- 172.21.2.104:0/3260846460 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a800d8c0 con 0x7f66ac067460 2021-07-11T04:02:19.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- 172.21.2.104:0/3260846460 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66ac067460 msgr2=0x7f66ac146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 --2- 172.21.2.104:0/3260846460 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66ac067460 0x7f66ac146bd0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7f66ac062740 tx=0x7f66a8006eb0).stop 2021-07-11T04:02:19.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- 172.21.2.104:0/3260846460 shutdown_connections 2021-07-11T04:02:19.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 --2- 172.21.2.104:0/3260846460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 0x7f66ac0ab7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 --2- 172.21.2.104:0/3260846460 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66ac067460 0x7f66ac146bd0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- 172.21.2.104:0/3260846460 >> 172.21.2.104:0/3260846460 conn(0x7f66ac061070 msgr2=0x7f66ac05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- 172.21.2.104:0/3260846460 shutdown_connections 2021-07-11T04:02:19.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- 172.21.2.104:0/3260846460 wait complete. 2021-07-11T04:02:19.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 Processor -- start 2021-07-11T04:02:19.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- start start 2021-07-11T04:02:19.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66ac067460 0x7f66ac0a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 0x7f66ac155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66ac0b08f0 0x7f66ac156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac0a1fb0 con 0x7f66ac067460 2021-07-11T04:02:19.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac09f670 con 0x7f66ac0b08f0 2021-07-11T04:02:19.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66ac09f2e0 con 0x7f66ac0b0330 2021-07-11T04:02:19.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b1b50700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66ac067460 0x7f66ac0a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b1b50700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66ac067460 0x7f66ac0a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51202/0 (socket says 172.21.2.104:51202) 2021-07-11T04:02:19.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b1b50700 1 -- 172.21.2.104:0/1557301435 learned_addr learned my addr 172.21.2.104:0/1557301435 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:19.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b134f700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 0x7f66ac155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b2351700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66ac0b08f0 0x7f66ac156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b1b50700 1 -- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 msgr2=0x7f66ac155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b1b50700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 0x7f66ac155e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b1b50700 1 -- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66ac0b08f0 msgr2=0x7f66ac156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b1b50700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66ac0b08f0 0x7f66ac156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b1b50700 1 -- 172.21.2.104:0/1557301435 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a800b1e0 con 0x7f66ac067460 2021-07-11T04:02:19.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b134f700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 0x7f66ac155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:02:19.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b1b50700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66ac067460 0x7f66ac0a1130 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7f66ac062740 tx=0x7f66a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f669affd700 1 -- 172.21.2.104:0/1557301435 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a8021070 con 0x7f66ac067460 2021-07-11T04:02:19.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f669affd700 1 -- 172.21.2.104:0/1557301435 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a8000e90 con 0x7f66ac067460 2021-07-11T04:02:19.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- 172.21.2.104:0/1557301435 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ac09f8a0 con 0x7f66ac067460 2021-07-11T04:02:19.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- 172.21.2.104:0/1557301435 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66ac0afaa0 con 0x7f66ac067460 2021-07-11T04:02:19.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f669affd700 1 -- 172.21.2.104:0/1557301435 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f66a8011a50 con 0x7f66ac067460 2021-07-11T04:02:19.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f66b3db4700 1 -- 172.21.2.104:0/1557301435 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6688003410 con 0x7f66ac067460 2021-07-11T04:02:19.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.008+0000 7f669affd700 1 -- 172.21.2.104:0/1557301435 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f66a800f070 con 0x7f66ac067460 2021-07-11T04:02:19.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.012+0000 7f669affd700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f668c053260 0x7f668c055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.012+0000 7f669affd700 1 -- 172.21.2.104:0/1557301435 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f66a806ca00 con 0x7f66ac067460 2021-07-11T04:02:19.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.012+0000 7f66b134f700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f668c053260 0x7f668c055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.012+0000 7f66b134f700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f668c053260 0x7f668c055720 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f669c00a000 tx=0x7f669c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.012+0000 7f669affd700 1 -- 172.21.2.104:0/1557301435 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f66a8043bd0 con 0x7f66ac067460 2021-07-11T04:02:19.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.092+0000 7f66b3db4700 1 -- 172.21.2.104:0/1557301435 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66880036c0 con 0x7f66ac067460 2021-07-11T04:02:19.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f669affd700 1 -- 172.21.2.104:0/1557301435 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f66a80471f0 con 0x7f66ac067460 2021-07-11T04:02:19.094 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:19.095 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":149,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:16.677257+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:02:14.500311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-07-11T04:02:15.641534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:19.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 -- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f668c053260 msgr2=0x7f668c055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f668c053260 0x7f668c055720 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f669c00a000 tx=0x7f669c008040).stop 2021-07-11T04:02:19.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 -- 172.21.2.104:0/1557301435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66ac067460 msgr2=0x7f66ac0a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66ac067460 0x7f66ac0a1130 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 rx=0x7f66ac062740 tx=0x7f66a8015000).stop 2021-07-11T04:02:19.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 -- 172.21.2.104:0/1557301435 shutdown_connections 2021-07-11T04:02:19.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66ac0b08f0 0x7f66ac156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66ac0b0330 0x7f66ac155e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f668c053260 0x7f668c055720 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 --2- 172.21.2.104:0/1557301435 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66ac067460 0x7f66ac0a1130 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 -- 172.21.2.104:0/1557301435 >> 172.21.2.104:0/1557301435 conn(0x7f66ac061070 msgr2=0x7f66ac0ab700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 -- 172.21.2.104:0/1557301435 shutdown_connections 2021-07-11T04:02:19.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.096+0000 7f66b3db4700 1 -- 172.21.2.104:0/1557301435 wait complete. 2021-07-11T04:02:19.102 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:02:14.500311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:02:15.641534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:02:19.103 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:02:19.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0adf91b700 1 Processor -- start 2021-07-11T04:02:19.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0adf91b700 1 -- start start 2021-07-11T04:02:19.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0adf91b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ad8144440 0x7f0ad8144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0adf91b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0ad8065420 0x7f0ad8144e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0adf91b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad8145350 con 0x7f0ad8060100 2021-07-11T04:02:19.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0adf91b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad8065c80 con 0x7f0ad8065420 2021-07-11T04:02:19.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0adf91b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad80604e0 con 0x7f0ad8144440 2021-07-11T04:02:19.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0adceb6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ad8144440 0x7f0ad8144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0addeb8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0ad8065420 0x7f0ad8144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0addeb8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0ad8065420 0x7f0ad8144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:58774/0 (socket says 172.21.2.104:58774) 2021-07-11T04:02:19.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0adceb6700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ad8144440 0x7f0ad8144970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:54376/0 (socket says 172.21.2.104:54376) 2021-07-11T04:02:19.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.156+0000 7f0addeb8700 1 -- 172.21.2.104:0/1630169889 learned_addr learned my addr 172.21.2.104:0/1630169889 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:19.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adceb6700 1 -- 172.21.2.104:0/1630169889 >> v2:172.21.2.105:3300/0 conn(0x7f0ad8065420 msgr2=0x7f0ad8144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:19.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adceb6700 1 --2- 172.21.2.104:0/1630169889 >> v2:172.21.2.105:3300/0 conn(0x7f0ad8065420 0x7f0ad8144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adceb6700 1 -- 172.21.2.104:0/1630169889 >> v1:172.21.2.105:6789/0 conn(0x7f0ad8060100 legacy=0x7f0ad8064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adceb6700 1 -- 172.21.2.104:0/1630169889 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad80b23b0 con 0x7f0ad8144440 2021-07-11T04:02:19.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0addeb8700 1 --2- 172.21.2.104:0/1630169889 >> v2:172.21.2.105:3300/0 conn(0x7f0ad8065420 0x7f0ad8144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:02:19.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adceb6700 1 --2- 172.21.2.104:0/1630169889 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ad8144440 0x7f0ad8144970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0acc002f70 tx=0x7f0acc006df0).ready entity=mon.2 client_cookie=8102a4e725282fa6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0acbfff700 1 -- 172.21.2.104:0/1630169889 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0acc00d570 con 0x7f0ad8144440 2021-07-11T04:02:19.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0acbfff700 1 -- 172.21.2.104:0/1630169889 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0acc000e90 con 0x7f0ad8144440 2021-07-11T04:02:19.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0acbfff700 1 -- 172.21.2.104:0/1630169889 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0acc00d8c0 con 0x7f0ad8144440 2021-07-11T04:02:19.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 -- 172.21.2.104:0/1630169889 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ad8144440 msgr2=0x7f0ad8144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 --2- 172.21.2.104:0/1630169889 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ad8144440 0x7f0ad8144970 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0acc002f70 tx=0x7f0acc006df0).stop 2021-07-11T04:02:19.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 -- 172.21.2.104:0/1630169889 shutdown_connections 2021-07-11T04:02:19.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 --2- 172.21.2.104:0/1630169889 >> v2:172.21.2.105:3300/0 conn(0x7f0ad8065420 0x7f0ad8144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 --2- 172.21.2.104:0/1630169889 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ad8144440 0x7f0ad8144970 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 -- 172.21.2.104:0/1630169889 >> 172.21.2.104:0/1630169889 conn(0x7f0ad805cf60 msgr2=0x7f0ad805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 -- 172.21.2.104:0/1630169889 shutdown_connections 2021-07-11T04:02:19.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 -- 172.21.2.104:0/1630169889 wait complete. 2021-07-11T04:02:19.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 Processor -- start 2021-07-11T04:02:19.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 -- start start 2021-07-11T04:02:19.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ad8060100 0x7f0ad813f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ad8065420 0x7f0ad80b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ad814e3c0 0x7f0ad814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad814dfd0 con 0x7f0ad8060100 2021-07-11T04:02:19.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad8140600 con 0x7f0ad814e3c0 2021-07-11T04:02:19.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0adf91b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad813c630 con 0x7f0ad8065420 2021-07-11T04:02:19.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0add6b7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ad8060100 0x7f0ad813f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0add6b7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ad8060100 0x7f0ad813f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51218/0 (socket says 172.21.2.104:51218) 2021-07-11T04:02:19.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0add6b7700 1 -- 172.21.2.104:0/976850010 learned_addr learned my addr 172.21.2.104:0/976850010 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:19.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0add6b7700 1 -- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ad8065420 msgr2=0x7f0ad80b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0add6b7700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ad8065420 0x7f0ad80b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0add6b7700 1 -- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ad814e3c0 msgr2=0x7f0ad814e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0add6b7700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ad814e3c0 0x7f0ad814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0add6b7700 1 -- 172.21.2.104:0/976850010 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad0009530 con 0x7f0ad8060100 2021-07-11T04:02:19.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.160+0000 7f0add6b7700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ad8060100 0x7f0ad813f6d0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f0ad8069be0 tx=0x7f0ad4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0aca7fc700 1 -- 172.21.2.104:0/976850010 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ad400ce30 con 0x7f0ad8060100 2021-07-11T04:02:19.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0aca7fc700 1 -- 172.21.2.104:0/976850010 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad400f470 con 0x7f0ad8060100 2021-07-11T04:02:19.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0aca7fc700 1 -- 172.21.2.104:0/976850010 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ad400f7d0 con 0x7f0ad8060100 2021-07-11T04:02:19.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0adf91b700 1 -- 172.21.2.104:0/976850010 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0acc00b180 con 0x7f0ad8060100 2021-07-11T04:02:19.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0adf91b700 1 -- 172.21.2.104:0/976850010 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad813cb70 con 0x7f0ad8060100 2021-07-11T04:02:19.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0adf91b700 1 -- 172.21.2.104:0/976850010 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad8057350 con 0x7f0ad8060100 2021-07-11T04:02:19.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0aca7fc700 1 -- 172.21.2.104:0/976850010 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0ad4019030 con 0x7f0ad8060100 2021-07-11T04:02:19.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0aca7fc700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ab80532d0 0x7f0ab8055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0aca7fc700 1 -- 172.21.2.104:0/976850010 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0ad406aaa0 con 0x7f0ad8060100 2021-07-11T04:02:19.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0adceb6700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ab80532d0 0x7f0ab8055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0adceb6700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ab80532d0 0x7f0ab8055790 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f0acc002f70 tx=0x7f0acc014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.164+0000 7f0aca7fc700 1 -- 172.21.2.104:0/976850010 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0ad4041c70 con 0x7f0ad8060100 2021-07-11T04:02:19.248 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.248+0000 7f0adf91b700 1 -- 172.21.2.104:0/976850010 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0ad814e120 con 0x7f0ad8060100 2021-07-11T04:02:19.249 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.248+0000 7f0aca7fc700 1 -- 172.21.2.104:0/976850010 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v149) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0ad4045290 con 0x7f0ad8060100 2021-07-11T04:02:19.249 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:02:19.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.248+0000 7f0adf91b700 1 -- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ab80532d0 msgr2=0x7f0ab8055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.248+0000 7f0adf91b700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ab80532d0 0x7f0ab8055790 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f0acc002f70 tx=0x7f0acc014000).stop 2021-07-11T04:02:19.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.248+0000 7f0adf91b700 1 -- 172.21.2.104:0/976850010 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ad8060100 msgr2=0x7f0ad813f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.248+0000 7f0adf91b700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ad8060100 0x7f0ad813f6d0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 rx=0x7f0ad8069be0 tx=0x7f0ad4007c50).stop 2021-07-11T04:02:19.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.252+0000 7f0adf91b700 1 -- 172.21.2.104:0/976850010 shutdown_connections 2021-07-11T04:02:19.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.252+0000 7f0adf91b700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ad814e3c0 0x7f0ad814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.252+0000 7f0adf91b700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ad8065420 0x7f0ad80b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.252+0000 7f0adf91b700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0ab80532d0 0x7f0ab8055790 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.252+0000 7f0adf91b700 1 --2- 172.21.2.104:0/976850010 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ad8060100 0x7f0ad813f6d0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.252+0000 7f0adf91b700 1 -- 172.21.2.104:0/976850010 >> 172.21.2.104:0/976850010 conn(0x7f0ad805cf60 msgr2=0x7f0ad80b6700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.252+0000 7f0adf91b700 1 -- 172.21.2.104:0/976850010 shutdown_connections 2021-07-11T04:02:19.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.252+0000 7f0adf91b700 1 -- 172.21.2.104:0/976850010 wait complete. 2021-07-11T04:02:19.256 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:02:19.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.308+0000 7fa9ad6e3700 1 Processor -- start 2021-07-11T04:02:19.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.308+0000 7fa9ad6e3700 1 -- start start 2021-07-11T04:02:19.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.308+0000 7fa9ad6e3700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa9a80603d0 0x7fa9a80aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.308+0000 7fa9ad6e3700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a8061990 con 0x7fa9a80a9ec0 2021-07-11T04:02:19.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.308+0000 7fa9ad6e3700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a8061c30 con 0x7fa9a813f790 2021-07-11T04:02:19.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.308+0000 7fa9ad6e3700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a8060b90 con 0x7fa9a80603d0 2021-07-11T04:02:19.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.308+0000 7fa9a6ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fa9a80a9ec0 0x7fa9a813ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:47992/0 (socket says 172.21.2.104:47992) 2021-07-11T04:02:19.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.308+0000 7fa9a6ffd700 1 -- 172.21.2.104:0/2718766086 learned_addr learned my addr 172.21.2.104:0/2718766086 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:19.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257098756 0 0) 0x7fa9a8061990 con 0x7fa9a80a9ec0 2021-07-11T04:02:19.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa988004370 con 0x7fa9a80a9ec0 2021-07-11T04:02:19.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2170338411 0 0) 0x7fa988004370 con 0x7fa9a80a9ec0 2021-07-11T04:02:19.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 >> v2:172.21.2.105:3300/0 conn(0x7fa9a80603d0 msgr2=0x7fa9a80aa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:19.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 --2- 172.21.2.104:0/2718766086 >> v2:172.21.2.105:3300/0 conn(0x7fa9a80603d0 0x7fa9a80aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 >> v1:172.21.2.105:6789/0 conn(0x7fa9a813f790 legacy=0x7fa9a814a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:19.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8060e30 con 0x7fa9a80a9ec0 2021-07-11T04:02:19.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fa998001660 con 0x7fa9a80a9ec0 2021-07-11T04:02:19.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 >> v1:172.21.2.104:6789/0 conn(0x7fa9a80a9ec0 legacy=0x7fa9a813ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 --2- 172.21.2.104:0/2718766086 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 0x7fa988004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 --2- 172.21.2.104:0/2718766086 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa988005610 0x7fa988005a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 --2- 172.21.2.104:0/2718766086 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa988006180 0x7fa98800c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a8060b90 con 0x7fa9880048e0 2021-07-11T04:02:19.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98800d000 con 0x7fa988006180 2021-07-11T04:02:19.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98800d1c0 con 0x7fa988005610 2021-07-11T04:02:19.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 --2- 172.21.2.104:0/2718766086 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 0x7fa988004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a77fe700 1 --2- 172.21.2.104:0/2718766086 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa988006180 0x7fa98800c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 -- 172.21.2.104:0/2718766086 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa988005610 msgr2=0x7fa988005a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:19.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 --2- 172.21.2.104:0/2718766086 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa988005610 0x7fa988005a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 -- 172.21.2.104:0/2718766086 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa988006180 msgr2=0x7fa98800c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:19.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 --2- 172.21.2.104:0/2718766086 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa988006180 0x7fa98800c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 -- 172.21.2.104:0/2718766086 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8060e30 con 0x7fa9880048e0 2021-07-11T04:02:19.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a77fe700 1 --2- 172.21.2.104:0/2718766086 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa988006180 0x7fa98800c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:19.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a77fe700 1 -- 172.21.2.104:0/2718766086 reap_dead start 2021-07-11T04:02:19.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 --2- 172.21.2.104:0/2718766086 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 0x7fa988004ce0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7fa998001020 tx=0x7fa998001e20).ready entity=mon.0 client_cookie=aca1fad94a324bfe server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa9980070d0 con 0x7fa9880048e0 2021-07-11T04:02:19.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa998007230 con 0x7fa9880048e0 2021-07-11T04:02:19.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a5ffb700 1 -- 172.21.2.104:0/2718766086 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa99800c9c0 con 0x7fa9880048e0 2021-07-11T04:02:19.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/2718766086 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 msgr2=0x7fa988004ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 --2- 172.21.2.104:0/2718766086 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 0x7fa988004ce0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7fa998001020 tx=0x7fa998001e20).stop 2021-07-11T04:02:19.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/2718766086 shutdown_connections 2021-07-11T04:02:19.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 --2- 172.21.2.104:0/2718766086 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 0x7fa988004ce0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/2718766086 >> 172.21.2.104:0/2718766086 conn(0x7fa9a80b2970 msgr2=0x7fa9a80b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/2718766086 shutdown_connections 2021-07-11T04:02:19.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/2718766086 wait complete. 2021-07-11T04:02:19.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 Processor -- start 2021-07-11T04:02:19.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 -- start start 2021-07-11T04:02:19.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 0x7fa9a80b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9a813c090 0x7fa9a813c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9a8151660 0x7fa9a8151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80646b0 con 0x7fa9880048e0 2021-07-11T04:02:19.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80afbd0 con 0x7fa9a8151660 2021-07-11T04:02:19.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9ad6e3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a80648e0 con 0x7fa9a813c090 2021-07-11T04:02:19.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 0x7fa9a80b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 0x7fa9a80b4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51236/0 (socket says 172.21.2.104:51236) 2021-07-11T04:02:19.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 -- 172.21.2.104:0/253711134 learned_addr learned my addr 172.21.2.104:0/253711134 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:19.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 -- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9a813c090 msgr2=0x7fa9a813c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9a813c090 0x7fa9a813c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 -- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9a8151660 msgr2=0x7fa9a8151b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9a8151660 0x7fa9a8151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 -- 172.21.2.104:0/253711134 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8066910 con 0x7fa9880048e0 2021-07-11T04:02:19.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.312+0000 7fa9a6ffd700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 0x7fa9a80b4d30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7fa998001020 tx=0x7fa998017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.316+0000 7fa99ffff700 1 -- 172.21.2.104:0/253711134 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa99800ce20 con 0x7fa9880048e0 2021-07-11T04:02:19.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.316+0000 7fa99ffff700 1 -- 172.21.2.104:0/253711134 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa998004520 con 0x7fa9880048e0 2021-07-11T04:02:19.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.316+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/253711134 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a80afe70 con 0x7fa9880048e0 2021-07-11T04:02:19.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.316+0000 7fa99ffff700 1 -- 172.21.2.104:0/253711134 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa998004880 con 0x7fa9880048e0 2021-07-11T04:02:19.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.316+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/253711134 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a80b0410 con 0x7fa9880048e0 2021-07-11T04:02:19.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.316+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/253711134 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a8066d80 con 0x7fa9880048e0 2021-07-11T04:02:19.332 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.316+0000 7fa99ffff700 1 -- 172.21.2.104:0/253711134 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa9980049e0 con 0x7fa9880048e0 2021-07-11T04:02:19.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.316+0000 7fa99ffff700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa98c0531a0 0x7fa98c055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.316+0000 7fa99ffff700 1 -- 172.21.2.104:0/253711134 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa99806e980 con 0x7fa9880048e0 2021-07-11T04:02:19.333 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.316+0000 7fa9a67fc700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa98c0531a0 0x7fa98c055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.316+0000 7fa9a67fc700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa98c0531a0 0x7fa98c055660 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fa990002f70 tx=0x7fa990006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.334 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.320+0000 7fa99ffff700 1 -- 172.21.2.104:0/253711134 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa998045bb0 con 0x7fa9880048e0 2021-07-11T04:02:19.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/253711134 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa9a8057350 con 0x7fa9880048e0 2021-07-11T04:02:19.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa99ffff700 1 -- 172.21.2.104:0/253711134 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v149) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa9980491d0 con 0x7fa9880048e0 2021-07-11T04:02:19.400 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:19.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa98c0531a0 msgr2=0x7fa98c055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa98c0531a0 0x7fa98c055660 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7fa990002f70 tx=0x7fa990006a80).stop 2021-07-11T04:02:19.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/253711134 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 msgr2=0x7fa9a80b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 0x7fa9a80b4d30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7fa998001020 tx=0x7fa998017000).stop 2021-07-11T04:02:19.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/253711134 shutdown_connections 2021-07-11T04:02:19.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa9a8151660 0x7fa9a8151b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa9a813c090 0x7fa9a813c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa98c0531a0 0x7fa98c055660 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 --2- 172.21.2.104:0/253711134 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa9880048e0 0x7fa9a80b4d30 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/253711134 >> 172.21.2.104:0/253711134 conn(0x7fa9a80b2970 msgr2=0x7fa9a813d170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/253711134 shutdown_connections 2021-07-11T04:02:19.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.400+0000 7fa9ad6e3700 1 -- 172.21.2.104:0/253711134 wait complete. 2021-07-11T04:02:19.407 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:02:19.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae3419700 1 Processor -- start 2021-07-11T04:02:19.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae3419700 1 -- start start 2021-07-11T04:02:19.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae3419700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9adc0ad040 0x7f9adc0b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae3419700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9adc05d760 0x7f9adc13f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae3419700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9adc0b5f40 con 0x7f9adc0afaf0 2021-07-11T04:02:19.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae3419700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc0ad420 con 0x7f9adc05d760 2021-07-11T04:02:19.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae3419700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9adc0aeb60 con 0x7f9adc0ad040 2021-07-11T04:02:19.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae09b4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9adc05d760 0x7f9adc13f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae09b4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9adc05d760 0x7f9adc13f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51246/0 (socket says 172.21.2.104:51246) 2021-07-11T04:02:19.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae09b4700 1 -- 172.21.2.104:0/1918159486 learned_addr learned my addr 172.21.2.104:0/1918159486 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:19.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae09b4700 1 -- 172.21.2.104:0/1918159486 >> v2:172.21.2.105:3300/0 conn(0x7f9adc0ad040 msgr2=0x7f9adc0b5a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:19.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae09b4700 1 --2- 172.21.2.104:0/1918159486 >> v2:172.21.2.105:3300/0 conn(0x7f9adc0ad040 0x7f9adc0b5a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae09b4700 1 -- 172.21.2.104:0/1918159486 >> v1:172.21.2.104:6789/0 conn(0x7f9adc0afaf0 legacy=0x7f9adc13f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:19.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae09b4700 1 -- 172.21.2.104:0/1918159486 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9adc0aee00 con 0x7f9adc05d760 2021-07-11T04:02:19.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae09b4700 1 --2- 172.21.2.104:0/1918159486 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9adc05d760 0x7f9adc13f070 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f9adc062900 tx=0x7f9ad0006eb0).ready entity=mon.0 client_cookie=86059d571d25552a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9acffff700 1 -- 172.21.2.104:0/1918159486 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ad000d5c0 con 0x7f9adc05d760 2021-07-11T04:02:19.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9acffff700 1 -- 172.21.2.104:0/1918159486 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0000e90 con 0x7f9adc05d760 2021-07-11T04:02:19.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9acffff700 1 -- 172.21.2.104:0/1918159486 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ad000d910 con 0x7f9adc05d760 2021-07-11T04:02:19.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae3419700 1 -- 172.21.2.104:0/1918159486 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9adc05d760 msgr2=0x7f9adc13f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.460+0000 7f9ae3419700 1 --2- 172.21.2.104:0/1918159486 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9adc05d760 0x7f9adc13f070 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f9adc062900 tx=0x7f9ad0006eb0).stop 2021-07-11T04:02:19.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 -- 172.21.2.104:0/1918159486 shutdown_connections 2021-07-11T04:02:19.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 --2- 172.21.2.104:0/1918159486 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9adc05d760 0x7f9adc13f070 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 --2- 172.21.2.104:0/1918159486 >> v2:172.21.2.105:3300/0 conn(0x7f9adc0ad040 0x7f9adc0b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 -- 172.21.2.104:0/1918159486 >> 172.21.2.104:0/1918159486 conn(0x7f9adc05c490 msgr2=0x7f9adc05c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 -- 172.21.2.104:0/1918159486 shutdown_connections 2021-07-11T04:02:19.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 -- 172.21.2.104:0/1918159486 wait complete. 2021-07-11T04:02:19.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 Processor -- start 2021-07-11T04:02:19.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 -- start start 2021-07-11T04:02:19.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9adc05d760 0x7f9adc13d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9adc0ad040 0x7f9adc13d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9adc0afaf0 0x7f9adc156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1403c0 con 0x7f9adc0ad040 2021-07-11T04:02:19.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc1405f0 con 0x7f9adc05d760 2021-07-11T04:02:19.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc140890 con 0x7f9adc0afaf0 2021-07-11T04:02:19.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae09b4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9adc0ad040 0x7f9adc13d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae09b4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9adc0ad040 0x7f9adc13d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51252/0 (socket says 172.21.2.104:51252) 2021-07-11T04:02:19.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae09b4700 1 -- 172.21.2.104:0/1995610777 learned_addr learned my addr 172.21.2.104:0/1995610777 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:19.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae09b4700 1 -- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9adc0afaf0 msgr2=0x7f9adc156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae09b4700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9adc0afaf0 0x7f9adc156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae09b4700 1 -- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9adc05d760 msgr2=0x7f9adc13d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae09b4700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9adc05d760 0x7f9adc13d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae09b4700 1 -- 172.21.2.104:0/1995610777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad000b1e0 con 0x7f9adc0ad040 2021-07-11T04:02:19.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae09b4700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9adc0ad040 0x7f9adc13d8c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7f9adc062900 tx=0x7f9ad0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ace7fc700 1 -- 172.21.2.104:0/1995610777 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ad0021070 con 0x7f9adc0ad040 2021-07-11T04:02:19.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ace7fc700 1 -- 172.21.2.104:0/1995610777 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad0000e90 con 0x7f9adc0ad040 2021-07-11T04:02:19.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ace7fc700 1 -- 172.21.2.104:0/1995610777 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ad0006be0 con 0x7f9adc0ad040 2021-07-11T04:02:19.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 -- 172.21.2.104:0/1995610777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc142160 con 0x7f9adc0ad040 2021-07-11T04:02:19.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 -- 172.21.2.104:0/1995610777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc156570 con 0x7f9adc0ad040 2021-07-11T04:02:19.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae3419700 1 -- 172.21.2.104:0/1995610777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc057350 con 0x7f9adc0ad040 2021-07-11T04:02:19.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ace7fc700 1 -- 172.21.2.104:0/1995610777 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9ad000f070 con 0x7f9adc0ad040 2021-07-11T04:02:19.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ace7fc700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9abc053380 0x7f9abc055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ace7fc700 1 -- 172.21.2.104:0/1995610777 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9ad006c920 con 0x7f9adc0ad040 2021-07-11T04:02:19.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.464+0000 7f9ae11b5700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9abc053380 0x7f9abc055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.468+0000 7f9ae11b5700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9abc053380 0x7f9abc055840 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f9ad8002fb0 tx=0x7f9ad80069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.468+0000 7f9ace7fc700 1 -- 172.21.2.104:0/1995610777 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9ad0043b50 con 0x7f9adc0ad040 2021-07-11T04:02:19.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.548+0000 7f9ae3419700 1 -- 172.21.2.104:0/1995610777 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9adc139b40 con 0x7f9adc0ad040 2021-07-11T04:02:19.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ace7fc700 1 -- 172.21.2.104:0/1995610777 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v149) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9ad0047170 con 0x7f9adc0ad040 2021-07-11T04:02:19.551 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:19.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 -- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9abc053380 msgr2=0x7f9abc055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9abc053380 0x7f9abc055840 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f9ad8002fb0 tx=0x7f9ad80069d0).stop 2021-07-11T04:02:19.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 -- 172.21.2.104:0/1995610777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9adc0ad040 msgr2=0x7f9adc13d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9adc0ad040 0x7f9adc13d8c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7f9adc062900 tx=0x7f9ad0015000).stop 2021-07-11T04:02:19.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 -- 172.21.2.104:0/1995610777 shutdown_connections 2021-07-11T04:02:19.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9adc0afaf0 0x7f9adc156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9abc053380 0x7f9abc055840 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9adc0ad040 0x7f9adc13d8c0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 --2- 172.21.2.104:0/1995610777 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9adc05d760 0x7f9adc13d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 -- 172.21.2.104:0/1995610777 >> 172.21.2.104:0/1995610777 conn(0x7f9adc05c490 msgr2=0x7f9adc13c080 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 -- 172.21.2.104:0/1995610777 shutdown_connections 2021-07-11T04:02:19.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.552+0000 7f9ae3419700 1 -- 172.21.2.104:0/1995610777 wait complete. 2021-07-11T04:02:19.559 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:02:19.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.612+0000 7f6ff43b8700 1 Processor -- start 2021-07-11T04:02:19.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.612+0000 7f6ff43b8700 1 -- start start 2021-07-11T04:02:19.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.612+0000 7f6ff43b8700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6fec0b12d0 0x7f6fec0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.612+0000 7f6ff43b8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fec0b22a0 0x7f6fec141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.612+0000 7f6ff43b8700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec0a77d0 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.612+0000 7f6ff43b8700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fec0b1740 con 0x7f6fec0b12d0 2021-07-11T04:02:19.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.612+0000 7f6ff43b8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0b0650 con 0x7f6fec0b22a0 2021-07-11T04:02:19.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.612+0000 7f6ff1953700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6fec0b12d0 0x7f6fec0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.612+0000 7f6ff2955700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fec0b22a0 0x7f6fec141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.612+0000 7f6ff1953700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6fec0b12d0 0x7f6fec0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:58818/0 (socket says 172.21.2.104:58818) 2021-07-11T04:02:19.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.612+0000 7f6ff1953700 1 -- 172.21.2.104:0/3452599196 learned_addr learned my addr 172.21.2.104:0/3452599196 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:19.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 -- 172.21.2.104:0/3452599196 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fec0b22a0 msgr2=0x7f6fec141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:19.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 --2- 172.21.2.104:0/3452599196 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fec0b22a0 0x7f6fec141250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 -- 172.21.2.104:0/3452599196 >> v1:172.21.2.105:6789/0 conn(0x7f6fec0b1ec0 legacy=0x7f6fec0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 -- 172.21.2.104:0/3452599196 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec0b08f0 con 0x7f6fec0b12d0 2021-07-11T04:02:19.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff2955700 1 --2- 172.21.2.104:0/3452599196 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fec0b22a0 0x7f6fec141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:02:19.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 --2- 172.21.2.104:0/3452599196 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fec0b12d0 0x7f6fec0affb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f6fe0002f70 tx=0x7f6fe0006eb0).ready entity=mon.1 client_cookie=61cbc32f89c966ca server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1152700 1 -- 172.21.2.104:0/3452599196 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe000d570 con 0x7f6fec0b12d0 2021-07-11T04:02:19.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1152700 1 -- 172.21.2.104:0/3452599196 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0000e90 con 0x7f6fec0b12d0 2021-07-11T04:02:19.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3452599196 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fec0b12d0 msgr2=0x7f6fec0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1152700 1 -- 172.21.2.104:0/3452599196 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe000d900 con 0x7f6fec0b12d0 2021-07-11T04:02:19.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 --2- 172.21.2.104:0/3452599196 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fec0b12d0 0x7f6fec0affb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7f6fe0002f70 tx=0x7f6fe0006eb0).stop 2021-07-11T04:02:19.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3452599196 shutdown_connections 2021-07-11T04:02:19.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 --2- 172.21.2.104:0/3452599196 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fec0b22a0 0x7f6fec141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 --2- 172.21.2.104:0/3452599196 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fec0b12d0 0x7f6fec0affb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3452599196 >> 172.21.2.104:0/3452599196 conn(0x7f6fec05f6b0 msgr2=0x7f6fec05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3452599196 shutdown_connections 2021-07-11T04:02:19.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3452599196 wait complete. 2021-07-11T04:02:19.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 Processor -- start 2021-07-11T04:02:19.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 -- start start 2021-07-11T04:02:19.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fec0b12d0 0x7f6fec14a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fec0b1ec0 0x7f6fec14ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fec0b22a0 0x7f6fec0b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec149080 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec149320 con 0x7f6fec0b22a0 2021-07-11T04:02:19.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff43b8700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec1495c0 con 0x7f6fec0b12d0 2021-07-11T04:02:19.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fec0b1ec0 0x7f6fec14ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fec0b1ec0 0x7f6fec14ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51268/0 (socket says 172.21.2.104:51268) 2021-07-11T04:02:19.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 -- 172.21.2.104:0/3897738476 learned_addr learned my addr 172.21.2.104:0/3897738476 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:19.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff2955700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fec0b22a0 0x7f6fec0b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff2154700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fec0b12d0 0x7f6fec14a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 -- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fec0b12d0 msgr2=0x7f6fec14a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.628 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fec0b12d0 0x7f6fec14a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 -- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fec0b22a0 msgr2=0x7f6fec0b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fec0b22a0 0x7f6fec0b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 -- 172.21.2.104:0/3897738476 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe4009470 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6ff1953700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fec0b1ec0 0x7f6fec14ade0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7f6fe0002f70 tx=0x7f6fe0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.616+0000 7f6fdf7fe700 1 -- 172.21.2.104:0/3897738476 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe0011860 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.620+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3897738476 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe000b1e0 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.620+0000 7f6fdf7fe700 1 -- 172.21.2.104:0/3897738476 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe00119c0 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.620+0000 7f6fdf7fe700 1 -- 172.21.2.104:0/3897738476 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6fe0011d20 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.620+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3897738476 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec14a590 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.620+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3897738476 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec057350 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.620+0000 7f6fdf7fe700 1 -- 172.21.2.104:0/3897738476 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6fe000f070 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.620+0000 7f6fdf7fe700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fcc0531f0 0x7f6fcc0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.620+0000 7f6fdf7fe700 1 -- 172.21.2.104:0/3897738476 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6fe006ce00 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.620+0000 7f6ff2154700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fcc0531f0 0x7f6fcc0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.620+0000 7f6ff2154700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fcc0531f0 0x7f6fcc0556b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f6fe800b000 tx=0x7f6fe8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.620+0000 7f6fdf7fe700 1 -- 172.21.2.104:0/3897738476 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6fe00440a0 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3897738476 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6fec0ac760 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6fdf7fe700 1 -- 172.21.2.104:0/3897738476 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v149) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6fe00476c0 con 0x7f6fec0b1ec0 2021-07-11T04:02:19.704 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:19.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fcc0531f0 msgr2=0x7f6fcc0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fcc0531f0 0x7f6fcc0556b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f6fe800b000 tx=0x7f6fe8009040).stop 2021-07-11T04:02:19.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3897738476 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fec0b1ec0 msgr2=0x7f6fec14ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fec0b1ec0 0x7f6fec14ade0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7f6fe0002f70 tx=0x7f6fe0015000).stop 2021-07-11T04:02:19.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3897738476 shutdown_connections 2021-07-11T04:02:19.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6fec0b22a0 0x7f6fec0b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6fcc0531f0 0x7f6fcc0556b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6fec0b1ec0 0x7f6fec14ade0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 --2- 172.21.2.104:0/3897738476 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6fec0b12d0 0x7f6fec14a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3897738476 >> 172.21.2.104:0/3897738476 conn(0x7f6fec05f6b0 msgr2=0x7f6fec0aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3897738476 shutdown_connections 2021-07-11T04:02:19.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.704+0000 7f6ff43b8700 1 -- 172.21.2.104:0/3897738476 wait complete. 2021-07-11T04:02:19.711 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:02:19.712 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:19.763 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 Processor -- start 2021-07-11T04:02:19.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 -- start start 2021-07-11T04:02:19.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d580b7810 0x7f2d581445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.764 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2d58144a90 0x7f2d58064f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d580b31d0 con 0x7f2d580b1170 2021-07-11T04:02:19.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d58144ef0 con 0x7f2d58144a90 2021-07-11T04:02:19.765 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d580b7bf0 con 0x7f2d580b7810 2021-07-11T04:02:19.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d56d9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2d58144a90 0x7f2d58064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d55d9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f2d580b1170 0x7f2d58064870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48044/0 (socket says 172.21.2.104:48044) 2021-07-11T04:02:19.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d55d9b700 1 -- 172.21.2.104:0/3044705941 learned_addr learned my addr 172.21.2.104:0/3044705941 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:19.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d56d9d700 1 -- 172.21.2.104:0/3044705941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d580b7810 msgr2=0x7f2d581445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:19.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d56d9d700 1 --2- 172.21.2.104:0/3044705941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d580b7810 0x7f2d581445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d56d9d700 1 -- 172.21.2.104:0/3044705941 >> v1:172.21.2.104:6789/0 conn(0x7f2d580b1170 legacy=0x7f2d58064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d56d9d700 1 -- 172.21.2.104:0/3044705941 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d580675c0 con 0x7f2d58144a90 2021-07-11T04:02:19.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d56d9d700 1 --2- 172.21.2.104:0/3044705941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d58144a90 0x7f2d58064f80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f2d500032f0 tx=0x7f2d500091d0).ready entity=mon.0 client_cookie=86c39027874e37b2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5559a700 1 -- 172.21.2.104:0/3044705941 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d50009ea0 con 0x7f2d58144a90 2021-07-11T04:02:19.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5559a700 1 -- 172.21.2.104:0/3044705941 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5000b8a0 con 0x7f2d58144a90 2021-07-11T04:02:19.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5559a700 1 -- 172.21.2.104:0/3044705941 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d5000ba10 con 0x7f2d58144a90 2021-07-11T04:02:19.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 -- 172.21.2.104:0/3044705941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d58144a90 msgr2=0x7f2d58064f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 --2- 172.21.2.104:0/3044705941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d58144a90 0x7f2d58064f80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 rx=0x7f2d500032f0 tx=0x7f2d500091d0).stop 2021-07-11T04:02:19.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 -- 172.21.2.104:0/3044705941 shutdown_connections 2021-07-11T04:02:19.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 --2- 172.21.2.104:0/3044705941 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d58144a90 0x7f2d58064f80 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 --2- 172.21.2.104:0/3044705941 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d580b7810 0x7f2d581445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 -- 172.21.2.104:0/3044705941 >> 172.21.2.104:0/3044705941 conn(0x7f2d5805f6b0 msgr2=0x7f2d5805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 -- 172.21.2.104:0/3044705941 shutdown_connections 2021-07-11T04:02:19.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 -- 172.21.2.104:0/3044705941 wait complete. 2021-07-11T04:02:19.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.764+0000 7f2d5c891700 1 Processor -- start 2021-07-11T04:02:19.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d5c891700 1 -- start start 2021-07-11T04:02:19.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d5c891700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d580b1170 0x7f2d581439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d5c891700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d580b7810 0x7f2d5813dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d5c891700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d58144a90 0x7f2d5813e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d5c891700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d58061f30 con 0x7f2d580b7810 2021-07-11T04:02:19.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d5c891700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d580aa600 con 0x7f2d58144a90 2021-07-11T04:02:19.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d5c891700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d58062160 con 0x7f2d580b1170 2021-07-11T04:02:19.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d55d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d580b7810 0x7f2d5813dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d55d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d580b7810 0x7f2d5813dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51282/0 (socket says 172.21.2.104:51282) 2021-07-11T04:02:19.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d55d9b700 1 -- 172.21.2.104:0/2899507474 learned_addr learned my addr 172.21.2.104:0/2899507474 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:19.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d55d9b700 1 -- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d580b1170 msgr2=0x7f2d581439b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d55d9b700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d580b1170 0x7f2d581439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d55d9b700 1 -- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d58144a90 msgr2=0x7f2d5813e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d55d9b700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d58144a90 0x7f2d5813e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d55d9b700 1 -- 172.21.2.104:0/2899507474 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d50009470 con 0x7f2d580b7810 2021-07-11T04:02:19.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d55d9b700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d580b7810 0x7f2d5813dbd0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f2d58060740 tx=0x7f2d40007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d4f7fe700 1 -- 172.21.2.104:0/2899507474 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d4000ce30 con 0x7f2d580b7810 2021-07-11T04:02:19.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d4f7fe700 1 -- 172.21.2.104:0/2899507474 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d400072e0 con 0x7f2d580b7810 2021-07-11T04:02:19.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d4f7fe700 1 -- 172.21.2.104:0/2899507474 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2d40005800 con 0x7f2d580b7810 2021-07-11T04:02:19.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d5c891700 1 -- 172.21.2.104:0/2899507474 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d58062400 con 0x7f2d580b7810 2021-07-11T04:02:19.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d5c891700 1 -- 172.21.2.104:0/2899507474 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d58061620 con 0x7f2d580b7810 2021-07-11T04:02:19.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d5c891700 1 -- 172.21.2.104:0/2899507474 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d58062590 con 0x7f2d580b7810 2021-07-11T04:02:19.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d4f7fe700 1 -- 172.21.2.104:0/2899507474 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2d40010070 con 0x7f2d580b7810 2021-07-11T04:02:19.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d4f7fe700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d3c053180 0x7f2d3c055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:19.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d4f7fe700 1 -- 172.21.2.104:0/2899507474 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2d4006abc0 con 0x7f2d580b7810 2021-07-11T04:02:19.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.768+0000 7f2d5659c700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d3c053180 0x7f2d3c055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:19.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.772+0000 7f2d5659c700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d3c053180 0x7f2d3c055640 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f2d48002f70 tx=0x7f2d48006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:19.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.772+0000 7f2d4f7fe700 1 -- 172.21.2.104:0/2899507474 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2d40041df0 con 0x7f2d580b7810 2021-07-11T04:02:19.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 -- 172.21.2.104:0/2899507474 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d58057350 con 0x7f2d580b7810 2021-07-11T04:02:19.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d4f7fe700 1 -- 172.21.2.104:0/2899507474 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f2d40018030 con 0x7f2d580b7810 2021-07-11T04:02:19.880 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:19.881 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/696025904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":696025904},{"type":"v1","addr":"172.21.2.104:6833","nonce":696025904}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8412,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/438569403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":438569403},{"type":"v1","addr":"172.21.2.104:6835","nonce":438569403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:16.677336+0000","modified":"2021-07-11T04:02:17.681052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8415},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8415":{"gid":8415,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.2.105:6835/130583442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":130583442},{"type":"v1","addr":"172.21.2.105:6835","nonce":130583442}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-07-11T04:02:19.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 -- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d3c053180 msgr2=0x7f2d3c055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d3c053180 0x7f2d3c055640 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f2d48002f70 tx=0x7f2d48006a80).stop 2021-07-11T04:02:19.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 -- 172.21.2.104:0/2899507474 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d580b7810 msgr2=0x7f2d5813dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:19.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d580b7810 0x7f2d5813dbd0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f2d58060740 tx=0x7f2d40007c40).stop 2021-07-11T04:02:19.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 -- 172.21.2.104:0/2899507474 shutdown_connections 2021-07-11T04:02:19.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2d58144a90 0x7f2d5813e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2d3c053180 0x7f2d3c055640 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2d580b7810 0x7f2d5813dbd0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:19.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 --2- 172.21.2.104:0/2899507474 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2d580b1170 0x7f2d581439b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:19.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 -- 172.21.2.104:0/2899507474 >> 172.21.2.104:0/2899507474 conn(0x7f2d5805f6b0 msgr2=0x7f2d580b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:19.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 -- 172.21.2.104:0/2899507474 shutdown_connections 2021-07-11T04:02:19.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:19.880+0000 7f2d5c891700 1 -- 172.21.2.104:0/2899507474 wait complete. 2021-07-11T04:02:19.886 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 74 2021-07-11T04:02:20.190 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:02:16.677336+0000', 'modified': '2021-07-11T04:02:17.681052+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8415}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8415': {'gid': 8415, 'name': 'b', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.105:6835/130583442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.105:6834', 'nonce': 130583442}, {'type': 'v1', 'addr': '172.21.2.105:6835', 'nonce': 130583442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:02:20.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:02:20.191 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2021-07-11T04:02:20.254 INFO:teuthology.orchestra.run.gibba005.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":8415,"want_state":"up:active","state":"up:active","rank_uptime":3.5706023070000001,"mdsmap_epoch":74,"osdmap_epoch":149,"osdmap_epoch_barrier":149,"uptime":7.7595411399999996} 2021-07-11T04:02:20.254 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 8415, "mdsmap_epoch": 74, "osdmap_epoch": 149, "osdmap_epoch_barrier": 149, "rank_uptime": 3.570602307, "state": "up:active", "uptime": 7.75954114, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:02:20.255 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:02:20.255 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T04:02:20.256 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T04:02:20.256 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:20.256 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:20.262 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:02:20.315 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:20.316 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:20.322 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:20.322 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:20.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:20.324 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:02:20.328 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:02:20.329 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:20.374 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:02:20.384 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:20.385 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:20.391 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:20.392 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:20.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:20.394 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:02:20.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:2021-07-11T04:02:20.410+0000 7f60ce0e8080 -1 init, newargv = 0x558b72b91a10 newargc=9 2021-07-11T04:02:20.413 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15808]: starting ceph client 2021-07-11T04:02:20.431 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15808]: starting fuse 2021-07-11T04:02:21.517 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:02:21.529 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:21.530 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:21.579 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:21.580 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:21.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:21.581 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:02:21.586 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T04:02:21.587 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:02:21.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:21.588 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T04:02:21.588 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T04:02:21.588 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T04:02:21.589 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T04:02:21.589 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T04:02:21.590 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:02:21.590 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T04:02:21.590 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T04:02:21.591 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T04:02:21.591 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:02:21.592 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T04:02:21.592 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T04:02:21.592 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T04:02:21.593 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T04:02:21.593 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:02:21.594 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T04:02:21.594 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:02:21.594 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:02:21.595 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T04:02:21.595 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:02:21.596 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:02:21.596 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T04:02:21.596 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T04:02:21.625 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15808.asok 2021-07-11T04:02:21.627 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15808.asok 2021-07-11T04:02:21.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:21.628 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15808.asok status 2021-07-11T04:02:21.735 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T04:02:21.736 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T04:02:21.736 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:02:21.737 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:02:21.737 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T04:02:21.737 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T04:02:21.737 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T04:02:21.737 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15808", 2021-07-11T04:02:21.738 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T04:02:21.738 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:02:21.738 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T04:02:21.738 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T04:02:21.745 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 8571, 2021-07-11T04:02:21.746 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T04:02:21.746 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T04:02:21.746 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T04:02:21.746 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 8571 2021-07-11T04:02:21.747 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:02:21.747 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:02:21.747 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:02:21.748 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:02:21.748 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3671950995 2021-07-11T04:02:21.748 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T04:02:21.748 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:02:21.749 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:02:21.749 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:02:21.749 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:02:21.750 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 3671950995 2021-07-11T04:02:21.750 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:02:21.750 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.8571 172.21.2.119:0/3671950995", 2021-07-11T04:02:21.750 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/3671950995", 2021-07-11T04:02:21.751 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T04:02:21.751 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 74, 2021-07-11T04:02:21.751 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 149, 2021-07-11T04:02:21.751 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:02:21.751 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T04:02:21.752 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T04:02:21.752 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T04:02:21.753 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:21.754 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:21.754 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:21.760 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:02:21.761 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:02:21.761 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:21.761 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:21.769 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:21.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 Processor -- start 2021-07-11T04:02:21.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 -- start start 2021-07-11T04:02:21.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f51300aa860 0x7f51300b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f51300a9da0 0x7f51300aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51300ac370 con 0x7f51300acf30 2021-07-11T04:02:21.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300abb50 con 0x7f51300aa860 2021-07-11T04:02:21.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300aac40 con 0x7f51300a9da0 2021-07-11T04:02:21.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512e59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f51300aa860 0x7f51300b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:21.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512ed9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f51300acf30 0x7f51301434b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48056/0 (socket says 172.21.2.104:48056) 2021-07-11T04:02:21.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512ed9d700 1 -- 172.21.2.104:0/261817197 learned_addr learned my addr 172.21.2.104:0/261817197 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:21.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512dd9b700 1 --2- 172.21.2.104:0/261817197 >> v2:172.21.2.105:3300/0 conn(0x7f51300a9da0 0x7f51300aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:21.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512e59c700 1 -- 172.21.2.104:0/261817197 >> v2:172.21.2.105:3300/0 conn(0x7f51300a9da0 msgr2=0x7f51300aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:21.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512e59c700 1 --2- 172.21.2.104:0/261817197 >> v2:172.21.2.105:3300/0 conn(0x7f51300a9da0 0x7f51300aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:21.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512e59c700 1 -- 172.21.2.104:0/261817197 >> v1:172.21.2.104:6789/0 conn(0x7f51300acf30 legacy=0x7f51301434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:21.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512e59c700 1 -- 172.21.2.104:0/261817197 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300b16b0 con 0x7f51300aa860 2021-07-11T04:02:21.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512e59c700 1 --2- 172.21.2.104:0/261817197 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51300aa860 0x7f51300b11c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7f5118002f70 tx=0x7f5118006df0).ready entity=mon.0 client_cookie=dc20a4b955bdfc68 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:21.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512d59a700 1 -- 172.21.2.104:0/261817197 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f511800d5c0 con 0x7f51300aa860 2021-07-11T04:02:21.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512d59a700 1 -- 172.21.2.104:0/261817197 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5118000e90 con 0x7f51300aa860 2021-07-11T04:02:21.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f512d59a700 1 -- 172.21.2.104:0/261817197 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f511800d910 con 0x7f51300aa860 2021-07-11T04:02:21.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 -- 172.21.2.104:0/261817197 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51300aa860 msgr2=0x7f51300b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:21.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 --2- 172.21.2.104:0/261817197 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51300aa860 0x7f51300b11c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7f5118002f70 tx=0x7f5118006df0).stop 2021-07-11T04:02:21.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 -- 172.21.2.104:0/261817197 shutdown_connections 2021-07-11T04:02:21.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 --2- 172.21.2.104:0/261817197 >> v2:172.21.2.105:3300/0 conn(0x7f51300a9da0 0x7f51300aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:21.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 --2- 172.21.2.104:0/261817197 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51300aa860 0x7f51300b11c0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:21.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 -- 172.21.2.104:0/261817197 >> 172.21.2.104:0/261817197 conn(0x7f51300b64d0 msgr2=0x7f51300b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:21.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 -- 172.21.2.104:0/261817197 shutdown_connections 2021-07-11T04:02:21.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 -- 172.21.2.104:0/261817197 wait complete. 2021-07-11T04:02:21.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 Processor -- start 2021-07-11T04:02:21.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.824+0000 7f5134d21700 1 -- start start 2021-07-11T04:02:21.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f5134d21700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51300a9da0 0x7f51300b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f5134d21700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f51300aa860 0x7f5130064950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f5134d21700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f51300acf30 0x7f5130065a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f5134d21700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130146260 con 0x7f51300a9da0 2021-07-11T04:02:21.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f5134d21700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130144ab0 con 0x7f51300acf30 2021-07-11T04:02:21.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f5134d21700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51301412b0 con 0x7f51300aa860 2021-07-11T04:02:21.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f512e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51300a9da0 0x7f51300b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:21.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f512e59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51300a9da0 0x7f51300b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51290/0 (socket says 172.21.2.104:51290) 2021-07-11T04:02:21.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f512e59c700 1 -- 172.21.2.104:0/3736149460 learned_addr learned my addr 172.21.2.104:0/3736149460 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:21.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f512e59c700 1 -- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f51300aa860 msgr2=0x7f5130064950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:21.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f512e59c700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f51300aa860 0x7f5130064950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:21.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f512e59c700 1 -- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f51300acf30 msgr2=0x7f5130065a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:21.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f512e59c700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f51300acf30 0x7f5130065a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:21.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f512e59c700 1 -- 172.21.2.104:0/3736149460 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511800b180 con 0x7f51300a9da0 2021-07-11T04:02:21.843 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f512e59c700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51300a9da0 0x7f51300b0320 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7f5118002f70 tx=0x7f5118014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:21.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f51277fe700 1 -- 172.21.2.104:0/3736149460 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5118012d40 con 0x7f51300a9da0 2021-07-11T04:02:21.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f51277fe700 1 -- 172.21.2.104:0/3736149460 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511801e430 con 0x7f51300a9da0 2021-07-11T04:02:21.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f51277fe700 1 -- 172.21.2.104:0/3736149460 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5118012ea0 con 0x7f51300a9da0 2021-07-11T04:02:21.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f5134d21700 1 -- 172.21.2.104:0/3736149460 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5130066440 con 0x7f51300a9da0 2021-07-11T04:02:21.844 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f5134d21700 1 -- 172.21.2.104:0/3736149460 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5130066980 con 0x7f51300a9da0 2021-07-11T04:02:21.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f5134d21700 1 -- 172.21.2.104:0/3736149460 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5130057350 con 0x7f51300a9da0 2021-07-11T04:02:21.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f51277fe700 1 -- 172.21.2.104:0/3736149460 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f511800f030 con 0x7f51300a9da0 2021-07-11T04:02:21.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f51277fe700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51140531f0 0x7f51140556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f51277fe700 1 -- 172.21.2.104:0/3736149460 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f511806c4d0 con 0x7f51300a9da0 2021-07-11T04:02:21.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f512dd9b700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51140531f0 0x7f51140556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:21.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.828+0000 7f512dd9b700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51140531f0 0x7f51140556b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f512000a000 tx=0x7f5120008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:21.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.832+0000 7f51277fe700 1 -- 172.21.2.104:0/3736149460 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5118043810 con 0x7f51300a9da0 2021-07-11T04:02:21.912 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.912+0000 7f5134d21700 1 -- 172.21.2.104:0/3736149460 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51300698e0 con 0x7f51300a9da0 2021-07-11T04:02:21.913 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.912+0000 7f51277fe700 1 -- 172.21.2.104:0/3736149460 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f511802fed0 con 0x7f51300a9da0 2021-07-11T04:02:21.913 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:21.914 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":149,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:16.677257+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:02:14.500311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":28,"pool_name":"cephfs_data","create_time":"2021-07-11T04:02:15.641534+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"149","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:21.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 -- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51140531f0 msgr2=0x7f51140556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:21.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51140531f0 0x7f51140556b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f512000a000 tx=0x7f5120008040).stop 2021-07-11T04:02:21.915 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 -- 172.21.2.104:0/3736149460 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51300a9da0 msgr2=0x7f51300b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:21.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51300a9da0 0x7f51300b0320 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7f5118002f70 tx=0x7f5118014000).stop 2021-07-11T04:02:21.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 -- 172.21.2.104:0/3736149460 shutdown_connections 2021-07-11T04:02:21.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f51300acf30 0x7f5130065a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:21.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f51300aa860 0x7f5130064950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:21.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f51140531f0 0x7f51140556b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:21.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 --2- 172.21.2.104:0/3736149460 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f51300a9da0 0x7f51300b0320 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:21.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 -- 172.21.2.104:0/3736149460 >> 172.21.2.104:0/3736149460 conn(0x7f51300b64d0 msgr2=0x7f51300b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:21.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 -- 172.21.2.104:0/3736149460 shutdown_connections 2021-07-11T04:02:21.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.916+0000 7f5134d21700 1 -- 172.21.2.104:0/3736149460 wait complete. 2021-07-11T04:02:21.922 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:02:14.500311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:02:15.641534+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '149', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:02:21.923 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:02:21.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 Processor -- start 2021-07-11T04:02:21.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 -- start start 2021-07-11T04:02:21.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc5ac0b3710 0x7fc5ac0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5ac0a9c10 0x7fc5ac0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ac143920 con 0x7fc5ac0b05b0 2021-07-11T04:02:21.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ac143bc0 con 0x7fc5ac0b3710 2021-07-11T04:02:21.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac143e60 con 0x7fc5ac0a9c10 2021-07-11T04:02:21.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b164e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc5ac0b3710 0x7fc5ac0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:21.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b164e700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc5ac0b3710 0x7fc5ac0b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51300/0 (socket says 172.21.2.104:51300) 2021-07-11T04:02:21.984 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b164e700 1 -- 172.21.2.104:0/48549151 learned_addr learned my addr 172.21.2.104:0/48549151 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:21.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b164e700 1 -- 172.21.2.104:0/48549151 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5ac0a9c10 msgr2=0x7fc5ac0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:21.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b164e700 1 --2- 172.21.2.104:0/48549151 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5ac0a9c10 0x7fc5ac0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:21.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b164e700 1 -- 172.21.2.104:0/48549151 >> v1:172.21.2.105:6789/0 conn(0x7fc5ac0b05b0 legacy=0x7fc5ac13d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:21.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b164e700 1 -- 172.21.2.104:0/48549151 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac144100 con 0x7fc5ac0b3710 2021-07-11T04:02:21.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b164e700 1 --2- 172.21.2.104:0/48549151 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5ac0b3710 0x7fc5ac0b0990 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7fc5ac06aa50 tx=0x7fc5a0006eb0).ready entity=mon.0 client_cookie=e84d1c0c9ba84188 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:21.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b0e4d700 1 -- 172.21.2.104:0/48549151 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5a000d570 con 0x7fc5ac0b3710 2021-07-11T04:02:21.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b0e4d700 1 -- 172.21.2.104:0/48549151 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a0000e90 con 0x7fc5ac0b3710 2021-07-11T04:02:21.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b0e4d700 1 -- 172.21.2.104:0/48549151 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5a000d8c0 con 0x7fc5ac0b3710 2021-07-11T04:02:21.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 -- 172.21.2.104:0/48549151 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5ac0b3710 msgr2=0x7fc5ac0b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:21.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 --2- 172.21.2.104:0/48549151 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5ac0b3710 0x7fc5ac0b0990 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 rx=0x7fc5ac06aa50 tx=0x7fc5a0006eb0).stop 2021-07-11T04:02:21.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 -- 172.21.2.104:0/48549151 shutdown_connections 2021-07-11T04:02:21.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 --2- 172.21.2.104:0/48549151 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5ac0a9c10 0x7fc5ac0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:21.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 --2- 172.21.2.104:0/48549151 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5ac0b3710 0x7fc5ac0b0990 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:21.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 -- 172.21.2.104:0/48549151 >> 172.21.2.104:0/48549151 conn(0x7fc5ac0607c0 msgr2=0x7fc5ac060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:21.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 -- 172.21.2.104:0/48549151 shutdown_connections 2021-07-11T04:02:21.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.976+0000 7fc5b40b3700 1 -- 172.21.2.104:0/48549151 wait complete. 2021-07-11T04:02:21.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b40b3700 1 Processor -- start 2021-07-11T04:02:21.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b40b3700 1 -- start start 2021-07-11T04:02:21.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b40b3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5ac0a9c10 0x7fc5ac05f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b40b3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5ac0b05b0 0x7fc5ac13f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b40b3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5ac0b3710 0x7fc5ac13f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b40b3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0ab390 con 0x7fc5ac0b3710 2021-07-11T04:02:21.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b40b3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac13e520 con 0x7fc5ac0b05b0 2021-07-11T04:02:21.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b40b3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac13de90 con 0x7fc5ac0a9c10 2021-07-11T04:02:21.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b2650700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5ac0b3710 0x7fc5ac13f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:21.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b2650700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5ac0b3710 0x7fc5ac13f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51308/0 (socket says 172.21.2.104:51308) 2021-07-11T04:02:21.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b2650700 1 -- 172.21.2.104:0/2374308841 learned_addr learned my addr 172.21.2.104:0/2374308841 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:21.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b2650700 1 -- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5ac0a9c10 msgr2=0x7fc5ac05f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:21.992 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b2650700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5ac0a9c10 0x7fc5ac05f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:21.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b2650700 1 -- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5ac0b05b0 msgr2=0x7fc5ac13f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:21.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b2650700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5ac0b05b0 0x7fc5ac13f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:21.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b2650700 1 -- 172.21.2.104:0/2374308841 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a000b1e0 con 0x7fc5ac0b3710 2021-07-11T04:02:21.993 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b2650700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5ac0b3710 0x7fc5ac13f7a0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7fc5a40032f0 tx=0x7fc5a4009250).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:21.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc59effd700 1 -- 172.21.2.104:0/2374308841 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5a400f7e0 con 0x7fc5ac0b3710 2021-07-11T04:02:21.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc59effd700 1 -- 172.21.2.104:0/2374308841 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a400f940 con 0x7fc5ac0b3710 2021-07-11T04:02:21.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b40b3700 1 -- 172.21.2.104:0/2374308841 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac060400 con 0x7fc5ac0b3710 2021-07-11T04:02:21.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc59effd700 1 -- 172.21.2.104:0/2374308841 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc5a400fca0 con 0x7fc5ac0b3710 2021-07-11T04:02:21.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b40b3700 1 -- 172.21.2.104:0/2374308841 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac13fe30 con 0x7fc5ac0b3710 2021-07-11T04:02:21.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b40b3700 1 -- 172.21.2.104:0/2374308841 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac13eab0 con 0x7fc5ac0b3710 2021-07-11T04:02:21.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc59effd700 1 -- 172.21.2.104:0/2374308841 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc5a4010480 con 0x7fc5ac0b3710 2021-07-11T04:02:21.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc59effd700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc58c0532d0 0x7fc58c055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:21.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc59effd700 1 -- 172.21.2.104:0/2374308841 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc5a406bc90 con 0x7fc5ac0b3710 2021-07-11T04:02:21.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.980+0000 7fc5b1e4f700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc58c0532d0 0x7fc58c055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:21.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.984+0000 7fc5b1e4f700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc58c0532d0 0x7fc58c055790 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fc5a8000f70 tx=0x7fc5a80077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:21.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:21.984+0000 7fc59effd700 1 -- 172.21.2.104:0/2374308841 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc5a4042e60 con 0x7fc5ac0b3710 2021-07-11T04:02:22.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.064+0000 7fc5b40b3700 1 -- 172.21.2.104:0/2374308841 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc5ac13e690 con 0x7fc5ac0b3710 2021-07-11T04:02:22.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.064+0000 7fc59effd700 1 -- 172.21.2.104:0/2374308841 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v149) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc5a400fe00 con 0x7fc5ac0b3710 2021-07-11T04:02:22.066 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:02:22.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 -- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc58c0532d0 msgr2=0x7fc58c055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc58c0532d0 0x7fc58c055790 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7fc5a8000f70 tx=0x7fc5a80077f0).stop 2021-07-11T04:02:22.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 -- 172.21.2.104:0/2374308841 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5ac0b3710 msgr2=0x7fc5ac13f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5ac0b3710 0x7fc5ac13f7a0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7fc5a40032f0 tx=0x7fc5a4009250).stop 2021-07-11T04:02:22.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 -- 172.21.2.104:0/2374308841 shutdown_connections 2021-07-11T04:02:22.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc58c0532d0 0x7fc58c055790 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5ac0b3710 0x7fc5ac13f7a0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5ac0b05b0 0x7fc5ac13f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 --2- 172.21.2.104:0/2374308841 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5ac0a9c10 0x7fc5ac05f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 -- 172.21.2.104:0/2374308841 >> 172.21.2.104:0/2374308841 conn(0x7fc5ac0607c0 msgr2=0x7fc5ac142ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:22.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 -- 172.21.2.104:0/2374308841 shutdown_connections 2021-07-11T04:02:22.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.068+0000 7fc5b40b3700 1 -- 172.21.2.104:0/2374308841 wait complete. 2021-07-11T04:02:22.073 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:02:22.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 Processor -- start 2021-07-11T04:02:22.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- start start 2021-07-11T04:02:22.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f87a00b12d0 0x7f87a00affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f87a00b22a0 0x7f87a0141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a00a77d0 con 0x7f87a00b1ec0 2021-07-11T04:02:22.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a00b1740 con 0x7f87a00b12d0 2021-07-11T04:02:22.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a00b0650 con 0x7f87a00b22a0 2021-07-11T04:02:22.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f879ffff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f87a00b12d0 0x7f87a00affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f879ffff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f87a00b12d0 0x7f87a00affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51316/0 (socket says 172.21.2.104:51316) 2021-07-11T04:02:22.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f879ffff700 1 -- 172.21.2.104:0/4186551138 learned_addr learned my addr 172.21.2.104:0/4186551138 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:22.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f879ffff700 1 -- 172.21.2.104:0/4186551138 >> v2:172.21.2.105:3300/0 conn(0x7f87a00b22a0 msgr2=0x7f87a0141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:22.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f879ffff700 1 --2- 172.21.2.104:0/4186551138 >> v2:172.21.2.105:3300/0 conn(0x7f87a00b22a0 0x7f87a0141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f879ffff700 1 -- 172.21.2.104:0/4186551138 >> v1:172.21.2.105:6789/0 conn(0x7f87a00b1ec0 legacy=0x7f87a00a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:22.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f879ffff700 1 -- 172.21.2.104:0/4186551138 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a00b08f0 con 0x7f87a00b12d0 2021-07-11T04:02:22.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f879ffff700 1 --2- 172.21.2.104:0/4186551138 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87a00b12d0 0x7f87a00affb0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f87a00ad2d0 tx=0x7f8788006eb0).ready entity=mon.0 client_cookie=3ae16ca5dfc319ae server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f879f7fe700 1 -- 172.21.2.104:0/4186551138 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f878800d5c0 con 0x7f87a00b12d0 2021-07-11T04:02:22.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f879f7fe700 1 -- 172.21.2.104:0/4186551138 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788000e90 con 0x7f87a00b12d0 2021-07-11T04:02:22.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f879f7fe700 1 -- 172.21.2.104:0/4186551138 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f878800d910 con 0x7f87a00b12d0 2021-07-11T04:02:22.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- 172.21.2.104:0/4186551138 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87a00b12d0 msgr2=0x7f87a00affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 --2- 172.21.2.104:0/4186551138 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87a00b12d0 0x7f87a00affb0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f87a00ad2d0 tx=0x7f8788006eb0).stop 2021-07-11T04:02:22.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- 172.21.2.104:0/4186551138 shutdown_connections 2021-07-11T04:02:22.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 --2- 172.21.2.104:0/4186551138 >> v2:172.21.2.105:3300/0 conn(0x7f87a00b22a0 0x7f87a0141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 --2- 172.21.2.104:0/4186551138 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87a00b12d0 0x7f87a00affb0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- 172.21.2.104:0/4186551138 >> 172.21.2.104:0/4186551138 conn(0x7f87a005f6b0 msgr2=0x7f87a005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:22.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- 172.21.2.104:0/4186551138 shutdown_connections 2021-07-11T04:02:22.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- 172.21.2.104:0/4186551138 wait complete. 2021-07-11T04:02:22.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 Processor -- start 2021-07-11T04:02:22.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- start start 2021-07-11T04:02:22.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87a00b12d0 0x7f87a015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87a00b1ec0 0x7f87a00ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f87a00b22a0 0x7f87a00abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00ac1b0 con 0x7f87a00b12d0 2021-07-11T04:02:22.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a015aba0 con 0x7f87a00b22a0 2021-07-11T04:02:22.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a6f81700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a00aa170 con 0x7f87a00b1ec0 2021-07-11T04:02:22.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a4d1d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87a00b12d0 0x7f87a015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a4d1d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87a00b12d0 0x7f87a015a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51318/0 (socket says 172.21.2.104:51318) 2021-07-11T04:02:22.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.128+0000 7f87a4d1d700 1 -- 172.21.2.104:0/2115260116 learned_addr learned my addr 172.21.2.104:0/2115260116 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:22.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f87a4d1d700 1 -- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87a00b1ec0 msgr2=0x7f87a00ab7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:22.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f87a4d1d700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87a00b1ec0 0x7f87a00ab7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f87a4d1d700 1 -- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f87a00b22a0 msgr2=0x7f87a00abcc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:22.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f87a4d1d700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f87a00b22a0 0x7f87a00abcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f87a4d1d700 1 -- 172.21.2.104:0/2115260116 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878800b1e0 con 0x7f87a00b12d0 2021-07-11T04:02:22.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f87a4d1d700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87a00b12d0 0x7f87a015a3a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f8790000f70 tx=0x7f8790007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f879dffb700 1 -- 172.21.2.104:0/2115260116 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f879000ce80 con 0x7f87a00b12d0 2021-07-11T04:02:22.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f879dffb700 1 -- 172.21.2.104:0/2115260116 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8790013450 con 0x7f87a00b12d0 2021-07-11T04:02:22.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f87a6f81700 1 -- 172.21.2.104:0/2115260116 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a00aa410 con 0x7f87a00b12d0 2021-07-11T04:02:22.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f879dffb700 1 -- 172.21.2.104:0/2115260116 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f87900137b0 con 0x7f87a00b12d0 2021-07-11T04:02:22.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f87a6f81700 1 -- 172.21.2.104:0/2115260116 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a015a890 con 0x7f87a00b12d0 2021-07-11T04:02:22.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f87a6f81700 1 -- 172.21.2.104:0/2115260116 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a00aa830 con 0x7f87a00b12d0 2021-07-11T04:02:22.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f879dffb700 1 -- 172.21.2.104:0/2115260116 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f879001a060 con 0x7f87a00b12d0 2021-07-11T04:02:22.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f879dffb700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f87840531f0 0x7f87840556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f879dffb700 1 -- 172.21.2.104:0/2115260116 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f879006afc0 con 0x7f87a00b12d0 2021-07-11T04:02:22.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f879ffff700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f87840531f0 0x7f87840556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.132+0000 7f879ffff700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f87840531f0 0x7f87840556b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f87a00ad2d0 tx=0x7f8788015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.136+0000 7f879dffb700 1 -- 172.21.2.104:0/2115260116 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f87900421f0 con 0x7f87a00b12d0 2021-07-11T04:02:22.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 -- 172.21.2.104:0/2115260116 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f87a0057350 con 0x7f87a00b12d0 2021-07-11T04:02:22.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f879dffb700 1 -- 172.21.2.104:0/2115260116 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v149) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8790018020 con 0x7f87a00b12d0 2021-07-11T04:02:22.217 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:22.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 -- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f87840531f0 msgr2=0x7f87840556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f87840531f0 0x7f87840556b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f87a00ad2d0 tx=0x7f8788015000).stop 2021-07-11T04:02:22.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 -- 172.21.2.104:0/2115260116 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87a00b12d0 msgr2=0x7f87a015a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87a00b12d0 0x7f87a015a3a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7f8790000f70 tx=0x7f8790007c30).stop 2021-07-11T04:02:22.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 -- 172.21.2.104:0/2115260116 shutdown_connections 2021-07-11T04:02:22.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f87a00b22a0 0x7f87a00abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f87a00b1ec0 0x7f87a00ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f87840531f0 0x7f87840556b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 --2- 172.21.2.104:0/2115260116 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f87a00b12d0 0x7f87a015a3a0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 -- 172.21.2.104:0/2115260116 >> 172.21.2.104:0/2115260116 conn(0x7f87a005f6b0 msgr2=0x7f87a0141020 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:22.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 -- 172.21.2.104:0/2115260116 shutdown_connections 2021-07-11T04:02:22.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.216+0000 7f87a6f81700 1 -- 172.21.2.104:0/2115260116 wait complete. 2021-07-11T04:02:22.224 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:02:22.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d444e700 1 Processor -- start 2021-07-11T04:02:22.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d444e700 1 -- start start 2021-07-11T04:02:22.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d444e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa4cc0b1ec0 0x7fa4cc0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d444e700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fa4cc0affb0 0x7fa4cc0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d444e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4cc0b2790 con 0x7fa4cc0a70c0 2021-07-11T04:02:22.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d444e700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4cc0b03f0 con 0x7fa4cc0affb0 2021-07-11T04:02:22.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d444e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0b0650 con 0x7fa4cc0b1ec0 2021-07-11T04:02:22.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d29eb700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fa4cc0a70c0 0x7fa4cc141250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48094/0 (socket says 172.21.2.104:48094) 2021-07-11T04:02:22.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d29eb700 1 -- 172.21.2.104:0/2834399689 learned_addr learned my addr 172.21.2.104:0/2834399689 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:22.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531455138 0 0) 0x7fa4cc0b2790 con 0x7fa4cc0a70c0 2021-07-11T04:02:22.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b4003260 con 0x7fa4cc0a70c0 2021-07-11T04:02:22.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1185118067 0 0) 0x7fa4b4003260 con 0x7fa4cc0a70c0 2021-07-11T04:02:22.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa4cc0b1ec0 msgr2=0x7fa4cc0b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:22.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 --2- 172.21.2.104:0/2834399689 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa4cc0b1ec0 0x7fa4cc0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 >> v2:172.21.2.105:3300/0 conn(0x7fa4cc0affb0 msgr2=0x7fa4cc0b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:22.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 --2- 172.21.2.104:0/2834399689 >> v2:172.21.2.105:3300/0 conn(0x7fa4cc0affb0 0x7fa4cc0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc0b08f0 con 0x7fa4cc0a70c0 2021-07-11T04:02:22.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fa4c40019e0 con 0x7fa4cc0a70c0 2021-07-11T04:02:22.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 >> v1:172.21.2.104:6789/0 conn(0x7fa4cc0a70c0 legacy=0x7fa4cc141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 --2- 172.21.2.104:0/2834399689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4b40037d0 0x7fa4b4003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 --2- 172.21.2.104:0/2834399689 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa4cc0b1ec0 0x7fa4b4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 --2- 172.21.2.104:0/2834399689 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa4b4004bf0 0x7fa4b400b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0b03f0 con 0x7fa4b40037d0 2021-07-11T04:02:22.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0b0650 con 0x7fa4b4004bf0 2021-07-11T04:02:22.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b4005300 con 0x7fa4cc0b1ec0 2021-07-11T04:02:22.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d21ea700 1 --2- 172.21.2.104:0/2834399689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4b40037d0 0x7fa4b4003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d21ea700 1 -- 172.21.2.104:0/2834399689 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa4cc0b1ec0 msgr2=0x7fa4b4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:22.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d21ea700 1 --2- 172.21.2.104:0/2834399689 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa4cc0b1ec0 0x7fa4b4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d21ea700 1 -- 172.21.2.104:0/2834399689 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa4b4004bf0 msgr2=0x7fa4b400b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:22.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.276+0000 7fa4d21ea700 1 --2- 172.21.2.104:0/2834399689 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa4b4004bf0 0x7fa4b400b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d21ea700 1 -- 172.21.2.104:0/2834399689 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b4003260 con 0x7fa4b40037d0 2021-07-11T04:02:22.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d21ea700 1 --2- 172.21.2.104:0/2834399689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4b40037d0 0x7fa4b4003bf0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 rx=0x7fa4c8002f70 tx=0x7fa4c8006ca0).ready entity=mon.0 client_cookie=d7dd3d461c51f625 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c8012410 con 0x7fa4b40037d0 2021-07-11T04:02:22.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c8012570 con 0x7fa4b40037d0 2021-07-11T04:02:22.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d11e8700 1 -- 172.21.2.104:0/2834399689 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c80128d0 con 0x7fa4b40037d0 2021-07-11T04:02:22.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 -- 172.21.2.104:0/2834399689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4b40037d0 msgr2=0x7fa4b4003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 --2- 172.21.2.104:0/2834399689 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4b40037d0 0x7fa4b4003bf0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 rx=0x7fa4c8002f70 tx=0x7fa4c8006ca0).stop 2021-07-11T04:02:22.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d29eb700 1 -- 172.21.2.104:0/2834399689 reap_dead start 2021-07-11T04:02:22.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 -- 172.21.2.104:0/2834399689 shutdown_connections 2021-07-11T04:02:22.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 -- 172.21.2.104:0/2834399689 >> 172.21.2.104:0/2834399689 conn(0x7fa4cc05f6b0 msgr2=0x7fa4cc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:22.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 -- 172.21.2.104:0/2834399689 shutdown_connections 2021-07-11T04:02:22.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 -- 172.21.2.104:0/2834399689 wait complete. 2021-07-11T04:02:22.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 Processor -- start 2021-07-11T04:02:22.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 -- start start 2021-07-11T04:02:22.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4cc0b3550 0x7fa4cc0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa4cc1470a0 0x7fa4cc0ab410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa4cc13a820 0x7fa4cc13acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0b3930 con 0x7fa4cc0b3550 2021-07-11T04:02:22.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0ac520 con 0x7fa4cc13a820 2021-07-11T04:02:22.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc0aa220 con 0x7fa4cc1470a0 2021-07-11T04:02:22.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d21ea700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4cc0b3550 0x7fa4cc0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d21ea700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4cc0b3550 0x7fa4cc0b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51338/0 (socket says 172.21.2.104:51338) 2021-07-11T04:02:22.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d21ea700 1 -- 172.21.2.104:0/3608403366 learned_addr learned my addr 172.21.2.104:0/3608403366 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:22.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d21ea700 1 -- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa4cc1470a0 msgr2=0x7fa4cc0ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d21ea700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa4cc1470a0 0x7fa4cc0ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d21ea700 1 -- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa4cc13a820 msgr2=0x7fa4cc13acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d21ea700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa4cc13a820 0x7fa4cc13acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d21ea700 1 -- 172.21.2.104:0/3608403366 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc0aa4c0 con 0x7fa4cc0b3550 2021-07-11T04:02:22.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d21ea700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4cc0b3550 0x7fa4cc0b4b00 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7fa4c8002f70 tx=0x7fa4c8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4bf7fe700 1 -- 172.21.2.104:0/3608403366 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c80117b0 con 0x7fa4cc0b3550 2021-07-11T04:02:22.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4bf7fe700 1 -- 172.21.2.104:0/3608403366 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c8011910 con 0x7fa4cc0b3550 2021-07-11T04:02:22.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4bf7fe700 1 -- 172.21.2.104:0/3608403366 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fa4c8011c70 con 0x7fa4cc0b3550 2021-07-11T04:02:22.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 -- 172.21.2.104:0/3608403366 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc0aa760 con 0x7fa4cc0b3550 2021-07-11T04:02:22.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.280+0000 7fa4d444e700 1 -- 172.21.2.104:0/3608403366 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4cc0abfc0 con 0x7fa4cc0b3550 2021-07-11T04:02:22.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.284+0000 7fa4d444e700 1 -- 172.21.2.104:0/3608403366 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4cc057350 con 0x7fa4cc0b3550 2021-07-11T04:02:22.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.284+0000 7fa4bf7fe700 1 -- 172.21.2.104:0/3608403366 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fa4c800f030 con 0x7fa4cc0b3550 2021-07-11T04:02:22.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.284+0000 7fa4bf7fe700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4ac053270 0x7fa4ac055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.284+0000 7fa4bf7fe700 1 -- 172.21.2.104:0/3608403366 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fa4c806c530 con 0x7fa4cc0b3550 2021-07-11T04:02:22.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.284+0000 7fa4d19e9700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4ac053270 0x7fa4ac055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.284+0000 7fa4d19e9700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4ac053270 0x7fa4ac055730 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fa4c0002f70 tx=0x7fa4c0006a80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.284+0000 7fa4bf7fe700 1 -- 172.21.2.104:0/3608403366 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fa4c8043760 con 0x7fa4cc0b3550 2021-07-11T04:02:22.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 -- 172.21.2.104:0/3608403366 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa4cc13b340 con 0x7fa4cc0b3550 2021-07-11T04:02:22.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4bf7fe700 1 -- 172.21.2.104:0/3608403366 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v149) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa4cc13b340 con 0x7fa4cc0b3550 2021-07-11T04:02:22.368 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:22.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 -- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4ac053270 msgr2=0x7fa4ac055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4ac053270 0x7fa4ac055730 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7fa4c0002f70 tx=0x7fa4c0006a80).stop 2021-07-11T04:02:22.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 -- 172.21.2.104:0/3608403366 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4cc0b3550 msgr2=0x7fa4cc0b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4cc0b3550 0x7fa4cc0b4b00 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7fa4c8002f70 tx=0x7fa4c8015000).stop 2021-07-11T04:02:22.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 -- 172.21.2.104:0/3608403366 shutdown_connections 2021-07-11T04:02:22.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fa4cc13a820 0x7fa4cc13acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fa4cc1470a0 0x7fa4cc0ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fa4ac053270 0x7fa4ac055730 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 --2- 172.21.2.104:0/3608403366 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fa4cc0b3550 0x7fa4cc0b4b00 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 -- 172.21.2.104:0/3608403366 >> 172.21.2.104:0/3608403366 conn(0x7fa4cc05f6b0 msgr2=0x7fa4cc13daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:22.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 -- 172.21.2.104:0/3608403366 shutdown_connections 2021-07-11T04:02:22.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.368+0000 7fa4d444e700 1 -- 172.21.2.104:0/3608403366 wait complete. 2021-07-11T04:02:22.375 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:02:22.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabefd36700 1 Processor -- start 2021-07-11T04:02:22.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabefd36700 1 -- start start 2021-07-11T04:02:22.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabefd36700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fabe80b1ec0 0x7fabe80b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabefd36700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe80a77d0 con 0x7fabe80affb0 2021-07-11T04:02:22.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabefd36700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabe80b03f0 con 0x7fabe80b22a0 2021-07-11T04:02:22.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabefd36700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80b0650 con 0x7fabe80b1ec0 2021-07-11T04:02:22.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabed2d1700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fabe80affb0 0x7fabe80a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48118/0 (socket says 172.21.2.104:48118) 2021-07-11T04:02:22.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabed2d1700 1 -- 172.21.2.104:0/2457203921 learned_addr learned my addr 172.21.2.104:0/2457203921 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:22.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514653735 0 0) 0x7fabe80a77d0 con 0x7fabe80affb0 2021-07-11T04:02:22.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd0003260 con 0x7fabe80affb0 2021-07-11T04:02:22.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabedad2700 1 --2- 172.21.2.104:0/2457203921 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fabe80b1ec0 0x7fabe80b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 619730335 0 0) 0x7fabd0003260 con 0x7fabe80affb0 2021-07-11T04:02:22.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fabe80b1ec0 msgr2=0x7fabe80b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:22.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 --2- 172.21.2.104:0/2457203921 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fabe80b1ec0 0x7fabe80b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 >> v1:172.21.2.105:6789/0 conn(0x7fabe80b22a0 legacy=0x7fabe8141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:22.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80b08f0 con 0x7fabe80affb0 2021-07-11T04:02:22.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fabe4001660 con 0x7fabe80affb0 2021-07-11T04:02:22.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 >> v1:172.21.2.104:6789/0 conn(0x7fabe80affb0 legacy=0x7fabe80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 --2- 172.21.2.104:0/2457203921 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabd0003800 0x7fabd0003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 --2- 172.21.2.104:0/2457203921 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fabd0004500 0x7fabd0004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 --2- 172.21.2.104:0/2457203921 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fabd0004f90 0x7fabd000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80b0650 con 0x7fabd0003800 2021-07-11T04:02:22.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd000be50 con 0x7fabd0004f90 2021-07-11T04:02:22.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd000c010 con 0x7fabd0004500 2021-07-11T04:02:22.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabedad2700 1 --2- 172.21.2.104:0/2457203921 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabd0003800 0x7fabd0003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabedad2700 1 -- 172.21.2.104:0/2457203921 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fabd0004500 msgr2=0x7fabd0004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:22.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabedad2700 1 --2- 172.21.2.104:0/2457203921 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fabd0004500 0x7fabd0004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabedad2700 1 -- 172.21.2.104:0/2457203921 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fabd0004f90 msgr2=0x7fabd000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:22.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabedad2700 1 --2- 172.21.2.104:0/2457203921 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fabd0004f90 0x7fabd000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabedad2700 1 -- 172.21.2.104:0/2457203921 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80b08f0 con 0x7fabd0003800 2021-07-11T04:02:22.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.428+0000 7fabedad2700 1 --2- 172.21.2.104:0/2457203921 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabd0003800 0x7fabd0003c00 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7fabd8007000 tx=0x7fabd8008040).ready entity=mon.0 client_cookie=e5f21b9d5c211062 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabd8012420 con 0x7fabd0003800 2021-07-11T04:02:22.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd800c030 con 0x7fabd0003800 2021-07-11T04:02:22.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabecad0700 1 -- 172.21.2.104:0/2457203921 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabd8012770 con 0x7fabd0003800 2021-07-11T04:02:22.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- 172.21.2.104:0/2457203921 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabd0003800 msgr2=0x7fabd0003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 --2- 172.21.2.104:0/2457203921 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabd0003800 0x7fabd0003c00 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 rx=0x7fabd8007000 tx=0x7fabd8008040).stop 2021-07-11T04:02:22.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabee2d3700 1 -- 172.21.2.104:0/2457203921 reap_dead start 2021-07-11T04:02:22.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- 172.21.2.104:0/2457203921 shutdown_connections 2021-07-11T04:02:22.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- 172.21.2.104:0/2457203921 >> 172.21.2.104:0/2457203921 conn(0x7fabe805f6b0 msgr2=0x7fabe805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:22.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- 172.21.2.104:0/2457203921 shutdown_connections 2021-07-11T04:02:22.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- 172.21.2.104:0/2457203921 wait complete. 2021-07-11T04:02:22.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 Processor -- start 2021-07-11T04:02:22.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- start start 2021-07-11T04:02:22.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fabe80b3ae0 0x7fabe81462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabe8149c90 0x7fabe815a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fabe815a610 0x7fabe815aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80b5e50 con 0x7fabe8149c90 2021-07-11T04:02:22.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80b5430 con 0x7fabe815a610 2021-07-11T04:02:22.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabe80b4930 con 0x7fabe80b3ae0 2021-07-11T04:02:22.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabed2d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabe8149c90 0x7fabe815a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabed2d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabe8149c90 0x7fabe815a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51360/0 (socket says 172.21.2.104:51360) 2021-07-11T04:02:22.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabed2d1700 1 -- 172.21.2.104:0/1916569121 learned_addr learned my addr 172.21.2.104:0/1916569121 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:22.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabed2d1700 1 -- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fabe80b3ae0 msgr2=0x7fabe81462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabed2d1700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fabe80b3ae0 0x7fabe81462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabed2d1700 1 -- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fabe815a610 msgr2=0x7fabe815aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabed2d1700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fabe815a610 0x7fabe815aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabed2d1700 1 -- 172.21.2.104:0/1916569121 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe80b3ec0 con 0x7fabe8149c90 2021-07-11T04:02:22.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabed2d1700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabe8149c90 0x7fabe815a120 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7fabe80b4310 tx=0x7fabe4001ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabd6ffd700 1 -- 172.21.2.104:0/1916569121 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabe4007170 con 0x7fabe8149c90 2021-07-11T04:02:22.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabd6ffd700 1 -- 172.21.2.104:0/1916569121 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe40072d0 con 0x7fabe8149c90 2021-07-11T04:02:22.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- 172.21.2.104:0/1916569121 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe815af60 con 0x7fabe8149c90 2021-07-11T04:02:22.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabd6ffd700 1 -- 172.21.2.104:0/1916569121 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fabe40049e0 con 0x7fabe8149c90 2021-07-11T04:02:22.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- 172.21.2.104:0/1916569121 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe80b4690 con 0x7fabe8149c90 2021-07-11T04:02:22.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabefd36700 1 -- 172.21.2.104:0/1916569121 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe8057350 con 0x7fabe8149c90 2021-07-11T04:02:22.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabd6ffd700 1 -- 172.21.2.104:0/1916569121 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fabe401b060 con 0x7fabe8149c90 2021-07-11T04:02:22.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabd6ffd700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fabc80532b0 0x7fabc8055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.432+0000 7fabd6ffd700 1 -- 172.21.2.104:0/1916569121 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fabe406b1b0 con 0x7fabe8149c90 2021-07-11T04:02:22.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.436+0000 7fabedad2700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fabc80532b0 0x7fabc8055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.436+0000 7fabedad2700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fabc80532b0 0x7fabc8055770 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fabd8007000 tx=0x7fabd8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.436+0000 7fabd6ffd700 1 -- 172.21.2.104:0/1916569121 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fabe40424f0 con 0x7fabe8149c90 2021-07-11T04:02:22.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 -- 172.21.2.104:0/1916569121 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fabe80ac760 con 0x7fabe8149c90 2021-07-11T04:02:22.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabd6ffd700 1 -- 172.21.2.104:0/1916569121 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v149) v1 ==== 100+0+10 (secure 0 0 0) 0x7fabe4019070 con 0x7fabe8149c90 2021-07-11T04:02:22.519 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:22.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 -- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fabc80532b0 msgr2=0x7fabc8055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fabc80532b0 0x7fabc8055770 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fabd8007000 tx=0x7fabd8008040).stop 2021-07-11T04:02:22.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 -- 172.21.2.104:0/1916569121 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabe8149c90 msgr2=0x7fabe815a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabe8149c90 0x7fabe815a120 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7fabe80b4310 tx=0x7fabe4001ae0).stop 2021-07-11T04:02:22.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 -- 172.21.2.104:0/1916569121 shutdown_connections 2021-07-11T04:02:22.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fabe815a610 0x7fabe815aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fabc80532b0 0x7fabc8055770 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fabe8149c90 0x7fabe815a120 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 --2- 172.21.2.104:0/1916569121 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fabe80b3ae0 0x7fabe81462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 -- 172.21.2.104:0/1916569121 >> 172.21.2.104:0/1916569121 conn(0x7fabe805f6b0 msgr2=0x7fabe80a82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:22.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 -- 172.21.2.104:0/1916569121 shutdown_connections 2021-07-11T04:02:22.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.520+0000 7fabefd36700 1 -- 172.21.2.104:0/1916569121 wait complete. 2021-07-11T04:02:22.527 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:02:22.527 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:22.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 Processor -- start 2021-07-11T04:02:22.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- start start 2021-07-11T04:02:22.580 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f5a380b1170 0x7f5a38141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5a38141920 0x7f5a38141db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a380b31d0 con 0x7f5a380b7810 2021-07-11T04:02:22.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a381422a0 con 0x7f5a380b1170 2021-07-11T04:02:22.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a380b7bf0 con 0x7f5a38141920 2021-07-11T04:02:22.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a377fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f5a380b7810 0x7f5a381445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48134/0 (socket says 172.21.2.104:48134) 2021-07-11T04:02:22.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a377fe700 1 -- 172.21.2.104:0/3517528868 learned_addr learned my addr 172.21.2.104:0/3517528868 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:22.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a36ffd700 1 --2- 172.21.2.104:0/3517528868 >> v2:172.21.2.104:3300/0 conn(0x7f5a380b1170 0x7f5a38141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a36ffd700 1 -- 172.21.2.104:0/3517528868 >> v2:172.21.2.105:3300/0 conn(0x7f5a38141920 msgr2=0x7f5a38141db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:22.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a36ffd700 1 --2- 172.21.2.104:0/3517528868 >> v2:172.21.2.105:3300/0 conn(0x7f5a38141920 0x7f5a38141db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a36ffd700 1 -- 172.21.2.104:0/3517528868 >> v1:172.21.2.104:6789/0 conn(0x7f5a380b7810 legacy=0x7f5a381445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a36ffd700 1 -- 172.21.2.104:0/3517528868 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a380675c0 con 0x7f5a380b1170 2021-07-11T04:02:22.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a36ffd700 1 --2- 172.21.2.104:0/3517528868 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a380b1170 0x7f5a38141430 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 rx=0x7f5a28002f70 tx=0x7f5a28006df0).ready entity=mon.0 client_cookie=9bbe22cfd0c8eb6c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a367fc700 1 -- 172.21.2.104:0/3517528868 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a2800d5c0 con 0x7f5a380b1170 2021-07-11T04:02:22.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a367fc700 1 -- 172.21.2.104:0/3517528868 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a28000e90 con 0x7f5a380b1170 2021-07-11T04:02:22.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a367fc700 1 -- 172.21.2.104:0/3517528868 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a2800d910 con 0x7f5a380b1170 2021-07-11T04:02:22.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- 172.21.2.104:0/3517528868 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a380b1170 msgr2=0x7f5a38141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 --2- 172.21.2.104:0/3517528868 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a380b1170 0x7f5a38141430 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 rx=0x7f5a28002f70 tx=0x7f5a28006df0).stop 2021-07-11T04:02:22.585 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- 172.21.2.104:0/3517528868 shutdown_connections 2021-07-11T04:02:22.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 --2- 172.21.2.104:0/3517528868 >> v2:172.21.2.105:3300/0 conn(0x7f5a38141920 0x7f5a38141db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 --2- 172.21.2.104:0/3517528868 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a380b1170 0x7f5a38141430 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- 172.21.2.104:0/3517528868 >> 172.21.2.104:0/3517528868 conn(0x7f5a3805f6b0 msgr2=0x7f5a3805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:22.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- 172.21.2.104:0/3517528868 shutdown_connections 2021-07-11T04:02:22.586 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- 172.21.2.104:0/3517528868 wait complete. 2021-07-11T04:02:22.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 Processor -- start 2021-07-11T04:02:22.589 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- start start 2021-07-11T04:02:22.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a380b1170 0x7f5a380ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5a380b7810 0x7f5a38143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a38141920 0x7f5a380a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a38151050 con 0x7f5a380b1170 2021-07-11T04:02:22.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a380b0950 con 0x7f5a380b7810 2021-07-11T04:02:22.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a3e203700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a381503d0 con 0x7f5a38141920 2021-07-11T04:02:22.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a377fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a380b1170 0x7f5a380ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a377fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a380b1170 0x7f5a380ad4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51372/0 (socket says 172.21.2.104:51372) 2021-07-11T04:02:22.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.580+0000 7f5a377fe700 1 -- 172.21.2.104:0/1139038832 learned_addr learned my addr 172.21.2.104:0/1139038832 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:22.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a377fe700 1 -- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a38141920 msgr2=0x7f5a380a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a377fe700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a38141920 0x7f5a380a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a377fe700 1 -- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5a380b7810 msgr2=0x7f5a38143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a377fe700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5a380b7810 0x7f5a38143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a377fe700 1 -- 172.21.2.104:0/1139038832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2800b180 con 0x7f5a380b1170 2021-07-11T04:02:22.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a377fe700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a380b1170 0x7f5a380ad4c0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f5a38067ca0 tx=0x7f5a20007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a34ff9700 1 -- 172.21.2.104:0/1139038832 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a2000ce30 con 0x7f5a380b1170 2021-07-11T04:02:22.594 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a34ff9700 1 -- 172.21.2.104:0/1139038832 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a200072e0 con 0x7f5a380b1170 2021-07-11T04:02:22.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a3e203700 1 -- 172.21.2.104:0/1139038832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a38150670 con 0x7f5a380b1170 2021-07-11T04:02:22.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a34ff9700 1 -- 172.21.2.104:0/1139038832 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5a20005800 con 0x7f5a380b1170 2021-07-11T04:02:22.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a3e203700 1 -- 172.21.2.104:0/1139038832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a38150bb0 con 0x7f5a380b1170 2021-07-11T04:02:22.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a3e203700 1 -- 172.21.2.104:0/1139038832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a38150d70 con 0x7f5a380b1170 2021-07-11T04:02:22.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a34ff9700 1 -- 172.21.2.104:0/1139038832 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5a20010070 con 0x7f5a380b1170 2021-07-11T04:02:22.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a34ff9700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a1c0531f0 0x7f5a1c0556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:22.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a34ff9700 1 -- 172.21.2.104:0/1139038832 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5a2006aba0 con 0x7f5a380b1170 2021-07-11T04:02:22.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a36ffd700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a1c0531f0 0x7f5a1c0556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:22.596 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.584+0000 7f5a36ffd700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a1c0531f0 0x7f5a1c0556b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f5a28002f70 tx=0x7f5a28014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:22.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.588+0000 7f5a34ff9700 1 -- 172.21.2.104:0/1139038832 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5a20041ee0 con 0x7f5a380b1170 2021-07-11T04:02:22.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.692+0000 7f5a3e203700 1 -- 172.21.2.104:0/1139038832 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a38057350 con 0x7f5a380b1170 2021-07-11T04:02:22.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.692+0000 7f5a34ff9700 1 -- 172.21.2.104:0/1139038832 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f5a20045500 con 0x7f5a380b1170 2021-07-11T04:02:22.694 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:22.695 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/696025904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":696025904},{"type":"v1","addr":"172.21.2.104:6833","nonce":696025904}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8412,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/438569403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":438569403},{"type":"v1","addr":"172.21.2.104:6835","nonce":438569403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:16.677336+0000","modified":"2021-07-11T04:02:17.681052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8415},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8415":{"gid":8415,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.2.105:6835/130583442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":130583442},{"type":"v1","addr":"172.21.2.105:6835","nonce":130583442}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-07-11T04:02:22.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 -- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a1c0531f0 msgr2=0x7f5a1c0556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a1c0531f0 0x7f5a1c0556b0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f5a28002f70 tx=0x7f5a28014000).stop 2021-07-11T04:02:22.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 -- 172.21.2.104:0/1139038832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a380b1170 msgr2=0x7f5a380ad4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:22.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a380b1170 0x7f5a380ad4c0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 rx=0x7f5a38067ca0 tx=0x7f5a20007c40).stop 2021-07-11T04:02:22.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 -- 172.21.2.104:0/1139038832 shutdown_connections 2021-07-11T04:02:22.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5a38141920 0x7f5a380a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5a380b7810 0x7f5a38143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:22.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5a1c0531f0 0x7f5a1c0556b0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 --2- 172.21.2.104:0/1139038832 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5a380b1170 0x7f5a380ad4c0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:22.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 -- 172.21.2.104:0/1139038832 >> 172.21.2.104:0/1139038832 conn(0x7f5a3805f6b0 msgr2=0x7f5a380b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:22.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 -- 172.21.2.104:0/1139038832 shutdown_connections 2021-07-11T04:02:22.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:22.696+0000 7f5a3e203700 1 -- 172.21.2.104:0/1139038832 wait complete. 2021-07-11T04:02:22.699 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 74 2021-07-11T04:02:23.004 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:02:16.677336+0000', 'modified': '2021-07-11T04:02:17.681052+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8415}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8415': {'gid': 8415, 'name': 'b', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.105:6835/130583442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.105:6834', 'nonce': 130583442}, {'type': 'v1', 'addr': '172.21.2.105:6835', 'nonce': 130583442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:02:23.005 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:02:23.006 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2021-07-11T04:02:23.070 INFO:teuthology.orchestra.run.gibba005.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":8415,"want_state":"up:active","state":"up:active","rank_uptime":6.3864250040000003,"mdsmap_epoch":74,"osdmap_epoch":149,"osdmap_epoch_barrier":149,"uptime":10.575362481999999} 2021-07-11T04:02:23.070 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 8415, "mdsmap_epoch": 74, "osdmap_epoch": 149, "osdmap_epoch_barrier": 149, "rank_uptime": 6.386425004, "state": "up:active", "uptime": 10.575362482, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:02:23.071 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:02:23.071 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-07-11T04:02:23.072 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba023.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-07-11T04:02:23.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:23.072 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:23.083 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:02:23.134 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:23.135 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:23.141 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:23.141 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:23.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:23.143 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:02:23.147 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:02:23.147 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:23.189 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:02:23.199 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:23.199 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:23.204 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:23.205 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:23.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:23.206 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:02:23.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:2021-07-11T04:02:23.222+0000 7fa71b607080 -1 init, newargv = 0x56457cf1b990 newargc=9 2021-07-11T04:02:23.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15280]: starting ceph client 2021-07-11T04:02:23.242 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15280]: starting fuse 2021-07-11T04:02:24.333 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:02:24.345 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:24.345 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:24.395 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:24.395 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:24.396 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:24.397 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:02:24.404 INFO:teuthology.orchestra.run.gibba023.stdout:52 2021-07-11T04:02:24.404 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:02:24.405 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:24.405 DEBUG:teuthology.orchestra.run.gibba023:> sudo python3 -c ' 2021-07-11T04:02:24.406 DEBUG:teuthology.orchestra.run.gibba023:> import glob 2021-07-11T04:02:24.406 DEBUG:teuthology.orchestra.run.gibba023:> import re 2021-07-11T04:02:24.406 DEBUG:teuthology.orchestra.run.gibba023:> import os 2021-07-11T04:02:24.407 DEBUG:teuthology.orchestra.run.gibba023:> import subprocess 2021-07-11T04:02:24.407 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:24.408 DEBUG:teuthology.orchestra.run.gibba023:> def find_socket(client_name): 2021-07-11T04:02:24.408 DEBUG:teuthology.orchestra.run.gibba023:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-07-11T04:02:24.409 DEBUG:teuthology.orchestra.run.gibba023:> files = glob.glob(asok_path) 2021-07-11T04:02:24.409 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:24.409 DEBUG:teuthology.orchestra.run.gibba023:> # Given a non-glob path, it better be there 2021-07-11T04:02:24.410 DEBUG:teuthology.orchestra.run.gibba023:> if "*" not in asok_path: 2021-07-11T04:02:24.410 DEBUG:teuthology.orchestra.run.gibba023:> assert(len(files) == 1) 2021-07-11T04:02:24.411 DEBUG:teuthology.orchestra.run.gibba023:> return files[0] 2021-07-11T04:02:24.411 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:24.412 DEBUG:teuthology.orchestra.run.gibba023:> for f in files: 2021-07-11T04:02:24.412 DEBUG:teuthology.orchestra.run.gibba023:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:02:24.413 DEBUG:teuthology.orchestra.run.gibba023:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:02:24.413 DEBUG:teuthology.orchestra.run.gibba023:> return f 2021-07-11T04:02:24.414 DEBUG:teuthology.orchestra.run.gibba023:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:02:24.414 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:24.415 DEBUG:teuthology.orchestra.run.gibba023:> print(find_socket("client.1")) 2021-07-11T04:02:24.415 DEBUG:teuthology.orchestra.run.gibba023:> ' 2021-07-11T04:02:24.483 INFO:teuthology.orchestra.run.gibba023.stdout:/var/run/ceph/ceph-client.1.15280.asok 2021-07-11T04:02:24.485 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15280.asok 2021-07-11T04:02:24.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:24.486 DEBUG:teuthology.orchestra.run.gibba023:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15280.asok status 2021-07-11T04:02:24.533 INFO:teuthology.orchestra.run.gibba023.stdout:{ 2021-07-11T04:02:24.534 INFO:teuthology.orchestra.run.gibba023.stdout: "metadata": { 2021-07-11T04:02:24.534 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:02:24.534 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:02:24.534 INFO:teuthology.orchestra.run.gibba023.stdout: "entity_id": "1", 2021-07-11T04:02:24.535 INFO:teuthology.orchestra.run.gibba023.stdout: "hostname": "gibba023", 2021-07-11T04:02:24.535 INFO:teuthology.orchestra.run.gibba023.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-07-11T04:02:24.535 INFO:teuthology.orchestra.run.gibba023.stdout: "pid": "15280", 2021-07-11T04:02:24.535 INFO:teuthology.orchestra.run.gibba023.stdout: "root": "/" 2021-07-11T04:02:24.535 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:24.536 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_count": 0, 2021-07-11T04:02:24.536 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_pinned_count": 0, 2021-07-11T04:02:24.536 INFO:teuthology.orchestra.run.gibba023.stdout: "id": 5467, 2021-07-11T04:02:24.536 INFO:teuthology.orchestra.run.gibba023.stdout: "inst": { 2021-07-11T04:02:24.537 INFO:teuthology.orchestra.run.gibba023.stdout: "name": { 2021-07-11T04:02:24.537 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "client", 2021-07-11T04:02:24.537 INFO:teuthology.orchestra.run.gibba023.stdout: "num": 5467 2021-07-11T04:02:24.537 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:24.537 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:02:24.538 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:02:24.538 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:02:24.538 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 3937492417 2021-07-11T04:02:24.538 INFO:teuthology.orchestra.run.gibba023.stdout: } 2021-07-11T04:02:24.539 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:24.539 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:02:24.539 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:02:24.539 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:02:24.539 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 3937492417 2021-07-11T04:02:24.540 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:24.540 INFO:teuthology.orchestra.run.gibba023.stdout: "inst_str": "client.5467 172.21.2.123:0/3937492417", 2021-07-11T04:02:24.540 INFO:teuthology.orchestra.run.gibba023.stdout: "addr_str": "172.21.2.123:0/3937492417", 2021-07-11T04:02:24.540 INFO:teuthology.orchestra.run.gibba023.stdout: "inode_count": 1, 2021-07-11T04:02:24.540 INFO:teuthology.orchestra.run.gibba023.stdout: "mds_epoch": 74, 2021-07-11T04:02:24.541 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch": 149, 2021-07-11T04:02:24.541 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:02:24.541 INFO:teuthology.orchestra.run.gibba023.stdout: "blacklisted": false, 2021-07-11T04:02:24.541 INFO:teuthology.orchestra.run.gibba023.stdout: "fs_name": "cephfs" 2021-07-11T04:02:24.541 INFO:teuthology.orchestra.run.gibba023.stdout:} 2021-07-11T04:02:24.543 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:24.543 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:24.544 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:24.551 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:02:24.552 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:24.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:24.553 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:24.568 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-07-11T04:02:24.568 DEBUG:teuthology.orchestra.run.gibba019:> cd /home/ubuntu/cephtest/mnt.0 && sudo getfattr --only-values -n ceph.dir.rctime . 2021-07-11T04:02:24.600 INFO:teuthology.orchestra.run.gibba019.stdout:1625976144.566667075 2021-07-11T04:02:24.601 INFO:tasks.cephfs.test_misc:rctime = 1625976144.566667075 2021-07-11T04:02:24.601 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:02:24.602 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:24.615 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:24.645 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:24.659 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:24.687 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:02:24.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:24.688 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:24.689 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:24.700 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:02:24.702 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:02:24.704 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba019.front.sepia.ceph.com... 2021-07-11T04:02:24.704 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:02:24.705 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:24.758 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15808]: fuse finished with error 0 and tester_r 0 2021-07-11T04:02:24.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:24.760 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:24.761 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:24.765 INFO:teuthology.orchestra.run.gibba019.stdout:ext2/ext3 2021-07-11T04:02:24.765 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:02:24.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:02:24.825 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:24.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:24.835 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:24.836 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:24.847 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:02:24.848 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:24.848 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba023.front.sepia.ceph.com... 2021-07-11T04:02:24.849 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:02:24.849 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:24.911 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15280]: fuse finished with error 0 and tester_r 0 2021-07-11T04:02:24.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:24.914 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:24.915 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:24.924 INFO:teuthology.orchestra.run.gibba023.stdout:ext2/ext3 2021-07-11T04:02:24.925 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:02:25.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:02:25.029 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:25.075 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:02:25.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 Processor -- start 2021-07-11T04:02:25.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 -- start start 2021-07-11T04:02:25.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ff80b1db0 0x7f0ff80b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0ff81483a0 0x7f0ff80b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0ff80a70c0 0x7f0ff8144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff80a90c0 con 0x7f0ff80a70c0 2021-07-11T04:02:25.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff80b0350 con 0x7f0ff81483a0 2021-07-11T04:02:25.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80b05f0 con 0x7f0ff80b1db0 2021-07-11T04:02:25.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffe954700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0ff80a70c0 0x7f0ff8144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:25.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffe954700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f0ff80a70c0 0x7f0ff8144720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51384/0 (socket says 172.21.2.104:51384) 2021-07-11T04:02:25.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffe954700 1 -- 172.21.2.104:0/1111852096 learned_addr learned my addr 172.21.2.104:0/1111852096 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:25.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffe954700 1 -- 172.21.2.104:0/1111852096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ff80b1db0 msgr2=0x7f0ff80b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:25.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffe954700 1 --2- 172.21.2.104:0/1111852096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ff80b1db0 0x7f0ff80b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffe954700 1 -- 172.21.2.104:0/1111852096 >> v2:172.21.2.105:3300/0 conn(0x7f0ff81483a0 msgr2=0x7f0ff80b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:25.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffe954700 1 --2- 172.21.2.104:0/1111852096 >> v2:172.21.2.105:3300/0 conn(0x7f0ff81483a0 0x7f0ff80b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffe954700 1 -- 172.21.2.104:0/1111852096 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff80b0890 con 0x7f0ff80a70c0 2021-07-11T04:02:25.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffe954700 1 --2- 172.21.2.104:0/1111852096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ff80a70c0 0x7f0ff8144720 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f0ff00032f0 tx=0x7f0ff0009290).ready entity=mon.0 client_cookie=22d35601361a064f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:25.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffd151700 1 -- 172.21.2.104:0/1111852096 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ff000b870 con 0x7f0ff80a70c0 2021-07-11T04:02:25.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffd151700 1 -- 172.21.2.104:0/1111852096 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff000b9d0 con 0x7f0ff80a70c0 2021-07-11T04:02:25.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f0ffd151700 1 -- 172.21.2.104:0/1111852096 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0ff000bb40 con 0x7f0ff80a70c0 2021-07-11T04:02:25.136 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 -- 172.21.2.104:0/1111852096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ff80a70c0 msgr2=0x7f0ff8144720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 --2- 172.21.2.104:0/1111852096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ff80a70c0 0x7f0ff8144720 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 rx=0x7f0ff00032f0 tx=0x7f0ff0009290).stop 2021-07-11T04:02:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 -- 172.21.2.104:0/1111852096 shutdown_connections 2021-07-11T04:02:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 --2- 172.21.2.104:0/1111852096 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ff80a70c0 0x7f0ff8144720 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 --2- 172.21.2.104:0/1111852096 >> v2:172.21.2.105:3300/0 conn(0x7f0ff81483a0 0x7f0ff80b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 --2- 172.21.2.104:0/1111852096 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ff80b1db0 0x7f0ff80b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 -- 172.21.2.104:0/1111852096 >> 172.21.2.104:0/1111852096 conn(0x7f0ff805f6b0 msgr2=0x7f0ff805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 -- 172.21.2.104:0/1111852096 shutdown_connections 2021-07-11T04:02:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.132+0000 7f10003b7700 1 -- 172.21.2.104:0/1111852096 wait complete. 2021-07-11T04:02:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f10003b7700 1 Processor -- start 2021-07-11T04:02:25.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f10003b7700 1 -- start start 2021-07-11T04:02:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f10003b7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ff80a70c0 0x7f0ff80ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f10003b7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ff80b1db0 0x7f0ff811d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f10003b7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ff81483a0 0x7f0ff8157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f10003b7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80b4b60 con 0x7f0ff80b1db0 2021-07-11T04:02:25.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f10003b7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8156850 con 0x7f0ff80a70c0 2021-07-11T04:02:25.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f10003b7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8063350 con 0x7f0ff81483a0 2021-07-11T04:02:25.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0ffd952700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ff80b1db0 0x7f0ff811d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:25.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0ffd952700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ff80b1db0 0x7f0ff811d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51390/0 (socket says 172.21.2.104:51390) 2021-07-11T04:02:25.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0ffd952700 1 -- 172.21.2.104:0/1498237252 learned_addr learned my addr 172.21.2.104:0/1498237252 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:25.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0ffd952700 1 -- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ff81483a0 msgr2=0x7f0ff8157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0ffd952700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ff81483a0 0x7f0ff8157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0ffd952700 1 -- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ff80a70c0 msgr2=0x7f0ff80ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0ffd952700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ff80a70c0 0x7f0ff80ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0ffd952700 1 -- 172.21.2.104:0/1498237252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0009530 con 0x7f0ff80b1db0 2021-07-11T04:02:25.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0ffd952700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ff80b1db0 0x7f0ff811d280 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 rx=0x7f0fe8002f70 tx=0x7f0fe8006da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:25.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0fe77fe700 1 -- 172.21.2.104:0/1498237252 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fe8009e00 con 0x7f0ff80b1db0 2021-07-11T04:02:25.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0fe77fe700 1 -- 172.21.2.104:0/1498237252 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe800f510 con 0x7f0ff80b1db0 2021-07-11T04:02:25.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0fe77fe700 1 -- 172.21.2.104:0/1498237252 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0fe800f680 con 0x7f0ff80b1db0 2021-07-11T04:02:25.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f10003b7700 1 -- 172.21.2.104:0/1498237252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff80635f0 con 0x7f0ff80b1db0 2021-07-11T04:02:25.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f10003b7700 1 -- 172.21.2.104:0/1498237252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff8065f20 con 0x7f0ff80b1db0 2021-07-11T04:02:25.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f10003b7700 1 -- 172.21.2.104:0/1498237252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff80631a0 con 0x7f0ff80b1db0 2021-07-11T04:02:25.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0fe77fe700 1 -- 172.21.2.104:0/1498237252 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0fe8017020 con 0x7f0ff80b1db0 2021-07-11T04:02:25.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0fe77fe700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fd8053300 0x7f0fd80557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0fe77fe700 1 -- 172.21.2.104:0/1498237252 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f0fe806a760 con 0x7f0ff80b1db0 2021-07-11T04:02:25.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.136+0000 7f0ffe153700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fd8053300 0x7f0fd80557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:25.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.140+0000 7f0ffe153700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fd8053300 0x7f0fd80557c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f0ff4002f70 tx=0x7f0ff4006a90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:25.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.140+0000 7f0fe77fe700 1 -- 172.21.2.104:0/1498237252 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f0fe8041990 con 0x7f0ff80b1db0 2021-07-11T04:02:25.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.220+0000 7f10003b7700 1 -- 172.21.2.104:0/1498237252 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f0ff8057350 con 0x7f0ff80b1db0 2021-07-11T04:02:25.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.220+0000 7f0fe77fe700 1 -- 172.21.2.104:0/1498237252 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v149) v1 ==== 81+0+33659 (secure 0 0 0) 0x7f0fe8028390 con 0x7f0ff80b1db0 2021-07-11T04:02:25.222 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:25.222 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:02:25.222 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 149, 2021-07-11T04:02:25.223 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:02:25.223 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:02:25.223 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:02:16.677257+0000", 2021-07-11T04:02:25.223 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:02:25.224 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:02:25.224 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:02:25.224 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:02:25.224 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:02:25.224 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:02:25.225 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:02:25.225 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:02:25.225 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:02:25.225 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:25.225 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:02:25.226 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:02:25.226 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:02:25.226 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:02:25.226 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:02:25.227 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 28, 2021-07-11T04:02:25.227 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:02:25.227 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:02:25.227 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:02:25.228 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:02:25.228 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:02:25.229 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.230 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:02:25.230 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:02:25.230 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:02:25.231 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:25.231 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:25.231 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:25.231 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:25.232 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:25.232 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:25.232 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:25.232 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:25.232 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:02:25.233 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:02:25.233 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:02:25.233 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:02:25.233 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:02:25.233 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:25.234 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:25.234 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:25.234 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:25.234 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:25.235 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:25.235 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:25.235 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.235 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:02:25.235 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:25.236 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:25.236 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:25.236 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:25.236 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:25.236 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:25.237 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:25.237 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:25.237 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:25.237 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:25.237 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:25.238 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:25.238 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:25.239 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:25.240 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:25.240 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:25.241 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:25.241 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:25.241 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:25.241 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:25.241 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:25.242 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:25.242 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:25.242 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:25.242 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:25.243 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:25.243 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.243 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:25.243 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:25.243 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:25.244 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:25.244 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:25.244 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:25.244 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:25.244 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:25.245 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:25.245 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:25.245 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:25.245 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:02:25.245 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:02:25.246 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.246 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:25.246 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:02:25.246 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.247 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.247 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.247 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:02:25.247 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:02:25.247 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:02:25.248 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:25.248 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:25.248 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:25.248 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:25.249 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:25.250 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:25.250 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:25.251 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:25.251 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:02:25.251 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:02:25.251 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:02:25.252 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:02:25.252 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:02:25.252 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:25.252 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:25.253 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:25.253 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:25.253 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:25.253 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:25.254 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:25.254 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.254 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:02:25.254 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:25.255 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:25.255 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:25.255 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:25.255 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:25.255 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:25.256 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:25.256 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:25.256 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:25.256 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:25.257 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:25.257 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:25.257 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:25.257 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:25.258 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:25.258 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:25.258 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:25.258 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:25.258 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:25.259 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:25.259 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:25.259 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:25.259 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:25.260 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:25.260 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:25.260 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:25.260 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.260 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:25.261 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:25.261 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:25.261 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:25.261 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:25.262 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:25.262 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:25.262 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:25.262 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:25.263 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:25.263 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:25.263 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:02:25.263 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:25.264 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:02:25.264 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.264 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.264 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.265 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 27, 2021-07-11T04:02:25.265 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T04:02:25.265 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:02:14.500311+0000", 2021-07-11T04:02:25.266 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:25.266 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:25.267 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:25.267 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:25.267 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:25.267 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:25.268 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:25.268 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:25.268 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:02:25.268 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:02:25.269 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:02:25.269 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:02:25.270 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:02:25.270 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:25.270 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:25.270 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:25.270 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:25.271 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:25.271 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:25.271 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:25.271 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.272 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "149", 2021-07-11T04:02:25.272 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:25.272 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:25.272 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:25.273 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:25.273 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:25.273 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:25.273 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:25.274 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:25.274 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:25.274 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:25.274 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:25.275 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:25.275 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:25.275 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:25.275 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:25.276 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:25.276 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:25.276 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:25.276 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:25.276 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:25.277 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:25.277 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:25.277 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:25.277 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:25.277 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:25.278 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:25.278 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.278 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:25.278 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:25.279 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:25.279 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:25.279 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:25.279 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:25.279 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:25.280 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:25.280 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:25.280 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:25.280 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:25.280 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:02:25.281 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T04:02:25.281 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T04:02:25.281 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T04:02:25.281 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.282 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:25.282 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:02:25.282 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T04:02:25.282 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.282 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.283 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.283 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.283 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 28, 2021-07-11T04:02:25.284 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T04:02:25.284 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:02:15.641534+0000", 2021-07-11T04:02:25.284 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:25.284 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:25.284 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:25.285 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:25.285 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:25.285 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:25.285 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:25.286 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:25.286 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:02:25.286 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:02:25.286 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:02:25.286 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:02:25.287 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:02:25.287 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:25.287 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:25.287 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:25.288 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:25.288 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:25.288 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:25.288 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:25.288 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.289 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "149", 2021-07-11T04:02:25.289 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:25.289 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:25.289 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:25.290 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:25.290 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:25.290 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:25.290 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:25.290 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:25.291 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:25.291 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:25.291 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:25.291 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:25.292 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:25.292 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:25.292 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:25.292 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:25.292 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:25.293 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:25.293 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:25.293 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:25.294 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:25.294 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:25.294 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:25.294 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:25.294 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:25.295 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:25.295 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.295 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:25.295 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:25.296 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:25.296 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:25.296 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:25.296 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:25.297 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:25.297 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:25.297 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:25.297 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:25.298 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:25.298 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:02:25.298 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:25.298 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:02:25.298 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T04:02:25.299 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.299 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.299 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.299 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:25.299 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:02:25.300 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.300 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:02:25.300 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:02:25.300 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:25.301 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:25.301 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:25.301 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:25.301 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:25.301 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:25.302 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:25.302 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 146, 2021-07-11T04:02:25.302 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:25.302 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:25.303 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:25.303 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.303 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.303 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.304 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:02:25.304 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:25.304 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.304 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.305 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.305 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:02:25.305 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:25.305 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.305 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.306 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.306 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:25.306 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.306 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.307 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.307 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:02:25.307 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:25.307 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.308 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.308 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.308 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:02:25.308 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:25.308 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.309 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.309 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.309 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:25.309 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.309 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.310 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.310 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:02:25.310 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:25.310 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.311 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.311 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.311 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:02:25.311 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:25.311 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.312 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.312 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.312 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:25.312 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.313 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.313 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.313 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:02:25.313 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:25.314 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.314 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.314 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.314 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:02:25.315 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:25.315 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.315 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.315 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.316 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:02:25.316 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:02:25.316 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:02:25.316 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:02:25.317 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:25.317 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:25.317 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:25.317 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.318 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.318 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.318 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:02:25.318 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:02:25.319 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:25.319 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:25.319 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:25.319 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:25.319 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:25.320 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:25.320 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:25.320 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 146, 2021-07-11T04:02:25.320 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:25.321 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:25.321 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:25.321 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.321 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.321 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.322 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:02:25.322 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:25.322 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.322 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.322 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.323 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:02:25.323 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:25.323 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.323 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.324 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.325 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:25.325 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.325 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.326 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.326 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:02:25.326 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:25.326 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.326 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.327 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.327 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:02:25.327 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:25.327 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.328 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.328 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.328 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:25.328 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.328 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.329 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.329 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:02:25.329 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:25.329 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.330 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.330 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.330 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:02:25.330 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:25.331 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.331 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.331 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.331 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:25.331 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.332 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.332 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.332 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:02:25.332 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:25.333 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.333 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.333 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.334 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:02:25.334 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:25.334 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.334 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.335 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.335 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:02:25.335 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:02:25.335 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:02:25.336 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:02:25.336 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:25.336 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:25.336 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:25.336 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.337 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.337 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.337 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:02:25.337 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:02:25.338 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:25.338 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:25.338 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:25.338 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:25.339 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:25.339 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:25.339 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:25.339 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 145, 2021-07-11T04:02:25.339 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:25.340 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:25.340 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:25.340 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.340 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.341 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.341 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:02:25.341 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:25.341 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.341 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.342 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.342 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:02:25.342 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:25.342 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.343 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.343 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.343 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:25.343 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.344 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.344 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.344 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:02:25.345 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:25.345 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.345 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.345 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.345 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:02:25.346 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:25.346 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.346 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.346 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.347 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:25.347 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.347 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.347 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.347 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:02:25.348 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:25.348 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.348 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.348 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.349 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:02:25.349 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:25.349 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.349 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.349 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.350 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:25.350 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.350 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.350 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.351 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:02:25.351 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:25.351 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.351 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.352 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.352 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:02:25.352 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:25.352 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.352 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.353 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.353 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:02:25.353 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:02:25.353 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:02:25.354 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:02:25.354 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:25.354 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:25.355 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:25.355 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.355 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.355 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.356 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:02:25.356 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:02:25.356 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:25.356 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:25.357 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:25.357 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:25.357 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:25.357 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:25.357 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:25.358 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 136, 2021-07-11T04:02:25.358 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:25.358 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:25.358 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:25.359 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.359 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.359 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.359 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:02:25.359 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:25.360 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.360 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.360 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.360 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:02:25.361 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:25.361 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.361 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.361 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.361 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:25.362 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.362 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.362 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.362 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:02:25.363 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:25.363 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.363 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.363 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.363 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:02:25.364 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:25.364 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.364 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.365 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.365 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:25.365 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.365 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.366 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.366 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:02:25.366 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:25.366 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.367 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.367 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.367 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:02:25.367 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:25.367 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.368 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.368 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.368 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:25.368 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.369 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.369 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.369 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:02:25.369 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:25.369 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.370 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.370 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.370 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:02:25.370 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:25.371 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.371 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.371 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.371 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:02:25.371 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:02:25.372 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:02:25.372 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:02:25.372 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:25.372 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:25.373 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:25.373 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.373 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.373 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.374 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:02:25.374 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:02:25.374 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:25.374 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:25.375 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:25.375 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:25.375 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:25.375 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:25.376 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:25.376 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 145, 2021-07-11T04:02:25.376 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:25.376 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:25.377 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:25.377 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.377 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.377 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.378 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:02:25.378 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:25.378 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.378 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.379 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.379 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:02:25.379 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:25.379 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.379 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.380 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.380 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:25.380 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.380 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.381 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.381 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:02:25.381 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:25.381 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.382 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.382 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.382 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:02:25.382 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:25.383 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.383 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.383 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.383 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:25.383 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.384 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.384 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.384 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:02:25.384 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:25.385 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.385 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.385 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.386 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:02:25.386 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:25.386 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.386 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.387 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.387 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:25.387 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.387 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.388 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.388 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:02:25.388 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:25.388 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.388 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.389 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.389 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:02:25.389 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:25.389 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.390 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.390 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.390 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:02:25.390 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:02:25.390 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:02:25.391 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:02:25.391 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:25.391 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:25.391 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:25.392 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.392 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.392 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.392 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:02:25.392 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:02:25.393 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:25.393 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:25.393 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:25.393 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:25.394 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:25.394 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:25.394 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:25.394 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 146, 2021-07-11T04:02:25.394 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:25.395 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:25.395 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:25.397 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.397 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.397 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.397 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:02:25.398 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:25.398 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.398 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.398 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.398 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:02:25.399 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:25.399 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.399 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.399 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.400 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:25.400 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.400 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.400 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.400 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:02:25.401 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:25.401 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.401 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.401 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.402 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:02:25.402 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:25.402 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.402 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.403 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.403 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:25.403 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.403 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.403 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.404 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:02:25.404 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:25.404 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.404 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.405 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.405 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:02:25.405 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:25.406 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.406 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.406 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.406 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:25.409 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.409 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.409 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.410 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:02:25.410 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:25.410 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.410 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.411 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.411 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:02:25.411 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:25.411 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.412 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.412 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.412 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:02:25.412 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:02:25.413 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:02:25.413 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:02:25.413 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:25.413 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:25.414 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:25.414 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.414 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.414 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.415 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:02:25.415 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:02:25.415 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:25.415 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:25.415 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:25.416 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:25.416 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:25.416 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:25.416 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:25.417 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 146, 2021-07-11T04:02:25.417 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:25.417 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:25.417 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:25.417 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.418 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.418 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.418 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:02:25.418 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:25.418 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.419 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.419 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.419 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:02:25.419 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:25.420 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.420 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.420 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.420 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:25.420 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.421 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.421 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.421 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:02:25.421 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:25.422 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.422 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.422 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.422 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:02:25.423 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:25.423 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.423 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.423 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.424 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:25.424 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.424 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.424 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.424 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:02:25.425 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:25.425 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.425 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.425 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.426 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:02:25.426 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:25.426 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.426 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.426 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.427 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:25.427 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.427 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.427 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.427 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:02:25.428 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:25.428 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.428 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.428 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.429 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:02:25.429 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:25.429 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.429 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.429 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.430 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:02:25.430 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:02:25.430 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:02:25.430 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:02:25.431 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:25.431 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:25.431 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:25.431 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.431 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.432 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.432 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:02:25.432 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:02:25.432 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:25.433 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:25.433 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:25.433 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:25.433 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:25.434 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:25.434 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:25.434 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 146, 2021-07-11T04:02:25.434 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:25.435 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:25.435 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:25.435 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.435 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.436 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.436 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:02:25.436 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:25.436 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.436 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.437 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.437 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:02:25.437 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:25.437 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.438 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.438 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.438 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:25.438 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.439 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.439 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.439 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:02:25.439 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:25.439 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.440 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.440 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.440 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:02:25.440 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:25.441 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.441 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.441 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.441 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:25.441 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.442 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.442 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.442 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:02:25.442 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:25.443 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.443 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.443 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.443 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:02:25.444 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:25.444 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.444 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.444 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.444 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:25.445 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:25.445 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.445 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:25.445 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:02:25.446 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:25.446 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.446 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.446 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:25.446 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:02:25.447 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:25.447 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.447 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.447 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.448 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:02:25.448 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:02:25.448 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:02:25.448 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:02:25.448 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:25.449 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:25.449 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:25.449 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:25.449 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.450 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:25.450 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:02:25.450 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.450 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:02:25.450 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:25.451 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:25.451 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:25.451 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:25.451 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:25.451 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:02:25.452 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:25.452 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.452 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.453 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:02:25.453 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:25.453 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:25.453 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:25.453 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:25.454 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:25.454 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:02:25.454 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:25.454 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.455 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.455 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:02:25.455 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:25.455 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:25.455 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:25.456 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:25.456 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:25.456 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:02:25.456 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:25.456 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.457 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.457 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:02:25.457 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:25.457 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:25.458 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:25.458 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:25.458 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:25.458 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:02:25.459 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:25.459 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.459 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.459 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:02:25.459 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:25.460 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:25.460 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:25.460 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:25.460 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:25.460 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:02:25.461 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:25.461 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.461 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.461 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:02:25.462 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:25.462 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:25.462 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:25.462 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:25.463 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:25.463 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:02:25.463 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:25.463 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.464 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.464 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:02:25.464 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:25.464 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:25.464 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:25.465 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:25.465 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:25.465 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:02:25.465 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:25.465 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.466 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:25.466 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:02:25.466 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:25.466 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:25.467 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:25.467 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:25.467 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:25.467 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:02:25.467 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:25.468 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.468 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:25.468 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:02:25.468 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:02:25.469 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:02:25.469 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:02:25.469 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": {}, 2021-07-11T04:02:25.469 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:02:25.469 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:02:25.470 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:02:25.470 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:02:25.470 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:02:25.470 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:02:25.471 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:02:25.471 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:02:25.471 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:25.471 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:25.472 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:02:25.472 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:02:25.472 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:02:25.472 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:02:25.473 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:02:25.473 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:02:25.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 -- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fd8053300 msgr2=0x7f0fd80557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fd8053300 0x7f0fd80557c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f0ff4002f70 tx=0x7f0ff4006a90).stop 2021-07-11T04:02:25.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 -- 172.21.2.104:0/1498237252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ff80b1db0 msgr2=0x7f0ff811d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ff80b1db0 0x7f0ff811d280 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 rx=0x7f0fe8002f70 tx=0x7f0fe8006da0).stop 2021-07-11T04:02:25.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 -- 172.21.2.104:0/1498237252 shutdown_connections 2021-07-11T04:02:25.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0ff81483a0 0x7f0ff8157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0fd8053300 0x7f0fd80557c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:25.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0ff80b1db0 0x7f0ff811d280 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:25.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 --2- 172.21.2.104:0/1498237252 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0ff80a70c0 0x7f0ff80ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 -- 172.21.2.104:0/1498237252 >> 172.21.2.104:0/1498237252 conn(0x7f0ff805f6b0 msgr2=0x7f0ff8146f40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:25.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 -- 172.21.2.104:0/1498237252 shutdown_connections 2021-07-11T04:02:25.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.224+0000 7f10003b7700 1 -- 172.21.2.104:0/1498237252 wait complete. 2021-07-11T04:02:25.478 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:25.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f6930993700 1 Processor -- start 2021-07-11T04:02:25.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f6930993700 1 -- start start 2021-07-11T04:02:25.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f6930993700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f692c067180 0x7f692c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f6930993700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f692c0b1970 0x7f692c05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f6930993700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692c0a7a90 con 0x7f692c0a76b0 2021-07-11T04:02:25.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f6930993700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692c147620 con 0x7f692c0b1970 2021-07-11T04:02:25.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f6930993700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c1478c0 con 0x7f692c067180 2021-07-11T04:02:25.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f692ad9d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f692c0b1970 0x7f692c05fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:25.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f6929d9b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f692c067180 0x7f692c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:25.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f692ad9d700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f692c0b1970 0x7f692c05fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:58958/0 (socket says 172.21.2.104:58958) 2021-07-11T04:02:25.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f692ad9d700 1 -- 172.21.2.104:0/415232241 learned_addr learned my addr 172.21.2.104:0/415232241 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:25.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.540+0000 7f692a59c700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f692c0a76b0 0x7f692c066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:47532/0 (socket says 172.21.2.104:47532) 2021-07-11T04:02:25.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 -- 172.21.2.104:0/415232241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f692c067180 msgr2=0x7f692c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:25.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 --2- 172.21.2.104:0/415232241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f692c067180 0x7f692c05f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:25.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 -- 172.21.2.104:0/415232241 >> v1:172.21.2.105:6789/0 conn(0x7f692c0a76b0 legacy=0x7f692c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 -- 172.21.2.104:0/415232241 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c147b60 con 0x7f692c0b1970 2021-07-11T04:02:25.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 --2- 172.21.2.104:0/415232241 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f692c0b1970 0x7f692c05fba0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f69240032f0 tx=0x7f6924009290).ready entity=mon.1 client_cookie=b4e05d069877fe1a server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:25.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692959a700 1 -- 172.21.2.104:0/415232241 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f692400b8a0 con 0x7f692c0b1970 2021-07-11T04:02:25.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692959a700 1 -- 172.21.2.104:0/415232241 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692400ba00 con 0x7f692c0b1970 2021-07-11T04:02:25.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692959a700 1 -- 172.21.2.104:0/415232241 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f692400bb70 con 0x7f692c0b1970 2021-07-11T04:02:25.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 -- 172.21.2.104:0/415232241 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f692c0b1970 msgr2=0x7f692c05fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 --2- 172.21.2.104:0/415232241 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f692c0b1970 0x7f692c05fba0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f69240032f0 tx=0x7f6924009290).stop 2021-07-11T04:02:25.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 -- 172.21.2.104:0/415232241 shutdown_connections 2021-07-11T04:02:25.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 --2- 172.21.2.104:0/415232241 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f692c0b1970 0x7f692c05fba0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:25.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 --2- 172.21.2.104:0/415232241 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f692c067180 0x7f692c05f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:25.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 -- 172.21.2.104:0/415232241 >> 172.21.2.104:0/415232241 conn(0x7f692c0b53e0 msgr2=0x7f692c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:25.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 -- 172.21.2.104:0/415232241 shutdown_connections 2021-07-11T04:02:25.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 -- 172.21.2.104:0/415232241 wait complete. 2021-07-11T04:02:25.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 Processor -- start 2021-07-11T04:02:25.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 -- start start 2021-07-11T04:02:25.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f692c067180 0x7f692c149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f692c0a76b0 0x7f692c156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f692c0b1970 0x7f692c156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0ac520 con 0x7f692c0b1970 2021-07-11T04:02:25.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0ac750 con 0x7f692c0a76b0 2021-07-11T04:02:25.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0ac9f0 con 0x7f692c067180 2021-07-11T04:02:25.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f692c0b1970 0x7f692c156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:25.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f692c0b1970 0x7f692c156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51400/0 (socket says 172.21.2.104:51400) 2021-07-11T04:02:25.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 -- 172.21.2.104:0/3105525041 learned_addr learned my addr 172.21.2.104:0/3105525041 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:25.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 -- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f692c067180 msgr2=0x7f692c149b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f692c067180 0x7f692c149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 -- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f692c0a76b0 msgr2=0x7f692c156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f692c0a76b0 0x7f692c156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 -- 172.21.2.104:0/3105525041 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924009530 con 0x7f692c0b1970 2021-07-11T04:02:25.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f692ad9d700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f692c0b1970 0x7f692c156f20 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7f69240032f0 tx=0x7f6924016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:25.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f69237fe700 1 -- 172.21.2.104:0/3105525041 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6924011ca0 con 0x7f692c0b1970 2021-07-11T04:02:25.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f69237fe700 1 -- 172.21.2.104:0/3105525041 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6924011e00 con 0x7f692c0b1970 2021-07-11T04:02:25.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 -- 172.21.2.104:0/3105525041 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c0acc90 con 0x7f692c0b1970 2021-07-11T04:02:25.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f6930993700 1 -- 172.21.2.104:0/3105525041 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c157410 con 0x7f692c0b1970 2021-07-11T04:02:25.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.544+0000 7f69237fe700 1 -- 172.21.2.104:0/3105525041 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6924020670 con 0x7f692c0b1970 2021-07-11T04:02:25.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.548+0000 7f6930993700 1 -- 172.21.2.104:0/3105525041 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692c057350 con 0x7f692c0b1970 2021-07-11T04:02:25.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.548+0000 7f69237fe700 1 -- 172.21.2.104:0/3105525041 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f692400f070 con 0x7f692c0b1970 2021-07-11T04:02:25.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.548+0000 7f69237fe700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6910053300 0x7f69100557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.548+0000 7f69237fe700 1 -- 172.21.2.104:0/3105525041 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f692406c0f0 con 0x7f692c0b1970 2021-07-11T04:02:25.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.548+0000 7f692a59c700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6910053300 0x7f69100557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:25.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.548+0000 7f692a59c700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6910053300 0x7f69100557c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f692c148420 tx=0x7f691c0077c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:25.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.548+0000 7f69237fe700 1 -- 172.21.2.104:0/3105525041 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f69240433d0 con 0x7f692c0b1970 2021-07-11T04:02:25.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.656+0000 7f6930993700 1 -- 172.21.2.104:0/3105525041 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f692c0698e0 con 0x7f692c0b1970 2021-07-11T04:02:25.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f69237fe700 1 -- 172.21.2.104:0/3105525041 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+2486 (secure 0 0 0) 0x7f6924015070 con 0x7f692c0b1970 2021-07-11T04:02:25.659 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:25.659 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":74,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/696025904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":696025904},{"type":"v1","addr":"172.21.2.104:6833","nonce":696025904}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8412,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/438569403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":438569403},{"type":"v1","addr":"172.21.2.104:6835","nonce":438569403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:16.677336+0000","modified":"2021-07-11T04:02:17.681052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8415},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8415":{"gid":8415,"name":"b","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.2.105:6835/130583442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":130583442},{"type":"v1","addr":"172.21.2.105:6835","nonce":130583442}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-07-11T04:02:25.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 -- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6910053300 msgr2=0x7f69100557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6910053300 0x7f69100557c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f692c148420 tx=0x7f691c0077c0).stop 2021-07-11T04:02:25.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 -- 172.21.2.104:0/3105525041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f692c0b1970 msgr2=0x7f692c156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f692c0b1970 0x7f692c156f20 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 rx=0x7f69240032f0 tx=0x7f6924016000).stop 2021-07-11T04:02:25.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 -- 172.21.2.104:0/3105525041 shutdown_connections 2021-07-11T04:02:25.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6910053300 0x7f69100557c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:25.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f692c0b1970 0x7f692c156f20 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:25.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f692c0a76b0 0x7f692c156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 --2- 172.21.2.104:0/3105525041 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f692c067180 0x7f692c149b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 -- 172.21.2.104:0/3105525041 >> 172.21.2.104:0/3105525041 conn(0x7f692c0b53e0 msgr2=0x7f692c14df10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:25.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 -- 172.21.2.104:0/3105525041 shutdown_connections 2021-07-11T04:02:25.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.660+0000 7f6930993700 1 -- 172.21.2.104:0/3105525041 wait complete. 2021-07-11T04:02:25.663 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 74 2021-07-11T04:02:25.666 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T04:02:25.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe910b1a700 1 Processor -- start 2021-07-11T04:02:25.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe910b1a700 1 -- start start 2021-07-11T04:02:25.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe910b1a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe90c067180 0x7fe90c05f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe910b1a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe90c0b1970 0x7fe90c05fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe910b1a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c0a7a90 con 0x7fe90c0a76b0 2021-07-11T04:02:25.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe910b1a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90c147620 con 0x7fe90c0b1970 2021-07-11T04:02:25.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe910b1a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c1478c0 con 0x7fe90c067180 2021-07-11T04:02:25.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe90ad9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe90c0b1970 0x7fe90c05fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:25.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe90ad9d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe90c0b1970 0x7fe90c05fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51410/0 (socket says 172.21.2.104:51410) 2021-07-11T04:02:25.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe90ad9d700 1 -- 172.21.2.104:0/1460099614 learned_addr learned my addr 172.21.2.104:0/1460099614 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:25.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe90ad9d700 1 -- 172.21.2.104:0/1460099614 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe90c067180 msgr2=0x7fe90c05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:25.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe90ad9d700 1 --2- 172.21.2.104:0/1460099614 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe90c067180 0x7fe90c05f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe90ad9d700 1 -- 172.21.2.104:0/1460099614 >> v1:172.21.2.104:6789/0 conn(0x7fe90c0a76b0 legacy=0x7fe90c066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe90ad9d700 1 -- 172.21.2.104:0/1460099614 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c147b60 con 0x7fe90c0b1970 2021-07-11T04:02:25.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe90ad9d700 1 --2- 172.21.2.104:0/1460099614 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe90c0b1970 0x7fe90c05fba0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7fe9040032f0 tx=0x7fe904009290).ready entity=mon.0 client_cookie=b2ab018ca73c1c92 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:25.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe90959a700 1 -- 172.21.2.104:0/1460099614 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe90400b8a0 con 0x7fe90c0b1970 2021-07-11T04:02:25.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe90959a700 1 -- 172.21.2.104:0/1460099614 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe90400ba00 con 0x7fe90c0b1970 2021-07-11T04:02:25.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe90959a700 1 -- 172.21.2.104:0/1460099614 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe90400bb70 con 0x7fe90c0b1970 2021-07-11T04:02:25.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe910b1a700 1 -- 172.21.2.104:0/1460099614 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe90c0b1970 msgr2=0x7fe90c05fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.716+0000 7fe910b1a700 1 --2- 172.21.2.104:0/1460099614 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe90c0b1970 0x7fe90c05fba0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 rx=0x7fe9040032f0 tx=0x7fe904009290).stop 2021-07-11T04:02:25.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 -- 172.21.2.104:0/1460099614 shutdown_connections 2021-07-11T04:02:25.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 --2- 172.21.2.104:0/1460099614 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe90c0b1970 0x7fe90c05fba0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:25.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 --2- 172.21.2.104:0/1460099614 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe90c067180 0x7fe90c05f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 -- 172.21.2.104:0/1460099614 >> 172.21.2.104:0/1460099614 conn(0x7fe90c0b53e0 msgr2=0x7fe90c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:25.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 -- 172.21.2.104:0/1460099614 shutdown_connections 2021-07-11T04:02:25.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 -- 172.21.2.104:0/1460099614 wait complete. 2021-07-11T04:02:25.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 Processor -- start 2021-07-11T04:02:25.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 -- start start 2021-07-11T04:02:25.727 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe90c067180 0x7fe90c146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe90c0a76b0 0x7fe90c0af090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe90c0b1970 0x7fe90c0b4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0a9650 con 0x7fe90c0a76b0 2021-07-11T04:02:25.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0ab6f0 con 0x7fe90c0b1970 2021-07-11T04:02:25.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0ab990 con 0x7fe90c067180 2021-07-11T04:02:25.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe909d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe90c0a76b0 0x7fe90c0af090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:25.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe909d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe90c0a76b0 0x7fe90c0af090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51418/0 (socket says 172.21.2.104:51418) 2021-07-11T04:02:25.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe909d9b700 1 -- 172.21.2.104:0/3639639346 learned_addr learned my addr 172.21.2.104:0/3639639346 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:25.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe909d9b700 1 -- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe90c067180 msgr2=0x7fe90c146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe909d9b700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe90c067180 0x7fe90c146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe909d9b700 1 -- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe90c0b1970 msgr2=0x7fe90c0b4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:25.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe909d9b700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe90c0b1970 0x7fe90c0b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:25.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe909d9b700 1 -- 172.21.2.104:0/3639639346 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe904009530 con 0x7fe90c0a76b0 2021-07-11T04:02:25.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe909d9b700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe90c0a76b0 0x7fe90c0af090 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7fe8fc002f70 tx=0x7fe8fc006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:25.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe9037fe700 1 -- 172.21.2.104:0/3639639346 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8fc013460 con 0x7fe90c0a76b0 2021-07-11T04:02:25.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe9037fe700 1 -- 172.21.2.104:0/3639639346 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc0135c0 con 0x7fe90c0a76b0 2021-07-11T04:02:25.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 -- 172.21.2.104:0/3639639346 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90c0abc30 con 0x7fe90c0a76b0 2021-07-11T04:02:25.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe9037fe700 1 -- 172.21.2.104:0/3639639346 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe8fc013920 con 0x7fe90c0a76b0 2021-07-11T04:02:25.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 -- 172.21.2.104:0/3639639346 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90c0ae0f0 con 0x7fe90c0a76b0 2021-07-11T04:02:25.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe910b1a700 1 -- 172.21.2.104:0/3639639346 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90c057350 con 0x7fe90c0a76b0 2021-07-11T04:02:25.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe9037fe700 1 -- 172.21.2.104:0/3639639346 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe8fc01a060 con 0x7fe90c0a76b0 2021-07-11T04:02:25.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.720+0000 7fe9037fe700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8f00531f0 0x7fe8f00556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:25.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.724+0000 7fe9037fe700 1 -- 172.21.2.104:0/3639639346 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe8fc06aef0 con 0x7fe90c0a76b0 2021-07-11T04:02:25.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.724+0000 7fe90a59c700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8f00531f0 0x7fe8f00556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:25.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.724+0000 7fe90a59c700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8f00531f0 0x7fe8f00556b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fe90c148420 tx=0x7fe8f40077d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:25.733 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.724+0000 7fe9037fe700 1 -- 172.21.2.104:0/3639639346 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe8fc042230 con 0x7fe90c0a76b0 2021-07-11T04:02:25.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:25.832+0000 7fe910b1a700 1 -- 172.21.2.104:0/3639639346 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe90c0698e0 con 0x7fe90c0a76b0 2021-07-11T04:02:26.387 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T04:02:26.388+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T04:02:26.397 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.396+0000 7fe9037fe700 1 -- 172.21.2.104:0/3639639346 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe8fc048090 con 0x7fe90c0a76b0 2021-07-11T04:02:26.399 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 -- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8f00531f0 msgr2=0x7fe8f00556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8f00531f0 0x7fe8f00556b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fe90c148420 tx=0x7fe8f40077d0).stop 2021-07-11T04:02:26.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 -- 172.21.2.104:0/3639639346 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe90c0a76b0 msgr2=0x7fe90c0af090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe90c0a76b0 0x7fe90c0af090 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 rx=0x7fe8fc002f70 tx=0x7fe8fc006ec0).stop 2021-07-11T04:02:26.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 -- 172.21.2.104:0/3639639346 shutdown_connections 2021-07-11T04:02:26.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe90c0b1970 0x7fe90c0b4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:26.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe8f00531f0 0x7fe8f00556b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:26.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe90c0a76b0 0x7fe90c0af090 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:26.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 --2- 172.21.2.104:0/3639639346 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe90c067180 0x7fe90c146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:26.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 -- 172.21.2.104:0/3639639346 >> 172.21.2.104:0/3639639346 conn(0x7fe90c0b53e0 msgr2=0x7fe90c0604e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:26.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 -- 172.21.2.104:0/3639639346 shutdown_connections 2021-07-11T04:02:26.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.400+0000 7fe910b1a700 1 -- 172.21.2.104:0/3639639346 wait complete. 2021-07-11T04:02:26.403 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T04:02:26.406 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:26.411 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T04:02:26.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 Processor -- start 2021-07-11T04:02:26.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 -- start start 2021-07-11T04:02:26.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f3c840b7810 0x7f3c841445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c840b31d0 con 0x7f3c840b1170 2021-07-11T04:02:26.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c840b7bf0 con 0x7f3c84144a90 2021-07-11T04:02:26.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c84144f20 con 0x7f3c840b7810 2021-07-11T04:02:26.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c827fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f3c840b1170 0x7f3c84064870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48194/0 (socket says 172.21.2.104:48194) 2021-07-11T04:02:26.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c827fc700 1 -- 172.21.2.104:0/2549906472 learned_addr learned my addr 172.21.2.104:0/2549906472 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:26.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3999080580 0 0) 0x7f3c840b31d0 con 0x7f3c840b1170 2021-07-11T04:02:26.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c68003260 con 0x7f3c840b1170 2021-07-11T04:02:26.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c82ffd700 1 --2- 172.21.2.104:0/2549906472 >> v2:172.21.2.105:3300/0 conn(0x7f3c840b7810 0x7f3c841445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:26.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1988522491 0 0) 0x7f3c68003260 con 0x7f3c840b1170 2021-07-11T04:02:26.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 >> v2:172.21.2.105:3300/0 conn(0x7f3c840b7810 msgr2=0x7f3c841445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:26.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 --2- 172.21.2.104:0/2549906472 >> v2:172.21.2.105:3300/0 conn(0x7f3c840b7810 0x7f3c841445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:26.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 >> v1:172.21.2.105:6789/0 conn(0x7f3c84144a90 legacy=0x7f3c84064f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:26.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c840675c0 con 0x7f3c840b1170 2021-07-11T04:02:26.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f3c78001660 con 0x7f3c840b1170 2021-07-11T04:02:26.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 >> v1:172.21.2.104:6789/0 conn(0x7f3c840b1170 legacy=0x7f3c84064870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 --2- 172.21.2.104:0/2549906472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 0x7f3c68003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 --2- 172.21.2.104:0/2549906472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3c68004500 0x7f3c68004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 --2- 172.21.2.104:0/2549906472 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c68004ec0 0x7f3c6800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c84144f20 con 0x7f3c68003750 2021-07-11T04:02:26.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6800bd80 con 0x7f3c68004ec0 2021-07-11T04:02:26.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6800bf40 con 0x7f3c68004500 2021-07-11T04:02:26.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c82ffd700 1 --2- 172.21.2.104:0/2549906472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 0x7f3c68003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:26.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c82ffd700 1 -- 172.21.2.104:0/2549906472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3c68004500 msgr2=0x7f3c68004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:26.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c82ffd700 1 --2- 172.21.2.104:0/2549906472 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3c68004500 0x7f3c68004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:26.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c82ffd700 1 -- 172.21.2.104:0/2549906472 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c68004ec0 msgr2=0x7f3c6800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:26.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c82ffd700 1 --2- 172.21.2.104:0/2549906472 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c68004ec0 0x7f3c6800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:26.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c82ffd700 1 -- 172.21.2.104:0/2549906472 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c68003260 con 0x7f3c68003750 2021-07-11T04:02:26.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c837fe700 1 -- 172.21.2.104:0/2549906472 reap_dead start 2021-07-11T04:02:26.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c82ffd700 1 --2- 172.21.2.104:0/2549906472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 0x7f3c68003b50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7f3c7400a000 tx=0x7f3c74008040).ready entity=mon.0 client_cookie=d7b7c600e5752d0d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:26.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c740128a0 con 0x7f3c68003750 2021-07-11T04:02:26.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c74012a00 con 0x7f3c68003750 2021-07-11T04:02:26.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c81ffb700 1 -- 172.21.2.104:0/2549906472 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c74012d60 con 0x7f3c68003750 2021-07-11T04:02:26.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 -- 172.21.2.104:0/2549906472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 msgr2=0x7f3c68003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 --2- 172.21.2.104:0/2549906472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 0x7f3c68003b50 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7f3c7400a000 tx=0x7f3c74008040).stop 2021-07-11T04:02:26.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 -- 172.21.2.104:0/2549906472 shutdown_connections 2021-07-11T04:02:26.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 --2- 172.21.2.104:0/2549906472 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 0x7f3c68003b50 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:26.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 -- 172.21.2.104:0/2549906472 >> 172.21.2.104:0/2549906472 conn(0x7f3c8405f6b0 msgr2=0x7f3c8405fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:26.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 -- 172.21.2.104:0/2549906472 shutdown_connections 2021-07-11T04:02:26.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.516+0000 7f3c89383700 1 -- 172.21.2.104:0/2549906472 wait complete. 2021-07-11T04:02:26.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c89383700 1 Processor -- start 2021-07-11T04:02:26.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c89383700 1 -- start start 2021-07-11T04:02:26.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c89383700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 0x7f3c840adef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c89383700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c84156970 0x7f3c84120990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c89383700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3c84156df0 0x7f3c84120e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c89383700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c84121370 con 0x7f3c68003750 2021-07-11T04:02:26.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c89383700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840b01a0 con 0x7f3c84156970 2021-07-11T04:02:26.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c89383700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840ae850 con 0x7f3c84156df0 2021-07-11T04:02:26.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c82ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 0x7f3c840adef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:26.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c82ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 0x7f3c840adef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51436/0 (socket says 172.21.2.104:51436) 2021-07-11T04:02:26.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c82ffd700 1 -- 172.21.2.104:0/163089597 learned_addr learned my addr 172.21.2.104:0/163089597 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:26.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c82ffd700 1 -- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3c84156df0 msgr2=0x7f3c84120e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c82ffd700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3c84156df0 0x7f3c84120e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:26.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c82ffd700 1 -- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c84156970 msgr2=0x7f3c84120990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c82ffd700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c84156970 0x7f3c84120990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:26.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c82ffd700 1 -- 172.21.2.104:0/163089597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c840aeaf0 con 0x7f3c68003750 2021-07-11T04:02:26.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c82ffd700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 0x7f3c840adef0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7f3c7400a000 tx=0x7f3c7401c000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:26.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c73fff700 1 -- 172.21.2.104:0/163089597 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c7401bb40 con 0x7f3c68003750 2021-07-11T04:02:26.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c73fff700 1 -- 172.21.2.104:0/163089597 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c74000e90 con 0x7f3c68003750 2021-07-11T04:02:26.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c89383700 1 -- 172.21.2.104:0/163089597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c840ad1b0 con 0x7f3c68003750 2021-07-11T04:02:26.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c89383700 1 -- 172.21.2.104:0/163089597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8411f210 con 0x7f3c68003750 2021-07-11T04:02:26.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c73fff700 1 -- 172.21.2.104:0/163089597 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3c7401be90 con 0x7f3c68003750 2021-07-11T04:02:26.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c89383700 1 -- 172.21.2.104:0/163089597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c841448e0 con 0x7f3c68003750 2021-07-11T04:02:26.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c73fff700 1 -- 172.21.2.104:0/163089597 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3c7401a3d0 con 0x7f3c68003750 2021-07-11T04:02:26.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c73fff700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c60053260 0x7f3c60055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c73fff700 1 -- 172.21.2.104:0/163089597 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f3c7406bec0 con 0x7f3c68003750 2021-07-11T04:02:26.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.520+0000 7f3c827fc700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c60053260 0x7f3c60055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:26.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.524+0000 7f3c827fc700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c60053260 0x7f3c60055720 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f3c84060740 tx=0x7f3c84065ad0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:26.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.524+0000 7f3c73fff700 1 -- 172.21.2.104:0/163089597 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3c74043040 con 0x7f3c68003750 2021-07-11T04:02:26.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 -- 172.21.2.104:0/163089597 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c840adc90 con 0x7f3c68003750 2021-07-11T04:02:26.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c73fff700 1 -- 172.21.2.104:0/163089597 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2129 (secure 0 0 0) 0x7f3c74046660 con 0x7f3c68003750 2021-07-11T04:02:26.632 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:26.633 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":75,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/696025904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":696025904},{"type":"v1","addr":"172.21.2.104:6833","nonce":696025904}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8412,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/438569403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":438569403},{"type":"v1","addr":"172.21.2.104:6835","nonce":438569403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:16.677336+0000","modified":"2021-07-11T04:02:26.390198+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":150,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":14}]} 2021-07-11T04:02:26.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 -- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c60053260 msgr2=0x7f3c60055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c60053260 0x7f3c60055720 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f3c84060740 tx=0x7f3c84065ad0).stop 2021-07-11T04:02:26.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 -- 172.21.2.104:0/163089597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 msgr2=0x7f3c840adef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 0x7f3c840adef0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 rx=0x7f3c7400a000 tx=0x7f3c7401c000).stop 2021-07-11T04:02:26.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 -- 172.21.2.104:0/163089597 shutdown_connections 2021-07-11T04:02:26.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3c84156df0 0x7f3c84120e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:26.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3c84156970 0x7f3c84120990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:26.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3c60053260 0x7f3c60055720 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:26.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 --2- 172.21.2.104:0/163089597 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3c68003750 0x7f3c840adef0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:26.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 -- 172.21.2.104:0/163089597 >> 172.21.2.104:0/163089597 conn(0x7f3c8405f6b0 msgr2=0x7f3c840b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:26.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 -- 172.21.2.104:0/163089597 shutdown_connections 2021-07-11T04:02:26.636 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.632+0000 7f3c89383700 1 -- 172.21.2.104:0/163089597 wait complete. 2021-07-11T04:02:26.636 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 75 2021-07-11T04:02:26.639 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T04:02:26.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f44090f9700 1 Processor -- start 2021-07-11T04:02:26.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f44090f9700 1 -- start start 2021-07-11T04:02:26.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f44090f9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f440414ab80 0x7f440405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f44090f9700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f44040aff10 0x7f440414af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f44090f9700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4404061360 con 0x7f44040ac9b0 2021-07-11T04:02:26.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f44090f9700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440414b5e0 con 0x7f44040aff10 2021-07-11T04:02:26.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f44090f9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440414b880 con 0x7f440414ab80 2021-07-11T04:02:26.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f440259c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f44040aff10 0x7f440414af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:26.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f440259c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f44040aff10 0x7f440414af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:59004/0 (socket says 172.21.2.104:59004) 2021-07-11T04:02:26.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f440259c700 1 -- 172.21.2.104:0/4168027136 learned_addr learned my addr 172.21.2.104:0/4168027136 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:26.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f4402d9d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f440414ab80 0x7f440405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:26.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f4402d9d700 1 -- 172.21.2.104:0/4168027136 >> v2:172.21.2.105:3300/0 conn(0x7f44040aff10 msgr2=0x7f440414af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:26.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f4402d9d700 1 --2- 172.21.2.104:0/4168027136 >> v2:172.21.2.105:3300/0 conn(0x7f44040aff10 0x7f440414af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:26.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f4402d9d700 1 -- 172.21.2.104:0/4168027136 >> v1:172.21.2.105:6789/0 conn(0x7f44040ac9b0 legacy=0x7f44040af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f4402d9d700 1 -- 172.21.2.104:0/4168027136 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44040aed20 con 0x7f440414ab80 2021-07-11T04:02:26.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.692+0000 7f440259c700 1 --2- 172.21.2.104:0/4168027136 >> v2:172.21.2.105:3300/0 conn(0x7f44040aff10 0x7f440414af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-07-11T04:02:26.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f4402d9d700 1 --2- 172.21.2.104:0/4168027136 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f440414ab80 0x7f440405f6b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f43ec002f70 tx=0x7f43ec006df0).ready entity=mon.2 client_cookie=67c375e883d7c163 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:26.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f4401d9b700 1 -- 172.21.2.104:0/4168027136 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43ec00d570 con 0x7f440414ab80 2021-07-11T04:02:26.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f4401d9b700 1 -- 172.21.2.104:0/4168027136 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec000e90 con 0x7f440414ab80 2021-07-11T04:02:26.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 -- 172.21.2.104:0/4168027136 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f440414ab80 msgr2=0x7f440405f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 --2- 172.21.2.104:0/4168027136 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f440414ab80 0x7f440405f6b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f43ec002f70 tx=0x7f43ec006df0).stop 2021-07-11T04:02:26.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 -- 172.21.2.104:0/4168027136 shutdown_connections 2021-07-11T04:02:26.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 --2- 172.21.2.104:0/4168027136 >> v2:172.21.2.105:3300/0 conn(0x7f44040aff10 0x7f440414af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:26.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 --2- 172.21.2.104:0/4168027136 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f440414ab80 0x7f440405f6b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:26.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 -- 172.21.2.104:0/4168027136 >> 172.21.2.104:0/4168027136 conn(0x7f44040b5570 msgr2=0x7f44040b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:26.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 -- 172.21.2.104:0/4168027136 shutdown_connections 2021-07-11T04:02:26.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 -- 172.21.2.104:0/4168027136 wait complete. 2021-07-11T04:02:26.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 Processor -- start 2021-07-11T04:02:26.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 -- start start 2021-07-11T04:02:26.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f44040ac9b0 0x7f440414a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f44040aff10 0x7f4404142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f440414ab80 0x7f4404156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4404143340 con 0x7f44040aff10 2021-07-11T04:02:26.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4404143590 con 0x7f44040ac9b0 2021-07-11T04:02:26.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4404143830 con 0x7f440414ab80 2021-07-11T04:02:26.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f440259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f44040aff10 0x7f4404142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:26.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f440259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f44040aff10 0x7f4404142e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51452/0 (socket says 172.21.2.104:51452) 2021-07-11T04:02:26.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f440259c700 1 -- 172.21.2.104:0/1510245861 learned_addr learned my addr 172.21.2.104:0/1510245861 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:26.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f440259c700 1 -- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f440414ab80 msgr2=0x7f4404156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f440259c700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f440414ab80 0x7f4404156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:26.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f440259c700 1 -- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f44040ac9b0 msgr2=0x7f440414a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:26.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f4402d9d700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f44040ac9b0 0x7f440414a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:26.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f440259c700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f44040ac9b0 0x7f440414a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:26.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f440259c700 1 -- 172.21.2.104:0/1510245861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f400b1e0 con 0x7f44040aff10 2021-07-11T04:02:26.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f440259c700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f44040aff10 0x7f4404142e50 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7f43f400c000 tx=0x7f43f4006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:26.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f43fbfff700 1 -- 172.21.2.104:0/1510245861 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43f400c670 con 0x7f44040aff10 2021-07-11T04:02:26.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f43fbfff700 1 -- 172.21.2.104:0/1510245861 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f400c7d0 con 0x7f44040aff10 2021-07-11T04:02:26.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f43fbfff700 1 -- 172.21.2.104:0/1510245861 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f43f400cb30 con 0x7f44040aff10 2021-07-11T04:02:26.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 -- 172.21.2.104:0/1510245861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ec00b180 con 0x7f44040aff10 2021-07-11T04:02:26.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.696+0000 7f44090f9700 1 -- 172.21.2.104:0/1510245861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4404156580 con 0x7f44040aff10 2021-07-11T04:02:26.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.700+0000 7f44090f9700 1 -- 172.21.2.104:0/1510245861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4404057350 con 0x7f44040aff10 2021-07-11T04:02:26.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.700+0000 7f43fbfff700 1 -- 172.21.2.104:0/1510245861 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f43f400cc90 con 0x7f44040aff10 2021-07-11T04:02:26.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.700+0000 7f43fbfff700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43e8053300 0x7f43e80557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:26.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.700+0000 7f43fbfff700 1 -- 172.21.2.104:0/1510245861 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f43f406bc50 con 0x7f44040aff10 2021-07-11T04:02:26.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.700+0000 7f4402d9d700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43e8053300 0x7f43e80557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:26.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.700+0000 7f4402d9d700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43e8053300 0x7f43e80557c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f43ec015000 tx=0x7f43ec014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:26.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.700+0000 7f43fbfff700 1 -- 172.21.2.104:0/1510245861 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f43f4042e30 con 0x7f44040aff10 2021-07-11T04:02:26.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:26.808+0000 7f44090f9700 1 -- 172.21.2.104:0/1510245861 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4404044650 con 0x7f44040aff10 2021-07-11T04:02:27.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f43fbfff700 1 -- 172.21.2.104:0/1510245861 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7f43f4046450 con 0x7f44040aff10 2021-07-11T04:02:27.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 -- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43e8053300 msgr2=0x7f43e80557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:27.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43e8053300 0x7f43e80557c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f43ec015000 tx=0x7f43ec014000).stop 2021-07-11T04:02:27.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 -- 172.21.2.104:0/1510245861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f44040aff10 msgr2=0x7f4404142e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:27.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f44040aff10 0x7f4404142e50 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7f43f400c000 tx=0x7f43f4006eb0).stop 2021-07-11T04:02:27.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 -- 172.21.2.104:0/1510245861 shutdown_connections 2021-07-11T04:02:27.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f440414ab80 0x7f4404156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:27.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f43e8053300 0x7f43e80557c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:27.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f44040aff10 0x7f4404142e50 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:27.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 --2- 172.21.2.104:0/1510245861 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f44040ac9b0 0x7f440414a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:27.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 -- 172.21.2.104:0/1510245861 >> 172.21.2.104:0/1510245861 conn(0x7f44040b5570 msgr2=0x7f44041489b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:27.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 -- 172.21.2.104:0/1510245861 shutdown_connections 2021-07-11T04:02:27.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.416+0000 7f44090f9700 1 -- 172.21.2.104:0/1510245861 wait complete. 2021-07-11T04:02:27.423 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T04:02:27.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95afc75700 1 Processor -- start 2021-07-11T04:02:27.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95afc75700 1 -- start start 2021-07-11T04:02:27.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95afc75700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95a8060100 0x7f95a80b4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:27.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95afc75700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f95a8067300 0x7f95a81438f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:27.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95afc75700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a80aabb0 con 0x7f95a80b4c40 2021-07-11T04:02:27.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95afc75700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a80604e0 con 0x7f95a8067300 2021-07-11T04:02:27.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95afc75700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a8066ea0 con 0x7f95a8060100 2021-07-11T04:02:27.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95ae212700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f95a80b4c40 0x7f95a8144510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48224/0 (socket says 172.21.2.104:48224) 2021-07-11T04:02:27.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95ae212700 1 -- 172.21.2.104:0/2668240088 learned_addr learned my addr 172.21.2.104:0/2668240088 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:27.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398345754 0 0) 0x7f95a80aabb0 con 0x7f95a80b4c40 2021-07-11T04:02:27.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9590003260 con 0x7f95a80b4c40 2021-07-11T04:02:27.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2449418175 0 0) 0x7f9590003260 con 0x7f95a80b4c40 2021-07-11T04:02:27.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95a8060100 msgr2=0x7f95a80b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:27.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 --2- 172.21.2.104:0/2668240088 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95a8060100 0x7f95a80b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:27.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 >> v2:172.21.2.105:3300/0 conn(0x7f95a8067300 msgr2=0x7f95a81438f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:27.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 --2- 172.21.2.104:0/2668240088 >> v2:172.21.2.105:3300/0 conn(0x7f95a8067300 0x7f95a81438f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:27.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80b33b0 con 0x7f95a80b4c40 2021-07-11T04:02:27.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f95a00019e0 con 0x7f95a80b4c40 2021-07-11T04:02:27.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 >> v1:172.21.2.104:6789/0 conn(0x7f95a80b4c40 legacy=0x7f95a8144510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:27.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 --2- 172.21.2.104:0/2668240088 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95900037d0 0x7f9590003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:27.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 --2- 172.21.2.104:0/2668240088 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9590004500 0x7f9590004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:27.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 --2- 172.21.2.104:0/2668240088 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95a8060100 0x7f959000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:27.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80604e0 con 0x7f9590004500 2021-07-11T04:02:27.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a8066ea0 con 0x7f95900037d0 2021-07-11T04:02:27.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9590005330 con 0x7f95a8060100 2021-07-11T04:02:27.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95ad210700 1 --2- 172.21.2.104:0/2668240088 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9590004500 0x7f9590004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:27.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95ad210700 1 -- 172.21.2.104:0/2668240088 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95a8060100 msgr2=0x7f959000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:27.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95ad210700 1 --2- 172.21.2.104:0/2668240088 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95a8060100 0x7f959000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:27.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95ad210700 1 -- 172.21.2.104:0/2668240088 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95900037d0 msgr2=0x7f9590003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:27.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95ad210700 1 --2- 172.21.2.104:0/2668240088 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95900037d0 0x7f9590003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:27.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95ad210700 1 -- 172.21.2.104:0/2668240088 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80b33b0 con 0x7f9590004500 2021-07-11T04:02:27.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.476+0000 7f95ad210700 1 --2- 172.21.2.104:0/2668240088 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9590004500 0x7f9590004980 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f9598002f70 tx=0x7f9598006ca0).ready entity=mon.0 client_cookie=62b995aa813d304c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:27.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9598012410 con 0x7f9590004500 2021-07-11T04:02:27.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9598012570 con 0x7f9590004500 2021-07-11T04:02:27.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95aca0f700 1 -- 172.21.2.104:0/2668240088 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95980128d0 con 0x7f9590004500 2021-07-11T04:02:27.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- 172.21.2.104:0/2668240088 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9590004500 msgr2=0x7f9590004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:27.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 --2- 172.21.2.104:0/2668240088 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9590004500 0x7f9590004980 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 rx=0x7f9598002f70 tx=0x7f9598006ca0).stop 2021-07-11T04:02:27.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95ae212700 1 -- 172.21.2.104:0/2668240088 reap_dead start 2021-07-11T04:02:27.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- 172.21.2.104:0/2668240088 shutdown_connections 2021-07-11T04:02:27.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- 172.21.2.104:0/2668240088 >> 172.21.2.104:0/2668240088 conn(0x7f95a805cf30 msgr2=0x7f95a805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:27.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- 172.21.2.104:0/2668240088 shutdown_connections 2021-07-11T04:02:27.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- 172.21.2.104:0/2668240088 wait complete. 2021-07-11T04:02:27.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 Processor -- start 2021-07-11T04:02:27.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- start start 2021-07-11T04:02:27.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95a80afb20 0x7f95a8155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:27.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95a813b930 0x7f95a8156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:27.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95a8156930 0x7f95a80ac380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:27.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80ae810 con 0x7f95a80afb20 2021-07-11T04:02:27.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80aff00 con 0x7f95a813b930 2021-07-11T04:02:27.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80adcb0 con 0x7f95a8156930 2021-07-11T04:02:27.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95ada11700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95a80afb20 0x7f95a8155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:27.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95ada11700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95a80afb20 0x7f95a8155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51468/0 (socket says 172.21.2.104:51468) 2021-07-11T04:02:27.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95ada11700 1 -- 172.21.2.104:0/2309157541 learned_addr learned my addr 172.21.2.104:0/2309157541 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:27.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95ada11700 1 -- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95a8156930 msgr2=0x7f95a80ac380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:27.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95ada11700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95a8156930 0x7f95a80ac380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:27.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95ada11700 1 -- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95a813b930 msgr2=0x7f95a8156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:27.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95ada11700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95a813b930 0x7f95a8156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:27.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95ada11700 1 -- 172.21.2.104:0/2309157541 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a80aea40 con 0x7f95a80afb20 2021-07-11T04:02:27.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95ada11700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95a80afb20 0x7f95a8155f50 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f95a4002f70 tx=0x7f95a4006e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:27.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f9596ffd700 1 -- 172.21.2.104:0/2309157541 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95a4013460 con 0x7f95a80afb20 2021-07-11T04:02:27.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f9596ffd700 1 -- 172.21.2.104:0/2309157541 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a40135c0 con 0x7f95a80afb20 2021-07-11T04:02:27.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- 172.21.2.104:0/2309157541 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a80adf10 con 0x7f95a80afb20 2021-07-11T04:02:27.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- 172.21.2.104:0/2309157541 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a813ec50 con 0x7f95a80afb20 2021-07-11T04:02:27.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f9596ffd700 1 -- 172.21.2.104:0/2309157541 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f95a4013920 con 0x7f95a80afb20 2021-07-11T04:02:27.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f95afc75700 1 -- 172.21.2.104:0/2309157541 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a8057350 con 0x7f95a80afb20 2021-07-11T04:02:27.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f9596ffd700 1 -- 172.21.2.104:0/2309157541 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f95a401a060 con 0x7f95a80afb20 2021-07-11T04:02:27.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f9596ffd700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9588053300 0x7f95880557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:27.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.480+0000 7f9596ffd700 1 -- 172.21.2.104:0/2309157541 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f95a406af40 con 0x7f95a80afb20 2021-07-11T04:02:27.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.484+0000 7f95ad210700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9588053300 0x7f95880557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:27.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.484+0000 7f95ad210700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9588053300 0x7f95880557c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f9598002f70 tx=0x7f9598015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:27.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.484+0000 7f9596ffd700 1 -- 172.21.2.104:0/2309157541 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f95a4042230 con 0x7f95a80afb20 2021-07-11T04:02:27.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:27.564+0000 7f95afc75700 1 -- 172.21.2.104:0/2309157541 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f95a80ae290 con 0x7f95a80afb20 2021-07-11T04:02:28.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f9596ffd700 1 -- 172.21.2.104:0/2309157541 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v151) v1 ==== 185+0+0 (secure 0 0 0) 0x7f95a4018020 con 0x7f95a80afb20 2021-07-11T04:02:28.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 -- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9588053300 msgr2=0x7f95880557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:28.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9588053300 0x7f95880557c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f9598002f70 tx=0x7f9598015000).stop 2021-07-11T04:02:28.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 -- 172.21.2.104:0/2309157541 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95a80afb20 msgr2=0x7f95a8155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:28.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95a80afb20 0x7f95a8155f50 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7f95a4002f70 tx=0x7f95a4006e40).stop 2021-07-11T04:02:28.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 -- 172.21.2.104:0/2309157541 shutdown_connections 2021-07-11T04:02:28.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f95a8156930 0x7f95a80ac380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:28.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f95a813b930 0x7f95a8156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:28.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9588053300 0x7f95880557c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:28.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 --2- 172.21.2.104:0/2309157541 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f95a80afb20 0x7f95a8155f50 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:28.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 -- 172.21.2.104:0/2309157541 >> 172.21.2.104:0/2309157541 conn(0x7f95a805cf30 msgr2=0x7f95a8142df0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:28.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 -- 172.21.2.104:0/2309157541 shutdown_connections 2021-07-11T04:02:28.423 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.420+0000 7f95afc75700 1 -- 172.21.2.104:0/2309157541 wait complete. 2021-07-11T04:02:28.424 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T04:02:28.427 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T04:02:28.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 Processor -- start 2021-07-11T04:02:28.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 -- start start 2021-07-11T04:02:28.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f28fc1483a0 0x7f28fc0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:28.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f28fc0a70c0 0x7f28fc144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:28.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28fc0b0350 con 0x7f28fc0b1db0 2021-07-11T04:02:28.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28fc0a90c0 con 0x7f28fc1483a0 2021-07-11T04:02:28.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28fc0b05f0 con 0x7f28fc0a70c0 2021-07-11T04:02:28.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f28fa7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f28fc1483a0 0x7f28fc0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:28.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f28fa7fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f28fc1483a0 0x7f28fc0b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51478/0 (socket says 172.21.2.104:51478) 2021-07-11T04:02:28.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f28fa7fc700 1 -- 172.21.2.104:0/1218128089 learned_addr learned my addr 172.21.2.104:0/1218128089 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:28.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f28fa7fc700 1 -- 172.21.2.104:0/1218128089 >> v2:172.21.2.105:3300/0 conn(0x7f28fc0a70c0 msgr2=0x7f28fc144720 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:28.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f28fa7fc700 1 --2- 172.21.2.104:0/1218128089 >> v2:172.21.2.105:3300/0 conn(0x7f28fc0a70c0 0x7f28fc144720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:28.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f28fa7fc700 1 -- 172.21.2.104:0/1218128089 >> v1:172.21.2.105:6789/0 conn(0x7f28fc0b1db0 legacy=0x7f28fc0b2190 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:28.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f28fa7fc700 1 -- 172.21.2.104:0/1218128089 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc0b0890 con 0x7f28fc1483a0 2021-07-11T04:02:28.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f28fa7fc700 1 --2- 172.21.2.104:0/1218128089 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28fc1483a0 0x7f28fc0b0fd0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 rx=0x7f28fc0ace60 tx=0x7f28ec006eb0).ready entity=mon.0 client_cookie=8e269664d61f858d server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:28.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f28f9ffb700 1 -- 172.21.2.104:0/1218128089 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28ec00d5c0 con 0x7f28fc1483a0 2021-07-11T04:02:28.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f28f9ffb700 1 -- 172.21.2.104:0/1218128089 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ec000e90 con 0x7f28fc1483a0 2021-07-11T04:02:28.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f28f9ffb700 1 -- 172.21.2.104:0/1218128089 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28ec00d950 con 0x7f28fc1483a0 2021-07-11T04:02:28.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 -- 172.21.2.104:0/1218128089 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28fc1483a0 msgr2=0x7f28fc0b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:28.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 --2- 172.21.2.104:0/1218128089 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28fc1483a0 0x7f28fc0b0fd0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 rx=0x7f28fc0ace60 tx=0x7f28ec006eb0).stop 2021-07-11T04:02:28.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 -- 172.21.2.104:0/1218128089 shutdown_connections 2021-07-11T04:02:28.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 --2- 172.21.2.104:0/1218128089 >> v2:172.21.2.105:3300/0 conn(0x7f28fc0a70c0 0x7f28fc144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:28.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 --2- 172.21.2.104:0/1218128089 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28fc1483a0 0x7f28fc0b0fd0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:28.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 -- 172.21.2.104:0/1218128089 >> 172.21.2.104:0/1218128089 conn(0x7f28fc05f6b0 msgr2=0x7f28fc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:28.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 -- 172.21.2.104:0/1218128089 shutdown_connections 2021-07-11T04:02:28.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.480+0000 7f290198b700 1 -- 172.21.2.104:0/1218128089 wait complete. 2021-07-11T04:02:28.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f290198b700 1 Processor -- start 2021-07-11T04:02:28.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f290198b700 1 -- start start 2021-07-11T04:02:28.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f290198b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28fc0a70c0 0x7f28fc147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:28.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f290198b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28fc0b1db0 0x7f28fc156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:28.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f290198b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28fc1483a0 0x7f28fc1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:28.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f290198b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc145b60 con 0x7f28fc0a70c0 2021-07-11T04:02:28.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f290198b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc145dc0 con 0x7f28fc1483a0 2021-07-11T04:02:28.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f290198b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc146060 con 0x7f28fc0b1db0 2021-07-11T04:02:28.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28faffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28fc0a70c0 0x7f28fc147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:28.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28faffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28fc0a70c0 0x7f28fc147040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51480/0 (socket says 172.21.2.104:51480) 2021-07-11T04:02:28.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28faffd700 1 -- 172.21.2.104:0/2419910884 learned_addr learned my addr 172.21.2.104:0/2419910884 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:28.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28faffd700 1 -- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28fc0b1db0 msgr2=0x7f28fc156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:28.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28faffd700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28fc0b1db0 0x7f28fc156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:28.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28faffd700 1 -- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28fc1483a0 msgr2=0x7f28fc1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:28.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28faffd700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28fc1483a0 0x7f28fc1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:28.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28faffd700 1 -- 172.21.2.104:0/2419910884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec00b1e0 con 0x7f28fc0a70c0 2021-07-11T04:02:28.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28faffd700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28fc0a70c0 0x7f28fc147040 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f28f0002fb0 tx=0x7f28f0006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:28.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28ebfff700 1 -- 172.21.2.104:0/2419910884 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28f0013460 con 0x7f28fc0a70c0 2021-07-11T04:02:28.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28ebfff700 1 -- 172.21.2.104:0/2419910884 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f00135c0 con 0x7f28fc0a70c0 2021-07-11T04:02:28.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f290198b700 1 -- 172.21.2.104:0/2419910884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc1467d0 con 0x7f28fc0a70c0 2021-07-11T04:02:28.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28ebfff700 1 -- 172.21.2.104:0/2419910884 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f28f0013920 con 0x7f28fc0a70c0 2021-07-11T04:02:28.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f290198b700 1 -- 172.21.2.104:0/2419910884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc156920 con 0x7f28fc0a70c0 2021-07-11T04:02:28.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f290198b700 1 -- 172.21.2.104:0/2419910884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc057350 con 0x7f28fc0a70c0 2021-07-11T04:02:28.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28ebfff700 1 -- 172.21.2.104:0/2419910884 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f28f000f070 con 0x7f28fc0a70c0 2021-07-11T04:02:28.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28ebfff700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28d80531f0 0x7f28d80556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:28.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28ebfff700 1 -- 172.21.2.104:0/2419910884 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5717+0+0 (secure 0 0 0) 0x7f28f006ae30 con 0x7f28fc0a70c0 2021-07-11T04:02:28.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.484+0000 7f28fa7fc700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28d80531f0 0x7f28d80556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:28.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.488+0000 7f28fa7fc700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28d80531f0 0x7f28d80556b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f28fc0ace60 tx=0x7f28ec015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:28.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.488+0000 7f28ebfff700 1 -- 172.21.2.104:0/2419910884 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f28f00422d0 con 0x7f28fc0a70c0 2021-07-11T04:02:28.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:28.572+0000 7f290198b700 1 -- 172.21.2.104:0/2419910884 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28fc044650 con 0x7f28fc0a70c0 2021-07-11T04:02:29.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f28ebfff700 1 -- 172.21.2.104:0/2419910884 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v152) v1 ==== 173+0+0 (secure 0 0 0) 0x7f28f00458f0 con 0x7f28fc0a70c0 2021-07-11T04:02:29.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 -- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28d80531f0 msgr2=0x7f28d80556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:29.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28d80531f0 0x7f28d80556b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f28fc0ace60 tx=0x7f28ec015000).stop 2021-07-11T04:02:29.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 -- 172.21.2.104:0/2419910884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28fc0a70c0 msgr2=0x7f28fc147040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:29.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28fc0a70c0 0x7f28fc147040 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 rx=0x7f28f0002fb0 tx=0x7f28f0006e10).stop 2021-07-11T04:02:29.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 -- 172.21.2.104:0/2419910884 shutdown_connections 2021-07-11T04:02:29.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f28fc1483a0 0x7f28fc1572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:29.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f28fc0b1db0 0x7f28fc156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:29.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f28d80531f0 0x7f28d80556b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 --2- 172.21.2.104:0/2419910884 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f28fc0a70c0 0x7f28fc147040 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 -- 172.21.2.104:0/2419910884 >> 172.21.2.104:0/2419910884 conn(0x7f28fc05f6b0 msgr2=0x7f28fc1479d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:29.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 -- 172.21.2.104:0/2419910884 shutdown_connections 2021-07-11T04:02:29.436 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.432+0000 7f290198b700 1 -- 172.21.2.104:0/2419910884 wait complete. 2021-07-11T04:02:29.437 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T04:02:29.441 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2021-07-11T04:02:29.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.492+0000 7fd9dd292700 1 Processor -- start 2021-07-11T04:02:29.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.492+0000 7fd9dd292700 1 -- start start 2021-07-11T04:02:29.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.492+0000 7fd9dd292700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9d80b12d0 0x7fd9d80affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.492+0000 7fd9dd292700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd9d80b22a0 0x7fd9d8141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.492+0000 7fd9dd292700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d80a77d0 con 0x7fd9d80b1ec0 2021-07-11T04:02:29.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.492+0000 7fd9dd292700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d80b1740 con 0x7fd9d80b22a0 2021-07-11T04:02:29.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.492+0000 7fd9dd292700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d80b0650 con 0x7fd9d80b12d0 2021-07-11T04:02:29.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.492+0000 7fd9d77fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd9d80b22a0 0x7fd9d8141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:29.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.492+0000 7fd9d77fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fd9d80b22a0 0x7fd9d8141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:59048/0 (socket says 172.21.2.104:59048) 2021-07-11T04:02:29.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.492+0000 7fd9d77fe700 1 -- 172.21.2.104:0/737917619 learned_addr learned my addr 172.21.2.104:0/737917619 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:29.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.492+0000 7fd9d67fc700 1 --2- 172.21.2.104:0/737917619 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9d80b12d0 0x7fd9d80affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:29.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d77fe700 1 -- 172.21.2.104:0/737917619 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9d80b12d0 msgr2=0x7fd9d80affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:29.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d77fe700 1 --2- 172.21.2.104:0/737917619 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9d80b12d0 0x7fd9d80affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d77fe700 1 -- 172.21.2.104:0/737917619 >> v1:172.21.2.105:6789/0 conn(0x7fd9d80b1ec0 legacy=0x7fd9d80a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:29.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d77fe700 1 -- 172.21.2.104:0/737917619 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d80b08f0 con 0x7fd9d80b22a0 2021-07-11T04:02:29.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d77fe700 1 --2- 172.21.2.104:0/737917619 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9d80b22a0 0x7fd9d8141250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fd9d00032f0 tx=0x7fd9d0009290).ready entity=mon.1 client_cookie=3e29c275d6955b2c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:29.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d5ffb700 1 -- 172.21.2.104:0/737917619 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9d000b8a0 con 0x7fd9d80b22a0 2021-07-11T04:02:29.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d5ffb700 1 -- 172.21.2.104:0/737917619 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d000ba00 con 0x7fd9d80b22a0 2021-07-11T04:02:29.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d5ffb700 1 -- 172.21.2.104:0/737917619 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9d000bb70 con 0x7fd9d80b22a0 2021-07-11T04:02:29.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 -- 172.21.2.104:0/737917619 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9d80b22a0 msgr2=0x7fd9d8141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:29.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 --2- 172.21.2.104:0/737917619 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9d80b22a0 0x7fd9d8141250 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fd9d00032f0 tx=0x7fd9d0009290).stop 2021-07-11T04:02:29.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 -- 172.21.2.104:0/737917619 shutdown_connections 2021-07-11T04:02:29.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 --2- 172.21.2.104:0/737917619 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9d80b22a0 0x7fd9d8141250 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 --2- 172.21.2.104:0/737917619 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9d80b12d0 0x7fd9d80affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 -- 172.21.2.104:0/737917619 >> 172.21.2.104:0/737917619 conn(0x7fd9d805f6b0 msgr2=0x7fd9d805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:29.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 -- 172.21.2.104:0/737917619 shutdown_connections 2021-07-11T04:02:29.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 -- 172.21.2.104:0/737917619 wait complete. 2021-07-11T04:02:29.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 Processor -- start 2021-07-11T04:02:29.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 -- start start 2021-07-11T04:02:29.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9d80b12d0 0x7fd9d80a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9d80b1ec0 0x7fd9d8157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9d80b22a0 0x7fd9d80ab550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d8149d90 con 0x7fd9d80b12d0 2021-07-11T04:02:29.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d814a030 con 0x7fd9d80b1ec0 2021-07-11T04:02:29.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d814a2d0 con 0x7fd9d80b22a0 2021-07-11T04:02:29.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9d80b12d0 0x7fd9d80a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:29.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d6ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9d80b12d0 0x7fd9d80a9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51496/0 (socket says 172.21.2.104:51496) 2021-07-11T04:02:29.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d6ffd700 1 -- 172.21.2.104:0/4262160749 learned_addr learned my addr 172.21.2.104:0/4262160749 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:29.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d67fc700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9d80b1ec0 0x7fd9d8157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:29.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d77fe700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9d80b22a0 0x7fd9d80ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:29.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d6ffd700 1 -- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9d80b22a0 msgr2=0x7fd9d80ab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:29.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d6ffd700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9d80b22a0 0x7fd9d80ab550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d6ffd700 1 -- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9d80b1ec0 msgr2=0x7fd9d8157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:29.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d6ffd700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9d80b1ec0 0x7fd9d8157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d6ffd700 1 -- 172.21.2.104:0/4262160749 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d0009530 con 0x7fd9d80b12d0 2021-07-11T04:02:29.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9d6ffd700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9d80b12d0 0x7fd9d80a9bc0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7fd9d806a0f0 tx=0x7fd9c8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:29.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9cffff700 1 -- 172.21.2.104:0/4262160749 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9c8007030 con 0x7fd9d80b12d0 2021-07-11T04:02:29.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9cffff700 1 -- 172.21.2.104:0/4262160749 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c8007190 con 0x7fd9d80b12d0 2021-07-11T04:02:29.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 -- 172.21.2.104:0/4262160749 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d814a570 con 0x7fd9d80b12d0 2021-07-11T04:02:29.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9dd292700 1 -- 172.21.2.104:0/4262160749 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d8157980 con 0x7fd9d80b12d0 2021-07-11T04:02:29.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.496+0000 7fd9cffff700 1 -- 172.21.2.104:0/4262160749 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd9c8006510 con 0x7fd9d80b12d0 2021-07-11T04:02:29.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.500+0000 7fd9dd292700 1 -- 172.21.2.104:0/4262160749 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d8057350 con 0x7fd9d80b12d0 2021-07-11T04:02:29.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.500+0000 7fd9cffff700 1 -- 172.21.2.104:0/4262160749 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd9c8010030 con 0x7fd9d80b12d0 2021-07-11T04:02:29.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.500+0000 7fd9cffff700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9bc053300 0x7fd9bc0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.500+0000 7fd9cffff700 1 -- 172.21.2.104:0/4262160749 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd9c806adc0 con 0x7fd9d80b12d0 2021-07-11T04:02:29.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.500+0000 7fd9d67fc700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9bc053300 0x7fd9bc0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:29.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.500+0000 7fd9d67fc700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9bc053300 0x7fd9bc0557c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7fd9c000b000 tx=0x7fd9c0006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:29.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.500+0000 7fd9cffff700 1 -- 172.21.2.104:0/4262160749 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd9c80423e0 con 0x7fd9d80b12d0 2021-07-11T04:02:29.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.584+0000 7fd9dd292700 1 -- 172.21.2.104:0/4262160749 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd9d80b2ed0 con 0x7fd9d80b12d0 2021-07-11T04:02:29.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9cffff700 1 -- 172.21.2.104:0/4262160749 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd9c8045a00 con 0x7fd9d80b12d0 2021-07-11T04:02:29.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 -- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9bc053300 msgr2=0x7fd9bc0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:29.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9bc053300 0x7fd9bc0557c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7fd9c000b000 tx=0x7fd9c0006df0).stop 2021-07-11T04:02:29.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 -- 172.21.2.104:0/4262160749 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9d80b12d0 msgr2=0x7fd9d80a9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:29.590 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9d80b12d0 0x7fd9d80a9bc0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7fd9d806a0f0 tx=0x7fd9c8007c10).stop 2021-07-11T04:02:29.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 -- 172.21.2.104:0/4262160749 shutdown_connections 2021-07-11T04:02:29.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd9d80b22a0 0x7fd9d80ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd9d80b1ec0 0x7fd9d8157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.591 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd9bc053300 0x7fd9bc0557c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 --2- 172.21.2.104:0/4262160749 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd9d80b12d0 0x7fd9d80a9bc0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 -- 172.21.2.104:0/4262160749 >> 172.21.2.104:0/4262160749 conn(0x7fd9d805f6b0 msgr2=0x7fd9d81404c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:29.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 -- 172.21.2.104:0/4262160749 shutdown_connections 2021-07-11T04:02:29.592 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.588+0000 7fd9dd292700 1 -- 172.21.2.104:0/4262160749 wait complete. 2021-07-11T04:02:29.596 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime' 2021-07-11T04:02:29.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 Processor -- start 2021-07-11T04:02:29.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 -- start start 2021-07-11T04:02:29.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f2a500a76b0 0x7f2a50066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f2a500b0060 0x7f2a5005fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a500a7a90 con 0x7f2a500b1970 2021-07-11T04:02:29.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a50147620 con 0x7f2a500b0060 2021-07-11T04:02:29.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a501478c0 con 0x7f2a500a76b0 2021-07-11T04:02:29.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a4e7fc700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f2a500b1970 0x7f2a5005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48274/0 (socket says 172.21.2.104:48274) 2021-07-11T04:02:29.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a4e7fc700 1 -- 172.21.2.104:0/2102396262 learned_addr learned my addr 172.21.2.104:0/2102396262 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:29.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a4f7fe700 1 --2- 172.21.2.104:0/2102396262 >> v2:172.21.2.104:3300/0 conn(0x7f2a500b0060 0x7f2a5005fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:29.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a4f7fe700 1 -- 172.21.2.104:0/2102396262 >> v2:172.21.2.105:3300/0 conn(0x7f2a500a76b0 msgr2=0x7f2a50066a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:29.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a4f7fe700 1 --2- 172.21.2.104:0/2102396262 >> v2:172.21.2.105:3300/0 conn(0x7f2a500a76b0 0x7f2a50066a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:29.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a4f7fe700 1 -- 172.21.2.104:0/2102396262 >> v1:172.21.2.104:6789/0 conn(0x7f2a500b1970 legacy=0x7f2a5005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:29.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a4f7fe700 1 -- 172.21.2.104:0/2102396262 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a50147b60 con 0x7f2a500b0060 2021-07-11T04:02:29.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a4f7fe700 1 --2- 172.21.2.104:0/2102396262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a500b0060 0x7f2a5005fdc0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f2a480032f0 tx=0x7f2a480091d0).ready entity=mon.0 client_cookie=f462d1c135ea7340 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:29.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a4dffb700 1 -- 172.21.2.104:0/2102396262 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a48009ea0 con 0x7f2a500b0060 2021-07-11T04:02:29.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a4dffb700 1 -- 172.21.2.104:0/2102396262 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4800b870 con 0x7f2a500b0060 2021-07-11T04:02:29.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a4dffb700 1 -- 172.21.2.104:0/2102396262 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a4800ba20 con 0x7f2a500b0060 2021-07-11T04:02:29.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 -- 172.21.2.104:0/2102396262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a500b0060 msgr2=0x7f2a5005fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:29.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 --2- 172.21.2.104:0/2102396262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a500b0060 0x7f2a5005fdc0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 rx=0x7f2a480032f0 tx=0x7f2a480091d0).stop 2021-07-11T04:02:29.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 -- 172.21.2.104:0/2102396262 shutdown_connections 2021-07-11T04:02:29.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 --2- 172.21.2.104:0/2102396262 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a500b0060 0x7f2a5005fdc0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:29.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 --2- 172.21.2.104:0/2102396262 >> v2:172.21.2.105:3300/0 conn(0x7f2a500a76b0 0x7f2a50066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:29.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 -- 172.21.2.104:0/2102396262 >> 172.21.2.104:0/2102396262 conn(0x7f2a500b53e0 msgr2=0x7f2a500b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:29.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 -- 172.21.2.104:0/2102396262 shutdown_connections 2021-07-11T04:02:29.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 -- 172.21.2.104:0/2102396262 wait complete. 2021-07-11T04:02:29.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 Processor -- start 2021-07-11T04:02:29.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.648+0000 7f2a55858700 1 -- start start 2021-07-11T04:02:29.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a55858700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a500a76b0 0x7f2a500aaf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a55858700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2a500b0060 0x7f2a501569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a55858700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2a500b1970 0x7f2a50156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a55858700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5014a400 con 0x7f2a500a76b0 2021-07-11T04:02:29.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a55858700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500ac6a0 con 0x7f2a500b1970 2021-07-11T04:02:29.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a55858700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a500ac940 con 0x7f2a500b0060 2021-07-11T04:02:29.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a4effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a500a76b0 0x7f2a500aaf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:29.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a4effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a500a76b0 0x7f2a500aaf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51508/0 (socket says 172.21.2.104:51508) 2021-07-11T04:02:29.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a4effd700 1 -- 172.21.2.104:0/1280036967 learned_addr learned my addr 172.21.2.104:0/1280036967 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:29.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a4effd700 1 -- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2a500b0060 msgr2=0x7f2a501569a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:29.660 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a4effd700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2a500b0060 0x7f2a501569a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:29.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a4effd700 1 -- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2a500b1970 msgr2=0x7f2a50156e90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:29.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a4effd700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2a500b1970 0x7f2a50156e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:29.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a4effd700 1 -- 172.21.2.104:0/1280036967 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a48009470 con 0x7f2a500a76b0 2021-07-11T04:02:29.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a4effd700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a500a76b0 0x7f2a500aaf00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7f2a40002fb0 tx=0x7f2a40006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:29.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a47fff700 1 -- 172.21.2.104:0/1280036967 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a40013460 con 0x7f2a500a76b0 2021-07-11T04:02:29.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a47fff700 1 -- 172.21.2.104:0/1280036967 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a400135c0 con 0x7f2a500a76b0 2021-07-11T04:02:29.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a55858700 1 -- 172.21.2.104:0/1280036967 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a500aef90 con 0x7f2a500a76b0 2021-07-11T04:02:29.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a55858700 1 -- 172.21.2.104:0/1280036967 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a500ad150 con 0x7f2a500a76b0 2021-07-11T04:02:29.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a47fff700 1 -- 172.21.2.104:0/1280036967 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f2a40013920 con 0x7f2a500a76b0 2021-07-11T04:02:29.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a55858700 1 -- 172.21.2.104:0/1280036967 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a28003410 con 0x7f2a500a76b0 2021-07-11T04:02:29.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a47fff700 1 -- 172.21.2.104:0/1280036967 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f2a4000f070 con 0x7f2a500a76b0 2021-07-11T04:02:29.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a47fff700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a34053240 0x7f2a34055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:29.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a47fff700 1 -- 172.21.2.104:0/1280036967 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2a4006ad20 con 0x7f2a500a76b0 2021-07-11T04:02:29.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a4e7fc700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a34053240 0x7f2a34055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:29.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.652+0000 7f2a4e7fc700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a34053240 0x7f2a34055700 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f2a5006a0e0 tx=0x7f2a380077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:29.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.656+0000 7f2a47fff700 1 -- 172.21.2.104:0/1280036967 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f2a400421d0 con 0x7f2a500a76b0 2021-07-11T04:02:29.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:29.768+0000 7f2a55858700 1 -- 172.21.2.104:0/1280036967 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]} v 0) v1 -- 0x7f2a28003c40 con 0x7f2a500a76b0 2021-07-11T04:02:30.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a47fff700 1 -- 172.21.2.104:0/1280036967 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_misc.TestMisc.test_root_rctime"]}]=0 v407) v1 ==== 127+0+0 (secure 0 0 0) 0x7f2a400457f0 con 0x7f2a500a76b0 2021-07-11T04:02:30.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 -- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a34053240 msgr2=0x7f2a34055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:30.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a34053240 0x7f2a34055700 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f2a5006a0e0 tx=0x7f2a380077f0).stop 2021-07-11T04:02:30.437 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 -- 172.21.2.104:0/1280036967 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a500a76b0 msgr2=0x7f2a500aaf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:30.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a500a76b0 0x7f2a500aaf00 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7f2a40002fb0 tx=0x7f2a40006df0).stop 2021-07-11T04:02:30.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 -- 172.21.2.104:0/1280036967 shutdown_connections 2021-07-11T04:02:30.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f2a500b1970 0x7f2a50156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:30.438 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f2a500b0060 0x7f2a501569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:30.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2a34053240 0x7f2a34055700 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:30.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 --2- 172.21.2.104:0/1280036967 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f2a500a76b0 0x7f2a500aaf00 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:30.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 -- 172.21.2.104:0/1280036967 >> 172.21.2.104:0/1280036967 conn(0x7f2a500b53e0 msgr2=0x7f2a5014df10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:30.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 -- 172.21.2.104:0/1280036967 shutdown_connections 2021-07-11T04:02:30.440 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.436+0000 7f2a55858700 1 -- 172.21.2.104:0/1280036967 wait complete. 2021-07-11T04:02:30.444 INFO:tasks.cephfs_test_runner:test_root_rctime (tasks.cephfs.test_misc.TestMisc) ... ok 2021-07-11T04:02:30.444 INFO:tasks.cephfs_test_runner:Starting test: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:02:30.445 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs' 2021-07-11T04:02:30.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f840ae99700 1 Processor -- start 2021-07-11T04:02:30.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f840ae99700 1 -- start start 2021-07-11T04:02:30.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f840ae99700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f84040b1db0 0x7f84040b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:30.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f840ae99700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84040b0350 con 0x7f84040a70c0 2021-07-11T04:02:30.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f840ae99700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84040b05f0 con 0x7f84041483a0 2021-07-11T04:02:30.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f840ae99700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84040a90c0 con 0x7f84040b1db0 2021-07-11T04:02:30.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f8409436700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f84040a70c0 0x7f8404144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48286/0 (socket says 172.21.2.104:48286) 2021-07-11T04:02:30.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f8409436700 1 -- 172.21.2.104:0/2870882693 learned_addr learned my addr 172.21.2.104:0/2870882693 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:30.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f8408c35700 1 --2- 172.21.2.104:0/2870882693 >> v2:172.21.2.104:3300/0 conn(0x7f84040b1db0 0x7f84040b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:30.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f8408c35700 1 -- 172.21.2.104:0/2870882693 >> v1:172.21.2.105:6789/0 conn(0x7f84041483a0 legacy=0x7f84040b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:30.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f8408c35700 1 -- 172.21.2.104:0/2870882693 >> v1:172.21.2.104:6789/0 conn(0x7f84040a70c0 legacy=0x7f8404144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:30.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f8408c35700 1 -- 172.21.2.104:0/2870882693 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84040b0890 con 0x7f84040b1db0 2021-07-11T04:02:30.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f8408c35700 1 --2- 172.21.2.104:0/2870882693 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f84040b1db0 0x7f84040b0fd0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 rx=0x7f8404147730 tx=0x7f83ec006df0).ready entity=mon.0 client_cookie=710806dfc7d858f0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:30.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f84037fe700 1 -- 172.21.2.104:0/2870882693 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83ec00d5c0 con 0x7f84040b1db0 2021-07-11T04:02:30.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f84037fe700 1 -- 172.21.2.104:0/2870882693 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec000e90 con 0x7f84040b1db0 2021-07-11T04:02:30.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f84037fe700 1 -- 172.21.2.104:0/2870882693 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83ec00d910 con 0x7f84040b1db0 2021-07-11T04:02:30.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f840ae99700 1 -- 172.21.2.104:0/2870882693 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f84040b1db0 msgr2=0x7f84040b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:30.519 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.512+0000 7f840ae99700 1 --2- 172.21.2.104:0/2870882693 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f84040b1db0 0x7f84040b0fd0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 rx=0x7f8404147730 tx=0x7f83ec006df0).stop 2021-07-11T04:02:30.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 -- 172.21.2.104:0/2870882693 shutdown_connections 2021-07-11T04:02:30.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 --2- 172.21.2.104:0/2870882693 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f84040b1db0 0x7f84040b0fd0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:30.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 -- 172.21.2.104:0/2870882693 >> 172.21.2.104:0/2870882693 conn(0x7f840405f6b0 msgr2=0x7f840405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:30.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 -- 172.21.2.104:0/2870882693 shutdown_connections 2021-07-11T04:02:30.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 -- 172.21.2.104:0/2870882693 wait complete. 2021-07-11T04:02:30.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 Processor -- start 2021-07-11T04:02:30.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 -- start start 2021-07-11T04:02:30.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f84040a70c0 0x7f84040adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:30.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f84040b1db0 0x7f8404156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:30.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f84041483a0 0x7f8404157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:30.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8404156b60 con 0x7f84041483a0 2021-07-11T04:02:30.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840415a0d0 con 0x7f84040b1db0 2021-07-11T04:02:30.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840414b420 con 0x7f84040a70c0 2021-07-11T04:02:30.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8409436700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f84041483a0 0x7f8404157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:30.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8409436700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f84041483a0 0x7f8404157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51522/0 (socket says 172.21.2.104:51522) 2021-07-11T04:02:30.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8409436700 1 -- 172.21.2.104:0/3338581299 learned_addr learned my addr 172.21.2.104:0/3338581299 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:30.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8409436700 1 -- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f84040a70c0 msgr2=0x7f84040adfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:30.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8409436700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f84040a70c0 0x7f84040adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:30.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8409436700 1 -- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f84040b1db0 msgr2=0x7f8404156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:30.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8409436700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f84040b1db0 0x7f8404156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:30.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8409436700 1 -- 172.21.2.104:0/3338581299 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec00b180 con 0x7f84041483a0 2021-07-11T04:02:30.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8409436700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f84041483a0 0x7f8404157440 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7f84040ace60 tx=0x7f83fc007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:30.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8401ffb700 1 -- 172.21.2.104:0/3338581299 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83fc005ba0 con 0x7f84041483a0 2021-07-11T04:02:30.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8401ffb700 1 -- 172.21.2.104:0/3338581299 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc005d00 con 0x7f84041483a0 2021-07-11T04:02:30.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 -- 172.21.2.104:0/3338581299 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840414b6c0 con 0x7f84041483a0 2021-07-11T04:02:30.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 -- 172.21.2.104:0/3338581299 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84040ae540 con 0x7f84041483a0 2021-07-11T04:02:30.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8401ffb700 1 -- 172.21.2.104:0/3338581299 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f83fc0072c0 con 0x7f84041483a0 2021-07-11T04:02:30.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f840ae99700 1 -- 172.21.2.104:0/3338581299 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84040a9c50 con 0x7f84041483a0 2021-07-11T04:02:30.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8401ffb700 1 -- 172.21.2.104:0/3338581299 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f83fc01a060 con 0x7f84041483a0 2021-07-11T04:02:30.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8401ffb700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83e8053260 0x7f83e8055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:30.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8401ffb700 1 -- 172.21.2.104:0/3338581299 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f83fc06adf0 con 0x7f84041483a0 2021-07-11T04:02:30.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.516+0000 7f8408c35700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83e8053260 0x7f83e8055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:30.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.520+0000 7f8408c35700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83e8053260 0x7f83e8055720 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f8404147730 tx=0x7f83ec014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:30.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.520+0000 7f8401ffb700 1 -- 172.21.2.104:0/3338581299 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f83fc042300 con 0x7f84041483a0 2021-07-11T04:02:30.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:30.628+0000 7f840ae99700 1 -- 172.21.2.104:0/3338581299 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]} v 0) v1 -- 0x7f8404057350 con 0x7f84041483a0 2021-07-11T04:02:31.445 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.444+0000 7f8401ffb700 1 -- 172.21.2.104:0/3338581299 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_misc.TestMisc.test_statfs_on_deleted_fs"]}]=0 v408) v1 ==== 139+0+0 (secure 0 0 0) 0x7f83fc018070 con 0x7f84041483a0 2021-07-11T04:02:31.446 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.444+0000 7f840ae99700 1 -- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83e8053260 msgr2=0x7f83e8055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:31.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.444+0000 7f840ae99700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83e8053260 0x7f83e8055720 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f8404147730 tx=0x7f83ec014000).stop 2021-07-11T04:02:31.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.444+0000 7f840ae99700 1 -- 172.21.2.104:0/3338581299 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f84041483a0 msgr2=0x7f8404157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:31.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.448+0000 7f840ae99700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f84041483a0 0x7f8404157440 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7f84040ace60 tx=0x7f83fc007f40).stop 2021-07-11T04:02:31.447 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.448+0000 7f840ae99700 1 -- 172.21.2.104:0/3338581299 shutdown_connections 2021-07-11T04:02:31.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.448+0000 7f840ae99700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f83e8053260 0x7f83e8055720 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:31.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.448+0000 7f840ae99700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f84041483a0 0x7f8404157440 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:31.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.448+0000 7f840ae99700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f84040b1db0 0x7f8404156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.448+0000 7f840ae99700 1 --2- 172.21.2.104:0/3338581299 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f84040a70c0 0x7f84040adfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.448+0000 7f840ae99700 1 -- 172.21.2.104:0/3338581299 >> 172.21.2.104:0/3338581299 conn(0x7f840405f6b0 msgr2=0x7f84040b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:31.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.448+0000 7f840ae99700 1 -- 172.21.2.104:0/3338581299 shutdown_connections 2021-07-11T04:02:31.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.448+0000 7f840ae99700 1 -- 172.21.2.104:0/3338581299 wait complete. 2021-07-11T04:02:31.455 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2021-07-11T04:02:31.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.508+0000 7fe74efbe700 1 Processor -- start 2021-07-11T04:02:31.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- start start 2021-07-11T04:02:31.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe7480b1ec0 0x7fe7480b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe7480b22a0 0x7fe748141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7480b0420 con 0x7fe7480affb0 2021-07-11T04:02:31.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7480a77d0 con 0x7fe7480b1ec0 2021-07-11T04:02:31.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7480b0650 con 0x7fe7480b22a0 2021-07-11T04:02:31.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe747fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fe7480affb0 0x7fe7480a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48298/0 (socket says 172.21.2.104:48298) 2021-07-11T04:02:31.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe7480b1ec0 0x7fe7480b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:31.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe747fff700 1 -- 172.21.2.104:0/4156633567 learned_addr learned my addr 172.21.2.104:0/4156633567 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:31.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 -- 172.21.2.104:0/4156633567 >> v2:172.21.2.105:3300/0 conn(0x7fe7480b22a0 msgr2=0x7fe748141250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:31.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 --2- 172.21.2.104:0/4156633567 >> v2:172.21.2.105:3300/0 conn(0x7fe7480b22a0 0x7fe748141250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 -- 172.21.2.104:0/4156633567 >> v1:172.21.2.104:6789/0 conn(0x7fe7480affb0 legacy=0x7fe7480a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:31.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 -- 172.21.2.104:0/4156633567 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7480b08f0 con 0x7fe7480b1ec0 2021-07-11T04:02:31.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 --2- 172.21.2.104:0/4156633567 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7480b1ec0 0x7fe7480b12d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7fe738002f70 tx=0x7fe738006df0).ready entity=mon.0 client_cookie=c07af1260cf37ce9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:31.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe7477fe700 1 -- 172.21.2.104:0/4156633567 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe73800d5c0 con 0x7fe7480b1ec0 2021-07-11T04:02:31.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe7477fe700 1 -- 172.21.2.104:0/4156633567 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe738000e90 con 0x7fe7480b1ec0 2021-07-11T04:02:31.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe7477fe700 1 -- 172.21.2.104:0/4156633567 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe73800d950 con 0x7fe7480b1ec0 2021-07-11T04:02:31.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- 172.21.2.104:0/4156633567 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7480b1ec0 msgr2=0x7fe7480b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:31.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 --2- 172.21.2.104:0/4156633567 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7480b1ec0 0x7fe7480b12d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7fe738002f70 tx=0x7fe738006df0).stop 2021-07-11T04:02:31.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- 172.21.2.104:0/4156633567 shutdown_connections 2021-07-11T04:02:31.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 --2- 172.21.2.104:0/4156633567 >> v2:172.21.2.105:3300/0 conn(0x7fe7480b22a0 0x7fe748141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 --2- 172.21.2.104:0/4156633567 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7480b1ec0 0x7fe7480b12d0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:31.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- 172.21.2.104:0/4156633567 >> 172.21.2.104:0/4156633567 conn(0x7fe74805f6b0 msgr2=0x7fe74805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:31.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- 172.21.2.104:0/4156633567 shutdown_connections 2021-07-11T04:02:31.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- 172.21.2.104:0/4156633567 wait complete. 2021-07-11T04:02:31.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 Processor -- start 2021-07-11T04:02:31.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- start start 2021-07-11T04:02:31.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7480affb0 0x7fe7480a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe7480b1ec0 0x7fe74814afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe7480b22a0 0x7fe74814b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe748149a80 con 0x7fe7480affb0 2021-07-11T04:02:31.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe748149d20 con 0x7fe7480b1ec0 2021-07-11T04:02:31.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe748149fc0 con 0x7fe7480b22a0 2021-07-11T04:02:31.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7480affb0 0x7fe7480a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:31.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7480affb0 0x7fe7480a9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51538/0 (socket says 172.21.2.104:51538) 2021-07-11T04:02:31.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 -- 172.21.2.104:0/193406865 learned_addr learned my addr 172.21.2.104:0/193406865 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:31.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 -- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe7480b22a0 msgr2=0x7fe74814b4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:31.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe7480b22a0 0x7fe74814b4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 -- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe7480b1ec0 msgr2=0x7fe74814afe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:31.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe7480b1ec0 0x7fe74814afe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 -- 172.21.2.104:0/193406865 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73800b180 con 0x7fe7480affb0 2021-07-11T04:02:31.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74cd5a700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7480affb0 0x7fe7480a9a80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7fe738002f70 tx=0x7fe738014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:31.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe745ffb700 1 -- 172.21.2.104:0/193406865 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe738006b20 con 0x7fe7480affb0 2021-07-11T04:02:31.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe745ffb700 1 -- 172.21.2.104:0/193406865 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73801e430 con 0x7fe7480affb0 2021-07-11T04:02:31.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- 172.21.2.104:0/193406865 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74814a260 con 0x7fe7480affb0 2021-07-11T04:02:31.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe745ffb700 1 -- 172.21.2.104:0/193406865 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe73801e790 con 0x7fe7480affb0 2021-07-11T04:02:31.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.512+0000 7fe74efbe700 1 -- 172.21.2.104:0/193406865 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7480ab6c0 con 0x7fe7480affb0 2021-07-11T04:02:31.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.516+0000 7fe74efbe700 1 -- 172.21.2.104:0/193406865 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74814a3f0 con 0x7fe7480affb0 2021-07-11T04:02:31.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.516+0000 7fe745ffb700 1 -- 172.21.2.104:0/193406865 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe73800f030 con 0x7fe7480affb0 2021-07-11T04:02:31.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.516+0000 7fe745ffb700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe72c053260 0x7fe72c055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.516+0000 7fe745ffb700 1 -- 172.21.2.104:0/193406865 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe73806c390 con 0x7fe7480affb0 2021-07-11T04:02:31.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.516+0000 7fe747fff700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe72c053260 0x7fe72c055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:31.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.516+0000 7fe747fff700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe72c053260 0x7fe72c055720 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fe7480ad2d0 tx=0x7fe730007790).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:31.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.516+0000 7fe745ffb700 1 -- 172.21.2.104:0/193406865 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe738043950 con 0x7fe7480affb0 2021-07-11T04:02:31.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.600+0000 7fe74efbe700 1 -- 172.21.2.104:0/193406865 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe748057350 con 0x7fe7480affb0 2021-07-11T04:02:31.603 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe745ffb700 1 -- 172.21.2.104:0/193406865 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe738046f70 con 0x7fe7480affb0 2021-07-11T04:02:31.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 -- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe72c053260 msgr2=0x7fe72c055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:31.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe72c053260 0x7fe72c055720 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fe7480ad2d0 tx=0x7fe730007790).stop 2021-07-11T04:02:31.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 -- 172.21.2.104:0/193406865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7480affb0 msgr2=0x7fe7480a9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:31.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7480affb0 0x7fe7480a9a80 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7fe738002f70 tx=0x7fe738014000).stop 2021-07-11T04:02:31.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 -- 172.21.2.104:0/193406865 shutdown_connections 2021-07-11T04:02:31.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe7480b22a0 0x7fe74814b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe7480b1ec0 0x7fe74814afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe72c053260 0x7fe72c055720 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:31.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 --2- 172.21.2.104:0/193406865 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe7480affb0 0x7fe7480a9a80 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:31.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 -- 172.21.2.104:0/193406865 >> 172.21.2.104:0/193406865 conn(0x7fe74805f6b0 msgr2=0x7fe74813cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:31.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 -- 172.21.2.104:0/193406865 shutdown_connections 2021-07-11T04:02:31.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.604+0000 7fe74efbe700 1 -- 172.21.2.104:0/193406865 wait complete. 2021-07-11T04:02:31.611 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:02:31.612 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:31.619 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:31.628 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:31.642 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:02:31.655 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:02:31.655 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:31.656 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:31.656 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:31.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:02:31.666 INFO:teuthology.orchestra.run.gibba019.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.0': No such file or directory 2021-07-11T04:02:31.666 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.0 2021-07-11T04:02:31.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:31.667 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:31.667 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:31.703 INFO:teuthology.orchestra.run.gibba023.stderr:stat: cannot read file system information for '/home/ubuntu/cephtest/mnt.1': No such file or directory 2021-07-11T04:02:31.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:02:31.705 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:31.706 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:02:31.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f7542643700 1 Processor -- start 2021-07-11T04:02:31.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f7542643700 1 -- start start 2021-07-11T04:02:31.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f7542643700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f753c0b1db0 0x7f753c0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f7542643700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f753c1483a0 0x7f753c0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f7542643700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753c0b0350 con 0x7f753c0a70c0 2021-07-11T04:02:31.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f7542643700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753c0a90c0 con 0x7f753c0b1db0 2021-07-11T04:02:31.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f7542643700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0b05f0 con 0x7f753c1483a0 2021-07-11T04:02:31.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f7540be0700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f753c0a70c0 0x7f753c144720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48312/0 (socket says 172.21.2.104:48312) 2021-07-11T04:02:31.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f7540be0700 1 -- 172.21.2.104:0/4168400279 learned_addr learned my addr 172.21.2.104:0/4168400279 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:31.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f753bfff700 1 --2- 172.21.2.104:0/4168400279 >> v2:172.21.2.104:3300/0 conn(0x7f753c0b1db0 0x7f753c0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:31.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f753bfff700 1 -- 172.21.2.104:0/4168400279 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f753c1483a0 msgr2=0x7f753c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:31.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f753bfff700 1 --2- 172.21.2.104:0/4168400279 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f753c1483a0 0x7f753c0b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f753bfff700 1 -- 172.21.2.104:0/4168400279 >> v1:172.21.2.104:6789/0 conn(0x7f753c0a70c0 legacy=0x7f753c144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:31.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f753affd700 1 -- 172.21.2.104:0/4168400279 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356913446 0 0) 0x7f753c0b0350 con 0x7f753c0a70c0 2021-07-11T04:02:31.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f753bfff700 1 -- 172.21.2.104:0/4168400279 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c0b0890 con 0x7f753c0b1db0 2021-07-11T04:02:31.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f753bfff700 1 --2- 172.21.2.104:0/4168400279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f753c0b1db0 0x7f753c0b0fd0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7f752c002f70 tx=0x7f752c006df0).ready entity=mon.0 client_cookie=5cef33ddc4b0fe19 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:31.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f753affd700 1 -- 172.21.2.104:0/4168400279 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f752c00d570 con 0x7f753c0b1db0 2021-07-11T04:02:31.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f753affd700 1 -- 172.21.2.104:0/4168400279 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752c000e90 con 0x7f753c0b1db0 2021-07-11T04:02:31.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f753affd700 1 -- 172.21.2.104:0/4168400279 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f752c00d8c0 con 0x7f753c0b1db0 2021-07-11T04:02:31.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f7542643700 1 -- 172.21.2.104:0/4168400279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f753c0b1db0 msgr2=0x7f753c0b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:31.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.764+0000 7f7542643700 1 --2- 172.21.2.104:0/4168400279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f753c0b1db0 0x7f753c0b0fd0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7f752c002f70 tx=0x7f752c006df0).stop 2021-07-11T04:02:31.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 -- 172.21.2.104:0/4168400279 shutdown_connections 2021-07-11T04:02:31.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 --2- 172.21.2.104:0/4168400279 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f753c1483a0 0x7f753c0b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 --2- 172.21.2.104:0/4168400279 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f753c0b1db0 0x7f753c0b0fd0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:31.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 -- 172.21.2.104:0/4168400279 >> 172.21.2.104:0/4168400279 conn(0x7f753c05f6b0 msgr2=0x7f753c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:31.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 -- 172.21.2.104:0/4168400279 shutdown_connections 2021-07-11T04:02:31.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 -- 172.21.2.104:0/4168400279 wait complete. 2021-07-11T04:02:31.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 Processor -- start 2021-07-11T04:02:31.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 -- start start 2021-07-11T04:02:31.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f753c0a70c0 0x7f753c14a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f753c0b1db0 0x7f753c1566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f753c1483a0 0x7f753c156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0b5be0 con 0x7f753c1483a0 2021-07-11T04:02:31.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0b46b0 con 0x7f753c0b1db0 2021-07-11T04:02:31.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0b51a0 con 0x7f753c0a70c0 2021-07-11T04:02:31.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7540be0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f753c1483a0 0x7f753c156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:31.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7540be0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f753c1483a0 0x7f753c156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51554/0 (socket says 172.21.2.104:51554) 2021-07-11T04:02:31.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7540be0700 1 -- 172.21.2.104:0/731112433 learned_addr learned my addr 172.21.2.104:0/731112433 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:31.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7540be0700 1 -- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f753c0a70c0 msgr2=0x7f753c14a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:31.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7540be0700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f753c0a70c0 0x7f753c14a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7540be0700 1 -- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f753c0b1db0 msgr2=0x7f753c1566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:31.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7540be0700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f753c0b1db0 0x7f753c1566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:31.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7540be0700 1 -- 172.21.2.104:0/731112433 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c00b180 con 0x7f753c1483a0 2021-07-11T04:02:31.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7540be0700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f753c1483a0 0x7f753c156b90 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7f7534000f40 tx=0x7f7534007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:31.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f75397fa700 1 -- 172.21.2.104:0/731112433 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7534005c70 con 0x7f753c1483a0 2021-07-11T04:02:31.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 -- 172.21.2.104:0/731112433 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753c0b5740 con 0x7f753c1483a0 2021-07-11T04:02:31.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f75397fa700 1 -- 172.21.2.104:0/731112433 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7534005dd0 con 0x7f753c1483a0 2021-07-11T04:02:31.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f75397fa700 1 -- 172.21.2.104:0/731112433 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f75340072c0 con 0x7f753c1483a0 2021-07-11T04:02:31.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 -- 172.21.2.104:0/731112433 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c14e9d0 con 0x7f753c1483a0 2021-07-11T04:02:31.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f7542643700 1 -- 172.21.2.104:0/731112433 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753c0b5000 con 0x7f753c1483a0 2021-07-11T04:02:31.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f75397fa700 1 -- 172.21.2.104:0/731112433 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f753401a060 con 0x7f753c1483a0 2021-07-11T04:02:31.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.768+0000 7f75397fa700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75200531a0 0x7f7520055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:31.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.772+0000 7f75397fa700 1 -- 172.21.2.104:0/731112433 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f753406ad30 con 0x7f753c1483a0 2021-07-11T04:02:31.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.772+0000 7f753bfff700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75200531a0 0x7f7520055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:31.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.772+0000 7f753bfff700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75200531a0 0x7f7520055660 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f752c002f70 tx=0x7f752c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:31.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.772+0000 7f75397fa700 1 -- 172.21.2.104:0/731112433 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f753c0b5000 con 0x7f753c1483a0 2021-07-11T04:02:31.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.856+0000 7f7542643700 1 -- 172.21.2.104:0/731112433 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f753c0b5000 con 0x7f753c1483a0 2021-07-11T04:02:31.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.856+0000 7f75397fa700 1 -- 172.21.2.104:0/731112433 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v152) v1 ==== 81+0+28835 (secure 0 0 0) 0x7f75340500c0 con 0x7f753c1483a0 2021-07-11T04:02:31.857 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:31.858 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:02:31.859 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 152, 2021-07-11T04:02:31.859 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:02:31.859 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:02:31.859 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:02:29.422185+0000", 2021-07-11T04:02:31.860 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:02:31.860 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:02:31.860 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:02:31.860 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:02:31.861 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:02:31.861 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:02:31.861 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:02:31.861 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:02:31.861 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:02:31.862 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:31.862 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:02:31.862 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:02:31.866 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:02:31.866 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:02:31.867 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:02:31.867 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 28, 2021-07-11T04:02:31.867 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:02:31.868 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:02:31.868 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:02:31.868 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:02:31.869 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:02:31.869 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.869 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:02:31.870 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:02:31.870 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:02:31.870 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:31.871 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:31.871 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:31.871 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:31.871 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:31.871 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:31.872 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:31.872 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:31.872 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:02:31.872 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:02:31.873 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:02:31.873 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:02:31.873 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:02:31.874 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:31.874 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:31.874 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:31.874 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:31.875 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:31.875 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:31.875 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:31.875 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.875 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:02:31.876 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:31.876 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:31.876 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:31.876 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:31.877 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:31.877 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:31.883 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:31.883 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:31.883 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:31.884 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:31.884 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:31.885 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:31.885 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:31.886 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:31.886 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:31.887 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:31.887 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:31.887 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:31.888 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:31.888 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:31.888 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:31.889 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:31.889 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:31.890 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:31.890 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:31.891 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:31.891 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.892 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:31.892 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:31.892 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:31.893 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:31.893 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:31.894 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:31.894 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:31.895 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:31.895 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:31.895 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:31.896 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:31.896 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:02:31.896 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:02:31.897 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.897 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:31.898 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:02:31.898 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.899 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.899 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.901 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:02:31.901 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:02:31.903 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:02:31.904 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:31.904 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:31.905 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:31.905 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:31.905 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:31.905 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:31.909 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:31.910 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:31.910 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:02:31.911 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:02:31.912 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:02:31.912 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:02:31.913 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:02:31.913 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:31.913 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:31.914 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:31.914 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:31.914 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:31.914 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:31.914 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:31.915 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.915 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:02:31.915 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:31.915 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:31.916 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:31.916 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:31.916 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:31.916 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:31.916 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:31.917 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:31.917 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:31.917 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:31.917 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:31.918 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:31.918 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:31.918 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:31.918 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:31.919 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:31.919 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:31.919 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:31.919 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:31.919 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:31.920 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:31.920 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:31.920 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:31.921 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:31.921 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:31.921 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:31.921 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.922 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:31.922 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:31.922 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:31.922 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:31.923 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:31.923 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:31.923 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:31.923 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:31.924 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:31.924 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:31.924 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:31.924 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:02:31.925 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:31.925 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:02:31.925 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.925 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.925 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:31.926 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:02:31.926 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.926 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:02:31.926 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:02:31.926 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:31.927 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:31.929 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:31.929 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:31.929 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:31.930 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:31.930 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:31.930 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 146, 2021-07-11T04:02:31.930 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:31.931 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:31.931 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:31.931 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.931 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.932 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.932 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:02:31.932 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:31.932 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.932 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.933 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.933 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:02:31.933 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:31.933 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.934 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.934 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.934 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:31.934 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.935 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.935 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.935 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:02:31.935 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:31.936 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.936 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.936 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.936 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:02:31.937 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:31.937 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.937 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.937 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.938 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:31.938 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.938 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.938 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.938 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:02:31.939 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:31.939 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.939 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.939 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.940 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:02:31.940 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:31.940 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.940 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.941 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.941 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:31.941 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.941 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.942 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.942 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:02:31.942 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:31.942 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.942 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.943 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.943 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:02:31.943 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:31.943 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.944 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.944 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.944 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:02:31.944 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:02:31.944 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:02:31.945 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:02:31.945 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:31.945 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:31.945 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:31.946 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.946 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.946 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.946 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:02:31.946 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:02:31.947 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:31.947 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:31.947 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:31.947 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:31.948 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:31.948 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:31.948 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:31.948 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 146, 2021-07-11T04:02:31.949 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:31.949 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:31.949 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:31.949 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.950 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.950 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.950 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:02:31.950 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:31.951 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.951 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.951 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.951 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:02:31.952 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:31.952 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.952 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.952 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.953 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:31.953 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.953 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.953 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.953 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:02:31.954 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:31.954 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.954 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.954 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.955 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:02:31.955 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:31.955 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.955 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.956 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.956 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:31.956 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.956 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.957 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.957 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:02:31.957 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:31.957 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.957 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.958 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.958 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:02:31.958 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:31.958 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.959 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.959 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.959 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:31.959 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.959 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.960 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.960 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:02:31.960 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:31.960 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.961 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.961 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.961 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:02:31.961 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:31.961 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.962 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.962 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.962 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:02:31.963 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:02:31.963 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:02:31.963 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:02:31.963 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:31.964 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:31.964 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:31.964 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.964 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.965 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.965 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:02:31.965 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:02:31.965 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:31.965 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:31.966 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:31.966 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:31.966 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:31.966 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:31.967 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:31.967 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 145, 2021-07-11T04:02:31.967 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:31.967 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:31.967 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:31.968 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.968 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.968 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.968 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:02:31.969 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:31.969 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.969 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.969 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.969 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:02:31.970 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:31.970 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.970 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.970 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.971 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:31.971 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.971 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.971 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.971 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:02:31.972 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:31.972 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.972 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.972 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.972 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:02:31.973 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:31.973 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.973 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.973 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.974 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:31.974 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.974 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.974 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.974 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:02:31.975 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:31.975 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.975 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.975 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.976 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:02:31.976 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:31.976 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.976 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.976 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.977 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:31.977 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.977 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.977 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.977 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:02:31.978 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:31.978 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.978 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.978 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.979 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:02:31.979 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:31.979 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.979 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.979 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.980 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:02:31.980 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:02:31.980 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:02:31.980 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:02:31.981 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:31.981 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:31.981 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:31.981 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.981 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.982 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.982 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:02:31.982 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:02:31.982 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:31.982 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:31.983 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:31.983 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:31.983 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:31.983 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:31.984 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:31.984 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 136, 2021-07-11T04:02:31.984 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:31.984 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:31.984 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:31.985 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.985 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.985 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.985 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:02:31.985 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:31.986 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.986 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.986 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.986 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:02:31.987 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:31.987 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.987 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.987 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.987 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:31.988 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.988 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.988 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.988 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:02:31.989 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:31.989 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.989 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.989 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.989 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:02:31.990 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:31.990 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.990 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.990 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.991 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:31.991 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.991 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.991 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.991 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:02:31.992 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:31.992 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.992 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.992 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.993 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:02:31.993 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:31.993 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.993 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.993 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.994 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:31.994 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:31.994 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.995 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:31.995 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:02:31.995 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:31.995 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.996 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.996 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:31.996 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:02:31.996 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:31.996 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:31.997 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.997 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.997 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:02:31.997 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:02:31.998 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:02:31.998 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:02:31.998 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:31.998 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:31.999 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:31.999 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:31.999 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:31.999 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:31.999 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:02:32.000 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:02:32.000 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:32.000 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:32.000 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:32.001 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:32.001 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:32.001 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:32.001 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:32.002 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 145, 2021-07-11T04:02:32.002 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:32.002 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:32.002 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:32.002 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.003 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.003 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.003 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:02:32.003 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:32.004 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.004 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.004 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.004 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:02:32.004 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:32.005 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.005 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.005 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.005 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:32.006 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.006 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.006 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.006 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:02:32.007 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:32.007 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.007 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.007 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.007 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:02:32.008 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:32.008 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.008 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.008 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.008 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:32.009 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.009 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.009 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.009 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:02:32.010 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:32.010 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.010 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.010 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.011 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:02:32.011 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:32.011 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.011 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.011 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.012 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:32.012 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.012 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.012 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.013 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:02:32.013 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:32.013 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.013 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.014 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.014 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:02:32.014 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:32.014 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.014 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.015 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.015 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:02:32.015 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:02:32.015 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:02:32.016 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:02:32.016 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:32.016 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:32.016 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:32.016 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.017 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.017 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.017 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:02:32.017 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:02:32.018 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:32.018 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:32.018 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:32.018 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:32.019 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:32.019 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:32.019 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:32.019 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 146, 2021-07-11T04:02:32.019 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:32.020 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:32.020 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:32.020 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.020 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.021 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.021 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:02:32.021 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:32.021 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.021 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.022 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.022 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:02:32.022 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:32.022 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.023 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.023 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.023 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:32.023 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.024 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.024 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.024 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:02:32.024 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:32.024 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.025 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.025 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.025 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:02:32.025 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:32.026 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.026 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.026 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.026 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:32.026 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.027 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.027 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.027 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:02:32.027 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:32.028 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.028 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.028 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.028 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:02:32.028 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:32.029 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.029 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.029 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.029 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:32.030 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.030 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.030 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.030 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:02:32.031 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:32.031 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.031 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.031 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.032 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:02:32.032 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:32.032 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.032 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.032 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.033 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:02:32.033 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:02:32.033 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:02:32.033 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:02:32.034 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:32.034 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:32.034 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:32.034 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.034 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.035 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.035 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:02:32.035 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:02:32.035 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:32.036 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:32.036 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:32.036 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:32.036 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:32.036 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:32.037 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:32.037 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 146, 2021-07-11T04:02:32.037 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:32.037 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:32.038 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:32.038 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.038 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.038 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.038 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:02:32.039 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:32.039 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.039 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.039 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.040 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:02:32.040 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:32.040 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.040 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.040 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.041 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:32.041 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.041 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.041 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.042 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:02:32.042 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:32.042 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.042 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.043 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.043 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:02:32.043 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:32.043 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.044 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.044 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.044 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:32.044 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.044 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.045 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.045 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:02:32.045 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:32.045 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.045 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.046 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.046 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:02:32.046 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:32.046 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.047 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.047 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.047 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:32.047 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.047 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.048 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.048 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:02:32.048 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:32.048 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.049 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.049 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.049 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:02:32.049 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:32.049 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.050 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.050 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.050 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:02:32.050 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:02:32.051 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:02:32.051 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:02:32.051 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:32.051 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:32.051 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:32.052 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.052 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.052 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.052 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:02:32.052 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:02:32.053 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:32.053 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:32.053 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:32.053 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:32.054 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:32.054 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:32.054 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:32.054 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 146, 2021-07-11T04:02:32.054 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:32.055 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:32.055 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:32.055 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.055 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.056 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.056 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:02:32.056 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:32.056 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.056 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.057 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.057 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:02:32.057 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:32.057 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.058 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.058 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.058 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:32.058 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.058 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.059 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.059 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:02:32.059 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:32.059 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.060 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.060 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.060 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:02:32.060 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:32.060 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.061 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.061 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.061 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:32.061 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.062 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.062 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.062 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:02:32.062 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:32.063 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.063 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.063 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.063 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:02:32.063 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:32.064 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.064 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.064 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.064 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:32.065 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:32.065 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.065 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:32.065 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:02:32.065 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:32.066 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.066 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.066 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:32.066 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:02:32.067 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:32.067 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.067 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.067 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.067 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:02:32.068 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:02:32.068 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:02:32.068 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:02:32.068 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:32.069 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:32.069 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:32.069 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:32.069 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.069 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:32.070 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:02:32.070 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.070 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:02:32.070 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:32.071 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:32.071 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:32.071 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:32.071 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:32.071 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:02:32.072 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:32.072 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.072 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.072 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:02:32.073 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:32.074 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:32.074 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:32.074 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:32.074 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:32.075 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:02:32.075 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:32.075 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.075 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.075 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:02:32.076 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:32.076 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:32.076 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:32.076 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:32.077 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:32.077 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:02:32.077 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:32.077 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.078 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.078 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:02:32.078 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:32.078 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:32.078 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:32.079 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:32.079 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:32.079 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:02:32.079 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:32.080 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.080 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.080 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:02:32.080 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:32.080 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:32.081 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:32.081 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:32.081 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:32.081 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:02:32.081 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:32.082 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.082 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.082 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:02:32.082 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:32.083 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:32.083 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:32.083 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:32.083 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:32.083 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:02:32.084 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:32.084 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.084 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.084 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:02:32.085 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:32.085 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:32.085 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:32.085 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:32.085 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:32.086 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:02:32.086 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:32.086 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.086 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:32.086 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:02:32.087 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:32.087 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:32.087 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:32.087 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:32.088 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:32.088 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:02:32.088 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:32.088 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.088 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:32.089 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:02:32.089 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:02:32.089 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:02:32.089 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:02:32.090 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": { 2021-07-11T04:02:32.090 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.105:6835/130583442": "2021-07-12T04:02:25.834056+0000", 2021-07-11T04:02:32.090 INFO:teuthology.orchestra.run.gibba004.stdout: "172.21.2.105:6834/130583442": "2021-07-12T04:02:25.834056+0000" 2021-07-11T04:02:32.090 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.090 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:02:32.091 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:02:32.091 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:02:32.091 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:02:32.091 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:02:32.092 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:02:32.092 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:02:32.092 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:02:32.092 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:32.092 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:32.093 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:02:32.093 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:02:32.093 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:02:32.093 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:02:32.093 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:02:32.094 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:02:32.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 -- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75200531a0 msgr2=0x7f7520055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75200531a0 0x7f7520055660 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f752c002f70 tx=0x7f752c015000).stop 2021-07-11T04:02:32.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 -- 172.21.2.104:0/731112433 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f753c1483a0 msgr2=0x7f753c156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f753c1483a0 0x7f753c156b90 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 rx=0x7f7534000f40 tx=0x7f7534007d80).stop 2021-07-11T04:02:32.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 -- 172.21.2.104:0/731112433 shutdown_connections 2021-07-11T04:02:32.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f75200531a0 0x7f7520055660 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f753c1483a0 0x7f753c156b90 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f753c0b1db0 0x7f753c1566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 --2- 172.21.2.104:0/731112433 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f753c0a70c0 0x7f753c14a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 -- 172.21.2.104:0/731112433 >> 172.21.2.104:0/731112433 conn(0x7f753c05f6b0 msgr2=0x7f753c147bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:32.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 -- 172.21.2.104:0/731112433 shutdown_connections 2021-07-11T04:02:32.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:31.860+0000 7f7542643700 1 -- 172.21.2.104:0/731112433 wait complete. 2021-07-11T04:02:32.100 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:32.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20aed3d700 1 Processor -- start 2021-07-11T04:02:32.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20aed3d700 1 -- start start 2021-07-11T04:02:32.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20aed3d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f20a80a76b0 0x7f20a8066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20aed3d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20a80b1970 0x7f20a8066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20aed3d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a8067450 con 0x7f20a80b0060 2021-07-11T04:02:32.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20aed3d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a80a7a90 con 0x7f20a80a76b0 2021-07-11T04:02:32.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20aed3d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a8147620 con 0x7f20a80b1970 2021-07-11T04:02:32.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20ad2da700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f20a80b0060 0x7f20a805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48330/0 (socket says 172.21.2.104:48330) 2021-07-11T04:02:32.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20ad2da700 1 -- 172.21.2.104:0/3944276538 learned_addr learned my addr 172.21.2.104:0/3944276538 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:32.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20acad9700 1 --2- 172.21.2.104:0/3944276538 >> v2:172.21.2.104:3300/0 conn(0x7f20a80a76b0 0x7f20a8066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20acad9700 1 -- 172.21.2.104:0/3944276538 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20a80b1970 msgr2=0x7f20a8066f60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:32.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20acad9700 1 --2- 172.21.2.104:0/3944276538 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20a80b1970 0x7f20a8066f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20acad9700 1 -- 172.21.2.104:0/3944276538 >> v1:172.21.2.104:6789/0 conn(0x7f20a80b0060 legacy=0x7f20a805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20acad9700 1 -- 172.21.2.104:0/3944276538 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a81478c0 con 0x7f20a80a76b0 2021-07-11T04:02:32.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20acad9700 1 --2- 172.21.2.104:0/3944276538 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20a80a76b0 0x7f20a8066a70 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 rx=0x7f20a0001490 tx=0x7f2098006df0).ready entity=mon.0 client_cookie=ccc14e091ad28c1e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:32.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.160+0000 7f20a77fe700 1 -- 172.21.2.104:0/3944276538 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f209800d570 con 0x7f20a80a76b0 2021-07-11T04:02:32.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a77fe700 1 -- 172.21.2.104:0/3944276538 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2098000e90 con 0x7f20a80a76b0 2021-07-11T04:02:32.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a77fe700 1 -- 172.21.2.104:0/3944276538 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f209800d8c0 con 0x7f20a80a76b0 2021-07-11T04:02:32.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- 172.21.2.104:0/3944276538 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20a80a76b0 msgr2=0x7f20a8066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 --2- 172.21.2.104:0/3944276538 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20a80a76b0 0x7f20a8066a70 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 rx=0x7f20a0001490 tx=0x7f2098006df0).stop 2021-07-11T04:02:32.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- 172.21.2.104:0/3944276538 shutdown_connections 2021-07-11T04:02:32.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 --2- 172.21.2.104:0/3944276538 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20a80b1970 0x7f20a8066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 --2- 172.21.2.104:0/3944276538 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20a80a76b0 0x7f20a8066a70 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- 172.21.2.104:0/3944276538 >> 172.21.2.104:0/3944276538 conn(0x7f20a80b53e0 msgr2=0x7f20a80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:32.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- 172.21.2.104:0/3944276538 shutdown_connections 2021-07-11T04:02:32.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- 172.21.2.104:0/3944276538 wait complete. 2021-07-11T04:02:32.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 Processor -- start 2021-07-11T04:02:32.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- start start 2021-07-11T04:02:32.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20a80a76b0 0x7f20a80b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20a80b0060 0x7f20a80abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20a80b1970 0x7f20a80aac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80ac2c0 con 0x7f20a80b0060 2021-07-11T04:02:32.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80b4430 con 0x7f20a80b1970 2021-07-11T04:02:32.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80adb90 con 0x7f20a80a76b0 2021-07-11T04:02:32.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20a80b0060 0x7f20a80abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a7fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20a80b0060 0x7f20a80abdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51566/0 (socket says 172.21.2.104:51566) 2021-07-11T04:02:32.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a7fff700 1 -- 172.21.2.104:0/642911369 learned_addr learned my addr 172.21.2.104:0/642911369 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:32.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a7fff700 1 -- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20a80a76b0 msgr2=0x7f20a80b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a7fff700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20a80a76b0 0x7f20a80b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a7fff700 1 -- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20a80b1970 msgr2=0x7f20a80aac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a7fff700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20a80b1970 0x7f20a80aac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a7fff700 1 -- 172.21.2.104:0/642911369 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209800b180 con 0x7f20a80b0060 2021-07-11T04:02:32.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a7fff700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20a80b0060 0x7f20a80abdd0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f209c000ed0 tx=0x7f209c006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:32.178 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a5ffb700 1 -- 172.21.2.104:0/642911369 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f209c012410 con 0x7f20a80b0060 2021-07-11T04:02:32.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a5ffb700 1 -- 172.21.2.104:0/642911369 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c012570 con 0x7f20a80b0060 2021-07-11T04:02:32.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- 172.21.2.104:0/642911369 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a80ade30 con 0x7f20a80b0060 2021-07-11T04:02:32.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a5ffb700 1 -- 172.21.2.104:0/642911369 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f209c0128d0 con 0x7f20a80b0060 2021-07-11T04:02:32.179 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- 172.21.2.104:0/642911369 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a80ab930 con 0x7f20a80b0060 2021-07-11T04:02:32.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20aed3d700 1 -- 172.21.2.104:0/642911369 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a80abaf0 con 0x7f20a80b0060 2021-07-11T04:02:32.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a5ffb700 1 -- 172.21.2.104:0/642911369 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f209c00f030 con 0x7f20a80b0060 2021-07-11T04:02:32.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a5ffb700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2088053350 0x7f2088055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.180 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.164+0000 7f20a5ffb700 1 -- 172.21.2.104:0/642911369 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f209c06af30 con 0x7f20a80b0060 2021-07-11T04:02:32.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.168+0000 7f20acad9700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2088053350 0x7f2088055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.168+0000 7f20acad9700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2088053350 0x7f2088055810 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f20a0001490 tx=0x7f2098014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:32.181 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.168+0000 7f20a5ffb700 1 -- 172.21.2.104:0/642911369 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f209c042440 con 0x7f20a80b0060 2021-07-11T04:02:32.276 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 -- 172.21.2.104:0/642911369 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20a8057350 con 0x7f20a80b0060 2021-07-11T04:02:32.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20a5ffb700 1 -- 172.21.2.104:0/642911369 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+1556 (secure 0 0 0) 0x7f209c045a60 con 0x7f20a80b0060 2021-07-11T04:02:32.277 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:32.277 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/1893950346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":1893950346},{"type":"v1","addr":"172.21.2.105:6835","nonce":1893950346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/696025904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":696025904},{"type":"v1","addr":"172.21.2.104:6833","nonce":696025904}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8412,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/438569403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":438569403},{"type":"v1","addr":"172.21.2.104:6835","nonce":438569403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[]} 2021-07-11T04:02:32.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 -- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2088053350 msgr2=0x7f2088055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2088053350 0x7f2088055810 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f20a0001490 tx=0x7f2098014000).stop 2021-07-11T04:02:32.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 -- 172.21.2.104:0/642911369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20a80b0060 msgr2=0x7f20a80abdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20a80b0060 0x7f20a80abdd0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 rx=0x7f209c000ed0 tx=0x7f209c006d70).stop 2021-07-11T04:02:32.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 -- 172.21.2.104:0/642911369 shutdown_connections 2021-07-11T04:02:32.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f20a80b1970 0x7f20a80aac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f2088053350 0x7f2088055810 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f20a80b0060 0x7f20a80abdd0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 --2- 172.21.2.104:0/642911369 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f20a80a76b0 0x7f20a80b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 -- 172.21.2.104:0/642911369 >> 172.21.2.104:0/642911369 conn(0x7f20a80b53e0 msgr2=0x7f20a8145c00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:32.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 -- 172.21.2.104:0/642911369 shutdown_connections 2021-07-11T04:02:32.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.276+0000 7f20aed3d700 1 -- 172.21.2.104:0/642911369 wait complete. 2021-07-11T04:02:32.281 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 77 2021-07-11T04:02:32.284 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:32.335 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f81577ca700 1 Processor -- start 2021-07-11T04:02:32.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f81577ca700 1 -- start start 2021-07-11T04:02:32.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f81577ca700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f81500a76b0 0x7f8150066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.336 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f81577ca700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500b1970 0x7f8150066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f81577ca700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8150067450 con 0x7f81500b0060 2021-07-11T04:02:32.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f81577ca700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81500a7a90 con 0x7f81500a76b0 2021-07-11T04:02:32.337 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f81577ca700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8150147620 con 0x7f81500b1970 2021-07-11T04:02:32.345 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155d67700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f81500b0060 0x7f815005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48342/0 (socket says 172.21.2.104:48342) 2021-07-11T04:02:32.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155d67700 1 -- 172.21.2.104:0/3108555104 learned_addr learned my addr 172.21.2.104:0/3108555104 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:32.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339308192 0 0) 0x7f8150067450 con 0x7f81500b0060 2021-07-11T04:02:32.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8154d65700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500b1970 0x7f8150066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.346 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8138003260 con 0x7f81500b0060 2021-07-11T04:02:32.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155566700 1 --2- 172.21.2.104:0/3108555104 >> v2:172.21.2.105:3300/0 conn(0x7f81500a76b0 0x7f8150066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 967878606 0 0) 0x7f8138003260 con 0x7f81500b0060 2021-07-11T04:02:32.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500b1970 msgr2=0x7f8150066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:32.347 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500b1970 0x7f8150066f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 >> v2:172.21.2.105:3300/0 conn(0x7f81500a76b0 msgr2=0x7f8150066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:32.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 --2- 172.21.2.104:0/3108555104 >> v2:172.21.2.105:3300/0 conn(0x7f81500a76b0 0x7f8150066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81501478c0 con 0x7f81500b0060 2021-07-11T04:02:32.348 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f81480019e0 con 0x7f81500b0060 2021-07-11T04:02:32.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 >> v1:172.21.2.104:6789/0 conn(0x7f81500b0060 legacy=0x7f815005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81380037d0 0x7f8138003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500b1970 0x7f8138004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81380049f0 0x7f8138004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.349 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500a7a90 con 0x7f81380049f0 2021-07-11T04:02:32.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8150147620 con 0x7f81380037d0 2021-07-11T04:02:32.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813800b7d0 con 0x7f81500b1970 2021-07-11T04:02:32.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155d67700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81380049f0 0x7f8138004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155566700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81380037d0 0x7f8138003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.350 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8154d65700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500b1970 0x7f8138004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155d67700 1 -- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500b1970 msgr2=0x7f8138004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:32.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155d67700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500b1970 0x7f8138004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155d67700 1 -- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81380037d0 msgr2=0x7f8138003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:32.351 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155d67700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f81380037d0 0x7f8138003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155d67700 1 -- 172.21.2.104:0/3108555104 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8138003260 con 0x7f81380049f0 2021-07-11T04:02:32.352 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8154d65700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500b1970 0x7f8138004500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:32.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155d67700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81380049f0 0x7f8138004e70 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7f81480016f0 tx=0x7f8148002100).ready entity=mon.0 client_cookie=eb7797e6c3509fcb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:32.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f814800cbb0 con 0x7f81380049f0 2021-07-11T04:02:32.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814800cd10 con 0x7f81380049f0 2021-07-11T04:02:32.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8143fff700 1 -- 172.21.2.104:0/3108555104 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8148005b80 con 0x7f81380049f0 2021-07-11T04:02:32.353 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f81577ca700 1 -- 172.21.2.104:0/3108555104 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81380049f0 msgr2=0x7f8138004e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f81577ca700 1 --2- 172.21.2.104:0/3108555104 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81380049f0 0x7f8138004e70 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 rx=0x7f81480016f0 tx=0x7f8148002100).stop 2021-07-11T04:02:32.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.336+0000 7f8155d67700 1 -- 172.21.2.104:0/3108555104 reap_dead start 2021-07-11T04:02:32.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 -- 172.21.2.104:0/3108555104 shutdown_connections 2021-07-11T04:02:32.354 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 -- 172.21.2.104:0/3108555104 >> 172.21.2.104:0/3108555104 conn(0x7f81500b53e0 msgr2=0x7f81500b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:32.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 -- 172.21.2.104:0/3108555104 shutdown_connections 2021-07-11T04:02:32.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 -- 172.21.2.104:0/3108555104 wait complete. 2021-07-11T04:02:32.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 Processor -- start 2021-07-11T04:02:32.355 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 -- start start 2021-07-11T04:02:32.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8150146740 0x7f81500af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81500aef90 0x7f81500ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500aa560 0x7f8150144600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.356 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500ac860 con 0x7f81500aef90 2021-07-11T04:02:32.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8150146b20 con 0x7f8150146740 2021-07-11T04:02:32.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81500ac250 con 0x7f81500aa560 2021-07-11T04:02:32.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f8154d65700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81500aef90 0x7f81500ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.357 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f8154d65700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81500aef90 0x7f81500ab110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51586/0 (socket says 172.21.2.104:51586) 2021-07-11T04:02:32.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f8154d65700 1 -- 172.21.2.104:0/1352265667 learned_addr learned my addr 172.21.2.104:0/1352265667 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:32.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f8154d65700 1 -- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500aa560 msgr2=0x7f8150144600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f8154d65700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500aa560 0x7f8150144600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f8154d65700 1 -- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8150146740 msgr2=0x7f81500af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.358 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f8154d65700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8150146740 0x7f81500af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f8154d65700 1 -- 172.21.2.104:0/1352265667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81500ac5c0 con 0x7f81500aef90 2021-07-11T04:02:32.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f8154d65700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81500aef90 0x7f81500ab110 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f814c00b000 tx=0x7f814c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:32.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81427fc700 1 -- 172.21.2.104:0/1352265667 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f814c017070 con 0x7f81500aef90 2021-07-11T04:02:32.359 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81427fc700 1 -- 172.21.2.104:0/1352265667 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c000e90 con 0x7f81500aef90 2021-07-11T04:02:32.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 -- 172.21.2.104:0/1352265667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81500acb30 con 0x7f81500aef90 2021-07-11T04:02:32.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81427fc700 1 -- 172.21.2.104:0/1352265667 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f814c011980 con 0x7f81500aef90 2021-07-11T04:02:32.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 -- 172.21.2.104:0/1352265667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81500aaa10 con 0x7f81500aef90 2021-07-11T04:02:32.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81577ca700 1 -- 172.21.2.104:0/1352265667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8150057350 con 0x7f81500aef90 2021-07-11T04:02:32.360 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81427fc700 1 -- 172.21.2.104:0/1352265667 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f814c011ae0 con 0x7f81500aef90 2021-07-11T04:02:32.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81427fc700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81300532b0 0x7f8130055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.340+0000 7f81427fc700 1 -- 172.21.2.104:0/1352265667 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f814c06b290 con 0x7f81500aef90 2021-07-11T04:02:32.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.344+0000 7f8155566700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81300532b0 0x7f8130055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.344+0000 7f8155566700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81300532b0 0x7f8130055770 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f814400c000 tx=0x7f8144008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:32.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.344+0000 7f81427fc700 1 -- 172.21.2.104:0/1352265667 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f814c0428b0 con 0x7f81500aef90 2021-07-11T04:02:32.452 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.452+0000 7f81577ca700 1 -- 172.21.2.104:0/1352265667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81500698e0 con 0x7f81500aef90 2021-07-11T04:02:32.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.452+0000 7f81427fc700 1 -- 172.21.2.104:0/1352265667 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+1556 (secure 0 0 0) 0x7f814c045ed0 con 0x7f81500aef90 2021-07-11T04:02:32.454 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:32.454 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":77,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/1893950346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":1893950346},{"type":"v1","addr":"172.21.2.105:6835","nonce":1893950346}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":77},{"gid":8409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.104:6833/696025904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":696025904},{"type":"v1","addr":"172.21.2.104:6833","nonce":696025904}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":73},{"gid":8412,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6835/438569403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":438569403},{"type":"v1","addr":"172.21.2.104:6835","nonce":438569403}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":72}],"filesystems":[]} 2021-07-11T04:02:32.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 -- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81300532b0 msgr2=0x7f8130055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81300532b0 0x7f8130055770 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f814400c000 tx=0x7f8144008040).stop 2021-07-11T04:02:32.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 -- 172.21.2.104:0/1352265667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81500aef90 msgr2=0x7f81500ab110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81500aef90 0x7f81500ab110 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 rx=0x7f814c00b000 tx=0x7f814c008040).stop 2021-07-11T04:02:32.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 -- 172.21.2.104:0/1352265667 shutdown_connections 2021-07-11T04:02:32.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f81500aa560 0x7f8150144600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f81300532b0 0x7f8130055770 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f81500aef90 0x7f81500ab110 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 --2- 172.21.2.104:0/1352265667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8150146740 0x7f81500af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 -- 172.21.2.104:0/1352265667 >> 172.21.2.104:0/1352265667 conn(0x7f81500b53e0 msgr2=0x7f81500b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:32.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 -- 172.21.2.104:0/1352265667 shutdown_connections 2021-07-11T04:02:32.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.456+0000 7f81577ca700 1 -- 172.21.2.104:0/1352265667 wait complete. 2021-07-11T04:02:32.458 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 77 2021-07-11T04:02:32.461 INFO:tasks.ceph.mds.a:Restarting daemon 2021-07-11T04:02:32.461 INFO:tasks.ceph.mds.a:Stopping old one... 2021-07-11T04:02:32.462 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T04:02:32.462 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:02:32.462 INFO:tasks.ceph.mds.c:Restarting daemon 2021-07-11T04:02:32.463 INFO:tasks.ceph.mds.c:Stopping old one... 2021-07-11T04:02:32.463 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T04:02:32.463 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:02:32.464 INFO:tasks.ceph.mds.b:Restarting daemon 2021-07-11T04:02:32.464 INFO:tasks.ceph.mds.b:Stopping old one... 2021-07-11T04:02:32.464 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T04:02:32.464 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:02:32.468 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T04:02:32.468 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2021-07-11T04:02:32.469 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T04:02:32.470 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2021-07-11T04:02:32.471 INFO:tasks.ceph.mds.a:Started 2021-07-11T04:02:32.472 DEBUG:teuthology.parallel:result is None 2021-07-11T04:02:32.473 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T04:02:32.473 DEBUG:teuthology.orchestra.run.gibba005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2021-07-11T04:02:32.474 INFO:tasks.ceph.mds.c:Started 2021-07-11T04:02:32.474 DEBUG:teuthology.parallel:result is None 2021-07-11T04:02:32.478 INFO:tasks.ceph.mds.b:Started 2021-07-11T04:02:32.478 DEBUG:teuthology.parallel:result is None 2021-07-11T04:02:32.479 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blacklist clear 2021-07-11T04:02:32.504 INFO:tasks.ceph.mds.a.gibba004.stdout:starting mds.a at 2021-07-11T04:02:32.505 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T04:02:32.515 INFO:tasks.ceph.mds.b.gibba005.stdout:starting mds.b at 2021-07-11T04:02:32.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.532+0000 7fe314ff7700 1 Processor -- start 2021-07-11T04:02:32.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- start start 2021-07-11T04:02:32.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe310143540 0x7fe3100a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fe31005f6b0 0x7fe310143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3100a8b60 0x7fe3100b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3100ab3e0 con 0x7fe310143540 2021-07-11T04:02:32.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3100b1b90 con 0x7fe31005f6b0 2021-07-11T04:02:32.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe310143fa0 con 0x7fe3100a8b60 2021-07-11T04:02:32.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30e59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe310143540 0x7fe3100a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30e59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe310143540 0x7fe3100a9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51616/0 (socket says 172.21.2.104:51616) 2021-07-11T04:02:32.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30e59c700 1 -- 172.21.2.104:0/431800836 learned_addr learned my addr 172.21.2.104:0/431800836 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:32.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30e59c700 1 -- 172.21.2.104:0/431800836 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3100a8b60 msgr2=0x7fe3100b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:32.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30e59c700 1 --2- 172.21.2.104:0/431800836 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3100a8b60 0x7fe3100b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30e59c700 1 -- 172.21.2.104:0/431800836 >> v2:172.21.2.105:3300/0 conn(0x7fe31005f6b0 msgr2=0x7fe310143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:32.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30e59c700 1 --2- 172.21.2.104:0/431800836 >> v2:172.21.2.105:3300/0 conn(0x7fe31005f6b0 0x7fe310143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30e59c700 1 -- 172.21.2.104:0/431800836 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe310144240 con 0x7fe310143540 2021-07-11T04:02:32.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30e59c700 1 --2- 172.21.2.104:0/431800836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe310143540 0x7fe3100a9300 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7fe3100ac320 tx=0x7fe300006f40).ready entity=mon.0 client_cookie=60699952e7fd347c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:32.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30d59a700 1 -- 172.21.2.104:0/431800836 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe30000da20 con 0x7fe310143540 2021-07-11T04:02:32.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30d59a700 1 -- 172.21.2.104:0/431800836 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe300000e90 con 0x7fe310143540 2021-07-11T04:02:32.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30d59a700 1 -- 172.21.2.104:0/431800836 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe30000ddb0 con 0x7fe310143540 2021-07-11T04:02:32.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- 172.21.2.104:0/431800836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe310143540 msgr2=0x7fe3100a9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 --2- 172.21.2.104:0/431800836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe310143540 0x7fe3100a9300 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 rx=0x7fe3100ac320 tx=0x7fe300006f40).stop 2021-07-11T04:02:32.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- 172.21.2.104:0/431800836 shutdown_connections 2021-07-11T04:02:32.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 --2- 172.21.2.104:0/431800836 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3100a8b60 0x7fe3100b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 --2- 172.21.2.104:0/431800836 >> v2:172.21.2.105:3300/0 conn(0x7fe31005f6b0 0x7fe310143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 --2- 172.21.2.104:0/431800836 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe310143540 0x7fe3100a9300 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:32.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- 172.21.2.104:0/431800836 >> 172.21.2.104:0/431800836 conn(0x7fe3100b53e0 msgr2=0x7fe3100b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:32.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- 172.21.2.104:0/431800836 shutdown_connections 2021-07-11T04:02:32.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- 172.21.2.104:0/431800836 wait complete. 2021-07-11T04:02:32.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 Processor -- start 2021-07-11T04:02:32.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- start start 2021-07-11T04:02:32.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe31005f6b0 0x7fe31013b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3100a8b60 0x7fe310141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe310143540 0x7fe3100ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe310152430 con 0x7fe310143540 2021-07-11T04:02:32.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3101526d0 con 0x7fe31005f6b0 2021-07-11T04:02:32.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe310142480 con 0x7fe3100a8b60 2021-07-11T04:02:32.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe310143540 0x7fe3100ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe310143540 0x7fe3100ac590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51626/0 (socket says 172.21.2.104:51626) 2021-07-11T04:02:32.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30ed9d700 1 -- 172.21.2.104:0/2264004781 learned_addr learned my addr 172.21.2.104:0/2264004781 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:32.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30ed9d700 1 -- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3100a8b60 msgr2=0x7fe310141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30ed9d700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3100a8b60 0x7fe310141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30ed9d700 1 -- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe31005f6b0 msgr2=0x7fe31013b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:32.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30ed9d700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe31005f6b0 0x7fe31013b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:32.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30ed9d700 1 -- 172.21.2.104:0/2264004781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30000b460 con 0x7fe310143540 2021-07-11T04:02:32.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe30ed9d700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe310143540 0x7fe3100ac590 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 rx=0x7fe3080032f0 tx=0x7fe3080091a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:32.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe3077fe700 1 -- 172.21.2.104:0/2264004781 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe308005c30 con 0x7fe310143540 2021-07-11T04:02:32.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe3077fe700 1 -- 172.21.2.104:0/2264004781 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe308005d90 con 0x7fe310143540 2021-07-11T04:02:32.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe314ff7700 1 -- 172.21.2.104:0/2264004781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe310142720 con 0x7fe310143540 2021-07-11T04:02:32.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.536+0000 7fe3077fe700 1 -- 172.21.2.104:0/2264004781 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe308006220 con 0x7fe310143540 2021-07-11T04:02:32.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.540+0000 7fe314ff7700 1 -- 172.21.2.104:0/2264004781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3100acc20 con 0x7fe310143540 2021-07-11T04:02:32.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.540+0000 7fe314ff7700 1 -- 172.21.2.104:0/2264004781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3101428b0 con 0x7fe310143540 2021-07-11T04:02:32.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.540+0000 7fe3077fe700 1 -- 172.21.2.104:0/2264004781 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe30800f030 con 0x7fe310143540 2021-07-11T04:02:32.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.540+0000 7fe3077fe700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2f4053330 0x7fe2f40557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:32.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.540+0000 7fe3077fe700 1 -- 172.21.2.104:0/2264004781 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe30806b460 con 0x7fe310143540 2021-07-11T04:02:32.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.540+0000 7fe30e59c700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2f4053330 0x7fe2f40557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:32.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.540+0000 7fe30e59c700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2f4053330 0x7fe2f40557f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fe3100ac320 tx=0x7fe30001a000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:32.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.540+0000 7fe3077fe700 1 -- 172.21.2.104:0/2264004781 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe308042970 con 0x7fe310143540 2021-07-11T04:02:32.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:32.632+0000 7fe314ff7700 1 -- 172.21.2.104:0/2264004781 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd blacklist clear"} v 0) v1 -- 0x7fe310057350 con 0x7fe310143540 2021-07-11T04:02:33.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.532+0000 7fe3077fe700 1 -- 172.21.2.104:0/2264004781 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd blacklist clear"}]=0 removed all blacklist entries v155) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe308045f90 con 0x7fe310143540 2021-07-11T04:02:33.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 -- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2f4053330 msgr2=0x7fe2f40557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2f4053330 0x7fe2f40557f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fe3100ac320 tx=0x7fe30001a000).stop 2021-07-11T04:02:33.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 -- 172.21.2.104:0/2264004781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe310143540 msgr2=0x7fe3100ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe310143540 0x7fe3100ac590 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 rx=0x7fe3080032f0 tx=0x7fe3080091a0).stop 2021-07-11T04:02:33.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 -- 172.21.2.104:0/2264004781 shutdown_connections 2021-07-11T04:02:33.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe2f4053330 0x7fe2f40557f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe310143540 0x7fe3100ac590 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe3100a8b60 0x7fe310141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:33.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 --2- 172.21.2.104:0/2264004781 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe31005f6b0 0x7fe31013b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:33.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 -- 172.21.2.104:0/2264004781 >> 172.21.2.104:0/2264004781 conn(0x7fe3100b53e0 msgr2=0x7fe3100b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:33.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 -- 172.21.2.104:0/2264004781 shutdown_connections 2021-07-11T04:02:33.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.540+0000 7fe314ff7700 1 -- 172.21.2.104:0/2264004781 wait complete. 2021-07-11T04:02:33.544 INFO:teuthology.orchestra.run.gibba004.stderr: removed all blacklist entries 2021-07-11T04:02:33.557 INFO:tasks.cephfs.cephfs_test_case:['0', '1'] 2021-07-11T04:02:33.558 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2021-07-11T04:02:33.684 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.684+0000 7fcb4df49700 1 Processor -- start 2021-07-11T04:02:33.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.684+0000 7fcb4df49700 1 -- start start 2021-07-11T04:02:33.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.684+0000 7fcb4df49700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480b0fd0 0x7fcb48144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.684+0000 7fcb4df49700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb480b0350 con 0x7fcb481483a0 2021-07-11T04:02:33.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.684+0000 7fcb4df49700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb480b05f0 con 0x7fcb480b1db0 2021-07-11T04:02:33.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.684+0000 7fcb4df49700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb480a90c0 con 0x7fcb480b0fd0 2021-07-11T04:02:33.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.684+0000 7fcb46ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fcb481483a0 0x7fcb480a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48404/0 (socket says 172.21.2.104:48404) 2021-07-11T04:02:33.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.684+0000 7fcb46ffd700 1 -- 172.21.2.104:0/1798440523 learned_addr learned my addr 172.21.2.104:0/1798440523 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:33.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.684+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 556231040 0 0) 0x7fcb480b0350 con 0x7fcb481483a0 2021-07-11T04:02:33.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.684+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb28003260 con 0x7fcb481483a0 2021-07-11T04:02:33.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.684+0000 7fcb47fff700 1 --2- 172.21.2.104:0/1798440523 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480b0fd0 0x7fcb48144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:33.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2930837671 0 0) 0x7fcb28003260 con 0x7fcb481483a0 2021-07-11T04:02:33.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480b0fd0 msgr2=0x7fcb48144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:33.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 --2- 172.21.2.104:0/1798440523 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480b0fd0 0x7fcb48144720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.698 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 >> v1:172.21.2.105:6789/0 conn(0x7fcb480b1db0 legacy=0x7fcb480b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:33.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480b0890 con 0x7fcb481483a0 2021-07-11T04:02:33.699 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fcb30001660 con 0x7fcb481483a0 2021-07-11T04:02:33.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 >> v1:172.21.2.104:6789/0 conn(0x7fcb481483a0 legacy=0x7fcb480a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 --2- 172.21.2.104:0/1798440523 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb28003800 0x7fcb28003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 --2- 172.21.2.104:0/1798440523 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb28004500 0x7fcb28004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 --2- 172.21.2.104:0/1798440523 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480b0fd0 0x7fcb28004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2800b8e0 con 0x7fcb28004500 2021-07-11T04:02:33.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2800ba50 con 0x7fcb28003800 2021-07-11T04:02:33.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2800bc10 con 0x7fcb480b0fd0 2021-07-11T04:02:33.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb46ffd700 1 --2- 172.21.2.104:0/1798440523 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb28004500 0x7fcb28004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:33.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb477fe700 1 --2- 172.21.2.104:0/1798440523 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb28003800 0x7fcb28003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:33.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb46ffd700 1 -- 172.21.2.104:0/1798440523 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480b0fd0 msgr2=0x7fcb28004ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:33.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb46ffd700 1 --2- 172.21.2.104:0/1798440523 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480b0fd0 0x7fcb28004ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:33.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb46ffd700 1 -- 172.21.2.104:0/1798440523 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb28003800 msgr2=0x7fcb28003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:33.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb46ffd700 1 --2- 172.21.2.104:0/1798440523 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb28003800 0x7fcb28003c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb46ffd700 1 -- 172.21.2.104:0/1798440523 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb28003260 con 0x7fcb28004500 2021-07-11T04:02:33.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb46ffd700 1 --2- 172.21.2.104:0/1798440523 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb28004500 0x7fcb28004980 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7fcb480613a0 tx=0x7fcb480b04e0).ready entity=mon.0 client_cookie=5b71fc994336f545 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:33.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb300070d0 con 0x7fcb28004500 2021-07-11T04:02:33.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb30007230 con 0x7fcb28004500 2021-07-11T04:02:33.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb467fc700 1 -- 172.21.2.104:0/1798440523 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb3000c8c0 con 0x7fcb28004500 2021-07-11T04:02:33.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 -- 172.21.2.104:0/1798440523 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb28004500 msgr2=0x7fcb28004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 --2- 172.21.2.104:0/1798440523 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb28004500 0x7fcb28004980 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7fcb480613a0 tx=0x7fcb480b04e0).stop 2021-07-11T04:02:33.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb47fff700 1 -- 172.21.2.104:0/1798440523 reap_dead start 2021-07-11T04:02:33.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 -- 172.21.2.104:0/1798440523 shutdown_connections 2021-07-11T04:02:33.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 -- 172.21.2.104:0/1798440523 >> 172.21.2.104:0/1798440523 conn(0x7fcb4805f6b0 msgr2=0x7fcb4805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:33.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 -- 172.21.2.104:0/1798440523 shutdown_connections 2021-07-11T04:02:33.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 -- 172.21.2.104:0/1798440523 wait complete. 2021-07-11T04:02:33.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 Processor -- start 2021-07-11T04:02:33.712 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 -- start start 2021-07-11T04:02:33.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480ae170 0x7fcb48159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb480ab950 0x7fcb480640b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.713 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb480645a0 0x7fcb48066d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb480ae550 con 0x7fcb480645a0 2021-07-11T04:02:33.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb480aa880 con 0x7fcb480ab950 2021-07-11T04:02:33.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb4df49700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb480aab20 con 0x7fcb480ae170 2021-07-11T04:02:33.715 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb47fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb480645a0 0x7fcb48066d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:33.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb47fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb480645a0 0x7fcb48066d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51650/0 (socket says 172.21.2.104:51650) 2021-07-11T04:02:33.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb47fff700 1 -- 172.21.2.104:0/1053351614 learned_addr learned my addr 172.21.2.104:0/1053351614 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:33.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb46ffd700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb480ab950 0x7fcb480640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:33.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb477fe700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480ae170 0x7fcb48159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:33.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb47fff700 1 -- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480ae170 msgr2=0x7fcb48159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb47fff700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480ae170 0x7fcb48159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb47fff700 1 -- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb480ab950 msgr2=0x7fcb480640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb47fff700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb480ab950 0x7fcb480640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.688+0000 7fcb47fff700 1 -- 172.21.2.104:0/1053351614 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480aadc0 con 0x7fcb480645a0 2021-07-11T04:02:33.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb47fff700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb480645a0 0x7fcb48066d20 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 rx=0x7fcb40000b60 tx=0x7fcb40008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:33.720 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb44ff9700 1 -- 172.21.2.104:0/1053351614 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb4000c070 con 0x7fcb480645a0 2021-07-11T04:02:33.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb44ff9700 1 -- 172.21.2.104:0/1053351614 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb400135e0 con 0x7fcb480645a0 2021-07-11T04:02:33.721 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb4df49700 1 -- 172.21.2.104:0/1053351614 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb480aa500 con 0x7fcb480645a0 2021-07-11T04:02:33.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb44ff9700 1 -- 172.21.2.104:0/1053351614 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb40013940 con 0x7fcb480645a0 2021-07-11T04:02:33.722 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb4df49700 1 -- 172.21.2.104:0/1053351614 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb48062960 con 0x7fcb480645a0 2021-07-11T04:02:33.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb4df49700 1 -- 172.21.2.104:0/1053351614 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb48057350 con 0x7fcb480645a0 2021-07-11T04:02:33.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb44ff9700 1 -- 172.21.2.104:0/1053351614 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcb4001b030 con 0x7fcb480645a0 2021-07-11T04:02:33.723 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb44ff9700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb2c053330 0x7fcb2c0557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb44ff9700 1 -- 172.21.2.104:0/1053351614 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcb4006a4a0 con 0x7fcb480645a0 2021-07-11T04:02:33.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb477fe700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb2c053330 0x7fcb2c0557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:33.724 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.692+0000 7fcb477fe700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb2c053330 0x7fcb2c0557f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fcb3800c000 tx=0x7fcb38009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:33.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.696+0000 7fcb44ff9700 1 -- 172.21.2.104:0/1053351614 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcb40041ab0 con 0x7fcb480645a0 2021-07-11T04:02:33.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 -- 172.21.2.104:0/1053351614 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcb48044650 con 0x7fcb480645a0 2021-07-11T04:02:33.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb44ff9700 1 -- 172.21.2.104:0/1053351614 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) v1 ==== 80+0+4255 (secure 0 0 0) 0x7fcb40068e60 con 0x7fcb480645a0 2021-07-11T04:02:33.805 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:33.805 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:02:33.805 INFO:teuthology.orchestra.run.gibba004.stdout: "auth_dump": [ 2021-07-11T04:02:33.806 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.806 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.a", 2021-07-11T04:02:33.806 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgPyAaNhAASO+FqBqBG3+wmwWwJRcs2Q==", 2021-07-11T04:02:33.806 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.807 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:02:33.807 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.807 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.807 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.808 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.808 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.808 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.808 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.b", 2021-07-11T04:02:33.808 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgb7Q2ARAAjVq3Ouf6dosQdJxfjWs89w==", 2021-07-11T04:02:33.809 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.809 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:02:33.809 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.809 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.810 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.810 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.810 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.810 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.810 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mds.c", 2021-07-11T04:02:33.811 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg4AQtOBAAgIU7DOOIQaCbyktjz40mLA==", 2021-07-11T04:02:33.811 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.811 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:02:33.811 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.812 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.813 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.813 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.813 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.814 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.814 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.0", 2021-07-11T04:02:33.814 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgJoeKLRAAojRKC4CMmee4I156fu0i9A==", 2021-07-11T04:02:33.814 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.815 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.815 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.815 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.816 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.816 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.816 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.816 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.1", 2021-07-11T04:02:33.817 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAOa+pgWI9ZDxAApRdFYsvEOzPbYiKQwNp+Zw==", 2021-07-11T04:02:33.817 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.817 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.817 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.818 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.818 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.818 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.818 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.818 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.2", 2021-07-11T04:02:33.819 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAQa+pgAVCYLxAAzO0RCYlkGJILQWMZQ9gyTQ==", 2021-07-11T04:02:33.819 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.819 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.819 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.819 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.820 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.820 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.820 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.820 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.3", 2021-07-11T04:02:33.821 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQATa+pgQNF5EhAA16me63Tfkm8KQvraq6iDpw==", 2021-07-11T04:02:33.821 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.821 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.821 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.821 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.822 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.822 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.824 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.824 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.4", 2021-07-11T04:02:33.824 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAWa+pgkHUyIBAA3fwK3ysgXAV1iedoS4dMwQ==", 2021-07-11T04:02:33.824 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.825 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.825 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.827 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.827 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.827 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.827 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.828 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.5", 2021-07-11T04:02:33.828 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAZa+pg+HPrABAA4bGIdosYwMP18qUFpCmgZg==", 2021-07-11T04:02:33.828 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.828 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.828 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.829 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.829 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.829 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.829 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.830 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.6", 2021-07-11T04:02:33.830 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAba+pgaJfpIRAAbuVTV2F0qER7s6fZunIcsQ==", 2021-07-11T04:02:33.830 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.830 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.830 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.831 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.831 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.832 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.832 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.832 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "osd.7", 2021-07-11T04:02:33.832 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAea+pgSVuDBBAAAp0D0ONJGSn8M4zR/hKwkA==", 2021-07-11T04:02:33.833 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.833 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.833 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.834 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.834 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.835 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.835 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.835 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.0", 2021-07-11T04:02:33.835 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgF4KvBBAAUnFsmJKholIXTnda0iNitw==", 2021-07-11T04:02:33.835 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.836 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:02:33.836 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:02:33.836 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:02:33.837 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.837 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.837 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.838 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.1", 2021-07-11T04:02:33.838 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQALa+pgjJYiBxAAV22WFjo/j3UEg3IaUdv1Yg==", 2021-07-11T04:02:33.838 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.838 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow", 2021-07-11T04:02:33.838 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow r", 2021-07-11T04:02:33.839 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow rw pool=cephfs_data" 2021-07-11T04:02:33.839 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.839 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.839 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.840 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "client.admin", 2021-07-11T04:02:33.840 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pg6m28GhAAYKFEAmJfRQP54+DVhZETfg==", 2021-07-11T04:02:33.840 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.840 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:02:33.840 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr": "allow *", 2021-07-11T04:02:33.841 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow *", 2021-07-11T04:02:33.841 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.841 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.841 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.842 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.842 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.x", 2021-07-11T04:02:33.842 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgzOnyMRAAwN7Fp3srdvbZQDl2ObhmcQ==", 2021-07-11T04:02:33.842 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.842 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:02:33.843 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:02:33.843 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.843 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.844 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:33.844 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:33.845 INFO:teuthology.orchestra.run.gibba004.stdout: "entity": "mgr.y", 2021-07-11T04:02:33.845 INFO:teuthology.orchestra.run.gibba004.stdout: "key": "AQAKa+pgwe7VLhAA2N+ikNQ2+lLSoCpHOP5DnQ==", 2021-07-11T04:02:33.845 INFO:teuthology.orchestra.run.gibba004.stdout: "caps": { 2021-07-11T04:02:33.845 INFO:teuthology.orchestra.run.gibba004.stdout: "mds": "allow *", 2021-07-11T04:02:33.845 INFO:teuthology.orchestra.run.gibba004.stdout: "mon": "allow profile mgr", 2021-07-11T04:02:33.846 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": "allow *" 2021-07-11T04:02:33.846 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.846 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:33.846 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:33.847 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:02:33.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 -- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb2c053330 msgr2=0x7fcb2c0557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb2c053330 0x7fcb2c0557f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7fcb3800c000 tx=0x7fcb38009040).stop 2021-07-11T04:02:33.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 -- 172.21.2.104:0/1053351614 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb480645a0 msgr2=0x7fcb48066d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb480645a0 0x7fcb48066d20 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 rx=0x7fcb40000b60 tx=0x7fcb40008040).stop 2021-07-11T04:02:33.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 -- 172.21.2.104:0/1053351614 shutdown_connections 2021-07-11T04:02:33.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb2c053330 0x7fcb2c0557f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb480645a0 0x7fcb48066d20 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb480ab950 0x7fcb480640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 --2- 172.21.2.104:0/1053351614 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcb480ae170 0x7fcb48159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 -- 172.21.2.104:0/1053351614 >> 172.21.2.104:0/1053351614 conn(0x7fcb4805f6b0 msgr2=0x7fcb480af330 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:33.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 -- 172.21.2.104:0/1053351614 shutdown_connections 2021-07-11T04:02:33.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.804+0000 7fcb4df49700 1 -- 172.21.2.104:0/1053351614 wait complete. 2021-07-11T04:02:33.852 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:33.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 Processor -- start 2021-07-11T04:02:33.907 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 -- start start 2021-07-11T04:02:33.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6940143540 0x7f69400a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f69400a8b60 0x7f6940143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69400b14c0 con 0x7f694005f6b0 2021-07-11T04:02:33.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69400ab3e0 con 0x7f69400a8b60 2021-07-11T04:02:33.916 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69400b1b90 con 0x7f6940143540 2021-07-11T04:02:33.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f693ffff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f694005f6b0 0x7f69400b0db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48424/0 (socket says 172.21.2.104:48424) 2021-07-11T04:02:33.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f693ffff700 1 -- 172.21.2.104:0/1568861649 learned_addr learned my addr 172.21.2.104:0/1568861649 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:33.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6945631700 1 --2- 172.21.2.104:0/1568861649 >> v2:172.21.2.104:3300/0 conn(0x7f69400a8b60 0x7f6940143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:33.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6945631700 1 -- 172.21.2.104:0/1568861649 >> v2:172.21.2.105:3300/0 conn(0x7f6940143540 msgr2=0x7f69400a9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:33.917 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6945631700 1 --2- 172.21.2.104:0/1568861649 >> v2:172.21.2.105:3300/0 conn(0x7f6940143540 0x7f69400a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:33.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6945631700 1 -- 172.21.2.104:0/1568861649 >> v1:172.21.2.104:6789/0 conn(0x7f694005f6b0 legacy=0x7f69400b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6945631700 1 -- 172.21.2.104:0/1568861649 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6940143fa0 con 0x7f69400a8b60 2021-07-11T04:02:33.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6945631700 1 --2- 172.21.2.104:0/1568861649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69400a8b60 0x7f6940143920 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f69380032f0 tx=0x7f69380091d0).ready entity=mon.0 client_cookie=fc7be4b661dd5560 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:33.918 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f693f7fe700 1 -- 172.21.2.104:0/1568861649 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6938009ea0 con 0x7f69400a8b60 2021-07-11T04:02:33.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f693f7fe700 1 -- 172.21.2.104:0/1568861649 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693800b870 con 0x7f69400a8b60 2021-07-11T04:02:33.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f693f7fe700 1 -- 172.21.2.104:0/1568861649 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f693800b9e0 con 0x7f69400a8b60 2021-07-11T04:02:33.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 -- 172.21.2.104:0/1568861649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69400a8b60 msgr2=0x7f6940143920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 --2- 172.21.2.104:0/1568861649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69400a8b60 0x7f6940143920 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f69380032f0 tx=0x7f69380091d0).stop 2021-07-11T04:02:33.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 -- 172.21.2.104:0/1568861649 shutdown_connections 2021-07-11T04:02:33.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 --2- 172.21.2.104:0/1568861649 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f69400a8b60 0x7f6940143920 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 --2- 172.21.2.104:0/1568861649 >> v2:172.21.2.105:3300/0 conn(0x7f6940143540 0x7f69400a9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:33.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 -- 172.21.2.104:0/1568861649 >> 172.21.2.104:0/1568861649 conn(0x7f69400b53e0 msgr2=0x7f69400b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:33.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 -- 172.21.2.104:0/1568861649 shutdown_connections 2021-07-11T04:02:33.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.908+0000 7f6947094700 1 -- 172.21.2.104:0/1568861649 wait complete. 2021-07-11T04:02:33.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6947094700 1 Processor -- start 2021-07-11T04:02:33.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6947094700 1 -- start start 2021-07-11T04:02:33.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6947094700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f694005f6b0 0x7f6940065d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6947094700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69400a8b60 0x7f69400b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6947094700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6940143540 0x7f69401462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6947094700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6940064610 con 0x7f6940143540 2021-07-11T04:02:33.926 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6947094700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6940064840 con 0x7f694005f6b0 2021-07-11T04:02:33.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6947094700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6940064ae0 con 0x7f69400a8b60 2021-07-11T04:02:33.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6945631700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6940143540 0x7f69401462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:33.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6945631700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6940143540 0x7f69401462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51664/0 (socket says 172.21.2.104:51664) 2021-07-11T04:02:33.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6945631700 1 -- 172.21.2.104:0/3088073532 learned_addr learned my addr 172.21.2.104:0/3088073532 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:33.927 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6944e30700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f694005f6b0 0x7f6940065d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:33.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6945631700 1 -- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69400a8b60 msgr2=0x7f69400b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6945631700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69400a8b60 0x7f69400b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:33.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6945631700 1 -- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f694005f6b0 msgr2=0x7f6940065d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6945631700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f694005f6b0 0x7f6940065d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:33.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6945631700 1 -- 172.21.2.104:0/3088073532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6938009470 con 0x7f6940143540 2021-07-11T04:02:33.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6944e30700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f694005f6b0 0x7f6940065d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:33.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6945631700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6940143540 0x7f69401462a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7f69380032f0 tx=0x7f6938016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:33.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f693dffb700 1 -- 172.21.2.104:0/3088073532 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6938021480 con 0x7f6940143540 2021-07-11T04:02:33.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f693dffb700 1 -- 172.21.2.104:0/3088073532 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69380215e0 con 0x7f6940143540 2021-07-11T04:02:33.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f693dffb700 1 -- 172.21.2.104:0/3088073532 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6938021940 con 0x7f6940143540 2021-07-11T04:02:33.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6947094700 1 -- 172.21.2.104:0/3088073532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69400656c0 con 0x7f6940143540 2021-07-11T04:02:33.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6947094700 1 -- 172.21.2.104:0/3088073532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69400652d0 con 0x7f6940143540 2021-07-11T04:02:33.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6947094700 1 -- 172.21.2.104:0/3088073532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6940064ca0 con 0x7f6940143540 2021-07-11T04:02:33.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f693dffb700 1 -- 172.21.2.104:0/3088073532 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f693800f070 con 0x7f6940143540 2021-07-11T04:02:33.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f693dffb700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69240531a0 0x7f6924055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:33.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f693dffb700 1 -- 172.21.2.104:0/3088073532 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f693806c680 con 0x7f6940143540 2021-07-11T04:02:33.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.912+0000 7f6944e30700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69240531a0 0x7f6924055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:33.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.916+0000 7f6944e30700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69240531a0 0x7f6924055660 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f6930007000 tx=0x7f6930008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:33.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.916+0000 7f693dffb700 1 -- 172.21.2.104:0/3088073532 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6938043da0 con 0x7f6940143540 2021-07-11T04:02:33.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.996+0000 7f6947094700 1 -- 172.21.2.104:0/3088073532 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6940057350 con 0x7f6940143540 2021-07-11T04:02:33.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:33.996+0000 7f693dffb700 1 -- 172.21.2.104:0/3088073532 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+13191 (secure 0 0 0) 0x7f6938015030 con 0x7f6940143540 2021-07-11T04:02:33.997 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:33.998 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":155,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:33.518015+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":145,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:33.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 -- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69240531a0 msgr2=0x7f6924055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:33.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69240531a0 0x7f6924055660 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f6930007000 tx=0x7f6930008040).stop 2021-07-11T04:02:34.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 -- 172.21.2.104:0/3088073532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6940143540 msgr2=0x7f69401462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6940143540 0x7f69401462a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7f69380032f0 tx=0x7f6938016000).stop 2021-07-11T04:02:34.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 -- 172.21.2.104:0/3088073532 shutdown_connections 2021-07-11T04:02:34.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f69240531a0 0x7f6924055660 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6940143540 0x7f69401462a0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f69400a8b60 0x7f69400b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 --2- 172.21.2.104:0/3088073532 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f694005f6b0 0x7f6940065d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 -- 172.21.2.104:0/3088073532 >> 172.21.2.104:0/3088073532 conn(0x7f69400b53e0 msgr2=0x7f6940141370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:34.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 -- 172.21.2.104:0/3088073532 shutdown_connections 2021-07-11T04:02:34.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.000+0000 7f6947094700 1 -- 172.21.2.104:0/3088073532 wait complete. 2021-07-11T04:02:34.005 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-07-11T04:02:34.005 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:02:34.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed263d3700 1 Processor -- start 2021-07-11T04:02:34.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed263d3700 1 -- start start 2021-07-11T04:02:34.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed263d3700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fed200b05b0 0x7fed200b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed263d3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fed200a9c10 0x7fed200b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed263d3700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed20143920 con 0x7fed200b3710 2021-07-11T04:02:34.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed263d3700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed20143bc0 con 0x7fed200b05b0 2021-07-11T04:02:34.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed263d3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed20143e60 con 0x7fed200a9c10 2021-07-11T04:02:34.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed1ffff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fed200b05b0 0x7fed200b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed24970700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fed200a9c10 0x7fed200b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed24970700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fed200a9c10 0x7fed200b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:54834/0 (socket says 172.21.2.104:54834) 2021-07-11T04:02:34.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed24970700 1 -- 172.21.2.104:0/2995360953 learned_addr learned my addr 172.21.2.104:0/2995360953 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:34.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed1ffff700 1 -- 172.21.2.104:0/2995360953 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fed200a9c10 msgr2=0x7fed200b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:34.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed1ffff700 1 --2- 172.21.2.104:0/2995360953 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fed200a9c10 0x7fed200b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed1ffff700 1 -- 172.21.2.104:0/2995360953 >> v1:172.21.2.105:6789/0 conn(0x7fed200b3710 legacy=0x7fed2013d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed1ffff700 1 -- 172.21.2.104:0/2995360953 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed20144100 con 0x7fed200b05b0 2021-07-11T04:02:34.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.060+0000 7fed1ffff700 1 --2- 172.21.2.104:0/2995360953 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fed200b05b0 0x7fed200b0990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fed08002f70 tx=0x7fed08006e80).ready entity=mon.1 client_cookie=f28744b8534811be server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1effd700 1 -- 172.21.2.104:0/2995360953 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed0800b7d0 con 0x7fed200b05b0 2021-07-11T04:02:34.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1effd700 1 -- 172.21.2.104:0/2995360953 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0800b930 con 0x7fed200b05b0 2021-07-11T04:02:34.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1effd700 1 -- 172.21.2.104:0/2995360953 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed0800bc90 con 0x7fed200b05b0 2021-07-11T04:02:34.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 -- 172.21.2.104:0/2995360953 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fed200b05b0 msgr2=0x7fed200b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 --2- 172.21.2.104:0/2995360953 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fed200b05b0 0x7fed200b0990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fed08002f70 tx=0x7fed08006e80).stop 2021-07-11T04:02:34.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 -- 172.21.2.104:0/2995360953 shutdown_connections 2021-07-11T04:02:34.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 --2- 172.21.2.104:0/2995360953 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fed200a9c10 0x7fed200b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 --2- 172.21.2.104:0/2995360953 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fed200b05b0 0x7fed200b0990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 -- 172.21.2.104:0/2995360953 >> 172.21.2.104:0/2995360953 conn(0x7fed200607c0 msgr2=0x7fed20060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:34.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 -- 172.21.2.104:0/2995360953 shutdown_connections 2021-07-11T04:02:34.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 -- 172.21.2.104:0/2995360953 wait complete. 2021-07-11T04:02:34.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 Processor -- start 2021-07-11T04:02:34.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 -- start start 2021-07-11T04:02:34.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fed200a9c10 0x7fed20140200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fed200b05b0 0x7fed20146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fed200b3710 0x7fed201317f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20146580 con 0x7fed200a9c10 2021-07-11T04:02:34.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20145e40 con 0x7fed200b3710 2021-07-11T04:02:34.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed20145900 con 0x7fed200b05b0 2021-07-11T04:02:34.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fed200a9c10 0x7fed20140200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fed200a9c10 0x7fed20140200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51674/0 (socket says 172.21.2.104:51674) 2021-07-11T04:02:34.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1ffff700 1 -- 172.21.2.104:0/3538853227 learned_addr learned my addr 172.21.2.104:0/3538853227 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:34.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1ffff700 1 -- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fed200b05b0 msgr2=0x7fed20146090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1ffff700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fed200b05b0 0x7fed20146090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1ffff700 1 -- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fed200b3710 msgr2=0x7fed201317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1ffff700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fed200b3710 0x7fed201317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1ffff700 1 -- 172.21.2.104:0/3538853227 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0800d410 con 0x7fed200a9c10 2021-07-11T04:02:34.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1ffff700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fed200a9c10 0x7fed20140200 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7fed201466c0 tx=0x7fed200b04d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1d7fa700 1 -- 172.21.2.104:0/3538853227 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed08019420 con 0x7fed200a9c10 2021-07-11T04:02:34.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1d7fa700 1 -- 172.21.2.104:0/3538853227 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed08000e90 con 0x7fed200a9c10 2021-07-11T04:02:34.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed1d7fa700 1 -- 172.21.2.104:0/3538853227 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fed08019770 con 0x7fed200a9c10 2021-07-11T04:02:34.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 -- 172.21.2.104:0/3538853227 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed20145ba0 con 0x7fed200a9c10 2021-07-11T04:02:34.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.064+0000 7fed263d3700 1 -- 172.21.2.104:0/3538853227 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed200af480 con 0x7fed200a9c10 2021-07-11T04:02:34.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.068+0000 7fed263d3700 1 -- 172.21.2.104:0/3538853227 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed20057350 con 0x7fed200a9c10 2021-07-11T04:02:34.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.068+0000 7fed1d7fa700 1 -- 172.21.2.104:0/3538853227 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fed0802a070 con 0x7fed200a9c10 2021-07-11T04:02:34.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.068+0000 7fed1d7fa700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fed04053350 0x7fed04055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.068+0000 7fed1d7fa700 1 -- 172.21.2.104:0/3538853227 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fed0806cb80 con 0x7fed200a9c10 2021-07-11T04:02:34.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.068+0000 7fed1f7fe700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fed04053350 0x7fed04055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.068+0000 7fed1f7fe700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fed04053350 0x7fed04055810 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fed2006aa50 tx=0x7fed10007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.068+0000 7fed1d7fa700 1 -- 172.21.2.104:0/3538853227 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fed080440e0 con 0x7fed200a9c10 2021-07-11T04:02:34.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.152+0000 7fed263d3700 1 -- 172.21.2.104:0/3538853227 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fed20044650 con 0x7fed200a9c10 2021-07-11T04:02:34.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.152+0000 7fed1d7fa700 1 -- 172.21.2.104:0/3538853227 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v155) v1 ==== 110+0+10 (secure 0 0 0) 0x7fed08014030 con 0x7fed200a9c10 2021-07-11T04:02:34.153 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:02:34.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.152+0000 7fed263d3700 1 -- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fed04053350 msgr2=0x7fed04055810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.152+0000 7fed263d3700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fed04053350 0x7fed04055810 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7fed2006aa50 tx=0x7fed10007810).stop 2021-07-11T04:02:34.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.152+0000 7fed263d3700 1 -- 172.21.2.104:0/3538853227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fed200a9c10 msgr2=0x7fed20140200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.152+0000 7fed263d3700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fed200a9c10 0x7fed20140200 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7fed201466c0 tx=0x7fed200b04d0).stop 2021-07-11T04:02:34.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.156+0000 7fed263d3700 1 -- 172.21.2.104:0/3538853227 shutdown_connections 2021-07-11T04:02:34.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.156+0000 7fed263d3700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fed200b3710 0x7fed201317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.156+0000 7fed263d3700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fed200b05b0 0x7fed20146090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.156+0000 7fed263d3700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fed04053350 0x7fed04055810 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.156+0000 7fed263d3700 1 --2- 172.21.2.104:0/3538853227 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fed200a9c10 0x7fed20140200 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.156+0000 7fed263d3700 1 -- 172.21.2.104:0/3538853227 >> 172.21.2.104:0/3538853227 conn(0x7fed200607c0 msgr2=0x7fed200b56e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:34.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.156+0000 7fed263d3700 1 -- 172.21.2.104:0/3538853227 shutdown_connections 2021-07-11T04:02:34.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.156+0000 7fed263d3700 1 -- 172.21.2.104:0/3538853227 wait complete. 2021-07-11T04:02:34.161 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:02:34.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ada143700 1 Processor -- start 2021-07-11T04:02:34.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ada143700 1 -- start start 2021-07-11T04:02:34.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ada143700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1ad4065420 0x7f1ad414c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ada143700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad40604e0 con 0x7f1ad4060100 2021-07-11T04:02:34.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ada143700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad40b23b0 con 0x7f1ad4144440 2021-07-11T04:02:34.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ada143700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad4065c80 con 0x7f1ad4065420 2021-07-11T04:02:34.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ad3fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1ad4065420 0x7f1ad414c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ad37fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1ad4060100 0x7f1ad4064d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48450/0 (socket says 172.21.2.104:48450) 2021-07-11T04:02:34.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ad37fe700 1 -- 172.21.2.104:0/3635861574 learned_addr learned my addr 172.21.2.104:0/3635861574 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:34.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ad3fff700 1 -- 172.21.2.104:0/3635861574 >> v1:172.21.2.105:6789/0 conn(0x7f1ad4144440 legacy=0x7f1ad4144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:34.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ad3fff700 1 -- 172.21.2.104:0/3635861574 >> v1:172.21.2.104:6789/0 conn(0x7f1ad4060100 legacy=0x7f1ad4064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ad27fc700 1 -- 172.21.2.104:0/3635861574 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024823831 0 0) 0x7f1ad40604e0 con 0x7f1ad4060100 2021-07-11T04:02:34.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.212+0000 7f1ad3fff700 1 -- 172.21.2.104:0/3635861574 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad40b2650 con 0x7f1ad4065420 2021-07-11T04:02:34.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad3fff700 1 --2- 172.21.2.104:0/3635861574 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad4065420 0x7f1ad414c7f0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7f1ad405f160 tx=0x7f1acc0091d0).ready entity=mon.0 client_cookie=f3308b1423373b5e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad27fc700 1 -- 172.21.2.104:0/3635861574 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1acc009ea0 con 0x7f1ad4065420 2021-07-11T04:02:34.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad27fc700 1 -- 172.21.2.104:0/3635861574 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc00b860 con 0x7f1ad4065420 2021-07-11T04:02:34.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad27fc700 1 -- 172.21.2.104:0/3635861574 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1acc00b9d0 con 0x7f1ad4065420 2021-07-11T04:02:34.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- 172.21.2.104:0/3635861574 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad4065420 msgr2=0x7f1ad414c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 --2- 172.21.2.104:0/3635861574 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad4065420 0x7f1ad414c7f0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7f1ad405f160 tx=0x7f1acc0091d0).stop 2021-07-11T04:02:34.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- 172.21.2.104:0/3635861574 shutdown_connections 2021-07-11T04:02:34.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 --2- 172.21.2.104:0/3635861574 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad4065420 0x7f1ad414c7f0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- 172.21.2.104:0/3635861574 >> 172.21.2.104:0/3635861574 conn(0x7f1ad405cf60 msgr2=0x7f1ad405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:34.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- 172.21.2.104:0/3635861574 shutdown_connections 2021-07-11T04:02:34.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- 172.21.2.104:0/3635861574 wait complete. 2021-07-11T04:02:34.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 Processor -- start 2021-07-11T04:02:34.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- start start 2021-07-11T04:02:34.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad4060100 0x7f1ad40b6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ad4065420 0x7f1ad40b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40b0880 0x7f1ad413a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40ae7c0 con 0x7f1ad40b0880 2021-07-11T04:02:34.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40aea60 con 0x7f1ad4065420 2021-07-11T04:02:34.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40adda0 con 0x7f1ad4060100 2021-07-11T04:02:34.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40b0880 0x7f1ad413a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad3fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40b0880 0x7f1ad413a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51690/0 (socket says 172.21.2.104:51690) 2021-07-11T04:02:34.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad3fff700 1 -- 172.21.2.104:0/3291237013 learned_addr learned my addr 172.21.2.104:0/3291237013 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:34.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad3fff700 1 -- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad4060100 msgr2=0x7f1ad40b6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad3fff700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad4060100 0x7f1ad40b6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad3fff700 1 -- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ad4065420 msgr2=0x7f1ad40b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad3fff700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ad4065420 0x7f1ad40b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad3fff700 1 -- 172.21.2.104:0/3291237013 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc009470 con 0x7f1ad40b0880 2021-07-11T04:02:34.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad3fff700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40b0880 0x7f1ad413a7d0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7f1ad405f160 tx=0x7f1acc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad0ff9700 1 -- 172.21.2.104:0/3291237013 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1acc011e50 con 0x7f1ad40b0880 2021-07-11T04:02:34.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad0ff9700 1 -- 172.21.2.104:0/3291237013 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc021440 con 0x7f1ad40b0880 2021-07-11T04:02:34.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad0ff9700 1 -- 172.21.2.104:0/3291237013 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1acc0217a0 con 0x7f1ad40b0880 2021-07-11T04:02:34.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- 172.21.2.104:0/3291237013 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad40ae040 con 0x7f1ad40b0880 2021-07-11T04:02:34.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- 172.21.2.104:0/3291237013 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad40af560 con 0x7f1ad40b0880 2021-07-11T04:02:34.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ada143700 1 -- 172.21.2.104:0/3291237013 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad40ae1d0 con 0x7f1ad40b0880 2021-07-11T04:02:34.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.216+0000 7f1ad0ff9700 1 -- 172.21.2.104:0/3291237013 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1acc027020 con 0x7f1ad40b0880 2021-07-11T04:02:34.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.220+0000 7f1ad0ff9700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab8053300 0x7f1ab80557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.220+0000 7f1ad0ff9700 1 -- 172.21.2.104:0/3291237013 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1acc06c6a0 con 0x7f1ad40b0880 2021-07-11T04:02:34.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.220+0000 7f1ad37fe700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab8053300 0x7f1ab80557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.220+0000 7f1ad37fe700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab8053300 0x7f1ab80557c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f1abc000bc0 tx=0x7f1abc0076c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.220+0000 7f1ad0ff9700 1 -- 172.21.2.104:0/3291237013 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1acc043c00 con 0x7f1ad40b0880 2021-07-11T04:02:34.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.300+0000 7f1ada143700 1 -- 172.21.2.104:0/3291237013 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1ad4057350 con 0x7f1ad40b0880 2021-07-11T04:02:34.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.300+0000 7f1ad0ff9700 1 -- 172.21.2.104:0/3291237013 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v155) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1acc0150a0 con 0x7f1ad40b0880 2021-07-11T04:02:34.302 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:34.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 -- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab8053300 msgr2=0x7f1ab80557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab8053300 0x7f1ab80557c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f1abc000bc0 tx=0x7f1abc0076c0).stop 2021-07-11T04:02:34.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 -- 172.21.2.104:0/3291237013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40b0880 msgr2=0x7f1ad413a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40b0880 0x7f1ad413a7d0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7f1ad405f160 tx=0x7f1acc016000).stop 2021-07-11T04:02:34.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 -- 172.21.2.104:0/3291237013 shutdown_connections 2021-07-11T04:02:34.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab8053300 0x7f1ab80557c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40b0880 0x7f1ad413a7d0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ad4065420 0x7f1ad40b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 --2- 172.21.2.104:0/3291237013 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad4060100 0x7f1ad40b6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 -- 172.21.2.104:0/3291237013 >> 172.21.2.104:0/3291237013 conn(0x7f1ad405cf60 msgr2=0x7f1ad413f160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:34.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 -- 172.21.2.104:0/3291237013 shutdown_connections 2021-07-11T04:02:34.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.304+0000 7f1ada143700 1 -- 172.21.2.104:0/3291237013 wait complete. 2021-07-11T04:02:34.309 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2021-07-11T04:02:34.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd135fda700 1 Processor -- start 2021-07-11T04:02:34.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd135fda700 1 -- start start 2021-07-11T04:02:34.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd135fda700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd1300b1db0 0x7fd1300b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd135fda700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1300a70c0 0x7fd130144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd135fda700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1300b0350 con 0x7fd1301483a0 2021-07-11T04:02:34.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd135fda700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1300a90c0 con 0x7fd1300b1db0 2021-07-11T04:02:34.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd135fda700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1300b05f0 con 0x7fd1300a70c0 2021-07-11T04:02:34.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd12f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd1300b1db0 0x7fd1300b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd12f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fd1300b1db0 0x7fd1300b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51698/0 (socket says 172.21.2.104:51698) 2021-07-11T04:02:34.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd12f7fe700 1 -- 172.21.2.104:0/2950220975 learned_addr learned my addr 172.21.2.104:0/2950220975 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:34.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd12f7fe700 1 -- 172.21.2.104:0/2950220975 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1300a70c0 msgr2=0x7fd130144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:34.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd12f7fe700 1 --2- 172.21.2.104:0/2950220975 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1300a70c0 0x7fd130144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd12f7fe700 1 -- 172.21.2.104:0/2950220975 >> v1:172.21.2.105:6789/0 conn(0x7fd1301483a0 legacy=0x7fd1300b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:34.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.360+0000 7fd12f7fe700 1 -- 172.21.2.104:0/2950220975 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1300b0890 con 0x7fd1300b1db0 2021-07-11T04:02:34.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12f7fe700 1 --2- 172.21.2.104:0/2950220975 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1300b1db0 0x7fd1300b0fd0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7fd130147730 tx=0x7fd120006eb0).ready entity=mon.0 client_cookie=ff255ca1df37c66 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12e7fc700 1 -- 172.21.2.104:0/2950220975 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd12000d570 con 0x7fd1300b1db0 2021-07-11T04:02:34.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12e7fc700 1 -- 172.21.2.104:0/2950220975 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd120000e90 con 0x7fd1300b1db0 2021-07-11T04:02:34.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12e7fc700 1 -- 172.21.2.104:0/2950220975 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd12000d900 con 0x7fd1300b1db0 2021-07-11T04:02:34.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- 172.21.2.104:0/2950220975 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1300b1db0 msgr2=0x7fd1300b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 --2- 172.21.2.104:0/2950220975 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1300b1db0 0x7fd1300b0fd0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 rx=0x7fd130147730 tx=0x7fd120006eb0).stop 2021-07-11T04:02:34.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- 172.21.2.104:0/2950220975 shutdown_connections 2021-07-11T04:02:34.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 --2- 172.21.2.104:0/2950220975 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1300a70c0 0x7fd130144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 --2- 172.21.2.104:0/2950220975 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1300b1db0 0x7fd1300b0fd0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- 172.21.2.104:0/2950220975 >> 172.21.2.104:0/2950220975 conn(0x7fd13005f6b0 msgr2=0x7fd13005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:34.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- 172.21.2.104:0/2950220975 shutdown_connections 2021-07-11T04:02:34.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- 172.21.2.104:0/2950220975 wait complete. 2021-07-11T04:02:34.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 Processor -- start 2021-07-11T04:02:34.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- start start 2021-07-11T04:02:34.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd1300a70c0 0x7fd130064630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1300b1db0 0x7fd1301439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1301483a0 0x7fd130143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd130144380 con 0x7fd1300b1db0 2021-07-11T04:02:34.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd130144920 con 0x7fd1300a70c0 2021-07-11T04:02:34.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd130144bc0 con 0x7fd1301483a0 2021-07-11T04:02:34.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1300b1db0 0x7fd1301439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12effd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1300b1db0 0x7fd1301439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51706/0 (socket says 172.21.2.104:51706) 2021-07-11T04:02:34.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12effd700 1 -- 172.21.2.104:0/2302495067 learned_addr learned my addr 172.21.2.104:0/2302495067 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:34.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12effd700 1 -- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1301483a0 msgr2=0x7fd130143e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12effd700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1301483a0 0x7fd130143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12effd700 1 -- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd1300a70c0 msgr2=0x7fd130064630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12effd700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd1300a70c0 0x7fd130064630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12effd700 1 -- 172.21.2.104:0/2302495067 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12000b1e0 con 0x7fd1300b1db0 2021-07-11T04:02:34.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12effd700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1300b1db0 0x7fd1301439a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7fd118002f30 tx=0x7fd118006d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12cff9700 1 -- 172.21.2.104:0/2302495067 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd118009e00 con 0x7fd1300b1db0 2021-07-11T04:02:34.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12cff9700 1 -- 172.21.2.104:0/2302495067 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11800f510 con 0x7fd1300b1db0 2021-07-11T04:02:34.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- 172.21.2.104:0/2302495067 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd130144e60 con 0x7fd1300b1db0 2021-07-11T04:02:34.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12cff9700 1 -- 172.21.2.104:0/2302495067 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fd11800f680 con 0x7fd1300b1db0 2021-07-11T04:02:34.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- 172.21.2.104:0/2302495067 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd130067540 con 0x7fd1300b1db0 2021-07-11T04:02:34.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd135fda700 1 -- 172.21.2.104:0/2302495067 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd130057350 con 0x7fd1300b1db0 2021-07-11T04:02:34.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.364+0000 7fd12cff9700 1 -- 172.21.2.104:0/2302495067 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fd11800f070 con 0x7fd1300b1db0 2021-07-11T04:02:34.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.368+0000 7fd12cff9700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd114053300 0x7fd1140557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.368+0000 7fd12cff9700 1 -- 172.21.2.104:0/2302495067 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd11806aa20 con 0x7fd1300b1db0 2021-07-11T04:02:34.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.368+0000 7fd12f7fe700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd114053300 0x7fd1140557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.368+0000 7fd12f7fe700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd114053300 0x7fd1140557c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fd130147730 tx=0x7fd120015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.368+0000 7fd12cff9700 1 -- 172.21.2.104:0/2302495067 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fd118042090 con 0x7fd1300b1db0 2021-07-11T04:02:34.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.452+0000 7fd135fda700 1 -- 172.21.2.104:0/2302495067 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd130044650 con 0x7fd1300b1db0 2021-07-11T04:02:34.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.452+0000 7fd12cff9700 1 -- 172.21.2.104:0/2302495067 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v155) v1 ==== 84+0+143 (secure 0 0 0) 0x7fd1180456b0 con 0x7fd1300b1db0 2021-07-11T04:02:34.454 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:34.454 INFO:teuthology.orchestra.run.gibba004.stdout:[ 2021-07-11T04:02:34.454 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:34.454 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 1, 2021-07-11T04:02:34.455 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "device_health_metrics" 2021-07-11T04:02:34.455 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:34.455 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:34.455 INFO:teuthology.orchestra.run.gibba004.stdout: "poolnum": 2, 2021-07-11T04:02:34.455 INFO:teuthology.orchestra.run.gibba004.stdout: "poolname": "rbd" 2021-07-11T04:02:34.456 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:34.456 INFO:teuthology.orchestra.run.gibba004.stdout:] 2021-07-11T04:02:34.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.452+0000 7fd135fda700 1 -- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd114053300 msgr2=0x7fd1140557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.452+0000 7fd135fda700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd114053300 0x7fd1140557c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fd130147730 tx=0x7fd120015000).stop 2021-07-11T04:02:34.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.452+0000 7fd135fda700 1 -- 172.21.2.104:0/2302495067 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1300b1db0 msgr2=0x7fd1301439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.452+0000 7fd135fda700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1300b1db0 0x7fd1301439a0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7fd118002f30 tx=0x7fd118006d60).stop 2021-07-11T04:02:34.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.456+0000 7fd135fda700 1 -- 172.21.2.104:0/2302495067 shutdown_connections 2021-07-11T04:02:34.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.456+0000 7fd135fda700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fd1301483a0 0x7fd130143e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.456+0000 7fd135fda700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fd114053300 0x7fd1140557c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.456+0000 7fd135fda700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fd1300b1db0 0x7fd1301439a0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.456+0000 7fd135fda700 1 --2- 172.21.2.104:0/2302495067 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fd1300a70c0 0x7fd130064630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.456+0000 7fd135fda700 1 -- 172.21.2.104:0/2302495067 >> 172.21.2.104:0/2302495067 conn(0x7fd13005f6b0 msgr2=0x7fd1300aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:34.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.456+0000 7fd135fda700 1 -- 172.21.2.104:0/2302495067 shutdown_connections 2021-07-11T04:02:34.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.456+0000 7fd135fda700 1 -- 172.21.2.104:0/2302495067 wait complete. 2021-07-11T04:02:34.461 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2021-07-11T04:02:34.461 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2021-07-11T04:02:34.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.516+0000 7fce8280a700 1 Processor -- start 2021-07-11T04:02:34.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.516+0000 7fce8280a700 1 -- start start 2021-07-11T04:02:34.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fce7c0b12d0 0x7fce7c0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce7c0b22a0 0x7fce7c141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c0a77d0 con 0x7fce7c0b1ec0 2021-07-11T04:02:34.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c0b1740 con 0x7fce7c0b12d0 2021-07-11T04:02:34.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0b0650 con 0x7fce7c0b22a0 2021-07-11T04:02:34.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7b7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fce7c0b12d0 0x7fce7c0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7b7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fce7c0b12d0 0x7fce7c0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51710/0 (socket says 172.21.2.104:51710) 2021-07-11T04:02:34.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7b7fe700 1 -- 172.21.2.104:0/1794353752 learned_addr learned my addr 172.21.2.104:0/1794353752 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:34.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7b7fe700 1 -- 172.21.2.104:0/1794353752 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce7c0b22a0 msgr2=0x7fce7c141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:34.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7b7fe700 1 --2- 172.21.2.104:0/1794353752 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce7c0b22a0 0x7fce7c141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7b7fe700 1 -- 172.21.2.104:0/1794353752 >> v1:172.21.2.105:6789/0 conn(0x7fce7c0b1ec0 legacy=0x7fce7c0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:34.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7b7fe700 1 -- 172.21.2.104:0/1794353752 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c0b08f0 con 0x7fce7c0b12d0 2021-07-11T04:02:34.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7b7fe700 1 --2- 172.21.2.104:0/1794353752 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce7c0b12d0 0x7fce7c0affb0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7fce7c0b2f50 tx=0x7fce6c006eb0).ready entity=mon.0 client_cookie=92a3d8413b086965 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7affd700 1 -- 172.21.2.104:0/1794353752 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce6c00d570 con 0x7fce7c0b12d0 2021-07-11T04:02:34.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7affd700 1 -- 172.21.2.104:0/1794353752 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c000e90 con 0x7fce7c0b12d0 2021-07-11T04:02:34.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7affd700 1 -- 172.21.2.104:0/1794353752 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce6c00d8c0 con 0x7fce7c0b12d0 2021-07-11T04:02:34.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- 172.21.2.104:0/1794353752 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce7c0b12d0 msgr2=0x7fce7c0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 --2- 172.21.2.104:0/1794353752 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce7c0b12d0 0x7fce7c0affb0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7fce7c0b2f50 tx=0x7fce6c006eb0).stop 2021-07-11T04:02:34.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- 172.21.2.104:0/1794353752 shutdown_connections 2021-07-11T04:02:34.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 --2- 172.21.2.104:0/1794353752 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce7c0b22a0 0x7fce7c141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:34.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 --2- 172.21.2.104:0/1794353752 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce7c0b12d0 0x7fce7c0affb0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- 172.21.2.104:0/1794353752 >> 172.21.2.104:0/1794353752 conn(0x7fce7c05f6b0 msgr2=0x7fce7c05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:34.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- 172.21.2.104:0/1794353752 shutdown_connections 2021-07-11T04:02:34.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- 172.21.2.104:0/1794353752 wait complete. 2021-07-11T04:02:34.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 Processor -- start 2021-07-11T04:02:34.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- start start 2021-07-11T04:02:34.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce7c0b12d0 0x7fce7c148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce7c0b1ec0 0x7fce7c14b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce7c0b22a0 0x7fce7c0aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c14afe0 con 0x7fce7c0b22a0 2021-07-11T04:02:34.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c14a2a0 con 0x7fce7c0b12d0 2021-07-11T04:02:34.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce8280a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c14a4d0 con 0x7fce7c0b1ec0 2021-07-11T04:02:34.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce80da7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce7c0b22a0 0x7fce7c0aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7bfff700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce7c0b12d0 0x7fce7c148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7b7fe700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce7c0b1ec0 0x7fce7c14b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7bfff700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce7c0b12d0 0x7fce7c148310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:59276/0 (socket says 172.21.2.104:59276) 2021-07-11T04:02:34.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce7bfff700 1 -- 172.21.2.104:0/3388185704 learned_addr learned my addr 172.21.2.104:0/3388185704 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:34.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce80da7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce7c0b22a0 0x7fce7c0aafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51720/0 (socket says 172.21.2.104:51720) 2021-07-11T04:02:34.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce80da7700 1 -- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce7c0b1ec0 msgr2=0x7fce7c14b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce80da7700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce7c0b1ec0 0x7fce7c14b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce80da7700 1 -- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce7c0b12d0 msgr2=0x7fce7c148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:34.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce80da7700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce7c0b12d0 0x7fce7c148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:34.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.520+0000 7fce80da7700 1 -- 172.21.2.104:0/3388185704 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c00b1e0 con 0x7fce7c0b22a0 2021-07-11T04:02:34.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce80da7700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce7c0b22a0 0x7fce7c0aafe0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7fce740032f0 tx=0x7fce740090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce797fa700 1 -- 172.21.2.104:0/3388185704 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7400f510 con 0x7fce7c0b22a0 2021-07-11T04:02:34.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce797fa700 1 -- 172.21.2.104:0/3388185704 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7400f670 con 0x7fce7c0b22a0 2021-07-11T04:02:34.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce8280a700 1 -- 172.21.2.104:0/3388185704 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c14a770 con 0x7fce7c0b22a0 2021-07-11T04:02:34.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce797fa700 1 -- 172.21.2.104:0/3388185704 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fce7400f7e0 con 0x7fce7c0b22a0 2021-07-11T04:02:34.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce8280a700 1 -- 172.21.2.104:0/3388185704 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c149ae0 con 0x7fce7c0b22a0 2021-07-11T04:02:34.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce8280a700 1 -- 172.21.2.104:0/3388185704 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c14a900 con 0x7fce7c0b22a0 2021-07-11T04:02:34.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce797fa700 1 -- 172.21.2.104:0/3388185704 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fce7400f070 con 0x7fce7c0b22a0 2021-07-11T04:02:34.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce797fa700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce600531f0 0x7fce600556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:34.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce797fa700 1 -- 172.21.2.104:0/3388185704 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fce7406ab80 con 0x7fce7c0b22a0 2021-07-11T04:02:34.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce7bfff700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce600531f0 0x7fce600556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:34.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce7bfff700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce600531f0 0x7fce600556b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fce6400d000 tx=0x7fce64009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:34.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.524+0000 7fce797fa700 1 -- 172.21.2.104:0/3388185704 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fce740422a0 con 0x7fce7c0b22a0 2021-07-11T04:02:34.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:34.608+0000 7fce8280a700 1 -- 172.21.2.104:0/3388185704 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fce7c057350 con 0x7fce7c0b22a0 2021-07-11T04:02:35.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.472+0000 7fce797fa700 1 -- 172.21.2.104:0/3388185704 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v156) v1 ==== 133+0+0 (secure 0 0 0) 0x7fce740458c0 con 0x7fce7c0b22a0 2021-07-11T04:02:35.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 -- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce600531f0 msgr2=0x7fce600556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:35.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce600531f0 0x7fce600556b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fce6400d000 tx=0x7fce64009040).stop 2021-07-11T04:02:35.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 -- 172.21.2.104:0/3388185704 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce7c0b22a0 msgr2=0x7fce7c0aafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:35.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce7c0b22a0 0x7fce7c0aafe0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7fce740032f0 tx=0x7fce740090f0).stop 2021-07-11T04:02:35.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 -- 172.21.2.104:0/3388185704 shutdown_connections 2021-07-11T04:02:35.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fce600531f0 0x7fce600556b0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:35.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fce7c0b22a0 0x7fce7c0aafe0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:35.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fce7c0b1ec0 0x7fce7c14b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:35.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 --2- 172.21.2.104:0/3388185704 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fce7c0b12d0 0x7fce7c148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:35.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 -- 172.21.2.104:0/3388185704 >> 172.21.2.104:0/3388185704 conn(0x7fce7c05f6b0 msgr2=0x7fce7c13edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:35.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 -- 172.21.2.104:0/3388185704 shutdown_connections 2021-07-11T04:02:35.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.476+0000 7fce8280a700 1 -- 172.21.2.104:0/3388185704 wait complete. 2021-07-11T04:02:35.478 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' created 2021-07-11T04:02:35.481 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2021-07-11T04:02:35.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 Processor -- start 2021-07-11T04:02:35.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 -- start start 2021-07-11T04:02:35.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbfd40b1ec0 0x7fbfd40b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:35.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbfd40affb0 0x7fbfd40b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:35.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fbfd40a70c0 0x7fbfd4141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:35.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd40b2790 con 0x7fbfd40affb0 2021-07-11T04:02:35.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfd40b03f0 con 0x7fbfd40a70c0 2021-07-11T04:02:35.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40b0650 con 0x7fbfd40b1ec0 2021-07-11T04:02:35.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfd8b01700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbfd40affb0 0x7fbfd40b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:35.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfd8b01700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fbfd40affb0 0x7fbfd40b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51728/0 (socket says 172.21.2.104:51728) 2021-07-11T04:02:35.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfd8b01700 1 -- 172.21.2.104:0/3355241458 learned_addr learned my addr 172.21.2.104:0/3355241458 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:35.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfd8b01700 1 -- 172.21.2.104:0/3355241458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbfd40b1ec0 msgr2=0x7fbfd40b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:35.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfd8b01700 1 --2- 172.21.2.104:0/3355241458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbfd40b1ec0 0x7fbfd40b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:35.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfd8b01700 1 -- 172.21.2.104:0/3355241458 >> v2:172.21.2.105:3300/0 conn(0x7fbfd40a70c0 msgr2=0x7fbfd4141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:35.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfd8b01700 1 --2- 172.21.2.104:0/3355241458 >> v2:172.21.2.105:3300/0 conn(0x7fbfd40a70c0 0x7fbfd4141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:35.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfd8b01700 1 -- 172.21.2.104:0/3355241458 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40b08f0 con 0x7fbfd40affb0 2021-07-11T04:02:35.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfd8b01700 1 --2- 172.21.2.104:0/3355241458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbfd40affb0 0x7fbfd40b22a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7fbfc8002f70 tx=0x7fbfc8006eb0).ready entity=mon.0 client_cookie=16e54bfdf42c0f9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:35.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfc7fff700 1 -- 172.21.2.104:0/3355241458 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc800d570 con 0x7fbfd40affb0 2021-07-11T04:02:35.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfc7fff700 1 -- 172.21.2.104:0/3355241458 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc8000e90 con 0x7fbfd40affb0 2021-07-11T04:02:35.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfc7fff700 1 -- 172.21.2.104:0/3355241458 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc800d900 con 0x7fbfd40affb0 2021-07-11T04:02:35.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 -- 172.21.2.104:0/3355241458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbfd40affb0 msgr2=0x7fbfd40b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:35.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 --2- 172.21.2.104:0/3355241458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbfd40affb0 0x7fbfd40b22a0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7fbfc8002f70 tx=0x7fbfc8006eb0).stop 2021-07-11T04:02:35.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 -- 172.21.2.104:0/3355241458 shutdown_connections 2021-07-11T04:02:35.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 --2- 172.21.2.104:0/3355241458 >> v2:172.21.2.105:3300/0 conn(0x7fbfd40a70c0 0x7fbfd4141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:35.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 --2- 172.21.2.104:0/3355241458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbfd40affb0 0x7fbfd40b22a0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:35.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 --2- 172.21.2.104:0/3355241458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbfd40b1ec0 0x7fbfd40b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:35.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 -- 172.21.2.104:0/3355241458 >> 172.21.2.104:0/3355241458 conn(0x7fbfd405f6b0 msgr2=0x7fbfd405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:35.551 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 -- 172.21.2.104:0/3355241458 shutdown_connections 2021-07-11T04:02:35.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.536+0000 7fbfdb566700 1 -- 172.21.2.104:0/3355241458 wait complete. 2021-07-11T04:02:35.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfdb566700 1 Processor -- start 2021-07-11T04:02:35.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfdb566700 1 -- start start 2021-07-11T04:02:35.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfdb566700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbfd40a70c0 0x7fbfd413ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:35.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfdb566700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbfd40affb0 0x7fbfd40ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:35.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfdb566700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbfd40b1ec0 0x7fbfd40aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:35.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfdb566700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40b4fc0 con 0x7fbfd40affb0 2021-07-11T04:02:35.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfdb566700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40aefb0 con 0x7fbfd40a70c0 2021-07-11T04:02:35.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfdb566700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd40ad660 con 0x7fbfd40b1ec0 2021-07-11T04:02:35.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfd8b01700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbfd40affb0 0x7fbfd40ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:35.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfd8b01700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbfd40affb0 0x7fbfd40ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51734/0 (socket says 172.21.2.104:51734) 2021-07-11T04:02:35.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfd8b01700 1 -- 172.21.2.104:0/1390296129 learned_addr learned my addr 172.21.2.104:0/1390296129 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:35.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfd8b01700 1 -- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbfd40b1ec0 msgr2=0x7fbfd40aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:35.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfd8b01700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbfd40b1ec0 0x7fbfd40aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:35.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfd8b01700 1 -- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbfd40a70c0 msgr2=0x7fbfd413ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:35.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfd8b01700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbfd40a70c0 0x7fbfd413ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:35.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfd8b01700 1 -- 172.21.2.104:0/1390296129 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc800b1e0 con 0x7fbfd40affb0 2021-07-11T04:02:35.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfd8b01700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbfd40affb0 0x7fbfd40ae5d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7fbfc8002f70 tx=0x7fbfc8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:35.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfc67fc700 1 -- 172.21.2.104:0/1390296129 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc8021070 con 0x7fbfd40affb0 2021-07-11T04:02:35.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfc67fc700 1 -- 172.21.2.104:0/1390296129 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc8000e90 con 0x7fbfd40affb0 2021-07-11T04:02:35.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfdb566700 1 -- 172.21.2.104:0/1390296129 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd40ad900 con 0x7fbfd40affb0 2021-07-11T04:02:35.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfc67fc700 1 -- 172.21.2.104:0/1390296129 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fbfc8011ad0 con 0x7fbfd40affb0 2021-07-11T04:02:35.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfdb566700 1 -- 172.21.2.104:0/1390296129 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd41403f0 con 0x7fbfd40affb0 2021-07-11T04:02:35.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfdb566700 1 -- 172.21.2.104:0/1390296129 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd40b4de0 con 0x7fbfd40affb0 2021-07-11T04:02:35.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfc67fc700 1 -- 172.21.2.104:0/1390296129 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fbfc800f070 con 0x7fbfd40affb0 2021-07-11T04:02:35.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfc67fc700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbfb40531f0 0x7fbfb40556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:35.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfc67fc700 1 -- 172.21.2.104:0/1390296129 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5603+0+0 (secure 0 0 0) 0x7fbfc806d380 con 0x7fbfd40affb0 2021-07-11T04:02:35.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.540+0000 7fbfd9302700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbfb40531f0 0x7fbfb40556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:35.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.544+0000 7fbfd9302700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbfb40531f0 0x7fbfb40556b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fbfd0002f70 tx=0x7fbfd0006990).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:35.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.544+0000 7fbfc67fc700 1 -- 172.21.2.104:0/1390296129 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fbfc8044940 con 0x7fbfd40affb0 2021-07-11T04:02:35.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:35.624+0000 7fbfdb566700 1 -- 172.21.2.104:0/1390296129 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fbfd4057350 con 0x7fbfd40affb0 2021-07-11T04:02:36.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.484+0000 7fbfc67fc700 1 -- 172.21.2.104:0/1390296129 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v157) v1 ==== 125+0+0 (secure 0 0 0) 0x7fbfc8047f60 con 0x7fbfd40affb0 2021-07-11T04:02:36.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 -- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbfb40531f0 msgr2=0x7fbfb40556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbfb40531f0 0x7fbfb40556b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fbfd0002f70 tx=0x7fbfd0006990).stop 2021-07-11T04:02:36.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 -- 172.21.2.104:0/1390296129 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbfd40affb0 msgr2=0x7fbfd40ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbfd40affb0 0x7fbfd40ae5d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7fbfc8002f70 tx=0x7fbfc8015000).stop 2021-07-11T04:02:36.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 -- 172.21.2.104:0/1390296129 shutdown_connections 2021-07-11T04:02:36.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fbfd40b1ec0 0x7fbfd40aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fbfb40531f0 0x7fbfb40556b0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fbfd40affb0 0x7fbfd40ae5d0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 --2- 172.21.2.104:0/1390296129 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fbfd40a70c0 0x7fbfd413ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 -- 172.21.2.104:0/1390296129 >> 172.21.2.104:0/1390296129 conn(0x7fbfd405f6b0 msgr2=0x7fbfd4145e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:36.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 -- 172.21.2.104:0/1390296129 shutdown_connections 2021-07-11T04:02:36.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.488+0000 7fbfdb566700 1 -- 172.21.2.104:0/1390296129 wait complete. 2021-07-11T04:02:36.491 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' created 2021-07-11T04:02:36.495 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2021-07-11T04:02:36.552 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad9102700 1 Processor -- start 2021-07-11T04:02:36.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad9102700 1 -- start start 2021-07-11T04:02:36.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad9102700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad40b2660 0x7f1ad405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad9102700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad40b2a40 con 0x7f1ad40973d0 2021-07-11T04:02:36.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad9102700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad4097b80 con 0x7f1ad405c490 2021-07-11T04:02:36.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad9102700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4097e20 con 0x7f1ad40b2660 2021-07-11T04:02:36.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad359e700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1ad40973d0 0x7f1ad413f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48510/0 (socket says 172.21.2.104:48510) 2021-07-11T04:02:36.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad359e700 1 -- 172.21.2.104:0/2513948621 learned_addr learned my addr 172.21.2.104:0/2513948621 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:36.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad2d9d700 1 --2- 172.21.2.104:0/2513948621 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad40b2660 0x7f1ad405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461282610 0 0) 0x7f1ad40b2a40 con 0x7f1ad40973d0 2021-07-11T04:02:36.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ab8003260 con 0x7f1ad40973d0 2021-07-11T04:02:36.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1753982050 0 0) 0x7f1ab8003260 con 0x7f1ad40973d0 2021-07-11T04:02:36.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad40b2660 msgr2=0x7f1ad405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:36.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 --2- 172.21.2.104:0/2513948621 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad40b2660 0x7f1ad405f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 >> v1:172.21.2.105:6789/0 conn(0x7f1ad405c490 legacy=0x7f1ad4096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad40980c0 con 0x7f1ad40973d0 2021-07-11T04:02:36.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f1acc001a00 con 0x7f1ad40973d0 2021-07-11T04:02:36.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 >> v1:172.21.2.104:6789/0 conn(0x7f1ad40973d0 legacy=0x7f1ad413f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 --2- 172.21.2.104:0/2513948621 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad40b2660 0x7f1ab8003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 --2- 172.21.2.104:0/2513948621 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ab8003db0 0x7f1ab8004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 --2- 172.21.2.104:0/2513948621 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ab8004a40 0x7f1ab8004ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4097e20 con 0x7f1ab8003db0 2021-07-11T04:02:36.564 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab800b910 con 0x7f1ab8004a40 2021-07-11T04:02:36.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab800bad0 con 0x7f1ad40b2660 2021-07-11T04:02:36.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad259c700 1 --2- 172.21.2.104:0/2513948621 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ab8003db0 0x7f1ab8004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.565 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad259c700 1 -- 172.21.2.104:0/2513948621 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad40b2660 msgr2=0x7f1ab8003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:36.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad259c700 1 --2- 172.21.2.104:0/2513948621 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad40b2660 0x7f1ab8003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad259c700 1 -- 172.21.2.104:0/2513948621 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ab8004a40 msgr2=0x7f1ab8004ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:36.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad259c700 1 --2- 172.21.2.104:0/2513948621 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ab8004a40 0x7f1ab8004ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad259c700 1 -- 172.21.2.104:0/2513948621 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab8003260 con 0x7f1ab8003db0 2021-07-11T04:02:36.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad259c700 1 --2- 172.21.2.104:0/2513948621 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ab8003db0 0x7f1ab8004500 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7f1ad40b20e0 tx=0x7f1ac8007a00).ready entity=mon.0 client_cookie=8db6859b5d238e5b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:36.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ac8007030 con 0x7f1ab8003db0 2021-07-11T04:02:36.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac8007190 con 0x7f1ab8003db0 2021-07-11T04:02:36.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad1d9b700 1 -- 172.21.2.104:0/2513948621 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ac8006510 con 0x7f1ab8003db0 2021-07-11T04:02:36.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad9102700 1 -- 172.21.2.104:0/2513948621 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ab8003db0 msgr2=0x7f1ab8004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad9102700 1 --2- 172.21.2.104:0/2513948621 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ab8003db0 0x7f1ab8004500 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7f1ad40b20e0 tx=0x7f1ac8007a00).stop 2021-07-11T04:02:36.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.552+0000 7f1ad359e700 1 -- 172.21.2.104:0/2513948621 reap_dead start 2021-07-11T04:02:36.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 -- 172.21.2.104:0/2513948621 shutdown_connections 2021-07-11T04:02:36.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 -- 172.21.2.104:0/2513948621 >> 172.21.2.104:0/2513948621 conn(0x7f1ad4062f10 msgr2=0x7f1ad40632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:36.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 -- 172.21.2.104:0/2513948621 shutdown_connections 2021-07-11T04:02:36.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 -- 172.21.2.104:0/2513948621 wait complete. 2021-07-11T04:02:36.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 Processor -- start 2021-07-11T04:02:36.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 -- start start 2021-07-11T04:02:36.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40ab180 0x7f1ad40ab560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ad413e7f0 0x7f1ad413ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad40aea10 0x7f1ad40aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40aa080 con 0x7f1ad40ab180 2021-07-11T04:02:36.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40a9de0 con 0x7f1ad413e7f0 2021-07-11T04:02:36.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40ac0f0 con 0x7f1ad40aea10 2021-07-11T04:02:36.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad2d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40ab180 0x7f1ad40ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad2d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40ab180 0x7f1ad40ab560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51750/0 (socket says 172.21.2.104:51750) 2021-07-11T04:02:36.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad2d9d700 1 -- 172.21.2.104:0/3190405890 learned_addr learned my addr 172.21.2.104:0/3190405890 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:36.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad259c700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ad413e7f0 0x7f1ad413ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad2d9d700 1 -- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad40aea10 msgr2=0x7f1ad40aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad2d9d700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad40aea10 0x7f1ad40aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad2d9d700 1 -- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ad413e7f0 msgr2=0x7f1ad413ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad2d9d700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ad413e7f0 0x7f1ad413ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad2d9d700 1 -- 172.21.2.104:0/3190405890 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad409ba10 con 0x7f1ad40ab180 2021-07-11T04:02:36.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad259c700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ad413e7f0 0x7f1ad413ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:36.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad2d9d700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40ab180 0x7f1ad40ab560 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7f1ac400a000 tx=0x7f1ac4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:36.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ac3fff700 1 -- 172.21.2.104:0/3190405890 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ac4016070 con 0x7f1ad40ab180 2021-07-11T04:02:36.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ac3fff700 1 -- 172.21.2.104:0/3190405890 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac4000e90 con 0x7f1ad40ab180 2021-07-11T04:02:36.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 -- 172.21.2.104:0/3190405890 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad413f160 con 0x7f1ad40ab180 2021-07-11T04:02:36.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 -- 172.21.2.104:0/3190405890 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad40a9bf0 con 0x7f1ad40ab180 2021-07-11T04:02:36.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ac3fff700 1 -- 172.21.2.104:0/3190405890 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1ac4011970 con 0x7f1ad40ab180 2021-07-11T04:02:36.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ad9102700 1 -- 172.21.2.104:0/3190405890 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad40aa2f0 con 0x7f1ad40ab180 2021-07-11T04:02:36.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ac3fff700 1 -- 172.21.2.104:0/3190405890 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1ac4011ad0 con 0x7f1ad40ab180 2021-07-11T04:02:36.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ac3fff700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab00531d0 0x7f1ab0055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.556+0000 7f1ac3fff700 1 -- 172.21.2.104:0/3190405890 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5953+0+0 (secure 0 0 0) 0x7f1ac406afd0 con 0x7f1ad40ab180 2021-07-11T04:02:36.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.560+0000 7f1ad259c700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab00531d0 0x7f1ab0055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.560+0000 7f1ad259c700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab00531d0 0x7f1ab0055690 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f1ac8016000 tx=0x7f1ac8013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:36.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.560+0000 7f1ac3fff700 1 -- 172.21.2.104:0/3190405890 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1ac4042270 con 0x7f1ad40ab180 2021-07-11T04:02:36.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.672+0000 7f1ad9102700 1 -- 172.21.2.104:0/3190405890 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f1ad4057350 con 0x7f1ad40ab180 2021-07-11T04:02:36.677 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.676+0000 7f1ac3fff700 1 -- 172.21.2.104:0/3190405890 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 29 and data pool 30 v79) v1 ==== 189+0+0 (secure 0 0 0) 0x7f1ac4045890 con 0x7f1ad40ab180 2021-07-11T04:02:36.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 -- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab00531d0 msgr2=0x7f1ab0055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab00531d0 0x7f1ab0055690 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f1ac8016000 tx=0x7f1ac8013000).stop 2021-07-11T04:02:36.679 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 -- 172.21.2.104:0/3190405890 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40ab180 msgr2=0x7f1ad40ab560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40ab180 0x7f1ad40ab560 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7f1ac400a000 tx=0x7f1ac4008040).stop 2021-07-11T04:02:36.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 -- 172.21.2.104:0/3190405890 shutdown_connections 2021-07-11T04:02:36.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1ad40aea10 0x7f1ad40aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.680 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1ad413e7f0 0x7f1ad413ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1ab00531d0 0x7f1ab0055690 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 --2- 172.21.2.104:0/3190405890 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1ad40ab180 0x7f1ad40ab560 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 -- 172.21.2.104:0/3190405890 >> 172.21.2.104:0/3190405890 conn(0x7f1ad4062f10 msgr2=0x7f1ad4060be0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:36.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 -- 172.21.2.104:0/3190405890 shutdown_connections 2021-07-11T04:02:36.681 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.680+0000 7f1ad9102700 1 -- 172.21.2.104:0/3190405890 wait complete. 2021-07-11T04:02:36.682 INFO:teuthology.orchestra.run.gibba004.stderr:new fs with metadata pool 29 and data pool 30 2021-07-11T04:02:36.686 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:36.736 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc920895700 1 Processor -- start 2021-07-11T04:02:36.737 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc920895700 1 -- start start 2021-07-11T04:02:36.741 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc920895700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc91c0b4710 0x7fc91c13fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc920895700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc91c1439a0 0x7fc91c140290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc920895700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91c0b6910 con 0x7fc91c061680 2021-07-11T04:02:36.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc920895700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91c0b9680 con 0x7fc91c0b4710 2021-07-11T04:02:36.742 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc920895700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0b6bb0 con 0x7fc91c1439a0 2021-07-11T04:02:36.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc91ad9d700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc91c061680 0x7fc91c0b9d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48532/0 (socket says 172.21.2.104:48532) 2021-07-11T04:02:36.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc91ad9d700 1 -- 172.21.2.104:0/1073782748 learned_addr learned my addr 172.21.2.104:0/1073782748 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:36.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085105180 0 0) 0x7fc91c0b6910 con 0x7fc91c061680 2021-07-11T04:02:36.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc900003260 con 0x7fc91c061680 2021-07-11T04:02:36.743 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc91a59c700 1 --2- 172.21.2.104:0/1073782748 >> v2:172.21.2.105:3300/0 conn(0x7fc91c0b4710 0x7fc91c13fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc91b59e700 1 --2- 172.21.2.104:0/1073782748 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc91c1439a0 0x7fc91c140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2321268170 0 0) 0x7fc900003260 con 0x7fc91c061680 2021-07-11T04:02:36.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc91c1439a0 msgr2=0x7fc91c140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:36.744 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 --2- 172.21.2.104:0/1073782748 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc91c1439a0 0x7fc91c140290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 >> v2:172.21.2.105:3300/0 conn(0x7fc91c0b4710 msgr2=0x7fc91c13fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:36.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 --2- 172.21.2.104:0/1073782748 >> v2:172.21.2.105:3300/0 conn(0x7fc91c0b4710 0x7fc91c13fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c0b6e50 con 0x7fc91c061680 2021-07-11T04:02:36.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fc910001660 con 0x7fc91c061680 2021-07-11T04:02:36.745 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 >> v1:172.21.2.104:6789/0 conn(0x7fc91c061680 legacy=0x7fc91c0b9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 --2- 172.21.2.104:0/1073782748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc900003750 0x7fc900003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 --2- 172.21.2.104:0/1073782748 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc900004500 0x7fc900004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 --2- 172.21.2.104:0/1073782748 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc91c1439a0 0x7fc900004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.746 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0b6bb0 con 0x7fc900003750 2021-07-11T04:02:36.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90000b750 con 0x7fc900004500 2021-07-11T04:02:36.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90000b8e0 con 0x7fc91c1439a0 2021-07-11T04:02:36.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc91ad9d700 1 --2- 172.21.2.104:0/1073782748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc900003750 0x7fc900003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.747 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc91ad9d700 1 -- 172.21.2.104:0/1073782748 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc91c1439a0 msgr2=0x7fc900004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:36.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc91ad9d700 1 --2- 172.21.2.104:0/1073782748 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc91c1439a0 0x7fc900004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc91ad9d700 1 -- 172.21.2.104:0/1073782748 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc900004500 msgr2=0x7fc900004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:36.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc91ad9d700 1 --2- 172.21.2.104:0/1073782748 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc900004500 0x7fc900004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.736+0000 7fc91ad9d700 1 -- 172.21.2.104:0/1073782748 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc900003260 con 0x7fc900003750 2021-07-11T04:02:36.748 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91ad9d700 1 --2- 172.21.2.104:0/1073782748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc900003750 0x7fc900003b70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 rx=0x7fc910001370 tx=0x7fc91c067700).ready entity=mon.0 client_cookie=93f7c4310f062c7c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:36.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc9100071a0 con 0x7fc900003750 2021-07-11T04:02:36.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc910007300 con 0x7fc900003750 2021-07-11T04:02:36.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc919d9b700 1 -- 172.21.2.104:0/1073782748 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc91000cbb0 con 0x7fc900003750 2021-07-11T04:02:36.749 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- 172.21.2.104:0/1073782748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc900003750 msgr2=0x7fc900003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 --2- 172.21.2.104:0/1073782748 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc900003750 0x7fc900003b70 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 rx=0x7fc910001370 tx=0x7fc91c067700).stop 2021-07-11T04:02:36.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91b59e700 1 -- 172.21.2.104:0/1073782748 reap_dead start 2021-07-11T04:02:36.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- 172.21.2.104:0/1073782748 shutdown_connections 2021-07-11T04:02:36.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- 172.21.2.104:0/1073782748 >> 172.21.2.104:0/1073782748 conn(0x7fc91c06adf0 msgr2=0x7fc91c05f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:36.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- 172.21.2.104:0/1073782748 shutdown_connections 2021-07-11T04:02:36.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- 172.21.2.104:0/1073782748 wait complete. 2021-07-11T04:02:36.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 Processor -- start 2021-07-11T04:02:36.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- start start 2021-07-11T04:02:36.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc91c146870 0x7fc91c152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc91c153120 0x7fc91c155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc91c155f80 0x7fc91c156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c1568f0 con 0x7fc91c153120 2021-07-11T04:02:36.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0b0200 con 0x7fc91c155f80 2021-07-11T04:02:36.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c0b04a0 con 0x7fc91c146870 2021-07-11T04:02:36.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91a59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc91c153120 0x7fc91c155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91a59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc91c153120 0x7fc91c155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51778/0 (socket says 172.21.2.104:51778) 2021-07-11T04:02:36.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91a59c700 1 -- 172.21.2.104:0/518185483 learned_addr learned my addr 172.21.2.104:0/518185483 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:36.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91a59c700 1 -- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc91c146870 msgr2=0x7fc91c152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91a59c700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc91c146870 0x7fc91c152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91a59c700 1 -- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc91c155f80 msgr2=0x7fc91c156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91a59c700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc91c155f80 0x7fc91c156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91a59c700 1 -- 172.21.2.104:0/518185483 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c152790 con 0x7fc91c153120 2021-07-11T04:02:36.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91a59c700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc91c153120 0x7fc91c155a90 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 rx=0x7fc90c00a000 tx=0x7fc90c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:36.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc90bfff700 1 -- 172.21.2.104:0/518185483 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc90c016070 con 0x7fc91c153120 2021-07-11T04:02:36.755 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc90bfff700 1 -- 172.21.2.104:0/518185483 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c000e90 con 0x7fc91c153120 2021-07-11T04:02:36.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- 172.21.2.104:0/518185483 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91c116280 con 0x7fc91c153120 2021-07-11T04:02:36.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc90bfff700 1 -- 172.21.2.104:0/518185483 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc90c011a40 con 0x7fc91c153120 2021-07-11T04:02:36.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- 172.21.2.104:0/518185483 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91c0afef0 con 0x7fc91c153120 2021-07-11T04:02:36.756 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc920895700 1 -- 172.21.2.104:0/518185483 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91c146df0 con 0x7fc91c153120 2021-07-11T04:02:36.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc90bfff700 1 -- 172.21.2.104:0/518185483 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc90c011ba0 con 0x7fc91c153120 2021-07-11T04:02:36.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc90bfff700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc8f80531d0 0x7fc8f8055690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc90bfff700 1 -- 172.21.2.104:0/518185483 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc90c06b210 con 0x7fc91c153120 2021-07-11T04:02:36.757 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.740+0000 7fc91ad9d700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc8f80531d0 0x7fc8f8055690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.744+0000 7fc91ad9d700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc8f80531d0 0x7fc8f8055690 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fc910001370 tx=0x7fc910013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:36.758 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.744+0000 7fc90bfff700 1 -- 172.21.2.104:0/518185483 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc90c042550 con 0x7fc91c153120 2021-07-11T04:02:36.827 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 -- 172.21.2.104:0/518185483 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc91c152920 con 0x7fc91c153120 2021-07-11T04:02:36.828 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc90bfff700 1 -- 172.21.2.104:0/518185483 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16065 (secure 0 0 0) 0x7fc90c045b70 con 0x7fc91c153120 2021-07-11T04:02:36.828 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:36.828 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":159,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:36.673863+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:02:34.620986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-07-11T04:02:35.639038+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:36.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 -- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc8f80531d0 msgr2=0x7fc8f8055690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc8f80531d0 0x7fc8f8055690 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7fc910001370 tx=0x7fc910013000).stop 2021-07-11T04:02:36.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 -- 172.21.2.104:0/518185483 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc91c153120 msgr2=0x7fc91c155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.830 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc91c153120 0x7fc91c155a90 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 rx=0x7fc90c00a000 tx=0x7fc90c008040).stop 2021-07-11T04:02:36.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 -- 172.21.2.104:0/518185483 shutdown_connections 2021-07-11T04:02:36.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc91c155f80 0x7fc91c156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc8f80531d0 0x7fc8f8055690 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.831 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc91c153120 0x7fc91c155a90 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 --2- 172.21.2.104:0/518185483 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc91c146870 0x7fc91c152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 -- 172.21.2.104:0/518185483 >> 172.21.2.104:0/518185483 conn(0x7fc91c06adf0 msgr2=0x7fc91c0b84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:36.832 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 -- 172.21.2.104:0/518185483 shutdown_connections 2021-07-11T04:02:36.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.828+0000 7fc920895700 1 -- 172.21.2.104:0/518185483 wait complete. 2021-07-11T04:02:36.836 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:36.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152f2ac700 1 Processor -- start 2021-07-11T04:02:36.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152f2ac700 1 -- start start 2021-07-11T04:02:36.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152f2ac700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f15280b1970 0x7f152805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152f2ac700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15280a7a90 con 0x7f1528067180 2021-07-11T04:02:36.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152f2ac700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1528147620 con 0x7f15280a76b0 2021-07-11T04:02:36.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152f2ac700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15281478c0 con 0x7f15280b1970 2021-07-11T04:02:36.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152c847700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1528067180 0x7f152805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48552/0 (socket says 172.21.2.104:48552) 2021-07-11T04:02:36.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152c847700 1 -- 172.21.2.104:0/3561388624 learned_addr learned my addr 172.21.2.104:0/3561388624 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:36.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578260373 0 0) 0x7f15280a7a90 con 0x7f1528067180 2021-07-11T04:02:36.893 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1510003260 con 0x7f1528067180 2021-07-11T04:02:36.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152d849700 1 --2- 172.21.2.104:0/3561388624 >> v2:172.21.2.105:3300/0 conn(0x7f15280b1970 0x7f152805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1561516983 0 0) 0x7f1510003260 con 0x7f1528067180 2021-07-11T04:02:36.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 >> v2:172.21.2.105:3300/0 conn(0x7f15280b1970 msgr2=0x7f152805fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:36.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 --2- 172.21.2.104:0/3561388624 >> v2:172.21.2.105:3300/0 conn(0x7f15280b1970 0x7f152805fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 >> v1:172.21.2.105:6789/0 conn(0x7f15280a76b0 legacy=0x7f1528066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:36.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1528147b60 con 0x7f1528067180 2021-07-11T04:02:36.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f1524001660 con 0x7f1528067180 2021-07-11T04:02:36.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 >> v1:172.21.2.104:6789/0 conn(0x7f1528067180 legacy=0x7f152805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 --2- 172.21.2.104:0/3561388624 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1510003820 0x7f1510003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 --2- 172.21.2.104:0/3561388624 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1510004500 0x7f1510004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 --2- 172.21.2.104:0/3561388624 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 0x7f151000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15281478c0 con 0x7f1510004f90 2021-07-11T04:02:36.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151000be80 con 0x7f1510003820 2021-07-11T04:02:36.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151000c040 con 0x7f1510004500 2021-07-11T04:02:36.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152d849700 1 --2- 172.21.2.104:0/3561388624 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 0x7f151000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152d849700 1 -- 172.21.2.104:0/3561388624 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1510004500 msgr2=0x7f1510004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:36.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152d849700 1 --2- 172.21.2.104:0/3561388624 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1510004500 0x7f1510004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152d849700 1 -- 172.21.2.104:0/3561388624 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1510003820 msgr2=0x7f1510003c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:36.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152d849700 1 --2- 172.21.2.104:0/3561388624 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1510003820 0x7f1510003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152d849700 1 -- 172.21.2.104:0/3561388624 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1510003260 con 0x7f1510004f90 2021-07-11T04:02:36.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152d849700 1 -- 172.21.2.104:0/3561388624 reap_dead start 2021-07-11T04:02:36.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152d849700 1 --2- 172.21.2.104:0/3561388624 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 0x7f151000b750 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x7f152000a000 tx=0x7f1520008040).ready entity=mon.0 client_cookie=e7b5892c5faeea87 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:36.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f152000c070 con 0x7f1510004f90 2021-07-11T04:02:36.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f152000e7e0 con 0x7f1510004f90 2021-07-11T04:02:36.900 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f1517fff700 1 -- 172.21.2.104:0/3561388624 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f152000eb40 con 0x7f1510004f90 2021-07-11T04:02:36.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152f2ac700 1 -- 172.21.2.104:0/3561388624 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 msgr2=0x7f151000b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.888+0000 7f152f2ac700 1 --2- 172.21.2.104:0/3561388624 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 0x7f151000b750 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 rx=0x7f152000a000 tx=0x7f1520008040).stop 2021-07-11T04:02:36.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 -- 172.21.2.104:0/3561388624 shutdown_connections 2021-07-11T04:02:36.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 --2- 172.21.2.104:0/3561388624 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 0x7f151000b750 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.901 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 -- 172.21.2.104:0/3561388624 >> 172.21.2.104:0/3561388624 conn(0x7f15280b53e0 msgr2=0x7f15280b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:36.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 -- 172.21.2.104:0/3561388624 shutdown_connections 2021-07-11T04:02:36.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 -- 172.21.2.104:0/3561388624 wait complete. 2021-07-11T04:02:36.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 Processor -- start 2021-07-11T04:02:36.902 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 -- start start 2021-07-11T04:02:36.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 0x7f152814e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f15280aba20 0x7f15280adf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.903 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f15280aa0f0 0x7f15280ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280aa700 con 0x7f1510004f90 2021-07-11T04:02:36.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280ae910 con 0x7f15280aa0f0 2021-07-11T04:02:36.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15280ab450 con 0x7f15280aba20 2021-07-11T04:02:36.904 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152d048700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 0x7f152814e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152d048700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 0x7f152814e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51798/0 (socket says 172.21.2.104:51798) 2021-07-11T04:02:36.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152d048700 1 -- 172.21.2.104:0/3456505700 learned_addr learned my addr 172.21.2.104:0/3456505700 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:36.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152d048700 1 -- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f15280aba20 msgr2=0x7f15280adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.905 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152d048700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f15280aba20 0x7f15280adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152d048700 1 -- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f15280aa0f0 msgr2=0x7f15280ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152d048700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f15280aa0f0 0x7f15280ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.906 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152d048700 1 -- 172.21.2.104:0/3456505700 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15280ab6f0 con 0x7f1510004f90 2021-07-11T04:02:36.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152d048700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 0x7f152814e5c0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 rx=0x7f1518001fa0 tx=0x7f1518007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:36.908 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f15167fc700 1 -- 172.21.2.104:0/3456505700 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f151800ce80 con 0x7f1510004f90 2021-07-11T04:02:36.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f15167fc700 1 -- 172.21.2.104:0/3456505700 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1518000e90 con 0x7f1510004f90 2021-07-11T04:02:36.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 -- 172.21.2.104:0/3456505700 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15280aa930 con 0x7f1510004f90 2021-07-11T04:02:36.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 -- 172.21.2.104:0/3456505700 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15280aeb30 con 0x7f1510004f90 2021-07-11T04:02:36.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f15167fc700 1 -- 172.21.2.104:0/3456505700 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1518007430 con 0x7f1510004f90 2021-07-11T04:02:36.909 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152f2ac700 1 -- 172.21.2.104:0/3456505700 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1528057350 con 0x7f1510004f90 2021-07-11T04:02:36.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f15167fc700 1 -- 172.21.2.104:0/3456505700 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f151801a060 con 0x7f1510004f90 2021-07-11T04:02:36.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f15167fc700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1508053380 0x7f1508055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:36.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f15167fc700 1 -- 172.21.2.104:0/3456505700 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f151806ae10 con 0x7f1510004f90 2021-07-11T04:02:36.910 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.892+0000 7f152c847700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1508053380 0x7f1508055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:36.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.896+0000 7f152c847700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1508053380 0x7f1508055840 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f152806a180 tx=0x7f15280a8c80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:36.911 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.896+0000 7f15167fc700 1 -- 172.21.2.104:0/3456505700 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1518042040 con 0x7f1510004f90 2021-07-11T04:02:36.978 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 -- 172.21.2.104:0/3456505700 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15280698e0 con 0x7f1510004f90 2021-07-11T04:02:36.979 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f15167fc700 1 -- 172.21.2.104:0/3456505700 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f1518018020 con 0x7f1510004f90 2021-07-11T04:02:36.979 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:36.980 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":159,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:36.673863+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:02:34.620986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-07-11T04:02:35.639038+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:36.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 -- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1508053380 msgr2=0x7f1508055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1508053380 0x7f1508055840 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f152806a180 tx=0x7f15280a8c80).stop 2021-07-11T04:02:36.981 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 -- 172.21.2.104:0/3456505700 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 msgr2=0x7f152814e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:36.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 0x7f152814e5c0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 rx=0x7f1518001fa0 tx=0x7f1518007c30).stop 2021-07-11T04:02:36.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 -- 172.21.2.104:0/3456505700 shutdown_connections 2021-07-11T04:02:36.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f15280aa0f0 0x7f15280ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f15280aba20 0x7f15280adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:36.982 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1508053380 0x7f1508055840 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 --2- 172.21.2.104:0/3456505700 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1510004f90 0x7f152814e5c0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:36.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 -- 172.21.2.104:0/3456505700 >> 172.21.2.104:0/3456505700 conn(0x7f15280b53e0 msgr2=0x7f152805fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:36.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 -- 172.21.2.104:0/3456505700 shutdown_connections 2021-07-11T04:02:36.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:36.980+0000 7f152f2ac700 1 -- 172.21.2.104:0/3456505700 wait complete. 2021-07-11T04:02:36.987 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2021-07-11T04:02:37.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.040+0000 7eff48030700 1 Processor -- start 2021-07-11T04:02:37.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.040+0000 7eff48030700 1 -- start start 2021-07-11T04:02:37.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.040+0000 7eff48030700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff400b1ec0 0x7eff400b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.040+0000 7eff48030700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7eff400b22a0 0x7eff40141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.040+0000 7eff48030700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff400a77d0 con 0x7eff400affb0 2021-07-11T04:02:37.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.040+0000 7eff48030700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff400b03f0 con 0x7eff400b22a0 2021-07-11T04:02:37.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.040+0000 7eff48030700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff400b0650 con 0x7eff400b1ec0 2021-07-11T04:02:37.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.040+0000 7eff465cd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7eff400b22a0 0x7eff40141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.040+0000 7eff465cd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7eff400b22a0 0x7eff40141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51806/0 (socket says 172.21.2.104:51806) 2021-07-11T04:02:37.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.040+0000 7eff465cd700 1 -- 172.21.2.104:0/4011796439 learned_addr learned my addr 172.21.2.104:0/4011796439 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:37.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff465cd700 1 -- 172.21.2.104:0/4011796439 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff400b1ec0 msgr2=0x7eff400b12d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:37.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff465cd700 1 --2- 172.21.2.104:0/4011796439 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff400b1ec0 0x7eff400b12d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:37.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff465cd700 1 -- 172.21.2.104:0/4011796439 >> v1:172.21.2.105:6789/0 conn(0x7eff400affb0 legacy=0x7eff400a70c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:37.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff465cd700 1 -- 172.21.2.104:0/4011796439 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400b08f0 con 0x7eff400b22a0 2021-07-11T04:02:37.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff465cd700 1 --2- 172.21.2.104:0/4011796439 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff400b22a0 0x7eff40141250 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7eff4006a0f0 tx=0x7eff38009290).ready entity=mon.0 client_cookie=e2ae551abbf4e3d0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:37.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff44dca700 1 -- 172.21.2.104:0/4011796439 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff3800b860 con 0x7eff400b22a0 2021-07-11T04:02:37.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff44dca700 1 -- 172.21.2.104:0/4011796439 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3800b9c0 con 0x7eff400b22a0 2021-07-11T04:02:37.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff44dca700 1 -- 172.21.2.104:0/4011796439 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff3800bb70 con 0x7eff400b22a0 2021-07-11T04:02:37.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- 172.21.2.104:0/4011796439 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff400b22a0 msgr2=0x7eff40141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 --2- 172.21.2.104:0/4011796439 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff400b22a0 0x7eff40141250 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7eff4006a0f0 tx=0x7eff38009290).stop 2021-07-11T04:02:37.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- 172.21.2.104:0/4011796439 shutdown_connections 2021-07-11T04:02:37.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 --2- 172.21.2.104:0/4011796439 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff400b22a0 0x7eff40141250 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 --2- 172.21.2.104:0/4011796439 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff400b1ec0 0x7eff400b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:37.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- 172.21.2.104:0/4011796439 >> 172.21.2.104:0/4011796439 conn(0x7eff4005f6b0 msgr2=0x7eff4005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:37.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- 172.21.2.104:0/4011796439 shutdown_connections 2021-07-11T04:02:37.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- 172.21.2.104:0/4011796439 wait complete. 2021-07-11T04:02:37.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 Processor -- start 2021-07-11T04:02:37.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- start start 2021-07-11T04:02:37.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff400affb0 0x7eff40140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff400b1ec0 0x7eff4015aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff400b22a0 0x7eff40157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff400adee0 con 0x7eff400b1ec0 2021-07-11T04:02:37.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff400ad9a0 con 0x7eff400affb0 2021-07-11T04:02:37.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff400adc40 con 0x7eff400b22a0 2021-07-11T04:02:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff455cb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff400b1ec0 0x7eff4015aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff455cb700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff400b1ec0 0x7eff4015aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51812/0 (socket says 172.21.2.104:51812) 2021-07-11T04:02:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff455cb700 1 -- 172.21.2.104:0/1298510345 learned_addr learned my addr 172.21.2.104:0/1298510345 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff455cb700 1 -- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff400b22a0 msgr2=0x7eff40157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff455cb700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff400b22a0 0x7eff40157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:37.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff455cb700 1 -- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff400affb0 msgr2=0x7eff40140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff455cb700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff400affb0 0x7eff40140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:37.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff455cb700 1 -- 172.21.2.104:0/1298510345 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff38009530 con 0x7eff400b1ec0 2021-07-11T04:02:37.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff455cb700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff400b1ec0 0x7eff4015aa40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7eff3c002fb0 tx=0x7eff3c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:37.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff2effd700 1 -- 172.21.2.104:0/1298510345 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff3c013460 con 0x7eff400b1ec0 2021-07-11T04:02:37.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff2effd700 1 -- 172.21.2.104:0/1298510345 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c0135c0 con 0x7eff400b1ec0 2021-07-11T04:02:37.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- 172.21.2.104:0/1298510345 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff40156ca0 con 0x7eff400b1ec0 2021-07-11T04:02:37.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- 172.21.2.104:0/1298510345 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff400ae560 con 0x7eff400b1ec0 2021-07-11T04:02:37.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff2effd700 1 -- 172.21.2.104:0/1298510345 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7eff3c013920 con 0x7eff400b1ec0 2021-07-11T04:02:37.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.044+0000 7eff48030700 1 -- 172.21.2.104:0/1298510345 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c003410 con 0x7eff400b1ec0 2021-07-11T04:02:37.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.048+0000 7eff2effd700 1 -- 172.21.2.104:0/1298510345 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7eff3c00f070 con 0x7eff400b1ec0 2021-07-11T04:02:37.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.048+0000 7eff2effd700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff200532d0 0x7eff20055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.048+0000 7eff2effd700 1 -- 172.21.2.104:0/1298510345 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6069+0+0 (secure 0 0 0) 0x7eff3c06afc0 con 0x7eff400b1ec0 2021-07-11T04:02:37.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.048+0000 7eff45dcc700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff200532d0 0x7eff20055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.048+0000 7eff45dcc700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff200532d0 0x7eff20055790 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7eff30002fb0 tx=0x7eff300069d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:37.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.048+0000 7eff2effd700 1 -- 172.21.2.104:0/1298510345 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7eff3c0421f0 con 0x7eff400b1ec0 2021-07-11T04:02:37.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.156+0000 7eff48030700 1 -- 172.21.2.104:0/1298510345 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7eff1c003c40 con 0x7eff400b1ec0 2021-07-11T04:02:37.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.688+0000 7eff2effd700 1 -- 172.21.2.104:0/1298510345 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) v1 ==== 118+0+0 (secure 0 0 0) 0x7eff3c045810 con 0x7eff400b1ec0 2021-07-11T04:02:37.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 -- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff200532d0 msgr2=0x7eff20055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff200532d0 0x7eff20055790 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7eff30002fb0 tx=0x7eff300069d0).stop 2021-07-11T04:02:37.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 -- 172.21.2.104:0/1298510345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff400b1ec0 msgr2=0x7eff4015aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff400b1ec0 0x7eff4015aa40 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 rx=0x7eff3c002fb0 tx=0x7eff3c006df0).stop 2021-07-11T04:02:37.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 -- 172.21.2.104:0/1298510345 shutdown_connections 2021-07-11T04:02:37.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7eff400b22a0 0x7eff40157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:37.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7eff200532d0 0x7eff20055790 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7eff400b1ec0 0x7eff4015aa40 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 --2- 172.21.2.104:0/1298510345 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7eff400affb0 0x7eff40140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:37.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 -- 172.21.2.104:0/1298510345 >> 172.21.2.104:0/1298510345 conn(0x7eff4005f6b0 msgr2=0x7eff401477a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:37.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 -- 172.21.2.104:0/1298510345 shutdown_connections 2021-07-11T04:02:37.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.692+0000 7eff48030700 1 -- 172.21.2.104:0/1298510345 wait complete. 2021-07-11T04:02:37.698 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:37.750 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.748+0000 7f917f4f2700 1 Processor -- start 2021-07-11T04:02:37.751 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.748+0000 7f917f4f2700 1 -- start start 2021-07-11T04:02:37.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917f4f2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780b2660 0x7f917805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917f4f2700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f917805c490 0x7f9178096cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.752 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917f4f2700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91780b2a40 con 0x7f917805c8d0 2021-07-11T04:02:37.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917f4f2700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9178097b80 con 0x7f917805c490 2021-07-11T04:02:37.753 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917f4f2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9178097e20 con 0x7f91780b2660 2021-07-11T04:02:37.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917da8f700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f917805c8d0 0x7f917813f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48588/0 (socket says 172.21.2.104:48588) 2021-07-11T04:02:37.754 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917da8f700 1 -- 172.21.2.104:0/660549855 learned_addr learned my addr 172.21.2.104:0/660549855 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:37.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917ca8d700 1 --2- 172.21.2.104:0/660549855 >> v2:172.21.2.105:3300/0 conn(0x7f917805c490 0x7f9178096cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917d28e700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780b2660 0x7f917805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060009728 0 0) 0x7f91780b2a40 con 0x7f917805c8d0 2021-07-11T04:02:37.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9160003260 con 0x7f917805c8d0 2021-07-11T04:02:37.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 571402320 0 0) 0x7f9160003260 con 0x7f917805c8d0 2021-07-11T04:02:37.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780b2660 msgr2=0x7f917805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:37.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780b2660 0x7f917805f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 >> v2:172.21.2.105:3300/0 conn(0x7f917805c490 msgr2=0x7f9178096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:37.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 --2- 172.21.2.104:0/660549855 >> v2:172.21.2.105:3300/0 conn(0x7f917805c490 0x7f9178096cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91780980c0 con 0x7f917805c8d0 2021-07-11T04:02:37.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f91700019e0 con 0x7f917805c8d0 2021-07-11T04:02:37.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 >> v1:172.21.2.104:6789/0 conn(0x7f917805c8d0 legacy=0x7f917813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9160003750 0x7f9160003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9160004500 0x7f9160004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.775 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780b2660 0x7f9160004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9178097b80 con 0x7f9160003750 2021-07-11T04:02:37.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9178097e20 con 0x7f9160004500 2021-07-11T04:02:37.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916000b750 con 0x7f91780b2660 2021-07-11T04:02:37.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917ca8d700 1 --2- 172.21.2.104:0/660549855 >> v2:172.21.2.105:3300/0 conn(0x7f917805c490 0x7f9178096cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:02:37.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917d28e700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9160003750 0x7f9160003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917da8f700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780b2660 0x7f9160004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917d28e700 1 -- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780b2660 msgr2=0x7f9160004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:37.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917ca8d700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9160004500 0x7f9160004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917d28e700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780b2660 0x7f9160004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917d28e700 1 -- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9160004500 msgr2=0x7f9160004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:37.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917d28e700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9160004500 0x7f9160004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917d28e700 1 -- 172.21.2.104:0/660549855 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9160003260 con 0x7f9160003750 2021-07-11T04:02:37.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917ca8d700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9160004500 0x7f9160004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:37.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917d28e700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9160003750 0x7f9160003b70 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f917400c000 tx=0x7f9174006df0).ready entity=mon.0 client_cookie=42d9ffdb60f25179 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:37.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f917400e7d0 con 0x7f9160003750 2021-07-11T04:02:37.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917400e930 con 0x7f9160003750 2021-07-11T04:02:37.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f9167fff700 1 -- 172.21.2.104:0/660549855 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f917400ec90 con 0x7f9160003750 2021-07-11T04:02:37.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917f4f2700 1 -- 172.21.2.104:0/660549855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9160003750 msgr2=0x7f9160003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917f4f2700 1 --2- 172.21.2.104:0/660549855 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9160003750 0x7f9160003b70 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f917400c000 tx=0x7f9174006df0).stop 2021-07-11T04:02:37.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917da8f700 1 -- 172.21.2.104:0/660549855 reap_dead start 2021-07-11T04:02:37.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917f4f2700 1 -- 172.21.2.104:0/660549855 shutdown_connections 2021-07-11T04:02:37.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917f4f2700 1 -- 172.21.2.104:0/660549855 >> 172.21.2.104:0/660549855 conn(0x7f9178062f10 msgr2=0x7f91780632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:37.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917f4f2700 1 -- 172.21.2.104:0/660549855 shutdown_connections 2021-07-11T04:02:37.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.752+0000 7f917f4f2700 1 -- 172.21.2.104:0/660549855 wait complete. 2021-07-11T04:02:37.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917f4f2700 1 Processor -- start 2021-07-11T04:02:37.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917f4f2700 1 -- start start 2021-07-11T04:02:37.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917f4f2700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f91780af3f0 0x7f9178063bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917f4f2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91780950c0 0x7f91780640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917f4f2700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780651a0 0x7f9178156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917f4f2700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91780675b0 con 0x7f91780950c0 2021-07-11T04:02:37.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917f4f2700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91780ac000 con 0x7f91780af3f0 2021-07-11T04:02:37.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917f4f2700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91780af7d0 con 0x7f91780651a0 2021-07-11T04:02:37.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917ca8d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91780950c0 0x7f91780640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917ca8d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91780950c0 0x7f91780640e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51832/0 (socket says 172.21.2.104:51832) 2021-07-11T04:02:37.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917ca8d700 1 -- 172.21.2.104:0/504939701 learned_addr learned my addr 172.21.2.104:0/504939701 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:37.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917d28e700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f91780af3f0 0x7f9178063bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917da8f700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780651a0 0x7f9178156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917ca8d700 1 -- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780651a0 msgr2=0x7f9178156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917ca8d700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780651a0 0x7f9178156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917ca8d700 1 -- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f91780af3f0 msgr2=0x7f9178063bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917ca8d700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f91780af3f0 0x7f9178063bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917ca8d700 1 -- 172.21.2.104:0/504939701 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91780ac6f0 con 0x7f91780950c0 2021-07-11T04:02:37.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917da8f700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780651a0 0x7f9178156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-07-11T04:02:37.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917ca8d700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91780950c0 0x7f91780640e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 rx=0x7f9168008000 tx=0x7f9168008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:37.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f91667fc700 1 -- 172.21.2.104:0/504939701 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f916800c070 con 0x7f91780950c0 2021-07-11T04:02:37.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f91667fc700 1 -- 172.21.2.104:0/504939701 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9168000e90 con 0x7f91780950c0 2021-07-11T04:02:37.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917f4f2700 1 -- 172.21.2.104:0/504939701 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9178067800 con 0x7f91780950c0 2021-07-11T04:02:37.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917f4f2700 1 -- 172.21.2.104:0/504939701 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91780645d0 con 0x7f91780950c0 2021-07-11T04:02:37.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f91667fc700 1 -- 172.21.2.104:0/504939701 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f91680135e0 con 0x7f91780950c0 2021-07-11T04:02:37.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917f4f2700 1 -- 172.21.2.104:0/504939701 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9154003410 con 0x7f91780950c0 2021-07-11T04:02:37.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f91667fc700 1 -- 172.21.2.104:0/504939701 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9168019020 con 0x7f91780950c0 2021-07-11T04:02:37.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f91667fc700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f91580531f0 0x7f91580556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f91667fc700 1 -- 172.21.2.104:0/504939701 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f916806aba0 con 0x7f91780950c0 2021-07-11T04:02:37.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.756+0000 7f917d28e700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f91580531f0 0x7f91580556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.760+0000 7f917d28e700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f91580531f0 0x7f91580556b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f917401a000 tx=0x7f9174019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:37.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.760+0000 7f91667fc700 1 -- 172.21.2.104:0/504939701 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9168041ee0 con 0x7f91780950c0 2021-07-11T04:02:37.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.868+0000 7f917f4f2700 1 -- 172.21.2.104:0/504939701 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9154003c40 con 0x7f91780950c0 2021-07-11T04:02:37.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.868+0000 7f91667fc700 1 -- 172.21.2.104:0/504939701 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f9168017030 con 0x7f91780950c0 2021-07-11T04:02:37.873 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:37.873 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":80,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5132,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/770706140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":770706140},{"type":"v1","addr":"172.21.2.105:6835","nonce":770706140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":79},{"gid":8718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/488029782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":488029782},{"type":"v1","addr":"172.21.2.104:6833","nonce":488029782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:36.673961+0000","modified":"2021-07-11T04:02:37.679104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8721":{"gid":8721,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/2638676458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2638676458},{"type":"v1","addr":"172.21.2.104:6835","nonce":2638676458}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-07-11T04:02:37.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 -- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f91580531f0 msgr2=0x7f91580556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f91580531f0 0x7f91580556b0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f917401a000 tx=0x7f9174019000).stop 2021-07-11T04:02:37.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 -- 172.21.2.104:0/504939701 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91780950c0 msgr2=0x7f91780640e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91780950c0 0x7f91780640e0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 rx=0x7f9168008000 tx=0x7f9168008040).stop 2021-07-11T04:02:37.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 -- 172.21.2.104:0/504939701 shutdown_connections 2021-07-11T04:02:37.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f91780651a0 0x7f9178156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f91580531f0 0x7f91580556b0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f91780950c0 0x7f91780640e0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 --2- 172.21.2.104:0/504939701 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f91780af3f0 0x7f9178063bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 -- 172.21.2.104:0/504939701 >> 172.21.2.104:0/504939701 conn(0x7f9178062f10 msgr2=0x7f91780b49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:37.879 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 -- 172.21.2.104:0/504939701 shutdown_connections 2021-07-11T04:02:37.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.872+0000 7f917f4f2700 1 -- 172.21.2.104:0/504939701 wait complete. 2021-07-11T04:02:37.880 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 80 2021-07-11T04:02:37.882 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:37.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f167eb64700 1 Processor -- start 2021-07-11T04:02:37.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f167eb64700 1 -- start start 2021-07-11T04:02:37.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f167eb64700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16780b1170 0x7f1678141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f167eb64700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780b31d0 con 0x7f1678141920 2021-07-11T04:02:37.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f167eb64700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16780b7bf0 con 0x7f16780b7810 2021-07-11T04:02:37.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f167eb64700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16780675c0 con 0x7f16780b1170 2021-07-11T04:02:37.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f167d101700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f1678141920 0x7f167814bb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48610/0 (socket says 172.21.2.104:48610) 2021-07-11T04:02:37.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f167d101700 1 -- 172.21.2.104:0/732395235 learned_addr learned my addr 172.21.2.104:0/732395235 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:37.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226028259 0 0) 0x7f16780b31d0 con 0x7f1678141920 2021-07-11T04:02:37.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1658003260 con 0x7f1678141920 2021-07-11T04:02:37.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 159829055 0 0) 0x7f1658003260 con 0x7f1678141920 2021-07-11T04:02:37.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f1677fff700 1 --2- 172.21.2.104:0/732395235 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16780b1170 0x7f1678141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16780b1170 msgr2=0x7f1678141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:37.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 --2- 172.21.2.104:0/732395235 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16780b1170 0x7f1678141430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:37.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 >> v1:172.21.2.105:6789/0 conn(0x7f16780b7810 legacy=0x7f16781445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:37.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16780aae70 con 0x7f1678141920 2021-07-11T04:02:37.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f16700019e0 con 0x7f1678141920 2021-07-11T04:02:37.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 >> v1:172.21.2.104:6789/0 conn(0x7f1678141920 legacy=0x7f167814bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 --2- 172.21.2.104:0/732395235 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1658003800 0x7f1658003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 --2- 172.21.2.104:0/732395235 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1658004500 0x7f1658004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 --2- 172.21.2.104:0/732395235 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1658005070 0x7f165800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780675c0 con 0x7f1658004500 2021-07-11T04:02:37.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165800bed0 con 0x7f1658005070 2021-07-11T04:02:37.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f165800c090 con 0x7f1658003800 2021-07-11T04:02:37.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f1677fff700 1 --2- 172.21.2.104:0/732395235 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16780b1170 0x7f1678141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:37.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.936+0000 7f1677fff700 1 --2- 172.21.2.104:0/732395235 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1658004500 0x7f1658004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f1677fff700 1 -- 172.21.2.104:0/732395235 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1658003800 msgr2=0x7f1658003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:37.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f1677fff700 1 --2- 172.21.2.104:0/732395235 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1658003800 0x7f1658003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:37.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f1677fff700 1 -- 172.21.2.104:0/732395235 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1658005070 msgr2=0x7f165800b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:37.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f1677fff700 1 --2- 172.21.2.104:0/732395235 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1658005070 0x7f165800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:37.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f1677fff700 1 -- 172.21.2.104:0/732395235 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1658003260 con 0x7f1658004500 2021-07-11T04:02:37.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f1677fff700 1 --2- 172.21.2.104:0/732395235 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1658004500 0x7f1658004980 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7f1660000b60 tx=0x7f1660008040).ready entity=mon.0 client_cookie=96381f40a4396fc6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:37.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1660011890 con 0x7f1658004500 2021-07-11T04:02:37.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1660015070 con 0x7f1658004500 2021-07-11T04:02:37.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f16777fe700 1 -- 172.21.2.104:0/732395235 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1660011be0 con 0x7f1658004500 2021-07-11T04:02:37.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 -- 172.21.2.104:0/732395235 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1658004500 msgr2=0x7f1658004980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 --2- 172.21.2.104:0/732395235 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1658004500 0x7f1658004980 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7f1660000b60 tx=0x7f1660008040).stop 2021-07-11T04:02:37.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167d101700 1 -- 172.21.2.104:0/732395235 reap_dead start 2021-07-11T04:02:37.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 -- 172.21.2.104:0/732395235 shutdown_connections 2021-07-11T04:02:37.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 -- 172.21.2.104:0/732395235 >> 172.21.2.104:0/732395235 conn(0x7f167805f6b0 msgr2=0x7f167805fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:37.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 -- 172.21.2.104:0/732395235 shutdown_connections 2021-07-11T04:02:37.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 -- 172.21.2.104:0/732395235 wait complete. 2021-07-11T04:02:37.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 Processor -- start 2021-07-11T04:02:37.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 -- start start 2021-07-11T04:02:37.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1678140f80 0x7f1678144760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.950 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16781519c0 0x7f1678151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f167813d3f0 0x7f167813d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16780ac470 con 0x7f167813d3f0 2021-07-11T04:02:37.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678141360 con 0x7f1678140f80 2021-07-11T04:02:37.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1678143c40 con 0x7f16781519c0 2021-07-11T04:02:37.951 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167d101700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f167813d3f0 0x7f167813d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167d101700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f167813d3f0 0x7f167813d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51854/0 (socket says 172.21.2.104:51854) 2021-07-11T04:02:37.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167d101700 1 -- 172.21.2.104:0/2158374466 learned_addr learned my addr 172.21.2.104:0/2158374466 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:37.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167d101700 1 -- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16781519c0 msgr2=0x7f1678151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.952 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167d101700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16781519c0 0x7f1678151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:37.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167d101700 1 -- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1678140f80 msgr2=0x7f1678144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:37.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167d101700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1678140f80 0x7f1678144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:37.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167d101700 1 -- 172.21.2.104:0/2158374466 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1678143ee0 con 0x7f167813d3f0 2021-07-11T04:02:37.953 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167d101700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f167813d3f0 0x7f167813d870 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 rx=0x7f16700016f0 tx=0x7f1678066a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:37.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f1675ffb700 1 -- 172.21.2.104:0/2158374466 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1670004b70 con 0x7f167813d3f0 2021-07-11T04:02:37.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f1675ffb700 1 -- 172.21.2.104:0/2158374466 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1670004cd0 con 0x7f167813d3f0 2021-07-11T04:02:37.955 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f1675ffb700 1 -- 172.21.2.104:0/2158374466 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f16700072c0 con 0x7f167813d3f0 2021-07-11T04:02:37.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 -- 172.21.2.104:0/2158374466 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1678144180 con 0x7f167813d3f0 2021-07-11T04:02:37.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.940+0000 7f167eb64700 1 -- 172.21.2.104:0/2158374466 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1678152310 con 0x7f167813d3f0 2021-07-11T04:02:37.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.944+0000 7f167eb64700 1 -- 172.21.2.104:0/2158374466 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1678057350 con 0x7f167813d3f0 2021-07-11T04:02:37.956 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.944+0000 7f1675ffb700 1 -- 172.21.2.104:0/2158374466 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f167001c070 con 0x7f167813d3f0 2021-07-11T04:02:37.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.944+0000 7f1675ffb700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f165c0531a0 0x7f165c055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:37.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.944+0000 7f1675ffb700 1 -- 172.21.2.104:0/2158374466 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f167006aa40 con 0x7f167813d3f0 2021-07-11T04:02:37.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.944+0000 7f167c900700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f165c0531a0 0x7f165c055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:37.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.944+0000 7f167c900700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f165c0531a0 0x7f165c055660 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f1668001fa0 tx=0x7f16680077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:37.957 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:37.944+0000 7f1675ffb700 1 -- 172.21.2.104:0/2158374466 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1670041c70 con 0x7f167813d3f0 2021-07-11T04:02:38.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 -- 172.21.2.104:0/2158374466 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16780698e0 con 0x7f167813d3f0 2021-07-11T04:02:38.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f1675ffb700 1 -- 172.21.2.104:0/2158374466 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f167001a030 con 0x7f167813d3f0 2021-07-11T04:02:38.029 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:38.029 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":160,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:37.677517+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:02:34.620986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-07-11T04:02:35.639038+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:38.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 -- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f165c0531a0 msgr2=0x7f165c055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f165c0531a0 0x7f165c055660 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f1668001fa0 tx=0x7f16680077f0).stop 2021-07-11T04:02:38.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 -- 172.21.2.104:0/2158374466 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f167813d3f0 msgr2=0x7f167813d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f167813d3f0 0x7f167813d870 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 rx=0x7f16700016f0 tx=0x7f1678066a60).stop 2021-07-11T04:02:38.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 -- 172.21.2.104:0/2158374466 shutdown_connections 2021-07-11T04:02:38.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f165c0531a0 0x7f165c055660 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f167813d3f0 0x7f167813d870 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f16781519c0 0x7f1678151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 --2- 172.21.2.104:0/2158374466 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1678140f80 0x7f1678144760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 -- 172.21.2.104:0/2158374466 >> 172.21.2.104:0/2158374466 conn(0x7f167805f6b0 msgr2=0x7f16780b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:38.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 -- 172.21.2.104:0/2158374466 shutdown_connections 2021-07-11T04:02:38.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.028+0000 7f167eb64700 1 -- 172.21.2.104:0/2158374466 wait complete. 2021-07-11T04:02:38.036 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:02:38.092 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a90d1700 1 Processor -- start 2021-07-11T04:02:38.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a90d1700 1 -- start start 2021-07-11T04:02:38.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a90d1700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f66a40a76b0 0x7f66a4066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a90d1700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f66a40b1970 0x7f66a4066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.093 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a90d1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a4067450 con 0x7f66a40b0060 2021-07-11T04:02:38.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a90d1700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a40a7a90 con 0x7f66a40b1970 2021-07-11T04:02:38.094 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a90d1700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a4147620 con 0x7f66a40a76b0 2021-07-11T04:02:38.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a359e700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f66a40b0060 0x7f66a405f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48628/0 (socket says 172.21.2.104:48628) 2021-07-11T04:02:38.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f66a40b1970 0x7f66a4066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a359e700 1 -- 172.21.2.104:0/1987187335 learned_addr learned my addr 172.21.2.104:0/1987187335 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:38.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a259c700 1 -- 172.21.2.104:0/1987187335 >> v2:172.21.2.105:3300/0 conn(0x7f66a40a76b0 msgr2=0x7f66a4066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:38.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a259c700 1 --2- 172.21.2.104:0/1987187335 >> v2:172.21.2.105:3300/0 conn(0x7f66a40a76b0 0x7f66a4066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a259c700 1 -- 172.21.2.104:0/1987187335 >> v1:172.21.2.104:6789/0 conn(0x7f66a40b0060 legacy=0x7f66a405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a259c700 1 -- 172.21.2.104:0/1987187335 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a41478c0 con 0x7f66a40b1970 2021-07-11T04:02:38.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a259c700 1 --2- 172.21.2.104:0/1987187335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66a40b1970 0x7f66a4066f60 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f669c001490 tx=0x7f668c006df0).ready entity=mon.0 client_cookie=bb286aa7c84599bb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:38.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a1d9b700 1 -- 172.21.2.104:0/1987187335 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f668c00d570 con 0x7f66a40b1970 2021-07-11T04:02:38.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a1d9b700 1 -- 172.21.2.104:0/1987187335 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c000e90 con 0x7f66a40b1970 2021-07-11T04:02:38.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a1d9b700 1 -- 172.21.2.104:0/1987187335 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f668c00d8c0 con 0x7f66a40b1970 2021-07-11T04:02:38.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a90d1700 1 -- 172.21.2.104:0/1987187335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66a40b1970 msgr2=0x7f66a4066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.092+0000 7f66a90d1700 1 --2- 172.21.2.104:0/1987187335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66a40b1970 0x7f66a4066f60 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f669c001490 tx=0x7f668c006df0).stop 2021-07-11T04:02:38.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 -- 172.21.2.104:0/1987187335 shutdown_connections 2021-07-11T04:02:38.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 --2- 172.21.2.104:0/1987187335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66a40b1970 0x7f66a4066f60 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 --2- 172.21.2.104:0/1987187335 >> v2:172.21.2.105:3300/0 conn(0x7f66a40a76b0 0x7f66a4066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 -- 172.21.2.104:0/1987187335 >> 172.21.2.104:0/1987187335 conn(0x7f66a40b53e0 msgr2=0x7f66a40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:38.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 -- 172.21.2.104:0/1987187335 shutdown_connections 2021-07-11T04:02:38.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 -- 172.21.2.104:0/1987187335 wait complete. 2021-07-11T04:02:38.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 Processor -- start 2021-07-11T04:02:38.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 -- start start 2021-07-11T04:02:38.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66a40a76b0 0x7f66a414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66a40b0060 0x7f66a40ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66a40b1970 0x7f66a40aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a40ad910 con 0x7f66a40a76b0 2021-07-11T04:02:38.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a40abd80 con 0x7f66a40b0060 2021-07-11T04:02:38.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a40aa440 con 0x7f66a40b1970 2021-07-11T04:02:38.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a2d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66a40a76b0 0x7f66a414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a2d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66a40a76b0 0x7f66a414b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51868/0 (socket says 172.21.2.104:51868) 2021-07-11T04:02:38.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a2d9d700 1 -- 172.21.2.104:0/4275494980 learned_addr learned my addr 172.21.2.104:0/4275494980 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:38.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a259c700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66a40b0060 0x7f66a40ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a2d9d700 1 -- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66a40b1970 msgr2=0x7f66a40aac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a2d9d700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66a40b1970 0x7f66a40aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a2d9d700 1 -- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66a40b0060 msgr2=0x7f66a40ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a2d9d700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66a40b0060 0x7f66a40ae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a2d9d700 1 -- 172.21.2.104:0/4275494980 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c00b180 con 0x7f66a40a76b0 2021-07-11T04:02:38.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a259c700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66a40b0060 0x7f66a40ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:38.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a2d9d700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66a40a76b0 0x7f66a414b3b0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 rx=0x7f66a40abfe0 tx=0x7f6694006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:38.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f669bfff700 1 -- 172.21.2.104:0/4275494980 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6694013460 con 0x7f66a40a76b0 2021-07-11T04:02:38.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f669bfff700 1 -- 172.21.2.104:0/4275494980 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66940135c0 con 0x7f66a40a76b0 2021-07-11T04:02:38.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 -- 172.21.2.104:0/4275494980 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a40aee90 con 0x7f66a40a76b0 2021-07-11T04:02:38.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f669bfff700 1 -- 172.21.2.104:0/4275494980 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6694013920 con 0x7f66a40a76b0 2021-07-11T04:02:38.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 -- 172.21.2.104:0/4275494980 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a414e210 con 0x7f66a40a76b0 2021-07-11T04:02:38.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f66a90d1700 1 -- 172.21.2.104:0/4275494980 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a4057350 con 0x7f66a40a76b0 2021-07-11T04:02:38.107 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.096+0000 7f669bfff700 1 -- 172.21.2.104:0/4275494980 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f669401a060 con 0x7f66a40a76b0 2021-07-11T04:02:38.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.100+0000 7f669bfff700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6688053210 0x7f66880556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.100+0000 7f669bfff700 1 -- 172.21.2.104:0/4275494980 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f669406aec0 con 0x7f66a40a76b0 2021-07-11T04:02:38.109 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.100+0000 7f66a259c700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6688053210 0x7f66880556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.100+0000 7f66a259c700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6688053210 0x7f66880556d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f668c013000 tx=0x7f668c014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:38.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.100+0000 7f669bfff700 1 -- 172.21.2.104:0/4275494980 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6694042200 con 0x7f66a40a76b0 2021-07-11T04:02:38.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.208+0000 7f66a90d1700 1 -- 172.21.2.104:0/4275494980 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7f66a414a910 con 0x7f66a40a76b0 2021-07-11T04:02:38.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f669bfff700 1 -- 172.21.2.104:0/4275494980 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v36) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6694018070 con 0x7f66a40a76b0 2021-07-11T04:02:38.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 -- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6688053210 msgr2=0x7f66880556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6688053210 0x7f66880556d0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f668c013000 tx=0x7f668c014000).stop 2021-07-11T04:02:38.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 -- 172.21.2.104:0/4275494980 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66a40a76b0 msgr2=0x7f66a414b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66a40a76b0 0x7f66a414b3b0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 rx=0x7f66a40abfe0 tx=0x7f6694006ec0).stop 2021-07-11T04:02:38.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 -- 172.21.2.104:0/4275494980 shutdown_connections 2021-07-11T04:02:38.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f66a40b1970 0x7f66a40aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f66a40b0060 0x7f66a40ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6688053210 0x7f66880556d0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 --2- 172.21.2.104:0/4275494980 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f66a40a76b0 0x7f66a414b3b0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 -- 172.21.2.104:0/4275494980 >> 172.21.2.104:0/4275494980 conn(0x7f66a40b53e0 msgr2=0x7f66a4149930 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:38.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 -- 172.21.2.104:0/4275494980 shutdown_connections 2021-07-11T04:02:38.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.216+0000 7f66a90d1700 1 -- 172.21.2.104:0/4275494980 wait complete. 2021-07-11T04:02:38.219 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.0 2021-07-11T04:02:38.223 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.1 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2021-07-11T04:02:38.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.276+0000 7faca5469700 1 Processor -- start 2021-07-11T04:02:38.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.276+0000 7faca5469700 1 -- start start 2021-07-11T04:02:38.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faca00b1970 0x7faca005fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca00a7a90 con 0x7faca00a76b0 2021-07-11T04:02:38.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca0147620 con 0x7faca0067180 2021-07-11T04:02:38.280 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca01478c0 con 0x7faca00b1970 2021-07-11T04:02:38.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9effd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7faca00a76b0 0x7faca0066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48640/0 (socket says 172.21.2.104:48640) 2021-07-11T04:02:38.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9f7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faca00b1970 0x7faca005fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9effd700 1 -- 172.21.2.104:0/4152963500 learned_addr learned my addr 172.21.2.104:0/4152963500 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:38.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9f7fe700 1 -- 172.21.2.104:0/4152963500 >> v1:172.21.2.105:6789/0 conn(0x7faca0067180 legacy=0x7faca005f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:38.281 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9f7fe700 1 -- 172.21.2.104:0/4152963500 >> v1:172.21.2.104:6789/0 conn(0x7faca00a76b0 legacy=0x7faca0066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:38.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9f7fe700 1 -- 172.21.2.104:0/4152963500 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca0147b60 con 0x7faca00b1970 2021-07-11T04:02:38.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9f7fe700 1 --2- 172.21.2.104:0/4152963500 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faca00b1970 0x7faca005fdc0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7faca006a0e0 tx=0x7fac980091d0).ready entity=mon.0 client_cookie=60885a732db4a981 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:38.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9dffb700 1 -- 172.21.2.104:0/4152963500 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac98009ea0 con 0x7faca00b1970 2021-07-11T04:02:38.282 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9dffb700 1 -- 172.21.2.104:0/4152963500 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9800b7f0 con 0x7faca00b1970 2021-07-11T04:02:38.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9dffb700 1 -- 172.21.2.104:0/4152963500 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac9800b960 con 0x7faca00b1970 2021-07-11T04:02:38.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- 172.21.2.104:0/4152963500 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faca00b1970 msgr2=0x7faca005fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 --2- 172.21.2.104:0/4152963500 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faca00b1970 0x7faca005fdc0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7faca006a0e0 tx=0x7fac980091d0).stop 2021-07-11T04:02:38.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- 172.21.2.104:0/4152963500 shutdown_connections 2021-07-11T04:02:38.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 --2- 172.21.2.104:0/4152963500 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faca00b1970 0x7faca005fdc0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- 172.21.2.104:0/4152963500 >> 172.21.2.104:0/4152963500 conn(0x7faca00b53e0 msgr2=0x7faca00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:38.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- 172.21.2.104:0/4152963500 shutdown_connections 2021-07-11T04:02:38.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- 172.21.2.104:0/4152963500 wait complete. 2021-07-11T04:02:38.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 Processor -- start 2021-07-11T04:02:38.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- start start 2021-07-11T04:02:38.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faca0067180 0x7faca00a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faca00a76b0 0x7faca00ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faca00b1970 0x7faca01491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca00aa020 con 0x7faca00a76b0 2021-07-11T04:02:38.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca00ad620 con 0x7faca0067180 2021-07-11T04:02:38.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca00ad840 con 0x7faca00b1970 2021-07-11T04:02:38.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faca00a76b0 0x7faca00ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9e7fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faca00a76b0 0x7faca00ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51882/0 (socket says 172.21.2.104:51882) 2021-07-11T04:02:38.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9e7fc700 1 -- 172.21.2.104:0/317039547 learned_addr learned my addr 172.21.2.104:0/317039547 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:38.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9e7fc700 1 -- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faca00b1970 msgr2=0x7faca01491f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9e7fc700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faca00b1970 0x7faca01491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9e7fc700 1 -- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faca0067180 msgr2=0x7faca00a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9e7fc700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faca0067180 0x7faca00a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9e7fc700 1 -- 172.21.2.104:0/317039547 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac98009470 con 0x7faca00a76b0 2021-07-11T04:02:38.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac9e7fc700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faca00a76b0 0x7faca00ad130 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 rx=0x7faca00addb0 tx=0x7fac88006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:38.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac97fff700 1 -- 172.21.2.104:0/317039547 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac88013460 con 0x7faca00a76b0 2021-07-11T04:02:38.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac97fff700 1 -- 172.21.2.104:0/317039547 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac880135c0 con 0x7faca00a76b0 2021-07-11T04:02:38.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- 172.21.2.104:0/317039547 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca00adae0 con 0x7faca00a76b0 2021-07-11T04:02:38.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7fac97fff700 1 -- 172.21.2.104:0/317039547 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fac88013920 con 0x7faca00a76b0 2021-07-11T04:02:38.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.280+0000 7faca5469700 1 -- 172.21.2.104:0/317039547 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca01496e0 con 0x7faca00a76b0 2021-07-11T04:02:38.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.284+0000 7faca5469700 1 -- 172.21.2.104:0/317039547 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca0057350 con 0x7faca00a76b0 2021-07-11T04:02:38.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.284+0000 7fac97fff700 1 -- 172.21.2.104:0/317039547 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fac8800f070 con 0x7faca00a76b0 2021-07-11T04:02:38.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.284+0000 7fac97fff700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac84053160 0x7fac84055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.284+0000 7fac97fff700 1 -- 172.21.2.104:0/317039547 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fac8806af00 con 0x7faca00a76b0 2021-07-11T04:02:38.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.284+0000 7fac9effd700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac84053160 0x7fac84055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.284+0000 7fac9effd700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac84053160 0x7fac84055620 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fac90004520 tx=0x7fac90007900).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:38.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.284+0000 7fac97fff700 1 -- 172.21.2.104:0/317039547 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fac88042240 con 0x7faca00a76b0 2021-07-11T04:02:38.395 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.396+0000 7faca5469700 1 -- 172.21.2.104:0/317039547 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) v1 -- 0x7faca00698e0 con 0x7faca00a76b0 2021-07-11T04:02:38.398 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7fac97fff700 1 -- 172.21.2.104:0/317039547 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.1", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.1 v37) v1 ==== 184+0+0 (secure 0 0 0) 0x7fac88045860 con 0x7faca00a76b0 2021-07-11T04:02:38.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 -- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac84053160 msgr2=0x7fac84055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac84053160 0x7fac84055620 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fac90004520 tx=0x7fac90007900).stop 2021-07-11T04:02:38.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 -- 172.21.2.104:0/317039547 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faca00a76b0 msgr2=0x7faca00ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faca00a76b0 0x7faca00ad130 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 rx=0x7faca00addb0 tx=0x7fac88006de0).stop 2021-07-11T04:02:38.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 -- 172.21.2.104:0/317039547 shutdown_connections 2021-07-11T04:02:38.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faca00b1970 0x7faca01491f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fac84053160 0x7fac84055620 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faca00a76b0 0x7faca00ad130 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 --2- 172.21.2.104:0/317039547 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faca0067180 0x7faca00a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 -- 172.21.2.104:0/317039547 >> 172.21.2.104:0/317039547 conn(0x7faca00b53e0 msgr2=0x7faca01447b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:38.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 -- 172.21.2.104:0/317039547 shutdown_connections 2021-07-11T04:02:38.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.400+0000 7faca5469700 1 -- 172.21.2.104:0/317039547 wait complete. 2021-07-11T04:02:38.404 INFO:teuthology.orchestra.run.gibba004.stderr:updated caps for client.1 2021-07-11T04:02:38.406 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:38.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 Processor -- start 2021-07-11T04:02:38.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 -- start start 2021-07-11T04:02:38.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33980a76b0 0x7f3398066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f33980b1970 0x7f3398066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f33980b0060 0x7f339805f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3398067450 con 0x7f33980b1970 2021-07-11T04:02:38.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33980a7a90 con 0x7f33980b0060 2021-07-11T04:02:38.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3398147620 con 0x7f33980a76b0 2021-07-11T04:02:38.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f3397fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f33980b1970 0x7f3398066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f3397fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f33980b1970 0x7f3398066f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51890/0 (socket says 172.21.2.104:51890) 2021-07-11T04:02:38.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f3397fff700 1 -- 172.21.2.104:0/202435657 learned_addr learned my addr 172.21.2.104:0/202435657 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:38.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f3397fff700 1 -- 172.21.2.104:0/202435657 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33980a76b0 msgr2=0x7f3398066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:38.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f3397fff700 1 --2- 172.21.2.104:0/202435657 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33980a76b0 0x7f3398066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f3397fff700 1 -- 172.21.2.104:0/202435657 >> v2:172.21.2.105:3300/0 conn(0x7f33980b0060 msgr2=0x7f339805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:38.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339d451700 1 --2- 172.21.2.104:0/202435657 >> v2:172.21.2.105:3300/0 conn(0x7f33980b0060 0x7f339805f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f3397fff700 1 --2- 172.21.2.104:0/202435657 >> v2:172.21.2.105:3300/0 conn(0x7f33980b0060 0x7f339805f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f3397fff700 1 -- 172.21.2.104:0/202435657 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33981478c0 con 0x7f33980b1970 2021-07-11T04:02:38.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339d451700 1 --2- 172.21.2.104:0/202435657 >> v2:172.21.2.105:3300/0 conn(0x7f33980b0060 0x7f339805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:38.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f3397fff700 1 --2- 172.21.2.104:0/202435657 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33980b1970 0x7f3398066f60 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7f3380002f70 tx=0x7f3380006eb0).ready entity=mon.0 client_cookie=eecdbc1f4ec046b2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:38.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f33977fe700 1 -- 172.21.2.104:0/202435657 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338000d570 con 0x7f33980b1970 2021-07-11T04:02:38.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f33977fe700 1 -- 172.21.2.104:0/202435657 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3380000e90 con 0x7f33980b1970 2021-07-11T04:02:38.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f33977fe700 1 -- 172.21.2.104:0/202435657 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f338000d900 con 0x7f33980b1970 2021-07-11T04:02:38.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 -- 172.21.2.104:0/202435657 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33980b1970 msgr2=0x7f3398066f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 --2- 172.21.2.104:0/202435657 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33980b1970 0x7f3398066f60 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7f3380002f70 tx=0x7f3380006eb0).stop 2021-07-11T04:02:38.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 -- 172.21.2.104:0/202435657 shutdown_connections 2021-07-11T04:02:38.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 --2- 172.21.2.104:0/202435657 >> v2:172.21.2.105:3300/0 conn(0x7f33980b0060 0x7f339805f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 --2- 172.21.2.104:0/202435657 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33980b1970 0x7f3398066f60 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 --2- 172.21.2.104:0/202435657 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33980a76b0 0x7f3398066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 -- 172.21.2.104:0/202435657 >> 172.21.2.104:0/202435657 conn(0x7f33980b53e0 msgr2=0x7f33980b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:38.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 -- 172.21.2.104:0/202435657 shutdown_connections 2021-07-11T04:02:38.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.460+0000 7f339eeb4700 1 -- 172.21.2.104:0/202435657 wait complete. 2021-07-11T04:02:38.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339eeb4700 1 Processor -- start 2021-07-11T04:02:38.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339eeb4700 1 -- start start 2021-07-11T04:02:38.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339eeb4700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33980a76b0 0x7f33980aa630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339eeb4700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33980b0060 0x7f33980aab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339eeb4700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33980b1970 0x7f3398156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339eeb4700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33980a99e0 con 0x7f33980b0060 2021-07-11T04:02:38.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339eeb4700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339814c350 con 0x7f33980a76b0 2021-07-11T04:02:38.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339eeb4700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3398141770 con 0x7f33980b1970 2021-07-11T04:02:38.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3397fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33980b0060 0x7f33980aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3397fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33980b0060 0x7f33980aab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51894/0 (socket says 172.21.2.104:51894) 2021-07-11T04:02:38.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3397fff700 1 -- 172.21.2.104:0/3359592722 learned_addr learned my addr 172.21.2.104:0/3359592722 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:38.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339cc50700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33980a76b0 0x7f33980aa630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3397fff700 1 -- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33980b1970 msgr2=0x7f3398156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3397fff700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33980b1970 0x7f3398156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3397fff700 1 -- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33980a76b0 msgr2=0x7f33980aa630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3397fff700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33980a76b0 0x7f33980aa630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3397fff700 1 -- 172.21.2.104:0/3359592722 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338000b1e0 con 0x7f33980b0060 2021-07-11T04:02:38.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3397fff700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33980b0060 0x7f33980aab20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 rx=0x7f33980aa3e0 tx=0x7f3380015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:38.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3395ffb700 1 -- 172.21.2.104:0/3359592722 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3380021070 con 0x7f33980b0060 2021-07-11T04:02:38.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3395ffb700 1 -- 172.21.2.104:0/3359592722 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3380000e90 con 0x7f33980b0060 2021-07-11T04:02:38.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339eeb4700 1 -- 172.21.2.104:0/3359592722 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339814c590 con 0x7f33980b0060 2021-07-11T04:02:38.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3395ffb700 1 -- 172.21.2.104:0/3359592722 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3380011e90 con 0x7f33980b0060 2021-07-11T04:02:38.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339eeb4700 1 -- 172.21.2.104:0/3359592722 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3398157330 con 0x7f33980b0060 2021-07-11T04:02:38.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339eeb4700 1 -- 172.21.2.104:0/3359592722 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339814e9b0 con 0x7f33980b0060 2021-07-11T04:02:38.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3395ffb700 1 -- 172.21.2.104:0/3359592722 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f338000f070 con 0x7f33980b0060 2021-07-11T04:02:38.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3395ffb700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f337c053160 0x7f337c055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:38.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f3395ffb700 1 -- 172.21.2.104:0/3359592722 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f338006c9b0 con 0x7f33980b0060 2021-07-11T04:02:38.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.464+0000 7f339cc50700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f337c053160 0x7f337c055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:38.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.468+0000 7f339cc50700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f337c053160 0x7f337c055620 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f338800a000 tx=0x7f3388008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:38.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.468+0000 7f3395ffb700 1 -- 172.21.2.104:0/3359592722 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3380043c90 con 0x7f33980b0060 2021-07-11T04:02:38.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.572+0000 7f339eeb4700 1 -- 172.21.2.104:0/3359592722 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3398057350 con 0x7f33980b0060 2021-07-11T04:02:38.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f3395ffb700 1 -- 172.21.2.104:0/3359592722 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f33800472b0 con 0x7f33980b0060 2021-07-11T04:02:38.575 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:38.575 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":80,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5132,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/770706140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":770706140},{"type":"v1","addr":"172.21.2.105:6835","nonce":770706140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":79},{"gid":8718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/488029782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":488029782},{"type":"v1","addr":"172.21.2.104:6833","nonce":488029782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:36.673961+0000","modified":"2021-07-11T04:02:37.679104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8721":{"gid":8721,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/2638676458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2638676458},{"type":"v1","addr":"172.21.2.104:6835","nonce":2638676458}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-07-11T04:02:38.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 -- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f337c053160 msgr2=0x7f337c055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f337c053160 0x7f337c055620 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f338800a000 tx=0x7f3388008040).stop 2021-07-11T04:02:38.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 -- 172.21.2.104:0/3359592722 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33980b0060 msgr2=0x7f33980aab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:38.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33980b0060 0x7f33980aab20 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 rx=0x7f33980aa3e0 tx=0x7f3380015000).stop 2021-07-11T04:02:38.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 -- 172.21.2.104:0/3359592722 shutdown_connections 2021-07-11T04:02:38.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f33980b1970 0x7f3398156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:38.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f337c053160 0x7f337c055620 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f33980b0060 0x7f33980aab20 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 --2- 172.21.2.104:0/3359592722 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f33980a76b0 0x7f33980aa630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:38.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 -- 172.21.2.104:0/3359592722 >> 172.21.2.104:0/3359592722 conn(0x7f33980b53e0 msgr2=0x7f3398149b10 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:38.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 -- 172.21.2.104:0/3359592722 shutdown_connections 2021-07-11T04:02:38.579 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:38.576+0000 7f339eeb4700 1 -- 172.21.2.104:0/3359592722 wait complete. 2021-07-11T04:02:38.579 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 80 2021-07-11T04:02:38.887 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:02:36.673961+0000', 'modified': '2021-07-11T04:02:37.679104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8721}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8721': {'gid': 8721, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/2638676458', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 2638676458}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 2638676458}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:02:38.888 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:02:38.888 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:02:38.952 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":8721,"want_state":"up:active","state":"up:active","rank_uptime":2.2733369840000002,"mdsmap_epoch":80,"osdmap_epoch":161,"osdmap_epoch_barrier":160,"uptime":6.4418320969999998} 2021-07-11T04:02:38.953 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 8721, "mdsmap_epoch": 80, "osdmap_epoch": 161, "osdmap_epoch_barrier": 160, "rank_uptime": 2.273336984, "state": "up:active", "uptime": 6.441832097, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:02:38.954 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:39.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b86195700 1 Processor -- start 2021-07-11T04:02:39.004 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b86195700 1 -- start start 2021-07-11T04:02:39.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b86195700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f4b801481b0 0x7f4b80061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b86195700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4b800b2780 0x7f4b8014ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b86195700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b8014aa40 con 0x7f4b800b31e0 2021-07-11T04:02:39.007 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b86195700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b801451f0 con 0x7f4b800b2780 2021-07-11T04:02:39.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b86195700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b800b7650 con 0x7f4b801481b0 2021-07-11T04:02:39.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b7ffff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4b800b2780 0x7f4b8014ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b7ffff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f4b800b2780 0x7f4b8014ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51902/0 (socket says 172.21.2.104:51902) 2021-07-11T04:02:39.008 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b7ffff700 1 -- 172.21.2.104:0/747462189 learned_addr learned my addr 172.21.2.104:0/747462189 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b7ffff700 1 -- 172.21.2.104:0/747462189 >> v2:172.21.2.105:3300/0 conn(0x7f4b801481b0 msgr2=0x7f4b80061190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:39.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b7ffff700 1 --2- 172.21.2.104:0/747462189 >> v2:172.21.2.105:3300/0 conn(0x7f4b801481b0 0x7f4b80061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.009 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b7ffff700 1 -- 172.21.2.104:0/747462189 >> v1:172.21.2.105:6789/0 conn(0x7f4b800b31e0 legacy=0x7f4b80147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:39.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b7ffff700 1 -- 172.21.2.104:0/747462189 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b800b78f0 con 0x7f4b800b2780 2021-07-11T04:02:39.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b7ffff700 1 --2- 172.21.2.104:0/747462189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b800b2780 0x7f4b8014ae70 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7f4b80062900 tx=0x7f4b78009290).ready entity=mon.0 client_cookie=6dceb43203f76555 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b7e7fc700 1 -- 172.21.2.104:0/747462189 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b7800b830 con 0x7f4b800b2780 2021-07-11T04:02:39.010 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b7e7fc700 1 -- 172.21.2.104:0/747462189 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7800b990 con 0x7f4b800b2780 2021-07-11T04:02:39.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b7e7fc700 1 -- 172.21.2.104:0/747462189 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b7800bb40 con 0x7f4b800b2780 2021-07-11T04:02:39.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b86195700 1 -- 172.21.2.104:0/747462189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b800b2780 msgr2=0x7f4b8014ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.004+0000 7f4b86195700 1 --2- 172.21.2.104:0/747462189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b800b2780 0x7f4b8014ae70 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7f4b80062900 tx=0x7f4b78009290).stop 2021-07-11T04:02:39.011 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 -- 172.21.2.104:0/747462189 shutdown_connections 2021-07-11T04:02:39.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 --2- 172.21.2.104:0/747462189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b800b2780 0x7f4b8014ae70 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 --2- 172.21.2.104:0/747462189 >> v2:172.21.2.105:3300/0 conn(0x7f4b801481b0 0x7f4b80061190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 -- 172.21.2.104:0/747462189 >> 172.21.2.104:0/747462189 conn(0x7f4b8005f6b0 msgr2=0x7f4b8005fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 -- 172.21.2.104:0/747462189 shutdown_connections 2021-07-11T04:02:39.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 -- 172.21.2.104:0/747462189 wait complete. 2021-07-11T04:02:39.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 Processor -- start 2021-07-11T04:02:39.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 -- start start 2021-07-11T04:02:39.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b800b2780 0x7f4b800b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b800b31e0 0x7f4b801440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b801481b0 0x7f4b801445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b800abc50 con 0x7f4b801481b0 2021-07-11T04:02:39.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b800ab470 con 0x7f4b800b2780 2021-07-11T04:02:39.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b800ab710 con 0x7f4b800b31e0 2021-07-11T04:02:39.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b801481b0 0x7f4b801445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b801481b0 0x7f4b801445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51910/0 (socket says 172.21.2.104:51910) 2021-07-11T04:02:39.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7ffff700 1 -- 172.21.2.104:0/2845588999 learned_addr learned my addr 172.21.2.104:0/2845588999 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7ffff700 1 -- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b800b31e0 msgr2=0x7f4b801440f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7ffff700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b800b31e0 0x7f4b801440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7ffff700 1 -- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b800b2780 msgr2=0x7f4b800b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7ffff700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b800b2780 0x7f4b800b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7ffff700 1 -- 172.21.2.104:0/2845588999 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b78009530 con 0x7f4b801481b0 2021-07-11T04:02:39.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7ffff700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b801481b0 0x7f4b801445e0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7f4b80062900 tx=0x7f4b78016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7cff9700 1 -- 172.21.2.104:0/2845588999 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b78011cc0 con 0x7f4b801481b0 2021-07-11T04:02:39.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7cff9700 1 -- 172.21.2.104:0/2845588999 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b78011e20 con 0x7f4b801481b0 2021-07-11T04:02:39.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 -- 172.21.2.104:0/2845588999 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b800ab9b0 con 0x7f4b801481b0 2021-07-11T04:02:39.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 -- 172.21.2.104:0/2845588999 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b800b6f50 con 0x7f4b801481b0 2021-07-11T04:02:39.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7cff9700 1 -- 172.21.2.104:0/2845588999 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4b78020620 con 0x7f4b801481b0 2021-07-11T04:02:39.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b86195700 1 -- 172.21.2.104:0/2845588999 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b80156770 con 0x7f4b801481b0 2021-07-11T04:02:39.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7cff9700 1 -- 172.21.2.104:0/2845588999 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4b78026020 con 0x7f4b801481b0 2021-07-11T04:02:39.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7cff9700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b640532d0 0x7f4b64055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.008+0000 7f4b7cff9700 1 -- 172.21.2.104:0/2845588999 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f4b7806c3a0 con 0x7f4b801481b0 2021-07-11T04:02:39.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.012+0000 7f4b7f7fe700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b640532d0 0x7f4b64055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.012+0000 7f4b7f7fe700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b640532d0 0x7f4b64055790 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f4b68000f70 tx=0x7f4b680077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.012+0000 7f4b7cff9700 1 -- 172.21.2.104:0/2845588999 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4b80156770 con 0x7f4b801481b0 2021-07-11T04:02:39.095 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 -- 172.21.2.104:0/2845588999 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b80156770 con 0x7f4b801481b0 2021-07-11T04:02:39.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b7cff9700 1 -- 172.21.2.104:0/2845588999 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f4b80156770 con 0x7f4b801481b0 2021-07-11T04:02:39.096 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:39.096 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":161,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:38.690916+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:02:34.620986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-07-11T04:02:35.639038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:39.097 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 -- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b640532d0 msgr2=0x7f4b64055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b640532d0 0x7f4b64055790 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f4b68000f70 tx=0x7f4b680077f0).stop 2021-07-11T04:02:39.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 -- 172.21.2.104:0/2845588999 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b801481b0 msgr2=0x7f4b801445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b801481b0 0x7f4b801445e0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7f4b80062900 tx=0x7f4b78016000).stop 2021-07-11T04:02:39.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 -- 172.21.2.104:0/2845588999 shutdown_connections 2021-07-11T04:02:39.098 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4b640532d0 0x7f4b64055790 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4b801481b0 0x7f4b801445e0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4b800b31e0 0x7f4b801440f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 --2- 172.21.2.104:0/2845588999 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4b800b2780 0x7f4b800b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.099 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 -- 172.21.2.104:0/2845588999 >> 172.21.2.104:0/2845588999 conn(0x7f4b8005f6b0 msgr2=0x7f4b80066aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 -- 172.21.2.104:0/2845588999 shutdown_connections 2021-07-11T04:02:39.100 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.096+0000 7f4b86195700 1 -- 172.21.2.104:0/2845588999 wait complete. 2021-07-11T04:02:39.104 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:02:34.620986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:02:35.639038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:02:39.105 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:02:39.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.156+0000 7f026da8c700 1 Processor -- start 2021-07-11T04:02:39.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.156+0000 7f026da8c700 1 -- start start 2021-07-11T04:02:39.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f02680ad040 0x7f02680b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f026805d760 0x7f026813f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02680b5f40 con 0x7f02680afaf0 2021-07-11T04:02:39.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02680ad420 con 0x7f02680ad040 2021-07-11T04:02:39.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02680aeb60 con 0x7f026805d760 2021-07-11T04:02:39.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f02677fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f02680ad040 0x7f02680b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f0266ffd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f026805d760 0x7f026813f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f02677fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f02680ad040 0x7f02680b5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:59474/0 (socket says 172.21.2.104:59474) 2021-07-11T04:02:39.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f0266ffd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f026805d760 0x7f026813f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:55082/0 (socket says 172.21.2.104:55082) 2021-07-11T04:02:39.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f02677fe700 1 -- 172.21.2.104:0/2959467836 learned_addr learned my addr 172.21.2.104:0/2959467836 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f0267fff700 1 --1- 172.21.2.104:0/2959467836 >> v1:172.21.2.105:6789/0 conn(0x7f02680afaf0 0x7f026813f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:48054/0 (socket says 172.21.2.104:48054) 2021-07-11T04:02:39.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f02677fe700 1 -- 172.21.2.104:0/2959467836 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f026805d760 msgr2=0x7f026813f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:39.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f02677fe700 1 --2- 172.21.2.104:0/2959467836 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f026805d760 0x7f026813f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f02677fe700 1 -- 172.21.2.104:0/2959467836 >> v1:172.21.2.105:6789/0 conn(0x7f02680afaf0 legacy=0x7f026813f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f02677fe700 1 -- 172.21.2.104:0/2959467836 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02680aee00 con 0x7f02680ad040 2021-07-11T04:02:39.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f02677fe700 1 --2- 172.21.2.104:0/2959467836 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02680ad040 0x7f02680b5a50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f025c002f70 tx=0x7f025c006eb0).ready entity=mon.1 client_cookie=f2cb0d61168e4c38 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f02667fc700 1 -- 172.21.2.104:0/2959467836 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f025c00d570 con 0x7f02680ad040 2021-07-11T04:02:39.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f02667fc700 1 -- 172.21.2.104:0/2959467836 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c000e90 con 0x7f02680ad040 2021-07-11T04:02:39.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f02667fc700 1 -- 172.21.2.104:0/2959467836 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f025c00d900 con 0x7f02680ad040 2021-07-11T04:02:39.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 -- 172.21.2.104:0/2959467836 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02680ad040 msgr2=0x7f02680b5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 --2- 172.21.2.104:0/2959467836 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02680ad040 0x7f02680b5a50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f025c002f70 tx=0x7f025c006eb0).stop 2021-07-11T04:02:39.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 -- 172.21.2.104:0/2959467836 shutdown_connections 2021-07-11T04:02:39.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 --2- 172.21.2.104:0/2959467836 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f026805d760 0x7f026813f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 --2- 172.21.2.104:0/2959467836 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02680ad040 0x7f02680b5a50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 -- 172.21.2.104:0/2959467836 >> 172.21.2.104:0/2959467836 conn(0x7f026805c490 msgr2=0x7f026805c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 -- 172.21.2.104:0/2959467836 shutdown_connections 2021-07-11T04:02:39.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.160+0000 7f026da8c700 1 -- 172.21.2.104:0/2959467836 wait complete. 2021-07-11T04:02:39.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f026da8c700 1 Processor -- start 2021-07-11T04:02:39.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f026da8c700 1 -- start start 2021-07-11T04:02:39.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f026da8c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f026805d760 0x7f0268066950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.170 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f026da8c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02680ad040 0x7f026813f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f026da8c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02680afaf0 0x7f026813c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f026da8c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0268067150 con 0x7f026805d760 2021-07-11T04:02:39.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f026da8c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0268131770 con 0x7f02680ad040 2021-07-11T04:02:39.171 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f026da8c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0268066e40 con 0x7f02680afaf0 2021-07-11T04:02:39.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f02677fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f026805d760 0x7f0268066950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f02677fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f026805d760 0x7f0268066950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51922/0 (socket says 172.21.2.104:51922) 2021-07-11T04:02:39.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f02677fe700 1 -- 172.21.2.104:0/3081551450 learned_addr learned my addr 172.21.2.104:0/3081551450 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f0267fff700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02680afaf0 0x7f026813c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.172 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f0266ffd700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02680ad040 0x7f026813f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f02677fe700 1 -- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02680afaf0 msgr2=0x7f026813c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f02677fe700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02680afaf0 0x7f026813c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f02677fe700 1 -- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02680ad040 msgr2=0x7f026813f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.173 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f02677fe700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02680ad040 0x7f026813f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f02677fe700 1 -- 172.21.2.104:0/3081551450 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c00b1e0 con 0x7f026805d760 2021-07-11T04:02:39.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f02677fe700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f026805d760 0x7f0268066950 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7f0268067290 tx=0x7f025c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f0264ff9700 1 -- 172.21.2.104:0/3081551450 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f025c021070 con 0x7f026805d760 2021-07-11T04:02:39.174 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f0264ff9700 1 -- 172.21.2.104:0/3081551450 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c000e90 con 0x7f026805d760 2021-07-11T04:02:39.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f0264ff9700 1 -- 172.21.2.104:0/3081551450 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f025c011880 con 0x7f026805d760 2021-07-11T04:02:39.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f026da8c700 1 -- 172.21.2.104:0/3081551450 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026813f9b0 con 0x7f026805d760 2021-07-11T04:02:39.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f026da8c700 1 -- 172.21.2.104:0/3081551450 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026813f540 con 0x7f026805d760 2021-07-11T04:02:39.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f026da8c700 1 -- 172.21.2.104:0/3081551450 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026813f810 con 0x7f026805d760 2021-07-11T04:02:39.175 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f0264ff9700 1 -- 172.21.2.104:0/3081551450 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f025c00f070 con 0x7f026805d760 2021-07-11T04:02:39.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f0264ff9700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f02480531a0 0x7f0248055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f0264ff9700 1 -- 172.21.2.104:0/3081551450 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f025c06c770 con 0x7f026805d760 2021-07-11T04:02:39.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f0266ffd700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f02480531a0 0x7f0248055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.176 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.164+0000 7f0266ffd700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f02480531a0 0x7f0248055660 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f025800b000 tx=0x7f0258008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.177 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.168+0000 7f0264ff9700 1 -- 172.21.2.104:0/3081551450 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f026813f810 con 0x7f026805d760 2021-07-11T04:02:39.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 -- 172.21.2.104:0/3081551450 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f026813f810 con 0x7f026805d760 2021-07-11T04:02:39.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f0264ff9700 1 -- 172.21.2.104:0/3081551450 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v161) v1 ==== 110+0+10 (secure 0 0 0) 0x7f026813f810 con 0x7f026805d760 2021-07-11T04:02:39.251 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:02:39.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 -- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f02480531a0 msgr2=0x7f0248055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f02480531a0 0x7f0248055660 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f025800b000 tx=0x7f0258008040).stop 2021-07-11T04:02:39.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 -- 172.21.2.104:0/3081551450 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f026805d760 msgr2=0x7f0268066950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f026805d760 0x7f0268066950 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7f0268067290 tx=0x7f025c015000).stop 2021-07-11T04:02:39.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 -- 172.21.2.104:0/3081551450 shutdown_connections 2021-07-11T04:02:39.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f02680afaf0 0x7f026813c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f02680ad040 0x7f026813f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f02480531a0 0x7f0248055660 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.254 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 --2- 172.21.2.104:0/3081551450 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f026805d760 0x7f0268066950 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 -- 172.21.2.104:0/3081551450 >> 172.21.2.104:0/3081551450 conn(0x7f026805c490 msgr2=0x7f026813a840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 -- 172.21.2.104:0/3081551450 shutdown_connections 2021-07-11T04:02:39.255 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.252+0000 7f026da8c700 1 -- 172.21.2.104:0/3081551450 wait complete. 2021-07-11T04:02:39.259 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:02:39.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f75a5cc7700 1 Processor -- start 2021-07-11T04:02:39.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f75a5cc7700 1 -- start start 2021-07-11T04:02:39.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f75a5cc7700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f75a00b1ec0 0x7f75a00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f75a5cc7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75a00affb0 0x7f75a00b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f75a5cc7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f75a00a70c0 0x7f75a0141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f75a5cc7700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a00b0420 con 0x7f75a00a70c0 2021-07-11T04:02:39.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f75a5cc7700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75a00b2790 con 0x7f75a00b1ec0 2021-07-11T04:02:39.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f75a5cc7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00b0650 con 0x7f75a00affb0 2021-07-11T04:02:39.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759ffff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f75a00a70c0 0x7f75a0141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759ffff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f75a00a70c0 0x7f75a0141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51928/0 (socket says 172.21.2.104:51928) 2021-07-11T04:02:39.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759ffff700 1 -- 172.21.2.104:0/3562716024 learned_addr learned my addr 172.21.2.104:0/3562716024 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759ffff700 1 -- 172.21.2.104:0/3562716024 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75a00affb0 msgr2=0x7f75a00b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:39.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759ffff700 1 --2- 172.21.2.104:0/3562716024 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75a00affb0 0x7f75a00b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759ffff700 1 -- 172.21.2.104:0/3562716024 >> v2:172.21.2.105:3300/0 conn(0x7f75a00b1ec0 msgr2=0x7f75a00b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:39.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759ffff700 1 --2- 172.21.2.104:0/3562716024 >> v2:172.21.2.105:3300/0 conn(0x7f75a00b1ec0 0x7f75a00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759ffff700 1 -- 172.21.2.104:0/3562716024 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a00b08f0 con 0x7f75a00a70c0 2021-07-11T04:02:39.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759ffff700 1 --2- 172.21.2.104:0/3562716024 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a00a70c0 0x7f75a0141250 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7f75980032f0 tx=0x7f7598009290).ready entity=mon.0 client_cookie=1ad197f30157a267 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759e7fc700 1 -- 172.21.2.104:0/3562716024 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f759800b870 con 0x7f75a00a70c0 2021-07-11T04:02:39.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759e7fc700 1 -- 172.21.2.104:0/3562716024 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759800b9d0 con 0x7f75a00a70c0 2021-07-11T04:02:39.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f759e7fc700 1 -- 172.21.2.104:0/3562716024 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f759800bb40 con 0x7f75a00a70c0 2021-07-11T04:02:39.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f75a5cc7700 1 -- 172.21.2.104:0/3562716024 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a00a70c0 msgr2=0x7f75a0141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.312+0000 7f75a5cc7700 1 --2- 172.21.2.104:0/3562716024 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a00a70c0 0x7f75a0141250 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7f75980032f0 tx=0x7f7598009290).stop 2021-07-11T04:02:39.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 -- 172.21.2.104:0/3562716024 shutdown_connections 2021-07-11T04:02:39.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 --2- 172.21.2.104:0/3562716024 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a00a70c0 0x7f75a0141250 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 --2- 172.21.2.104:0/3562716024 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75a00affb0 0x7f75a00b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 --2- 172.21.2.104:0/3562716024 >> v2:172.21.2.105:3300/0 conn(0x7f75a00b1ec0 0x7f75a00b12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 -- 172.21.2.104:0/3562716024 >> 172.21.2.104:0/3562716024 conn(0x7f75a005f6b0 msgr2=0x7f75a005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 -- 172.21.2.104:0/3562716024 shutdown_connections 2021-07-11T04:02:39.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 -- 172.21.2.104:0/3562716024 wait complete. 2021-07-11T04:02:39.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 Processor -- start 2021-07-11T04:02:39.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 -- start start 2021-07-11T04:02:39.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75a00a70c0 0x7f75a0156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75a00affb0 0x7f75a0157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a00b1ec0 0x7f75a015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00ac340 con 0x7f75a00b1ec0 2021-07-11T04:02:39.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00ad420 con 0x7f75a00a70c0 2021-07-11T04:02:39.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a00ad6c0 con 0x7f75a00affb0 2021-07-11T04:02:39.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a00b1ec0 0x7f75a015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759ffff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a00b1ec0 0x7f75a015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51932/0 (socket says 172.21.2.104:51932) 2021-07-11T04:02:39.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759ffff700 1 -- 172.21.2.104:0/341632772 learned_addr learned my addr 172.21.2.104:0/341632772 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759f7fe700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75a00a70c0 0x7f75a0156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759effd700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75a00affb0 0x7f75a0157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759ffff700 1 -- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75a00affb0 msgr2=0x7f75a0157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759ffff700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75a00affb0 0x7f75a0157420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759ffff700 1 -- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75a00a70c0 msgr2=0x7f75a0156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759ffff700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75a00a70c0 0x7f75a0156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759ffff700 1 -- 172.21.2.104:0/341632772 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7598009530 con 0x7f75a00b1ec0 2021-07-11T04:02:39.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759ffff700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a00b1ec0 0x7f75a015a1f0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7f75980032f0 tx=0x7f7598016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759cff9700 1 -- 172.21.2.104:0/341632772 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7598011c80 con 0x7f75a00b1ec0 2021-07-11T04:02:39.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759cff9700 1 -- 172.21.2.104:0/341632772 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598011de0 con 0x7f75a00b1ec0 2021-07-11T04:02:39.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 -- 172.21.2.104:0/341632772 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75a00ad960 con 0x7f75a00b1ec0 2021-07-11T04:02:39.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 -- 172.21.2.104:0/341632772 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75a0140ba0 con 0x7f75a00b1ec0 2021-07-11T04:02:39.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759cff9700 1 -- 172.21.2.104:0/341632772 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7598020620 con 0x7f75a00b1ec0 2021-07-11T04:02:39.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f75a5cc7700 1 -- 172.21.2.104:0/341632772 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75a0057350 con 0x7f75a00b1ec0 2021-07-11T04:02:39.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759cff9700 1 -- 172.21.2.104:0/341632772 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f759800f070 con 0x7f75a00b1ec0 2021-07-11T04:02:39.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.316+0000 7f759cff9700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7584053300 0x7f75840557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.320+0000 7f759cff9700 1 -- 172.21.2.104:0/341632772 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f759806c5e0 con 0x7f75a00b1ec0 2021-07-11T04:02:39.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.320+0000 7f759f7fe700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7584053300 0x7f75840557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.320+0000 7f759f7fe700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7584053300 0x7f75840557c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f758800a000 tx=0x7f7588008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.320+0000 7f759cff9700 1 -- 172.21.2.104:0/341632772 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7598043920 con 0x7f75a00b1ec0 2021-07-11T04:02:39.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 -- 172.21.2.104:0/341632772 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f75a00ac760 con 0x7f75a00b1ec0 2021-07-11T04:02:39.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f759cff9700 1 -- 172.21.2.104:0/341632772 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v161) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7598015020 con 0x7f75a00b1ec0 2021-07-11T04:02:39.404 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:39.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 -- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7584053300 msgr2=0x7f75840557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7584053300 0x7f75840557c0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f758800a000 tx=0x7f7588008040).stop 2021-07-11T04:02:39.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 -- 172.21.2.104:0/341632772 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a00b1ec0 msgr2=0x7f75a015a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a00b1ec0 0x7f75a015a1f0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 rx=0x7f75980032f0 tx=0x7f7598016000).stop 2021-07-11T04:02:39.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 -- 172.21.2.104:0/341632772 shutdown_connections 2021-07-11T04:02:39.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7584053300 0x7f75840557c0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f75a00b1ec0 0x7f75a015a1f0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f75a00affb0 0x7f75a0157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 --2- 172.21.2.104:0/341632772 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f75a00a70c0 0x7f75a0156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 -- 172.21.2.104:0/341632772 >> 172.21.2.104:0/341632772 conn(0x7f75a005f6b0 msgr2=0x7f75a014df90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 -- 172.21.2.104:0/341632772 shutdown_connections 2021-07-11T04:02:39.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.404+0000 7f75a5cc7700 1 -- 172.21.2.104:0/341632772 wait complete. 2021-07-11T04:02:39.411 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:02:39.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 Processor -- start 2021-07-11T04:02:39.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 -- start start 2021-07-11T04:02:39.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f709c060100 0x7f709c064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f709c144440 0x7f709c065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709c0604e0 con 0x7f709c0b38d0 2021-07-11T04:02:39.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709c0656f0 con 0x7f709c144440 2021-07-11T04:02:39.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c065c80 con 0x7f709c060100 2021-07-11T04:02:39.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f7099d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f709c144440 0x7f709c065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f7099d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f709c144440 0x7f709c065200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51942/0 (socket says 172.21.2.104:51942) 2021-07-11T04:02:39.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f7099d9b700 1 -- 172.21.2.104:0/2341991037 learned_addr learned my addr 172.21.2.104:0/2341991037 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f7099d9b700 1 -- 172.21.2.104:0/2341991037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f709c060100 msgr2=0x7f709c064d10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:39.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f7099d9b700 1 --2- 172.21.2.104:0/2341991037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f709c060100 0x7f709c064d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f7099d9b700 1 -- 172.21.2.104:0/2341991037 >> v1:172.21.2.105:6789/0 conn(0x7f709c0b38d0 legacy=0x7f709c144970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:39.469 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f7099d9b700 1 -- 172.21.2.104:0/2341991037 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709c0b23b0 con 0x7f709c144440 2021-07-11T04:02:39.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f7099d9b700 1 --2- 172.21.2.104:0/2341991037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f709c144440 0x7f709c065200 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f709c069be0 tx=0x7f7084006eb0).ready entity=mon.0 client_cookie=d11688e6f8952ebc server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f709959a700 1 -- 172.21.2.104:0/2341991037 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f708400d570 con 0x7f709c144440 2021-07-11T04:02:39.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f709959a700 1 -- 172.21.2.104:0/2341991037 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7084000e90 con 0x7f709c144440 2021-07-11T04:02:39.470 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f709959a700 1 -- 172.21.2.104:0/2341991037 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f708400d8c0 con 0x7f709c144440 2021-07-11T04:02:39.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 -- 172.21.2.104:0/2341991037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f709c144440 msgr2=0x7f709c065200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 --2- 172.21.2.104:0/2341991037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f709c144440 0x7f709c065200 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f709c069be0 tx=0x7f7084006eb0).stop 2021-07-11T04:02:39.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 -- 172.21.2.104:0/2341991037 shutdown_connections 2021-07-11T04:02:39.471 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 --2- 172.21.2.104:0/2341991037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f709c144440 0x7f709c065200 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 --2- 172.21.2.104:0/2341991037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f709c060100 0x7f709c064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 -- 172.21.2.104:0/2341991037 >> 172.21.2.104:0/2341991037 conn(0x7f709c05cf60 msgr2=0x7f709c05f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 -- 172.21.2.104:0/2341991037 shutdown_connections 2021-07-11T04:02:39.472 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.464+0000 7f70a0a16700 1 -- 172.21.2.104:0/2341991037 wait complete. 2021-07-11T04:02:39.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70a0a16700 1 Processor -- start 2021-07-11T04:02:39.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70a0a16700 1 -- start start 2021-07-11T04:02:39.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70a0a16700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f709c060100 0x7f709c1469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70a0a16700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f709c0b38d0 0x7f709c154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.473 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70a0a16700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f709c141f50 0x7f709c154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70a0a16700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c1463f0 con 0x7f709c0b38d0 2021-07-11T04:02:39.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70a0a16700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f709c145eb0 con 0x7f709c060100 2021-07-11T04:02:39.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70a0a16700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709c146150 con 0x7f709c141f50 2021-07-11T04:02:39.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f7099d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f709c0b38d0 0x7f709c154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.474 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f7099d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f709c0b38d0 0x7f709c154130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51946/0 (socket says 172.21.2.104:51946) 2021-07-11T04:02:39.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f7099d9b700 1 -- 172.21.2.104:0/2637646103 learned_addr learned my addr 172.21.2.104:0/2637646103 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f7099d9b700 1 -- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f709c141f50 msgr2=0x7f709c154620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f7099d9b700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f709c141f50 0x7f709c154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.475 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f7099d9b700 1 -- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f709c060100 msgr2=0x7f709c1469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f7099d9b700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f709c060100 0x7f709c1469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f7099d9b700 1 -- 172.21.2.104:0/2637646103 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708400b1e0 con 0x7f709c0b38d0 2021-07-11T04:02:39.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f7099d9b700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f709c0b38d0 0x7f709c154130 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f709c069be0 tx=0x7f7084015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70937fe700 1 -- 172.21.2.104:0/2637646103 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7084011860 con 0x7f709c0b38d0 2021-07-11T04:02:39.476 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70937fe700 1 -- 172.21.2.104:0/2637646103 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70840119c0 con 0x7f709c0b38d0 2021-07-11T04:02:39.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70a0a16700 1 -- 172.21.2.104:0/2637646103 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f709c153c90 con 0x7f709c0b38d0 2021-07-11T04:02:39.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70a0a16700 1 -- 172.21.2.104:0/2637646103 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f709c142460 con 0x7f709c0b38d0 2021-07-11T04:02:39.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70937fe700 1 -- 172.21.2.104:0/2637646103 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7084011d20 con 0x7f709c0b38d0 2021-07-11T04:02:39.477 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70a0a16700 1 -- 172.21.2.104:0/2637646103 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f709c153e20 con 0x7f709c0b38d0 2021-07-11T04:02:39.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70937fe700 1 -- 172.21.2.104:0/2637646103 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f708400f070 con 0x7f709c0b38d0 2021-07-11T04:02:39.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70937fe700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70800532d0 0x7f7080055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f70937fe700 1 -- 172.21.2.104:0/2637646103 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f708406cf20 con 0x7f709c0b38d0 2021-07-11T04:02:39.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f709a59c700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70800532d0 0x7f7080055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.479 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.468+0000 7f709a59c700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70800532d0 0x7f7080055790 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f708c002fb0 tx=0x7f708c0069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.472+0000 7f70937fe700 1 -- 172.21.2.104:0/2637646103 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7084044190 con 0x7f709c0b38d0 2021-07-11T04:02:39.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 -- 172.21.2.104:0/2637646103 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f709c057350 con 0x7f709c0b38d0 2021-07-11T04:02:39.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70937fe700 1 -- 172.21.2.104:0/2637646103 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v161) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7084047770 con 0x7f709c0b38d0 2021-07-11T04:02:39.556 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:39.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 -- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70800532d0 msgr2=0x7f7080055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70800532d0 0x7f7080055790 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f708c002fb0 tx=0x7f708c0069b0).stop 2021-07-11T04:02:39.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 -- 172.21.2.104:0/2637646103 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f709c0b38d0 msgr2=0x7f709c154130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f709c0b38d0 0x7f709c154130 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f709c069be0 tx=0x7f7084015000).stop 2021-07-11T04:02:39.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 -- 172.21.2.104:0/2637646103 shutdown_connections 2021-07-11T04:02:39.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f709c141f50 0x7f709c154620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f70800532d0 0x7f7080055790 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f709c0b38d0 0x7f709c154130 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 --2- 172.21.2.104:0/2637646103 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f709c060100 0x7f709c1469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 -- 172.21.2.104:0/2637646103 >> 172.21.2.104:0/2637646103 conn(0x7f709c05cf60 msgr2=0x7f709c140640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 -- 172.21.2.104:0/2637646103 shutdown_connections 2021-07-11T04:02:39.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.556+0000 7f70a0a16700 1 -- 172.21.2.104:0/2637646103 wait complete. 2021-07-11T04:02:39.563 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:02:39.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 Processor -- start 2021-07-11T04:02:39.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 -- start start 2021-07-11T04:02:39.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f90c00b1ec0 0x7f90c00b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90c00affb0 0x7f90c00b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f90c00a70c0 0x7f90c0141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c00b0420 con 0x7f90c00b1ec0 2021-07-11T04:02:39.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c00b2790 con 0x7f90c00a70c0 2021-07-11T04:02:39.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c00b0650 con 0x7f90c00affb0 2021-07-11T04:02:39.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90bf7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f90c00b1ec0 0x7f90c00b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90bf7fe700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f90c00b1ec0 0x7f90c00b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51954/0 (socket says 172.21.2.104:51954) 2021-07-11T04:02:39.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90bf7fe700 1 -- 172.21.2.104:0/2463140199 learned_addr learned my addr 172.21.2.104:0/2463140199 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90bf7fe700 1 -- 172.21.2.104:0/2463140199 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90c00affb0 msgr2=0x7f90c00b22a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:39.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90bf7fe700 1 --2- 172.21.2.104:0/2463140199 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90c00affb0 0x7f90c00b22a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90bf7fe700 1 -- 172.21.2.104:0/2463140199 >> v2:172.21.2.105:3300/0 conn(0x7f90c00a70c0 msgr2=0x7f90c0141250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:39.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90bf7fe700 1 --2- 172.21.2.104:0/2463140199 >> v2:172.21.2.105:3300/0 conn(0x7f90c00a70c0 0x7f90c0141250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90bf7fe700 1 -- 172.21.2.104:0/2463140199 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c00b08f0 con 0x7f90c00b1ec0 2021-07-11T04:02:39.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90bf7fe700 1 --2- 172.21.2.104:0/2463140199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90c00b1ec0 0x7f90c00b12d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7f90a8002f70 tx=0x7f90a8006eb0).ready entity=mon.0 client_cookie=dc0c5b216e37ee63 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90be7fc700 1 -- 172.21.2.104:0/2463140199 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90a800d570 con 0x7f90c00b1ec0 2021-07-11T04:02:39.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90be7fc700 1 -- 172.21.2.104:0/2463140199 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a8000e90 con 0x7f90c00b1ec0 2021-07-11T04:02:39.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90be7fc700 1 -- 172.21.2.104:0/2463140199 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90a800d900 con 0x7f90c00b1ec0 2021-07-11T04:02:39.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 -- 172.21.2.104:0/2463140199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90c00b1ec0 msgr2=0x7f90c00b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 --2- 172.21.2.104:0/2463140199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90c00b1ec0 0x7f90c00b12d0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 rx=0x7f90a8002f70 tx=0x7f90a8006eb0).stop 2021-07-11T04:02:39.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 -- 172.21.2.104:0/2463140199 shutdown_connections 2021-07-11T04:02:39.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 --2- 172.21.2.104:0/2463140199 >> v2:172.21.2.105:3300/0 conn(0x7f90c00a70c0 0x7f90c0141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 --2- 172.21.2.104:0/2463140199 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90c00affb0 0x7f90c00b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 --2- 172.21.2.104:0/2463140199 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90c00b1ec0 0x7f90c00b12d0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 -- 172.21.2.104:0/2463140199 >> 172.21.2.104:0/2463140199 conn(0x7f90c005f6b0 msgr2=0x7f90c005fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 -- 172.21.2.104:0/2463140199 shutdown_connections 2021-07-11T04:02:39.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.616+0000 7f90c5cac700 1 -- 172.21.2.104:0/2463140199 wait complete. 2021-07-11T04:02:39.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90c5cac700 1 Processor -- start 2021-07-11T04:02:39.635 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90c5cac700 1 -- start start 2021-07-11T04:02:39.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90c5cac700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90c00a70c0 0x7f90c014e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90c5cac700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90c00affb0 0x7f90c00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90c5cac700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90c00b1ec0 0x7f90c0142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.638 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90c5cac700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c00adce0 con 0x7f90c00affb0 2021-07-11T04:02:39.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90c5cac700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c015b0c0 con 0x7f90c00a70c0 2021-07-11T04:02:39.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90c5cac700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c00adf50 con 0x7f90c00b1ec0 2021-07-11T04:02:39.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90beffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90c00affb0 0x7f90c00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.639 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90beffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90c00affb0 0x7f90c00ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51962/0 (socket says 172.21.2.104:51962) 2021-07-11T04:02:39.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90beffd700 1 -- 172.21.2.104:0/2980054491 learned_addr learned my addr 172.21.2.104:0/2980054491 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90beffd700 1 -- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90c00b1ec0 msgr2=0x7f90c0142b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90beffd700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90c00b1ec0 0x7f90c0142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90beffd700 1 -- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90c00a70c0 msgr2=0x7f90c014e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.640 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90beffd700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90c00a70c0 0x7f90c014e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90beffd700 1 -- 172.21.2.104:0/2980054491 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a800b1e0 con 0x7f90c00affb0 2021-07-11T04:02:39.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90beffd700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90c00affb0 0x7f90c00ae690 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7f90b0002fb0 tx=0x7f90b0006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90bcff9700 1 -- 172.21.2.104:0/2980054491 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90b0013460 con 0x7f90c00affb0 2021-07-11T04:02:39.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90bcff9700 1 -- 172.21.2.104:0/2980054491 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b00135c0 con 0x7f90c00affb0 2021-07-11T04:02:39.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90c5cac700 1 -- 172.21.2.104:0/2980054491 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c00ae1f0 con 0x7f90c00affb0 2021-07-11T04:02:39.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90bcff9700 1 -- 172.21.2.104:0/2980054491 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f90b0013920 con 0x7f90c00affb0 2021-07-11T04:02:39.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90c5cac700 1 -- 172.21.2.104:0/2980054491 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c0143050 con 0x7f90c00affb0 2021-07-11T04:02:39.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90c5cac700 1 -- 172.21.2.104:0/2980054491 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c0145440 con 0x7f90c00affb0 2021-07-11T04:02:39.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90bcff9700 1 -- 172.21.2.104:0/2980054491 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f90b000f070 con 0x7f90c00affb0 2021-07-11T04:02:39.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90bcff9700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90a40532d0 0x7f90a4055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90bcff9700 1 -- 172.21.2.104:0/2980054491 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f90b006af00 con 0x7f90c00affb0 2021-07-11T04:02:39.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.620+0000 7f90bf7fe700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90a40532d0 0x7f90a4055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.624+0000 7f90bf7fe700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90a40532d0 0x7f90a4055790 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f90a8002f70 tx=0x7f90a8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.624+0000 7f90bcff9700 1 -- 172.21.2.104:0/2980054491 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f90b0042240 con 0x7f90c00affb0 2021-07-11T04:02:39.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.704+0000 7f90c5cac700 1 -- 172.21.2.104:0/2980054491 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90c00ae380 con 0x7f90c00affb0 2021-07-11T04:02:39.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.704+0000 7f90bcff9700 1 -- 172.21.2.104:0/2980054491 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v161) v1 ==== 100+0+10 (secure 0 0 0) 0x7f90b0045860 con 0x7f90c00affb0 2021-07-11T04:02:39.707 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:39.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 -- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90a40532d0 msgr2=0x7f90a4055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90a40532d0 0x7f90a4055790 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f90a8002f70 tx=0x7f90a8015000).stop 2021-07-11T04:02:39.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 -- 172.21.2.104:0/2980054491 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90c00affb0 msgr2=0x7f90c00ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90c00affb0 0x7f90c00ae690 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7f90b0002fb0 tx=0x7f90b0006e10).stop 2021-07-11T04:02:39.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 -- 172.21.2.104:0/2980054491 shutdown_connections 2021-07-11T04:02:39.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f90c00b1ec0 0x7f90c0142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f90a40532d0 0x7f90a4055790 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f90c00affb0 0x7f90c00ae690 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 --2- 172.21.2.104:0/2980054491 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f90c00a70c0 0x7f90c014e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 -- 172.21.2.104:0/2980054491 >> 172.21.2.104:0/2980054491 conn(0x7f90c005f6b0 msgr2=0x7f90c0145fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 -- 172.21.2.104:0/2980054491 shutdown_connections 2021-07-11T04:02:39.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.708+0000 7f90c5cac700 1 -- 172.21.2.104:0/2980054491 wait complete. 2021-07-11T04:02:39.714 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:02:39.715 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:39.766 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.764+0000 7fee15199700 1 Processor -- start 2021-07-11T04:02:39.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.764+0000 7fee15199700 1 -- start start 2021-07-11T04:02:39.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fee10060d00 0x7fee100aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee100afd80 0x7fee10143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.767 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee100671e0 con 0x7fee10066e00 2021-07-11T04:02:39.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee100b0190 con 0x7fee10060d00 2021-07-11T04:02:39.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee100a9450 con 0x7fee100afd80 2021-07-11T04:02:39.771 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0e59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fee10060d00 0x7fee100aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0e59c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fee10060d00 0x7fee100aaf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51968/0 (socket says 172.21.2.104:51968) 2021-07-11T04:02:39.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0e59c700 1 -- 172.21.2.104:0/2544794637 learned_addr learned my addr 172.21.2.104:0/2544794637 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0e59c700 1 -- 172.21.2.104:0/2544794637 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee100afd80 msgr2=0x7fee10143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:39.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0e59c700 1 --2- 172.21.2.104:0/2544794637 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee100afd80 0x7fee10143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.772 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0e59c700 1 -- 172.21.2.104:0/2544794637 >> v1:172.21.2.105:6789/0 conn(0x7fee10066e00 legacy=0x7fee100aa800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:39.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0e59c700 1 -- 172.21.2.104:0/2544794637 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee100b03b0 con 0x7fee10060d00 2021-07-11T04:02:39.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0e59c700 1 --2- 172.21.2.104:0/2544794637 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee10060d00 0x7fee100aaf10 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7fee100623b0 tx=0x7fee00009290).ready entity=mon.0 client_cookie=c54763690e84d233 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0dd9b700 1 -- 172.21.2.104:0/2544794637 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee0000b830 con 0x7fee10060d00 2021-07-11T04:02:39.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0dd9b700 1 -- 172.21.2.104:0/2544794637 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0000b990 con 0x7fee10060d00 2021-07-11T04:02:39.774 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0dd9b700 1 -- 172.21.2.104:0/2544794637 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fee0000bb00 con 0x7fee10060d00 2021-07-11T04:02:39.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- 172.21.2.104:0/2544794637 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee10060d00 msgr2=0x7fee100aaf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 --2- 172.21.2.104:0/2544794637 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee10060d00 0x7fee100aaf10 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7fee100623b0 tx=0x7fee00009290).stop 2021-07-11T04:02:39.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- 172.21.2.104:0/2544794637 shutdown_connections 2021-07-11T04:02:39.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 --2- 172.21.2.104:0/2544794637 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee100afd80 0x7fee10143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 --2- 172.21.2.104:0/2544794637 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee10060d00 0x7fee100aaf10 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- 172.21.2.104:0/2544794637 >> 172.21.2.104:0/2544794637 conn(0x7fee100b53e0 msgr2=0x7fee100b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- 172.21.2.104:0/2544794637 shutdown_connections 2021-07-11T04:02:39.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- 172.21.2.104:0/2544794637 wait complete. 2021-07-11T04:02:39.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 Processor -- start 2021-07-11T04:02:39.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- start start 2021-07-11T04:02:39.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee10060d00 0x7fee101439c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee10066e00 0x7fee101557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fee100afd80 0x7fee100ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee100aefe0 con 0x7fee10060d00 2021-07-11T04:02:39.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee100af280 con 0x7fee100afd80 2021-07-11T04:02:39.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee10147d30 con 0x7fee10066e00 2021-07-11T04:02:39.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee10060d00 0x7fee101439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0ed9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee10060d00 0x7fee101439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51974/0 (socket says 172.21.2.104:51974) 2021-07-11T04:02:39.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0ed9d700 1 -- 172.21.2.104:0/690073189 learned_addr learned my addr 172.21.2.104:0/690073189 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:39.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0ed9d700 1 -- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee10066e00 msgr2=0x7fee101557b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0ed9d700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee10066e00 0x7fee101557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0ed9d700 1 -- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fee100afd80 msgr2=0x7fee100ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0ed9d700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fee100afd80 0x7fee100ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0ed9d700 1 -- 172.21.2.104:0/690073189 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee00009530 con 0x7fee10060d00 2021-07-11T04:02:39.782 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee0ed9d700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee10060d00 0x7fee101439c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7fedf8002fb0 tx=0x7fedf8006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee07fff700 1 -- 172.21.2.104:0/690073189 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedf8013460 con 0x7fee10060d00 2021-07-11T04:02:39.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee07fff700 1 -- 172.21.2.104:0/690073189 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf80135c0 con 0x7fee10060d00 2021-07-11T04:02:39.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- 172.21.2.104:0/690073189 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee100adf30 con 0x7fee10060d00 2021-07-11T04:02:39.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee07fff700 1 -- 172.21.2.104:0/690073189 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fedf8013920 con 0x7fee10060d00 2021-07-11T04:02:39.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.768+0000 7fee15199700 1 -- 172.21.2.104:0/690073189 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1014a700 con 0x7fee10060d00 2021-07-11T04:02:39.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.772+0000 7fee15199700 1 -- 172.21.2.104:0/690073189 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee100aee20 con 0x7fee10060d00 2021-07-11T04:02:39.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.772+0000 7fee07fff700 1 -- 172.21.2.104:0/690073189 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fedf800f070 con 0x7fee10060d00 2021-07-11T04:02:39.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.772+0000 7fee07fff700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedf4053160 0x7fedf4055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:39.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.772+0000 7fee07fff700 1 -- 172.21.2.104:0/690073189 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fedf806aef0 con 0x7fee10060d00 2021-07-11T04:02:39.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.772+0000 7fee0e59c700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedf4053160 0x7fedf4055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:39.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.772+0000 7fee0e59c700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedf4053160 0x7fedf4055620 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fee100623b0 tx=0x7fee00016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:39.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.772+0000 7fee07fff700 1 -- 172.21.2.104:0/690073189 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fedf8042230 con 0x7fee10060d00 2021-07-11T04:02:39.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.880+0000 7fee15199700 1 -- 172.21.2.104:0/690073189 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee10147ef0 con 0x7fee10060d00 2021-07-11T04:02:39.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.880+0000 7fee07fff700 1 -- 172.21.2.104:0/690073189 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2489 (secure 0 0 0) 0x7fedf8048090 con 0x7fee10060d00 2021-07-11T04:02:39.883 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:39.883 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":80,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5132,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/770706140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":770706140},{"type":"v1","addr":"172.21.2.105:6835","nonce":770706140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":79},{"gid":8718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/488029782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":488029782},{"type":"v1","addr":"172.21.2.104:6833","nonce":488029782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:36.673961+0000","modified":"2021-07-11T04:02:37.679104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8721":{"gid":8721,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/2638676458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2638676458},{"type":"v1","addr":"172.21.2.104:6835","nonce":2638676458}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-07-11T04:02:39.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 -- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedf4053160 msgr2=0x7fedf4055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedf4053160 0x7fedf4055620 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fee100623b0 tx=0x7fee00016000).stop 2021-07-11T04:02:39.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 -- 172.21.2.104:0/690073189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee10060d00 msgr2=0x7fee101439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:39.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee10060d00 0x7fee101439c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7fedf8002fb0 tx=0x7fedf8006e10).stop 2021-07-11T04:02:39.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 -- 172.21.2.104:0/690073189 shutdown_connections 2021-07-11T04:02:39.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fee100afd80 0x7fee100ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fee10066e00 0x7fee101557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:39.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fedf4053160 0x7fedf4055620 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 --2- 172.21.2.104:0/690073189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fee10060d00 0x7fee101439c0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:39.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 -- 172.21.2.104:0/690073189 >> 172.21.2.104:0/690073189 conn(0x7fee100b53e0 msgr2=0x7fee1013f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:39.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 -- 172.21.2.104:0/690073189 shutdown_connections 2021-07-11T04:02:39.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:39.884+0000 7fee15199700 1 -- 172.21.2.104:0/690073189 wait complete. 2021-07-11T04:02:39.889 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 80 2021-07-11T04:02:40.196 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:02:36.673961+0000', 'modified': '2021-07-11T04:02:37.679104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8721}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8721': {'gid': 8721, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/2638676458', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 2638676458}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 2638676458}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:02:40.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:02:40.198 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:02:40.297 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":8721,"want_state":"up:active","state":"up:active","rank_uptime":3.6100561199999999,"mdsmap_epoch":80,"osdmap_epoch":161,"osdmap_epoch_barrier":160,"uptime":7.7785505029999999} 2021-07-11T04:02:40.299 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 8721, "mdsmap_epoch": 80, "osdmap_epoch": 161, "osdmap_epoch_barrier": 160, "rank_uptime": 3.61005612, "state": "up:active", "uptime": 7.778550503, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:02:40.305 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:02:40.306 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2021-07-11T04:02:40.307 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.0 at ubuntu@gibba019.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0... 2021-07-11T04:02:40.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:40.308 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:40.323 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:02:40.338 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:40.339 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:40.391 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:40.393 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:40.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:40.393 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:02:40.402 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:02:40.403 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.0 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:40.447 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:02:40.461 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:40.462 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:40.471 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:40.473 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:40.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:40.474 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:02:40.490 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15894]: starting ceph client2021-07-11T04:02:40.487+0000 7f2790dd4080 -1 init, newargv = 0x55b0348f7b30 newargc=9 2021-07-11T04:02:40.491 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr: 2021-07-11T04:02:40.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.gibba019.stderr:ceph-fuse[15894]: starting fuse 2021-07-11T04:02:41.598 DEBUG:teuthology.orchestra.run.gibba019:> sudo modprobe fuse 2021-07-11T04:02:41.610 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:41.611 DEBUG:teuthology.orchestra.run.gibba019:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:41.617 INFO:teuthology.orchestra.run.gibba019.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:41.617 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:41.618 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:41.619 DEBUG:teuthology.orchestra.run.gibba019:> ls /sys/fs/fuse/connections 2021-07-11T04:02:41.623 INFO:teuthology.orchestra.run.gibba019.stdout:52 2021-07-11T04:02:41.624 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:02:41.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:41.624 DEBUG:teuthology.orchestra.run.gibba019:> sudo python3 -c ' 2021-07-11T04:02:41.625 DEBUG:teuthology.orchestra.run.gibba019:> import glob 2021-07-11T04:02:41.625 DEBUG:teuthology.orchestra.run.gibba019:> import re 2021-07-11T04:02:41.626 DEBUG:teuthology.orchestra.run.gibba019:> import os 2021-07-11T04:02:41.626 DEBUG:teuthology.orchestra.run.gibba019:> import subprocess 2021-07-11T04:02:41.627 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:02:41.627 DEBUG:teuthology.orchestra.run.gibba019:> def find_socket(client_name): 2021-07-11T04:02:41.627 DEBUG:teuthology.orchestra.run.gibba019:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2021-07-11T04:02:41.628 DEBUG:teuthology.orchestra.run.gibba019:> files = glob.glob(asok_path) 2021-07-11T04:02:41.628 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:02:41.629 DEBUG:teuthology.orchestra.run.gibba019:> # Given a non-glob path, it better be there 2021-07-11T04:02:41.629 DEBUG:teuthology.orchestra.run.gibba019:> if "*" not in asok_path: 2021-07-11T04:02:41.629 DEBUG:teuthology.orchestra.run.gibba019:> assert(len(files) == 1) 2021-07-11T04:02:41.630 DEBUG:teuthology.orchestra.run.gibba019:> return files[0] 2021-07-11T04:02:41.630 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:02:41.631 DEBUG:teuthology.orchestra.run.gibba019:> for f in files: 2021-07-11T04:02:41.631 DEBUG:teuthology.orchestra.run.gibba019:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:02:41.632 DEBUG:teuthology.orchestra.run.gibba019:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:02:41.632 DEBUG:teuthology.orchestra.run.gibba019:> return f 2021-07-11T04:02:41.633 DEBUG:teuthology.orchestra.run.gibba019:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:02:41.633 DEBUG:teuthology.orchestra.run.gibba019:> 2021-07-11T04:02:41.633 DEBUG:teuthology.orchestra.run.gibba019:> print(find_socket("client.0")) 2021-07-11T04:02:41.634 DEBUG:teuthology.orchestra.run.gibba019:> ' 2021-07-11T04:02:41.695 INFO:teuthology.orchestra.run.gibba019.stdout:/var/run/ceph/ceph-client.0.15894.asok 2021-07-11T04:02:41.696 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.15894.asok 2021-07-11T04:02:41.697 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:41.697 DEBUG:teuthology.orchestra.run.gibba019:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.15894.asok status 2021-07-11T04:02:41.745 INFO:teuthology.orchestra.run.gibba019.stdout:{ 2021-07-11T04:02:41.745 INFO:teuthology.orchestra.run.gibba019.stdout: "metadata": { 2021-07-11T04:02:41.746 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:02:41.746 INFO:teuthology.orchestra.run.gibba019.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:02:41.746 INFO:teuthology.orchestra.run.gibba019.stdout: "entity_id": "0", 2021-07-11T04:02:41.746 INFO:teuthology.orchestra.run.gibba019.stdout: "hostname": "gibba019", 2021-07-11T04:02:41.747 INFO:teuthology.orchestra.run.gibba019.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2021-07-11T04:02:41.747 INFO:teuthology.orchestra.run.gibba019.stdout: "pid": "15894", 2021-07-11T04:02:41.747 INFO:teuthology.orchestra.run.gibba019.stdout: "root": "/" 2021-07-11T04:02:41.747 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:02:41.748 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_count": 0, 2021-07-11T04:02:41.748 INFO:teuthology.orchestra.run.gibba019.stdout: "dentry_pinned_count": 0, 2021-07-11T04:02:41.748 INFO:teuthology.orchestra.run.gibba019.stdout: "id": 5581, 2021-07-11T04:02:41.748 INFO:teuthology.orchestra.run.gibba019.stdout: "inst": { 2021-07-11T04:02:41.748 INFO:teuthology.orchestra.run.gibba019.stdout: "name": { 2021-07-11T04:02:41.749 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "client", 2021-07-11T04:02:41.749 INFO:teuthology.orchestra.run.gibba019.stdout: "num": 5581 2021-07-11T04:02:41.749 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:02:41.749 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:02:41.749 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:02:41.750 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:02:41.750 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 758190324 2021-07-11T04:02:41.750 INFO:teuthology.orchestra.run.gibba019.stdout: } 2021-07-11T04:02:41.751 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:02:41.751 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": { 2021-07-11T04:02:41.751 INFO:teuthology.orchestra.run.gibba019.stdout: "type": "v1", 2021-07-11T04:02:41.752 INFO:teuthology.orchestra.run.gibba019.stdout: "addr": "172.21.2.119:0", 2021-07-11T04:02:41.752 INFO:teuthology.orchestra.run.gibba019.stdout: "nonce": 758190324 2021-07-11T04:02:41.752 INFO:teuthology.orchestra.run.gibba019.stdout: }, 2021-07-11T04:02:41.752 INFO:teuthology.orchestra.run.gibba019.stdout: "inst_str": "client.5581 172.21.2.119:0/758190324", 2021-07-11T04:02:41.752 INFO:teuthology.orchestra.run.gibba019.stdout: "addr_str": "172.21.2.119:0/758190324", 2021-07-11T04:02:41.753 INFO:teuthology.orchestra.run.gibba019.stdout: "inode_count": 1, 2021-07-11T04:02:41.753 INFO:teuthology.orchestra.run.gibba019.stdout: "mds_epoch": 80, 2021-07-11T04:02:41.753 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch": 161, 2021-07-11T04:02:41.753 INFO:teuthology.orchestra.run.gibba019.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:02:41.754 INFO:teuthology.orchestra.run.gibba019.stdout: "blacklisted": false, 2021-07-11T04:02:41.754 INFO:teuthology.orchestra.run.gibba019.stdout: "fs_name": "cephfs" 2021-07-11T04:02:41.754 INFO:teuthology.orchestra.run.gibba019.stdout:} 2021-07-11T04:02:41.755 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:41.756 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:41.756 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:41.762 INFO:teuthology.orchestra.run.gibba019.stdout:fuseblk 2021-07-11T04:02:41.762 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2021-07-11T04:02:41.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:41.763 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:02:41.775 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:02:41.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.828+0000 7f3639803700 1 Processor -- start 2021-07-11T04:02:41.829 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.828+0000 7f3639803700 1 -- start start 2021-07-11T04:02:41.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.828+0000 7f3639803700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f363405f6b0 0x7f363413fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.828+0000 7f3639803700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36340b2a40 con 0x7f36340b2660 2021-07-11T04:02:41.833 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.828+0000 7f3639803700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3634097b80 con 0x7f36340973d0 2021-07-11T04:02:41.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.828+0000 7f3639803700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3634097e20 con 0x7f363405f6b0 2021-07-11T04:02:41.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.828+0000 7f3632ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f36340b2660 0x7f3634096cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48748/0 (socket says 172.21.2.104:48748) 2021-07-11T04:02:41.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.828+0000 7f3632ffd700 1 -- 172.21.2.104:0/3754902371 learned_addr learned my addr 172.21.2.104:0/3754902371 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:41.834 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087104428 0 0) 0x7f36340b2a40 con 0x7f36340b2660 2021-07-11T04:02:41.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3614003260 con 0x7f36340b2660 2021-07-11T04:02:41.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36337fe700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f363405f6b0 0x7f363413fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:41.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 86119140 0 0) 0x7f3614003260 con 0x7f36340b2660 2021-07-11T04:02:41.835 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f363405f6b0 msgr2=0x7f363413fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:41.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f363405f6b0 0x7f363413fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:41.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 >> v1:172.21.2.105:6789/0 conn(0x7f36340973d0 legacy=0x7f363413f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:41.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36340980c0 con 0x7f36340b2660 2021-07-11T04:02:41.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f361c001680 con 0x7f36340b2660 2021-07-11T04:02:41.836 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 >> v1:172.21.2.104:6789/0 conn(0x7f36340b2660 legacy=0x7f3634096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:41.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f363405f6b0 0x7f3614003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3614003e20 0x7f3614004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3614004b10 0x7f3614004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.837 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3634097e20 con 0x7f3614004b10 2021-07-11T04:02:41.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3614005960 con 0x7f3614003e20 2021-07-11T04:02:41.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3614005b20 con 0x7f363405f6b0 2021-07-11T04:02:41.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36337fe700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3614004b10 0x7f3614004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:41.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36327fc700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3614003e20 0x7f3614004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:41.838 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3632ffd700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f363405f6b0 0x7f3614003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:41.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36337fe700 1 -- 172.21.2.104:0/3754902371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f363405f6b0 msgr2=0x7f3614003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:41.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36337fe700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f363405f6b0 0x7f3614003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:41.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36337fe700 1 -- 172.21.2.104:0/3754902371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3614003e20 msgr2=0x7f3614004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:41.839 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36337fe700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3614003e20 0x7f3614004500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:41.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36337fe700 1 -- 172.21.2.104:0/3754902371 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614003260 con 0x7f3614004b10 2021-07-11T04:02:41.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36337fe700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3614004b10 0x7f3614004f90 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7f362c007000 tx=0x7f362c008040).ready entity=mon.0 client_cookie=cd0e97a6bdd56c74 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:41.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f362c00c640 con 0x7f3614004b10 2021-07-11T04:02:41.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c00c7a0 con 0x7f3614004b10 2021-07-11T04:02:41.840 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3631ffb700 1 -- 172.21.2.104:0/3754902371 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f362c00cb00 con 0x7f3614004b10 2021-07-11T04:02:41.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 -- 172.21.2.104:0/3754902371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3614004b10 msgr2=0x7f3614004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:41.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 --2- 172.21.2.104:0/3754902371 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3614004b10 0x7f3614004f90 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7f362c007000 tx=0x7f362c008040).stop 2021-07-11T04:02:41.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36337fe700 1 -- 172.21.2.104:0/3754902371 reap_dead start 2021-07-11T04:02:41.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 -- 172.21.2.104:0/3754902371 shutdown_connections 2021-07-11T04:02:41.841 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 -- 172.21.2.104:0/3754902371 >> 172.21.2.104:0/3754902371 conn(0x7f3634062f10 msgr2=0x7f36340632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:41.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 -- 172.21.2.104:0/3754902371 shutdown_connections 2021-07-11T04:02:41.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 -- 172.21.2.104:0/3754902371 wait complete. 2021-07-11T04:02:41.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 Processor -- start 2021-07-11T04:02:41.842 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 -- start start 2021-07-11T04:02:41.845 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f36340abd00 0x7f3634098c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3634099100 0x7f3634066e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3634099580 0x7f3634067310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3634064660 con 0x7f3634099100 2021-07-11T04:02:41.846 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363413e920 con 0x7f36340abd00 2021-07-11T04:02:41.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f3639803700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3634065110 con 0x7f3634099580 2021-07-11T04:02:41.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36327fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3634099100 0x7f3634066e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:41.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36327fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3634099100 0x7f3634066e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:51996/0 (socket says 172.21.2.104:51996) 2021-07-11T04:02:41.847 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.832+0000 7f36327fc700 1 -- 172.21.2.104:0/3925445284 learned_addr learned my addr 172.21.2.104:0/3925445284 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:41.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f36327fc700 1 -- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3634099580 msgr2=0x7f3634067310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:41.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f36327fc700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3634099580 0x7f3634067310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:41.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f36327fc700 1 -- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f36340abd00 msgr2=0x7f3634098c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:41.848 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f36327fc700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f36340abd00 0x7f3634098c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:41.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f36327fc700 1 -- 172.21.2.104:0/3925445284 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363413fe00 con 0x7f3634099100 2021-07-11T04:02:41.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f36327fc700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3634099100 0x7f3634066e20 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7f362400b000 tx=0x7f3624009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:41.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f362bfff700 1 -- 172.21.2.104:0/3925445284 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3624015630 con 0x7f3634099100 2021-07-11T04:02:41.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f362bfff700 1 -- 172.21.2.104:0/3925445284 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3624000e90 con 0x7f3634099100 2021-07-11T04:02:41.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f3639803700 1 -- 172.21.2.104:0/3925445284 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3634065340 con 0x7f3634099100 2021-07-11T04:02:41.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f3639803700 1 -- 172.21.2.104:0/3925445284 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3634065880 con 0x7f3634099100 2021-07-11T04:02:41.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f362bfff700 1 -- 172.21.2.104:0/3925445284 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3624015980 con 0x7f3634099100 2021-07-11T04:02:41.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f3639803700 1 -- 172.21.2.104:0/3925445284 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36340ac280 con 0x7f3634099100 2021-07-11T04:02:41.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f362bfff700 1 -- 172.21.2.104:0/3925445284 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3624015ae0 con 0x7f3634099100 2021-07-11T04:02:41.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f362bfff700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3618053260 0x7f3618055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f362bfff700 1 -- 172.21.2.104:0/3925445284 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f362406b340 con 0x7f3634099100 2021-07-11T04:02:41.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f3632ffd700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3618053260 0x7f3618055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:41.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.836+0000 7f3632ffd700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3618053260 0x7f3618055720 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f361c00b000 tx=0x7f3634096a20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:41.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.840+0000 7f362bfff700 1 -- 172.21.2.104:0/3925445284 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f3624042570 con 0x7f3634099100 2021-07-11T04:02:41.919 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 -- 172.21.2.104:0/3925445284 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3634057350 con 0x7f3634099100 2021-07-11T04:02:41.920 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f362bfff700 1 -- 172.21.2.104:0/3925445284 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f3624045b90 con 0x7f3634099100 2021-07-11T04:02:41.921 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:41.921 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":161,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:38.690916+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2021-07-11T04:02:34.620986+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":30,"pool_name":"cephfs_data","create_time":"2021-07-11T04:02:35.639038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:02:41.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 -- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3618053260 msgr2=0x7f3618055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:41.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3618053260 0x7f3618055720 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f361c00b000 tx=0x7f3634096a20).stop 2021-07-11T04:02:41.922 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 -- 172.21.2.104:0/3925445284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3634099100 msgr2=0x7f3634066e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:41.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3634099100 0x7f3634066e20 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7f362400b000 tx=0x7f3624009040).stop 2021-07-11T04:02:41.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 -- 172.21.2.104:0/3925445284 shutdown_connections 2021-07-11T04:02:41.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f3634099580 0x7f3634067310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:41.923 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3618053260 0x7f3618055720 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:41.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3634099100 0x7f3634066e20 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:41.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 --2- 172.21.2.104:0/3925445284 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f36340abd00 0x7f3634098c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:41.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 -- 172.21.2.104:0/3925445284 >> 172.21.2.104:0/3925445284 conn(0x7f3634062f10 msgr2=0x7f3634060210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:41.924 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 -- 172.21.2.104:0/3925445284 shutdown_connections 2021-07-11T04:02:41.925 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.920+0000 7f3639803700 1 -- 172.21.2.104:0/3925445284 wait complete. 2021-07-11T04:02:41.928 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-07-11T03:53:10.374230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-07-11T03:53:18.251635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2021-07-11T04:02:34.620986+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2021-07-11T04:02:35.639038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '161', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-07-11T04:02:41.929 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-07-11T04:02:41.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 Processor -- start 2021-07-11T04:02:41.983 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 -- start start 2021-07-11T04:02:41.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe9bc0b12d0 0x7fe9bc0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9bc0b22a0 0x7fe9bc141250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.985 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc0a77d0 con 0x7fe9bc0b1ec0 2021-07-11T04:02:41.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc0b1740 con 0x7fe9bc0b12d0 2021-07-11T04:02:41.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0b0650 con 0x7fe9bc0b22a0 2021-07-11T04:02:41.986 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9b9d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe9bc0b12d0 0x7fe9bc0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:41.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9b9d9b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fe9bc0b12d0 0x7fe9bc0affb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52002/0 (socket says 172.21.2.104:52002) 2021-07-11T04:02:41.987 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9b9d9b700 1 -- 172.21.2.104:0/1400521812 learned_addr learned my addr 172.21.2.104:0/1400521812 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:41.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9b9d9b700 1 -- 172.21.2.104:0/1400521812 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9bc0b22a0 msgr2=0x7fe9bc141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:41.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9b9d9b700 1 --2- 172.21.2.104:0/1400521812 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9bc0b22a0 0x7fe9bc141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:41.988 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9b9d9b700 1 -- 172.21.2.104:0/1400521812 >> v1:172.21.2.105:6789/0 conn(0x7fe9bc0b1ec0 legacy=0x7fe9bc0a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:41.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9b9d9b700 1 -- 172.21.2.104:0/1400521812 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc0b08f0 con 0x7fe9bc0b12d0 2021-07-11T04:02:41.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9b9d9b700 1 --2- 172.21.2.104:0/1400521812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9bc0b12d0 0x7fe9bc0affb0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 rx=0x7fe9bc0b2f50 tx=0x7fe9ac006eb0).ready entity=mon.0 client_cookie=2e1a9c11718ec426 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:41.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9b959a700 1 -- 172.21.2.104:0/1400521812 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9ac00d570 con 0x7fe9bc0b12d0 2021-07-11T04:02:41.989 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9b959a700 1 -- 172.21.2.104:0/1400521812 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac000e90 con 0x7fe9bc0b12d0 2021-07-11T04:02:41.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9b959a700 1 -- 172.21.2.104:0/1400521812 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9ac00d8c0 con 0x7fe9bc0b12d0 2021-07-11T04:02:41.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 -- 172.21.2.104:0/1400521812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9bc0b12d0 msgr2=0x7fe9bc0affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:41.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 --2- 172.21.2.104:0/1400521812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9bc0b12d0 0x7fe9bc0affb0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 rx=0x7fe9bc0b2f50 tx=0x7fe9ac006eb0).stop 2021-07-11T04:02:41.990 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 -- 172.21.2.104:0/1400521812 shutdown_connections 2021-07-11T04:02:41.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 --2- 172.21.2.104:0/1400521812 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9bc0b22a0 0x7fe9bc141250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:41.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 --2- 172.21.2.104:0/1400521812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9bc0b12d0 0x7fe9bc0affb0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:41.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 -- 172.21.2.104:0/1400521812 >> 172.21.2.104:0/1400521812 conn(0x7fe9bc05f6b0 msgr2=0x7fe9bc05fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:41.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 -- 172.21.2.104:0/1400521812 shutdown_connections 2021-07-11T04:02:41.991 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 -- 172.21.2.104:0/1400521812 wait complete. 2021-07-11T04:02:41.994 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 Processor -- start 2021-07-11T04:02:41.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.984+0000 7fe9c0d28700 1 -- start start 2021-07-11T04:02:41.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9c0d28700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9bc0b12d0 0x7fe9bc148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.995 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9c0d28700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9bc0b1ec0 0x7fe9bc0aa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9c0d28700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9bc0b22a0 0x7fe9bc148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:41.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9c0d28700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc14a400 con 0x7fe9bc0b12d0 2021-07-11T04:02:41.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9c0d28700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc149e50 con 0x7fe9bc0b22a0 2021-07-11T04:02:41.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9c0d28700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc14aa80 con 0x7fe9bc0b1ec0 2021-07-11T04:02:41.996 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ba59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9bc0b12d0 0x7fe9bc148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:41.997 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ba59c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9bc0b12d0 0x7fe9bc148320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52006/0 (socket says 172.21.2.104:52006) 2021-07-11T04:02:41.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ba59c700 1 -- 172.21.2.104:0/2570503260 learned_addr learned my addr 172.21.2.104:0/2570503260 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:41.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ba59c700 1 -- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9bc0b1ec0 msgr2=0x7fe9bc0aa6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:41.998 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ba59c700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9bc0b1ec0 0x7fe9bc0aa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:41.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ba59c700 1 -- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9bc0b22a0 msgr2=0x7fe9bc148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:41.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ba59c700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9bc0b22a0 0x7fe9bc148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:41.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ba59c700 1 -- 172.21.2.104:0/2570503260 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac00b1e0 con 0x7fe9bc0b12d0 2021-07-11T04:02:41.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ba59c700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9bc0b12d0 0x7fe9bc148320 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 rx=0x7fe9bc149f90 tx=0x7fe9b0007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:41.999 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ab7fe700 1 -- 172.21.2.104:0/2570503260 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b000ce70 con 0x7fe9bc0b12d0 2021-07-11T04:02:42.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ab7fe700 1 -- 172.21.2.104:0/2570503260 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b0013450 con 0x7fe9bc0b12d0 2021-07-11T04:02:42.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9c0d28700 1 -- 172.21.2.104:0/2570503260 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc0ab8c0 con 0x7fe9bc0b12d0 2021-07-11T04:02:42.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9c0d28700 1 -- 172.21.2.104:0/2570503260 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc14ace0 con 0x7fe9bc0b12d0 2021-07-11T04:02:42.000 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ab7fe700 1 -- 172.21.2.104:0/2570503260 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fe9b00137b0 con 0x7fe9bc0b12d0 2021-07-11T04:02:42.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9c0d28700 1 -- 172.21.2.104:0/2570503260 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9bc057350 con 0x7fe9bc0b12d0 2021-07-11T04:02:42.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ab7fe700 1 -- 172.21.2.104:0/2570503260 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fe9b001a060 con 0x7fe9bc0b12d0 2021-07-11T04:02:42.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ab7fe700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe9980532d0 0x7fe998055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9ab7fe700 1 -- 172.21.2.104:0/2570503260 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe9b006af70 con 0x7fe9bc0b12d0 2021-07-11T04:02:42.001 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9b9d9b700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe9980532d0 0x7fe998055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.988+0000 7fe9b9d9b700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe9980532d0 0x7fe998055790 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fe9bc0b2f50 tx=0x7fe9ac015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.002 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:41.992+0000 7fe9ab7fe700 1 -- 172.21.2.104:0/2570503260 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fe9b00421a0 con 0x7fe9bc0b12d0 2021-07-11T04:02:42.078 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.076+0000 7fe9c0d28700 1 -- 172.21.2.104:0/2570503260 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe9bc0ac760 con 0x7fe9bc0b12d0 2021-07-11T04:02:42.079 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.076+0000 7fe9ab7fe700 1 -- 172.21.2.104:0/2570503260 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v161) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe9b0018020 con 0x7fe9bc0b12d0 2021-07-11T04:02:42.080 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 1 2021-07-11T04:02:42.081 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 -- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe9980532d0 msgr2=0x7fe998055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe9980532d0 0x7fe998055790 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fe9bc0b2f50 tx=0x7fe9ac015000).stop 2021-07-11T04:02:42.082 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 -- 172.21.2.104:0/2570503260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9bc0b12d0 msgr2=0x7fe9bc148320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9bc0b12d0 0x7fe9bc148320 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 rx=0x7fe9bc149f90 tx=0x7fe9b0007c30).stop 2021-07-11T04:02:42.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 -- 172.21.2.104:0/2570503260 shutdown_connections 2021-07-11T04:02:42.083 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fe9bc0b22a0 0x7fe9bc148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fe9bc0b1ec0 0x7fe9bc0aa6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fe9980532d0 0x7fe998055790 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.084 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 --2- 172.21.2.104:0/2570503260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fe9bc0b12d0 0x7fe9bc148320 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 -- 172.21.2.104:0/2570503260 >> 172.21.2.104:0/2570503260 conn(0x7fe9bc05f6b0 msgr2=0x7fe9bc13edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:42.085 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 -- 172.21.2.104:0/2570503260 shutdown_connections 2021-07-11T04:02:42.086 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.080+0000 7fe9c0d28700 1 -- 172.21.2.104:0/2570503260 wait complete. 2021-07-11T04:02:42.088 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2021-07-11T04:02:42.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1da067a700 1 Processor -- start 2021-07-11T04:02:42.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1da067a700 1 -- start start 2021-07-11T04:02:42.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1da067a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1d98060100 0x7f1d98064d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1da067a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d98144440 0x7f1d98065200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1da067a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1d980b38d0 0x7f1d98144970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1da067a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d980656f0 con 0x7f1d980b38d0 2021-07-11T04:02:42.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1da067a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d98065c80 con 0x7f1d98060100 2021-07-11T04:02:42.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1da067a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980604e0 con 0x7f1d98144440 2021-07-11T04:02:42.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1d9ec17700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1d980b38d0 0x7f1d98144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1d9ec17700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f1d980b38d0 0x7f1d98144970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52016/0 (socket says 172.21.2.104:52016) 2021-07-11T04:02:42.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1d9ec17700 1 -- 172.21.2.104:0/588611246 learned_addr learned my addr 172.21.2.104:0/588611246 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:42.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1d9ec17700 1 -- 172.21.2.104:0/588611246 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d98144440 msgr2=0x7f1d98065200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:42.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1d9ec17700 1 --2- 172.21.2.104:0/588611246 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d98144440 0x7f1d98065200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1d9ec17700 1 -- 172.21.2.104:0/588611246 >> v2:172.21.2.105:3300/0 conn(0x7f1d98060100 msgr2=0x7f1d98064d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:42.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1d9ec17700 1 --2- 172.21.2.104:0/588611246 >> v2:172.21.2.105:3300/0 conn(0x7f1d98060100 0x7f1d98064d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1d9ec17700 1 -- 172.21.2.104:0/588611246 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d980b23b0 con 0x7f1d980b38d0 2021-07-11T04:02:42.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.140+0000 7f1d9ec17700 1 --2- 172.21.2.104:0/588611246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d980b38d0 0x7f1d98144970 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 rx=0x7f1d900032f0 tx=0x7f1d90009290).ready entity=mon.0 client_cookie=a29a139ad8946e23 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9d414700 1 -- 172.21.2.104:0/588611246 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d9000ba60 con 0x7f1d980b38d0 2021-07-11T04:02:42.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9d414700 1 -- 172.21.2.104:0/588611246 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9000bbc0 con 0x7f1d980b38d0 2021-07-11T04:02:42.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9d414700 1 -- 172.21.2.104:0/588611246 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d90010560 con 0x7f1d980b38d0 2021-07-11T04:02:42.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- 172.21.2.104:0/588611246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d980b38d0 msgr2=0x7f1d98144970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 --2- 172.21.2.104:0/588611246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d980b38d0 0x7f1d98144970 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 rx=0x7f1d900032f0 tx=0x7f1d90009290).stop 2021-07-11T04:02:42.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- 172.21.2.104:0/588611246 shutdown_connections 2021-07-11T04:02:42.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 --2- 172.21.2.104:0/588611246 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d980b38d0 0x7f1d98144970 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 --2- 172.21.2.104:0/588611246 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d98144440 0x7f1d98065200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 --2- 172.21.2.104:0/588611246 >> v2:172.21.2.105:3300/0 conn(0x7f1d98060100 0x7f1d98064d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- 172.21.2.104:0/588611246 >> 172.21.2.104:0/588611246 conn(0x7f1d9805cf60 msgr2=0x7f1d9805f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:42.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- 172.21.2.104:0/588611246 shutdown_connections 2021-07-11T04:02:42.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- 172.21.2.104:0/588611246 wait complete. 2021-07-11T04:02:42.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 Processor -- start 2021-07-11T04:02:42.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- start start 2021-07-11T04:02:42.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d98060100 0x7f1d98159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.154 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d980b38d0 0x7f1d98143350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d980ab970 0x7f1d980ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980aaaa0 con 0x7f1d980ab970 2021-07-11T04:02:42.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980aa2c0 con 0x7f1d980b38d0 2021-07-11T04:02:42.155 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d980aa560 con 0x7f1d98060100 2021-07-11T04:02:42.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9ec17700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d980ab970 0x7f1d980ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9ec17700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d980ab970 0x7f1d980ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52024/0 (socket says 172.21.2.104:52024) 2021-07-11T04:02:42.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9ec17700 1 -- 172.21.2.104:0/1819713512 learned_addr learned my addr 172.21.2.104:0/1819713512 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:42.156 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9ec17700 1 -- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d98060100 msgr2=0x7f1d98159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9e416700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d98060100 0x7f1d98159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9ec17700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d98060100 0x7f1d98159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9ec17700 1 -- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d980b38d0 msgr2=0x7f1d98143350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.157 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9ec17700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d980b38d0 0x7f1d98143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9ec17700 1 -- 172.21.2.104:0/1819713512 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d90009530 con 0x7f1d980ab970 2021-07-11T04:02:42.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d9ec17700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d980ab970 0x7f1d980ad420 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 rx=0x7f1d900032f0 tx=0x7f1d90018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/1819713512 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d90017da0 con 0x7f1d980ab970 2021-07-11T04:02:42.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/1819713512 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d90018420 con 0x7f1d980ab970 2021-07-11T04:02:42.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- 172.21.2.104:0/1819713512 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d980aa800 con 0x7f1d980ab970 2021-07-11T04:02:42.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/1819713512 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1d90018780 con 0x7f1d980ab970 2021-07-11T04:02:42.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- 172.21.2.104:0/1819713512 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d980b15d0 con 0x7f1d980ab970 2021-07-11T04:02:42.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1da067a700 1 -- 172.21.2.104:0/1819713512 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d98057350 con 0x7f1d980ab970 2021-07-11T04:02:42.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/1819713512 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1d90027070 con 0x7f1d980ab970 2021-07-11T04:02:42.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d8b7fe700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d78053300 0x7f1d780557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.144+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/1819713512 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f1d9006c6b0 con 0x7f1d980ab970 2021-07-11T04:02:42.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.148+0000 7f1d9e416700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d78053300 0x7f1d780557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.148+0000 7f1d9e416700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d78053300 0x7f1d780557c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f1d8c007000 tx=0x7f1d8c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.148+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/1819713512 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1d900439f0 con 0x7f1d980ab970 2021-07-11T04:02:42.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 -- 172.21.2.104:0/1819713512 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1d9813b590 con 0x7f1d980ab970 2021-07-11T04:02:42.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1d8b7fe700 1 -- 172.21.2.104:0/1819713512 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v161) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1d90047010 con 0x7f1d980ab970 2021-07-11T04:02:42.232 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:42.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 -- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d78053300 msgr2=0x7f1d780557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d78053300 0x7f1d780557c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f1d8c007000 tx=0x7f1d8c008040).stop 2021-07-11T04:02:42.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 -- 172.21.2.104:0/1819713512 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d980ab970 msgr2=0x7f1d980ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d980ab970 0x7f1d980ad420 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 rx=0x7f1d900032f0 tx=0x7f1d90018000).stop 2021-07-11T04:02:42.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 -- 172.21.2.104:0/1819713512 shutdown_connections 2021-07-11T04:02:42.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1d78053300 0x7f1d780557c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.234 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f1d980ab970 0x7f1d980ad420 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f1d980b38d0 0x7f1d98143350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 --2- 172.21.2.104:0/1819713512 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1d98060100 0x7f1d98159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 -- 172.21.2.104:0/1819713512 >> 172.21.2.104:0/1819713512 conn(0x7f1d9805cf60 msgr2=0x7f1d9813fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:42.235 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 -- 172.21.2.104:0/1819713512 shutdown_connections 2021-07-11T04:02:42.236 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.232+0000 7f1da067a700 1 -- 172.21.2.104:0/1819713512 wait complete. 2021-07-11T04:02:42.239 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2021-07-11T04:02:42.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.292+0000 7f133caa1700 1 Processor -- start 2021-07-11T04:02:42.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.292+0000 7f133caa1700 1 -- start start 2021-07-11T04:02:42.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.292+0000 7f133caa1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380b1ec0 0x7f13380b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.292+0000 7f133caa1700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13380a77d0 con 0x7f13380affb0 2021-07-11T04:02:42.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.292+0000 7f133caa1700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13380b03f0 con 0x7f13380b22a0 2021-07-11T04:02:42.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.292+0000 7f133caa1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13380b0650 con 0x7f13380b1ec0 2021-07-11T04:02:42.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.292+0000 7f1335d9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f13380affb0 0x7f13380a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48800/0 (socket says 172.21.2.104:48800) 2021-07-11T04:02:42.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.292+0000 7f1335d9b700 1 -- 172.21.2.104:0/1959189360 learned_addr learned my addr 172.21.2.104:0/1959189360 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:42.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686069921 0 0) 0x7f13380a77d0 con 0x7f13380affb0 2021-07-11T04:02:42.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f131c003260 con 0x7f13380affb0 2021-07-11T04:02:42.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2269361392 0 0) 0x7f131c003260 con 0x7f13380affb0 2021-07-11T04:02:42.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380b1ec0 msgr2=0x7f13380b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:42.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380b1ec0 0x7f13380b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 >> v1:172.21.2.105:6789/0 conn(0x7f13380b22a0 legacy=0x7f1338141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:42.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13380b08f0 con 0x7f13380affb0 2021-07-11T04:02:42.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f132c001660 con 0x7f13380affb0 2021-07-11T04:02:42.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 >> v1:172.21.2.104:6789/0 conn(0x7f13380affb0 legacy=0x7f13380a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380b1ec0 0x7f131c003780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f131c003e20 0x7f131c004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f131c004b10 0x7f131c004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13380b0650 con 0x7f131c003e20 2021-07-11T04:02:42.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c00b960 con 0x7f131c004b10 2021-07-11T04:02:42.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c00bb20 con 0x7f13380b1ec0 2021-07-11T04:02:42.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f1335d9b700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f131c003e20 0x7f131c004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133659c700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380b1ec0 0x7f131c003780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f1336d9d700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f131c004b10 0x7f131c004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f1335d9b700 1 -- 172.21.2.104:0/1959189360 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380b1ec0 msgr2=0x7f131c003780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:42.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f1335d9b700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380b1ec0 0x7f131c003780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f1335d9b700 1 -- 172.21.2.104:0/1959189360 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f131c004b10 msgr2=0x7f131c004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:42.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f1335d9b700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f131c004b10 0x7f131c004f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f1335d9b700 1 -- 172.21.2.104:0/1959189360 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c003260 con 0x7f131c003e20 2021-07-11T04:02:42.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133659c700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380b1ec0 0x7f131c003780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:42.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f1335d9b700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f131c003e20 0x7f131c004500 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7f133806a190 tx=0x7f13380b2a00).ready entity=mon.0 client_cookie=1229421cdcc247aa server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f132c0071a0 con 0x7f131c003e20 2021-07-11T04:02:42.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c007300 con 0x7f131c003e20 2021-07-11T04:02:42.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133559a700 1 -- 172.21.2.104:0/1959189360 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f132c00cae0 con 0x7f131c003e20 2021-07-11T04:02:42.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133caa1700 1 -- 172.21.2.104:0/1959189360 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f131c003e20 msgr2=0x7f131c004500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133caa1700 1 --2- 172.21.2.104:0/1959189360 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f131c003e20 0x7f131c004500 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 rx=0x7f133806a190 tx=0x7f13380b2a00).stop 2021-07-11T04:02:42.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f1336d9d700 1 -- 172.21.2.104:0/1959189360 reap_dead start 2021-07-11T04:02:42.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133caa1700 1 -- 172.21.2.104:0/1959189360 shutdown_connections 2021-07-11T04:02:42.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133caa1700 1 -- 172.21.2.104:0/1959189360 >> 172.21.2.104:0/1959189360 conn(0x7f133805f6b0 msgr2=0x7f133805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:42.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133caa1700 1 -- 172.21.2.104:0/1959189360 shutdown_connections 2021-07-11T04:02:42.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133caa1700 1 -- 172.21.2.104:0/1959189360 wait complete. 2021-07-11T04:02:42.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133caa1700 1 Processor -- start 2021-07-11T04:02:42.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133caa1700 1 -- start start 2021-07-11T04:02:42.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.296+0000 7f133caa1700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13381470e0 0x7f133815a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f133caa1700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13380ac340 0x7f133815a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f133caa1700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380aa000 0x7f133813a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f133caa1700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133814ac50 con 0x7f13380ac340 2021-07-11T04:02:42.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f133caa1700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13380abf10 con 0x7f13381470e0 2021-07-11T04:02:42.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f133caa1700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13381474c0 con 0x7f13380aa000 2021-07-11T04:02:42.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f1335d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13380ac340 0x7f133815a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f1335d9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13380ac340 0x7f133815a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52042/0 (socket says 172.21.2.104:52042) 2021-07-11T04:02:42.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f1335d9b700 1 -- 172.21.2.104:0/675692189 learned_addr learned my addr 172.21.2.104:0/675692189 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:42.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f1335d9b700 1 -- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380aa000 msgr2=0x7f133813a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f1335d9b700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380aa000 0x7f133813a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f1335d9b700 1 -- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13381470e0 msgr2=0x7f133815a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f1335d9b700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13381470e0 0x7f133815a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f1335d9b700 1 -- 172.21.2.104:0/675692189 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133814ae80 con 0x7f13380ac340 2021-07-11T04:02:42.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f1335d9b700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13380ac340 0x7f133815a760 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 rx=0x7f133806a190 tx=0x7f132c013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f13277fe700 1 -- 172.21.2.104:0/675692189 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f132c001c70 con 0x7f13380ac340 2021-07-11T04:02:42.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f13277fe700 1 -- 172.21.2.104:0/675692189 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c007450 con 0x7f13380ac340 2021-07-11T04:02:42.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f13277fe700 1 -- 172.21.2.104:0/675692189 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f132c0049e0 con 0x7f13380ac340 2021-07-11T04:02:42.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f133caa1700 1 -- 172.21.2.104:0/675692189 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13380aa950 con 0x7f13380ac340 2021-07-11T04:02:42.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f133caa1700 1 -- 172.21.2.104:0/675692189 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1338149dc0 con 0x7f13380ac340 2021-07-11T04:02:42.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f133caa1700 1 -- 172.21.2.104:0/675692189 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13380ac050 con 0x7f13380ac340 2021-07-11T04:02:42.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f13277fe700 1 -- 172.21.2.104:0/675692189 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f132c01c070 con 0x7f13380ac340 2021-07-11T04:02:42.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f13277fe700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1314053380 0x7f1314055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f13277fe700 1 -- 172.21.2.104:0/675692189 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f132c06b460 con 0x7f13380ac340 2021-07-11T04:02:42.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f133659c700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1314053380 0x7f1314055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.300+0000 7f133659c700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1314053380 0x7f1314055840 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f1328007000 tx=0x7f1328008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.304+0000 7f13277fe700 1 -- 172.21.2.104:0/675692189 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f132c0427a0 con 0x7f13380ac340 2021-07-11T04:02:42.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 -- 172.21.2.104:0/675692189 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1338057350 con 0x7f13380ac340 2021-07-11T04:02:42.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f13277fe700 1 -- 172.21.2.104:0/675692189 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v161) v1 ==== 104+0+10 (secure 0 0 0) 0x7f132c045dc0 con 0x7f13380ac340 2021-07-11T04:02:42.385 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:42.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 -- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1314053380 msgr2=0x7f1314055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1314053380 0x7f1314055840 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f1328007000 tx=0x7f1328008040).stop 2021-07-11T04:02:42.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 -- 172.21.2.104:0/675692189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13380ac340 msgr2=0x7f133815a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13380ac340 0x7f133815a760 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 rx=0x7f133806a190 tx=0x7f132c013000).stop 2021-07-11T04:02:42.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 -- 172.21.2.104:0/675692189 shutdown_connections 2021-07-11T04:02:42.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f13380aa000 0x7f133813a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1314053380 0x7f1314055840 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f13380ac340 0x7f133815a760 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 --2- 172.21.2.104:0/675692189 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f13381470e0 0x7f133815a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 -- 172.21.2.104:0/675692189 >> 172.21.2.104:0/675692189 conn(0x7f133805f6b0 msgr2=0x7f13380af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:42.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 -- 172.21.2.104:0/675692189 shutdown_connections 2021-07-11T04:02:42.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.384+0000 7f133caa1700 1 -- 172.21.2.104:0/675692189 wait complete. 2021-07-11T04:02:42.395 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2021-07-11T04:02:42.448 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 Processor -- start 2021-07-11T04:02:42.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 -- start start 2021-07-11T04:02:42.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f085c0603d0 0x7f085c0aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.449 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f085c061990 con 0x7f085c0a9ec0 2021-07-11T04:02:42.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f085c061c30 con 0x7f085c13f790 2021-07-11T04:02:42.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f085c060b90 con 0x7f085c0603d0 2021-07-11T04:02:42.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0860c9b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f085c0a9ec0 0x7f085c13ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48816/0 (socket says 172.21.2.104:48816) 2021-07-11T04:02:42.450 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0860c9b700 1 -- 172.21.2.104:0/3230073644 learned_addr learned my addr 172.21.2.104:0/3230073644 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:42.451 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f085bfff700 1 --2- 172.21.2.104:0/3230073644 >> v2:172.21.2.104:3300/0 conn(0x7f085c0603d0 0x7f085c0aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f085bfff700 1 -- 172.21.2.104:0/3230073644 >> v1:172.21.2.105:6789/0 conn(0x7f085c13f790 legacy=0x7f085c14a020 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:42.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f085bfff700 1 -- 172.21.2.104:0/3230073644 >> v1:172.21.2.104:6789/0 conn(0x7f085c0a9ec0 legacy=0x7f085c13ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f085bfff700 1 -- 172.21.2.104:0/3230073644 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c060e30 con 0x7f085c0603d0 2021-07-11T04:02:42.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f085bfff700 1 --2- 172.21.2.104:0/3230073644 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f085c0603d0 0x7f085c0aa2a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7f085c0aaaf0 tx=0x7f0850006e80).ready entity=mon.0 client_cookie=669fb67fe48fb0dd server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f085b7fe700 1 -- 172.21.2.104:0/3230073644 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f085000a880 con 0x7f085c0603d0 2021-07-11T04:02:42.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f085b7fe700 1 -- 172.21.2.104:0/3230073644 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0850000e90 con 0x7f085c0603d0 2021-07-11T04:02:42.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f085b7fe700 1 -- 172.21.2.104:0/3230073644 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f085000abd0 con 0x7f085c0603d0 2021-07-11T04:02:42.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 -- 172.21.2.104:0/3230073644 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f085c0603d0 msgr2=0x7f085c0aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 --2- 172.21.2.104:0/3230073644 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f085c0603d0 0x7f085c0aa2a0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 rx=0x7f085c0aaaf0 tx=0x7f0850006e80).stop 2021-07-11T04:02:42.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 -- 172.21.2.104:0/3230073644 shutdown_connections 2021-07-11T04:02:42.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 --2- 172.21.2.104:0/3230073644 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f085c0603d0 0x7f085c0aa2a0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 -- 172.21.2.104:0/3230073644 >> 172.21.2.104:0/3230073644 conn(0x7f085c0b2970 msgr2=0x7f085c0b2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:42.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 -- 172.21.2.104:0/3230073644 shutdown_connections 2021-07-11T04:02:42.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.448+0000 7f0862eff700 1 -- 172.21.2.104:0/3230073644 wait complete. 2021-07-11T04:02:42.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0862eff700 1 Processor -- start 2021-07-11T04:02:42.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0862eff700 1 -- start start 2021-07-11T04:02:42.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0862eff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f085c0603d0 0x7f085c13c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0862eff700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f085c0a9ec0 0x7f085c0ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0862eff700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f085c13f790 0x7f085c0aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.462 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0862eff700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c14ef10 con 0x7f085c0603d0 2021-07-11T04:02:42.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0862eff700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c14f1b0 con 0x7f085c0a9ec0 2021-07-11T04:02:42.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0862eff700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0ae2c0 con 0x7f085c13f790 2021-07-11T04:02:42.463 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0860c9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f085c0603d0 0x7f085c13c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0860c9b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f085c0603d0 0x7f085c13c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52054/0 (socket says 172.21.2.104:52054) 2021-07-11T04:02:42.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0860c9b700 1 -- 172.21.2.104:0/2001995576 learned_addr learned my addr 172.21.2.104:0/2001995576 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:42.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0860c9b700 1 -- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f085c13f790 msgr2=0x7f085c0aec50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:42.464 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0860c9b700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f085c13f790 0x7f085c0aec50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0860c9b700 1 -- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f085c0a9ec0 msgr2=0x7f085c0ae760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:02:42.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0860c9b700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f085c0a9ec0 0x7f085c0ae760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0860c9b700 1 -- 172.21.2.104:0/2001995576 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085000c310 con 0x7f085c0603d0 2021-07-11T04:02:42.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0860c9b700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f085c0603d0 0x7f085c13c720 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 rx=0x7f085c0abc40 tx=0x7f084c007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.465 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0859ffb700 1 -- 172.21.2.104:0/2001995576 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f084c00ce30 con 0x7f085c0603d0 2021-07-11T04:02:42.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0859ffb700 1 -- 172.21.2.104:0/2001995576 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c0072e0 con 0x7f085c0603d0 2021-07-11T04:02:42.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0859ffb700 1 -- 172.21.2.104:0/2001995576 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f084c005800 con 0x7f085c0603d0 2021-07-11T04:02:42.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0862eff700 1 -- 172.21.2.104:0/2001995576 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085c1419c0 con 0x7f085c0603d0 2021-07-11T04:02:42.466 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0862eff700 1 -- 172.21.2.104:0/2001995576 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c0af2e0 con 0x7f085c0603d0 2021-07-11T04:02:42.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0862eff700 1 -- 172.21.2.104:0/2001995576 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085c057350 con 0x7f085c0603d0 2021-07-11T04:02:42.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0859ffb700 1 -- 172.21.2.104:0/2001995576 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f084c010070 con 0x7f085c0603d0 2021-07-11T04:02:42.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0859ffb700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f083c0532b0 0x7f083c055770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f0859ffb700 1 -- 172.21.2.104:0/2001995576 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f084c06aa40 con 0x7f085c0603d0 2021-07-11T04:02:42.467 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.452+0000 7f085bfff700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f083c0532b0 0x7f083c055770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.456+0000 7f085bfff700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f083c0532b0 0x7f083c055770 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f085c0aaaf0 tx=0x7f0850017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.468 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.456+0000 7f0859ffb700 1 -- 172.21.2.104:0/2001995576 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f084c041d80 con 0x7f085c0603d0 2021-07-11T04:02:42.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.536+0000 7f0862eff700 1 -- 172.21.2.104:0/2001995576 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f085c0ae430 con 0x7f085c0603d0 2021-07-11T04:02:42.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0859ffb700 1 -- 172.21.2.104:0/2001995576 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v161) v1 ==== 100+0+10 (secure 0 0 0) 0x7f084c018030 con 0x7f085c0603d0 2021-07-11T04:02:42.539 INFO:teuthology.orchestra.run.gibba004.stdout:pg_num: 8 2021-07-11T04:02:42.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 -- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f083c0532b0 msgr2=0x7f083c055770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f083c0532b0 0x7f083c055770 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f085c0aaaf0 tx=0x7f0850017000).stop 2021-07-11T04:02:42.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 -- 172.21.2.104:0/2001995576 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f085c0603d0 msgr2=0x7f085c13c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f085c0603d0 0x7f085c13c720 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 rx=0x7f085c0abc40 tx=0x7f084c007c00).stop 2021-07-11T04:02:42.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 -- 172.21.2.104:0/2001995576 shutdown_connections 2021-07-11T04:02:42.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f085c13f790 0x7f085c0aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f085c0a9ec0 0x7f085c0ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f083c0532b0 0x7f083c055770 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 --2- 172.21.2.104:0/2001995576 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f085c0603d0 0x7f085c13c720 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 -- 172.21.2.104:0/2001995576 >> 172.21.2.104:0/2001995576 conn(0x7f085c0b2970 msgr2=0x7f085c0b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:42.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 -- 172.21.2.104:0/2001995576 shutdown_connections 2021-07-11T04:02:42.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.540+0000 7f0862eff700 1 -- 172.21.2.104:0/2001995576 wait complete. 2021-07-11T04:02:42.546 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2021-07-11T04:02:42.547 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:42.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.596+0000 7f22ebb25700 1 Processor -- start 2021-07-11T04:02:42.605 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 -- start start 2021-07-11T04:02:42.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22e40a76b0 0x7f22e4066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40a7a90 con 0x7f22e40b0060 2021-07-11T04:02:42.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e4147620 con 0x7f22e40b1970 2021-07-11T04:02:42.606 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e41478c0 con 0x7f22e40a76b0 2021-07-11T04:02:42.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ea0c2700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f22e40b0060 0x7f22e4151f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48830/0 (socket says 172.21.2.104:48830) 2021-07-11T04:02:42.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ea0c2700 1 -- 172.21.2.104:0/1176734528 learned_addr learned my addr 172.21.2.104:0/1176734528 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:42.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335106188 0 0) 0x7f22e40a7a90 con 0x7f22e40b0060 2021-07-11T04:02:42.607 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22cc003260 con 0x7f22e40b0060 2021-07-11T04:02:42.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3408693260 0 0) 0x7f22cc003260 con 0x7f22e40b0060 2021-07-11T04:02:42.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22e40a76b0 msgr2=0x7f22e4066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:42.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 --2- 172.21.2.104:0/1176734528 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22e40a76b0 0x7f22e4066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 >> v1:172.21.2.105:6789/0 conn(0x7f22e40b1970 legacy=0x7f22e405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:42.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4147b60 con 0x7f22e40b0060 2021-07-11T04:02:42.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f22dc0019e0 con 0x7f22e40b0060 2021-07-11T04:02:42.609 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 >> v1:172.21.2.104:6789/0 conn(0x7f22e40b0060 legacy=0x7f22e4151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 --2- 172.21.2.104:0/1176734528 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22cc003800 0x7f22cc003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 --2- 172.21.2.104:0/1176734528 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22cc004500 0x7f22cc004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 --2- 172.21.2.104:0/1176734528 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22e40a76b0 0x7f22cc00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.610 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e41478c0 con 0x7f22cc003800 2021-07-11T04:02:42.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc005290 con 0x7f22cc004500 2021-07-11T04:02:42.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc00bc40 con 0x7f22e40a76b0 2021-07-11T04:02:42.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e98c1700 1 --2- 172.21.2.104:0/1176734528 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22cc003800 0x7f22cc003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e98c1700 1 -- 172.21.2.104:0/1176734528 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22e40a76b0 msgr2=0x7f22cc00b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:42.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e98c1700 1 --2- 172.21.2.104:0/1176734528 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22e40a76b0 0x7f22cc00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e98c1700 1 -- 172.21.2.104:0/1176734528 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22cc004500 msgr2=0x7f22cc004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:42.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e98c1700 1 --2- 172.21.2.104:0/1176734528 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22cc004500 0x7f22cc004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e98c1700 1 -- 172.21.2.104:0/1176734528 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc003260 con 0x7f22cc003800 2021-07-11T04:02:42.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e98c1700 1 --2- 172.21.2.104:0/1176734528 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22cc003800 0x7f22cc003c00 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 rx=0x7f22e0002f70 tx=0x7f22e0006ca0).ready entity=mon.0 client_cookie=e36b897cc4742ee6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22e0012410 con 0x7f22cc003800 2021-07-11T04:02:42.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e0012570 con 0x7f22cc003800 2021-07-11T04:02:42.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22e88bf700 1 -- 172.21.2.104:0/1176734528 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22e00128d0 con 0x7f22cc003800 2021-07-11T04:02:42.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 -- 172.21.2.104:0/1176734528 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22cc003800 msgr2=0x7f22cc003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 --2- 172.21.2.104:0/1176734528 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22cc003800 0x7f22cc003c00 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 rx=0x7f22e0002f70 tx=0x7f22e0006ca0).stop 2021-07-11T04:02:42.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ea0c2700 1 -- 172.21.2.104:0/1176734528 reap_dead start 2021-07-11T04:02:42.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 -- 172.21.2.104:0/1176734528 shutdown_connections 2021-07-11T04:02:42.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 -- 172.21.2.104:0/1176734528 >> 172.21.2.104:0/1176734528 conn(0x7f22e40b53e0 msgr2=0x7f22e40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:42.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 -- 172.21.2.104:0/1176734528 shutdown_connections 2021-07-11T04:02:42.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 -- 172.21.2.104:0/1176734528 wait complete. 2021-07-11T04:02:42.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 Processor -- start 2021-07-11T04:02:42.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.600+0000 7f22ebb25700 1 -- start start 2021-07-11T04:02:42.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22ebb25700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22e4146790 0x7f22e4144740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22ebb25700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22e40ac090 0x7f22e4144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22ebb25700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22e40af4f0 0x7f22e40ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22ebb25700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4146b70 con 0x7f22e40ac090 2021-07-11T04:02:42.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22ebb25700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40aadb0 con 0x7f22e40af4f0 2021-07-11T04:02:42.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22ebb25700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40ab050 con 0x7f22e4146790 2021-07-11T04:02:42.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22e90c0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22e40ac090 0x7f22e4144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22e90c0700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22e40ac090 0x7f22e4144c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52078/0 (socket says 172.21.2.104:52078) 2021-07-11T04:02:42.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22e90c0700 1 -- 172.21.2.104:0/1264837469 learned_addr learned my addr 172.21.2.104:0/1264837469 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:42.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22e90c0700 1 -- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22e4146790 msgr2=0x7f22e4144740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22e90c0700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22e4146790 0x7f22e4144740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22e90c0700 1 -- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22e40af4f0 msgr2=0x7f22e40ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22e90c0700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22e40af4f0 0x7f22e40ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22e90c0700 1 -- 172.21.2.104:0/1264837469 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40ab3c0 con 0x7f22e40ac090 2021-07-11T04:02:42.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22e90c0700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22e40ac090 0x7f22e4144c30 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7f22d4000f70 tx=0x7f22d4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22d2ffd700 1 -- 172.21.2.104:0/1264837469 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22d4013460 con 0x7f22e40ac090 2021-07-11T04:02:42.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22d2ffd700 1 -- 172.21.2.104:0/1264837469 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d40135c0 con 0x7f22e40ac090 2021-07-11T04:02:42.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22ebb25700 1 -- 172.21.2.104:0/1264837469 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e40ab660 con 0x7f22e40ac090 2021-07-11T04:02:42.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22ebb25700 1 -- 172.21.2.104:0/1264837469 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e40af220 con 0x7f22e40ac090 2021-07-11T04:02:42.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22d2ffd700 1 -- 172.21.2.104:0/1264837469 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f22d4013920 con 0x7f22e40ac090 2021-07-11T04:02:42.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22ebb25700 1 -- 172.21.2.104:0/1264837469 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e40ab7f0 con 0x7f22e40ac090 2021-07-11T04:02:42.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22d2ffd700 1 -- 172.21.2.104:0/1264837469 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f22d401a060 con 0x7f22e40ac090 2021-07-11T04:02:42.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22d2ffd700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22c4053240 0x7f22c4055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:42.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22d2ffd700 1 -- 172.21.2.104:0/1264837469 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f22d406af80 con 0x7f22e40ac090 2021-07-11T04:02:42.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.604+0000 7f22e98c1700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22c4053240 0x7f22c4055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:42.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.608+0000 7f22e98c1700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22c4053240 0x7f22c4055700 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f22e0002f70 tx=0x7f22e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:42.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.608+0000 7f22d2ffd700 1 -- 172.21.2.104:0/1264837469 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f22d40421b0 con 0x7f22e40ac090 2021-07-11T04:02:42.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 -- 172.21.2.104:0/1264837469 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22e4057350 con 0x7f22e40ac090 2021-07-11T04:02:42.714 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22d2ffd700 1 -- 172.21.2.104:0/1264837469 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f22d4018020 con 0x7f22e40ac090 2021-07-11T04:02:42.715 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:42.715 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":80,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5132,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/770706140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":770706140},{"type":"v1","addr":"172.21.2.105:6835","nonce":770706140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":79},{"gid":8718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/488029782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":488029782},{"type":"v1","addr":"172.21.2.104:6833","nonce":488029782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:36.673961+0000","modified":"2021-07-11T04:02:37.679104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8721":{"gid":8721,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/2638676458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2638676458},{"type":"v1","addr":"172.21.2.104:6835","nonce":2638676458}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-07-11T04:02:42.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 -- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22c4053240 msgr2=0x7f22c4055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.716 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22c4053240 0x7f22c4055700 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f22e0002f70 tx=0x7f22e0015000).stop 2021-07-11T04:02:42.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 -- 172.21.2.104:0/1264837469 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22e40ac090 msgr2=0x7f22e4144c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:42.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22e40ac090 0x7f22e4144c30 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7f22d4000f70 tx=0x7f22d4006df0).stop 2021-07-11T04:02:42.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 -- 172.21.2.104:0/1264837469 shutdown_connections 2021-07-11T04:02:42.717 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f22e40af4f0 0x7f22e40ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f22c4053240 0x7f22c4055700 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f22e40ac090 0x7f22e4144c30 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:42.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 --2- 172.21.2.104:0/1264837469 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f22e4146790 0x7f22e4144740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:42.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 -- 172.21.2.104:0/1264837469 >> 172.21.2.104:0/1264837469 conn(0x7f22e40b53e0 msgr2=0x7f22e40b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:42.718 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 -- 172.21.2.104:0/1264837469 shutdown_connections 2021-07-11T04:02:42.719 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:42.716+0000 7f22ebb25700 1 -- 172.21.2.104:0/1264837469 wait complete. 2021-07-11T04:02:42.719 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 80 2021-07-11T04:02:43.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2021-07-11T04:02:36.673961+0000', 'modified': '2021-07-11T04:02:37.679104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'min_compat_client': '0 (unknown)', 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8721}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8721': {'gid': 8721, 'name': 'c', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.2.104:6835/2638676458', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.2.104:6834', 'nonce': 2638676458}, {'type': 'v1', 'addr': '172.21.2.104:6835', 'nonce': 2638676458}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138292840890367, 'flags': 0}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2021-07-11T04:02:43.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2021-07-11T04:02:43.026 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2021-07-11T04:02:43.096 INFO:teuthology.orchestra.run.gibba004.stdout:{"cluster_fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","whoami":0,"id":8721,"want_state":"up:active","state":"up:active","rank_uptime":6.4172759700000004,"mdsmap_epoch":80,"osdmap_epoch":161,"osdmap_epoch_barrier":160,"uptime":10.585770740999999} 2021-07-11T04:02:43.097 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", "id": 8721, "mdsmap_epoch": 80, "osdmap_epoch": 161, "osdmap_epoch_barrier": 160, "rank_uptime": 6.41727597, "state": "up:active", "uptime": 10.585770741, "want_state": "up:active", "whoami": 0 } 2021-07-11T04:02:43.097 INFO:tasks.cephfs.mount:Ready to start FuseMount... 2021-07-11T04:02:43.098 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2021-07-11T04:02:43.098 INFO:tasks.cephfs.fuse_mount:Mounting ceph-fuse client.1 at ubuntu@gibba023.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.1... 2021-07-11T04:02:43.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:43.099 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec mkdir -m 0000 -p /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:43.110 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:02:43.122 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:43.123 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:43.158 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:02:43.160 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:02:43.163 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:02:43.167 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:02:43.172 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:43.173 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:43.174 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:43.174 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:02:43.179 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2021-07-11T04:02:43.179 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --name client.1 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:43.245 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:02:43.257 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:43.258 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:43.263 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:43.263 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:43.264 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:43.265 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:02:43.289 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15366]: starting ceph client 2021-07-11T04:02:43.289 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:2021-07-11T04:02:43.282+0000 7fd493d53080 -1 init, newargv = 0x55cae0784b00 newargc=9 2021-07-11T04:02:43.300 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15366]: starting fuse 2021-07-11T04:02:44.369 DEBUG:teuthology.orchestra.run.gibba023:> sudo modprobe fuse 2021-07-11T04:02:44.383 INFO:teuthology.orchestra.run:Running command with timeout 30 2021-07-11T04:02:44.384 DEBUG:teuthology.orchestra.run.gibba023:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2021-07-11T04:02:44.434 INFO:teuthology.orchestra.run.gibba023.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2021-07-11T04:02:44.435 DEBUG:teuthology.orchestra.run:got remote process result: 32 2021-07-11T04:02:44.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:44.436 DEBUG:teuthology.orchestra.run.gibba023:> ls /sys/fs/fuse/connections 2021-07-11T04:02:44.441 INFO:teuthology.orchestra.run.gibba023.stdout:52 2021-07-11T04:02:44.442 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [52] 2021-07-11T04:02:44.442 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:44.443 DEBUG:teuthology.orchestra.run.gibba023:> sudo python3 -c ' 2021-07-11T04:02:44.443 DEBUG:teuthology.orchestra.run.gibba023:> import glob 2021-07-11T04:02:44.443 DEBUG:teuthology.orchestra.run.gibba023:> import re 2021-07-11T04:02:44.444 DEBUG:teuthology.orchestra.run.gibba023:> import os 2021-07-11T04:02:44.444 DEBUG:teuthology.orchestra.run.gibba023:> import subprocess 2021-07-11T04:02:44.445 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:44.445 DEBUG:teuthology.orchestra.run.gibba023:> def find_socket(client_name): 2021-07-11T04:02:44.445 DEBUG:teuthology.orchestra.run.gibba023:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2021-07-11T04:02:44.446 DEBUG:teuthology.orchestra.run.gibba023:> files = glob.glob(asok_path) 2021-07-11T04:02:44.446 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:44.447 DEBUG:teuthology.orchestra.run.gibba023:> # Given a non-glob path, it better be there 2021-07-11T04:02:44.447 DEBUG:teuthology.orchestra.run.gibba023:> if "*" not in asok_path: 2021-07-11T04:02:44.447 DEBUG:teuthology.orchestra.run.gibba023:> assert(len(files) == 1) 2021-07-11T04:02:44.448 DEBUG:teuthology.orchestra.run.gibba023:> return files[0] 2021-07-11T04:02:44.448 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:44.449 DEBUG:teuthology.orchestra.run.gibba023:> for f in files: 2021-07-11T04:02:44.449 DEBUG:teuthology.orchestra.run.gibba023:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2021-07-11T04:02:44.449 DEBUG:teuthology.orchestra.run.gibba023:> if os.path.exists("/proc/{0}".format(pid)): 2021-07-11T04:02:44.450 DEBUG:teuthology.orchestra.run.gibba023:> return f 2021-07-11T04:02:44.450 DEBUG:teuthology.orchestra.run.gibba023:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2021-07-11T04:02:44.451 DEBUG:teuthology.orchestra.run.gibba023:> 2021-07-11T04:02:44.451 DEBUG:teuthology.orchestra.run.gibba023:> print(find_socket("client.1")) 2021-07-11T04:02:44.451 DEBUG:teuthology.orchestra.run.gibba023:> ' 2021-07-11T04:02:44.478 INFO:teuthology.orchestra.run.gibba023.stdout:/var/run/ceph/ceph-client.1.15366.asok 2021-07-11T04:02:44.480 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.15366.asok 2021-07-11T04:02:44.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:44.481 DEBUG:teuthology.orchestra.run.gibba023:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.15366.asok status 2021-07-11T04:02:44.589 INFO:teuthology.orchestra.run.gibba023.stdout:{ 2021-07-11T04:02:44.589 INFO:teuthology.orchestra.run.gibba023.stdout: "metadata": { 2021-07-11T04:02:44.589 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_sha1": "45c6565d67f116c9b3509842b718398a5b38cf73", 2021-07-11T04:02:44.590 INFO:teuthology.orchestra.run.gibba023.stdout: "ceph_version": "ceph version 15.2.13-130-g45c6565d (45c6565d67f116c9b3509842b718398a5b38cf73) octopus (stable)", 2021-07-11T04:02:44.590 INFO:teuthology.orchestra.run.gibba023.stdout: "entity_id": "1", 2021-07-11T04:02:44.590 INFO:teuthology.orchestra.run.gibba023.stdout: "hostname": "gibba023", 2021-07-11T04:02:44.590 INFO:teuthology.orchestra.run.gibba023.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2021-07-11T04:02:44.591 INFO:teuthology.orchestra.run.gibba023.stdout: "pid": "15366", 2021-07-11T04:02:44.591 INFO:teuthology.orchestra.run.gibba023.stdout: "root": "/" 2021-07-11T04:02:44.591 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:44.591 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_count": 0, 2021-07-11T04:02:44.591 INFO:teuthology.orchestra.run.gibba023.stdout: "dentry_pinned_count": 0, 2021-07-11T04:02:44.592 INFO:teuthology.orchestra.run.gibba023.stdout: "id": 5590, 2021-07-11T04:02:44.592 INFO:teuthology.orchestra.run.gibba023.stdout: "inst": { 2021-07-11T04:02:44.592 INFO:teuthology.orchestra.run.gibba023.stdout: "name": { 2021-07-11T04:02:44.592 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "client", 2021-07-11T04:02:44.593 INFO:teuthology.orchestra.run.gibba023.stdout: "num": 5590 2021-07-11T04:02:44.593 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:44.593 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:02:44.593 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:02:44.593 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:02:44.594 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 2674548137 2021-07-11T04:02:44.594 INFO:teuthology.orchestra.run.gibba023.stdout: } 2021-07-11T04:02:44.594 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:44.594 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": { 2021-07-11T04:02:44.595 INFO:teuthology.orchestra.run.gibba023.stdout: "type": "v1", 2021-07-11T04:02:44.595 INFO:teuthology.orchestra.run.gibba023.stdout: "addr": "172.21.2.123:0", 2021-07-11T04:02:44.595 INFO:teuthology.orchestra.run.gibba023.stdout: "nonce": 2674548137 2021-07-11T04:02:44.595 INFO:teuthology.orchestra.run.gibba023.stdout: }, 2021-07-11T04:02:44.595 INFO:teuthology.orchestra.run.gibba023.stdout: "inst_str": "client.5590 172.21.2.123:0/2674548137", 2021-07-11T04:02:44.596 INFO:teuthology.orchestra.run.gibba023.stdout: "addr_str": "172.21.2.123:0/2674548137", 2021-07-11T04:02:44.596 INFO:teuthology.orchestra.run.gibba023.stdout: "inode_count": 1, 2021-07-11T04:02:44.596 INFO:teuthology.orchestra.run.gibba023.stdout: "mds_epoch": 80, 2021-07-11T04:02:44.596 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch": 161, 2021-07-11T04:02:44.596 INFO:teuthology.orchestra.run.gibba023.stdout: "osd_epoch_barrier": 0, 2021-07-11T04:02:44.597 INFO:teuthology.orchestra.run.gibba023.stdout: "blacklisted": false, 2021-07-11T04:02:44.597 INFO:teuthology.orchestra.run.gibba023.stdout: "fs_name": "cephfs" 2021-07-11T04:02:44.599 INFO:teuthology.orchestra.run.gibba023.stdout:} 2021-07-11T04:02:44.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:44.601 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:44.601 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:44.607 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:02:44.607 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:44.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:44.608 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo chmod 1777 /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:44.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:44.617 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:44.618 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:44.666 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:02:44.666 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:44.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:44.667 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:44.667 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:44.673 INFO:teuthology.orchestra.run.gibba023.stdout:fuseblk 2021-07-11T04:02:44.674 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2021-07-11T04:02:44.674 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@gibba023.front.sepia.ceph.com... 2021-07-11T04:02:44.675 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-07-11T04:02:44.675 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec sudo fusermount -u /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:44.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.gibba023.stderr:ceph-fuse[15366]: fuse finished with error 0 and tester_r 0 2021-07-11T04:02:44.685 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:02:44.686 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:02:44.686 DEBUG:teuthology.orchestra.run.gibba023:> ' -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:44.736 INFO:teuthology.orchestra.run.gibba023.stdout:ext2/ext3 2021-07-11T04:02:44.737 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse not mounted, got fs type 'ext2/ext3' 2021-07-11T04:02:44.738 INFO:teuthology.orchestra.run:waiting for 900 2021-07-11T04:02:50.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-07-11T04:02:50.840 DEBUG:teuthology.orchestra.run.gibba023:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2021-07-11T04:02:50.850 INFO:teuthology.orchestra.run:Running command with timeout 10800 2021-07-11T04:02:50.851 DEBUG:teuthology.orchestra.run.gibba019:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2021-07-11T04:02:50.870 INFO:teuthology.orchestra.run.gibba019.stdout: File: "." 2021-07-11T04:02:50.870 INFO:teuthology.orchestra.run.gibba019.stdout: ID: 0 Namelen: 255 Type: fuseblk 2021-07-11T04:02:50.871 INFO:teuthology.orchestra.run.gibba019.stdout:Block size: 4194304 Fundamental block size: 4194304 2021-07-11T04:02:50.871 INFO:teuthology.orchestra.run.gibba019.stdout:Blocks: Total: 86526 Free: 86526 Available: 86526 2021-07-11T04:02:50.871 INFO:teuthology.orchestra.run.gibba019.stdout:Inodes: Total: 1 Free: 0 2021-07-11T04:02:50.872 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2021-07-11T04:02:50.928 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faab0161700 1 Processor -- start 2021-07-11T04:02:50.929 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faab0161700 1 -- start start 2021-07-11T04:02:50.930 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faab0161700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faaa80b2200 0x7faaa80abe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:50.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faab0161700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7faaa80affd0 0x7faaa80ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:50.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faab0161700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faaa81416a0 0x7faaa8141b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:50.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faab0161700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa80ac7e0 con 0x7faaa80b2200 2021-07-11T04:02:50.931 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faab0161700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa8145250 con 0x7faaa80affd0 2021-07-11T04:02:50.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faab0161700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81454f0 con 0x7faaa81416a0 2021-07-11T04:02:50.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faaadefd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faaa80b2200 0x7faaa80abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:50.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faaadefd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7faaa80b2200 0x7faaa80abe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52082/0 (socket says 172.21.2.104:52082) 2021-07-11T04:02:50.932 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faaadefd700 1 -- 172.21.2.104:0/716611752 learned_addr learned my addr 172.21.2.104:0/716611752 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:50.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faaadefd700 1 -- 172.21.2.104:0/716611752 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faaa81416a0 msgr2=0x7faaa8141b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:50.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faaadefd700 1 --2- 172.21.2.104:0/716611752 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faaa81416a0 0x7faaa8141b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:50.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faaadefd700 1 -- 172.21.2.104:0/716611752 >> v2:172.21.2.105:3300/0 conn(0x7faaa80affd0 msgr2=0x7faaa80ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:50.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faaadefd700 1 --2- 172.21.2.104:0/716611752 >> v2:172.21.2.105:3300/0 conn(0x7faaa80affd0 0x7faaa80ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:50.933 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faaadefd700 1 -- 172.21.2.104:0/716611752 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa8141200 con 0x7faaa80b2200 2021-07-11T04:02:50.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.928+0000 7faaadefd700 1 --2- 172.21.2.104:0/716611752 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faaa80b2200 0x7faaa80abe00 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 rx=0x7faa98002f70 tx=0x7faa98006eb0).ready entity=mon.0 client_cookie=21f6cc1e1bcb00cf server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:50.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaacefb700 1 -- 172.21.2.104:0/716611752 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa9800d570 con 0x7faaa80b2200 2021-07-11T04:02:50.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaacefb700 1 -- 172.21.2.104:0/716611752 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa98000e90 con 0x7faaa80b2200 2021-07-11T04:02:50.934 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaacefb700 1 -- 172.21.2.104:0/716611752 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faa9800d900 con 0x7faaa80b2200 2021-07-11T04:02:50.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- 172.21.2.104:0/716611752 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faaa80b2200 msgr2=0x7faaa80abe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:50.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 --2- 172.21.2.104:0/716611752 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faaa80b2200 0x7faaa80abe00 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 rx=0x7faa98002f70 tx=0x7faa98006eb0).stop 2021-07-11T04:02:50.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- 172.21.2.104:0/716611752 shutdown_connections 2021-07-11T04:02:50.935 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 --2- 172.21.2.104:0/716611752 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faaa81416a0 0x7faaa8141b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:50.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 --2- 172.21.2.104:0/716611752 >> v2:172.21.2.105:3300/0 conn(0x7faaa80affd0 0x7faaa80ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:50.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 --2- 172.21.2.104:0/716611752 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faaa80b2200 0x7faaa80abe00 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:50.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- 172.21.2.104:0/716611752 >> 172.21.2.104:0/716611752 conn(0x7faaa805f6b0 msgr2=0x7faaa805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:50.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- 172.21.2.104:0/716611752 shutdown_connections 2021-07-11T04:02:50.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- 172.21.2.104:0/716611752 wait complete. 2021-07-11T04:02:50.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 Processor -- start 2021-07-11T04:02:50.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- start start 2021-07-11T04:02:50.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faaa80affd0 0x7faaa80b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:50.941 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faaa80b2200 0x7faaa813df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:50.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faaa81416a0 0x7faaa8150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:50.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80677f0 con 0x7faaa81416a0 2021-07-11T04:02:50.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80b3450 con 0x7faaa80affd0 2021-07-11T04:02:50.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa813c390 con 0x7faaa80b2200 2021-07-11T04:02:50.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaae6fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faaa81416a0 0x7faaa8150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:50.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaae6fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faaa81416a0 0x7faaa8150680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52092/0 (socket says 172.21.2.104:52092) 2021-07-11T04:02:50.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaae6fe700 1 -- 172.21.2.104:0/435196218 learned_addr learned my addr 172.21.2.104:0/435196218 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:50.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaae6fe700 1 -- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faaa80b2200 msgr2=0x7faaa813df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:50.943 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaae6fe700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faaa80b2200 0x7faaa813df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:50.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaae6fe700 1 -- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faaa80affd0 msgr2=0x7faaa80b4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:50.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaae6fe700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faaa80affd0 0x7faaa80b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:50.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaae6fe700 1 -- 172.21.2.104:0/435196218 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9800b1e0 con 0x7faaa81416a0 2021-07-11T04:02:50.944 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faaae6fe700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faaa81416a0 0x7faaa8150680 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 rx=0x7faaa00032f0 tx=0x7faaa0006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:50.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faa96ffd700 1 -- 172.21.2.104:0/435196218 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaa000f510 con 0x7faaa81416a0 2021-07-11T04:02:50.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faa96ffd700 1 -- 172.21.2.104:0/435196218 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa000f670 con 0x7faaa81416a0 2021-07-11T04:02:50.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- 172.21.2.104:0/435196218 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa8146b80 con 0x7faaa81416a0 2021-07-11T04:02:50.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- 172.21.2.104:0/435196218 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa8147020 con 0x7faaa81416a0 2021-07-11T04:02:50.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faa96ffd700 1 -- 172.21.2.104:0/435196218 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7faaa000f7e0 con 0x7faaa81416a0 2021-07-11T04:02:50.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faab0161700 1 -- 172.21.2.104:0/435196218 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa813c590 con 0x7faaa81416a0 2021-07-11T04:02:50.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faa96ffd700 1 -- 172.21.2.104:0/435196218 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7faaa000f070 con 0x7faaa81416a0 2021-07-11T04:02:50.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faa96ffd700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa88053240 0x7faa88055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:50.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.932+0000 7faa96ffd700 1 -- 172.21.2.104:0/435196218 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7faaa006b200 con 0x7faaa81416a0 2021-07-11T04:02:50.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.936+0000 7faaadefd700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa88053240 0x7faa88055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:50.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.936+0000 7faaadefd700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa88053240 0x7faa88055700 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7faa98002f70 tx=0x7faa98015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:50.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:50.936+0000 7faa96ffd700 1 -- 172.21.2.104:0/435196218 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7faaa0042430 con 0x7faaa81416a0 2021-07-11T04:02:51.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.016+0000 7faab0161700 1 -- 172.21.2.104:0/435196218 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7faaa8062960 con 0x7faaa81416a0 2021-07-11T04:02:51.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faa96ffd700 1 -- 172.21.2.104:0/435196218 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v161) v1 ==== 81+0+33659 (secure 0 0 0) 0x7faaa0045a50 con 0x7faaa81416a0 2021-07-11T04:02:51.019 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:51.019 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:02:51.020 INFO:teuthology.orchestra.run.gibba004.stdout: "epoch": 161, 2021-07-11T04:02:51.020 INFO:teuthology.orchestra.run.gibba004.stdout: "fsid": "57c05e2e-f3b6-453c-878e-145bef41bd6c", 2021-07-11T04:02:51.020 INFO:teuthology.orchestra.run.gibba004.stdout: "created": "2021-07-11T03:53:08.172646+0000", 2021-07-11T04:02:51.020 INFO:teuthology.orchestra.run.gibba004.stdout: "modified": "2021-07-11T04:02:38.690916+0000", 2021-07-11T04:02:51.020 INFO:teuthology.orchestra.run.gibba004.stdout: "last_up_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:02:51.021 INFO:teuthology.orchestra.run.gibba004.stdout: "last_in_change": "2021-07-11T03:53:14.396616+0000", 2021-07-11T04:02:51.021 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2021-07-11T04:02:51.021 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_num": 5799936, 2021-07-11T04:02:51.021 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_set": [ 2021-07-11T04:02:51.022 INFO:teuthology.orchestra.run.gibba004.stdout: "pglog_hardlimit", 2021-07-11T04:02:51.022 INFO:teuthology.orchestra.run.gibba004.stdout: "purged_snapdirs", 2021-07-11T04:02:51.022 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_deletes", 2021-07-11T04:02:51.022 INFO:teuthology.orchestra.run.gibba004.stdout: "sortbitwise" 2021-07-11T04:02:51.023 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:51.023 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_version": 4, 2021-07-11T04:02:51.023 INFO:teuthology.orchestra.run.gibba004.stdout: "full_ratio": 0.94999998807907104, 2021-07-11T04:02:51.023 INFO:teuthology.orchestra.run.gibba004.stdout: "backfillfull_ratio": 0.89999997615814209, 2021-07-11T04:02:51.023 INFO:teuthology.orchestra.run.gibba004.stdout: "nearfull_ratio": 0.85000002384185791, 2021-07-11T04:02:51.024 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_snapshot": "", 2021-07-11T04:02:51.024 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_max": 30, 2021-07-11T04:02:51.024 INFO:teuthology.orchestra.run.gibba004.stdout: "max_osd": 8, 2021-07-11T04:02:51.024 INFO:teuthology.orchestra.run.gibba004.stdout: "require_min_compat_client": "jewel", 2021-07-11T04:02:51.025 INFO:teuthology.orchestra.run.gibba004.stdout: "min_compat_client": "jewel", 2021-07-11T04:02:51.025 INFO:teuthology.orchestra.run.gibba004.stdout: "require_osd_release": "octopus", 2021-07-11T04:02:51.025 INFO:teuthology.orchestra.run.gibba004.stdout: "pools": [ 2021-07-11T04:02:51.025 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.026 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 1, 2021-07-11T04:02:51.026 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "device_health_metrics", 2021-07-11T04:02:51.026 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:10.374230+0000", 2021-07-11T04:02:51.026 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:51.026 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:51.028 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:51.028 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:51.028 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:51.029 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:51.029 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:51.029 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:51.029 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 1, 2021-07-11T04:02:51.029 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 1, 2021-07-11T04:02:51.030 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 1, 2021-07-11T04:02:51.030 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 1, 2021-07-11T04:02:51.030 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 1, 2021-07-11T04:02:51.030 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:51.031 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:51.031 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:51.031 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:51.031 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:51.031 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:51.032 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:51.032 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.032 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "16", 2021-07-11T04:02:51.032 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:51.033 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:51.033 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:51.033 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:51.033 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:51.033 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:51.034 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:51.034 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:51.034 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:51.034 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:51.034 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:51.035 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:51.035 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:51.035 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:51.035 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:51.036 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:51.036 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:51.036 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:51.036 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:51.036 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:51.037 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:51.037 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:51.037 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:51.037 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:51.037 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:51.038 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:51.038 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.038 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:51.038 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:51.039 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:51.039 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:51.039 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:51.039 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:51.039 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:51.040 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:51.040 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:51.040 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:51.040 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:51.040 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:02:51.041 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 1 2021-07-11T04:02:51.041 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.041 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:51.041 INFO:teuthology.orchestra.run.gibba004.stdout: "mgr_devicehealth": {} 2021-07-11T04:02:51.041 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.042 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.042 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.042 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 2, 2021-07-11T04:02:51.042 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "rbd", 2021-07-11T04:02:51.043 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T03:53:18.251635+0000", 2021-07-11T04:02:51.043 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:51.043 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:51.043 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:51.043 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:51.044 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:51.044 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:51.044 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:51.044 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:51.044 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:02:51.045 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:02:51.045 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:02:51.045 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:02:51.045 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:02:51.045 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:51.046 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:51.046 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:51.046 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:51.046 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:51.047 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:51.047 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:51.047 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.047 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "19", 2021-07-11T04:02:51.047 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:51.048 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:51.048 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:51.048 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:51.048 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:51.048 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:51.049 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:51.049 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:51.049 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:51.050 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:51.050 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:51.050 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:51.050 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:51.050 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:51.051 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:51.051 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:51.051 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:51.051 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:51.051 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:51.052 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:51.052 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:51.052 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:51.052 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:51.053 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:51.053 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:51.053 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:51.053 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.053 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:51.054 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:51.054 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:51.054 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:51.054 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:51.054 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:51.055 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:51.055 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:51.055 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:51.055 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:51.056 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:51.056 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:02:51.056 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:51.056 INFO:teuthology.orchestra.run.gibba004.stdout: "rbd": {} 2021-07-11T04:02:51.056 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.057 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.057 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.057 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 29, 2021-07-11T04:02:51.057 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_metadata", 2021-07-11T04:02:51.057 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:02:34.620986+0000", 2021-07-11T04:02:51.058 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:51.058 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:51.058 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:51.058 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:51.058 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:51.059 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:51.059 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:51.059 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:51.059 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:02:51.060 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:02:51.060 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:02:51.060 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:02:51.060 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:02:51.060 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:51.061 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:51.061 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:51.061 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:51.061 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:51.061 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:51.062 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:51.062 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.062 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "159", 2021-07-11T04:02:51.062 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:51.063 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:51.063 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:51.063 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:51.063 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:51.063 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:51.064 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:51.064 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:51.064 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:51.064 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:51.064 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:51.065 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:51.065 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:51.065 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:51.065 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:51.065 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:51.066 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:51.066 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:51.066 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:51.066 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:51.067 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:51.067 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:51.067 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:51.067 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:51.067 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:51.068 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:51.068 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.068 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:51.068 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:51.068 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:51.069 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:51.069 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:51.069 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:51.069 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:51.069 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:51.070 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:51.070 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:51.070 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:51.070 INFO:teuthology.orchestra.run.gibba004.stdout: "options": { 2021-07-11T04:02:51.071 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_bias": 4, 2021-07-11T04:02:51.071 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_min": 16, 2021-07-11T04:02:51.071 INFO:teuthology.orchestra.run.gibba004.stdout: "recovery_priority": 5 2021-07-11T04:02:51.071 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.071 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:51.072 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:02:51.072 INFO:teuthology.orchestra.run.gibba004.stdout: "metadata": "cephfs" 2021-07-11T04:02:51.072 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.072 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.072 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.073 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.073 INFO:teuthology.orchestra.run.gibba004.stdout: "pool": 30, 2021-07-11T04:02:51.073 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_name": "cephfs_data", 2021-07-11T04:02:51.073 INFO:teuthology.orchestra.run.gibba004.stdout: "create_time": "2021-07-11T04:02:35.639038+0000", 2021-07-11T04:02:51.074 INFO:teuthology.orchestra.run.gibba004.stdout: "flags": 1, 2021-07-11T04:02:51.074 INFO:teuthology.orchestra.run.gibba004.stdout: "flags_names": "hashpspool", 2021-07-11T04:02:51.074 INFO:teuthology.orchestra.run.gibba004.stdout: "type": 1, 2021-07-11T04:02:51.074 INFO:teuthology.orchestra.run.gibba004.stdout: "size": 2, 2021-07-11T04:02:51.074 INFO:teuthology.orchestra.run.gibba004.stdout: "min_size": 1, 2021-07-11T04:02:51.075 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_rule": 0, 2021-07-11T04:02:51.075 INFO:teuthology.orchestra.run.gibba004.stdout: "object_hash": 2, 2021-07-11T04:02:51.075 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_autoscale_mode": "off", 2021-07-11T04:02:51.075 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num": 8, 2021-07-11T04:02:51.075 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num": 8, 2021-07-11T04:02:51.076 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_placement_num_target": 8, 2021-07-11T04:02:51.076 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_target": 8, 2021-07-11T04:02:51.076 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_num_pending": 8, 2021-07-11T04:02:51.076 INFO:teuthology.orchestra.run.gibba004.stdout: "last_pg_merge_meta": { 2021-07-11T04:02:51.076 INFO:teuthology.orchestra.run.gibba004.stdout: "source_pgid": "0.0", 2021-07-11T04:02:51.077 INFO:teuthology.orchestra.run.gibba004.stdout: "ready_epoch": 0, 2021-07-11T04:02:51.077 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_started": 0, 2021-07-11T04:02:51.077 INFO:teuthology.orchestra.run.gibba004.stdout: "last_epoch_clean": 0, 2021-07-11T04:02:51.077 INFO:teuthology.orchestra.run.gibba004.stdout: "source_version": "0'0", 2021-07-11T04:02:51.078 INFO:teuthology.orchestra.run.gibba004.stdout: "target_version": "0'0" 2021-07-11T04:02:51.078 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.078 INFO:teuthology.orchestra.run.gibba004.stdout: "last_change": "161", 2021-07-11T04:02:51.078 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend": "0", 2021-07-11T04:02:51.078 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_prenautilus": "0", 2021-07-11T04:02:51.079 INFO:teuthology.orchestra.run.gibba004.stdout: "last_force_op_resend_preluminous": "0", 2021-07-11T04:02:51.079 INFO:teuthology.orchestra.run.gibba004.stdout: "auid": 0, 2021-07-11T04:02:51.079 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_mode": "selfmanaged", 2021-07-11T04:02:51.079 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_seq": 0, 2021-07-11T04:02:51.079 INFO:teuthology.orchestra.run.gibba004.stdout: "snap_epoch": 0, 2021-07-11T04:02:51.080 INFO:teuthology.orchestra.run.gibba004.stdout: "pool_snaps": [], 2021-07-11T04:02:51.080 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps": "[]", 2021-07-11T04:02:51.080 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_bytes": 0, 2021-07-11T04:02:51.080 INFO:teuthology.orchestra.run.gibba004.stdout: "quota_max_objects": 0, 2021-07-11T04:02:51.081 INFO:teuthology.orchestra.run.gibba004.stdout: "tiers": [], 2021-07-11T04:02:51.081 INFO:teuthology.orchestra.run.gibba004.stdout: "tier_of": -1, 2021-07-11T04:02:51.081 INFO:teuthology.orchestra.run.gibba004.stdout: "read_tier": -1, 2021-07-11T04:02:51.081 INFO:teuthology.orchestra.run.gibba004.stdout: "write_tier": -1, 2021-07-11T04:02:51.081 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_mode": "none", 2021-07-11T04:02:51.082 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_bytes": 0, 2021-07-11T04:02:51.082 INFO:teuthology.orchestra.run.gibba004.stdout: "target_max_objects": 0, 2021-07-11T04:02:51.082 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_ratio_micro": 400000, 2021-07-11T04:02:51.082 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2021-07-11T04:02:51.083 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_target_full_ratio_micro": 800000, 2021-07-11T04:02:51.083 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_flush_age": 0, 2021-07-11T04:02:51.083 INFO:teuthology.orchestra.run.gibba004.stdout: "cache_min_evict_age": 0, 2021-07-11T04:02:51.083 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profile": "", 2021-07-11T04:02:51.084 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_params": { 2021-07-11T04:02:51.084 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "none" 2021-07-11T04:02:51.084 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.084 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_period": 0, 2021-07-11T04:02:51.085 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_count": 0, 2021-07-11T04:02:51.085 INFO:teuthology.orchestra.run.gibba004.stdout: "use_gmt_hitset": true, 2021-07-11T04:02:51.085 INFO:teuthology.orchestra.run.gibba004.stdout: "min_read_recency_for_promote": 0, 2021-07-11T04:02:51.085 INFO:teuthology.orchestra.run.gibba004.stdout: "min_write_recency_for_promote": 0, 2021-07-11T04:02:51.085 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_grade_decay_rate": 0, 2021-07-11T04:02:51.086 INFO:teuthology.orchestra.run.gibba004.stdout: "hit_set_search_last_n": 0, 2021-07-11T04:02:51.086 INFO:teuthology.orchestra.run.gibba004.stdout: "grade_table": [], 2021-07-11T04:02:51.086 INFO:teuthology.orchestra.run.gibba004.stdout: "stripe_width": 0, 2021-07-11T04:02:51.086 INFO:teuthology.orchestra.run.gibba004.stdout: "expected_num_objects": 0, 2021-07-11T04:02:51.087 INFO:teuthology.orchestra.run.gibba004.stdout: "fast_read": false, 2021-07-11T04:02:51.087 INFO:teuthology.orchestra.run.gibba004.stdout: "options": {}, 2021-07-11T04:02:51.087 INFO:teuthology.orchestra.run.gibba004.stdout: "application_metadata": { 2021-07-11T04:02:51.087 INFO:teuthology.orchestra.run.gibba004.stdout: "cephfs": { 2021-07-11T04:02:51.087 INFO:teuthology.orchestra.run.gibba004.stdout: "data": "cephfs" 2021-07-11T04:02:51.088 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.088 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.088 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.088 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:51.089 INFO:teuthology.orchestra.run.gibba004.stdout: "osds": [ 2021-07-11T04:02:51.089 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.089 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:02:51.089 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "a9f3dcb9-1813-4b31-8816-f0e71b6104b6", 2021-07-11T04:02:51.089 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:51.090 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:51.090 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:51.090 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:51.090 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:51.091 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:51.091 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:51.091 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 157, 2021-07-11T04:02:51.091 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:51.091 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:51.092 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:51.092 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.092 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.092 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.092 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6800", 2021-07-11T04:02:51.093 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:51.093 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.093 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.093 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.094 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6801", 2021-07-11T04:02:51.094 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:51.094 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.095 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.095 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.095 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:51.095 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.095 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.096 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.096 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6802", 2021-07-11T04:02:51.096 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:51.096 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.097 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.097 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.097 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6803", 2021-07-11T04:02:51.097 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:51.097 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.098 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.098 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.098 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:51.098 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.099 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.099 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.099 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6806", 2021-07-11T04:02:51.099 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:51.099 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.100 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.100 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.100 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6807", 2021-07-11T04:02:51.100 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:51.101 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.101 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.101 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.101 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:51.101 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.102 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.102 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.102 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6804", 2021-07-11T04:02:51.102 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:51.102 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.103 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.103 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.103 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6805", 2021-07-11T04:02:51.103 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15411 2021-07-11T04:02:51.104 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.104 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.104 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.104 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6801/15411", 2021-07-11T04:02:51.104 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6803/15411", 2021-07-11T04:02:51.105 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6807/15411", 2021-07-11T04:02:51.105 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6805/15411", 2021-07-11T04:02:51.105 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:51.105 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:51.106 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:51.106 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.106 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.106 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.106 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:02:51.107 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "61e6a59d-2129-494a-9723-2c792e38c8ed", 2021-07-11T04:02:51.107 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:51.107 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:51.107 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:51.107 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:51.108 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:51.108 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:51.108 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:51.109 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 156, 2021-07-11T04:02:51.109 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:51.109 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:51.109 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:51.109 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.110 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.110 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.110 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6808", 2021-07-11T04:02:51.110 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:51.111 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.111 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.111 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.111 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6809", 2021-07-11T04:02:51.111 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:51.112 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.112 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.112 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.112 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:51.112 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.113 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.113 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.113 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6810", 2021-07-11T04:02:51.113 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:51.114 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.114 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.114 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.114 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6811", 2021-07-11T04:02:51.114 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:51.115 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.115 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.115 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.115 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:51.116 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.116 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.116 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.116 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6814", 2021-07-11T04:02:51.116 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:51.117 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.117 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.117 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.117 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6815", 2021-07-11T04:02:51.118 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:51.118 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.118 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.118 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.119 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:51.119 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.119 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.119 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.120 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6812", 2021-07-11T04:02:51.120 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:51.120 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.120 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.120 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.121 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6813", 2021-07-11T04:02:51.121 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15412 2021-07-11T04:02:51.121 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.122 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.122 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.122 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6809/15412", 2021-07-11T04:02:51.123 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6811/15412", 2021-07-11T04:02:51.123 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6815/15412", 2021-07-11T04:02:51.123 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6813/15412", 2021-07-11T04:02:51.123 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:51.124 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:51.124 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:51.124 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.124 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.124 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.125 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:02:51.125 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "9b13c5df-5184-4f5e-a553-cc41d8143a5f", 2021-07-11T04:02:51.125 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:51.125 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:51.125 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:51.126 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:51.126 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:51.126 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:51.126 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:51.127 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 157, 2021-07-11T04:02:51.127 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:51.127 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:51.127 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:51.127 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.128 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.128 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.128 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6816", 2021-07-11T04:02:51.128 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:51.129 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.129 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.129 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.129 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6817", 2021-07-11T04:02:51.129 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:51.130 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.130 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.130 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.130 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:51.131 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.131 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.131 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.131 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6818", 2021-07-11T04:02:51.132 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:51.132 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.132 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.132 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.132 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6819", 2021-07-11T04:02:51.133 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:51.133 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.133 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.133 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.133 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:51.134 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.134 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.134 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.134 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6822", 2021-07-11T04:02:51.135 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:51.135 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.135 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.135 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.136 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6823", 2021-07-11T04:02:51.136 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:51.136 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.136 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.136 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.137 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:51.137 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.137 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.137 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.137 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6820", 2021-07-11T04:02:51.138 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:51.138 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.138 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.138 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.139 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6821", 2021-07-11T04:02:51.139 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15413 2021-07-11T04:02:51.139 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.139 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.139 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.140 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6817/15413", 2021-07-11T04:02:51.140 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6819/15413", 2021-07-11T04:02:51.140 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6823/15413", 2021-07-11T04:02:51.140 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6821/15413", 2021-07-11T04:02:51.141 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:51.141 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:51.141 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:51.141 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.141 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.142 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.142 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:02:51.142 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "b75b5b84-a78e-4b67-8044-8ab08825f5fb", 2021-07-11T04:02:51.142 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:51.142 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:51.143 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:51.143 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:51.143 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:51.143 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:51.144 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:51.144 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 157, 2021-07-11T04:02:51.144 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:51.144 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:51.144 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:51.145 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.145 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.145 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.145 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6824", 2021-07-11T04:02:51.146 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:51.146 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.146 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.146 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.146 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6825", 2021-07-11T04:02:51.147 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:51.147 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.147 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.147 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.148 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:51.148 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.148 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.148 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.148 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6826", 2021-07-11T04:02:51.149 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:51.149 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.149 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.149 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.149 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6827", 2021-07-11T04:02:51.150 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:51.150 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.150 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.150 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.151 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:51.151 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.151 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.151 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.151 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6830", 2021-07-11T04:02:51.152 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:51.152 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.152 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.152 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.153 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6831", 2021-07-11T04:02:51.153 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:51.153 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.153 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.153 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.154 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:51.154 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.154 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.154 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.155 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6828", 2021-07-11T04:02:51.155 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:51.155 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.155 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.155 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.156 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.104:6829", 2021-07-11T04:02:51.156 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15456 2021-07-11T04:02:51.156 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.156 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.156 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.157 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.104:6825/15456", 2021-07-11T04:02:51.157 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.104:6827/15456", 2021-07-11T04:02:51.157 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.104:6831/15456", 2021-07-11T04:02:51.157 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.104:6829/15456", 2021-07-11T04:02:51.158 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:51.158 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:51.158 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:51.158 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.158 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.159 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.159 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:02:51.159 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "70205843-19d1-4c4f-a9e4-6aa9ebfa4c28", 2021-07-11T04:02:51.159 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:51.160 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:51.160 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:51.160 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:51.160 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:51.160 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:51.161 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:51.161 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 156, 2021-07-11T04:02:51.161 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:51.161 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:51.161 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:51.162 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.162 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.162 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.162 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6802", 2021-07-11T04:02:51.163 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:51.163 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.163 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.163 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.163 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6803", 2021-07-11T04:02:51.164 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:51.164 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.164 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.164 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.164 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:51.165 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.165 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.165 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.165 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6804", 2021-07-11T04:02:51.166 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:51.166 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.166 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.166 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.166 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6805", 2021-07-11T04:02:51.167 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:51.167 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.167 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.167 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.168 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:51.168 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.168 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.168 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.168 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6808", 2021-07-11T04:02:51.169 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:51.169 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.169 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.169 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.169 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6809", 2021-07-11T04:02:51.170 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:51.170 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.170 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.170 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.171 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:51.171 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.171 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.171 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.171 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6806", 2021-07-11T04:02:51.172 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:51.172 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.172 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.172 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.173 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6807", 2021-07-11T04:02:51.173 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15645 2021-07-11T04:02:51.173 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.173 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.173 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.174 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6803/15645", 2021-07-11T04:02:51.174 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6805/15645", 2021-07-11T04:02:51.174 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6809/15645", 2021-07-11T04:02:51.174 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6807/15645", 2021-07-11T04:02:51.175 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:51.175 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:51.175 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:51.175 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.175 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.176 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.176 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:02:51.176 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "e19543c5-2378-49c8-a626-c71dbbb2a718", 2021-07-11T04:02:51.176 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:51.176 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:51.177 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:51.177 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:51.177 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:51.177 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:51.178 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:51.178 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 159, 2021-07-11T04:02:51.178 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:51.178 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:51.178 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:51.179 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.179 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.179 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.179 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6812", 2021-07-11T04:02:51.179 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:51.180 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.180 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.180 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.180 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6815", 2021-07-11T04:02:51.181 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:51.181 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.181 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.181 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.181 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:51.182 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.182 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.182 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.182 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6818", 2021-07-11T04:02:51.183 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:51.183 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.183 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.183 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.183 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6821", 2021-07-11T04:02:51.184 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:51.184 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.184 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.184 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.184 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:51.185 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.185 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.185 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.185 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6830", 2021-07-11T04:02:51.186 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:51.186 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.186 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.186 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.186 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6833", 2021-07-11T04:02:51.187 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:51.187 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.187 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.187 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.188 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:51.188 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.188 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.188 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.188 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6824", 2021-07-11T04:02:51.189 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:51.189 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.189 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.189 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.189 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6827", 2021-07-11T04:02:51.190 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15646 2021-07-11T04:02:51.190 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.190 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.190 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.191 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6815/15646", 2021-07-11T04:02:51.191 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6821/15646", 2021-07-11T04:02:51.191 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6833/15646", 2021-07-11T04:02:51.191 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6827/15646", 2021-07-11T04:02:51.191 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:51.192 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:51.192 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:51.192 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.192 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.193 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.193 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:02:51.193 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "fc1e39b4-4c50-4f6a-a8ff-19687bf442b2", 2021-07-11T04:02:51.193 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:51.193 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:51.194 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:51.194 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:51.194 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:51.194 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:51.194 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:51.195 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 157, 2021-07-11T04:02:51.195 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:51.195 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:51.195 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:51.196 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.196 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.196 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.196 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6810", 2021-07-11T04:02:51.196 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:51.197 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.197 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.197 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.197 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6813", 2021-07-11T04:02:51.198 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:51.198 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.198 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.198 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.198 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:51.199 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.199 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.199 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.199 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6816", 2021-07-11T04:02:51.199 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:51.200 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.200 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.200 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.200 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6819", 2021-07-11T04:02:51.201 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:51.201 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.201 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.201 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.201 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:51.202 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.202 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.202 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.202 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6829", 2021-07-11T04:02:51.203 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:51.203 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.203 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.203 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.203 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6832", 2021-07-11T04:02:51.204 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:51.204 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.204 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.204 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.204 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:51.205 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.205 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.205 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.205 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6822", 2021-07-11T04:02:51.206 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:51.206 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.206 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.206 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.206 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6825", 2021-07-11T04:02:51.207 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15647 2021-07-11T04:02:51.207 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.207 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.207 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.208 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6813/15647", 2021-07-11T04:02:51.208 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6819/15647", 2021-07-11T04:02:51.208 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6832/15647", 2021-07-11T04:02:51.208 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6825/15647", 2021-07-11T04:02:51.208 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:51.209 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:51.209 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:51.209 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.209 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.209 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.210 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:02:51.210 INFO:teuthology.orchestra.run.gibba004.stdout: "uuid": "2df0639d-90fe-4158-80ae-24cf0f10dc86", 2021-07-11T04:02:51.210 INFO:teuthology.orchestra.run.gibba004.stdout: "up": 1, 2021-07-11T04:02:51.210 INFO:teuthology.orchestra.run.gibba004.stdout: "in": 1, 2021-07-11T04:02:51.211 INFO:teuthology.orchestra.run.gibba004.stdout: "weight": 1, 2021-07-11T04:02:51.211 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_affinity": 1, 2021-07-11T04:02:51.211 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_begin": 0, 2021-07-11T04:02:51.211 INFO:teuthology.orchestra.run.gibba004.stdout: "last_clean_end": 0, 2021-07-11T04:02:51.211 INFO:teuthology.orchestra.run.gibba004.stdout: "up_from": 14, 2021-07-11T04:02:51.212 INFO:teuthology.orchestra.run.gibba004.stdout: "up_thru": 157, 2021-07-11T04:02:51.212 INFO:teuthology.orchestra.run.gibba004.stdout: "down_at": 0, 2021-07-11T04:02:51.212 INFO:teuthology.orchestra.run.gibba004.stdout: "lost_at": 0, 2021-07-11T04:02:51.212 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addrs": { 2021-07-11T04:02:51.212 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.213 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.213 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.213 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6811", 2021-07-11T04:02:51.213 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:51.214 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.214 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.214 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.214 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6814", 2021-07-11T04:02:51.214 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:51.215 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.215 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.215 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.215 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addrs": { 2021-07-11T04:02:51.215 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.216 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.216 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.216 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6817", 2021-07-11T04:02:51.216 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:51.217 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.217 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.217 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.217 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6820", 2021-07-11T04:02:51.217 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:51.218 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.218 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.218 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.218 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addrs": { 2021-07-11T04:02:51.219 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.219 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.219 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.219 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6828", 2021-07-11T04:02:51.220 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:51.220 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.220 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.220 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.220 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6831", 2021-07-11T04:02:51.221 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:51.221 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.221 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.221 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.221 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addrs": { 2021-07-11T04:02:51.222 INFO:teuthology.orchestra.run.gibba004.stdout: "addrvec": [ 2021-07-11T04:02:51.222 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.222 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v2", 2021-07-11T04:02:51.222 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6823", 2021-07-11T04:02:51.223 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:51.223 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.223 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.223 INFO:teuthology.orchestra.run.gibba004.stdout: "type": "v1", 2021-07-11T04:02:51.223 INFO:teuthology.orchestra.run.gibba004.stdout: "addr": "172.21.2.105:6826", 2021-07-11T04:02:51.224 INFO:teuthology.orchestra.run.gibba004.stdout: "nonce": 15656 2021-07-11T04:02:51.224 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.224 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.224 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.224 INFO:teuthology.orchestra.run.gibba004.stdout: "public_addr": "172.21.2.105:6814/15656", 2021-07-11T04:02:51.225 INFO:teuthology.orchestra.run.gibba004.stdout: "cluster_addr": "172.21.2.105:6820/15656", 2021-07-11T04:02:51.225 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_back_addr": "172.21.2.105:6831/15656", 2021-07-11T04:02:51.225 INFO:teuthology.orchestra.run.gibba004.stdout: "heartbeat_front_addr": "172.21.2.105:6826/15656", 2021-07-11T04:02:51.225 INFO:teuthology.orchestra.run.gibba004.stdout: "state": [ 2021-07-11T04:02:51.226 INFO:teuthology.orchestra.run.gibba004.stdout: "exists", 2021-07-11T04:02:51.226 INFO:teuthology.orchestra.run.gibba004.stdout: "up" 2021-07-11T04:02:51.226 INFO:teuthology.orchestra.run.gibba004.stdout: ] 2021-07-11T04:02:51.226 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.226 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:51.227 INFO:teuthology.orchestra.run.gibba004.stdout: "osd_xinfo": [ 2021-07-11T04:02:51.227 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.227 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 0, 2021-07-11T04:02:51.227 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:51.228 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:51.228 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:51.228 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:51.228 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:51.228 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.990606+0000", 2021-07-11T04:02:51.229 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:51.229 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.229 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.229 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 1, 2021-07-11T04:02:51.229 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:51.230 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:51.230 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:51.230 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:51.230 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:51.231 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.951362+0000", 2021-07-11T04:02:51.231 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:51.231 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.231 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.231 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 2, 2021-07-11T04:02:51.232 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:51.232 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:51.232 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:51.232 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:51.233 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:51.233 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.132692+0000", 2021-07-11T04:02:51.233 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:51.233 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.233 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.234 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 3, 2021-07-11T04:02:51.234 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:51.234 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:51.234 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:51.234 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:51.235 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:51.235 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.157951+0000", 2021-07-11T04:02:51.235 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:51.235 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.236 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.236 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 4, 2021-07-11T04:02:51.236 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:51.236 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:51.236 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:51.237 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:51.237 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:51.237 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:12.704587+0000", 2021-07-11T04:02:51.237 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:51.237 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.238 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.238 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 5, 2021-07-11T04:02:51.238 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:51.238 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:51.239 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:51.239 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:51.239 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:51.239 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.008971+0000", 2021-07-11T04:02:51.239 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:51.240 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.240 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.240 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 6, 2021-07-11T04:02:51.240 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:51.240 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:51.241 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:51.241 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:51.241 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:51.241 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.062419+0000", 2021-07-11T04:02:51.242 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:51.242 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.242 INFO:teuthology.orchestra.run.gibba004.stdout: { 2021-07-11T04:02:51.242 INFO:teuthology.orchestra.run.gibba004.stdout: "osd": 7, 2021-07-11T04:02:51.242 INFO:teuthology.orchestra.run.gibba004.stdout: "down_stamp": "0.000000", 2021-07-11T04:02:51.243 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_probability": 0, 2021-07-11T04:02:51.243 INFO:teuthology.orchestra.run.gibba004.stdout: "laggy_interval": 0, 2021-07-11T04:02:51.243 INFO:teuthology.orchestra.run.gibba004.stdout: "features": 4540138292840890367, 2021-07-11T04:02:51.243 INFO:teuthology.orchestra.run.gibba004.stdout: "old_weight": 0, 2021-07-11T04:02:51.244 INFO:teuthology.orchestra.run.gibba004.stdout: "last_purged_snaps_scrub": "2021-07-11T03:53:13.104834+0000", 2021-07-11T04:02:51.244 INFO:teuthology.orchestra.run.gibba004.stdout: "dead_epoch": 0 2021-07-11T04:02:51.244 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.244 INFO:teuthology.orchestra.run.gibba004.stdout: ], 2021-07-11T04:02:51.244 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap": [], 2021-07-11T04:02:51.245 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_upmap_items": [], 2021-07-11T04:02:51.245 INFO:teuthology.orchestra.run.gibba004.stdout: "pg_temp": [], 2021-07-11T04:02:51.245 INFO:teuthology.orchestra.run.gibba004.stdout: "primary_temp": [], 2021-07-11T04:02:51.245 INFO:teuthology.orchestra.run.gibba004.stdout: "blacklist": {}, 2021-07-11T04:02:51.246 INFO:teuthology.orchestra.run.gibba004.stdout: "erasure_code_profiles": { 2021-07-11T04:02:51.246 INFO:teuthology.orchestra.run.gibba004.stdout: "default": { 2021-07-11T04:02:51.246 INFO:teuthology.orchestra.run.gibba004.stdout: "crush-failure-domain": "osd", 2021-07-11T04:02:51.246 INFO:teuthology.orchestra.run.gibba004.stdout: "k": "2", 2021-07-11T04:02:51.246 INFO:teuthology.orchestra.run.gibba004.stdout: "m": "1", 2021-07-11T04:02:51.247 INFO:teuthology.orchestra.run.gibba004.stdout: "plugin": "jerasure", 2021-07-11T04:02:51.247 INFO:teuthology.orchestra.run.gibba004.stdout: "ruleset-failure-domain": "osd", 2021-07-11T04:02:51.247 INFO:teuthology.orchestra.run.gibba004.stdout: "technique": "reed_sol_van" 2021-07-11T04:02:51.247 INFO:teuthology.orchestra.run.gibba004.stdout: } 2021-07-11T04:02:51.247 INFO:teuthology.orchestra.run.gibba004.stdout: }, 2021-07-11T04:02:51.248 INFO:teuthology.orchestra.run.gibba004.stdout: "removed_snaps_queue": [], 2021-07-11T04:02:51.248 INFO:teuthology.orchestra.run.gibba004.stdout: "new_removed_snaps": [], 2021-07-11T04:02:51.248 INFO:teuthology.orchestra.run.gibba004.stdout: "new_purged_snaps": [], 2021-07-11T04:02:51.248 INFO:teuthology.orchestra.run.gibba004.stdout: "crush_node_flags": {}, 2021-07-11T04:02:51.249 INFO:teuthology.orchestra.run.gibba004.stdout: "device_class_flags": {} 2021-07-11T04:02:51.249 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:02:51.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 -- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa88053240 msgr2=0x7faa88055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:51.250 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa88053240 0x7faa88055700 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7faa98002f70 tx=0x7faa98015000).stop 2021-07-11T04:02:51.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 -- 172.21.2.104:0/435196218 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faaa81416a0 msgr2=0x7faaa8150680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:51.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faaa81416a0 0x7faaa8150680 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 rx=0x7faaa00032f0 tx=0x7faaa0006f90).stop 2021-07-11T04:02:51.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 -- 172.21.2.104:0/435196218 shutdown_connections 2021-07-11T04:02:51.251 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7faa88053240 0x7faa88055700 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:51.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7faaa81416a0 0x7faaa8150680 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:51.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7faaa80b2200 0x7faaa813df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:51.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 --2- 172.21.2.104:0/435196218 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7faaa80affd0 0x7faaa80b4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:51.252 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 -- 172.21.2.104:0/435196218 >> 172.21.2.104:0/435196218 conn(0x7faaa805f6b0 msgr2=0x7faaa81447f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:51.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 -- 172.21.2.104:0/435196218 shutdown_connections 2021-07-11T04:02:51.253 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.020+0000 7faab0161700 1 -- 172.21.2.104:0/435196218 wait complete. 2021-07-11T04:02:51.254 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:51.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.308+0000 7f0362962700 1 Processor -- start 2021-07-11T04:02:51.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.308+0000 7f0362962700 1 -- start start 2021-07-11T04:02:51.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f035c143540 0x7f035c0a9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f035c0a8b60 0x7f035c143920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035c0b14c0 con 0x7f035c05f6b0 2021-07-11T04:02:51.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035c0ab3e0 con 0x7f035c0a8b60 2021-07-11T04:02:51.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0b1b90 con 0x7f035c143540 2021-07-11T04:02:51.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0360eff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f035c0a8b60 0x7f035c143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:51.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0360eff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f035c0a8b60 0x7f035c143920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52098/0 (socket says 172.21.2.104:52098) 2021-07-11T04:02:51.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0360eff700 1 -- 172.21.2.104:0/780247073 learned_addr learned my addr 172.21.2.104:0/780247073 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:51.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0360eff700 1 -- 172.21.2.104:0/780247073 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f035c143540 msgr2=0x7f035c0a9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:51.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0360eff700 1 --2- 172.21.2.104:0/780247073 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f035c143540 0x7f035c0a9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:51.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0360eff700 1 -- 172.21.2.104:0/780247073 >> v1:172.21.2.105:6789/0 conn(0x7f035c05f6b0 legacy=0x7f035c0b0db0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:51.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0360eff700 1 -- 172.21.2.104:0/780247073 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035c143fa0 con 0x7f035c0a8b60 2021-07-11T04:02:51.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0360eff700 1 --2- 172.21.2.104:0/780247073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f035c0a8b60 0x7f035c143920 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f035c0611c0 tx=0x7f0354009290).ready entity=mon.0 client_cookie=83e0b2d79a7a4b9c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:51.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f035affd700 1 -- 172.21.2.104:0/780247073 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f035400b860 con 0x7f035c0a8b60 2021-07-11T04:02:51.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f035affd700 1 -- 172.21.2.104:0/780247073 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035400b9c0 con 0x7f035c0a8b60 2021-07-11T04:02:51.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f035affd700 1 -- 172.21.2.104:0/780247073 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f035400bb70 con 0x7f035c0a8b60 2021-07-11T04:02:51.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- 172.21.2.104:0/780247073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f035c0a8b60 msgr2=0x7f035c143920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:51.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 --2- 172.21.2.104:0/780247073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f035c0a8b60 0x7f035c143920 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 rx=0x7f035c0611c0 tx=0x7f0354009290).stop 2021-07-11T04:02:51.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- 172.21.2.104:0/780247073 shutdown_connections 2021-07-11T04:02:51.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 --2- 172.21.2.104:0/780247073 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f035c0a8b60 0x7f035c143920 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:51.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 --2- 172.21.2.104:0/780247073 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f035c143540 0x7f035c0a9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:51.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- 172.21.2.104:0/780247073 >> 172.21.2.104:0/780247073 conn(0x7f035c0b53e0 msgr2=0x7f035c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:51.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- 172.21.2.104:0/780247073 shutdown_connections 2021-07-11T04:02:51.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- 172.21.2.104:0/780247073 wait complete. 2021-07-11T04:02:51.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 Processor -- start 2021-07-11T04:02:51.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- start start 2021-07-11T04:02:51.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f035c05f6b0 0x7f035c142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f035c0a8b60 0x7f035c141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f035c143540 0x7f035c155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c142280 con 0x7f035c05f6b0 2021-07-11T04:02:51.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0add90 con 0x7f035c143540 2021-07-11T04:02:51.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f0362962700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035c1539b0 con 0x7f035c0a8b60 2021-07-11T04:02:51.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f035bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f035c05f6b0 0x7f035c142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:51.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f035bfff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f035c05f6b0 0x7f035c142c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52102/0 (socket says 172.21.2.104:52102) 2021-07-11T04:02:51.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.312+0000 7f035bfff700 1 -- 172.21.2.104:0/3118714654 learned_addr learned my addr 172.21.2.104:0/3118714654 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:51.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f035bfff700 1 -- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f035c0a8b60 msgr2=0x7f035c141820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:51.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f035bfff700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f035c0a8b60 0x7f035c141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:51.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f035bfff700 1 -- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f035c143540 msgr2=0x7f035c155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:51.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f035bfff700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f035c143540 0x7f035c155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:51.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f035bfff700 1 -- 172.21.2.104:0/3118714654 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0354009530 con 0x7f035c05f6b0 2021-07-11T04:02:51.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f035bfff700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f035c05f6b0 0x7f035c142c40 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 rx=0x7f035c153af0 tx=0x7f034c006e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:51.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f03597fa700 1 -- 172.21.2.104:0/3118714654 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f034c00e470 con 0x7f035c05f6b0 2021-07-11T04:02:51.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f03597fa700 1 -- 172.21.2.104:0/3118714654 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c00e5d0 con 0x7f035c05f6b0 2021-07-11T04:02:51.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f0362962700 1 -- 172.21.2.104:0/3118714654 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035c0ac8b0 con 0x7f035c05f6b0 2021-07-11T04:02:51.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f0362962700 1 -- 172.21.2.104:0/3118714654 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c153510 con 0x7f035c05f6b0 2021-07-11T04:02:51.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f03597fa700 1 -- 172.21.2.104:0/3118714654 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f034c00e930 con 0x7f035c05f6b0 2021-07-11T04:02:51.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f0362962700 1 -- 172.21.2.104:0/3118714654 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035c141ee0 con 0x7f035c05f6b0 2021-07-11T04:02:51.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f03597fa700 1 -- 172.21.2.104:0/3118714654 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f034c018030 con 0x7f035c05f6b0 2021-07-11T04:02:51.325 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f03597fa700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f03400532d0 0x7f0340055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f03597fa700 1 -- 172.21.2.104:0/3118714654 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f034c06aad0 con 0x7f035c05f6b0 2021-07-11T04:02:51.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f035b7fe700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f03400532d0 0x7f0340055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:51.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.316+0000 7f035b7fe700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f03400532d0 0x7f0340055790 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f0344002fb0 tx=0x7f03440069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:51.326 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.320+0000 7f03597fa700 1 -- 172.21.2.104:0/3118714654 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f034c041db0 con 0x7f035c05f6b0 2021-07-11T04:02:51.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.424+0000 7f0362962700 1 -- 172.21.2.104:0/3118714654 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f035c057350 con 0x7f035c05f6b0 2021-07-11T04:02:51.425 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.424+0000 7f03597fa700 1 -- 172.21.2.104:0/3118714654 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+2489 (secure 0 0 0) 0x7f034c0453d0 con 0x7f035c05f6b0 2021-07-11T04:02:51.426 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:51.426 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":80,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5132,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/770706140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":770706140},{"type":"v1","addr":"172.21.2.105:6835","nonce":770706140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":79},{"gid":8718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/488029782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":488029782},{"type":"v1","addr":"172.21.2.104:6833","nonce":488029782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:36.673961+0000","modified":"2021-07-11T04:02:37.679104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8721},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8721":{"gid":8721,"name":"c","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.2.104:6835/2638676458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6834","nonce":2638676458},{"type":"v1","addr":"172.21.2.104:6835","nonce":2638676458}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-07-11T04:02:51.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 -- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f03400532d0 msgr2=0x7f0340055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:51.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f03400532d0 0x7f0340055790 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f0344002fb0 tx=0x7f03440069b0).stop 2021-07-11T04:02:51.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 -- 172.21.2.104:0/3118714654 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f035c05f6b0 msgr2=0x7f035c142c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:51.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f035c05f6b0 0x7f035c142c40 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 rx=0x7f035c153af0 tx=0x7f034c006e50).stop 2021-07-11T04:02:51.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 -- 172.21.2.104:0/3118714654 shutdown_connections 2021-07-11T04:02:51.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f035c143540 0x7f035c155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:51.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f035c0a8b60 0x7f035c141820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:51.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f03400532d0 0x7f0340055790 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:51.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 --2- 172.21.2.104:0/3118714654 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f035c05f6b0 0x7f035c142c40 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:51.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 -- 172.21.2.104:0/3118714654 >> 172.21.2.104:0/3118714654 conn(0x7f035c0b53e0 msgr2=0x7f035c0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:51.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 -- 172.21.2.104:0/3118714654 shutdown_connections 2021-07-11T04:02:51.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.428+0000 7f0362962700 1 -- 172.21.2.104:0/3118714654 wait complete. 2021-07-11T04:02:51.430 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 80 2021-07-11T04:02:51.433 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2021-07-11T04:02:51.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.484+0000 7f5c19bee700 1 Processor -- start 2021-07-11T04:02:51.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.484+0000 7f5c19bee700 1 -- start start 2021-07-11T04:02:51.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.484+0000 7f5c19bee700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c14067180 0x7f5c1405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.484+0000 7f5c19bee700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5c140b1970 0x7f5c1405fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.484+0000 7f5c19bee700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c140a7a90 con 0x7f5c140a76b0 2021-07-11T04:02:51.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.484+0000 7f5c19bee700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c14147620 con 0x7f5c140b1970 2021-07-11T04:02:51.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.484+0000 7f5c19bee700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c141478c0 con 0x7f5c14067180 2021-07-11T04:02:51.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c12ffd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c14067180 0x7f5c1405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:51.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c13fff700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f5c140b1970 0x7f5c1405fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:51.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c137fe700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f5c140a76b0 0x7f5c14066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:48248/0 (socket says 172.21.2.104:48248) 2021-07-11T04:02:51.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c12ffd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c14067180 0x7f5c1405f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:55278/0 (socket says 172.21.2.104:55278) 2021-07-11T04:02:51.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c13fff700 1 --2- 172.21.2.104:0/3823171105 >> v2:172.21.2.105:3300/0 conn(0x7f5c140b1970 0x7f5c1405fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:59672/0 (socket says 172.21.2.104:59672) 2021-07-11T04:02:51.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c137fe700 1 -- 172.21.2.104:0/3823171105 learned_addr learned my addr 172.21.2.104:0/3823171105 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:51.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c12ffd700 1 -- 172.21.2.104:0/3823171105 >> v2:172.21.2.105:3300/0 conn(0x7f5c140b1970 msgr2=0x7f5c1405fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:51.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c12ffd700 1 --2- 172.21.2.104:0/3823171105 >> v2:172.21.2.105:3300/0 conn(0x7f5c140b1970 0x7f5c1405fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:51.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c12ffd700 1 -- 172.21.2.104:0/3823171105 >> v1:172.21.2.105:6789/0 conn(0x7f5c140a76b0 legacy=0x7f5c14066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:51.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c12ffd700 1 -- 172.21.2.104:0/3823171105 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c14147b60 con 0x7f5c14067180 2021-07-11T04:02:51.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c12ffd700 1 --2- 172.21.2.104:0/3823171105 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c14067180 0x7f5c1405f6b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f5c04002f70 tx=0x7f5c04006df0).ready entity=mon.2 client_cookie=90e2934dd2e1deb5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:51.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c127fc700 1 -- 172.21.2.104:0/3823171105 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0400d570 con 0x7f5c14067180 2021-07-11T04:02:51.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c127fc700 1 -- 172.21.2.104:0/3823171105 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c04000e90 con 0x7f5c14067180 2021-07-11T04:02:51.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c127fc700 1 -- 172.21.2.104:0/3823171105 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5c0400d8c0 con 0x7f5c14067180 2021-07-11T04:02:51.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c19bee700 1 -- 172.21.2.104:0/3823171105 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c14067180 msgr2=0x7f5c1405f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:51.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c19bee700 1 --2- 172.21.2.104:0/3823171105 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c14067180 0x7f5c1405f6b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f5c04002f70 tx=0x7f5c04006df0).stop 2021-07-11T04:02:51.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c19bee700 1 -- 172.21.2.104:0/3823171105 shutdown_connections 2021-07-11T04:02:51.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c19bee700 1 --2- 172.21.2.104:0/3823171105 >> v2:172.21.2.105:3300/0 conn(0x7f5c140b1970 0x7f5c1405fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:51.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c19bee700 1 --2- 172.21.2.104:0/3823171105 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c14067180 0x7f5c1405f6b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:51.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c19bee700 1 -- 172.21.2.104:0/3823171105 >> 172.21.2.104:0/3823171105 conn(0x7f5c140b53e0 msgr2=0x7f5c140b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:51.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c19bee700 1 -- 172.21.2.104:0/3823171105 shutdown_connections 2021-07-11T04:02:51.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c19bee700 1 -- 172.21.2.104:0/3823171105 wait complete. 2021-07-11T04:02:51.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c19bee700 1 Processor -- start 2021-07-11T04:02:51.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.488+0000 7f5c19bee700 1 -- start start 2021-07-11T04:02:51.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c19bee700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c14067180 0x7f5c140ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c19bee700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c140a76b0 0x7f5c140abea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c19bee700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c140b1970 0x7f5c140aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c19bee700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c140abc00 con 0x7f5c14067180 2021-07-11T04:02:51.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c19bee700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c140aed30 con 0x7f5c140b1970 2021-07-11T04:02:51.497 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c19bee700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c140aefd0 con 0x7f5c140a76b0 2021-07-11T04:02:51.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c137fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c14067180 0x7f5c140ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:51.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c137fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c14067180 0x7f5c140ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52120/0 (socket says 172.21.2.104:52120) 2021-07-11T04:02:51.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c137fe700 1 -- 172.21.2.104:0/2338729025 learned_addr learned my addr 172.21.2.104:0/2338729025 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:51.498 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c13fff700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c140b1970 0x7f5c140aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:51.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c12ffd700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c140a76b0 0x7f5c140abea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:51.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c137fe700 1 -- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c140a76b0 msgr2=0x7f5c140abea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:51.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c137fe700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c140a76b0 0x7f5c140abea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:51.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c137fe700 1 -- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c140b1970 msgr2=0x7f5c140aa2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:51.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c137fe700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c140b1970 0x7f5c140aa2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:51.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c137fe700 1 -- 172.21.2.104:0/2338729025 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0400b180 con 0x7f5c14067180 2021-07-11T04:02:51.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c12ffd700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c140a76b0 0x7f5c140abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:51.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c137fe700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c14067180 0x7f5c140ac860 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 rx=0x7f5c14148420 tx=0x7f5bfc007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:51.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c10ff9700 1 -- 172.21.2.104:0/2338729025 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bfc00ce30 con 0x7f5c14067180 2021-07-11T04:02:51.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c10ff9700 1 -- 172.21.2.104:0/2338729025 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bfc0072e0 con 0x7f5c14067180 2021-07-11T04:02:51.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c10ff9700 1 -- 172.21.2.104:0/2338729025 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5bfc0057c0 con 0x7f5c14067180 2021-07-11T04:02:51.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c19bee700 1 -- 172.21.2.104:0/2338729025 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c140ae8c0 con 0x7f5c14067180 2021-07-11T04:02:51.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c19bee700 1 -- 172.21.2.104:0/2338729025 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c140a9ec0 con 0x7f5c14067180 2021-07-11T04:02:51.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c19bee700 1 -- 172.21.2.104:0/2338729025 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c140aeb90 con 0x7f5c14067180 2021-07-11T04:02:51.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c10ff9700 1 -- 172.21.2.104:0/2338729025 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5bfc010070 con 0x7f5c14067180 2021-07-11T04:02:51.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c10ff9700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bf8053270 0x7f5bf8055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:51.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c10ff9700 1 -- 172.21.2.104:0/2338729025 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f5bfc06ab90 con 0x7f5c14067180 2021-07-11T04:02:51.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.492+0000 7f5c12ffd700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bf8053270 0x7f5bf8055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:51.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.496+0000 7f5c12ffd700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bf8053270 0x7f5bf8055730 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f5c04013000 tx=0x7f5c04014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:51.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.496+0000 7f5c10ff9700 1 -- 172.21.2.104:0/2338729025 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5c140aeb90 con 0x7f5c14067180 2021-07-11T04:02:51.602 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:51.604+0000 7f5c19bee700 1 -- 172.21.2.104:0/2338729025 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5c140aeb90 con 0x7f5c14067180 2021-07-11T04:02:52.402 INFO:tasks.ceph.mon.a.gibba004.stderr:2021-07-11T04:02:52.400+0000 7f206c14c700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-07-11T04:02:52.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.412+0000 7f5c10ff9700 1 -- 172.21.2.104:0/2338729025 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5c140aeb90 con 0x7f5c14067180 2021-07-11T04:02:52.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 -- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bf8053270 msgr2=0x7f5bf8055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:52.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bf8053270 0x7f5bf8055730 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f5c04013000 tx=0x7f5c04014000).stop 2021-07-11T04:02:52.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 -- 172.21.2.104:0/2338729025 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c14067180 msgr2=0x7f5c140ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:52.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c14067180 0x7f5c140ac860 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 rx=0x7f5c14148420 tx=0x7f5bfc007c30).stop 2021-07-11T04:02:52.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 -- 172.21.2.104:0/2338729025 shutdown_connections 2021-07-11T04:02:52.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f5c140b1970 0x7f5c140aa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:52.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f5c140a76b0 0x7f5c140abea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:52.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5bf8053270 0x7f5bf8055730 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:52.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 --2- 172.21.2.104:0/2338729025 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f5c14067180 0x7f5c140ac860 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:52.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 -- 172.21.2.104:0/2338729025 >> 172.21.2.104:0/2338729025 conn(0x7f5c140b53e0 msgr2=0x7f5c14148a00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:52.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 -- 172.21.2.104:0/2338729025 shutdown_connections 2021-07-11T04:02:52.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.416+0000 7f5c19bee700 1 -- 172.21.2.104:0/2338729025 wait complete. 2021-07-11T04:02:52.419 INFO:teuthology.orchestra.run.gibba004.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2021-07-11T04:02:52.424 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-07-11T04:02:52.426 INFO:tasks.ceph.mds.c.gibba004.stdout:starting mds.c at 2021-07-11T04:02:52.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.476+0000 7fc5aa054700 1 Processor -- start 2021-07-11T04:02:52.478 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.476+0000 7fc5aa054700 1 -- start start 2021-07-11T04:02:52.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc5a40b7810 0x7fc5a41445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.480 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5a40b1170 0x7fc5a4144a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a40b31d0 con 0x7fc5a4064870 2021-07-11T04:02:52.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a40b7bf0 con 0x7fc5a40b7810 2021-07-11T04:02:52.481 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a40675c0 con 0x7fc5a40b1170 2021-07-11T04:02:52.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a3fff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc5a4064870 0x7fc5a4064cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48906/0 (socket says 172.21.2.104:48906) 2021-07-11T04:02:52.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a3fff700 1 -- 172.21.2.104:0/682441716 learned_addr learned my addr 172.21.2.104:0/682441716 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:52.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 --2- 172.21.2.104:0/682441716 >> v2:172.21.2.104:3300/0 conn(0x7fc5a40b7810 0x7fc5a41445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:52.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a27fc700 1 -- 172.21.2.104:0/682441716 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500525604 0 0) 0x7fc5a40b31d0 con 0x7fc5a4064870 2021-07-11T04:02:52.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 -- 172.21.2.104:0/682441716 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5a40b1170 msgr2=0x7fc5a4144a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:02:52.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 --2- 172.21.2.104:0/682441716 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5a40b1170 0x7fc5a4144a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:52.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 -- 172.21.2.104:0/682441716 >> v1:172.21.2.104:6789/0 conn(0x7fc5a4064870 legacy=0x7fc5a4064cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:52.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 -- 172.21.2.104:0/682441716 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a40aae70 con 0x7fc5a40b7810 2021-07-11T04:02:52.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 --2- 172.21.2.104:0/682441716 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5a40b7810 0x7fc5a41445a0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 rx=0x7fc58c002f70 tx=0x7fc58c006df0).ready entity=mon.0 client_cookie=3025c2e1bda46e5e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:52.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a27fc700 1 -- 172.21.2.104:0/682441716 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c00d570 con 0x7fc5a40b7810 2021-07-11T04:02:52.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a27fc700 1 -- 172.21.2.104:0/682441716 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c000e90 con 0x7fc5a40b7810 2021-07-11T04:02:52.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a27fc700 1 -- 172.21.2.104:0/682441716 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c00d900 con 0x7fc5a40b7810 2021-07-11T04:02:52.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- 172.21.2.104:0/682441716 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5a40b7810 msgr2=0x7fc5a41445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:52.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 --2- 172.21.2.104:0/682441716 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5a40b7810 0x7fc5a41445a0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 rx=0x7fc58c002f70 tx=0x7fc58c006df0).stop 2021-07-11T04:02:52.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- 172.21.2.104:0/682441716 shutdown_connections 2021-07-11T04:02:52.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 --2- 172.21.2.104:0/682441716 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5a40b1170 0x7fc5a4144a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:52.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 --2- 172.21.2.104:0/682441716 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5a40b7810 0x7fc5a41445a0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:52.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- 172.21.2.104:0/682441716 >> 172.21.2.104:0/682441716 conn(0x7fc5a405f6b0 msgr2=0x7fc5a405fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:52.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- 172.21.2.104:0/682441716 shutdown_connections 2021-07-11T04:02:52.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- 172.21.2.104:0/682441716 wait complete. 2021-07-11T04:02:52.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 Processor -- start 2021-07-11T04:02:52.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- start start 2021-07-11T04:02:52.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5a4064870 0x7fc5a4144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5a40b1170 0x7fc5a4152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5a40b7810 0x7fc5a4146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a4146b90 con 0x7fc5a4064870 2021-07-11T04:02:52.490 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a4152d30 con 0x7fc5a40b7810 2021-07-11T04:02:52.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a41468f0 con 0x7fc5a40b1170 2021-07-11T04:02:52.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5a4064870 0x7fc5a4144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:52.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5a4064870 0x7fc5a4144f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52146/0 (socket says 172.21.2.104:52146) 2021-07-11T04:02:52.491 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 -- 172.21.2.104:0/4122062122 learned_addr learned my addr 172.21.2.104:0/4122062122 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:52.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 -- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5a40b1170 msgr2=0x7fc5a4152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:52.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5a40b1170 0x7fc5a4152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:52.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 -- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5a40b7810 msgr2=0x7fc5a4146dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:52.492 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5a40b7810 0x7fc5a4146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:52.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 -- 172.21.2.104:0/4122062122 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c00b180 con 0x7fc5a4064870 2021-07-11T04:02:52.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a37fe700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5a4064870 0x7fc5a4144f90 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 rx=0x7fc58c002f70 tx=0x7fc58c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:52.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a0ff9700 1 -- 172.21.2.104:0/4122062122 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c011ce0 con 0x7fc5a4064870 2021-07-11T04:02:52.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a0ff9700 1 -- 172.21.2.104:0/4122062122 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c011e40 con 0x7fc5a4064870 2021-07-11T04:02:52.493 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- 172.21.2.104:0/4122062122 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a41458d0 con 0x7fc5a4064870 2021-07-11T04:02:52.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5a0ff9700 1 -- 172.21.2.104:0/4122062122 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc58c01d640 con 0x7fc5a4064870 2021-07-11T04:02:52.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.480+0000 7fc5aa054700 1 -- 172.21.2.104:0/4122062122 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a4146620 con 0x7fc5a4064870 2021-07-11T04:02:52.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.484+0000 7fc5aa054700 1 -- 172.21.2.104:0/4122062122 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a4057350 con 0x7fc5a4064870 2021-07-11T04:02:52.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.484+0000 7fc5a0ff9700 1 -- 172.21.2.104:0/4122062122 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc58c00f070 con 0x7fc5a4064870 2021-07-11T04:02:52.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.484+0000 7fc5a0ff9700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc5880531f0 0x7fc5880556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.484+0000 7fc5a0ff9700 1 -- 172.21.2.104:0/4122062122 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6155+0+0 (secure 0 0 0) 0x7fc58c06d140 con 0x7fc5a4064870 2021-07-11T04:02:52.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.484+0000 7fc5a2ffd700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc5880531f0 0x7fc5880556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:52.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.484+0000 7fc5a2ffd700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc5880531f0 0x7fc5880556b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fc594002fb0 tx=0x7fc5940069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:52.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.484+0000 7fc5a0ff9700 1 -- 172.21.2.104:0/4122062122 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc58c044430 con 0x7fc5a4064870 2021-07-11T04:02:52.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 -- 172.21.2.104:0/4122062122 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5a4145d70 con 0x7fc5a4064870 2021-07-11T04:02:52.595 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5a0ff9700 1 -- 172.21.2.104:0/4122062122 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+2129 (secure 0 0 0) 0x7fc58c047a50 con 0x7fc5a4064870 2021-07-11T04:02:52.596 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:02:52.596 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":81,"default_fscid":15,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5132,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.2.105:6835/770706140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6834","nonce":770706140},{"type":"v1","addr":"172.21.2.105:6835","nonce":770706140}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":79},{"gid":8718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.2.104:6833/488029782","addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6832","nonce":488029782},{"type":"v1","addr":"172.21.2.104:6833","nonce":488029782}]},"join_fscid":-1,"export_targets":[],"features":4540138292840890367,"flags":0,"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":19,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-07-11T04:02:36.673961+0000","modified":"2021-07-11T04:02:52.404205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"min_compat_client":"0 (unknown)","max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":162,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":15}]} 2021-07-11T04:02:52.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 -- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc5880531f0 msgr2=0x7fc5880556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:52.597 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc5880531f0 0x7fc5880556b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fc594002fb0 tx=0x7fc5940069b0).stop 2021-07-11T04:02:52.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 -- 172.21.2.104:0/4122062122 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5a4064870 msgr2=0x7fc5a4144f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:52.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5a4064870 0x7fc5a4144f90 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 rx=0x7fc58c002f70 tx=0x7fc58c015000).stop 2021-07-11T04:02:52.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 -- 172.21.2.104:0/4122062122 shutdown_connections 2021-07-11T04:02:52.598 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc5a40b7810 0x7fc5a4146dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:52.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc5a40b1170 0x7fc5a4152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:52.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc5880531f0 0x7fc5880556b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:52.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 --2- 172.21.2.104:0/4122062122 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc5a4064870 0x7fc5a4144f90 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:52.599 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 -- 172.21.2.104:0/4122062122 >> 172.21.2.104:0/4122062122 conn(0x7fc5a405f6b0 msgr2=0x7fc5a4140d70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:52.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 -- 172.21.2.104:0/4122062122 shutdown_connections 2021-07-11T04:02:52.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.596+0000 7fc5aa054700 1 -- 172.21.2.104:0/4122062122 wait complete. 2021-07-11T04:02:52.600 INFO:teuthology.orchestra.run.gibba004.stderr:dumped fsmap epoch 81 2021-07-11T04:02:52.603 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2021-07-11T04:02:52.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e7c99700 1 Processor -- start 2021-07-11T04:02:52.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e7c99700 1 -- start start 2021-07-11T04:02:52.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e7c99700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f58e00b1c60 0x7f58e00af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e7c99700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f58e00b22f0 0x7f58e013cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e7c99700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e00b28b0 con 0x7f58e0061d20 2021-07-11T04:02:52.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e7c99700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e0062100 con 0x7f58e00b1c60 2021-07-11T04:02:52.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e7c99700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e00b2ae0 con 0x7f58e00b22f0 2021-07-11T04:02:52.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e5234700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f58e00b1c60 0x7f58e00af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:52.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e5234700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f58e00b1c60 0x7f58e00af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52150/0 (socket says 172.21.2.104:52150) 2021-07-11T04:02:52.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e5234700 1 -- 172.21.2.104:0/3428388278 learned_addr learned my addr 172.21.2.104:0/3428388278 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:52.661 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e5234700 1 -- 172.21.2.104:0/3428388278 >> v2:172.21.2.105:3300/0 conn(0x7f58e00b22f0 msgr2=0x7f58e013cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:52.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e5234700 1 --2- 172.21.2.104:0/3428388278 >> v2:172.21.2.105:3300/0 conn(0x7f58e00b22f0 0x7f58e013cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:52.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e5234700 1 -- 172.21.2.104:0/3428388278 >> v1:172.21.2.105:6789/0 conn(0x7f58e0061d20 legacy=0x7f58e00afc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:52.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e5234700 1 -- 172.21.2.104:0/3428388278 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e00b2d80 con 0x7f58e00b1c60 2021-07-11T04:02:52.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e5234700 1 --2- 172.21.2.104:0/3428388278 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f58e00b1c60 0x7f58e00af690 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7f58e00b6590 tx=0x7f58dc006f40).ready entity=mon.0 client_cookie=41331383fbc179a2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:52.662 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e4a33700 1 -- 172.21.2.104:0/3428388278 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58dc00dc30 con 0x7f58e00b1c60 2021-07-11T04:02:52.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e4a33700 1 -- 172.21.2.104:0/3428388278 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc000e90 con 0x7f58e00b1c60 2021-07-11T04:02:52.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e4a33700 1 -- 172.21.2.104:0/3428388278 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58dc00be90 con 0x7f58e00b1c60 2021-07-11T04:02:52.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e7c99700 1 -- 172.21.2.104:0/3428388278 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f58e00b1c60 msgr2=0x7f58e00af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:52.663 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.656+0000 7f58e7c99700 1 --2- 172.21.2.104:0/3428388278 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f58e00b1c60 0x7f58e00af690 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7f58e00b6590 tx=0x7f58dc006f40).stop 2021-07-11T04:02:52.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 -- 172.21.2.104:0/3428388278 shutdown_connections 2021-07-11T04:02:52.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 --2- 172.21.2.104:0/3428388278 >> v2:172.21.2.105:3300/0 conn(0x7f58e00b22f0 0x7f58e013cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:52.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 --2- 172.21.2.104:0/3428388278 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f58e00b1c60 0x7f58e00af690 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:52.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 -- 172.21.2.104:0/3428388278 >> 172.21.2.104:0/3428388278 conn(0x7f58e00639b0 msgr2=0x7f58e0063d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:52.664 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 -- 172.21.2.104:0/3428388278 shutdown_connections 2021-07-11T04:02:52.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 -- 172.21.2.104:0/3428388278 wait complete. 2021-07-11T04:02:52.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 Processor -- start 2021-07-11T04:02:52.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 -- start start 2021-07-11T04:02:52.665 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f58e0061d20 0x7f58e00b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f58e00b1c60 0x7f58e0143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f58e00b22f0 0x7f58e00666a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e00647c0 con 0x7f58e00b1c60 2021-07-11T04:02:52.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e0064a60 con 0x7f58e0061d20 2021-07-11T04:02:52.666 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e0064d00 con 0x7f58e00b22f0 2021-07-11T04:02:52.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e5234700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f58e00b1c60 0x7f58e0143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:52.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e5234700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f58e00b1c60 0x7f58e0143c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52158/0 (socket says 172.21.2.104:52158) 2021-07-11T04:02:52.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e5234700 1 -- 172.21.2.104:0/2291544864 learned_addr learned my addr 172.21.2.104:0/2291544864 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:52.667 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e5234700 1 -- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f58e00b22f0 msgr2=0x7f58e00666a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:52.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e5234700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f58e00b22f0 0x7f58e00666a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:52.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e5234700 1 -- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f58e0061d20 msgr2=0x7f58e00b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:52.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e5234700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f58e0061d20 0x7f58e00b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:52.668 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e5234700 1 -- 172.21.2.104:0/2291544864 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc00b460 con 0x7f58e00b1c60 2021-07-11T04:02:52.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e5234700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f58e00b1c60 0x7f58e0143c80 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7f58e00b6590 tx=0x7f58e013ff40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:52.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58ceffd700 1 -- 172.21.2.104:0/2291544864 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58dc022430 con 0x7f58e00b1c60 2021-07-11T04:02:52.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58ceffd700 1 -- 172.21.2.104:0/2291544864 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc022590 con 0x7f58e00b1c60 2021-07-11T04:02:52.669 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 -- 172.21.2.104:0/2291544864 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e0064fa0 con 0x7f58e00b1c60 2021-07-11T04:02:52.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 -- 172.21.2.104:0/2291544864 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e0144170 con 0x7f58e00b1c60 2021-07-11T04:02:52.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58ceffd700 1 -- 172.21.2.104:0/2291544864 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f58dc0228f0 con 0x7f58e00b1c60 2021-07-11T04:02:52.670 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58e7c99700 1 -- 172.21.2.104:0/2291544864 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc003410 con 0x7f58e00b1c60 2021-07-11T04:02:52.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58ceffd700 1 -- 172.21.2.104:0/2291544864 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f58dc029070 con 0x7f58e00b1c60 2021-07-11T04:02:52.672 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58ceffd700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f58c0053300 0x7f58c00557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:52.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.660+0000 7f58ceffd700 1 -- 172.21.2.104:0/2291544864 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f58dc06c990 con 0x7f58e00b1c60 2021-07-11T04:02:52.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.664+0000 7f58e5a35700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f58c0053300 0x7f58c00557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:52.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.664+0000 7f58e5a35700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f58c0053300 0x7f58c00557c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f58d0002f90 tx=0x7f58d00069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:52.673 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.664+0000 7f58ceffd700 1 -- 172.21.2.104:0/2291544864 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f58dc043b70 con 0x7f58e00b1c60 2021-07-11T04:02:52.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:52.772+0000 7f58e7c99700 1 -- 172.21.2.104:0/2291544864 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f58bc003c40 con 0x7f58e00b1c60 2021-07-11T04:02:53.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58ceffd700 1 -- 172.21.2.104:0/2291544864 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 104+0+0 (secure 0 0 0) 0x7f58dc027020 con 0x7f58e00b1c60 2021-07-11T04:02:53.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 -- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f58c0053300 msgr2=0x7f58c00557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:53.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f58c0053300 0x7f58c00557c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f58d0002f90 tx=0x7f58d00069b0).stop 2021-07-11T04:02:53.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 -- 172.21.2.104:0/2291544864 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f58e00b1c60 msgr2=0x7f58e0143c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:53.428 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f58e00b1c60 0x7f58e0143c80 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7f58e00b6590 tx=0x7f58e013ff40).stop 2021-07-11T04:02:53.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 -- 172.21.2.104:0/2291544864 shutdown_connections 2021-07-11T04:02:53.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f58e00b22f0 0x7f58e00666a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:53.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f58c0053300 0x7f58c00557c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:53.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f58e00b1c60 0x7f58e0143c80 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:53.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 --2- 172.21.2.104:0/2291544864 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f58e0061d20 0x7f58e00b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:53.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 -- 172.21.2.104:0/2291544864 >> 172.21.2.104:0/2291544864 conn(0x7f58e00639b0 msgr2=0x7f58e013b600 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:53.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 -- 172.21.2.104:0/2291544864 shutdown_connections 2021-07-11T04:02:53.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.424+0000 7f58e7c99700 1 -- 172.21.2.104:0/2291544864 wait complete. 2021-07-11T04:02:53.434 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2021-07-11T04:02:53.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.496+0000 7f5949e00700 1 Processor -- start 2021-07-11T04:02:53.499 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.496+0000 7f5949e00700 1 -- start start 2021-07-11T04:02:53.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.496+0000 7f5949e00700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f59440b1db0 0x7f59440b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:53.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.496+0000 7f5949e00700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f59440a70c0 0x7f5944144720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:53.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.496+0000 7f5949e00700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59440b0350 con 0x7f59441483a0 2021-07-11T04:02:53.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.496+0000 7f5949e00700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59440a90c0 con 0x7f59440b1db0 2021-07-11T04:02:53.500 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.496+0000 7f5949e00700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59440b05f0 con 0x7f59440a70c0 2021-07-11T04:02:53.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5942ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f59441483a0 0x7f59440b2190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48932/0 (socket says 172.21.2.104:48932) 2021-07-11T04:02:53.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5942ffd700 1 -- 172.21.2.104:0/2072593186 learned_addr learned my addr 172.21.2.104:0/2072593186 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:53.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f59437fe700 1 --2- 172.21.2.104:0/2072593186 >> v2:172.21.2.104:3300/0 conn(0x7f59440b1db0 0x7f59440b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:53.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f59427fc700 1 -- 172.21.2.104:0/2072593186 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 505669246 0 0) 0x7f59440b0350 con 0x7f59441483a0 2021-07-11T04:02:53.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f59427fc700 1 -- 172.21.2.104:0/2072593186 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5930003260 con 0x7f59441483a0 2021-07-11T04:02:53.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f59437fe700 1 -- 172.21.2.104:0/2072593186 >> v2:172.21.2.105:3300/0 conn(0x7f59440a70c0 msgr2=0x7f5944144720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:53.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f59437fe700 1 --2- 172.21.2.104:0/2072593186 >> v2:172.21.2.105:3300/0 conn(0x7f59440a70c0 0x7f5944144720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:53.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f59437fe700 1 -- 172.21.2.104:0/2072593186 >> v1:172.21.2.104:6789/0 conn(0x7f59441483a0 legacy=0x7f59440b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:53.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f59437fe700 1 -- 172.21.2.104:0/2072593186 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440b0890 con 0x7f59440b1db0 2021-07-11T04:02:53.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f59437fe700 1 --2- 172.21.2.104:0/2072593186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59440b1db0 0x7f59440b0fd0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 rx=0x7f5944147730 tx=0x7f5934006df0).ready entity=mon.0 client_cookie=bccac46c55231bb5 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:53.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f59427fc700 1 -- 172.21.2.104:0/2072593186 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f593400d5c0 con 0x7f59440b1db0 2021-07-11T04:02:53.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f59427fc700 1 -- 172.21.2.104:0/2072593186 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5934000e90 con 0x7f59440b1db0 2021-07-11T04:02:53.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f59427fc700 1 -- 172.21.2.104:0/2072593186 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f593400d910 con 0x7f59440b1db0 2021-07-11T04:02:53.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 -- 172.21.2.104:0/2072593186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59440b1db0 msgr2=0x7f59440b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:53.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 --2- 172.21.2.104:0/2072593186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59440b1db0 0x7f59440b0fd0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 rx=0x7f5944147730 tx=0x7f5934006df0).stop 2021-07-11T04:02:53.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 -- 172.21.2.104:0/2072593186 shutdown_connections 2021-07-11T04:02:53.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 --2- 172.21.2.104:0/2072593186 >> v2:172.21.2.105:3300/0 conn(0x7f59440a70c0 0x7f5944144720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:53.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 --2- 172.21.2.104:0/2072593186 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59440b1db0 0x7f59440b0fd0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:53.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 -- 172.21.2.104:0/2072593186 >> 172.21.2.104:0/2072593186 conn(0x7f594405f6b0 msgr2=0x7f594405fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:53.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 -- 172.21.2.104:0/2072593186 shutdown_connections 2021-07-11T04:02:53.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 -- 172.21.2.104:0/2072593186 wait complete. 2021-07-11T04:02:53.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 Processor -- start 2021-07-11T04:02:53.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 -- start start 2021-07-11T04:02:53.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f59440a70c0 0x7f5944064f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:53.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59440b1db0 0x7f59440aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:53.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59441483a0 0x7f5944156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:53.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5944144360 con 0x7f59441483a0 2021-07-11T04:02:53.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59440657a0 con 0x7f59440a70c0 2021-07-11T04:02:53.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594411d390 con 0x7f59440b1db0 2021-07-11T04:02:53.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5943fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59441483a0 0x7f5944156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:53.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5943fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59441483a0 0x7f5944156830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52172/0 (socket says 172.21.2.104:52172) 2021-07-11T04:02:53.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5943fff700 1 -- 172.21.2.104:0/1760817328 learned_addr learned my addr 172.21.2.104:0/1760817328 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:53.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5943fff700 1 -- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59440b1db0 msgr2=0x7f59440aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:53.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5943fff700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59440b1db0 0x7f59440aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:53.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5943fff700 1 -- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f59440a70c0 msgr2=0x7f5944064f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:53.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5943fff700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f59440a70c0 0x7f5944064f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:53.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5943fff700 1 -- 172.21.2.104:0/1760817328 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593400b180 con 0x7f59441483a0 2021-07-11T04:02:53.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5943fff700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59441483a0 0x7f5944156830 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 rx=0x7f593c0032f0 tx=0x7f593c009220).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:53.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5940ff9700 1 -- 172.21.2.104:0/1760817328 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f593c00d580 con 0x7f59441483a0 2021-07-11T04:02:53.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5940ff9700 1 -- 172.21.2.104:0/1760817328 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c00d6e0 con 0x7f59441483a0 2021-07-11T04:02:53.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 -- 172.21.2.104:0/1760817328 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594411d630 con 0x7f59441483a0 2021-07-11T04:02:53.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5940ff9700 1 -- 172.21.2.104:0/1760817328 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f593c00da40 con 0x7f59441483a0 2021-07-11T04:02:53.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.500+0000 7f5949e00700 1 -- 172.21.2.104:0/1760817328 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5944065440 con 0x7f59441483a0 2021-07-11T04:02:53.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.504+0000 7f5949e00700 1 -- 172.21.2.104:0/1760817328 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5944157680 con 0x7f59441483a0 2021-07-11T04:02:53.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.504+0000 7f5940ff9700 1 -- 172.21.2.104:0/1760817328 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f593c00dba0 con 0x7f59441483a0 2021-07-11T04:02:53.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.504+0000 7f5940ff9700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5924053270 0x7f5924055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:53.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.504+0000 7f5940ff9700 1 -- 172.21.2.104:0/1760817328 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6155+0+0 (secure 0 0 0) 0x7f593c06b890 con 0x7f59441483a0 2021-07-11T04:02:53.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.504+0000 7f59437fe700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5924053270 0x7f5924055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:53.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.504+0000 7f59437fe700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5924053270 0x7f5924055730 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f5944147730 tx=0x7f5934014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:53.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.508+0000 7f5940ff9700 1 -- 172.21.2.104:0/1760817328 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f5944157680 con 0x7f59441483a0 2021-07-11T04:02:53.588 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:53.588+0000 7f5949e00700 1 -- 172.21.2.104:0/1760817328 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5944157680 con 0x7f59441483a0 2021-07-11T04:02:54.427 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5940ff9700 1 -- 172.21.2.104:0/1760817328 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v163) v1 ==== 185+0+0 (secure 0 0 0) 0x7f5944157680 con 0x7f59441483a0 2021-07-11T04:02:54.429 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 -- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5924053270 msgr2=0x7f5924055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:54.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5924053270 0x7f5924055730 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f5944147730 tx=0x7f5934014000).stop 2021-07-11T04:02:54.430 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 -- 172.21.2.104:0/1760817328 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59441483a0 msgr2=0x7f5944156830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:54.431 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59441483a0 0x7f5944156830 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 rx=0x7f593c0032f0 tx=0x7f593c009220).stop 2021-07-11T04:02:54.432 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 -- 172.21.2.104:0/1760817328 shutdown_connections 2021-07-11T04:02:54.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5924053270 0x7f5924055730 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:54.433 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f59441483a0 0x7f5944156830 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:54.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f59440b1db0 0x7f59440aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:54.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 --2- 172.21.2.104:0/1760817328 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f59440a70c0 0x7f5944064f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:54.434 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 -- 172.21.2.104:0/1760817328 >> 172.21.2.104:0/1760817328 conn(0x7f594405f6b0 msgr2=0x7f59440aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:54.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 -- 172.21.2.104:0/1760817328 shutdown_connections 2021-07-11T04:02:54.435 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.428+0000 7f5949e00700 1 -- 172.21.2.104:0/1760817328 wait complete. 2021-07-11T04:02:54.436 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_metadata' removed 2021-07-11T04:02:54.438 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete cephfs_data cephfs_data --yes-i-really-really-mean-it 2021-07-11T04:02:54.494 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 Processor -- start 2021-07-11T04:02:54.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 -- start start 2021-07-11T04:02:54.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc47c0b04a0 0x7fc47c0add30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:54.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0b0880 0x7fc47c1405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:54.495 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47c0b2af0 con 0x7fc47c0aa120 2021-07-11T04:02:54.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47c0b2770 con 0x7fc47c0b04a0 2021-07-11T04:02:54.496 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0b2d20 con 0x7fc47c0b0880 2021-07-11T04:02:54.501 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47bfff700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fc47c0aa120 0x7fc47c0ae220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:48948/0 (socket says 172.21.2.104:48948) 2021-07-11T04:02:54.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47bfff700 1 -- 172.21.2.104:0/640536778 learned_addr learned my addr 172.21.2.104:0/640536778 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:02:54.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36114861 0 0) 0x7fc47c0b2af0 con 0x7fc47c0aa120 2021-07-11T04:02:54.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc468003260 con 0x7fc47c0aa120 2021-07-11T04:02:54.502 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038937289 0 0) 0x7fc468003260 con 0x7fc47c0aa120 2021-07-11T04:02:54.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0b0880 msgr2=0x7fc47c1405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:54.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc481328700 1 --2- 172.21.2.104:0/640536778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0b0880 0x7fc47c1405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:54.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 --2- 172.21.2.104:0/640536778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0b0880 0x7fc47c1405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:54.503 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 >> v2:172.21.2.105:3300/0 conn(0x7fc47c0b04a0 msgr2=0x7fc47c0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:54.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc480b27700 1 --2- 172.21.2.104:0/640536778 >> v2:172.21.2.105:3300/0 conn(0x7fc47c0b04a0 0x7fc47c0add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:54.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 --2- 172.21.2.104:0/640536778 >> v2:172.21.2.105:3300/0 conn(0x7fc47c0b04a0 0x7fc47c0add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:54.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c0a9b70 con 0x7fc47c0aa120 2021-07-11T04:02:54.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fc4740019e0 con 0x7fc47c0aa120 2021-07-11T04:02:54.504 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 >> v1:172.21.2.104:6789/0 conn(0x7fc47c0aa120 legacy=0x7fc47c0ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:54.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 --2- 172.21.2.104:0/640536778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4680037d0 0x7fc468003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:54.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 --2- 172.21.2.104:0/640536778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0b0880 0x7fc468004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:54.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 --2- 172.21.2.104:0/640536778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc468004bf0 0x7fc46800d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:54.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0b2770 con 0x7fc4680037d0 2021-07-11T04:02:54.505 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0b2d20 con 0x7fc468004bf0 2021-07-11T04:02:54.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc468005330 con 0x7fc47c0b0880 2021-07-11T04:02:54.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc480b27700 1 --2- 172.21.2.104:0/640536778 >> v2:172.21.2.105:3300/0 conn(0x7fc47c0b04a0 0x7fc47c0add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:02:54.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc480b27700 1 --2- 172.21.2.104:0/640536778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4680037d0 0x7fc468003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:54.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47bfff700 1 --2- 172.21.2.104:0/640536778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0b0880 0x7fc468004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:54.506 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc480b27700 1 -- 172.21.2.104:0/640536778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0b0880 msgr2=0x7fc468004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:54.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc480b27700 1 --2- 172.21.2.104:0/640536778 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0b0880 0x7fc468004500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:54.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc480b27700 1 -- 172.21.2.104:0/640536778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc468004bf0 msgr2=0x7fc46800d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:02:54.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc480b27700 1 --2- 172.21.2.104:0/640536778 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc468004bf0 0x7fc46800d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:54.507 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc480b27700 1 -- 172.21.2.104:0/640536778 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468003260 con 0x7fc4680037d0 2021-07-11T04:02:54.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc480b27700 1 --2- 172.21.2.104:0/640536778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4680037d0 0x7fc468003bf0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7fc46c007000 tx=0x7fc46c008040).ready entity=mon.0 client_cookie=269dcd2ce99c00eb server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:54.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46c015070 con 0x7fc4680037d0 2021-07-11T04:02:54.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c011410 con 0x7fc4680037d0 2021-07-11T04:02:54.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc47b7fe700 1 -- 172.21.2.104:0/640536778 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46c011770 con 0x7fc4680037d0 2021-07-11T04:02:54.508 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 -- 172.21.2.104:0/640536778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4680037d0 msgr2=0x7fc468003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:54.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 --2- 172.21.2.104:0/640536778 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc4680037d0 0x7fc468003bf0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7fc46c007000 tx=0x7fc46c008040).stop 2021-07-11T04:02:54.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc481328700 1 -- 172.21.2.104:0/640536778 reap_dead start 2021-07-11T04:02:54.509 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 -- 172.21.2.104:0/640536778 shutdown_connections 2021-07-11T04:02:54.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 -- 172.21.2.104:0/640536778 >> 172.21.2.104:0/640536778 conn(0x7fc47c0607c0 msgr2=0x7fc47c060ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:54.510 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 -- 172.21.2.104:0/640536778 shutdown_connections 2021-07-11T04:02:54.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 -- 172.21.2.104:0/640536778 wait complete. 2021-07-11T04:02:54.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 Processor -- start 2021-07-11T04:02:54.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.496+0000 7fc482d8b700 1 -- start start 2021-07-11T04:02:54.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc482d8b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c140180 0x7fc47c142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:54.511 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc482d8b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc47c1427c0 0x7fc47c142c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:54.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc482d8b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c151700 0x7fc47c151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:54.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc482d8b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c13e8e0 con 0x7fc47c151700 2021-07-11T04:02:54.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc482d8b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c144c60 con 0x7fc47c1427c0 2021-07-11T04:02:54.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc482d8b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c140560 con 0x7fc47c140180 2021-07-11T04:02:54.512 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc481328700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c151700 0x7fc47c151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:54.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc481328700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c151700 0x7fc47c151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52192/0 (socket says 172.21.2.104:52192) 2021-07-11T04:02:54.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc481328700 1 -- 172.21.2.104:0/4214233196 learned_addr learned my addr 172.21.2.104:0/4214233196 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:02:54.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc481328700 1 -- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c140180 msgr2=0x7fc47c142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:54.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc481328700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c140180 0x7fc47c142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:54.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc481328700 1 -- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc47c1427c0 msgr2=0x7fc47c142c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:54.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc481328700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc47c1427c0 0x7fc47c142c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:54.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc481328700 1 -- 172.21.2.104:0/4214233196 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c14ed40 con 0x7fc47c151700 2021-07-11T04:02:54.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc481328700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c151700 0x7fc47c151b80 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 rx=0x7fc474001b40 tx=0x7fc470008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:54.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc479ffb700 1 -- 172.21.2.104:0/4214233196 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc47000ad00 con 0x7fc47c151700 2021-07-11T04:02:54.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc479ffb700 1 -- 172.21.2.104:0/4214233196 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47000ae60 con 0x7fc47c151700 2021-07-11T04:02:54.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc482d8b700 1 -- 172.21.2.104:0/4214233196 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47c14efe0 con 0x7fc47c151700 2021-07-11T04:02:54.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc479ffb700 1 -- 172.21.2.104:0/4214233196 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc470011600 con 0x7fc47c151700 2021-07-11T04:02:54.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc482d8b700 1 -- 172.21.2.104:0/4214233196 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47c145090 con 0x7fc47c151700 2021-07-11T04:02:54.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc482d8b700 1 -- 172.21.2.104:0/4214233196 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47c144dd0 con 0x7fc47c151700 2021-07-11T04:02:54.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc479ffb700 1 -- 172.21.2.104:0/4214233196 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc470011970 con 0x7fc47c151700 2021-07-11T04:02:54.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc479ffb700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053270 0x7fc45c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:02:54.516 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc479ffb700 1 -- 172.21.2.104:0/4214233196 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5717+0+0 (secure 0 0 0) 0x7fc47006b1f0 con 0x7fc47c151700 2021-07-11T04:02:54.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc480b27700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053270 0x7fc45c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:02:54.517 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.500+0000 7fc480b27700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053270 0x7fc45c055730 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fc46c007000 tx=0x7fc46c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:02:54.518 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.504+0000 7fc479ffb700 1 -- 172.21.2.104:0/4214233196 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc47c144dd0 con 0x7fc47c151700 2021-07-11T04:02:54.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:54.584+0000 7fc482d8b700 1 -- 172.21.2.104:0/4214233196 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc47c144dd0 con 0x7fc47c151700 2021-07-11T04:02:55.439 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc479ffb700 1 -- 172.21.2.104:0/4214233196 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v164) v1 ==== 173+0+0 (secure 0 0 0) 0x7fc47c144dd0 con 0x7fc47c151700 2021-07-11T04:02:55.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 -- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053270 msgr2=0x7fc45c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:55.441 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053270 0x7fc45c055730 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fc46c007000 tx=0x7fc46c008040).stop 2021-07-11T04:02:55.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 -- 172.21.2.104:0/4214233196 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c151700 msgr2=0x7fc47c151b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:02:55.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c151700 0x7fc47c151b80 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 rx=0x7fc474001b40 tx=0x7fc470008040).stop 2021-07-11T04:02:55.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 -- 172.21.2.104:0/4214233196 shutdown_connections 2021-07-11T04:02:55.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053270 0x7fc45c055730 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:55.442 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c151700 0x7fc47c151b80 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:02:55.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc47c1427c0 0x7fc47c142c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:55.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 --2- 172.21.2.104:0/4214233196 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c140180 0x7fc47c142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:02:55.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 -- 172.21.2.104:0/4214233196 >> 172.21.2.104:0/4214233196 conn(0x7fc47c0607c0 msgr2=0x7fc47c0b1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:02:55.443 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 -- 172.21.2.104:0/4214233196 shutdown_connections 2021-07-11T04:02:55.444 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:02:55.440+0000 7fc482d8b700 1 -- 172.21.2.104:0/4214233196 wait complete. 2021-07-11T04:02:55.444 INFO:teuthology.orchestra.run.gibba004.stderr:pool 'cephfs_data' removed 2021-07-11T04:02:55.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2021-07-11T04:02:55.448 DEBUG:teuthology.orchestra.run.gibba019:> cd /home/ubuntu/cephtest/mnt.0 && sudo bash -c 'stat -f .' 2021-07-11T04:03:13.208 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:03:13.212 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:03:13.215 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:03:13.219 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:03:25.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:03:25.553 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:03:25.554 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:03:43.262 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:03:43.266 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:03:43.270 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:03:43.275 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:03:55.455 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-07-11T04:03:55.461 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-07-11T04:04:13.321 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:04:13.326 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:04:13.329 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:04:13.333 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:04:43.372 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:04:43.376 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:04:43.379 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:04:43.382 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:05:13.422 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:05:13.427 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:05:13.431 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:05:13.435 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:05:43.475 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:05:43.485 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:05:43.489 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:05:43.493 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:06:13.533 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:06:13.537 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:06:13.539 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:06:13.543 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:06:43.584 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:06:43.588 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:06:43.591 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:06:43.595 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:07:13.649 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:07:13.653 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:07:13.659 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:07:13.668 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:07:43.737 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:07:43.742 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:07:43.745 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:07:43.749 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:08:13.797 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:08:13.801 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:08:13.804 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:08:13.807 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:08:43.863 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:08:43.868 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:08:43.871 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:08:43.875 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:09:13.915 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:09:13.919 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:09:13.923 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:09:13.926 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:09:43.976 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:09:43.980 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:09:43.983 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:09:43.986 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:10:14.028 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:10:14.032 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:10:14.039 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:10:14.043 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:10:44.103 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:10:44.107 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:10:44.110 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:10:44.118 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:11:14.182 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:11:14.186 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:11:14.189 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:11:14.192 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:11:44.237 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:11:44.242 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:11:44.245 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:11:44.249 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:12:14.307 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:12:14.311 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:12:14.316 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:12:14.319 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:12:44.385 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:12:44.389 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:12:44.392 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:12:44.395 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:13:14.435 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:13:14.439 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:13:14.443 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:13:14.446 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:13:44.507 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:13:44.512 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:13:44.518 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:13:44.522 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:14:14.585 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:14:14.589 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:14:14.592 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:14:14.596 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:14:44.663 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:14:44.667 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:14:44.670 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:14:44.674 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:15:14.717 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:15:14.721 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:15:14.723 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:15:14.727 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:15:44.767 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:15:44.771 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:15:44.774 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:15:44.777 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:16:14.826 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:16:14.830 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:16:14.833 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:16:14.838 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:16:44.879 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:16:44.889 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:16:44.897 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:16:44.913 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:17:14.968 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:17:14.973 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:17:14.976 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:17:14.980 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:17:45.089 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:17:45.093 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:17:45.095 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:17:45.099 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:18:15.137 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:18:15.140 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:18:15.142 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:18:15.146 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:18:25.559 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-07-11T04:18:25.560 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-07-11T04:18:25.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:18:25.571 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2021-07-11T04:18:25.572 DEBUG:teuthology.orchestra.run.gibba004:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:18:25.586 DEBUG:teuthology.orchestra.run.gibba005:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:18:25.600 DEBUG:teuthology.orchestra.run.gibba019:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:18:25.614 DEBUG:teuthology.orchestra.run.gibba023:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2021-07-11T04:18:25.628 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2021-07-11T04:18:25.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:18:25.629 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:18:25.629 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:18:45.192 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:18:45.201 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:18:45.212 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:18:45.222 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:19:15.270 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:19:15.274 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:19:15.277 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:19:15.280 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:19:45.323 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:19:45.336 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:19:45.339 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:19:45.343 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:20:15.383 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:20:15.388 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:20:15.391 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:20:15.394 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:20:45.439 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:20:45.443 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:20:45.447 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:20:45.451 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:21:15.498 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:21:15.506 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:21:15.514 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:21:15.520 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:21:45.564 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:21:45.568 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:21:45.571 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:21:45.575 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:22:15.644 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:22:15.648 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:22:15.651 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:22:15.655 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:22:45.715 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:22:45.719 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:22:45.722 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:22:45.726 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:23:15.769 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:23:15.774 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:23:15.777 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:23:15.780 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:23:45.841 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:23:45.845 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:23:45.848 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:23:45.852 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:24:15.897 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:24:15.901 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:24:15.904 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:24:15.907 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:24:45.947 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:24:45.951 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:24:45.954 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:24:45.957 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:25:15.999 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:25:16.003 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:25:16.007 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:25:16.010 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:25:46.049 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:25:46.053 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:25:46.056 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:25:46.060 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:26:16.134 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:26:16.143 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:26:16.147 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:26:16.150 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:26:46.199 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:26:46.203 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:26:46.206 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:26:46.210 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:27:16.280 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:27:16.283 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:27:16.287 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:27:16.291 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:27:46.358 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:27:46.361 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:27:46.365 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:27:46.368 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:28:16.427 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:28:16.430 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:28:16.435 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:28:16.441 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:28:46.506 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:28:46.510 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:28:46.513 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:28:46.516 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:29:16.559 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:29:16.563 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:29:16.566 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:29:16.571 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:29:46.656 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:29:46.660 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:29:46.663 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:29:46.667 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:30:16.723 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:30:16.727 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:30:16.730 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:30:16.733 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:30:46.875 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:30:46.879 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:30:46.882 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:30:46.886 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:31:16.925 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:31:16.928 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:31:16.935 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:31:16.938 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:31:46.977 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:31:46.994 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:31:47.013 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:31:47.031 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:32:17.096 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:32:17.111 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:32:17.118 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:32:17.127 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:32:47.234 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:32:47.268 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:32:47.272 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:32:47.277 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:33:17.327 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:33:17.350 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:33:17.373 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:33:17.399 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:33:25.662 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-07-11T04:33:25.664 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-07-11T04:33:25.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:33:25.671 INFO:tasks.cephfs_test_runner:test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) ... ERROR 2021-07-11T04:33:25.673 INFO:tasks.cephfs_test_runner:ERROR 2021-07-11T04:33:25.674 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.674 INFO:tasks.cephfs_test_runner:====================================================================== 2021-07-11T04:33:25.675 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:33:25.675 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-07-11T04:33:25.676 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-07-11T04:33:25.676 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-07-11T04:33:25.677 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-07-11T04:33:25.678 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-07-11T04:33:25.678 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-07-11T04:33:25.679 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-07-11T04:33:25.680 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.680 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-07-11T04:33:25.682 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.682 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-07-11T04:33:25.683 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2021-07-11T04:33:25.683 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2021-07-11T04:33:25.684 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2021-07-11T04:33:25.684 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2021-07-11T04:33:25.685 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-07-11T04:33:25.685 INFO:tasks.cephfs_test_runner: proc.wait() 2021-07-11T04:33:25.686 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 148, in wait 2021-07-11T04:33:25.687 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-07-11T04:33:25.687 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-07-11T04:33:25.688 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-07-11T04:33:25.688 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-07-11T04:33:25.689 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-07-11T04:33:25.689 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-07-11T04:33:25.690 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-07-11T04:33:25.691 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-07-11T04:33:25.692 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-07-11T04:33:25.693 INFO:tasks.cephfs_test_runner: for line in f: 2021-07-11T04:33:25.693 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-07-11T04:33:25.694 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-07-11T04:33:25.694 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-07-11T04:33:25.695 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-07-11T04:33:25.696 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-07-11T04:33:25.697 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-07-11T04:33:25.697 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-07-11T04:33:25.698 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-07-11T04:33:25.699 INFO:tasks.cephfs_test_runner:socket.timeout 2021-07-11T04:33:25.699 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.700 INFO:tasks.cephfs_test_runner:====================================================================== 2021-07-11T04:33:25.701 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:33:25.702 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-07-11T04:33:25.702 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-07-11T04:33:25.703 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-07-11T04:33:25.704 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-07-11T04:33:25.704 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-07-11T04:33:25.705 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-07-11T04:33:25.705 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-07-11T04:33:25.706 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.707 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-07-11T04:33:25.707 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.708 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-07-11T04:33:25.708 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2021-07-11T04:33:25.709 INFO:tasks.cephfs_test_runner: m.teardown() 2021-07-11T04:33:25.709 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2021-07-11T04:33:25.710 INFO:tasks.cephfs_test_runner: self.umount() 2021-07-11T04:33:25.710 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2021-07-11T04:33:25.711 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2021-07-11T04:33:25.712 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-07-11T04:33:25.712 INFO:tasks.cephfs_test_runner: proc.wait() 2021-07-11T04:33:25.713 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 148, in wait 2021-07-11T04:33:25.714 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-07-11T04:33:25.715 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-07-11T04:33:25.715 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-07-11T04:33:25.716 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-07-11T04:33:25.716 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-07-11T04:33:25.717 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-07-11T04:33:25.717 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-07-11T04:33:25.718 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-07-11T04:33:25.718 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-07-11T04:33:25.719 INFO:tasks.cephfs_test_runner: for line in f: 2021-07-11T04:33:25.720 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-07-11T04:33:25.720 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-07-11T04:33:25.721 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-07-11T04:33:25.721 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-07-11T04:33:25.722 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-07-11T04:33:25.723 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-07-11T04:33:25.726 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-07-11T04:33:25.727 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-07-11T04:33:25.727 INFO:tasks.cephfs_test_runner:socket.timeout 2021-07-11T04:33:25.728 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.728 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-07-11T04:33:25.729 INFO:tasks.cephfs_test_runner:Ran 12 tests in 2385.910s 2021-07-11T04:33:25.729 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.730 INFO:tasks.cephfs_test_runner:FAILED (errors=2) 2021-07-11T04:33:25.730 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.731 INFO:tasks.cephfs_test_runner:====================================================================== 2021-07-11T04:33:25.732 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:33:25.732 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-07-11T04:33:25.733 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-07-11T04:33:25.734 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-07-11T04:33:25.734 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-07-11T04:33:25.735 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-07-11T04:33:25.735 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-07-11T04:33:25.736 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-07-11T04:33:25.737 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.737 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-07-11T04:33:25.738 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.738 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-07-11T04:33:25.739 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/test_misc.py", line 28, in test_statfs_on_deleted_fs 2021-07-11T04:33:25.739 INFO:tasks.cephfs_test_runner: self.mount_a.umount_wait(force=True) 2021-07-11T04:33:25.740 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 326, in umount_wait 2021-07-11T04:33:25.740 INFO:tasks.cephfs_test_runner: if not (self.is_mounted() and self.fuse_daemon): 2021-07-11T04:33:25.741 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-07-11T04:33:25.741 INFO:tasks.cephfs_test_runner: proc.wait() 2021-07-11T04:33:25.742 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 148, in wait 2021-07-11T04:33:25.742 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-07-11T04:33:25.743 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-07-11T04:33:25.743 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-07-11T04:33:25.744 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-07-11T04:33:25.745 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-07-11T04:33:25.745 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-07-11T04:33:25.745 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-07-11T04:33:25.746 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-07-11T04:33:25.747 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-07-11T04:33:25.748 INFO:tasks.cephfs_test_runner: for line in f: 2021-07-11T04:33:25.748 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-07-11T04:33:25.749 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-07-11T04:33:25.750 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-07-11T04:33:25.750 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-07-11T04:33:25.751 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-07-11T04:33:25.751 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-07-11T04:33:25.752 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-07-11T04:33:25.752 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-07-11T04:33:25.753 INFO:tasks.cephfs_test_runner:socket.timeout 2021-07-11T04:33:25.753 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.755 INFO:tasks.cephfs_test_runner:====================================================================== 2021-07-11T04:33:25.755 INFO:tasks.cephfs_test_runner:ERROR: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:33:25.755 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2021-07-11T04:33:25.756 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-07-11T04:33:25.756 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr 2021-07-11T04:33:25.756 INFO:tasks.cephfs_test_runner: out = self.in_stderr_buffer.read(nbytes, self.timeout) 2021-07-11T04:33:25.756 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read 2021-07-11T04:33:25.757 INFO:tasks.cephfs_test_runner: raise PipeTimeout() 2021-07-11T04:33:25.757 INFO:tasks.cephfs_test_runner:paramiko.buffered_pipe.PipeTimeout 2021-07-11T04:33:25.757 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.757 INFO:tasks.cephfs_test_runner:During handling of the above exception, another exception occurred: 2021-07-11T04:33:25.758 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.758 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2021-07-11T04:33:25.758 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/cephfs_test_case.py", line 175, in tearDown 2021-07-11T04:33:25.759 INFO:tasks.cephfs_test_runner: m.teardown() 2021-07-11T04:33:25.759 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 408, in teardown 2021-07-11T04:33:25.759 INFO:tasks.cephfs_test_runner: self.umount() 2021-07-11T04:33:25.759 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 261, in umount 2021-07-11T04:33:25.760 INFO:tasks.cephfs_test_runner: if not self.is_mounted(): 2021-07-11T04:33:25.760 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted 2021-07-11T04:33:25.760 INFO:tasks.cephfs_test_runner: proc.wait() 2021-07-11T04:33:25.760 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 148, in wait 2021-07-11T04:33:25.761 INFO:tasks.cephfs_test_runner: greenlet.get(block=True,timeout=60) 2021-07-11T04:33:25.761 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get 2021-07-11T04:33:25.761 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception 2021-07-11T04:33:25.762 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise 2021-07-11T04:33:25.762 INFO:tasks.cephfs_test_runner: raise value.with_traceback(tb) 2021-07-11T04:33:25.762 INFO:tasks.cephfs_test_runner: File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run 2021-07-11T04:33:25.762 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to 2021-07-11T04:33:25.763 INFO:tasks.cephfs_test_runner: copy_to_log(src, logger, capture=stream, quiet=quiet) 2021-07-11T04:33:25.763 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log 2021-07-11T04:33:25.763 INFO:tasks.cephfs_test_runner: for line in f: 2021-07-11T04:33:25.763 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ 2021-07-11T04:33:25.764 INFO:tasks.cephfs_test_runner: line = self.readline() 2021-07-11T04:33:25.764 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline 2021-07-11T04:33:25.764 INFO:tasks.cephfs_test_runner: new_data = self._read(n) 2021-07-11T04:33:25.764 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read 2021-07-11T04:33:25.765 INFO:tasks.cephfs_test_runner: return self.channel.recv_stderr(size) 2021-07-11T04:33:25.765 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr 2021-07-11T04:33:25.765 INFO:tasks.cephfs_test_runner: raise socket.timeout() 2021-07-11T04:33:25.766 INFO:tasks.cephfs_test_runner:socket.timeout 2021-07-11T04:33:25.766 INFO:tasks.cephfs_test_runner: 2021-07-11T04:33:25.766 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/run_tasks.py", line 94, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:33:25.858 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=46d3241dd84041ffb4290121306f6096 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/run_tasks.py", line 94, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc), test_statfs_on_deleted_fs (tasks.cephfs.test_misc.TestMisc) 2021-07-11T04:33:25.860 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2021-07-11T04:33:25.867 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2021-07-11T04:33:25.892 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2021-07-11T04:33:25.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2021-07-11T04:33:25.893 DEBUG:teuthology.orchestra.run.gibba019:> (cd /home/ubuntu/cephtest && exec stat --file-system '--printf=%T 2021-07-11T04:33:25.894 DEBUG:teuthology.orchestra.run.gibba019:> ' -- /home/ubuntu/cephtest/mnt.0) 2021-07-11T04:33:47.461 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:33:47.471 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:33:47.475 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:33:47.479 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:34:17.545 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:34:17.549 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:34:17.551 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:34:17.555 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:34:47.644 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:34:47.649 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:34:47.654 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:34:47.658 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:35:17.707 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:35:17.713 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:35:17.716 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:35:17.722 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:35:47.863 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:35:47.867 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:35:47.870 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:35:47.874 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:36:17.922 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:36:17.927 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:36:17.930 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:36:17.935 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:36:47.986 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:36:47.990 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:36:47.994 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:36:47.998 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:37:18.064 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:37:18.069 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:37:18.073 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:37:18.076 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:37:48.145 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:37:48.150 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:37:48.154 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:37:48.158 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:38:18.202 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:38:18.206 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:38:18.210 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:38:18.213 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:38:48.261 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:38:48.269 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:38:48.274 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:38:48.279 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:39:18.321 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:39:18.326 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:39:18.330 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:39:18.337 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:39:48.392 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:39:48.412 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:39:48.430 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:39:48.439 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:40:18.491 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:40:18.495 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:40:18.499 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:40:18.502 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:40:48.675 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:40:48.680 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:40:48.684 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:40:48.689 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:41:18.738 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:41:18.742 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:41:18.745 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:41:18.749 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:41:48.790 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:41:48.795 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:41:48.802 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:41:48.805 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:42:18.859 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:42:18.868 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:42:18.871 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:42:18.874 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:42:48.939 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:42:48.942 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:42:48.946 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:42:48.949 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:43:19.001 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:43:19.005 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:43:19.008 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:43:19.011 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:43:49.052 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:43:49.056 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:43:49.059 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:43:49.063 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:44:19.117 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:44:19.125 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:44:19.131 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:44:19.141 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:44:49.186 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:44:49.190 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:44:49.193 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:44:49.196 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:45:19.261 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:45:19.265 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:45:19.270 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:45:19.273 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:45:49.431 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:45:49.458 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:45:49.460 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:45:49.464 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:46:19.508 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:46:19.512 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:46:19.515 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:46:19.519 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:46:49.601 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:46:49.610 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:46:49.614 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:46:49.619 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:47:19.664 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:47:19.669 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:47:19.672 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:47:19.676 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:47:49.733 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:47:49.737 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:47:49.740 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:47:49.744 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:48:19.797 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:48:19.801 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:48:19.805 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:48:19.809 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:48:25.925 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-07-11T04:48:25.940 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-07-11T04:48:25.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:48:25.969 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/run_tasks.py", line 176, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-07-11T04:48:25.981 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2021-07-11T04:48:25.989 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2021-07-11T04:48:25.990 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:48:26.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1078215700 1 Processor -- start 2021-07-11T04:48:26.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1078215700 1 -- start start 2021-07-11T04:48:26.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1078215700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f1070067180 0x7f107005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1078215700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f10700b1970 0x7f107005fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1078215700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10700a7a90 con 0x7f10700a76b0 2021-07-11T04:48:26.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1078215700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1070147620 con 0x7f10700b1970 2021-07-11T04:48:26.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1078215700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10701478c0 con 0x7f1070067180 2021-07-11T04:48:26.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f10767b2700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f10700b1970 0x7f107005fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:26.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f10767b2700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f10700b1970 0x7f107005fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52230/0 (socket says 172.21.2.104:52230) 2021-07-11T04:48:26.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f10767b2700 1 -- 172.21.2.104:0/697314335 learned_addr learned my addr 172.21.2.104:0/697314335 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:26.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f10767b2700 1 -- 172.21.2.104:0/697314335 >> v2:172.21.2.105:3300/0 conn(0x7f1070067180 msgr2=0x7f107005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:26.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f10767b2700 1 --2- 172.21.2.104:0/697314335 >> v2:172.21.2.105:3300/0 conn(0x7f1070067180 0x7f107005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f10767b2700 1 -- 172.21.2.104:0/697314335 >> v1:172.21.2.105:6789/0 conn(0x7f10700a76b0 legacy=0x7f1070066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:26.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f10767b2700 1 -- 172.21.2.104:0/697314335 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1070147b60 con 0x7f10700b1970 2021-07-11T04:48:26.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f10767b2700 1 --2- 172.21.2.104:0/697314335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10700b1970 0x7f107005fba0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7f107006a0e0 tx=0x7f1068009290).ready entity=mon.0 client_cookie=4af9b801f5598a15 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:26.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1074faf700 1 -- 172.21.2.104:0/697314335 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f106800b830 con 0x7f10700b1970 2021-07-11T04:48:26.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1074faf700 1 -- 172.21.2.104:0/697314335 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106800b990 con 0x7f10700b1970 2021-07-11T04:48:26.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1074faf700 1 -- 172.21.2.104:0/697314335 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f106800bb00 con 0x7f10700b1970 2021-07-11T04:48:26.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1078215700 1 -- 172.21.2.104:0/697314335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10700b1970 msgr2=0x7f107005fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.052+0000 7f1078215700 1 --2- 172.21.2.104:0/697314335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10700b1970 0x7f107005fba0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7f107006a0e0 tx=0x7f1068009290).stop 2021-07-11T04:48:26.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 -- 172.21.2.104:0/697314335 shutdown_connections 2021-07-11T04:48:26.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 --2- 172.21.2.104:0/697314335 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10700b1970 0x7f107005fba0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:26.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 --2- 172.21.2.104:0/697314335 >> v2:172.21.2.105:3300/0 conn(0x7f1070067180 0x7f107005f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 -- 172.21.2.104:0/697314335 >> 172.21.2.104:0/697314335 conn(0x7f10700b53e0 msgr2=0x7f10700b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:26.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 -- 172.21.2.104:0/697314335 shutdown_connections 2021-07-11T04:48:26.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 -- 172.21.2.104:0/697314335 wait complete. 2021-07-11T04:48:26.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 Processor -- start 2021-07-11T04:48:26.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 -- start start 2021-07-11T04:48:26.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1070067180 0x7f10700b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10700a76b0 0x7f10700ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.069 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10700b1970 0x7f10700aaa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10700ab8b0 con 0x7f10700b1970 2021-07-11T04:48:26.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10700abae0 con 0x7f10700a76b0 2021-07-11T04:48:26.070 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10700abd80 con 0x7f1070067180 2021-07-11T04:48:26.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f10767b2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10700b1970 0x7f10700aaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:26.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f10767b2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10700b1970 0x7f10700aaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52236/0 (socket says 172.21.2.104:52236) 2021-07-11T04:48:26.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f10767b2700 1 -- 172.21.2.104:0/2700846584 learned_addr learned my addr 172.21.2.104:0/2700846584 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:26.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f10767b2700 1 -- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1070067180 msgr2=0x7f10700b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f10767b2700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1070067180 0x7f10700b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f10767b2700 1 -- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10700a76b0 msgr2=0x7f10700ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f10767b2700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10700a76b0 0x7f10700ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f10767b2700 1 -- 172.21.2.104:0/2700846584 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1068009530 con 0x7f10700b1970 2021-07-11T04:48:26.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f10767b2700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10700b1970 0x7f10700aaa00 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7f107006a0e0 tx=0x7f1068016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:26.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f105effd700 1 -- 172.21.2.104:0/2700846584 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1068011c60 con 0x7f10700b1970 2021-07-11T04:48:26.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f105effd700 1 -- 172.21.2.104:0/2700846584 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1068011dc0 con 0x7f10700b1970 2021-07-11T04:48:26.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f105effd700 1 -- 172.21.2.104:0/2700846584 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1068020610 con 0x7f10700b1970 2021-07-11T04:48:26.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 -- 172.21.2.104:0/2700846584 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10700ac020 con 0x7f10700b1970 2021-07-11T04:48:26.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 -- 172.21.2.104:0/2700846584 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10700ae190 con 0x7f10700b1970 2021-07-11T04:48:26.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.056+0000 7f1078215700 1 -- 172.21.2.104:0/2700846584 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1070057350 con 0x7f10700b1970 2021-07-11T04:48:26.075 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.060+0000 7f105effd700 1 -- 172.21.2.104:0/2700846584 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f1068027070 con 0x7f10700b1970 2021-07-11T04:48:26.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.060+0000 7f105effd700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10500531a0 0x7f1050055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.076 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.060+0000 7f105effd700 1 -- 172.21.2.104:0/2700846584 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f106806ba70 con 0x7f10700b1970 2021-07-11T04:48:26.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.060+0000 7f1075fb1700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10500531a0 0x7f1050055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:26.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.060+0000 7f1075fb1700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10500531a0 0x7f1050055660 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f106c000f70 tx=0x7f106c0077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:26.077 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.060+0000 7f105effd700 1 -- 172.21.2.104:0/2700846584 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f1068042f80 con 0x7f10700b1970 2021-07-11T04:48:26.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 -- 172.21.2.104:0/2700846584 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10700698e0 con 0x7f10700b1970 2021-07-11T04:48:26.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f105effd700 1 -- 172.21.2.104:0/2700846584 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+13320 (secure 0 0 0) 0x7f1068015070 con 0x7f10700b1970 2021-07-11T04:48:26.147 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:48:26.148 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":164,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:55.429316+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.104:6835/2638676458":"2021-07-12T04:02:51.605702+0000","172.21.2.104:6834/2638676458":"2021-07-12T04:02:51.605702+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:48:26.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 -- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10500531a0 msgr2=0x7f1050055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10500531a0 0x7f1050055660 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f106c000f70 tx=0x7f106c0077f0).stop 2021-07-11T04:48:26.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 -- 172.21.2.104:0/2700846584 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10700b1970 msgr2=0x7f10700aaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10700b1970 0x7f10700aaa00 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7f107006a0e0 tx=0x7f1068016000).stop 2021-07-11T04:48:26.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 -- 172.21.2.104:0/2700846584 shutdown_connections 2021-07-11T04:48:26.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f10500531a0 0x7f1050055660 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:26.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f10700b1970 0x7f10700aaa00 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:26.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f10700a76b0 0x7f10700ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 --2- 172.21.2.104:0/2700846584 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1070067180 0x7f10700b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 -- 172.21.2.104:0/2700846584 >> 172.21.2.104:0/2700846584 conn(0x7f10700b53e0 msgr2=0x7f1070148a00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:26.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 -- 172.21.2.104:0/2700846584 shutdown_connections 2021-07-11T04:48:26.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.144+0000 7f1078215700 1 -- 172.21.2.104:0/2700846584 wait complete. 2021-07-11T04:48:26.154 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-07-11T04:48:26.206 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f48a52cd700 1 Processor -- start 2021-07-11T04:48:26.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f48a52cd700 1 -- start start 2021-07-11T04:48:26.207 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f48a52cd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48a00a76b0 0x7f48a0066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f48a52cd700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f48a00b0060 0x7f48a005fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f48a52cd700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a00a7a90 con 0x7f48a00b1970 2021-07-11T04:48:26.208 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f48a52cd700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a0147620 con 0x7f48a00b0060 2021-07-11T04:48:26.209 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f48a52cd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a01478c0 con 0x7f48a00a76b0 2021-07-11T04:48:26.210 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f489f7fe700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f48a00b0060 0x7f48a005fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:26.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f489effd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48a00a76b0 0x7f48a0066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:26.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f489e7fc700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f48a00b1970 0x7f48a005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:48378/0 (socket says 172.21.2.104:48378) 2021-07-11T04:48:26.211 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f489e7fc700 1 -- 172.21.2.104:0/542589700 learned_addr learned my addr 172.21.2.104:0/542589700 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:48:26.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f489effd700 1 -- 172.21.2.104:0/542589700 >> v2:172.21.2.105:3300/0 conn(0x7f48a00b0060 msgr2=0x7f48a005fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:26.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f489effd700 1 --2- 172.21.2.104:0/542589700 >> v2:172.21.2.105:3300/0 conn(0x7f48a00b0060 0x7f48a005fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:26.212 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f489effd700 1 -- 172.21.2.104:0/542589700 >> v1:172.21.2.105:6789/0 conn(0x7f48a00b1970 legacy=0x7f48a005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f489effd700 1 -- 172.21.2.104:0/542589700 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a0147b60 con 0x7f48a00a76b0 2021-07-11T04:48:26.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.204+0000 7f489effd700 1 --2- 172.21.2.104:0/542589700 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48a00a76b0 0x7f48a0066a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f4890002f70 tx=0x7f4890006df0).ready entity=mon.2 client_cookie=69e4a9709ecf40e2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:26.213 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489dffb700 1 -- 172.21.2.104:0/542589700 <== mon.2 v2:172.21.2.105:3301/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f489000d570 con 0x7f48a00a76b0 2021-07-11T04:48:26.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489dffb700 1 -- 172.21.2.104:0/542589700 <== mon.2 v2:172.21.2.105:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4890000e90 con 0x7f48a00a76b0 2021-07-11T04:48:26.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489dffb700 1 -- 172.21.2.104:0/542589700 <== mon.2 v2:172.21.2.105:3301/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f489000d8c0 con 0x7f48a00a76b0 2021-07-11T04:48:26.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- 172.21.2.104:0/542589700 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48a00a76b0 msgr2=0x7f48a0066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 --2- 172.21.2.104:0/542589700 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48a00a76b0 0x7f48a0066a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f4890002f70 tx=0x7f4890006df0).stop 2021-07-11T04:48:26.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- 172.21.2.104:0/542589700 shutdown_connections 2021-07-11T04:48:26.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 --2- 172.21.2.104:0/542589700 >> v2:172.21.2.105:3300/0 conn(0x7f48a00b0060 0x7f48a005fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:26.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 --2- 172.21.2.104:0/542589700 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48a00a76b0 0x7f48a0066a70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:26.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- 172.21.2.104:0/542589700 >> 172.21.2.104:0/542589700 conn(0x7f48a00b53e0 msgr2=0x7f48a00b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:26.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- 172.21.2.104:0/542589700 shutdown_connections 2021-07-11T04:48:26.216 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- 172.21.2.104:0/542589700 wait complete. 2021-07-11T04:48:26.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 Processor -- start 2021-07-11T04:48:26.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- start start 2021-07-11T04:48:26.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48a00a76b0 0x7f48a005f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48a00b0060 0x7f48a0157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48a00b1970 0x7f48a00ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a01568c0 con 0x7f48a00b1970 2021-07-11T04:48:26.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a014d920 con 0x7f48a00b0060 2021-07-11T04:48:26.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a0156af0 con 0x7f48a00a76b0 2021-07-11T04:48:26.221 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48a00b1970 0x7f48a00ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:26.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48a00b1970 0x7f48a00ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52248/0 (socket says 172.21.2.104:52248) 2021-07-11T04:48:26.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489f7fe700 1 -- 172.21.2.104:0/2836169310 learned_addr learned my addr 172.21.2.104:0/2836169310 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:26.222 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489f7fe700 1 -- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48a00a76b0 msgr2=0x7f48a005f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489f7fe700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48a00a76b0 0x7f48a005f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489f7fe700 1 -- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48a00b0060 msgr2=0x7f48a0157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.223 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489f7fe700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48a00b0060 0x7f48a0157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489f7fe700 1 -- 172.21.2.104:0/2836169310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489000b180 con 0x7f48a00b1970 2021-07-11T04:48:26.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f489f7fe700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48a00b1970 0x7f48a00ae490 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 rx=0x7f4898006e00 tx=0x7f48980091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:26.224 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f4897fff700 1 -- 172.21.2.104:0/2836169310 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f489800c850 con 0x7f48a00b1970 2021-07-11T04:48:26.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f4897fff700 1 -- 172.21.2.104:0/2836169310 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489800c9b0 con 0x7f48a00b1970 2021-07-11T04:48:26.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- 172.21.2.104:0/2836169310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a0156d90 con 0x7f48a00b1970 2021-07-11T04:48:26.225 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f4897fff700 1 -- 172.21.2.104:0/2836169310 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f489800cd10 con 0x7f48a00b1970 2021-07-11T04:48:26.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- 172.21.2.104:0/2836169310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a00ae1c0 con 0x7f48a00b1970 2021-07-11T04:48:26.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f48a52cd700 1 -- 172.21.2.104:0/2836169310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a00aeb20 con 0x7f48a00b1970 2021-07-11T04:48:26.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f4897fff700 1 -- 172.21.2.104:0/2836169310 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4898022070 con 0x7f48a00b1970 2021-07-11T04:48:26.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.208+0000 7f4897fff700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48840531f0 0x7f48840556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.212+0000 7f4897fff700 1 -- 172.21.2.104:0/2836169310 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f489806b910 con 0x7f48a00b1970 2021-07-11T04:48:26.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.212+0000 7f489effd700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48840531f0 0x7f48840556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:26.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.212+0000 7f489effd700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48840531f0 0x7f48840556b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f4890002f70 tx=0x7f4890014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:26.228 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.212+0000 7f4897fff700 1 -- 172.21.2.104:0/2836169310 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f4898042f30 con 0x7f48a00b1970 2021-07-11T04:48:26.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.292+0000 7f48a52cd700 1 -- 172.21.2.104:0/2836169310 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48a014d6c0 con 0x7f48840531f0 2021-07-11T04:48:26.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.296+0000 7f4897fff700 1 -- 172.21.2.104:0/2836169310 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66465 (secure 0 0 0) 0x7f489000d570 con 0x7f48840531f0 2021-07-11T04:48:26.303 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:48:26.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 -- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48840531f0 msgr2=0x7f48840556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48840531f0 0x7f48840556b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f4890002f70 tx=0x7f4890014000).stop 2021-07-11T04:48:26.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 -- 172.21.2.104:0/2836169310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48a00b1970 msgr2=0x7f48a00ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48a00b1970 0x7f48a00ae490 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 rx=0x7f4898006e00 tx=0x7f48980091d0).stop 2021-07-11T04:48:26.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 -- 172.21.2.104:0/2836169310 shutdown_connections 2021-07-11T04:48:26.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f48840531f0 0x7f48840556b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:26.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f48a00b1970 0x7f48a00ae490 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:26.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f48a00b0060 0x7f48a0157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 --2- 172.21.2.104:0/2836169310 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f48a00a76b0 0x7f48a005f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 -- 172.21.2.104:0/2836169310 >> 172.21.2.104:0/2836169310 conn(0x7f48a00b53e0 msgr2=0x7f48a014a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:26.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 -- 172.21.2.104:0/2836169310 shutdown_connections 2021-07-11T04:48:26.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.300+0000 7f48a52cd700 1 -- 172.21.2.104:0/2836169310 wait complete. 2021-07-11T04:48:26.309 INFO:teuthology.orchestra.run.gibba004.stderr:dumped all 2021-07-11T04:48:26.310 INFO:teuthology.orchestra.run.gibba004.stdout:{"pg_ready":true,"pg_map":{"version":1812,"stamp":"2021-07-11T04:48:25.816815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404944,"kb_used_data":16272,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746569776,"statfs":{"total":773094113280,"available":764487450624,"internally_reserved":8589934592,"allocated":16662528,"data_stored":7731989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005166"},"pg_stats":[{"pgid":"1.0","version":"16'2","reported_seq":"159","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:54.424625+0000","last_change":"2021-07-11T03:53:15.964238+0000","last_active":"2021-07-11T04:02:54.424625+0000","last_peered":"2021-07-11T04:02:54.424625+0000","last_clean":"2021-07-11T04:02:54.424625+0000","last_became_active":"2021-07-11T03:53:15.962801+0000","last_became_peered":"2021-07-11T03:53:15.962801+0000","last_unstale":"2021-07-11T04:02:54.424625+0000","last_undegraded":"2021-07-11T04:02:54.424625+0000","last_fullsized":"2021-07-11T04:02:54.424625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:11.361771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:11.361771+0000","last_clean_scrub_stamp":"2021-07-11T03:53:11.361771+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"138","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:56.106047+0000","last_change":"2021-07-11T03:53:20.153399+0000","last_active":"2021-07-11T04:02:56.106047+0000","last_peered":"2021-07-11T04:02:56.106047+0000","last_clean":"2021-07-11T04:02:56.106047+0000","last_became_active":"2021-07-11T03:53:20.153135+0000","last_became_peered":"2021-07-11T03:53:20.153135+0000","last_unstale":"2021-07-11T04:02:56.106047+0000","last_undegraded":"2021-07-11T04:02:56.106047+0000","last_fullsized":"2021-07-11T04:02:56.106047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"154","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:54.424626+0000","last_change":"2021-07-11T03:53:20.152489+0000","last_active":"2021-07-11T04:02:54.424626+0000","last_peered":"2021-07-11T04:02:54.424626+0000","last_clean":"2021-07-11T04:02:54.424626+0000","last_became_active":"2021-07-11T03:53:20.152282+0000","last_became_peered":"2021-07-11T03:53:20.152282+0000","last_unstale":"2021-07-11T04:02:54.424626+0000","last_undegraded":"2021-07-11T04:02:54.424626+0000","last_fullsized":"2021-07-11T04:02:54.424626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"142","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:54.716394+0000","last_change":"2021-07-11T03:53:20.153985+0000","last_active":"2021-07-11T04:02:54.716394+0000","last_peered":"2021-07-11T04:02:54.716394+0000","last_clean":"2021-07-11T04:02:54.716394+0000","last_became_active":"2021-07-11T03:53:20.153695+0000","last_became_peered":"2021-07-11T03:53:20.153695+0000","last_unstale":"2021-07-11T04:02:54.716394+0000","last_undegraded":"2021-07-11T04:02:54.716394+0000","last_fullsized":"2021-07-11T04:02:54.716394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"138","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:56.106021+0000","last_change":"2021-07-11T03:53:20.153402+0000","last_active":"2021-07-11T04:02:56.106021+0000","last_peered":"2021-07-11T04:02:56.106021+0000","last_clean":"2021-07-11T04:02:56.106021+0000","last_became_active":"2021-07-11T03:53:20.153235+0000","last_became_peered":"2021-07-11T03:53:20.153235+0000","last_unstale":"2021-07-11T04:02:56.106021+0000","last_undegraded":"2021-07-11T04:02:56.106021+0000","last_fullsized":"2021-07-11T04:02:56.106021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"136","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:55.312108+0000","last_change":"2021-07-11T03:53:20.286895+0000","last_active":"2021-07-11T04:02:55.312108+0000","last_peered":"2021-07-11T04:02:55.312108+0000","last_clean":"2021-07-11T04:02:55.312108+0000","last_became_active":"2021-07-11T03:53:20.286695+0000","last_became_peered":"2021-07-11T03:53:20.286695+0000","last_unstale":"2021-07-11T04:02:55.312108+0000","last_undegraded":"2021-07-11T04:02:55.312108+0000","last_fullsized":"2021-07-11T04:02:55.312108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"154","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:54.424678+0000","last_change":"2021-07-11T03:53:20.288329+0000","last_active":"2021-07-11T04:02:54.424678+0000","last_peered":"2021-07-11T04:02:54.424678+0000","last_clean":"2021-07-11T04:02:54.424678+0000","last_became_active":"2021-07-11T03:53:20.287190+0000","last_became_peered":"2021-07-11T03:53:20.287190+0000","last_unstale":"2021-07-11T04:02:54.424678+0000","last_undegraded":"2021-07-11T04:02:54.424678+0000","last_fullsized":"2021-07-11T04:02:54.424678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"136","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:55.312181+0000","last_change":"2021-07-11T03:53:20.153427+0000","last_active":"2021-07-11T04:02:55.312181+0000","last_peered":"2021-07-11T04:02:55.312181+0000","last_clean":"2021-07-11T04:02:55.312181+0000","last_became_active":"2021-07-11T03:53:20.153186+0000","last_became_peered":"2021-07-11T03:53:20.153186+0000","last_unstale":"2021-07-11T04:02:55.312181+0000","last_undegraded":"2021-07-11T04:02:55.312181+0000","last_fullsized":"2021-07-11T04:02:55.312181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"136","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:55.905701+0000","last_change":"2021-07-11T03:53:20.153252+0000","last_active":"2021-07-11T04:02:55.905701+0000","last_peered":"2021-07-11T04:02:55.905701+0000","last_clean":"2021-07-11T04:02:55.905701+0000","last_became_active":"2021-07-11T03:53:20.153046+0000","last_became_peered":"2021-07-11T03:53:20.153046+0000","last_unstale":"2021-07-11T04:02:55.905701+0000","last_undegraded":"2021-07-11T04:02:55.905701+0000","last_fullsized":"2021-07-11T04:02:55.905701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542809,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2150000000000001,"15min":1.2410000000000001},"min":{"1min":0.46800000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.623,"5min":1.653,"15min":1.8620000000000001},"last":1.177},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2529999999999999,"15min":1.2350000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.313},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.542}]},{"osd":1,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2609999999999999,"15min":1.2589999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.33700000000000002},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.22,"15min":1.2370000000000001},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.5940000000000001,"5min":1.77,"15min":1.8400000000000001},"last":1.47}]},{"osd":2,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1850000000000001,"15min":1.198},"min":{"1min":0.371,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.6579999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.256},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.288,"15min":1.2629999999999999},"min":{"1min":0.53800000000000003,"5min":0.307,"15min":0.307},"max":{"1min":1.804,"5min":1.835,"15min":1.849},"last":1.6379999999999999}]},{"osd":3,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.44800000000000001,"5min":0.39200000000000002,"15min":0.34999999999999998},"max":{"1min":1.5620000000000001,"5min":1.867,"15min":1.944},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.337,"15min":1.343},"min":{"1min":0.51900000000000002,"5min":0.36499999999999999,"15min":0.34399999999999997},"max":{"1min":1.7470000000000001,"5min":1.847,"15min":1.974},"last":1.5740000000000001}]},{"osd":4,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.879,"15min":0.876},"min":{"1min":0.28299999999999997,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.4590000000000001,"5min":1.496,"15min":1.627},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.92300000000000004},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.2210000000000001,"5min":1.595,"15min":1.7250000000000001},"last":0.80900000000000005}]},{"osd":5,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.94499999999999995,"15min":0.92900000000000005},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.952},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.995,"5min":0.97899999999999998,"15min":0.97199999999999998},"min":{"1min":0.249,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.042,"15min":1.0429999999999999},"min":{"1min":0.29799999999999999,"5min":0.247,"15min":0.247},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.024},"min":{"1min":0.27700000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.5860000000000001,"5min":1.724,"15min":1.8129999999999999},"last":0.88200000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542809,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050616,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321224,"statfs":{"total":96636764160,"available":95560933376,"internally_reserved":1073741824,"allocated":2080768,"data_stored":966194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.262,"15min":1.244},"min":{"1min":0.84099999999999997,"5min":0.55900000000000005,"15min":0.31900000000000001},"max":{"1min":1.6060000000000001,"5min":1.79,"15min":1.976},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.2829999999999999,"15min":1.2589999999999999},"min":{"1min":0.92400000000000004,"5min":0.57599999999999996,"15min":0.36099999999999999},"max":{"1min":1.5069999999999999,"5min":1.821,"15min":1.849},"last":1.2190000000000001}]},{"osd":1,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.236,"15min":1.2250000000000001},"min":{"1min":0.93700000000000006,"5min":0.60999999999999999,"15min":0.28100000000000003},"max":{"1min":1.631,"5min":1.873,"15min":1.873},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.2729999999999999,"15min":1.2370000000000001},"min":{"1min":0.90600000000000003,"5min":0.58599999999999997,"15min":0.27600000000000002},"max":{"1min":1.653,"5min":1.786,"15min":1.786},"last":1.4470000000000001}]},{"osd":2,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2549999999999999,"15min":1.256},"min":{"1min":0.87,"5min":0.622,"15min":0.33200000000000002},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":2.0169999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2569999999999999,"15min":1.2370000000000001},"min":{"1min":0.86199999999999999,"5min":0.63100000000000001,"15min":0.29499999999999998},"max":{"1min":1.53,"5min":1.6439999999999999,"15min":1.7789999999999999},"last":1.3220000000000001}]},{"osd":3,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.337,"15min":1.3100000000000001},"min":{"1min":0.93000000000000005,"5min":0.59999999999999998,"15min":0.29999999999999999},"max":{"1min":1.6890000000000001,"5min":1.8320000000000001,"15min":1.8460000000000001},"last":1.228},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.375,"15min":1.3320000000000001},"min":{"1min":0.90000000000000002,"5min":0.59599999999999997,"15min":0.34999999999999998},"max":{"1min":1.921,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.2889999999999999}]},{"osd":4,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92400000000000004,"15min":0.90400000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.17799999999999999},"max":{"1min":1.4770000000000001,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.95799999999999996,"15min":0.92400000000000004},"min":{"1min":0.216,"5min":0.216,"15min":0.183},"max":{"1min":1.8680000000000001,"5min":1.913,"15min":1.913},"last":0.95899999999999996}]},{"osd":5,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93200000000000005,"15min":0.91900000000000004},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.17599999999999999},"max":{"1min":1.6020000000000001,"5min":1.6879999999999999,"15min":1.7789999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.012,"15min":0.98299999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.16},"max":{"1min":1.7430000000000001,"5min":1.871,"15min":1.9019999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.05,"15min":1.002},"min":{"1min":0.24299999999999999,"5min":0.218,"15min":0.19800000000000001},"max":{"1min":1.883,"5min":2.024,"15min":2.024},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.0649999999999999,"15min":1.0209999999999999},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.86099999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542809,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.347,"15min":1.306},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.76,"5min":1.8,"15min":1.843},"last":1.264},{"interface":"front","average":{"1min":1.385,"5min":1.363,"15min":1.3180000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.8100000000000001,"5min":1.96,"15min":1.96},"last":1.2250000000000001}]},{"osd":1,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.288,"15min":1.25},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.7110000000000001,"5min":1.7270000000000001,"15min":1.9359999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.292,"15min":1.258},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6990000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.2969999999999999}]},{"osd":2,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2809999999999999,"15min":1.238},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.645,"5min":1.8169999999999999,"15min":1.837},"last":1.365},{"interface":"front","average":{"1min":1.284,"5min":1.2729999999999999,"15min":1.252},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.6890000000000001,"5min":1.843,"15min":1.843},"last":1.2030000000000001}]},{"osd":3,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2589999999999999,"15min":1.2569999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3109999999999999,"15min":1.238},"min":{"1min":0.442,"5min":0.442,"15min":0.378},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.94299999999999995}]},{"osd":4,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.90100000000000002,"15min":0.878},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.218},"max":{"1min":1.512,"5min":1.595,"15min":1.657},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.071,"5min":0.96299999999999997,"15min":0.92600000000000005},"min":{"1min":0.29599999999999999,"5min":0.24299999999999999,"15min":0.21299999999999999},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.96699999999999997,"15min":0.92900000000000005},"min":{"1min":0.35799999999999998,"5min":0.27100000000000002,"15min":0.23400000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.01,"15min":0.97699999999999998},"min":{"1min":0.219,"5min":0.218,"15min":0.218},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.96299999999999997}]},{"osd":7,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.012,"15min":0.97999999999999998},"min":{"1min":0.33400000000000002,"5min":0.26400000000000001,"15min":0.23100000000000001},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.9970000000000001},"last":1.012},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.0940000000000001,"15min":1.0269999999999999},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.22},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.879}]}]},{"osd":4,"up_from":14,"seq":60129542810,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050616,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321224,"statfs":{"total":96636764160,"available":95560933376,"internally_reserved":1073741824,"allocated":2080768,"data_stored":966194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3220000000000001,"15min":1.2969999999999999},"min":{"1min":1.0329999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3100000000000001,"15min":1.2909999999999999},"min":{"1min":1.0569999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.732,"5min":1.798,"15min":1.8799999999999999},"last":1.254}]},{"osd":1,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.236,"15min":1.224},"min":{"1min":0.91600000000000004,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.641,"5min":1.6759999999999999,"15min":1.766},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.218,"15min":1.234},"min":{"1min":0.89800000000000002,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":1.6279999999999999,"5min":1.7230000000000001,"15min":1.79},"last":1.282}]},{"osd":2,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2769999999999999,"15min":1.26},"min":{"1min":0.99199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.506,"5min":1.6579999999999999,"15min":1.8129999999999999},"last":1.304},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.272,"15min":1.2729999999999999},"min":{"1min":1.0229999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.6679999999999999,"5min":1.758,"15min":1.758},"last":1.5109999999999999}]},{"osd":3,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2070000000000001,"15min":1.22},"min":{"1min":0.92600000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.728},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.248,"15min":1.2430000000000001},"min":{"1min":0.97499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5649999999999999,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.484}]},{"osd":5,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.84499999999999997,"15min":0.85399999999999998},"min":{"1min":0.55500000000000005,"5min":0.24099999999999999,"15min":0.22600000000000001},"max":{"1min":1.2989999999999999,"5min":1.5720000000000001,"15min":1.6459999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.91000000000000003,"15min":0.90000000000000002},"min":{"1min":0.63600000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.143,"5min":1.5109999999999999,"15min":1.6819999999999999},"last":0.92500000000000004}]},{"osd":6,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.91200000000000003},"min":{"1min":0.57699999999999996,"5min":0.27600000000000002,"15min":0.216},"max":{"1min":1.2929999999999999,"5min":1.613,"15min":1.7749999999999999},"last":1},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.95699999999999996,"15min":0.96399999999999997},"min":{"1min":0.69099999999999995,"5min":0.22,"15min":0.22},"max":{"1min":1.1539999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.032}]},{"osd":7,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.52900000000000003,"5min":0.26400000000000001,"15min":0.23100000000000001},"max":{"1min":1.3660000000000001,"5min":1.5860000000000001,"15min":1.7330000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0649999999999999,"15min":1.026},"min":{"1min":0.61699999999999999,"5min":0.26600000000000001,"15min":0.22},"max":{"1min":1.3260000000000001,"5min":1.6950000000000001,"15min":1.782},"last":1.1870000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542809,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.28699999999999998,"5min":0.27600000000000002,"15min":0.23300000000000001},"max":{"1min":0.97199999999999998,"5min":1.1379999999999999,"15min":1.4359999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80300000000000005,"15min":0.80200000000000005},"min":{"1min":0.27600000000000002,"5min":0.221,"15min":0.221},"max":{"1min":1.0309999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.88200000000000001}]},{"osd":1,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.30299999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93300000000000005,"5min":1.081,"15min":1.081},"last":0.754},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.83799999999999997,"15min":0.82599999999999996},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.22900000000000001},"max":{"1min":1.109,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.93300000000000005}]},{"osd":2,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.83199999999999996,"15min":0.81699999999999995},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.216},"max":{"1min":1.089,"5min":1.1819999999999999,"15min":1.625},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.89200000000000002,"15min":0.88500000000000001},"min":{"1min":0.32200000000000001,"5min":0.248,"15min":0.248},"max":{"1min":1.3080000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.82799999999999996}]},{"osd":4,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.361,"15min":1.3340000000000001},"min":{"1min":0.81799999999999995,"5min":0.72099999999999997,"15min":0.36399999999999999},"max":{"1min":1.6719999999999999,"5min":1.859,"15min":1.859},"last":1.669},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.339,"15min":1.3380000000000001},"min":{"1min":0.70299999999999996,"5min":0.68300000000000005,"15min":0.34799999999999998},"max":{"1min":1.6839999999999999,"5min":1.873,"15min":1.873},"last":1.712}]},{"osd":5,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.419,"15min":1.3799999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.42899999999999999},"max":{"1min":1.7789999999999999,"5min":1.875,"15min":1.875},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.3759999999999999,"15min":1.3660000000000001},"min":{"1min":0.73299999999999998,"5min":0.66700000000000004,"15min":0.41899999999999998},"max":{"1min":1.667,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.6220000000000001}]},{"osd":6,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.3420000000000001,"15min":1.3109999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.433},"max":{"1min":1.524,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.452},{"interface":"front","average":{"1min":1.304,"5min":1.3580000000000001,"15min":1.345},"min":{"1min":0.80600000000000005,"5min":0.67400000000000004,"15min":0.33700000000000002},"max":{"1min":1.5409999999999999,"5min":1.905,"15min":1.905},"last":1.4119999999999999}]},{"osd":7,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.361,"15min":1.341},"min":{"1min":0.753,"5min":0.70099999999999996,"15min":0.44700000000000001},"max":{"1min":1.75,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3720000000000001,"15min":1.3600000000000001},"min":{"1min":0.67300000000000004,"5min":0.64600000000000002,"15min":0.39900000000000002},"max":{"1min":1.708,"5min":1.7849999999999999,"15min":1.7869999999999999},"last":1.5489999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542810,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.83499999999999996,"15min":0.81999999999999995},"min":{"1min":0.58499999999999996,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":1.2430000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.88200000000000001,"15min":0.86299999999999999},"min":{"1min":0.61099999999999999,"5min":0.27300000000000002,"15min":0.252},"max":{"1min":1.157,"5min":1.417,"15min":1.417},"last":1.028}]},{"osd":1,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.44400000000000001,"5min":0.245,"15min":0.245},"max":{"1min":1.083,"5min":1.3500000000000001,"15min":1.5469999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.83499999999999996,"15min":0.82099999999999995},"min":{"1min":0.52100000000000002,"5min":0.26900000000000002,"15min":0.23799999999999999},"max":{"1min":1.089,"5min":1.2849999999999999,"15min":1.3},"last":0.81000000000000005}]},{"osd":3,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80500000000000005,"15min":0.79300000000000004},"min":{"1min":0.41799999999999998,"5min":0.23599999999999999,"15min":0.22900000000000001},"max":{"1min":1.0349999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.878},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.89500000000000002,"15min":0.88200000000000001},"min":{"1min":0.49099999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2010000000000001,"5min":1.375,"15min":1.573},"last":0.84299999999999997}]},{"osd":4,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3160000000000001,"15min":1.296},"min":{"1min":0.86299999999999999,"5min":0.66300000000000003,"15min":0.44800000000000001},"max":{"1min":1.538,"5min":1.794,"15min":1.877},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.395,"15min":1.3540000000000001},"min":{"1min":0.73899999999999999,"5min":0.623,"15min":0.48599999999999999},"max":{"1min":1.7210000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.1240000000000001}]},{"osd":5,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3640000000000001,"15min":1.335},"min":{"1min":0.82399999999999995,"5min":0.67200000000000004,"15min":0.441},"max":{"1min":1.6319999999999999,"5min":1.966,"15min":1.966},"last":1.411},{"interface":"front","average":{"1min":1.371,"5min":1.3580000000000001,"15min":1.335},"min":{"1min":0.90200000000000002,"5min":0.63800000000000001,"15min":0.41499999999999998},"max":{"1min":1.5820000000000001,"5min":1.7829999999999999,"15min":1.956},"last":1.385}]},{"osd":6,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.383,"15min":1.3560000000000001},"min":{"1min":0.80300000000000005,"5min":0.64400000000000002,"15min":0.52600000000000002},"max":{"1min":1.667,"5min":1.794,"15min":1.8360000000000001},"last":1.244},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.357},"min":{"1min":0.78600000000000003,"5min":0.58499999999999996,"15min":0.47599999999999998},"max":{"1min":1.7130000000000001,"5min":1.9079999999999999,"15min":2.0550000000000002},"last":1.536}]},{"osd":7,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3779999999999999,"15min":1.3560000000000001},"min":{"1min":0.76800000000000002,"5min":0.71599999999999997,"15min":0.47199999999999998},"max":{"1min":1.6040000000000001,"5min":1.992,"15min":1.992},"last":1.179},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3759999999999999,"15min":1.3460000000000001},"min":{"1min":0.85299999999999998,"5min":0.65300000000000002,"15min":0.53500000000000003},"max":{"1min":1.6899999999999999,"5min":1.8149999999999999,"15min":1.964},"last":1.49}]}]},{"osd":1,"up_from":14,"seq":60129542810,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79400000000000004,"15min":0.80100000000000005},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.1359999999999999,"5min":1.1459999999999999,"15min":1.2709999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85399999999999998,"15min":0.85899999999999999},"min":{"1min":0.26200000000000001,"5min":0.247,"15min":0.22600000000000001},"max":{"1min":1.1319999999999999,"5min":1.1519999999999999,"15min":1.1970000000000001},"last":0.83499999999999996}]},{"osd":2,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":1.151,"5min":1.216,"15min":1.216},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.78500000000000003,"15min":0.80900000000000005},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":1.069,"5min":1.234,"15min":1.234},"last":0.86599999999999999}]},{"osd":3,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.83099999999999996,"15min":0.86299999999999999},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.23100000000000001},"max":{"1min":1.111,"5min":1.143,"15min":1.3600000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.77700000000000002,"15min":0.78400000000000003},"min":{"1min":0.28799999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1160000000000001,"5min":1.2050000000000001,"15min":1.258},"last":0.90900000000000003}]},{"osd":4,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.321,"15min":1.3169999999999999},"min":{"1min":0.69399999999999995,"5min":0.51900000000000002,"15min":0.29599999999999999},"max":{"1min":1.6699999999999999,"5min":1.887,"15min":1.887},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.3280000000000001,"15min":1.321},"min":{"1min":0.65900000000000003,"5min":0.48799999999999999,"15min":0.28899999999999998},"max":{"1min":1.6499999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.296}]},{"osd":5,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3300000000000001,"15min":1.3240000000000001},"min":{"1min":0.78700000000000003,"5min":0.46700000000000003,"15min":0.25900000000000001},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.3140000000000001,"15min":1.304},"min":{"1min":0.73499999999999999,"5min":0.432,"15min":0.26400000000000001},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.452}]},{"osd":6,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.415,"15min":1.391},"min":{"1min":0.71699999999999997,"5min":0.44700000000000001,"15min":0.313},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.405},{"interface":"front","average":{"1min":1.48,"5min":1.391,"15min":1.393},"min":{"1min":0.74299999999999999,"5min":0.498,"15min":0.30599999999999999},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.379}]},{"osd":7,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3009999999999999,"15min":1.3169999999999999},"min":{"1min":0.63900000000000001,"5min":0.40999999999999998,"15min":0.34599999999999997},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.8600000000000001},"last":1.55},{"interface":"front","average":{"1min":1.363,"5min":1.343,"15min":1.3360000000000001},"min":{"1min":0.64600000000000002,"5min":0.55300000000000005,"15min":0.27100000000000002},"max":{"1min":1.7709999999999999,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.349}]}]},{"osd":0,"up_from":14,"seq":60129542808,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050612,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321228,"statfs":{"total":96636764160,"available":95560937472,"internally_reserved":1073741824,"allocated":2076672,"data_stored":965616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76400000000000001,"15min":0.78400000000000003},"min":{"1min":0.25600000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.94299999999999995,"5min":1.099,"15min":1.1890000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.26300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":1.1379999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.85399999999999998}]},{"osd":2,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81000000000000005,"15min":0.81899999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.21099999999999999},"max":{"1min":1.0800000000000001,"5min":1.129,"15min":1.2789999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.90000000000000002,"15min":0.91900000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.253},"max":{"1min":1.0720000000000001,"5min":1.252,"15min":1.4219999999999999},"last":0.97199999999999998}]},{"osd":3,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73399999999999999,"15min":0.76300000000000001},"min":{"1min":0.33400000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.454},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.219},"max":{"1min":1.095,"5min":1.1619999999999999,"15min":1.339},"last":1.0289999999999999}]},{"osd":4,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.329,"15min":1.3620000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.6359999999999999,"5min":1.7470000000000001,"15min":2.1030000000000002},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.331,"15min":1.343},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":1.6499999999999999,"5min":1.72,"15min":1.8899999999999999},"last":1.456}]},{"osd":5,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.321,"15min":1.3320000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.6459999999999999,"5min":1.8280000000000001,"15min":1.9399999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.33500000000000002},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":2.0289999999999999},"last":1.4410000000000001}]},{"osd":6,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3069999999999999,"15min":1.347},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6950000000000001,"5min":1.7929999999999999,"15min":1.944},"last":1.391},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2949999999999999,"15min":1.3240000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.5649999999999999,"5min":1.6950000000000001,"15min":1.8580000000000001},"last":1.6950000000000001}]},{"osd":7,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.335,"15min":1.353},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6479999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.349,"15min":1.363},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.39300000000000002},"max":{"1min":1.6779999999999999,"5min":1.8169999999999999,"15min":1.9199999999999999},"last":1.53}]}]}]}} 2021-07-11T04:48:26.313 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-07-11T04:48:26.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f156635a700 1 Processor -- start 2021-07-11T04:48:26.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f156635a700 1 -- start start 2021-07-11T04:48:26.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f156635a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f15600b2660 0x7f156005f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f156635a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1560096cc0 0x7f1560097100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f156635a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15600b2a40 con 0x7f156013f560 2021-07-11T04:48:26.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f156635a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15600975f0 con 0x7f15600b2660 2021-07-11T04:48:26.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f156635a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1560097b80 con 0x7f1560096cc0 2021-07-11T04:48:26.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f155ffff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f15600b2660 0x7f156005f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:26.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f15648f7700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f156013f560 0x7f156013f9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49024/0 (socket says 172.21.2.104:49024) 2021-07-11T04:48:26.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f15648f7700 1 -- 172.21.2.104:0/2182300861 learned_addr learned my addr 172.21.2.104:0/2182300861 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:48:26.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f155ffff700 1 -- 172.21.2.104:0/2182300861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1560096cc0 msgr2=0x7f1560097100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:48:26.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f155ffff700 1 --2- 172.21.2.104:0/2182300861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1560096cc0 0x7f1560097100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.371 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f155ffff700 1 -- 172.21.2.104:0/2182300861 >> v1:172.21.2.104:6789/0 conn(0x7f156013f560 legacy=0x7f156013f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f155ffff700 1 -- 172.21.2.104:0/2182300861 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1560097e20 con 0x7f15600b2660 2021-07-11T04:48:26.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f155ffff700 1 --2- 172.21.2.104:0/2182300861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15600b2660 0x7f156005f6b0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 rx=0x7f1548002f70 tx=0x7f1548006e80).ready entity=mon.0 client_cookie=af10a28e1e0067cf server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:26.372 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f155effd700 1 -- 172.21.2.104:0/2182300861 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154800ba70 con 0x7f15600b2660 2021-07-11T04:48:26.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f155effd700 1 -- 172.21.2.104:0/2182300861 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1548000e90 con 0x7f15600b2660 2021-07-11T04:48:26.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f155effd700 1 -- 172.21.2.104:0/2182300861 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f154800be00 con 0x7f15600b2660 2021-07-11T04:48:26.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f156635a700 1 -- 172.21.2.104:0/2182300861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15600b2660 msgr2=0x7f156005f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.373 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.360+0000 7f156635a700 1 --2- 172.21.2.104:0/2182300861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15600b2660 0x7f156005f6b0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 rx=0x7f1548002f70 tx=0x7f1548006e80).stop 2021-07-11T04:48:26.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 -- 172.21.2.104:0/2182300861 shutdown_connections 2021-07-11T04:48:26.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 --2- 172.21.2.104:0/2182300861 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1560096cc0 0x7f1560097100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 --2- 172.21.2.104:0/2182300861 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15600b2660 0x7f156005f6b0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:26.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 -- 172.21.2.104:0/2182300861 >> 172.21.2.104:0/2182300861 conn(0x7f1560062f10 msgr2=0x7f15600632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:26.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 -- 172.21.2.104:0/2182300861 shutdown_connections 2021-07-11T04:48:26.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 -- 172.21.2.104:0/2182300861 wait complete. 2021-07-11T04:48:26.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 Processor -- start 2021-07-11T04:48:26.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 -- start start 2021-07-11T04:48:26.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1560096cc0 0x7f15600af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15600b2660 0x7f156009a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f156013f560 0x7f15600adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15600ae480 con 0x7f15600b2660 2021-07-11T04:48:26.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15600ae720 con 0x7f156013f560 2021-07-11T04:48:26.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1560155e90 con 0x7f1560096cc0 2021-07-11T04:48:26.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15600b2660 0x7f156009a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:26.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15600b2660 0x7f156009a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52258/0 (socket says 172.21.2.104:52258) 2021-07-11T04:48:26.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155f7fe700 1 -- 172.21.2.104:0/1961009412 learned_addr learned my addr 172.21.2.104:0/1961009412 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:26.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155f7fe700 1 -- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1560096cc0 msgr2=0x7f15600af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155f7fe700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1560096cc0 0x7f15600af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155f7fe700 1 -- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f156013f560 msgr2=0x7f15600adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155f7fe700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f156013f560 0x7f15600adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155f7fe700 1 -- 172.21.2.104:0/1961009412 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154800d410 con 0x7f15600b2660 2021-07-11T04:48:26.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155f7fe700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15600b2660 0x7f156009a5e0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 rx=0x7f1550002fb0 tx=0x7f1550006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:26.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155d7fa700 1 -- 172.21.2.104:0/1961009412 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1550013460 con 0x7f15600b2660 2021-07-11T04:48:26.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155d7fa700 1 -- 172.21.2.104:0/1961009412 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15500135c0 con 0x7f15600b2660 2021-07-11T04:48:26.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155d7fa700 1 -- 172.21.2.104:0/1961009412 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f1550013920 con 0x7f15600b2660 2021-07-11T04:48:26.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 -- 172.21.2.104:0/1961009412 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1560156130 con 0x7f15600b2660 2021-07-11T04:48:26.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 -- 172.21.2.104:0/1961009412 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15600aef80 con 0x7f15600b2660 2021-07-11T04:48:26.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f156635a700 1 -- 172.21.2.104:0/1961009412 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15601562c0 con 0x7f15600b2660 2021-07-11T04:48:26.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155d7fa700 1 -- 172.21.2.104:0/1961009412 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f155000f070 con 0x7f15600b2660 2021-07-11T04:48:26.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155d7fa700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1544053240 0x7f1544055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:26.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155d7fa700 1 -- 172.21.2.104:0/1961009412 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f155006ada0 con 0x7f15600b2660 2021-07-11T04:48:26.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.364+0000 7f155ffff700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1544053240 0x7f1544055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:26.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.368+0000 7f155ffff700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1544053240 0x7f1544055700 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f1548002f70 tx=0x7f1548006e80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:26.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.368+0000 7f155d7fa700 1 -- 172.21.2.104:0/1961009412 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f15500422b0 con 0x7f15600b2660 2021-07-11T04:48:26.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 -- 172.21.2.104:0/1961009412 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1560057350 con 0x7f15600b2660 2021-07-11T04:48:26.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f155d7fa700 1 -- 172.21.2.104:0/1961009412 <== mon.0 v2:172.21.2.104:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+13320 (secure 0 0 0) 0x7f15500458d0 con 0x7f15600b2660 2021-07-11T04:48:26.456 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:48:26.456 INFO:teuthology.orchestra.run.gibba004.stdout:{"epoch":164,"fsid":"57c05e2e-f3b6-453c-878e-145bef41bd6c","created":"2021-07-11T03:53:08.172646+0000","modified":"2021-07-11T04:02:55.429316+0000","last_up_change":"2021-07-11T03:53:14.396616+0000","last_in_change":"2021-07-11T03:53:14.396616+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-07-11T03:53:10.374230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-07-11T03:53:18.251635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"a9f3dcb9-1813-4b31-8816-f0e71b6104b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6800","nonce":15411},{"type":"v1","addr":"172.21.2.104:6801","nonce":15411}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6802","nonce":15411},{"type":"v1","addr":"172.21.2.104:6803","nonce":15411}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6806","nonce":15411},{"type":"v1","addr":"172.21.2.104:6807","nonce":15411}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6804","nonce":15411},{"type":"v1","addr":"172.21.2.104:6805","nonce":15411}]},"public_addr":"172.21.2.104:6801/15411","cluster_addr":"172.21.2.104:6803/15411","heartbeat_back_addr":"172.21.2.104:6807/15411","heartbeat_front_addr":"172.21.2.104:6805/15411","state":["exists","up"]},{"osd":1,"uuid":"61e6a59d-2129-494a-9723-2c792e38c8ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6808","nonce":15412},{"type":"v1","addr":"172.21.2.104:6809","nonce":15412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6810","nonce":15412},{"type":"v1","addr":"172.21.2.104:6811","nonce":15412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6814","nonce":15412},{"type":"v1","addr":"172.21.2.104:6815","nonce":15412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6812","nonce":15412},{"type":"v1","addr":"172.21.2.104:6813","nonce":15412}]},"public_addr":"172.21.2.104:6809/15412","cluster_addr":"172.21.2.104:6811/15412","heartbeat_back_addr":"172.21.2.104:6815/15412","heartbeat_front_addr":"172.21.2.104:6813/15412","state":["exists","up"]},{"osd":2,"uuid":"9b13c5df-5184-4f5e-a553-cc41d8143a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6816","nonce":15413},{"type":"v1","addr":"172.21.2.104:6817","nonce":15413}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6818","nonce":15413},{"type":"v1","addr":"172.21.2.104:6819","nonce":15413}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6822","nonce":15413},{"type":"v1","addr":"172.21.2.104:6823","nonce":15413}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6820","nonce":15413},{"type":"v1","addr":"172.21.2.104:6821","nonce":15413}]},"public_addr":"172.21.2.104:6817/15413","cluster_addr":"172.21.2.104:6819/15413","heartbeat_back_addr":"172.21.2.104:6823/15413","heartbeat_front_addr":"172.21.2.104:6821/15413","state":["exists","up"]},{"osd":3,"uuid":"b75b5b84-a78e-4b67-8044-8ab08825f5fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6824","nonce":15456},{"type":"v1","addr":"172.21.2.104:6825","nonce":15456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6826","nonce":15456},{"type":"v1","addr":"172.21.2.104:6827","nonce":15456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6830","nonce":15456},{"type":"v1","addr":"172.21.2.104:6831","nonce":15456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.104:6828","nonce":15456},{"type":"v1","addr":"172.21.2.104:6829","nonce":15456}]},"public_addr":"172.21.2.104:6825/15456","cluster_addr":"172.21.2.104:6827/15456","heartbeat_back_addr":"172.21.2.104:6831/15456","heartbeat_front_addr":"172.21.2.104:6829/15456","state":["exists","up"]},{"osd":4,"uuid":"70205843-19d1-4c4f-a9e4-6aa9ebfa4c28","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6802","nonce":15645},{"type":"v1","addr":"172.21.2.105:6803","nonce":15645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6804","nonce":15645},{"type":"v1","addr":"172.21.2.105:6805","nonce":15645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6808","nonce":15645},{"type":"v1","addr":"172.21.2.105:6809","nonce":15645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6806","nonce":15645},{"type":"v1","addr":"172.21.2.105:6807","nonce":15645}]},"public_addr":"172.21.2.105:6803/15645","cluster_addr":"172.21.2.105:6805/15645","heartbeat_back_addr":"172.21.2.105:6809/15645","heartbeat_front_addr":"172.21.2.105:6807/15645","state":["exists","up"]},{"osd":5,"uuid":"e19543c5-2378-49c8-a626-c71dbbb2a718","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":159,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6812","nonce":15646},{"type":"v1","addr":"172.21.2.105:6815","nonce":15646}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6818","nonce":15646},{"type":"v1","addr":"172.21.2.105:6821","nonce":15646}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6830","nonce":15646},{"type":"v1","addr":"172.21.2.105:6833","nonce":15646}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6824","nonce":15646},{"type":"v1","addr":"172.21.2.105:6827","nonce":15646}]},"public_addr":"172.21.2.105:6815/15646","cluster_addr":"172.21.2.105:6821/15646","heartbeat_back_addr":"172.21.2.105:6833/15646","heartbeat_front_addr":"172.21.2.105:6827/15646","state":["exists","up"]},{"osd":6,"uuid":"fc1e39b4-4c50-4f6a-a8ff-19687bf442b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6810","nonce":15647},{"type":"v1","addr":"172.21.2.105:6813","nonce":15647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6816","nonce":15647},{"type":"v1","addr":"172.21.2.105:6819","nonce":15647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6829","nonce":15647},{"type":"v1","addr":"172.21.2.105:6832","nonce":15647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6822","nonce":15647},{"type":"v1","addr":"172.21.2.105:6825","nonce":15647}]},"public_addr":"172.21.2.105:6813/15647","cluster_addr":"172.21.2.105:6819/15647","heartbeat_back_addr":"172.21.2.105:6832/15647","heartbeat_front_addr":"172.21.2.105:6825/15647","state":["exists","up"]},{"osd":7,"uuid":"2df0639d-90fe-4158-80ae-24cf0f10dc86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6811","nonce":15656},{"type":"v1","addr":"172.21.2.105:6814","nonce":15656}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6817","nonce":15656},{"type":"v1","addr":"172.21.2.105:6820","nonce":15656}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6828","nonce":15656},{"type":"v1","addr":"172.21.2.105:6831","nonce":15656}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.2.105:6823","nonce":15656},{"type":"v1","addr":"172.21.2.105:6826","nonce":15656}]},"public_addr":"172.21.2.105:6814/15656","cluster_addr":"172.21.2.105:6820/15656","heartbeat_back_addr":"172.21.2.105:6831/15656","heartbeat_front_addr":"172.21.2.105:6826/15656","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.990606+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.951362+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.132692+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.157951+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:12.704587+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.008971+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.062419+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292840890367,"old_weight":0,"last_purged_snaps_scrub":"2021-07-11T03:53:13.104834+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{"172.21.2.104:6835/2638676458":"2021-07-12T04:02:51.605702+0000","172.21.2.104:6834/2638676458":"2021-07-12T04:02:51.605702+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2021-07-11T04:48:26.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 -- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1544053240 msgr2=0x7f1544055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1544053240 0x7f1544055700 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f1548002f70 tx=0x7f1548006e80).stop 2021-07-11T04:48:26.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 -- 172.21.2.104:0/1961009412 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15600b2660 msgr2=0x7f156009a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:26.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15600b2660 0x7f156009a5e0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 rx=0x7f1550002fb0 tx=0x7f1550006df0).stop 2021-07-11T04:48:26.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 -- 172.21.2.104:0/1961009412 shutdown_connections 2021-07-11T04:48:26.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f156013f560 0x7f15600adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f1544053240 0x7f1544055700 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:26.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f15600b2660 0x7f156009a5e0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:26.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 --2- 172.21.2.104:0/1961009412 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f1560096cc0 0x7f15600af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:26.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 -- 172.21.2.104:0/1961009412 >> 172.21.2.104:0/1961009412 conn(0x7f1560062f10 msgr2=0x7f1560060210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:26.460 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 -- 172.21.2.104:0/1961009412 shutdown_connections 2021-07-11T04:48:26.461 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:26.452+0000 7f156635a700 1 -- 172.21.2.104:0/1961009412 wait complete. 2021-07-11T04:48:27.465 INFO:tasks.ceph:Scrubbing osd.0 2021-07-11T04:48:27.466 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2021-07-11T04:48:27.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.520+0000 7f545526c700 1 Processor -- start 2021-07-11T04:48:27.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.520+0000 7f545526c700 1 -- start start 2021-07-11T04:48:27.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.520+0000 7f545526c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f54500a76b0 0x7f5450066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.520+0000 7f545526c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f54500b1970 0x7f5450066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.520+0000 7f545526c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5450067450 con 0x7f54500b0060 2021-07-11T04:48:27.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54500a7a90 con 0x7f54500a76b0 2021-07-11T04:48:27.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5450147620 con 0x7f54500b1970 2021-07-11T04:48:27.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544f7fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f54500b0060 0x7f545005f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49034/0 (socket says 172.21.2.104:49034) 2021-07-11T04:48:27.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544f7fe700 1 -- 172.21.2.104:0/2014575581 learned_addr learned my addr 172.21.2.104:0/2014575581 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:48:27.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544effd700 1 --2- 172.21.2.104:0/2014575581 >> v2:172.21.2.104:3300/0 conn(0x7f54500a76b0 0x7f5450066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544effd700 1 -- 172.21.2.104:0/2014575581 >> v2:172.21.2.105:3300/0 conn(0x7f54500b1970 msgr2=0x7f5450066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:27.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544effd700 1 --2- 172.21.2.104:0/2014575581 >> v2:172.21.2.105:3300/0 conn(0x7f54500b1970 0x7f5450066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544effd700 1 -- 172.21.2.104:0/2014575581 >> v1:172.21.2.104:6789/0 conn(0x7f54500b0060 legacy=0x7f545005f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544effd700 1 -- 172.21.2.104:0/2014575581 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54501478c0 con 0x7f54500a76b0 2021-07-11T04:48:27.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544effd700 1 --2- 172.21.2.104:0/2014575581 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54500a76b0 0x7f5450066a70 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 rx=0x7f5440002f70 tx=0x7f5440006df0).ready entity=mon.0 client_cookie=c4bbbf4cf12d0d8c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544dffb700 1 -- 172.21.2.104:0/2014575581 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f544000d5c0 con 0x7f54500a76b0 2021-07-11T04:48:27.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544dffb700 1 -- 172.21.2.104:0/2014575581 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5440000e90 con 0x7f54500a76b0 2021-07-11T04:48:27.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544dffb700 1 -- 172.21.2.104:0/2014575581 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f544000d910 con 0x7f54500a76b0 2021-07-11T04:48:27.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 -- 172.21.2.104:0/2014575581 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54500a76b0 msgr2=0x7f5450066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 --2- 172.21.2.104:0/2014575581 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54500a76b0 0x7f5450066a70 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 rx=0x7f5440002f70 tx=0x7f5440006df0).stop 2021-07-11T04:48:27.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 -- 172.21.2.104:0/2014575581 shutdown_connections 2021-07-11T04:48:27.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 --2- 172.21.2.104:0/2014575581 >> v2:172.21.2.105:3300/0 conn(0x7f54500b1970 0x7f5450066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 --2- 172.21.2.104:0/2014575581 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54500a76b0 0x7f5450066a70 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 -- 172.21.2.104:0/2014575581 >> 172.21.2.104:0/2014575581 conn(0x7f54500b53e0 msgr2=0x7f54500b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:27.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 -- 172.21.2.104:0/2014575581 shutdown_connections 2021-07-11T04:48:27.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 -- 172.21.2.104:0/2014575581 wait complete. 2021-07-11T04:48:27.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 Processor -- start 2021-07-11T04:48:27.538 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 -- start start 2021-07-11T04:48:27.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54500a76b0 0x7f54500b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54500b0060 0x7f5450157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.539 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54500b1970 0x7f54500ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54500a9c80 con 0x7f54500b1970 2021-07-11T04:48:27.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5450157680 con 0x7f54500b0060 2021-07-11T04:48:27.540 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f545526c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54500a9740 con 0x7f54500a76b0 2021-07-11T04:48:27.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54500b1970 0x7f54500ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544f7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54500b1970 0x7f54500ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52274/0 (socket says 172.21.2.104:52274) 2021-07-11T04:48:27.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544f7fe700 1 -- 172.21.2.104:0/884114812 learned_addr learned my addr 172.21.2.104:0/884114812 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:27.541 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544effd700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54500a76b0 0x7f54500b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.524+0000 7f544e7fc700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54500b0060 0x7f5450157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f544f7fe700 1 -- 172.21.2.104:0/884114812 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54500a76b0 msgr2=0x7f54500b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.542 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f544f7fe700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f54500a76b0 0x7f54500b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f544f7fe700 1 -- 172.21.2.104:0/884114812 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54500b0060 msgr2=0x7f5450157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f544f7fe700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f54500b0060 0x7f5450157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.543 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f544f7fe700 1 -- 172.21.2.104:0/884114812 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544000b180 con 0x7f54500b1970 2021-07-11T04:48:27.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f544f7fe700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54500b1970 0x7f54500ae690 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 rx=0x7f545006a0e0 tx=0x7f5448007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f5447fff700 1 -- 172.21.2.104:0/884114812 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f5448005ba0 con 0x7f54500b1970 2021-07-11T04:48:27.544 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f5447fff700 1 -- 172.21.2.104:0/884114812 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5448005d00 con 0x7f54500b1970 2021-07-11T04:48:27.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f545526c700 1 -- 172.21.2.104:0/884114812 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54500a99e0 con 0x7f54500b1970 2021-07-11T04:48:27.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f545526c700 1 -- 172.21.2.104:0/884114812 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54500ac520 con 0x7f54500b1970 2021-07-11T04:48:27.545 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f5447fff700 1 -- 172.21.2.104:0/884114812 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f54480072c0 con 0x7f54500b1970 2021-07-11T04:48:27.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f545526c700 1 -- 172.21.2.104:0/884114812 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f54500ae3f0 con 0x7f54500b1970 2021-07-11T04:48:27.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f5447fff700 1 -- 172.21.2.104:0/884114812 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f5448010070 con 0x7f54500b1970 2021-07-11T04:48:27.546 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f5447fff700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5434053240 0x7f5434055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f5447fff700 1 -- 172.21.2.104:0/884114812 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f544806ac10 con 0x7f54500b1970 2021-07-11T04:48:27.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f5447fff700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x7f5434059a40 0x7f543405be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f5447fff700 1 -- 172.21.2.104:0/884114812 --> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f543405c4c0 con 0x7f5434059a40 2021-07-11T04:48:27.547 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f5447fff700 1 -- 172.21.2.104:0/884114812 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=164) v2 ==== 24+0+0 (secure 0 0 0) 0x7f544806f240 con 0x7f54500b1970 2021-07-11T04:48:27.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f544e7fc700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x7f5434059a40 0x7f543405be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f544effd700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5434053240 0x7f5434055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.548 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f544e7fc700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x7f5434059a40 0x7f543405be60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f544effd700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5434053240 0x7f5434055700 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f5440017000 tx=0x7f5440014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.549 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.528+0000 7f5447fff700 1 -- 172.21.2.104:0/884114812 <== osd.0 v2:172.21.2.104:6800/15411 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f543405c4c0 con 0x7f5434059a40 2021-07-11T04:48:27.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.536+0000 7f545526c700 1 -- 172.21.2.104:0/884114812 --> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f54500aee20 con 0x7f5434059a40 2021-07-11T04:48:27.550 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.536+0000 7f5447fff700 1 -- 172.21.2.104:0/884114812 <== osd.0 v2:172.21.2.104:6800/15411 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f54500aee20 con 0x7f5434059a40 2021-07-11T04:48:27.551 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:48:27.551 INFO:teuthology.orchestra.run.gibba004.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-07-11T04:48:27.552 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:48:27.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.536+0000 7f545526c700 1 -- 172.21.2.104:0/884114812 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x7f5434059a40 msgr2=0x7f543405be60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.536+0000 7f545526c700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.104:6800/15411,v1:172.21.2.104:6801/15411] conn(0x7f5434059a40 0x7f543405be60 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.536+0000 7f545526c700 1 -- 172.21.2.104:0/884114812 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5434053240 msgr2=0x7f5434055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.536+0000 7f545526c700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f5434053240 0x7f5434055700 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f5440017000 tx=0x7f5440014000).stop 2021-07-11T04:48:27.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.536+0000 7f545526c700 1 -- 172.21.2.104:0/884114812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54500b1970 msgr2=0x7f54500ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.555 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.536+0000 7f545526c700 1 --2- 172.21.2.104:0/884114812 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f54500b1970 0x7f54500ae690 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 rx=0x7f545006a0e0 tx=0x7f5448007f00).stop 2021-07-11T04:48:27.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.536+0000 7f544f7fe700 1 -- 172.21.2.104:0/884114812 reap_dead start 2021-07-11T04:48:27.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.540+0000 7f545526c700 1 -- 172.21.2.104:0/884114812 shutdown_connections 2021-07-11T04:48:27.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.540+0000 7f545526c700 1 -- 172.21.2.104:0/884114812 >> 172.21.2.104:0/884114812 conn(0x7f54500b53e0 msgr2=0x7f5450149b90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:27.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.540+0000 7f545526c700 1 -- 172.21.2.104:0/884114812 shutdown_connections 2021-07-11T04:48:27.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.540+0000 7f545526c700 1 -- 172.21.2.104:0/884114812 wait complete. 2021-07-11T04:48:27.557 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2021-07-11T04:48:27.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.608+0000 7f326b1fc700 1 Processor -- start 2021-07-11T04:48:27.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.608+0000 7f326b1fc700 1 -- start start 2021-07-11T04:48:27.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.608+0000 7f326b1fc700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f32640b2660 0x7f326405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.608+0000 7f326b1fc700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3264096cc0 0x7f3264097130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.616 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.608+0000 7f326b1fc700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3264097620 con 0x7f326413f560 2021-07-11T04:48:27.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.608+0000 7f326b1fc700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32640b2a40 con 0x7f3264096cc0 2021-07-11T04:48:27.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.608+0000 7f326b1fc700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3264097b80 con 0x7f32640b2660 2021-07-11T04:48:27.617 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3263fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3264096cc0 0x7f3264097130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3263fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f3264096cc0 0x7f3264097130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52282/0 (socket says 172.21.2.104:52282) 2021-07-11T04:48:27.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3263fff700 1 -- 172.21.2.104:0/3459342991 learned_addr learned my addr 172.21.2.104:0/3459342991 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:27.618 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3263fff700 1 -- 172.21.2.104:0/3459342991 >> v2:172.21.2.105:3300/0 conn(0x7f32640b2660 msgr2=0x7f326405f6b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:48:27.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3263fff700 1 --2- 172.21.2.104:0/3459342991 >> v2:172.21.2.105:3300/0 conn(0x7f32640b2660 0x7f326405f6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3263fff700 1 -- 172.21.2.104:0/3459342991 >> v1:172.21.2.105:6789/0 conn(0x7f326413f560 legacy=0x7f326413f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:27.619 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3263fff700 1 -- 172.21.2.104:0/3459342991 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3264097e20 con 0x7f3264096cc0 2021-07-11T04:48:27.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3263fff700 1 --2- 172.21.2.104:0/3459342991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3264096cc0 0x7f3264097130 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 rx=0x7f32640986e0 tx=0x7f3258006e40).ready entity=mon.0 client_cookie=fb805a6d61640044 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f32637fe700 1 -- 172.21.2.104:0/3459342991 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f325800bcb0 con 0x7f3264096cc0 2021-07-11T04:48:27.620 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f32637fe700 1 -- 172.21.2.104:0/3459342991 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3258000e90 con 0x7f3264096cc0 2021-07-11T04:48:27.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f32637fe700 1 -- 172.21.2.104:0/3459342991 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f32580153f0 con 0x7f3264096cc0 2021-07-11T04:48:27.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- 172.21.2.104:0/3459342991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3264096cc0 msgr2=0x7f3264097130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.621 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 --2- 172.21.2.104:0/3459342991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3264096cc0 0x7f3264097130 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 rx=0x7f32640986e0 tx=0x7f3258006e40).stop 2021-07-11T04:48:27.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- 172.21.2.104:0/3459342991 shutdown_connections 2021-07-11T04:48:27.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 --2- 172.21.2.104:0/3459342991 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f3264096cc0 0x7f3264097130 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 --2- 172.21.2.104:0/3459342991 >> v2:172.21.2.105:3300/0 conn(0x7f32640b2660 0x7f326405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.622 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- 172.21.2.104:0/3459342991 >> 172.21.2.104:0/3459342991 conn(0x7f3264062f10 msgr2=0x7f32640632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:27.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- 172.21.2.104:0/3459342991 shutdown_connections 2021-07-11T04:48:27.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- 172.21.2.104:0/3459342991 wait complete. 2021-07-11T04:48:27.623 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 Processor -- start 2021-07-11T04:48:27.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- start start 2021-07-11T04:48:27.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3264096cc0 0x7f32640aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.624 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f32640b2660 0x7f326414a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f326413f560 0x7f3264156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326409ba20 con 0x7f326413f560 2021-07-11T04:48:27.625 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32640af430 con 0x7f3264096cc0 2021-07-11T04:48:27.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32640aa9b0 con 0x7f32640b2660 2021-07-11T04:48:27.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3269799700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f326413f560 0x7f3264156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.626 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3269799700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f326413f560 0x7f3264156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52288/0 (socket says 172.21.2.104:52288) 2021-07-11T04:48:27.627 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3269799700 1 -- 172.21.2.104:0/3788274977 learned_addr learned my addr 172.21.2.104:0/3788274977 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:27.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3269799700 1 -- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f32640b2660 msgr2=0x7f326414a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3269799700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f32640b2660 0x7f326414a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.629 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3269799700 1 -- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3264096cc0 msgr2=0x7f32640aa380 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:48:27.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3269799700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3264096cc0 0x7f32640aa380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3269799700 1 -- 172.21.2.104:0/3788274977 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325800d4d0 con 0x7f326413f560 2021-07-11T04:48:27.630 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3269799700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f326413f560 0x7f3264156280 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 rx=0x7f325c003290 tx=0x7f325c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3261ffb700 1 -- 172.21.2.104:0/3788274977 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f325c00f510 con 0x7f326413f560 2021-07-11T04:48:27.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3261ffb700 1 -- 172.21.2.104:0/3788274977 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c00f670 con 0x7f326413f560 2021-07-11T04:48:27.631 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- 172.21.2.104:0/3788274977 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32640aac50 con 0x7f326413f560 2021-07-11T04:48:27.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3261ffb700 1 -- 172.21.2.104:0/3788274977 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f325c00f7e0 con 0x7f326413f560 2021-07-11T04:48:27.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- 172.21.2.104:0/3788274977 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3264099620 con 0x7f326413f560 2021-07-11T04:48:27.632 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f326b1fc700 1 -- 172.21.2.104:0/3788274977 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3264057350 con 0x7f326413f560 2021-07-11T04:48:27.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.612+0000 7f3261ffb700 1 -- 172.21.2.104:0/3788274977 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f325c00f070 con 0x7f326413f560 2021-07-11T04:48:27.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.616+0000 7f3261ffb700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3244053160 0x7f3244055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.633 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.616+0000 7f3261ffb700 1 -- 172.21.2.104:0/3788274977 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f325c06ae10 con 0x7f326413f560 2021-07-11T04:48:27.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.616+0000 7f3268f98700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3244053160 0x7f3244055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.616+0000 7f3268f98700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3244053160 0x7f3244055620 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f3254002fb0 tx=0x7f3254006950).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.634 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.616+0000 7f3261ffb700 1 -- 172.21.2.104:0/3788274977 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f325c042430 con 0x7f326413f560 2021-07-11T04:48:27.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.700+0000 7f326b1fc700 1 -- 172.21.2.104:0/3788274977 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f3264155c60 con 0x7f3244053160 2021-07-11T04:48:27.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.700+0000 7f3261ffb700 1 -- 172.21.2.104:0/3788274977 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f325400b0e0 con 0x7f3244053160 2021-07-11T04:48:27.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 -- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3244053160 msgr2=0x7f3244055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3244053160 0x7f3244055620 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f3254002fb0 tx=0x7f3254006950).stop 2021-07-11T04:48:27.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 -- 172.21.2.104:0/3788274977 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f326413f560 msgr2=0x7f3264156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f326413f560 0x7f3264156280 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 rx=0x7f325c003290 tx=0x7f325c0090f0).stop 2021-07-11T04:48:27.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 -- 172.21.2.104:0/3788274977 shutdown_connections 2021-07-11T04:48:27.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3244053160 0x7f3244055620 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f326413f560 0x7f3264156280 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f32640b2660 0x7f326414a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 --2- 172.21.2.104:0/3788274977 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f3264096cc0 0x7f32640aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 -- 172.21.2.104:0/3788274977 >> 172.21.2.104:0/3788274977 conn(0x7f3264062f10 msgr2=0x7f3264094250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:27.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 -- 172.21.2.104:0/3788274977 shutdown_connections 2021-07-11T04:48:27.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.704+0000 7f326b1fc700 1 -- 172.21.2.104:0/3788274977 wait complete. 2021-07-11T04:48:27.712 INFO:teuthology.orchestra.run.gibba004.stderr:instructed osd(s) 0 to deep-scrub 2021-07-11T04:48:27.713 INFO:tasks.ceph:Scrubbing osd.1 2021-07-11T04:48:27.714 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2021-07-11T04:48:27.768 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f47af53e700 1 Processor -- start 2021-07-11T04:48:27.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f47af53e700 1 -- start start 2021-07-11T04:48:27.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f47af53e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a80a76b0 0x7f47a8066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.769 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f47af53e700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a80a7a90 con 0x7f47a80b0060 2021-07-11T04:48:27.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f47af53e700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47a8147620 con 0x7f47a80b1970 2021-07-11T04:48:27.770 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f47af53e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a81478c0 con 0x7f47a80a76b0 2021-07-11T04:48:27.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f47adadb700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f47a80b0060 0x7f47a8151f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49066/0 (socket says 172.21.2.104:49066) 2021-07-11T04:48:27.776 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f47adadb700 1 -- 172.21.2.104:0/780650831 learned_addr learned my addr 172.21.2.104:0/780650831 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:48:27.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298531948 0 0) 0x7f47a80a7a90 con 0x7f47a80b0060 2021-07-11T04:48:27.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4790003260 con 0x7f47a80b0060 2021-07-11T04:48:27.777 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f47ad2da700 1 --2- 172.21.2.104:0/780650831 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a80a76b0 0x7f47a8066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3781847539 0 0) 0x7f4790003260 con 0x7f47a80b0060 2021-07-11T04:48:27.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a80a76b0 msgr2=0x7f47a8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:27.778 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 --2- 172.21.2.104:0/780650831 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a80a76b0 0x7f47a8066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 >> v1:172.21.2.105:6789/0 conn(0x7f47a80b1970 legacy=0x7f47a805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:27.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a8147b60 con 0x7f47a80b0060 2021-07-11T04:48:27.779 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f47a00019e0 con 0x7f47a80b0060 2021-07-11T04:48:27.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 >> v1:172.21.2.104:6789/0 conn(0x7f47a80b0060 legacy=0x7f47a8151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 --2- 172.21.2.104:0/780650831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4790003800 0x7f4790003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.780 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 --2- 172.21.2.104:0/780650831 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a80a76b0 0x7f4790004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 --2- 172.21.2.104:0/780650831 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4790004bf0 0x7f479000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a81478c0 con 0x7f4790003800 2021-07-11T04:48:27.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47900052a0 con 0x7f4790004bf0 2021-07-11T04:48:27.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479000bc40 con 0x7f47a80a76b0 2021-07-11T04:48:27.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.764+0000 7f47ad2da700 1 --2- 172.21.2.104:0/780650831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4790003800 0x7f4790003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 -- 172.21.2.104:0/780650831 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a80a76b0 msgr2=0x7f4790004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:27.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 --2- 172.21.2.104:0/780650831 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a80a76b0 0x7f4790004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 -- 172.21.2.104:0/780650831 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4790004bf0 msgr2=0x7f479000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:27.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 --2- 172.21.2.104:0/780650831 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4790004bf0 0x7f479000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 -- 172.21.2.104:0/780650831 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4790003260 con 0x7f4790003800 2021-07-11T04:48:27.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 --2- 172.21.2.104:0/780650831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4790003800 0x7f4790003c00 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 rx=0x7f47a4007000 tx=0x7f47a4008040).ready entity=mon.0 client_cookie=d363b35f1f9edfc8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47a4012470 con 0x7f4790003800 2021-07-11T04:48:27.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a400c030 con 0x7f4790003800 2021-07-11T04:48:27.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f4797fff700 1 -- 172.21.2.104:0/780650831 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47a40127c0 con 0x7f4790003800 2021-07-11T04:48:27.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- 172.21.2.104:0/780650831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4790003800 msgr2=0x7f4790003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 --2- 172.21.2.104:0/780650831 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4790003800 0x7f4790003c00 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 rx=0x7f47a4007000 tx=0x7f47a4008040).stop 2021-07-11T04:48:27.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47adadb700 1 -- 172.21.2.104:0/780650831 reap_dead start 2021-07-11T04:48:27.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- 172.21.2.104:0/780650831 shutdown_connections 2021-07-11T04:48:27.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- 172.21.2.104:0/780650831 >> 172.21.2.104:0/780650831 conn(0x7f47a80b53e0 msgr2=0x7f47a80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:27.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- 172.21.2.104:0/780650831 shutdown_connections 2021-07-11T04:48:27.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- 172.21.2.104:0/780650831 wait complete. 2021-07-11T04:48:27.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 Processor -- start 2021-07-11T04:48:27.790 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- start start 2021-07-11T04:48:27.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a8146790 0x7f47a8066820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f47a80af310 0x7f47a80ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a80aead0 0x7f47a8144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8146b70 con 0x7f47a8146790 2021-07-11T04:48:27.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80aa990 con 0x7f47a80af310 2021-07-11T04:48:27.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47a80aac30 con 0x7f47a80aead0 2021-07-11T04:48:27.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a8146790 0x7f47a8066820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a8146790 0x7f47a8066820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52310/0 (socket says 172.21.2.104:52310) 2021-07-11T04:48:27.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 -- 172.21.2.104:0/361560636 learned_addr learned my addr 172.21.2.104:0/361560636 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:27.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 -- 172.21.2.104:0/361560636 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a80aead0 msgr2=0x7f47a8144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f47a80aead0 0x7f47a8144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 -- 172.21.2.104:0/361560636 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f47a80af310 msgr2=0x7f47a80ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f47a80af310 0x7f47a80ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 -- 172.21.2.104:0/361560636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80aaed0 con 0x7f47a8146790 2021-07-11T04:48:27.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47ad2da700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a8146790 0x7f47a8066820 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 rx=0x7f47a4007000 tx=0x7f47a4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47967fc700 1 -- 172.21.2.104:0/361560636 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47a401c8e0 con 0x7f47a8146790 2021-07-11T04:48:27.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47967fc700 1 -- 172.21.2.104:0/361560636 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a4000e90 con 0x7f47a8146790 2021-07-11T04:48:27.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- 172.21.2.104:0/361560636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a80ae630 con 0x7f47a8146790 2021-07-11T04:48:27.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47967fc700 1 -- 172.21.2.104:0/361560636 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f47a401cc30 con 0x7f47a8146790 2021-07-11T04:48:27.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- 172.21.2.104:0/361560636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a80aef60 con 0x7f47a8146790 2021-07-11T04:48:27.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47af53e700 1 -- 172.21.2.104:0/361560636 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47a80ae7c0 con 0x7f47a8146790 2021-07-11T04:48:27.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47967fc700 1 -- 172.21.2.104:0/361560636 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f47a402b070 con 0x7f47a8146790 2021-07-11T04:48:27.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47967fc700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f47880532d0 0x7f4788055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47967fc700 1 -- 172.21.2.104:0/361560636 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f47a406cf00 con 0x7f47a8146790 2021-07-11T04:48:27.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47967fc700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] conn(0x7f4788059ad0 0x7f478805bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.799 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47967fc700 1 -- 172.21.2.104:0/361560636 --> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f478805c4b0 con 0x7f4788059ad0 2021-07-11T04:48:27.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.768+0000 7f47967fc700 1 -- 172.21.2.104:0/361560636 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=164) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47a406d210 con 0x7f47a8146790 2021-07-11T04:48:27.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.772+0000 7f47adadb700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] conn(0x7f4788059ad0 0x7f478805bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.772+0000 7f47acad9700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f47880532d0 0x7f4788055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.772+0000 7f47adadb700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] conn(0x7f4788059ad0 0x7f478805bef0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.772+0000 7f47967fc700 1 -- 172.21.2.104:0/361560636 <== osd.1 v2:172.21.2.104:6808/15412 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f478805c4b0 con 0x7f4788059ad0 2021-07-11T04:48:27.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.772+0000 7f47acad9700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f47880532d0 0x7f4788055790 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f4798000f70 tx=0x7f47980069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47af53e700 1 -- 172.21.2.104:0/361560636 --> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f47a8146b70 con 0x7f4788059ad0 2021-07-11T04:48:27.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47967fc700 1 -- 172.21.2.104:0/361560636 <== osd.1 v2:172.21.2.104:6808/15412 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f47a8146b70 con 0x7f4788059ad0 2021-07-11T04:48:27.804 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:48:27.804 INFO:teuthology.orchestra.run.gibba004.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-07-11T04:48:27.805 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:48:27.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47af53e700 1 -- 172.21.2.104:0/361560636 >> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] conn(0x7f4788059ad0 msgr2=0x7f478805bef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47af53e700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.104:6808/15412,v1:172.21.2.104:6809/15412] conn(0x7f4788059ad0 0x7f478805bef0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47af53e700 1 -- 172.21.2.104:0/361560636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f47880532d0 msgr2=0x7f4788055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47af53e700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f47880532d0 0x7f4788055790 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f4798000f70 tx=0x7f47980069b0).stop 2021-07-11T04:48:27.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47af53e700 1 -- 172.21.2.104:0/361560636 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a8146790 msgr2=0x7f47a8066820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47af53e700 1 --2- 172.21.2.104:0/361560636 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f47a8146790 0x7f47a8066820 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 rx=0x7f47a4007000 tx=0x7f47a4008040).stop 2021-07-11T04:48:27.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47adadb700 1 -- 172.21.2.104:0/361560636 reap_dead start 2021-07-11T04:48:27.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47af53e700 1 -- 172.21.2.104:0/361560636 shutdown_connections 2021-07-11T04:48:27.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47af53e700 1 -- 172.21.2.104:0/361560636 >> 172.21.2.104:0/361560636 conn(0x7f47a80b53e0 msgr2=0x7f47a8149f00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:27.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47af53e700 1 -- 172.21.2.104:0/361560636 shutdown_connections 2021-07-11T04:48:27.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.780+0000 7f47af53e700 1 -- 172.21.2.104:0/361560636 wait complete. 2021-07-11T04:48:27.810 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2021-07-11T04:48:27.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 Processor -- start 2021-07-11T04:48:27.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- start start 2021-07-11T04:48:27.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0aa860 0x7fc47c0b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc47c0a9da0 0x7fc47c0aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc47c0ac370 con 0x7fc47c0acf30 2021-07-11T04:48:27.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc47c0abb50 con 0x7fc47c0a9da0 2021-07-11T04:48:27.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0aac40 con 0x7fc47c0aa860 2021-07-11T04:48:27.880 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47affd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc47c0a9da0 0x7fc47c0aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47affd700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7fc47c0a9da0 0x7fc47c0aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52322/0 (socket says 172.21.2.104:52322) 2021-07-11T04:48:27.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47affd700 1 -- 172.21.2.104:0/3132138942 learned_addr learned my addr 172.21.2.104:0/3132138942 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:27.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47affd700 1 -- 172.21.2.104:0/3132138942 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0aa860 msgr2=0x7fc47c0b11c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:48:27.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47affd700 1 --2- 172.21.2.104:0/3132138942 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0aa860 0x7fc47c0b11c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47affd700 1 -- 172.21.2.104:0/3132138942 >> v1:172.21.2.105:6789/0 conn(0x7fc47c0acf30 legacy=0x7fc47c1434b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:48:27.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47affd700 1 -- 172.21.2.104:0/3132138942 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c0b16b0 con 0x7fc47c0a9da0 2021-07-11T04:48:27.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47affd700 1 --2- 172.21.2.104:0/3132138942 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c0a9da0 0x7fc47c0aca40 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 rx=0x7fc47c0ac920 tx=0x7fc470006f40).ready entity=mon.0 client_cookie=fff80f69ed9b8b56 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47a7fc700 1 -- 172.21.2.104:0/3132138942 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc47000dc30 con 0x7fc47c0a9da0 2021-07-11T04:48:27.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47a7fc700 1 -- 172.21.2.104:0/3132138942 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc470000e90 con 0x7fc47c0a9da0 2021-07-11T04:48:27.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47a7fc700 1 -- 172.21.2.104:0/3132138942 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc47000be90 con 0x7fc47c0a9da0 2021-07-11T04:48:27.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- 172.21.2.104:0/3132138942 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c0a9da0 msgr2=0x7fc47c0aca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 --2- 172.21.2.104:0/3132138942 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c0a9da0 0x7fc47c0aca40 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 rx=0x7fc47c0ac920 tx=0x7fc470006f40).stop 2021-07-11T04:48:27.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- 172.21.2.104:0/3132138942 shutdown_connections 2021-07-11T04:48:27.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 --2- 172.21.2.104:0/3132138942 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c0a9da0 0x7fc47c0aca40 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 --2- 172.21.2.104:0/3132138942 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0aa860 0x7fc47c0b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- 172.21.2.104:0/3132138942 >> 172.21.2.104:0/3132138942 conn(0x7fc47c0b64d0 msgr2=0x7fc47c0b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:27.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- 172.21.2.104:0/3132138942 shutdown_connections 2021-07-11T04:48:27.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- 172.21.2.104:0/3132138942 wait complete. 2021-07-11T04:48:27.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 Processor -- start 2021-07-11T04:48:27.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- start start 2021-07-11T04:48:27.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c0a9da0 0x7fc47c064c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc47c0aa860 0x7fc47c065410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0acf30 0x7fc47c0a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c0609c0 con 0x7fc47c0a9da0 2021-07-11T04:48:27.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c060c60 con 0x7fc47c0aa860 2021-07-11T04:48:27.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc482189700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c060f00 con 0x7fc47c0acf30 2021-07-11T04:48:27.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c0a9da0 0x7fc47c064c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.891 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47b7fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c0a9da0 0x7fc47c064c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52324/0 (socket says 172.21.2.104:52324) 2021-07-11T04:48:27.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.872+0000 7fc47b7fe700 1 -- 172.21.2.104:0/204957260 learned_addr learned my addr 172.21.2.104:0/204957260 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:27.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc47b7fe700 1 -- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0acf30 msgr2=0x7fc47c0a00f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:48:27.892 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc47b7fe700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0acf30 0x7fc47c0a00f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc47b7fe700 1 -- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc47c0aa860 msgr2=0x7fc47c065410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:48:27.894 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc47b7fe700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc47c0aa860 0x7fc47c065410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc47b7fe700 1 -- 172.21.2.104:0/204957260 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47000b460 con 0x7fc47c0a9da0 2021-07-11T04:48:27.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc47b7fe700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c0a9da0 0x7fc47c064c30 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 rx=0x7fc46c002fb0 tx=0x7fc46c006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.895 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc478ff9700 1 -- 172.21.2.104:0/204957260 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46c013460 con 0x7fc47c0a9da0 2021-07-11T04:48:27.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc478ff9700 1 -- 172.21.2.104:0/204957260 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46c0135c0 con 0x7fc47c0a9da0 2021-07-11T04:48:27.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc482189700 1 -- 172.21.2.104:0/204957260 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47c0afc30 con 0x7fc47c0a9da0 2021-07-11T04:48:27.896 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc478ff9700 1 -- 172.21.2.104:0/204957260 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc46c013920 con 0x7fc47c0a9da0 2021-07-11T04:48:27.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc482189700 1 -- 172.21.2.104:0/204957260 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47c156920 con 0x7fc47c0a9da0 2021-07-11T04:48:27.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc482189700 1 -- 172.21.2.104:0/204957260 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc47c057350 con 0x7fc47c0a9da0 2021-07-11T04:48:27.897 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc478ff9700 1 -- 172.21.2.104:0/204957260 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc46c00f070 con 0x7fc47c0a9da0 2021-07-11T04:48:27.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc478ff9700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053300 0x7fc45c0557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:27.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc478ff9700 1 -- 172.21.2.104:0/204957260 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc46c06ac10 con 0x7fc47c0a9da0 2021-07-11T04:48:27.898 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc47affd700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053300 0x7fc45c0557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:27.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc47affd700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053300 0x7fc45c0557c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fc47c0ac920 tx=0x7fc47c0ac290).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:27.899 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.876+0000 7fc478ff9700 1 -- 172.21.2.104:0/204957260 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc46c042230 con 0x7fc47c0a9da0 2021-07-11T04:48:27.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.960+0000 7fc482189700 1 -- 172.21.2.104:0/204957260 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc47c060840 con 0x7fc45c053300 2021-07-11T04:48:27.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.960+0000 7fc478ff9700 1 -- 172.21.2.104:0/204957260 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc47000db10 con 0x7fc45c053300 2021-07-11T04:48:27.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 -- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053300 msgr2=0x7fc45c0557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053300 0x7fc45c0557c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fc47c0ac920 tx=0x7fc47c0ac290).stop 2021-07-11T04:48:27.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 -- 172.21.2.104:0/204957260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c0a9da0 msgr2=0x7fc47c064c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:27.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c0a9da0 0x7fc47c064c30 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 rx=0x7fc46c002fb0 tx=0x7fc46c006e10).stop 2021-07-11T04:48:27.968 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 -- 172.21.2.104:0/204957260 shutdown_connections 2021-07-11T04:48:27.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc47c0acf30 0x7fc47c0a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc47c0aa860 0x7fc47c065410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:27.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc45c053300 0x7fc45c0557c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.969 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 --2- 172.21.2.104:0/204957260 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc47c0a9da0 0x7fc47c064c30 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:27.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 -- 172.21.2.104:0/204957260 >> 172.21.2.104:0/204957260 conn(0x7fc47c0b64d0 msgr2=0x7fc47c143bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:27.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 -- 172.21.2.104:0/204957260 shutdown_connections 2021-07-11T04:48:27.970 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:27.964+0000 7fc482189700 1 -- 172.21.2.104:0/204957260 wait complete. 2021-07-11T04:48:27.971 INFO:teuthology.orchestra.run.gibba004.stderr:instructed osd(s) 1 to deep-scrub 2021-07-11T04:48:27.973 INFO:tasks.ceph:Scrubbing osd.2 2021-07-11T04:48:27.974 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2021-07-11T04:48:28.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4e85f700 1 Processor -- start 2021-07-11T04:48:28.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4e85f700 1 -- start start 2021-07-11T04:48:28.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4e85f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4d480a76b0 0x7f4d48066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4e85f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d480a7a90 con 0x7f4d480b1970 2021-07-11T04:48:28.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4e85f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d48147620 con 0x7f4d480b0060 2021-07-11T04:48:28.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4e85f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d481478c0 con 0x7f4d480a76b0 2021-07-11T04:48:28.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d477fe700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f4d480b1970 0x7f4d4805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49102/0 (socket says 172.21.2.104:49102) 2021-07-11T04:48:28.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d477fe700 1 -- 172.21.2.104:0/536859059 learned_addr learned my addr 172.21.2.104:0/536859059 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:48:28.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818202387 0 0) 0x7f4d480a7a90 con 0x7f4d480b1970 2021-07-11T04:48:28.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d28003260 con 0x7f4d480b1970 2021-07-11T04:48:28.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d47fff700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4d480a76b0 0x7f4d48066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1999159569 0 0) 0x7f4d28003260 con 0x7f4d480b1970 2021-07-11T04:48:28.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4d480a76b0 msgr2=0x7f4d48066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4d480a76b0 0x7f4d48066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 >> v1:172.21.2.105:6789/0 conn(0x7f4d480b0060 legacy=0x7f4d48151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d48147b60 con 0x7f4d480b1970 2021-07-11T04:48:28.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f4d38001660 con 0x7f4d480b1970 2021-07-11T04:48:28.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 >> v1:172.21.2.104:6789/0 conn(0x7f4d480b1970 legacy=0x7f4d4805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4d480a76b0 0x7f4d28003800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4d28003dc0 0x7f4d28004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4d28004a40 0x7f4d28004ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d481478c0 con 0x7f4d28004a40 2021-07-11T04:48:28.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2800b910 con 0x7f4d28003dc0 2021-07-11T04:48:28.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2800bad0 con 0x7f4d480a76b0 2021-07-11T04:48:28.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4cdfc700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4d28004a40 0x7f4d28004ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d47fff700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4d480a76b0 0x7f4d28003800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d477fe700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4d28003dc0 0x7f4d28004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4cdfc700 1 -- 172.21.2.104:0/536859059 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4d480a76b0 msgr2=0x7f4d28003800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4cdfc700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4d480a76b0 0x7f4d28003800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4cdfc700 1 -- 172.21.2.104:0/536859059 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4d28003dc0 msgr2=0x7f4d28004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4cdfc700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4d28003dc0 0x7f4d28004500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4cdfc700 1 -- 172.21.2.104:0/536859059 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d48147b60 con 0x7f4d28004a40 2021-07-11T04:48:28.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4cdfc700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4d28004a40 0x7f4d28004ec0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7f4d40002320 tx=0x7f4d40007d80).ready entity=mon.0 client_cookie=65c3a2a506c92353 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d40002510 con 0x7f4d28004a40 2021-07-11T04:48:28.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d40006890 con 0x7f4d28004a40 2021-07-11T04:48:28.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d46ffd700 1 -- 172.21.2.104:0/536859059 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d40006bf0 con 0x7f4d28004a40 2021-07-11T04:48:28.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4e85f700 1 -- 172.21.2.104:0/536859059 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4d28004a40 msgr2=0x7f4d28004ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4e85f700 1 --2- 172.21.2.104:0/536859059 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4d28004a40 0x7f4d28004ec0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7f4d40002320 tx=0x7f4d40007d80).stop 2021-07-11T04:48:28.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.024+0000 7f4d4cdfc700 1 -- 172.21.2.104:0/536859059 reap_dead start 2021-07-11T04:48:28.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 -- 172.21.2.104:0/536859059 shutdown_connections 2021-07-11T04:48:28.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 -- 172.21.2.104:0/536859059 >> 172.21.2.104:0/536859059 conn(0x7f4d480b53e0 msgr2=0x7f4d480b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 -- 172.21.2.104:0/536859059 shutdown_connections 2021-07-11T04:48:28.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 -- 172.21.2.104:0/536859059 wait complete. 2021-07-11T04:48:28.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 Processor -- start 2021-07-11T04:48:28.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 -- start start 2021-07-11T04:48:28.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4d4814cc30 0x7f4d4814d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4d4815a5e0 0x7f4d4814c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4d48156530 0x7f4d481569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4814c810 con 0x7f4d4815a5e0 2021-07-11T04:48:28.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d48159c00 con 0x7f4d48156530 2021-07-11T04:48:28.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d48159ea0 con 0x7f4d4814cc30 2021-07-11T04:48:28.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d477fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4d4815a5e0 0x7f4d4814c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d477fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4d4815a5e0 0x7f4d4814c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52346/0 (socket says 172.21.2.104:52346) 2021-07-11T04:48:28.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d477fe700 1 -- 172.21.2.104:0/3062629864 learned_addr learned my addr 172.21.2.104:0/3062629864 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:28.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d477fe700 1 -- 172.21.2.104:0/3062629864 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4d4814cc30 msgr2=0x7f4d4814d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d477fe700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f4d4814cc30 0x7f4d4814d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d477fe700 1 -- 172.21.2.104:0/3062629864 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4d48156530 msgr2=0x7f4d481569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d477fe700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f4d48156530 0x7f4d481569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d477fe700 1 -- 172.21.2.104:0/3062629864 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4815a140 con 0x7f4d4815a5e0 2021-07-11T04:48:28.058 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d477fe700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4d4815a5e0 0x7f4d4814c320 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 rx=0x7f4d38008000 tx=0x7f4d38009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d457fa700 1 -- 172.21.2.104:0/3062629864 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d38007170 con 0x7f4d4815a5e0 2021-07-11T04:48:28.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d457fa700 1 -- 172.21.2.104:0/3062629864 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d380072d0 con 0x7f4d4815a5e0 2021-07-11T04:48:28.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 -- 172.21.2.104:0/3062629864 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4814e8e0 con 0x7f4d4815a5e0 2021-07-11T04:48:28.060 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d457fa700 1 -- 172.21.2.104:0/3062629864 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f4d380049e0 con 0x7f4d4815a5e0 2021-07-11T04:48:28.061 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 -- 172.21.2.104:0/3062629864 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d480aa990 con 0x7f4d4815a5e0 2021-07-11T04:48:28.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4e85f700 1 -- 172.21.2.104:0/3062629864 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4d48159a80 con 0x7f4d4815a5e0 2021-07-11T04:48:28.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d457fa700 1 -- 172.21.2.104:0/3062629864 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f4d38004b40 con 0x7f4d4815a5e0 2021-07-11T04:48:28.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d457fa700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4d2c053270 0x7f4d2c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d457fa700 1 -- 172.21.2.104:0/3062629864 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4d3806c410 con 0x7f4d4815a5e0 2021-07-11T04:48:28.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d457fa700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] conn(0x7f4d2c0599f0 0x7f4d2c05be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d457fa700 1 -- 172.21.2.104:0/3062629864 --> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d2c05c3d0 con 0x7f4d2c0599f0 2021-07-11T04:48:28.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d457fa700 1 -- 172.21.2.104:0/3062629864 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=164) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d3806c7f0 con 0x7f4d4815a5e0 2021-07-11T04:48:28.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.028+0000 7f4d4cdfc700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] conn(0x7f4d2c0599f0 0x7f4d2c05be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.032+0000 7f4d4cdfc700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] conn(0x7f4d2c0599f0 0x7f4d2c05be10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.032+0000 7f4d47fff700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4d2c053270 0x7f4d2c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.066 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.032+0000 7f4d457fa700 1 -- 172.21.2.104:0/3062629864 <== osd.2 v2:172.21.2.104:6816/15413 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f4d2c05c3d0 con 0x7f4d2c0599f0 2021-07-11T04:48:28.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.032+0000 7f4d47fff700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4d2c053270 0x7f4d2c055730 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f4d30008000 tx=0x7f4d30008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.067 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4e85f700 1 -- 172.21.2.104:0/3062629864 --> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4d48057350 con 0x7f4d2c0599f0 2021-07-11T04:48:28.068 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d457fa700 1 -- 172.21.2.104:0/3062629864 <== osd.2 v2:172.21.2.104:6816/15413 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4d48057350 con 0x7f4d2c0599f0 2021-07-11T04:48:28.068 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:48:28.069 INFO:teuthology.orchestra.run.gibba004.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-07-11T04:48:28.069 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:48:28.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4e85f700 1 -- 172.21.2.104:0/3062629864 >> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] conn(0x7f4d2c0599f0 msgr2=0x7f4d2c05be10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.071 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4e85f700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.104:6816/15413,v1:172.21.2.104:6817/15413] conn(0x7f4d2c0599f0 0x7f4d2c05be10 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4e85f700 1 -- 172.21.2.104:0/3062629864 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4d2c053270 msgr2=0x7f4d2c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4e85f700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f4d2c053270 0x7f4d2c055730 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f4d30008000 tx=0x7f4d30008040).stop 2021-07-11T04:48:28.072 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4e85f700 1 -- 172.21.2.104:0/3062629864 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4d4815a5e0 msgr2=0x7f4d4814c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4e85f700 1 --2- 172.21.2.104:0/3062629864 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f4d4815a5e0 0x7f4d4814c320 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 rx=0x7f4d38008000 tx=0x7f4d38009040).stop 2021-07-11T04:48:28.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4cdfc700 1 -- 172.21.2.104:0/3062629864 reap_dead start 2021-07-11T04:48:28.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4e85f700 1 -- 172.21.2.104:0/3062629864 shutdown_connections 2021-07-11T04:48:28.073 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4e85f700 1 -- 172.21.2.104:0/3062629864 >> 172.21.2.104:0/3062629864 conn(0x7f4d480b53e0 msgr2=0x7f4d4814a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4e85f700 1 -- 172.21.2.104:0/3062629864 shutdown_connections 2021-07-11T04:48:28.074 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.040+0000 7f4d4e85f700 1 -- 172.21.2.104:0/3062629864 wait complete. 2021-07-11T04:48:28.075 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2021-07-11T04:48:28.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.132+0000 7f12dda7b700 1 Processor -- start 2021-07-11T04:48:28.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.132+0000 7f12dda7b700 1 -- start start 2021-07-11T04:48:28.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f12d80aa860 0x7f12d80b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12d80a9da0 0x7f12d80aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f12d80acf30 0x7f12d81434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d80abb50 con 0x7f12d80acf30 2021-07-11T04:48:28.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d80ac370 con 0x7f12d80aa860 2021-07-11T04:48:28.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80aac40 con 0x7f12d80a9da0 2021-07-11T04:48:28.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d7fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f12d80acf30 0x7f12d81434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d7fff700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f12d80acf30 0x7f12d81434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52358/0 (socket says 172.21.2.104:52358) 2021-07-11T04:48:28.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d7fff700 1 -- 172.21.2.104:0/1703537407 learned_addr learned my addr 172.21.2.104:0/1703537407 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:28.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d7fff700 1 -- 172.21.2.104:0/1703537407 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12d80a9da0 msgr2=0x7f12d80aca40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:48:28.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d7fff700 1 --2- 172.21.2.104:0/1703537407 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12d80a9da0 0x7f12d80aca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d7fff700 1 -- 172.21.2.104:0/1703537407 >> v2:172.21.2.105:3300/0 conn(0x7f12d80aa860 msgr2=0x7f12d80b11c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:48:28.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d7fff700 1 --2- 172.21.2.104:0/1703537407 >> v2:172.21.2.105:3300/0 conn(0x7f12d80aa860 0x7f12d80b11c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d7fff700 1 -- 172.21.2.104:0/1703537407 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d80b16b0 con 0x7f12d80acf30 2021-07-11T04:48:28.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d7fff700 1 --2- 172.21.2.104:0/1703537407 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d80acf30 0x7f12d81434b0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 rx=0x7f12d8067210 tx=0x7f12d000b530).ready entity=mon.0 client_cookie=41eb04ee5176811 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d67fc700 1 -- 172.21.2.104:0/1703537407 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12d000bea0 con 0x7f12d80acf30 2021-07-11T04:48:28.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d67fc700 1 -- 172.21.2.104:0/1703537407 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d0010470 con 0x7f12d80acf30 2021-07-11T04:48:28.147 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d67fc700 1 -- 172.21.2.104:0/1703537407 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12d00107d0 con 0x7f12d80acf30 2021-07-11T04:48:28.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- 172.21.2.104:0/1703537407 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d80acf30 msgr2=0x7f12d81434b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 --2- 172.21.2.104:0/1703537407 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d80acf30 0x7f12d81434b0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 rx=0x7f12d8067210 tx=0x7f12d000b530).stop 2021-07-11T04:48:28.148 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- 172.21.2.104:0/1703537407 shutdown_connections 2021-07-11T04:48:28.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 --2- 172.21.2.104:0/1703537407 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d80acf30 0x7f12d81434b0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.149 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 --2- 172.21.2.104:0/1703537407 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12d80a9da0 0x7f12d80aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.150 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 --2- 172.21.2.104:0/1703537407 >> v2:172.21.2.105:3300/0 conn(0x7f12d80aa860 0x7f12d80b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- 172.21.2.104:0/1703537407 >> 172.21.2.104:0/1703537407 conn(0x7f12d80b64d0 msgr2=0x7f12d80b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.151 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- 172.21.2.104:0/1703537407 shutdown_connections 2021-07-11T04:48:28.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- 172.21.2.104:0/1703537407 wait complete. 2021-07-11T04:48:28.152 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 Processor -- start 2021-07-11T04:48:28.153 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- start start 2021-07-11T04:48:28.158 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d80a9da0 0x7f12d8065fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12d80aa860 0x7f12d809eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.159 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12d80acf30 0x7f12d809f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d8066490 con 0x7f12d80a9da0 2021-07-11T04:48:28.160 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d80a9440 con 0x7f12d80aa860 2021-07-11T04:48:28.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d814eb10 con 0x7f12d80acf30 2021-07-11T04:48:28.161 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d80a9da0 0x7f12d8065fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.162 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d77fe700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d80a9da0 0x7f12d8065fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52362/0 (socket says 172.21.2.104:52362) 2021-07-11T04:48:28.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d77fe700 1 -- 172.21.2.104:0/3718310458 learned_addr learned my addr 172.21.2.104:0/3718310458 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:28.163 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d77fe700 1 -- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12d80acf30 msgr2=0x7f12d809f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d77fe700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12d80acf30 0x7f12d809f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d77fe700 1 -- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12d80aa860 msgr2=0x7f12d809eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.164 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d77fe700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12d80aa860 0x7f12d809eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d77fe700 1 -- 172.21.2.104:0/3718310458 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d000b7d0 con 0x7f12d80a9da0 2021-07-11T04:48:28.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d77fe700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d80a9da0 0x7f12d8065fa0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7f12c0002fb0 tx=0x7f12c0006e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.165 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d4ff9700 1 -- 172.21.2.104:0/3718310458 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12c0013460 con 0x7f12d80a9da0 2021-07-11T04:48:28.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- 172.21.2.104:0/3718310458 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d814edb0 con 0x7f12d80a9da0 2021-07-11T04:48:28.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d4ff9700 1 -- 172.21.2.104:0/3718310458 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c00135c0 con 0x7f12d80a9da0 2021-07-11T04:48:28.166 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12d4ff9700 1 -- 172.21.2.104:0/3718310458 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f12c0013920 con 0x7f12d80a9da0 2021-07-11T04:48:28.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.136+0000 7f12dda7b700 1 -- 172.21.2.104:0/3718310458 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d809f050 con 0x7f12d80a9da0 2021-07-11T04:48:28.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.140+0000 7f12dda7b700 1 -- 172.21.2.104:0/3718310458 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d814e940 con 0x7f12d80a9da0 2021-07-11T04:48:28.167 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.140+0000 7f12d4ff9700 1 -- 172.21.2.104:0/3718310458 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f12c000f070 con 0x7f12d80a9da0 2021-07-11T04:48:28.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.140+0000 7f12d4ff9700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12bc053240 0x7f12bc055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.140+0000 7f12d4ff9700 1 -- 172.21.2.104:0/3718310458 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f12c006ad80 con 0x7f12d80a9da0 2021-07-11T04:48:28.168 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.140+0000 7f12d6ffd700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12bc053240 0x7f12bc055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.140+0000 7f12d6ffd700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12bc053240 0x7f12bc055700 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f12c8002fb0 tx=0x7f12c80069b0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.169 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.140+0000 7f12d4ff9700 1 -- 172.21.2.104:0/3718310458 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f12c0042290 con 0x7f12d80a9da0 2021-07-11T04:48:28.226 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 -- 172.21.2.104:0/3718310458 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f12d80a9580 con 0x7f12bc053240 2021-07-11T04:48:28.227 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12d4ff9700 1 -- 172.21.2.104:0/3718310458 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f12d80a9580 con 0x7f12bc053240 2021-07-11T04:48:28.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 -- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12bc053240 msgr2=0x7f12bc055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12bc053240 0x7f12bc055700 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f12c8002fb0 tx=0x7f12c80069b0).stop 2021-07-11T04:48:28.229 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 -- 172.21.2.104:0/3718310458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d80a9da0 msgr2=0x7f12d8065fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.230 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d80a9da0 0x7f12d8065fa0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7f12c0002fb0 tx=0x7f12c0006e10).stop 2021-07-11T04:48:28.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 -- 172.21.2.104:0/3718310458 shutdown_connections 2021-07-11T04:48:28.231 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f12d80acf30 0x7f12d809f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f12d80aa860 0x7f12d809eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f12bc053240 0x7f12bc055700 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.232 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 --2- 172.21.2.104:0/3718310458 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f12d80a9da0 0x7f12d8065fa0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 -- 172.21.2.104:0/3718310458 >> 172.21.2.104:0/3718310458 conn(0x7f12d80b64d0 msgr2=0x7f12d8143bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 -- 172.21.2.104:0/3718310458 shutdown_connections 2021-07-11T04:48:28.233 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.224+0000 7f12dda7b700 1 -- 172.21.2.104:0/3718310458 wait complete. 2021-07-11T04:48:28.234 INFO:teuthology.orchestra.run.gibba004.stderr:instructed osd(s) 2 to deep-scrub 2021-07-11T04:48:28.235 INFO:tasks.ceph:Scrubbing osd.3 2021-07-11T04:48:28.236 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2021-07-11T04:48:28.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 Processor -- start 2021-07-11T04:48:28.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 -- start start 2021-07-11T04:48:28.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0b0c0b1170 0x7f0b0c141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f0b0c141920 0x7f0b0c141d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b0c0b31d0 con 0x7f0b0c0b7810 2021-07-11T04:48:28.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b0c0b7bf0 con 0x7f0b0c141920 2021-07-11T04:48:28.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0675c0 con 0x7f0b0c0b1170 2021-07-11T04:48:28.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b11e3b700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f0b0c0b7810 0x7f0b0c1445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49136/0 (socket says 172.21.2.104:49136) 2021-07-11T04:48:28.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b11e3b700 1 -- 172.21.2.104:0/1083740894 learned_addr learned my addr 172.21.2.104:0/1083740894 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:48:28.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7856087 0 0) 0x7f0b0c0b31d0 con 0x7f0b0c0b7810 2021-07-11T04:48:28.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0af4003260 con 0x7f0b0c0b7810 2021-07-11T04:48:28.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2267608917 0 0) 0x7f0af4003260 con 0x7f0b0c0b7810 2021-07-11T04:48:28.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0b0c0b1170 msgr2=0x7f0b0c141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 --2- 172.21.2.104:0/1083740894 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0b0c0b1170 0x7f0b0c141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 >> v2:172.21.2.105:3300/0 conn(0x7f0b0c141920 msgr2=0x7f0b0c141d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 --2- 172.21.2.104:0/1083740894 >> v2:172.21.2.105:3300/0 conn(0x7f0b0c141920 0x7f0b0c141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c142270 con 0x7f0b0c0b7810 2021-07-11T04:48:28.302 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f0b00001660 con 0x7f0b0c0b7810 2021-07-11T04:48:28.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 >> v1:172.21.2.104:6789/0 conn(0x7f0b0c0b7810 legacy=0x7f0b0c1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 --2- 172.21.2.104:0/1083740894 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0af40037d0 0x7f0af4003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.303 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 --2- 172.21.2.104:0/1083740894 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0b0c0b1170 0x7f0af4004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 --2- 172.21.2.104:0/1083740894 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0af4004b10 0x7f0af4004f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0b7bf0 con 0x7f0af40037d0 2021-07-11T04:48:28.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0675c0 con 0x7f0af4004b10 2021-07-11T04:48:28.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af400fe20 con 0x7f0b0c0b1170 2021-07-11T04:48:28.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b11e3b700 1 --2- 172.21.2.104:0/1083740894 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0af40037d0 0x7f0af4003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b11e3b700 1 -- 172.21.2.104:0/1083740894 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0b0c0b1170 msgr2=0x7f0af4004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b11e3b700 1 --2- 172.21.2.104:0/1083740894 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0b0c0b1170 0x7f0af4004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b11e3b700 1 -- 172.21.2.104:0/1083740894 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0af4004b10 msgr2=0x7f0af4004f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b11e3b700 1 --2- 172.21.2.104:0/1083740894 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0af4004b10 0x7f0af4004f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b11e3b700 1 -- 172.21.2.104:0/1083740894 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c142270 con 0x7f0af40037d0 2021-07-11T04:48:28.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b11e3b700 1 --2- 172.21.2.104:0/1083740894 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0af40037d0 0x7f0af4003bf0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 rx=0x7f0b00001370 tx=0x7f0b00001e40).ready entity=mon.0 client_cookie=790bce2007e5b9a8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b000070d0 con 0x7f0af40037d0 2021-07-11T04:48:28.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b00007230 con 0x7f0af40037d0 2021-07-11T04:48:28.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b10e39700 1 -- 172.21.2.104:0/1083740894 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b0000c980 con 0x7f0af40037d0 2021-07-11T04:48:28.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 -- 172.21.2.104:0/1083740894 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0af40037d0 msgr2=0x7f0af4003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 --2- 172.21.2.104:0/1083740894 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0af40037d0 0x7f0af4003bf0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 rx=0x7f0b00001370 tx=0x7f0b00001e40).stop 2021-07-11T04:48:28.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1263c700 1 -- 172.21.2.104:0/1083740894 reap_dead start 2021-07-11T04:48:28.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 -- 172.21.2.104:0/1083740894 shutdown_connections 2021-07-11T04:48:28.309 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 -- 172.21.2.104:0/1083740894 >> 172.21.2.104:0/1083740894 conn(0x7f0b0c05f6b0 msgr2=0x7f0b0c05fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 -- 172.21.2.104:0/1083740894 shutdown_connections 2021-07-11T04:48:28.310 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.288+0000 7f0b1409f700 1 -- 172.21.2.104:0/1083740894 wait complete. 2021-07-11T04:48:28.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1409f700 1 Processor -- start 2021-07-11T04:48:28.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1409f700 1 -- start start 2021-07-11T04:48:28.311 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1409f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0b0c145ef0 0x7f0b0c13f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1409f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0b0c05d0e0 0x7f0b0c13fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.312 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1409f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0b0c14e0b0 0x7f0b0c14e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1409f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0abd90 con 0x7f0b0c05d0e0 2021-07-11T04:48:28.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1409f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c14dd00 con 0x7f0b0c145ef0 2021-07-11T04:48:28.313 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1409f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c0ac4c0 con 0x7f0b0c14e0b0 2021-07-11T04:48:28.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1163a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0b0c05d0e0 0x7f0b0c13fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.314 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1163a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0b0c05d0e0 0x7f0b0c13fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52382/0 (socket says 172.21.2.104:52382) 2021-07-11T04:48:28.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1163a700 1 -- 172.21.2.104:0/4027804680 learned_addr learned my addr 172.21.2.104:0/4027804680 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:28.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1163a700 1 -- 172.21.2.104:0/4027804680 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0b0c14e0b0 msgr2=0x7f0b0c14e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.315 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1163a700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f0b0c14e0b0 0x7f0b0c14e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1163a700 1 -- 172.21.2.104:0/4027804680 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0b0c145ef0 msgr2=0x7f0b0c13f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1163a700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f0b0c145ef0 0x7f0b0c13f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.316 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1163a700 1 -- 172.21.2.104:0/4027804680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c1512b0 con 0x7f0b0c05d0e0 2021-07-11T04:48:28.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1163a700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0b0c05d0e0 0x7f0b0c13fcf0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 rx=0x7f0b0c0b0850 tx=0x7f0b08006e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0afeffd700 1 -- 172.21.2.104:0/4027804680 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b0800e470 con 0x7f0b0c05d0e0 2021-07-11T04:48:28.317 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0afeffd700 1 -- 172.21.2.104:0/4027804680 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b08000e90 con 0x7f0b0c05d0e0 2021-07-11T04:48:28.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0afeffd700 1 -- 172.21.2.104:0/4027804680 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f0b0800e800 con 0x7f0b0c05d0e0 2021-07-11T04:48:28.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1409f700 1 -- 172.21.2.104:0/4027804680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c1462d0 con 0x7f0b0c05d0e0 2021-07-11T04:48:28.318 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1409f700 1 -- 172.21.2.104:0/4027804680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c0ac220 con 0x7f0b0c05d0e0 2021-07-11T04:48:28.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1409f700 1 -- 172.21.2.104:0/4027804680 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0b0c14de50 con 0x7f0b0c05d0e0 2021-07-11T04:48:28.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0afeffd700 1 -- 172.21.2.104:0/4027804680 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f0b0800f070 con 0x7f0b0c05d0e0 2021-07-11T04:48:28.319 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0afeffd700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0aec053180 0x7f0aec055640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0afeffd700 1 -- 172.21.2.104:0/4027804680 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0b0806a540 con 0x7f0b0c05d0e0 2021-07-11T04:48:28.320 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0afeffd700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] conn(0x7f0aec0599a0 0x7f0aec05bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0afeffd700 1 -- 172.21.2.104:0/4027804680 --> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0aec05c420 con 0x7f0aec0599a0 2021-07-11T04:48:28.321 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0afeffd700 1 -- 172.21.2.104:0/4027804680 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=164) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b0806cbd0 con 0x7f0b0c05d0e0 2021-07-11T04:48:28.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1263c700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] conn(0x7f0aec0599a0 0x7f0aec05bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b11e3b700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0aec053180 0x7f0aec055640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.322 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.292+0000 7f0b1263c700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] conn(0x7f0aec0599a0 0x7f0aec05bdc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.296+0000 7f0afeffd700 1 -- 172.21.2.104:0/4027804680 <== osd.3 v2:172.21.2.104:6824/15456 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f0aec05c420 con 0x7f0aec0599a0 2021-07-11T04:48:28.323 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.296+0000 7f0b11e3b700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0aec053180 0x7f0aec055640 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f0b00001370 tx=0x7f0b00015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1409f700 1 -- 172.21.2.104:0/4027804680 --> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0b0c057350 con 0x7f0aec0599a0 2021-07-11T04:48:28.324 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0afeffd700 1 -- 172.21.2.104:0/4027804680 <== osd.3 v2:172.21.2.104:6824/15456 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0b0c057350 con 0x7f0aec0599a0 2021-07-11T04:48:28.325 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:48:28.325 INFO:teuthology.orchestra.run.gibba004.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-07-11T04:48:28.326 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:48:28.327 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1409f700 1 -- 172.21.2.104:0/4027804680 >> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] conn(0x7f0aec0599a0 msgr2=0x7f0aec05bdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1409f700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.104:6824/15456,v1:172.21.2.104:6825/15456] conn(0x7f0aec0599a0 0x7f0aec05bdc0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1409f700 1 -- 172.21.2.104:0/4027804680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0aec053180 msgr2=0x7f0aec055640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.328 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1409f700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f0aec053180 0x7f0aec055640 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f0b00001370 tx=0x7f0b00015000).stop 2021-07-11T04:48:28.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1409f700 1 -- 172.21.2.104:0/4027804680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0b0c05d0e0 msgr2=0x7f0b0c13fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1409f700 1 --2- 172.21.2.104:0/4027804680 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f0b0c05d0e0 0x7f0b0c13fcf0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 rx=0x7f0b0c0b0850 tx=0x7f0b08006e70).stop 2021-07-11T04:48:28.329 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1263c700 1 -- 172.21.2.104:0/4027804680 reap_dead start 2021-07-11T04:48:28.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1409f700 1 -- 172.21.2.104:0/4027804680 shutdown_connections 2021-07-11T04:48:28.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1409f700 1 -- 172.21.2.104:0/4027804680 >> 172.21.2.104:0/4027804680 conn(0x7f0b0c05f6b0 msgr2=0x7f0b0c0b60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.330 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1409f700 1 -- 172.21.2.104:0/4027804680 shutdown_connections 2021-07-11T04:48:28.331 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.304+0000 7f0b1409f700 1 -- 172.21.2.104:0/4027804680 wait complete. 2021-07-11T04:48:28.331 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2021-07-11T04:48:28.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbb0068700 1 Processor -- start 2021-07-11T04:48:28.392 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbb0068700 1 -- start start 2021-07-11T04:48:28.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbb0068700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcba80a76b0 0x7fcba8066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbb0068700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fcba80b0060 0x7fcba805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.393 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbb0068700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba80a7a90 con 0x7fcba80b1970 2021-07-11T04:48:28.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbb0068700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba8147620 con 0x7fcba80b0060 2021-07-11T04:48:28.394 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbb0068700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba81478c0 con 0x7fcba80a76b0 2021-07-11T04:48:28.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbad603700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fcba80b1970 0x7fcba805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49160/0 (socket says 172.21.2.104:49160) 2021-07-11T04:48:28.400 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbad603700 1 -- 172.21.2.104:0/3606020037 learned_addr learned my addr 172.21.2.104:0/3606020037 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:48:28.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425865859 0 0) 0x7fcba80a7a90 con 0x7fcba80b1970 2021-07-11T04:48:28.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb90003260 con 0x7fcba80b1970 2021-07-11T04:48:28.401 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1857139686 0 0) 0x7fcb90003260 con 0x7fcba80b1970 2021-07-11T04:48:28.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcba80a76b0 msgr2=0x7fcba8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbade04700 1 --2- 172.21.2.104:0/3606020037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcba80a76b0 0x7fcba8066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.402 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbae605700 1 --2- 172.21.2.104:0/3606020037 >> v2:172.21.2.105:3300/0 conn(0x7fcba80b0060 0x7fcba805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 --2- 172.21.2.104:0/3606020037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcba80a76b0 0x7fcba8066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 >> v2:172.21.2.105:3300/0 conn(0x7fcba80b0060 msgr2=0x7fcba805fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.403 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 --2- 172.21.2.104:0/3606020037 >> v2:172.21.2.105:3300/0 conn(0x7fcba80b0060 0x7fcba805fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba8147b60 con 0x7fcba80b1970 2021-07-11T04:48:28.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fcba4001660 con 0x7fcba80b1970 2021-07-11T04:48:28.404 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 >> v1:172.21.2.104:6789/0 conn(0x7fcba80b1970 legacy=0x7fcba805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 --2- 172.21.2.104:0/3606020037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb900037d0 0x7fcb90003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 --2- 172.21.2.104:0/3606020037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcba80a76b0 0x7fcb90004500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.405 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 --2- 172.21.2.104:0/3606020037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb90004bf0 0x7fcb9000b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba8147620 con 0x7fcb900037d0 2021-07-11T04:48:28.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba81478c0 con 0x7fcb90004bf0 2021-07-11T04:48:28.406 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb90005300 con 0x7fcba80a76b0 2021-07-11T04:48:28.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbade04700 1 --2- 172.21.2.104:0/3606020037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb900037d0 0x7fcb90003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbade04700 1 -- 172.21.2.104:0/3606020037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcba80a76b0 msgr2=0x7fcb90004500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.407 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbade04700 1 --2- 172.21.2.104:0/3606020037 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcba80a76b0 0x7fcb90004500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbade04700 1 -- 172.21.2.104:0/3606020037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb90004bf0 msgr2=0x7fcb9000b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbade04700 1 --2- 172.21.2.104:0/3606020037 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcb90004bf0 0x7fcb9000b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.408 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbade04700 1 -- 172.21.2.104:0/3606020037 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb90003260 con 0x7fcb900037d0 2021-07-11T04:48:28.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbade04700 1 --2- 172.21.2.104:0/3606020037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb900037d0 0x7fcb90003bf0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 rx=0x7fcb9c007000 tx=0x7fcb9c008040).ready entity=mon.0 client_cookie=33a2fa975358ae8f server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb9c012470 con 0x7fcb900037d0 2021-07-11T04:48:28.409 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c00c030 con 0x7fcb900037d0 2021-07-11T04:48:28.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbace02700 1 -- 172.21.2.104:0/3606020037 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb9c0127c0 con 0x7fcb900037d0 2021-07-11T04:48:28.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbb0068700 1 -- 172.21.2.104:0/3606020037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb900037d0 msgr2=0x7fcb90003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.410 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbb0068700 1 --2- 172.21.2.104:0/3606020037 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcb900037d0 0x7fcb90003bf0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 rx=0x7fcb9c007000 tx=0x7fcb9c008040).stop 2021-07-11T04:48:28.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.388+0000 7fcbae605700 1 -- 172.21.2.104:0/3606020037 reap_dead start 2021-07-11T04:48:28.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 -- 172.21.2.104:0/3606020037 shutdown_connections 2021-07-11T04:48:28.411 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 -- 172.21.2.104:0/3606020037 >> 172.21.2.104:0/3606020037 conn(0x7fcba80b53e0 msgr2=0x7fcba80b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 -- 172.21.2.104:0/3606020037 shutdown_connections 2021-07-11T04:48:28.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 -- 172.21.2.104:0/3606020037 wait complete. 2021-07-11T04:48:28.412 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 Processor -- start 2021-07-11T04:48:28.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 -- start start 2021-07-11T04:48:28.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcba80af0a0 0x7fcba81571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.413 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcba80a98f0 0x7fcba8066420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcba8066910 0x7fcba80a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba81576e0 con 0x7fcba80af0a0 2021-07-11T04:48:28.414 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80ae0c0 con 0x7fcba80a98f0 2021-07-11T04:48:28.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80af480 con 0x7fcba8066910 2021-07-11T04:48:28.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbade04700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcba80af0a0 0x7fcba81571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.415 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbade04700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcba80af0a0 0x7fcba81571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52404/0 (socket says 172.21.2.104:52404) 2021-07-11T04:48:28.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbade04700 1 -- 172.21.2.104:0/2645626626 learned_addr learned my addr 172.21.2.104:0/2645626626 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:28.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbade04700 1 -- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcba8066910 msgr2=0x7fcba80a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.416 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbade04700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcba8066910 0x7fcba80a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbade04700 1 -- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcba80a98f0 msgr2=0x7fcba8066420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbade04700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcba80a98f0 0x7fcba8066420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.417 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbade04700 1 -- 172.21.2.104:0/2645626626 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba8066da0 con 0x7fcba80af0a0 2021-07-11T04:48:28.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbade04700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcba80af0a0 0x7fcba81571f0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7fcb9c007000 tx=0x7fcb9c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcb9affd700 1 -- 172.21.2.104:0/2645626626 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb9c01c8e0 con 0x7fcba80af0a0 2021-07-11T04:48:28.418 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcb9affd700 1 -- 172.21.2.104:0/2645626626 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c000e90 con 0x7fcba80af0a0 2021-07-11T04:48:28.419 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 -- 172.21.2.104:0/2645626626 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba80625d0 con 0x7fcba80af0a0 2021-07-11T04:48:28.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcb9affd700 1 -- 172.21.2.104:0/2645626626 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fcb9c01cc30 con 0x7fcba80af0a0 2021-07-11T04:48:28.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 -- 172.21.2.104:0/2645626626 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba8062b10 con 0x7fcba80af0a0 2021-07-11T04:48:28.420 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbb0068700 1 -- 172.21.2.104:0/2645626626 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba8062f50 con 0x7fcba80af0a0 2021-07-11T04:48:28.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcb9affd700 1 -- 172.21.2.104:0/2645626626 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fcb9c02b070 con 0x7fcba80af0a0 2021-07-11T04:48:28.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcb9affd700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb88053160 0x7fcb88055620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.421 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcb9affd700 1 -- 172.21.2.104:0/2645626626 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcb9c06c790 con 0x7fcba80af0a0 2021-07-11T04:48:28.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.392+0000 7fcbad603700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb88053160 0x7fcb88055620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.396+0000 7fcbad603700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb88053160 0x7fcb88055620 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fcba4001370 tx=0x7fcba40014c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.422 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.396+0000 7fcb9affd700 1 -- 172.21.2.104:0/2645626626 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fcb9c043db0 con 0x7fcba80af0a0 2021-07-11T04:48:28.482 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 -- 172.21.2.104:0/2645626626 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcba8057350 con 0x7fcb88053160 2021-07-11T04:48:28.483 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcb9affd700 1 -- 172.21.2.104:0/2645626626 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcba8057350 con 0x7fcb88053160 2021-07-11T04:48:28.484 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 -- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb88053160 msgr2=0x7fcb88055620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb88053160 0x7fcb88055620 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fcba4001370 tx=0x7fcba40014c0).stop 2021-07-11T04:48:28.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 -- 172.21.2.104:0/2645626626 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcba80af0a0 msgr2=0x7fcba81571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.485 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcba80af0a0 0x7fcba81571f0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7fcb9c007000 tx=0x7fcb9c008040).stop 2021-07-11T04:48:28.486 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 -- 172.21.2.104:0/2645626626 shutdown_connections 2021-07-11T04:48:28.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fcba8066910 0x7fcba80a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.487 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fcba80a98f0 0x7fcba8066420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fcb88053160 0x7fcb88055620 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 --2- 172.21.2.104:0/2645626626 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fcba80af0a0 0x7fcba81571f0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.488 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 -- 172.21.2.104:0/2645626626 >> 172.21.2.104:0/2645626626 conn(0x7fcba80b53e0 msgr2=0x7fcba805ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 -- 172.21.2.104:0/2645626626 shutdown_connections 2021-07-11T04:48:28.489 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.480+0000 7fcbb0068700 1 -- 172.21.2.104:0/2645626626 wait complete. 2021-07-11T04:48:28.489 INFO:teuthology.orchestra.run.gibba004.stderr:instructed osd(s) 3 to deep-scrub 2021-07-11T04:48:28.499 INFO:tasks.ceph:Scrubbing osd.4 2021-07-11T04:48:28.500 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2021-07-11T04:48:28.553 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.548+0000 7f3727c4d700 1 Processor -- start 2021-07-11T04:48:28.554 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.548+0000 7f3727c4d700 1 -- start start 2021-07-11T04:48:28.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f37200ad150 0x7f37200b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.556 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37200ac370 con 0x7f37200aa860 2021-07-11T04:48:28.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37200aac40 con 0x7f37200a9da0 2021-07-11T04:48:28.557 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37200abb50 con 0x7f37200ad150 2021-07-11T04:48:28.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37259e9700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f37200aa860 0x7f37200aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49178/0 (socket says 172.21.2.104:49178) 2021-07-11T04:48:28.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37259e9700 1 -- 172.21.2.104:0/2516110112 learned_addr learned my addr 172.21.2.104:0/2516110112 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:48:28.558 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37251e8700 1 --2- 172.21.2.104:0/2516110112 >> v2:172.21.2.104:3300/0 conn(0x7f37200ad150 0x7f37200b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37251e8700 1 -- 172.21.2.104:0/2516110112 >> v1:172.21.2.105:6789/0 conn(0x7f37200a9da0 legacy=0x7f3720140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37251e8700 1 -- 172.21.2.104:0/2516110112 >> v1:172.21.2.104:6789/0 conn(0x7f37200aa860 legacy=0x7f37200aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.559 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37251e8700 1 -- 172.21.2.104:0/2516110112 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37200b16b0 con 0x7f37200ad150 2021-07-11T04:48:28.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37251e8700 1 --2- 172.21.2.104:0/2516110112 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f37200ad150 0x7f37200b11c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 rx=0x7f37200b63e0 tx=0x7f371c006df0).ready entity=mon.0 client_cookie=c6a7792f1f6f70d8 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37249e7700 1 -- 172.21.2.104:0/2516110112 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f371c014070 con 0x7f37200ad150 2021-07-11T04:48:28.560 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37249e7700 1 -- 172.21.2.104:0/2516110112 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f371c000e90 con 0x7f37200ad150 2021-07-11T04:48:28.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37249e7700 1 -- 172.21.2.104:0/2516110112 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f371c00fbb0 con 0x7f37200ad150 2021-07-11T04:48:28.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- 172.21.2.104:0/2516110112 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f37200ad150 msgr2=0x7f37200b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.561 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 --2- 172.21.2.104:0/2516110112 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f37200ad150 0x7f37200b11c0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 rx=0x7f37200b63e0 tx=0x7f371c006df0).stop 2021-07-11T04:48:28.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- 172.21.2.104:0/2516110112 shutdown_connections 2021-07-11T04:48:28.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 --2- 172.21.2.104:0/2516110112 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f37200ad150 0x7f37200b11c0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.562 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- 172.21.2.104:0/2516110112 >> 172.21.2.104:0/2516110112 conn(0x7f37200b64d0 msgr2=0x7f37200b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- 172.21.2.104:0/2516110112 shutdown_connections 2021-07-11T04:48:28.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- 172.21.2.104:0/2516110112 wait complete. 2021-07-11T04:48:28.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 Processor -- start 2021-07-11T04:48:28.563 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- start start 2021-07-11T04:48:28.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f37200a9da0 0x7f3720156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.566 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f37200aa860 0x7f37200afa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f37200ad150 0x7f372009e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3720065810 con 0x7f37200a9da0 2021-07-11T04:48:28.567 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3720065ce0 con 0x7f37200aa860 2021-07-11T04:48:28.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37200a1010 con 0x7f37200ad150 2021-07-11T04:48:28.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37259e9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f37200a9da0 0x7f3720156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.568 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37259e9700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f37200a9da0 0x7f3720156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52418/0 (socket says 172.21.2.104:52418) 2021-07-11T04:48:28.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37259e9700 1 -- 172.21.2.104:0/1120688623 learned_addr learned my addr 172.21.2.104:0/1120688623 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:28.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37259e9700 1 -- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f37200ad150 msgr2=0x7f372009e8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:48:28.569 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37259e9700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f37200ad150 0x7f372009e8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37259e9700 1 -- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f37200aa860 msgr2=0x7f37200afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37259e9700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f37200aa860 0x7f37200afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.570 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37259e9700 1 -- 172.21.2.104:0/1120688623 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c00b180 con 0x7f37200a9da0 2021-07-11T04:48:28.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f37259e9700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f37200a9da0 0x7f3720156a20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 rx=0x7f37200b2e10 tx=0x7f3714007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3712ffd700 1 -- 172.21.2.104:0/1120688623 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f371400ce30 con 0x7f37200a9da0 2021-07-11T04:48:28.571 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3712ffd700 1 -- 172.21.2.104:0/1120688623 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37140072e0 con 0x7f37200a9da0 2021-07-11T04:48:28.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3712ffd700 1 -- 172.21.2.104:0/1120688623 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f3714005800 con 0x7f37200a9da0 2021-07-11T04:48:28.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- 172.21.2.104:0/1120688623 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3720065a40 con 0x7f37200a9da0 2021-07-11T04:48:28.572 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.552+0000 7f3727c4d700 1 -- 172.21.2.104:0/1120688623 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37200668e0 con 0x7f37200a9da0 2021-07-11T04:48:28.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.556+0000 7f3727c4d700 1 -- 172.21.2.104:0/1120688623 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3720066ae0 con 0x7f37200a9da0 2021-07-11T04:48:28.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.556+0000 7f3712ffd700 1 -- 172.21.2.104:0/1120688623 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f3714010070 con 0x7f37200a9da0 2021-07-11T04:48:28.573 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.556+0000 7f3712ffd700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3708053270 0x7f3708055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.556+0000 7f3712ffd700 1 -- 172.21.2.104:0/1120688623 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f371406af20 con 0x7f37200a9da0 2021-07-11T04:48:28.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.556+0000 7f3712ffd700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] conn(0x7f37080599f0 0x7f370805be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.574 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.556+0000 7f3712ffd700 1 -- 172.21.2.104:0/1120688623 --> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f370805c3d0 con 0x7f37080599f0 2021-07-11T04:48:28.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.556+0000 7f3712ffd700 1 -- 172.21.2.104:0/1120688623 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=164) v2 ==== 24+0+0 (secure 0 0 0) 0x7f371406b300 con 0x7f37200a9da0 2021-07-11T04:48:28.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.556+0000 7f37251e8700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3708053270 0x7f3708055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.575 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.556+0000 7f37261ea700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] conn(0x7f37080599f0 0x7f370805be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.556+0000 7f37261ea700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] conn(0x7f37080599f0 0x7f370805be10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.556+0000 7f37251e8700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3708053270 0x7f3708055730 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f37200b63e0 tx=0x7f371c014000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.576 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.560+0000 7f3712ffd700 1 -- 172.21.2.104:0/1120688623 <== osd.4 v2:172.21.2.105:6802/15645 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f370805c3d0 con 0x7f37080599f0 2021-07-11T04:48:28.577 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3727c4d700 1 -- 172.21.2.104:0/1120688623 --> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f372009efe0 con 0x7f37080599f0 2021-07-11T04:48:28.578 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3712ffd700 1 -- 172.21.2.104:0/1120688623 <== osd.4 v2:172.21.2.105:6802/15645 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f372009efe0 con 0x7f37080599f0 2021-07-11T04:48:28.578 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:48:28.579 INFO:teuthology.orchestra.run.gibba004.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-07-11T04:48:28.579 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:48:28.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3727c4d700 1 -- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] conn(0x7f37080599f0 msgr2=0x7f370805be10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3727c4d700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:6802/15645,v1:172.21.2.105:6803/15645] conn(0x7f37080599f0 0x7f370805be10 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.581 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3727c4d700 1 -- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3708053270 msgr2=0x7f3708055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3727c4d700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f3708053270 0x7f3708055730 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f37200b63e0 tx=0x7f371c014000).stop 2021-07-11T04:48:28.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3727c4d700 1 -- 172.21.2.104:0/1120688623 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f37200a9da0 msgr2=0x7f3720156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.582 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3727c4d700 1 --2- 172.21.2.104:0/1120688623 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f37200a9da0 0x7f3720156a20 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 rx=0x7f37200b2e10 tx=0x7f3714007c40).stop 2021-07-11T04:48:28.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f37261ea700 1 -- 172.21.2.104:0/1120688623 reap_dead start 2021-07-11T04:48:28.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3727c4d700 1 -- 172.21.2.104:0/1120688623 shutdown_connections 2021-07-11T04:48:28.583 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3727c4d700 1 -- 172.21.2.104:0/1120688623 >> 172.21.2.104:0/1120688623 conn(0x7f37200b64d0 msgr2=0x7f37200618e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3727c4d700 1 -- 172.21.2.104:0/1120688623 shutdown_connections 2021-07-11T04:48:28.584 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.568+0000 7f3727c4d700 1 -- 172.21.2.104:0/1120688623 wait complete. 2021-07-11T04:48:28.585 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2021-07-11T04:48:28.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.632+0000 7f7feb2ca700 1 Processor -- start 2021-07-11T04:48:28.637 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.632+0000 7f7feb2ca700 1 -- start start 2021-07-11T04:48:28.641 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.632+0000 7f7feb2ca700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7fe40aa860 0x7f7fe40b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.632+0000 7f7feb2ca700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fe40a9da0 0x7f7fe40aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.632+0000 7f7feb2ca700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe40ac370 con 0x7f7fe40acf30 2021-07-11T04:48:28.642 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.632+0000 7f7feb2ca700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe40abb50 con 0x7f7fe40aa860 2021-07-11T04:48:28.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.632+0000 7f7feb2ca700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe40aac40 con 0x7f7fe40a9da0 2021-07-11T04:48:28.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.632+0000 7f7fe9066700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7fe40aa860 0x7f7fe40b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.643 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.632+0000 7f7fe9066700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f7fe40aa860 0x7f7fe40b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52426/0 (socket says 172.21.2.104:52426) 2021-07-11T04:48:28.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.632+0000 7f7fe9066700 1 -- 172.21.2.104:0/2876446926 learned_addr learned my addr 172.21.2.104:0/2876446926 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:28.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9066700 1 -- 172.21.2.104:0/2876446926 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fe40a9da0 msgr2=0x7f7fe40aca40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:48:28.644 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9066700 1 --2- 172.21.2.104:0/2876446926 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fe40a9da0 0x7f7fe40aca40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9066700 1 -- 172.21.2.104:0/2876446926 >> v1:172.21.2.105:6789/0 conn(0x7f7fe40acf30 legacy=0x7f7fe41434b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2021-07-11T04:48:28.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9066700 1 -- 172.21.2.104:0/2876446926 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe40b16b0 con 0x7f7fe40aa860 2021-07-11T04:48:28.645 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9066700 1 --2- 172.21.2.104:0/2876446926 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fe40aa860 0x7f7fe40b11c0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 rx=0x7f7fe40ac920 tx=0x7f7fd8006eb0).ready entity=mon.0 client_cookie=482c1f4f70c80c1c server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fd7fff700 1 -- 172.21.2.104:0/2876446926 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fd800d570 con 0x7f7fe40aa860 2021-07-11T04:48:28.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fd7fff700 1 -- 172.21.2.104:0/2876446926 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd8000e90 con 0x7f7fe40aa860 2021-07-11T04:48:28.646 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fd7fff700 1 -- 172.21.2.104:0/2876446926 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fd800d8c0 con 0x7f7fe40aa860 2021-07-11T04:48:28.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2876446926 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fe40aa860 msgr2=0x7f7fe40b11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 --2- 172.21.2.104:0/2876446926 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fe40aa860 0x7f7fe40b11c0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 rx=0x7f7fe40ac920 tx=0x7f7fd8006eb0).stop 2021-07-11T04:48:28.647 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2876446926 shutdown_connections 2021-07-11T04:48:28.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 --2- 172.21.2.104:0/2876446926 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fe40a9da0 0x7f7fe40aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 --2- 172.21.2.104:0/2876446926 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fe40aa860 0x7f7fe40b11c0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.648 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2876446926 >> 172.21.2.104:0/2876446926 conn(0x7f7fe40b64d0 msgr2=0x7f7fe40b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2876446926 shutdown_connections 2021-07-11T04:48:28.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2876446926 wait complete. 2021-07-11T04:48:28.649 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 Processor -- start 2021-07-11T04:48:28.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- start start 2021-07-11T04:48:28.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7fe40a9da0 0x7f7fe4142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.650 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fe40aa860 0x7f7fe4146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fe40acf30 0x7f7fe4145100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe40663d0 con 0x7f7fe40acf30 2021-07-11T04:48:28.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe41402c0 con 0x7f7fe40a9da0 2021-07-11T04:48:28.651 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe40658b0 con 0x7f7fe40aa860 2021-07-11T04:48:28.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9867700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fe40acf30 0x7f7fe4145100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.652 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9867700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fe40acf30 0x7f7fe4145100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52436/0 (socket says 172.21.2.104:52436) 2021-07-11T04:48:28.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9867700 1 -- 172.21.2.104:0/2023562061 learned_addr learned my addr 172.21.2.104:0/2023562061 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:28.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9867700 1 -- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fe40aa860 msgr2=0x7f7fe4146ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:48:28.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9867700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fe40aa860 0x7f7fe4146ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.653 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9867700 1 -- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7fe40a9da0 msgr2=0x7f7fe4142ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:48:28.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9867700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7fe40a9da0 0x7f7fe4142ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9867700 1 -- 172.21.2.104:0/2023562061 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd800b1e0 con 0x7f7fe40acf30 2021-07-11T04:48:28.654 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fe9867700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fe40acf30 0x7f7fe4145100 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 rx=0x7f7fdc001250 tx=0x7f7fdc0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fd67fc700 1 -- 172.21.2.104:0/2023562061 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fdc0014e0 con 0x7f7fe40acf30 2021-07-11T04:48:28.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fd67fc700 1 -- 172.21.2.104:0/2023562061 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fdc001640 con 0x7f7fe40acf30 2021-07-11T04:48:28.655 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2023562061 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe4065e10 con 0x7f7fe40acf30 2021-07-11T04:48:28.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fd67fc700 1 -- 172.21.2.104:0/2023562061 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f7fdc0019a0 con 0x7f7fe40acf30 2021-07-11T04:48:28.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2023562061 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe4146fc0 con 0x7f7fe40acf30 2021-07-11T04:48:28.656 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2023562061 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe4057350 con 0x7f7fe40acf30 2021-07-11T04:48:28.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fd67fc700 1 -- 172.21.2.104:0/2023562061 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f7fdc018030 con 0x7f7fe40acf30 2021-07-11T04:48:28.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fd67fc700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7fc4053330 0x7f7fc40557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.657 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.636+0000 7f7fd67fc700 1 -- 172.21.2.104:0/2023562061 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7fdc06aaa0 con 0x7f7fe40acf30 2021-07-11T04:48:28.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.640+0000 7f7fe9066700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7fc4053330 0x7f7fc40557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.658 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.640+0000 7f7fe9066700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7fc4053330 0x7f7fc40557f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f7fe40ac920 tx=0x7f7fd8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.659 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.640+0000 7f7fd67fc700 1 -- 172.21.2.104:0/2023562061 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f7fdc042170 con 0x7f7fe40acf30 2021-07-11T04:48:28.725 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.720+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2023562061 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fe40698e0 con 0x7f7fc4053330 2021-07-11T04:48:28.726 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7fd67fc700 1 -- 172.21.2.104:0/2023562061 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7fe40698e0 con 0x7f7fc4053330 2021-07-11T04:48:28.728 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7fc4053330 msgr2=0x7f7fc40557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7fc4053330 0x7f7fc40557f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f7fe40ac920 tx=0x7f7fd8015000).stop 2021-07-11T04:48:28.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2023562061 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fe40acf30 msgr2=0x7f7fe4145100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.729 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fe40acf30 0x7f7fe4145100 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 rx=0x7f7fdc001250 tx=0x7f7fdc0090f0).stop 2021-07-11T04:48:28.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2023562061 shutdown_connections 2021-07-11T04:48:28.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f7fc4053330 0x7f7fc40557f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.730 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f7fe40acf30 0x7f7fe4145100 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f7fe40aa860 0x7f7fe4146ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 --2- 172.21.2.104:0/2023562061 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f7fe40a9da0 0x7f7fe4142ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.731 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2023562061 >> 172.21.2.104:0/2023562061 conn(0x7f7fe40b64d0 msgr2=0x7f7fe405fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2023562061 shutdown_connections 2021-07-11T04:48:28.732 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.724+0000 7f7feb2ca700 1 -- 172.21.2.104:0/2023562061 wait complete. 2021-07-11T04:48:28.733 INFO:teuthology.orchestra.run.gibba004.stderr:instructed osd(s) 4 to deep-scrub 2021-07-11T04:48:28.735 INFO:tasks.ceph:Scrubbing osd.5 2021-07-11T04:48:28.736 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2021-07-11T04:48:28.788 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed8842700 1 Processor -- start 2021-07-11T04:48:28.789 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed8842700 1 -- start start 2021-07-11T04:48:28.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed8842700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9ed40a76b0 0x7f9ed4066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed8842700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f9ed40b0060 0x7f9ed405fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.791 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed8842700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed40a7a90 con 0x7f9ed40b1970 2021-07-11T04:48:28.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed8842700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed4147620 con 0x7f9ed40a76b0 2021-07-11T04:48:28.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed8842700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed41478c0 con 0x7f9ed40b0060 2021-07-11T04:48:28.792 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9ed40a76b0 0x7f9ed4066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f9ed40a76b0 0x7f9ed4066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52440/0 (socket says 172.21.2.104:52440) 2021-07-11T04:48:28.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed259c700 1 -- 172.21.2.104:0/2927240517 learned_addr learned my addr 172.21.2.104:0/2927240517 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:28.793 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed259c700 1 -- 172.21.2.104:0/2927240517 >> v2:172.21.2.105:3300/0 conn(0x7f9ed40b0060 msgr2=0x7f9ed405fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed259c700 1 --2- 172.21.2.104:0/2927240517 >> v2:172.21.2.105:3300/0 conn(0x7f9ed40b0060 0x7f9ed405fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed259c700 1 -- 172.21.2.104:0/2927240517 >> v1:172.21.2.105:6789/0 conn(0x7f9ed40b1970 legacy=0x7f9ed405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.784+0000 7f9ed259c700 1 -- 172.21.2.104:0/2927240517 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed4147b60 con 0x7f9ed40a76b0 2021-07-11T04:48:28.794 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed259c700 1 --2- 172.21.2.104:0/2927240517 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ed40a76b0 0x7f9ed4066a70 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7f9ed406a0e0 tx=0x7f9ec4006eb0).ready entity=mon.0 client_cookie=4da12e166b2e2c3b server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed159a700 1 -- 172.21.2.104:0/2927240517 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ec400d5c0 con 0x7f9ed40a76b0 2021-07-11T04:48:28.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed159a700 1 -- 172.21.2.104:0/2927240517 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec4000e90 con 0x7f9ed40a76b0 2021-07-11T04:48:28.795 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed159a700 1 -- 172.21.2.104:0/2927240517 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ec400d910 con 0x7f9ed40a76b0 2021-07-11T04:48:28.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 -- 172.21.2.104:0/2927240517 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ed40a76b0 msgr2=0x7f9ed4066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 --2- 172.21.2.104:0/2927240517 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ed40a76b0 0x7f9ed4066a70 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7f9ed406a0e0 tx=0x7f9ec4006eb0).stop 2021-07-11T04:48:28.796 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 -- 172.21.2.104:0/2927240517 shutdown_connections 2021-07-11T04:48:28.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 --2- 172.21.2.104:0/2927240517 >> v2:172.21.2.105:3300/0 conn(0x7f9ed40b0060 0x7f9ed405fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 --2- 172.21.2.104:0/2927240517 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ed40a76b0 0x7f9ed4066a70 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.797 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 -- 172.21.2.104:0/2927240517 >> 172.21.2.104:0/2927240517 conn(0x7f9ed40b53e0 msgr2=0x7f9ed40b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 -- 172.21.2.104:0/2927240517 shutdown_connections 2021-07-11T04:48:28.798 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 -- 172.21.2.104:0/2927240517 wait complete. 2021-07-11T04:48:28.800 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 Processor -- start 2021-07-11T04:48:28.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 -- start start 2021-07-11T04:48:28.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ed40a76b0 0x7f9ed40ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.801 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ed40b0060 0x7f9ed4159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ed40b1970 0x7f9ed415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed415a5f0 con 0x7f9ed40a76b0 2021-07-11T04:48:28.802 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed41567e0 con 0x7f9ed40b1970 2021-07-11T04:48:28.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed4156a80 con 0x7f9ed40b0060 2021-07-11T04:48:28.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ed40a76b0 0x7f9ed40ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.803 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed259c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ed40a76b0 0x7f9ed40ad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52448/0 (socket says 172.21.2.104:52448) 2021-07-11T04:48:28.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed259c700 1 -- 172.21.2.104:0/4081197075 learned_addr learned my addr 172.21.2.104:0/4081197075 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:28.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed259c700 1 -- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ed40b0060 msgr2=0x7f9ed4159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed259c700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f9ed40b0060 0x7f9ed4159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.804 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed259c700 1 -- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ed40b1970 msgr2=0x7f9ed415a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed259c700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f9ed40b1970 0x7f9ed415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed259c700 1 -- 172.21.2.104:0/4081197075 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec400b1e0 con 0x7f9ed40a76b0 2021-07-11T04:48:28.805 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed259c700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ed40a76b0 0x7f9ed40ad060 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7f9ed406a0e0 tx=0x7f9ec4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ecb7fe700 1 -- 172.21.2.104:0/4081197075 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ec4006be0 con 0x7f9ed40a76b0 2021-07-11T04:48:28.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ecb7fe700 1 -- 172.21.2.104:0/4081197075 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec4011ce0 con 0x7f9ed40a76b0 2021-07-11T04:48:28.806 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ecb7fe700 1 -- 172.21.2.104:0/4081197075 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9ec401d640 con 0x7f9ed40a76b0 2021-07-11T04:48:28.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 -- 172.21.2.104:0/4081197075 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed4156d20 con 0x7f9ed40a76b0 2021-07-11T04:48:28.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.788+0000 7f9ed8842700 1 -- 172.21.2.104:0/4081197075 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed4157460 con 0x7f9ed40a76b0 2021-07-11T04:48:28.807 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.792+0000 7f9ed8842700 1 -- 172.21.2.104:0/4081197075 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9ed41571c0 con 0x7f9ed40a76b0 2021-07-11T04:48:28.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.792+0000 7f9ecb7fe700 1 -- 172.21.2.104:0/4081197075 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9ec400f070 con 0x7f9ed40a76b0 2021-07-11T04:48:28.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.792+0000 7f9ecb7fe700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9eb8053300 0x7f9eb80557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.808 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.792+0000 7f9ecb7fe700 1 -- 172.21.2.104:0/4081197075 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9ec4045fa0 con 0x7f9ed40a76b0 2021-07-11T04:48:28.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.792+0000 7f9ecb7fe700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] conn(0x7f9eb8059a80 0x7f9eb805bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.792+0000 7f9ecb7fe700 1 -- 172.21.2.104:0/4081197075 --> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9eb805c460 con 0x7f9eb8059a80 2021-07-11T04:48:28.809 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.792+0000 7f9ecb7fe700 1 -- 172.21.2.104:0/4081197075 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=164) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ec4002f50 con 0x7f9ed40a76b0 2021-07-11T04:48:28.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.792+0000 7f9ed1d9b700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9eb8053300 0x7f9eb80557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.792+0000 7f9ed2d9d700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] conn(0x7f9eb8059a80 0x7f9eb805bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.792+0000 7f9ed1d9b700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9eb8053300 0x7f9eb80557c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f9ebc002f10 tx=0x7f9ebc006930).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.810 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.792+0000 7f9ed2d9d700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] conn(0x7f9eb8059a80 0x7f9eb805bea0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.811 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.796+0000 7f9ecb7fe700 1 -- 172.21.2.104:0/4081197075 <== osd.5 v2:172.21.2.105:6812/15646 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f9eb805c460 con 0x7f9eb8059a80 2021-07-11T04:48:28.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed8842700 1 -- 172.21.2.104:0/4081197075 --> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9ed40adb00 con 0x7f9eb8059a80 2021-07-11T04:48:28.812 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ecb7fe700 1 -- 172.21.2.104:0/4081197075 <== osd.5 v2:172.21.2.105:6812/15646 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9ed40adb00 con 0x7f9eb8059a80 2021-07-11T04:48:28.813 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:48:28.813 INFO:teuthology.orchestra.run.gibba004.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-07-11T04:48:28.813 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:48:28.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed8842700 1 -- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] conn(0x7f9eb8059a80 msgr2=0x7f9eb805bea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.815 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed8842700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:6812/15646,v1:172.21.2.105:6815/15646] conn(0x7f9eb8059a80 0x7f9eb805bea0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed8842700 1 -- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9eb8053300 msgr2=0x7f9eb80557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed8842700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9eb8053300 0x7f9eb80557c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f9ebc002f10 tx=0x7f9ebc006930).stop 2021-07-11T04:48:28.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed8842700 1 -- 172.21.2.104:0/4081197075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ed40a76b0 msgr2=0x7f9ed40ad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.816 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed8842700 1 --2- 172.21.2.104:0/4081197075 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f9ed40a76b0 0x7f9ed40ad060 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7f9ed406a0e0 tx=0x7f9ec4015000).stop 2021-07-11T04:48:28.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed2d9d700 1 -- 172.21.2.104:0/4081197075 reap_dead start 2021-07-11T04:48:28.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed8842700 1 -- 172.21.2.104:0/4081197075 shutdown_connections 2021-07-11T04:48:28.817 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed8842700 1 -- 172.21.2.104:0/4081197075 >> 172.21.2.104:0/4081197075 conn(0x7f9ed40b53e0 msgr2=0x7f9ed4066820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed8842700 1 -- 172.21.2.104:0/4081197075 shutdown_connections 2021-07-11T04:48:28.818 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.804+0000 7f9ed8842700 1 -- 172.21.2.104:0/4081197075 wait complete. 2021-07-11T04:48:28.819 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2021-07-11T04:48:28.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.864+0000 7f677bd1a700 1 Processor -- start 2021-07-11T04:48:28.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.864+0000 7f677bd1a700 1 -- start start 2021-07-11T04:48:28.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f67740973d0 0x7f677405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f677413f560 0x7f677413f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67740b2a40 con 0x7f67740b2660 2021-07-11T04:48:28.872 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6774097b80 con 0x7f677413f560 2021-07-11T04:48:28.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6774097e20 con 0x7f67740973d0 2021-07-11T04:48:28.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f677413f560 0x7f677413f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.873 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f6779ab6700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f67740b2660 0x7f6774096cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49224/0 (socket says 172.21.2.104:49224) 2021-07-11T04:48:28.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f6779ab6700 1 -- 172.21.2.104:0/2071396547 learned_addr learned my addr 172.21.2.104:0/2071396547 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:48:28.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 -- 172.21.2.104:0/2071396547 >> v2:172.21.2.105:3300/0 conn(0x7f67740973d0 msgr2=0x7f677405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:28.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 --2- 172.21.2.104:0/2071396547 >> v2:172.21.2.105:3300/0 conn(0x7f67740973d0 0x7f677405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.874 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 -- 172.21.2.104:0/2071396547 >> v1:172.21.2.104:6789/0 conn(0x7f67740b2660 legacy=0x7f6774096cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 -- 172.21.2.104:0/2071396547 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740980c0 con 0x7f677413f560 2021-07-11T04:48:28.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 --2- 172.21.2.104:0/2071396547 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f677413f560 0x7f677413f9b0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 rx=0x7f676c0032f0 tx=0x7f676c00b470).ready entity=mon.0 client_cookie=d5a5990dff33d3af server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.875 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f6778ab4700 1 -- 172.21.2.104:0/2071396547 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f676c00ddf0 con 0x7f677413f560 2021-07-11T04:48:28.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f6778ab4700 1 -- 172.21.2.104:0/2071396547 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c010470 con 0x7f677413f560 2021-07-11T04:48:28.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f6778ab4700 1 -- 172.21.2.104:0/2071396547 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f676c0107d0 con 0x7f677413f560 2021-07-11T04:48:28.876 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- 172.21.2.104:0/2071396547 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f677413f560 msgr2=0x7f677413f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 --2- 172.21.2.104:0/2071396547 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f677413f560 0x7f677413f9b0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 rx=0x7f676c0032f0 tx=0x7f676c00b470).stop 2021-07-11T04:48:28.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- 172.21.2.104:0/2071396547 shutdown_connections 2021-07-11T04:48:28.877 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 --2- 172.21.2.104:0/2071396547 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f677413f560 0x7f677413f9b0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 --2- 172.21.2.104:0/2071396547 >> v2:172.21.2.105:3300/0 conn(0x7f67740973d0 0x7f677405f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- 172.21.2.104:0/2071396547 >> 172.21.2.104:0/2071396547 conn(0x7f6774062f10 msgr2=0x7f67740632f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- 172.21.2.104:0/2071396547 shutdown_connections 2021-07-11T04:48:28.878 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- 172.21.2.104:0/2071396547 wait complete. 2021-07-11T04:48:28.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 Processor -- start 2021-07-11T04:48:28.881 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- start start 2021-07-11T04:48:28.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67740973d0 0x7f67740950a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67740b2660 0x7f67740af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f677413f560 0x7f677414e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.882 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677414f260 con 0x7f677413f560 2021-07-11T04:48:28.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67740abdb0 con 0x7f67740973d0 2021-07-11T04:48:28.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677414e730 con 0x7f67740b2660 2021-07-11T04:48:28.883 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f677413f560 0x7f677414e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f677413f560 0x7f677414e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52466/0 (socket says 172.21.2.104:52466) 2021-07-11T04:48:28.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 -- 172.21.2.104:0/3722794051 learned_addr learned my addr 172.21.2.104:0/3722794051 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:28.884 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 -- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67740b2660 msgr2=0x7f67740af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67740b2660 0x7f67740af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 -- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67740973d0 msgr2=0x7f67740950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.885 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67740973d0 0x7f67740950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 -- 172.21.2.104:0/3722794051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c00b710 con 0x7f677413f560 2021-07-11T04:48:28.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677a2b7700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f677413f560 0x7f677414e240 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7f676c0032f0 tx=0x7f6774096a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f6766ffd700 1 -- 172.21.2.104:0/3722794051 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f676c010ea0 con 0x7f677413f560 2021-07-11T04:48:28.886 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f6766ffd700 1 -- 172.21.2.104:0/3722794051 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c018610 con 0x7f677413f560 2021-07-11T04:48:28.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f6766ffd700 1 -- 172.21.2.104:0/3722794051 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f676c018970 con 0x7f677413f560 2021-07-11T04:48:28.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- 172.21.2.104:0/3722794051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67740ab7d0 con 0x7f677413f560 2021-07-11T04:48:28.887 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.868+0000 7f677bd1a700 1 -- 172.21.2.104:0/3722794051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67740abfe0 con 0x7f677413f560 2021-07-11T04:48:28.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.872+0000 7f677bd1a700 1 -- 172.21.2.104:0/3722794051 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67740ab3d0 con 0x7f677413f560 2021-07-11T04:48:28.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.872+0000 7f6766ffd700 1 -- 172.21.2.104:0/3722794051 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f676c021070 con 0x7f677413f560 2021-07-11T04:48:28.888 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.872+0000 7f6766ffd700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67540531f0 0x7f67540556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:28.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.872+0000 7f6766ffd700 1 -- 172.21.2.104:0/3722794051 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f676c06ead0 con 0x7f677413f560 2021-07-11T04:48:28.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.872+0000 7f6779ab6700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67540531f0 0x7f67540556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:28.889 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.872+0000 7f6779ab6700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67540531f0 0x7f67540556b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f67740b3350 tx=0x7f6768007890).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:28.890 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.872+0000 7f6766ffd700 1 -- 172.21.2.104:0/3722794051 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f67740ab3d0 con 0x7f677413f560 2021-07-11T04:48:28.960 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.956+0000 7f677bd1a700 1 -- 172.21.2.104:0/3722794051 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f67740ab3d0 con 0x7f67540531f0 2021-07-11T04:48:28.961 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.956+0000 7f6766ffd700 1 -- 172.21.2.104:0/3722794051 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f67740ab3d0 con 0x7f67540531f0 2021-07-11T04:48:28.963 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 -- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67540531f0 msgr2=0x7f67540556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67540531f0 0x7f67540556b0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f67740b3350 tx=0x7f6768007890).stop 2021-07-11T04:48:28.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 -- 172.21.2.104:0/3722794051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f677413f560 msgr2=0x7f677414e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:28.964 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f677413f560 0x7f677414e240 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7f676c0032f0 tx=0x7f6774096a20).stop 2021-07-11T04:48:28.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 -- 172.21.2.104:0/3722794051 shutdown_connections 2021-07-11T04:48:28.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f67540531f0 0x7f67540556b0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.965 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f677413f560 0x7f677414e240 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:28.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f67740b2660 0x7f67740af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 --2- 172.21.2.104:0/3722794051 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f67740973d0 0x7f67740950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:28.966 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 -- 172.21.2.104:0/3722794051 >> 172.21.2.104:0/3722794051 conn(0x7f6774062f10 msgr2=0x7f677409ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:28.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 -- 172.21.2.104:0/3722794051 shutdown_connections 2021-07-11T04:48:28.967 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:28.960+0000 7f677bd1a700 1 -- 172.21.2.104:0/3722794051 wait complete. 2021-07-11T04:48:28.967 INFO:teuthology.orchestra.run.gibba004.stderr:instructed osd(s) 5 to deep-scrub 2021-07-11T04:48:28.969 INFO:tasks.ceph:Scrubbing osd.6 2021-07-11T04:48:28.970 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2021-07-11T04:48:29.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb73c0ab700 1 Processor -- start 2021-07-11T04:48:29.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb73c0ab700 1 -- start start 2021-07-11T04:48:29.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb73c0ab700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fb734067180 0x7fb73405f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb73c0ab700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7340a7a90 con 0x7fb7340b1970 2021-07-11T04:48:29.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb73c0ab700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb734147620 con 0x7fb7340a76b0 2021-07-11T04:48:29.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb73c0ab700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7341478c0 con 0x7fb734067180 2021-07-11T04:48:29.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb73a648700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7fb7340b1970 0x7fb73405fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49238/0 (socket says 172.21.2.104:49238) 2021-07-11T04:48:29.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb73a648700 1 -- 172.21.2.104:0/3523935948 learned_addr learned my addr 172.21.2.104:0/3523935948 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:48:29.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129513459 0 0) 0x7fb7340a7a90 con 0x7fb7340b1970 2021-07-11T04:48:29.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb71c003260 con 0x7fb7340b1970 2021-07-11T04:48:29.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb739646700 1 --2- 172.21.2.104:0/3523935948 >> v2:172.21.2.105:3300/0 conn(0x7fb734067180 0x7fb73405f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4260276119 0 0) 0x7fb71c003260 con 0x7fb7340b1970 2021-07-11T04:48:29.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 >> v2:172.21.2.105:3300/0 conn(0x7fb734067180 msgr2=0x7fb73405f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:29.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 --2- 172.21.2.104:0/3523935948 >> v2:172.21.2.105:3300/0 conn(0x7fb734067180 0x7fb73405f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.035 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 >> v1:172.21.2.105:6789/0 conn(0x7fb7340a76b0 legacy=0x7fb734066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:29.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb734147b60 con 0x7fb7340b1970 2021-07-11T04:48:29.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7fb72c0019e0 con 0x7fb7340b1970 2021-07-11T04:48:29.036 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 >> v1:172.21.2.104:6789/0 conn(0x7fb7340b1970 legacy=0x7fb73405fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb71c003820 0x7fb71c003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb71c004500 0x7fb71c004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.037 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 0x7fb71c00b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7341478c0 con 0x7fb71c004f90 2021-07-11T04:48:29.038 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c00be80 con 0x7fb71c003820 2021-07-11T04:48:29.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c00c040 con 0x7fb71c004500 2021-07-11T04:48:29.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb73a648700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 0x7fb71c00b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.039 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb739e47700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb71c003820 0x7fb71c003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb739646700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb71c004500 0x7fb71c004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.020+0000 7fb73a648700 1 -- 172.21.2.104:0/3523935948 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb71c004500 msgr2=0x7fb71c004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:29.040 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73a648700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb71c004500 0x7fb71c004980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73a648700 1 -- 172.21.2.104:0/3523935948 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb71c003820 msgr2=0x7fb71c003c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:29.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73a648700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb71c003820 0x7fb71c003c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.041 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73a648700 1 -- 172.21.2.104:0/3523935948 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c003260 con 0x7fb71c004f90 2021-07-11T04:48:29.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739646700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb71c004500 0x7fb71c004980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:48:29.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73a648700 1 -- 172.21.2.104:0/3523935948 reap_dead start 2021-07-11T04:48:29.042 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73a648700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 0x7fb71c00b750 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7fb73406a180 tx=0x7fb7340a8c80).ready entity=mon.0 client_cookie=e9b4cd5b135c6d9 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb72c00cb70 con 0x7fb71c004f90 2021-07-11T04:48:29.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c00ccd0 con 0x7fb71c004f90 2021-07-11T04:48:29.043 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb738e45700 1 -- 172.21.2.104:0/3523935948 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb72c005b80 con 0x7fb71c004f90 2021-07-11T04:48:29.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- 172.21.2.104:0/3523935948 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 msgr2=0x7fb71c00b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 0x7fb71c00b750 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7fb73406a180 tx=0x7fb7340a8c80).stop 2021-07-11T04:48:29.044 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- 172.21.2.104:0/3523935948 shutdown_connections 2021-07-11T04:48:29.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 --2- 172.21.2.104:0/3523935948 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 0x7fb71c00b750 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- 172.21.2.104:0/3523935948 >> 172.21.2.104:0/3523935948 conn(0x7fb7340b53e0 msgr2=0x7fb7340b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:29.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- 172.21.2.104:0/3523935948 shutdown_connections 2021-07-11T04:48:29.045 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- 172.21.2.104:0/3523935948 wait complete. 2021-07-11T04:48:29.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 Processor -- start 2021-07-11T04:48:29.046 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- start start 2021-07-11T04:48:29.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 0x7fb7340ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7340af520 0x7fb7340ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7340aa280 0x7fb7340ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.047 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340aad10 con 0x7fb71c004f90 2021-07-11T04:48:29.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340ab5d0 con 0x7fb7340aa280 2021-07-11T04:48:29.048 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340ab870 con 0x7fb7340af520 2021-07-11T04:48:29.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739e47700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 0x7fb7340ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739e47700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 0x7fb7340ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52482/0 (socket says 172.21.2.104:52482) 2021-07-11T04:48:29.049 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739e47700 1 -- 172.21.2.104:0/774766087 learned_addr learned my addr 172.21.2.104:0/774766087 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:29.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73a648700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7340aa280 0x7fb7340ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739646700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7340af520 0x7fb7340ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.050 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739e47700 1 -- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7340af520 msgr2=0x7fb7340ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739e47700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fb7340af520 0x7fb7340ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739e47700 1 -- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7340aa280 msgr2=0x7fb7340ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.051 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739e47700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fb7340aa280 0x7fb7340ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739e47700 1 -- 172.21.2.104:0/774766087 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7340abb50 con 0x7fb71c004f90 2021-07-11T04:48:29.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739e47700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 0x7fb7340ac860 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7fb73000b000 tx=0x7fb730009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.052 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb726ffd700 1 -- 172.21.2.104:0/774766087 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb73000eea0 con 0x7fb71c004f90 2021-07-11T04:48:29.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb726ffd700 1 -- 172.21.2.104:0/774766087 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb730000e90 con 0x7fb71c004f90 2021-07-11T04:48:29.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb726ffd700 1 -- 172.21.2.104:0/774766087 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fb730015820 con 0x7fb71c004f90 2021-07-11T04:48:29.053 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- 172.21.2.104:0/774766087 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7340aa790 con 0x7fb71c004f90 2021-07-11T04:48:29.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- 172.21.2.104:0/774766087 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb734144690 con 0x7fb71c004f90 2021-07-11T04:48:29.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb73c0ab700 1 -- 172.21.2.104:0/774766087 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7340acf70 con 0x7fb71c004f90 2021-07-11T04:48:29.054 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb726ffd700 1 -- 172.21.2.104:0/774766087 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fb730015980 con 0x7fb71c004f90 2021-07-11T04:48:29.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb726ffd700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb7140531f0 0x7fb7140556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb726ffd700 1 -- 172.21.2.104:0/774766087 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb73006aed0 con 0x7fb71c004f90 2021-07-11T04:48:29.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb726ffd700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] conn(0x7fb714059970 0x7fb71405bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.055 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb726ffd700 1 -- 172.21.2.104:0/774766087 --> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb71405c3f0 con 0x7fb714059970 2021-07-11T04:48:29.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb726ffd700 1 -- 172.21.2.104:0/774766087 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=164) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb73006d560 con 0x7fb71c004f90 2021-07-11T04:48:29.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.024+0000 7fb739646700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb7140531f0 0x7fb7140556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.056 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.028+0000 7fb73a648700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] conn(0x7fb714059970 0x7fb71405bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.028+0000 7fb739646700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb7140531f0 0x7fb7140556b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7fb728009000 tx=0x7fb728008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.028+0000 7fb73a648700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] conn(0x7fb714059970 0x7fb71405bd90 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.057 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.028+0000 7fb726ffd700 1 -- 172.21.2.104:0/774766087 <== osd.6 v2:172.21.2.105:6810/15647 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7fb71405c3f0 con 0x7fb714059970 2021-07-11T04:48:29.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73c0ab700 1 -- 172.21.2.104:0/774766087 --> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb734144fd0 con 0x7fb714059970 2021-07-11T04:48:29.059 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb726ffd700 1 -- 172.21.2.104:0/774766087 <== osd.6 v2:172.21.2.105:6810/15647 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb734144fd0 con 0x7fb714059970 2021-07-11T04:48:29.060 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:48:29.060 INFO:teuthology.orchestra.run.gibba004.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-07-11T04:48:29.060 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:48:29.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73c0ab700 1 -- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] conn(0x7fb714059970 msgr2=0x7fb71405bd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73c0ab700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:6810/15647,v1:172.21.2.105:6813/15647] conn(0x7fb714059970 0x7fb71405bd90 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.062 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73c0ab700 1 -- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb7140531f0 msgr2=0x7fb7140556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73c0ab700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fb7140531f0 0x7fb7140556b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7fb728009000 tx=0x7fb728008040).stop 2021-07-11T04:48:29.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73c0ab700 1 -- 172.21.2.104:0/774766087 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 msgr2=0x7fb7340ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.063 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73c0ab700 1 --2- 172.21.2.104:0/774766087 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fb71c004f90 0x7fb7340ac860 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7fb73000b000 tx=0x7fb730009040).stop 2021-07-11T04:48:29.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73a648700 1 -- 172.21.2.104:0/774766087 reap_dead start 2021-07-11T04:48:29.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73c0ab700 1 -- 172.21.2.104:0/774766087 shutdown_connections 2021-07-11T04:48:29.064 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73c0ab700 1 -- 172.21.2.104:0/774766087 >> 172.21.2.104:0/774766087 conn(0x7fb7340b53e0 msgr2=0x7fb73414a820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:29.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73c0ab700 1 -- 172.21.2.104:0/774766087 shutdown_connections 2021-07-11T04:48:29.065 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.036+0000 7fb73c0ab700 1 -- 172.21.2.104:0/774766087 wait complete. 2021-07-11T04:48:29.066 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2021-07-11T04:48:29.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.120+0000 7f999feca700 1 Processor -- start 2021-07-11T04:48:29.125 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.120+0000 7f999feca700 1 -- start start 2021-07-11T04:48:29.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.120+0000 7f999feca700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99980ad150 0x7f99980b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.127 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.120+0000 7f999feca700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f99980a9da0 0x7f9998140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.120+0000 7f999feca700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99980ac370 con 0x7f99980aa860 2021-07-11T04:48:29.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.120+0000 7f999feca700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99980abb50 con 0x7f99980a9da0 2021-07-11T04:48:29.128 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.120+0000 7f999feca700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99980aac40 con 0x7f99980ad150 2021-07-11T04:48:29.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.120+0000 7f999e467700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f99980a9da0 0x7f9998140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.120+0000 7f999e467700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f99980a9da0 0x7f9998140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52496/0 (socket says 172.21.2.104:52496) 2021-07-11T04:48:29.129 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.120+0000 7f999e467700 1 -- 172.21.2.104:0/1184473430 learned_addr learned my addr 172.21.2.104:0/1184473430 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:29.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 -- 172.21.2.104:0/1184473430 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99980ad150 msgr2=0x7f99980b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:29.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 --2- 172.21.2.104:0/1184473430 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99980ad150 0x7f99980b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:29.130 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 -- 172.21.2.104:0/1184473430 >> v1:172.21.2.105:6789/0 conn(0x7f99980aa860 legacy=0x7f99980aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:29.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 -- 172.21.2.104:0/1184473430 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99980b16b0 con 0x7f99980a9da0 2021-07-11T04:48:29.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 --2- 172.21.2.104:0/1184473430 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f99980a9da0 0x7f9998140340 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 rx=0x7f99980ac920 tx=0x7f999000b530).ready entity=mon.0 client_cookie=7473be35e3cc5d4e server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.131 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999cc64700 1 -- 172.21.2.104:0/1184473430 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f999000be70 con 0x7f99980a9da0 2021-07-11T04:48:29.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999cc64700 1 -- 172.21.2.104:0/1184473430 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9990010470 con 0x7f99980a9da0 2021-07-11T04:48:29.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999cc64700 1 -- 172.21.2.104:0/1184473430 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9990010810 con 0x7f99980a9da0 2021-07-11T04:48:29.132 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- 172.21.2.104:0/1184473430 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f99980a9da0 msgr2=0x7f9998140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 --2- 172.21.2.104:0/1184473430 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f99980a9da0 0x7f9998140340 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 rx=0x7f99980ac920 tx=0x7f999000b530).stop 2021-07-11T04:48:29.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- 172.21.2.104:0/1184473430 shutdown_connections 2021-07-11T04:48:29.133 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 --2- 172.21.2.104:0/1184473430 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f99980a9da0 0x7f9998140340 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 --2- 172.21.2.104:0/1184473430 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99980ad150 0x7f99980b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:29.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- 172.21.2.104:0/1184473430 >> 172.21.2.104:0/1184473430 conn(0x7f99980b64d0 msgr2=0x7f99980b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:29.134 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- 172.21.2.104:0/1184473430 shutdown_connections 2021-07-11T04:48:29.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- 172.21.2.104:0/1184473430 wait complete. 2021-07-11T04:48:29.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 Processor -- start 2021-07-11T04:48:29.135 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- start start 2021-07-11T04:48:29.137 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99980a9da0 0x7f9998064700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f99980aa860 0x7f9998145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.138 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f99980ad150 0x7f9998065e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99980a9880 con 0x7f99980ad150 2021-07-11T04:48:29.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99980ab6a0 con 0x7f99980aa860 2021-07-11T04:48:29.139 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99980ad610 con 0x7f99980a9da0 2021-07-11T04:48:29.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f99980ad150 0x7f9998065e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f99980ad150 0x7f9998065e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52498/0 (socket says 172.21.2.104:52498) 2021-07-11T04:48:29.140 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 -- 172.21.2.104:0/50856554 learned_addr learned my addr 172.21.2.104:0/50856554 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:29.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 -- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99980a9da0 msgr2=0x7f9998064700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99980a9da0 0x7f9998064700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:29.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 -- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f99980aa860 msgr2=0x7f9998145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.141 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f99980aa860 0x7f9998145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:29.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 -- 172.21.2.104:0/50856554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999000b7d0 con 0x7f99980ad150 2021-07-11T04:48:29.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999e467700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f99980ad150 0x7f9998065e70 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 rx=0x7f9998065a30 tx=0x7f99980ac290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.142 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f998affd700 1 -- 172.21.2.104:0/50856554 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f999001d070 con 0x7f99980ad150 2021-07-11T04:48:29.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f998affd700 1 -- 172.21.2.104:0/50856554 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9990022600 con 0x7f99980ad150 2021-07-11T04:48:29.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- 172.21.2.104:0/50856554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9998063ae0 con 0x7f99980ad150 2021-07-11T04:48:29.143 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f998affd700 1 -- 172.21.2.104:0/50856554 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f9990022960 con 0x7f99980ad150 2021-07-11T04:48:29.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- 172.21.2.104:0/50856554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9998064260 con 0x7f99980ad150 2021-07-11T04:48:29.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f999feca700 1 -- 172.21.2.104:0/50856554 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9998064420 con 0x7f99980ad150 2021-07-11T04:48:29.144 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f998affd700 1 -- 172.21.2.104:0/50856554 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f9990029070 con 0x7f99980ad150 2021-07-11T04:48:29.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f998affd700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9978053380 0x7f9978055840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.124+0000 7f998affd700 1 -- 172.21.2.104:0/50856554 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f999006c9f0 con 0x7f99980ad150 2021-07-11T04:48:29.145 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.128+0000 7f999dc66700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9978053380 0x7f9978055840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.128+0000 7f999dc66700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9978053380 0x7f9978055840 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f9994000f70 tx=0x7f99940077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.146 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.128+0000 7f998affd700 1 -- 172.21.2.104:0/50856554 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f9990043fb0 con 0x7f99980ad150 2021-07-11T04:48:29.214 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.208+0000 7f999feca700 1 -- 172.21.2.104:0/50856554 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f99980655e0 con 0x7f9978053380 2021-07-11T04:48:29.215 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f998affd700 1 -- 172.21.2.104:0/50856554 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f99940133c0 con 0x7f9978053380 2021-07-11T04:48:29.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 -- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9978053380 msgr2=0x7f9978055840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9978053380 0x7f9978055840 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f9994000f70 tx=0x7f99940077f0).stop 2021-07-11T04:48:29.217 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 -- 172.21.2.104:0/50856554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f99980ad150 msgr2=0x7f9998065e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f99980ad150 0x7f9998065e70 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 rx=0x7f9998065a30 tx=0x7f99980ac290).stop 2021-07-11T04:48:29.218 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 -- 172.21.2.104:0/50856554 shutdown_connections 2021-07-11T04:48:29.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f9978053380 0x7f9978055840 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f99980ad150 0x7f9998065e70 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f99980aa860 0x7f9998145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:29.219 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 --2- 172.21.2.104:0/50856554 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f99980a9da0 0x7f9998064700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:29.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 -- 172.21.2.104:0/50856554 >> 172.21.2.104:0/50856554 conn(0x7f99980b64d0 msgr2=0x7f9998066840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:29.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 -- 172.21.2.104:0/50856554 shutdown_connections 2021-07-11T04:48:29.220 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.212+0000 7f999feca700 1 -- 172.21.2.104:0/50856554 wait complete. 2021-07-11T04:48:29.221 INFO:teuthology.orchestra.run.gibba004.stderr:instructed osd(s) 6 to deep-scrub 2021-07-11T04:48:29.223 INFO:tasks.ceph:Scrubbing osd.7 2021-07-11T04:48:29.223 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2021-07-11T04:48:29.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb8505700 1 Processor -- start 2021-07-11T04:48:29.277 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb8505700 1 -- start start 2021-07-11T04:48:29.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb8505700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f8fb00b1170 0x7f8fb0141430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.278 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb8505700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb00b31d0 con 0x7f8fb0141920 2021-07-11T04:48:29.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb8505700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb00b7bf0 con 0x7f8fb00b7810 2021-07-11T04:48:29.279 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb8505700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb00675c0 con 0x7f8fb00b1170 2021-07-11T04:48:29.283 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb6aa2700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f8fb0141920 0x7f8fb014bb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49276/0 (socket says 172.21.2.104:49276) 2021-07-11T04:48:29.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb6aa2700 1 -- 172.21.2.104:0/3950694216 learned_addr learned my addr 172.21.2.104:0/3950694216 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:48:29.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb5aa0700 1 --2- 172.21.2.104:0/3950694216 >> v2:172.21.2.104:3300/0 conn(0x7f8fb00b1170 0x7f8fb0141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.284 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb5aa0700 1 -- 172.21.2.104:0/3950694216 >> v1:172.21.2.105:6789/0 conn(0x7f8fb00b7810 legacy=0x7f8fb01445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:29.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb5aa0700 1 -- 172.21.2.104:0/3950694216 >> v1:172.21.2.104:6789/0 conn(0x7f8fb0141920 legacy=0x7f8fb014bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb5aa0700 1 -- 172.21.2.104:0/3950694216 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb00aae70 con 0x7f8fb00b1170 2021-07-11T04:48:29.285 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.272+0000 7f8fb529f700 1 -- 172.21.2.104:0/3950694216 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364142683 0 0) 0x7f8fb00b31d0 con 0x7f8fb0141920 2021-07-11T04:48:29.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb5aa0700 1 --2- 172.21.2.104:0/3950694216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8fb00b1170 0x7f8fb0141430 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 rx=0x7f8fb0060740 tx=0x7f8fa4006df0).ready entity=mon.0 client_cookie=7ddb64aba5f1ac1 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb529f700 1 -- 172.21.2.104:0/3950694216 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa400d5c0 con 0x7f8fb00b1170 2021-07-11T04:48:29.286 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb529f700 1 -- 172.21.2.104:0/3950694216 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa4000e90 con 0x7f8fb00b1170 2021-07-11T04:48:29.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb529f700 1 -- 172.21.2.104:0/3950694216 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa400d910 con 0x7f8fb00b1170 2021-07-11T04:48:29.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- 172.21.2.104:0/3950694216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8fb00b1170 msgr2=0x7f8fb0141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.287 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 --2- 172.21.2.104:0/3950694216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8fb00b1170 0x7f8fb0141430 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 rx=0x7f8fb0060740 tx=0x7f8fa4006df0).stop 2021-07-11T04:48:29.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- 172.21.2.104:0/3950694216 shutdown_connections 2021-07-11T04:48:29.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 --2- 172.21.2.104:0/3950694216 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8fb00b1170 0x7f8fb0141430 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.288 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- 172.21.2.104:0/3950694216 >> 172.21.2.104:0/3950694216 conn(0x7f8fb005f6b0 msgr2=0x7f8fb005fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:29.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- 172.21.2.104:0/3950694216 shutdown_connections 2021-07-11T04:48:29.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- 172.21.2.104:0/3950694216 wait complete. 2021-07-11T04:48:29.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 Processor -- start 2021-07-11T04:48:29.289 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- start start 2021-07-11T04:48:29.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8fb00b1170 0x7f8fb00a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8fb00b7810 0x7f8fb0143510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.290 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8fb0141920 0x7f8fb0156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb0156a70 con 0x7f8fb0141920 2021-07-11T04:48:29.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb00621b0 con 0x7f8fb00b1170 2021-07-11T04:48:29.291 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb0061c70 con 0x7f8fb00b7810 2021-07-11T04:48:29.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb6aa2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8fb0141920 0x7f8fb0156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb6aa2700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8fb0141920 0x7f8fb0156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52516/0 (socket says 172.21.2.104:52516) 2021-07-11T04:48:29.292 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb6aa2700 1 -- 172.21.2.104:0/1948777697 learned_addr learned my addr 172.21.2.104:0/1948777697 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:29.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb6aa2700 1 -- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8fb00b7810 msgr2=0x7f8fb0143510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb6aa2700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8fb00b7810 0x7f8fb0143510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:29.293 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb6aa2700 1 -- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8fb00b1170 msgr2=0x7f8fb00a85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb6aa2700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8fb00b1170 0x7f8fb00a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:29.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb6aa2700 1 -- 172.21.2.104:0/1948777697 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa400b180 con 0x7f8fb0141920 2021-07-11T04:48:29.294 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb6aa2700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8fb0141920 0x7f8fb0156ed0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 rx=0x7f8fa8000f40 tx=0x7f8fa8001380).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fa37fe700 1 -- 172.21.2.104:0/1948777697 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa8005ba0 con 0x7f8fb0141920 2021-07-11T04:48:29.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fa37fe700 1 -- 172.21.2.104:0/1948777697 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8005d00 con 0x7f8fb0141920 2021-07-11T04:48:29.295 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- 172.21.2.104:0/1948777697 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb0061f10 con 0x7f8fb0141920 2021-07-11T04:48:29.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fa37fe700 1 -- 172.21.2.104:0/1948777697 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8fa80072c0 con 0x7f8fb0141920 2021-07-11T04:48:29.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- 172.21.2.104:0/1948777697 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb01440e0 con 0x7f8fb0141920 2021-07-11T04:48:29.296 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fb8505700 1 -- 172.21.2.104:0/1948777697 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8fb0062850 con 0x7f8fb0141920 2021-07-11T04:48:29.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.276+0000 7f8fa37fe700 1 -- 172.21.2.104:0/1948777697 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8fa801a060 con 0x7f8fb0141920 2021-07-11T04:48:29.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.280+0000 7f8fa37fe700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8f900531f0 0x7f8f900556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.297 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.280+0000 7f8fa37fe700 1 -- 172.21.2.104:0/1948777697 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8fa806ad60 con 0x7f8fb0141920 2021-07-11T04:48:29.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.280+0000 7f8fa37fe700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] conn(0x7f8f90059970 0x7f8f9005bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.280+0000 7f8fa37fe700 1 -- 172.21.2.104:0/1948777697 --> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8f9005c3f0 con 0x7f8f90059970 2021-07-11T04:48:29.298 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.280+0000 7f8fa37fe700 1 -- 172.21.2.104:0/1948777697 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_get_version_reply(handle=1 version=164) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fa806d3f0 con 0x7f8fb0141920 2021-07-11T04:48:29.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.280+0000 7f8fb62a1700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8f900531f0 0x7f8f900556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.280+0000 7f8fb5aa0700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] conn(0x7f8f90059970 0x7f8f9005bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.299 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.280+0000 7f8fb62a1700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8f900531f0 0x7f8f900556b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f8fac000f70 tx=0x7f8fac0077f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.300 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.280+0000 7f8fb5aa0700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] conn(0x7f8f90059970 0x7f8f9005bd90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.280+0000 7f8fa37fe700 1 -- 172.21.2.104:0/1948777697 <== osd.7 v2:172.21.2.105:6811/15656 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23441 (crc 0 0 0) 0x7f8f9005c3f0 con 0x7f8f90059970 2021-07-11T04:48:29.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb8505700 1 -- 172.21.2.104:0/1948777697 --> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8fb0157680 con 0x7f8f90059970 2021-07-11T04:48:29.301 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fa37fe700 1 -- 172.21.2.104:0/1948777697 <== osd.7 v2:172.21.2.105:6811/15656 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8fb0157680 con 0x7f8f90059970 2021-07-11T04:48:29.302 INFO:teuthology.orchestra.run.gibba004.stdout:{ 2021-07-11T04:48:29.302 INFO:teuthology.orchestra.run.gibba004.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2021-07-11T04:48:29.303 INFO:teuthology.orchestra.run.gibba004.stdout:} 2021-07-11T04:48:29.304 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb8505700 1 -- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] conn(0x7f8f90059970 msgr2=0x7f8f9005bd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb8505700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:6811/15656,v1:172.21.2.105:6814/15656] conn(0x7f8f90059970 0x7f8f9005bd90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb8505700 1 -- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8f900531f0 msgr2=0x7f8f900556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.305 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb8505700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8f900531f0 0x7f8f900556b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f8fac000f70 tx=0x7f8fac0077f0).stop 2021-07-11T04:48:29.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb8505700 1 -- 172.21.2.104:0/1948777697 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8fb0141920 msgr2=0x7f8fb0156ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.306 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb8505700 1 --2- 172.21.2.104:0/1948777697 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8fb0141920 0x7f8fb0156ed0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 rx=0x7f8fa8000f40 tx=0x7f8fa8001380).stop 2021-07-11T04:48:29.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb6aa2700 1 -- 172.21.2.104:0/1948777697 reap_dead start 2021-07-11T04:48:29.307 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb8505700 1 -- 172.21.2.104:0/1948777697 shutdown_connections 2021-07-11T04:48:29.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb8505700 1 -- 172.21.2.104:0/1948777697 >> 172.21.2.104:0/1948777697 conn(0x7f8fb005f6b0 msgr2=0x7f8fb00b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:29.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb8505700 1 -- 172.21.2.104:0/1948777697 shutdown_connections 2021-07-11T04:48:29.308 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.292+0000 7f8fb8505700 1 -- 172.21.2.104:0/1948777697 wait complete. 2021-07-11T04:48:29.309 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2021-07-11T04:48:29.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.356+0000 7f8c6f019700 1 Processor -- start 2021-07-11T04:48:29.361 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.356+0000 7f8c6f019700 1 -- start start 2021-07-11T04:48:29.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.356+0000 7f8c6f019700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680aa860 0x7f8c680b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.356+0000 7f8c6f019700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8c680ad150 0x7f8c68140340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.362 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.356+0000 7f8c6f019700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c680ac370 con 0x7f8c680a9da0 2021-07-11T04:48:29.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.356+0000 7f8c6f019700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c680abb50 con 0x7f8c680ad150 2021-07-11T04:48:29.363 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.356+0000 7f8c6f019700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680aac40 con 0x7f8c680aa860 2021-07-11T04:48:29.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6d5b6700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8c680ad150 0x7f8c68140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.364 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6cdb5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680aa860 0x7f8c680b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6d5b6700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f8c680ad150 0x7f8c68140340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:60082/0 (socket says 172.21.2.104:60082) 2021-07-11T04:48:29.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c67fff700 1 --1- >> v1:172.21.2.105:6789/0 conn(0x7f8c680a9da0 0x7f8c680aca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:48664/0 (socket says 172.21.2.104:48664) 2021-07-11T04:48:29.365 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6cdb5700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680aa860 0x7f8c680b11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:55686/0 (socket says 172.21.2.104:55686) 2021-07-11T04:48:29.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6d5b6700 1 -- 172.21.2.104:0/2610190116 learned_addr learned my addr 172.21.2.104:0/2610190116 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:29.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6d5b6700 1 -- 172.21.2.104:0/2610190116 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680aa860 msgr2=0x7f8c680b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:29.366 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6d5b6700 1 --2- 172.21.2.104:0/2610190116 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680aa860 0x7f8c680b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6d5b6700 1 -- 172.21.2.104:0/2610190116 >> v1:172.21.2.105:6789/0 conn(0x7f8c680a9da0 legacy=0x7f8c680aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6d5b6700 1 -- 172.21.2.104:0/2610190116 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c680b16b0 con 0x7f8c680ad150 2021-07-11T04:48:29.367 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6d5b6700 1 --2- 172.21.2.104:0/2610190116 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680ad150 0x7f8c68140340 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f8c600032f0 tx=0x7f8c60009290).ready entity=mon.1 client_cookie=2b3d188cc339adc3 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c677fe700 1 -- 172.21.2.104:0/2610190116 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c6000b8a0 con 0x7f8c680ad150 2021-07-11T04:48:29.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c677fe700 1 -- 172.21.2.104:0/2610190116 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6000ba00 con 0x7f8c680ad150 2021-07-11T04:48:29.368 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c677fe700 1 -- 172.21.2.104:0/2610190116 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c6000bb70 con 0x7f8c680ad150 2021-07-11T04:48:29.369 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6f019700 1 -- 172.21.2.104:0/2610190116 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680ad150 msgr2=0x7f8c68140340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.370 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6f019700 1 --2- 172.21.2.104:0/2610190116 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680ad150 0x7f8c68140340 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f8c600032f0 tx=0x7f8c60009290).stop 2021-07-11T04:48:29.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6f019700 1 -- 172.21.2.104:0/2610190116 shutdown_connections 2021-07-11T04:48:29.374 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6f019700 1 --2- 172.21.2.104:0/2610190116 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680ad150 0x7f8c68140340 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.375 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6f019700 1 --2- 172.21.2.104:0/2610190116 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680aa860 0x7f8c680b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6f019700 1 -- 172.21.2.104:0/2610190116 >> 172.21.2.104:0/2610190116 conn(0x7f8c680b64d0 msgr2=0x7f8c680b68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:29.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6f019700 1 -- 172.21.2.104:0/2610190116 shutdown_connections 2021-07-11T04:48:29.376 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6f019700 1 -- 172.21.2.104:0/2610190116 wait complete. 2021-07-11T04:48:29.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6f019700 1 Processor -- start 2021-07-11T04:48:29.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.360+0000 7f8c6f019700 1 -- start start 2021-07-11T04:48:29.377 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6f019700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680a9da0 0x7f8c68064840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6f019700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680aa860 0x7f8c68153ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6f019700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680ad150 0x7f8c68153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.378 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6f019700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680656e0 con 0x7f8c680aa860 2021-07-11T04:48:29.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6f019700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c681544a0 con 0x7f8c680ad150 2021-07-11T04:48:29.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6f019700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c680afe30 con 0x7f8c680a9da0 2021-07-11T04:48:29.379 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c67fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680aa860 0x7f8c68153ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c67fff700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680aa860 0x7f8c68153ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52528/0 (socket says 172.21.2.104:52528) 2021-07-11T04:48:29.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c67fff700 1 -- 172.21.2.104:0/2317694310 learned_addr learned my addr 172.21.2.104:0/2317694310 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:29.380 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6d5b6700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680ad150 0x7f8c68153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6cdb5700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680a9da0 0x7f8c68064840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c67fff700 1 -- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680a9da0 msgr2=0x7f8c68064840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.381 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c67fff700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680a9da0 0x7f8c68064840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c67fff700 1 -- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680ad150 msgr2=0x7f8c68153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c67fff700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680ad150 0x7f8c68153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.382 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c67fff700 1 -- 172.21.2.104:0/2317694310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c60009530 con 0x7f8c680aa860 2021-07-11T04:48:29.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6d5b6700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680ad150 0x7f8c68153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:48:29.383 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c67fff700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680aa860 0x7f8c68153ac0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7f8c681545e0 tx=0x7f8c50007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.384 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c65ffb700 1 -- 172.21.2.104:0/2317694310 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c5000ce20 con 0x7f8c680aa860 2021-07-11T04:48:29.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c65ffb700 1 -- 172.21.2.104:0/2317694310 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c500072e0 con 0x7f8c680aa860 2021-07-11T04:48:29.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c65ffb700 1 -- 172.21.2.104:0/2317694310 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f8c50007450 con 0x7f8c680aa860 2021-07-11T04:48:29.385 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6f019700 1 -- 172.21.2.104:0/2317694310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c680651a0 con 0x7f8c680aa860 2021-07-11T04:48:29.386 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6f019700 1 -- 172.21.2.104:0/2317694310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c680a9320 con 0x7f8c680aa860 2021-07-11T04:48:29.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6f019700 1 -- 172.21.2.104:0/2317694310 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c680affa0 con 0x7f8c680aa860 2021-07-11T04:48:29.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c65ffb700 1 -- 172.21.2.104:0/2317694310 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f8c50010070 con 0x7f8c680aa860 2021-07-11T04:48:29.387 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c65ffb700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053240 0x7f8c4c055700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c65ffb700 1 -- 172.21.2.104:0/2317694310 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8c5006a9f0 con 0x7f8c680aa860 2021-07-11T04:48:29.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.364+0000 7f8c6cdb5700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053240 0x7f8c4c055700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.388 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.368+0000 7f8c6cdb5700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053240 0x7f8c4c055700 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f8c58008000 tx=0x7f8c58008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.389 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.368+0000 7f8c65ffb700 1 -- 172.21.2.104:0/2317694310 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f8c50041f00 con 0x7f8c680aa860 2021-07-11T04:48:29.453 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.448+0000 7f8c6f019700 1 -- 172.21.2.104:0/2317694310 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c68060cb0 con 0x7f8c4c053240 2021-07-11T04:48:29.454 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c65ffb700 1 -- 172.21.2.104:0/2317694310 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8c68060cb0 con 0x7f8c4c053240 2021-07-11T04:48:29.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 -- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053240 msgr2=0x7f8c4c055700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.455 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053240 0x7f8c4c055700 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f8c58008000 tx=0x7f8c58008040).stop 2021-07-11T04:48:29.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 -- 172.21.2.104:0/2317694310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680aa860 msgr2=0x7f8c68153ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680aa860 0x7f8c68153ac0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7f8c681545e0 tx=0x7f8c50007c40).stop 2021-07-11T04:48:29.456 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 -- 172.21.2.104:0/2317694310 shutdown_connections 2021-07-11T04:48:29.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f8c680ad150 0x7f8c68153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f8c4c053240 0x7f8c4c055700 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.457 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f8c680aa860 0x7f8c68153ac0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 --2- 172.21.2.104:0/2317694310 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f8c680a9da0 0x7f8c68064840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 -- 172.21.2.104:0/2317694310 >> 172.21.2.104:0/2317694310 conn(0x7f8c680b64d0 msgr2=0x7f8c680605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:29.458 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 -- 172.21.2.104:0/2317694310 shutdown_connections 2021-07-11T04:48:29.459 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.452+0000 7f8c6f019700 1 -- 172.21.2.104:0/2317694310 wait complete. 2021-07-11T04:48:29.459 INFO:teuthology.orchestra.run.gibba004.stderr:instructed osd(s) 7 to deep-scrub 2021-07-11T04:48:29.462 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-07-11T04:48:29.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10907f700 1 Processor -- start 2021-07-11T04:48:29.513 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10907f700 1 -- start start 2021-07-11T04:48:29.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10907f700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff1041393d0 0x7ff1041430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10907f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff10405d760 0x7ff1040b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.514 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10907f700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1040604e0 con 0x7ff104060100 2021-07-11T04:48:29.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10907f700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1040b16b0 con 0x7ff1041393d0 2021-07-11T04:48:29.515 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10907f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040ac400 con 0x7ff10405d760 2021-07-11T04:48:29.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff1041393d0 0x7ff1041430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.520 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10259c700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7ff1041393d0 0x7ff1041430d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52540/0 (socket says 172.21.2.104:52540) 2021-07-11T04:48:29.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10259c700 1 -- 172.21.2.104:0/3131544105 learned_addr learned my addr 172.21.2.104:0/3131544105 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:29.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10259c700 1 -- 172.21.2.104:0/3131544105 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff10405d760 msgr2=0x7ff1040b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:29.521 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10259c700 1 --2- 172.21.2.104:0/3131544105 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff10405d760 0x7ff1040b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:29.522 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10259c700 1 -- 172.21.2.104:0/3131544105 >> v1:172.21.2.105:6789/0 conn(0x7ff104060100 legacy=0x7ff104138cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:29.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.508+0000 7ff10259c700 1 -- 172.21.2.104:0/3131544105 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1040ac6a0 con 0x7ff1041393d0 2021-07-11T04:48:29.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10259c700 1 --2- 172.21.2.104:0/3131544105 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1041393d0 0x7ff1041430d0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 rx=0x7ff104061a70 tx=0x7ff0fc009290).ready entity=mon.0 client_cookie=354d25c662a238da server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.523 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff101d9b700 1 -- 172.21.2.104:0/3131544105 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0fc00b860 con 0x7ff1041393d0 2021-07-11T04:48:29.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff101d9b700 1 -- 172.21.2.104:0/3131544105 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc00b9c0 con 0x7ff1041393d0 2021-07-11T04:48:29.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff101d9b700 1 -- 172.21.2.104:0/3131544105 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0fc00bb70 con 0x7ff1041393d0 2021-07-11T04:48:29.524 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- 172.21.2.104:0/3131544105 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1041393d0 msgr2=0x7ff1041430d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 --2- 172.21.2.104:0/3131544105 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1041393d0 0x7ff1041430d0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 rx=0x7ff104061a70 tx=0x7ff0fc009290).stop 2021-07-11T04:48:29.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- 172.21.2.104:0/3131544105 shutdown_connections 2021-07-11T04:48:29.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 --2- 172.21.2.104:0/3131544105 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff10405d760 0x7ff1040b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:48:29.525 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 --2- 172.21.2.104:0/3131544105 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff1041393d0 0x7ff1041430d0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- 172.21.2.104:0/3131544105 >> 172.21.2.104:0/3131544105 conn(0x7ff104061070 msgr2=0x7ff10405f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:29.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- 172.21.2.104:0/3131544105 shutdown_connections 2021-07-11T04:48:29.526 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- 172.21.2.104:0/3131544105 wait complete. 2021-07-11T04:48:29.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 Processor -- start 2021-07-11T04:48:29.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- start start 2021-07-11T04:48:29.527 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff10405d760 0x7ff1040b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.528 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff104060100 0x7ff1040adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1041393d0 0x7ff104143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff104144510 con 0x7ff10405d760 2021-07-11T04:48:29.529 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040ae180 con 0x7ff1041393d0 2021-07-11T04:48:29.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1040af660 con 0x7ff104060100 2021-07-11T04:48:29.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff102d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff10405d760 0x7ff1040b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff102d9d700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff10405d760 0x7ff1040b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52546/0 (socket says 172.21.2.104:52546) 2021-07-11T04:48:29.530 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff102d9d700 1 -- 172.21.2.104:0/294730767 learned_addr learned my addr 172.21.2.104:0/294730767 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:29.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10259c700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff104060100 0x7ff1040adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10359e700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1041393d0 0x7ff104143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.531 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff102d9d700 1 -- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff104060100 msgr2=0x7ff1040adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff102d9d700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff104060100 0x7ff1040adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff102d9d700 1 -- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1041393d0 msgr2=0x7ff104143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.532 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff102d9d700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1041393d0 0x7ff104143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff102d9d700 1 -- 172.21.2.104:0/294730767 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc009530 con 0x7ff10405d760 2021-07-11T04:48:29.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10359e700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1041393d0 0x7ff104143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-07-11T04:48:29.533 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff102d9d700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff10405d760 0x7ff1040b3f20 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 rx=0x7ff0ec000f70 tx=0x7ff0ec007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff0fbfff700 1 -- 172.21.2.104:0/294730767 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0ec00ce80 con 0x7ff10405d760 2021-07-11T04:48:29.534 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff0fbfff700 1 -- 172.21.2.104:0/294730767 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ec013450 con 0x7ff10405d760 2021-07-11T04:48:29.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- 172.21.2.104:0/294730767 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff104144740 con 0x7ff10405d760 2021-07-11T04:48:29.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- 172.21.2.104:0/294730767 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1040afab0 con 0x7ff10405d760 2021-07-11T04:48:29.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff0fbfff700 1 -- 172.21.2.104:0/294730767 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7ff0ec0137b0 con 0x7ff10405d760 2021-07-11T04:48:29.535 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff10907f700 1 -- 172.21.2.104:0/294730767 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff104057350 con 0x7ff10405d760 2021-07-11T04:48:29.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.512+0000 7ff0fbfff700 1 -- 172.21.2.104:0/294730767 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7ff0ec01a060 con 0x7ff10405d760 2021-07-11T04:48:29.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.516+0000 7ff0fbfff700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e80532d0 0x7ff0e8055790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:29.536 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.516+0000 7ff0fbfff700 1 -- 172.21.2.104:0/294730767 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff0ec06abf0 con 0x7ff10405d760 2021-07-11T04:48:29.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.516+0000 7ff10259c700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e80532d0 0x7ff0e8055790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:29.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.516+0000 7ff10259c700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e80532d0 0x7ff0e8055790 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7ff0fc009000 tx=0x7ff0fc016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:29.537 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.516+0000 7ff0fbfff700 1 -- 172.21.2.104:0/294730767 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7ff0ec042100 con 0x7ff10405d760 2021-07-11T04:48:29.600 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.596+0000 7ff10907f700 1 -- 172.21.2.104:0/294730767 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1041425b0 con 0x7ff0e80532d0 2021-07-11T04:48:29.608 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.604+0000 7ff0fbfff700 1 -- 172.21.2.104:0/294730767 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66465 (secure 0 0 0) 0x7ff1041425b0 con 0x7ff0e80532d0 2021-07-11T04:48:29.610 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:48:29.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.604+0000 7ff10907f700 1 -- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e80532d0 msgr2=0x7ff0e8055790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.611 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.604+0000 7ff10907f700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e80532d0 0x7ff0e8055790 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7ff0fc009000 tx=0x7ff0fc016000).stop 2021-07-11T04:48:29.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.604+0000 7ff10907f700 1 -- 172.21.2.104:0/294730767 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff10405d760 msgr2=0x7ff1040b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:29.612 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.604+0000 7ff10907f700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff10405d760 0x7ff1040b3f20 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 rx=0x7ff0ec000f70 tx=0x7ff0ec007c30).stop 2021-07-11T04:48:29.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.608+0000 7ff10907f700 1 -- 172.21.2.104:0/294730767 shutdown_connections 2021-07-11T04:48:29.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.608+0000 7ff10907f700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7ff1041393d0 0x7ff104143d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.613 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.608+0000 7ff10907f700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7ff104060100 0x7ff1040adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.608+0000 7ff10907f700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7ff0e80532d0 0x7ff0e8055790 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.608+0000 7ff10907f700 1 --2- 172.21.2.104:0/294730767 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7ff10405d760 0x7ff1040b3f20 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:29.614 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.608+0000 7ff10907f700 1 -- 172.21.2.104:0/294730767 >> 172.21.2.104:0/294730767 conn(0x7ff104061070 msgr2=0x7ff1040b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:29.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.608+0000 7ff10907f700 1 -- 172.21.2.104:0/294730767 shutdown_connections 2021-07-11T04:48:29.615 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:29.608+0000 7ff10907f700 1 -- 172.21.2.104:0/294730767 wait complete. 2021-07-11T04:48:29.615 INFO:teuthology.orchestra.run.gibba004.stderr:dumped all 2021-07-11T04:48:29.616 INFO:teuthology.orchestra.run.gibba004.stdout:{"pg_ready":true,"pg_map":{"version":1813,"stamp":"2021-07-11T04:48:27.817724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404944,"kb_used_data":16272,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746569776,"statfs":{"total":773094113280,"available":764487450624,"internally_reserved":8589934592,"allocated":16662528,"data_stored":7731989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005287"},"pg_stats":[{"pgid":"1.0","version":"16'2","reported_seq":"159","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:54.424625+0000","last_change":"2021-07-11T03:53:15.964238+0000","last_active":"2021-07-11T04:02:54.424625+0000","last_peered":"2021-07-11T04:02:54.424625+0000","last_clean":"2021-07-11T04:02:54.424625+0000","last_became_active":"2021-07-11T03:53:15.962801+0000","last_became_peered":"2021-07-11T03:53:15.962801+0000","last_unstale":"2021-07-11T04:02:54.424625+0000","last_undegraded":"2021-07-11T04:02:54.424625+0000","last_fullsized":"2021-07-11T04:02:54.424625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:11.361771+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:11.361771+0000","last_clean_scrub_stamp":"2021-07-11T03:53:11.361771+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"138","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:56.106047+0000","last_change":"2021-07-11T03:53:20.153399+0000","last_active":"2021-07-11T04:02:56.106047+0000","last_peered":"2021-07-11T04:02:56.106047+0000","last_clean":"2021-07-11T04:02:56.106047+0000","last_became_active":"2021-07-11T03:53:20.153135+0000","last_became_peered":"2021-07-11T03:53:20.153135+0000","last_unstale":"2021-07-11T04:02:56.106047+0000","last_undegraded":"2021-07-11T04:02:56.106047+0000","last_fullsized":"2021-07-11T04:02:56.106047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"154","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:54.424626+0000","last_change":"2021-07-11T03:53:20.152489+0000","last_active":"2021-07-11T04:02:54.424626+0000","last_peered":"2021-07-11T04:02:54.424626+0000","last_clean":"2021-07-11T04:02:54.424626+0000","last_became_active":"2021-07-11T03:53:20.152282+0000","last_became_peered":"2021-07-11T03:53:20.152282+0000","last_unstale":"2021-07-11T04:02:54.424626+0000","last_undegraded":"2021-07-11T04:02:54.424626+0000","last_fullsized":"2021-07-11T04:02:54.424626+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"142","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:54.716394+0000","last_change":"2021-07-11T03:53:20.153985+0000","last_active":"2021-07-11T04:02:54.716394+0000","last_peered":"2021-07-11T04:02:54.716394+0000","last_clean":"2021-07-11T04:02:54.716394+0000","last_became_active":"2021-07-11T03:53:20.153695+0000","last_became_peered":"2021-07-11T03:53:20.153695+0000","last_unstale":"2021-07-11T04:02:54.716394+0000","last_undegraded":"2021-07-11T04:02:54.716394+0000","last_fullsized":"2021-07-11T04:02:54.716394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"138","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:56.106021+0000","last_change":"2021-07-11T03:53:20.153402+0000","last_active":"2021-07-11T04:02:56.106021+0000","last_peered":"2021-07-11T04:02:56.106021+0000","last_clean":"2021-07-11T04:02:56.106021+0000","last_became_active":"2021-07-11T03:53:20.153235+0000","last_became_peered":"2021-07-11T03:53:20.153235+0000","last_unstale":"2021-07-11T04:02:56.106021+0000","last_undegraded":"2021-07-11T04:02:56.106021+0000","last_fullsized":"2021-07-11T04:02:56.106021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"136","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:55.312108+0000","last_change":"2021-07-11T03:53:20.286895+0000","last_active":"2021-07-11T04:02:55.312108+0000","last_peered":"2021-07-11T04:02:55.312108+0000","last_clean":"2021-07-11T04:02:55.312108+0000","last_became_active":"2021-07-11T03:53:20.286695+0000","last_became_peered":"2021-07-11T03:53:20.286695+0000","last_unstale":"2021-07-11T04:02:55.312108+0000","last_undegraded":"2021-07-11T04:02:55.312108+0000","last_fullsized":"2021-07-11T04:02:55.312108+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"154","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:54.424678+0000","last_change":"2021-07-11T03:53:20.288329+0000","last_active":"2021-07-11T04:02:54.424678+0000","last_peered":"2021-07-11T04:02:54.424678+0000","last_clean":"2021-07-11T04:02:54.424678+0000","last_became_active":"2021-07-11T03:53:20.287190+0000","last_became_peered":"2021-07-11T03:53:20.287190+0000","last_unstale":"2021-07-11T04:02:54.424678+0000","last_undegraded":"2021-07-11T04:02:54.424678+0000","last_fullsized":"2021-07-11T04:02:54.424678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"136","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:55.312181+0000","last_change":"2021-07-11T03:53:20.153427+0000","last_active":"2021-07-11T04:02:55.312181+0000","last_peered":"2021-07-11T04:02:55.312181+0000","last_clean":"2021-07-11T04:02:55.312181+0000","last_became_active":"2021-07-11T03:53:20.153186+0000","last_became_peered":"2021-07-11T03:53:20.153186+0000","last_unstale":"2021-07-11T04:02:55.312181+0000","last_undegraded":"2021-07-11T04:02:55.312181+0000","last_fullsized":"2021-07-11T04:02:55.312181+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"136","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:55.905701+0000","last_change":"2021-07-11T03:53:20.153252+0000","last_active":"2021-07-11T04:02:55.905701+0000","last_peered":"2021-07-11T04:02:55.905701+0000","last_clean":"2021-07-11T04:02:55.905701+0000","last_became_active":"2021-07-11T03:53:20.153046+0000","last_became_peered":"2021-07-11T03:53:20.153046+0000","last_unstale":"2021-07-11T04:02:55.905701+0000","last_undegraded":"2021-07-11T04:02:55.905701+0000","last_fullsized":"2021-07-11T04:02:55.905701+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542809,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2150000000000001,"15min":1.2410000000000001},"min":{"1min":0.46800000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.623,"5min":1.653,"15min":1.8620000000000001},"last":1.177},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2529999999999999,"15min":1.2350000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.313},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.542}]},{"osd":1,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2609999999999999,"15min":1.2589999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.33700000000000002},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.22,"15min":1.2370000000000001},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.5940000000000001,"5min":1.77,"15min":1.8400000000000001},"last":1.47}]},{"osd":2,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1850000000000001,"15min":1.198},"min":{"1min":0.371,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.6579999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.256},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.288,"15min":1.2629999999999999},"min":{"1min":0.53800000000000003,"5min":0.307,"15min":0.307},"max":{"1min":1.804,"5min":1.835,"15min":1.849},"last":1.6379999999999999}]},{"osd":3,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.44800000000000001,"5min":0.39200000000000002,"15min":0.34999999999999998},"max":{"1min":1.5620000000000001,"5min":1.867,"15min":1.944},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.337,"15min":1.343},"min":{"1min":0.51900000000000002,"5min":0.36499999999999999,"15min":0.34399999999999997},"max":{"1min":1.7470000000000001,"5min":1.847,"15min":1.974},"last":1.5740000000000001}]},{"osd":4,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.879,"15min":0.876},"min":{"1min":0.28299999999999997,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.4590000000000001,"5min":1.496,"15min":1.627},"last":1.1259999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.92300000000000004},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.2210000000000001,"5min":1.595,"15min":1.7250000000000001},"last":0.80900000000000005}]},{"osd":5,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.94499999999999995,"15min":0.92900000000000005},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.952},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.995,"5min":0.97899999999999998,"15min":0.97199999999999998},"min":{"1min":0.249,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.042,"15min":1.0429999999999999},"min":{"1min":0.29799999999999999,"5min":0.247,"15min":0.247},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.3169999999999999},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.024},"min":{"1min":0.27700000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.5860000000000001,"5min":1.724,"15min":1.8129999999999999},"last":0.88200000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542809,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050616,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321224,"statfs":{"total":96636764160,"available":95560933376,"internally_reserved":1073741824,"allocated":2080768,"data_stored":966194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.262,"15min":1.244},"min":{"1min":0.84099999999999997,"5min":0.55900000000000005,"15min":0.31900000000000001},"max":{"1min":1.6060000000000001,"5min":1.79,"15min":1.976},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.2829999999999999,"15min":1.2589999999999999},"min":{"1min":0.92400000000000004,"5min":0.57599999999999996,"15min":0.36099999999999999},"max":{"1min":1.5069999999999999,"5min":1.821,"15min":1.849},"last":1.2190000000000001}]},{"osd":1,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.236,"15min":1.2250000000000001},"min":{"1min":0.93700000000000006,"5min":0.60999999999999999,"15min":0.28100000000000003},"max":{"1min":1.631,"5min":1.873,"15min":1.873},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.2729999999999999,"15min":1.2370000000000001},"min":{"1min":0.90600000000000003,"5min":0.58599999999999997,"15min":0.27600000000000002},"max":{"1min":1.653,"5min":1.786,"15min":1.786},"last":1.4470000000000001}]},{"osd":2,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2549999999999999,"15min":1.256},"min":{"1min":0.87,"5min":0.622,"15min":0.33200000000000002},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":2.0169999999999999},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2569999999999999,"15min":1.2370000000000001},"min":{"1min":0.86199999999999999,"5min":0.63100000000000001,"15min":0.29499999999999998},"max":{"1min":1.53,"5min":1.6439999999999999,"15min":1.7789999999999999},"last":1.3220000000000001}]},{"osd":3,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.337,"15min":1.3100000000000001},"min":{"1min":0.93000000000000005,"5min":0.59999999999999998,"15min":0.29999999999999999},"max":{"1min":1.6890000000000001,"5min":1.8320000000000001,"15min":1.8460000000000001},"last":1.228},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.375,"15min":1.3320000000000001},"min":{"1min":0.90000000000000002,"5min":0.59599999999999997,"15min":0.34999999999999998},"max":{"1min":1.921,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.2889999999999999}]},{"osd":4,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92400000000000004,"15min":0.90400000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.17799999999999999},"max":{"1min":1.4770000000000001,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.95799999999999996,"15min":0.92400000000000004},"min":{"1min":0.216,"5min":0.216,"15min":0.183},"max":{"1min":1.8680000000000001,"5min":1.913,"15min":1.913},"last":0.95899999999999996}]},{"osd":5,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93200000000000005,"15min":0.91900000000000004},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.17599999999999999},"max":{"1min":1.6020000000000001,"5min":1.6879999999999999,"15min":1.7789999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.012,"15min":0.98299999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.16},"max":{"1min":1.7430000000000001,"5min":1.871,"15min":1.9019999999999999},"last":0.82899999999999996}]},{"osd":7,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.05,"15min":1.002},"min":{"1min":0.24299999999999999,"5min":0.218,"15min":0.19800000000000001},"max":{"1min":1.883,"5min":2.024,"15min":2.024},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.0649999999999999,"15min":1.0209999999999999},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.86099999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542809,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.347,"15min":1.306},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":1.76,"5min":1.8,"15min":1.843},"last":1.264},{"interface":"front","average":{"1min":1.385,"5min":1.363,"15min":1.3180000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.8100000000000001,"5min":1.96,"15min":1.96},"last":1.2250000000000001}]},{"osd":1,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.288,"15min":1.25},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":1.7110000000000001,"5min":1.7270000000000001,"15min":1.9359999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.292,"15min":1.258},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6990000000000001,"5min":2.0430000000000001,"15min":2.0430000000000001},"last":1.2969999999999999}]},{"osd":2,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2809999999999999,"15min":1.238},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.645,"5min":1.8169999999999999,"15min":1.837},"last":1.365},{"interface":"front","average":{"1min":1.284,"5min":1.2729999999999999,"15min":1.252},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.6890000000000001,"5min":1.843,"15min":1.843},"last":1.2030000000000001}]},{"osd":3,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2589999999999999,"15min":1.2569999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.3839999999999999,"5min":1.3109999999999999,"15min":1.238},"min":{"1min":0.442,"5min":0.442,"15min":0.378},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.94299999999999995}]},{"osd":4,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.90100000000000002,"15min":0.878},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.218},"max":{"1min":1.512,"5min":1.595,"15min":1.657},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.071,"5min":0.96299999999999997,"15min":0.92600000000000005},"min":{"1min":0.29599999999999999,"5min":0.24299999999999999,"15min":0.21299999999999999},"max":{"1min":1.855,"5min":1.855,"15min":1.855},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.96699999999999997,"15min":0.92900000000000005},"min":{"1min":0.35799999999999998,"5min":0.27100000000000002,"15min":0.23400000000000001},"max":{"1min":1.8029999999999999,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.01,"15min":0.97699999999999998},"min":{"1min":0.219,"5min":0.218,"15min":0.218},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.96299999999999997}]},{"osd":7,"last update":"Sun Jul 11 04:47:25 2021","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.012,"15min":0.97999999999999998},"min":{"1min":0.33400000000000002,"5min":0.26400000000000001,"15min":0.23100000000000001},"max":{"1min":1.7649999999999999,"5min":1.7649999999999999,"15min":1.9970000000000001},"last":1.012},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.0940000000000001,"15min":1.0269999999999999},"min":{"1min":0.33800000000000002,"5min":0.27000000000000002,"15min":0.22},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.879}]}]},{"osd":4,"up_from":14,"seq":60129542810,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050616,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321224,"statfs":{"total":96636764160,"available":95560933376,"internally_reserved":1073741824,"allocated":2080768,"data_stored":966194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3220000000000001,"15min":1.2969999999999999},"min":{"1min":1.0329999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3100000000000001,"15min":1.2909999999999999},"min":{"1min":1.0569999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.732,"5min":1.798,"15min":1.8799999999999999},"last":1.254}]},{"osd":1,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.236,"15min":1.224},"min":{"1min":0.91600000000000004,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.641,"5min":1.6759999999999999,"15min":1.766},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.218,"15min":1.234},"min":{"1min":0.89800000000000002,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":1.6279999999999999,"5min":1.7230000000000001,"15min":1.79},"last":1.282}]},{"osd":2,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2769999999999999,"15min":1.26},"min":{"1min":0.99199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.506,"5min":1.6579999999999999,"15min":1.8129999999999999},"last":1.304},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.272,"15min":1.2729999999999999},"min":{"1min":1.0229999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.6679999999999999,"5min":1.758,"15min":1.758},"last":1.5109999999999999}]},{"osd":3,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2070000000000001,"15min":1.22},"min":{"1min":0.92600000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.728},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.248,"15min":1.2430000000000001},"min":{"1min":0.97499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5649999999999999,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.484}]},{"osd":5,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.84499999999999997,"15min":0.85399999999999998},"min":{"1min":0.55500000000000005,"5min":0.24099999999999999,"15min":0.22600000000000001},"max":{"1min":1.2989999999999999,"5min":1.5720000000000001,"15min":1.6459999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.91000000000000003,"15min":0.90000000000000002},"min":{"1min":0.63600000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.143,"5min":1.5109999999999999,"15min":1.6819999999999999},"last":0.92500000000000004}]},{"osd":6,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.91200000000000003},"min":{"1min":0.57699999999999996,"5min":0.27600000000000002,"15min":0.216},"max":{"1min":1.2929999999999999,"5min":1.613,"15min":1.7749999999999999},"last":1},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.95699999999999996,"15min":0.96399999999999997},"min":{"1min":0.69099999999999995,"5min":0.22,"15min":0.22},"max":{"1min":1.1539999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":1.032}]},{"osd":7,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.52900000000000003,"5min":0.26400000000000001,"15min":0.23100000000000001},"max":{"1min":1.3660000000000001,"5min":1.5860000000000001,"15min":1.7330000000000001},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0649999999999999,"15min":1.026},"min":{"1min":0.61699999999999999,"5min":0.26600000000000001,"15min":0.22},"max":{"1min":1.3260000000000001,"5min":1.6950000000000001,"15min":1.782},"last":1.1870000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542809,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.28699999999999998,"5min":0.27600000000000002,"15min":0.23300000000000001},"max":{"1min":0.97199999999999998,"5min":1.1379999999999999,"15min":1.4359999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80300000000000005,"15min":0.80200000000000005},"min":{"1min":0.27600000000000002,"5min":0.221,"15min":0.221},"max":{"1min":1.0309999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.88200000000000001}]},{"osd":1,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.77000000000000002,"15min":0.77200000000000002},"min":{"1min":0.30299999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93300000000000005,"5min":1.081,"15min":1.081},"last":0.754},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.83799999999999997,"15min":0.82599999999999996},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.22900000000000001},"max":{"1min":1.109,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.93300000000000005}]},{"osd":2,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.83199999999999996,"15min":0.81699999999999995},"min":{"1min":0.29599999999999999,"5min":0.27700000000000002,"15min":0.216},"max":{"1min":1.089,"5min":1.1819999999999999,"15min":1.625},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.89200000000000002,"15min":0.88500000000000001},"min":{"1min":0.32200000000000001,"5min":0.248,"15min":0.248},"max":{"1min":1.3080000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.82799999999999996}]},{"osd":4,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.361,"15min":1.3340000000000001},"min":{"1min":0.81799999999999995,"5min":0.72099999999999997,"15min":0.36399999999999999},"max":{"1min":1.6719999999999999,"5min":1.859,"15min":1.859},"last":1.669},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.339,"15min":1.3380000000000001},"min":{"1min":0.70299999999999996,"5min":0.68300000000000005,"15min":0.34799999999999998},"max":{"1min":1.6839999999999999,"5min":1.873,"15min":1.873},"last":1.712}]},{"osd":5,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.419,"15min":1.3799999999999999},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.42899999999999999},"max":{"1min":1.7789999999999999,"5min":1.875,"15min":1.875},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.3759999999999999,"15min":1.3660000000000001},"min":{"1min":0.73299999999999998,"5min":0.66700000000000004,"15min":0.41899999999999998},"max":{"1min":1.667,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.6220000000000001}]},{"osd":6,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.3420000000000001,"15min":1.3109999999999999},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.433},"max":{"1min":1.524,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.452},{"interface":"front","average":{"1min":1.304,"5min":1.3580000000000001,"15min":1.345},"min":{"1min":0.80600000000000005,"5min":0.67400000000000004,"15min":0.33700000000000002},"max":{"1min":1.5409999999999999,"5min":1.905,"15min":1.905},"last":1.4119999999999999}]},{"osd":7,"last update":"Sun Jul 11 04:47:39 2021","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.361,"15min":1.341},"min":{"1min":0.753,"5min":0.70099999999999996,"15min":0.44700000000000001},"max":{"1min":1.75,"5min":1.8700000000000001,"15min":1.8700000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3720000000000001,"15min":1.3600000000000001},"min":{"1min":0.67300000000000004,"5min":0.64600000000000002,"15min":0.39900000000000002},"max":{"1min":1.708,"5min":1.7849999999999999,"15min":1.7869999999999999},"last":1.5489999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542810,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.83499999999999996,"15min":0.81999999999999995},"min":{"1min":0.58499999999999996,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":1.2430000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.88200000000000001,"15min":0.86299999999999999},"min":{"1min":0.61099999999999999,"5min":0.27300000000000002,"15min":0.252},"max":{"1min":1.157,"5min":1.417,"15min":1.417},"last":1.028}]},{"osd":1,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.44400000000000001,"5min":0.245,"15min":0.245},"max":{"1min":1.083,"5min":1.3500000000000001,"15min":1.5469999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.83499999999999996,"15min":0.82099999999999995},"min":{"1min":0.52100000000000002,"5min":0.26900000000000002,"15min":0.23799999999999999},"max":{"1min":1.089,"5min":1.2849999999999999,"15min":1.3},"last":0.81000000000000005}]},{"osd":3,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80500000000000005,"15min":0.79300000000000004},"min":{"1min":0.41799999999999998,"5min":0.23599999999999999,"15min":0.22900000000000001},"max":{"1min":1.0349999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.878},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.89500000000000002,"15min":0.88200000000000001},"min":{"1min":0.49099999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2010000000000001,"5min":1.375,"15min":1.573},"last":0.84299999999999997}]},{"osd":4,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3160000000000001,"15min":1.296},"min":{"1min":0.86299999999999999,"5min":0.66300000000000003,"15min":0.44800000000000001},"max":{"1min":1.538,"5min":1.794,"15min":1.877},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.395,"15min":1.3540000000000001},"min":{"1min":0.73899999999999999,"5min":0.623,"15min":0.48599999999999999},"max":{"1min":1.7210000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.1240000000000001}]},{"osd":5,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3640000000000001,"15min":1.335},"min":{"1min":0.82399999999999995,"5min":0.67200000000000004,"15min":0.441},"max":{"1min":1.6319999999999999,"5min":1.966,"15min":1.966},"last":1.411},{"interface":"front","average":{"1min":1.371,"5min":1.3580000000000001,"15min":1.335},"min":{"1min":0.90200000000000002,"5min":0.63800000000000001,"15min":0.41499999999999998},"max":{"1min":1.5820000000000001,"5min":1.7829999999999999,"15min":1.956},"last":1.385}]},{"osd":6,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.383,"15min":1.3560000000000001},"min":{"1min":0.80300000000000005,"5min":0.64400000000000002,"15min":0.52600000000000002},"max":{"1min":1.667,"5min":1.794,"15min":1.8360000000000001},"last":1.244},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.357},"min":{"1min":0.78600000000000003,"5min":0.58499999999999996,"15min":0.47599999999999998},"max":{"1min":1.7130000000000001,"5min":1.9079999999999999,"15min":2.0550000000000002},"last":1.536}]},{"osd":7,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3779999999999999,"15min":1.3560000000000001},"min":{"1min":0.76800000000000002,"5min":0.71599999999999997,"15min":0.47199999999999998},"max":{"1min":1.6040000000000001,"5min":1.992,"15min":1.992},"last":1.179},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3759999999999999,"15min":1.3460000000000001},"min":{"1min":0.85299999999999998,"5min":0.65300000000000002,"15min":0.53500000000000003},"max":{"1min":1.6899999999999999,"5min":1.8149999999999999,"15min":1.964},"last":1.49}]}]},{"osd":1,"up_from":14,"seq":60129542810,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.79400000000000004,"15min":0.80100000000000005},"min":{"1min":0.20599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.1359999999999999,"5min":1.1459999999999999,"15min":1.2709999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85399999999999998,"15min":0.85899999999999999},"min":{"1min":0.26200000000000001,"5min":0.247,"15min":0.22600000000000001},"max":{"1min":1.1319999999999999,"5min":1.1519999999999999,"15min":1.1970000000000001},"last":0.83499999999999996}]},{"osd":2,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.79200000000000004,"15min":0.80800000000000005},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":1.151,"5min":1.216,"15min":1.216},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.78500000000000003,"15min":0.80900000000000005},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":1.069,"5min":1.234,"15min":1.234},"last":0.86599999999999999}]},{"osd":3,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.83099999999999996,"15min":0.86299999999999999},"min":{"1min":0.26300000000000001,"5min":0.23999999999999999,"15min":0.23100000000000001},"max":{"1min":1.111,"5min":1.143,"15min":1.3600000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.77700000000000002,"15min":0.78400000000000003},"min":{"1min":0.28799999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.1160000000000001,"5min":1.2050000000000001,"15min":1.258},"last":0.90900000000000003}]},{"osd":4,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.321,"15min":1.3169999999999999},"min":{"1min":0.69399999999999995,"5min":0.51900000000000002,"15min":0.29599999999999999},"max":{"1min":1.6699999999999999,"5min":1.887,"15min":1.887},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.3280000000000001,"15min":1.321},"min":{"1min":0.65900000000000003,"5min":0.48799999999999999,"15min":0.28899999999999998},"max":{"1min":1.6499999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.296}]},{"osd":5,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.3300000000000001,"15min":1.3240000000000001},"min":{"1min":0.78700000000000003,"5min":0.46700000000000003,"15min":0.25900000000000001},"max":{"1min":2.0059999999999998,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.3140000000000001,"15min":1.304},"min":{"1min":0.73499999999999999,"5min":0.432,"15min":0.26400000000000001},"max":{"1min":2.2280000000000002,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.452}]},{"osd":6,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.415,"15min":1.391},"min":{"1min":0.71699999999999997,"5min":0.44700000000000001,"15min":0.313},"max":{"1min":2.2970000000000002,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.405},{"interface":"front","average":{"1min":1.48,"5min":1.391,"15min":1.393},"min":{"1min":0.74299999999999999,"5min":0.498,"15min":0.30599999999999999},"max":{"1min":2.3410000000000002,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.379}]},{"osd":7,"last update":"Sun Jul 11 04:47:43 2021","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.3009999999999999,"15min":1.3169999999999999},"min":{"1min":0.63900000000000001,"5min":0.40999999999999998,"15min":0.34599999999999997},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.8600000000000001},"last":1.55},{"interface":"front","average":{"1min":1.363,"5min":1.343,"15min":1.3360000000000001},"min":{"1min":0.64600000000000002,"5min":0.55300000000000005,"15min":0.27100000000000002},"max":{"1min":1.7709999999999999,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.349}]}]},{"osd":0,"up_from":14,"seq":60129542808,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050612,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321228,"statfs":{"total":96636764160,"available":95560937472,"internally_reserved":1073741824,"allocated":2076672,"data_stored":965616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76400000000000001,"15min":0.78400000000000003},"min":{"1min":0.25600000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.94299999999999995,"5min":1.099,"15min":1.1890000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.26300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":1.1379999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.85399999999999998}]},{"osd":2,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81000000000000005,"15min":0.81899999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.21099999999999999},"max":{"1min":1.0800000000000001,"5min":1.129,"15min":1.2789999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.90000000000000002,"15min":0.91900000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.253},"max":{"1min":1.0720000000000001,"5min":1.252,"15min":1.4219999999999999},"last":0.97199999999999998}]},{"osd":3,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73399999999999999,"15min":0.76300000000000001},"min":{"1min":0.33400000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.454},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.219},"max":{"1min":1.095,"5min":1.1619999999999999,"15min":1.339},"last":1.0289999999999999}]},{"osd":4,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.329,"15min":1.3620000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.6359999999999999,"5min":1.7470000000000001,"15min":2.1030000000000002},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.367,"5min":1.331,"15min":1.343},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":1.6499999999999999,"5min":1.72,"15min":1.8899999999999999},"last":1.456}]},{"osd":5,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.321,"15min":1.3320000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.6459999999999999,"5min":1.8280000000000001,"15min":1.9399999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.33500000000000002},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":2.0289999999999999},"last":1.4410000000000001}]},{"osd":6,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3069999999999999,"15min":1.347},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6950000000000001,"5min":1.7929999999999999,"15min":1.944},"last":1.391},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2949999999999999,"15min":1.3240000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.5649999999999999,"5min":1.6950000000000001,"15min":1.8580000000000001},"last":1.6950000000000001}]},{"osd":7,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.335,"15min":1.353},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6479999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.349,"15min":1.363},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.39300000000000002},"max":{"1min":1.6779999999999999,"5min":1.8169999999999999,"15min":1.9199999999999999},"last":1.53}]}]}]}} 2021-07-11T04:48:29.622 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2021-07-11T03:53:11.361771+0000 time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=3, tm_min=53, tm_sec=11, tm_wday=6, tm_yday=192, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=4, tm_min=48, tm_sec=26, tm_wday=6, tm_yday=192, tm_isdst=0) 2021-07-11T04:48:29.623 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2021-07-11T03:53:19.125309+0000 time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=3, tm_min=53, tm_sec=19, tm_wday=6, tm_yday=192, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=4, tm_min=48, tm_sec=26, tm_wday=6, tm_yday=192, tm_isdst=0) 2021-07-11T04:48:29.623 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2021-07-11T03:53:19.125309+0000 time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=3, tm_min=53, tm_sec=19, tm_wday=6, tm_yday=192, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=4, tm_min=48, tm_sec=26, tm_wday=6, tm_yday=192, tm_isdst=0) 2021-07-11T04:48:29.624 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2021-07-11T03:53:19.125309+0000 time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=3, tm_min=53, tm_sec=19, tm_wday=6, tm_yday=192, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=4, tm_min=48, tm_sec=26, tm_wday=6, tm_yday=192, tm_isdst=0) 2021-07-11T04:48:29.624 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2021-07-11T03:53:19.125309+0000 time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=3, tm_min=53, tm_sec=19, tm_wday=6, tm_yday=192, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=4, tm_min=48, tm_sec=26, tm_wday=6, tm_yday=192, tm_isdst=0) 2021-07-11T04:48:29.625 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2021-07-11T03:53:19.125309+0000 time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=3, tm_min=53, tm_sec=19, tm_wday=6, tm_yday=192, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=4, tm_min=48, tm_sec=26, tm_wday=6, tm_yday=192, tm_isdst=0) 2021-07-11T04:48:29.626 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2021-07-11T03:53:19.125309+0000 time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=3, tm_min=53, tm_sec=19, tm_wday=6, tm_yday=192, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=4, tm_min=48, tm_sec=26, tm_wday=6, tm_yday=192, tm_isdst=0) 2021-07-11T04:48:29.626 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2021-07-11T03:53:19.125309+0000 time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=3, tm_min=53, tm_sec=19, tm_wday=6, tm_yday=192, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=4, tm_min=48, tm_sec=26, tm_wday=6, tm_yday=192, tm_isdst=0) 2021-07-11T04:48:29.627 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2021-07-11T03:53:19.125309+0000 time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=3, tm_min=53, tm_sec=19, tm_wday=6, tm_yday=192, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=4, tm_min=48, tm_sec=26, tm_wday=6, tm_yday=192, tm_isdst=0) 2021-07-11T04:48:29.628 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2021-07-11T04:48:49.629 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-07-11T04:48:49.685 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.680+0000 7fc3a04ec700 1 Processor -- start 2021-07-11T04:48:49.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.680+0000 7fc3a04ec700 1 -- start start 2021-07-11T04:48:49.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.680+0000 7fc3a04ec700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc3980a76b0 0x7fc398066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:49.686 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.680+0000 7fc3a04ec700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3980b0060 0x7fc39805fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:49.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.680+0000 7fc3a04ec700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3980a7a90 con 0x7fc3980b1970 2021-07-11T04:48:49.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.680+0000 7fc3a04ec700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc398147620 con 0x7fc3980a76b0 2021-07-11T04:48:49.687 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.680+0000 7fc3a04ec700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3981478c0 con 0x7fc3980b0060 2021-07-11T04:48:49.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39ea89700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3980b0060 0x7fc39805fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:49.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39e288700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc3980a76b0 0x7fc398066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:49.689 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39e288700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7fc3980a76b0 0x7fc398066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3300/0 says I am v2:172.21.2.104:60110/0 (socket says 172.21.2.104:60110) 2021-07-11T04:48:49.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39ea89700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3980b0060 0x7fc39805fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.105:3301/0 says I am v2:172.21.2.104:55714/0 (socket says 172.21.2.104:55714) 2021-07-11T04:48:49.690 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39e288700 1 -- 172.21.2.104:0/1467845750 learned_addr learned my addr 172.21.2.104:0/1467845750 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:49.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39da87700 1 --1- 172.21.2.104:0/1467845750 >> v1:172.21.2.105:6789/0 conn(0x7fc3980b1970 0x7fc39805f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.105:6789/0 says I am v1:172.21.2.104:48688/0 (socket says 172.21.2.104:48688) 2021-07-11T04:48:49.691 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39e288700 1 -- 172.21.2.104:0/1467845750 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3980b0060 msgr2=0x7fc39805fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:48:49.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39e288700 1 --2- 172.21.2.104:0/1467845750 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3980b0060 0x7fc39805fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:49.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39e288700 1 -- 172.21.2.104:0/1467845750 >> v1:172.21.2.105:6789/0 conn(0x7fc3980b1970 legacy=0x7fc39805f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:49.692 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39e288700 1 -- 172.21.2.104:0/1467845750 --> v2:172.21.2.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc398147b60 con 0x7fc3980a76b0 2021-07-11T04:48:49.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39e288700 1 --2- 172.21.2.104:0/1467845750 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3980a76b0 0x7fc398066a70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fc38c002f70 tx=0x7fc38c006eb0).ready entity=mon.1 client_cookie=a049080e343e2fa6 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:49.693 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39d286700 1 -- 172.21.2.104:0/1467845750 <== mon.1 v2:172.21.2.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc38c00d570 con 0x7fc3980a76b0 2021-07-11T04:48:49.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39d286700 1 -- 172.21.2.104:0/1467845750 <== mon.1 v2:172.21.2.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c000e90 con 0x7fc3980a76b0 2021-07-11T04:48:49.694 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc39d286700 1 -- 172.21.2.104:0/1467845750 <== mon.1 v2:172.21.2.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc38c00d900 con 0x7fc3980a76b0 2021-07-11T04:48:49.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc3a04ec700 1 -- 172.21.2.104:0/1467845750 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3980a76b0 msgr2=0x7fc398066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:49.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc3a04ec700 1 --2- 172.21.2.104:0/1467845750 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3980a76b0 0x7fc398066a70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fc38c002f70 tx=0x7fc38c006eb0).stop 2021-07-11T04:48:49.695 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc3a04ec700 1 -- 172.21.2.104:0/1467845750 shutdown_connections 2021-07-11T04:48:49.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc3a04ec700 1 --2- 172.21.2.104:0/1467845750 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3980b0060 0x7fc39805fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:49.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc3a04ec700 1 --2- 172.21.2.104:0/1467845750 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3980a76b0 0x7fc398066a70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:49.696 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc3a04ec700 1 -- 172.21.2.104:0/1467845750 >> 172.21.2.104:0/1467845750 conn(0x7fc3980b53e0 msgr2=0x7fc3980b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:49.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc3a04ec700 1 -- 172.21.2.104:0/1467845750 shutdown_connections 2021-07-11T04:48:49.697 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc3a04ec700 1 -- 172.21.2.104:0/1467845750 wait complete. 2021-07-11T04:48:49.700 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc3a04ec700 1 Processor -- start 2021-07-11T04:48:49.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.684+0000 7fc3a04ec700 1 -- start start 2021-07-11T04:48:49.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc3a04ec700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3980a76b0 0x7fc398149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:49.701 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc3a04ec700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3980b0060 0x7fc3980ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:49.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc3a04ec700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3980b1970 0x7fc398159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:49.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc3a04ec700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3980aace0 con 0x7fc3980b0060 2021-07-11T04:48:49.702 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc3a04ec700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3980abef0 con 0x7fc3980b1970 2021-07-11T04:48:49.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc3a04ec700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3980ac190 con 0x7fc3980a76b0 2021-07-11T04:48:49.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39da87700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3980b0060 0x7fc3980ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:49.703 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39da87700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3980b0060 0x7fc3980ab3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52558/0 (socket says 172.21.2.104:52558) 2021-07-11T04:48:49.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39da87700 1 -- 172.21.2.104:0/3424257667 learned_addr learned my addr 172.21.2.104:0/3424257667 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:48:49.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39e288700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3980a76b0 0x7fc398149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:49.704 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39ea89700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3980b1970 0x7fc398159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:49.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39da87700 1 -- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3980a76b0 msgr2=0x7fc398149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:49.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39da87700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3980a76b0 0x7fc398149ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:49.705 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39da87700 1 -- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3980b1970 msgr2=0x7fc398159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:49.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39da87700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3980b1970 0x7fc398159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:49.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39da87700 1 -- 172.21.2.104:0/3424257667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c00b1e0 con 0x7fc3980b0060 2021-07-11T04:48:49.706 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39da87700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3980b0060 0x7fc3980ab3b0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 rx=0x7fc39806a0e0 tx=0x7fc394007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:49.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc38b7fe700 1 -- 172.21.2.104:0/3424257667 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc39400ce30 con 0x7fc3980b0060 2021-07-11T04:48:49.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc38b7fe700 1 -- 172.21.2.104:0/3424257667 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc394007170 con 0x7fc3980b0060 2021-07-11T04:48:49.707 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc38b7fe700 1 -- 172.21.2.104:0/3424257667 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7fc394004730 con 0x7fc3980b0060 2021-07-11T04:48:49.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc3a04ec700 1 -- 172.21.2.104:0/3424257667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3980aaf10 con 0x7fc3980b0060 2021-07-11T04:48:49.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc3a04ec700 1 -- 172.21.2.104:0/3424257667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39815a2e0 con 0x7fc3980b0060 2021-07-11T04:48:49.708 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc3a04ec700 1 -- 172.21.2.104:0/3424257667 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3980ab0a0 con 0x7fc3980b0060 2021-07-11T04:48:49.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc38b7fe700 1 -- 172.21.2.104:0/3424257667 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7fc394004890 con 0x7fc3980b0060 2021-07-11T04:48:49.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc38b7fe700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc378053260 0x7fc378055720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:48:49.709 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc38b7fe700 1 -- 172.21.2.104:0/3424257667 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc39406aad0 con 0x7fc3980b0060 2021-07-11T04:48:49.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.688+0000 7fc39e288700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc378053260 0x7fc378055720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:48:49.710 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.692+0000 7fc39e288700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc378053260 0x7fc378055720 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7fc38c019000 tx=0x7fc38c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:48:49.711 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.692+0000 7fc38b7fe700 1 -- 172.21.2.104:0/3424257667 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7fc394041f80 con 0x7fc3980b0060 2021-07-11T04:48:49.773 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.768+0000 7fc3a04ec700 1 -- 172.21.2.104:0/3424257667 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc398057350 con 0x7fc378053260 2021-07-11T04:48:49.781 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.776+0000 7fc38b7fe700 1 -- 172.21.2.104:0/3424257667 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66479 (secure 0 0 0) 0x7fc398057350 con 0x7fc378053260 2021-07-11T04:48:49.782 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:48:49.783 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 -- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc378053260 msgr2=0x7fc378055720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:49.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc378053260 0x7fc378055720 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7fc38c019000 tx=0x7fc38c015000).stop 2021-07-11T04:48:49.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 -- 172.21.2.104:0/3424257667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3980b0060 msgr2=0x7fc3980ab3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:48:49.784 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3980b0060 0x7fc3980ab3b0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 rx=0x7fc39806a0e0 tx=0x7fc394007be0).stop 2021-07-11T04:48:49.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 -- 172.21.2.104:0/3424257667 shutdown_connections 2021-07-11T04:48:49.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7fc3980b1970 0x7fc398159df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:49.785 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7fc378053260 0x7fc378055720 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:49.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7fc3980b0060 0x7fc3980ab3b0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:49.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 --2- 172.21.2.104:0/3424257667 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7fc3980a76b0 0x7fc398149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:48:49.786 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 -- 172.21.2.104:0/3424257667 >> 172.21.2.104:0/3424257667 conn(0x7fc3980b53e0 msgr2=0x7fc3980b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:48:49.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 -- 172.21.2.104:0/3424257667 shutdown_connections 2021-07-11T04:48:49.787 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:48:49.780+0000 7fc3a04ec700 1 -- 172.21.2.104:0/3424257667 wait complete. 2021-07-11T04:48:49.787 INFO:teuthology.orchestra.run.gibba004.stderr:dumped all 2021-07-11T04:48:49.789 INFO:teuthology.orchestra.run.gibba004.stdout:{"pg_ready":true,"pg_map":{"version":1823,"stamp":"2021-07-11T04:48:47.827259+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9833,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404944,"kb_used_data":16272,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746569776,"statfs":{"total":773094113280,"available":764487450624,"internally_reserved":8589934592,"allocated":16662528,"data_stored":7731989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9833,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005542"},"pg_stats":[{"pgid":"1.0","version":"164'4","reported_seq":"164","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:42.401398+0000","last_change":"2021-07-11T04:48:42.401658+0000","last_active":"2021-07-11T04:48:42.401398+0000","last_peered":"2021-07-11T04:48:42.401398+0000","last_clean":"2021-07-11T04:48:42.401398+0000","last_became_active":"2021-07-11T03:53:15.962801+0000","last_became_peered":"2021-07-11T03:53:15.962801+0000","last_unstale":"2021-07-11T04:48:42.401398+0000","last_undegraded":"2021-07-11T04:48:42.401398+0000","last_fullsized":"2021-07-11T04:48:42.401398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"164'4","last_scrub_stamp":"2021-07-11T04:48:42.401397+0000","last_deep_scrub":"164'4","last_deep_scrub_stamp":"2021-07-11T04:48:42.401397+0000","last_clean_scrub_stamp":"2021-07-11T04:48:42.401397+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9833,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"141","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:29.961462+0000","last_change":"2021-07-11T04:48:29.961750+0000","last_active":"2021-07-11T04:48:29.961462+0000","last_peered":"2021-07-11T04:48:29.961462+0000","last_clean":"2021-07-11T04:48:29.961462+0000","last_became_active":"2021-07-11T03:53:20.153135+0000","last_became_peered":"2021-07-11T03:53:20.153135+0000","last_unstale":"2021-07-11T04:48:29.961462+0000","last_undegraded":"2021-07-11T04:48:29.961462+0000","last_fullsized":"2021-07-11T04:48:29.961462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:29.961461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:29.961461+0000","last_clean_scrub_stamp":"2021-07-11T04:48:29.961461+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"157","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:45.481818+0000","last_change":"2021-07-11T04:48:45.482186+0000","last_active":"2021-07-11T04:48:45.481818+0000","last_peered":"2021-07-11T04:48:45.481818+0000","last_clean":"2021-07-11T04:48:45.481818+0000","last_became_active":"2021-07-11T03:53:20.152282+0000","last_became_peered":"2021-07-11T03:53:20.152282+0000","last_unstale":"2021-07-11T04:48:45.481818+0000","last_undegraded":"2021-07-11T04:48:45.481818+0000","last_fullsized":"2021-07-11T04:48:45.481818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:45.481817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:45.481817+0000","last_clean_scrub_stamp":"2021-07-11T04:48:45.481817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"145","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:32.137673+0000","last_change":"2021-07-11T04:48:32.137727+0000","last_active":"2021-07-11T04:48:32.137673+0000","last_peered":"2021-07-11T04:48:32.137673+0000","last_clean":"2021-07-11T04:48:32.137673+0000","last_became_active":"2021-07-11T03:53:20.153695+0000","last_became_peered":"2021-07-11T03:53:20.153695+0000","last_unstale":"2021-07-11T04:48:32.137673+0000","last_undegraded":"2021-07-11T04:48:32.137673+0000","last_fullsized":"2021-07-11T04:48:32.137673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:32.137673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:32.137673+0000","last_clean_scrub_stamp":"2021-07-11T04:48:32.137673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"141","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:28.999475+0000","last_change":"2021-07-11T04:48:28.999543+0000","last_active":"2021-07-11T04:48:28.999475+0000","last_peered":"2021-07-11T04:48:28.999475+0000","last_clean":"2021-07-11T04:48:28.999475+0000","last_became_active":"2021-07-11T03:53:20.153235+0000","last_became_peered":"2021-07-11T03:53:20.153235+0000","last_unstale":"2021-07-11T04:48:28.999475+0000","last_undegraded":"2021-07-11T04:48:28.999475+0000","last_fullsized":"2021-07-11T04:48:28.999475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:28.999474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:28.999474+0000","last_clean_scrub_stamp":"2021-07-11T04:48:28.999474+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"139","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:38.839037+0000","last_change":"2021-07-11T04:48:38.839082+0000","last_active":"2021-07-11T04:48:38.839037+0000","last_peered":"2021-07-11T04:48:38.839037+0000","last_clean":"2021-07-11T04:48:38.839037+0000","last_became_active":"2021-07-11T03:53:20.286695+0000","last_became_peered":"2021-07-11T03:53:20.286695+0000","last_unstale":"2021-07-11T04:48:38.839037+0000","last_undegraded":"2021-07-11T04:48:38.839037+0000","last_fullsized":"2021-07-11T04:48:38.839037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:38.839037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:38.839037+0000","last_clean_scrub_stamp":"2021-07-11T04:48:38.839037+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"154","reported_epoch":"163","state":"active+clean","last_fresh":"2021-07-11T04:02:54.424678+0000","last_change":"2021-07-11T03:53:20.288329+0000","last_active":"2021-07-11T04:02:54.424678+0000","last_peered":"2021-07-11T04:02:54.424678+0000","last_clean":"2021-07-11T04:02:54.424678+0000","last_became_active":"2021-07-11T03:53:20.287190+0000","last_became_peered":"2021-07-11T03:53:20.287190+0000","last_unstale":"2021-07-11T04:02:54.424678+0000","last_undegraded":"2021-07-11T04:02:54.424678+0000","last_fullsized":"2021-07-11T04:02:54.424678+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T03:53:19.125309+0000","last_clean_scrub_stamp":"2021-07-11T03:53:19.125309+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"139","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:40.801640+0000","last_change":"2021-07-11T04:48:40.801712+0000","last_active":"2021-07-11T04:48:40.801640+0000","last_peered":"2021-07-11T04:48:40.801640+0000","last_clean":"2021-07-11T04:48:40.801640+0000","last_became_active":"2021-07-11T03:53:20.153186+0000","last_became_peered":"2021-07-11T03:53:20.153186+0000","last_unstale":"2021-07-11T04:48:40.801640+0000","last_undegraded":"2021-07-11T04:48:40.801640+0000","last_fullsized":"2021-07-11T04:48:40.801640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:40.801639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:40.801639+0000","last_clean_scrub_stamp":"2021-07-11T04:48:40.801639+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"139","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:29.862634+0000","last_change":"2021-07-11T04:48:29.862689+0000","last_active":"2021-07-11T04:48:29.862634+0000","last_peered":"2021-07-11T04:48:29.862634+0000","last_clean":"2021-07-11T04:48:29.862634+0000","last_became_active":"2021-07-11T03:53:20.153046+0000","last_became_peered":"2021-07-11T03:53:20.153046+0000","last_unstale":"2021-07-11T04:48:29.862634+0000","last_undegraded":"2021-07-11T04:48:29.862634+0000","last_fullsized":"2021-07-11T04:48:29.862634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:29.862634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:29.862634+0000","last_clean_scrub_stamp":"2021-07-11T04:48:29.862634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9833,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542813,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.2150000000000001,"15min":1.2410000000000001},"min":{"1min":0.46800000000000003,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.623,"5min":1.653,"15min":1.8620000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2529999999999999,"15min":1.2350000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.313},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.242}]},{"osd":1,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2609999999999999,"15min":1.2589999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.33700000000000002},"max":{"1min":1.847,"5min":1.847,"15min":1.847},"last":1.369},{"interface":"front","average":{"1min":1.1930000000000001,"5min":1.22,"15min":1.2370000000000001},"min":{"1min":0.439,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.5940000000000001,"5min":1.77,"15min":1.8400000000000001},"last":1.1799999999999999}]},{"osd":2,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.153,"5min":1.1850000000000001,"15min":1.198},"min":{"1min":0.371,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.6579999999999999,"5min":1.7290000000000001,"15min":1.7290000000000001},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.288,"15min":1.2629999999999999},"min":{"1min":0.53800000000000003,"5min":0.307,"15min":0.307},"max":{"1min":1.804,"5min":1.835,"15min":1.849},"last":1.2689999999999999}]},{"osd":3,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.266,"15min":1.2569999999999999},"min":{"1min":0.44800000000000001,"5min":0.39200000000000002,"15min":0.34999999999999998},"max":{"1min":1.5620000000000001,"5min":1.867,"15min":1.944},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.337,"15min":1.343},"min":{"1min":0.51900000000000002,"5min":0.36499999999999999,"15min":0.34399999999999997},"max":{"1min":1.7470000000000001,"5min":1.847,"15min":1.974},"last":1.226}]},{"osd":4,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.879,"15min":0.876},"min":{"1min":0.28299999999999997,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.4590000000000001,"5min":1.496,"15min":1.627},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.92800000000000005,"15min":0.92300000000000004},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.2210000000000001,"5min":1.595,"15min":1.7250000000000001},"last":0.93899999999999995}]},{"osd":5,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.94499999999999995,"15min":0.92900000000000005},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.952},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.995,"5min":0.97899999999999998,"15min":0.97199999999999998},"min":{"1min":0.249,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.919,"5min":1.919,"15min":1.919},"last":0.99099999999999999}]},{"osd":6,"last update":"Sun Jul 11 04:48:00 2021","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.042,"15min":1.0429999999999999},"min":{"1min":0.29799999999999999,"5min":0.247,"15min":0.247},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.0069999999999999,"15min":1.024},"min":{"1min":0.27700000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.5860000000000001,"5min":1.724,"15min":1.8129999999999999},"last":0.85699999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050616,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321224,"statfs":{"total":96636764160,"available":95560933376,"internally_reserved":1073741824,"allocated":2080768,"data_stored":966194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.262,"15min":1.244},"min":{"1min":0.84099999999999997,"5min":0.55900000000000005,"15min":0.31900000000000001},"max":{"1min":1.6060000000000001,"5min":1.79,"15min":1.976},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.331,"5min":1.2829999999999999,"15min":1.2589999999999999},"min":{"1min":0.92400000000000004,"5min":0.57599999999999996,"15min":0.36099999999999999},"max":{"1min":1.5069999999999999,"5min":1.821,"15min":1.849},"last":1.282}]},{"osd":1,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.236,"15min":1.2250000000000001},"min":{"1min":0.93700000000000006,"5min":0.60999999999999999,"15min":0.28100000000000003},"max":{"1min":1.631,"5min":1.873,"15min":1.873},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.2729999999999999,"15min":1.2370000000000001},"min":{"1min":0.90600000000000003,"5min":0.58599999999999997,"15min":0.27600000000000002},"max":{"1min":1.653,"5min":1.786,"15min":1.786},"last":1.4239999999999999}]},{"osd":2,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2549999999999999,"15min":1.256},"min":{"1min":0.87,"5min":0.622,"15min":0.33200000000000002},"max":{"1min":1.7889999999999999,"5min":1.7889999999999999,"15min":2.0169999999999999},"last":1.351},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2569999999999999,"15min":1.2370000000000001},"min":{"1min":0.86199999999999999,"5min":0.63100000000000001,"15min":0.29499999999999998},"max":{"1min":1.53,"5min":1.6439999999999999,"15min":1.7789999999999999},"last":1.129}]},{"osd":3,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.337,"15min":1.3100000000000001},"min":{"1min":0.93000000000000005,"5min":0.59999999999999998,"15min":0.29999999999999999},"max":{"1min":1.6890000000000001,"5min":1.8320000000000001,"15min":1.8460000000000001},"last":1.512},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.375,"15min":1.3320000000000001},"min":{"1min":0.90000000000000002,"5min":0.59599999999999997,"15min":0.34999999999999998},"max":{"1min":1.921,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.397}]},{"osd":4,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.92400000000000004,"15min":0.90400000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.17799999999999999},"max":{"1min":1.4770000000000001,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.0900000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.95799999999999996,"15min":0.92400000000000004},"min":{"1min":0.216,"5min":0.216,"15min":0.183},"max":{"1min":1.8680000000000001,"5min":1.913,"15min":1.913},"last":1.0349999999999999}]},{"osd":5,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.93200000000000005,"15min":0.91900000000000004},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.17599999999999999},"max":{"1min":1.6020000000000001,"5min":1.6879999999999999,"15min":1.7789999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.044,"5min":1.012,"15min":0.98299999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.16},"max":{"1min":1.7430000000000001,"5min":1.871,"15min":1.9019999999999999},"last":0.89400000000000002}]},{"osd":7,"last update":"Sun Jul 11 04:47:55 2021","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.05,"15min":1.002},"min":{"1min":0.24299999999999999,"5min":0.218,"15min":0.19800000000000001},"max":{"1min":1.883,"5min":2.024,"15min":2.024},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.0649999999999999,"15min":1.0209999999999999},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":0.99199999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542813,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.32700000000000001,"5min":0.246,"15min":0.246},"max":{"1min":1.639,"5min":1.8,"15min":1.843},"last":1.371},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.337,"15min":1.3120000000000001},"min":{"1min":0.38200000000000001,"5min":0.317,"15min":0.317},"max":{"1min":1.5820000000000001,"5min":1.8100000000000001,"15min":1.96},"last":1.5900000000000001}]},{"osd":1,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.252,"15min":1.2390000000000001},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.48,"5min":1.7270000000000001,"15min":1.9359999999999999},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.238,"15min":1.2470000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.5189999999999999,"5min":1.716,"15min":2.0430000000000001},"last":1.748}]},{"osd":2,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.2450000000000001,"15min":1.2290000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.482,"5min":1.8169999999999999,"15min":1.837},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.242,"15min":1.2430000000000001},"min":{"1min":0.375,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.508,"5min":1.6890000000000001,"15min":1.843},"last":1.165}]},{"osd":3,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.232,"15min":1.246},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.421,"5min":1.907,"15min":1.907},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.264,"15min":1.2310000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.378},"max":{"1min":1.3620000000000001,"5min":1.99,"15min":1.99},"last":1.4550000000000001}]},{"osd":4,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":1.4019999999999999,"5min":1.595,"15min":1.657},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.93600000000000005,"15min":0.92000000000000004},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.21299999999999999},"max":{"1min":1.2949999999999999,"5min":1.855,"15min":1.855},"last":0.79700000000000004}]},{"osd":6,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.94999999999999996,"15min":0.92500000000000004},"min":{"1min":0.249,"5min":0.249,"15min":0.23400000000000001},"max":{"1min":1.1950000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.002,"15min":0.97699999999999998},"min":{"1min":0.26800000000000002,"5min":0.218,"15min":0.218},"max":{"1min":1.381,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.92300000000000004}]},{"osd":7,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.98999999999999999,"15min":0.97399999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.363,"5min":1.7649999999999999,"15min":1.9219999999999999},"last":0.874},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.0800000000000001,"15min":1.028},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.22},"max":{"1min":1.4850000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.1000000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542814,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050616,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321224,"statfs":{"total":96636764160,"available":95560933376,"internally_reserved":1073741824,"allocated":2080768,"data_stored":966194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3220000000000001,"15min":1.2969999999999999},"min":{"1min":1.0329999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":1.204},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3100000000000001,"15min":1.2909999999999999},"min":{"1min":1.0569999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.732,"5min":1.798,"15min":1.8799999999999999},"last":1.2989999999999999}]},{"osd":1,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.236,"15min":1.224},"min":{"1min":0.91600000000000004,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.641,"5min":1.6759999999999999,"15min":1.766},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.218,"15min":1.234},"min":{"1min":0.89800000000000002,"5min":0.377,"15min":0.35899999999999999},"max":{"1min":1.6279999999999999,"5min":1.7230000000000001,"15min":1.79},"last":1.2869999999999999}]},{"osd":2,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2769999999999999,"15min":1.26},"min":{"1min":0.99199999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.506,"5min":1.6579999999999999,"15min":1.8129999999999999},"last":1.395},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.272,"15min":1.2729999999999999},"min":{"1min":1.0229999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.6679999999999999,"5min":1.758,"15min":1.758},"last":1.4339999999999999}]},{"osd":3,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.2070000000000001,"15min":1.22},"min":{"1min":0.92600000000000005,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.728},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.248,"15min":1.2430000000000001},"min":{"1min":0.97499999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.5649999999999999,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.369}]},{"osd":5,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.84499999999999997,"15min":0.85399999999999998},"min":{"1min":0.55500000000000005,"5min":0.24099999999999999,"15min":0.22600000000000001},"max":{"1min":1.2989999999999999,"5min":1.5720000000000001,"15min":1.6459999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.92500000000000004,"5min":0.91000000000000003,"15min":0.90000000000000002},"min":{"1min":0.63600000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.143,"5min":1.5109999999999999,"15min":1.6819999999999999},"last":1.089}]},{"osd":6,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89600000000000002,"15min":0.91200000000000003},"min":{"1min":0.57699999999999996,"5min":0.27600000000000002,"15min":0.216},"max":{"1min":1.2929999999999999,"5min":1.613,"15min":1.7749999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.95699999999999996,"15min":0.96399999999999997},"min":{"1min":0.69099999999999995,"5min":0.22,"15min":0.22},"max":{"1min":1.1539999999999999,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.82299999999999995}]},{"osd":7,"last update":"Sun Jul 11 04:47:59 2021","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.99099999999999999},"min":{"1min":0.52900000000000003,"5min":0.26400000000000001,"15min":0.23100000000000001},"max":{"1min":1.3660000000000001,"5min":1.5860000000000001,"15min":1.7330000000000001},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0649999999999999,"15min":1.026},"min":{"1min":0.61699999999999999,"5min":0.26600000000000001,"15min":0.22},"max":{"1min":1.3260000000000001,"5min":1.6950000000000001,"15min":1.782},"last":1.1499999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542813,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0509999999999999,"5min":1.133,"15min":1.4359999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.78000000000000003,"15min":0.79900000000000004},"min":{"1min":0.28599999999999998,"5min":0.221,"15min":0.221},"max":{"1min":1.1240000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.87}]},{"osd":1,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.755,"15min":0.77000000000000002},"min":{"1min":0.29599999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.94999999999999996,"5min":1.0700000000000001,"15min":1.081},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81499999999999995,"15min":0.82199999999999995},"min":{"1min":0.246,"5min":0.246,"15min":0.23599999999999999},"max":{"1min":1.085,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.0629999999999999}]},{"osd":2,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.81899999999999995,"15min":0.81699999999999995},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.216},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.625},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.876,"15min":0.88200000000000001},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":1.143,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.91200000000000003}]},{"osd":4,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3500000000000001,"15min":1.337},"min":{"1min":0.91800000000000004,"5min":0.72099999999999997,"15min":0.36399999999999999},"max":{"1min":1.7370000000000001,"5min":1.7989999999999999,"15min":1.859},"last":1.8680000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.3120000000000001,"15min":1.3380000000000001},"min":{"1min":0.88400000000000001,"5min":0.68300000000000005,"15min":0.34799999999999998},"max":{"1min":1.712,"5min":1.873,"15min":1.873},"last":1.1870000000000001}]},{"osd":5,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.385,"15min":1.3799999999999999},"min":{"1min":0.88,"5min":0.71999999999999997,"15min":0.42899999999999999},"max":{"1min":1.6459999999999999,"5min":1.7789999999999999,"15min":1.875},"last":1.351},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.359,"15min":1.3660000000000001},"min":{"1min":0.86699999999999999,"5min":0.66700000000000004,"15min":0.436},"max":{"1min":1.702,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.528}]},{"osd":6,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3200000000000001,"15min":1.3109999999999999},"min":{"1min":0.89900000000000002,"5min":0.68700000000000006,"15min":0.49199999999999999},"max":{"1min":1.546,"5min":1.6830000000000001,"15min":1.6859999999999999},"last":1.23},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.321,"15min":1.3420000000000001},"min":{"1min":0.90400000000000003,"5min":0.67400000000000004,"15min":0.33700000000000002},"max":{"1min":1.502,"5min":1.7090000000000001,"15min":1.905},"last":1.4350000000000001}]},{"osd":7,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.327,"15min":1.339},"min":{"1min":0.80700000000000005,"5min":0.70099999999999996,"15min":0.44700000000000001},"max":{"1min":1.7330000000000001,"5min":1.855,"15min":1.8700000000000001},"last":1.734},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.347,"15min":1.361},"min":{"1min":0.85999999999999999,"5min":0.64600000000000002,"15min":0.39900000000000002},"max":{"1min":1.7070000000000001,"5min":1.7849999999999999,"15min":1.7869999999999999},"last":1.645}]}]},{"osd":2,"up_from":14,"seq":60129542814,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.83499999999999996,"15min":0.81999999999999995},"min":{"1min":0.58499999999999996,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":1.2430000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.88200000000000001,"15min":0.86299999999999999},"min":{"1min":0.61099999999999999,"5min":0.27300000000000002,"15min":0.252},"max":{"1min":1.157,"5min":1.417,"15min":1.417},"last":1.1899999999999999}]},{"osd":1,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.44400000000000001,"5min":0.245,"15min":0.245},"max":{"1min":1.083,"5min":1.3500000000000001,"15min":1.5469999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.83499999999999996,"15min":0.82099999999999995},"min":{"1min":0.52100000000000002,"5min":0.26900000000000002,"15min":0.23799999999999999},"max":{"1min":1.089,"5min":1.2849999999999999,"15min":1.3},"last":1.0529999999999999}]},{"osd":3,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80500000000000005,"15min":0.79300000000000004},"min":{"1min":0.41799999999999998,"5min":0.23599999999999999,"15min":0.22900000000000001},"max":{"1min":1.0349999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.89500000000000002,"15min":0.88200000000000001},"min":{"1min":0.49099999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2010000000000001,"5min":1.375,"15min":1.573},"last":0.95699999999999996}]},{"osd":4,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3160000000000001,"15min":1.296},"min":{"1min":0.86299999999999999,"5min":0.66300000000000003,"15min":0.44800000000000001},"max":{"1min":1.538,"5min":1.794,"15min":1.877},"last":1.484},{"interface":"front","average":{"1min":1.417,"5min":1.395,"15min":1.3540000000000001},"min":{"1min":0.73899999999999999,"5min":0.623,"15min":0.48599999999999999},"max":{"1min":1.7210000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.4399999999999999}]},{"osd":5,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3640000000000001,"15min":1.335},"min":{"1min":0.82399999999999995,"5min":0.67200000000000004,"15min":0.441},"max":{"1min":1.6319999999999999,"5min":1.966,"15min":1.966},"last":1.456},{"interface":"front","average":{"1min":1.371,"5min":1.3580000000000001,"15min":1.335},"min":{"1min":0.90200000000000002,"5min":0.63800000000000001,"15min":0.41499999999999998},"max":{"1min":1.5820000000000001,"5min":1.7829999999999999,"15min":1.956},"last":1.6599999999999999}]},{"osd":6,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.383,"15min":1.3560000000000001},"min":{"1min":0.80300000000000005,"5min":0.64400000000000002,"15min":0.52600000000000002},"max":{"1min":1.667,"5min":1.794,"15min":1.8360000000000001},"last":1.534},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.357},"min":{"1min":0.78600000000000003,"5min":0.58499999999999996,"15min":0.47599999999999998},"max":{"1min":1.7130000000000001,"5min":1.9079999999999999,"15min":2.0550000000000002},"last":1.5640000000000001}]},{"osd":7,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3779999999999999,"15min":1.3560000000000001},"min":{"1min":0.76800000000000002,"5min":0.71599999999999997,"15min":0.47199999999999998},"max":{"1min":1.6040000000000001,"5min":1.992,"15min":1.992},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3759999999999999,"15min":1.3460000000000001},"min":{"1min":0.85299999999999998,"5min":0.65300000000000002,"15min":0.53500000000000003},"max":{"1min":1.6899999999999999,"5min":1.8149999999999999,"15min":1.964},"last":1.726}]}]},{"osd":1,"up_from":14,"seq":60129542814,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78900000000000003,"15min":0.80000000000000004},"min":{"1min":0.32000000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.026,"5min":1.1459999999999999,"15min":1.2709999999999999},"last":0.874},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83999999999999997,"15min":0.85399999999999998},"min":{"1min":0.374,"5min":0.247,"15min":0.22600000000000001},"max":{"1min":1.125,"5min":1.1519999999999999,"15min":1.1870000000000001},"last":0.95499999999999996}]},{"osd":2,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.79800000000000004},"min":{"1min":0.35499999999999998,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":1.05,"5min":1.151,"15min":1.216},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.38,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":1.02,"5min":1.234,"15min":1.234},"last":0.75700000000000001}]},{"osd":3,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.83299999999999996,"15min":0.85899999999999999},"min":{"1min":0.34699999999999998,"5min":0.23999999999999999,"15min":0.23100000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.3600000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.40699999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.95699999999999996,"5min":1.2050000000000001,"15min":1.258},"last":0.70999999999999996}]},{"osd":4,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3120000000000001,"15min":1.3169999999999999},"min":{"1min":0.73299999999999998,"5min":0.51900000000000002,"15min":0.29599999999999999},"max":{"1min":1.607,"5min":1.887,"15min":1.887},"last":1.347},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.331,"15min":1.3200000000000001},"min":{"1min":0.67900000000000005,"5min":0.48799999999999999,"15min":0.28899999999999998},"max":{"1min":1.6879999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.4730000000000001}]},{"osd":5,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3380000000000001,"15min":1.3260000000000001},"min":{"1min":0.65700000000000003,"5min":0.46700000000000003,"15min":0.25900000000000001},"max":{"1min":1.776,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.3180000000000001,"15min":1.304},"min":{"1min":0.748,"5min":0.432,"15min":0.26400000000000001},"max":{"1min":1.7350000000000001,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.321}]},{"osd":6,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4079999999999999,"15min":1.3919999999999999},"min":{"1min":0.80700000000000005,"5min":0.44700000000000001,"15min":0.313},"max":{"1min":1.9199999999999999,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3859999999999999,"15min":1.387},"min":{"1min":0.66800000000000004,"5min":0.498,"15min":0.30599999999999999},"max":{"1min":1.6599999999999999,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.6599999999999999}]},{"osd":7,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.304,"15min":1.3160000000000001},"min":{"1min":0.67300000000000004,"5min":0.40999999999999998,"15min":0.34599999999999997},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.264},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3320000000000001,"15min":1.333},"min":{"1min":0.627,"5min":0.55300000000000005,"15min":0.27100000000000002},"max":{"1min":1.8129999999999999,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.431}]}]},{"osd":0,"up_from":14,"seq":60129542812,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050612,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321228,"statfs":{"total":96636764160,"available":95560937472,"internally_reserved":1073741824,"allocated":2076672,"data_stored":965616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76400000000000001,"15min":0.78400000000000003},"min":{"1min":0.25600000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.94299999999999995,"5min":1.099,"15min":1.1890000000000001},"last":1.038},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.26300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":1.1379999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.99299999999999999}]},{"osd":2,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81000000000000005,"15min":0.81899999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.21099999999999999},"max":{"1min":1.0800000000000001,"5min":1.129,"15min":1.2789999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.90000000000000002,"15min":0.91900000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.253},"max":{"1min":1.0720000000000001,"5min":1.252,"15min":1.4219999999999999},"last":0.79900000000000004}]},{"osd":3,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73399999999999999,"15min":0.76300000000000001},"min":{"1min":0.33400000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.454},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.219},"max":{"1min":1.095,"5min":1.1619999999999999,"15min":1.339},"last":1.0820000000000001}]},{"osd":4,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.329,"15min":1.3620000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.6359999999999999,"5min":1.7470000000000001,"15min":2.1030000000000002},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.331,"15min":1.343},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":1.6499999999999999,"5min":1.72,"15min":1.8899999999999999},"last":1.2909999999999999}]},{"osd":5,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.321,"15min":1.3320000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.6459999999999999,"5min":1.8280000000000001,"15min":1.9399999999999999},"last":1.264},{"interface":"front","average":{"1min":1.357,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.33500000000000002},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":2.0289999999999999},"last":1.2549999999999999}]},{"osd":6,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3069999999999999,"15min":1.347},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6950000000000001,"5min":1.7929999999999999,"15min":1.944},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2949999999999999,"15min":1.3240000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.5649999999999999,"5min":1.6950000000000001,"15min":1.8580000000000001},"last":1.337}]},{"osd":7,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.335,"15min":1.353},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6479999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.349,"15min":1.363},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.39300000000000002},"max":{"1min":1.6779999999999999,"5min":1.8169999999999999,"15min":1.9199999999999999},"last":1.2070000000000001}]}]}]}} 2021-07-11T04:48:49.791 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2021-07-11T03:53:19.125309+0000 time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=3, tm_min=53, tm_sec=19, tm_wday=6, tm_yday=192, tm_isdst=-1) <= time.struct_time(tm_year=2021, tm_mon=7, tm_mday=11, tm_hour=4, tm_min=48, tm_sec=26, tm_wday=6, tm_yday=192, tm_isdst=0) 2021-07-11T04:48:49.792 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2021-07-11T04:48:49.857 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:48:49.860 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:48:49.863 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:48:49.867 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:09.794 DEBUG:teuthology.orchestra.run.gibba004:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2021-07-11T04:49:09.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f47031ca700 1 Processor -- start 2021-07-11T04:49:09.849 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f47031ca700 1 -- start start 2021-07-11T04:49:09.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f47031ca700 1 --2- >> v2:172.21.2.104:3300/0 conn(0x7f46fc0a76b0 0x7f46fc066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:09.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f47031ca700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f46fc0b1970 0x7f46fc066f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:09.850 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f47031ca700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46fc067450 con 0x7f46fc0b0060 2021-07-11T04:49:09.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f47031ca700 1 -- --> v2:172.21.2.104:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc0a7a90 con 0x7f46fc0a76b0 2021-07-11T04:49:09.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f47031ca700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46fc147620 con 0x7f46fc0b1970 2021-07-11T04:49:09.851 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f4701767700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f46fc0b0060 0x7f46fc05f6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49332/0 (socket says 172.21.2.104:49332) 2021-07-11T04:49:09.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f4701767700 1 -- 172.21.2.104:0/117647706 learned_addr learned my addr 172.21.2.104:0/117647706 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:49:09.852 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f4700f66700 1 --2- 172.21.2.104:0/117647706 >> v2:172.21.2.104:3300/0 conn(0x7f46fc0a76b0 0x7f46fc066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:49:09.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f46fb7fe700 1 -- 172.21.2.104:0/117647706 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77148650 0 0) 0x7f46fc067450 con 0x7f46fc0b0060 2021-07-11T04:49:09.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f46fb7fe700 1 -- 172.21.2.104:0/117647706 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46dc003260 con 0x7f46fc0b0060 2021-07-11T04:49:09.856 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f4700f66700 1 -- 172.21.2.104:0/117647706 >> v2:172.21.2.105:3300/0 conn(0x7f46fc0b1970 msgr2=0x7f46fc066f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:49:09.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f4700f66700 1 --2- 172.21.2.104:0/117647706 >> v2:172.21.2.105:3300/0 conn(0x7f46fc0b1970 0x7f46fc066f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:49:09.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f4700f66700 1 -- 172.21.2.104:0/117647706 >> v1:172.21.2.104:6789/0 conn(0x7f46fc0b0060 legacy=0x7f46fc05f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:49:09.857 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f4700f66700 1 -- 172.21.2.104:0/117647706 --> v2:172.21.2.104:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46fc1478c0 con 0x7f46fc0a76b0 2021-07-11T04:49:09.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.844+0000 7f4700f66700 1 --2- 172.21.2.104:0/117647706 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46fc0a76b0 0x7f46fc066a70 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 rx=0x7f46e4002f70 tx=0x7f46e4006df0).ready entity=mon.0 client_cookie=90de6ab330e6d0e2 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:49:09.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f46fb7fe700 1 -- 172.21.2.104:0/117647706 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46e400d570 con 0x7f46fc0a76b0 2021-07-11T04:49:09.858 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f46fb7fe700 1 -- 172.21.2.104:0/117647706 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e4000e90 con 0x7f46fc0a76b0 2021-07-11T04:49:09.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- 172.21.2.104:0/117647706 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46fc0a76b0 msgr2=0x7f46fc066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:49:09.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f46fb7fe700 1 -- 172.21.2.104:0/117647706 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46e400d8c0 con 0x7f46fc0a76b0 2021-07-11T04:49:09.859 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 --2- 172.21.2.104:0/117647706 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46fc0a76b0 0x7f46fc066a70 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 rx=0x7f46e4002f70 tx=0x7f46e4006df0).stop 2021-07-11T04:49:09.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- 172.21.2.104:0/117647706 shutdown_connections 2021-07-11T04:49:09.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 --2- 172.21.2.104:0/117647706 >> v2:172.21.2.105:3300/0 conn(0x7f46fc0b1970 0x7f46fc066f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:49:09.860 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 --2- 172.21.2.104:0/117647706 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46fc0a76b0 0x7f46fc066a70 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:49:09.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- 172.21.2.104:0/117647706 >> 172.21.2.104:0/117647706 conn(0x7f46fc0b53e0 msgr2=0x7f46fc0b57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:49:09.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- 172.21.2.104:0/117647706 shutdown_connections 2021-07-11T04:49:09.861 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- 172.21.2.104:0/117647706 wait complete. 2021-07-11T04:49:09.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 Processor -- start 2021-07-11T04:49:09.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- start start 2021-07-11T04:49:09.862 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46fc0a76b0 0x7f46fc0aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:09.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46fc0b0060 0x7f46fc156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:09.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46fc0b1970 0x7f46fc156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:09.863 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0ab650 con 0x7f46fc0a76b0 2021-07-11T04:49:09.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0ab8f0 con 0x7f46fc0b1970 2021-07-11T04:49:09.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46fc0abb90 con 0x7f46fc0b0060 2021-07-11T04:49:09.864 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f4700f66700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46fc0a76b0 0x7f46fc0aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:49:09.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f4700f66700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46fc0a76b0 0x7f46fc0aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52570/0 (socket says 172.21.2.104:52570) 2021-07-11T04:49:09.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f4700f66700 1 -- 172.21.2.104:0/532499947 learned_addr learned my addr 172.21.2.104:0/532499947 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:49:09.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f4700f66700 1 -- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46fc0b0060 msgr2=0x7f46fc156ae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:49:09.865 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f4700f66700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46fc0b0060 0x7f46fc156ae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:49:09.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f4700f66700 1 -- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46fc0b1970 msgr2=0x7f46fc156fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2021-07-11T04:49:09.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f4700f66700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46fc0b1970 0x7f46fc156fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:49:09.866 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f4700f66700 1 -- 172.21.2.104:0/532499947 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e400b180 con 0x7f46fc0a76b0 2021-07-11T04:49:09.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f4700f66700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46fc0a76b0 0x7f46fc0aa4c0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 rx=0x7f46e4002f70 tx=0x7f46e4014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:49:09.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f46f9ffb700 1 -- 172.21.2.104:0/532499947 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46e400de20 con 0x7f46fc0a76b0 2021-07-11T04:49:09.867 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f46f9ffb700 1 -- 172.21.2.104:0/532499947 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e401e430 con 0x7f46fc0a76b0 2021-07-11T04:49:09.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f46f9ffb700 1 -- 172.21.2.104:0/532499947 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f46e401e790 con 0x7f46fc0a76b0 2021-07-11T04:49:09.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- 172.21.2.104:0/532499947 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46fc156640 con 0x7f46fc0a76b0 2021-07-11T04:49:09.868 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- 172.21.2.104:0/532499947 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46fc0ae1c0 con 0x7f46fc0a76b0 2021-07-11T04:49:09.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f47031ca700 1 -- 172.21.2.104:0/532499947 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc057350 con 0x7f46fc0a76b0 2021-07-11T04:49:09.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f46f9ffb700 1 -- 172.21.2.104:0/532499947 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f46e400f030 con 0x7f46fc0a76b0 2021-07-11T04:49:09.869 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f46f9ffb700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46e00531a0 0x7f46e0055660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:09.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.848+0000 7f46f9ffb700 1 -- 172.21.2.104:0/532499947 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f46e406c280 con 0x7f46fc0a76b0 2021-07-11T04:49:09.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.852+0000 7f46fbfff700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46e00531a0 0x7f46e0055660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:49:09.870 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.852+0000 7f46fbfff700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46e00531a0 0x7f46e0055660 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f46ec002f70 tx=0x7f46ec006960).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:49:09.871 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.852+0000 7f46f9ffb700 1 -- 172.21.2.104:0/532499947 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f46e4043840 con 0x7f46fc0a76b0 2021-07-11T04:49:09.936 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.932+0000 7f47031ca700 1 -- 172.21.2.104:0/532499947 --> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f46fc0698e0 con 0x7f46e00531a0 2021-07-11T04:49:09.942 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f46f9ffb700 1 -- 172.21.2.104:0/532499947 <== mgr.4100 v2:172.21.2.105:6800/15271 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+66357 (secure 0 0 0) 0x7f46fc0698e0 con 0x7f46e00531a0 2021-07-11T04:49:09.944 INFO:teuthology.orchestra.run.gibba004.stdout: 2021-07-11T04:49:09.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 -- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46e00531a0 msgr2=0x7f46e0055660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:49:09.945 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46e00531a0 0x7f46e0055660 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f46ec002f70 tx=0x7f46ec006960).stop 2021-07-11T04:49:09.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 -- 172.21.2.104:0/532499947 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46fc0a76b0 msgr2=0x7f46fc0aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:49:09.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46fc0a76b0 0x7f46fc0aa4c0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 rx=0x7f46e4002f70 tx=0x7f46e4014000).stop 2021-07-11T04:49:09.946 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 -- 172.21.2.104:0/532499947 shutdown_connections 2021-07-11T04:49:09.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f46fc0b1970 0x7f46fc156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:49:09.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f46fc0b0060 0x7f46fc156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:49:09.947 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f46e00531a0 0x7f46e0055660 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:49:09.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 --2- 172.21.2.104:0/532499947 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f46fc0a76b0 0x7f46fc0aa4c0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:49:09.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 -- 172.21.2.104:0/532499947 >> 172.21.2.104:0/532499947 conn(0x7f46fc0b53e0 msgr2=0x7f46fc0a7e40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:49:09.948 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 -- 172.21.2.104:0/532499947 shutdown_connections 2021-07-11T04:49:09.949 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:09.940+0000 7f47031ca700 1 -- 172.21.2.104:0/532499947 wait complete. 2021-07-11T04:49:09.949 INFO:teuthology.orchestra.run.gibba004.stderr:dumped all 2021-07-11T04:49:09.950 INFO:teuthology.orchestra.run.gibba004.stdout:{"pg_ready":true,"pg_map":{"version":1834,"stamp":"2021-07-11T04:49:09.837763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9833,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":8404944,"kb_used_data":16272,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746569776,"statfs":{"total":773094113280,"available":764487450624,"internally_reserved":8589934592,"allocated":16662528,"data_stored":7731989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005949"},"pg_stats":[{"pgid":"1.0","version":"164'4","reported_seq":"164","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:42.401398+0000","last_change":"2021-07-11T04:48:42.401658+0000","last_active":"2021-07-11T04:48:42.401398+0000","last_peered":"2021-07-11T04:48:42.401398+0000","last_clean":"2021-07-11T04:48:42.401398+0000","last_became_active":"2021-07-11T03:53:15.962801+0000","last_became_peered":"2021-07-11T03:53:15.962801+0000","last_unstale":"2021-07-11T04:48:42.401398+0000","last_undegraded":"2021-07-11T04:48:42.401398+0000","last_fullsized":"2021-07-11T04:48:42.401398+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"164'4","last_scrub_stamp":"2021-07-11T04:48:42.401397+0000","last_deep_scrub":"164'4","last_deep_scrub_stamp":"2021-07-11T04:48:42.401397+0000","last_clean_scrub_stamp":"2021-07-11T04:48:42.401397+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9833,"num_omap_keys":2,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"141","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:29.961462+0000","last_change":"2021-07-11T04:48:29.961750+0000","last_active":"2021-07-11T04:48:29.961462+0000","last_peered":"2021-07-11T04:48:29.961462+0000","last_clean":"2021-07-11T04:48:29.961462+0000","last_became_active":"2021-07-11T03:53:20.153135+0000","last_became_peered":"2021-07-11T03:53:20.153135+0000","last_unstale":"2021-07-11T04:48:29.961462+0000","last_undegraded":"2021-07-11T04:48:29.961462+0000","last_fullsized":"2021-07-11T04:48:29.961462+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:29.961461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:29.961461+0000","last_clean_scrub_stamp":"2021-07-11T04:48:29.961461+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"157","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:45.481818+0000","last_change":"2021-07-11T04:48:45.482186+0000","last_active":"2021-07-11T04:48:45.481818+0000","last_peered":"2021-07-11T04:48:45.481818+0000","last_clean":"2021-07-11T04:48:45.481818+0000","last_became_active":"2021-07-11T03:53:20.152282+0000","last_became_peered":"2021-07-11T03:53:20.152282+0000","last_unstale":"2021-07-11T04:48:45.481818+0000","last_undegraded":"2021-07-11T04:48:45.481818+0000","last_fullsized":"2021-07-11T04:48:45.481818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:45.481817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:45.481817+0000","last_clean_scrub_stamp":"2021-07-11T04:48:45.481817+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"145","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:32.137673+0000","last_change":"2021-07-11T04:48:32.137727+0000","last_active":"2021-07-11T04:48:32.137673+0000","last_peered":"2021-07-11T04:48:32.137673+0000","last_clean":"2021-07-11T04:48:32.137673+0000","last_became_active":"2021-07-11T03:53:20.153695+0000","last_became_peered":"2021-07-11T03:53:20.153695+0000","last_unstale":"2021-07-11T04:48:32.137673+0000","last_undegraded":"2021-07-11T04:48:32.137673+0000","last_fullsized":"2021-07-11T04:48:32.137673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:32.137673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:32.137673+0000","last_clean_scrub_stamp":"2021-07-11T04:48:32.137673+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"141","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:28.999475+0000","last_change":"2021-07-11T04:48:28.999543+0000","last_active":"2021-07-11T04:48:28.999475+0000","last_peered":"2021-07-11T04:48:28.999475+0000","last_clean":"2021-07-11T04:48:28.999475+0000","last_became_active":"2021-07-11T03:53:20.153235+0000","last_became_peered":"2021-07-11T03:53:20.153235+0000","last_unstale":"2021-07-11T04:48:28.999475+0000","last_undegraded":"2021-07-11T04:48:28.999475+0000","last_fullsized":"2021-07-11T04:48:28.999475+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:28.999474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:28.999474+0000","last_clean_scrub_stamp":"2021-07-11T04:48:28.999474+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"139","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:38.839037+0000","last_change":"2021-07-11T04:48:38.839082+0000","last_active":"2021-07-11T04:48:38.839037+0000","last_peered":"2021-07-11T04:48:38.839037+0000","last_clean":"2021-07-11T04:48:38.839037+0000","last_became_active":"2021-07-11T03:53:20.286695+0000","last_became_peered":"2021-07-11T03:53:20.286695+0000","last_unstale":"2021-07-11T04:48:38.839037+0000","last_undegraded":"2021-07-11T04:48:38.839037+0000","last_fullsized":"2021-07-11T04:48:38.839037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:38.839037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:38.839037+0000","last_clean_scrub_stamp":"2021-07-11T04:48:38.839037+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"157","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:46.541886+0000","last_change":"2021-07-11T04:48:46.541939+0000","last_active":"2021-07-11T04:48:46.541886+0000","last_peered":"2021-07-11T04:48:46.541886+0000","last_clean":"2021-07-11T04:48:46.541886+0000","last_became_active":"2021-07-11T03:53:20.287190+0000","last_became_peered":"2021-07-11T03:53:20.287190+0000","last_unstale":"2021-07-11T04:48:46.541886+0000","last_undegraded":"2021-07-11T04:48:46.541886+0000","last_fullsized":"2021-07-11T04:48:46.541886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:46.541886+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:46.541886+0000","last_clean_scrub_stamp":"2021-07-11T04:48:46.541886+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"139","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:40.801640+0000","last_change":"2021-07-11T04:48:40.801712+0000","last_active":"2021-07-11T04:48:40.801640+0000","last_peered":"2021-07-11T04:48:40.801640+0000","last_clean":"2021-07-11T04:48:40.801640+0000","last_became_active":"2021-07-11T03:53:20.153186+0000","last_became_peered":"2021-07-11T03:53:20.153186+0000","last_unstale":"2021-07-11T04:48:40.801640+0000","last_undegraded":"2021-07-11T04:48:40.801640+0000","last_fullsized":"2021-07-11T04:48:40.801640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:40.801639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:40.801639+0000","last_clean_scrub_stamp":"2021-07-11T04:48:40.801639+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"139","reported_epoch":"164","state":"active+clean","last_fresh":"2021-07-11T04:48:29.862634+0000","last_change":"2021-07-11T04:48:29.862689+0000","last_active":"2021-07-11T04:48:29.862634+0000","last_peered":"2021-07-11T04:48:29.862634+0000","last_clean":"2021-07-11T04:48:29.862634+0000","last_became_active":"2021-07-11T03:53:20.153046+0000","last_became_peered":"2021-07-11T03:53:20.153046+0000","last_unstale":"2021-07-11T04:48:29.862634+0000","last_undegraded":"2021-07-11T04:48:29.862634+0000","last_fullsized":"2021-07-11T04:48:29.862634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-07-11T04:48:29.862634+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-07-11T04:48:29.862634+0000","last_clean_scrub_stamp":"2021-07-11T04:48:29.862634+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9833,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542817,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:49:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.206,"15min":1.2370000000000001},"min":{"1min":0.70899999999999996,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.8620000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.252,"15min":1.2330000000000001},"min":{"1min":0.67300000000000004,"5min":0.34799999999999998,"15min":0.313},"max":{"1min":1.669,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Sun Jul 11 04:49:01 2021","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.242,"15min":1.2490000000000001},"min":{"1min":0.66200000000000003,"5min":0.34000000000000002,"15min":0.33700000000000002},"max":{"1min":1.542,"5min":1.847,"15min":1.847},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.288,"5min":1.2310000000000001,"15min":1.236},"min":{"1min":0.65500000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8400000000000001},"last":0.65500000000000003}]},{"osd":2,"last update":"Sun Jul 11 04:49:01 2021","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1699999999999999,"15min":1.1899999999999999},"min":{"1min":0.63900000000000001,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.266,"15min":1.2609999999999999},"min":{"1min":0.63200000000000001,"5min":0.307,"15min":0.307},"max":{"1min":1.657,"5min":1.835,"15min":1.849},"last":0.63200000000000001}]},{"osd":3,"last update":"Sun Jul 11 04:49:01 2021","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.2509999999999999,"15min":1.2470000000000001},"min":{"1min":0.65800000000000003,"5min":0.39200000000000002,"15min":0.34999999999999998},"max":{"1min":1.6160000000000001,"5min":1.8100000000000001,"15min":1.944},"last":0.65800000000000003},{"interface":"front","average":{"1min":1.298,"5min":1.304,"15min":1.339},"min":{"1min":0.65800000000000003,"5min":0.36499999999999999,"15min":0.34399999999999997},"max":{"1min":1.748,"5min":1.748,"15min":1.974},"last":0.68799999999999994}]},{"osd":4,"last update":"Sun Jul 11 04:49:01 2021","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.86699999999999999,"15min":0.86799999999999999},"min":{"1min":0.27300000000000002,"5min":0.246,"15min":0.23200000000000001},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.90100000000000002,"15min":0.91100000000000003},"min":{"1min":0.28199999999999997,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.21,"5min":1.595,"15min":1.7250000000000001},"last":0.28199999999999997}]},{"osd":5,"last update":"Sun Jul 11 04:49:01 2021","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.93000000000000005,"15min":0.92100000000000004},"min":{"1min":0.23999999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.952},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.95599999999999996,"15min":0.96699999999999997},"min":{"1min":0.25900000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.53,"5min":1.919,"15min":1.919},"last":0.29199999999999998}]},{"osd":6,"last update":"Sun Jul 11 04:49:01 2021","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0369999999999999,"15min":1.034},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.247},"max":{"1min":1.7,"5min":1.984,"15min":1.984},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.98399999999999999,"15min":1.006},"min":{"1min":0.249,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.657,"5min":1.724,"15min":1.8129999999999999},"last":0.308}]}]},{"osd":6,"up_from":14,"seq":60129542817,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050616,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321224,"statfs":{"total":96636764160,"available":95560933376,"internally_reserved":1073741824,"allocated":2080768,"data_stored":966194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:57 2021","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.26,"15min":1.2490000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.6040000000000001,"5min":1.6060000000000001,"15min":1.976},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2869999999999999,"15min":1.2549999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.577,"5min":1.821,"15min":1.849},"last":0.69399999999999995}]},{"osd":1,"last update":"Sun Jul 11 04:48:57 2021","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.264,"15min":1.23},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.7270000000000001,"5min":1.873,"15min":1.873},"last":1.018},{"interface":"front","average":{"1min":1.27,"5min":1.284,"15min":1.2430000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.5489999999999999,"5min":1.786,"15min":1.786},"last":1.079}]},{"osd":2,"last update":"Sun Jul 11 04:48:57 2021","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.276,"15min":1.2609999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.6639999999999999,"5min":1.7889999999999999,"15min":2.0169999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.258,"5min":1.272,"15min":1.238},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.591,"5min":1.6439999999999999,"15min":1.7789999999999999},"last":1.1220000000000001}]},{"osd":3,"last update":"Sun Jul 11 04:48:57 2021","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.339,"15min":1.3129999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.7130000000000001,"5min":1.8320000000000001,"15min":1.8460000000000001},"last":1.272},{"interface":"front","average":{"1min":1.343,"5min":1.383,"15min":1.333},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.78,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.052}]},{"osd":4,"last update":"Sun Jul 11 04:48:57 2021","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.93500000000000005,"15min":0.90900000000000003},"min":{"1min":0.28699999999999998,"5min":0.254,"15min":0.189},"max":{"1min":1.341,"5min":1.7889999999999999,"15min":1.7889999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.92800000000000005},"min":{"1min":0.30499999999999999,"5min":0.216,"15min":0.183},"max":{"1min":1.6279999999999999,"5min":1.913,"15min":1.913},"last":0.94299999999999995}]},{"osd":5,"last update":"Sun Jul 11 04:48:57 2021","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.94099999999999995,"15min":0.92300000000000004},"min":{"1min":0.35099999999999998,"5min":0.23200000000000001,"15min":0.17599999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.7789999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":1.012,"15min":0.98899999999999999},"min":{"1min":0.38200000000000001,"5min":0.223,"15min":0.16},"max":{"1min":1.484,"5min":1.871,"15min":1.9019999999999999},"last":0.97099999999999997}]},{"osd":7,"last update":"Sun Jul 11 04:48:57 2021","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":1.0760000000000001,"15min":1.01},"min":{"1min":0.33200000000000002,"5min":0.218,"15min":0.19800000000000001},"max":{"1min":1.9370000000000001,"5min":2.024,"15min":2.024},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.085,"15min":1.034},"min":{"1min":0.39000000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":1.2330000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542817,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.3120000000000001,"15min":1.3},"min":{"1min":0.32700000000000001,"5min":0.246,"15min":0.246},"max":{"1min":1.639,"5min":1.8,"15min":1.843},"last":0.28399999999999997},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.337,"15min":1.3120000000000001},"min":{"1min":0.38200000000000001,"5min":0.317,"15min":0.317},"max":{"1min":1.5820000000000001,"5min":1.8100000000000001,"15min":1.96},"last":0.28100000000000003}]},{"osd":1,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.252,"15min":1.2390000000000001},"min":{"1min":0.28999999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.48,"5min":1.7270000000000001,"15min":1.9359999999999999},"last":0.25900000000000001},{"interface":"front","average":{"1min":1.119,"5min":1.238,"15min":1.2470000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.5189999999999999,"5min":1.716,"15min":2.0430000000000001},"last":0.249}]},{"osd":2,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.2450000000000001,"15min":1.2290000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.482,"5min":1.8169999999999999,"15min":1.837},"last":0.28799999999999998},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.242,"15min":1.2430000000000001},"min":{"1min":0.375,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.508,"5min":1.6890000000000001,"15min":1.843},"last":0.30299999999999999}]},{"osd":3,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.232,"15min":1.246},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.421,"5min":1.907,"15min":1.907},"last":0.316},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.264,"15min":1.2310000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.378},"max":{"1min":1.3620000000000001,"5min":1.99,"15min":1.99},"last":0.29799999999999999}]},{"osd":4,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.89900000000000002,"15min":0.88300000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":1.4019999999999999,"5min":1.595,"15min":1.657},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.93600000000000005,"15min":0.92000000000000004},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.21299999999999999},"max":{"1min":1.2949999999999999,"5min":1.855,"15min":1.855},"last":0.221}]},{"osd":6,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.94999999999999996,"15min":0.92500000000000004},"min":{"1min":0.249,"5min":0.249,"15min":0.23400000000000001},"max":{"1min":1.1950000000000001,"5min":1.8029999999999999,"15min":1.8029999999999999},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.96999999999999997,"5min":1.002,"15min":0.97699999999999998},"min":{"1min":0.26800000000000002,"5min":0.218,"15min":0.218},"max":{"1min":1.381,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.29199999999999998}]},{"osd":7,"last update":"Sun Jul 11 04:48:28 2021","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.98999999999999999,"15min":0.97399999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":1.363,"5min":1.7649999999999999,"15min":1.9219999999999999},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.99099999999999999,"5min":1.0800000000000001,"15min":1.028},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.22},"max":{"1min":1.4850000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.307}]}]},{"osd":4,"up_from":14,"seq":60129542818,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":1050616,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321224,"statfs":{"total":96636764160,"available":95560933376,"internally_reserved":1073741824,"allocated":2080768,"data_stored":966194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3069999999999999,"15min":1.29},"min":{"1min":0.56999999999999995,"5min":0.54300000000000004,"15min":0.34399999999999997},"max":{"1min":1.7410000000000001,"5min":1.823,"15min":1.823},"last":1.724},{"interface":"front","average":{"1min":1.125,"5min":1.2929999999999999,"15min":1.282},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.38500000000000001},"max":{"1min":1.429,"5min":1.798,"15min":1.8799999999999999},"last":1.476}]},{"osd":1,"last update":"Sun Jul 11 04:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.24,"15min":1.22},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.35399999999999998},"max":{"1min":1.4990000000000001,"5min":1.6759999999999999,"15min":1.766},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.103,"5min":1.21,"15min":1.228},"min":{"1min":0.45900000000000002,"5min":0.40100000000000002,"15min":0.35899999999999999},"max":{"1min":1.482,"5min":1.7230000000000001,"15min":1.79},"last":1.167}]},{"osd":2,"last update":"Sun Jul 11 04:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.282,"15min":1.256},"min":{"1min":0.502,"5min":0.499,"15min":0.35899999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.8129999999999999},"last":1.641},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.2729999999999999,"15min":1.27},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.37},"max":{"1min":1.5409999999999999,"5min":1.758,"15min":1.758},"last":1.575}]},{"osd":3,"last update":"Sun Jul 11 04:48:59 2021","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.1990000000000001,"15min":1.21},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.39400000000000002},"max":{"1min":1.4379999999999999,"5min":1.679,"15min":1.728},"last":1.819},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.256,"15min":1.2390000000000001},"min":{"1min":0.54800000000000004,"5min":0.443,"15min":0.36399999999999999},"max":{"1min":1.585,"5min":1.7470000000000001,"15min":1.7470000000000001},"last":1.542}]},{"osd":5,"last update":"Sun Jul 11 04:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.83499999999999996,"15min":0.84599999999999997},"min":{"1min":0.26600000000000001,"5min":0.252,"15min":0.22600000000000001},"max":{"1min":1.1990000000000001,"5min":1.2989999999999999,"15min":1.6459999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.90600000000000003,"15min":0.89100000000000001},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.22500000000000001},"max":{"1min":1.397,"5min":1.5109999999999999,"15min":1.5109999999999999},"last":1.0369999999999999}]},{"osd":6,"last update":"Sun Jul 11 04:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.88600000000000001,"15min":0.89700000000000002},"min":{"1min":0.30299999999999999,"5min":0.27600000000000002,"15min":0.216},"max":{"1min":1.115,"5min":1.613,"15min":1.7749999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.94699999999999995,"15min":0.95499999999999996},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.22},"max":{"1min":1.3440000000000001,"5min":1.6319999999999999,"15min":1.6319999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Sun Jul 11 04:48:59 2021","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.97599999999999998,"15min":0.97899999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.23100000000000001},"max":{"1min":1.4490000000000001,"5min":1.5860000000000001,"15min":1.7330000000000001},"last":1.381},{"interface":"front","average":{"1min":0.875,"5min":1.0529999999999999,"15min":1.0209999999999999},"min":{"1min":0.28499999999999998,"5min":0.26600000000000001,"15min":0.22},"max":{"1min":1.468,"5min":1.6950000000000001,"15min":1.782},"last":0.89000000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542817,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.72399999999999998,"15min":0.72199999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0509999999999999,"5min":1.133,"15min":1.4359999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.78000000000000003,"15min":0.79900000000000004},"min":{"1min":0.28599999999999998,"5min":0.221,"15min":0.221},"max":{"1min":1.1240000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.84799999999999998}]},{"osd":1,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.755,"15min":0.77000000000000002},"min":{"1min":0.29599999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.94999999999999996,"5min":1.0700000000000001,"15min":1.081},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81499999999999995,"15min":0.82199999999999995},"min":{"1min":0.246,"5min":0.246,"15min":0.23599999999999999},"max":{"1min":1.085,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.93100000000000005}]},{"osd":2,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.81899999999999995,"15min":0.81699999999999995},"min":{"1min":0.29799999999999999,"5min":0.27700000000000002,"15min":0.216},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.625},"last":1.0029999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.876,"15min":0.88200000000000001},"min":{"1min":0.27300000000000002,"5min":0.248,"15min":0.248},"max":{"1min":1.143,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.046}]},{"osd":4,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3500000000000001,"15min":1.337},"min":{"1min":0.91800000000000004,"5min":0.72099999999999997,"15min":0.36399999999999999},"max":{"1min":1.7370000000000001,"5min":1.7989999999999999,"15min":1.859},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.3120000000000001,"15min":1.3380000000000001},"min":{"1min":0.88400000000000001,"5min":0.68300000000000005,"15min":0.34799999999999998},"max":{"1min":1.712,"5min":1.873,"15min":1.873},"last":0.90300000000000002}]},{"osd":5,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.385,"15min":1.3799999999999999},"min":{"1min":0.88,"5min":0.71999999999999997,"15min":0.42899999999999999},"max":{"1min":1.6459999999999999,"5min":1.7789999999999999,"15min":1.875},"last":0.98399999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.359,"15min":1.3660000000000001},"min":{"1min":0.86699999999999999,"5min":0.66700000000000004,"15min":0.436},"max":{"1min":1.702,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":1.1120000000000001}]},{"osd":6,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.3200000000000001,"15min":1.3109999999999999},"min":{"1min":0.89900000000000002,"5min":0.68700000000000006,"15min":0.49199999999999999},"max":{"1min":1.546,"5min":1.6830000000000001,"15min":1.6859999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.321,"15min":1.3420000000000001},"min":{"1min":0.90400000000000003,"5min":0.67400000000000004,"15min":0.33700000000000002},"max":{"1min":1.502,"5min":1.7090000000000001,"15min":1.905},"last":1.1910000000000001}]},{"osd":7,"last update":"Sun Jul 11 04:48:41 2021","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.327,"15min":1.339},"min":{"1min":0.80700000000000005,"5min":0.70099999999999996,"15min":0.44700000000000001},"max":{"1min":1.7330000000000001,"5min":1.855,"15min":1.8700000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.347,"15min":1.361},"min":{"1min":0.85999999999999999,"5min":0.64600000000000002,"15min":0.39900000000000002},"max":{"1min":1.7070000000000001,"5min":1.7849999999999999,"15min":1.7869999999999999},"last":1.3220000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542818,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.83499999999999996,"15min":0.81999999999999995},"min":{"1min":0.58499999999999996,"5min":0.26600000000000001,"15min":0.26100000000000001},"max":{"1min":1.2430000000000001,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.88200000000000001,"15min":0.86299999999999999},"min":{"1min":0.61099999999999999,"5min":0.27300000000000002,"15min":0.252},"max":{"1min":1.157,"5min":1.417,"15min":1.417},"last":0.77800000000000002}]},{"osd":1,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.44400000000000001,"5min":0.245,"15min":0.245},"max":{"1min":1.083,"5min":1.3500000000000001,"15min":1.5469999999999999},"last":0.877},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.83499999999999996,"15min":0.82099999999999995},"min":{"1min":0.52100000000000002,"5min":0.26900000000000002,"15min":0.23799999999999999},"max":{"1min":1.089,"5min":1.2849999999999999,"15min":1.3},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.80500000000000005,"15min":0.79300000000000004},"min":{"1min":0.41799999999999998,"5min":0.23599999999999999,"15min":0.22900000000000001},"max":{"1min":1.0349999999999999,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.96199999999999997,"5min":0.89500000000000002,"15min":0.88200000000000001},"min":{"1min":0.49099999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.2010000000000001,"5min":1.375,"15min":1.573},"last":1.1759999999999999}]},{"osd":4,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.3160000000000001,"15min":1.296},"min":{"1min":0.86299999999999999,"5min":0.66300000000000003,"15min":0.44800000000000001},"max":{"1min":1.538,"5min":1.794,"15min":1.877},"last":1.357},{"interface":"front","average":{"1min":1.417,"5min":1.395,"15min":1.3540000000000001},"min":{"1min":0.73899999999999999,"5min":0.623,"15min":0.48599999999999999},"max":{"1min":1.7210000000000001,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.4419999999999999}]},{"osd":5,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3640000000000001,"15min":1.335},"min":{"1min":0.82399999999999995,"5min":0.67200000000000004,"15min":0.441},"max":{"1min":1.6319999999999999,"5min":1.966,"15min":1.966},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.3580000000000001,"15min":1.335},"min":{"1min":0.90200000000000002,"5min":0.63800000000000001,"15min":0.41499999999999998},"max":{"1min":1.5820000000000001,"5min":1.7829999999999999,"15min":1.956},"last":1.0960000000000001}]},{"osd":6,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.383,"15min":1.3560000000000001},"min":{"1min":0.80300000000000005,"5min":0.64400000000000002,"15min":0.52600000000000002},"max":{"1min":1.667,"5min":1.794,"15min":1.8360000000000001},"last":1.123},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.357},"min":{"1min":0.78600000000000003,"5min":0.58499999999999996,"15min":0.47599999999999998},"max":{"1min":1.7130000000000001,"5min":1.9079999999999999,"15min":2.0550000000000002},"last":1.395}]},{"osd":7,"last update":"Sun Jul 11 04:48:13 2021","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3779999999999999,"15min":1.3560000000000001},"min":{"1min":0.76800000000000002,"5min":0.71599999999999997,"15min":0.47199999999999998},"max":{"1min":1.6040000000000001,"5min":1.992,"15min":1.992},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.3759999999999999,"15min":1.3460000000000001},"min":{"1min":0.85299999999999998,"5min":0.65300000000000002,"15min":0.53500000000000003},"max":{"1min":1.6899999999999999,"5min":1.8149999999999999,"15min":1.964},"last":1.5109999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542818,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050620,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321220,"statfs":{"total":96636764160,"available":95560929280,"internally_reserved":1073741824,"allocated":2084864,"data_stored":966797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.78900000000000003,"15min":0.80000000000000004},"min":{"1min":0.32000000000000001,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":1.026,"5min":1.1459999999999999,"15min":1.2709999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.83999999999999997,"15min":0.85399999999999998},"min":{"1min":0.374,"5min":0.247,"15min":0.22600000000000001},"max":{"1min":1.125,"5min":1.1519999999999999,"15min":1.1870000000000001},"last":0.80700000000000005}]},{"osd":2,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.79800000000000004},"min":{"1min":0.35499999999999998,"5min":0.245,"15min":0.24399999999999999},"max":{"1min":1.05,"5min":1.151,"15min":1.216},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.77100000000000002,"15min":0.80800000000000005},"min":{"1min":0.38,"5min":0.23200000000000001,"15min":0.217},"max":{"1min":1.02,"5min":1.234,"15min":1.234},"last":0.77300000000000002}]},{"osd":3,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":0.86799999999999999,"5min":0.83299999999999996,"15min":0.85899999999999999},"min":{"1min":0.34699999999999998,"5min":0.23999999999999999,"15min":0.23100000000000001},"max":{"1min":1.171,"5min":1.171,"15min":1.3600000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.76500000000000001,"15min":0.78000000000000003},"min":{"1min":0.40699999999999997,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.95699999999999996,"5min":1.2050000000000001,"15min":1.258},"last":1.0549999999999999}]},{"osd":4,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3120000000000001,"15min":1.3169999999999999},"min":{"1min":0.73299999999999998,"5min":0.51900000000000002,"15min":0.29599999999999999},"max":{"1min":1.607,"5min":1.887,"15min":1.887},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.331,"15min":1.3200000000000001},"min":{"1min":0.67900000000000005,"5min":0.48799999999999999,"15min":0.28899999999999998},"max":{"1min":1.6879999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.651}]},{"osd":5,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3380000000000001,"15min":1.3260000000000001},"min":{"1min":0.65700000000000003,"5min":0.46700000000000003,"15min":0.25900000000000001},"max":{"1min":1.776,"5min":2.0059999999999998,"15min":2.0059999999999998},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.3180000000000001,"15min":1.304},"min":{"1min":0.748,"5min":0.432,"15min":0.26400000000000001},"max":{"1min":1.7350000000000001,"5min":2.2280000000000002,"15min":2.2280000000000002},"last":1.599}]},{"osd":6,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4079999999999999,"15min":1.3919999999999999},"min":{"1min":0.80700000000000005,"5min":0.44700000000000001,"15min":0.313},"max":{"1min":1.9199999999999999,"5min":2.2970000000000002,"15min":2.2970000000000002},"last":1.55},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3859999999999999,"15min":1.387},"min":{"1min":0.66800000000000004,"5min":0.498,"15min":0.30599999999999999},"max":{"1min":1.6599999999999999,"5min":2.3410000000000002,"15min":2.3410000000000002},"last":1.47}]},{"osd":7,"last update":"Sun Jul 11 04:48:44 2021","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.304,"15min":1.3160000000000001},"min":{"1min":0.67300000000000004,"5min":0.40999999999999998,"15min":0.34599999999999997},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.498},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3320000000000001,"15min":1.333},"min":{"1min":0.627,"5min":0.55300000000000005,"15min":0.27100000000000002},"max":{"1min":1.8129999999999999,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.829}]}]},{"osd":0,"up_from":14,"seq":60129542816,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1050612,"kb_used_data":2028,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93321228,"statfs":{"total":96636764160,"available":95560937472,"internally_reserved":1073741824,"allocated":2076672,"data_stored":965616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76400000000000001,"15min":0.78400000000000003},"min":{"1min":0.25600000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.94299999999999995,"5min":1.099,"15min":1.1890000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.82399999999999995,"15min":0.84099999999999997},"min":{"1min":0.26300000000000001,"5min":0.222,"15min":0.222},"max":{"1min":1.1379999999999999,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.92900000000000005}]},{"osd":2,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.81000000000000005,"15min":0.81899999999999995},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.21099999999999999},"max":{"1min":1.0800000000000001,"5min":1.129,"15min":1.2789999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.90000000000000002,"15min":0.91900000000000004},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.253},"max":{"1min":1.0720000000000001,"5min":1.252,"15min":1.4219999999999999},"last":1.032}]},{"osd":3,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73399999999999999,"15min":0.76300000000000001},"min":{"1min":0.33400000000000002,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.94499999999999995,"5min":1.0249999999999999,"15min":1.454},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.872,"5min":0.83599999999999997,"15min":0.85199999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.219},"max":{"1min":1.095,"5min":1.1619999999999999,"15min":1.339},"last":0.86199999999999999}]},{"osd":4,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.329,"15min":1.3620000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.6359999999999999,"5min":1.7470000000000001,"15min":2.1030000000000002},"last":1.198},{"interface":"front","average":{"1min":1.367,"5min":1.331,"15min":1.343},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.41899999999999998},"max":{"1min":1.6499999999999999,"5min":1.72,"15min":1.8899999999999999},"last":1.345}]},{"osd":5,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.321,"15min":1.3320000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.6459999999999999,"5min":1.8280000000000001,"15min":1.9399999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.3140000000000001,"15min":1.3440000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.33500000000000002},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":2.0289999999999999},"last":1.284}]},{"osd":6,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3069999999999999,"15min":1.347},"min":{"1min":0.46200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.6950000000000001,"5min":1.7929999999999999,"15min":1.944},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.2949999999999999,"15min":1.3240000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.5649999999999999,"5min":1.6950000000000001,"15min":1.8580000000000001},"last":1.3979999999999999}]},{"osd":7,"last update":"Sun Jul 11 04:48:06 2021","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.335,"15min":1.353},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.6479999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.49},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.349,"15min":1.363},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.39300000000000002},"max":{"1min":1.6779999999999999,"5min":1.8169999999999999,"15min":1.9199999999999999},"last":1.5680000000000001}]}]}]}} 2021-07-11T04:49:09.953 DEBUG:teuthology.orchestra.run.gibba004:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2021-07-11T04:49:10.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.000+0000 7f6b6e04c700 1 Processor -- start 2021-07-11T04:49:10.005 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.000+0000 7f6b6e04c700 1 -- start start 2021-07-11T04:49:10.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.000+0000 7f6b6e04c700 1 --2- >> v2:172.21.2.105:3300/0 conn(0x7f6b680b1ec0 0x7f6b680b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:10.012 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.000+0000 7f6b6e04c700 1 -- --> v1:172.21.2.104:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b680b0420 con 0x7f6b680affb0 2021-07-11T04:49:10.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.000+0000 7f6b6e04c700 1 -- --> v1:172.21.2.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b680a77d0 con 0x7f6b680b22a0 2021-07-11T04:49:10.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.000+0000 7f6b6e04c700 1 -- --> v2:172.21.2.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b680b0650 con 0x7f6b680b1ec0 2021-07-11T04:49:10.013 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.000+0000 7f6b66ffd700 1 --1- >> v1:172.21.2.104:6789/0 conn(0x7f6b680affb0 0x7f6b680a70c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.2.104:6789/0 says I am v1:172.21.2.104:49348/0 (socket says 172.21.2.104:49348) 2021-07-11T04:49:10.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.000+0000 7f6b66ffd700 1 -- 172.21.2.104:0/702748902 learned_addr learned my addr 172.21.2.104:0/702748902 (peer_addr_for_me v1:172.21.2.104:0/0) 2021-07-11T04:49:10.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 <== mon.0 v1:172.21.2.104:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2754391515 0 0) 0x7f6b680b0420 con 0x7f6b680affb0 2021-07-11T04:49:10.014 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 --> v1:172.21.2.104:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b48003260 con 0x7f6b680affb0 2021-07-11T04:49:10.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 <== mon.0 v1:172.21.2.104:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2687866599 0 0) 0x7f6b48003260 con 0x7f6b680affb0 2021-07-11T04:49:10.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 >> v2:172.21.2.105:3300/0 conn(0x7f6b680b1ec0 msgr2=0x7f6b680b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:49:10.015 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 --2- 172.21.2.104:0/702748902 >> v2:172.21.2.105:3300/0 conn(0x7f6b680b1ec0 0x7f6b680b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-07-11T04:49:10.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 >> v1:172.21.2.105:6789/0 conn(0x7f6b680b22a0 legacy=0x7f6b68141250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:49:10.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 --> v1:172.21.2.104:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b680b08f0 con 0x7f6b680affb0 2021-07-11T04:49:10.016 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 <== mon.0 v1:172.21.2.104:6789/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (unknown 2456773854 0 0) 0x7f6b58001660 con 0x7f6b680affb0 2021-07-11T04:49:10.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 >> v1:172.21.2.104:6789/0 conn(0x7f6b680affb0 legacy=0x7f6b680a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:49:10.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 --2- 172.21.2.104:0/702748902 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b480037d0 0x7f6b48003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:10.017 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 --2- 172.21.2.104:0/702748902 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b48004500 0x7f6b48004980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:10.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 --2- 172.21.2.104:0/702748902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b48005070 0x7f6b4800b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:10.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680b0650 con 0x7f6b48005070 2021-07-11T04:49:10.018 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4800bef0 con 0x7f6b480037d0 2021-07-11T04:49:10.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4800c0b0 con 0x7f6b48004500 2021-07-11T04:49:10.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b67fff700 1 --2- 172.21.2.104:0/702748902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b48005070 0x7f6b4800b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:49:10.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b66ffd700 1 --2- 172.21.2.104:0/702748902 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b48004500 0x7f6b48004980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:49:10.019 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b677fe700 1 --2- 172.21.2.104:0/702748902 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b480037d0 0x7f6b48003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:49:10.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b67fff700 1 -- 172.21.2.104:0/702748902 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b48004500 msgr2=0x7f6b48004980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:49:10.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b67fff700 1 --2- 172.21.2.104:0/702748902 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b48004500 0x7f6b48004980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:49:10.020 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b67fff700 1 -- 172.21.2.104:0/702748902 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b480037d0 msgr2=0x7f6b48003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2021-07-11T04:49:10.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b67fff700 1 --2- 172.21.2.104:0/702748902 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b480037d0 0x7f6b48003bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-07-11T04:49:10.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b67fff700 1 -- 172.21.2.104:0/702748902 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b680b08f0 con 0x7f6b48005070 2021-07-11T04:49:10.021 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b67fff700 1 -- 172.21.2.104:0/702748902 reap_dead start 2021-07-11T04:49:10.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b67fff700 1 --2- 172.21.2.104:0/702748902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b48005070 0x7f6b4800b750 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 rx=0x7f6b600012f0 tx=0x7f6b60007d60).ready entity=mon.0 client_cookie=86c8d634bbad1e79 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:49:10.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b600014c0 con 0x7f6b48005070 2021-07-11T04:49:10.022 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b60006870 con 0x7f6b48005070 2021-07-11T04:49:10.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b667fc700 1 -- 172.21.2.104:0/702748902 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b60006bd0 con 0x7f6b48005070 2021-07-11T04:49:10.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 -- 172.21.2.104:0/702748902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b48005070 msgr2=0x7f6b4800b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:49:10.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 --2- 172.21.2.104:0/702748902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b48005070 0x7f6b4800b750 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 rx=0x7f6b600012f0 tx=0x7f6b60007d60).stop 2021-07-11T04:49:10.023 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 -- 172.21.2.104:0/702748902 shutdown_connections 2021-07-11T04:49:10.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 --2- 172.21.2.104:0/702748902 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b48005070 0x7f6b4800b750 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:49:10.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 -- 172.21.2.104:0/702748902 >> 172.21.2.104:0/702748902 conn(0x7f6b6805f6b0 msgr2=0x7f6b6805fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:49:10.024 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 -- 172.21.2.104:0/702748902 shutdown_connections 2021-07-11T04:49:10.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 -- 172.21.2.104:0/702748902 wait complete. 2021-07-11T04:49:10.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 Processor -- start 2021-07-11T04:49:10.025 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 -- start start 2021-07-11T04:49:10.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 --2- >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b48005070 0x7f6b68149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:10.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b6814cf30 0x7f6b68159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:10.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 --2- >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b6815a110 0x7f6b6814ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:10.026 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 -- --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680abb80 con 0x7f6b6814cf30 2021-07-11T04:49:10.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 -- --> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6815aad0 con 0x7f6b6815a110 2021-07-11T04:49:10.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b6e04c700 1 -- --> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6814c730 con 0x7f6b48005070 2021-07-11T04:49:10.027 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b66ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b6814cf30 0x7f6b68159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:49:10.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b66ffd700 1 --2- >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b6814cf30 0x7f6b68159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.2.104:3300/0 says I am v2:172.21.2.104:52590/0 (socket says 172.21.2.104:52590) 2021-07-11T04:49:10.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b66ffd700 1 -- 172.21.2.104:0/3688234303 learned_addr learned my addr 172.21.2.104:0/3688234303 (peer_addr_for_me v2:172.21.2.104:0/0) 2021-07-11T04:49:10.028 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b67fff700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b6815a110 0x7f6b6814ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:49:10.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b66ffd700 1 -- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b48005070 msgr2=0x7f6b68149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:49:10.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b66ffd700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b48005070 0x7f6b68149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:49:10.029 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b66ffd700 1 -- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b6815a110 msgr2=0x7f6b6814ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:49:10.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b66ffd700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b6815a110 0x7f6b6814ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:49:10.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b66ffd700 1 -- 172.21.2.104:0/3688234303 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6814c9d0 con 0x7f6b6814cf30 2021-07-11T04:49:10.030 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.004+0000 7f6b66ffd700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b6814cf30 0x7f6b68159c20 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 rx=0x7f6b68149bc0 tx=0x7f6b58009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:49:10.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.008+0000 7f6b64ff9700 1 -- 172.21.2.104:0/3688234303 <== mon.0 v2:172.21.2.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b58007170 con 0x7f6b6814cf30 2021-07-11T04:49:10.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.008+0000 7f6b64ff9700 1 -- 172.21.2.104:0/3688234303 <== mon.0 v2:172.21.2.104:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b580072d0 con 0x7f6b6814cf30 2021-07-11T04:49:10.031 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.008+0000 7f6b6e04c700 1 -- 172.21.2.104:0/3688234303 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6814cc70 con 0x7f6b6814cf30 2021-07-11T04:49:10.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.008+0000 7f6b64ff9700 1 -- 172.21.2.104:0/3688234303 <== mon.0 v2:172.21.2.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 383+0+0 (secure 0 0 0) 0x7f6b58004920 con 0x7f6b6814cf30 2021-07-11T04:49:10.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.008+0000 7f6b6e04c700 1 -- 172.21.2.104:0/3688234303 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b68148930 con 0x7f6b6814cf30 2021-07-11T04:49:10.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.008+0000 7f6b6e04c700 1 -- 172.21.2.104:0/3688234303 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b68057350 con 0x7f6b6814cf30 2021-07-11T04:49:10.032 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.008+0000 7f6b64ff9700 1 -- 172.21.2.104:0/3688234303 <== mon.0 v2:172.21.2.104:3300/0 4 ==== mgrmap(e 5) v1 ==== 65520+0+0 (secure 0 0 0) 0x7f6b58004a80 con 0x7f6b6814cf30 2021-07-11T04:49:10.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.008+0000 7f6b64ff9700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b4c053270 0x7f6b4c055730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-07-11T04:49:10.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.008+0000 7f6b64ff9700 1 -- 172.21.2.104:0/3688234303 <== mon.0 v2:172.21.2.104:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6b5806b690 con 0x7f6b6814cf30 2021-07-11T04:49:10.033 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.008+0000 7f6b677fe700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b4c053270 0x7f6b4c055730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-07-11T04:49:10.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.008+0000 7f6b677fe700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b4c053270 0x7f6b4c055730 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f6b5000a000 tx=0x7f6b50008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-07-11T04:49:10.034 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.012+0000 7f6b64ff9700 1 -- 172.21.2.104:0/3688234303 <== mon.0 v2:172.21.2.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+138451 (secure 0 0 0) 0x7f6b58042cb0 con 0x7f6b6814cf30 2021-07-11T04:49:10.096 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.092+0000 7f6b6e04c700 1 -- 172.21.2.104:0/3688234303 --> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6b680ac760 con 0x7f6b6814cf30 2021-07-11T04:49:10.101 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.096+0000 7f6b64ff9700 1 -- 172.21.2.104:0/3688234303 <== mon.0 v2:172.21.2.104:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6b58042cb0 con 0x7f6b6814cf30 2021-07-11T04:49:10.102 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.096+0000 7f6b64ff9700 1 -- 172.21.2.104:0/3688234303 <== mon.0 v2:172.21.2.104:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v25)=0 v25) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6b580462d0 con 0x7f6b6814cf30 2021-07-11T04:49:10.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 -- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b4c053270 msgr2=0x7f6b4c055730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:49:10.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b4c053270 0x7f6b4c055730 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f6b5000a000 tx=0x7f6b50008040).stop 2021-07-11T04:49:10.103 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 -- 172.21.2.104:0/3688234303 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b6814cf30 msgr2=0x7f6b68159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-07-11T04:49:10.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b6814cf30 0x7f6b68159c20 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 rx=0x7f6b68149bc0 tx=0x7f6b58009040).stop 2021-07-11T04:49:10.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 -- 172.21.2.104:0/3688234303 shutdown_connections 2021-07-11T04:49:10.104 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:3300/0,v1:172.21.2.105:6789/0] conn(0x7f6b6815a110 0x7f6b6814ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:49:10.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:6800/15271,v1:172.21.2.105:6801/15271] conn(0x7f6b4c053270 0x7f6b4c055730 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:49:10.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.104:3300/0,v1:172.21.2.104:6789/0] conn(0x7f6b6814cf30 0x7f6b68159c20 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-07-11T04:49:10.105 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 --2- 172.21.2.104:0/3688234303 >> [v2:172.21.2.105:3301/0,v1:172.21.2.105:6790/0] conn(0x7f6b48005070 0x7f6b68149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-07-11T04:49:10.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 -- 172.21.2.104:0/3688234303 >> 172.21.2.104:0/3688234303 conn(0x7f6b6805f6b0 msgr2=0x7f6b68141640 unknown :-1 s=STATE_NONE l=0).mark_down 2021-07-11T04:49:10.106 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 -- 172.21.2.104:0/3688234303 shutdown_connections 2021-07-11T04:49:10.110 INFO:teuthology.orchestra.run.gibba004.stderr:2021-07-11T04:49:10.100+0000 7f6b6e04c700 1 -- 172.21.2.104:0/3688234303 wait complete. 2021-07-11T04:49:10.112 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/ceph.py", line 1839, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/run_tasks.py", line 176, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-07-11T04:49:10.115 INFO:teuthology.misc:Shutting down mds daemons... 2021-07-11T04:49:10.116 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2021-07-11T04:49:10.116 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:10.122 INFO:tasks.ceph.mds.a:Stopped 2021-07-11T04:49:10.123 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2021-07-11T04:49:10.124 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:10.129 INFO:tasks.ceph.mds.c:Stopped 2021-07-11T04:49:10.130 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2021-07-11T04:49:10.131 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:10.137 INFO:tasks.ceph.mds.b:Stopped 2021-07-11T04:49:10.138 INFO:teuthology.misc:Shutting down osd daemons... 2021-07-11T04:49:10.138 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2021-07-11T04:49:10.139 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:13.299 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2021-07-11T04:49:16.242 INFO:tasks.ceph.osd.0:Stopped 2021-07-11T04:49:16.243 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2021-07-11T04:49:16.244 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:16.310 INFO:tasks.ceph.osd.1:Stopped 2021-07-11T04:49:16.311 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2021-07-11T04:49:16.312 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:19.309 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2021-07-11T04:49:19.309 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2021-07-11T04:49:19.915 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:19.918 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:19.921 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:19.924 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:22.414 INFO:tasks.ceph.osd.2:Stopped 2021-07-11T04:49:22.415 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2021-07-11T04:49:22.416 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:25.218 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2021-07-11T04:49:25.219 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2021-07-11T04:49:28.519 INFO:tasks.ceph.osd.3:Stopped 2021-07-11T04:49:28.521 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2021-07-11T04:49:28.522 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:28.594 INFO:tasks.ceph.osd.4:Stopped 2021-07-11T04:49:28.595 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2021-07-11T04:49:28.595 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:30.924 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2021-07-11T04:49:30.925 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2021-07-11T04:49:34.698 INFO:tasks.ceph.osd.5:Stopped 2021-07-11T04:49:34.699 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2021-07-11T04:49:34.700 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:34.798 INFO:tasks.ceph.osd.6:Stopped 2021-07-11T04:49:34.799 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2021-07-11T04:49:34.799 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:34.886 INFO:tasks.ceph.osd.7:Stopped 2021-07-11T04:49:34.886 INFO:teuthology.misc:Shutting down mgr daemons... 2021-07-11T04:49:34.887 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2021-07-11T04:49:34.888 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:34.899 INFO:tasks.ceph.mgr.y:Stopped 2021-07-11T04:49:34.900 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2021-07-11T04:49:34.900 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:34.911 INFO:tasks.ceph.mgr.x:Stopped 2021-07-11T04:49:34.912 INFO:teuthology.misc:Shutting down mon daemons... 2021-07-11T04:49:34.913 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2021-07-11T04:49:34.913 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:34.952 INFO:tasks.ceph.mon.a:Stopped 2021-07-11T04:49:34.953 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2021-07-11T04:49:34.954 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:34.965 INFO:tasks.ceph.mon.b:Stopped 2021-07-11T04:49:34.966 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2021-07-11T04:49:34.967 INFO:teuthology.orchestra.run:waiting for 300 2021-07-11T04:49:34.982 INFO:tasks.ceph.mon.c:Stopped 2021-07-11T04:49:34.983 INFO:tasks.ceph:Checking cluster log for badness... 2021-07-11T04:49:34.984 DEBUG:teuthology.orchestra.run.gibba004:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'has not responded to cap revoke by MDS for over' | egrep -v MDS_CLIENT_LATE_RELEASE | egrep -v 'responding to mclientcaps' | head -n 1 2021-07-11T04:49:34.995 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T04:49:34.995 DEBUG:teuthology.orchestra.run.gibba004:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2021-07-11T04:49:35.232 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T04:49:35.233 DEBUG:teuthology.orchestra.run.gibba004:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2021-07-11T04:49:35.358 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T04:49:35.359 DEBUG:teuthology.orchestra.run.gibba004:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2021-07-11T04:49:35.500 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T04:49:35.501 DEBUG:teuthology.orchestra.run.gibba004:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2021-07-11T04:49:35.634 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T04:49:35.635 DEBUG:teuthology.orchestra.run.gibba005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2021-07-11T04:49:35.856 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T04:49:35.857 DEBUG:teuthology.orchestra.run.gibba005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2021-07-11T04:49:35.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2021-07-11T04:49:35.977 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T04:49:35.978 DEBUG:teuthology.orchestra.run.gibba005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2021-07-11T04:49:36.069 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T04:49:36.070 DEBUG:teuthology.orchestra.run.gibba005:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2021-07-11T04:49:36.170 INFO:tasks.ceph:Archiving mon data... 2021-07-11T04:49:36.173 DEBUG:teuthology.misc:Transferring archived files from gibba004:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/data/mon.a.tgz 2021-07-11T04:49:36.174 DEBUG:teuthology.orchestra.run.gibba004:> mktemp 2021-07-11T04:49:36.183 INFO:teuthology.orchestra.run.gibba004.stdout:/tmp/tmp.748EHqdCTb 2021-07-11T04:49:36.184 DEBUG:teuthology.orchestra.run.gibba004:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.748EHqdCTb 2021-07-11T04:49:36.362 DEBUG:teuthology.orchestra.run.gibba004:> sudo chmod 0666 /tmp/tmp.748EHqdCTb 2021-07-11T04:49:36.377 DEBUG:teuthology.orchestra.remote:gibba004:/tmp/tmp.748EHqdCTb is 501KB 2021-07-11T04:49:36.446 DEBUG:teuthology.orchestra.run.gibba004:> rm -fr /tmp/tmp.748EHqdCTb 2021-07-11T04:49:36.450 DEBUG:teuthology.misc:Transferring archived files from gibba005:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/data/mon.b.tgz 2021-07-11T04:49:36.451 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T04:49:36.461 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.sKZtUTvRx4 2021-07-11T04:49:36.462 DEBUG:teuthology.orchestra.run.gibba005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.sKZtUTvRx4 2021-07-11T04:49:36.608 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.sKZtUTvRx4 2021-07-11T04:49:36.661 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.sKZtUTvRx4 is 552KB 2021-07-11T04:49:36.784 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.sKZtUTvRx4 2021-07-11T04:49:36.795 DEBUG:teuthology.misc:Transferring archived files from gibba005:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/data/mon.c.tgz 2021-07-11T04:49:36.795 DEBUG:teuthology.orchestra.run.gibba005:> mktemp 2021-07-11T04:49:36.805 INFO:teuthology.orchestra.run.gibba005.stdout:/tmp/tmp.4E4RRAs55P 2021-07-11T04:49:36.805 DEBUG:teuthology.orchestra.run.gibba005:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.4E4RRAs55P 2021-07-11T04:49:36.996 DEBUG:teuthology.orchestra.run.gibba005:> sudo chmod 0666 /tmp/tmp.4E4RRAs55P 2021-07-11T04:49:37.008 DEBUG:teuthology.orchestra.remote:gibba005:/tmp/tmp.4E4RRAs55P is 552KB 2021-07-11T04:49:37.075 DEBUG:teuthology.orchestra.run.gibba005:> rm -fr /tmp/tmp.4E4RRAs55P 2021-07-11T04:49:37.085 INFO:tasks.ceph:Cleaning ceph cluster... 2021-07-11T04:49:37.087 DEBUG:teuthology.orchestra.run.gibba004:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-07-11T04:49:37.092 DEBUG:teuthology.orchestra.run.gibba005:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-07-11T04:49:37.129 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-07-11T04:49:37.134 DEBUG:teuthology.orchestra.run.gibba023:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2021-07-11T04:49:37.147 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2021-07-11T04:49:37.148 DEBUG:teuthology.orchestra.run.gibba004:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-07-11T04:49:37.151 DEBUG:teuthology.orchestra.run.gibba005:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-07-11T04:49:37.154 DEBUG:teuthology.orchestra.run.gibba019:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-07-11T04:49:37.157 DEBUG:teuthology.orchestra.run.gibba023:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2021-07-11T04:49:37.166 INFO:teuthology.orchestra.run.gibba004.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-07-11T04:49:37.171 INFO:teuthology.orchestra.run.gibba005.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-07-11T04:49:37.171 INFO:teuthology.orchestra.run.gibba019.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-07-11T04:49:37.172 INFO:teuthology.orchestra.run.gibba023.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2021-07-11T04:49:37.184 INFO:tasks.ceph:Archiving crash dumps... 2021-07-11T04:49:37.186 DEBUG:teuthology.misc:Transferring archived files from gibba004:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba004/crash 2021-07-11T04:49:37.188 DEBUG:teuthology.orchestra.run.gibba004:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-07-11T04:49:37.230 DEBUG:teuthology.misc:Transferring archived files from gibba005:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba005/crash 2021-07-11T04:49:37.232 DEBUG:teuthology.orchestra.run.gibba005:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-07-11T04:49:37.244 DEBUG:teuthology.misc:Transferring archived files from gibba019:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba019/crash 2021-07-11T04:49:37.246 DEBUG:teuthology.orchestra.run.gibba019:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-07-11T04:49:37.273 DEBUG:teuthology.misc:Transferring archived files from gibba023:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba023/crash 2021-07-11T04:49:37.278 DEBUG:teuthology.orchestra.run.gibba023:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2021-07-11T04:49:37.305 INFO:tasks.ceph:Shutting down logrotate 2021-07-11T04:49:37.306 DEBUG:teuthology.orchestra.run.gibba004:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:37.308 DEBUG:teuthology.orchestra.run.gibba005:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:37.311 DEBUG:teuthology.orchestra.run.gibba019:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:37.315 DEBUG:teuthology.orchestra.run.gibba023:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:37.365 DEBUG:teuthology.orchestra.run.gibba004:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:37.400 DEBUG:teuthology.orchestra.run.gibba005:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:37.422 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:37.444 DEBUG:teuthology.orchestra.run.gibba023:> sudo rm /etc/logrotate.d/ceph-test.conf 2021-07-11T04:49:37.466 INFO:tasks.ceph:Compressing logs... 2021-07-11T04:49:37.467 DEBUG:teuthology.orchestra.run.gibba004:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-07-11T04:49:37.472 DEBUG:teuthology.orchestra.run.gibba005:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-07-11T04:49:37.475 DEBUG:teuthology.orchestra.run.gibba019:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-07-11T04:49:37.479 DEBUG:teuthology.orchestra.run.gibba023:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-07-11T04:49:48.386 INFO:tasks.ceph:Archiving logs... 2021-07-11T04:49:48.387 DEBUG:teuthology.misc:Transferring archived files from gibba004:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba004/log 2021-07-11T04:49:48.390 DEBUG:teuthology.orchestra.run.gibba004:> sudo tar cz -f - -C /var/log/ceph -- . 2021-07-11T04:49:51.437 DEBUG:teuthology.misc:Transferring archived files from gibba005:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba005/log 2021-07-11T04:49:51.443 DEBUG:teuthology.orchestra.run.gibba005:> sudo tar cz -f - -C /var/log/ceph -- . 2021-07-11T04:49:53.038 DEBUG:teuthology.misc:Transferring archived files from gibba019:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba019/log 2021-07-11T04:49:53.040 DEBUG:teuthology.orchestra.run.gibba019:> sudo tar cz -f - -C /var/log/ceph -- . 2021-07-11T04:49:53.332 DEBUG:teuthology.misc:Transferring archived files from gibba023:/var/log/ceph to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba023/log 2021-07-11T04:49:53.333 DEBUG:teuthology.orchestra.run.gibba023:> sudo tar cz -f - -C /var/log/ceph -- . 2021-07-11T04:49:53.370 DEBUG:teuthology.run_tasks:Unwinding manager install 2021-07-11T04:49:53.388 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/task/install/__init__.py", line 619, in task yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/run_tasks.py", line 176, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/ceph_fuse.py", line 159, in task if mount.is_mounted(): File "/home/teuthworker/src/github.com_ceph_ceph_54bff90266c0649369e90717e0c4df7b0276c9e1/qa/tasks/cephfs/fuse_mount.py", line 205, in is_mounted proc.wait() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 148, in wait greenlet.get(block=True,timeout=60) File "src/gevent/greenlet.py", line 803, in gevent._gevent_cgreenlet.Greenlet.get File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/gevent/_compat.py", line 65, in reraise raise value.with_traceback(tb) File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_24e3280d7d1fc156628af95b24b27a29e38b2766/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-07-11T04:49:53.391 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2021-07-11T04:49:53.392 DEBUG:teuthology.orchestra.run.gibba004:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-07-11T04:49:53.396 DEBUG:teuthology.orchestra.run.gibba005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-07-11T04:49:53.400 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-07-11T04:49:53.403 DEBUG:teuthology.orchestra.run.gibba023:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2021-07-11T04:49:53.489 DEBUG:teuthology.run_tasks:Unwinding manager clock 2021-07-11T04:49:53.501 INFO:teuthology.task.clock:Checking final clock skew... 2021-07-11T04:49:53.502 DEBUG:teuthology.orchestra.run.gibba004:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-07-11T04:49:53.541 INFO:teuthology.orchestra.run.gibba004.stdout: remote refid st t when poll reach delay offset jitter 2021-07-11T04:49:53.542 INFO:teuthology.orchestra.run.gibba004.stdout:============================================================================== 2021-07-11T04:49:53.542 INFO:teuthology.orchestra.run.gibba004.stdout:-hv01.front.sepi 162.248.241.94 3 u 85 128 377 0.196 6.727 0.654 2021-07-11T04:49:53.542 INFO:teuthology.orchestra.run.gibba004.stdout:*hv02.front.sepi 163.237.218.18 2 u 100 128 377 0.262 0.279 0.062 2021-07-11T04:49:53.543 INFO:teuthology.orchestra.run.gibba004.stdout:+hv03.front.sepi 74.6.168.72 3 u 83 128 377 0.228 -0.530 0.518 2021-07-11T04:49:53.543 INFO:teuthology.orchestra.run.gibba004.stdout:+hv04.front.sepi 144.172.126.201 3 u 82 128 377 0.219 2.724 0.125 2021-07-11T04:49:53.544 DEBUG:teuthology.orchestra.run.gibba005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-07-11T04:49:53.563 INFO:teuthology.orchestra.run.gibba005.stdout: remote refid st t when poll reach delay offset jitter 2021-07-11T04:49:53.564 INFO:teuthology.orchestra.run.gibba005.stdout:============================================================================== 2021-07-11T04:49:53.564 INFO:teuthology.orchestra.run.gibba005.stdout:-hv01.front.sepi 162.248.241.94 3 u 22 128 377 0.235 11.798 1.268 2021-07-11T04:49:53.564 INFO:teuthology.orchestra.run.gibba005.stdout:*hv02.front.sepi 163.237.218.18 2 u 21 128 377 0.204 4.175 0.461 2021-07-11T04:49:53.565 INFO:teuthology.orchestra.run.gibba005.stdout:+hv03.front.sepi 74.6.168.72 3 u 27 128 377 0.183 2.133 1.130 2021-07-11T04:49:53.565 INFO:teuthology.orchestra.run.gibba005.stdout:+hv04.front.sepi 144.172.126.201 3 u 4 128 377 0.212 5.916 0.789 2021-07-11T04:49:53.566 DEBUG:teuthology.orchestra.run.gibba019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-07-11T04:49:53.587 INFO:teuthology.orchestra.run.gibba019.stdout: remote refid st t when poll reach delay offset jitter 2021-07-11T04:49:53.588 INFO:teuthology.orchestra.run.gibba019.stdout:============================================================================== 2021-07-11T04:49:53.589 INFO:teuthology.orchestra.run.gibba019.stdout:-hv01.front.sepi 162.248.241.94 3 u 12 128 377 0.225 11.973 4.098 2021-07-11T04:49:53.589 INFO:teuthology.orchestra.run.gibba019.stdout:*hv02.front.sepi 163.237.218.18 2 u 77 128 377 0.226 6.811 2.328 2021-07-11T04:49:53.590 INFO:teuthology.orchestra.run.gibba019.stdout:+hv03.front.sepi 74.6.168.72 3 u 82 128 377 0.231 8.835 2.959 2021-07-11T04:49:53.590 INFO:teuthology.orchestra.run.gibba019.stdout:+hv04.front.sepi 144.172.126.201 3 u 26 128 377 0.249 12.686 2.733 2021-07-11T04:49:53.591 DEBUG:teuthology.orchestra.run.gibba023:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-07-11T04:49:53.614 INFO:teuthology.orchestra.run.gibba023.stdout: remote refid st t when poll reach delay offset jitter 2021-07-11T04:49:53.615 INFO:teuthology.orchestra.run.gibba023.stdout:============================================================================== 2021-07-11T04:49:53.616 INFO:teuthology.orchestra.run.gibba023.stdout:-hv01.front.sepi 162.248.241.94 3 u 51 128 377 0.272 8.158 1.079 2021-07-11T04:49:53.617 INFO:teuthology.orchestra.run.gibba023.stdout:*hv02.front.sepi 163.237.218.18 2 u 126 128 377 0.277 1.926 0.265 2021-07-11T04:49:53.617 INFO:teuthology.orchestra.run.gibba023.stdout:+hv03.front.sepi 74.6.168.72 3 u 64 128 377 0.265 1.346 0.545 2021-07-11T04:49:53.618 INFO:teuthology.orchestra.run.gibba023.stdout:+hv04.front.sepi 144.172.126.201 3 u 80 128 377 0.252 4.582 0.271 2021-07-11T04:49:53.618 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2021-07-11T04:49:53.633 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2021-07-11T04:49:53.634 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2021-07-11T04:49:53.648 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2021-07-11T04:49:53.664 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2021-07-11T04:49:53.678 INFO:teuthology.task.internal:Duration was 3792.418013 seconds 2021-07-11T04:49:53.680 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2021-07-11T04:49:53.697 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2021-07-11T04:49:53.698 DEBUG:teuthology.orchestra.run.gibba004:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-07-11T04:49:53.703 DEBUG:teuthology.orchestra.run.gibba005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-07-11T04:49:53.706 DEBUG:teuthology.orchestra.run.gibba019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-07-11T04:49:53.710 DEBUG:teuthology.orchestra.run.gibba023:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2021-07-11T04:49:53.765 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2021-07-11T04:49:53.766 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba004.front.sepia.ceph.com 2021-07-11T04:49:53.767 DEBUG:teuthology.orchestra.run.gibba004:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-07-11T04:49:53.776 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba005.front.sepia.ceph.com 2021-07-11T04:49:53.777 DEBUG:teuthology.orchestra.run.gibba005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-07-11T04:49:53.791 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba019.front.sepia.ceph.com 2021-07-11T04:49:53.792 DEBUG:teuthology.orchestra.run.gibba019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-07-11T04:49:53.807 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@gibba023.front.sepia.ceph.com 2021-07-11T04:49:53.808 DEBUG:teuthology.orchestra.run.gibba023:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2021-07-11T04:49:53.817 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2021-07-11T04:49:53.818 DEBUG:teuthology.orchestra.run.gibba004:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-07-11T04:49:53.821 DEBUG:teuthology.orchestra.run.gibba005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-07-11T04:49:53.824 DEBUG:teuthology.orchestra.run.gibba019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-07-11T04:49:53.827 DEBUG:teuthology.orchestra.run.gibba023:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2021-07-11T04:49:53.856 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2021-07-11T04:49:53.870 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2021-07-11T04:49:53.871 DEBUG:teuthology.orchestra.run.gibba004:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-07-11T04:49:53.880 DEBUG:teuthology.orchestra.run.gibba005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-07-11T04:49:53.893 DEBUG:teuthology.orchestra.run.gibba019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-07-11T04:49:53.904 DEBUG:teuthology.orchestra.run.gibba023:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2021-07-11T04:49:53.917 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2021-07-11T04:49:53.929 DEBUG:teuthology.orchestra.run.gibba004:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-07-11T04:49:53.936 DEBUG:teuthology.orchestra.run.gibba005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-07-11T04:49:53.939 DEBUG:teuthology.orchestra.run.gibba019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-07-11T04:49:53.944 INFO:teuthology.orchestra.run.gibba004.stdout:kernel.core_pattern = core 2021-07-11T04:49:53.947 DEBUG:teuthology.orchestra.run.gibba023:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2021-07-11T04:49:53.952 INFO:teuthology.orchestra.run.gibba019.stdout:kernel.core_pattern = core 2021-07-11T04:49:53.955 INFO:teuthology.orchestra.run.gibba005.stdout:kernel.core_pattern = core 2021-07-11T04:49:53.960 INFO:teuthology.orchestra.run.gibba023.stdout:kernel.core_pattern = core 2021-07-11T04:49:54.003 DEBUG:teuthology.orchestra.run.gibba004:> test -e /home/ubuntu/cephtest/archive/coredump 2021-07-11T04:49:54.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:49:54.009 DEBUG:teuthology.orchestra.run.gibba005:> test -e /home/ubuntu/cephtest/archive/coredump 2021-07-11T04:49:54.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:49:54.023 DEBUG:teuthology.orchestra.run.gibba019:> test -e /home/ubuntu/cephtest/archive/coredump 2021-07-11T04:49:54.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:49:54.033 DEBUG:teuthology.orchestra.run.gibba023:> test -e /home/ubuntu/cephtest/archive/coredump 2021-07-11T04:49:54.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-07-11T04:49:54.040 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2021-07-11T04:49:54.053 INFO:teuthology.task.internal:Transferring archived files... 2021-07-11T04:49:54.055 DEBUG:teuthology.misc:Transferring archived files from gibba004:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba004 2021-07-11T04:49:54.056 DEBUG:teuthology.orchestra.run.gibba004:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-07-11T04:49:54.078 DEBUG:teuthology.misc:Transferring archived files from gibba005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba005 2021-07-11T04:49:54.079 DEBUG:teuthology.orchestra.run.gibba005:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-07-11T04:49:54.110 DEBUG:teuthology.misc:Transferring archived files from gibba019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba019 2021-07-11T04:49:54.111 DEBUG:teuthology.orchestra.run.gibba019:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-07-11T04:49:54.146 DEBUG:teuthology.misc:Transferring archived files from gibba023:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2021-07-09_04:10:02-fs-octopus-distro-basic-gibba/6260912/remote/gibba023 2021-07-11T04:49:54.147 DEBUG:teuthology.orchestra.run.gibba023:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2021-07-11T04:49:54.167 INFO:teuthology.task.internal:Removing archive directory... 2021-07-11T04:49:54.168 DEBUG:teuthology.orchestra.run.gibba004:> rm -rf -- /home/ubuntu/cephtest/archive 2021-07-11T04:49:54.171 DEBUG:teuthology.orchestra.run.gibba005:> rm -rf -- /home/ubuntu/cephtest/archive 2021-07-11T04:49:54.174 DEBUG:teuthology.orchestra.run.gibba019:> rm -rf -- /home/ubuntu/cephtest/archive 2021-07-11T04:49:54.178 DEBUG:teuthology.orchestra.run.gibba023:> rm -rf -- /home/ubuntu/cephtest/archive 2021-07-11T04:49:54.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2021-07-11T04:49:54.198 INFO:teuthology.task.internal:Not uploading archives. 2021-07-11T04:49:54.199 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2021-07-11T04:49:54.213 INFO:teuthology.task.internal:Tidying up after the test... 2021-07-11T04:49:54.214 DEBUG:teuthology.orchestra.run.gibba004:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-07-11T04:49:54.224 DEBUG:teuthology.orchestra.run.gibba005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-07-11T04:49:54.227 DEBUG:teuthology.orchestra.run.gibba019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-07-11T04:49:54.231 INFO:teuthology.orchestra.run.gibba004.stdout: 524322 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 11 04:49 /home/ubuntu/cephtest 2021-07-11T04:49:54.232 DEBUG:teuthology.orchestra.run.gibba023:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2021-07-11T04:49:54.234 INFO:teuthology.orchestra.run.gibba005.stdout: 524322 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 11 04:49 /home/ubuntu/cephtest 2021-07-11T04:49:54.239 INFO:teuthology.orchestra.run.gibba023.stdout: 524322 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 11 04:49 /home/ubuntu/cephtest 2021-07-11T15:41:25.301 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-07-11T15:41:25.338 DEBUG:teuthology.task.console_log:Killing console logger for gibba004 2021-07-11T15:41:25.340 DEBUG:teuthology.task.console_log:Killing console logger for gibba005 2021-07-11T15:41:25.341 DEBUG:teuthology.task.console_log:Killing console logger for gibba019 2021-07-11T15:41:25.342 DEBUG:teuthology.task.console_log:Killing console logger for gibba023 2021-07-11T15:41:25.342 DEBUG:teuthology.exit:Finished running handlers